After creating a Node.js script to automatically post Instagram Reels and one to automatically generate thumbnails for a Instagram Reels, what comes next? Automatically generating the content of the video, of course!

In this post I’ll show you how to use OpenAI’s GPT-3.5 to generate the content of a coding tutorial video. I’ll use the same Node.js script I used in the previous posts, plus a create-react-app that will be used to type in the code from the generated tutorial.

Prompting engineeering

The first question I had to ask myself was: how can I prompt GPT-3.5 to generate a coding tutorial? I mean, I can’t just ask it to generate a coding tutorial, right? I need to give it some context, some information, something to work with.

In this new era of working with language models, it’s weird that if you want the model to do someting, you just have to ask for it. So, I started by creating a list of things I wanted to be in the video, like:

• A title
• A description
• The code itself
• A list of ideas I’ve used before
• A list of things to avoid

And then I started to think about how I could use this information to prompt GPT-3.5 to generate a coding tutorial. I came up with the following prompt:

Take a deep breath, relax, and think analytically about the problem you’re trying to solve. I have a JSON file for a script for a video coding tutorial for Instagram Reels, which looks like this: <JSON_EXAMPLE_HERE> Please use this template to generate a new script for a video coding tutorial for Instagram Reels. Avoid using the following ideas: <USED_IDEAS> And avoid the following topics: <AVOID_TOPICS> Make sure to only answer me with a JSON file with the same structure as the one I gave you and nothing else.

After crafting this prompt, I tested it a few time on the ChatGPT, and it was looking good, but as expected, sometimes the output wasn’t a valid JSON format, and that’s when I’ve learned about function calling.

Function calling

Function calling is a new feature of the OpenAI API that allows you to call a function from the prompt, and it helps it to make sure the output from GPT is what you expect.

In my case, I created a function called generateTutorialViaOpenAi that receives the JSON file with the script for the video and returns a new JSON file with the script for the video.

const OpenAI = require('openai');

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

const generateTutorialViaOpenAi = async (prompt) => {
  const schema = {
    type: 'object',
    required: ['commands', 'files', 'description'],
    properties: {
      description: {
        type: 'string',
        description: 'A nice and catchy description for the tutorial.',
      },
      commands: {
        type: 'string',
        description: 'A string with the code and commands to use for the tutorial. It must not be empty.',
      },
      files: {
        type: 'array',
        description: 'The list of files that are going to be used in the tutorial. One of them MUST be index.html',
        items: {
          type: 'object',
          description: 'A file that is going to be used in the tutorial',
          required: ['name', 'content'],
          properties: {
            name: {
              type: 'string',
              description: 'The name of the file',
            },
            content: {
              type: 'string',
              description: 'The content of the file. Should be empty for the file that is going to be typed from the script.',
            },
          },
        },
      },
    },
  };

  const reelsCommands = await openai.chat.completions.create({
    model: 'gpt-3.5-turbo',
    messages: [
      { role: 'system', content: 'You are a helpful programming professor.' },
      { role: 'user', content: prompt },
    ],
    functions: [{ name: 'generate_tutorial', parameters: schema }],
    function_call: { name: 'generate_tutorial' },
  });

  return reelsCommands;
};

As you can see, the schema object is the key here, making sure the output from GPT is what I expect.

Generating the video

Now that I have a JSON file generated on demand by OpenAI, I need to somehow convert it into a video, and this is where the Puppeteer.js script comes into play.

I’m not going to get into too much detail here because this is actually the most complex part, and it deserves its own post one day, but in a nutshell, I’m using Puppeteer.js to open a headless Chrome browser, load a React app, and using react-ace for the coding highlighting, then with Puppeteer.js I’m typing away the code and taking a screenshot of the page and saving it as a PNG file by using the ‘Page.screencastFrame’.

const allFrames = [];
const browser = await puppeteer.launch();
const [page] = await browser.pages();
const client = await page.target().createCDPSession();
client.on('Page.screencastFrame', async ({ data, sessionId }) => {
  await client.send('Page.screencastFrameAck', { sessionId });
  allFrames.push(data);
});

After that, I’m using ffmpeg to convert the PNG files into a video.

const ffmpeg = require('fluent-ffmpeg');
const ffmpegPath = require('ffmpeg-static');
const ffprobe = require('ffprobe-static');

ffmpeg.setFfmpegPath(ffmpegPath);
ffmpeg.setFfprobePath(ffprobe.path);

const createVideoFromFrames = async (
  fps,
  frames,
  outputPath
) => {
  const tempOutput = `${outputPath}.temp`;
  await new Promise((resolve, reject) => {
    ffmpeg()
      .input(frames)
      .inputFPS(fps)
      .toFormat('mp4')
      .videoCodec('libx264')
      .videoBitrate('5000k')
      .addOption('-crf', '23')
      .addOption('-g', '30')
      .addOption('-profile:v', 'high')
      .addOption('-level', '4.0')
      .on('end', resolve)
      .on('error', reject)
      .save(tempOutput);
  });

  await new Promise((resolve, reject) => {
    ffmpeg(tempOutput)
      .fps(FPS)
      .on('end', resolve)
      .on('error', reject)
      .output(outputPath)
      .run();
  });
};

Putting it all together

Now that I have all the pieces, I just need to put them together. I created a new Node.js script that will:

• Generate a new JSON file with the script for the video
• Generate a new video with the script for the video
• Generate a new thumbnail for the video
• Post the video to Instagram Reels

const generateTutorialViaOpenAi = require('./generate-tutorial-via-openai');
const generateImageForPost = require('./generate-image-for-post');
const generateVideoForPost = require('./generate-video-for-post');
const postVideoToInstagramReels = require('./post-video-to-instagram-reels');
const prompt = require('./prompt');

const generateTutorial = async () => {
  const reelsCommands = await generateTutorialViaOpenAi(prompt);
  const { description, commands, files } = reelsCommands.data.choices[0].value;

  const outputPath = `./output/${Date.now()}`;
  const thumbnailPath = `${outputPath}/thumbnail.png`;
  const videoPath = `${outputPath}/video.mp4`;

  await generateImageForPost(
    './assets/background.png',
    './assets/foreground.png',
    description,
    thumbnailPath
  );

  await generateVideoForPost(commands, files, videoPath);

  await postVideoToInstagramReels(description, videoPath, thumbnailPath);
};

generateTutorial();

The actual code I’m using is a little bit more complex, but this is the gist of it.

Proof of concept

Of course after doing all this work I had to test it, so I created a new Instagram account that will post a new tutorial every day, and always avoid existing ideas and topics, well, at least in theory…

… but the reality ended up being that the generated tutorials are not that good, and I’m not sure if it’s because of the prompt I’m using, or because of the model itself, but I’m sure it’s a combination of both. For some reason it keeps generating the same tutorials, always related to coloring things using CSS.

I have been running this account for 3 months now, and it got only 38 followers as of today and 466 accounts reached in the past 30 days, so I’m not sure if it’s worth it to keep it running, even though it’s fully automated and the total cost so far was only U$1.00.

I think the best approach here is a hybrid one, where I generate my own scripts and use my Node.js code to generate the video and post it to Instagram Reels, which already automates a lot of the work.

Conclusion

I’m really excited about this new era of working with language models. I’m sure we’ll see a lot of cool stuff being created in the next few years.

Even though this project didn’t work as expected, I learned a lot about OpenAI’s API and how to use it, and I’m sure I’ll be using it in the future.

I hope you enjoyed this post, and don’t forget to check how to automatically post Instagram Reels using Node.js and how to automatically generate thumbnails for your videos with Node.js.