Dynamically Creating a Personalised Video

You know how you get custom videos which shows your whole year on a platform? Spotify has Spotify Wrapped, Monzo has Year in Monzo - and Facebook has Year in Review.

Ever wondered how they we’re made? Because it seems really inefficient for a company like Spotify to hire a team of editors to hand make a video for all of their millions of users. This would take up a lot of time, and money.

The way this is actually done is that they would effectively create a template, and then automatically render out the video, but for each user they would modify properties or even compositions to match the users data.

This is a massive improvement, as while the time it takes to render the video would stay the same, the time it takes to swap out images, text and alter properties of layers is considerably faster with a computer than a human.

What else could this be used for?

A while ago - I came across an API that returned information about the COVID-19 pandemic - and I made a video which compares the amount of cases by country. It was Yakko’s World from Animaniacs with the number displayed on each of the countries - but a comparison video nonetheless.

I created a Node script which would first call the API to get the data, and then it would create a JSON string, which would then be used by a piece of software which would effectively update my After Effects composition.

For a one minute and 48 second clip - it took three minutes and six seconds in total from running the command to the .mp4 file being rendered. That’s definitely faster than me manually finding the data and then updating them within After Effects.

I made the video for a joke, and for me to try out this kind of thing, and I don’t really intend on releasing it - but things like this could be rendered quickly if say new data came in - or I wanted to create a new video every day, I could trigger a cron job to run the command.

So how do you do it?

In order to make something like a “Year in Review” - We’re going to use Adobe After Effects, a tool called nexrender, managed by a Node script.

Adobe After Effects would be used to make the template, and nexrender would render the video based on a JSON string, which would be created by a Node script.

Creating the template

Just to be clear, I am by no means an After Effects expert - (I make websites for my job, not motion graphics or visual effects 😁)

I struggled to think of what I could do to create a “review video” - so naturally, I came up with something that doesn’t make a whole lot of sense.

After a while - I created four compositions, and put them together into a single one to create some kind of “You liked this the most!” highlight.

video: ./template.mp4

The timing is wrong, I’ve reused some of the animations and It’s quite short - but I think for the sake of a demonstration it’s pretty good.

This template has quite a bit going on, mostly to do with text - but there is also an image in there too - Here is a breakdown of everything that would need changing:

• The person’s name
• The name of what they liked
• The number of likes
• The like description
• The liked image

In terms of what you need to take into account of your After Effects project - you need to ensure the layers are appropriately labelled, as they will be used by Nexrender to change their properties.

For example, I made sure that the name of the person has a layer name of “personName”

Using Nexrender

Nexrender can be installed as an global NPM package.

For this demonstration, you’d also need to include two additional plugins. Encode, which encodes the output, and copy, which will copy the result file to a file path of our choice.

To install Nexrender, and the two plugins, you can run this one command:

npm i -g @nexrender/cli @nexrender/action-encode @nexrender/action-copy

The way nexrender works is that it takes a set of instructions (or job) from a JSON file. The job file would contain everything nexrender needs to apply changes to an After Effects file and render it.

To start, a job JSON must have a template object, which defines the After Effects project and which composition to render.

{
  "template": {
    "src": "file://path/to/after/efects/project.aep",
    "composition": "Main"
  }
}

By default - nexrender will actually delete rendered files, so it’s important to define what to do after the project has been rendered. In our case, we’ll encode the video to MP4, and then move it to a location on my machine.

In the same JSON file, after the template object, add another object:

{
  "template": {
    "src": "file://path/to/after/efects/project.aep",
    "composition": "Main"
  },
  "actions": {
    "postrender": [
      {
        "module": "@nexrender/action-encode",
        "preset": "mp4",
        "output": "encoded.mp4"
      },
      {
        "module": "@nexrender/action-copy",
        "input": "encoded.mp4",
        "output": "C:///path/to/rendered/video.mp4"
      }
    ]
  }
}

Once done, you should be ready to render the project - this can be done by running:

nexrender-cli --file job.json

Nexrender may give an error when you run it for the first time, however this can be fixed by running your terminal as an administrator.

You should see an .mp4 file created, which is your rendered After Effects composition!

We still haven’t really looked into the more dynamic and personalisation of the video though - in order to do that, we need to add some assets.

Within the same job JSON file, create a new array named assets which will contain objects about what we need to modify on the project.

For example, the first piece of text that will change would be the persons full name on the first scene.

By adding an asset of “data” - we can modify the “Source Text” value of a layer.

"assets": [
  {
    "type": "data",
    "layerName": "PersonName",
    "property": "Source Text",
    "value": "Someone's Name"
  }
]

This block will work by setting the “Source Text” on the “PersonName” layer as “Someone’s Name”.

When running the nexrender-cli you should see the “PersonName” value has been changed!

While many other of the changes are similar - there is the image which we also would need to change.

This can be done by downloading an image, and adding another asset block to changing a layers source.

{
  "src": "file:///path/to/flowers-image.jpg",
  "type": "image",
  "layerName": "LikedImage"
}

And when we render again, we’ll notice that the image has changed!

Using Node

We’re able to easily change layers and their properties - but it would be better if we could even automate that!

Thankfully - nexrender comes functionality which allows you to easily run a job using node.

First - you’ll need to install the NPM package locally by running:

npm install @nexrender/core --save

Next create a Javascript file which we will use to run nexrender.

Inside the file, import nexrender, and make an async method that renders out the parsed JSON from our existing job JSON file.

const { render } = require("@nexrender/core");

const renderComposition = async () => {
  const result = await render(JSON.parse(/* JSON STRING */))
}

renderComposition();

Running node index results in exactly the same output, apart from we’re running from Node instead of the nexrender CLI!

From this - we can use Node to create our JSON string for us, and then render that.

User Data

The way I intend the script to work, is to get all of the data needed to create the video, and then one by one generate the video.

For the sake of this example, I’m going to use a standard Javascript object array, but you can also use things like databases.

You could also host it on a server, and pass parameters to it via an API - keep in mind you’d still need a version of After Effects installed on that server.

Inside of the javascript file, I’ve created an array which contains the required data for each of the users.

const users = [
  {
    PersonName: 'Lisa Terry',
    LikedTitle: 'Flowers',
    Number: '2302',
    LikedDescription: 'That\'s a lot.',
    LikedImage: './flowers.jpg',
  },
  {
    PersonName: 'Eric Gomez',
    LikedTitle: 'Pizza',
    Number: '4026',
    LikedDescription: 'And who can blame you!',
    LikedImage: './pizza.jpg',
  },
  {
    PersonName: 'Camila Newman',
    LikedTitle: 'Cats',
    Number: '2549',
    LikedDescription: 'Puuurrrfect!',
    LikedImage: './cat.jpg',
  }
]

I’ve named the key of the layer to update to make things a bit more easier.

Next is to modify our renderComposition method to accept a user, and modify our job JSON.

const renderComposition = async (user) => {
  // Base object
  const jobObject = {
    "template": {
      "src": "file://path/to/after/efects/project.aep",
      "composition": "Main"
    },
    "assets": [],
    "actions":{
      "postrender": [
        {
          "module": "@nexrender/action-encode",
          "preset": "mp4",
          "output": "encoded.mp4"
        },
        {
          "module": "@nexrender/action-copy",
          "input": "encoded.mp4",
          "output": `C:///path/to/rendered/video-${user.PersonName}.mp4`
        }
      ]
    }
  }

  // Apply dynamic values
  // Get the layer names.
  const layers = Object.keys(user);

  layers.forEach((layer) => {
    // Images are added differently
    if (layer === 'LikedImage') {
      jobObject.assets.push({
        src: `file:///path/to/${user[layer]}`,
        type: 'image',
        layerName: layer
      })
    } else {
      jobObject.assets.push({
        type: 'data',
        layerName: layer,
        property: 'Source Text',
        value: user[layer]
      })
    }
  })

  const result = await render(jobObject)
  console.log(result);
}

There is a lot going on here - so let’s break it down a bit!

The first bit of code is part of our existing job JSON. I’ve replaced our assets as an empty array - as we’ll populate them based on the user’s values.

I’ve also set the output file name to contain the users name, this is because nexrender will overwrite anything that already has the same name.

The forEach method goes through each of the users data properties - and pushes an object to an array, these would be our asset as they were objects earlier.

Finally - nexrender will render the job, which would have had it’s assets updated by the forEach method.

The very last step is to call our renderComposition method, for each of our three users.

const forLoop = async () => {
  for (let i = 0; i < users.length; i++) {
    await renderComposition(users[i]);
  }
}

forLoop();

This method will go through our users array, and execute the renderComposition method - passing through the user.

Now when we run node index - it should render three dynamically generated videos!

In our case - we had someone who liked flowers 🌼:

Someone who likes pizza 🍕:

And someone who likes cats 🐈:

Total time it took to render all three videos?

44 seconds. Which is probably faster than doing them all by hand anyway…

Wrapping Up

Nexrender is a tool which can easily modify an Adobe After Effects project, and with Node you’re able to create the JSON file required based on data.

You can also do alot more with nexrender such as change effects - but that’s for another time 😊.

Further Reading

• https://github.com/inlife/nexrender - Nexrender GitHub repo