beat sync MP4 with canvas-sketch

about.md

MP4 from frame sequence + audio file

See the resulting 2K video on Vimeo

Let's say you export a sequence of PNGs from canvas-sketch animation into a tmp/ folder. If you have an audio file, you can use the below FFMPEG command to mix the frames and audio into a high quality MP4 file that can be uploaded to Twitter, Vimeo, etc.

Beat-Sync Generative Art

Let's say your canvas-sketch program isn't an animation, but instead exports hundreds of variations on a generative algorithm. You want to mash them all together to create a video that feels somewhat in sync with the music.

• Cut your MP3 file with Audacity or similar program so that it begins on a beat
• Find the BPM of your track with a tool like GetSongBPM.com
• Use a BPM to Frames calculator (e.g. 2 or 4 frames per beat) to find an ideal value for the --fps flag
• Modify the below shell command accordingly

Batch File Sizing & Renaming

I have some not-yet-released tools for batch renaming and resizing images in a folder into a sequence of filenames like 000.png, 001.png, etc. I'll try to publish these soon, but for now this guide works on regular frame sequences exported from canvas-sketch.

ffmpeg-with-sound.sh

ffmpeg -framerate 24 -i tmp/%03d.png -i audio.mp3 -shortest \
  -y -c:v libx264 \
  -tune animation \
  -preset slow -crf 18 -profile:v high -pix_fmt yuvj420p \
  -acodec aac -ar 44100 -ac 2 \
  -x264opts colormatrix=bt709 output.mp4