converting a processing sketch with audio and visuals to a video on ubuntu

I really like generative art, so I have been playing with processing for a while. Processing is an open source language (on top of java), that gives the possibilty to create images, animations, with added interactivity. First I created a simulation of raindrops, and because I wanted it to be easily configurable I used processing.js: processing implemented on top of javascript. So that becomes native processing in the browser. Allowing to interact with HTML and javascript objects easily. Publishing the sketch is just: give people the link.

Next-up I wanted to create a music visualisation. Interact with the music. I had the perfect piece of music in my head: "Endless Season" by Ken Ishii. Now processing.js does not interact with music. There is HTML5 audio, but it still is very experimental, and I did not find any API for processing music, reacting to and analysing music played. Processing (the java version) has an excellent library for this: Minim (actually more than one, but I ended up using that one). When the sketch was finished, I wanted to share it, convert it to a video. What were the options.

Use Processing itself

There are two ways to convert a processing sketch to a movie, from within processing itself:

• use MovieMaker: this requires quicktime and unfortunately does not work on Ubuntu
• when each frame is drawn, do saveFrame and then convert all frames to a movie afterwards. While in theory this should work, saving the images slowed down my sketch, and ultimately screwed up the sync with the audio. My frame-rate was not consistent enough. For straightforward stuff this does not matter, but I needed it to sync with the audio.
• There is a third option: GSVideo, but frankly, that seemed to damn hard for me, so I skipped that. So I needed an alternative approach. If it runs correctly on my screen, couldn't I just record it on my screen?

Use some sort of screenrecording/screencast software

To record my desktop, on ubuntu, including the sound from processing proved to have some issues:

• processing (i.e. java/JDK 6) does not use ALSA to create the sound, but address the hardware devices directly
• I do not want to record my entire desktop, but a specific part, of a specific size
• I want to share my video on vimeo, so it has to follow certain guidelines The first proved to be the hardest.

Recording the system audio out together with the video

On ubuntu, I found one approach to work very well for me:

• use gtk-recordmydesktop
• use PulseAudio mixer, it will allow to take the sound output as input to record
• and record away! :) But, unfortunately, since java does not ALSA but uses the hardware devices directly, PulseAudio was unable to capture the sounds. However, by accident I found out that if you export your processing sketch to an applet, and run the applet in the browser, it does use ALSA and can be recorded perfectly. Awesome. Part one solved.

Recording a specific part of the screen

gtk-recordmydesktop allows to specify an area of the screen to record, but somewhat akwardly.

Now, for exporting to vimeo, it had to follow certain fixed, optimal sizes. E.g. 640×480 for 4:3 SD video, 640×360 for 16:9 SD video, and 1280×720 or 1920×1080 for HD. And that is hard to do if you are trying to position the recording box manually.

But, as I found out here, when using recordmydesktop from the commandline, you can send those options along:

  recordmydesktop -x=0 -y=0 --width=320 --height=240 

So, if you open the Advanced->Misc, and look for the Extra options field, there you can fill in the same options, and when you press record gtk-recordmydesktop will show the bounding box that is recorded.

Preparing your video for uploading to vimeo

To upload your video to vimeo, you have make sure two things are correct:

• the screensize, which we discussed before
• the video format

gtk-recordmydesktop creates an Ogg Theora file, and unfortunately vimeo does not accept that format yet. Converting that to MP4 was hell, until I found Arista Transcoder. Using Arista to create an MP4 is easy (but you have to know it):

• open Arista Transcode
• create a new conversion
• select your file, by default called out.ogv, as the source
• select Sony Playstation - PSP as the device
• press create!

This will create a file called out.mp4 which is just perfect for vimeo, including the sound.

The result

Twotoned from DIXIS on Vimeo.

Recent comments

• sonsun on Removing VBA Password in AutoCAD 04/05/23
• cccc on [Bootstrap 4] Closing a modal programmatically 28/06/22
• Rollins Duke on Removing VBA Password in AutoCAD 06/05/22
• Greg Tarsa on [rails 4.1] when the rails command just hangs 12/03/22
• gfbh tg on starting a new rails3 site using rspec2 and jquery 01/02/22

Tags