Video Art: Experimental Animation and Video Techniques in Linux

http://www.linuxjournal.com/content/experimental-animation-and-video-techniques-linux


 Animation and video editing in Linux can be treacherous territory. Anyone who has tried working in these media probably has experienced the frustration of rendering a huge file for an hour only to see the program crash before the export is finished. A bevy of tools and applications for manipulating video exist for Linux, and some are more mature than others.
The most mainstream of GUI applications have been covered quite a bit in other Linux-related articles on the Web and in print, including in previous issues of Linux Journal. Some of these names may ring familiar to you: Kino, PiTiVi, Openshot, Cinelerra, Kdenlive and Open Movie Editor.
Although I refer to these nonlinear editors (NLEs) from time to time here, the main purpose of this article is to introduce some video effects and techniques you may not have thought of before. If you are producing a film or animation in a conventional format, such as a DVD or a Web video, you most likely will want to employ a suitable NLE at some point in your process. Many of the ideas I present in this article are experimental.

Video Editing

LiVES
LiVES is primarily a VJ (video jockey) tool for performing live audio-visual effects, but it also can encode and export video via its MPlayer back end. The interface has two modes: clip editor and multitrack editor. The clip editor view is more suitable for live VJ sets, while you'll probably lean toward the multitrack view if using LiVES as your NLE.
Figure 1. LiVES in the Clip Editor View
LiVES is highly extensible. In addition to the built-in effects, you can apply custom RFX (rendered/real-time effects) plugins. Several of these scripts are available for download from the LiVES Web site. You also can share LiVES' real-time effects with other applications using the frei0r effects API.
The number of options and the advanced effects in LiVES are comparable to those of Cinelerra, but I strongly recommend LiVES over the latter. Cinelerra is indeed a powerful video editor, but the interface is antiquated and difficult to use. Although LiVES can seem foreign to new users, it is not hard to become acquainted with it.
ZS4
ZS4, formerly known as Zweistein, is a unique—and quite strange—video editor and compositor. The developers of ZS4, who go by the name "t@b", are a duo of musicians who use their own software to create music videos. They are hard to pinpoint on the Web, as they use several sites for different purposes.
I admit that I was confused by the existence of both zs4.net and zs4.org, as well as the Viagra advertisement lines that appeared in Google search results at the zs4.net domain. The two sites both contain download links for ZS4 as well as some other software.
If you plan to use ZS4, I recommend downloading the t@b Media Converter and/or installing Avidemux, as ZS4 is picky about importing video files. Most videos are not compatible out of the box, so it may be necessary to convert them to a format ZS4 can work with.
Working with ZS4 can be frustrating at first because the interface is far from intuitive. Menus are not where you would expect them to be, and you might find yourself aimlessly clicking your cursor in different places to accomplish a simple task, such as dragging a media file into the timeline. The media viewing windows are vaguely labeled "rectangles". To show or hide a track, you click on + or - instead of the typical open- or closed-eye icon.
It took me years to gather the patience to learn my way around this program. So yes, the GUI needs some serious work if it ever is going to reach a mass audience, but it doesn't seem like mainstream appeal is a major concern for the eccentric-minded developers.
So, why tell you about a bizarre-looking application that hasn't been updated in years when there are plenty of other video editors for Linux? Well, for all ZS4's graphical quirks, it can accomplish some very interesting compositing effects.
Figure 2. Tiling Effects in ZS4

 Animation

GIMP
The famous GNU Image Manipulation Program can create animations as well as still images. Because it is a full-featured image editing program, you can use it to create an animation entirely from scratch.
In order to import a prepared image sequence into GIMP, click File→Open as Layers... or press Ctrl-Alt-o. The Open Image dialog allows you to select multiple files, which then will appear as layers.
Figure 3. An animation in progress that I made by tracing reference photos of faces from the Psychological Image Collection at Stirling (PICS).
In the example shown in Figure 3, I imported a series of reference photos into GIMP and traced over them in cyan and then in black. I eventually deleted the reference photos and blue layers, leaving only the black-lined drawings that I planned to use for my final animation.
To finish my animation, I exported the layers as a GIF and specified animation parameters in the export dialog. Because I wanted to use the animation in a video, I had to turn the animated GIF into a video file. I ultimately chose to do this by way of screen recording, but that is not the only option.

From Stills to Movies

Let's say you have a sequence of images, or perhaps an animated GIF, that you want to make into a video file. There are several ways to go about this.
Stopmotion
Stopmotion started as a student project under the Skolelinux/Debian-edu organization in 2005. Although it hasn't been updated since 2008, I find it to be a handy tool for anyone working with frame-by-frame animation. You might have trouble finding Stopmotion in your distribution's repositories if you aren't using a DEB- or RPM-based package manager, but you can, of course, compile it from source on any distribution; that's how I set it up in Sabayon Linux.
Stopmotion is simple and to the point, with a nice drag-and-drop interface. It's not designed for heavy post-production or for drawing and adding effects to frames. Rather, the point is to give users an easy way to arrange images sequentially and export them into a video file.
The video import and export options are limited only by your imagination (and your knowledge of the command line). If you know how to use FFmpeg and/or MEncoder to convert image sequences to video, you can pass your desired command-line arguments to Stopmotion, which is essentially a GUI for those programs. Stopmotion also gives you several choices of video capture commands for grabbing video from your Webcam or another attached device.
One cool feature I didn't know about until I read the user's handbook was the option to add sound. You can set a sound clip to start at any given frame by double-clicking on it. The audio I added to my sequence didn't play in the exported AVI, but maybe you'll have better luck.
If you want to perform more-advanced editing on your individual frames, Stopmotion has a button to open a selected frame in GIMP. You also can export your data into Cinelerra for video editing.
Figure 4. Animating a Sequence of Faces in Stopmotion

 Command Line

There are several ways to turn frames into motion via the command line.
jpegtoavi
jpegtoavi is a simple C script that does exactly what its name suggests—converts a sequence of *.jpg files into an AVI movie. If your images are not in the JPEG format, you first can convert them using the convert command from ImageMagick:

convert image.png image.jpg

If you need to convert a batch of images in a folder, ImageMagick grants you about a gazillion different methods. One of these is to cd to that directory and do:

convert *.png image.jpg

The new filenames will be numbered automatically.
Once you have your folder of sequenced JPEG files, you can employ jpegtoavi. A basic usage template from the man page is:

jpegtoavi -f fps width height img1 [img2 ... imgN]

Although jpegtoavi is nice for simple tasks, minimal documentation exists. I was surprised to find that none of the main Web sites hosting downloads of the software provided any type of wiki or instructions beyond what already was in the man page and README.
You can do more-advanced encoding with FFmpeg and MEncoder, both of which are heavily documented on-line and in their man pages. These programs both rely on libavcodec and have many overlapping uses, but the command formats are different. For this article, I cover only FFmpeg.
This will convert a folder of GIF files sequenced as "image-001", "image-002" and so forth into an MP4 movie file with a framerate of 10 frames per second and a reasonably high bitrate of 1800:

ffmpeg -r 10 -b 1800 -i image-%03d.gif movie.mp4

Make sure your files are named properly, because encoding will stop early if the program encounters a gap in the number sequence.

Animated GIFs

If you're a citizen of the Internet, you've no doubt noticed the recent proliferation of animated GIFs on sites like Tumblr.com. Now that more people have access to high-bandwidth network connections, the GIF art form is not so limited in resolution and number of frames as it was in the 1990s when tiny GIF animations originally rose to popularity in Geocities and Angelfire home pages. Modern GIF animations often display entire scenes from movies.
So, are you ready to pimp out some mad GIF skills?
With ImageMagick, it's easy to fashion an animated GIF from a sequence of non-GIF images:

cd /path/to/image/folder ; convert *.jpg animation.gif

The mother of all command-line GIF manipulation programs though is Gifsicle. Your images must already be in the GIF format to use it.
To create a GIF that animates just once, do:

gifsicle image1.gif image2.gif image3.gif > animation.gif

In most cases, you'll want your animated GIF to loop endlessly. You also may want to specify parameters, such as framerate. Try this for a dithered animation that loops at 10 frames per second:

gifsicle --loopcount=0 --delay 10 --dither image1.gif image2.gif image3.gif > animation.gif

You also can use Gifsicle in reverse mode—that is, to extract the individual frames from an animated GIF. Just use the --explode argument:

gifsicle --explode animation.gif

Now, go out (and by "out", I mean to your nearest terminal) and explore all the neat tricks you can do with Gifsicle!
Here's one more to wet your feet:
Take a ready-made animated GIF with a white background and make it transparent:

gifsicle --transparent '#FFFFFF' --disposal 2 animation.gif > animation-transparent.gif

 Abstraction

For most of us, the notion of animation brings to mind deliberate, structured sequences. In this section, I introduce some less-traditional ways of creating mind-blowing, computer-generated effects without having to know languages like Processing or Pure Data (both of these are very powerful, but not everyone who wants to animate knows how to code).
In my own work with video, screen recording tools have been indispensable. Sometimes I use them to capture animations I make in Pencil, because the movie export feature is broken in the version I use. Other times, I just want to capture some cool imagery on my screen without worrying about proprietary copyrights, so I take screen recordings of free software.
My preferred screen recorder is the bare-bones, command-line version of recordMyDesktop. Your distribution's repositories also might provide the graphical front ends GTK-recordmydesktop and QT-recordmydesktop, but I find those to be buggy and prone to crashes when recording long scenes. You can record your entire screen with:

recordmydesktop screenrecording.ogv

The recording will start as soon as you enter the command, and it will stop when you press Ctrl-c. Read the man page for more options, such as recording a specific window (tip: find a window's ID with xwininfo).
Electric Sheep
If you aren't familiar with the trippiest screensaver in the world, go on-line and look up some images of Electric Sheep. The software artist Scott Draves created Electric Sheep as a dynamic, collaborative fractal flame animation that runs on and by thousands of computers worldwide. Networked data determines the mutations of the various "sheep" in the animation, and users can vote on and contribute their own sheep. And because it's all free, anyone can use the images generated in this android dream.
So how do you take a screen recording of a screensaver? Well, guess what: Electric Sheep is a binary. Just enter electricsheep into your terminal and watch the magic in MPlayer.
If you want to create your own sheep, check out the program Qosmic.
Figure 5. Using recordMyDesktop to Capture Electric Sheep
XaoS
XaoS is a real-time, interactive fractal zoomer that will capture the eye of mathematicians and VJs alike. You can change the fractal formulae and colors with many different parameters and filters. My favorite is the Pseudo-3D filter, which extrudes lines to generate what looks like a surreal landscape. Using the left and right mouse buttons, you can zoom in and out as if flying a plane over the "terrain".
Figure 6. XaoS with the Pseudo-3D Filter Applied
Fyre
Fyre is a program that generates and animates Peter de Jong maps. You don't need a screen recorder to make animations with this; you can enter key frames and render an AVI file directly from the program. As you can see from the screenshot shown in Figure 7, Peter de Jong maps make for some neat, abstract images.
Figure 7. Animating in Fyre

Alphas and More to Look Out For

Unfortunately, there is not enough space in this article or in my brain to cover all the new video-related Linux software that's in development. In lieu of a complete list, I'll provide you with the names of a few projects that I expect to be worth checking out for both developers and end users.
Auteur
Auteur is one cool new kid on the block. I first heard of this project in an episode of the podcast "The Bad Apples" (which has since been re-branded as "GNU World Order"), produced by Seth Kenlon, aka Klaatu, who is also a developer on the Auteur team. Klaatu noted the absence of a truly solid nonlinear video editor for Linux, so he set out to make one with all the features he felt existing software was lacking. The Web site currently says that the project is frozen due to lack of programmers—so programmers, why not help out with a promising alpha?
Figure 8. Testing Out Auteur
VLMC
The folks behind the VLC media player have a nascent project called VLMC (VideoLAN Movie Creator). The latest releases are still basic and not quite stable, but knowing what the VideoLAN team is capable of, I am sure this will mature into a serious nonlinear video editor. They currently are looking for contributors.
Pencil
Pencil is a traditional 2-D animation program, which, although still in beta, already fills a gaping hole in the sphere of Linux animation tools. It allows drawing in both vector and bitmap formats as well as importing images and sounds. My trials with Pencil have been basic but mostly satisfactory, although the video export feature appears broken in Linux. I have compensated for that and made some cool videos anyway simply by taking a screen recording during animation playback in Pencil. There is an active community of Pencil users who post animations on the Pencil Web site's gallery. Pencil is similar to Synfig Studio, but I find the interface easier to navigate.
Figure 9. An Animation I Made in Pencil
Puredyne
Puredyne is a multimedia Linux distribution based on Ubuntu and Debian Live, specifically designed for real-time audio-visual processing. Many of the tools and APIs I haven't had the verbal real estate to cover in this article (such as FreeJ, Gephex and DataDada) are included either in the distribution itself or in optional modules.
And, there you have it, animators and filmmakers. I hope this article inspires a cool music video or two!

Resources

LiVES: http://lives.sourceforge.net
Frei0r: http://piksel.org/frei0r
ZS4 Video Compositing: http://zs4.org
t@b Software: http://www.thugsatbay.com/tab/?q=software
ZS4 Download: http://zs4.net/download
GIMP: http://www.gimp.org
Psychological Image Collection at Stirling (PICS): http://pics.psych.stir.ac.uk
Stopmotion: http://stopmotion.bjoernen.com
JPEG to MJPEG-AVI Converter: http://sourceforge.net/projects/jpegtoavi
FFmpeg: http://ffmpeg.org
Gifsicle: http://www.lcdf.org/gifsicle
recordMyDesktop: http://recordmydesktop.sourceforge.net
Electric Sheep: http://www.electricsheep.org
GNU XaoS: http://wmi.math.u-szeged.hu/xaos/doku.php
Fyre: http://fyre.navi.cx
Auteur Non-Linear Editor: http://auteur-editor.info
VLMC: http://www.videolan.org/vlmc
Pencil—Traditional Animation Software: http://www.pencil-animation.org

Puredyne: http://www.puredyne.org