Legacy AVI was fairly straightforward as we only need to do one seek to get the index. ODML AVI was somewhat messy because we had to perform multiple seeks as there can be multiple indexes spread across the file. As such, within the GStreamer framework, we have to handle the multiple seeks in the streaming thread rather than in the seek thread as we might prefer. This is because we cannot block the seek thread.

Check out the head commit of git master or the next release of GStreamer to try out this work. If you find bugs, file them on bugzilla as normal for GStreamer.

I have been working for Collabora Multimedia since October. It’s been challenging, interesting, rewarding and definitely very enjoyable. They’re a fun bunch to work with.

Most of my previous experience has been with codecs, but I have been working on demuxers since joining the company. I also didn’t really have any prior experience with multi-threaded code and GStreamer certainly is. As such, debugging has been more difficult than usual but I’m seeing potential issues in code before I hit them now, so that’s progress.

I’ve been working on progressive download support for mov/mp4/3gp in GStreamer. The changes were pushed into the master branch not too long ago and I welcome testing of the code and filing of bug reports for any issues you encounter.

The feature was both engaging and problematic to implement. Support for progressive download required a new concept in GStreamer (though I didn’t know that when I started ) – seeking in push mode. ‘What is push mode?’ you might ask…

A GStreamer element has the possibility to support one or both of pull and push modes. Elements are modules that fit together to make up a ‘pipeline’ e.g. an element might be a file source, demuxer, decoder, scaler, volume normaliser, encoder, muxer, file sink, etc..

In pull mode, the element is allowed to seek around as much as it likes. Reads do not have to be contiguous and consecutive because the data from upstream is available with random access. This works just fine for locally stored files for example where seeks are relatively cheap.

In push mode, the data is received contiguously and consecutively – like a stream. Imagine streaming a live radio broadcast – the data comes when it comes and without some cache of the stream somewhere, you can’t seek backwards and you certainly can’t seek forwards to data that does not yet exist.

Now, for progressive download (think YouTube) a file is stored on an HTTP server. One wishes to view the file as quickly as possible after requesting to start playing. mov/mp4/3gp files often have the information necessary to begin playing stored at the end of the file. These formats are handled by the qtdemux element in GStreamer.

This is a problem. Normally, to get the data from the end of the file, one would have to wait for the entire file to download, storing the data locally on the playback device, then reading the data from the end of the file and commencing playback. It should be clear that for any file of significant size compared to the downstream bandwidth available, this approach is slow. Also, on many devices, it may not be possible to store all that data locally. And finally, there was a limit in qtdemux push mode that if this data was outside the first 10MB, we would abort. I assume this was to avoid the above-mentioned issues.

Thankfully, HTTP servers often support ‘range requests’ – asking for data starting and ending at certain byte offsets. We can locate the byte offset of the data necessary to begin playing the file, use a range request to obtain the data (this is a push mode seek!), parse the information and begin playing. This requires much less data upfront and so allows for much faster start up without having to perform too many seeks (which HTTP servers don’t really like…).

But then we want full functionality when playing a stream from some push mode source that doesn’t allow random access like an HTTP server. We want to be able to seek around in the stream to go to the point that we want to watch/hear.

Most container formats have some form of seek index that allows translation of a time position into a byte offset in the file/stream. For mov/mp4/3gp, this information is amongst the same data that we obtained to begin playing the file in the first instance. So, if a seek request to 10 minutes 31 seconds is made, we can build the index from said data, translate the seek time stamp into a byte offset and push the byte offset seek request upstream to the source element. And then… voila! Progressive download with seek support.

It should be noted that this method will only work for HTTP servers supporting range requests.

Of course, better would be for all files intended for distribution over the web to have all information necessary to play at the beginning, or failing that, that all HTTP servers support unlimited range requests and trust that software obtaining data from them does not abuse that privilege.

My brother got married.

FFmpeg finally has channel layout support for AAC and Vorbis. I’ve published my work-in-progress HE AAC v1 (Spectral Band Replication) code in FFmpeg’s Summer of Code repository (though it has nothing to do with Google SoC).

A collaboration focused on open source software between two universities in Umeå, Sweden and Vaasa, Finland called OpenKvarken held a seminar about VoIP and related technologies. I did two talks there titled: An Open Standard IPTV Implementation in MythTV and Improving FFmpeg for High Definition Video Conferencing.

The latter refers to ffmpeg-mt – multi-threaded decoding for macroblock-based codecs as developed by Alexander Strange. OpenKvarken should be funding the merge of this code into FFmpeg trunk.

I should be getting a new job with Collabora working to improve the quality of open source multimedia on the whole. It’s a very exciting venture for me and it may lead to more regular blog posts.