robert.swain

Since my last post I have worked on applying a similar approach of making HTTP range requests to obtain indexes necessary for seeking in the Flash Video format version 1 (‘old’ FLV, if you will) and AVI (both legacy and ODML) in GStreamer.

Legacy AVI was fairly straightforward as we only need to do one seek to get the index. ODML AVI was somewhat messy because we had to perform multiple seeks as there can be multiple indexes spread across the file. As such, within the GStreamer framework, we have to handle the multiple seeks in the streaming thread rather than in the seek thread as we might prefer. This is because we cannot block the seek thread.

Check out the head commit of git master or the next release of GStreamer to try out this work. If you find bugs, file them on bugzilla as normal for GStreamer.

So, I’ve been rather busy again over the past months and have not had time to blog. However, as I’ve been busy I have things to blog about!

I have been working for Collabora Multimedia since October. It’s been challenging, interesting, rewarding and definitely very enjoyable. They’re a fun bunch to work with.

Most of my previous experience has been with codecs, but I have been working on demuxers since joining the company. I also didn’t really have any prior experience with multi-threaded code and GStreamer certainly is. As such, debugging has been more difficult than usual but I’m seeing potential issues in code before I hit them now, so that’s progress.

I’ve been working on progressive download support for mov/mp4/3gp in GStreamer. The changes were pushed into the master branch not too long ago and I welcome testing of the code and filing of bug reports for any issues you encounter.

The feature was both engaging and problematic to implement. Support for progressive download required a new concept in GStreamer (though I didn’t know that when I started ) – seeking in push mode. ‘What is push mode?’ you might ask…

A GStreamer element has the possibility to support one or both of pull and push modes. Elements are modules that fit together to make up a ‘pipeline’ e.g. an element might be a file source, demuxer, decoder, scaler, volume normaliser, encoder, muxer, file sink, etc..

In pull mode, the element is allowed to seek around as much as it likes. Reads do not have to be contiguous and consecutive because the data from upstream is available with random access. This works just fine for locally stored files for example where seeks are relatively cheap.

In push mode, the data is received contiguously and consecutively – like a stream. Imagine streaming a live radio broadcast – the data comes when it comes and without some cache of the stream somewhere, you can’t seek backwards and you certainly can’t seek forwards to data that does not yet exist.

Now, for progressive download (think YouTube) a file is stored on an HTTP server. One wishes to view the file as quickly as possible after requesting to start playing. mov/mp4/3gp files often have the information necessary to begin playing stored at the end of the file. These formats are handled by the qtdemux element in GStreamer.

This is a problem. Normally, to get the data from the end of the file, one would have to wait for the entire file to download, storing the data locally on the playback device, then reading the data from the end of the file and commencing playback. It should be clear that for any file of significant size compared to the downstream bandwidth available, this approach is slow. Also, on many devices, it may not be possible to store all that data locally. And finally, there was a limit in qtdemux push mode that if this data was outside the first 10MB, we would abort. I assume this was to avoid the above-mentioned issues.

Thankfully, HTTP servers often support ‘range requests’ – asking for data starting and ending at certain byte offsets. We can locate the byte offset of the data necessary to begin playing the file, use a range request to obtain the data (this is a push mode seek!), parse the information and begin playing. This requires much less data upfront and so allows for much faster start up without having to perform too many seeks (which HTTP servers don’t really like…).

But then we want full functionality when playing a stream from some push mode source that doesn’t allow random access like an HTTP server. We want to be able to seek around in the stream to go to the point that we want to watch/hear.

Most container formats have some form of seek index that allows translation of a time position into a byte offset in the file/stream. For mov/mp4/3gp, this information is amongst the same data that we obtained to begin playing the file in the first instance. So, if a seek request to 10 minutes 31 seconds is made, we can build the index from said data, translate the seek time stamp into a byte offset and push the byte offset seek request upstream to the source element. And then… voila! Progressive download with seek support.

It should be noted that this method will only work for HTTP servers supporting range requests.

Of course, better would be for all files intended for distribution over the web to have all information necessary to play at the beginning, or failing that, that all HTTP servers support unlimited range requests and trust that software obtaining data from them does not abuse that privilege.

Well, I’ve been quite busy over the past months, despite my lack of blog posts.

My brother got married.

FFmpeg finally has channel layout support for AAC and Vorbis. I’ve published my work-in-progress HE AAC v1 (Spectral Band Replication) code in FFmpeg’s Summer of Code repository (though it has nothing to do with Google SoC).

A collaboration focused on open source software between two universities in Umeå, Sweden and Vaasa, Finland called OpenKvarken held a seminar about VoIP and related technologies. I did two talks there titled: An Open Standard IPTV Implementation in MythTV and Improving FFmpeg for High Definition Video Conferencing.

The latter refers to ffmpeg-mt – multi-threaded decoding for macroblock-based codecs as developed by Alexander Strange. OpenKvarken should be funding the merge of this code into FFmpeg trunk.

I should be getting a new job with Collabora working to improve the quality of open source multimedia on the whole. It’s a very exciting venture for me and it may lead to more regular blog posts.

There’s been a lot of noise recently about Google Chrome shipping with H.264 decoder support. They’re actually using FFmpeg for this and a few other things. <disclaimer>These comments on my blog do not necessarily represent the views of all FFmpeg project members.</disclaimer> Personally, I’m glad they’re shipping with H.264 support.

Theora was obsolete when it was published and its authors (Monty from Xiph) agreed with this. The encoder implementations are crap, though getting better and, the most annoying noise for me coming from the outside world is that Theora is as good as or better than H.264. Theora is NOT as good as or better than H.264! Specification-wise, Theora cannot compete with H.264. Implementation-wise, in terms of FOSS, Theora (current or Thusnelda) does not compete with x264. There are no non-FOSS Theora implementations and x264 is one of the best H.264 encoders in the world and has few rivals.

I’m all for digital media that is free (in all senses) to create, distribute and consume. I don’t think Theora is the answer.

Well, it’s just over a month since I last posted and the PC is still running.

I’ve updated the motherboard BIOS once but it hasn’t improved the situation with slow Windows installation, nor with the on-board LAN chip not working properly. The chip works fine in Windows 7 which I tried out on a spare drive but it doesn’t work properly in Linux (2.6.29/2.6.30).

I’ve been using a an old 100Mbps 3COM 905 PCI card I had set aside to be used in a VIA Mini-ITX router box. That project hasn’t progressed for a while since we discovered that the issues we’d been having with our ADSL connection for years were in fact just due to a bad microfilter which we had replaced a while ago and so thought was fine.

That little machine is a VIA C3 500MHz. We’ve populated it with RAM and an SD card reader that connects to an IDE socket. I’ve ordered a new ethernet ADSL modem and D-Link Gigabit PCI card. The gigabit card is going in my desktop machine and I’ll move the 3COM back to its original home in the router box. I’m intending to run Smoothwall saving log files to a ram disk to reduce wear on the SD card.

I’ve had a few issues with the Hauppauge WinTV Nova-TD 500 PCI card. After a bit of uptime with the DVB tuners being idle, sometimes one of them would show no signal, sometimes both. I had added the options usbcore autosuspend=-1 line to my /etc/modprobe.d/options file and so thought I had this covered. I later realised, while perusing the output of lsmod, that usbcore had been compiled into the kernel and was not a module so I had to add the option usbcore.autosuspend=-1 to my kernel options in the grub menu.lst. Since then they’ve been OK.

I’ve switched to Claws Mail from Icedove because Thunderbird has always had some annoying little issues that get under my skin. When writing e-mails, if I pause mid-paragraph while thinking what I’m going to write, it sometimes inserts some new paragraph character. It did this in OS X too and I went to the Thunderbird developers to complain about it once but they had no idea to what I was referring. It’s also kind of slow. Claws is working quite well so far, though the interface is a bit clunky in places.

I’ve installed Google Chrome for amd64 Linux and OS X and I’ll be tracking its progress. Hopefully they’ll get the two essentials completed for all platforms soon – Flash plugin support and some form of ad blocking. I heard about the latter being supported via a plugin API in Windows but no noise on other platforms yet. Chrome is certainly much faster than Firefox.

Out of curiosity, as mentioned, I tried Windows 7 on my desktop PC. It really does feel like Vista done right. There are lots of tiny little tweaks to the user interface and such that make it easier to use. Hidden system tray icons can be dragged between being visible or an expandable ‘bin’ of hidden icons. Simple, but effective. Performance-wise it has come along in leaps and bounds it seems.

I tried GTA IV for the PC to see if it faired any better on the new monster. It seems that a Core i7 920 and 6GB DDR3 is not enough for GTA IV. It demands sacrifices to the gaming Gods and a monster graphics card too. In short, my 7800GT is the weak link in my machine so I won’t be playing GTA IV multiplayer with it. Oh well. I still have it on the Xbox 360 so maybe I’ll get Live! Gold for some racing fun. That reminds me, I’d like a fun driving game for the 360. Any recommendations?