Debian: /usr/bin/ld: cannot find -lGL

May 4th, 2009 No comments

While building MythTV svn trunk in Debian unstable, I stumbled across this error message:

“/usr/bin/ld: cannot find -lGL”

$ ls -l /usr/lib | grep GL showed that indeed, libGL.so was not present. I was a little confused as I had libgl1-mesa-dev installed. Over to the Debian package website I went to check which packages have libGL.so and I noted nvidia-glx-dev. If you have an NVIDIA card and need libGL.so in Debian, don’t forget to install nvidia-glx-dev. :)

Categories: Uncategorized Tags:

New PC

May 3rd, 2009 No comments

I recently purchased a new PC. I had been using my Apple MacBook for quite some time without any desktop PC. And no, I’m not an Apple fan boy, but it has served its purpose quite well.

For what it’s worth, I would probably recommend that any every day desktop user use Mac OS X rather than Windows if you can’t stomach Linux and don’t have some tech savvy person to maintain a Linux machine for you. My parents have been using Linux for a couple of years and my Dad used to get my Mum to set the VCR. :) He doesn’t have a problem anymore though as I gifted him a DVR the other year and he’s quite happy using the EPG and hitting the record button to schedule a recording. Also, my Aunt and Uncle are using Linux. I’m infecting the world with more Linux users.

Anyway, the new machine and my experiences with it:

  • Intel Core i7 920 (quad core 2.66GHz 8MB L3)
  • 3×2GB Corsair DDR3 1333MHz CAS9 RAM
  • Gigabyte EX58-UD3R motherboard
  • 2×1TB Samsung Spinpoint F1 SATA hard drives
  • LG DVD-RW
  • Antec P183 case
  • Corsair HX620W modular PSU
  • Noctua NH-U12P SE1366 tower heatpipe CPU cooler

That’s all the new stuff and I’m using an NVIDIA GeForce 7800GT PCI-E graphics card I’ve had for a couple of years.

So, how did it fair in my OSes of choice? The main issue is the on-board NIC. These days, if you don’t have a network connection, your OS is not particularly useful. It’s so limiting to not be able to connect to any other machines outside the one you’re using.

Anyway, I don’t know if it’s just a really awful choice of hardware on Gigabyte’s part for the EX58 series of boards, if the BIOS is doing something wrong, if it’s a bus issue, or if the drivers are just shoddy. For your information, the Gigabyte EX58 series of motherboards seem to use a Realtek RTL8111 with RTL8168 or RTL8169 that uses the PCI-E bus. All I know is that the on-board NIC does not work properly at all. In Windows, I had some difficulty installing the drivers and then, when installed, the interface would not recognise that I had connected a cat5 cable.

I didn’t want to use Windows anyway, I just wanted to see how well GTA IV performed on the Core i7. :) In Debian Lenny (Linux 2.6.26) and Ubuntu Jaunty (2.6.28), the on-board NIC sometimes appeared during installation and sometimes didn’t. When it had been detected and had the driver loaded, it didn’t seem to indulge any traffic.

Måns Rullgård, a prominent FFmpeg developer, has recently purchased a Core i7 machine too. He bought the Gigabyte EX58 Extreme and experienced the same issues. He gave up and used a spare PCI NIC he had. Thankfully I had an old 100Mbps 3com card lying around unused so I stuck that in and that seems to work.

With what else did I have problems? The usual Xorg ‘nv’ driver issues with my 7800GT. I look forward to that code being binned in all distributions. At least for my card it’s a piece of crap. It displays GDM mostly fine, with a few corrupt pixels, but then when arriving at the desktop, it is unusable. The cursor can be moved around, but clicking on things invokes no response. ctrl-alt-backspace doesn’t work (yes, I know about dontzap and it’s not that) nor switching to some other VT. It’s as good as hard locked, just with a cursor. Hopefully nouveau will be better. :)

You can see my previous post for my Hauppauge woes. Other than that, I think this machine runs very well. I have my 2×1TB drives set up with a mirrored boot partition (this doesn’t cause boot issues because the partitions can be read separately as if they had no mirror), a 6GB swap partition each (the kernel automagically stripes across the drives if you set them to have the same priority in fstab) and the rest is part of an LVM volume group.

Needless to say, it’s ridiculously fast and I shouldn’t need to upgrade for some years at least. Even one core of this Core i7 is multiple times faster than the Core Duo 1.83GHz I had been using. It’s immense. :)

Categories: Uncategorized Tags:

Hauppauge WinTV Nova-TD 500 PCI in Linux and MythTV

May 2nd, 2009 No comments

I recently purchased a Hauppauge WinTV Nova-TD 500 PCI to use in a Linux machine. I was planning to use Debian testing/unstable AMD64, and may still do so, but at the moment I’ve settled on Ubuntu Jaunty.

I don’t think I managed to get the card working in Debian stable (lenny). However, the card was picked up and the two tuners were assigned devices when using Ubuntu Jaunty.

Unfortunately, it still didn’t pick up the IR device so the remote didn’t work. I spent quite some time trying to figure this out but eventually it was solved by simply compiling and installing new dvb modules from the v4l-dvb mercurial repository as there was some IR-related code missing from the dibcom 0700 source that is now present in the head of the mercurial repository.

I also added a file to block HAL from picking up on the IR device as this supposedly causes problems. I don’t know if it does, but I did it anyway. See here for information on the fdi file that blocks HAL.

Other than that, Ubuntu does a pretty good job of getting the LIRC settings right. The only thing that’s needed is a ~/.lircrc which I created from here.

And finally, to get the MythTV frontend to run without complaints (it has an aversion to pulseaudio at the moment) I had to disable pulseaudio completely.

If MythTV is too difficult to set up for you, I have also used me-tv and a fellow FFmpeg developer also works on a couple of programs with great potential called Showtime and Tvheadend. It functions in a similar way to MythTV in that you run Tvheadend as a DVR backend and then run Showtime as a graphical frontend for actually consuming the media. It does need some polish and some more features, but it’s sooooo easy to set up I will definitely be following it’s progress and maybe contributing to the code myself.

Categories: Uncategorized Tags:

bit rate, file size, quality misunderstandings

May 1st, 2009 2 comments

It seems quite a few people have been confusing how bit rate, file size and quality are inter-related. A number of people have been using the -sameq option from the ffmpeg CLI and expecting it to output a file of the same size and quality as the original, but they observe it creates files that are much larger. So, what do the terms mean and what’s going on?

When specifying the -b <bit rate in bits per second> option to ffmpeg, one is requesting that the video encoder target the specified bit rate as an average over the entire video stream. If you have some audio or video file, playing it back takes some amount of time i.e. it has a duration. The file also has a size in bytes. The overall average bit rate is simply the number of bits per second averaged over the duration of the entire file. So:

overall average bit rate = file size in bytes * bits per byte / duration in seconds

It should be noted that the file may contain, for example, an audio stream and a video stream. So in fact we observe something more like:

overall average bit rate = average audio bit rate + average video bit rate + average overhead bit rate = file size in bytes * bits per byte / duration in seconds

The audio and video bit rates being constituents of the overall average should be clear, but what is this overhead? Metadata tags (like the artist, album and title of a music file) could contribute to this, but they are a fixed size regardless of the duration of the file.

The rest of the overhead, beyond metadata and header information about the streams within the file, consists of various information that is part of the container format the file is using. These are things to note where one section of data begins, the time at which it should be displayed/presented to the user and other bits and pieces of information. This normally does not contribute a large amount to the overall average bit rate, but it’s worth being aware of to avoid confusion later.

Bit rate need not be the overall average however. The term bit rate simply refers to an amount of data with a corresponding duration for transmission/playback or so.

So, now we know how the bit rate and file size (if producing a file) are related, but where does quality come into the equations? There are two main ways to affect the quality of, for example, a video that one wishes to encode.

Most video encoders have a plethora of options that affect this thing called ‘quality’. Quality is subjective and that makes it difficult to quantify. The options that are supposed to improve quality, usually make the encoding process slower. This leads to a speed/quality trade off. That is, you have to consider the relation between two questions:

  • How long am I willing to wait to encode this file?
  • How good do I need the quality to be?

Normally the answer to the second question is “as good as it can be” within the constraints of the answer to the first question. More simply, people want the best quality from whatever amount of encoding time they deem reasonable for their job.

The other way that one can affect the perceived quality of the resulting video is to affect the bit rate used to encode it. If you try to encode 1080p video at 200 kilobits per second, it will look awful no matter what encoder and options you use. If you try to encode 1080p video at 200 megabits per second, it will probably look good no matter what encoder and options you use.

How should one approach making the decisions about what encoding options to use? Normally bit rate/file size are the most major consideration, whether streaming with an upper bound on the bandwidth available when transmitting the stream or creating a file that needs to fit on a 700MB CD. This will impose your first constraint, unless you have some terabytes of storage and don’t really care about the size too much.

After that you need to consider how long you’re willing to wait to encode something as this will impact on the options you use with your encoder of choice to obtain the best quality within your more major constraints. If the quality is not good enough within your constraints, you may need more powerful hardware or to re-evaluate the available bandwidth/storage space.

Finally we come to the -sameq option in ffmpeg. There are a number of different ways to control how the bits are allocated throughout the video. Normally one wants to observe constant quality throughout a video stream. If some frames are good and some bad, it is actually more noticeable and annoying than if all frames do not look quite as good but are consistent.

Some frames are more difficult to compress than others and so need more bits to maintain that consistent level of perceived quality. Conversely, some are easier to compress and so need less bits.

If one conducts two passes when encoding a video stream, this benefits the encoder because it already has a log of the relative complexity of each frame so it can better predict how to allocate the bits.

Traditionally, if conducting only one pass with a variable bit rate, the encoder has to try to guess based on whatever information it can from frames that have already been encoded. There are some ideas around to try to improve the bit allocation when doing one pass but I won’t discuss those here.

MPEG, and other, video codecs use a method called quantisation. This topic is also a bit too complex to discuss here. The quantiser values used in MPEG-1/-2/-4 ASP are 1, 2, …, 31 and for H.264 they are 1, 2, …, 51. Quantisers are how the encoder controls the allocation of bits to frames and parts of frames. The method of quantisation used in H.264 is different to the older codecs.

Confused? Don’t be. A frame compressed using a high quantiser will use less bits and have lower perceived quality than a frame compressed using a low quantiser. One can encode using a constant quantiser in ffmpeg by using the -qscale <quantiser> option. For MPEG-4 ASP (-vcodec mpeg4) using a value somewhere in the region of 3-5 for -qscale should produce good quality. For H.264 (-vcodec libx264) a value somewhere around the late teens to mid 20s should suffice.

So what does the -sameq option do? It uses the quantisers used in the source video stream to encode the output video stream. If converting from MPEG-4 ASP (e.g. Xvid or DivX) to H.264 (e.g. x264) this will inflate the file size considerably because of the different way these formats conduct quantisation.

Regardless, I would pretty much never recommend the use of -sameq. It does not mean “encode at the same quality as the source file”, nor does it mean “encode resulting in a file of similar size to the source file”. It just means “encode using the same quantisers” and the resulting video size and quality will likely be very unpredictable.

The conclusion: use -b with one or two passes. Or, if you’re using libx264 and you’re more concerned about quality than file size/bit rate and only want to do one pass, try -crf which uses an intelligent way of maintaining constant quality while using only one pass.

Categories: Uncategorized Tags:

Updated the FFmpeg x264 encoding guide

April 7th, 2009 No comments

After receiving some criticism of my FFmpeg x264 encoding guide a day or two ago, I decided to make the guide a bit more detailed so that readers could better understand how to construct the necessary command lines to best convert streams to x264 using FFmpeg.

I also moved the ‘old, manual way’ of explicitly specifying all options to a separate page but I won’t link to it in this post because I don’t want people to use it. :)

Categories: Uncategorized Tags: