Fixing AVIs for playback on Bravia TVs

So I bought a newish Sony Bravia TV (2011 model). It has all the Smart TV gumpf including ability to connect to a DLNA server in the house. So then I played around with linux based DLNA servers. Tried ushare, then finally settled on minidlna, though I need to reboot minidlna every now and then. But the bigger problem with DLNA streaming with the Bravia is that won't play all my AVIs. I've been using an old Xbox with XBMC for many years and it could play pretty much anything, but when I first tried the Bravia out, maybe 20% of my AVIs would spit up 'Playback not available' on the TV.

Being the curious type, I started to look at whether there was anything consistent with the AVIs that did not play. You can get mplayer to spit out the details with this;

mplayer -endpos 0 -vo null -ao null somevideo.avi

What I discovered is that the ones that would not play all had a commonality to do with the AUDIO codec information. For example if you grep for AUDIO in the output of the mplayer example above, you might get something like;

AUDIO: 48000 Hz, 2 ch, floatle, 128.0 kbit/4.17% (ratio: 16000->384000)

OR

AUDIO: 48000 Hz, 2 ch, floatle, 127.4 kbit/4.15% (ratio: 15927->384000)

All the ones that the Bravia could not play had a sort of exact whole number kbit rate. eg. the AVI with 128.0 kbit in the AUDIO codec info would not play, but the one with 127.4 kbit would play. I found other numbers that seemingly would deter the Bravia from playback; 112.0 kbit and possibly 96.0 kbit.

So then I tried using mencoder in copy mode to see whether that would 'clean up' the AVI files to make them playable and sure enough it actually worked. The Bravia could play them.

mencoder -ovc copy -oac copy -o somevideo-modded.avi somevideo.avi

mencoder in copy mode is incredibly quick. It only takes a few seconds on a 350MB AVI file on a modern machine.

I have actually found a few other AVI's that don't have a magic whole number for the AUDIO codec info and they don't play on the Bravia for some reason .... but running them through mencoder in copy mode does seem to fix them as well.

Playing with dirty_ratio and dirty_background_ratio

So I have a linux KVM server (SL 6.2 64 bit) that is a bit old and has hardware limits that prevent it having more than 3GB of RAM. It has one of those Seagate XT drives in it that has a SSD cache. With quite a few VMs running where the total sum of the RAM allowances for these VMs is bigger than 3GB, then the performance of the box is rubbish. Lots of stalls and I end up shutting down an idle VM to try and get it useful again. What was I thinking? Overcommitting and it falls apart? ;-)

If I run something like atop while the box is thrashing you can see that its used up all its RAM and there is a lot of paging activity. The thing is, compared to other OS's linux tends to try and push pages that haven't been used for a while to disk, and it has two thresholds; dirty_background_ratio and dirty_ratio ; The background one is sort of the percentage of dirty pages in RAM that you have to have before the VM starts flushing them out. The latter 'dirty_ratio' is a higher percentage threshold. If the amount of dirty pages reaches that percentage then the kernel will block processes from executing until the dirty_ratio gets below that percentage. So hitting the higher percentage is kinda irritating.

There's a good article at rackerhacker about these settings, and there is a note at the end about using 1% as the lower threshold and 100% as the higher one (with the caveat that you really should have a good power supply and RAID card). It might seem weird to have 1% as the lower threshold, as this logically increases disk activity, but with a memory constrained machine you want to start pushing dirty pages out sooner, and having the higher threshold as high as possible means you're telling the OS "Please don't block processes unless its an absolute disaster".

So me, attempting to be marginally conservative tried 3% and 80% for dirty_background_ratio and dirty_ratio respectively and hey it does make quite a difference. A lot of people tend to play with the swappiness setting too, but I've let it as the default of 60%

This article is also good

2.6 kernel on old 2.4 system

Ok, for whatever reasons I had an old Redhat 9.0 system (kernel 2.4.20 or thereabouts) that I wanted to put a 2.6 kernel on. Now Redhat 9.0 is really old. Like 'last century' old. There are a few articles on the net about migrating Redhat 9.0 to a 2.6 kernel including this one, but generally its hard to find decent info.

Anyway, I followed that article but took a stab and downloaded kernel linux-2.6.27.62.tar.bz2. I ended up compiling and installing module-init-tools-3.4.1 as its hard to find really old copies of module-init-tools. I gave up finding the right mkinitrd that might work, and (as per that article) decided I would forgo having an initrd.

Anyway the basic deal is to compile the kernel like so;

cd /usr/src

tar xvjf ~blah/linux-2.6.27.62.tar.bz2

cd linux-2.6.27.62

make mrproper

make oldconfig # I just pretty much held down the ENTER key to take the default answers

make menuconfig # adjusted and disabled a lot of stuff. But I made sure that loopback, ext3 journalling and my network card driver were all part of the kernel (ie. not modules)

make # this is the long bit

make modules_install

# The make install seems to install the kernel and an initd into /boot, but I could never get the initrd to work right.

make install

# And then vi /boot/grub/menu.lst and adjust so there is no initrd line and the root device is hard coded.

#

title Red Hat Linux (2.6.27.62)

root (hd0,0)

kernel /vmlinuz-2.6.27.62 ro root=/dev/hda2 debug

And now I reboot, and make sure to select the 2.6.27 kernel and it starts to boot and seems to mount the root filesystem, but then gets some weird panic about 'init'. I did the usual googling and trawling forums ... and decided "I'll try an older kernel".

So I download kernel 2.6.18, go through all the same setup again, compile it, install it and reboot .... and this time it works fine. There must be something specific in more recent kernels that disagrees with an old 2.4 based system.

Now, a missing piece is that this system is running as a virtual instance. From experience, I know that linux 2.4 and early 2.6 based systems do not virtualise well. They tend to suffer from bad clock drift problems (google for 'vmware timekeeping' for an explanation). If you are trying to virtualise one of these older systems and it contains a database then you are asking for trouble as databases do not like non-linear time. So, initially I did virtualise this Redhat 9.0 system on KVM. You could tell something was not quite right immediately, as the KVM host system was showing 10% CPU use for this virtual machine even though inside the virtual machine there was nothing happening. This same virtual machine, upgraded to a 2.6.18 kernel is lucky to hit 1% CPU use while idle. Note: I am also using a 100Hz tick on this 2.6.18 kernel and so far the timekeeping seems OK (but generally the time keeping problems only really occur when the VM is under load).

Linux LVM2 and md arrays

For some time now, I've been running a debian server with several disks in a RAID5 md array with LVM2 layered over the top. At the time this seemed like the usual path that people were taking; ie. Use the md tools to create your redundant drive array, and use LVM2 over the top to get a flexible volume management system. Coming from a Veritas Volume Manager background, this always seemed a bit dodgy (Veritas manages the redundancy and the logical volumes). However, this setup has proved reliable and straight forward to work on. Admitedly I haven't had a drive failure, but I've never had a situation where the LVM2 or md information needs some form of repair.

I still don't mirror my root filesystem ... but thats a personal choice.

At the time I setup this disk array, there were some notes indicating that the Linux device-mapper stuff (which effectively implements the core of LVM2) would implement mirroring. It looks like that is the case now, though it's not exactly mentioned in many places on the net. The main resource seems to be redhat who I think maintain the LVM2 stuff.

Anyway, you can now do mirroring without an underlying md array. It's really quite simple. If you have another drive you need to pvcreate at least a partition on it and then vgextend and finally lvconvert to do the actual mirroring. Here's an example from an LVM'd debian setup

pvcreate /dev/hdd1
vgextend debian /dev/hdd1
lvconvert --corelog -m1 /dev/debian/usr

Not too sure what the corelog does. I kept on getting an insufficient extents error until I added it. You can do a 'lvs' to see the progress of the mirror. It works pretty well. Like most LVM operations you don't need to umount anything while they're running.

But you still need to use md if you want RAID5. I don't think the device mapper implements RAID5 yet. One thing that is good about current (post kernel 2.6.16) md setups is that you can add a disk to a RAID5 array and dynamically expand it (google for mdadm raid5 reshaping). If you understand how RAID5 works, you'll realise that adding a disk is a complicated process involving restriping data across all the disks in the array. When I set up my array this reshaping feature was not available so I deliberately split each drive up into about 5 or 6 partitions to create multiple RAID5 instances .... which suffice to say allows me to add another drive and reshape it myself (it's a long process, but doable)

Compiling kernel 2.6.20 for a Core Duo

Am just trying out the latest 2.6.20.6 kernel on Debian Sarge, and the first problem I hit was having the 'make bzImage' fail very early on with an error similar to:

cc1: error: invalid option `tune=i686'
make[1]: *** [arch/i386/kernel/asm-offsets.s] Error 1
make: *** [prepare0] Error 2

It turns out you need to use gcc 3.4 or higher if you choose Core Duo as the processor type. There are gcc-3.4 debs in the normal repositories for Debian Sarge.

ADSL speeds

Just a quick note about ADSLv1 speeds. Here in NZ, ADSL2 and ADSL2+ are really just wishful thinking, so we're stuck with ADSLv1 at the moment. Its not too awful. Its essentially; up to 8Mbps down and up to 800kbps up depending on distance to the exchange and bunch of other factors. And actual throughput beyond your exchange is highly variable depending on how congested your exchange is.

For 4 or 5 years I've used an Alcatel Speedtouch USB modem attached to some form of linux box (more recently to an OPENWRT box). It has served me well, but has never managed download speeds higher than 2.3Mbps. Having had a few problems with my ADSL connection lately, I tried using a Dynalink RTA1025W ADSL2+ capable modem on the line, and the speed difference is staggering. This new ADSL router will max out at around 6Mbps ... which is rather fine and dandy.

The Dynalink is an interesting device. It seems to run some form of Montavista linux, and you can even enable SSH access via the web interface. Its a typical little busybox/uclibc type of setup. The only problem with the command line access is I can't work out how you save anything. This would be nice so you could add your own iptables rules and other stuff. One thing that I would like to work out is how to configure and save changes to the 4 port switch. It looks very similar to the one in the WRT54G devices whereby you can configure it to effectively look like multiple interfaces and hence have a true DMZ. The web interface kind of hints at the fact that you can do this but doesn't really tell you how.

icewm sloppy focus

If I'm not using beryl and some other window manager, I'm often using icewm. It's simple and it works. Most of the config is done by editing files in ~/.icewm (or a global location like /usr/share/icewm). I usually like sloppy focus rather than click to focus, so there are a couple of settings in the 'preferences' file you need to change to get sloppy focus going.

However, in the latest icewm (v1.2.30), they seemed to have changed how you configure the focus model. There is a menu item on the main start menu that allows you to set the focus model to 'Click to Focus', 'Sloppy Focus' or 'Custom'. I tried 'Sloppy Focus' and it mostly did what I wanted, except that in order to raise a window I needed to click on the window border rather than anywhere on the window, so I thought I'd try changing those 'preferences' settings to what I thought should work. I tried setting 'ClickToFocus=0' and 'RaiseOnFocus=0' but making those changes and restarting icewm didn't seem to do much. Then I finally tried changing that start menu item from 'Sloppy Focus' to 'Custom' ... and now my type of sloppy focus worked (ie. moving the mouse over a window activates it but won't raise it, and clicking anywhere in it raises it). It looks like you have to set it to 'Custom' for all the old focus settings in the preferences file to be recognised.

There's actually a new config file for that focus mode start menu item called 'focus_mode'. (eg. ~/.icewm/focus_mode ). If that file has 'FocusMode=0' in it then its set to 'Custom' mode.