Setting up shellinabox

So I was trying to find a web based terminal program. I'd used Ajaxterm a while back and it does work but is quite clunky. Anyway, I found shellinabox and set it up to test it. It's a straight configure , make , make install type installation. The first time I tried it using the "shellinaboxd -s /:LOGIN" example, it gave me an error about certificate.pem. I ended up following these instructions (repeated below) to generate the self signed certificate.pem;

# cd /var/lib/shellinabox
# openssl genrsa -des3 -out server.key 1024
# openssl req -new -key server.key -out server.csr
# cp server.key server.key.org
# openssl rsa -in server.key.org -out server.key
# openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt
# cat server.crt server.key > certificate.pem

And with shellinabox compiled, as root you can run it like so;

   shellinabox -c /var/lib/shellinabox -s /:LOGIN

Then point a browser at https://yourserver:4200 and you should have a login prompt. The first thing I noticed is that the terminal font is quite large. You're meant to be able to adjust this using a user supplied css file. eg.

   shellinaboxd -c /var/lib/shellinabox/ --user-css /:+notpablo.css -s /:LOGIN

I could never get the font and line height to adjust properly (and I read Issue 68 a few times). What ended up working for me was to change the styles.css directly and recompile shellinabox;

cd ~/shellinabox-2.10/shellinaboxvi styles.css and add in the font-size lines highlighted below

#vt100 #console, #vt100 #alt_console, #vt100 #cursor, #vt100 #lineheight {

font-family: "DejaVu Sans Mono", "Everson Mono", FreeMono, "Andale Mono", "Lucida Console", monospace;

font-size: 8pt;

}

#vt100 #lineheight {

position: absolute;

visibility: hidden;

}

#vt100 #cursor {

position: absolute;

font-size: 8pt;

left: 0px;

top: 0px;

overflow: hidden;

z-index: 1;

}

Installing Windows Phone 7 SDK

So I pretty much hate Windows, but I was curious about Windows Phone 7's SDK environment. So I had a go at setting up a dev environment. First the SDK requirements are quite high.

Supported operating systems: Windows 7, Windows Vista

• Windows® Vista® (x86 and x64) with Service Pack 2 – all editions except Starter Edition
• Windows 7 (x86 and x64) – all editions except Starter Edition
• Installation requires 4 GB of free disk space on the system drive.
• 3 GB RAM
• Windows Phone Emulator requires a DirectX 10 or above capable graphics card with a WDDM 1.1 driver

I must admit I glanced at that list and just saw the '3GB RAM". That pretty much meant I could not do this in a virtual machine as I don't have any box here with more than 4GB of RAM. I don't have a box that has Windows 7 on it natively either, so I found an old 80GB drive and begain setting up an old core 2 duo tower box I have. First I had trouble installing Windows 7 on the box. I knew I had installed Windows 7 before on it, and it eventually transpired that the Windows 7 install hated the USB audio device I had temporarily plugged in. Then I downloaded Windows 7 SP1, then I did the SDK install off this page.

The SDK install chugged along and eventually installed. Then I had a go at the MiniBrowser tutorial. That all went good until I went to run it in the emulator. The Windows Phone 7 emulator started up OK, but everytime I tried to get the mini browser to load a web page the screen was blank. Eventually I figured out that the warning that popped up before the emulator launched telling me that my graphics card was possibly not good enough actually meant that most of the emulator does not work at all. It seems that even though I had DirectX11, my graphics card was using WDDM 1.0 drivers and hence did not really want to work at all with the emulator (I reread the requirements which mentions that a WDDM 1.1 capable driver is required). I have an old Nvidia Geforce 7300 LE which I have always thought was overkill since I don't play any games on this system. However it is apparently not good enough to emulate a telephone ;-)

virsh - VM won't start

I have seen this quite a bit now. On my Scientific Linux 6.2 box running lots of KVM VM's I go to start up a VM using virsh and it doesn't start. I trawl through the VM's log in /var/log/libvirtd/qemu and the error is often a bit vague, but the most recent one said this;

Unknown ramblock "", cannot accept migration

It's kind of a useless piece of information until my brain clicks and remembers that the VM in question was running last time I shut down the server and SL's libvirt shutdown script tells all running VMs to effectively suspend. The theory is that when you reboot, the VM 'resumes' and is back where you left it. I find this resume is pretty much 50/50 that it actually works. I always end up dropping the suspend info and forcing the VM to do a clean boot. The magic virsh command to get it to drop the suspend info is;

managedsave-remove

So in virsh you do something like;

virsh# managedsave-remove <the-vm>

Then you should be able to start it;

virsh# start <the-vm>

It reminds me I need to work out how to get libvirt to try to shut the VMs down rather than suspending them.

Fixing AVIs for playback on Bravia TVs

So I bought a newish Sony Bravia TV (2011 model). It has all the Smart TV gumpf including ability to connect to a DLNA server in the house. So then I played around with linux based DLNA servers. Tried ushare, then finally settled on minidlna, though I need to reboot minidlna every now and then. But the bigger problem with DLNA streaming with the Bravia is that won't play all my AVIs. I've been using an old Xbox with XBMC for many years and it could play pretty much anything, but when I first tried the Bravia out, maybe 20% of my AVIs would spit up 'Playback not available' on the TV.

Being the curious type, I started to look at whether there was anything consistent with the AVIs that did not play. You can get mplayer to spit out the details with this;

mplayer -endpos 0 -vo null -ao null somevideo.avi

What I discovered is that the ones that would not play all had a commonality to do with the AUDIO codec information. For example if you grep for AUDIO in the output of the mplayer example above, you might get something like;

AUDIO: 48000 Hz, 2 ch, floatle, 128.0 kbit/4.17% (ratio: 16000->384000)

OR

AUDIO: 48000 Hz, 2 ch, floatle, 127.4 kbit/4.15% (ratio: 15927->384000)

All the ones that the Bravia could not play had a sort of exact whole number kbit rate. eg. the AVI with 128.0 kbit in the AUDIO codec info would not play, but the one with 127.4 kbit would play. I found other numbers that seemingly would deter the Bravia from playback; 112.0 kbit and possibly 96.0 kbit.

So then I tried using mencoder in copy mode to see whether that would 'clean up' the AVI files to make them playable and sure enough it actually worked. The Bravia could play them.

mencoder -ovc copy -oac copy -o somevideo-modded.avi somevideo.avi

mencoder in copy mode is incredibly quick. It only takes a few seconds on a 350MB AVI file on a modern machine.

I have actually found a few other AVI's that don't have a magic whole number for the AUDIO codec info and they don't play on the Bravia for some reason .... but running them through mencoder in copy mode does seem to fix them as well.

Playing with dirty_ratio and dirty_background_ratio

So I have a linux KVM server (SL 6.2 64 bit) that is a bit old and has hardware limits that prevent it having more than 3GB of RAM. It has one of those Seagate XT drives in it that has a SSD cache. With quite a few VMs running where the total sum of the RAM allowances for these VMs is bigger than 3GB, then the performance of the box is rubbish. Lots of stalls and I end up shutting down an idle VM to try and get it useful again. What was I thinking? Overcommitting and it falls apart? ;-)

If I run something like atop while the box is thrashing you can see that its used up all its RAM and there is a lot of paging activity. The thing is, compared to other OS's linux tends to try and push pages that haven't been used for a while to disk, and it has two thresholds; dirty_background_ratio and dirty_ratio ; The background one is sort of the percentage of dirty pages in RAM that you have to have before the VM starts flushing them out. The latter 'dirty_ratio' is a higher percentage threshold. If the amount of dirty pages reaches that percentage then the kernel will block processes from executing until the dirty_ratio gets below that percentage. So hitting the higher percentage is kinda irritating.

There's a good article at rackerhacker about these settings, and there is a note at the end about using 1% as the lower threshold and 100% as the higher one (with the caveat that you really should have a good power supply and RAID card). It might seem weird to have 1% as the lower threshold, as this logically increases disk activity, but with a memory constrained machine you want to start pushing dirty pages out sooner, and having the higher threshold as high as possible means you're telling the OS "Please don't block processes unless its an absolute disaster".

So me, attempting to be marginally conservative tried 3% and 80% for dirty_background_ratio and dirty_ratio respectively and hey it does make quite a difference. A lot of people tend to play with the swappiness setting too, but I've let it as the default of 60%

This article is also good

2.6 kernel on old 2.4 system

Ok, for whatever reasons I had an old Redhat 9.0 system (kernel 2.4.20 or thereabouts) that I wanted to put a 2.6 kernel on. Now Redhat 9.0 is really old. Like 'last century' old. There are a few articles on the net about migrating Redhat 9.0 to a 2.6 kernel including this one, but generally its hard to find decent info.

Anyway, I followed that article but took a stab and downloaded kernel linux-2.6.27.62.tar.bz2. I ended up compiling and installing module-init-tools-3.4.1 as its hard to find really old copies of module-init-tools. I gave up finding the right mkinitrd that might work, and (as per that article) decided I would forgo having an initrd.

Anyway the basic deal is to compile the kernel like so;

cd /usr/src

tar xvjf ~blah/linux-2.6.27.62.tar.bz2

cd linux-2.6.27.62

make mrproper

make oldconfig # I just pretty much held down the ENTER key to take the default answers

make menuconfig # adjusted and disabled a lot of stuff. But I made sure that loopback, ext3 journalling and my network card driver were all part of the kernel (ie. not modules)

make # this is the long bit

make modules_install

# The make install seems to install the kernel and an initd into /boot, but I could never get the initrd to work right.

make install

# And then vi /boot/grub/menu.lst and adjust so there is no initrd line and the root device is hard coded.

#

title Red Hat Linux (2.6.27.62)

root (hd0,0)

kernel /vmlinuz-2.6.27.62 ro root=/dev/hda2 debug

And now I reboot, and make sure to select the 2.6.27 kernel and it starts to boot and seems to mount the root filesystem, but then gets some weird panic about 'init'. I did the usual googling and trawling forums ... and decided "I'll try an older kernel".

So I download kernel 2.6.18, go through all the same setup again, compile it, install it and reboot .... and this time it works fine. There must be something specific in more recent kernels that disagrees with an old 2.4 based system.

Now, a missing piece is that this system is running as a virtual instance. From experience, I know that linux 2.4 and early 2.6 based systems do not virtualise well. They tend to suffer from bad clock drift problems (google for 'vmware timekeeping' for an explanation). If you are trying to virtualise one of these older systems and it contains a database then you are asking for trouble as databases do not like non-linear time. So, initially I did virtualise this Redhat 9.0 system on KVM. You could tell something was not quite right immediately, as the KVM host system was showing 10% CPU use for this virtual machine even though inside the virtual machine there was nothing happening. This same virtual machine, upgraded to a 2.6.18 kernel is lucky to hit 1% CPU use while idle. Note: I am also using a 100Hz tick on this 2.6.18 kernel and so far the timekeeping seems OK (but generally the time keeping problems only really occur when the VM is under load).

Linux LVM2 and md arrays

For some time now, I've been running a debian server with several disks in a RAID5 md array with LVM2 layered over the top. At the time this seemed like the usual path that people were taking; ie. Use the md tools to create your redundant drive array, and use LVM2 over the top to get a flexible volume management system. Coming from a Veritas Volume Manager background, this always seemed a bit dodgy (Veritas manages the redundancy and the logical volumes). However, this setup has proved reliable and straight forward to work on. Admitedly I haven't had a drive failure, but I've never had a situation where the LVM2 or md information needs some form of repair.

I still don't mirror my root filesystem ... but thats a personal choice.

At the time I setup this disk array, there were some notes indicating that the Linux device-mapper stuff (which effectively implements the core of LVM2) would implement mirroring. It looks like that is the case now, though it's not exactly mentioned in many places on the net. The main resource seems to be redhat who I think maintain the LVM2 stuff.

Anyway, you can now do mirroring without an underlying md array. It's really quite simple. If you have another drive you need to pvcreate at least a partition on it and then vgextend and finally lvconvert to do the actual mirroring. Here's an example from an LVM'd debian setup

pvcreate /dev/hdd1
vgextend debian /dev/hdd1
lvconvert --corelog -m1 /dev/debian/usr

Not too sure what the corelog does. I kept on getting an insufficient extents error until I added it. You can do a 'lvs' to see the progress of the mirror. It works pretty well. Like most LVM operations you don't need to umount anything while they're running.

But you still need to use md if you want RAID5. I don't think the device mapper implements RAID5 yet. One thing that is good about current (post kernel 2.6.16) md setups is that you can add a disk to a RAID5 array and dynamically expand it (google for mdadm raid5 reshaping). If you understand how RAID5 works, you'll realise that adding a disk is a complicated process involving restriping data across all the disks in the array. When I set up my array this reshaping feature was not available so I deliberately split each drive up into about 5 or 6 partitions to create multiple RAID5 instances .... which suffice to say allows me to add another drive and reshape it myself (it's a long process, but doable)