Archive for the 'UNIX' Category

CHM readers for Linux

Thursday, October 16th, 2008

The other day I needed a CHM reader. CHM files are Microsoft HTML compiled files and I knew getting a reader on Linux might be some what tricky.

After trying to open the file in Firefox (I thought I might get something after all it is ‘HTML’) and failing I realised that there was most likely a Firefox plugin for CHM functionality. Indeed there is, https://addons.mozilla.org/en-US/firefox/addon/3235

Automate Rapidshare free downloads - No more captchas

Monday, July 14th, 2008

Earlier this month the popular file sharing service Rapidshare decided to drop the use of captcha’s. This came as a huge surprise to myself because previously Rapidshare has used some of the most convaluded captcha puzzles ever! At points I would be Lucky to get 25% of captures used by Rapidshare right.

Dropping the use of captcha’s has paved the way for automated downloads but Rapidshare are aware of this and has lowered the download speed of free users to 500 kilobit per second.

Never less I would rather the lower speed and with the capability to have the damn thing automated. So here it is, my simple bash script to download a batch of files of Rapidshare. Hell if you only want to download 1 file its only a “one” linner.

URL=$(curl -s <URL TO RAPIDSHARE FILE>| grep "<form id=\"ff\" action=\"" | grep -o ‘http://[^"]*rar’);ourfile=$(curl -s -d "dl.start=Free" "$URL" | grep "document.dlf.action=" | grep -o ‘http://[^"]*rar’ | head -n 1); sleep 90;wget $ourfile;

Full version to download a batch of files specified in a file called input.txt, 1 file per line.

New Version (1/11/2008): Rapidshare has added a wait time in between file downloads. On top of your download
to start. This has been fixed.

Download the full .sh with some enhancments made by Tune. Also see comments for a Python version by Conman.

Download Itay’s version with some nice features. See his comment for the features.

while read line
do

URL=$(curl -s $line | grep "<form id=\"ff\" action=\"" | grep -o ‘http://[^"]*rar’);
ourfile=$(curl -s -d "dl.start=Free" "$URL" | grep "document.dlf.action=" | grep -o ‘http://[^"]*rar’ | head -n 1);
sleep 90; #70 secs for 200mb 50 secs for 90mb
wget $ourfile;

done < input.txt

Analysis

while read line
do
Code……………..
done < input.txt

Loop through all lines in the file input.txt, store each line in a variable called ‘line’, move to the next line.

URL=$(curl -s $line | grep "<form id=\"ff\" action=\"" | grep -o ‘http://[^"]*rar’);

Use curl to ‘download’ the page of the url we are processing. Pipe the downloaded pages source code through some greps to extract the action URL of the html form which we need to ’submit’ to trigger the download. Store this url in a variable called ‘URL’.

ourfile=$(curl -s -d "dl.start=Free" "$URL" | grep "document.dlf.action=" | grep -o ‘http://[^"]*rar’ | head -n 1);

Using curl again, ’submit’ a html form to our extracted URL, passing it the post data (-d switch in curl) of being a free user. Rapidshare then replies with a new page with a list of Rapidshare servers to download from, they are stored in lines of Javascript starting with ‘document.dlf.action=’. After the equal sign is the direct url to our file, so lets grab this with another grep!

We are now have a list all the Rapidshare servers holding our file like this.

http://rs202l3.rapidshare.com/files/50141870/1554876/centos-5.0.tar.part01.rar
http://rs202l33.rapidshare.com/files/50141870/1554876/centos-5.0.tar.part01.rar
http://rs202tl3.rapidshare.com/files/50141870/1554876/centos-5.0.tar.part01.rar
http://rs202cg.rapidshare.com/files/50141870/1554876/centos-5.0.tar.part01.rar
http://rs202l32.rapidshare.com/files/50141870/1554876/centos-5.0.tar.part01.rar
etc…..

We will try our luck with the first one, so filter this output through ‘head’ telling it to extract one line (head -n 1)

sleep 90;

Rapidshare still makes us wait for our file! So, we will just wait here for however long it takes which is 70 secs for a 200mb (new file size limit) 50 secs for 90mb (old file size limit).

wget $ourfile;

Finally the wait is over, luckily its the script doing the waiting, not us. Lets get our file. wget our file that is…

Closing comments, if you know how to improve the script please let me know. At the moment it will only work with .rar files though you could easily change this. A generic version would be nice but this is what most files are in on RS.

A version which uses proxys to download files in parallel would be good.

I dont like piping a grep command into a grep command so I’m sure this could be done with just one grep.

I use use both wget and curl, which here accentually do the same job, if you have one or the other you can fix the script up to only use either.

Tip: Set your batch download up an in a screen session and detach it so you can ssh in and monitor your downloads where ever you go. This will also let your downloads continue downloading when you close your terminal window or putty sesion.

Programs needed: Bash (duh!), head, grep, sleep and curl or wget

Note: I have required to increase the sleep time to 90 seconds. I am unsure how to calculate this time because it has seemed to change from when I first wrote this. It may well be RS has increased the time or the value is variable depending on how much you have previously downloaded. So 90 seconds seems like a safe value for now.

ABreakoutC - Another break out clone

Sunday, August 26th, 2007

Today I have finished the first release of a Breakout clone I have been working on for PC (Linux / Windows / PC).

If you dont know what Breakout is; it was one of the first arcade games ever made. It involves hitting a ball at blocks using a paddle which you can move left and right. Breaking all the blocks progresses you to the next level. Like latter versions of breakout, ABreakoutC includes power-ups which add special effects to the game.

ABreakoutC was my first game written in c++ using SDL which is a cross platform game/media library.

It and its source are released under the GPL3 and the graphics were made in entirely in GIMP.

The game was developed in Linux with the help of the Netbeans (best IDE in the world) c++ plugin.

Game Objects

abreakoutcitems.png

Controls

Pause/Resume - P
Start the ball moving - Space bar
Move paddle - Left key, right key

Screen shots

abreakoutc1.png abreakoutc2.png abreakoutc3.png

Downloads

Windows
x86Linux-32
Source

System requirments

1Ghz or better Celeron. 128 Meg or ram.
Note - You cant go higher than 60fps

If you want to design your own levels (let me know and I will talk you through it), provide music to the game or even add a power-up your welcome to. Getting a few people working on this would be great. Also I would like someone to do a Visual c++ build as mine is only MinGW.

———Update!———

If you have previous downloaded any version of the game please get version 002 (from the above locations) there was a bug which will cause what seems like a random crash on Windows which is now fixed.

For those that are interested it was a memory violation caused by…
if(ds[i]->blockDestroySprite1->status == Sprite::FINISHED){
delete ds[i]->blockDestroySprite1;
ds.erase(ds.begin() + i);
}

gs->apply_surface(ds[i]->location_X,ds[i]->location_Y,
ds[i]->blockDestroySprite1->spritesheet,ds[i]->blockDestroySprite1->getFrame());

As you can see I was deleting the sprite then trying to draw the sprite on certain instances. I have no idea why this error didnt occur on Linux (where I did all of my testing). I have now fixed all versions to delete the sprite after it is drawn.

It makes very little sense but Im not going to complain.

Monday, August 6th, 2007

Well today started out great.

Today was a ‘Bank holiday’ for NSW Banks and because I work in the Insurance industry I guess I got the day of as well. It makes very little sense but Im not going to complain.

All was going good until some time around 4:00pm I decided to run a program I hadnt worked on in a few weeks. The problem was it didnt work and not only did it not work it wouldnt go away and sat on 100% CPU.

So anyway I killed it. But the problem wasnt it, it was some other process it had spawned which was now running at 100% and owned by root.

So I logged on as root and not even thinking of the effects I kill -9ed it. At this point the kernel pretty much crashed. I had to hard reboot.

BIOS loaded. Grub tried to load. But failed. It failed with the Error: ‘error #17 cannot mount selected partition’. Great. Anyway, I pulled out my now trusty Live Distro SAM Linux and booted it. I went in to inspect my partitions all was fine. WTF? I inspected my grub.config. No issue there.

Then the magic happed. I shut down my PC (properly this time) and took out SAM and rebooted.

All was fine. It makes very little sense but Im not going to complain.

SAM Linux - Review

Saturday, July 7th, 2007

SAM Linux Live Distro.

Ok lets begin from the start. I insert the Live CD and boot. I come to the boot menu which gives me several options including to boot SAM Linux, boot into console mode, memory tests and a few other options.

Login user names and passwords are shown on the boot menu for both a normal user and root user which is good because on of the first live cds I tried I could never find the root password. Its great there is a boot to console mode, it cuts down on memory and loads a little faster. Also half the reason you want a live CD is so you can quickly go and repair a setting in VI (Emacs isn’t included) or backup a some files to a FTP server or ssh account or even a removable drive. So you can then format the non bootable Windows PC and reinstall an an OS again (hopefully Linux this time ;) but more on these things latter.

One thing that was disappointing is the network configuration Wizard isn’t included if you boot to a console only environment which means depending on your network type and network interface getting a connection could mean allot of configuration.

Also a nice feature was the ability to hide and show the kernel loading and the systems boot process by simply toggling it on and off with the escape key. Ive used some distros that try to be more user friendly by hiding this information but when something goes wrong like a drive doesn’t mount your left wondering why.

Once your logged in and the intro music plays your left with the Xfce Windows manager. Which I like to think is a light weight Gnome. I say this because Xfce 4 is based of the Gnome tool kit 2 (Gtk2). Xfce is perfect for a live cd because its not only light but a very powerful desktop manager.

Network

Before you get to the desktop you can opt to set up a network adapter through a convenient wizard. The wizard interestingly had options for GPRS, Edge and 3G which is I believe are wireless cell phone transport technologys. There were also for options for IDSN, modem, Ethernet, DSL and even Blue tooth! I also have a good feeling that wireless connections wouldnt be to difficult to get running provided you had the correct Windows driver.

Network tools

SAM comes with several networking tools which above others includes Firefox 2, gFTP, ssh (must have for me)/ Putty (which seems to be emulated through WINE), TightVNC, Skype, Gaim (must have for me), Xchat and Gnutella client (I would of preferred a Bittorrent client).

Multimedia

I dont really expect much multimedia capability’s to be on a live CD but SAM comes packed full of options. My media player of choice is Mplayer which as a bonus comes with a nice GUI. I was able to watch every multimedia file on the PC including a xvid/divx encoded movie and play a WMA encoded song with no extra codecs required, I would be interested to know if commercial DVD playback works straight out of the box as well and wasnt able to check this out as the live cd occupied my only drive. As for Mp3 players both Winamp(XMMS) and Itunes (Exaile) clones/alternatives are included.

Development / programming

This is a interesting section. Im in two minds weather you actually need development tools on a live CD. But as myself being a programmer I’m going to say its a good thing and here’s why.

Perl - Yes Perl is included, Perl is an amazing powerful OS scripting language and can be used to automate a complex task.

GCC - (Gnu Compiler Collection) I guess this could come in handy if you need to do some quick hacking.

Python - I think this was included primarily for Xchat. But hey its there! I even use a few Python programs my self Shuffle-db http://shuffle-db.sourceforge.net so I cant complain.

IDE - Disappointingly there is no good IDE and not even a editor with syntax highlighting included. On top of this the GNU debugger (GDB) isnt even included. Having said that, there is ‘Bluefish’ which seems to be a good web development IDE.

Windows support

As I mentioned earlier one thing I use a live cd for is to resolve issues with a computer which is either miss-configured or non booting or even to get important files before the final solution of a system wipe.

SAM Linux makes this ideal with its many networking applications and support for Windows programs and partitions.

Wine (the Windows emulator (sic)) is included which allows you to run handy Win32 system applications such as Regedit.exe to get rid of that malicious software or fix miss configured setting.

It also comes with fat32 read/write and NTFS read support.

Eye candy.

If you wish SAM Linux gives you the option of turning on Beryl and Compiz which gives you desktop effects that out amaze even that seen in MS Vista. Whats even move amazing is I didnt even have to install any ATI graphics drivers for my 9600 pro GFX card. This allowed me to play the 3D OpenGL game ‘Warsow’ which I had on another partition straight out of the box!

Speaking of FPS games - Tilda is also included, which is a Terminal emulator which works like the consoles in First Person Shooter games. So when you press the ‘~’ key a terminal scrolls down where you can execute bash commands and pressing ‘~’ again will send it scrolling back out of the way until you need it again.

Other notable programs

JAVA, The full Open Office 2 suit (which im using now), Gimp, Gnome Baker (CD/DVD burning) a PDF viewer and a archiving tool. Even the Flash player is installed.

Disk/Partition management

SAM Linux also comes with an Disk partitioning tool which can handle many types of file system which is always useful on a live cd.

System/Software management

System management is basically identical to Mandriva’s. Which I have found to be probably the best out of any distros.

Software management takes place in Synaptic which is a computer user interface and front-end to the Advanced Packaging Tool. I am still unsure how difficult it would be to get packaged software for PCLinux (which same Linux) hopefully you would be able to use the Mandriva packages because PCLinux is based of Mandriva. If this is not the case finding aditonal software would be a problem until the distribution becomes more popular.

Final comments

SAM Linux is a very exceptional live CD and would make and very nice light Linux OS if installed.

The inclusion of ‘commercial’ software such as flash and 3D GFX card drivers make the experience even better. SAM linux dosnt excel in any one area which makes it a hugely versatile portable OS.

I could install SAM to my HDD right now and not need any more additional software packages apart from GDB and a good IDE but for 90% of uses they could use it straight out of the box.

The combination of Xfce and Beryl and Compiz make for a smooth experience on the front end yet SAM includes enough versatility behind the scenes to get the ‘real’ work done when needed.

Full metal racket mp3’s

Wednesday, May 2nd, 2007

I have set up a process to record the weekly Australian heavy metal radio ‘Full metal racket” on JJJ which also broadcasts over the internet. I did this using Cron, Bash scripting, Mplayer using the -streamdump switch and PHP. Then I host the Mp3 files on my site for download on IPods or portable mp3 players for listening at our own leisure.
For those that are interested this is the bash script called DownloadFMR.sh:

1) #!/bin/sh
2) mp3filename=Full_Metal_Racket-`date +%y%m%d`
3) /home/slith/apps/MPlayer-1.0rc1/mplayer http://202.6.74.107:8060/triplej.mp3 -dumpstream -dumpfile /home/slith/emkay.unpointless.com/FMR/${mp3filename}.mp3

Here is my crontab; maybe theres a better way of stoping the recording than killing all instances of mplayer. Does anyone know?

0 5 * * 2 /home/slith/scripts/DownloadFMR.sh > /home/slith/mylogs/FMR.log
0 8 * * 2 kill -9 `ps -ef | grep DownloadFMR.sh | grep -v grep | awk ‘{print $2}’`
0 8 * * 2 kill -9 `ps -ef | grep mplayer | grep -v grep | awk ‘{print $2}’`

If all you care about is downloading the shows in Mp3 format you can find these here.

Full metal racket Slayer