Experiences in the community

Just another WordPress.com weblog

Jigdo-Jigsaw Downloader-II

This post continues where I left off with Jigdo-Jigsaw Downloader for downloading large image files (read ISO9660 DVDs or BD) as well as freshen it with a changing source.

First order of the day, A very Happy New Year to all of you. I hope you are at peace and may prosperity shine in whatever you do.

Now, getting back to where we were, the last time we were talking of this, we had done all the initial work and were just waiting for .jigdo to finish it work . Depending on your connection and your patience this could be from matter of hours to matter of days. I, as experience and as an e.g. took about 4 days to download the weekly Debian Squeeze DVD .iso file through jigdo . There are many reasons why it could have happened but it can happen.

Alternative Mirror :-

Anyways, here’s the next part. And this is also pretty interesting. Many a times for some reason or the other you will not able to get the same server . Either the server may be down or some link/path between you and server is not up. Although there is redundancy built within the network but there is a possibility and sometimes stuff happens. If something like that did happen, this is jigdo solves it :-

FINISHED --2010-12-24 00:11:19--
Downloaded: 3 files, 746K in 21s (35.5 KB/s)
Found 3 of the 4 files required by the template
Copied input files to temporary file `debian-testing-amd64-DVD-1.iso.tmp'
- repeat command and supply more files to continue

---------------------------------------------------
1 files not found in previous pass,trying
alternative download locations: --2010-12-24 00:11:19--
http://us.cdimage.debian.org/cdimage/snapshot/Debian/pool/main/h/hw-detect/disk-detect_1.82_all.udeb

Resolving us.cdimage.debian.org... 83.142.228.128, 2001:470:1f08:80b::2
Connecting to us.cdimage.debian.org|83.142.228.128|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 22044 (22K)[text/plain]
Saving to: `debian-testing-amd64-DVD-1.iso.tmpdir/us.cdimage.debian.org/cdimage/snapshot/Debian/pool/main/h/hw-detect/disk-detect_1.82_all.udeb'

Now this is clearly interesting. From where did it get that info. While I cannot confirm, I do see it happening through only one way, the .template file. While I cannot read that is the only thing that makes sense.

Lastly, when everything is good and proper you get a success message like this :-

100%[============================>] 22,044
32.1K/s   in 0.7s

2010-12-24 00:11:36 (32.1 KB/s) - `debian-testing-amd64-DVD-1.iso.tmpdir/us.cdimage.debian.org/cdimage/snapshot/Debian/pool/main/h/hw-detect/disk-detect_1.82_all.udeb' saved [22044/22044]
Found 1 of the 1 files required by the template
Successfully created `debian-testing-amd64-DVD-1.iso'
-----------------------------------------------------------------
Finished!
The fact that you got this far is a strong indication that `debian-testing-amd64-DVD-1.iso'was generated correctly. I will perform an additional, final check,which you can interrupt safely with Ctrl-C if you do not want to wait.

OK: Checksums match, image is good!

Now one can take a big sigh of relief. Our first work is done. This is the same thing that is or can be achieved by using a download manager, by a torrent and by every other way than why should I use jigdo ?

The answer lies in the next scheme of things which others cannot do, except for another free software tool called Rsync which we can talk about sometime later as well.

Now let’s say the next week you want to/have to download the next weekly to keep things current/flowing. Especially in these times where distributed version control systems are the norm, software changes quickly, new packages, updates and upgrades to packages, bug fixes a lot can change in a week. Now does that mean I would have to download the .iso all over again.

While in other tools yes, in jigsaw its simple. Just take some precautions and you are good to go. Some golden rules can take you a long way .

a. Always download in a new directory which is going to be used for that download
b.Never panic
c.Think 10 times before deleting a file.

Ok, with those ideas let’s see how are we going to freshen an archive, for our e.g. we will take the same .iso file only this time the date of the archive will be the upcoming 30/12/10 which happened few days back.

Simply do the following :-

$mkdir jigdo301210

$ls | grep jigdo271210
jigdo271210

$cd jigdo271210

~/jigdo271210$ ls -lh
total 4.4G
-rw-r--r-- 1 shirish shirish 1.4K Dec 28 20:43 debian
-rw-r--r-- 1 shirish shirish 4.4G Dec 28 21:24 debian-testing-amd64-DVD-1.iso
-rw-r--r-- 1 shirish shirish 116K Dec 28 02:04 debian-testing-amd64-DVD-1.jigdo
-rw-r--r-- 1 shirish shirish 18M Dec 28 02:04 debian-testing-amd64-DVD-1.template
-rw-r--r-- 1 shirish shirish 1.9M Dec 28 20:44 jigdo-file-cache.db

~/jigdo271210$ sudo mount debian-testing-amd64-DVD-1.iso /mnt/jigdo -0 loop
[sudo] password for shirish:

~/jigdo271210$ cd ../jigdo301210

~/jigdo301210$ jigdo-lite

Explanation of the Mounting process

Let’s run through those commands again so we explicitly are clear as to what we are doing.

$mkdir jigdo301210

The above is pretty straight-forward, we are simply making a directory. It is just like going to Nautilus, and being in some directory or even at the base and right-clicking and saying new folder. Folder and directory in Debian are one and the same.

$ls jigdo271210
jigdo271210

~/jigdo271210$ ls -lh
total 4.4G
-rw-r--r-- 1 shirish shirish 1.4K Dec 28 20:43 debian
-rw-r--r-- 1 shirish shirish 4.4G Dec 28 21:24 debian-testing-amd64-DVD-1.iso
-rw-r--r-- 1 shirish shirish 116K Dec 28 02:04 debian-testing-amd64-DVD-1.jigdo
-rw-r--r-- 1 shirish shirish 18M Dec 28 02:04 debian-testing-amd64-DVD-1.template
-rw-r--r-- 1 shirish shirish 1.9M Dec 28 20:44 jigdo-file-cache.db

This is again pretty simple. We are just listing the contents of the directory.

~/jigdo271210$ sudo mount debian-testing-amd64-DVD-1.iso /mnt/jigdo -0 loop
[sudo] password for shirish:

Ok, this one is a bit interesting. What we are doing here is known as mounting a loop device or simply stating we are doing a DVD drive emulation. It’s known as Virtual CD or Virtual Drive

This one is also simple as well. As we are at the base we do know that we made a directory last week. So just checking if the directory is there. One of the things you can do in most of the shells is just do something like the following :-

$ls jigdo- [TAB key ]

The tab key will simply show the files and directories which start with “jig” . This feature is known as tab-completion and is one of the most essential features which comes pretty handy when working on CLI (Command-Line Interface) . In this case the output would be :-

$ls jigdo [TAB key ]
jigdo271210/ jigdo301210//

Simply add couple of letters so that its unique and press tab key and it autocompletes, pretty nifty eh!

Anyways back to the job on hand.

$cd jigdo271210

Again simple, we are just changing/entering into the directory we made the last week.

~/jigdo271210$ sudo mount debian-testing-amd64-DVD-1.iso /mnt/jigdo -0 loop
[sudo] password for shirish:

Ok, this is somewhat interesting to share. What I am doing is known as ‘mounting a loop device. In most of GNU/Linux OSes and Macintosh you have this feature in-built. What we are doing is instead of burning the DVD on DVD media, we simply emulate or virtualize the operation that a DVD media does. Look at Virtual Drive for more explanations.

$cd ../jigdo301210

This is again nothing great. This is simply a shorter way of going to the new directory jigdo301210 .

~./jigdo301210$ls
~./jigdo301210$

We are simply checking that there is nothing in that directory. Nothing should be. It should be empty. We are on the last few steps.

~./jigdo301210$jigdo-lite

Ok, just like before we did, run jigdo-lite and make sure to give path of the .iso we mounted when it asks if there is some media we want to check. Either download the .jigdo file by http means (using web browser etc.) or use jigdo for both up to you. It will diff/check against the list and md5sum given in the .jigdo and .template files and give you a result something similar to this

Found 3732 of the 3861 files required by the template
Copied input files to temporary file `debian-testing-amd64-DVD-1.iso.tmp' - repeat command and supply more files to continue

-----------------------------------------------------------------

If you get something similar like the above, this means all is well. Just list the directory and see that an .iso.tmp file is created with today date. If that is so, then only three things left to be done.

First of all, immediately after you get that message, stop the process. Use CTRL+C combo to cancel the operation.Then umount the old .iso for whatever mount point you created in this case /mnt/jigdo and mount the new .iso.tmp file in its place and re-run jigdo-lite again using the same process as we did above as well as we did the last week.

~./jigdo301210$sudo umount /mnt/jigdo

~./jigdo301210$sudo mountdebian-testing-amd64-DVD-1.iso.tmp /mnt/jigdo

~./jigdo301210$jigdo-lite

Now lastly, depending on your disk space you can remove the old image from your hdd.

$rm -rf jigdo271210

While it would download the remaining files, my job is done. This really is a pretty useful tool to have to freshen up archives, do incremental backups and lots of other things which perhaps I may not also know or imagine. At the end, it’s just a tool we need our own imagination to use it.

Edit:- The last part in the series is put up here .


Single Post Navigation

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: