The long tail in a common’s man journey to debconf16 – 2

This is an extension of part 1 which I shared few days ago. This would be a longish one so please bear.

First of all somebody emailed me this link so in the future a layover at Doha Airport will be a bit expensive from before, approx INR 700/- added to the ticket costs 😦

Moving on, Let me share an experience I shared one of the last few days I had while I was in Cape Town –

Singer singing some great oldies from 60's , 70's till 90's.
Singer singing some great oldies from 60’s , 70’s till 90’s.

I had booked a place near Long Street, Cape Town using Bernelle’s help. What I had not known at that time that near Long Street there are free walking tours every couple of hours. I took part in all the tours and those were nice experiences. Where they start the walk, there was the gentleman pictured above. I was amazed by this gentleman’s rich voice. He strummed lot of classics from the 60’s, 70’s till the 90’s . I had two coffees and thought I was at a premium rock concert. It was a bitter-sweet experience for me because I could see that he has such prodigious talent and still he had to struggle to survive to make ends meet. I did my 2 bit but wish I could have done something more.

Side note – Before I forget there is one trick of feh which I use to view images without it getting very high-resolution (especially on my low-end systems) –

┌─[shirish@debian] - [/run/user/1000/gvfs/mtp:host=%5Busb%3A001%2C006%5D/Card/DCIM/Camera] - [4621]
└─[$] feh -g 1350x1000 .

This actually makes it far far easier to traverse through the 1000 odd photos of the trip that I have in my personal archive without doing any sort of conversion methodology. Btw, it took me time but finally was able to create an album at gallery.debconf.org . Haven’t been able to upload photos as came across an error which I have shared at https://lists.debconf.org/lurker/message/20161113.215659.fce58823.en.html

Moving on, here’s the funny story/experience I wanted to share –

could have been arrested ;)

What happened was this. This is from the Doha Airport. I had seen big buggies (ones similar to golf carts) which was ferrying people from end of the concourse to the other. I had been walking the whole day and even with the horizontal escalators and everything, it takes a toll. I was half-tired, half-sleepy and saw a buggy stationed. From behind it looked like the buggies I had seen. As there was no place to park my behind there, I entered into the buggy and sat there. Around 15-20 minutes later a Doha cop in another buggy came to me and asked me if something had happened ?

I had no clue what he was talking about. He told/shared/asked me in friendly tone whether I had committed a crime or wanted to report a crime. When I replied in negative to both, he asked then why I was sitting there. I replied it was for stretching my legs and it was the buggy which was being used to transport people from A. to B. He gently told me I had entered into the wrong one and it was actually a cop buggy. I couldn’t believe it. He did go his own way as he saw I was dead-tired. After 10-15 minutes, half-believingly I came out of the buggy and to my shock the gentleman was right. There was nothing to do but solder on to find a spot in this big airport. I shared this with few friends and family and managed to elicit few laughs hence sharing.

The somewhat sad one was I had met a couple with a baby. Now as shared before, Most Airports including the Doha Airport is Air-Conditioned/Climate-Controlled and is probably in mid-20’s so it was more than cold for me. The couples with the baby were from Asian sub-continent. From their clothes and the way they were, they were not very well off. I do remember them sharing that they had a death in the family and hence were going. I didn’t know at that point in time that there was something called bereavement fares and if they were able to take opportunity of those tickets. But this is besides the point . The issue was that their baby had been running a high-fever and the A/C was making matters worse. I had seen a pharmacy but no clinic in the airport. It was much later I came across http://dohahamadairport.com/airport-guide/facilities-services/medical-emergencies but as can be seen on the web-page it doesn’t tell whether the services are chargeable or not. I assume it would be paid, although in some of the ‘developed/industrialized’ countries it is rumoured not to be for simple ailments such as the baby was going through. Have no idea if that’s true or not. I also don’t know how it equates with travel insurance as well as most travel insurance is also supposed to help you in situations like these. I was concerned as it was a baby and babies as all know are very very fragile. If anybody has an idea or had similar experience would like to know specifically related to International Airport environment as it has ‘transit’ issues unlike in domestic airports where I don’t think it would be a bit more easy.

Now coming to my own inadequacies/lack of foresight which I had mentioned I will share, I had asked/queried and got to lead a Debian-installation workshop on the Open Debian Day. I had done a few earlier and had installed it a few times on my system and for my friends, relatives and some clients. The only bad experiences I had were to do with UEFI but even those in the jessie releases had got resolved quite a bit, so was pretty confident. The day before the Installfest was to happen, ‘Mensah Nyarko Yaa Dufie’ (one full name) of Ghana approached me to install Debian on her system. I had some older version of the Debian DVD either 8.1 or 8.3 and had known that 8.5 had been released just a few days back. Had seen pretty fast internet (as far as downloading Debian DVD) is concerned hence asked her to wait a bit while I downloaded the newest image. I sha256summed it to make sure that the image was bit-to-bit perfect.

Now I hadn’t bought a pen drive/disk from India as I was under the impression that in such conferences, pen drives should not be an issue. I had asked Bernelle privately before via e-mail as well and she had assured me that some pen-drives would be available. She gave me a handful of HP pen drives. The pen drives as we came know during our usage were somewhat flaky. It would pop out/lose connection even with the slightest nudge to the lappy.

Somehow I was able to transfer the image to the usb disk. As people say hindsight is 50:50 maybe it was not such a smart move on my part to download the big DVD image and maybe I should have got the netinstall iso . Be careful, the link I have just shared is of the old version, if you have good web link and want to try the newest stable netinstall head to cdimage.debian.org . Apart from that goof-up I dunno (still) of anyway to know if a copy from an .iso image to usb was successful or not and did it do correctly –

I did the following command –

sudo dd if=/path_to/debian-dvd.iso of=/dev/usb-mount-point

which is usually /dev/sdb on all of my systems . Her system was a brand new HP (don’t remember the model details) which she had bought just a few weeks/months before debconf. We tried a few times but it failed at installing the boot-loader stage. I asked Ritesh Raj Saraff (a friend and DD) and while he had some ideas, none of them worked. Ritesh later pointed out Steve McIntyre and shared he is part of the Debian-Installer team. At that point in time, I had no clue who Steve McIntyre was otherwise I probably would not have approached him. He quickly acquiesced to my request and shared that he would be there for the workshop. With load of my mind little bit, I apologized to mensah and asked her to be at the workshop the following day. I had no clue what was wrong at this point in time, whether it was the iso image in the usb disk or a UEFI issue. This also wasn’t good for my confidence but as somebody from the Debian-Installer team was there, I was somewhat relaxed.

Next day, some more people came for the Installfest. While I had made 2-3 copies, clearly it was not enough as more people came. I was in a frenzy and asked Deven Bansod, Keerthana Krishnan, Prabaharan Jaminy (the whole GSOC and Outreachy attendees) to volunteer to help out in making more iso images on usb disks. I introduced mensah to Steve McIntyre and we tried 2-3 times to get debian installed on the system but it didn’t move from the same place. Ritesh shared that dd had a memory leak and hence cat was a better way to do it. So we did –

$ cat debian.iso > /dev/sdb
and soon other machines had debian sporting on their desktops.

But mensa’s lappy wouldn’t get move from the boot-loader stage. Suddenly Steve had the bright idea (light bulb moment) that maybe the .iso is corrupted/usb disk is bad or something is incomplete. We started on another usb disk.

Now this is where I have a query – While I don’t want to compare, in Ubuntu there was an image self-checking mechanism where probably behind the scenes (backend) the checksums published in a file are compared with checksums generated by apps. which are on the .iso image. While it does extend your time, the end result is you know if there is some issue on the decompressed image on the usb disk. AFAIK we don’t have anything similar. The only two things I know is the wiki page and of course the various checksums of the image as shared at http://cdimage.debian.org/debian-cd/8.6.0/amd64/iso-cd/ or http://cdimage.debian.org/debian-cd/8.6.0/amd64/iso-dvd/

If anybody knows of any movement or a bug in the BTS which I can follow for the above issue please let me know.

This time Steve was able to install it without any issues. I asked him whether he had to make some specific FAT/Ex-FAT/NTFS partitions as some new UEFI-based lappies need one or more but he replied in the negative. While mensa did get her debian install, the GUI didn’t come while command-prompt was available. Then Steve added backports to the sources.list, got the new kernel, new Intel/Nvidia drivers (think it was one of those hybrid models IIRC) and she was able to boot into GNOME-Debian.

I didn’t saw any bug-reports about checksumming state of the applications before installation but did couple of reports about badblocks support and memory checking and from action on both bug-reports it is also need of the hour (although the earlier one has been marked as won’t fix :().

In this whole thing, I liked/appreciated the way Steve handled things, I intuitively understood/knew that he wasn’t just part of the Debian-installer team but someone better. I can’t explain it but it was there. A little investigation in the evening and it turned out that he had been Debian Project Leader for two consecutive years (2008 and 2009) . In hindsight it probably was a good thing I didn’t know that before otherwise I probably wouldn’t have interacted with him and it would have been my loss. To have been the DPL and still being so humble while technically being so proficient, I was amazed and didn’t know what to make of it.

Here i.e. in India, if somebody wins even the mohalla elections (neighbourhood elections) the person carries a big chip on her/is shoulder not just till he is on the seat but even beyond, and here was an example of a previous DPL asking time from one of the developers in a video if it’s possible in the next couple of days.

Lastly,last week have able to report 2 bugs upstream. The first one is of youtube-dl . It’s somewhat complicated hence will not go there atm. The second and more surprising one was from ‘nano’ our esteemed text-editor- Hopefully the bug will be fixed once a new version comes.

4 thoughts on “The long tail in a common’s man journey to debconf16 – 2

  1. Dear Ritesh, While I agree with verifying the integrity of the download, the problem lies when you have all sorts of combination of cheap, flaky usb cards and no knowledge when you do a dd or a cat or whatever that the image transferred, transferred properly or not.

    While you do have some ways in which you can ascertain the progress of the iso image on the usb card, there is no way to know the integrity of the now uncompressed packages, the installer, the works. Having some sort of checker, optionally to be run by the user would be better as he wouldn’t waste time to figure out what went wrong when an install is unsuccessful.

    If I have an integrity checker (something similar to what Ubuntu has/ad in their live CD to check the integrity of the live CD) would go a long way in keeping stress levels down when you know the image is good, especially if you are in low-bandwidth environments which you would find in most schools and small businesses.

    1. For flaky hardware, the tools in GNU/Linux are good enough. In common cases, the tools (cat/dd) will report you errors if the underneath Linux kernel senses one.

      On the other hand, if your payload already is tampered, the tools have nothing to do.

      As for the last part, about Ubuntu’s examples, I think we have much simpler approach.
      https://www.debian.org/CD/verify
      https://www.debian.org/CD/faq/#write-usb

      If you follow the links, you’d notice that we quote “dd” as the recommended example.
      I only suggested ‘cat’ because looking at the symptoms back then, I knew what the bug was. And ‘cat’ helps avoid that bug.

      But if your payload was corrupt already, irrespective of what tool you use, your installation may fail unpredictably, mostly with CRC errors.

    2. Ritesh,

      I think you have something there ” the tools (cat/dd) will report you errors if the underneath Linux kernel senses one.” – Ritesh

      This will happen only if the flaky hardware is not getting the bits due to bad surface but I dunno if cat/dd does do for instance simple checksums to ensure that the payload went through perfectly or not.

      What would be a much better option as I shared before is a checksum (say SHA-256 or better) which you could optionally run before running the whole gamut of debian-installer screens. Yes, this would need and involve somebody who has knowledge of d=i internals but if we have it, there is possibility of less chances of failure.

      On a side note, have been seeing lots of cheap chinese as well as ‘duplicate’ usb disks flooding the market. Having a bit of optional redundancy built-in should help our cause/case.

  2. Verifying the integrity of the download (in this case, an ISO image) is very important. Because, they usually are downloaded over http/ftp and the client that downloads them (usually a browser/download manager) doesn’t really do an integrity check of the payload.

    When using the package manager, one doesn’t have to worry of it because the package manager takes care of ensuring the integrity of the payload isn’t tampered.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.