This would be a sorta longish post with bits about both the red-tape and policy issues while changing a water pipe and later about a QA tool called adequate which entered debian archives quite some time before.
First of all a belated happy new year to all. For the last couple of months I have not been able to blog as I had been running around getting the water pipe changed and fixed. A little bit of history is in order as to why the change had to happen and things I found out.
To recap, we would have to rewind to the mid 1960’s. My maternal grandfather had been transferred to Pune on account of his job with the Military/Central Govt./Defense. As a civil servant and esp. with the defense establishment, it was a roaming job. My mother recalls they were shifted many times to different places while growing up.Till the time of transfer to Pune he had wanted to settle back in Delhi where we had an ancestral house and this place was an assignment.He got some place near to him first as he was the only one, then when the rest of the family showed up, he took a rented place which to this day is my present home.
Then cut to the 1978 and the Pune Municipal Corporation had started giving residential water connections. Till that time we used to take water from the community water connection and as people know it can be a tiring and sometimes a tense affair.
So with the permission of the landlord of the day, we got our own private water meter connection.The corporation was also very helpful (according to my mother) and we got our own tap and everything in the next 2-3 days. Due charges were paid for the same.
Cut to 2007 and when we first started having water issues. It was in the Newspaper that the city was going through a water crisis, so we thought it was due to that we had no water pressure.We live on the first floor which is 14 foot from the ground (or about 4.5 meters) of ground. We don’t have an overhead tank(as is nowadays a routine thing with most modern structures). We also don’t use a motor and before the crisis never needed to. Then 2 years back the lineman who comes every now and then to take water meter readings, commented on the fact that the water pressure was really low and it’s possible that the pipe had become rusty or something like that.
After quite a bit of back and forth came to know that the municipality shared that they do not do digging the pipeline anymore, we have to get a private plumber to replace the GI pipe (Galvanized Iron Pipe) on the main road and they would just give us the necessary permission was Rs. 6000/- for digging up the road as that permission is necessary as it’s Government property. The calculation being Rs. 60×100 feet @60/- per feet . That is how much the distance between my building and the Main water pipe which supplies water to all the neighborhood. The connection between the new pipe and the main water pipe would be done by the Corporation people themselves.
After receipt of the permission, it took them quite sometime and they came and dug a small hole where the water pipe was supposed to be (apparently) and they found three pipes of 1/2 inch diameter. I have to point, they didn’t find where the connection was made to the water pipe. In my innocence I asked if they didn’t have any plans of the underground pipes and to my horror, I was told there were no underground/water plans/maps.
Anyways, after the small hole they dug, they came to no conclusion as they didn’t know which line was ours. As it had been a great number of years, about 30+ years my mother also didn’t remember how it laid out and over the years the road had been tarred numerous number of time. Each layer of tar adding a bit of depth which also adds to the problem. My grandfather (god rest his soul) had passed a number of years ago so that information passed with him as well.
After a long period of deliberation, indecision and confusion (as we weren’t sure if the pipe had actually rusted or people were just trying to make money), we actually went for the change of the Pipe about a week back. I looked around in the market for different brands of GI Pipe (Galvanized Iron Pipe). I came to know that the GI Pipe is basically Iron when made into sheets being immersed in a zinc smelter so it has a uniform zinc coating. The number of brands are/were numerous and they are/were graded as ‘A’, ‘B’ and ‘C’ . The ‘C’ being the most thickest as well as most expensive while ‘B’ being the lighter and ‘A’ being the lightest. With the help of the plumber I zeroed in ‘Surya Prakash’. The pipe was brought @ of Rs. 30/- per feet and had bought 100 feet so it was Rs. 3000/-. The plumber charged Rs. 4400/- for the work and incidentals were another 1.5k/- . Overall changing the pipe cost us Rs. 6000+3000+4400+1500 = 15000/- and this is just till the entrance to the building.If I were to change the remaining pipe, add another 9000/- to it.
Now the question for me are two-fold. I am grateful that I earn a bit so I could afford this expense, but I know so many people around me who cannot. I don’t get this policy of the Municipal Corporation to ask people to dig even while taking money for permission. It makes sense for companies as they are digging for future profits and can mark those digs as expenses against future profits, but for averaged salaried people who are married and have a kid or two, this kind of expense is not something that could do easily, and the tax breaks that companies get etc. are not possible for the average person.
A point to make is where I live, I know a lot of low-income households who barely make Rs. 10,000/- a month for them this would be a crucification.The feature of most of the low-income households, they make less money and have more dependents on them although the situation around my place is better with the most households only going up 2-3 children at the most.If I think back, I still don’t know whether we actually needed to change the pipe, or the missing ferrule. I had to buy another ferrule and a connection was made. I also have to point that when we actually did the work, the small hole that the Corporation people had dug before, was dug by them at least twice larger and they did find where our old pipe and the connection to the Main pipe was. So, looking back, they didn’t do the work and perhaps just changing the ferrule would possibly been enough. I do know that the quality of Water pipes usually do deteriorate over time from before as had read and heard from people that somewhere while researching. Also at the end, there was quite a layer of dirt and perhaps (rust) as the color of water was brownish/reddish.
The only thing I learnt from this, you cannot do anything about the status quo, even if we change the leaders by voting, unless there is a way to change the processes we would remain as we are.
The biggest thing if it has to be done is to have maps of good quality of underground pipes, in the public domain shared under CC-0 and updated regularly. This is not a new idea at all. In fact there has been a proposal (for quite some time now) to add such data to OSM.
Slightly OT, but if I were to guess, that proposal in OSM is not being integrated because there doesn’t seem to be a good way to visualize the 3rd dimension in a 2d world. Let me explain, as of right now, the view is strictly 2d which is good for finding things such as going from A to B or finding distances between two places but once you have elevation (either upper or lower) then the third dimension comes into picture. So it’s no longer A-B but A-B-C or X-Y-Z. Now visualizing something like 5-15 foot up or down is a whole different kettle of fish. It would improve the accuracy of the maps a whole lot more but would make maps more complex as well as make the maps expensive to render as you would need some kind of 3D modeling software to render along with more requirements both on the client and server-side. But I’m sure this is the way to go at some point in the future. In fact, a small sample of what would be needed can be seen with this YouTube video. A small history about this video, this was done by Blacksburg, Virginia, U.S.A. when Google was going to do 1 Gig bandwidth to some city in U.S.A. and they had asked cities for their proposal as to what they would do if they had 1 Gigabyte of bandwidth on tap.
I know of some people who have made some maps, but due to limitations in software, they are not that pretty (as of yet) or that much useful.Another thing which would be immensely helpful are cheap, handheld GPR (Ground penetrating Radar). Some people would definitely go for it.
Anyways, let’s come to a different software altogether. I have been meaning to share about adequate for sometime now but for number of reasons (mostly lack of time) have not been able to blog about it. With my name appearing in the changelog recently, it did spur me to write about it.
The following is from /usr/share/doc/adequate/changelog.gz . This is from testing version and got this while updating/upgrading a while back.
adequate (0.8.2) unstable; urgency=low
* Improve the manual page:
+ Fix hyphens used as minus signs.
+ Add examples (closes: #726152)
Thanks to Shirish Agarwal for the bug report.
* Don't complain about files in /usr/share/paster_templates/ not being
* Fix reporting incompatible-licenses, and other ELF-related tags, against
wrong package (closes: #729031).
* Fix incorrect implementation of --tags for ELF-related tags.
-- Jakub Wilk Thu, 05 Dec 2013 10:04:22 +0100
Both the maintainer and upstream of this software is Jakub Wilk. Let’s see what the package is about from the description :-
$ aptitude show adequate
Automatically installed: no
Maintainer: Jakub Wilk
Uncompressed Size: 117 k
Depends: perl (>= 5.12), debconf
Description: Debian package quality testing tool
adequate checks packages installed on the system and reports bugs and policy violations.
The following checks are currently implemented:
* broken symlinks;
* missing copyright file;
* obsolete conffiles;
* Python modules not byte-compiled;
* /bin and /sbin binaries requiring /usr/lib libraries;
* missing libraries, undefined symbols, symbol size mismatches;
* license conflicts;
* program name collisions;
* missing alternatives.
Now the idea about the state of the system is to be as stable as possible vis-a-vis number of bugs in the system. So for the system administrator it’s in his/her best interest to keep them in the minimum and report if you see something not right/correct.Adequate is an effort in doing that and maintain software upto Debian’s standards. But it would not be possible to do that without people using adequate on a day-to-day basis and finding and reporting bugs when installing, upgrading packages and adequate running on each install and upgrade to see if any cruft is left behind or something like that.
The first thing to do is to go to /etc/apt/apt.conf.d/20adequate and toggle the Enabled to true. By default it’s turned to false. What is being done here is you are enabling the apt-hook for adequate which means as a result every time either an install or an upgrade happens, adequate will run and if an issue/bug is found it will trigger and you will see what the bug is on the console in highlighted format.
Now before making changes to the file itself, it’s good if you make a backup of the configuration file of adequate itself.
$ sudo cp /etc/apt/apt.conf.d/20adequate /etc/apt/apt.conf.d/20adequate.bak
The .bak extension tells apt to ignore the contents of the backup file you created.
From the apt.conf manpage, directories, last paragraph:-
The Ignore-Files-Silently list can be used to specify which files APT should silently ignore while parsing the files in the fragment directories. Per default a file which end with .disabled, ~, .bak or .dpkg-[a-z]+ is silently ignored. As seen in the last default value these patterns can use regular expression syntax.
Actually the .bak extension is pretty useful and I use it to always keep a backup of the default state of configuration files before making changes in case I make a mess of things and didn’t know what changes I did and it has worked out great so far.
The configuration file of the adequate apt-hook is 8-10 lines at the most so it’s small and easy.
Now you can run either adequate interactively (if you so desire) or you can let it be run automatically whenever any changes to your system state.
The default behavior whenever it encounters a bug is to give a blue screen and the bug in red so sys-admins who are conservative-minded (most of them are due to the various SLA’s involved) can diagnose and either fix or workaround the issue. Fixing would be putting up a patch with a bug telling the maintainer what the issue is/was and working towards that.
But some people perhaps do not like that blue screen and red color thing as it is irritating so I made a change in /etc/apt/apt.conf.d/20adequate
"adequate --help >/dev/null 2>&1 || exit 0; exec adequate --debconf --user nobody --pending";
See the above code. What that is telling your package manager (in my case aptitude) that post installation run adequate with the default setup.
Just making a small change here makes the output that much more human and machine-friendly as it doesn’t block the screen :-
"adequate --help >/dev/null 2>&1 || exit 0; DEBIAN_FRONTEND=readline exec adequate --debconf --user nobody --pending";
The only change I did was add DEBIAN_FRONTEND=readline so instead of flashing I get it a nice textual interface which I can use to do things as per convenience and it’s easier to parse.
The readline library is part of the readline-common package.
$ aptitude show readline-common
Automatically installed: no
Maintainer: Matthias Klose
Uncompressed Size: 81.9 k
Depends: dpkg (>= 1.15.4) | install-info
Conflicts: libreadline-common, libreadline5 (< 5.0-11)
Replaces: libreadline-common, libreadline4 (< 4.3-16), libreadline5 (< 5.0-11)
Description: GNU readline and history libraries, common files
The GNU readline library aids in the consistency of user interface across discrete programs that need to provide a command line interface.
The GNU history library provides a consistent user interface for recalling lines of previously typed input.
I have been using readline before as well. I just wish I knew how I could use readline more effectively because I’m sure there are more than enough places where it would make for better readability, output etc but that’s another topic for another day.
Anyways, that about does it for running adequate automatically when doing an installation or an upgrade etc.
But what about when you have some time to kill and want to make Debian better and don’t know how. Adequate to the rescue.
While you can just let it rip by saying :-
and adequate giving you a list of all potential issues/bugs, I usually go for broken-symlinks and obsolete-conffile as these are two of the more simpler bugs to report.
As the name shares, a broken-symlink is nothing but a broken Symbolic link. For those coming from a MS-Windows background it basically is similar to a shortcut.
An example of a symbolic link :-
$ ls -l /usr/share/tuxpaint/fonts/locale/ta.ttf
lrwxrwxrwx 1 root root 54 Dec 4 2011 /usr/share/tuxpaint/fonts/locale/ta.ttf -> ../../../fonts/truetype/ttf-tamil-fonts/TSCu_Comic.ttf
but this is a bad example as the real file doesn’t exist at the path shared. The real file should have been in a sub-directory called truetype in /usr/share/tuxpaint/ but there is no directory like that.
default_font.ttf FreeMonoOblique.ttf FreeSansBold.ttf FreeSerifBoldItalic.ttf FreeSerif.ttf
FreeMonoBoldOblique.ttf FreeMono.ttf FreeSansOblique.ttf FreeSerifBold.ttf locale
FreeMonoBold.ttf FreeSansBoldOblique.ttf FreeSans.ttf FreeSerifItalic.ttf
I actually came to know it’s a broken-symlink because adequate told me. These are some of the broken-symlinks adequate found on my system :-
$ adequate --all --tags broken-symlink
libgcj14:amd64: broken-symlink /usr/lib/jvm/java-1.5.0-gcj-4.8-amd64/jre/lib/security/java.security -> ../../../../../x86_64-linux-gnu/security/classpath.security
python-support: broken-symlink /usr/lib/python2.6/dist-packages/python-support.pth -> ../../pymodules/python2.6/.path
libjack-dev: broken-symlink /usr/lib/x86_64-linux-gnu/libjackserver.so -> libjackserver.so.0.0.28
fonts-lmodern: broken-symlink /usr/share/doc/fonts-lmodern/README-Latin-Modern-Math.TXT -> ../../texmf/doc/fonts/lm-math/README-Latin-Modern-Math.TXT
lazarus-src-1.0.10: broken-symlink /usr/lib/lazarus/1.0.10/components/chmhelp/lhelp/lhelp.app/Contents/MacOS/lhelp -> ../../../lhelp
libgraphviz-dev: broken-symlink /usr/lib/graphviz/libgvplugin_gtk.so -> libgvplugin_gtk.so.6.0.0
libgraphviz-dev: broken-symlink /usr/lib/graphviz/libgvplugin_gdk_pixbuf.so -> libgvplugin_gdk_pixbuf.so.6.0.0
tuxpaint-data: broken-symlink /usr/share/tuxpaint/fonts/locale/zh_TW.ttf -> ../../../fonts/truetype/arphic/uming.ttf
tuxpaint-data: broken-symlink /usr/share/tuxpaint/fonts/locale/el.ttf -> ../../../fonts/truetype/ttf-thryomanes/thryrg__.ttf
tuxpaint-data: broken-symlink /usr/share/tuxpaint/fonts/locale/th.ttf -> ../../../fonts/truetype/thai/Garuda-Bold.ttf
tuxpaint-data: broken-symlink /usr/share/tuxpaint/fonts/locale/ar.ttf -> ../../../fonts/truetype/ttf-arabeyes/ae_Nice.ttf
tuxpaint-data: broken-symlink /usr/share/tuxpaint/fonts/locale/ko.ttf -> ../../../fonts/truetype/baekmuk/gulim.ttf
tuxpaint-data: broken-symlink /usr/share/tuxpaint/fonts/locale/te.ttf -> ../../../fonts/truetype/ttf-telugu-fonts/Pothana2000.ttf
tuxpaint-data: broken-symlink /usr/share/tuxpaint/fonts/locale/zh_CN.ttf -> ../../../fonts/truetype/arphic/gbsn00lp.ttf
tuxpaint-data: broken-symlink /usr/share/tuxpaint/fonts/locale/gu.ttf -> ../../../fonts/truetype/ttf-gujarati-fonts/lohit_gu.ttf
tuxpaint-data: broken-symlink /usr/share/tuxpaint/fonts/locale/ta.ttf -> ../../../fonts/truetype/ttf-tamil-fonts/TSCu_Comic.ttf
openjdk-7-jdk:amd64: broken-symlink /usr/lib/jvm/java-7-openjdk-amd64/src.zip -> ../java-7-openjdk-common/src.zip
This is one of the bugs I reported found using adequate:-
Usertags: adequate broken-symlink
Adequate reported broken symlinks for gcj-4.8-jre-headless.
$ adequate gcj-4.8-jre-headless
On investigating further they were proved to be correct as there is no
x86_64-linux-gnu sub-directory as can be seen below.
So please fix the same.
— System Information:
Debian Release: jessie/sid
APT prefers testing
APT policy: (600, ‘testing’), (1, ‘experimental’), (1, ‘unstable’)
Architecture: amd64 (x86_64)
Kernel: Linux 3.10-3-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_IN, LC_CTYPE=en_IN (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Versions of packages gcj-4.8-jre-headless depends on:
ii gcc-4.8-base 4.8.1-10
ii gcj-4.8-jre-lib 4.8.1-10
ii libc6 2.17-93
ii libgcc1 1:4.8.1-10
ii libgcj14 4.8.1-10
ii zlib1g 1:1.2.8.dfsg-1
gcj-4.8-jre-headless recommends no packages.
Versions of packages gcj-4.8-jre-headless suggests:
ii fastjar 2:0.98-5
— no debconf information
I used reportbug (about which I shared before a few times on the blog). The interesting point are the two tags in the beginning of the bug-report.
Usertags: adequate broken-symlink
What you do by adding the two tags above, you are sharing the bug with debian-qa team and bringing the same to their notice and you are also sharing that what tool told you about the issue/bug/ticket.
I usually purposely run adequate with a package name and do investigate it manually to see if the package is broken or not (what adequate is telling/sharing about) before reporting it.
I wish I could share some more examples and details and I may have glossed over somethings due to time-constraints. I would like to thank both Jakub Wilk (the author as well as maintainer of adequate in Debian archives) and Paul Wise (who does a lot of Q&A work using adequate and I guess an evangelist of the tool).
In fact it was from Paul’s website when I first came to know about adequate about a year back and have been using it since then. This tool is not just good for sys-admins but probably also developers too, so they could just run their new shiny package through this before uploading it to the archive and see if anything pops up.