Different strokes

Delhi Metro - courtesy wikipedia.org

Statutory warning – It’s a long read.

I start by sharing I regret, I did not hold onto the Budget and Economics 101 blog post for one more day. I had been holding/thinking on to it for almost couple of weeks before posting, if I had just waited a day more, I would have been able to share an Indian Express story . While I thought that the work for the budget starts around 3 months before the budget, I came to learn from that article that it takes 6 months. As can be seen in the article, it is somewhat of a wasted opportunity, part of it probably due to the Government (irrespective of any political party, dynasty etc.) mismanagement.

What has not been stated in the article is what I had shared earlier, reading between the lines, it seems that the Government isn’t able to trust what it hears from its advisers and man on the street. Unlike Chanakya and many wise people before him who are credited with advising about good governance, that a good king is one who goes out in disguise, learns how his/er subjects are surviving, seeing what ills them and taking or even not taking corrective steps after seeing the problem from various angles. Of course it’s easier said then done, though lot of Indian kings did try and ran successful provinces. There were also some who were more interested in gambling, women and threw/frittered away their kingdoms.

The 6-month things while not being said in the Express article is probably more about checking and re-checking figures and sources to make sure they are able to read whatever pattern the various Big Businesses, Industry, Social Welfare schemes and people are saying I guess. And unless mass digitalization as well as overhaul of procedures, Right to Information (RTI) happens, don’t see any improvement in the way the information is collected, interpreted and shared with the public at large.

It would also require people who are able to figure out how things work sharing the inferences (right or wrong) through various media so there is discussion about figures and policy-making. Such researchers and their findings are sadly missing in Indian public discourses and only found in glossy coffee table books :(.

One of the most basic question for instance is, How much of any policy should be based on facts and figures and how much giving fillip to products and services needed in short to medium term ?

Also how much morality should play a part in Public Policy ?

Surprisingly, or probably not, most Indian budgets are populist by nature with some scientific basis but most of the times there is no dialog about how the FM came to some conclusion or Policy-making. I am guessing a huge part of that has also to do with basic illiteracy as well as Economic and Financial Illiteracy.

Just to share a well-known world-over example, one of the policies where the Government of India has been somewhat lethargic is wired broadband penetration. As have shared umpteen times, while superficially broadband penetration is happening, most of the penetration is the unreliable and more expensive mobile broadband penetration.

While this may come as a shock to many of the users of technology, BSNL, a Government company who provides broadband for almost 70-80% of the ADSL wired broadband subscribers gives 50:1 contention ratio to its customers.

One can now understand the pathetic speeds along with very old copper wiring (20 odd years) on which the network is running. The idea/idiom of running network using duct-tape seems pretty apt in here 😦

Now, the Government couple of years ago introduced FFTH Fiber-to-the-home but because the charges are so high, it’s not going anywhere. The Government could say 10% discount in your Income Tax rates if you get FFTH. This would force people to get FFTH and would also force BSNL to clean up its act. It has been documented that a percentage increase in broadband equals a similar percentage rise in GDP.

Having higher speeds of broadband would mean better quality of streaming video as well as all sorts of remote teaching and sharing of ideas which will give a lot of fillip to all sorts of IT peripherals in short, medium and long-term as well. Not to mention, all the software that will be invented/coded to take benefit of all that speed.

Although, realistically speaking I am cynical that the Government would bring something like this 😦

Moving on –

Behind a truck - Courtesy TheEconomist.com

Another interesting story which I had shared was a bit about World History

Now the Economist sort of confirmed how things are in Pakistan. What is and was interesting that the article is made by a politically left-leaning magazine which is for globalization, business among other things .

So, there seem to be only three options, either I and the magazine are correct or we both are reading it wrong.

The third and last option is that the United States realize that Pakistan can no longer be trusted as Pakistan is siding more and more with Chinese and Russians, hence the article. Atlhough it seems a somewhat far-fetched idea as I don’t see the magazine getting any brownie points with President Trump. Unless, ‘The Economist’ becomes more hawkish, more right-wingish due to the new establishment.

I can’t claim to have any major political understanding or expertise but it does seem that Pakistan is losing friends. Even UAE have been cautiously building bridges with us. Now how this will play out in the medium to long-term depends much on the personal equations of the two heads of state, happenings in geopolitics around the world and the two countries, decisions they take, it is a welcome opportunity as far they (the Saudis) have funds they want to invest and India can use those investments to make new infrastructure.

Now, I need a bit of help of Java and VCS (Version control system) experts . There is a small game project called Mars-Sim. I asked probably a few more questions than I should have and the result was that I was made a member of the game team even though I had shared with them that I’m a non-coder.

I think such a game is important as it’s foss. Both the game itself is foss as well as its build-tools with a basic wiki. Such a game would be useful not only to Debian but all free software distributions.

Journeying into the game

Unfortunately, the game as it is currently, doesn’t work with openjdk8 but private conversations with the devs. have shared they will work on getting it to work on OpenJDK 9 which though is sometime away.

Now as it is a game, I knew it would have multiple multimedia assets. It took me quite sometime to figure out where most of the multimedia assets are.

I was shocked to find that there aren’t any tool/s in Debian as well a GNU/Linux to know about types of content is there inside a directory and its sub-directories.

I framed it in a query and found a script as an answer . I renamed the script to file-extension-information.sh (for lack of imagination of better name).

After that, I downloaded a snapshot of the head of the project from https://sourceforge.net/p/mars-sim/code/HEAD/tree/ where it shows a link to download the snapshot.

https://sourceforge.net/code-snapshots/svn/m/ma/mars-sim/code/mars-sim-code-3847-trunk.zip

unzipped it and then ran the script on it –

[$] bash file-extension-information.sh mars-sim-code-3846-trunk
theme: 1770
dtd: 31915
py: 10815
project: 5627
JPG: 762476
fxml: 59490
vm: 876
dat: 15841044
java: 13052271
store: 1343
gitignore: 8
jpg: 3473416
md: 5156
lua: 57
gz: 1447
desktop: 281
wav: 83278
1: 2340
css: 323739
frag: 471
svg: 8948591
launch: 9404
index: 11520
iml: 27186
png: 3268773
json: 1217
ttf: 2861016
vert: 712
ogg: 12394801
prefs: 11541
properties: 186731
gradle: 611
classpath: 8538
pro: 687
groovy: 2711
form: 5780
txt: 50274
xml: 794365
js: 1465072
dll: 2268672
html: 1676452
gif: 38399
sum: 23040
(none): 1124
jsx: 32070

It gave me some idea of what sort of file were under the repository. I do wish the script defaulted to showing file-sizes in KB if not MB to better assess how the directory is made up but not a big loss .

The above listing told me that at the very least theme, JPG, dat, wav, png, ogg and lastly gif files.

For lack of better tools and to get an overview of where those multimedia assets used ncdu –

┌─[shirish@debian] - [~/games/mars-sim-code-3846-trunk] - [10210]
└─[$] ncdu mars-sim/

--- /home/shirish/games/mars-sim-code-3846-trunk/mars-sim --------------------------------------------------------------------------------------
46.2 MiB [##########] /mars-sim-ui
15.2 MiB [### ] /mars-sim-mapdata
8.3 MiB [# ] /mars-sim-core
2.1 MiB [ ] /mars-sim-service
500.0 KiB [ ] /mars-sim-main
188.0 KiB [ ] /mars-sim-android
72.0 KiB [ ] /mars-sim-network
16.0 KiB [ ] pom.xml
12.0 KiB [ ] /.settings
4.0 KiB [ ] mars-sim.store
4.0 KiB [ ] mars-sim.iml
4.0 KiB [ ] .project

I found that all the media is distributed randomly and posted a ticket about it. As I’m not even a java newbie, could somebody look at mokun’s comment and help out please ?

On the same project, there has been talk of migrating to github.com

Now whatever little I know of git, it makes a copy of the whole repository under .git/ folder/directory so having multimedia assets under git is a bad, bad idea, as each multimedia binary format file would be unique and no possibility of diff. between two binary files even though they may be the same file with some addition or subtraction from earlier version.

I did file a question but am unhappy with the answers given. Can anybody give some definitive answers if they have been able to do how I am proposing , if yes, how did they go about it ?

And lastly –

Immigrants of the United States in 2000 by country of birth

America was founded by immigrants. Everybody knows the story about American Indians, the originals of the land were over-powered by the European settlers. So any claim, then and now that immigration did not help United States is just a lie.

This came due to a conversation on #debconf by andrewsh –

[18:37:06] I’d be more than happy myself to apply for an US tourist not transit visa when I really need it, as a transit visa isn’t really useful, is just as costly as a tourist visa, and nearly as difficult to get as a tourist visa
[18:37:40] I’m not entirely sure I wish to transit through the US in its Trumplandia incarnation either
[18:38:07] likely to be more difficult and unfun

FWIW I am in complete agreement with Andrew’s assessment of how it might be with foreigners. It has been on my mind and thoughts for quite some time although andrewsh put it eloquently.

But as always I’m getting ahead of myself.

The conversation is because debconf this year would be in Canada. For many a cheap flight, one of the likely layovers/stopover can be the United States.

I actually would have gone one step further, even if it was cheap transit visa, it would equally be unfun as it would discriminate.

About couple of years back, a friend of mine while explaining what “visa” is, put it rather succinctly –

the visa officer looks at only 3 things –

a. Your financial position – something which tells that you can take care of your financial needs if things go south –

b. You are not looking to settle there unlawfully

c. You are not a criminal.

While costs do matter, what is disturbing more is the form of extremism being displayed therein. While Indians from the South Asian continent in US have been largely successful, love to be in peace (one-off incidents do and will happen anywhere) if I had to take a transit or tourist visa in this atmosphere, it would leave a bad taste in the mouth.

When one of my best friends is a Muslim, 20% of the population in India is made of Muslims and 99% of the time both of us co-exist in peace I simply can’t take any alternative ideology. Even in Freakonomics 2.0 the authors when they shared that it’s less than 0.1 percent of Muslims who are engaged in terrorist activities, if they were even 1 percent than all the world’s armed forces couldn’t fight them and couldn’t keep anyone safe. Which simply means that 99.99% of even all Muslims are good.

This resonates strongly with me for number of reasons. One of my uncles in early to late 80’s had an opportunity for work to visit Russia for official work. He went there and there were Secret Police after him all the time. While he didn’t know it, I later read it, that it was SOP (Standard Operating Procedure) when all and any foreigners came visiting the country, and not just foreigners, they had spies for their own citizens. Russka a book I read several years ago explained the paranoia beautifully.

While U.S. in those days was a more welcoming place for him.

I am thankful as well as find it strange that Canada and States have such different visa procedures. While Canada would simply look at the above things, probably discreetly inquire about you if you have been a bad boy/girl in any way and then make a decision which is fine. For United States, even for a transit visa I probably would have to go to Interview where my world view would probably be in conflict with the current American world view.

Interestingly, while I was looking at conversations on the web and one thing that is missing there is that nobody has talked about intelligence community. What Mr. Trump is saying in not so many words is that our intelligence even with all the e-mails we monitor and everything we do, we still can’t catch you. It almost seems like giving a back-handed compliment to the extremists saying you do a better job than our intelligence community.

This doesn’t mean that States doesn’t have interesting things to give to the world, Star Trek conventions, Grand Canyon (which probably would require me more than a month or more to explore even a little part), NASA, Intel, AMD, SpaceX, CES (when it’s held) and LPC (Linux Plumber’s conference where whose who come to think of roadmap for GNU/Linux). What I wouldn’t give to be a fly in the wall when LPC, CES happens in the States.

What I actually found very interesting is that in the current Canadian Government, if what I read and heard is true, then Justin Trudeau, the Prime Minister of Canada made 50 of his cabinet female. Just like in the article, studies even in Indian parliament have shown that when women are in power, questions about social justice, equality, common good get asked and policies made. If I do get the opportunity to be part of debconf, I would like to see, hear, watch, learn how the women cabinet is doing things. I am assuming that reporting and analysis standards of whatever decisions are more transparent and more people are engaged in the political process to know what their elected representatives are doing.

Mountain biking in British Columbia, Canada - source wikipedia.org

One another interesting point I came to know is that Canada is home to bicycling paths. While I stopped bicycling years ago 😦 as it has been becoming more and more dangerous to bicycle here in Pune as there is no demarcation for cyclists, I am sure lot of Canadians must be using this opportunity fully.

Lastly, on the debconf preparation stage, things have started becoming a bit more urgent and hectic. From a monthly IRC meet, it has now become a weekly meet. Both the wiki and the website are slowly taking up shape. http://deb.li/dc17kbp is a nice way to know/see progress of the activities happening .

One important decision that would be taken today is where people would stay during debconf. There are options between on-site and two places around the venue, one 1.9 km around, the other 5 km. mark. Each has its own good and bad points. It would be interesting to see which place gets selected and why.

Debian contributions and World History

Beware, this would be slightly longish.

Debian Contributions

In the last couple of weeks, was lucky to put up a patch against debian-policy which had been bothering me for a long-long time.

The problem statement is simple, man-pages historically were made by software engineers for software-engineers. The idea, probably then was you give the user some idea of what the software does and the rest the software engineer would garner from reading the source-code. But over period of time, the audience has changed. While there are still software engineers who use GNU/Linux for the technical excellence, the man-pages have not kept up with this new audience who perhaps are either not technically so sound that or they do not want to take the trouble to reading the source-code to understand how things flow. An ‘example’ or ‘examples’ in a man-page gives us (the lesser mortals) some insight as how the command works, how the logic flows.

A good example of a man-page is the ufw man-page –

EXAMPLES
Deny all access to port 53:

ufw deny 53

Allow all access to tcp port 80:

ufw allow 80/tcp

Allow all access from RFC1918 networks to this host:

ufw allow from 10.0.0.0/8
ufw allow from 172.16.0.0/12
ufw allow from 192.168.0.0/16

Deny access to udp port 514 from host 1.2.3.4:

ufw deny proto udp from 1.2.3.4 to any port 514

Allow access to udp 1.2.3.4 port 5469 from 1.2.3.5 port 5469:

ufw allow proto udp from 1.2.3.5 port 5469 to 1.2.3.4 port 5469

Now if we had man-pages like the above which give examples, then the user at least can try to accomplish whatever s/he is trying to do. I truly believe not having examples in a man-page kills 50% of your audience and people who could potentially use your tool.

Personal wishlist – The only thing (and this might be my failure) is we need a good way to search through a man-page. The only way I know is using ‘/’ and try to give a pattern. Lots of times it fails because I, the user doesn’t know the exact keyword which the documenter was using. What would be nice, great if we do have some sort of parser where I tell it, ‘$this is what I’m looking for’ and the parser tries the pattern + all its synonyms and whatever seems to be most relevant passage from the content, in this case – a manpage it tells me. It would make my life a lot easier while at the same time force people to document more and more.

I dunno if there has been any research or study of the relationship between good documentation and popularity of a program. I know there are lots of different tiny bits which make or break a program, one of which would definitely be documentation and in that a man-page IF it’s a command-line tool.

A query on Quora gives some indication https://www.quora.com/How-comprehensible-do-you-find-Unix-Linux-Man-pages although the low response rate tells its own story.

there have been projects like man2html and man2pdf and others which try to make the content more accessible to people who are not used to the man-page interface but till you don’t have ‘Examples’ the other things can work only so far. Also if anybody talks about X project which claims to solve this problem they will have to fight manpages who have been around like forever.

As can be seen in the patch, did some rookie mistakes as can be seen. I also filed a lintian bug at the same time. Hope the patch does get merged at some point in debian-policy and then a check introduced in lintian in some future release. I do agree with anarcat’s assertion that it should be at the level of the manpage missing level.

I am no coder but finding 14,000 binary packages without a manpage left me both shocked and surprised.

I came to know about manpage-alert from the devscripts package to know which all binary packages that have been installed but not have man-pages.

I hope to contribute a manpage or two if I across a package I’m somewhat comfortable with. I have made a beginning of sorts by running manpage alert and putting the output in a .txt file which I would grep through manually and see if something interesting jumps at me.

The learning garnered from putting the patch to the debian package resulted in another patch but this time for an upstream project altogether. As can be seen all are just baby-steps that even a non-coder can take.

Another couple of bugs I filed which were fixed were of a sim called ‘unknown-horizons’ . A 2D realtime strategy simulation. I had filed three bugs, two of which were fixed in 2 days, the 3rd I hope is also fixed soonish.

Lastly, I spent most of the week-end poring over packages who have left files in /etc/bash_completion.d/ . I spent almost 4-5 odd hours as each package in question as well as entries found in /etc/bash-completion.d/$filename I had to find which package it belonged to first –

[$] dpkg -S /etc/bash_completion.d/git-prompt

git: /etc/bash_completion.d/git-prompt

I know that dpkg-query also does the same –


[$] dpkg-query -S /etc/bash_completion.d/git-prompt

git: /etc/bash_completion.d/git-prompt

But I am used to plain dpkg although do know that dpkg-query can do lot more intimate searching in various ways than dpkg can.

Once the package name was established, first simulate the purge –

[$] sudo aptitude -s purge git

[sudo] password for shirish:
The following packages will be REMOVED:
git{p}
0 packages upgraded, 0 newly installed, 1 to remove and 14 not upgraded.
Need to get 0 B of archives. After unpacking 29.5 MB will be freed.
The following packages have unmet dependencies:
libgit-wrapper-perl : Depends: git but it is not going to be installed
git-extras : Depends: git (>= 1.7.0) but it is not going to be installed
bup : Depends: git but it is not going to be installed
git-remote-gcrypt : Depends: git but it is not going to be installed
git-svn : Depends: git (> 1:2.11.0) but it is not going to be installed
Depends: git ( 1:2.11.0) but it is not going to be installed
Depends: git (= 1:1.8.1) but it is not going to be installed
git-core : Depends: git (> 1:1.7.0.2) but it is not going to be installed
The following actions will resolve these dependencies:

Remove the following packages:
1) bup [0.29-2 (now, testing, unstable)]
2) fdroidserver [0.7.0-1 (now, testing, unstable)]
3) git-annex [6.20161012-1 (now, testing)]
4) git-core [1:2.11.0-2 (now, testing, unstable)]
5) git-extras [4.2.0-1 (now, testing, unstable)]
6) git-remote-gcrypt [1.0.1-1 (now, testing, unstable)]
7) git-repair [1.20151215-1 (now, unstable)]
8) git-svn [1:2.11.0-2 (now, testing, unstable)]
9) gitk [1:2.11.0-2 (now, testing, unstable)]
10) libgit-wrapper-perl [0.047-1 (now, testing, unstable)]
11) python3-git [2.1.0-1 (now, testing, unstable)]
12) svn2git [2.4.0-1 (now, testing, unstable)]

Leave the following dependencies unresolved:
13) devscripts recommends libgit-wrapper-perl
14) dh-make-perl recommends git
15) fdroidserver recommends git
16) git-annex recommends git-remote-gcrypt (>= 0.20130908-6)
17) gplaycli recommends fdroidserver
18) python-rope recommends git-core

Accept this solution? [Y/n/q/?] q
Abandoning all efforts to resolve these dependencies.
Abort.

Then I made a note of all the packages being affected, saw purging all of them wouldn’t call others (the Package dependency Hell), made the purge and then reinstalled anew.

The reason I did this is that many a times during upgrade, either during update/upgrade sometimes the correct action doesn’t happen. To take the git’s example itself, there were two files git-extras and git-prompt which were in /etc/bash_completion.d/ both of which were showing their source as git. Purging git and installing git afresh removed git-extras file and git-prompt is the only one remaining.

While blogging about the package, did try to grep through changelog.Debian.gz and changelog.gz in git –

┌─[shirish@debian] - [/usr/share/doc/git] - [10046]
└─[$] zless changelog.gz

and similarly –

┌─[shirish@debian] - [/usr/share/doc/git] - [10046]
└─[$] zless changelog.Debian.gz

But failed to find any mention of the now gone git-extras. Doing this with all the packages took considerable time as didn’t want to deal with any potential fallout later on. For instance, ufw (uncomplicated firewall) also had an entry in /etc/bash_completion.d/, hence before purging ufw, took backup of all the rules I have made, did a successful simulation

[$] sudo aptitude -s purge ufw gufw


The following packages will be REMOVED:
gufw{p} ufw{p}
0 packages upgraded, 0 newly installed, 2 to remove and 14 not upgraded.
Need to get 0 B of archives. After unpacking 4,224 kB will be freed.

Note: Using 'Simulate' mode.
Do you want to continue? [Y/n/?] y
Would download/install/remove packages.

purged the packages, reinstalled it and then re-added all the rules. Doing it all for various sundry packages, had to do it manually as there is no one size fits all solution.

A sensitive one was grub which still has an entry in /etc/bash_completion.d/grub. Doing it wrong could have resulted in a non-bootable situation. There are workarounds for that, but it would have taken quite a bit of time, energy, notes and bit of recall factor what I did the last time something like that happened. Doing it manually, being present meant I could do it rightly the first time.

So, was it worth it – It would be if the package maintainers do the needful and the remaining entries are moved out of /etc/bash_completion.d/ to /usr/share/bash-completions and some to my favourite /usr/share/zsh/vendor-completions/ – for instance –

┌─[shirish@debian] - [/usr/share/zsh/vendor-completions] - [10064]
└─[$] ll -h _youtube-dl

-rw-r--r-- 1 root root 3.2K 2016-12-01 08:48 _youtube-dl

But trying to get all or even major packages to use zsh-completions would be hard work and would take oodles of time and this concerns upstream stuff, also very much outside what I was sharing.

World History

Before, during and even after South-African experience, I was left wondering why India and South Africa, two countries who had similar histories at least the last couple of hundred years or more, the final result of Independence was so different for both the countries. It took me quite sometime to articulate that in a form of question , while the answers were interesting, from what little I know of India itself, if I were an Englishman I would never leave ‘Hindustan’. What the people answering failed to take into account was that in that era it was ‘Hindustan’ or un-divided India.

Pre-partition map of India
Pre-partition map of India

This map can be found at https://commons.wikimedia.org/wiki/File:British_Indian_Empire_1909_Imperial_Gazetteer_of_India.jpg and is part of quite a few Indian articles. I would urge people to look at the map in-depth. Except for the Central India Agency and Central India Provinces, most of the other regions were quite comfortable weather-wise.

Hence I can’t help but feel the assertion that Britishers didn’t like India (as to live here) slightly revolting. See the excerpt/take on Dale Kennedy ‘s The Magic Mountains: Hill Stations and the British Raj. Berkeley: University of California Press, 1996. xv + 264 pp. . A look at the list of hill stations of divided India is enough to tell that there were lot of places which either were founded by the Britishers or they chose to live there. And this is not all, there are supposed to lot of beautiful places even in Pakistan, especially in North East Frontier, Swat for instance. While today it’s infamous for Taliban and Islamic Terrorism, there was a time it was known for its beauty.

The second most difficult mountain in the world - K2, Pakistan
The second most difficult mountain in the world – K2, Pakistan

Trivia – After Everest, K2 is the smaller one although whatever I have read of people’s accounts, most people who ascended all 14 8,000 metre peaks say K2 is technically more tougher than Everest and after Everest has the highest casualty rate.

Also places like the disputed North half of Pakistan Occupied Kashmir, Gilgit–Baltistan, Extreme northern Punjab of Pakistan , Northern half of Khyber-Pakhtunkhawa province and Northern Balochistan all of these places would have been more than conducive to the Britishers as it is near to the British climate (snow and pleasant weather all year round). It really is a pity that Pakistan chose to become a terrorist state where it could have become one of the more toured places of Asia. I really feel nauseous and sad at the multiple chances that Pakistan frittered away, it could have been something else.