Experiences in the community

Just another WordPress.com weblog

Compilation and DVCS -1

Hi all,
A comment from an outsider’s point of view about the need for compiling programs and the process using various DVCS and making a compilation/development environment.

Before I begin, I am not a programmer nor much of a technical person.

Many years ago, when I started with GNU/Linux one of the things I hated about it apart from the slightly grubby interface was lack of games. By games I do/did not mean puzzles and casual games but a game with some rich experiences, a la turn-based RPG and its ilk. There are exceptions such as freedink but sadly there are few exceptions. Another quite feature-complete is freedroidrpg but that’s about it.

Let me take a moment to explain what do I look for when playing rpg games. Of course lot of it comes from my gaming history.

Expectations :-

a. Rich story
b. Good humor
c. Lots of quests
d. Experience building but in a fun way. Not just mindless grinding.
e. One of the ways which I think is/could be to have more rounded individuals. In most of the games he is just a hunter/gatherer. Ideally he should be a multi-tasker and when he improves those skills, it also could benefit in his fighting experience.

Anyways, I do not want to delve much into it as that would take atleast couple of pages if I were to go deeper in that.

While trying to discover new games I discovered that while full free and open-source games are not yet available, quite a few of them are under development and of course they need developers, story writers, artists,testers and documentation. While I have none of the skills of the first three, I felt I could help a bit in testing and documentation provided I found a way to run the latest upstream.

While for some time I tried using the latest release put up by the developers as development/alpha/beta release to the wider testing and gaming community. It became quickly apparent to me that this was not good enough. As developers are/were always in short supply many of these projects would take too long to make a release and even if the release was made many a times it used to be only for specific distributions and specific architecture (mostly either Fedora or/and Ubuntu 32-bit release along with Windows).

What was also frustrating for me if I did find any issue/s and reported it, sometimes it used to be reported as fixed in the trunk version (where the actual development of the game/project was happening) . Slowly I understood that I would need to learn to be able to download straight from the trunk, have a development build environment ready (even if I’m not programming anything), compile it (do something that would make the game work.) and then see if I find any glitches or not and report whatever I felt was an issue (or atleast discuss) with the developers about it. This also has another effect, it motivates the development of the game as well as the developers for putting time into developing a game.

The first project with which I was able to get a proper response and support is the project dawn-rpg.

It took me quite sometime to learn about subversion and the process called compilation.

One of the first things I learnt is to make a unique directory where you are going to play with games or other applications you wish to download. I made an arbitrary place at /usr/local/src/ where all the different games/projects I am going to download/compile will remain. I made sure that this directory and everything in it has normal user privileges :-

/usr/local$ ll -h
total 32K
drwxr-sr-x 2 root staff 4.0K Jun 15 19:19 bin
drwxrwsr-x 2 root staff 4.0K Dec 1 2010 etc
drwxrwsr-x 2 root staff 4.0K Dec 1 2010 games
drwxrwsr-x 3 root staff 4.0K Dec 25 2010 include
drwxrwsr-x 11 root staff 4.0K Jul 4 03:47 lib
lrwxrwxrwx 1 root staff 9 Dec 1 2010 man -> share/man
drwxrwsr-x 2 root staff 4.0K Dec 1 2010 sbin
drwxrwsr-x 17 root staff 4.0K Jul 4 23:10 share
drwxrwsr-x 49 shirish staff 4.0K Jul 7 03:32 src

As can be seen the only one which does not need root/sudo privileges is the src directory.

drwxrwsr-x 49 shirish staff 4.0K Jul 7 03:32 src

While I learning about it I also came to know that subversion was the most common tool which was used till around 5 years.

I also understood about few essential packages which are necessary for whatever game you wanna build/compile from some package repository :-

$ sudo aptitude install build-essential ccache libtool checkinstall subversion liblua5.1-0 liblua5.1-0-dev liblua50 liblualib50 lua5.1 libsdl-gfx1.2-4 libsdl-image1.2 libsdl-mixer1.2 libsdl-ttf2.0-0 libsdl1.2debian libfreetype6

For many a games the -dev packages are also required :-

$ sudo aptitude install libsdl-gfx1.2-dev libsdl-image1.2-dev libsdl-mixer1.2-dev libsdl-ttf2.0-dev libfreetype6-dev

While I’m not really familiar with either the LUA library or/and the SDL library, what little I have been able to understood is that while LUA is used mostly for the logic, AI and how and what options will happen in a game or an application, SDL on the other hand makes sure that the user is able to see the animation, the video and is able to use the mouse or the keyboard when playing the game. It has a supportive function but a very important one. While both can be used independently they are usually used together in games.

One of the other important library that is used from time to time is the openGL library .

$ sudo aptitude install libglu1-mesa libglu1-mesa-dev mesa-common-dev python-opengl

The reason why we made a specific directory is that each game/project would need its own directory. Continuing the above example of using dawn-rpg project as the example project, the first thing to be done is to download the latest/bleeding edge source to a new directory.

/usr/local/src$ svn checkout https://dawn-rpg.svn.sourceforge.net/svnroot/dawn-rpg/trunk dawn-rpg

This command simply makes a clone/duplicate of the source-code and any art that the project has into a new sub-directory called dawn-rpg.

When you start the process you see something like this on the command-line :-

A dawn-rpg/errorparser.sh
A dawn-rpg/mkinstalldirs
A dawn-rpg/AssetsLicenceInventory.txt
A dawn-rpg/Makefile.am
A dawn-rpg/Makefile.in
A dawn-rpg/configure

If and when the subversion is successfully cloned/duplicated to a directory in your system it would subversion would announce something like

Checked out revision 890

There are a couple of more commands that help in understanding about the project which I use with almost every project.

/usr/local/src/dawn-rpg$ svn info
Path: .
URL: https://dawn-rpg.svn.sourceforge.net/svnroot/dawn-rpg/trunk
Repository Root: https://dawn-rpg.svn.sourceforge.net/svnroot/dawn-rpg
Repository UUID: b4d8fe5f-8a90-43c9-9084-dcb629e63849
Revision: 890
Node Kind: directory
Schedule: normal
Last Changed Author: zaldron
Last Changed Rev: 890
Last Changed Date: 2011-06-26 23:04:23 +0530 (Sun, 26 Jun 2011)

Now as I tend to forget the canonical address of the repository (after playing/having more repos) its amazing to have this command at hand. The output gives quite a bit of info.

It shares :-

URL: https://dawn-rpg.svn.sourceforge.net/svnroot/dawn-rpg/trunk

The canonical address in case you, as a user forget about it. For me I have to not remember it longer.


Last Changed Author: zaldron
Last Changed Rev: 890
Last Changed Date: 2011-06-26 23:04:23 +0530 (Sun, 26 Jun 2011)

When was this project last touched/committed/some changes made to it and who was the author/committer who made the change.

But while this is good in itself, it still does not give the whole picture. Hence enter svn log

/usr/local/src/dawn-rpg$ svn log > locallog.txt

I dunno when svn log made it into the picture but seems it was an afterthought which other dvcs’es have quickly submerged/made it an essential feature. For what little I know about Subversion, Subversion keeps history on the server where the repository is rather than making and keeping it at your end/local/offline. Which means I have to be online anytime I want to svn log to see or check back what the history was. The above command simply asks for the whole log and sends the output from the server to a file called locallog.txt.

Now each time I have to sync to the server I just have to svn up . If you look at the svn help up (update command) you will find the following info. :-

$ svn help update
update (up): Bring changes from the repository into the working copy.
usage: update [PATH...]

If no revision is given, bring working copy up-to-date with HEAD rev.Else synchronize working copy to revision given by -r.

For each updated item a line will start with a character reporting the action taken.These characters have the following meaning:

A Added
D Deleted
U Updated
C Conflict
G Merged
E Existed

A character in the first column signifies an update to the actual file, while updates to the file's properties are shown in the second column. A 'B' in the third column signifies that the lock for the file has been broken or stolen.

If --force is used, unversioned obstructing paths in the working copy do not automatically cause a failure if the update attempts to add the same path. If the obstructing path is the same type (file or directory) as the corresponding path in the repository it becomes versioned but its contents are left 'as-is' in the working copy. This means that an obstructing directory's unversioned children may also obstruct and become versioned. For files, any content differences between the obstruction and the repository are treated like a local modification to the working copy. All properties from the repository are applied to the obstructing path. Obstructing paths are reported in the first column with code 'E'.

Use the --set-depth option to set a new working copy depth on the targets of this operation.

I do not want to go into the intricacies of subversion as frankly I do know much about it. Anyways as a user I just do a simple

/usr/local/src/dawn-rpg$svn up

to see if anything new has been committed or not. I am a pretty lazy person and I get bored easily so what I did was by taking help of the good folks of Debian forums added the following in my .bashrc :-

svn ()
{
if [ up = "" ] ; then
command svn update
command svn log > locallog.txt
else
command svn "$@"
fi
}

I do not understand much but from what I could/can read it simply makes the switch ‘up’ to do multiple things . It updates/freshens the working directory as well as downloads the whole history from the subversion repository.

I have to explain why the subversion log is important. When developers change something, there is a possibility to forget what the change is about and what files/directories were changed. For instance the game project dawn-rpg that we are using as an example, the directory features 2924 items (inclusive of both files and directories) totalling over 120+ MiB . So if you do not write about what change you made it confuses you, the developer, more if there are more developers as well as users. Also nobody knows who worked on it last so if there is an issue/bug those changes cannot be held back. I am sure there are a few more reasons but this is all I can/could remember on a spur of the moment.

This is the last commit made by the developer by the zaldron :-


-----------------------------------
r890 | zaldron | 2011-06-26 23:04:23 +0530 (Sun, 26 Jun 2011) | 1 line

Moved editor methods from header to src-file to cleanup header
-------------------------------------

While this is the first commit made to the project using subversion :-


This commit was generated by cvs2svn to compensate for changes in r2,
which included commits to RCS files with non-trunk default branches.

-----------------------------------
r1 | (no author) | 2009-06-08 22:17:11 +0530 (Mon, 08 Jun 2009) | 1 line

Standard project directories initialized by cvs2svn.
-------------------------------------

which means the project has been in development even before that and it was using a less feature-full version control system called cvs. I do not want to go into cvs vs subversion as there are countless articles about it and even subversion seems to be on the way out. Comparing version control systems is outside the scope of the article/post although I may do some informally while looking up some of them.

Anyways moving along, most of these projects/games/applications have a configure script. Remember the


A dawn-rpg/configure

which was one of the last files which I downloaded when downloading the source-code from the subversion repository for the first time.

Let’s look in the configure file/script to see what it does :-


! /bin/sh
# Guess values for system-dependent variables and create Makefiles.
# Generated by GNU Autoconf 2.68 for dawn 0.0.46.
#
# Report bugs to .
#
#
# Copyright (C) 1992, 1993, 1994, 1995, 1996, 1998, 1999, 2000, 2001,
# 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010 Free Software
# Foundation, Inc.
#
#
# This configure script is free software; the Free Software Foundation
# gives unlimited permission to copy, distribute and modify it.
## -------------------- ##
## M4sh Initialization. ##
## -------------------- ##

This is just the first comments for the configure script and tells/shares what it does. It basically approximates and tries to figure out if all the libraries and dependencies are there and what sort of environment is there. Many of the developers use gnu autoconf or sometimes gnu autoreconf to make the configure scripts. I will share about autoreconf in another project where one has to also do this as an additional step.

When you run the .configure script/file you get output something like this on the stdout/command-line :-


/usr/local/src/dawn-rpg$ ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking for g++... g++
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking for style of include used by make... GNU
checking dependency style of g++... gcc3
checking for gcc... gcc
checking whether we are using the GNU C compiler... yes
........
........
checking for strspn... yes
configure: creating ./config.status
config.status: creating Makefile
config.status: executing depfiles commands

I have truncated the output from the .configure script. The actual configure script output for the dawn-rpg project goes to 100 + lines and in some projects even more depending on how many things it has to check for. The basics it does is look and check if whatever package/library dependencies are written in the .configure script . The last three lines are the most interesting, the config.status file it generates, the makefile it generates. There is also a config.log file which is generated by it.

Let’s look at the three files to see what is in there. First up is the config.status file :-


#! /bin/bash
# Generated by configure.
# Run this file to recreate the current configuration.
# Compiler output produced by configure, useful for debugging
# configure, is in config.log if it exists.

The second one is config.log and this is perhaps one of the more important files :-

This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.

It was created by dawn configure 0.0.46, which was
generated by GNU Autoconf 2.68. Invocation command line was


$ ./configure

## --------- ##
## Platform. ##
## --------- ##

hostname = deb-home
uname -m = x86_64
uname -r = 2.6.39-2-amd64
uname -s = Linux
uname -v = #1 SMP Tue Jul 5 02:51:22 UTC 2011
.....
.....
#define HAVE_STRSPN 1

configure: exit 0

As tradition/make it easy for self I just truncated both config.status and config.log as both are quite long in themselves. The idea simply is that the configure script should end in success . When the .configure script ends with making a makefile and something called depfile commands (this I have no idea about) this means the configure script has done its work.

Also at the end of config.log the configure:exit 0 is doing nothing but telling that the configure script ended in success/ran successfully.

In case if there is an issue while running configure either due to a library missing or some difference between library versions or something else you might get output like/similar to :-

checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking for lt_dlinit in -lltdl... no
Adonthell requires libltdl. Exitting...

As can be seen in this specific .configure script instance of a project there was either a library not installed or some issue. Talking with the developers in this specific instance it was something in the configure script that was wrong and was fixed by the developer later on.

The last and perhaps most important file is the Makefile that is generated after running the .configure script. The Makefile uses instructions from both the configure script as well as the Makefile.in and Makefile.am files which were made by the developer :-


Makefile.in generated by automake 1.11.1 from Makefile.am.
# Makefile. Generated from Makefile.in by configure.

# Copyright (C) 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002,
# 2003, 2004, 2005, 2006, 2007, 2008, 2009 Free Software Foundation,
# Inc.
# This Makefile.in is free software; the Free Software Foundation
# gives unlimited permission to copy and/or distribute it,
# with or without modifications, as long as this notice is preserved.

# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY, to the extent permitted by law; without
# even the implied warranty of MERCHANTABILITY or FITNESS FOR A
# PARTICULAR PURPOSE.

As before I have just used the comments used in the beginning to try to explain what the file is about. Those who understand gcc and its brethren may be able to make more sense of it.

The best definition of a makefile is perhaps given by the make documentation.


To prepare to use `make', you must write a file called the
"makefile" that describes the relationships among files in your program and provides commands for updating each file. In a program, typically,the executable file is updated from object files, which are in turn made by compiling source files.

Once the configure script is run one runs the ‘make’ command. When you run that command it gives a lot of output which unless you are a software developer is simply not understandable. At least for me. The only thing to know here is that at the end it should simply throw you back at the command-line without giving/showing you any error. If it spits out any error then it means something is wrong somewhere and you need to connect with the developer for the same.

When you get errors during the make process you get output similar to this :-


/usr/include/c++/4.6/bits/stl_list.h:1628:1: error: expected declaration before '}' token
make[3]: *** [libadonthell_gui_la-decoration.lo] Error 1
make[3]: Leaving directory `/usr/local/src/adonthell-build/src/gui'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/usr/local/src/adonthell-build/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/usr/local/src/adonthell-build'
make: *** [all] Error 2

The point to note is the error and specifically the errors reported against make. This gives a hint that something is wrong somewhere, either at user-end or the developer end.

If you are a developer contributor you try to fix it, if not at least report it to the developer giving all the info. you have.

Once the make process finishes successfully you have a binary/executable of the game/application which you built.

While this concludes the first part of the post which stressed more on the compilation process, the second part would venture more into couple of more version control systems which are becoming more mainstream.

I have left quite a lot of switches when running either configure and make which could/would make for more faster/efficient programs but that needs more time and efforts than an average joe would make.

At the very end, there is another command which is also very important. This command is known as make clean. Using the same project dawn-rpg I get :-


/usr/local/src/dawn-rpg$ make clean
test -z "dawn-rpg " || rm -f dawn-rpg
rm -f *.o

The above is simply making sure that all shared libraries (*.o) are deleted within the directory making it pristine/clean. As told in the info document (partial extract below) one do/can make elaborate rules for cleaning depending on the complexity of the project/game/application.

As always the info make document has the best explanation of running make clean. From the doc :-


2.7 Rules for Cleaning the Directory
====================================

Compiling a program is not the only thing you might want to write rules
for. Makefiles commonly tell how to do a few other things besides
compiling a program: for example, how to delete all the object files
and executables so that the directory is `clean’.

It is and would be highly recommended to install the make-doc package if you want/intend to use make for your project/game.

$ aptitude show make-doc
Package: make-doc
State: installed
Automatically installed: no
Version: 3.81-5
Priority: optional
Section: non-free/doc
Maintainer: Manoj Srivastava
Uncompressed Size: 1,982 k
Depends: dpkg (>= 1.15.4) | install-info
Suggests: make
Replaces: make (< 3.80+3.81.rc2-1)
Description: Documentation for the GNU version of the "make" utility.
This package contains the documentation for GNU Make. The upstream sources
for this package are available at the location ftp://ftp.gnu.org/gnu/make/.
This package has been stripped down to only contain the documentation; since
the documentation is covered under a non-free license and thus has been
moved to non-free/doc. Make itself remains in Debian.

Tags: devel::doc, devel::lang:c, made-of::html, made-of::postscript,
role::documentation, suite::gnu

The icing on the cake is that the
package is maintained by a fellow Indian Manoj Srivastava.

That’s about it for now.

Single Post Navigation

One thought on “Compilation and DVCS -1

  1. Pingback: Compilation and DVCS-2 « Experiences in the community

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: