MySQL based authentication for SVN

Our company has been using SVN as it's primary source control system. In order to provide secure access to it's repository, SVN was configured to use HTTP based authentication using Apache server's WebDAV module (dav_svn in this case). With this configuration, a password file is created using htpasswd command, which consists of login credentials of all the users. Though it seems pretty straight forward, it has a some limitations that can't be avoided. One of such limitations is that there is no way the end users can change their passwords easily. The only option they have is to call the IT guys if they forget their passwords or the passwords initially assigned to them are really hard to remember.

We were hence decided to find some tool(s) to make end users life easier. During our research, we found that SVN can be authenticated using MySQL database. Since we already have a bug tracking tool using MySQL, our findings provided an additional benefit to the solution that we were looking for - we can have a Single Sign-on setup for SVN and a bug tracking tool.

Once we started digging up how to implement MySQL based authentication for SVN, we landed on a page on Rob Peck's blog. He shared his experience using mod_authnz_external, an Apache module, which allows authentication to be done using any script or program running on existing system. Such external script can be considered as a glue code, according to Rob. Based on his tutorial, we started setting up the environment to test our scenario as a proof of concept.

The configuration

A virtual machine instance was established with the following components;

• Ubuntu Server 64-bit (version 11.10)
• Apache (version 2.2.20)
• MySQL (version 5.1.58)
• SVN (version 1.6.12)
• PHP (version 5.3.6)

In order to match our current configuration, SVN server was then configured to use HTTP based authentication. After that, the following steps were performed;

1. Installing and configuring mod_authnz_external

Instead of getting binaries from third party websites, source was compiled after downloading the latest version of the module (mod_authnz_external-3.2.6, at the time of writing this blog). Installation was done by following the step by step instructions from the site - http://code.google.com/p/mod-auth-external/wiki/Installation

2. Sample MySQL database

A sample database similar to shown below was created.  Password was not encrypted in order to make everything as simple as possible.

Figure 1: Sample Database

3. The glue code (PHP script)

Among various modes of mod_authnz_external, pipe mode uses pwauth format, where it passes the username and password to stdin separating each with a newline. It then uses exit codes to return back to Apache server, which can be used to determine if the login was valid or not. With this information, it is easy to write a script which takes username and password as inputs and then query them against the MySQL database to check if user exists or not and return the exit code accordingly.

Figure 2: PHP script talking to the database

4. Updating dav_svn.conf

The final step is to modify Apache's configuration file to talk to mod_authnz_external. However, in order keep everything simple, dav_svn.conf file was modified in our case. The detail on how to modify configuration file can be found at  - http://code.google.com/p/mod-auth-external/wiki/ConfigApache22

The sample dav_svn.conf looks like as below;

Figure 3: Configuring Apache to talk to mod_authnz_external

Now it's time to restart Apache and SVN starts authenticating using MySQL database.

Cleanup Mac OS X Lion Launchpad

I've been using Mac OS X Lion for a while and I learned a lot about this operating system. One of best features of Lion is a Launchpad. Basically, Launchpad show all the applications that are installed in the Applications folder. Whenever you make an Alias of any application, Launchpad shows duplicate icon of the same application. It continues to show the duplicate icon even if you delete or move the alias. However, surprisingly enough, there is no way that you can delete the icon from Launchpad.

In such case, the only way you can clean-up Launchpad by resetting it and restoring in it's original state as mentioned below. You will however loose all your previous setting like folders etc in Launchpad.

• Fire up a Terminal window and go to your home folder if you are not already in it. (This can be done by the command $ cd ~)
• Go to Library/Application Support/Dock folder.
• Trash all the .db files.
• Restart the dock using killall -KILL Dock (command is case sensitive)

Installing Ruby on Rails in Linux Mint

I am adding new post to my blog almost after a year and half. It's not that I was not involved in any technical work or not learned any thing during this time, but for some strange reason I could not continue writing.

Anyway, here I am trying to learn Ruby on Rails (RoR) because it's been a long time since I've involved in real programming work. For the last few weeks I've been going through articles, tutorials, etc. on Ruby (and Rails) and learned a lot about its architecture. One thing I consistently doing while going through those materials is that when it came to installation and other configuration part, I used to skip the entire chapter. I wasn't realizing what I am missing until recently I tried to install Rails framework on my Linux Mint instance.

I got the ruby installed from Software Manager and rails using the command

$ sudo gem install rails

When rails is installed, I suddenly fired a command to create new website

$ rails new my_first_site -d mysql

Rails threw few strange error messages saying that it can't install mysql gem. Thinking that installing a MySQL server and client tool will help, I fired sudo apt-get install command to get them installed. I got disappointed when I still see those strange messages again. Realizing that it's time to get help from Google and I found the solution in no time. I actually needed MySQL library for Ruby. So, here it goes with the following command;

$ sudo apt-get install libmysql-ruby libmysqlclient-dev

Now it's time to install mysql gem.

$ sudo gem install mysql2

Rails created the new site properly and when I tried to run the server, it again threw strange error message. Don't know why but I always find messages in Linux very strange and cryptic. The message was about missing JavaScript engine ( though I find messages cryptic, I always read them correctly). Now, installing Node.js is simple;

$ sudo apt-get install nodejs

Phew!

Finally, I got the Ruby on Rails framework ready and I am all set for my practice session.

Update: Linux Mint doesn't have MySQL server and client pre-installed. The following command does the trick;

$ sudo apt-get install mysql-server

Multicasting

From the old issue of PCQuest magazine on video streaming;

Using unicast, video contents can be communicated to a single machine on a network—a peer-to-peer communication. For such communication, you need to specify the exact IP address of the target machine.

With broadcast, content is communicated to all the machines on the network. All machines receive the content even if they don’t intend to. Such content is delivered using broadcast address for a network. For example, the broadcast address for a 192.168.1.0 network is 192.168.1.255.

For video streaming, neither unicast nor broadcast may be suitable. You may not want to stream video to only one machine. You may like more than one machine to receive the video stream so that more users can watch the video. If you opt for broadcast, you may end up wasting network bandwidth by streaming heavy content like video to all machines and hence to users who might not be interested in watching the video.

This is where Multicast comes to the rescue. In multicast, the server streams the content to a particular IP in the range of 224.0.0.0 to 239.255.255.255. This IP does not fall in the range of the prescribed IP addresses for computer networks. Hence, content delivered to this IP is not received by any machine on the network. Only when a machine connects to this IP, will it be able to retrieve the content and more than one machine can connect to a multicast IP simultaneously.

Full article can be found here.

Letting Flex know about the date format

When you need to pass the date other than in the US format in Flex application, you can use the following code;

import mx.controls.DateField;

var d:Date = DateField.stringToDate("30-12-2007","DD-MM-YYYY");

Original source of this entry.

How to checkout single file from SVN

I've been using SVN for a long time and I was in the habit of checking out the whole folder even if I needed a single file. This habit of mine was working well until I had to check out the folder with around 200 files with hundreds of megabytes of data. I then realized what I've been missing. I tried to checkout single file but SVN wouldn't let me do that. From my perspective, it should be simple as checking out the folder, but it was not. There could be some reasonable explanation for this limitation but I needed to checkout the file and quickly finish my job. After spending few mins on the Internet, I found the solution and thought it would be wise to share it. So, here it goes.

1. Right click on the folder in which you would like to check out the file and select 'SVN Checkout...' from the menu.
2. In the Checkout screen, in URL of repository text box, fill the URL to the folder in the repository from which you would like to checkout the file.
3. Select 'Only this item' from the drop down menu from 'Checkout Depth' in the same screen. Click 'OK' button.
4. You can now see SVN overlay icon on the folder. Also, you will notice that the folder is still empty. Let's check out the single file now.
5. Now, right click again on the same folder and select 'TortoiseSVN -> Repo-browser' from the menu.
6. Repository Browser will take you directly to the folder from which you would like to checkout the file and you can see the files on the right hand side of the screen.
7. Right click on the desire file and select 'Update item to revision' from the menu. Click 'OK' on the next screen. Voila, you have successfully checked out single file.
8. If you have some more files (but not all) to checkout, repeat the last step.

Although it takes little time to check out a single or few files from the repository, I think it's worth doing this way rather than checking out the whole folder when the folder it huge in size. What do you think?

Dynamic columns in datagrid in Adobe Flex

When we set the dataprovider property of the datagrid, it automatically renders the columns with the default values. For instance, the column title is set as the node name if the dataprovider is XMLListCollection. But we would like to have more control on each column so that we can set it's title, width, and other properties as per our need. For this, we need to create each column dynamically at the run time by taking a first record from the dataprovider. In my same project, as mentioned in the last post, I was using XML output from PHP module to bind with the datagrid. However, this XML is also very dynamic in nature that it has variable numbers of nodes with variable node name based on the parameters selected. Depending on the output, I would also require to set some of the columns as read only when the grid is rendered. The overall idea here is to get the first child node of the XML and generate the array of an Objects with the desired properties set. And then an array of DataGridColumn objects is created and set the property as in the Object created in the first step. Below is the code sample;

private function bindData():void {     myGrid.dataProvider = xmlListCollection;     myGrid.columns = getColumns(getColumnDef(xmlListCollection)); }          //get the first item from the XML to define the columns private function getColumnDef(xmlData:XMLListCollection):Array {     var arrColDef:Array = new Array();     var node:XML = xmlData[0]; //get the first node of the XMLListCollection     var childNodes:XMLList = node.children(); //get its child nodes as an XMLList     var objColDef:Object;                  for each(var xmlColumn:XML in childNodes) //loop over the XMLList     {         objColDef = new Object();         objColDef.dataField = xmlColumn.localName();                          switch(objColDef.dataField)         {             case "name":                 objColDef.headerText = "Full Name";                 objColDef.width = 120;                 objColDef.editable = true;                 break;                                  case "address":                 objColDef.headerText = "Permanent Address";                 objColDef.width = 120;                 objColDef.editable = true;                 break;                                      case "hour":                 objColDef.headerText = "Hour";                 objColDef.width = 120;                 objColDef.editable = false;                 break;                                      case "min":                 objColDef.headerText = "Minute";                 objColDef.width = 120;                 objColDef.editable = false;                 break;                              default :                 objColDef.width = 120;                 objColDef.editable = true;                 break;         }                          arrColDef.push(objColDef);     }     return arrColDef;                     } //Generate the actual datagrid columns private function getColumns(colDef:Array):Array {     var dataGridColumn:DataGridColumn;     var arrColumns:Array = new Array();                  for each (var objColDef:Object in colDef)     {         dataGridColumn = new DataGridColumn();         dataGridColumn.dataField =  objColDef.dataField;         dataGridColumn.headerText = objColDef.headerText;         dataGridColumn.editable = objColDef.editable;         dataGridColumn.width =  objColDef.width;                          arrColumns.push(dataGridColumn);                     }          return arrColumns; }

Deleting multiple records in datagrid in Adobe Flex

Although it sound simple, I had a really hard time finding the solution to delete multiple records in datagrid in one of my projects. I was using XMLListCollection as a dataprovider and a single record could be deleted with removeItemAt(index) method. However, for the multiple selection, Flex stores the indices of the selected rows in an array. If I iterate through these indices and start removing item using the index, as soon as the first record is removed, XMLListCollection re-arrange itself and remaining items get the new index which is not valid against the existing indices in the array. Also, there is no way that we can find if the row is selected just by iterating through the collection of rows. The frustrating part was that Google was not able to provide any answer. So, after spending few hours with couple of failed attempts, something suddenly came to my mind. What-if I start removing item from the bottom of the collection, instead from the top. This will not change the indices of existing items. This worked for items selected in an order. However, if I select items in the datagrid randomly, it fails. Flex pushes the indices of the items to an array as they are selected. That means the indices are not in order. So, there is a method Array.sort() to sort an array. Also, this method accept sortOption to consider the values in the array as numeric. Finally it worked. Below is the sample code;

//get the selected indices in the array var sIndices:Array = myGrid.selectedIndices;                   //since the indices are pushed as they selected, we need to sort them in ascending order sIndices.sort(Array.NUMERIC);                   //get the highest index first and remove the item // i.e. start removing items from the bottom so that change in index won't give any problem for(var index:int = sIndices.length-1; index>=0; index--)      xmlListCollection.removeItemAt(sIndices[index]);

What amazed me the most is that I didn't find a single discussion in any of the websites, forums, blogs about this. Is it only me who want to have this kind of feature in the datagrid or the solution was so simple that people easily implemented it? :-)

Excluding config file from version control

Every application has some kind of configuration file to load settings. For instance, ASP.NET applications make use of web.config file to get the connection string for database operations. Similarly, desktop applications use INI file. While developing the application, each developer use configuration file as per their need. If the source is maintained in source control system, developers have to take extra care not to commit such configuration files. If somebody does it by mistake, everybody's file will be altered while updating from the source control. There is a simple way to avoid such situation. Don't put the configuration file under source control. Instead, put a template of the file, something like "app.config.tmpl". Then, after the initial checkout, have the users do a normal OS copy of the template to the proper filename, like "app.config" and have users customize the copy. The file is unversioned, so it will never be committed.

Retrieve full name from Active Directory using C#

Once upon a time, I was given a task to build a simple attendance system to replace the attendance register in our office. Starting from sourceforge.net, I tried to find open source application to fit our requirement. After spending few hours in the Internet, I could not convinced myself with the applications that I came across. Then I suddenly realized that we are using Active Directory to log in to domain and also thought that if I could get the user detail from AD somehow, I can build a system which does not require any user login. Finally, I was able to find the code to get the full name of the current Windows user and build a small prototype for attendance system. The logic behind this application is to get the user detail from Active Directory and simply log attendance detail to another database. This will reduce the overhead of loggin in to the system as well as user administration part. Below is the code to get the full name of Windows user;

using System.DirectoryServices;

public static string GetFullName(string strLogin)
    {
        string str = "";
        string strDomain;
        string strName;

        // Parse the string to check if domain name is present.
        int idx = strLogin.IndexOf('\\');
        if (idx == -1)
        {
            idx = strLogin.IndexOf('@');
        }

        if (idx != -1)
        {
            strDomain = strLogin.Substring(0, idx);
            strName = strLogin.Substring(idx + 1);
        }
        else
        {
            strDomain = Environment.MachineName;
            strName = strLogin;
        }

        DirectoryEntry obDirEntry = null;
        try
        {
            obDirEntry = new DirectoryEntry("WinNT://" + strDomain + "/" + strName);
            System.DirectoryServices.PropertyCollection coll = obDirEntry.Properties;
            object obVal = coll["FullName"].Value;
            str = obVal.ToString();
        }
        catch (Exception ex)
        {
            str = ex.Message;
        }
        return str;
    }

Some useful tools for Delphi developers

I was always fascinated by the tools that are available to .NET developers which help to increase the productivity, like unit testing framework (NUnit), documenation tool and automated build tool (NANT). So, I started looking for same for Delphi. Starting with the documentation tool, I tried few applications but not so promising. Lastly I found DelphiCodeToDoc and it worked well for me. Though there are few bugs, but the tool is great and it supports JavaDoc style documentation. Again, I found that the author of this application is offering a template for GExperts. Tried that and worked well. Next target was unit testing framework. The only one framework I found was DUnit. Also tried this tool and worked very well for me. Finally, I started searching for automated build tool for Delphi application. The one I tried was WANT. Due to the poor documentation, I was forced to learn the whole building process from ANT website and looking into WANT's own build script. In a few hours, I was able to write a script to build my first test application including unit test with DUnit. Super cool. Somebody said "Sky is the limit". Very true. Since my last unit test used GUIRunner, I wanted to try something with console. Could not find a clue how to do that. Well, after going through the build script of WANT again and again, I noticed that we need to create a DLL version of test application. Tried it and worked well. I will try explain this process in detail in my next post.

Celebrating new year

Believe it or not, we are in the year 2065 here in Nepal. If you do not know, we have our own calendar. I've been planning a vacation for new year for a long time. I wanted to go outside valley in last December but my son was sick that time. So, I did not want to miss our new year this time and made a plan. Our new year was on the 13th of April and we went to Godavari Village Resort to relax. Since we were going with the kid, I did not wanted to go far from the city. The resort is just 7 KM away and very peaceful. I can't explain how much I enjoyed with my family there. My wife and I decided to make a plan for such short trip for every new year (either in English or in Nepali year). Let's see how it goes.

Constitutional Assembly Election in Nepal

Today is the biggest day in the history of Nepal. Today, we are having an election for Constitutional Assembly for the first time. This election will select the group of people who will be responsible to write the new constitution of Nepal. Everybody is hoping for stable political situation and long lasting peace in the country after this election. Till now, everything is going well, except for few minor incidents in the remote part of the country. Also, I cast a vote today for the first time in my life. :-)

New Delphi Magazine

This is good news to all Delphi developers that there is a new magazine about Delphi, "Blaise Pascal". The printed version costs around 40 euro per year. But you can get the download version for FREE. However, you need to pay for the code or program if you decide to go for free version. I am taking this as a good news because I still remember closing down of Delphi magazine last year. I started thinking that nobody uses Delphi anymore but with this magazine (and other development), I already started seeing the light at the end of the tunnel.

Back again

I don't believe that it's been almost 7 months since my last post here. Anyway, lot of things happened in 7 months of time. During this time, I was given more responsibilities in the office. I am now part of a team which is responsible for auditing CMMI implementation. Also, I joined EMBA and my classes started from Feb 17th. We created a group in Google to share information regarding course materials and other stuffs. I am the manager of that group. After joining EMBA, I am feeling young again :-). I remember my collage day and enjoying a lot with new friends there. Most importantly, I now don't have time to watch TV or movies. I have to attend the classes on Saturdays. Although we also have classes on Sundays, it's only in the evening. Thank God. I did not have enough time to spend on photography, so I decided to sell my camera. I will probably save some money and go for dSLR in the near future. ;-) Bought a laptop recently. Now I don't have money for new camera. Damn! These days I am totally into management business. Haven't had chance to look into technical things and feeling myself way to far from the latest development in technology. The only thing I should be focusing now is 'Time Managment'. I sometime feel guilty for not giving enough time to my family. I hope Google will help me finding something from which I can learn to manage my time efficiently.

Management Lesson: Taking charge with programmers

I've been managing various projects for more than a year. I found this article interesting and think it will help those who are in project management business. Please read on.

If you want to communicate with a programmer, you have to take charge. Programmers are a tricky bunch sometimes. But you, not the programmer, are in charge of the project. Although the programmer is in charge of a large portion work, you’re the one responsible of the project if the project fails. You must establish dominance without being too aggressive. Establish five things through your early communications:

• Leadership: Leadership is focused on motivating, aligning objectives, and moving your project team to a destination. Assume that you’re leading the project and that your project team will follow.
• Management: Management is focused on getting results. As a project manager, your core focus is on getting the project successfully completed. Management of a group of programmers means you must see results.
• Discipline: When your programmers aren’t getting their work done as promised, don’t hesitate to discipline according to your human resources guidelines. Be careful about making snap judgments, thought. First, find out why they aren’t completing the work. Were your instructions unclear? Was there a miscommunication on your end? The problem could be yours and not the programmers’.
• Organization: Your ability to communicate, lead, manage, and discipline your project team centers on your organizational skills. Be organized and your project team will respect you for having everything on the ball.
• Balance: In all your decisions you must be fair. Your team of programmers will respect you even more if you show balance and fairness in all of your work assignments and disciplinary actions. Don’t play favorites.

Multiple instances of the application

I wanted to run multiple instances of Skype but couldn't find a way to do it. Although I have polygamy patch for my MSN messenger, I couldn't find similar application to patch Skype. After searching the Internet, I came to know that there is a command line tool to run any application with different Window's user. The command is as below; runas /user: XXX application_executable For me, the solution was just adding one more user to Windows XP system(and assigning password to it too) and by firing following command; runas /user:new_user skype.exe I am not sure if this command works for other version of Windows, but it worked with my Windows XP professional. :-)

Management Lesson: The Law of Diminishing Returns

Time is time. No one can buy more time. You can buy more labor if you think it will help your team do more work faster, but that’s not the same thing as adding time to a project. The law of diminishing returns dictates that adding labor doesn’t exponentially increase productivity; in fact, at some point productivity can even go backwards. For a real-life example, consider that two hardworking and experienced programmers working on a project. In order to finish the project on time, you decide to add one more programmer. Now the programmers may be completing the code more quickly, you decide to add six more programmers so that the project can be finished even sooner. You soon realize that although adding one programmer increased your productivity, adding six more only created chaos with creating a contentious environment. You reached the point of diminishing returns when you added six programmers. Bottom line: You can’t build a house in a single day even with 100 people.

Restructuring SVN repository

Sometime you may need to restructure your SVN repository. I was in the same kind of situation. We rely on SVN and we really trust this software. But the person who created repository made a mistake. He created 3 folders, 'trunk', 'tags' and 'branches' in the hard drive and then created repo in the 'trunk' folder. We didn't notice this until we need to branch our project. After going through some websites and forums, I found the simple method to restructure the repository.
1. First, dump the repository to some file

svnadmin dump /path/to/repo > dumpfile.txt

2. Move your repository folder to somewhere else in order to backup, just in case

mv /path/to/repo /path/to/repo_backup

3. Now, manually change the "Node-path:" and "Copy-to-path:" headers to insert "trunk/" into them

sed -e 's/^\(Node-path: \)\(.*\)/\1trunk\/\2/;s/^\(Copy-from-path:\)\(.*\)/\1trunk\/\2/' dumpfile > dumpfile.new 

If you find above command too hard, just open the dumpfile in any text editor and do find and replace. :-) You should be replacing "Node-path:" with "Node-path:trunk/" and "Copy-to-path:" with "Copy-to-path:trunk/". Simple huh! 4. Create new repository

svnadmin create /path/to/repo

5. Load modified dump file into newly created repository

svnadmin load newdumpfile.txt path/to/repo

6. Finally, import 'tags' and 'branches' folder in the repository. Now, you will see your repository with proper structure in which you can make branches and tags of your project.

Distributed Version Control with SVK

Imagine a situation where some software development company has it's offices located in different part of the world. This company also has central server where the version control software is running. The connection to this server is not always good. In such situation, if each office has its own server having version control system which is a mirror of the central system, the problem due to bad connection can be reduced. For this kind of situation, SVK is the tool everybody should use. SVK allows mirroring of existing remote repositories, creating local branches, working on these branches, and when everything is ready, merging them back into mirrored trunk, which updates the remote repository automatically. I've done some research on SVK for the real world situation mentioned above. The company I am currently working has it's offices in US, India and Nepal. The central server is located in US. But sometimes, due to the problem of connection or by other factors, the main repository is not available. To overcome this problem, we looked for the alternative of making mirror of central repository in both Nepal and India. By having the repository in both locations, access to the repository is local. No more slow connection or time out problem. Since our version control is Subversion, our task was to find the tool(s) which is best suitable for this system. Also, most of the development work is done in Microsoft Windows Platform, we use TortoiseSVN to update from and commit to the repository. It was an advantage to us that SVK works on subversion file system.

Plan of implementation
In order to make SVK work, the plan is to mirror the central repository in the local machine. Then the local branch is created. Instead of using the mirrored trunk, user will be accessing this branch. At the end of the day, when everybody is done with their work, the local branch is merged into mirrored trunk. At the same time, the central repository is automatically updated.

Installing SVK
SVK is written completely in Perl and kind of pain to install. Luckily, in my Fedora Core 4 system, Subversion was already installed with perl binding, all I needed to fire the command;

perl -MCPAN -e 'install SVK'

This command install SVK and all of it's many dependencies.

Setting up local 'depotmap'
To set up local repository, the following command does the trick;

svk depotmap --init

This will create a default depotmap. Alternatively, if you want to make a depotmap to the desire location, just issue the command

svk depotmap

This will open a default text editor and let you edit the depotmap name and the location.

Mirroring the central repository

svk mirror svn://mydomain.com/myproject/trunk //localrepo/trunk

To ensure that mirror has been setup, svk mirror --list command can be used. Now that the mirror has been set up, the local copy is sync with mirror using

svk sync //localrepo/trunk

Creating a local branch
The first thing need to be done before start working is to create a local branch. Doing this is same as in Subversion.

svk cp -m "local branch" //localrepo/trunk /localrepo/localbranch

You can check the branch by issuing the following command;

svk ls //localrepo/localbranch

Now user is able to work with the local branch using TortoiseSVN or anyother Subversion client tool.

Merging from local branch to the mirror
At the end of the day, after everybody is done with the work, local branch must be merged to the mirror. This can be done with the command svk smerge command.

svk smerge -C //localrepo/localbranch //localrepo/trunk

The C option checks for the conflict that may have. Now, if there is no conflict, the actual merger can be done using following command;

svk smerge -l //localrepo/localbranch //localrepo/trunk

This command also updates the repository automatically that the mirror points to.