The Right Stuff

For obvious reasons, a couple weeks ago my team made the switch to Git. Hosting a Git server on Windows is by all means possible, and there are two options:

1. Gitosis is a Python-based solution that provides basic Git hosting with per-repository permissions. Shannon Cornish has an excellent two-part guide how to set that up.
2. Gitolite, a Perl-based rewrite of Gitosis, is a more advanced Git server that has a lot more configuration options. For example, it’s possible to specify who is able to force a push to a Git branch, an operation that is possibly problematic when working in teams.

A notable aspect of both solutions is that repository configuration and permissions management is done through Git itself. Over time, you will build a versioned history of the server configuration. Without further ado, let’s get started!

Contents

You’ll see that we have to deal with Cygwin and SSH mostly. Gitolite’s installation is pretty easy and does not require a lot of work by itself. Getting the Windows Server in a condition where it handles SSH takes most of our time.

1. Installing Cygwin
2. Connecting Cygwin to Windows Security
3. Setting Up the SSH Server
4. Enabling SSH Client Access
5. Verifying SSH Password Access
   1. Creating Your SSH Identity
6. Making the SSH Server Aware of Your SSH Identity
7. Installing Gitolite

What You Need

1. A Windows Server (I’m using Windows Server 2008 x86) with permissions to log in as an Administrator.
2. An internet connection to download Cygwin.

Installing Cygwin

1. Download the Cygwin setup program to C:\Cygwin and launch it. For this guide, I’ve used the current version 1.7.2.
2. Select “Install from Internet”, click Next.
3. Leave Root Directory as the default, C:\Cygwin, and install for all users. Click Next.
4. Select C:\Cygwin\pkg as the Local Package Directory. Actually it doesn’t really matter what the directory is, you can delete it after the installation. Click Next.
5. Select the Internet Connection you prefer: Direct, IE Settings or enter a manual proxy. Click Next.
6. Select a mirror near your location, click Next.
7. Acknowledge the “Setup Alert” warning about your installation.
8. In the packages list, select the following packages by clicking on “Skip” text in the “New” column. When you clicked, the version that will be installed is displayed instead of “Skip”.
   • Net | openssh
   • Devel | git
9. Click Next and wait for the installer to complete.
10. You may choose to add icons to the Desktop and Start Menu. Click Complete.

I recommend leaving the setup.exe in place, as you can use the installer to add, remove or upgrade Cygwin packages later.

Repeat the process on your local machine, this time with an extended set of packages to install:

• Net | openssh
• Devel | git
• Devel | git-completion
• Devel | git-gui
• Devel | git-svn (if you want to commit to SVN)
• Devel | gitk
• Editors | vim

Connecting Cygwin to Windows Security

In preparation for the SSH server installation in the next section, we need to provide Cygwin with means to impersonate a SSH user as a Windows user with public key authentication. You can read more about integrating with Windows Security in the Cygwin documentation.

1. On the server, open C:\Cygwin in Explorer.
2. Locate Cygwin.bat, right-click and choose “Run as Administrator”.

   Copying skeleton files.
   These files are for the user to personalise their cygwin experience.
   
   They will never be overwritten nor automatically updated.
   
   `./.bashrc' -> `/home/Administrator//.bashrc'
   `./.bash_profile' -> `/home/Administrator//.bash_profile'
   `./.inputrc' -> `/home/Administrator//.inputrc'
   
   Administrator@GIT-SERVER ~
   $

3. Execute /bin/cyglsa-config

   Warning: Registering the Cygwin LSA authentication package requires
   administrator privileges!  You also have to reboot the machine to
   activate the change.
   
   Are you sure you want to continue? (yes/no)

4. Type yes.

   Cygwin LSA authentication package registered.
   
   Activating Cygwin's LSA authentication package requires to reboot.

5. Reboot the machine.

Setting Up the SSH Server

SSH will encrypt and authenticate connections to your Git repositories. SSH will use public key authentication to check if the user is permitted to access the server. Once the user got past the SSH security check Gitolite will take over handling the request.

When the Git server finished rebooting:

1. Open a new Cygwin Bash prompt by running C:\Cygwin\Cygwin.bat as Administrator.
2. Execute ssh-host-config

   Administrator@GIT-SERVER ~
   $ ssh-host-config
   *** Info: Generating /etc/ssh_host_key
   *** Info: Generating /etc/ssh_host_rsa_key
   *** Info: Generating /etc/ssh_host_dsa_key
   *** Info: Creating default /etc/ssh_config file
   *** Info: Creating default /etc/sshd_config file
   *** Info: Privilege separation is set to yes by default since OpenSSH 3.3.
   *** Info: However, this requires a non-privileged account called 'sshd'.
   *** Info: For more info on privilege separation read /usr/share/doc/openssh/README.privsep.
   *** Query: Should privilege separation be used? (yes/no)

3. Type yes.

   *** Info: Note that creating a new user requires that the current account have
   *** Info: Administrator privileges.  Should this script attempt to create a
   *** Query: new local account 'sshd'? (yes/no)

4. Type yes.

   *** Info: Updating /etc/sshd_config file
   
   
   *** Warning: The following functions require administrator privileges!
   
   *** Query: Do you want to install sshd as a service?
   *** Query: (Say "no" if it is already installed as a service) (yes/no)

5. Type yes.

   *** Query: Enter the value of CYGWIN for the daemon: []

6. Just hit the Return key.

   *** Info: On Windows Server 2003, Windows Vista, and above, the
   *** Info: SYSTEM account cannot setuid to other users -- a capability
   *** Info: sshd requires.  You need to have or to create a privileged
   *** Info: account.  This script will help you do so.
   
   *** Info: You appear to be running Windows 2003 Server or later.  On 2003
   *** Info: and later systems, it's not possible to use the LocalSystem
   *** Info: account for services that can change the user id without an
   *** Info: explicit password (such as passwordless logins [e.g. public key
   *** Info: authentication] via sshd).
   
   *** Info: If you want to enable that functionality, it's required to create
   *** Info: a new account with special privileges (unless a similar account
   *** Info: already exists). This account is then used to run these special
   *** Info: servers.
   
   *** Info: Note that creating a new user requires that the current account
   *** Info: have Administrator privileges itself.
   
   *** Info: No privileged account could be found.
   
   *** Info: This script plans to use 'cyg_server'.
   *** Info: 'cyg_server' will only be used by registered services.
   *** Query: Do you want to use a different name? (yes/no)

7. Type no.

   *** Query: Create new privileged user account 'cyg_server'? (yes/no)

8. Type yes.

   *** Info: Please enter a password for new user cyg_server.  Please be sure
   *** Info: that this password matches the password rules given on your system.
   *** Info: Entering no password will exit the configuration.
   *** Query: Please enter the password:

9. Type and confirm a secure password for the SSH service account. This account will later fork processes on behalf of the user logged in via SSH. You will see another slew of text (which you should read) and then a blinking prompt.
10. Open the Windows Firewall and create an exception for port 22/tcp. Thanks to Marcel Hoyer for the command line equivalent:

    netsh advfirewall firewall add rule dir=in action=allow localport=22 protocol=tcp name="Cygwin SSHD"

11. Execute sc start sshd

Enabling SSH Client Access

Next we will enable SSH access for the git user that will be used to access repositories.

1. Create a new Windows user account named git with a secure password. That user should have no password expiration. You can also delete any group membership.
2. In the Cygwin Bash prompt, execute mkpasswd -l -u git >> /etc/passwd
3. Close the Bash prompt (Ctrl + D) and log off from that machine. The rest of the setup process will be done from your machine.

Verifying SSH Password Access

1. On your workstation, open a Cygwin shell.
2. Execute ssh git@git-server

   you@YOUR-MACHINE ~
   $ ssh git@git-server
   The authenticity of host 'git-server (172.16.0.42)' can't be established.
   RSA key fingerprint is 13:16:ba:00:d3:ac:d6:f2:bf:36:f4:28:df:fc:d5:26.
   Are you sure you want to continue connecting (yes/no)?

3. Type yes.

   Warning: Permanently added 'git-server,172.16.0.42' (RSA) to the list of known hosts.
   git@git-server's password:

4. Enter the password for the git account and you will be presented with a prompt from git-server.

   Copying skeleton files.
   These files are for the user to personalise their cygwin experience.
   
   They will never be overwritten nor automatically updated.
   
   `./.bashrc' -> `/home/git//.bashrc'
   `./.bash_profile' -> `/home/git//.bash_profile'
   `./.inputrc' -> `/home/git//.inputrc'
   
   git@git-server ~
   $ 

5. Press Ctrl + D or execute logout to end the session and you’ll be back on your machine’s prompt.

Creating Your SSH Identity

The next steps to create a SSH identity are optional and only required if you do not already have one, for example to access GitHub. Execute the following commands on your local machine.

1. We’re about to generate a private and public key pair for you that will be used to authenticate SSH connections. Execute ssh-user-config

   *** Query: Shall I create an SSH1 RSA identity file for you? (yes/no)

2. Type no.

   *** Query: Shall I create an SSH2 RSA identity file for you? (yes/no)

3. Type yes.

   *** Info: Generating /home/agross/.ssh/id_rsa
   Enter passphrase (empty for no passphrase):

4. Type and confirm a passphrase. You can omit the passphrase if you want, but that makes you less secure when you loose your private key file.

   *** Query: Do you want to use this identity to login to this machine? (yes/no)

5. Type no. (Unless you want to remotely log in to your workstation with that key. Don't worry, this can be enabled later.)

   *** Query: Shall I create an SSH2 DSA identity file for you? (yes/no)

6. Type no.

   *** Info: Configuration finished. Have fun!

Making the SSH Server Aware of Your SSH Identity

In order to be able to log-in to the Git server as the git user using your SSH identity, execute ssh-copy-id git@git-server. This adds your public key to the list of authorized keys for the git account.

you@YOUR-MACHINE ~
$ ssh-copy-id git@git-server
git@git-server's password:
Now try logging into the machine, with "ssh 'git@git-server'", and check in:

  .ssh/authorized_keys

to make sure we haven't added extra keys that you weren't expecting.

Verifying that public key authentication works, on the next log-in you do not have to enter git@git-server’s password.

you@YOUR-MACHINE ~
$ ssh git@gitserver
Last login: Fri Mar 26 02:04:40 2010 from your-machine

git@git-server ~
$

You are now ready to install Gitolite!

Installing Gitolite

You can install Gitolite in two ways:

• User install
  Installs Gitolite remotely from your workstation. Upgrades to newer versions of Gitolite are easy and are run like the first-time installation. That is, you can just repeat the process outlined below, probably with a new Gitolite version. This installation method requires a SSH login, but we’ve just set-up things this way.
• System install
  Installs Gitolite on the server. Consider that the typical Windows installation where you have to RDP into the server and run things locally. The process is documented in the Gitolite docs.

Next, we’re going to install remotely using the user install method.

1. Open a Cygwin shell on your workstation.
2. Change into the temporary directory be executing cd /tmp
3. Next, clone the Gitolite repository from GitHub by executing git clone git://github.com/sitaramc/gitolite.git
4. Change into the directory containing the Gitolite installer by executing cd gitolite/src
5. As of the time of writing, the Gitolite master branch (the one you just cloned and checked-out) contains a bug preventing successful installation on Cygwin systems. We have to switch to the pu branch, which contains the fix. Execute git checkout -b pu --track origin/pu (Update: The fix is now in the master branch.)
6. Verify that you’re on the pu branch, the asterisk should be in front of the pu line.

   $ git branch
     master
   * pu

7. Install Gitolite for the git user by executing ./gl-easy-install git git-server your-name
   Optional: Execute ./gl-easy-install to see a list of command line options.
   your-name is your name as it will appear in the eventual Gitolite configuration. I went with agross.
8. You will be guided through the installation of Gitolite with every step being explained to you.
9. When you’re at the step which starts with the gitolite rc file needs to be edited by hand and you hit Return, you will be taken to the Vim editor.
   1. In the editor, press the letter i to enter Vim’s insert mode.
   2. Right at the top of the file, insert the following line: $ENV{PATH} = "/usr/local/bin:/bin:/usr/bin";
      This is Cygwin-specific to work around some path related issues.
   3. Press ESC to leave Vim’s insert mode.
   4. Type :w and hit Return to save the file.
   5. Apply any changes to the well-commented configuration you want to make.
      You can navigate using the cursor keys, and enter insert mode by pressing i again. Leave insert mode by hitting ESC.
   6. Type :wq and hit Return to save the file and exit Vim.  To dismiss any changes made in the last step and exit Vim, type :q! and hit the Return key.
10. Continue the installation.

Once the installation is finished, you will find the gitolite-admin repository in your home directory.

To add repositories or change permissions on existing repositories, please refer to the Gitolite documentation. The process uses Git itself, which is awesome:

1. Make changes to your copy of the gitolite-admin repository in your home directory.
2. Commit changes locally.
3. Push to the Gitolite server and let it handle the updated configuration.

Wrapping Up

This guide as been pretty long, longer than I wish it had been. Following Shannon Cornish’s example, I wanted it to be rather too verbose than too short. At least, I did appreciate the detail of Shannon’s instructions when I installed Gitosis back in December. I’ve just begun to grasp the power of Unix – leveraging a set of tiny programs to orchestrate a system.

With the setup you have now in place, you can do anything you like – it’s a complete Git server. However, if you want to publish your server on the internet there’s more you will want to take care of. I will go into that in a future post, detailing some of Cygwin’s security features that helped us reduce the number of attacks on our server. Also, I will take a look at how you can enable the Gitweb Repository Browser using the lighttpd web server.

Today we changed the URL of our Subversion server at work to a new domain. Subversion and TortoiseSVN offer a separate relocate command for that, which basically updates the local working copy metadata.

Not so for Git. Git keeps SVN metadata in two places: the commit log messages themselves which hold a git-svn-id entry for all commits that have been pushed to SVN, and in the .git/config file.

commit e82751b4872142679ba61e26fc0c57e97c698e8f
Author: agross 
Date:   Thu Jun 25 16:44:55 2009 +0000

    Adding FxCop to the code quality task

    git-svn-id: https://your.svn-server/svn/Crimson/trunk@67 8ed4a44c-bfb4-4748-a28a-fad9255c4788

[core]
	repositoryformatversion = 0
	filemode = false
	bare = false
	logallrefupdates = true
	ignorecase = true
[svn-remote "svn"]
	url = https://your.svn-server/svn/Crimson/trunk
	fetch = :refs/remotes/git-svn

To update the SVN URL it’s required to update the Git configuration file (an easy edit) and also to rewrite the commit log messages, updating the values of git-svn-id to reflect the new SVN server URL. The latter can be achieved with the git-filter-branch command which allows you to dissect the project history in interesting ways.

Having several local Git repositories to update, I went for the scripted solution. I found this article on how to change the SVN repository URL and added some scripting goodness to it (aside from fixing the syntactic errors). It worked pretty good for my ~10 repositories.

• You pass two arguments:
  • The old SVN URL, i.e. http://old.server
  • The new SVN URL, i.e. https://new.server
  • The old URL will be matched against git-svn-id entries with a regular expression, and the matched parts get replaced with the new URL.
• Rewrite the commit log messages, thus updating git-svn-id
• Create a backup copy of .git/config
• Replace the old SVN URL in .git/config
• Delete all metadata Git has aquired about SVN
• Rebase against SVN, recreating the SVN metadata

#!/bin/sh

# Must be called with two command-line args.
# Example: git-svn-relocate.sh http://old.server https://new.server
if [ $# -ne 2 ]  
then
  echo "Please invoke this script with two command-line arguments (old and new SVN URLs)."
  exit $E_NO_ARGS
fi  

# Prepare URLs for regex search and replace.
oldUrl=`echo $1 | awk '{gsub("[\\\.]", "\\\\\\\&");print}'`
newUrl=`echo $2 | awk '{gsub("[\\\&]", "\\\\\\\&");print}'`

filter="sed \"s|^git-svn-id: $oldUrl|git-svn-id: $newUrl|g\""
git filter-branch --msg-filter "$filter" -- --all

sed -i.backup -e "s|$oldUrl|$newUrl|g" .git/config

rm -rf .git/svn
git svn rebase

For about six month I’m using Git as my preferred Source Control Management system. That is, while my colleagues at work still work with Subversion and TortoiseSVN, I am leveraging the powers of Git behind the scenes. That is, I work on local (Git) feature branches that I commit to our SVN trunk every once in a while, typically after I finished working on a feature.

If you want to get started with Git, I recommend watching the GitCasts and try one of the Windows distributions, msysgit or Cygwin. (I switched to Cygwin after the msysgit team dropped SVN integration in their 1.6 release.)

Git heavily relies on the command line, and UI tools like gitk and TortoiseGit are not quite there yet. Using a CLI might be something a lot of Windows developers do not look forward to, but you still can configure support for some UI tools. For example, file diffs and merges can be customized to work with any tool (UI and CLI-based). I use SourceGear DiffMerge (free) as my primary diff and merge tool. It’s far better that what comes with TortoiseSVN or Git itself (again CLI-based).

I could only find sparse documentation on how to make DiffMerge work with Git, so I dove into the world of Shell programming and created a couple of simple scripts. Here is what I came up with.

diffmerge-diff.sh

#!/bin/sh

path="$(cygpath $1)"
old="$(cygpath --mixed --absolute "$2")"
new="$(cygpath --mixed --absolute "$5")"

#echo -e "path\n$path"
#echo -e "old\n$old"
#echo -e "new\n$new"

"/cygdrive/C/Tools/DiffMerge/diffmerge.exe" "$old" "$new" --title1="Old" --title2="New $path"

diffmerge-merge.sh

#!/bin/sh

localPath="$(cygpath --mixed --absolute "$2")"
basePath="$(cygpath --mixed --absolute "$1")"
remotePath="$(cygpath --mixed --absolute "$3")"
resultPath="$(cygpath --mixed --absolute "$4")"

if [ ! -f $basePath ]
then
    basePath="$(cygpath --mixed --absolute ~/diffmerge-empty)"
fi

#echo -ne "local\n$localPath\n"
#echo -ne "base\n$basePath\n"
#echo -ne "remote\n$remotePath\n"
#echo -ne "result\n$resultPath\n"

"/cygdrive/C/Tools/DiffMerge/diffmerge.exe" --merge --result="$resultPath" "$localPath" "$basePath" "$remotePath" --title1="Mine" --title2="Merged: $4" --title3="Theirs"

Setting up Git

1. Put the files above in your home directory, either C:\Cygwin\home\<username> (Cygwin) or C:\Users\<username> (msysgit). Make sure the line endings are LF-only (UNIX style) and the file is saved without the UTF byte-order mark (BOM).
2. Create an empty file named diffmerge-empty in the same directory.
3. Modify the scripts to point to the correct location of DiffMerge (see the last lines). Some more modifications will be needed if you’re from the msysgit crowd, as the cygpath utility is not available there.
4. Open a Git CLI and enter the following commands:

   git config --global merge.tool diffmerge
   git config --global mergetool.diffmerge.cmd "~/diffmerge-merge.sh \"\$BASE\" \"\$LOCAL\" \"\$REMOTE\" \"\$MERGED\""
   git config --global mergetool.diffmerge.trustExitCode false 
   git config --global diff.external "C:/Cygwin/home/<username>/diffmerge-diff.sh"

   This should add the following lines to your .gitconfig:

   [merge]
       tool = diffmerge
   [mergetool "diffmerge"]
       cmd = ~/diffmerge-merge.sh \"$BASE\" \"$LOCAL\" \"$REMOTE\" \"$MERGED\"
       trustExitCode = false
   [diff]
       external = C:/Cygwin/home/<username>/diffmerge-diff.sh

Using DiffMerge

Using DiffMerge is pretty easy, the normal git diff and git mergetool commands work as usual, but will now spawn DiffMerge instead of vi (which is the default). If anything does not work as expected, uncomment the #echo lines and use a tool like Process Monitor and Process Explorer to see where things go wrong.

Happy diffing!