yaaay! http is finally done!

doc/dev-status.mkd

@@ -2,7 +2,6 @@
 
 Not yet done (will be tackled in this order unless someone asks):
 
-  * smart http
   * svnserve (someone is testing it)
   * mechanism for ADCs using unchecked arguments -- this is not just a matter
     of writing it; I have to think about *how* it will be done.  (AFAIK the
@@ -33,3 +32,4 @@ Done:
   * migration documentation
   * distro packaging instructions
   * migration advice for common cases
+  * smart http

doc/g3-master-toc.mm

@@ -152,7 +152,6 @@
 </node>
 <node CREATED="1333301296248" ID="ID_571518549" MODIFIED="1333526198202" POSITION="right" TEXT="TBD">
 <node CREATED="1333327082853" ID="ID_488765250" MODIFIED="1333327089444" TEXT="log file format, LOG_EXTRA"/>
-<node CREATED="1333301298504" ID="ID_60946303" MODIFIED="1333301300418" TEXT="smart http"/>
 <node CREATED="1333301308136" ID="ID_1900285587" MODIFIED="1333301311863" TEXT="hub"/>
 <node CREATED="1333328274461" ID="ID_248606591" MODIFIED="1333328277083" TEXT="mob branches"/>
 <node CREATED="1333328277387" ID="ID_1027016949" MODIFIED="1333328280083" TEXT="password access"/>

doc/http.mkd

@@ -0,0 +1,70 @@
+# how to setup gitolite to use smart http mode
+
+**Note**: "smart http" refers to the feature that came with git 1.6.6, late
+2009 or so.  The base documentation for this is `man git-http-backend`.  Do
+**NOT** read `Documentation/howto/setup-git-server-over-http.txt` and think
+that is the same or even relevant -- that is from 2006 and is quite different
+(and arguably obsolete).
+
+## WARNINGS and important notes
+
+  * Please read [authentication versus authorisation][auth] first, and make
+    sure you understand what is gitolite's responsibility and what isn't.
+
+  * The 'gitolite' command (for example, 'gitolite compile', 'gitolite
+    query-rc', and so on) *can* be run on the server, but it's not
+    straightforward.  Assuming you installed using the exact same values as in
+    this document:
+
+      * get a shell by using, say, `su -s /bin/bash - apache`
+      * run `export HOME=$HOME/gitolite-home`
+      * run `export PATH=$PATH:$HOME/bin`
+
+    Now you can run `gitolite <subcommand>`
+
+  * I have tested only on stock Fedora 16; YDMV
+
+  * As before, I have not tried making repos available to both ssh *and* http
+    mode clients but it ought to work.  If you managed it, I'd appreciate a
+    doc patch describing how you did it.
+
+## additional requirements
+
+  * requires `GIT_PROJECT_ROOT` (see "man git-http-backend" for what this is)
+    set explicitly (i.e., it is no longer optional).  Please set it to some
+    place outside apache's `DOCUMENT_ROOT`.
+
+## assumptions:
+
+  * apache 2.x and git installed.
+  * httpd runs under the "apache" userid; adjust instructions below if not.
+  * similarly for "/var/www" and other file names/locations.
+
+## instructions
+
+The detailed instructions I used to have in g2 have now been replaced by a
+script called `t/smart-http.root-setup`.  **Do NOT run this script as is -- it
+is actually meant for my testing setup and deletes stuff**.  However, it does
+provide an excellent (and working!) narration of what you need to do to
+install gitolite in smart http mode.
+
+Make a copy of the script, go through it carefully, (possibly removing lines
+that delete files etc.), change values per your system, and only then run it.
+
+## usage
+
+Git URLs look like `http://user:password@server/git/reponame.git`.
+
+The custom commands, like "info", "expand" should be handled as follows.  The
+command name will come just after the `/git/`, followed by a `?`, followed by
+the arguments, with `+` representing a space.  Here are some examples:
+
+    # ssh git@server info
+    curl http://user:password@server/git/info
+    # ssh git@server info repopatt
+    curl http://user:password@server/git/info?repopatt
+    # ssh git@server info repopatt user1 user2
+    curl http://user:password@server/git/info?repopatt+user1+user2
+
+With a few nice shell aliases, you won't even notice the horrible convolutions
+here ;-)  See t/smart-http for a couple of useful ones.

doc/index.mkd

@@ -46,7 +46,7 @@ most people see:
   * Can be installed without root access, assuming git and perl are already
     installed.
   * Authentication is most commonly done using sshd, but you can also use
-    httpd if you prefer (this may require root access).
+    [http][] if you prefer (this may require root access).
 
 ## #contact contact
 

doc/install.mkd

@@ -45,8 +45,7 @@ Notes:
   * any Unix system with a posix compatible "sh".
   * git version 1.6.6 or later
   * perl 5.8.8 or later
-  * openssh (almost any version).  Optional if you're using the http backend
+  * openssh (almost any version).  Optional if you're using [smart http][http]
-    (which is still a TODO item!)
   * a dedicated Unix userid to be the hosting user, usually "git" but it can
     be any user, even your own normal one.  (If you're using an RPM/DEB the
     install probably created one called "gitolite").

doc/setup.mkd

@@ -37,5 +37,7 @@ after the install as well:
 
   * to replace a [lost admin key][lost-key].
 
+  * to setup gitolite for http mode (run 'gitolite setup -h' for more info)
+
 When in doubt, run 'gitolite setup' anyway; it doesn't do any harm, though it
 may take a minute or so if you have more than a few thousand repos!

src/gitolite-shell

@@ -8,6 +8,10 @@ use FindBin;
 BEGIN { $ENV{GL_BINDIR} = $FindBin::RealBin; }
 BEGIN { $ENV{GL_LIBDIR} = "$ENV{GL_BINDIR}/lib"; }
 use lib $ENV{GL_LIBDIR};
+
+# set HOME
+BEGIN { $ENV{HOME} = $ENV{GITOLITE_HTTP_HOME} if $ENV{GITOLITE_HTTP_HOME}; }
+
 use Gitolite::Rc;
 use Gitolite::Common;
 use Gitolite::Conf::Load;
@@ -48,7 +52,20 @@ sub in_file {
 }
 
 sub in_http {
-    _die 'http not yet implemented...';
+    http_setup_die_handler();
+
+    _die "GITOLITE_HTTP_HOME not set" unless $ENV{GITOLITE_HTTP_HOME};
+
+    _die "fallback to DAV not supported" if $ENV{REQUEST_METHOD} eq 'PROPFIND';
+
+    # fake out SSH_ORIGINAL_COMMAND and SSH_CONNECTION when called via http,
+    # so the rest of the code stays the same (except the exec at the end).
+    http_simulate_ssh_connection();
+
+    $ENV{REMOTE_USER} ||= $rc{HTTP_ANON_USER};
+    @ARGV = ( $ENV{REMOTE_USER} );
+
+    return 'http';
 }
 
 sub in_ssh {
@@ -104,6 +121,8 @@ sub main {
         gl_log( "pre_git", $repo, $user, $aa, 'any', "-> $ret" );
     }
 
+    exec $ENV{GIT_HTTP_BACKEND} if $ENV{REQUEST_URI};
+
     trigger( 'PRE_GIT', $repo, $user, $aa, 'any', $verb );
     my $repodir = "'$rc{GL_REPO_BASE}/$repo.git'";
     _system( "git", "shell", "-c", "$verb $repodir" );
@@ -146,3 +165,67 @@ sub sanity {
     _die "'$repo' ends with a '/'"         if $repo =~ m(/$);
     _die "'$repo' contains '..'"           if $repo =~ m(\.\.$);
 }
+
+# ----------------------------------------------------------------------
+# helper functions for "in_http"
+
+sub http_setup_die_handler {
+
+    $SIG{__DIE__} = sub {
+        my $service = ($ENV{SSH_ORIGINAL_COMMAND} =~ /git-receive-pack/ ?  'git-receive-pack' : 'git-upload-pack');
+        my $message = shift; chomp($message);
+        print STDERR "$message\n";
+
+        # format the service response, then the message.  With initial
+        # help from Ilari and then a more detailed email from Shawn...
+        $service = "# service=$service\n"; $message = "ERR $message\n";
+        $service = sprintf("%04X", length($service)+4) . "$service";        # no CRLF on this one
+        $message = sprintf("%04X", length($message)+4) . "$message";
+
+        http_print_headers();
+        print $service;
+        print "0000";       # flush-pkt, apparently
+        print $message;
+        print STDERR $service;
+        print STDERR $message;
+        exit 0;     # if it's ok for die_webcgi in git.git/http-backend.c, it's ok for me ;-)
+    }
+}
+
+sub http_simulate_ssh_connection {
+    # these patterns indicate normal git usage; see "services[]" in
+    # http-backend.c for how I got that.  Also note that "info" is overloaded;
+    # git uses "info/refs...", while gitolite uses "info" or "info?...".  So
+    # there's a "/" after info in the list below
+    if ($ENV{PATH_INFO} =~ m(^/(.*)/(HEAD$|info/refs$|objects/|git-(?:upload|receive)-pack$))) {
+        my $repo = $1;
+        my $verb = ($ENV{REQUEST_URI} =~ /git-receive-pack/) ?  'git-receive-pack' : 'git-upload-pack';
+        $ENV{SSH_ORIGINAL_COMMAND} = "$verb '$repo'";
+    } else {
+        # this is one of our custom commands; could be anything really,
+        # because of the adc feature
+        my ($verb) = ($ENV{PATH_INFO} =~ m(^/(\S+)));
+        my $args = $ENV{QUERY_STRING};
+        $args =~ s/\+/ /g;
+        $ENV{SSH_ORIGINAL_COMMAND} = $verb;
+        $ENV{SSH_ORIGINAL_COMMAND} .= " $args" if $args;
+        http_print_headers();  # in preparation for the eventual output!
+    }
+    $ENV{SSH_CONNECTION} = "$ENV{REMOTE_ADDR} $ENV{REMOTE_PORT} $ENV{SERVER_ADDR} $ENV{SERVER_PORT}";
+}
+
+my $http_headers_printed = 0;
+sub http_print_headers {
+    my($code, $text) = @_;
+
+    return if $http_headers_printed++;
+    $code ||= 200;
+    $text ||= "OK - gitolite";
+
+    $|++;
+    print "Status: $code $text\r\n";
+    print "Expires: Fri, 01 Jan 1980 00:00:00 GMT\r\n";
+    print "Pragma: no-cache\r\n";
+    print "Cache-Control: no-cache, max-age=0, must-revalidate\r\n";
+    print "\r\n";
+}

src/lib/Gitolite/Setup.pm

@@ -7,9 +7,11 @@ package Gitolite::Setup;
 Usage:  gitolite setup [<option>]
 
     -pk, --pubkey <file>        pubkey file name
+    -a, --admin <name>          admin name
 
-Setup gitolite, compile conf, and fixup hooks.  The pubkey is required on the
+Setup gitolite, compile conf, and fixup hooks.  Either the pubkey or the admin
-first run.
+name is required on the first run, depending on whether you're using ssh mode
+or http mode.
 
 Subsequent runs:
 
@@ -87,7 +89,7 @@ sub setup_glrc {
 
 sub setup_gladmin {
     my ( $admin, $pubkey, $argv ) = @_;
-    _die "no existing conf file found, '-a' required"
+    _die "no existing conf file found, '-pk' or '-a' required"
       if not $admin and not -f "$rc{GL_ADMIN_BASE}/conf/gitolite.conf";
 
     # reminder: 'admin files' are in ~/.gitolite, 'admin repo' is

t/smart-http

@@ -0,0 +1,90 @@
+#!/bin/bash
+
+die() { echo "$@"; exit 1; }
+
+# git clone `url u1 r1`
+url() {
+    echo http://$1:$1@localhost/git/$2.git
+}
+
+# `cmd sitaram info`
+cmd() {
+    c="curl http://$1:$1@localhost/git"
+    shift
+    c="$c/$1"
+    shift
+
+    if [ -n "$1" ]
+    then
+        c="$c?$1"
+        shift
+    fi
+    while [ -n "$1" ]
+    do
+        c="$c+$1"
+        shift
+    done
+
+    echo $c
+}
+
+export tmp=$(mktemp -d);
+trap "rm -rf $tmp" 0;
+cd $tmp
+
+tsh "plan 28"
+
+tsh "
+    ## ls-remote admin admin
+    git ls-remote `url admin gitolite-admin`
+        ok
+        /HEAD/
+        /refs.heads.master/
+    ## clone
+    git clone `url admin gitolite-admin`
+        ok
+        /Cloning into/
+    ls -al gitolite-admin/conf
+        /gitolite.conf/
+" || die "step 1"
+
+cd gitolite-admin
+echo repo t2 >> conf/gitolite.conf
+echo 'RW+  = u1 u2' >> conf/gitolite.conf
+
+tsh "
+    ## add, commit, push
+    git add conf/gitolite.conf
+        ok
+        !/./
+    git commit -m t2
+        ok
+        /1 file.*changed/
+    git push
+        ok
+        /Initialized.*var.www.gitolite-home.repositories.t2.git/
+        /To http:..admin:admin.localhost.git.gitolite-admin.git/
+        /master -. master/
+    ## various ls-remotes
+    git ls-remote `url u1 gitolite-admin`
+        !ok
+        /FATAL: R any gitolite-admin u1 DENIED by fallthru/
+    git ls-remote `url u1 t2`
+        ok
+        !/./
+    git ls-remote `url u2 t2`
+        ok
+        !/./
+    git ls-remote `url u3 t2`
+        !ok
+        /FATAL: R any t2 u3 DENIED by fallthru/
+    ## push to u1:t2
+    git push      `url u1 t2` master:master
+        ok
+        /To http:..u1:u1.localhost.git.t2.git/
+        /master -. master/
+    git ls-remote `url u2 t2`
+        ok
+        /HEAD/
+        /refs.heads.master/
+" || die "step 2"

t/smart-http.root-setup

@@ -0,0 +1,82 @@
+#!/bin/bash
+
+# ----------------------------------------------------------------------
+# please do not even LOOK at this file without reading doc/http.mkd
+# ----------------------------------------------------------------------
+
+die() { echo "$@"; exit 1; }
+
+# scare the sh*t out of people who run it blindly
+[ -f /tmp/gitolite-smart-http-test-OK ] || {
+    # scary message
+    echo '+ rm -rf /'
+    # lots of disk activity
+    find / >/dev/null 2>/dev/null
+    # and it he's still clueless, God bless!
+    echo 'root file system erased successfully.  Goodbye and God bless!'
+    exit 1
+}
+
+id | grep '=0(root)' || die "you must run this as root"
+
+# delete any existing apache conf for gitolite
+rm /etc/httpd/conf.d/gitolite.conf
+
+# build your "home within a home"
+cd ~apache
+rm -rf gitolite-home
+mkdir gitolite-home
+export GITOLITE_HTTP_HOME=$PWD/gitolite-home
+
+# get the gitolite sources
+cd gitolite-home
+git clone /tmp/gitolite.git gitolite-source
+# NOTE: I use a bare repo in /tmp for convenience; you'd use
+# 'git://github.com/sitaramc/gitolite'
+
+# make the bin directory, and add it to PATH
+cd gitolite-source
+mkdir         $GITOLITE_HTTP_HOME/bin
+./install -ln $GITOLITE_HTTP_HOME/bin
+export PATH=$PATH:$GITOLITE_HTTP_HOME/bin
+
+# come back to base, then run setup.  Notice that you have to point HOME to
+# the right place, even if it is just for this command
+cd $GITOLITE_HTTP_HOME
+HOME=$GITOLITE_HTTP_HOME gitolite setup -a admin
+
+# insert some essential lines at the beginning of the rc file
+echo '$ENV{GIT_HTTP_BACKEND} = "/usr/libexec/git-core/git-http-backend";' > 1
+echo '$ENV{PATH} .= ":$ENV{GITOLITE_HTTP_HOME}/bin";'  >> 1
+echo >> 1
+cat .gitolite.rc >> 1
+\mv 1 .gitolite.rc
+
+# fix up ownership
+chown -R apache.apache $GITOLITE_HTTP_HOME
+
+# create the apache config.  Note the trailing slashes on the 2 ScriptAlias
+# lines.  (The second one is optional for most sites).  NOTE: you also need to
+# give the AuthUserFile a better name/location than what I have below.
+cat <<EOF1 > /etc/httpd/conf.d/gitolite.conf
+SetEnv GIT_PROJECT_ROOT $GITOLITE_HTTP_HOME/repositories
+ScriptAlias /git/ $GITOLITE_HTTP_HOME/gitolite-source/src/gitolite-shell/
+ScriptAlias /gitmob/ $GITOLITE_HTTP_HOME/gitolite-source/src/gitolite-shell/
+SetEnv GITOLITE_HTTP_HOME $GITOLITE_HTTP_HOME
+SetEnv GIT_HTTP_EXPORT_ALL
+
+<Location /git>
+    AuthType Basic
+    AuthName "Private Git Access"
+    Require valid-user
+    AuthUserFile /tmp/gitolite-http-authuserfile
+</Location>
+EOF1
+
+# NOTE: this is for testing only
+htpasswd -bc /tmp/gitolite-http-authuserfile admin admin
+map "htpasswd -b /tmp/gitolite-http-authuserfile % %" u{1..6}
+
+# restart httpd to make it pick up all the new stuff
+service httpd restart
+