new 'subconf' feature to explicitly do delegation

(includes HOSTNAME substitution feature also...)

doc/big-config.mkd

@@ -308,16 +308,7 @@ this (note the clever date command that always gets you last months log file!)
 
 ### what are the downsides?
 
-There are some downsides.  The first one applies in all cases:
+There are some downsides.
-
-  * If you use the delegation feature, you can no longer define or extend
-    @groups in a fragment, for security reasons.  It will also not let you use
-    any group other than the @fragname itself (specifically, groups which
-    contained a subset of the allowed @fragname, which would work normally, do
-    not work now).
-
-    (If you didn't understand all that, you're probably not using delegation,
-    so feel free to ignore it!)
 
 The following apply if individual ("split") conf files are written, which in
 turn only happens if you used repo names instead of group names on the `repo`

doc/delegation.mkd

@@ -1,16 +1,14 @@
 ## delegating access control responsibilities
 
-[Thanks to jeromeag for forcing me to think through this...]
-
-----
-
 In this document:
 
   * <a href="#_lots_of_repos_lots_of_users">lots of repos, lots of users</a>
-  * <a href="#_splitting_up_the_set_of_repos_into_groups">splitting up the set of repos into groups</a>
+  * <a href="#_how_to_use_delegation">how to use delegation</a>
-  * <a href="#_delegating_ownership_of_groups_of_repos">delegating ownership of groups of repos</a>
+      * <a href="#_the_subconf_command">the subconf command</a>
-  * <a href="#_other_notes">other notes</a>
+      * <a href="#_backward_compatibility">backward compatibility</a>
-  * <a href="#_security_philosophy_note">security/philosophy note</a>
+  * <a href="#_security_notes">security notes</a>
+      * <a href="#_group_names">group names</a>
+      * <a href="#_delegating_pubkeys">delegating pubkeys</a>
 
 ----
 
@@ -36,96 +34,118 @@ repos, otherwise it doesn't scale.  It would also be nice if we could prevent
 an admin from creating access rules for *any* repo in the system -- i.e., set
 limits on what repos he can control.  This would be a nice "security" feature.
 
-Delegation offers a way to do all that.  Note that delegated admins cannot
+Delegation offers a way to do all that.  You can allow access control rules
-create or remove users, nor can they define new repos.  They can only define
+for a set of repos to be specified in a **subconf** file and allow someone (a
-access control rules for a set of repos they have been given authority for.
+**sub-admin**) to make changes within that file.  (Note: sub-admins cannot
+create or remove users).
 
-----
+<a name="_how_to_use_delegation"></a>
 
-It's easier to show how it all works with an example instead of long
+### how to use delegation
-descriptions.
 
-<a name="_splitting_up_the_set_of_repos_into_groups"></a>
+First, you group your repos however you want.  In the example below, I'm
+considering firefox and lynx (projects at the root of the gitolite server) as
+well as *any* repo inside the `browsers` subdirectory, as members of the
+`webbrowsers` group.  Similarly for the others.
 
-### splitting up the set of repos into groups
+    @webbrowsers        = firefox lynx browsers/..*
+    @webservers         = apache nginx servers/..*
+    @malwares           = conficker storm ms/..*
+        # side note: if anyone objects, we claim ms stands for "metasploit" ;-)
 
-To start with, recall that gitolite allows you to specify **groups** (of users
+Each of these groups is called a **subconf** from here on.
-or repos, same syntax).  So the basic idea is that the main config file
-(`conf/gitolite.conf` in your admin repo clone) will specify some repo groups:
 
-    # group your projects/repos however you want
+Then you designate a **sub-admin** to manage each subconf, and you ensure
-    @webbrowser_repos           = firefox lynx
+(using the standard gitolite feature of restricting pushes by names of changed
-    @webserver_repos            = apache nginx
+files) that a sub-admin can make changes only to her subconf file and nothing
-    @malware_repos              = conficker storm
+else.
 
-    # any other config as usual, including access control lines for any of the
+For example, Alice is in charge of all web browser development projects.
-    # above projects or groups
+Similarly, Bob takes care of web servers, and Mallory, as [tradition][abe]
-
+dictates, is in charge of malware ;-)
-<a name="_delegating_ownership_of_groups_of_repos"></a>
-
-### delegating ownership of groups of repos
-
-Once the repos are grouped, give each person charge of one or more groups.
-For example, Alice may be in charge of all web browser development projects,
-Bob takes care of web servers, and Mallory, as [tradition][abe] dictates, is
-in charge of malware ;-)
 
 [abe]: http://en.wikipedia.org/wiki/Alice_and_Bob#List_of_characters
 
-You do this by adding files with specific names to the `gitolite-admin` repo:
-
     # the admin repo access was probably like this to start with:
     repo gitolite-admin
-        RW+                                         = sitaram
+        RW+                                     = sitaram
     # now add these lines to the config for the admin repo
-        RW                                          = alice bob mallory
+        RW                                      = alice bob mallory
-        RW+ NAME/                                   = sitaram
+        RW+ NAME/                               = sitaram
-        RW  NAME/conf/fragments/webbrowser_repos    = alice
+        RW  NAME/conf/subs/webbrowsers          = alice
-        RW  NAME/conf/fragments/webserver_repos     = bob
+        RW  NAME/conf/subs/webservers           = bob
-        RW  NAME/conf/fragments/malware_repos       = mallory
+        RW  NAME/conf/subs/malwares             = mallory
 
-This uses gitolite's ability to restrict pushes by file/dir name being changed
+Finally, you tell gitolite to pull in these files using the "subconf" command
--- the syntax you see above ensures that, while "sitaram" does not have any
-NAME based restrictions, the other 3 users do.  See `conf/example.conf` for
-syntax and notes.
 
-As you can see, **for each repo group** you want to delegate authority over,
+    subconf "subs/*.conf"
-there's a rule for a **corresponding file** in `conf/fragments` in the
-`gitolite-admin` repo.  If you have write access to that file, you are allowed
-to define rules for repos in that repo group.
 
-In other words, we use gitolite's file/dir NAME-based permissions to "enforce"
+You can put this command anywhere in the main gitolite.conf file, but it's
-the separation between the delegated configs!
+best to put it at the end.
 
-Here's how to use this in practice:
+Now alice can clone the admin repo, add a file called `conf/subs/webbrowsers`
+with whatever access rules she wants for the repositories under her control,
+commit and push.
 
-  * Alice clones the `gitolite-admin` repo, and adds a file called
+And that's really all there is to it.
-    `conf/fragments/webbrowser_repos.conf`
 
-  * she writes in this file any access control rules for the "firefox" and
+<a name="_the_subconf_command"></a>
-    "lynx" repos.  She should not write access rules for any other project --
-    they will be ignored
 
-  * Alice then commits and pushes to the `gitolite-admin` repo
+#### the subconf command
 
-Naturally, a successful push invokes the post-update hook that the admin repo
+This command is much like the "include" command, but in addition it checks
-has, which eventually runs the compile script.  The **net effect** is as if
+that a subconf does not contain ACL rules for repos that are outside its
-you appended the contents of all the "fragment" files, in alphabetical order,
+purview.
-to the bottom of the main file.
 
-<a name="_other_notes"></a>
+In the above example, the `webbrowsers` subconf file can only have access
+control lines for firefox, lynx, and anything under "browsers/" because those
+are the elements of the `@webbrowsers` group.  (This is checked using a regex
+match, which is why "anything under browsers/" is written `browsers/..*`)
 
-### other notes
+In more precise terms:
 
-If you're in big-config mode (`GL_BIG_CONFIG` set to 1 in the rc file), a
+  * the subconf name is simply the basename of the conf file, without the
-fragment file cannot define any new groups; all groups have to be defined in
+    .conf extension, and
-the main config file or in a file included from it.
+  * the elements of an `@` group of the same name are then used to limit what
+    repos the subconf can have ACL lines for.
 
-----
+(Additional notes: it can also contain lines for an actual repo called
+`webbrowsers`, or, in big-config mode, for a group called `@webbrowsers`).
 
-<a name="_security_philosophy_note"></a>
+<a name="_backward_compatibility"></a>
 
-### security/philosophy note
+#### backward compatibility
+
+For backward compatibility, if no `subconf` commands have been seen at the end
+of processing the main config file, gitolite pretends you appended
+
+    subconf "conf/fragments/*.conf"
+
+to the end of the file.
+
+<a name="_security_notes"></a>
+
+### security notes
+
+<a name="_group_names"></a>
+
+#### group names
+
+You can use "@group"s defined in the main config file but do not attempt to
+redefine or extend them in your own subconf file.  If you must extend a group
+(say `@foo`) defined in the main config file, do this:
+
+    @myfoo  =   @foo
+    # now do whatever you want with @myfoo
+
+Group names you define in your subconf will not clash even if the exact same
+name is used in another subconf file, so you need not worry about that.
+
+<a name="_delegating_pubkeys"></a>
+
+#### delegating pubkeys
+
+Short answer: not gonna happen.
 
 The delegation feature is meant only for access control rules, not pubkeys.
 Adding/removing pubkeys is a much more significant event than changing branch
@@ -134,12 +154,11 @@ be allowed to do it.
 
 Gitolite's "userids" all live in the same namespace.  This is unlikely to
 change, so please don't ask -- it gets real complicated to do otherwise.
-Allowing delegated admins to add users means username collisions, which also
+Allowing sub-admins to add users means username collisions, which also means
-means security problems (admin-A creates a pubkey for Admin-B, thus gaining
+security problems (admin-A creates a pubkey for Admin-B, thus gaining access
-access to all of Admin-B's stuff).
+to all of Admin-B's stuff).
 
 If you feel the need to delegate even that, please just go the whole hog and
 give them separate gitolite instances!  It's pretty easy to setup the
-*software* itself system-wide, so that many users can use it without all the
+*software* itself system-wide, so that many users can use it; see the root
-"easy install" fuss.  See the "system install / user setup" section in
+install method in the install document.
-doc/1-INSTALL.mkd for details.

doc/gitolite.conf.mkd

@@ -80,6 +80,11 @@ config file exists.
 
 Files that have been already processed once are skipped, with a warning.
 
+<font color="gray">Advanced users: `subconf`, a command that is very closely
+related to `include`, is documented [here][subconf].</gray>
+
+[subconf]: http://sitaramc.github.com/gitolite/doc/delegation.html#_the_subconf_command
+
 <a name="_basic_access_control"></a>
 
 ### basic access control

src/gl-compile-conf

@@ -68,6 +68,9 @@ my %user_list = ();
 my %desc = ();
 my %owner = ();
 
+# backward compat for delegation
+my $subconf_seen = 0;
+
 # ----------------------------------------------------------------------------
 #       subroutines
 # ----------------------------------------------------------------------------
@@ -104,9 +107,11 @@ sub device_inode {
 # ----------------------------------------------------------------------------
 
 # detect recursion in include files; see processing of "include" statement later
-our %included;
+my %included;
 $included{device_inode("conf/gitolite.conf")}++;
 
+my %prefixed_groupname = ();
+
 sub check_fragment_repo_disallowed
 {
     # trying to set access for $repo (='foo')...
@@ -133,8 +138,6 @@ sub parse_conf_line
     # user or repo groups
     if ($line =~ /^(@\S+) = ?(.*)/)
     {
-        die "$ABRT defining groups is not allowed inside fragments\n"
-            if $GL_BIG_CONFIG and $fragment ne 'master';
         # store the members of each group as hash key.  Keep track of when
         # the group was *first* created by using $fragment as the *value*
         do { $groups{$1}{$_} ||= $fragment } for ( expand_list( split(' ', $2) ) );
@@ -193,6 +196,8 @@ sub parse_conf_line
             # trying to set access for $repo (='foo')...
             if (check_fragment_repo_disallowed( $fragment, $repo ))
             {
+                my $repo = $repo;
+                $repo =~ s/^\@$fragment\./locally modified \@/;
                 $ignored_p->{$fragment}{$repo} = 1;
                 next;
             }
@@ -252,17 +257,30 @@ sub parse_conf_line
             $repos{$repo}{HAS_CONFIG} = 1;
         }
     }
-    # include
+    # include and subconf.  subconf is just a special case of "include",
-    elsif ($line =~ /^include "(.+)"/)
+    # saying that the config parse should "switch" contexts
+    elsif ($line =~ /^(include|subconf) "(.+)"/)
     {
-        my $include_glob = $1;
+        my $include_glob = $2;
+        my $subconf = ( $1 eq 'subconf' );
+        die "$ABRT subconf $fragment attempting to run 'subconf'\n" if $subconf and $fragment ne 'master';
+
+        # substitute HOSTNAME word if GL_HOSTNAME defined, otherwise leave as is
+        $include_glob =~ s/\bHOSTNAME\b/$GL_HOSTNAME/ if $GL_HOSTNAME;
+
         for my $file (glob($include_glob =~ m(^/) ? $include_glob : "conf/$include_glob")) {
-            die "$ABRT included file not found: '$file'\n" unless -f $file;
+            warn("$WARN included file not found: '$file'\n"), next unless -f $file;
 
             my $file_id = device_inode($file);
             warn("$WARN $file already included\n"), next if ($included{$file_id}++);
 
-            parse_conf_file( $file, $fragment );
+            if ($subconf) {
+                die "$ABRT subconf filename should end in .conf\n" unless $file =~ /^.*\/(.*).conf$/;
+                parse_conf_file( $file, $1 );
+                $subconf_seen++;
+            } else {
+                parse_conf_file( $file, $fragment );
+            }
         }
     }
     # very simple syntax for the gitweb description of repo; one of:
@@ -318,8 +336,33 @@ sub parse_conf_file
         # skip blank lines
         next unless $line =~ /\S/;
 
+        # this is how we prevent subconf hacking; we internally prefix all
+        # group names *defined* in the subconf (also if they are later used)
+        # with the subconf name.
+
+        # rules for prefixing the subconf name: prefix it if the @group name
+        # has appeared earlier in this file on the *left side*.  Prefix all
+        # left side @group names regardless.
+
+        if ($fragment ne 'master') {
+            my $lhs = '';
+            # save 'foo' if it's an '@foo = list' line
+            $lhs = $1 if $line =~ /^@(\S+) = /;
+            # prefix all @group in the line
+            $line =~ s/(^| )(@\S+)(?= |$)/ $1 . ($prefixed_groupname{$fragment}{$2} || $2) /ge;
+            # now prefix the LHS and store it if needed
+            if ($lhs) {
+                $line =~ s/^@\S+ = /"\@$fragment.$lhs = "/e;
+                $prefixed_groupname{$fragment}{"\@$lhs"} = "\@$fragment.$lhs";
+            }
+        }
+
         parse_conf_line( $line, $fragment, \@repos, \%ignored );
     }
+    # backward compat for delegation
+    parse_conf_line( 'subconf "fragments/*.conf"', $fragment, \@repos, \%ignored )
+        if ($conffile eq $GL_CONF and $fragment eq 'master' and not $subconf_seen);
+
     for my $ig (sort keys %ignored)
     {
         warn "\n\t\t***** WARNING *****\n" .
@@ -331,33 +374,6 @@ sub parse_conf_file
 # parse the main config file
 parse_conf_file($GL_CONF, 'master');
 
-# parse any delegated fragments
-wrap_chdir($GL_ADMINDIR);
-for my $fragment_file (glob("conf/fragments/*.conf"))
-{
-    # we already check (elsewhere) that a fragment called "foo" will not try
-    # to specify access control for a repo whose name is not "foo" or is not
-    # part of a group called "foo" created by master
-
-    # meanwhile, I found a possible attack where the admin for group B creates
-    # a "convenience" group of (a subset of) his users, and then the admin for
-    # repo group A (alphabetically before B) adds himself to that same group
-    # in his own fragment.
-
-    # as a result, admin_A now has access to group B repos :(
-
-    # so now we lock the groups hash to the value it had after parsing
-    # "master", and localise any changes to it by this fragment so that they
-    # don't propagate to the next fragment.  Thus, each fragment now has only
-    # those groups that are defined in "master" and itself
-
-    local %groups = %groups;
-
-    my $fragment = $fragment_file;
-    $fragment =~ s/^conf\/fragments\/(.*).conf$/$1/;
-    parse_conf_file($fragment_file, $fragment);
-}
-
 # ----------------------------------------------------------------------------
 #       (that ends the config file compiler, though we postpone the writing
 #       for now to deal with the latest GL_BIG_CONFIG innovation!)

t/t05a-delegation

@@ -87,7 +87,7 @@ echo "
 " > conf/fragments/u3r.conf
 ugc u3 < /dev/null
 [[ $1 == 0 ]] && expect "u3r.conf attempting to set access for r2b"
-[[ $1 == 1 ]] && expect "defining groups is not allowed inside fragments"
+[[ $1 == 1 ]] && expect "u3r.conf attempting to set access for locally modified @u3r"
 [[ $1 == 1 ]] && notexpect "u3r.conf attempting to set access for r2b"
 [[ $1 == 0 ]] && notexpect "defining groups is not allowed inside fragments"
 git reset --hard origin/master &>/dev/null

t/t05b-delegation-wild

@@ -108,7 +108,7 @@ echo "
 " > conf/fragments/u3r.conf
 ugc u3 < /dev/null
 [[ $1 == 0 ]] && expect "u3r.conf attempting to set access for r2b"
-[[ $1 == 1 ]] && expect "defining groups is not allowed inside fragments"
+[[ $1 == 1 ]] && expect "u3r.conf attempting to set access for locally modified @u3r"
 [[ $1 == 1 ]] && notexpect "u3r.conf attempting to set access for r2b"
 [[ $1 == 0 ]] && notexpect "defining groups is not allowed inside fragments"
 git reset --hard origin/master &>/dev/null