Commit graph

134 commits

Author SHA1 Message Date
Sitaram Chamarty
91a8379f9f change some 'warn's to 'print STDERR' and vice versa
with warns now being logged, it's nice to make sure that anything that
could even vaguely be considered someone playing with the system, *or*
is otherwise noteworthy, be emitted as a 'warn' instead of as a 'print
STDERR'.  Similarly stuff that is clearly a syntactic warning or typo
should come from 'print STDERR', instead of from a 'warn'.
2011-03-08 14:20:35 +05:30
Sitaram Chamarty
d2cef2d05e doc fixes related to conf and rc getting their own doc files 2011-01-29 17:07:57 +05:30
Sitaram Chamarty
692552d146 gitolite v2.0rc1 -- please see new developer-notes doc 2011-01-16 07:26:13 +05:30
Sitaram Chamarty
10a30c961d (major change in big-config mode) split the compiled config file
Fedora's config has over 11,000 repositories and the compiled config
file is over 20 MB in size.  Although negligible on a server class
machine, on my laptop just parsing this file takes a good 2.5 seconds.

Even if you use GL_ALL_READ_ALL (see a couple of commits before this
one) to remove the overhead for 'read's, that's still a pretty big
overhead for writes.  And GL_ALL_READ_ALL is not really a solution for
most people anyway.

With this commit, using GL_BIG_CONFIG adds another optimisation; see
doc/big-config.mkd for details (look for the word "split config" to find
the section that talks about it).

----

Implementation notes:

  - the check for GL_NO_CREATE_REPOS has moved *into* the loop (which it
    completely bypassed earlier) so that write_1_compiled_conf can be
    called on each item
2011-01-02 11:30:29 +05:30
Sitaram Chamarty
7fc1e9459f (data format change) fix misnamed 'repo_config' to 'git_configs'
this change includes a minor data format change.  This should be mostly
transparent for upgrades though.
2011-01-02 10:21:39 +05:30
Sitaram Chamarty
270e0b9157 (minor) support saying 'repo.git' in the config file
but complain loudly so they stop doing this... :-)
2010-11-10 18:07:58 +05:30
Teemu Matilainen
6eb8558e8d compile: Allow empty @group definitions
Use case: group information is generated from an external system and
because of synchronization or authorization restrictions some groups
can be empty.

Signed-off-by: Teemu Matilainen <teemu.matilainen@reaktor.fi>
2010-11-10 12:38:15 +05:30
Sitaram Chamarty
047790140a custom perm categories in setperms (WARNING: PLEASE READ FULL COMMIT MESSAGE)
THE COMPILED CONFIG FILE FORMAT CHANGES WITH THIS VERSION.  PLEASE DO
NOT MIX VERSIONS OR DOWNGRADE.  Upgrading using normal gitolite upgrade
means should be fine, though.

Originally, we only allowed "R" and "RW" as categories of users supplied
to the `setperms` command.  These map respectively to "READERS" and
"WRITERS" in the access rules.

Now:

  - we prefer READERS instead of R and WRITERS instead of RW
  - we allow the admin to define other categories as she wishes
    (example: MANAGERS, TESTERS, etc).  These do not have abbreviations,
    however, so they must be supplied in full.

PLEASE, *PLEASE*, read the section in doc/wildcard-repositories.mkd for
more info.  This is a VERY powerful feature and if you're not careful
you could mess up the ACLs nicely.

Backward compat note: you can continue to use the "R" and "RW"
categories when running the "setperms" command, and gitolite will
internally convert them to READERS and WRITERS categories.

----

implementation notes:

  - new RC var called GL_WILDREPOS_PERM_CATS that is a space-sep list of
    the allowed categories in a gl-perms file; defaults to "R RW" if not
    specified

  - wild_repo_rights no longer returns $c, $r, $wC, where $r = $user if
    "R $user", $r = '@all' if "R @all", and similarly with $w and "RW".

    Instead it returns $c and a new hash that effectively gives the same
    info, but expanded to include any other valid categories (listed in
    GL_WILDREPOS_PERM_CATS)

  - consequently, the arguments that parse_acl takes also change the
    same way

  - (side note: R and RW are quietly converted to READERS and WRITERS;
    however, new categories that you define yourself do not have
    abbreviations)

  - setperms validates perms to make sure only allowed categories are
    used; however even if someone changed them behind the scenes,
    wild_repo_rights will also check.  This is necessary in case the
    admin tightened up GL_WILDREPOS_PERM_CATS after someone had already
    setperms-d his repos.

  - as a bonus, we eliminate all the post-Dumper shenanigans, at least
    for READERS and WRITERS.  Those two now look, to the compile script,
    just like any other usernames.
2010-11-06 21:03:34 +05:30
Sitaram Chamarty
8202ad6d8a (minor) allow @all to be combined with other items
This was a very old quirk/oddity.  Doing

    R       =   @all alice

would fail, but you could still do

    R       =   @all
    R       =   alice

Now we fixed it so it's consistent.

----

This also fixed a curious bug that no one ever caught:

    @all = u1 u2        # yes -- there was no check on redefining @all
    repo foo
        R   =   @all u3 # now would not fail because of defining @all

would have given only those 3 users R access to foo, not really @all
users!  This was because the previous failure message was an artifact of
not finding an expansion for @all, not a genuine "why are you saying
@all and then specifying some user explicitly" warning!
2010-10-24 18:27:56 +05:30
Sitaram Chamarty
cd0eac8c3f (minor) a slew of little docfixes 2010-10-23 23:01:12 +05:30
Sitaram Chamarty
66b65e5e1d (doh!) make gitolite.pm easier for packagers
should have done this long ago...
2010-10-23 17:43:19 +05:30
Sitaram Chamarty
7b633049be refactored and lifted out the line parse part from inside parse_conf_file
adapted from code by kpfleming@digium.com.  I basically cherry-picked
the top commit on "pu-work" (30068d1) on his fork at github, and made
some minor fixups to it
2010-09-04 15:03:06 +05:30
Sitaram Chamarty
1d566ac46b a bit of optimisation to help another one along
(thanks to reuss for telling me about the problem, although setting
BIG_CONFIG worked just as well in his case, and this patch was not
really needed)
2010-08-26 00:10:47 +05:30
Sitaram Chamarty
6e2db12302 allow @groups in setperms command also
This should hopefully be the final step in making wildrepos as close to
normal repos as possible.  You can now do pretty much anything with them
that you can do with normal repos [1]

Implementation notes:

  - compile puts out %groups into the compiled config file regardless of
    GL_BIG_CONFIG because this feature needs it
  - wild_repo_rights caches %groups because the part of the %groups hash
    we care about will not change between calls in the same run

----

[1] **except** use the full-blown config file syntax within the gl-perms
    file :-)  I don't plan to do that; it's too complicated! [2]

[2] yeah yeah I know -- famous last words!
2010-08-21 13:04:13 +05:30
Sitaram Chamarty
da210f21bd log elapsed time
I'm an idiot.  I say I won't do it, then I go and do it anyway.

Fortunately, in this case, the code and execution remain exactly the
same for people who do not set $GL_PERFLOGT in the rc file, so it's
tolerable.

<evil grin> People who want even more than this can contact Greg Lonnon
(see the mailing list archives at
http://groups.google.com/group/gitolite for an obfuscated but easy to
guess email address) ;-)
2010-08-17 22:35:16 +05:30
Sitaram Chamarty
79f0a5fd52 (big one!) more than one wildcard may match a repo...
plus it can also be matched by a normal repo line.  In other words, with

    repo foo/bar
        RW  =   u1

    repo foo/..*
        RW  =   u2

user u2 has access to foo/bar (the non-wild does not cause the wild to
be completely ignored any longer)

implementation notes:

    get_memberships:

      - no more highlander ("there can only be one") for patterns in
        @repo_plus
      - return $wild as a space-separated list of matched patterns

    collect_repo_patts:

      - as of the last change to this section of code it appears we
        weren't using the values anyway, but I had forgotten :-)

    repo_rights: (big change: $wild no longer implies $creator present,
    or vice versa)

      - new type of "creator" (like "was_sitaram") is now possible
2010-08-11 22:38:17 +05:30
Kevin P. Fleming
a5601970da make repo patterns work in fragment-named groups 2010-08-11 22:38:17 +05:30
Sitaram Chamarty
f21e7780a1 fix up gitweb, daemon, for wild + big-config
Implementation notes:

  - %repo_config is now "our", not "my"
  - collect_repo_patts now uses repo_rights to get the name of the wild
    card repo (if any) that pertains to the physical $repo, instead of
    all that new code (duh!)
  - new "can_read(repo, user)" sub (to help daemon and gitweb use)
  - the "convenience copy on steroids" thing now copies %repo_config
    also, not just %repos.  This makes setup_repo_configs simpler
  - $creator gets substituted into %groups also; we need that now that
    we (%repos and %groups) are working closer together :)
2010-08-11 22:38:16 +05:30
Sitaram Chamarty
509c73b888 gitweb/daemon now work for wild repos also
(thanks to Kevin Fleming for the need/use case)

TODO: tests
TODO: proper documentation; meanwhile, just read this:

  - you can give gitweb and daemon read rights to wild card repos also,
    and it'll all just work -- when a new repo is 'C'reated, it'll pick
    up those rights etc

  - you can assign descriptions (and owners) to individual repos as
    before, except now you can assign them to repos that actually were
    created from wild card patterns.  So for example, you can define
    rules for

        repo foo/..*

    and then assign descriptions like

        foo/repo1 = "repo one"
        foo/repo2 = "repo two"
        foo/dil "scott" = "scott's dilbert repo"

    However, this only works for repos that already exist, and only when
    you push the admin repo.

    Thumb rule: have the user create his wild repo, *then* add and push
    the admin config file with the description.  Not the other way
    around.

implementation notes:

  - wildcard support for git config revamped, refactored...

    it's not just git config that needs wildcard support.  daemon and
    gitweb access also will be needing it soon, so we start by factoring
    out the part that finds the "pattern" given a "real" repo name.

  - GL_NO_DAEMON_NO_GITWEB now gates more than just those two things;
    see doc/big-config.mkd for details

  - we trawl through $GL_REPO_BASE_ABS *once* only, collecting repo
    names and tying them to either the same name or to a wild pattern
    that the repo name was created from

  - nice little subs to setup gitweb, daemon, and git config

  - god bless $GL_REPOPATT and the day I decided to set that env var
    whenever a user hits a wild repo in any way :-)

  - the code in gl-compile-conf is very simple now.  Much nicer than
    before
2010-08-11 22:38:16 +05:30
Sitaram Chamarty
61802045d9 (mini refactor) clean up the repo_base_abs stuff
make it a sort of "super global" (an ENV var) all through, because
*everyone* seems to need it *and* this variable is pretty much constant
for the entire install
2010-08-11 22:38:16 +05:30
Sitaram Chamarty
d1d2c3e054 git config settings in wild repos: part 1
- new GL_GITCONFIG_WILD to gate it
  - new sub to do all the hard work (refactored from a few lines in
    compile)
  - split the call from "compile" into two sets -- first for non-wild,
    then for wild

This ensures that after a "compile" (admin push) all git configs are
applied.

TODO: apply them when a new wild repo is created by a user, and then on
the "fork" (admin-defined command)
2010-08-11 22:38:15 +05:30
Sitaram Chamarty
c8879264e6 separate out the code that sets up ~/.ssh/authorized_keys
NOTE: there are no *functional* changes in this for *normal*
    gitolite users.  It's just a chunk of code moving into a new
    subroutine etc.

KDE needs to populate the authkeys file from an LDAP store.  Other large
projects may have similar means to store keys, depending on how they do
their user provisioning so a generic solution is worth exploring.

This means that in these special cases
  - the gitolite-admin repo's keydir/ directory is not needed [1]
  - but they still need to create the authkeys file somehow

Implementation:

  - write a shim program to make the authkeys-generation code callable
    from the command line/shell.
  - set $GL_NO_SETUP_AUTHKEYS=1 in the rc file to disable authkey
    generation during a "compile" (admin repo push)

Expected usage of new program gl-setup-authkeys:

  - LDAP change triggers some script
  - this script collects all keys from LDAP, puts them in some
    directory, and then calls gl-setup-authkeys, passing it the name of
    the directory

ALSO PLEASE SEE COMMENTS AT THE TOP OF THE NEW PROGRAM IN THIS COMMIT
FOR SOME IMPORTANT DISCUSSION.

----

Footnotes:

[1] It doesn't make sense to use it if the keys will be maintained by
some other entity and can be called up as needed, and it adds an
unnecessary extra step.
2010-08-09 23:21:15 +05:30
Paweł Zuzelski
1315b1ad11 Better warning message for multi-keys per pubkey file
"WARNING: a pubkey file can only have one line (key); ignoring $pubkey"
message was a bit confusing, because elsewhere the docs claim multiple
keys are suported.  Added note on how to add multiple keys for single
user and pointer to the doc file concerned.
2010-08-09 23:21:13 +05:30
Sitaram Chamarty
a12eb15252 make compile atomic by writing a different and rename-ing when done 2010-07-31 00:47:18 +05:30
Sitaram Chamarty
0d0e7e5d72 (really big config) 2 new rc variables
...to prevent auto-creation of repos from config, and to prevent
processing of ssh keys.  Also doc update
2010-07-23 20:21:52 +05:30
Sitaram Chamarty
a430cc57c7 separating "push" from "create"
This is what I *should* have done back then; thanks to Jeff Mitchell for
pointing out a problem with the old method.

The old one is *definitely* a kludge.  <shamefaced grin>
2010-06-18 21:34:43 +05:30
Sitaram Chamarty
bf1a9720af (minor) be less noisy about pubkeys present but not used in config
The main use case is for people who give most people access via @all,
which is somewhat unusual but in some situations it probably makes
sense.

See also a related commit made a month or so ago (aa8da93).

Actually these two lint checks were made to help people spot typos in
the config, which sorta becomes meaningless if you have more than a few
such cases anyway, so for most people it should not matter that I am now
merely summarising the number of such cases if there are more then 10.
2010-06-18 16:50:45 +05:30
Sitaram Chamarty
080ec22ae9 compile: kill spurious "user ... not in config" warnings
this happens when users are given rights to a repo via a groupname, and
GL_BIG_CONFIG is in effect
2010-06-02 12:29:47 +05:30
Sitaram Chamarty
025de395dc (minor) 2010-05-21 21:32:55 +05:30
Sitaram Chamarty
fd85ee2c91 *try* to make upgrades resilient to format changes (pkg maintainers please read)
the commits leading up to v1.5 caused the data format to change (we
added a rule sequence number).

This in turn caused a problem for people who may have installed using
the "system install / user setup" mode of install (which includes people
who used RPM/DEB to install it) -- they would now have to *manually* run
"gl-setup" once after the rpm/deb upgrade.

This commit *tries* to mitigate this problem by recording a data format
version number in the compiled output file.  On any access to that file,
if the version number is not found or is found to be not equal to the
current version, gl-setup is run again.

The reason I say "*tries*" is that the exact command used to do this is
a bit of a hack for now.  However, if it works for Fedora and Debian,
I'm going to leave it at that :)
2010-05-21 14:40:03 +05:30
Sitaram Chamarty
be3d00079a Revert "allow setperms to override config file permissions"
This reverts commit 9612e3a4cc, since it
is no longer needed as of the rule sequencing changes we just made.

Conflicts:

	src/gl-compile-conf
2010-05-18 16:40:15 +05:30
Sitaram Chamarty
32056e0b7f (big one!) rule sequencing changes!
There were 2 problems with rule sequencing.

Eli had a use case where everyone is equal, but some are more equal than
the others ;-)  He wanted a way to say "everyone can create repos under
their own names, but only some people should be able to rewind their
branches".

Something like this would be ideal (follow the rules in sequence for
u1/u2/u3/u4, and you will see that the "deny" rule kicks in to prevent
u1/u2 from being able to rewind, although they can certainly delete
their branches):

    @private-owners = u1 u2
    @experienced-private-owners = u3 u4

    repo CREATOR/.*
      C   = @private-owners @experienced-private-owners
      RWD = CREATOR
      RW  = WRITERS
      R   = READERS
      -   = @private-owners
      RW+D = CREATOR

In normal gitolite this doesn't work because the CREATOR rules (which
get translated to "u1" at runtime) end up over-writing the "deny" rule
when u1 or u2 are the creators.  This over-writing happens directly at
the "do compiled.pm" step.

With big-config, this does not happen (because @private-owners does not
get expanded to u1 and u2), but the problem remains: the order of
picking up elements of repo_plus and user_plus is such that, again, the
RW+D wins (it appears before the "-" rule).

We fix all that by

  - making CREATOR complete to more than just the creator's name (for
    "u1", it now becomes "u1 - wild", which is actually illegal to use
    for real so there's no possibility of a name clash!)
  - maintaining a rule sequence number that is used to sort the rules
    eventually applied (this also resulted in the refex+perm hash
    becoming a list)
2010-05-18 16:36:06 +05:30
Sitaram Chamarty
aa8da93016 tone down the "ZOMG users without pubkeys" hysteria :) 2010-05-16 13:36:54 +05:30
Sitaram Chamarty
35750c1abe (big-config) update doc and rc, allow skipping gitweb/daemon
skipping gitweb/daemon has an enormous impact on speed of an admin-push!
2010-05-16 12:51:03 +05:30
Sitaram Chamarty
d11a27924b (big-config) compile: fragments in big-config
Since it is possible to do all sorts of shenanigans with wildcards and
repo groups, we

  - allow only a fragment called "foo" to set permissions for a group
    called "@foo", in addition to a repo called "foo"
  - forbid defining any groups within a fragment conf.  All "@foo = bar
    baz" must be done in the main config file now.

If this proves too limiting for anyone I'll worry about it then.
2010-05-14 21:44:03 +05:30
Sitaram Chamarty
0139fe0e97 (big-config) compile: dont complain about "@foo" not having a pubkey 2010-05-14 20:57:22 +05:30
Sitaram Chamarty
c8f83a03dd (big-config) compile: create new repos even if GL_BIG_CONFIG is set
...by expanding the groups of course
2010-05-14 20:57:05 +05:30
Sitaram Chamarty
cf0e568c89 (big-config) the new "big-config" for large setups
If you have many thousands of repos and users, neatly organised into
groups, etc., the normal gitolite fails.  (It actually runs out of
memory very fast while doing the "compile" when you push the config, due
to the number of combinations of repo/user being stored in the hash!)

This commit series will stop doing that if you set $GL_BIG_CONFIG = 1 in
the rc file.

Some notes:

  - deny rules will still work but somewhat differently -- now they must
    be placed all together in one place to work like before.  Ask me for
    details if you need to know before I get done with the docs

  - I've tested most of the important features, but not every single
    nuance

  - the update hook may be a tad less efficient now; we can try and
    tweak it later if needed but it shouldn't really hurt anything
    significantly even now

  - docs have not been written yet
2010-05-14 20:43:13 +05:30
Sitaram Chamarty
346b396840 (minor) move version check to the right place 2010-05-14 20:43:13 +05:30
Sitaram Chamarty
9612e3a4cc allow setperms to override config file permissions 2010-05-14 11:32:35 +05:30
Sitaram Chamarty
f1a942b7f7 (minor) more helpful message when the user forgot to set $GL_WILDREPOS
thanks to konrad for catching this

also make lack of WILDREPOS more noticable on compile
2010-04-29 19:27:01 +05:30
Sitaram Chamarty
c4cbfabd4c spelling cluestick...
Ouch!  How mortifying :)  I'd always thought this was one of the Brit/US
differences, but to find out that it really *isn't* a word... hmph!

Anyway, in the interest of not breaking existing wild repos, the
ownership file is still called "gl-creater".  Everything else has been
changed.

(...thanks to Sverre)
2010-04-29 19:25:39 +05:30
Sitaram Chamarty
38337551b1 (minor) helpful message when git isn't found in PATH on server 2010-04-20 21:21:42 +05:30
Sitaram Chamarty
45ecc518f0 WARNING: WE NOW REFUSE TO RUN IF GIT ON THE SERVER IS < 1.6.2
I just got tired of supporting old gits.  Sorry.  Had to happen sooner
or later.

I know you feel upset right now but later you'll thank me.
2010-04-19 21:57:47 +05:30
Sitaram Chamarty
2a776e56ad "D" must be combined with RW or RW+ (warning: minor backward compat breakage)
Having to specify "D" separately from RW or RW+ was cumbersome, and
although I don't actually use this feature, I can see the point.

One way to think of this is:

  - RW and RW+ were the only existing branch level rights
  - it doesnt make sense to have D rights without W (hence RW) rights
  - so we simply suffix a D to these if required.

Thus you can have RW, RW+, RWD, RW+D.

I hope the (hopefully few) of you who have started to use this feature
will convert your configs when you next upgrade to "pu".

I now regret pushing the previous syntax to master too quickly -- lots
of people use master only, and on the next promotion of pu the syntax
will change.  To reduce this exposure, this change will be promoted to
master very soon.
2010-04-15 06:37:35 +05:30
Sitaram Chamarty
8d55bd722c (minor fixup) 2010-04-14 09:49:09 +05:30
Sitaram Chamarty
9b35f84f55 fix bug in 7bfb367 that causes "@all.git" to be created! 2010-04-13 10:07:59 +05:30
Sitaram Chamarty
e0fe73ac18 compile: recurse through keydir/ for pubkeys 2010-04-10 09:05:50 +05:30
Sitaram Chamarty
246165537d new server-side program "gl-tool", subcommand "shell-add"
Previous implementations of "give shell access to some gitolite users"
feature were crap.  There was no easy/elegant way to ensure that someone
who had repo admin access would not manage to get himself shell access.

Giving someone shell access requires that you should have shell access
in the first place, so the simplest way is to enable it from the server
side only.

So now that we decided to do that, we may as well prepare for other,
future, commands by starting a server-side utility program with
sub-commands (the only current one being "shell-add")
2010-04-09 21:05:17 +05:30
Sitaram Chamarty
5deffee3cf security: gitolite admin can get shell access by using screwy pubkey name
example: keydir/sitaram@$(some-dangerous-command; echo hi).pub

(still won't get the reward; that is only if a non-admin user gets
privs!)
2010-04-09 16:48:46 +05:30