also
- new "WEB_INTERFACE" variable (defaults to 'gitweb')
- setup_web_access (write a plain list of repos out)
- add_del_web_access (add/delete a single repo from projects.list)
Without this, complex mirroring scenarios will be unpredictable. For
example (abbreviating "gitolite.mirror." to "gimo.") something as simple
as this will not give "foo" his different mirror setup
repo @all
config gimo.master = "frodo"
config gimo.slaves = "sam"
repo foo
config gimo.master = "sam"
config gimo.slaves = "frodo gollum"
repo foo bar
RW = u1
Even worse things happen when you have wild cards.
Now, however, they all come in the right sequence and the most recent
one takes effect (unlike ACL rules, where the first match wins, because
there you're trying to just find a match and get out, while here you're
just mindlessly applying config lines in the right order).
The KDE folks have a projects.list file that gets picked up by rsync and
sent elsewhere in some cron-ish manner. They noticed that occasionally
the file that went across was truncated, which indicates that maybe it
got picked up while it was still being written.
oh and this change also affects any files that get written using
wrap_print()
$ENV{GL_REPO_BASE_ABS} is meant to point to the same directory as
$REPO_BASE, except it is meant to be passed to hooks, ADCs and other
child programs. And since you can't be sure where the child program
starts in, this became an absolute path.
Gradually, however, I started using it wherever I needed an absolute
path (mostly in code that jumps around various directories to do stuff).
Which is silly, because there's no reason $REPO_BASE cannot also be made
an absolute, even if the rc file has a relative path.
So that's what I did now: made $REPO_BASE absolute very early on, and
then systematically changed all uses of the longer form to the shorter
form when appropriate. And so the only thing we now use the longer one
for is to pass to child programs.
(Implementation note: The actual change is not very big, but while I was
about it I decided to make the test suite able to test with an absolute
REPO_BASE also, which is why the commit seems so large.)
----
This all started with a complaint from Damien Regad. He had an
extremely odd setup where his bashrc changed PWD to something other than
$HOME before anything else ran. This caused those two variables to
beceom inconsistent, and he had a 1-line fix he wanted me to apply.
I generally don't like making special fixes for for non-standard setups,
and anyway all he had to do was set the full path to REPO_BASE in the rc
file to get around this. Which is what I told him and he very politely
left it at that.
However, this did get me thinking, and I soon realised I was needlessly
conflating "relative versus absolute" with "able to be passed to child
programs". Fixing that solved his problem also, as a side-effect.
So I guess this is all thanks to Damien!
The code that sets %projlist doesn't even run if GL_NO_DAEMON_NO_GITWEB
is set, so it doesn't make sense to then *use* that (empty) variable and
effectively wipe out the projects.list file.
Thanks to m0 for asking...
with warns now being logged, it's nice to make sure that anything that
could even vaguely be considered someone playing with the system, *or*
is otherwise noteworthy, be emitted as a 'warn' instead of as a 'print
STDERR'. Similarly stuff that is clearly a syntactic warning or typo
should come from 'print STDERR', instead of from a 'warn'.
Fedora's config has over 11,000 repositories and the compiled config
file is over 20 MB in size. Although negligible on a server class
machine, on my laptop just parsing this file takes a good 2.5 seconds.
Even if you use GL_ALL_READ_ALL (see a couple of commits before this
one) to remove the overhead for 'read's, that's still a pretty big
overhead for writes. And GL_ALL_READ_ALL is not really a solution for
most people anyway.
With this commit, using GL_BIG_CONFIG adds another optimisation; see
doc/big-config.mkd for details (look for the word "split config" to find
the section that talks about it).
----
Implementation notes:
- the check for GL_NO_CREATE_REPOS has moved *into* the loop (which it
completely bypassed earlier) so that write_1_compiled_conf can be
called on each item
Use case: group information is generated from an external system and
because of synchronization or authorization restrictions some groups
can be empty.
Signed-off-by: Teemu Matilainen <teemu.matilainen@reaktor.fi>
THE COMPILED CONFIG FILE FORMAT CHANGES WITH THIS VERSION. PLEASE DO
NOT MIX VERSIONS OR DOWNGRADE. Upgrading using normal gitolite upgrade
means should be fine, though.
Originally, we only allowed "R" and "RW" as categories of users supplied
to the `setperms` command. These map respectively to "READERS" and
"WRITERS" in the access rules.
Now:
- we prefer READERS instead of R and WRITERS instead of RW
- we allow the admin to define other categories as she wishes
(example: MANAGERS, TESTERS, etc). These do not have abbreviations,
however, so they must be supplied in full.
PLEASE, *PLEASE*, read the section in doc/wildcard-repositories.mkd for
more info. This is a VERY powerful feature and if you're not careful
you could mess up the ACLs nicely.
Backward compat note: you can continue to use the "R" and "RW"
categories when running the "setperms" command, and gitolite will
internally convert them to READERS and WRITERS categories.
----
implementation notes:
- new RC var called GL_WILDREPOS_PERM_CATS that is a space-sep list of
the allowed categories in a gl-perms file; defaults to "R RW" if not
specified
- wild_repo_rights no longer returns $c, $r, $wC, where $r = $user if
"R $user", $r = '@all' if "R @all", and similarly with $w and "RW".
Instead it returns $c and a new hash that effectively gives the same
info, but expanded to include any other valid categories (listed in
GL_WILDREPOS_PERM_CATS)
- consequently, the arguments that parse_acl takes also change the
same way
- (side note: R and RW are quietly converted to READERS and WRITERS;
however, new categories that you define yourself do not have
abbreviations)
- setperms validates perms to make sure only allowed categories are
used; however even if someone changed them behind the scenes,
wild_repo_rights will also check. This is necessary in case the
admin tightened up GL_WILDREPOS_PERM_CATS after someone had already
setperms-d his repos.
- as a bonus, we eliminate all the post-Dumper shenanigans, at least
for READERS and WRITERS. Those two now look, to the compile script,
just like any other usernames.
This was a very old quirk/oddity. Doing
R = @all alice
would fail, but you could still do
R = @all
R = alice
Now we fixed it so it's consistent.
----
This also fixed a curious bug that no one ever caught:
@all = u1 u2 # yes -- there was no check on redefining @all
repo foo
R = @all u3 # now would not fail because of defining @all
would have given only those 3 users R access to foo, not really @all
users! This was because the previous failure message was an artifact of
not finding an expansion for @all, not a genuine "why are you saying
@all and then specifying some user explicitly" warning!
adapted from code by kpfleming@digium.com. I basically cherry-picked
the top commit on "pu-work" (30068d1) on his fork at github, and made
some minor fixups to it
This should hopefully be the final step in making wildrepos as close to
normal repos as possible. You can now do pretty much anything with them
that you can do with normal repos [1]
Implementation notes:
- compile puts out %groups into the compiled config file regardless of
GL_BIG_CONFIG because this feature needs it
- wild_repo_rights caches %groups because the part of the %groups hash
we care about will not change between calls in the same run
----
[1] **except** use the full-blown config file syntax within the gl-perms
file :-) I don't plan to do that; it's too complicated! [2]
[2] yeah yeah I know -- famous last words!
I'm an idiot. I say I won't do it, then I go and do it anyway.
Fortunately, in this case, the code and execution remain exactly the
same for people who do not set $GL_PERFLOGT in the rc file, so it's
tolerable.
<evil grin> People who want even more than this can contact Greg Lonnon
(see the mailing list archives at
http://groups.google.com/group/gitolite for an obfuscated but easy to
guess email address) ;-)
plus it can also be matched by a normal repo line. In other words, with
repo foo/bar
RW = u1
repo foo/..*
RW = u2
user u2 has access to foo/bar (the non-wild does not cause the wild to
be completely ignored any longer)
implementation notes:
get_memberships:
- no more highlander ("there can only be one") for patterns in
@repo_plus
- return $wild as a space-separated list of matched patterns
collect_repo_patts:
- as of the last change to this section of code it appears we
weren't using the values anyway, but I had forgotten :-)
repo_rights: (big change: $wild no longer implies $creator present,
or vice versa)
- new type of "creator" (like "was_sitaram") is now possible
Implementation notes:
- %repo_config is now "our", not "my"
- collect_repo_patts now uses repo_rights to get the name of the wild
card repo (if any) that pertains to the physical $repo, instead of
all that new code (duh!)
- new "can_read(repo, user)" sub (to help daemon and gitweb use)
- the "convenience copy on steroids" thing now copies %repo_config
also, not just %repos. This makes setup_repo_configs simpler
- $creator gets substituted into %groups also; we need that now that
we (%repos and %groups) are working closer together :)
(thanks to Kevin Fleming for the need/use case)
TODO: tests
TODO: proper documentation; meanwhile, just read this:
- you can give gitweb and daemon read rights to wild card repos also,
and it'll all just work -- when a new repo is 'C'reated, it'll pick
up those rights etc
- you can assign descriptions (and owners) to individual repos as
before, except now you can assign them to repos that actually were
created from wild card patterns. So for example, you can define
rules for
repo foo/..*
and then assign descriptions like
foo/repo1 = "repo one"
foo/repo2 = "repo two"
foo/dil "scott" = "scott's dilbert repo"
However, this only works for repos that already exist, and only when
you push the admin repo.
Thumb rule: have the user create his wild repo, *then* add and push
the admin config file with the description. Not the other way
around.
implementation notes:
- wildcard support for git config revamped, refactored...
it's not just git config that needs wildcard support. daemon and
gitweb access also will be needing it soon, so we start by factoring
out the part that finds the "pattern" given a "real" repo name.
- GL_NO_DAEMON_NO_GITWEB now gates more than just those two things;
see doc/big-config.mkd for details
- we trawl through $GL_REPO_BASE_ABS *once* only, collecting repo
names and tying them to either the same name or to a wild pattern
that the repo name was created from
- nice little subs to setup gitweb, daemon, and git config
- god bless $GL_REPOPATT and the day I decided to set that env var
whenever a user hits a wild repo in any way :-)
- the code in gl-compile-conf is very simple now. Much nicer than
before
make it a sort of "super global" (an ENV var) all through, because
*everyone* seems to need it *and* this variable is pretty much constant
for the entire install
- new GL_GITCONFIG_WILD to gate it
- new sub to do all the hard work (refactored from a few lines in
compile)
- split the call from "compile" into two sets -- first for non-wild,
then for wild
This ensures that after a "compile" (admin push) all git configs are
applied.
TODO: apply them when a new wild repo is created by a user, and then on
the "fork" (admin-defined command)
NOTE: there are no *functional* changes in this for *normal*
gitolite users. It's just a chunk of code moving into a new
subroutine etc.
KDE needs to populate the authkeys file from an LDAP store. Other large
projects may have similar means to store keys, depending on how they do
their user provisioning so a generic solution is worth exploring.
This means that in these special cases
- the gitolite-admin repo's keydir/ directory is not needed [1]
- but they still need to create the authkeys file somehow
Implementation:
- write a shim program to make the authkeys-generation code callable
from the command line/shell.
- set $GL_NO_SETUP_AUTHKEYS=1 in the rc file to disable authkey
generation during a "compile" (admin repo push)
Expected usage of new program gl-setup-authkeys:
- LDAP change triggers some script
- this script collects all keys from LDAP, puts them in some
directory, and then calls gl-setup-authkeys, passing it the name of
the directory
ALSO PLEASE SEE COMMENTS AT THE TOP OF THE NEW PROGRAM IN THIS COMMIT
FOR SOME IMPORTANT DISCUSSION.
----
Footnotes:
[1] It doesn't make sense to use it if the keys will be maintained by
some other entity and can be called up as needed, and it adds an
unnecessary extra step.
"WARNING: a pubkey file can only have one line (key); ignoring $pubkey"
message was a bit confusing, because elsewhere the docs claim multiple
keys are suported. Added note on how to add multiple keys for single
user and pointer to the doc file concerned.
This is what I *should* have done back then; thanks to Jeff Mitchell for
pointing out a problem with the old method.
The old one is *definitely* a kludge. <shamefaced grin>
The main use case is for people who give most people access via @all,
which is somewhat unusual but in some situations it probably makes
sense.
See also a related commit made a month or so ago (aa8da93).
Actually these two lint checks were made to help people spot typos in
the config, which sorta becomes meaningless if you have more than a few
such cases anyway, so for most people it should not matter that I am now
merely summarising the number of such cases if there are more then 10.
the commits leading up to v1.5 caused the data format to change (we
added a rule sequence number).
This in turn caused a problem for people who may have installed using
the "system install / user setup" mode of install (which includes people
who used RPM/DEB to install it) -- they would now have to *manually* run
"gl-setup" once after the rpm/deb upgrade.
This commit *tries* to mitigate this problem by recording a data format
version number in the compiled output file. On any access to that file,
if the version number is not found or is found to be not equal to the
current version, gl-setup is run again.
The reason I say "*tries*" is that the exact command used to do this is
a bit of a hack for now. However, if it works for Fedora and Debian,
I'm going to leave it at that :)
There were 2 problems with rule sequencing.
Eli had a use case where everyone is equal, but some are more equal than
the others ;-) He wanted a way to say "everyone can create repos under
their own names, but only some people should be able to rewind their
branches".
Something like this would be ideal (follow the rules in sequence for
u1/u2/u3/u4, and you will see that the "deny" rule kicks in to prevent
u1/u2 from being able to rewind, although they can certainly delete
their branches):
@private-owners = u1 u2
@experienced-private-owners = u3 u4
repo CREATOR/.*
C = @private-owners @experienced-private-owners
RWD = CREATOR
RW = WRITERS
R = READERS
- = @private-owners
RW+D = CREATOR
In normal gitolite this doesn't work because the CREATOR rules (which
get translated to "u1" at runtime) end up over-writing the "deny" rule
when u1 or u2 are the creators. This over-writing happens directly at
the "do compiled.pm" step.
With big-config, this does not happen (because @private-owners does not
get expanded to u1 and u2), but the problem remains: the order of
picking up elements of repo_plus and user_plus is such that, again, the
RW+D wins (it appears before the "-" rule).
We fix all that by
- making CREATOR complete to more than just the creator's name (for
"u1", it now becomes "u1 - wild", which is actually illegal to use
for real so there's no possibility of a name clash!)
- maintaining a rule sequence number that is used to sort the rules
eventually applied (this also resulted in the refex+perm hash
becoming a list)
