Fedora's config has over 11,000 repositories and the compiled config
file is over 20 MB in size. Although negligible on a server class
machine, on my laptop just parsing this file takes a good 2.5 seconds.
Even if you use GL_ALL_READ_ALL (see a couple of commits before this
one) to remove the overhead for 'read's, that's still a pretty big
overhead for writes. And GL_ALL_READ_ALL is not really a solution for
most people anyway.
With this commit, using GL_BIG_CONFIG adds another optimisation; see
doc/big-config.mkd for details (look for the word "split config" to find
the section that talks about it).
----
Implementation notes:
- the check for GL_NO_CREATE_REPOS has moved *into* the loop (which it
completely bypassed earlier) so that write_1_compiled_conf can be
called on each item
(we quietly do not document the 'able' adc, which is now the most
"official" adc in the sense that it has a new test, t64-write-able!)
other notes: fix bug in 'able' (not setting $loc)
(thanks to a somewhat heated "discussion" with "abstrakt" on #git)
While I don't agree with everything he said, some improvements are
always possible (always, always!) in docs:
- move the "conventions used" section closer to the action
- add note about RPM/DEB using "gitolite" as the user, not "git"
- de-emphasise multiple gitolite hosting users at the top; refer
advanced users to the already present detailed section later instead
- in that section, add a bit of intro, and hand-wave the inconsistency
between its 2 sub-sections ;-)
----
Unrelated to the "discussion" today, someone else (running Arch? don't
remember) had a system where /usr/local/bin was not in $PATH for a
normal user, so I added a note about that.
THE COMPILED CONFIG FILE FORMAT CHANGES WITH THIS VERSION. PLEASE DO
NOT MIX VERSIONS OR DOWNGRADE. Upgrading using normal gitolite upgrade
means should be fine, though.
Originally, we only allowed "R" and "RW" as categories of users supplied
to the `setperms` command. These map respectively to "READERS" and
"WRITERS" in the access rules.
Now:
- we prefer READERS instead of R and WRITERS instead of RW
- we allow the admin to define other categories as she wishes
(example: MANAGERS, TESTERS, etc). These do not have abbreviations,
however, so they must be supplied in full.
PLEASE, *PLEASE*, read the section in doc/wildcard-repositories.mkd for
more info. This is a VERY powerful feature and if you're not careful
you could mess up the ACLs nicely.
Backward compat note: you can continue to use the "R" and "RW"
categories when running the "setperms" command, and gitolite will
internally convert them to READERS and WRITERS categories.
----
implementation notes:
- new RC var called GL_WILDREPOS_PERM_CATS that is a space-sep list of
the allowed categories in a gl-perms file; defaults to "R RW" if not
specified
- wild_repo_rights no longer returns $c, $r, $wC, where $r = $user if
"R $user", $r = '@all' if "R @all", and similarly with $w and "RW".
Instead it returns $c and a new hash that effectively gives the same
info, but expanded to include any other valid categories (listed in
GL_WILDREPOS_PERM_CATS)
- consequently, the arguments that parse_acl takes also change the
same way
- (side note: R and RW are quietly converted to READERS and WRITERS;
however, new categories that you define yourself do not have
abbreviations)
- setperms validates perms to make sure only allowed categories are
used; however even if someone changed them behind the scenes,
wild_repo_rights will also check. This is necessary in case the
admin tightened up GL_WILDREPOS_PERM_CATS after someone had already
setperms-d his repos.
- as a bonus, we eliminate all the post-Dumper shenanigans, at least
for READERS and WRITERS. Those two now look, to the compile script,
just like any other usernames.
Well from now on they will be called "YourName".
Even better quote from essial on #git (after literally typing in
"sitaram.pub" instead of substituting his name as the instructions [in
bold] tell him to do):
come on you know how ubuntu users are
if they see fixed width fonts inside a box they immediately copy-paste it
UBUNTU USERS: I DIDN'T SAY THAT, SOMEONE ELSE DID! For details see
http://colabti.org/irclogger/irclogger_log/git?date=2010-11-04#l2417
[Although, since you apparently are quite happy to use a system that
default installs mono I doubt these little jibes matter to you
anyway...]
[idea: distribute my own pubkey with gitolite and instantly get access
to every gitolite install that is not behind a firewall, anywhere in the
world. No one will notice or realise what I'm doing - MUAHAHAHAHA!!!]
- get_rights_and_owner normalises its arg1 by stripping .git if
supplied, then sets the variable "repo" to the result as a side
effect
- new "help" adc with some default text but main purpose is to allow
site local help text
- other adc's refer to 'help' adc when appropriate
- 'undelete' renamed to 'restore'; that's what the KDE "trashcan"
program calls that operation
- minor typo in sample script in documentation
- main adc doc points to contrib/adc/repo-deletion.README now
I had someone delete the admin repo on the server, then run gl-setup
again, and complain that included config files did not get restored.
There have been others (see below) before with similar demands, but
those at least had the excuse of being provoked by genuine mistakes.
This guy was intentionally breaking stuff server side.
Wish I could say he was stupid, but actually he was probably smarter
than I. Just that his idea of the limits of gitolite's responsibility
was vastly different from mine.
----
[1] There was this guy who, as root, went on a "chmod go-rwx" spree for
security, which bollixed up gitweb access to all his repos, so he tells
me gitolite should be able to fix all the permissions on the next admin
push at least? (That is, instead of just setting umask as it currently
does, it should go on a chmod spree just like he did).
[2] Then there was the guy who told me gitolite should re-create all the
"gl-creater" files for his wildcard repos because he was restoring from
a git push --mirror backup and that doesn't preserve those files? I
tried to tell him that a git push --mirror doesn't preserve "config" or
"description" or "info/exclude" or any of the other files that git (not
gitolite) maintains, but he didn't care -- losing those did not affect
him (or he never had them), but losing these affected access control,
and it's my fault.
The old method of passing in usergroup info had some problems, which are
now fixed. It is also much easier to use now -- no more "wrapper"
script, plus it should work identially whether you use sshd or httpd.
See doc/big-config.mkd for details on the new method.
----
Notes on problems with the old method:
The old method for passing in usergroup info consisted of tacking them
on as extra arguments to gl-auth-command, after the username.
However, there are some problems with this method.
Some actions in gitolite look for permissions for users other than the
invoking user. Determining permissions for gitweb and daemon is one.
An admin asking for "info" on some other user, is another.
However, the list of groups sent in via the command line
pertains only to the invoking user, so these actions don't work
correctly. They may even pick up the wrong permissions.
What it all boils down to is that we need group information for any user
dynamically, instead of being passed a (static) list just for the
invoking user.
- hardcode 0700 mode for GL_ADMINDIR tree (thanks to ma at
ibitsense.com) for catching this
- honor REPO_UMASK for GL_REPO_BASE_ABS creation
- plus a minor doc update
- 79f0a5f ("(big one!) more than one wildcard may match a repo...")
makes some of the dire warnings about this irrelevant
- d1d2c3e and ad64f99 ("git config settings in wild repos: part 1" and
"...part 2") makes this caveat also useless
While we were about it, we added a quick intro and tried to make some
other details a little clearer.
As usual there's more documentation than code.
Unlike usual, however, this isn't completely tested. Please read the
documentation for details of what works, what doesn't, what has been
tested, what hasn't, and so on.
- all anchors prefixed by AUTO_ now
- some bad links fixed (maybe still a few I didn't catch)
- misc wording changes/additions (support section to README,
"technical skills" section to install doc, etc).
because someone else found the doc overwhelming. However, the suggested
reading order (which so far existed only on the wiki) was probably a
good thing to have at the top of the README, and the disclaimers about
ssh may help keep my sanity a little longer ;-)
jefferai pointed out that some of the links about this were broken, and
a quick look showed that it was described in multiple places too.
Brought it all together...
conf/example.gitolite.rc
- "slave mode" flag to disable pushes and "list of slaves"
hooks/common/post-receive.mirrorpush
- code to push to the mirror, creating the repo if needed
src/mirror-shell
- shell for master pushing to a slave, because we don't actually want
to go through gitolite itself, yet we have to take care of
$REPO_BASE being wherever. And of course we have to set
GL_BYPASS_UPDATE_HOOK to 1 for the push to happen!
src/gl-mirror-sync
- manually runnable program to sync from current server to another
gl-emergency-addkey replaced by totally new gl-dont-panic, which does
more (including recovering from a botched push, not just lost keys), is
cleaner, and works for all install methods
Fedora, till now, had no hope in hell of running the info command. Why?
Because the output of the info command is semantically the same as the
output of the compile script *before* the big-config mode was created.
And we all know how _that_ went ;-)
So now you get to give "info" a partial reponame or a pattern, just like
in the case of "expand". And if you're under GL_BIG_CONFIG this pattern
is mandatory. And if you try to cheat it'll still stop after showing 5
entries to prevent (accidental?) DOSs
Anyway, see doc changes in this commit for more details.
More and more people are using one of the first 3 methods of install
(the ones that don't involve running "src/gl-easy-install" from the
client side) usualy due to RPM/DEB being available now.
Previously, the ending message on running that command was serving this
purpose, and so it never really got written down in so many words.
[thanks to antgel for catching this]
----
while we were there, we removed a now-obsolete section that talks about
how to use just one key; there are better methods now
This is what I *should* have done back then; thanks to Jeff Mitchell for
pointing out a problem with the old method.
The old one is *definitely* a kludge. <shamefaced grin>
The log message format has changed. All log messages now have a common
prefix (timestamp, user, IP). This is followed by $SSH_ORIGINAL_COMMAND
(or, in one special case, the name of the user's login shell). Any
further text appears after this (currently this only happens in the case
of a successful push -- one for each ref pushed successfully)
see doc/3 for details (look for "separating delete and rewind rights"
----
and for gerrit, this is one more thing it can do that we can too ;-)
[the original text was somewhat misleading. We mean "prevent someone
from creating a branch that they have permissions to push". That is
what is now possible, where it was not possible before.]
people will NOT read documentation, especially the bloody install
documentation. I'm about ready to throw in the towel and declare
gitolite unsupported, take-it-or-leave-it.
But I'm making one last attempt to refocus the install doc to better
suit the "I know I'm very smart and I dont have to read docs so it's
clearly your fault that I am not able to install gitolite" crowd.
As a bonus, though, I ended up making proper, hyper-linked, TOCs for
most of the docs, and moved a whole bunch of stuff around. Also finally
got some of the ssh stuff over from my git-notes repo because it really
belongs here.
the commits leading up to v1.5 caused the data format to change (we
added a rule sequence number).
This in turn caused a problem for people who may have installed using
the "system install / user setup" mode of install (which includes people
who used RPM/DEB to install it) -- they would now have to *manually* run
"gl-setup" once after the rpm/deb upgrade.
This commit *tries* to mitigate this problem by recording a data format
version number in the compiled output file. On any access to that file,
if the version number is not found or is found to be not equal to the
current version, gl-setup is run again.
The reason I say "*tries*" is that the exact command used to do this is
a bit of a hack for now. However, if it works for Fedora and Debian,
I'm going to leave it at that :)
There has been a format change to the compiled output file. As the
CHANGELOG says:
Upgrading to v1.5 from any version prior to v1.5 requires an extra
step for people who installed gitolite using the "system install /
user setup" method described in doc/0-INSTALL.mkd. For such
installations, after the administrator has upgraded gitolite
system-wide, each "gitolite host" user must run `gl-setup` once
(without any arguments).
This is *not* an issue if you installed using src/gl-easy-install.
Move the example code from doc/3 to contrib/gitweb/ and modify it
to work with both wildcard and non-wildcard setups.
Signed-off-by: Teemu Matilainen <teemu.matilainen@reaktor.fi>
(as if we didn't already have enough programs with the word "install" in
their names!)
Anyway, this does what an RPM or a DEB would do -- basically implement
the instructions in Appendix C of doc/0.
You can use this to do a system-wide install if your distro isn't as
smart, forward-looking, and uptodate as Fedora ;-)
Clone the repo somewhere, cd to it, and run, for example:
sudo src/gl-system-install /usr/local/bin /var/gitolite/conf /var/gitolite/hooks
or something like that. See doc/0 for details. Run without arguments
for help.
Ouch! How mortifying :) I'd always thought this was one of the Brit/US
differences, but to find out that it really *isn't* a word... hmph!
Anyway, in the interest of not breaking existing wild repos, the
ownership file is still called "gl-creater". Everything else has been
changed.
(...thanks to Sverre)
Having to specify "D" separately from RW or RW+ was cumbersome, and
although I don't actually use this feature, I can see the point.
One way to think of this is:
- RW and RW+ were the only existing branch level rights
- it doesnt make sense to have D rights without W (hence RW) rights
- so we simply suffix a D to these if required.
Thus you can have RW, RW+, RWD, RW+D.
I hope the (hopefully few) of you who have started to use this feature
will convert your configs when you next upgrade to "pu".
I now regret pushing the previous syntax to master too quickly -- lots
of people use master only, and on the next promotion of pu the syntax
will change. To reduce this exposure, this change will be promoted to
master very soon.
Previous implementations of "give shell access to some gitolite users"
feature were crap. There was no easy/elegant way to ensure that someone
who had repo admin access would not manage to get himself shell access.
Giving someone shell access requires that you should have shell access
in the first place, so the simplest way is to enable it from the server
side only.
So now that we decided to do that, we may as well prepare for other,
future, commands by starting a server-side utility program with
sub-commands (the only current one being "shell-add")
normally, RW+ means permission to rewind or delete.
Now, if you use "D" permission anywhere in a repo config, that means
"delete" and RW+ then means only "rewind", no delete.
- no need to put it at the end of the config file now, yeaaay!
- @all for @all is meaningless and not supported. People asking will
be told to get a life or use git-daemon.
- NAME/ limits for @all repos is ignored for efficiency reasons.
There are some disadvantages to the old-style personal branch scheme.
It only allows one specific pattern (of refname) to be used, forces that
pattern to be applicable to *all* repos in the entire config, and
requires editing the rc file (on the server) to be edited to achieve
this.
In other words, it's a very blunt instrument...
The new style depends on using lines like this within a specific repo
config:
RW+ personal/USER/ = @userlist
The important thing is that the "branch" name should contain `/USER/`
(including the slashes). Access is still determined by the right hand
side of course.
This gives you the following advantages:
- allow it only for repos that need it
- allow different patterns to be used for different repos
- allow *multiple* patterns; just add more than one such line
- allow the pattern to have suffixes (eg: foo/USER/bar)
the changes to cp/scp are because without "-p" they dont carry perms
across to existing files. So if you forgot to chmod +x your custom
hook and ran easy install, then after that you have to go to the server
side to fix the perms...
when repos are copied over from elsewhere, one had to run easy install
once again to make the new (OS-copied) repo contain the proper update
hook.
We eliminate this step now, using a new, empty, "hook" as a sentinel and
having "compile" check/fix all repos' hooks.
Since you have to add the repos to conf anyway, this makes it as
seamless as possible. The correct sequence now is
- (server) copy the repo at the OS level
- (admin clone) add it to conf/gitolite.conf, commit, push
(about this commit)
The install doc now describes both the ways of installing gitolite.
It also has a handy appendix for package maintainers describing what
they need to do.
(about the "dps" -- distro packaging support -- commit series)
This commit is the last in the chain meant to make gitolite more
friendly for package maintainers.
Frankly, I never really thought gitolite would get big enough or
important enough for someone to package it, and I always did just
the bare minimum I needed to get it working, first for myself, then
anyone who hopped onto #git and asked. As a result, it had some
quirks in terms of what is expected where and so on...
Luckily, it didn't take a lot of changes to fix it, and this series
of commits should help make it very easy to package gitolite for
system-wide use.
The wildrepos branch has been merged into master, and deleted. It will no
longer exist as a separate branch. Instead, a new variable
called $GL_WILDREPOS has been added which acts as a switch; when
off (which is the default), many wildrepos features are disabled.
(the "C" permissions, and the getperms (etc.) commands mainly).
Important: if you are using wildrepos, please set "$GL_WILDREPOS = 1;" in
the RC file when you upgrade to this version (or just before you do the
upgrade).
lots of conflicts, esp in gl-auth-command, due to refactoring the
"special commands" stuff on master
Conflicts:
doc/3-faq-tips-etc.mkd
src/gitolite.pm
src/gl-auth-command
src/gl-compile-conf
great idea by Robin Smidsrød: since users are already capable of
authenticating themselves to gitolite via ssh keys, use that to let them
set or change their own HTTP passwords (ie, run the "htpasswd" command
with the correct parameters on behalf of the "git" user on the server)
code, rc para, and documentation. In fact everything except... ahem...
testing ;-)
and while we're about it, we also reorganised the way these helper
commands (including the venerable "info" are called)
This is actually a pretty big deal, and I am seriously starting wonder
if calling this "gito*lite*" is justified anymore.
Anyway, in for a penny, in for a pound...
This patch implements a generic way to allow access control for external
commands, as long as they are invoked via ssh and present a server-side
command that contains enough information to make an access control
decision.
The first (and only, so far) such command implemented is rsync.
Please read the changes in this commit (at least the ones in conf/ and
doc/) carefully.
It's not clear whether $projectroot has or does not have a trailing
slash. Current code assumes it does, but we need to cater for it not
having one also. Otherwise the final reponame ends up with a leading
slash, once $projectroot has been stripped from the beginning of the
full repo path.
I know hardly anyone is using delegation, but if you find yourself
locked out from pushing because of this one little thing, do this:
* on your gitolite-admin clone, add the required lines per this patch,
and commit
* on the server, edit ~/.gitolite/conf/gitolite.conf-compiled.pm, and
delete the following line
'NAME_LIMITS' => 1
from the entry for "gitolite-admin" (if you don't know what that
means delete *all* such lines) and save the file
* back on your admin repo clone, do a push
I know hardly anyone is using delegation, but if you find yourself
locked out from pushing because of this one little thing, do this:
* on your gitolite-admin clone, add the required lines per this patch,
and commit
* on the server, edit ~/.gitolite/conf/gitolite.conf-compiled.pm, and
delete the following line
'NAME_LIMITS' => 1
from the entry for "gitolite-admin" (if you don't know what that
means delete *all* such lines) and save the file
* back on your admin repo clone, do a push
Stop conflating the privilege to push changes to the admin repo with the
privilege to get a shell on the server.
Please read doc/6 carefully before upgrading to this version. Also
please ensure that the gitolite key is *not* your only means to get a
command line on the server
This is a backward incompatible change. If you are using delegation and
you upgrade to this version, please do the following:
* change your gitolite.conf file to use the new syntax (see
doc/5-delegation.mkd in this commit)
* for each branch "foo" in the gitolite-admin repo, do this:
# (on "master" branch)
git checkout foo -- conf/fragments/foo.conf
* git add all those new fragments and commit to master
* delete all the branches on your clone and the server
# again, for each branch foo
git branch -D foo
git push origin :foo
Gitolite allows you to restrict changes by file/dir name. The syntax
for this used "PATH/" as a prefix to denote such file/dir patterns.
This has now been changed to "NAME/" because PATH is potentially
confusing.
While this is technically a backward-incompatible change, the feature
itself was hitherto undocumented, and only a few people were using it,
so I guess it's not that bad...
Also added documentation now.
