We previously said all mirroring features are disabled if GL_HOSTNAME is
not set.
But what if, after mirroring has been setup, and master/slaves defined
for a repo, a slave admin fat-fingers the RC file and accidentally
comments out GL_HOSTNAME? We might end up violating RULE NUMBER ONE!
- post-receive now just calls mirror-push
- mirror-push is a medium complex shell script (all that backgrounding
etc., can't be done so easily in God's first language!)
- mirror-shell is now a perl program that does a few different things
(receive mirror-pushes, command line re-sync, re-sync requests from a
slave, etc)
- auth-command changes to reject/redirect non-native pushes
Supporting git on DOS may be a fact of life, but it felt good to make
them use the "right" editor (hint: starts with v, ends with m, has three
letters), or at least something equivalent, to produce their setperms
input files.
I'd say "yes Fred I know. However, Unix doesn't work like that, and
when talking to a big, important, OS like Unix from your little
single-user workstation, you do have to go the extra mile. It's not
that different from going into a meeting with the CEO you know -- you
dress up a little for that meeting don't you?". And Fred would nod, a
little awed by the analogy. Maybe the awe was tinged with a wee bit of
anger but not much; he knows there's a hierarchy among OSs, just like
among people, and he knows where his OS stands in that hierarchy...
----
For the humour impaired, that was a joke. To start with, I don't know
anyone called Fred.
It *is* true that I tend to ignore DOS if at all possible, especially in
my *code*. But since I don't really use wildrepos at work, this issue
would never have come up for me, even in the DOS projects I manage.
In this case someone who's contributed a heck of a lot to the evolution
of gitolite asked, so here it is.
----
oh and in case you were wondering, DOS stands for "dominant operating
system" ;-)
Fix a problem with authkeys perms when REPO_UMASK is too loose.
(To duplicate it, run a fresh, non-root install, and when gl-setup pops
up an editor, change the REPO_UMASK to 0007 (from the default 0077).
You'll find that ~/.ssh/authorized_keys now has g+w set, causing sshd to
refuse key-based access.)
And before you ask, even though gl-setup does it, I won't fiddle with
the permissions of an existing file in *this* code. (gl-setup is run
manually by the admin, this one gets run on every push).
----
Side note: 9ad7ea4 was somewhat forced on me, and I didn't really agree
with parts of it. I have no idea why I gave in so easily, but it won't
happen again!
a warn/die may happen even before setup_environment() has been called,
so then log_it() tries to use uninitialised variables and it then adds
to the confusion complaining about *those*
make the arguments optional (with documented defaults) plus they need
not exist a priori, reducing one command (the silly mkdir!) that the
user has to run.
All this is preparatory to deprecating the from-client method. We've
even switched the test suite to 'non-root' method now
The KDE folks have a projects.list file that gets picked up by rsync and
sent elsewhere in some cron-ish manner. They noticed that occasionally
the file that went across was truncated, which indicates that maybe it
got picked up while it was still being written.
oh and this change also affects any files that get written using
wrap_print()
(thanks to yoonique dot net at gmail; he also tested that files with
some normal and some md5 passwords, such as would happen on an upgrade,
work fine)
git 1.7.4+ insists on these two being defined. So I reduce my support
load by forcing them if they were not set.
Much easier than explaining to people what should be obvious from the
error message.
$ENV{GL_REPO_BASE_ABS} is meant to point to the same directory as
$REPO_BASE, except it is meant to be passed to hooks, ADCs and other
child programs. And since you can't be sure where the child program
starts in, this became an absolute path.
Gradually, however, I started using it wherever I needed an absolute
path (mostly in code that jumps around various directories to do stuff).
Which is silly, because there's no reason $REPO_BASE cannot also be made
an absolute, even if the rc file has a relative path.
So that's what I did now: made $REPO_BASE absolute very early on, and
then systematically changed all uses of the longer form to the shorter
form when appropriate. And so the only thing we now use the longer one
for is to pass to child programs.
(Implementation note: The actual change is not very big, but while I was
about it I decided to make the test suite able to test with an absolute
REPO_BASE also, which is why the commit seems so large.)
----
This all started with a complaint from Damien Regad. He had an
extremely odd setup where his bashrc changed PWD to something other than
$HOME before anything else ran. This caused those two variables to
beceom inconsistent, and he had a 1-line fix he wanted me to apply.
I generally don't like making special fixes for for non-standard setups,
and anyway all he had to do was set the full path to REPO_BASE in the rc
file to get around this. Which is what I told him and he very politely
left it at that.
However, this did get me thinking, and I soon realised I was needlessly
conflating "relative versus absolute" with "able to be passed to child
programs". Fixing that solved his problem also, as a side-effect.
So I guess this is all thanks to Damien!
Technically this does not add any new information, but I'm hoping it
will help the folks just won't read what's on the screen otherwise.
The main impetus this time is git 1.7.4, which is strict about
user.email and user.name and rejects commits when those config variables
are not set. As a result, the number of times gl-easy-install hits a
fatal error and bombs out without completing its job, has increased
drastically.
The code that sets %projlist doesn't even run if GL_NO_DAEMON_NO_GITWEB
is set, so it doesn't make sense to then *use* that (empty) variable and
effectively wipe out the projects.list file.
Thanks to m0 for asking...
(suggested by cmyers and ryan_c on #gitolite)
Between wrap_print(), which now takes a list, and the new slurp(),
pretty much everything to do with 'cat' or 'echo' has been converted to
pure perl.
----
Personally, I consider these changes to be somewhat gratuitous, because
none of these had a security *or* a performance concern. But since the
amount of new perl code was not too high (just the slurp() function,
really), I figure it's not a big deal to do it.
with warns now being logged, it's nice to make sure that anything that
could even vaguely be considered someone playing with the system, *or*
is otherwise noteworthy, be emitted as a 'warn' instead of as a 'print
STDERR'. Similarly stuff that is clearly a syntactic warning or typo
should come from 'print STDERR', instead of from a 'warn'.
ryan-c on #gitolite (ryan.castellucci@gmail.com) found that if a user
types in
ssh git@server `echo -e "\033[2J"`
or eqvt, he can get raw ASCII control characters into gitolite's log
file. Then if a gitolite admin 'cat's the log file (instead of using a
pager, or uses a pager in raw mode like 'less -r'), those control
characters hit his screen and do stuff.
While clearing the screen etc is probably harmless and I would not have
bothered, we know that the old vt100 would allow the keyboard to be
remapped by the server sending control codes, and we're not really sure
which of the currently in use terminals emulate this.
And finally, I found somewhere that "PuTTY allows the server to send
control codes that let it take over the mouse". Scary...
(...of course, I hate putty/plink so I was sorely tempted to leave this
as is to punish people who use it <grin> but not really; I'd joke about
it but won't actually *do* it!)
Earlier, it wasn't as critical for gl-setup to be run with the full
path; the BINDIR deduction used to happen in almost every program. Now
it's a lot more important.
Apparently I never noticed that "/bin/bash -l gl-setup" does not set $0
to the correct, fq path. Adding a "-c" does, however...
[thanks to Jeff from the KDE team for finding this]
This shaves 3 seconds off of KDE's config compile time :-)
Yes, I know wrap_print has that extra print statement, but otherwise it
was lying around not earning its keep so I gave it a little side job :-)
you might wonder why these are different from all the other variables in
the rc file... it's just that I never thought people would want to
change these!
- allow a mob username to be defined; all unauthenticated access will
look to gitolite like this user (if you setup apache also properly)
- update doc with more details (some repeat stuff from `man
git-http-backend` but it's probably worth having everything in one
place
Fedora's config has over 11,000 repositories and the compiled config
file is over 20 MB in size. Although negligible on a server class
machine, on my laptop just parsing this file takes a good 2.5 seconds.
Even if you use GL_ALL_READ_ALL (see a couple of commits before this
one) to remove the overhead for 'read's, that's still a pretty big
overhead for writes. And GL_ALL_READ_ALL is not really a solution for
most people anyway.
With this commit, using GL_BIG_CONFIG adds another optimisation; see
doc/big-config.mkd for details (look for the word "split config" to find
the section that talks about it).
----
Implementation notes:
- the check for GL_NO_CREATE_REPOS has moved *into* the loop (which it
completely bypassed earlier) so that write_1_compiled_conf can be
called on each item
Fedora (for example) runs like this:
* each user has his own userid and login
* his/her ~/.ssh/authkeys file (containing only his/her key) has a
"command=" clause invoking just "gl-auth-command"
* trusted users have "gl-auth-command -s" meaning they can get a shell if
they want to
As a result, there is no specific $HOME where you can look for
.gitolite.rc. Hence this patch
----
Side note: in addition, Fedora may have one or more of the following
characteristics (writing them here for convenience; they're not directly
relevant to this patch):
* actual git repos are under "git" (or some such), and include the chmod g+s
(git init --shared) unix perms tricks for shared access
* but since they're coming through gl-auth, branch-level acls are in effect
* the gitolite config file is generated from some database and compiled (all
via cron)
* they keydir/ is empty; in fact they probably don't use the admin repo at
all, AFAIK
For sample code see new file contrib/adc/get-rights-and-owner.in-perl.
Despite the name, you can use similar code in a hook also -- comments in
that file will tell you how.
implementation notes:
- check_access now takes an optional last arg "dry_run", which is also
passes through to check_ref
- check_ref returns a "DENIED by ..." instead of die-ing if dry_run is
passed in
- as a side effect, cli_repo_rights is now just a stub calling
check_access (we kept it hanging around for backward compat -- too
much adc pain for too many people if we change it now)
perm categories (like READERS and WRITERS, or whatever you put in your
$GL_WILDREPOS_PERM_CATS) are *supposed* to "have no pubkeys"; don't warn
about them
thanks to Joe Schaefer at the ASF for catching it. Note that this new
pattern *may* be too restrictive -- if you're using this feature and
have a problem with the new pattern please email me.
See email to gitolite mailing list around this date (2010-11-28) for
more details.
They don't work if someone calls the script for example
su - gitolite -c gl-setup <key>
from a directory where "gitolite" user does not have permissions (e.g.
0700), then 'cd $od' fails and we stay in gitolite's $HOME.
[commit message changed by committer; author was more polite ;-)]
(we quietly do not document the 'able' adc, which is now the most
"official" adc in the sense that it has a new test, t64-write-able!)
other notes: fix bug in 'able' (not setting $loc)
- openssh 5.6 doesn't like "ssh user@host" with no command following
it, because they changed the rules for pty allocation failure.
I'm calling this a BSD compat change because BSD hit it first, but
really, the "ssh -T" will eventually be needed by Linuxes also, as
they start upgrading to openssh 5.6
- FreeBSD (and I presume the other BSDs also) *require* a "-t"
argument to mktemp (thanks to matias for finding this).
Note that on FreeBSD, -t is a prefix (the X's are taken literally,
and the real random stuff gets appended to the prefix), while on
Linux, it is a template (the X's are converted to random
characters). Thus, on BSD you will get names like
/tmp/tmp.XXXXXXXXXX.1BAEGkHm, whereas on Linux you'll get
/tmp/tmp.Aq7vbdNpGp or something.
Use case: group information is generated from an external system and
because of synchronization or authorization restrictions some groups
can be empty.
Signed-off-by: Teemu Matilainen <teemu.matilainen@reaktor.fi>
THE COMPILED CONFIG FILE FORMAT CHANGES WITH THIS VERSION. PLEASE DO
NOT MIX VERSIONS OR DOWNGRADE. Upgrading using normal gitolite upgrade
means should be fine, though.
Originally, we only allowed "R" and "RW" as categories of users supplied
to the `setperms` command. These map respectively to "READERS" and
"WRITERS" in the access rules.
Now:
- we prefer READERS instead of R and WRITERS instead of RW
- we allow the admin to define other categories as she wishes
(example: MANAGERS, TESTERS, etc). These do not have abbreviations,
however, so they must be supplied in full.
PLEASE, *PLEASE*, read the section in doc/wildcard-repositories.mkd for
more info. This is a VERY powerful feature and if you're not careful
you could mess up the ACLs nicely.
Backward compat note: you can continue to use the "R" and "RW"
categories when running the "setperms" command, and gitolite will
internally convert them to READERS and WRITERS categories.
----
implementation notes:
- new RC var called GL_WILDREPOS_PERM_CATS that is a space-sep list of
the allowed categories in a gl-perms file; defaults to "R RW" if not
specified
- wild_repo_rights no longer returns $c, $r, $wC, where $r = $user if
"R $user", $r = '@all' if "R @all", and similarly with $w and "RW".
Instead it returns $c and a new hash that effectively gives the same
info, but expanded to include any other valid categories (listed in
GL_WILDREPOS_PERM_CATS)
- consequently, the arguments that parse_acl takes also change the
same way
- (side note: R and RW are quietly converted to READERS and WRITERS;
however, new categories that you define yourself do not have
abbreviations)
- setperms validates perms to make sure only allowed categories are
used; however even if someone changed them behind the scenes,
wild_repo_rights will also check. This is necessary in case the
admin tightened up GL_WILDREPOS_PERM_CATS after someone had already
setperms-d his repos.
- as a bonus, we eliminate all the post-Dumper shenanigans, at least
for READERS and WRITERS. Those two now look, to the compile script,
just like any other usernames.
This was a very old quirk/oddity. Doing
R = @all alice
would fail, but you could still do
R = @all
R = alice
Now we fixed it so it's consistent.
----
This also fixed a curious bug that no one ever caught:
@all = u1 u2 # yes -- there was no check on redefining @all
repo foo
R = @all u3 # now would not fail because of defining @all
would have given only those 3 users R access to foo, not really @all
users! This was because the previous failure message was an artifact of
not finding an expansion for @all, not a genuine "why are you saying
@all and then specifying some user explicitly" warning!
The old method of passing in usergroup info had some problems, which are
now fixed. It is also much easier to use now -- no more "wrapper"
script, plus it should work identially whether you use sshd or httpd.
See doc/big-config.mkd for details on the new method.
----
Notes on problems with the old method:
The old method for passing in usergroup info consisted of tacking them
on as extra arguments to gl-auth-command, after the username.
However, there are some problems with this method.
Some actions in gitolite look for permissions for users other than the
invoking user. Determining permissions for gitweb and daemon is one.
An admin asking for "info" on some other user, is another.
However, the list of groups sent in via the command line
pertains only to the invoking user, so these actions don't work
correctly. They may even pick up the wrong permissions.
What it all boils down to is that we need group information for any user
dynamically, instead of being passed a (static) list just for the
invoking user.
- hardcode 0700 mode for GL_ADMINDIR tree (thanks to ma at
ibitsense.com) for catching this
- honor REPO_UMASK for GL_REPO_BASE_ABS creation
- plus a minor doc update
for people who don't get the continual reference to towels when talking
about the "gl-dont-panic" program, all I can say is that your education
is incomplete ;-)
TODO: if the verb doesn't actually contain "git-receive-pack", I am
assuming it is some sort of read. The list in services[] in
http-backend.c does not seem to look like any other verb is a "write";
need to check this with someone.
For normal git commands:
- PATH_INFO gives you the repo name
- REQUEST_URI gives you the verb
- we construct a fake SSH_ORIGINAL_COMMAND so the rest of the
processing does not have to change
For our special commands:
- PATH_INFO is actually the verb
- QUERY_STRING has the parameters
- we again fake out the SSH_ORIGINAL_COMMAND
- we print the extra HTTP headers in anticipation of the actual output
Either way, we also fake out the SSH_CONNECTION so that the IP address
can get logged ok
And of course REMOTE_USER is now the incoming userid
Finally, at the end, we exec GIT_HTTP_BACKEND instead of the normal one
adapted from code by kpfleming@digium.com. I basically cherry-picked
the top commit on "pu-work" (30068d1) on his fork at github, and made
some minor fixups to it
By default, @all does not include gitweb and daemon, but if that's what
you want, you can make it happen... see GL_ALL_INCLUDES_SPECIAL
variable in conf/example.gitolite.rc
This should hopefully be the final step in making wildrepos as close to
normal repos as possible. You can now do pretty much anything with them
that you can do with normal repos [1]
Implementation notes:
- compile puts out %groups into the compiled config file regardless of
GL_BIG_CONFIG because this feature needs it
- wild_repo_rights caches %groups because the part of the %groups hash
we care about will not change between calls in the same run
----
[1] **except** use the full-blown config file syntax within the gl-perms
file :-) I don't plan to do that; it's too complicated! [2]
[2] yeah yeah I know -- famous last words!
modifications:
- call setup_gitweb_access and setup_daemon_access from with
get_set_perms so when the user sets a perm explicitly it works
- in setup_gitweb_access, do not delete description file or
gitweb.owner if the repo is wild
- make the "fork" adc set gitweb.owner *and* call setperms using
GL_WILDREPOS_DEFPERMS
- add tests
bug fixes:
- gl-auth did not even *look* at GL_WILDREPOS_DEFPERMS when
auto-"C"reating a wild repo; fixed
- setup_gitweb_access did not delete the description file as
consistently as it deleted the owner
what will NOT work:
- removing gitweb permissions does not clear the name from
"projects.list". That's complicated, so just wait till the next
"compile" to make this happen
(thanks to Jefferai for driving this...)
----
mildly puzzling:
for some strange reason, after a "git ls-remote ...try3" in t58,
instead of not creating a "description" file, we started seeing a
73-byte file containing this message:
Unnamed repository; edit this file 'description' to name the repository.
I'm an idiot. I say I won't do it, then I go and do it anyway.
Fortunately, in this case, the code and execution remain exactly the
same for people who do not set $GL_PERFLOGT in the rc file, so it's
tolerable.
<evil grin> People who want even more than this can contact Greg Lonnon
(see the mailing list archives at
http://groups.google.com/group/gitolite for an obfuscated but easy to
guess email address) ;-)
