Commit graph

122 commits

Author SHA1 Message Date
Sitaram Chamarty 9ad7ea4e19 (minor) some changes to system(), ``, etc.
(suggested by cmyers and ryan_c on #gitolite)

Between wrap_print(), which now takes a list, and the new slurp(),
pretty much everything to do with 'cat' or 'echo' has been converted to
pure perl.

----

Personally, I consider these changes to be somewhat gratuitous, because
none of these had a security *or* a performance concern.  But since the
amount of new perl code was not too high (just the slurp() function,
really), I figure it's not a big deal to do it.
2011-03-09 20:11:30 +05:30
Sitaram Chamarty 91a8379f9f change some 'warn's to 'print STDERR' and vice versa
with warns now being logged, it's nice to make sure that anything that
could even vaguely be considered someone playing with the system, *or*
is otherwise noteworthy, be emitted as a 'warn' instead of as a 'print
STDERR'.  Similarly stuff that is clearly a syntactic warning or typo
should come from 'print STDERR', instead of from a 'warn'.
2011-03-08 14:20:35 +05:30
Carl Myers 59a1855e38 Added handler to ensure warn and die are logged
(with some extra changes by Sitaram)
2011-03-08 14:20:35 +05:30
Sitaram Chamarty ca913af6cd for cool 'cat's who use 'putty' :-)
ryan-c on #gitolite (ryan.castellucci@gmail.com) found that if a user
types in
        ssh git@server `echo -e "\033[2J"`
or eqvt, he can get raw ASCII control characters into gitolite's log
file.  Then if a gitolite admin 'cat's the log file (instead of using a
pager, or uses a pager in raw mode like 'less -r'), those control
characters hit his screen and do stuff.

While clearing the screen etc is probably harmless and I would not have
bothered, we know that the old vt100 would allow the keyboard to be
remapped by the server sending control codes, and we're not really sure
which of the currently in use terminals emulate this.

And finally, I found somewhere that "PuTTY allows the server to send
control codes that let it take over the mouse".  Scary...

(...of course, I hate putty/plink so I was sorely tempted to leave this
as is to punish people who use it <grin> but not really; I'd joke about
it but won't actually *do* it!)
2011-03-05 05:56:58 +05:30
Sitaram Chamarty 46528f0cc7 squelch useless "undefined" warnings
Normal users will never need this change but the big boys like to play
on the server side directly... and sometimes the SSH_CONNECTION var is
not set
2011-02-25 20:39:59 +05:30
Sitaram Chamarty 9a49487d0a minor bug in data format fixup code
Earlier, it wasn't as critical for gl-setup to be run with the full
path; the BINDIR deduction used to happen in almost every program.  Now
it's a lot more important.

Apparently I never noticed that "/bin/bash -l gl-setup" does not set $0
to the correct, fq path.  Adding a "-c" does, however...

[thanks to Jeff from the KDE team for finding this]
2011-02-25 06:20:41 +05:30
Sitaram Chamarty bc5e995078 reach out and touch someone
This shaves 3 seconds off of KDE's config compile time :-)

Yes, I know wrap_print has that extra print statement, but otherwise it
was lying around not earning its keep so I gave it a little side job :-)
2011-02-22 13:28:02 +05:30
Sitaram Chamarty a33f0f8504 security fix for optional ADC (admin-defined command) feature
Thanks to Dylan Simon for catching it...
2011-02-13 08:15:01 +05:30
Sitaram Chamarty 1c1ae6061d "git config foo.bar = 0" was not working; fixed
(because perl treats 0 as false and I'd not accounted for that)

thanks to idl0r for catching this
2011-02-06 07:14:25 +05:30
Sitaram Chamarty 1fce051ea1 add -prune to find commands
Apparently it makes a huge difference with some kinds of network drives
(guess which company's software ;-)

http://groups.google.com/group/gitolite/browse_thread/thread/66b888f11dc5a365
2011-01-28 04:55:04 +05:30
Sitaram Chamarty 692552d146 gitolite v2.0rc1 -- please see new developer-notes doc 2011-01-16 07:26:13 +05:30
Sitaram Chamarty d8789a3af0 get rid of wasted parse in wild_repo_rights 2011-01-12 00:37:09 +05:30
Sitaram Chamarty 10a30c961d (major change in big-config mode) split the compiled config file
Fedora's config has over 11,000 repositories and the compiled config
file is over 20 MB in size.  Although negligible on a server class
machine, on my laptop just parsing this file takes a good 2.5 seconds.

Even if you use GL_ALL_READ_ALL (see a couple of commits before this
one) to remove the overhead for 'read's, that's still a pretty big
overhead for writes.  And GL_ALL_READ_ALL is not really a solution for
most people anyway.

With this commit, using GL_BIG_CONFIG adds another optimisation; see
doc/big-config.mkd for details (look for the word "split config" to find
the section that talks about it).

----

Implementation notes:

  - the check for GL_NO_CREATE_REPOS has moved *into* the loop (which it
    completely bypassed earlier) so that write_1_compiled_conf can be
    called on each item
2011-01-02 11:30:29 +05:30
Sitaram Chamarty 7fc1e9459f (data format change) fix misnamed 'repo_config' to 'git_configs'
this change includes a minor data format change.  This should be mostly
transparent for upgrades though.
2011-01-02 10:21:39 +05:30
Sitaram Chamarty e98bde322e allow gitolite to be used when the users already have real IDs
Fedora (for example) runs like this:

  * each user has his own userid and login
  * his/her ~/.ssh/authkeys file (containing only his/her key) has a
    "command=" clause invoking just "gl-auth-command"
  * trusted users have "gl-auth-command -s" meaning they can get a shell if
    they want to

As a result, there is no specific $HOME where you can look for
.gitolite.rc.  Hence this patch

----

Side note: in addition, Fedora may have one or more of the following
characteristics (writing them here for convenience; they're not directly
relevant to this patch):

  * actual git repos are under "git" (or some such), and include the chmod g+s
    (git init --shared) unix perms tricks for shared access

  * but since they're coming through gl-auth, branch-level acls are in effect

  * the gitolite config file is generated from some database and compiled (all
    via cron)

  * they keydir/ is empty; in fact they probably don't use the admin repo at
    all, AFAIK
2010-12-28 13:58:53 +05:30
Sitaram Chamarty 5b9bf700cc allow access checks from ADC or hook
For sample code see new file contrib/adc/get-rights-and-owner.in-perl.
Despite the name, you can use similar code in a hook also -- comments in
that file will tell you how.

implementation notes:

  - check_access now takes an optional last arg "dry_run", which is also
    passes through to check_ref
  - check_ref returns a "DENIED by ..." instead of die-ing if dry_run is
    passed in

  - as a side effect, cli_repo_rights is now just a stub calling
    check_access (we kept it hanging around for backward compat -- too
    much adc pain for too many people if we change it now)
2010-12-25 12:40:41 +05:30
Sitaram Chamarty 45a696c419 fine tune "no pubkeys" warning
perm categories (like READERS and WRITERS, or whatever you put in your
$GL_WILDREPOS_PERM_CATS) are *supposed* to "have no pubkeys"; don't warn
about them
2010-12-25 05:58:26 +05:30
Sitaram Chamarty 2bbcc8239c tighten up pattern to check rsync arguments
thanks to Joe Schaefer at the ASF for catching it.  Note that this new
pattern *may* be too restrictive -- if you're using this feature and
have a problem with the new pattern please email me.

See email to gitolite mailing list around this date (2010-11-28) for
more details.
2010-11-28 20:12:39 +05:30
Sitaram Chamarty c1b27eae22 stick to regex metas that come with perl 5.8
we have a stated aim to work with any perl that git is happy with, which
means it should work with perl 5.8.  At least :)
2010-11-18 19:23:13 +05:30
Sitaram Chamarty 7313d48247 gitolite-down: disable write-access to take backups
(we quietly do not document the 'able' adc, which is now the most
"official" adc in the sense that it has a new test, t64-write-able!)

other notes: fix bug in 'able' (not setting $loc)
2010-11-16 22:57:23 +05:30
Sitaram Chamarty 047790140a custom perm categories in setperms (WARNING: PLEASE READ FULL COMMIT MESSAGE)
THE COMPILED CONFIG FILE FORMAT CHANGES WITH THIS VERSION.  PLEASE DO
NOT MIX VERSIONS OR DOWNGRADE.  Upgrading using normal gitolite upgrade
means should be fine, though.

Originally, we only allowed "R" and "RW" as categories of users supplied
to the `setperms` command.  These map respectively to "READERS" and
"WRITERS" in the access rules.

Now:

  - we prefer READERS instead of R and WRITERS instead of RW
  - we allow the admin to define other categories as she wishes
    (example: MANAGERS, TESTERS, etc).  These do not have abbreviations,
    however, so they must be supplied in full.

PLEASE, *PLEASE*, read the section in doc/wildcard-repositories.mkd for
more info.  This is a VERY powerful feature and if you're not careful
you could mess up the ACLs nicely.

Backward compat note: you can continue to use the "R" and "RW"
categories when running the "setperms" command, and gitolite will
internally convert them to READERS and WRITERS categories.

----

implementation notes:

  - new RC var called GL_WILDREPOS_PERM_CATS that is a space-sep list of
    the allowed categories in a gl-perms file; defaults to "R RW" if not
    specified

  - wild_repo_rights no longer returns $c, $r, $wC, where $r = $user if
    "R $user", $r = '@all' if "R @all", and similarly with $w and "RW".

    Instead it returns $c and a new hash that effectively gives the same
    info, but expanded to include any other valid categories (listed in
    GL_WILDREPOS_PERM_CATS)

  - consequently, the arguments that parse_acl takes also change the
    same way

  - (side note: R and RW are quietly converted to READERS and WRITERS;
    however, new categories that you define yourself do not have
    abbreviations)

  - setperms validates perms to make sure only allowed categories are
    used; however even if someone changed them behind the scenes,
    wild_repo_rights will also check.  This is necessary in case the
    admin tightened up GL_WILDREPOS_PERM_CATS after someone had already
    setperms-d his repos.

  - as a bonus, we eliminate all the post-Dumper shenanigans, at least
    for READERS and WRITERS.  Those two now look, to the compile script,
    just like any other usernames.
2010-11-06 21:03:34 +05:30
Sitaram Chamarty 6386d8ca2f gl-reflog adc gains 'usage', loses some crappy warnings
(actually these warnings can come in many other situations; I just
noticed them when testing this adc)
2010-10-28 19:34:46 +05:30
Sitaram Chamarty 84fe767b64 pubkeys and the pareto principle! 2010-10-26 20:30:10 +05:30
Sitaram Chamarty cb0a9bdb0c ADC "rmrepo" replaced by "rm" and "trash", with helper ADCs
helpers for 'rm': lock, unlock
helpers for 'trash': list-trash, undelete

common functions updated with local settings for ADCs as well
2010-10-22 17:43:09 +05:30
Sitaram Chamarty a0aecbb012 (minor) provide more info on symlink failure 2010-10-19 19:27:27 +05:30
Sitaram Chamarty db0485fa7e new method for passing usergroup info (warning: minor backward compat breakage)
The old method of passing in usergroup info had some problems, which are
now fixed.  It is also much easier to use now -- no more "wrapper"
script, plus it should work identially whether you use sshd or httpd.

See doc/big-config.mkd for details on the new method.

----

Notes on problems with the old method:

The old method for passing in usergroup info consisted of tacking them
on as extra arguments to gl-auth-command, after the username.

However, there are some problems with this method.

Some actions in gitolite look for permissions for users other than the
invoking user.  Determining permissions for gitweb and daemon is one.
An admin asking for "info" on some other user, is another.

However, the list of groups sent in via the command line
pertains only to the invoking user, so these actions don't work
correctly.  They may even pick up the wrong permissions.

What it all boils down to is that we need group information for any user
dynamically, instead of being passed a (static) list just for the
invoking user.
2010-10-07 20:20:39 +05:30
Sitaram Chamarty 8a980a60bb tighten up argument handling in ADCs even more
More and more people are using ADCs, which I originally wrote just for
adventure-loving people ;-)
2010-10-06 21:30:17 +05:30
Sitaram Chamarty b851011ce5 new RC file variable $GL_SITE_INFO to show site-specific info 2010-10-04 14:33:30 +05:30
Sitaram Chamarty 67a72a3f5b added support for a post-repo-create hook (gl-post-init)
...some people want to run a special function after a repo is created
2010-09-24 16:58:46 +05:30
Sitaram Chamarty 2349afdc78 (http) fixed up the first level error reporting even better
see http://permalink.gmane.org/gmane.comp.version-control.git/155787
2010-09-08 19:57:11 +05:30
Sitaram Chamarty 595410aa55 (http) first level error reporting now works
I didn't know that remote-curl.c requires 200 OK even if you want to
report an error.

With Ilari's patch at [1] you'll get a more readable message but it is
still good enough now.

[1]: http://permalink.gmane.org/gmane.comp.version-control.git/155464
2010-09-06 12:11:19 +05:30
Sitaram Chamarty f4e011226a (http) issues of $HOME, startup (birth!), and death
- deal with issues of HOME not being available...
  - "where_is_rc" finally has a purpose; see comment block before
    function
2010-09-05 20:47:06 +05:30
Sitaram Chamarty 52e0ed3488 (http) auth: handle REQUEST_URI and friends
TODO: if the verb doesn't actually contain "git-receive-pack", I am
assuming it is some sort of read.  The list in services[] in
http-backend.c does not seem to look like any other verb is a "write";
need to check this with someone.

For normal git commands:
  - PATH_INFO gives you the repo name
  - REQUEST_URI gives you the verb
  - we construct a fake SSH_ORIGINAL_COMMAND so the rest of the
    processing does not have to change

For our special commands:
  - PATH_INFO is actually the verb
  - QUERY_STRING has the parameters
  - we again fake out the SSH_ORIGINAL_COMMAND
  - we print the extra HTTP headers in anticipation of the actual output

Either way, we also fake out the SSH_CONNECTION so that the IP address
can get logged ok

And of course REMOTE_USER is now the incoming userid

Finally, at the end, we exec GIT_HTTP_BACKEND instead of the normal one
2010-09-05 20:47:01 +05:30
Sitaram Chamarty 6b9e75dfcb allow @all to also include gitweb and daemon
By default, @all does not include gitweb and daemon, but if that's what
you want, you can make it happen...  see GL_ALL_INCLUDES_SPECIAL
variable in conf/example.gitolite.rc
2010-08-24 21:38:46 +05:30
Sitaram Chamarty 2b066fc9f0 info/expand print better(?) permission codes
see doc/report-output.mkd for details
2010-08-24 17:51:33 +05:30
Sitaram Chamarty 6e2db12302 allow @groups in setperms command also
This should hopefully be the final step in making wildrepos as close to
normal repos as possible.  You can now do pretty much anything with them
that you can do with normal repos [1]

Implementation notes:

  - compile puts out %groups into the compiled config file regardless of
    GL_BIG_CONFIG because this feature needs it
  - wild_repo_rights caches %groups because the part of the %groups hash
    we care about will not change between calls in the same run

----

[1] **except** use the full-blown config file syntax within the gl-perms
    file :-)  I don't plan to do that; it's too complicated! [2]

[2] yeah yeah I know -- famous last words!
2010-08-21 13:04:13 +05:30
Sitaram Chamarty 3a8f32ac9b minor fixups to the "gitweb/daemon from setperms" code...
- fork was not printing a newline after the permissions
  - "add_del_line" to properly handle that damn projects.list file!
2010-08-20 23:09:11 +05:30
Sitaram Chamarty 648676faec gitweb and daemon should now work even from within setperms
modifications:

  - call setup_gitweb_access and setup_daemon_access from with
    get_set_perms so when the user sets a perm explicitly it works
  - in setup_gitweb_access, do not delete description file or
    gitweb.owner if the repo is wild
  - make the "fork" adc set gitweb.owner *and* call setperms using
    GL_WILDREPOS_DEFPERMS
  - add tests

bug fixes:

  - gl-auth did not even *look* at GL_WILDREPOS_DEFPERMS when
    auto-"C"reating a wild repo; fixed
  - setup_gitweb_access did not delete the description file as
    consistently as it deleted the owner

what will NOT work:

  - removing gitweb permissions does not clear the name from
    "projects.list".  That's complicated, so just wait till the next
    "compile" to make this happen

(thanks to Jefferai for driving this...)

----

mildly puzzling:

    for some strange reason, after a "git ls-remote ...try3" in t58,
    instead of not creating a "description" file, we started seeing a
    73-byte file containing this message:

    Unnamed repository; edit this file 'description' to name the repository.
2010-08-20 21:02:18 +05:30
Sitaram Chamarty 1b5294f26d make info and expand run faster
using a wee bit of local optimisation in an inner function
2010-08-20 19:50:20 +05:30
Sitaram Chamarty da210f21bd log elapsed time
I'm an idiot.  I say I won't do it, then I go and do it anyway.

Fortunately, in this case, the code and execution remain exactly the
same for people who do not set $GL_PERFLOGT in the rc file, so it's
tolerable.

<evil grin> People who want even more than this can contact Greg Lonnon
(see the mailing list archives at
http://groups.google.com/group/gitolite for an obfuscated but easy to
guess email address) ;-)
2010-08-17 22:35:16 +05:30
Sitaram Chamarty 79f0a5fd52 (big one!) more than one wildcard may match a repo...
plus it can also be matched by a normal repo line.  In other words, with

    repo foo/bar
        RW  =   u1

    repo foo/..*
        RW  =   u2

user u2 has access to foo/bar (the non-wild does not cause the wild to
be completely ignored any longer)

implementation notes:

    get_memberships:

      - no more highlander ("there can only be one") for patterns in
        @repo_plus
      - return $wild as a space-separated list of matched patterns

    collect_repo_patts:

      - as of the last change to this section of code it appears we
        weren't using the values anyway, but I had forgotten :-)

    repo_rights: (big change: $wild no longer implies $creator present,
    or vice versa)

      - new type of "creator" (like "was_sitaram") is now possible
2010-08-11 22:38:17 +05:30
Sitaram Chamarty f21e7780a1 fix up gitweb, daemon, for wild + big-config
Implementation notes:

  - %repo_config is now "our", not "my"
  - collect_repo_patts now uses repo_rights to get the name of the wild
    card repo (if any) that pertains to the physical $repo, instead of
    all that new code (duh!)
  - new "can_read(repo, user)" sub (to help daemon and gitweb use)
  - the "convenience copy on steroids" thing now copies %repo_config
    also, not just %repos.  This makes setup_repo_configs simpler
  - $creator gets substituted into %groups also; we need that now that
    we (%repos and %groups) are working closer together :)
2010-08-11 22:38:16 +05:30
Sitaram Chamarty 509c73b888 gitweb/daemon now work for wild repos also
(thanks to Kevin Fleming for the need/use case)

TODO: tests
TODO: proper documentation; meanwhile, just read this:

  - you can give gitweb and daemon read rights to wild card repos also,
    and it'll all just work -- when a new repo is 'C'reated, it'll pick
    up those rights etc

  - you can assign descriptions (and owners) to individual repos as
    before, except now you can assign them to repos that actually were
    created from wild card patterns.  So for example, you can define
    rules for

        repo foo/..*

    and then assign descriptions like

        foo/repo1 = "repo one"
        foo/repo2 = "repo two"
        foo/dil "scott" = "scott's dilbert repo"

    However, this only works for repos that already exist, and only when
    you push the admin repo.

    Thumb rule: have the user create his wild repo, *then* add and push
    the admin config file with the description.  Not the other way
    around.

implementation notes:

  - wildcard support for git config revamped, refactored...

    it's not just git config that needs wildcard support.  daemon and
    gitweb access also will be needing it soon, so we start by factoring
    out the part that finds the "pattern" given a "real" repo name.

  - GL_NO_DAEMON_NO_GITWEB now gates more than just those two things;
    see doc/big-config.mkd for details

  - we trawl through $GL_REPO_BASE_ABS *once* only, collecting repo
    names and tying them to either the same name or to a wild pattern
    that the repo name was created from

  - nice little subs to setup gitweb, daemon, and git config

  - god bless $GL_REPOPATT and the day I decided to set that env var
    whenever a user hits a wild repo in any way :-)

  - the code in gl-compile-conf is very simple now.  Much nicer than
    before
2010-08-11 22:38:16 +05:30
Sitaram Chamarty 61802045d9 (mini refactor) clean up the repo_base_abs stuff
make it a sort of "super global" (an ENV var) all through, because
*everyone* seems to need it *and* this variable is pretty much constant
for the entire install
2010-08-11 22:38:16 +05:30
Kevin P. Fleming 33d052dc7d it is not an error for a wildcard repository to not match any patterns... that just means it does not have any config keys defined 2010-08-11 22:38:15 +05:30
Sitaram Chamarty d1d2c3e054 git config settings in wild repos: part 1
- new GL_GITCONFIG_WILD to gate it
  - new sub to do all the hard work (refactored from a few lines in
    compile)
  - split the call from "compile" into two sets -- first for non-wild,
    then for wild

This ensures that after a "compile" (admin push) all git configs are
applied.

TODO: apply them when a new wild repo is created by a user, and then on
the "fork" (admin-defined command)
2010-08-11 22:38:15 +05:30
Sitaram Chamarty c8879264e6 separate out the code that sets up ~/.ssh/authorized_keys
NOTE: there are no *functional* changes in this for *normal*
    gitolite users.  It's just a chunk of code moving into a new
    subroutine etc.

KDE needs to populate the authkeys file from an LDAP store.  Other large
projects may have similar means to store keys, depending on how they do
their user provisioning so a generic solution is worth exploring.

This means that in these special cases
  - the gitolite-admin repo's keydir/ directory is not needed [1]
  - but they still need to create the authkeys file somehow

Implementation:

  - write a shim program to make the authkeys-generation code callable
    from the command line/shell.
  - set $GL_NO_SETUP_AUTHKEYS=1 in the rc file to disable authkey
    generation during a "compile" (admin repo push)

Expected usage of new program gl-setup-authkeys:

  - LDAP change triggers some script
  - this script collects all keys from LDAP, puts them in some
    directory, and then calls gl-setup-authkeys, passing it the name of
    the directory

ALSO PLEASE SEE COMMENTS AT THE TOP OF THE NEW PROGRAM IN THIS COMMIT
FOR SOME IMPORTANT DISCUSSION.

----

Footnotes:

[1] It doesn't make sense to use it if the keys will be maintained by
some other entity and can be called up as needed, and it adds an
unnecessary extra step.
2010-08-09 23:21:15 +05:30
Sitaram Chamarty fbb9dafbd1 gqt -- gitolite quick (re-)test
sometimes I want to quickly test a few lines of change within the context of
a currently-running/just-ran test, *without* doing the rollback etc.

Here's how you do that now:

  - in your source tree, make the change and then run:
        cp -a src hooks contrib/adc /some/tmp/place
  - go to the tester userid and re-run your tests like so:
        GQT=/some/tmp/place ./test-driver.sh
    it'll rollback as normal then overwrite src and hooks from $GQT

Also, there's now a "dbg" sub that can be used for quick printf-style
debugging.
2010-08-09 23:21:14 +05:30
Sitaram Chamarty f0c280cd38 allow "info" to have some chance of working on big-config setups!
Fedora, till now, had no hope in hell of running the info command.  Why?
Because the output of the info command is semantically the same as the
output of the compile script *before* the big-config mode was created.

And we all know how _that_ went ;-)

So now you get to give "info" a partial reponame or a pattern, just like
in the case of "expand".  And if you're under GL_BIG_CONFIG this pattern
is mandatory.  And if you try to cheat it'll still stop after showing 5
entries to prevent (accidental?) DOSs

Anyway, see doc changes in this commit for more details.
2010-07-31 01:30:43 +05:30
Sitaram Chamarty c25e05d87b distinguish "repo not found" from "no access" if the user has rights anyway
thanks to Jesse from the Fedora team for pointing this out.  They use
GL_NO_CREATE_REPOS, so sometimes the physical repo on disk doesn't exist
at the time the config file is written.

We're talking about non-wild repos only here, so this means it should
never happen to normal gitolite users.  But now -- in the rare case that
there is a disk-side problem -- people who have rights to a repo will
get a more specific error message.
2010-07-30 09:32:38 +05:30