gitolite/doc/mirroring.mkd
Sitaram Chamarty db70595b87 fixup to pushing wild repos permissions...
- fix docs to explicitly state that mirroring wild repos is a bad idea
    if the authentication data is not the same on the peers.

  - an important check against a malicious peer was missed out.  If
    people heed the warning above this check is not really needed but it
    is good for completeness.

  - warning about redirected pushes removed, thanks to Ronald Ip having
    tested it and reported it working.
2012-06-03 07:44:08 +05:30

236 lines
8.9 KiB
Markdown

# mirroring using gitolite
<font color="red">**WARNING** existing gitolite mirroring users please note:
**there are [significant changes][g2i-mirroring]** in syntax and usage
compared to g2. If you're not the kind who reads documentation before doing
serious system admin things, well... good luck!</font>
----
[[TOC]]
----
Mirroring is simple: you have one "master" server and one or more "slave"
servers. The slaves get updates only from the master; to the rest of the
world they are at best read-only.
Gitolite extends this simple notion in the following ways:
* Different masters and sets of slaves for different repos
This lets you do things like:
* Use the server closest to *most* of its developers as the master for
that repo.
* Not mirror a repo at all to some servers.
* Have repos that are purely local to a server (not mirrored at all).
* Negotiate mirroring with servers that are not even under your control.
* Push to a slave on demand or via cron (helps deal with bandwidth or
connectivity constraints).
* Pushes to a slave can be transparently forwarded to the real master.
Your developers need not worry about where a repo's master is -- they just
write to their local mirror for *all* repos, even if their local mirror is
only a slave for some.
## caveats
* Mirroring will never *create* a repo on a slave; it has to exist and be
prepared to receive updates from the master.
However, there is limited support for auto-creating wild card repos and
sending 'perms' info across, with the following caveats at present. (Some
of this text won't make sense unless you know what those features are).
* *WARNING: it does NOT make sense to mirror wild repos in setups where
the authentication data is not the same (i.e., where "alice" on the
master and "alice" on a slave maybe totally different people)*.
* This has only been minimally tested. For example, complex setups or
asymmetric configs on master and slave, etc. have NOT been tested.
* Permission changes will only propagate on the next 'git push'. Of
course, if you know the name of the slave server, you can run
ssh git@host mirror push slave-server-name repo-name
* Using 'perms' on a slave is allowed but will neither propagate nor
persist. They will be overwritten by whatever perms the master has
(even if it is an empty set) on the next 'git push'.
* As with lots of extra features in gitolite, smart http support is not
on my radar. Don't ask.
Please test it out and let me know if something surprising happens. Be
aware that I have been known to claim bugs are features if I don't have
time to fix them immediately :-)
* Mirroring is only for git repos. Ancillary files like gl-creator and
gl-perms in the repo directory are not mirrored; you must do that
separately. Files in the admin directory (like log files) are also not
mirrored.
* If you ever do a [bypass push][bypass], mirroring will not work.
Mirroring checks also will not work -- for example, you can push to a
slave, which is not usually a good idea. So don't bypass gitolite if the
repo is mirrored!
## setting up mirroring
This is in two parts: the initial setup and the rc file, followed by the conf
file settings and syntax.
### the initial setup and the rc file
On **each** server:
* Install gitolite normally. Make clones of the server's 'gitolite-admin'
repo on your workstation so you can admin them all from one place.
* Give the server a short, simple, "hostname" and set the HOSTNAME in the
rc file to this name, for example 'mars'.
* Run ssh-keygen if needed and get an ssh key pair for the server. Copy the
public key to a common area and name it after the host, but with 'server-'
prefixed. So the pubkey for server 'mars' would be stored as
'server-mars.pub'.
* Copy all keys to all the admin repo clones on your workstation and and add
them as usual. This is an `O(N^2)` operation ;-)
You may have guessed that the prefix 'server-' is special, and
distinguishes a human user from a mirroring peer.
* Create "host" aliases to refer to all other machines. See [here][ssh-ha]
for what/how.
The host alias for a host (in all other machines' `~/.ssh/config` files)
MUST be the same as the `HOSTNAME` in the referred host's
`~/.gitolite.rc`. Gitolite mirroring **requires** this consistency in
naming; things will NOT work otherwise.
Normally you should be able to build one common file and append it to all
the servers' `~/.ssh/config` files.
* The following **MUST** work for **each pair** of servers that must talk to
each other:
# on server mars
ssh phobos info
# the response MUST start with "hello, server-mars..."
Note the exact syntax used; variations like "ssh git@phobos.example.com
info" are NOT sufficient. That is why you need the ssh host aliases.
Check this command from *everywhere to everywhere else*, and make sure you
get expected results. **Do NOT proceed otherwise.**
* Setup the gitolite.conf file on all the servers. If the slaves are to be
exact copies of the master, you need to do the complete configuration only
on the master; the slaves can have just this:
repo gitolite-admin
RW+ = <some local admin>
option mirror.master = mars
option mirror.slaves = phobos
because on the first push to the master it will update all the slaves
anyway.
* When that is all done and tested, **enable mirroring** by going through
the rc file and uncommenting all the lines mentioning `Mirroring`.
### conf file settings and syntax
Mirroring is defined by the following [options][]. You can have different
settings for different repos, and of course some repos may not have any mirror
options at all -- they are then purely local.
repo foo
...access rules...
option mirror.master = mars
option mirror.slaves = phobos deimos
option mirror.redirectOK = all
The first line is easy, since a repo can have only one master.
The second is a space separated list of hosts that are all slaves. You can
have several slave lists, as long as the config key starts with
'mirror.slaves' and is unique. For example.
option mirror.slaves-1 = phobos deimos
option mirror.slaves-2 = io europa
option mirror.slaves-3 = ganymede callisto
Do not repeat a key; then only the last line for that key will be effective.
## redirected pushes
**Please read carefully; there are security implications if you enable this
for mirrors NOT under your control**.
Normally, a master, (and *only* a master), pushes to a slave, and the slaves
are "read-only" to the users. Gitolite allows a *slave* to receive pushes
from a user and transparently redirect them to the *master*.
This simplifies things for users in complex setups, letting them use their
local mirror for both fetch and push access to all repos.
Just remember that if you do this, **authentication** happens on the slave,
but **authorisation** is on the master. The master is trusting the slave to
authenticate the user correctly, *and* use the same authentication data (i.e.,
user alice on the slave should be guaranteed to be the same as user alice on
the master).
The syntax for enabling this is one of these:
option mirror.redirectOK = all
option mirror.redirectOK = phobos deimos
The first syntax trusts all valid slaves to redirect user pushes, while the
second one trusts only some slaves.
Note that you cannot redirect gitolite commands (like perms, etc).
## #sync manually synchronising a slave repo
You can use the `gitolite mirror push` command on a master to manually
synchronise any of its slaves. Try it with `-h` to get usage info.
Tip: if you want to do this to all the slaves, try this:
for s in `gitolite git-config -r reponame mirror.slave | cut -f3`
do
gitolite mirror push $s reponame
done
This command can also be run remotely; run `ssh git@host mirror -h` for
details.
## #HOSTNAME appendix A: HOSTNAME substitution
Wherever gitolite sees the word `%HOSTNAME`, it will replace it with the
HOSTNAME supplied in the rc file, if one was supplied. This lets you maintain
configurations for all servers in one repo, yet havethem act differently on
different servers, by saying something like:
subconf "%HOSTNAME/*.conf"
You can use it in other places also, for example:
RW+ VREF/NAME/subs/%HOSTNAME/ = @%HOSTNAME-admins
(you still have to define @mars-admins, @phobos-admins, etc., but the actual
VREF is now one line instead of one for each server!)
## appendix B: efficiency versus paranoia
If you're paranoid enough to use mirrors, you should be paranoid enough to
set this on each server, despite the possible CPU overhead:
git config --global receive.fsckObjects true