db70595b87
- fix docs to explicitly state that mirroring wild repos is a bad idea
if the authentication data is not the same on the peers.
- an important check against a malicious peer was missed out. If
people heed the warning above this check is not really needed but it
is good for completeness.
- warning about redirected pushes removed, thanks to Ronald Ip having
tested it and reported it working.
236 lines
8.9 KiB
Markdown
236 lines
8.9 KiB
Markdown
# mirroring using gitolite
|
|
|
|
<font color="red">**WARNING** existing gitolite mirroring users please note:
|
|
**there are [significant changes][g2i-mirroring]** in syntax and usage
|
|
compared to g2. If you're not the kind who reads documentation before doing
|
|
serious system admin things, well... good luck!</font>
|
|
|
|
----
|
|
|
|
[[TOC]]
|
|
|
|
----
|
|
|
|
Mirroring is simple: you have one "master" server and one or more "slave"
|
|
servers. The slaves get updates only from the master; to the rest of the
|
|
world they are at best read-only.
|
|
|
|
Gitolite extends this simple notion in the following ways:
|
|
|
|
* Different masters and sets of slaves for different repos
|
|
|
|
This lets you do things like:
|
|
|
|
* Use the server closest to *most* of its developers as the master for
|
|
that repo.
|
|
* Not mirror a repo at all to some servers.
|
|
* Have repos that are purely local to a server (not mirrored at all).
|
|
* Negotiate mirroring with servers that are not even under your control.
|
|
* Push to a slave on demand or via cron (helps deal with bandwidth or
|
|
connectivity constraints).
|
|
|
|
* Pushes to a slave can be transparently forwarded to the real master.
|
|
|
|
Your developers need not worry about where a repo's master is -- they just
|
|
write to their local mirror for *all* repos, even if their local mirror is
|
|
only a slave for some.
|
|
|
|
## caveats
|
|
|
|
* Mirroring will never *create* a repo on a slave; it has to exist and be
|
|
prepared to receive updates from the master.
|
|
|
|
However, there is limited support for auto-creating wild card repos and
|
|
sending 'perms' info across, with the following caveats at present. (Some
|
|
of this text won't make sense unless you know what those features are).
|
|
|
|
* *WARNING: it does NOT make sense to mirror wild repos in setups where
|
|
the authentication data is not the same (i.e., where "alice" on the
|
|
master and "alice" on a slave maybe totally different people)*.
|
|
|
|
* This has only been minimally tested. For example, complex setups or
|
|
asymmetric configs on master and slave, etc. have NOT been tested.
|
|
|
|
* Permission changes will only propagate on the next 'git push'. Of
|
|
course, if you know the name of the slave server, you can run
|
|
|
|
ssh git@host mirror push slave-server-name repo-name
|
|
|
|
* Using 'perms' on a slave is allowed but will neither propagate nor
|
|
persist. They will be overwritten by whatever perms the master has
|
|
(even if it is an empty set) on the next 'git push'.
|
|
|
|
* As with lots of extra features in gitolite, smart http support is not
|
|
on my radar. Don't ask.
|
|
|
|
Please test it out and let me know if something surprising happens. Be
|
|
aware that I have been known to claim bugs are features if I don't have
|
|
time to fix them immediately :-)
|
|
|
|
* Mirroring is only for git repos. Ancillary files like gl-creator and
|
|
gl-perms in the repo directory are not mirrored; you must do that
|
|
separately. Files in the admin directory (like log files) are also not
|
|
mirrored.
|
|
|
|
* If you ever do a [bypass push][bypass], mirroring will not work.
|
|
Mirroring checks also will not work -- for example, you can push to a
|
|
slave, which is not usually a good idea. So don't bypass gitolite if the
|
|
repo is mirrored!
|
|
|
|
## setting up mirroring
|
|
|
|
This is in two parts: the initial setup and the rc file, followed by the conf
|
|
file settings and syntax.
|
|
|
|
### the initial setup and the rc file
|
|
|
|
On **each** server:
|
|
|
|
* Install gitolite normally. Make clones of the server's 'gitolite-admin'
|
|
repo on your workstation so you can admin them all from one place.
|
|
|
|
* Give the server a short, simple, "hostname" and set the HOSTNAME in the
|
|
rc file to this name, for example 'mars'.
|
|
|
|
* Run ssh-keygen if needed and get an ssh key pair for the server. Copy the
|
|
public key to a common area and name it after the host, but with 'server-'
|
|
prefixed. So the pubkey for server 'mars' would be stored as
|
|
'server-mars.pub'.
|
|
|
|
* Copy all keys to all the admin repo clones on your workstation and and add
|
|
them as usual. This is an `O(N^2)` operation ;-)
|
|
|
|
You may have guessed that the prefix 'server-' is special, and
|
|
distinguishes a human user from a mirroring peer.
|
|
|
|
* Create "host" aliases to refer to all other machines. See [here][ssh-ha]
|
|
for what/how.
|
|
|
|
The host alias for a host (in all other machines' `~/.ssh/config` files)
|
|
MUST be the same as the `HOSTNAME` in the referred host's
|
|
`~/.gitolite.rc`. Gitolite mirroring **requires** this consistency in
|
|
naming; things will NOT work otherwise.
|
|
|
|
Normally you should be able to build one common file and append it to all
|
|
the servers' `~/.ssh/config` files.
|
|
|
|
* The following **MUST** work for **each pair** of servers that must talk to
|
|
each other:
|
|
|
|
# on server mars
|
|
ssh phobos info
|
|
# the response MUST start with "hello, server-mars..."
|
|
|
|
Note the exact syntax used; variations like "ssh git@phobos.example.com
|
|
info" are NOT sufficient. That is why you need the ssh host aliases.
|
|
|
|
Check this command from *everywhere to everywhere else*, and make sure you
|
|
get expected results. **Do NOT proceed otherwise.**
|
|
|
|
* Setup the gitolite.conf file on all the servers. If the slaves are to be
|
|
exact copies of the master, you need to do the complete configuration only
|
|
on the master; the slaves can have just this:
|
|
|
|
repo gitolite-admin
|
|
RW+ = <some local admin>
|
|
|
|
option mirror.master = mars
|
|
option mirror.slaves = phobos
|
|
|
|
because on the first push to the master it will update all the slaves
|
|
anyway.
|
|
|
|
* When that is all done and tested, **enable mirroring** by going through
|
|
the rc file and uncommenting all the lines mentioning `Mirroring`.
|
|
|
|
### conf file settings and syntax
|
|
|
|
Mirroring is defined by the following [options][]. You can have different
|
|
settings for different repos, and of course some repos may not have any mirror
|
|
options at all -- they are then purely local.
|
|
|
|
repo foo
|
|
...access rules...
|
|
|
|
option mirror.master = mars
|
|
option mirror.slaves = phobos deimos
|
|
option mirror.redirectOK = all
|
|
|
|
The first line is easy, since a repo can have only one master.
|
|
|
|
The second is a space separated list of hosts that are all slaves. You can
|
|
have several slave lists, as long as the config key starts with
|
|
'mirror.slaves' and is unique. For example.
|
|
|
|
option mirror.slaves-1 = phobos deimos
|
|
option mirror.slaves-2 = io europa
|
|
option mirror.slaves-3 = ganymede callisto
|
|
|
|
Do not repeat a key; then only the last line for that key will be effective.
|
|
|
|
## redirected pushes
|
|
|
|
**Please read carefully; there are security implications if you enable this
|
|
for mirrors NOT under your control**.
|
|
|
|
Normally, a master, (and *only* a master), pushes to a slave, and the slaves
|
|
are "read-only" to the users. Gitolite allows a *slave* to receive pushes
|
|
from a user and transparently redirect them to the *master*.
|
|
|
|
This simplifies things for users in complex setups, letting them use their
|
|
local mirror for both fetch and push access to all repos.
|
|
|
|
Just remember that if you do this, **authentication** happens on the slave,
|
|
but **authorisation** is on the master. The master is trusting the slave to
|
|
authenticate the user correctly, *and* use the same authentication data (i.e.,
|
|
user alice on the slave should be guaranteed to be the same as user alice on
|
|
the master).
|
|
|
|
The syntax for enabling this is one of these:
|
|
|
|
option mirror.redirectOK = all
|
|
option mirror.redirectOK = phobos deimos
|
|
|
|
The first syntax trusts all valid slaves to redirect user pushes, while the
|
|
second one trusts only some slaves.
|
|
|
|
Note that you cannot redirect gitolite commands (like perms, etc).
|
|
|
|
## #sync manually synchronising a slave repo
|
|
|
|
You can use the `gitolite mirror push` command on a master to manually
|
|
synchronise any of its slaves. Try it with `-h` to get usage info.
|
|
|
|
Tip: if you want to do this to all the slaves, try this:
|
|
|
|
for s in `gitolite git-config -r reponame mirror.slave | cut -f3`
|
|
do
|
|
gitolite mirror push $s reponame
|
|
done
|
|
|
|
This command can also be run remotely; run `ssh git@host mirror -h` for
|
|
details.
|
|
|
|
## #HOSTNAME appendix A: HOSTNAME substitution
|
|
|
|
Wherever gitolite sees the word `%HOSTNAME`, it will replace it with the
|
|
HOSTNAME supplied in the rc file, if one was supplied. This lets you maintain
|
|
configurations for all servers in one repo, yet havethem act differently on
|
|
different servers, by saying something like:
|
|
|
|
subconf "%HOSTNAME/*.conf"
|
|
|
|
You can use it in other places also, for example:
|
|
|
|
RW+ VREF/NAME/subs/%HOSTNAME/ = @%HOSTNAME-admins
|
|
|
|
(you still have to define @mars-admins, @phobos-admins, etc., but the actual
|
|
VREF is now one line instead of one for each server!)
|
|
|
|
## appendix B: efficiency versus paranoia
|
|
|
|
If you're paranoid enough to use mirrors, you should be paranoid enough to
|
|
set this on each server, despite the possible CPU overhead:
|
|
|
|
git config --global receive.fsckObjects true
|