de40461d9a
- explicit 'list' gives way to mindmap, ... - 'fm2mt.pl' to produce master-toc.mkd from the mindmap - mkdoc no longer ignores master-toc.mkd, calls fm2mt.pl itself and LOTS of changes to the actual docs
189 lines
7.1 KiB
Markdown
189 lines
7.1 KiB
Markdown
# mirroring using gitolite
|
|
|
|
**WARNING** existing gitolite mirroring users please note: **there are
|
|
significant changes** in syntax and usage compared to g2. The most important
|
|
change is that `config gitolite.mirror.master` changes to `option
|
|
mirror.master`, and similarly for 'slaves' and redirectOK. In addition, the
|
|
special value for 'redirectOK' changes from '1' to 'all'. The second big
|
|
change is the disappearance of the ambiguously named 'keys' (for example
|
|
`gitolite.mirror.hourly` etc., in the old mirroring doc) -- all that
|
|
complexity is yours to handle now :-)
|
|
|
|
----
|
|
|
|
Mirroring is simple: you have one "master" server and one or more "slave"
|
|
servers. The slaves get updates only from the master; to the rest of the
|
|
world they are at best read-only.
|
|
|
|
Gitolite extends this simple notion in the following ways:
|
|
|
|
* different masters and sets of slaves for different repos
|
|
|
|
This lets you do things like
|
|
|
|
* use the server closest to *most* of its developers as the master for
|
|
that repo
|
|
* not mirror a repo at all to some servers
|
|
* have repos that are purely local to a server (not mirrored at all)
|
|
* negotiate mirroring with servers that are not even under your control
|
|
* push to a slave on demand or via cron (helps deal with bandwidth or
|
|
connectivity constraints).
|
|
|
|
* pushes to a slave can be transparently forwarded to the real master
|
|
|
|
Your developers need not worry about where a repo's master is -- they just
|
|
write to their local mirror for *all* repos, even if their local mirror is
|
|
only a slave for some.
|
|
|
|
## caveats
|
|
|
|
* mirroring will never *create* a repo on a slave; it has to exist and be
|
|
prepared to receive updates from the master. (For example, [wild][] repos
|
|
must be created on the slave as well, otherwise they will not propagate).
|
|
|
|
* mirroring is only for git repos. Ancillary files like gl-creator and
|
|
gl-perms in the repo directory are not mirrored; you must do that
|
|
separately. Files in the admin directory (like log files) are also not
|
|
mirrored.
|
|
|
|
## setting up mirroring
|
|
|
|
This is in two parts: the initial setup and the rc file, followed by the conf
|
|
file settings and syntax.
|
|
|
|
### the initial setup and the rc file
|
|
|
|
On **each** server:
|
|
|
|
* install gitolite normally. Make clones of the server's 'gitolite-admin'
|
|
repo on your workstation so you can admin them all from one place.
|
|
|
|
* give the server a short, simple, "hostname" and set the HOSTNAME in the
|
|
rc file to this name, for example 'mars'.
|
|
|
|
* run ssh-keygen if needed and get an ssh key pair for the server. Copy the
|
|
public key to a common area and name it after the host, but with 'server-'
|
|
prefixed. So the pubkey for server 'mars' would be stored as
|
|
'server-mars.pub'.
|
|
|
|
* copy all keys to all the admin repo clones on your workstation and and add
|
|
them as usual. This is an `O(N^2)` operation ;-)
|
|
|
|
You may have guessed that the prefix 'server-' is special, and
|
|
distinguishes a human user from a mirroring peer.
|
|
|
|
* create "host" aliases to refer to all other machines. See [here][ssh-ha]
|
|
for what/how.
|
|
|
|
The host alias for a host (in all other machines' `~/.ssh/config` files)
|
|
MUST be the same as the `HOSTNAME` in the referred host's
|
|
`~/.gitolite.rc`. Gitolite mirroring **requires** this consistency in
|
|
naming; things will NOT work otherwise.
|
|
|
|
Normally you should be able to build one common file and append it to all
|
|
the servers' `~/.ssh/config` files.
|
|
|
|
* the following **MUST** work for **each pair** of servers that must talk to
|
|
each other:
|
|
|
|
# on server mars
|
|
ssh phobos info
|
|
# the response MUST start with "hello, server-mars..."
|
|
|
|
Note the exact syntax used; variations like "ssh git@phobos.example.com
|
|
info" are NOT sufficient. That is why you need the ssh host aliases.
|
|
|
|
Check this command from *everywhere to everywhere else*, and make sure you
|
|
get expected results. **Do NOT proceed otherwise.**
|
|
|
|
* setup the gitolite.conf file on all the servers. If the slaves are to be
|
|
exact copies of the master, you need to do the complete configuration only
|
|
on the master; the slaves can have just this:
|
|
|
|
repo gitolite-admin
|
|
RW+ = <some local admin>
|
|
|
|
option mirror.master = mars
|
|
option mirror.slaves = phobos
|
|
|
|
because on the first push to the master it will update all the slaves
|
|
anyway.
|
|
|
|
* when that is all done and tested, **enable mirroring** by going through
|
|
the rc file and uncommenting all the lines mentioning `Mirroring`.
|
|
|
|
### conf file settings and syntax
|
|
|
|
Mirroring is defined by the following [options][]. You can have different
|
|
settings for different repos, and of course some repos may not have any mirror
|
|
options at all -- they are then purely local.
|
|
|
|
repo foo
|
|
...access rules...
|
|
|
|
option mirror.master = mars
|
|
option mirror.slaves = phobos deimos
|
|
option mirror.redirectOK = all
|
|
|
|
The first line is easy, since a repo can have only one master.
|
|
|
|
The second is a space separated list of hosts that are all slaves. You can
|
|
have several slave lists, as long as the config key starts with
|
|
'mirror.slaves' and is unique. For example.
|
|
|
|
option mirror.slaves-1 = phobos deimos
|
|
option mirror.slaves-2 = io europa
|
|
option mirror.slaves-3 = ganymede callisto
|
|
|
|
Do not repeat a key; then only the last line for that key will be effective.
|
|
|
|
## redirected pushes
|
|
|
|
**Please read carefully; there are security implications if you enable this
|
|
for mirrors NOT under your control**.
|
|
|
|
Normally, a master, (and *only* a master), pushes to a slave, and the slaves
|
|
are "read-only" to the users. Gitolite allows a *slave* to receive pushes
|
|
from a user and transparently redirect them to the *master*.
|
|
|
|
This simplifies things for users in complex setups, letting them use their
|
|
local mirror for both fetch and push access to all repos.
|
|
|
|
Just remember that if you do this, **authentication** happens on the slave,
|
|
but **authorisation** is on the master. The master is trusting the slave to
|
|
authenticate the user correctly, *and* use the same authentication data (i.e.,
|
|
user alice on the slave should be guaranteed to be the same as user alice on
|
|
the master).
|
|
|
|
The syntax for enabling this is one of these:
|
|
|
|
option mirror.redirectOK = all
|
|
option mirror.redirectOK = phobos deimos
|
|
|
|
The first syntax trusts all valid slaves to redirect user pushes, while the
|
|
second one trusts only some slaves.
|
|
|
|
Note that you cannot redirect gitolite commands (like perms, etc).
|
|
|
|
## manually synchronising a slave repo
|
|
|
|
You can use the `gitolite mirror push` command on a master to manually
|
|
synchronise any of its slaves. Try it with `-h` to get usage info.
|
|
|
|
Tip: if you want to do this to all the slaves, try this:
|
|
|
|
for s in `gitolite git-config -r reponame mirror.slave | cut -f3`
|
|
do
|
|
gitolite mirror push $s reponame
|
|
done
|
|
|
|
This command can also be run remotely; run `ssh git@host mirror -h` for
|
|
details.
|
|
|
|
## appendix A: efficiency versus paranoia
|
|
|
|
If you're paranoid enough to use mirrors, you should be paranoid enough to
|
|
set this on each server, despite the possible CPU overhead:
|
|
|
|
git config --global receive.fsckObjects true
|