246165537d
Previous implementations of "give shell access to some gitolite users" feature were crap. There was no easy/elegant way to ensure that someone who had repo admin access would not manage to get himself shell access. Giving someone shell access requires that you should have shell access in the first place, so the simplest way is to enable it from the server side only. So now that we decided to do that, we may as well prepare for other, future, commands by starting a server-side utility program with sub-commands (the only current one being "shell-add")
409 lines
16 KiB
Markdown
409 lines
16 KiB
Markdown
# ssh troubleshooting
|
|
|
|
In this document:
|
|
|
|
* basic ssh troubleshooting
|
|
* passphrases versus passwords
|
|
* ssh-agent problems
|
|
* basic ssh troubleshooting for the main admin
|
|
* basic ssh troubleshooting for a normal user
|
|
* details
|
|
* files on the server
|
|
* files on client
|
|
* why two keys on client
|
|
* more complex ssh setups
|
|
* two gitolite servers to manage?
|
|
* giving shell access to gitolite users
|
|
|
|
----
|
|
|
|
This document should help you troubleshoot ssh-related problems in accessing
|
|
gitolite *after* the install has completed successfully.
|
|
|
|
In addition, I **strongly** recommend reading [this document][glb] -- it's a
|
|
very detailed look at how gitolite uses ssh's features on the server side.
|
|
Most people don't know ssh as well as they *think* they do; even if you don't
|
|
have any problems right now, it's worth skimming over.
|
|
|
|
In addition to both these documents, there's now a program called
|
|
`sshkeys-lint` that you can run on your client. Run it without arguments to
|
|
get help on how to run it and what inputs it needs.
|
|
|
|
Please also note that ssh problems don't always look like ssh problems. One
|
|
common example: when the remote says the repo you're trying to access "does
|
|
not appear to be a git repository", and yet you are sure it exists, you
|
|
haven't mis-spelled it, etc. Another example is being able to access
|
|
repositories using the full unix path (typically like
|
|
`git@server:repositories/reponame.git`, assuming default `$REPO_BASE` setting,
|
|
instead of specifying only the part below `$REPO_BASE`, i.e.,
|
|
`git@server:reponame.git`).
|
|
|
|
[Both these errors indicate that you managed to bypass gitolite completely and
|
|
are using your shell access -- instead of running via
|
|
`/some/path/gl-auth-command <your_username>` it is just going to bash and
|
|
working from there!]
|
|
|
|
<a name="basic"></a>
|
|
|
|
### basic ssh troubleshooting
|
|
|
|
[glb]: http://sitaramc.github.com/0-installing/9-gitolite-basics.html#IMPORTANT_overview_of_ssh
|
|
|
|
I assume the gitolite server is called "server" and the user hosting all the
|
|
gitolite repos is "git". I will also be using "sitaram" as the *gitolite
|
|
username* of the admin.
|
|
|
|
Unless specifically mentioned, all these commands are run on the user's or
|
|
admin's workstation, not on the server.
|
|
|
|
#### passphrases versus passwords
|
|
|
|
When you create an ssh keypair, you have the option of protecting it with a
|
|
passphrase. When you subsequently use that keypair to access a remote host,
|
|
your *local* ssh client needs to unlock the corresponding private key, and ssh
|
|
will probably ask for the passphrase you set when you created the keypair.
|
|
|
|
Do not confuse or mistake this prompt (`Enter passphrase for key
|
|
'/home/sitaram/.ssh/id_rsa':`) for a password prompt from the remote server!
|
|
|
|
You have two choices to avoid this prompt every time you try to access the
|
|
remote. The first is to create keypairs *without* a passphrase (just hit
|
|
enter when prompted for one). **Be sure to add a passphrase later, once
|
|
everything is working, using `ssh-keygen -p`**.
|
|
|
|
The second is to use `ssh-agent` (or `keychain`, which in turn uses
|
|
`ssh-agent`) or something like that to manage your keys. Other than the next
|
|
section, further discussion of this is out of scope of this document.
|
|
|
|
#### ssh-agent problems
|
|
|
|
1. Run `ssh-add -l`. If this responds with either "The agent has no
|
|
identities." or "Could not open a connection to your authentication
|
|
agent.", skip this section.
|
|
|
|
2. However, if it lists some keys, like this:
|
|
|
|
2048 fc:c1:48:1e:06:31:97:a4:8b:fc:37:b2:76:14:c7:53 /home/sitaram/.ssh/id_rsa (RSA)
|
|
2048 d2:e0:7f:fa:1a:89:22:41:bb:06:d9:ff:a7:27:36:5c /home/sitaram/.ssh/sitaram (RSA)
|
|
|
|
then run `ls ~/.ssh` and make sure that all the keypairs you have there
|
|
are represented in the `ssh-add -l` output.
|
|
|
|
3. If you find any keypairs in `~/.ssh` that are not represented in the
|
|
`ssh-add -l` output, add them. For instance, if `ssh-add -l` showed me
|
|
only the `id_rsa` key, but I also had a `sitaram` (and `sitaram.pub`)
|
|
keypair, I'd run `ssh-add ~/.ssh/sitaram` to add it.
|
|
|
|
This is because ssh-agent has a quirk: if `ssh-add -l` shows *any* keys at
|
|
all, ssh will only use those keys. Even if you explicitly specify an unlisted
|
|
key using `ssh -i` or an `identityfile` directive in the config file, it won't
|
|
use it.
|
|
|
|
#### basic ssh troubleshooting for the main admin
|
|
|
|
You're the "main admin" if you're trying to access gitolite from the same
|
|
workstation and user account where you ran the "easy install" command. You
|
|
should have two keypairs in your `~/.ssh` directory. The pair called `id_rsa`
|
|
(and `id_rsa.pub`) was probably the first one you created, and you used this
|
|
to get passwordless (pubkey based) access to the server (which was a
|
|
pre-requisite for running the easy install command).
|
|
|
|
The second keypair has the same name as the last argument in the easy install
|
|
command you ran (in my case, `sitaram` and `sitaram.pub`). It was probably
|
|
created by the easy install script, and is the key used for gitolite access.
|
|
|
|
In addition, you should have a "gitolite" paragraph in your `~/.ssh/config`,
|
|
looking something like this:
|
|
|
|
host gitolite
|
|
user git
|
|
hostname server
|
|
identityfile ~/.ssh/sitaram
|
|
|
|
If any of these are not true, you did something funky in your install; email
|
|
me or hop onto #git and hope for the best ;-)
|
|
|
|
Otherwise, run these checks:
|
|
|
|
1. `ssh git@server` should get you a command line.
|
|
|
|
If it asks you for a password, then your `id_rsa` keypair changed after
|
|
you ran the easy install, or someone fiddled with the
|
|
`~/.ssh/authorized_keys` file on the server.
|
|
|
|
If it prints [gitolite version and access info][myrights], you managed to
|
|
overwrite the `id_rsa` keypair with the `sitaram` keypair, or something
|
|
equally weird.
|
|
|
|
2. `ssh gitolite info` should print some [gitolite version and access
|
|
info][myrights]. If you get the output of the GNU info command instead,
|
|
you probably reused your `id_rsa` keypair as your `sitaram` keypair, or
|
|
overwrote the `sitaram` keypair with the `id_rsa` keypair.
|
|
|
|
There are many ways to fix this, depending on where and what the damage is.
|
|
The most generic way (and therefore time-taking) is to re-install gitolite
|
|
from scratch:
|
|
|
|
* make a backup of your gitolite-admin repo clone somewhere (basically your
|
|
"keydir/*.pub" and your "conf/gitolite.conf"). If necessary get these
|
|
files from the server's `~/.gitolite` directory.
|
|
* log on to the server somehow (using some other account, using a password,
|
|
su-ing in, etc) and delete `~/.ssh/authorized_keys`. Rename or move aside
|
|
`~/.gitolite` so that also looks like it is missing.
|
|
* back on your workstation, make sure you have 2 keypairs (`id_rsa` and
|
|
`sitaram`, along with corresponding `.pub` files). Create them if needed.
|
|
Also make sure they are *different* and not a copy of each other :-)
|
|
* install gitolite normally:
|
|
* run `ssh-copy-id -i ~/.ssh/id_rsa git@server` to get passwordless
|
|
access to the server. (Mac users may have to do this step manually)
|
|
* make sure `ssh git@server pwd` prints the `$HOME` of `git@server`
|
|
**without** asking for a password. Do not proceed till this works.
|
|
* run easy install again, (in my case: `cd gitolite-source;
|
|
src/gl-easy-install -q git server sitaram`)
|
|
* go to your gitolite-admin repo clone, and copy `conf/gitolite.conf` and
|
|
`keydir/*.pub` from your backup to this directory
|
|
* copy (be sure to overwrite!) `~/.ssh/sitaram.pub` also to keydir
|
|
* now `git add keydir; git commit; git push -f`
|
|
|
|
That's a long sequence but it should work.
|
|
|
|
#### basic ssh troubleshooting for a normal user
|
|
|
|
For a normal user, life is much simpler. They should have only one pubkey,
|
|
which was previously sent to the gitolite admin to add into the admin repo's
|
|
`keydir` as "user.pub", and then "user" given permissions to some repo.
|
|
|
|
`ssh git@server info` should get you [gitolite version and access
|
|
info][myrights]. If it asks you for a password, your pubkey was not sent to
|
|
the server properly. Check with your admin.
|
|
|
|
[myrights]: http://github.com/sitaramc/gitolite/blob/pu/doc/3-faq-tips-etc.mkd#myrights
|
|
|
|
If it gets you the GNU info command output, you have shell access. This means
|
|
you had command line access to the server *before* you were added as a
|
|
gitolite user. If you send that same key to your gitolite admin to include in
|
|
the admin repo, it won't work. For reasons why, see below.
|
|
|
|
<a name="details"></a>
|
|
|
|
### details
|
|
|
|
Here's how it all hangs together.
|
|
|
|
#### files on the server
|
|
|
|
* the authkeys file; this contains one line containing the pubkey of each
|
|
user who is permitted to login without a password.
|
|
|
|
Pubkey lines that give shell access look like this:
|
|
|
|
ssh-rsa AAAAB3NzaC[snip]uPjrUiAUew== /home/sitaram/.ssh/id_rsa
|
|
|
|
On a typical server there will be only one or two of these lines.
|
|
|
|
Note that the last bit (`/home/sitaram/.ssh/id_rsa`) is purely a *comment*
|
|
field and can be anything. Also, the actual lines are much longer, about
|
|
400 characters; I snipped 'em in the middle, as you can see.
|
|
|
|
In contrast, pubkey lines that give access to git repos hosted by gitolite
|
|
look like this:
|
|
|
|
command="[some path]src/gl-auth-command sitaram",[some restrictions] ssh-rsa AAAAB3NzaC[snip]s18OnB42oQ== sitaram@sita-lt
|
|
|
|
You will have many more of these lines -- one for every pubkey file in
|
|
`keydir/` of your gitolite-admin repo, with the corresponding username in
|
|
place of "sitaram" in the example above.
|
|
|
|
The "command=" at the beginning ensures that when someone with the
|
|
corresponding private key logs in, they don't get a shell. Instead, the
|
|
`gl-auth-command` program is run, and (in this example) is given the
|
|
argument `sitaram`. This is how gitolite is invoked, (and is told the
|
|
user logging in is "sitaram").
|
|
|
|
#### files on client
|
|
|
|
* default keypair; used to get shell access to servers. You would have
|
|
copied this pubkey to the gitolite server in order to log in without a
|
|
password. (On Linux systems you may have used `ssh-copy-id` to do that).
|
|
You would have done this *before* you ran the easy install script, because
|
|
otherwise easy install won't run!
|
|
|
|
~/.ssh/id_rsa
|
|
~/.ssh/id_rsa.pub
|
|
|
|
* gitolite keypair; the "sitaram" in this is the 3rd argument to the
|
|
`src/gl-easy-install` command you ran; the easy install script does the
|
|
rest
|
|
|
|
~/.ssh/sitaram
|
|
~/.ssh/sitaram.pub
|
|
|
|
* config file; this file has an entry for gitolite access:
|
|
|
|
~/.ssh/config
|
|
|
|
To understand why we need that, let's step back a bit. Normally, you
|
|
might expect to access gitolite repos like this:
|
|
|
|
ssh://git@server/reponame.git
|
|
|
|
But this won't work, because this ends up using the *default* keypair
|
|
(normally), which gives you a command line. Which means it won't invoke
|
|
the `gl-auth-command` program at all, and so none of gitolite's access
|
|
control will work.
|
|
|
|
<a name="altkey"></a>
|
|
|
|
You need to force ssh to use the *other* keypair when performing a git
|
|
operation. With normal ssh, that would be
|
|
|
|
ssh -i ~/.ssh/sitaram git@server
|
|
|
|
but git does not support putting an alternate keypair in the URL.
|
|
|
|
Luckily, ssh has a very convenient way of capturing all the connection
|
|
information (username, hostname, port number (if it's not the default 22),
|
|
and keypair to be used) in one "paragraph" of `~/.ssh/config`. This is
|
|
what the para looks like for us (the easy install script puts it there the
|
|
first time):
|
|
|
|
host gitolite
|
|
user git
|
|
hostname server
|
|
identityfile ~/.ssh/sitaram
|
|
|
|
(The "gitolite" can be anything you want of course; it's like a group name
|
|
for all the stuff below it). This ensures that typing
|
|
|
|
ssh gitolite
|
|
|
|
is equivalent to
|
|
|
|
ssh -i ~/.ssh/sitaram git@server
|
|
|
|
and therefore this:
|
|
|
|
git clone gitolite:reponame.git
|
|
|
|
now works as expected, invoking the special keypair instead of the default
|
|
one.
|
|
|
|
<a name="twokeys"></a>
|
|
|
|
#### why two keys on client
|
|
|
|
Why do I (the admin) need two **different** keypairs?
|
|
|
|
There are two types of access the admin will make to the server: a normal
|
|
login, to get a shell prompt, and gitolite access (clone/fetch/push etc). The
|
|
first access needs an authkeys line *without* any "command=" restrictions,
|
|
while the second requires a line *with* such a restriction.
|
|
|
|
And we can't use the same key for both because there is no way to disambiguate
|
|
them; the ssh server will always (*always*) pick the first one in sequence
|
|
when the key is offered by the ssh client.
|
|
|
|
So the next question is usually "I have other ways to get a shell on that
|
|
account, so why do I need a key for shell access at all?"
|
|
|
|
The answer to this is that the "easy install" script, being written for the
|
|
most general case, needs shell access via ssh to do its stuff.
|
|
|
|
If you really, really, want to get rid of the extra key, here's a transcript
|
|
that should have enough info to get you going (but it helps to know ssh well):
|
|
|
|
* on "sitaram" user, on my workstation
|
|
|
|
cd ~/.ssh
|
|
cp id_rsa sitaram
|
|
cp id_rsa.pub sitaram.pub
|
|
cd ~/gitolite-clone
|
|
src/gl-easy-install -q git my.git.server sitaram
|
|
|
|
that last command produces something like the following:
|
|
|
|
you are upgrading from (unknown) to v0.80-6-gdde8c4e
|
|
setting up keypair...
|
|
...reusing /home/sitaram/.ssh/sitaram.pub...
|
|
creating gitolite para in ~/.ssh/config...
|
|
finding/creating gitolite rc...
|
|
installing/upgrading...
|
|
Pseudo-terminal will not be allocated because stdin is not a terminal.
|
|
[master (root-commit) e717a89] start
|
|
2 files changed, 11 insertions(+), 0 deletions(-)
|
|
create mode 100644 conf/gitolite.conf
|
|
create mode 100644 keydir/sitaram.pub
|
|
cloning gitolite-admin repo...
|
|
Initialized empty Git repository in /home/sitaram/gitolite-admin/.git/
|
|
fatal: 'gitolite-admin.git' does not appear to be a git repository
|
|
fatal: The remote end hung up unexpectedly
|
|
|
|
notice that the final step (the clone of the newly created gitolite-admin
|
|
repo) failed, as expected
|
|
|
|
* now log on to the git hosting account (`git@my.git.server` in this
|
|
example), edit `~/.ssh/authorized_keys`, and delete the line with the
|
|
first occurrence of your key (this should be *before* the `# gitolite
|
|
start` line)
|
|
|
|
* now go back to your workstation and
|
|
|
|
git clone git@my.git.server:gitolite-admin
|
|
|
|
That should do it.
|
|
|
|
<a name="complex"></a>
|
|
|
|
### more complex ssh setups
|
|
|
|
What do you need to know in order to create more complex ssh setups (for
|
|
instance if you have *two* gitolite servers you are administering)?
|
|
|
|
#### two gitolite servers to manage?
|
|
|
|
* they can have the same key; no harm there (example, sitaram.pub)
|
|
|
|
* instead of just one ssh/config para, you now have two (assuming that the
|
|
remote user on both machines is called "git"):
|
|
|
|
host gitolite
|
|
user git
|
|
hostname server
|
|
identityfile ~/.ssh/sitaram
|
|
|
|
host gitolite2
|
|
user git
|
|
hostname server2
|
|
identityfile ~/.ssh/sitaram
|
|
|
|
* now access one server's repos as `gitolite:reponame.git` and the other
|
|
server's repos as `gitolite2:reponame.git`.
|
|
|
|
<a name="shell"></a>
|
|
|
|
### giving shell access to gitolite users
|
|
|
|
We've managed (thanks to an idea from Jesse Keating) to make it possible for a
|
|
single key to allow both gitolite access *and* shell access.
|
|
|
|
This is done by copying the pubkey (to which you want to give shell access) to
|
|
the server and running either
|
|
|
|
cd $HOME/.gitolite # assuming default $GL_ADMINDIR in ~/.gitolite.rc
|
|
src/gl-tool shell-add ~/foo.pub
|
|
|
|
or
|
|
|
|
gl-tool shell-add ~/foo.pub
|
|
|
|
The first method is to be used if you used the **user-install** mode, while
|
|
the second method is for the **system-install followed by user-setup** mode
|
|
(see doc/0-INSTALL.mkd, section on "install methods", for more on this)
|
|
|
|
**IMPORTANT UPGRADE NOTE**: previous implementations of this feature were
|
|
crap. There was no easy/elegant way to ensure that someone who had repo admin
|
|
access would not manage to get himself shell access.
|
|
|
|
Giving someone shell access requires that you should have shell access in the
|
|
first place, so the simplest way is to enable it from the server side only.
|