unfolding disasters/ posts/ Kerberos

Over the years I've watched Kerberos and related tools from afar, interested in the idea, but not interested enough to figure out the installation, configuration, etc. Well, in an attempt to secure assorted NFS mounts around my home, I decided to take the plunge today and install NFSv4 + Kerberos. Here are my notes for my Gentoo systems, mostly following the Kerberos install guide. I'll use the following settings for my examples:

  • Domain: d.net
  • Kerberos realm: R.EDU
  • Server: server.d.net
  • Client: client.d.net
  • User: jdoe (on both the client and server)

Setup the Kerberos server

Emerge the Kerberos server (app-crypt/mit-krb5) and PAM module:

# USE=-openldap emerge -av pam_krb5

-openldap breaks an OpenLDAP <-> Kerberos dependency loop.

Setup DNS to centralize service location management (krb manual):

# emacs /etc/bind/pri/d.net.zone
# /etc/init.d/named restart

I added the following entries to the $ORIGIN d.net. section of my zone file:

_kerberos TXT   "R.EDU"
kerberos  A     192.168.0.2
krb5      A     192.168.0.2
_kerberos-adm._tcp     SRV  0 0 749 krb5
_kerberos._udp         SRV  0 0 88  krb5
_kerberos-master._udp  SRV  0 0 88  krb5
_kpasswd._udp          SRV  0 0 464 krb5

Configure Kerberos and the KDC (krb manual):

# cp /etc/krb5.conf{.example,}
# emacs /etc/krb5.conf
# cat /etc/krb5.conf
[libdefaults]
        default_realm = R.EDU
        dns_fallback = yes
        kdc_ports = 88

[realms]
        R.EDU = {
                kdc = "server.d.net"  # HACK?
                admin_server = "server.d.net"  # DNS support not yet complete
        }

[domain_realm]
        .d.net = R.EDU
        d.net = R.EDU

[logging]
        kdc = FILE:/var/log/krb5/kdc.log
        admin_server = FILE:/var/log/krb5/kadmind.log
        default = FILE:/var/log/krb5/krblib.log
# cp /var/lib/krb5kdc/kdc.conf{.example,}
# emacs /var/lib/krb5kdc/kdc.conf
# cat /var/lib/krb5kdc/kdc.conf
[realms]
        R.EDU = {
                admin_server = server.d.net  # DNS support not yet complete
                database_name = /var/lib/krb5kdc/principal
                admin_keytab = FILE:/etc/krb5.keytab
                acl_file = /var/lib/krb5kdc/kadm5.acl
                key_stash_file = /var/lib/krb5kdc/.k5.R.EDU
                kdc_ports = 88
                max_life = 10h 0m 0s
                max_renewable_life = 7d 0h 0m 0s
        }

Create the database and stash file (krb manual):

# kdb5_util create -r R.EDU -s

Add administrators to the access control list (krb manual):

# emacs /var/lib/krb5kdc/kadm5.acl
# cat /var/lib/krb5kdc/kadm5.acl
jdoe/admin@R.EDU  x
# kadmin.local
kadmin.local:  add_principal jdoe/admin@R.EDU
WARNING: no policy specified for jdoe/admin@R.EDU; defaulting to no policy
Enter password for principal "jdoe/admin@R.EDU": 
Re-enter password for principal "jdoe/admin@R.EDU": 
Principal "jdoe/admin@R.EDU" created.
kadmin.local:  quit

Start the Kerberos daemons:

# /etc/init.d/mit-krb5kdc start
# /etc/init.d/mit-krb5kadmind start

Add them to your default runlevel with:

# eselect rc add /etc/init.d/mit-krb5kadmin default
# eselect rc add /etc/init.d/mit-krb5kadmind default

Add new principals (krb manual):

$ kadmin -p jdoe/admin
Authenticating as principal jdoe/admin with password.
Password for jdoe/admin@R.EDU: 
kadmin:  list_principals
...
kadmin:  add_principal jdoe
WARNING: no policy specified for jdoe@R.EDU; defaulting to no policy
Enter password for principal "jdoe@R.EDU": 
Re-enter password for principal "jdoe@R.EDU": 
Principal "jdoe@R.EDU" created.
kadmin:  quit

Now you can get your ticket granting ticket (TGT) with

$ kinit

and do all the other standard Kerberos stuff.

Setup the Kerberos client

Not much to do here, just

# emerge -av pam_krb5

and scp /etc/krb5.conf from your Kerberos server onto the client.

Check that everything works by running

$ kinit
Password for jdoe@R.EDU: 
$ klist 
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: jdoe@R.EDU

Valid starting     Expires            Service principal
06/02/11 10:32:30  06/02/11 20:32:30  krbtgt/R.EDU@R.EDU
        renew until 06/03/11 10:32:30

Setup the NFS server

Now we'll setup NFSv4 using Kerberos authentication. There don't seem to be authoritative docs, but there are a number of good tutorials (1, 2, 3, 4).

Emerge nfs-utils with the kerberos USE flag set (homepage). You may also want app-crypt/kstart (homepage) to automatically renew your server and client tickets. Now is also a good time to check your kernel config. I was missing CRYPTO_CTS, which lead to

error writing to downcall channel /proc/net/rpc/auth.rpcsec.context/channel: Invalid argument

If your realm is not your uppercased domain name, you probably also want a version of libnfsidmap >0.21 to avoid the

get_ids: failed to map name 'nfs/<fqdn>@REALM' to uid/gid: Invalid argument

bug (discussion).

Since we'll be running the NFS service, we'll need a nfs/<fqdn>@REALM principal for the service. Because we want that service to start automatically at boot, we neek to keep its key in a keytab file (krb manual).

# kadmin.local -p jdoe/admin
Authenticating as principal jdoe/admin with password.
Password for jdoe/admin@R.EDU: 
kadmin.local:  add_principal -randkey nfs/server.d.net
WARNING: no policy specified for nfs/server.d.net@R.EDU; defaulting to no policy
Principal "nfs/server.d.net@R.EDU" created.
kadmin.local:  ktadd nfs/server.d.net
Entry for principal nfs/server.d.net...
...
kadmin.local:  quit

You need use kadmin.local here (instead of kadmin) so the process has premission to create and edit the keytab file.

Read through /etc/idmapd.conf to see if you need to make any changes for your setup. I set Domain = d.net and Local-Realms = R.EDU. You probably also want to look through /etc/conf.d/nfs. I added -vvv to OPTS_RPC_GSSD, OPTS_RPC_IDMAPD, and OPTS_RPC_SVCGSSD to aid in debugging.

Setup your export filesystem. NFSv4 wants all its exports to live under a single root, so do something like:

# mkdir /export
# mkdir /export/home
# mount --bind /home /export/home

And then setup /etc/exports:

# cat /etc/exports
/export  *(rw,fsid=0,insecure,sec=krb5p,root_squash,no_subtree_check,crossmnt)
/export/a/ *(rw,insecure,sec=krb5p,root_squash,no_subtree_check)

Note that the syntax has changed somewhat, and there seem to have been a few versions of the NFSv4 syntax. exports(5) should contain good documentation for whatever version of nfs-utils you have installed on your system.

If you used mount --bind to populate /export, make sure you add appropriate entries to /etc/fstab so the mounts come up when you reboot.

# cat /etc/fstab
...
/home /export/home none rw,bind 0 0

Start the NFS server:

# /etc/init.d/nfs start

Add it to your default runlevel with:

# eselect rc add /etc/init.d/nfs default

Setup the NFS client

In order to use private (sec=krb5p) mounts, you'll need to enable RPCSEC_GSS_KRB5. Without it, you'll get error messages such as

gss_create: Pseudoflavor 390005 not found!

You'll also need nfs-utils here

# USE="kerberos" emerge -av nfs-utils

You'll need a client principal for secured mounts, so head back over to the server and run

server.d.net# kadmin.local
kadmin.local:  add_principal -randkey nfs/client.d.net
kadmin.local:  ktadd -k /tmp/krb5.keytab nfs/client.d.net
Entry for principal nfs/client.d.net ...
...
kadmin.local:  quit

Then scp the new keyfile over to /etc/krb5.keytab on the client and remove the temporary version from the host. You can list the keys in a keytab with klist -e -k /path/to/keytab if you find a keytab lying around but forget what's inside it.

On the client, you'll need gssd and idmapd running (both part of nfs-utils).

# /etc/init.d/rpc.gssd start
# /etc/init.d/rpc.idmapd start

There's no need to add these to your default runlevel, since they should be started automatically if you have NFSv4 entries in your /etc/fstab (I have no idea how that works).

Now test your mount:

$ sudo mkdir /tmp/mnt
$ sudo mount -v -t nfs4 -o sec=krb5p server:/ /tmp/mnt
mount.nfs4: timeout set for Thu Jun  2 10:44:46 2011
mount.nfs4: trying text-based options '...'
server:/ on /tmp/mnt type nfs4 (rw,sec=krb5p)
$ ls /tmp/mnt 
ls: cannot access /tmp/mnt: Permission denied
$ klist 
klist: No credentials cache found (ticket cache FILE:/tmp/krb5cc_1000)
$ kinit 
Password for jdoe@R.EDU: 
$ ls /tmp/mnt/
home

Note that if you kestroy your key, you can still access the files:

$ kdestroy 
$ klist 
klist: No credentials cache found (ticket cache FILE:/tmp/krb5cc_1000)
$ ls /tmp/mnt/
home

This is because your credentials have been cached in the client's kernel. On AIX there seems to be an nfsauthreset command to manually flush cached GSSAPI information. Linux support is waiting on a new key ring implementation.

Other stuff

If you hadn't had the kerberos USE flag set before, you should consider adding it to your /etc/make.conf and running

$ sudo emerge -av --deep --newuse --update @world

to get Kerberized versions of any packages you have installed (e.g. cups, curl, cvs, emacs, openssh, most SASL libraries, ...).

For details on using Kerberos with SSH, check out the excellent description in the SSH definative guide. The key elements are host/<fqdn>@REALM principals for each host (with keyfiles on each server) and appropriate enabling of the GSSAPI* options in sshd_config and ssh_config.

There's also suite of Kerberos-aware utilities in app-crypt/mit-krb5-appl (krcp, krlogin, krsh, ktelnet, and kftp). I don't use the non-Kerberized versions, so I haven't tried any of these.

If you're using MPD on an NFS-mounted music repository, you might be interested in my kinit-mpd.sh script for granting the mpd user access to the NFS-mounted music as the nobody principal.

For debugging, check out the KRB5_TRACE environment variable. I sent some patches upstream to integrate reverse DNS debugging into the KRB5_TRACE framework. The patches will go live with the next major krb5 release after the 1.10 series.

If you end up compiling from source, you can run the unit tests and check coverage with something like:

$ git clone git://github.com/krb5/krb5.git
$ cd krb5/src
$ util/reconf
$ mkdir ../build
$ cd ../build
$ ../src/configure --disable-rpath CFLAGS="-fprofile-arcs -ftest-coverage -O0" LIBS=-lgcov
$ make
$ make check
$ cd lib/krb5/os
$ gcov -o sn2princ.so.gcno ../../../lib/krb5/os/sn2princ.c
$ gcov -o sn2princ.so.gcno sn2princ.c
$ less sn2princ.c.gcov

Running configure from a separate directory creates a VPATH build, which avoids polluting the source directory with generated files.