Greg's arla page

Greg's arla page

This page is a notebook about my experiences with arla. I don't know if anybody else is interested, but here it is anyway.
This may look like a diary, but in fact I'll go back and change things as I see fit. In particular, I'm bound to make inaccurate statements, and I'll remove them without trace, or add new stuff without mention, things I don't do in my diary.

Wednesday, 17 November 2004

Finally got down to getting arla working. My initial attempts are on NetBSD because the FreeBSD port is broken (yet another casualty of the rebuild of the BIO layer). I plan to fix that, but it'll be easier if I have something functional to compare it to.

Going through old email, I discovered the directory /usr/arla/etc/. As installed, it contains a number of files:

CellServDB is a list of all AFS cells.
ThisCell contains a name for the cell of which we're a part. As installed, it contains the name stacken.kth.se.

According to my information, I should have been able to start arla with /usr/arla/sbin/startarla. That produced:

=== root@chili (/dev/ttyp0) /wantadilla/home/grog 3 -> /usr/arla/sbin/startarla
modload: /dev/ksyms: No such file or directory
modload: error initializing module: File exists

startarla proves to be a shell script. Looking through /var/log/messages, I saw:

Nov 17 12:30:29 chili /netbsd: nnpfs (arla 0.36.2): protocol version 18, using cdev: 168, syscall: 210

That certainly gives the impression that things are OK, but the output of modstat looks strange:

=== root@chili (/dev/ttyp0) /usr/arla/sbin 11 -> modstat
Type    Id   Offset Loadaddr Size Info     Rev Module Name
DEV       0  -1/168 cbe11000 0034 cbe1d260   2 nnpfs_mod

A bit of reading found the ksyms(4) driver, related to the pseudo-device ksyms. That's in my kernel (and in GENERIC), so it looks like the device node is not present. Unfortunately, MAKEDEV doesn't help. Looking on another NetBSD box gives me:

-bash-2.05b$ ls -l /dev/ksyms
cr--r--r--  1 root  wheel  101, 0 Apr 16  2004 /dev/ksyms

=== root@chili (/dev/ttyp0) /dev 19 -> ls -l ksyms 
crw-r--r--  1 root  wheel  101, 0 Nov 17 13:53 ksyms

After that, things progressed:

=== root@chili (/dev/ttyp0) /usr/arla/sbin 21 -> ./startarla 
modload: error initializing module: File exists

And then things hung for a while, long enough for me to fire up a tcpdump on my uplink gateway and find:

13:56:35.672024 chili.lemis.com.afs3-callback > fishburger.stacken.kth.se.afs3-vlserver:  rx data vldb call get-entry-by-name-n "root.afs [|vldb] (44)
13:56:35.971974 chili.lemis.com.afs3-callback > milko.stacken.kth.se.afs3-vlserver:  rx ack (50)
13:56:36.211922 chili.lemis.com.afs3-callback > milko.stacken.kth.se.afs3-vlserver:  rx data vldb call probe (32)
13:56:39.431379 chili.lemis.com.afs3-callback > fishburger.stacken.kth.se.afs3-vlserver:  rx ack (50)
13:56:39.451348 chili.lemis.com.afs3-callback > fishburger.stacken.kth.se.afs3-vlserver:  rx data vldb call get-entry-by-name-n "root.afs [|vldb] (44)
13:56:39.991260 chili.lemis.com.afs3-callback > milko.stacken.kth.se.afs3-vlserver:  rx data vldb call probe (32)
13:56:43.450815 chili.lemis.com.afs3-callback > milko.stacken.kth.se.afs3-vlserver:  rx data vldb call get-entry-by-name-n "root.afs [|vldb] (44)

Finally it came back with:

mkdir: /afs: File exists
mount_nnpfs: mount: Operation not supported
=== root@chili (/dev/ttyp0) /usr/arla/sbin 22 -> df
(long delay)
Filesystem        1048576-blocks     Used     Avail Capacity  Mounted on
/dev/wd0a                   1013       32       930     3%    /
/dev/wd0e                   4816     4004       570    87%    /usr
echunga:/                   5814     4627       722    86%    /echunga
echunga:/home               6329     3237      2585    55%    /echunga/home
zaphod:/src               150214    75775     62421    54%    /src
echunga:/dump              76285    73901      1621    97%    /dump
wantadilla:/                9912     3171      5948    34%    /wantadilla
wantadilla:/home           51895    38023      9720    79%    /wantadilla/home
wantadilla:/dumpa          76285    69388       794    98%    /dumpa
arla                           0        0         0     0%    /afs
=== root@chili (/dev/ttyp0) /usr/arla/sbin 23 -> ls -l /afs
ls: /afs: Operation not supported

More for fun than anything else, started tcpdump on sat-gw (a Linux box; I continually forget how to install packages on Linux), and saw:

=== root@sat-gw (/dev/pts/3) /home/grog 2 -> tcpdump host chili
User level filter, protocol ALL, datagram packet socket
tcpdump: listening on all devices
14:26:13.825593 sm200d < fishburger.stacken.kth.se.7003 > chili.lemis.com.7001: udp 65 (DF)
14:26:14.246839 sm200d < fishburger.stacken.kth.se.7003 > chili.lemis.com.7001: udp 28 (DF)
14:26:14.285801 sm200d < hot.stacken.kth.se.7003 > chili.lemis.com.7001: udp 28 (DF)
14:26:17.178692 sm200d < fishburger.stacken.kth.se.7003 > chili.lemis.com.7001: udp 65 (DF)

In other words, the packets are coming in (on interface sm200d), but they're not going out on interface eth0: they've been firewalled. After checking in /etc/services, added some rules to my iptables configuration:

# Allow AFS
iptables -A FORWARD -j ACCEPT -p udp --sport 7000:7011
iptables -A FORWARD -j ACCEPT -p udp --dport 7000:7011

After that things looked better:

=== grog@chili (/dev/ttyp3) /wantadilla/home/grog 1 -> ls /afs
alfvenlab.kth.se        cs.wisc.edu             ir.stanford.edu         nxs.se                  su.se
andrew.cmu.edu          dementia.org            isk.kth.se              openafs.org             telge.kth.se
arla.e.kth.se           desy.de                 it.kth.se               pdc.kth.se              tu-chemnitz.de
assv.net                dev.mit.edu             kth.se                  pi.infn.it              umich.edu
asu.edu                 e.kth.se                lsa.umich.edu           pitt.edu                umr.edu
athena.mit.edu          ece.cmu.edu             mech.kth.se             rpi.edu                 wam.umd.edu
cern.ch                 glue.umd.edu            msc.cornell.edu         sanchin.se
citi.umich.edu          iastate.edu             nada.kth.se             sipb.mit.edu
club.cc.cmu.edu         ies.auc.dk              ncsa.uiuc.edu           slac.stanford.edu
cmf.nrl.navy.mil        ipp-garching.mpg.de     northstar.dartmouth.edu stacken.kth.se

And on the firewall machine, I saw:

14:45:28.893951 sm200d < fishburger.stacken.kth.se.7003 > chili.lemis.com.7001: udp 28 (DF)
14:45:28.894070 eth0 > fishburger.stacken.kth.se.7003 > chili.lemis.com.7001: udp 28 (DF)
14:45:32.382124 sm200d < bzq-76-181.red.bezeqint.net.4652 > chili.lemis.com.1433: S 2474251459:2474251459(0) win 64240 <mss 1460,nop,nop,sackOK> (DF)
14:45:43.582089 sm200d < milko.stacken.kth.se.7003 > chili.lemis.com.7001: udp 504 (DF)
14:45:43.582164 eth0 > milko.stacken.kth.se.7003 > chili.lemis.com.7001: udp 504 (DF)
14:45:44.553362 sm200d < kvikklunsj.stacken.kth.se.7000 > chili.lemis.com.7001: udp 66

After that, things went slowly but surely:

=== grog@chili (/dev/ttyp3) /afs 4 -> ls -l
ls: .su.se: No such file or directory
ls: arla.e.kth.se: No such file or directory
ls: assv.net: No such file or directory
ls: citi.umich.edu: Connection timed out
ls: dementia.org: No such file or directory
ls: glue.umd.edu: Connection timed out
ls: kth.se: No such file or directory
ls: lsa.umich.edu: Connection timed out
ls: nxs.se: No such file or directory
ls: rpi.edu: Connection timed out
ls: sanchin.se: No such file or directory
ls: su.se: No such file or directory
total 1
drwxrwxrwx   6 root  wheel  2048 Nov  6  2003 alfvenlab.kth.se
drwxrwxrwx   2 root  wheel  6144 May 13  2004 andrew.cmu.edu
drwxrwxrwx   5 root  wheel  4096 Sep 16  2003 asu.edu
drwxr-xr-x   3 root  wheel  2048 Jun 27  1995 athena.mit.edu
drwxr-xr-x   3 root  wheel  2048 Jun 15 00:46 cern.ch
drwxrwxrwx   2 root  wheel  2048 Oct 21 00:31 club.cc.cmu.edu
drwxr-xr-x   4 root  wheel  4096 Aug  7 01:10 cmf.nrl.navy.mil
drwxr-xr-x   3 root  wheel  2048 Sep 23 23:44 cs.wisc.edu
drwxr-xr-x   7 root  wheel  2048 Sep  9 21:41 desy.de
drwxr-xr-x   2 root  wheel  2048 Apr 11  1997 dev.mit.edu
drwxrwxrwx   2 root  wheel  2048 Jan 20  2003 e.kth.se
drwxr-xr-x  12 root  wheel  2048 Oct 31  2003 ece.cmu.edu
drwxrwxrwx   4 root  wheel  2048 Aug 26 02:00 iastate.edu
drwxrwxrwx   2 root  wheel  2048 Feb 20  2004 ies.auc.dk
drwxr-xr-x  13 root  wheel  6144 Oct 19 23:28 ipp-garching.mpg.de
drwxr-xr-x   3 root  wheel  2048 Nov 11  2003 ir.stanford.edu
drwxr-xr-x   2 root  wheel  2048 Jun 13 16:16 isk.kth.se
drwxrwxrwx   3 root  wheel  2048 Dec 13  2003 it.kth.se
drwxr-xr-x   2 root  wheel  2048 Oct 15 16:24 mech.kth.se
drwxrwxrwx   2 root  300    2048 Sep 23 06:20 msc.cornell.edu
drwxr-xr-x   3 root  wheel  4096 Oct  2  2003 nada.kth.se
drwxrwxrwx   2 root  wheel  4096 Aug  5  2003 ncsa.uiuc.edu
drwxr-xr-x  20 root  wheel  4096 Oct  9 03:11 northstar.dartmouth.edu
drwxrwxrwx   2 root  wheel  2048 Jun 21  2001 openafs.org
drwxr-xr-x   5 root  wheel  2048 Sep 19  2003 pdc.kth.se
drwxrwxrwx   3 root  wheel  2048 Oct 18 21:51 pi.infn.it
drwxrwxrwx   3 root  wheel  6144 May  1  2004 pitt.edu
drwxrwxrwx   2 root  wheel  2048 Oct 13  1994 sipb.mit.edu
drwxrwxrwx   2 root  wheel  2048 Oct 14 03:52 slac.stanford.edu
drwxrwxrwx   4 root  wheel  4096 Jul 17 23:38 stacken.kth.se
drwxrwxrwx   3 root  wheel  2048 Mar 31  2001 telge.kth.se
dr-xr-xr-x   9 root  wheel  2048 Oct 21 02:09 tu-chemnitz.de
drwxr-xr-x   3 root  wheel  2048 Mar 14  1997 umich.edu
drwxr-xr-x   2 root  wheel  4096 Dec  6  2003 umr.edu
drwxr-xr-x   2 root  wheel  4096 Nov 24  2002 wam.umd.edu
=== grog@chili (/dev/ttyp3) /afs 5 ->

=== root@chili (/dev/ttyp0) /afs 30 -> ls -l andrew.cmu.edu
ls: sun4_413: No such file or directory
ls: sun4c_411: No such file or directory
ls: sun4m_413: No such file or directory
total 3
drwxrwxrwx   2 root   wheel     6144 May 13  2004 .recursivefindtrap
drwxrwxrwx   5 root   wheel     2048 Aug 27 01:50 acs
drwxrwxrwx   2 root   wheel     2048 Aug 27 03:03 adm
drwxrwxrwx   2 root   wheel     2048 Jul 12  2001 as
lrwxr-xr-x   1 16690  wheel        8 Feb 23  1993 asa -> acs/ducs
drwxrwxrwx   2 root   wheel     2048 Aug 27 03:30 bap
drwxrwxrwx   2 root   wheel     2048 Sep 25  2002 cas
drwxrwxrwx   4 root   wheel     2048 Aug 17  1988 cdec
-rw-r--r--   1 27773  10          22 Oct 17  1995 cds.afs.proxy
drwxrwxrwx   8 root   wheel     2048 Oct  2  1991 ce
drwxrwxrwx   8 root   wheel     2048 May 29  1991 cece

Greg's home page

Greg's diary

Greg's photos

$Id: arla.html,v 1.3 2008/02/10 23:50:42 grog Exp $