Initially I tried get a root shell using:

sudo -s

sadly after repeated password prompts I had no luck. Initially I thought I was just getting my password wrong – something I occasionally do with the amount of passwords I have to remember. However, after repeated attempts I came to the conclusion that something else was wrong.

I looked at the logs to see if they would show anything useful:



Apr 14 11:49:28 leo sudo:   benjsc : 1 incorrect password attempt ; TTY=pts/3 ; PWD=/owners/benjsc/home ; USER=root ; COMMAND=/bin/tcsh
Apr 14 11:51:09 leo sudo:   benjsc : 1 incorrect password attempt ; TTY=pts/4 ; PWD=/owners/benjsc/home ; USER=root ; COMMAND=/bin/tcsh
Apr 14 11:58:03 leo sudo:   benjsc : 1 incorrect password attempt ; TTY=pts/4 ; PWD=/owners/benjsc/home ; USER=root ; COMMAND=/bin/tcsh
Apr 14 11:58:16 leo sudo:   benjsc : 1 incorrect password attempt ; TTY=pts/4 ; PWD=/owners/benjsc/home ; USER=root ; COMMAND=/bin/tcsh

Once again nothing obvious showed up. At this point I started Googling to try and find the answer. There was a lot of articles about people trying the root password rather than the user password, but I wasn’t doing that. There was articles about people stuffing up the syntax of the sudoers file (you should use visudo to edit this). However, nothing had changed but the version of sudo. Eventually I found a post that solved the issue.

It turns out that the latest versions of sudo break pam support. Whilst the article refers to OPIE (One Time Keys), I’ve since confirmed that this issue affects machines without OPIE setup. Hence for anyone who tries to upgrade sudo to a later version and has problems with their password not being accepted, and the password prompt just continuing to reappear. Then you’ll need to patch sudo to work.

How, copy and paste the following on the console:

cd  /usr/ports/security/sudo
make extract
cd work/sudo*
patch << END
--- auth/pam.c.orig     2010-02-04 10:43:28.635212518 -0600
+++ auth/pam.c  2010-02-04 10:43:34.194558424 -0600
@@ -107,13 +107,6 @@
     }

     /*
-     * Set PAM_RUSER to the invoking user (the "from" user).
-     * We set PAM_RHOST to avoid a bug in Solaris 7 and below.
-     */
-    (void) pam_set_item(pamh, PAM_RUSER, user_name);
-    (void) pam_set_item(pamh, PAM_RHOST, user_host);
-
-    /*
      * Some versions of pam_lastlog have a bug that
      * will cause a crash if PAM_TTY is not set so if
      * there is no tty, set PAM_TTY to the empty string.

END
cd ../..
make build
make deinstall
make install

Initially developed under Linux the program was then shifted to it’s permanent location on a FreeBSD server. This is where we first started having problems. Initially we discovered the server didn’t have a native serial port. We fixed this using a USB to serial converter/dongle (FTDI Chipset). This was fine as FreeBSD has the ufdti kernel module. Upon loading the module new devices appears in /dev

crw-rw----  1 uucp  dialer    0, 157 Oct  6 08:39 /dev/cuaU0
crw-rw----  1 uucp  dialer    0, 158 Oct  6 08:39 /dev/cuaU0.init
crw-rw----  1 uucp  dialer    0, 159 Oct  6 08:39 /dev/cuaU0.lock
crw-rw-rw-  1 root  wheel     0, 154 Jan  8 10:50 /dev/ttyU0
crw-------  1 root  wheel     0, 155 Oct  6 08:39 /dev/ttyU0.init
crw-------  1 root  wheel     0, 156 Oct  6 08:39 /dev/ttyU0.lock

We attempted to connect to our device using screen (screen /dev/ttyU0 115200) and everything worked as expected. We could send AT commands to the device all ok.
We then stopped screen and ran our php program. It ended up hanging on a fgets call to the serial port. This is really strange we though.
Next we queried the port to find out what baud rate it was set at:

>stty -f /dev/ttyu0
speed 9600 baud;
lflags: echoe echoke echoctl
oflags: tab0
cflags: cs8 -parenb

Strange we thought as we’d just connected with screen at 115200. Under linux we use screen to set the baud rate, all other programs accessing the port use the port at 115200. So what had set it back to 9600 baud?
We tried to use stty to set the speed:

>stty -f /dev/ttyU0 speed 115200

>stty -f /dev/ttyu0
speed 9600 baud;
lflags: echoe echoke echoctl
oflags: tab0
cflags: cs8 -parenb

What on earth was happening? We set the speed to 115200 but directly quering the port again indicated it was still at 9600 baud? At this point we were perplexed.
Eventually we found the solution. The newer FreeBSD terminal drivers provide the *.init devices, in this case /dev/ttyU0.init . These devices indicate the terminal settings to be applied to the terminal when the device is closed. Whilst Linux leaves the device in the same state the last program put the port into, FreeBSD restores the terminals state to what ever is specified in the init file. So a quick command:

> stty -f /dev/ttyU0.init -icanon -isig -echo echoe echok echoke echoctl -icrnl -ixany -imaxbel ignpar -opost -onlcr -oxtabs cs8 -parenb -hupcl clocal

And then to check:

> stty -f /dev/ttyU0
speed 115200 baud;
lflags: -icanon -isig -echo echoe echok echoke echoctl
iflags: -icrnl -ixany -imaxbel ignpar
oflags: -opost -onlcr -oxtabs
cflags: cs8 -parenb -hupcl clocal

Excellent. The terminal was now configured exactly how we wanted. We ran the program and it worked like a charm!

For quite a while now I’ve been using Apple’s ICal server on FreeBSD and Sunbird/Lightning as a front end to the calendar. However, one thing that has always annoyed me was the lack of a web frontend to my calendars.

Well today after searching on a completely different topic I found a javascript front end to caldav. After only a few minutes setup I had the frontend up and running. Sadly however, It didn’t play happy with ICalServer. A little debugging of the caldav REPORT request and I had it querying correctly.

Now there was valid data populating the table… or was it. Looking a little closer there were bugs with the reoccurring events and also with the lastMonday function. A quick fix (after 20mins finding them) and now the calendar works great!

Best thing about it is I can now have a public caldav calendar which everyone can view whilst being able to update thing directly on my own writable version of the caldav calendar.

Note, this does not cover using the gpt based partition tables. If you want to use these, please refer to the following page: http://lulf.geeknest.org/blog/freebsd/Setting_up_a_zfs-only_system/ or booting zfs as root using a small ufs boot partition as provided by the instructions at: http://wiki.freebsd.org/ZFSOnRoot.

Below is the steps required to be able to setup the root zpool

1. Download a FreeBSD -current fixit cdrom snapshot later than 200901, as these have loader ZFS support
2. Burn the CD
3. Boot the CD
4. Setup any partitions you want – note you must setup the ‘a’ partition to cover the entire device as the loader will use this.
5. Select Fixit from the menu, and use the CDrom as a source
6. Create the pool and install the loader (see below)
7. Copy the required files to boot (see below)

Creating the root zpool and installing the Loader

The fixit cd has everything required to create a zpool, however by default none of the required modules are loaded. Hence they need to be loaded first:

cd /mnt2/boot/kernel
kldload ./opensolaris.ko

kldload ./zfs.ko

Once the modules have been loaded all the zfs tools (zpool,zfs,zdb) should now work. Let assume you want to install FreeBSD to /dev/ad4s2 (second partition on a sata disk). You can do this using:

    zpool create somename /dev/ad4s2

Where somename is the name of the pool you want to create. This creates a single zfs filesystem and a zfs pool of storage. To install the boot loader you need to do:

    # dd if=/mnt2/boot/zfsboot of=/dev/da0s1 count=1
    # dd if=/mnt2/boot/zfsboot of=/dev/da0s1 skip=1 seek=1024

The first line installs boot1, the second line installs boot2.  However, boot2 is responsible for loading boot3 (aka the loader – found in /boot/loader). Hence that must be put in place.

Copying the required files to boot

The easiest way to get things to the point where things are ready to boot is to copy all the files from /dist  - the live distribution. Before you do this, you might like to take advantage of zfs and create some subfilesystems so you can snapshot, monitor space, etc.

For instance creating a /usr and /var filesystem is often very handy:

    #zfs create somename/usr
    #zfs create somename/var

Now you can copy the base system:

    cp -a /dist/* /somename

This will install among other things:

• /somename/boot/kernel/kernel  - FreeBSD kernel
• /somename/boot/kernel/opensolaris.ko – zfs dependency
• /somename/boot/kernel/zfs.ko – module understanding zpools/zfs
• /somename/boot/loader  - the FreeBSD loader
  

At this point you need to replace the loader with one that understands zfs. You can download the loader from: (To be advise – see cavet below)
And if you have a usb stick copy it in place using:

   mount_msdos /dev/da0s1 /mnt
   cp /mnt/loader /somename/boot

Finally you have to tell FreeBSD where to mount filesystems on a standard boot:

   zfs set mountpoint=/var somename/var
   zfs set mountpoint=/usr somename/usr

At this point any command you type will now probably fail indicating it’s missing some shared library. This is because /usr has now changed. You can get around this by telling the loader where to find valid libraries:

   export LD_LIBRARY_PATH=/mnt2/lib

Next we need to build the zfs cache. This is used by zfs mount to automatically mount zfs filesystems by /etc/rc.d/zfs at boot time. It’s also used to determine if a filesystem is local to the system or belongs to an exported pool.

   mkdir /boot/zfs
   mkdir /somename/boot/zfs
   cd /
   zfs export somename
   zfs import -f somename
   cp /boot/zfs/zfs.cache /somename/boot/zfs/

Finally we tell the loader where we want to boot from and set the init scripts to automatically start all zfs filesystems:

    echo 'zfs_enable="YES"' > /somename/etc/rc.conf
    echo 'zfs_load="YES"' > /somename/boot/loader.conf
    echo 'vfs.root.mountfrom="zfs:somename"' >> /somename/boot/loader.conf

And set the root filesystem to a legacy mountpoint (so zfs mount -a won’t try and mount an already mounted filesystem)

     zfs set mountpoint=legacy somename

At this point you can reboot and things should now boot!
Update: 20090809

There has been a lot of updates to instructions along the way. There are now official ZFS on Root instructions available using GPT/MBR/other available at: http://wiki.freebsd.org/RootOnZFS

Hardware

• CPU: Core2Duo  E8200  @ 2.66GHz
• HDD:
  ad0: 114472MB <WDC WD1200JB-00FUA0 15.05R15> at ata0-master UDMA33
  ad4: 305245MB <WDC WD3200AAKS-00B3A0 01.03A01> at ata2-master SATA150
  ad6: 190782MB <Seagate ST3200822AS 3.01> at ata3-master SATA150
  ad7: 190782MB <Seagate ST3200822AS 3.01> at ata3-slave SATA150
• Video Card:
  NVideo GeForce 7300 LE
• Audio:
  Intel 82801G (ICH7 Family) High Definition Audio
• Capture Cards:
• • CX2388x TV Capture Chip (DVB-T)
  • Conexant (Was: Brooktree Corp)
    ’7610144D&REV_02\4&1F7DBC9F&0&09F0 TV Video Capture

With the hardware set and not likely to change (I refuse to buy hardware due to an O/S not working with what I’ve got), it was time to try and get things working. Whilst there is a mythtv port, it’s the old stable release of 0.18. There has been a lot of changes since then. Including the new libmythui library where groovy graphics features are available. I must admit I’m not coming in to this blind. I’ve been running mythtv under Linux for quite some time.  Hence I’m quite aware what needs to be done under Linux to get a working Mythtv setup. Hence I’ll break this article up into a number of different steps as below. Each one being a different blog entry.. some of these steps are going to take a while to get working!

1. Setting up required ports
2. Setting up capture cards
3. Setting up audio
4. Setting up Xorg
5. Installing Mythtv
6. Configuring Mythtv
7. Tweaking Mythtv

For anyone who wishes to follow this drama of getting things working below I provide some links I’ve found which useful in determining what might be possible. I also provide a little justification as to why I want Mythtv working under FreeBSD.

Why the Switch?

After having mythtv successfully running under various versions of ubuntu, I finally reached a point where Linux annoyed me enough to try and get rid of it in favour of FreeBSD. It’s not that Linux didn’t work, it’s just the amount of stuffing around I had to do to get things working was CRAZY! Sure installing mythtv was relatively easy but it’s all the little things that FreeBSD does so well that Linux doesn’t that made me want to change. Thinks like power management, cpu throttling, wireless that actually works!

These things are just easier under FreeBSD, there’s no config, overiding some config, linked to some default, using some crazy symlink farm. There’s not ‘volatile’ kernel modules which must go through crazy loading scripts because they are not GPL compliant. FreeBSD also is much easier to upgrade in place and has the ZFS filesystem. This is the main reason I wanted to switch. I use the same machine as a backup server with 2 disks in raid 1. The ability to daily snapshot at the filesystem level is just soooo nice!

FreeBSD Mythtv Links

Below are some links that I found useful when getting mythtv working under FreeBSD.

• http://mythtv.son.org/tiki-index.php
• http://wiki.freebsd.org/MythTV
• http://www.lemis.com/grog/HOWTO/mythtv-on-FreeBSD-setup.html

===>  Configuring for qt4-rcc-4.4.3
/bin/cp /data/usr/ports/devel/qt4-rcc/../../devel/qt4/files/configure /data/usr/ports/devel/qt4-rcc/work/qt-x11-opensource-src-4.4.3/src/tools/rcc/../../../
/usr/bin/sed -i.bak -e 's|target.path.*|target.path=/usr/local/bin|g'  /data/usr/ports/devel/qt4-rcc/work/qt-x11-opensource-src-4.4.3/src/tools/rcc/rcc.pro
/bin/mkdir -p /data/usr/ports/devel/qt4-rcc/work/qt-x11-opensource-src-4.4.3/src/tools/rcc/../../../mkspecs
/bin/ln -sf /usr/local/bin/qmake-qt4 /data/usr/ports/devel/qt4-rcc/work/qt-x11-opensource-src-4.4.3/src/tools/rcc/../../../bin/qmake

This is the Qt/X11 Open Source Edition.

   The specified system/compiler is not supported:

      /data/usr/ports/devel/qt4-rcc/work/qt-x11-opensource-src-4.4.3/mkspecs/freebsd-g++

   Please see the README file for a complete list.

===>  Script "configure" failed unexpectedly.
Please report the problem to kde@FreeBSD.org [maintainer] and attach the
"/data/usr/ports/devel/qt4-rcc/work/qt-x11-opensource-src-4.4.3/src/tools/rcc/../../..//config.log"
including the output of the failure of your make command. Also, it might be
a good idea to provide an overview of all packages installed on your system
(e.g. an `ls /var/db/pkg`).
*** Error code 1

After some brief Google searching, I found the issue. It turns out that a long time ago I had been using qt4 with another project I had been working on. With this project I’d defined:

setenv QMAKESPEC freebsd-g++
setenv QTDIR /usr/X11R6/

This was causing the build system to break. The post at:  http://mail.kde.org/pipermail/kde-freebsd/2008-August/003360.html gave the hint about this.

Once I undefined QMAKESPEC everything worked as expected!

This document describes how to setup both the server side and client side for a PPTP connection with MPPE encryption that works for windows, MacOsX and other freebsd boxes.

The basic process:

 - Install & configure pptpserver on the freebsd server
 - Configure ppp on the freebsd server
 - Setup the clients

1. Installing pptpserver

This part is handled really easily as it pptp server exists in the ports collections. Hence all you need to do is:

 cd /usr/port/net/poptop
 make install

To configure pptp modify /usr/local/etc/pptp.conf\\ Put the following lines in the file:

 localip 192.168.2.1
 remoteip 192.168.2.56-75

 # Listen on the outside interface only
 listen 130.220.37.202

Configure ppp on the freebsd server

Edit /etc/ppp/ppp.conf and set the following target only:

  pptp:
    set ifaddr 192.168.1.1 192.168.1.56-192.168.1.74 255.255.255.255
   set dns 192.168.0.1
    set nbns 192.168.0.1
   disable pap
   disable utmp
   disable passwdauth
   #enable chap     # MPPE Requirest chap81/MSChapV2
   enable MSChapV2
   enable mppe      # Enable Encrptions
   set log Phase Chat LCP IPCP CCP tun command  # Debugging
   set timeout 0   # Don't drop the connection

   #
   # Force 128 bit encryption with a key change every packet
   # MacOSX only works with stateless connections and the are more
   # secure anyway - just less efficient.
   set mppe 128 stateless

   # Disable compression - freebsd clients try to use it but it breaks mppe
   disable deflate pred1
   deny deflate pred1
   set server /var/run/pptp_ppp_%d "" 0700
   accept dns              # Enable clients to request dns details
   disable ipv6cp          # Disable ipv6
   enable proxy            # Enable proxying addresses on the local net for clients

Now modify/create /etc/ppp/ppp.secret and put in it:

 someuser  userpassword  192.168.1.75

Now ‘someuser’ can log in with the password ‘userpassword’ and will get an IP address of 192.168.1.75. If you don’t want to specify the ip, just leave the 3rd parameter off that line of the file.

Setup Clients

This section details how to setup various clients

Windows XP

 - Create a new VPN connection
 - Specify host
 - Specify Usename  / password
 - Hit connect

FreeBSD

Freebsd works with MPPE out the box. Simply setup the following in /etc/ppp.conf

 MYVPN:
   set authname someuser
   set authkey  userpassword
   disable pred1
   enable proxy
   disable ipv6cp
   set timeout 0
   add default HISADDR

Install pptpclient

   cd /usr/ports/net/pptpclient
   make install

Now run it with: pptp serverip MYVPN\\ ie:

  pptp  130.220.37.2 MYVPN

Mac OsX

Simply configure the GUI tool.

The first step in setting up TLS/Auth support is to install the required additional libraries. For SSL support (required by TLS) the following ports must be installed:

• security/cyrus-sasl
• security/cyrus-sasl-saslauthd//

These must be installed before sendmail is recompiled.

Sendmail in FreeBSD by default is not compiled with TLS/Auth support. In order to allow it to work with these features it must be recompiled. This is actually quite simple as long as the FreeBSD machine has the FreeBSD source collection (aka /usr/src). First you need to edit ///etc/make.conf// (//cp /etc/defaults/make.conf// if it doesn’t already exist. Edit the file so the following lines exist/are uncommented.

  # with SASLv2:
  SENDMAIL_CFLAGS=-I/usr/local/include -DSASL=2
  SENDMAIL_LDFLAGS=-L/usr/local/lib
  SENDMAIL_LDADD=-lsasl2

Next you need to recompile sendmail. Due to the way sendmail exists in the ports collections you must compile some of the libraries first. The following lines show the procedure for recompiling the libraries and sendmail.

1. Compile the smutil library

  cd /usr/src/lib/libsmutil
  make clean
  make obj
  make

1. Compile the smlibrary

  cd /usr/src/lib/libsm
  make clean
  make obj
  make

1. Compile Sendmail

  cd /usr/src/usr.sbin/sendmail
  make clean
  make obj
  make
  make install

At this point sendmail with TLS/Auth support is installed. Now sendmail must be configured to work with these features. Setting Up TLS Support

TLS stands for Transport Layer Security. It’s a bit like SSL (Socket Layer Security) in that it provides encryption between two points. The difference is that TLS provides it only in the data, SSL provides encryption of the headers as well.

Using TLS is a good idea. It provides encyrption for authentication purposes and also Trusted mail headers. Ie A Mail server can create a TLS connection between itself and another TLS server and this is reported in the mail headers and the mail headers are thus deemed ‘accurate’.

To setup TLS support you need to first generate a public/private key pair for use with the mail server. This is outside the scope of this document. Then you need to setup sendmail with the following options:

  define(`CERT_DIR', `MAIL_SETTINGS_DIR`'certs')
  define(`confCACERT_PATH', `CERT_DIR')
  define(`confCACERT', `CERT_DIR/CAcert.pem')
  define(`confSERVER_CERT', `CERT_DIR/MYcert.pem')
  define(`confSERVER_KEY', `CERT_DIR/MYkey.pem')
  define(`confCLIENT_CERT', `CERT_DIR/cert.pem')
  define(`confCLIENT_KEY', `CERT_DIR/MYkey.pem')

You must set both the client and the server key so that incomming mail can be encrypted and so that outgoing mail can be encrypted.

Once you have set this up, restart sendmail and test it (See below) Setting Up Auth Support

In order to use authentication support you must first add a few options to the sendmail configuration file. Authentication is used to allow relaying from domains that are not listed as relay domains provided authentication exists. Ie: as long as the authentication is successful, then the mail server is affectivly an open relay on that connection. This is great for roaming laptop users who want to send mail but are often in a different domain or on dialup.

To set this up add the following to your //sendmail.mc// file:

  define(`confAUTH_MECHANISMS',`GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN')
  TRUST_AUTH_MECH(`GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN');

This tells sendmail to support the listed authentication methods. There is also a ‘PLAIN’ option but this should be avoided as the password is sent in plain text. Even of TLS/SSL this is not a good idea.

With that option in place you now have to tell sendmail what it is allowed to relay based on authentication. To allow relays to anywhere provided authentication works add the following to the access file.

  From: * OK
  To: * RELAY

Make sure that you rebuild the access database (run make) then restart sendmail and authentication should be enabled so test it! Extra Sendmail Configuration Options

It is possible and highly recommended that sendmail be setup to force TLS for authentication purposes. Otherwise sendmail will quite happily accept plain text passwords across an unencrypted data stream. This is just a big security problem. Hence adding the following line to sendmail forces the use to use TLS/SSL before authentication is possible:

  define(`confAUTH_OPTIONS',`p,y')

Testing it all

After everything has been setup, it is extreemly important to test that the results are as expected. There are a number of tests you will want to run. These include the following: Test Test Implementation TLS Test

The following shows a telnet to the mail server, type the text in bold and look for the result in italics/bold

  $ telnet localhost 25
  220 mail.example.net ESMTP Sendmail 8.11.1/8.11.1; Sat, 19 May 2001 08:04:04 -0400
  EHLO localhost
  250-mail.example.net Hello IDENT:jose@[127.0.0.1], pleased to meet you
  250-ENHANCEDSTATUSCODES
  250-EXPN
  250-VERB
  250-8BITMIME
  250-SIZE
  250-DSN
  250-ONEX
  250-ETRN
  250-XUSR
  250-STARTTLS
  250-DELIVERBY
  250 HELP
  Auth Test

The following shows a telnet to the mail server, type the text in bold and look for the result in italics/bold

  $ telnet localhost 25
  220 mail.example.net ESMTP Sendmail 8.11.1/8.11.1; Sat, 19 May 2001 08:04:04 -0400
  EHLO localhost
  250-mail.example.net Hello IDENT:jose@[127.0.0.1], pleased to meet you
  250-ENHANCEDSTATUSCODES
  250-EXPN
  250-VERB
  250-8BITMIME
  250-SIZE
  250-DSN
  250-ONEX
  250-ETRN
  250-AUTH DIGEST-MD5 CRAM-MD5
  250-XUSR
  250-STARTTLS
  250-DELIVERBY
  250 HELP

You might find that the authentication methods supported don’t match what you selected. This is due to various libraries not supporting those methods of authentication.

Open Relay Test

http://www.relaycheck.com telnet relay-test.mail-abuse.org Authentication Test On the system it should work Off the system it should require authentication

  telnet server 25
  helo server
  mail from: user@server
  rcpt to: someotheruser@someotherserver

This should complain about relaying being denied without authorisation if not on the local machine

Authentication Test

With SSL & Authentications Enabled You should be able to email anywhere.

Last Updated: 20080521054508

This page documents the current state of the IWN driver for FreeBSD, the driver supports the Intel 4965AGN Wireless Card, often found in Intel Centrino based laptops. If your looking for the driver for the 3945 chipset check out the wpi page.

Note: A majorly updated version of the iwn driver has been committed to FreeBSD -Current (aka 8.0). This version is greatly advanced over the perforce version. If you intend to try the perforce version under 7.0 please make sure you are running 7.0-STABLE not 7.0-RELEASE

An up to date commentary on what I’m working on can often be found in my blog and this is a wiki page so you can check the page history for what has changed

Details on how to help debug the driver are in README file that’s in the download package.

For those of you wanting to try the perforce version of the driver, the script P4fetch.rb will help you easily obtain the files. The script was provided by Tom Evans.

History

• Iwn Committed to FreeBSD -Current (aka 8.0)

• Perforce Version (Not yet tarballed – available here)
  • Initial OpenBSD Import
  • Many locking additions
  • Lots of changes to get things working. At present, the card will talk with an unencrypted access point, pass packets but stops passing packets after a while.
  • Major overhaul by Sam Leffler

• Perforce (VAP Branch)
  • Major changes to support vap

Installation/Testing Instructions

At present installation of the driver is very much hands on.

To install the driver:

• Download from p4 using the scribe above
• Read the README file

FAQ / Build Issues

• No common FAQ’s as yet.

Outstanding issues

• Background scanning doesn’t yet exist.

Background

Frox is a groovy little ftp-proxy which has the advantage of being able to cache the files that it proxies. Whilst there is lots of ftp proxies that help ftp through a firewall, frox is the only ftp-proxy I have found that supports caching of files.

Even better than that, Frox has the ability to use an external cache. Hence it’s possible to setup frox so it passes all ftp transfers to squid. This allows a dedicated cache (squid) to do the hard work with frox being merely a relay.

Frox also has the ability to transparently provide ftp proxying. Hence you can provide cached transparent ftp proxying without the users even knowing about it. This proves to be a great benefit in a lab where many people use ubuntu and often do apt-get update. The first user grabs the data, the second gets it from the cache.

The Problem

Sadly whilst frox has all these features, no on has really worked on it for a number of years. Hence somethings don’t work as well as expected. Take for instance the fact I wanted transparent proxying using pf the [OpenBSD] packet filter. Sadly frox doesn’t support pf.. till now

The Setup

I wanted a transparent ftp proxy setup which would use my existing squid setup in order to provide caching ftp. I use pf for my firewall (FreeBSD supports: ipfw, ipfilter and pf) so it had to work with my existing firewall setup.

Initially I setup frox-0.7.18 and tried with both ipfw and ipfilter options enable to get transparent redirects working. I setup the following pf rule to redirect to frox:

# Redirect to frox on port 2121
rdr on xl0 proto tcp from $local_net to any port 21 \
        -> $my_gatewaybox port 2121

and the frox config file as given below. Sadly frox kept replying

421 Proxy tried to loop. Closing connection

I tried lots of things to no avail to fix this. Finally I sat down and looked at the code.

Frox PF Support

It turns out that frox was complaining about the loop due to the destination address being the localhost/gateway box rather than actual ftp server we wanted to contact. This was due to the pf redirect. Basically the following happens:

Normal Connection (No transparent proxying)

internalhost -> frox -> <tooltip>XQXFTPXQX|XQXFile Transport ProtocolXQX</tooltip> Server

Frox gets a connection request:

Connect:

• From: My internal Address
• To: This FTP Server

All is good

rdr connection (Transparent Proxying)

With Squid backend

internalhost -+ rdr
              |
              +- ftp-proxy (gateway box)
                      |
                      | External Cache (ie squid) -> Ftp Server

Frox gets a connection request:

Connect:

• From: internal host
• To: gatewaybox host

Without Squid Backend

internalhost -+ rdr        +-> <tooltip>XQXFTPXQX|XQXFile Transport ProtocolXQX</tooltip> Server
              |            |
              +- ftp-proxy +
                      |
                      | Frox Internal cache

Frox gets a connection request: Connect:

• From: internal host
• To: gateway box

Correcting the problem

Now the problem is simple. Frox never sees the ftp server address that the client intended to visit if we use the pf redirect. For ipfw, this is not the issue you can use something like:

ipfw fwd INTERNALGWADDRESS,2121 tcp from INTERNALNETWORK to any dst-port 21 out keep-state

and ipfw only does the redirect on the ‘out’ traffic. Hence frox sees valid information for the ftp server.

IPFilter works as in bsd.c frox has code that asks ipfilter for the external address. But frox has no code to do the same for pf.

Hence I worked on the code to provide the same functionality to frox for pf. The below patch is what I came up with and it works!

Installation

To use this code:

• Download frox-0.7.18.tar.gz from http://frox.sourceforge.net/
• Extract the code tar xvzf frox-0.7.18.tar.gz
• Run configure ./configure
• cd src
• Download the patch below to the file bsd.diff
• patch < bsd.diff
• Compile: make

Now your done, you can use a rule like:

# Redirect to frox on port 2121
rdr on xl0 proto tcp from $local_net to any port 21 \
        -> $my_gatewaybox port 2121

in pf.conf, and load the rule with pfctl -f pf.conf to make it work.

How you set frox running is up to you!

Hope this helps someone. –Benjsc 10:25, 23 May 2007 (EIT)

Frox PF Transparent Proxy Patch

The following patch allows frox to be used as a transparent ftp proxy with the OpenBSD pf packet filter.

The full patch is available at http://www.clearchain.com/~benjsc/download/frox/pf.patch

--- bsd.c.orig  Fri Feb  4 20:54:55 2005
+++ bsd.c       Wed Jul 25 01:25:16 2007
@@ -30,6 +30,16 @@
 #error --enable-transparent-data not supported under BSD
 #endif

+#ifdef PF
+#include <fcntl.h>
+#include <sys/ioctl.h>
+#include <net/if.h>
+#include <net/pfvar.h>
+
+static int natfd;
+#endif
+
+
 #ifdef IPFILTER
 #include <fcntl.h>
 #include <sys/ioctl.h>
@@ -51,6 +61,11 @@
        if(natfd < 0)
                write_log(ERROR, "Unable to initialise IPFilter");
 #endif
+#ifdef PF
+       natfd = open("/dev/pf", O_RDWR);
+       if (natfd == -1)
+               write_log(ERROR, "Unable to initialise PF");
+#endif
        return 0;
 }

@@ -61,6 +76,11 @@
 int get_orig_dest(int fd, struct sockaddr_in *addr)
 {
        socklen_t len;
+#ifdef PF
+    struct pfioc_natlook nl;
+    struct sockaddr_in from;
+    int r2;
+#endif
 #ifdef IPFILTER
        struct natlookup nat;
        struct sockaddr_in from;
@@ -99,6 +119,31 @@
                addr->sin_family = AF_INET;
                return r2;
        }
+#endif
+#ifdef PF
+       getpeername(fd, (struct sockaddr *) &from, &len);
+       memset(&nl, 0, sizeof(struct pfioc_natlook));
+       memcpy( &nl.daddr.v4, &to.sin_addr.s_addr, sizeof( nl.saddr.v4 ));
+       memcpy( &nl.saddr.v4, &from.sin_addr.s_addr, sizeof( nl.daddr.v4 ));
+       nl.dport = to.sin_port;
+       nl.sport = from.sin_port;
+       nl.af = AF_INET;
+       nl.proto = IPPROTO_TCP;
+       nl.direction = PF_INOUT;
+
+       if ( natfd > 0 ){
+           if (ioctl(natfd, DIOCNATLOOK, &nl)==-1){
+               write_log(ERROR, "Failed to lookup address");
+           }
+           else {
+               memset(addr, sizeof(*addr), 0);
+               memcpy(&addr->sin_addr.s_addr, &nl.rdaddr.v4.s_addr, sizeof(struct sockaddr_in));
+               addr->sin_len = sizeof(struct sockaddr_in);
+               addr->sin_port = nl.rdport;
+               addr->sin_family = AF_INET;
+               return r2;
+                  }
+          }
 #endif
        memcpy(addr, &to, len);
        return r1;

Frox Config File

The config file to use Frox as a transparent ftp proxy with pf.

# $ClearChain: machines/aquila/usr/local/etc/frox.conf,v 1.2 2006/05/23 11:29:15 benjsc Exp $
# Configuration file for frox transparent ftp-proxy.

# Send SIGHUP after editing and it will be reread. This will fail
# completely if we are chrooted and the config file isn't within the
# dir we have chrooted to, or if we have dropped priveleges and no
# longer have permission to read the config file! Some options cannot
# be reread - namely those which require special privelidges (ie.
# BindToDevice, Listen, Port, TransparentData) and the caching stuff.

####################################################################
# Network Options                                                  #
####################################################################

# Address to listen on - default is 0.0.0.0 If you are using an OS other
# than Linux and are doing transparent proxying then you will need to set
# this to the IP of a local interface. If using linux you could leave it
# commented out to listen on all local IPs.
#
# Listen firewall.localnet
# Change it with your ip!
Listen 192.168.154.1

# Port to listen on. Must be supplied.
#
Port 2121

# Whether to run from inetd. You should still define Port above, but
# it isn't used for much.
#
# FromInetd yes

# Stop frox from putting itself into the background. Use this if you want
# to run frox from supervise from djb's daemontools
#
#NoDetach yes

# A hack that should allow you to get away without putting resolver libraries
# into the chroot jail. The default is fine unless for some reason you have
# this hostname in /etc/hosts. If this sort of thing offends you, you may
# comment this out and copy resolver libraries into the chroot jail instead.
# See FAQ section 3.2 for details.
#
#ResolvLoadHack wontresolve.doesntexist.abc

# Another ftp proxy to forward on to. Frox will contact this ftp
# proxy, and send it a login name of the form "user@host:port" where
# host and port are the server frox should contact. If you set
# FTPProxyNoPort then frox will send logins of the form user@host
#
# FTPProxy 192.168.2.9:2222
# FTPProxyNoPort yes

# Pick the IP frox should use for outgoing connections. You probably don't
# need this, and it is not well tested.
#
#TcpOutgoingAddr 192.168.154.1

# Pick the IP that frox should send in PASV replies to the client. Defaults
# to the address frox received the control connection on which you shouldn't
# need to change unless you are doing NAT between frox and your clients, or
# are trying to tunnel connections using frox. See FAQ.
#
#PASVAddress 192.168.0.2

####################################################################
# General Options                                                  #
####################################################################
# User and group to drop priveliges to. This must be specified - if
# you really want to run as root (not a good idea) you must say so
# specifically, and have compiled with --enable-run-as-root.
#
User nobody
Group nogroup

# This is frox's working directory - it must be specified. Temporary
# files and sockets will be created here. If you are using local
# caching then the cache will be stored in this directory too. It
# should be owned by frox with permissions 700. By default frox will
# also chroot to this dir on startup. To avoid this you must specifically
# set DontChroot to Yes.
#
WorkingDir /tmp
DontChroot Yes

# Logging level. 0=No logging. 5=Critical errors only. 10= All errors.
# 15=Errors, other important stuf. 20= Errors, connections, cache
# hits/misses 25=Debug info including text of control session. By
# default frox will log through syslog as facility daemon. If you want
# frox to log to a file instead specify this in LogFile below. You may
# set LogFile to "stderr" if you wish it to log there. XferLogging
# defaults to on, and results in a one line log entry for each file
# transferred irrespective of the log level. You can turn this off
# below.
#
# LogFile /tmp/frox.log
# XferLogging no
#LogLevel 15

# File to store PID in. Default is not to. If this file is not within
# the Chroot directory then it cannot be deleted on exit, but will
# otherwise work fine.
#
PidFile /var/run/frox.pid

####################################################################
# Ftp Protocol Options                                             #
####################################################################

# Active --> Passive conversion. If set then all outgoing connections
# from the proxy will be passive <tooltip>XQXFTPXQX|XQXFile Transport ProtocolXQX</tooltip>, regardless of the type of the
# connection coming in. This makes firewalling a lot easier. Defaults
# to no.
#
APConv yes

# Passive --> Active conversion. If set then all outgoing connections
# from the proxy will be active <tooltip>XQXFTPXQX|XQXFile Transport ProtocolXQX</tooltip>, regardless of the type of the
# connection coming in. Defaults to no.
# DO NOT USE WITH APConv!
#
#PAConv yes

# Block PORT commands asking data to be sent to ports<1024 and
# prevent incoming control stream connections from port 20 to
# help depend against ftp bounce attacks. Defaults to on.
#
BounceDefend yes

# If true then only accept data connections from the hosts the control
# connections are to. Breaks the rfc, and defaults to off.
#
# SameAddress yes

# Normally frox strips out nonprintable characters from the control
# stream. This makes buffer overflow attacks on clients/servers much more
# difficult. If you download files that contain non english characters
# this may cause you problems (especially for big charsets like Chines).
# In that case turn on this option.
#
# AllowNonASCII yes

# Try to transparently proxy the data connections as well. Not
# necessary for most clients, and does increase security risks. N.V.
# You probably do _NOT_ need this option. It increases the complexity
# of what frox has to do, increases the difficulty of setting frox up
# correctly, and increases potential security risks. This has nothing
# to do with whether your clients will be transparently proxied. If
# you still want to use this option then read README.transdata for
# details.
#
# TransparentData yes

# Specify ranges for local ports to use for outgoing connections and
# for sending out in PORT commands. By default these are all between
# 40000 and 50000, but you might want to split them up if you have
# complicated firewalling rules.
#
# ControlPorts 40000-40999
# PassivePorts 41000-41999
# ActivePorts  42000-42999

# SSL/AUTH support. Frox must have been linked to the openssl libraries.
# This is currently experimental, and only tested against vsftpd
#
# UseSSL yes
# DataSSL no

####################################################################
# Caching Options                                                  #
####################################################################

# Caching options. There should be at most one CacheModule line, and
# Cache lines to give the options for that caching module. CacheModule
# is <tooltip>XQXHTTPXQX|XQXHyperText Transfer ProtocolXQX</tooltip> (rewrites ftp requests as <tooltip>XQXHTTPXQX|XQXHyperText Transfer ProtocolXQX</tooltip> and sends them to a <tooltip>XQXHTTPXQX|XQXHyperText Transfer ProtocolXQX</tooltip>
# proxy like squid), or local (cache files locally). The relevant
# module needs to have been compiled in at compile time. See FAQ for
# details. If there are no CacheModule lines then no caching will be
# done. "CacheModule None" explicitly requests no caching, and is
# useful to turn off caching within a subsection (below).
#
# CacheModule local
# CacheSize 400
#
CacheModule http
HTTPProxy 127.0.0.1:3128

# MinCacheSize 65536
# ForceHTTP no  # Set to yes to force http file retreiving even if
#               # file is not cacheable
#
# StrictCaching no  # Read FAQ for details.
# CacheOnFQDN yes   # Read FAQ for details.
#
# CacheAll no   # Set to yes to cache non anonymous ftp downloads

# Virus scanning -- see FAQ
#
# VirusScanner '"/usr/bin/viruscan" "--option" "%s"'
# VSOK 0
# VSProgressMsgs 30

####################################################################
# Access control                                                   #
####################################################################

# Allow non-transparent proxying support. The user can connect
# directly to frox, and give his username as user@host:port or
# user@host. Defaults to no. NTPAddress gives the address to which
# incoming connections must be addressed if the client is to be offered
# non-transparent proxying. For most people using this it will be the same
# as the Listen address above. If not given then all connections will be
# offered non transparent proxying. If you are not using transparent
# proxying at all then you should leave NTPAddress commented out.
#
#DoNTP yes
#NTPAddress 192.168.155.1:2121

# Number of seconds of no activity before closing session
# Defaults to 300
#
# Timeout 300

#Maximum number of processes to fork.
#
# MaxForks 0 # For debugging -- only one connection may be served.
MaxForks 10

# Maximum number of connections from a single host (IP address).
MaxForksPerHost 4

# Maximum number of bytes/second to be transferred over the data
# connection for each client. MaxTransferRate limits downloads and
# MaxUploadRate uploads. CacheDlRate is the rate for downloads of files
# that are cached locally - if not set these files will be downloaded at
# full speed.
#
# MaxTransferRate 4096
# CacheDlRate 8192
# MaxUploadRate 4096

# Access control lists:
# The format is: "ACL Allow|Deny SRC - DST [PORTS]"
# a dns name, or * to match everything.
#
# PORTS is a list of ports. If specified then the rule will only match
# if the destination port of the connection is in this list. This is
# likely only relevant if you are allowing non-transparent proxying of
# ftp connections (ie. DoNTP is enabled above). Specifying * is equivalent
# to not specifying anything - all ports will be matched
#
# Any connection that matches no rules will be denied. Since there are
# no rules by default you'll need to add something to let any
# connections happen at all (look at the last example if you are
# feeling lazy/not bothered by security).
#
# # Examples:
# # Allow local network to ftp to port 21 only, and block host ftp.evil
# ACL Deny * - ftp.evil
# ACL Allow 192.168.0.0/255.255.0.0 - * 21
#
# # Allow local network to ftp anywhere except certain dodgy ports. Network
# # admin's machine can ftp anywhere.
# ACL Allow admin.localnet - *
# ACL Deny * - * 1-20,22-1024,6000-6007,7100
# ACL Allow 192.168.0.0/16 - * *
#
# # You don't really believe in this security stuff, and just want
# # everything to work.
ACL Allow * - *

# Command control program: A bit like the idea of a squid redirector.
# By default the old interface is used so as not to break existing
# installations. The new interface is much more powerful, and is
# reccommended for new scripts -- set UseOldCCP to false to use it.
# See the FAQ for details.
#
# CCProgram /usr/local/lib/frox/bin/ccp
# UseOldCCP no

####################################################################
# Subsections                                                      #
####################################################################
# Matching rules the same as ACLS. Only some options can be specified
# in a subsection (currently the yes/no options, timeout, and caching
# options).
#
# SubSection * - ftp.dodgy.server
#  StrictCaching yes
# EndSection
#
# SubSection * - 10.0.0.0/24 # A low latency high bandwidth connection
#  MinCacheSize 4096
# EndSection
#
# Subsection * - ftp.localnetwork
# # To disable caching if it has been turned on in a parent section
#  CacheModule None
# EndSection