The Internet and Linux
The Internet and Linux
I usually use my Linux box and Freenetname to connect to the internet. Like most free ISPs, Freenetname does not officially support Linux for connections, and like most ISPs these days, uses PPP for modem connections. Almost any ISP that uses PPP or SLIP can probably be used with Linux - however, I have had trouble connecting to several.
I have had considerable trouble connecting to Free Online and FreeNet using Linux. Free Online just hangs up as soon as the username and password are sent (any username and password). FreeNet is very slow, and pppd fails after connecting as it can't determine the remote IP address. Both are fine under Windows, and Free Online was fine for a while and then suddenly stopped working. I suppose FreeNet uses a protocol unsupported by pppd, but I have no idea what's up with Free Online (sigh, unlimited webspace was very tempting :-)
In addition, I used to have an account with PlusNet, who suddenly decided that when they *said* 'unlimited time', they *meant* '45 minutes'. They also thought that changing the contract that customers had signed up to retrospectively and without consultation was a good idea (which, if nothing else, falls afoul of the provision in British contract law that states that a contract must be fair, never mind the other provisions). Suffice it to say that I don't use them any more. They're the same people who run Free Online, but when you're with them, they don't mind because they get money by the minute when you use the 0845 number. (PlusNet, incidentally, claim that they never said unlimited in the first place. Of course, I use a caching proxy server, which still has the original page saying unlimited fairly explicitly).
Update: I have recently retried with Free Online, and managed to get on as guest, so it looks like they sorted out their problems. I deleted the scripts for FreeNet in disgust a long time ago, so I haven't tried them recently. Oh yeah, my windows ppp driver went AWOL some time ago, so I now only connect using Linux... Freenetname have completely failed to give me money for the free advertising on this page too. *Sigh* a vast pile of money of my very own isn't too much to ask is it? ;-)
These days, I'll plug demon, who charge a bit more than some other ISPs, but they've been around for a while, provide a very reliable and consistent service, give you a static IP and don't complain when you actually use their service.
This description applies mainly to Linux. Other Unixes probably require other techniques - I'll investigate Solaris if anyone is interested... Anyway, the first thing to do is get pppd installed and working - this usually means installing networking software, and can mean recompiling the kernel to include ppp support. Next you need an account on your ISP of choice. Your milage may vary here, but I have had success with CallnetUK as well as Freenetname. Usually, you sign up using Windows [groan] (if you don't have a Windows partition, VMWare is your friend). It is important to find out and note down the DNS, mail and news servers - most free ISPs have a support section on their website that lists this information: if they don't, choose a different ISP.
Several files control a ppp session. In this case, I am using the ifup script (as supplied with RedHat 5.0) to initiate the connection. This means creating the files ifcfg-ppp0 and chat-ppp0, which contain the configuration information and start-up script respectively. Your distribution probably contains software to create these for you, but I prefer to create them myself - it's much easier to find out what went wrong. For some ISPs, you will need to set up PAP or CHAP as well. The setup I describe here was designed for Redhat Linux, and may not work on other distributions. You may wish to use another ppp tool anyway, in which case you can skip this section.
This file contains general configuration information for the connection,
and is located (on my system at least) in
/etc/sysconfig/network-scripts.
My freenetname configuration file contains:
PERSIST=yes DEFROUTE=yes ONBOOT=no INITSTRING=ATZ MODEMPORT=/dev/modem LINESPEED=115200 ESCAPECHARS=no DEFABORT=yes HARDFLOWCTL=yes DEVICE=ppp0 PPPOPTIONS= DEBUG=yes PAPNAME=achunter REMIP= IPADDR= BOOTPROTO=none MTU= MRU= DISCONNECTTIMEOUT= RETRYTIMEOUT= USERCTL=yes
Most of these options can be left the same for all ISPs. The options you
may be interested in fiddling with are DEBUG, which sets whether
or not debugging information is logged via syslogd, PAPNAME,
which sets your name under PAP, and DEVICE, for when you
have several ISPs to choose from.
This file contains the sequence of commands to be sent to the modem, and is usually located in the same directory as ifcfg-ppp0. It is run by chat(8), and is a usual source of problems, particularily as the automatic configuration programs such as netcfg don't always make a good job of writing it. For freenetname, my chat-ppp0 file contains:
'ABORT' 'BUSY'
| a collection of messages that cause the script to abort and hang up the modem |
'ABORT' 'ERROR'
| |
'ABORT' 'NO CARRIER'
| |
'ABORT' 'NO DIALTONE'
| |
'ABORT' 'Invalid Login'
| |
'ABORT' 'Login incorrect'
| |
'ABORT' 'Authentication failed.'
| |
'' 'ATZ'
| reset the modem. You may want to use '+++ATZ' here, (+++ gets the modems attention when it's already connected) |
'OK' 'ATM0W1S95=46S37=104'
| modem initialisation string for a Hayes Accura 33.6 modem. Substitute your own here. |
'OK' 'ATDT0845 080 0889'
| Dial the number |
'ogin:' 'username'
| Send your username. Note that netcfg adds a CONNECT line before this one - this causes a carriage return to be sent, which freenetname does not like. Your Milage May Vary with this, of course. Some ISPs just use 'username' and some ISPs use 'username@isp.somewhere.blop'. Freenetname uses the latter (that is, 'username@freenetname.co.uk'). You can have similar problems with the PAP authentication too. |
'ord:' 'password'
| Send your password |
'TIMEOUT' '5'
| Wait up to 5 seconds for the next line |
'~--' ''
| PPP data starts after this |
Some ISPs, like freenetname, need PAP authentication with PPP connections.
This is done by setting the PAPNAME in ifcfg-ppp0, and putting
a corresponding secret into /etc/ppp/pap-secrets. The file should look like
the following:
# pap-secrets # client server secret IP addresses username ppp0 password
Substitute your username and password as appropriate. The username should match
the value PAPNAME. pap-secrets should ideally only be readable
by root and the pppd daemon. chown root:daemon pap-secrets sets
the owners correctly, and chmod og-rwx pap-secrets sets the
permissions.
Make sure that DEBUG is set to yes in ifcfg-ppp0. This will
ensure you can see what's going on with chat & ppp in the logs
(/var/log/messages, usually). As root, do a ifup ppp0 to
enable the ppp0 interface. The logs will show the progress of the connection,
and will indicate when you have connected, or give an indication of the
problem:
Sep 19 17:52:49 chrysoprase ifup-ppp: pppd started for ppp5 on /dev/modem at 115200 Sep 19 17:52:49 chrysoprase pppd[1559]: pppd 2.3.5 started by root, uid 0 Sep 19 17:52:49 chrysoprase pppd[1559]: Removed stale lock on modem (pid 1080) Sep 19 17:52:50 chrysoprase chat[1561]: abort on (BUSY) Sep 19 17:52:50 chrysoprase chat[1561]: abort on (ERROR) Sep 19 17:52:50 chrysoprase chat[1561]: abort on (NO CARRIER) Sep 19 17:52:50 chrysoprase chat[1561]: abort on (NO DIALTONE) Sep 19 17:52:50 chrysoprase chat[1561]: abort on (Invalid Login) Sep 19 17:52:50 chrysoprase chat[1561]: abort on (Login incorrect) Sep 19 17:52:50 chrysoprase chat[1561]: abort on (Authentication failed.) Sep 19 17:52:50 chrysoprase chat[1561]: send (ATZ^M) Sep 19 17:52:50 chrysoprase chat[1561]: expect (OK) Sep 19 17:52:51 chrysoprase chat[1561]: ATZ^M^M Sep 19 17:52:51 chrysoprase chat[1561]: OK Sep 19 17:52:51 chrysoprase chat[1561]: -- got it Sep 19 17:52:51 chrysoprase chat[1561]: send (ATM0W1S95=46S37=104^M) Sep 19 17:52:52 chrysoprase chat[1561]: expect (OK) Sep 19 17:52:52 chrysoprase chat[1561]: ^M Sep 19 17:52:52 chrysoprase chat[1561]: ATM0W1S95=46S37=104^M^M Sep 19 17:52:52 chrysoprase chat[1561]: OK Sep 19 17:52:52 chrysoprase chat[1561]: -- got it Sep 19 17:52:52 chrysoprase chat[1561]: send (ATDT0845 080 0889^M) Sep 19 17:52:52 chrysoprase chat[1561]: expect (ogin:) Sep 19 17:52:52 chrysoprase chat[1561]: ^M Sep 19 17:53:16 chrysoprase chat[1561]: ATDT0845 080 0889^M^M Sep 19 17:53:16 chrysoprase chat[1561]: CARRIER 31200^M Sep 19 17:53:16 chrysoprase chat[1561]: ^M Sep 19 17:53:16 chrysoprase chat[1561]: PROTOCOL: LAP-M^M Sep 19 17:53:16 chrysoprase chat[1561]: ^M Sep 19 17:53:16 chrysoprase chat[1561]: COMPRESSION: V.42BIS^M Sep 19 17:53:16 chrysoprase chat[1561]: ^M Sep 19 17:53:16 chrysoprase chat[1561]: CONNECT 115200/ARQ^M Sep 19 17:53:17 chrysoprase chat[1561]: ^M Sep 19 17:53:17 chrysoprase chat[1561]: Login: Sep 19 17:53:17 chrysoprase chat[1561]: -- got it Sep 19 17:53:17 chrysoprase chat[1561]: send (username^M) Sep 19 17:53:17 chrysoprase chat[1561]: expect (ord:) Sep 19 17:53:17 chrysoprase chat[1561]: Password: Sep 19 17:53:17 chrysoprase chat[1561]: -- got it Sep 19 17:53:17 chrysoprase chat[1561]: send (password^M) Sep 19 17:53:17 chrysoprase chat[1561]: timeout set to 5 seconds Sep 19 17:53:17 chrysoprase chat[1561]: expect (~) Sep 19 17:53:18 chrysoprase chat[1561]: ^M Sep 19 17:53:18 chrysoprase chat[1561]: Entering PPP mode.^M Sep 19 17:53:18 chrysoprase chat[1561]: Async interface address is unnumbered (Loopback0)^M Sep 19 17:53:18 chrysoprase chat[1561]: Your IP address is 195.147.171.35. MTU is 1500 bytes^M Sep 19 17:53:18 chrysoprase chat[1561]: Header compression will match your system.^M Sep 19 17:53:18 chrysoprase chat[1561]: ^M Sep 19 17:53:21 chrysoprase chat[1561]: ~ Sep 19 17:53:21 chrysoprase chat[1561]: -- got it Sep 19 17:53:21 chrysoprase chat[1561]: send (^M) Sep 19 17:53:21 chrysoprase pppd[1559]: Serial connection established. Sep 19 17:53:22 chrysoprase pppd[1559]: Using interface ppp0 Sep 19 17:53:22 chrysoprase pppd[1559]: Connect: ppp0 <--> /dev/modem Sep 19 17:53:23 chrysoprase pppd[1559]: local IP address 195.147.171.35 Sep 19 17:53:23 chrysoprase pppd[1559]: remote IP address 195.147.160.11
The connection is established as soon as you get the local/remote ip
address messages. To test the connection, ping a computer somewhere on
the internet - if you haven't set the DNS servers up yet, you will have
to give an IP address. Problems are usually with chat - invalid passwords,
etc. A program that lets you talk direct to the modem like minicom is
good for sorting out these problems and editting the chat script. Take
down the connection with ifdown ppp0.
Once the connection has come up, pppd should set up routes appropriately,
so all that is left to do is to set up DNS so that you can refer to servers
by their names rather than their IP addresses. The DNS server is specified
by /etc/resolv.conf, and for freenetname, this file should contain
the following:
search nameserver 194.126.82.5 nameserver 193.126.86.9
Once this is set up, you can test it using nslookup. Run nslookup
and type the name of a server that you know (eg www.logicalshift.org.uk) - it
should come back with the IP address.
The web was originally designed for computers that are connected permanently to the internet. This means that when you disconnect from your ISP, the web disappears. Fear not! Salvation is here in the form of the proxy server wwwoffle, which stores each web page as you visit it, and allows you to view the stored version again as soon as you're offline. It also support fetching web pages and entire websites for later browsing. Excellent for when you're watching your phone bill. To use wwwoffle, compile and install the binaries, then start wwwoffle in your init sequence.
To start wwwoffle, I use an init script. These are scripts executed by
init on entering differing runlevels, and on Redhat Linux are stored
in /etc/rc.d/. This script will do the trick of
starting a wwwoffle installed in /usr/local - install it as
/etc/rc.d/init.d/wwwoffled, and then do a
ln -s /etc/rc.d/init.d/wwwoffled /etc/rc.d/rc3.d/S80wwwoffled
to start it in runlevel 3 (the default runlevel for Linux). You may want to
add a K25wwwoffled to /etc/rc.d/rc6.d/ in order to
shut wwwoffled down when you shut down the system.
Now, this is a tricky one. Most ISPs use POP and SMTP to transfer mail, and most Linux mail clients receive mail from the local mail spool, and send it using the local SMTP server or sendmail. This all assumes that your machine has a permanent connection to the internet. Now, some programs like kmail and balsa support POP fetching from an ISP and let you read mail offline, but require you to be online to read mail. The solution I describe here will allow you to use almost any Linux mail client software to both send and receive mail, it being transmitted when you connect to your ISP.
I use QMail to send mail. QMail is a sendmail replacement (it has less bugs and security holes, and it's somewhat easier to configure). As well as QMail, you will need a patched version of procmail, serialmail and ucspi. I recommend using mutt(1) as a mailer. If your distribution comes with qmail as a package, make sure sendmail is uninstalled, and then install qmail. If not, you're going to have to build from source.
There are comprehensive instructions provided with qmail on how to build it, but these assume you are providing a full mail server, and are not necessarily appropriate for your ISP. Basically, the following steps should give you a copy of QMail installed in /var/qmail, using Linux:
mkdir /var/qmail
INSTALL.ids):
groupadd nofiles useradd -g nofiles -d /var/qmail/alias alias useradd -g nofiles -d /var/qmail qmaild useradd -g nofiles -d /var/qmail qmaill useradd -g nofiles -d /var/qmail qmailp groupadd qmail useradd -g qmail -d /var/qmail qmailq useradd -g qmail -d /var/qmail qmailr useradd -g qmail -d /var/qmail qmails
make setup check
./config-fast chrysoprase.logicalshift.org.uk
The name you give after config-fast must be a valid hostname, usually the name of your ISPs SMTP mailserver (smtpmail.freenetname.co.uk, for example). If you have your own domain name, you can use something.domainname.foo instead if you like.
(cd ~alias; touch .qmail-postmaster .qmail-mailer-daemon .qmail-root) chmod 644 ~alias/.qmail*
This will install a fairly minimal qmail. It won't be functioning yet, but we're getting there...
QMail keeps its configuration in /var/qmail/control. You will need the following files in there (you can probably delete the others):
defaultdomain defaulthost locals me plusdomain smtpgreeting virtualdomains
The files should have contents as follows:
freenetname.co.uk). This might seem a weird, nay backward, way
of doing things, but it works ;-)
chrysoprase.logicalshift.org.uk logicalshift.org.uk freenetname.co.uk
.org.uk. If you are using a .co.uk address,
put that in instead.
:alias-ppp. This ensures
that when you send mail that it queued for later transmission.
You then need to create the ppp alias. First create maildir called pppdir
in ~alias (/var/qmail/alias). The program
maildirmake does this particular trick -
/var/qmail/bin/maildirmake ~alias/pppdir is the command
you want. Chown it to belong to alias
(chown -R alias:qmail ~alias/pppdir). Finally, create the file
~alias/.qmail-ppp-default, which specifies where the 'ppp' mail
goes. This file should contain the single line ./pppdir/. Now,
assuming that the control directory is set up as above, all outgoing mail
sent via qmail will go to ~alias/pppdir, and qmail won't attempt to send it
to the outside world. You can use qmail-inject to check
that this is working.
The next step is to install serialmail. This program provides various
utilities for manipulating the contents of QMail maildirs, of particular
interest to us is maildirsmtp, which sends the contents
via SMTP to the destinations specified in their Delivered-To: fields.
The control files above fool QMail into generating a suitable Return-Path:
field for this program. To actually send the mail, I use the following
shell script:
#!/bin/sh /usr/local/bin/maildirsmtp ~alias/pppdir alias-ppp- smtpmail.freenetname.co.uk freenetname.co.uk
Replace smtpmail.freenetname.co.uk with your SMTP server name.
freenetname.co.uk is the name passed in the HELO part
of the SMTP protocol. It's a good idea to pass something sensible here, and
some mail servers require your IP address or a specific host name here.
The easiest way to make this script work is to make
/usr/local/bin/maildirserial suid alias. The commands to do
this are as follows:
chown alias:qmail /usr/local/bin/serialsmtp chmod ug+s /usr/local/bin/serialsmtp
There are probably better ways of doing this, but this one has the advantage of being simple and easy.
Precisely how you do this depends on which program you use. I use mutt, which is fairly simple to set up to use QMail. Basically, you need to make the sendmail used by mutt point to QMail's version:
ln -sf /var/qmail/bin/sendmail /usr/sbin/sendmail ln -sf /var/qmail/bin/sendmail /usr/lib/sendmail
Note that these commands overwrite any versions of sendmail that might already be there. Finally, set the From: header field used by mutt to your email address (put this in .muttrc in your home directory):
my_hdr From: Andrew Hunter <andrew@logicalshift.org.uk>
And that's it! Now, sending a mail using mutt should create a new entry in ~alias/pppdir/new, and you should be able to use the script to actually transmit the mails across the internet. You can test that everything's working by creating an account at a free email provider (such as hotmail), and sending mail there.
If you have a demon account, they actually deliver the mail to you by SMTP. This seems to be an undocumented feature (in that I couldn't find any documentation for it), but I've been using it for the last few months without any problems - I found out about it because it kept producing stuff in my firewall logs. Provided you've set qmail up correctly, it should just work with no further intervention, assuming that you haven't blocked port 25 anywhere. This is a very nice feature, because mail gets delivered to the person it was mailed to, rather than the person in the To: header (yes, they are different :-). This means that mailing lists get through to the right user... also spam, depressingly... One of the things about spam is that it almost never has a To: address that is you. Setting up a procmail script that chucks mail that isn't addressed to you (in the Cc: or To: fields) into a different mailbox is a good way to filter out spam.
Fortunately, this is much easier - you use fetchmail to retrieve mail from a server, and procmail to deliver it. I recommend you start by creating a QMail maildir in your home directory and configuring qmail to deliver to it:
/var/qmail/bin/maildirmake ~/Maildir echo ./Maildir/ >.qmail
In a logon script (.bashrc or similar), set MAIL to point to this
directory - mutt will then read mail from here instead of /var/spool/mail.
Now, create a .fetchmailrc:
server mail.freenetname.co.uk proto pop3 user username pass password flush mda /usr/bin/procmail
Substitute your POP mail server, username and password as appropriate. This
file will ensure that when you run fetchmail, all mail will be
retrieved from the mail server by POP and delivered to users on your system by
procmail. You will also need a .procmailrc - for delivery just to you, the
following (editted for your home directory) is sufficient:
:0 /home/ahunter/Maildir/
Note that this assumes the version of procmail patched to use Maildirs.
Standard procmail can be used too, but you will have to stick with using
Mailbox delivery (this is left as an exercise for the reader ;-). Before going
any further, set the permissions correctly on .fetchmailrc and
.procmailrc: no other user may have read access to them.
chmod og-rwx .procmailrc .fetchmailrc
Having done all this, just run fetchmail (no arguments), and all
your mail will be fetched from the POP server and placed in your mailbox.
Normally you will want to check your mail as soon as you log on to your
ISP. On my Redhat system, at least, the script
/etc/ppp/ip-up.local is run as soon as the PPP connection
comes up. You can put anything you like in here, but to deliver and send
the mail, I use:
#!/bin/sh su mrmail /etc/ppp/mrmail
Here, mrmail is the user who has the .fetchmailrc
in his home directory. You should change this to your username.
/etc/ppp/mrmail is a sequence of commands to be run as mrmail.
I use the following sequence, where /home/mrmail/sendout is the
script for sending mail:
/home/mrmail/sendout /usr/bin/fetchmail
Putting websites up by ftp can be a bit of a pain, even with clever ftp clients. You have to upload directories individually, delete files manually and if you do an mput, you might be uploading large files that haven't changed. Yuck. Luckily, I have written a perl script to do precisely that :-) Go here.