steven n fettig's Jitterin' Thoughts

!!!WARNING!!!

This HOWTO (which should be named "misleading install guide that is too old to be taken seriously) no longer matches Oli's at pofo.de. Please direct your attention to lifewithqmail.org for better help and more details. (I'm only leaving this up until I can figure out how to gracefully remove it.)
SF - 2005-11-30

Chapter 1. Installation of the Necessary Programs

Copyright 2000, 2001, 2002, 2003 by Oliver Lehman

Translation 2002 by Steven N. Fettig
Modified: 6 August 2002 - Steven N. Fettig
Modified: 25 November 2002 - Steven N. Fettig
Modified: 21 January 2003 - Steven N. Fettig
Modified: 27 March 2003 - Steven N. Fettig
Modified: 18 February 2004 - Steven N. Fettig

The following HOWTO is provided for the installation of an email server. POP3, POP3 over ssl, IMAP4, and IMAP4 over ssl are the different service configurations in this document. Mail transport is done using SMTP. The authentication for the roaming use of SMTP is taken care of through a preauthorization check done when one authenticates via IMAP4 or POP3. It is assumed that for the purposes of the installation, one is using a current version of FreeBSD.

(Translators note: The translation of this document is more of a "recreation" of the original, not a true translation. It would have taken me much more time to use the exact language structure that Oliver Lehman has used, and I decided that the better purpose of this document would be to give non-German speaking peoples the opportunity to take a look at this wonderful HOWTO in English. So, this is a Steve-ization of Oliver's original writings and I offer my apologies where I have butchered his original document. I only hope to make this thorough and as close as possible to his original, but offer no promises ;) . I will also offer some personal notes, as I have found some issues with the setup of some services. Those notes will be in itallics and enclosed by [ ] brackets.)

This HOWTO was written with great care, however, there may be errors in the following document. I cannot be responsible for erroneous directions and their results and will not be held to any judicial proceedings. I am thankful for any suggestions or corrections.

!!!WARNING!!!

This HOWTO (which should be named "misleading install guide that is too old to be taken seriously) no longer matches Oli's at pofo.de. Please direct your attention to lifewithqmail.org for better help and more details. (I'm only leaving this up until I can figure out how to gracefully remove it.)
SF - 2005-11-30

!!!WARNING!!!

This HOWTO (which should be named "misleading install guide that is too old to be taken seriously) no longer matches Oli's at pofo.de. Please direct your attention to lifewithqmail.org for better help and more details. (I'm only leaving this up until I can figure out how to gracefully remove it.)
SF - 2005-11-30

!!!WARNING!!!

This HOWTO (which should be named "misleading install guide that is too old to be taken seriously) no longer matches Oli's at pofo.de. Please direct your attention to lifewithqmail.org for better help and more details. (I'm only leaving this up until I can figure out how to gracefully remove it.)
SF - 2005-11-30

I have received a number of emails over the past few months regarding adding a section on how to install auth smtp for qmail. To be quite honest, I haven't had the time to figure out how to do this. Plus, with my now 12 domains and 50 some users humming along beautifully on my current qmail server, I am very nervous about changing anything. (In fact, I'm downright paranoid about making any major changes - things have been smooth for 2 years now!) If anyone, however, knows how to add auth smtp to qmail as I have it installed, without hosing the current setup, I would appreciate the insight. A link to my address is in the nav bar above. Or, simply add comments to this posting... I'll see them in due time.

    I know it has been forever since I last wrote about working on auth-smtp for Oliver's qmail howto (it is still Oliver's considering he wrote the original ;)), but life has kept me busy beyond anything I could have ever imagined. I am still nowhere nearer to being a programmer than I was a year ago, so I have to desperately rely on other people's work and knowledge to get things with my email server to work properly. In the mean-time, I have come across a number howto's relating to auth-smtp. While I haven't tested any of it myself, I think if I publish the list I am working through, perhaps someone who has more time than me can come up with the installation instructions. I am making a huge move of my equipment in a few weeks (moving to a new location - new house, new infrastructure and most importantly - a backup generator!) and I expect at that time to test a new qmail server which includes the auth-smtp routine. There are apparently two main ways of going about installing/configuring auth-smpt. The most important issue for those of you using the qmail-vpopmail combination is that you realize that users must be authenticated using vchkpw and not checkpasswd. Here are some of the links I have been looking at (REMEMBER: this is for people who have a good concept of how to hack with *nix servers - not for the novice):
- qmail-smtpd-auth (the main link I will likely use to revise the howto)
- Interazioni: Qmail / vpopmail page
- Re: qmail-smtpd-auth
- EnderUNIX Software Development Team
Hopefully some of these links will help. Please, pretty please, comment if they do and if you have a simple list of directions that go along with the layout of the howto that Oliver originally wrote.

!!!WARNING!!!
This HOWTO - which should be renamed the "qmail quick, uninformed install guide" is horribly outdated. Please see lifewithqmail.org for better and nore up-to-date instructions or help!
SF 2005-11-30

(with support for vqadmin, qmailadmin and sqwebmail)

Steven N. Fettig
See Creative Commons Copyright for copyright details.
Rev 1: 14 May 2004

This howto is very similar to the one I started translating a few years back from Oliver Lehman and is intended for people running FreeBSD 4.9 and up (it may work on lower versions, but I have not tested it and make no implied guarantee that it will work either way). I have found that a number of items in Oliver's howto didn't fit my needs, so I altered the HOWTO accordingly. Firstly, I don't totally understand the ssl certs, so I have not taken the time to make sure they are compiled/created correctly. The qmail and patchset I install includes the tls version of qmail and the courier-imap version I install will handle connections over ssl. I have installed and reinstalled various versions of qmail, vpopmail and courier-imap on the test server and have hosed certain portions of the installation. I have things working well, however, for new domains and intend to do a rebuild with the ssl support.

Assumptions:
- You have a fair grasp of FreeBSD and how to move around the filesystem.
- You can install services/programs via the ports.
- You understand that incorrectly configured email systems can be used as spam gateways - if you don't understand how to test and make sure you have NOT set up an open relay, STOP. Do us all a favor and learn/test/learn before placing a server on the net that is open to spammers.
- I don't know what I am doing half of the time.
- You may need to modify parts of these instructions to get this service combination to work on your machine.
- The smtp-auth patch will only work with vpopmail 5.4 and above.
- I use /usr/ports/distfiles/qmail/ as my SRC directory for compiling and installing software not installed via the ports.

Chapter 1 - Installation of Services

1.1 qmail

Install netqmail-1.05-tls with smtpauth patch (installation instructions are right in the text of the beginning of the patch):

Set sendmail_enable="YES" to sendmail_sendmail="NONE" in /etc/rc.conf

1.2 ucspi-tcp

ucspi-tcp is used so that use of inetd can be avoided in setting up tcp port connections. There are plenty of resources showing why people don't like inetd and I suggest you look for them on google.com.
There is a diff patch applied to rblsmtpd.c from Alan Curry so that "rblsmtpd works with A records." Same as Oliver's instructions:

1.3 daemontools

Daemontools is used to start and log the services we are going to set up. In our case, daemontools only watches qmail, pop3 services and the smtp services. Courier-imap is started via sh scripts. I, personally, like daemontools, but people have made good arguments for and against its use. DJB is a person that people seem to love to hate or love to love, so some of the comments pro or contra are juvenile. So, decide for yourself whether you want to employ daemontools for other items.

1.4 vpopmail

vpopmail administers virtual domains. This allows you to set up multiple domains on one host and has many tools to go along with it (also from vpopmail's makers, inter7) that allow you to add and remove users/domains/etc. with very little effort.

1.5 courier-imap

courier-imap is the service used to allow for imap access to email accounts. It is also an inter7 invention.

If you want to make the ssl certificates you need to follow Oliver's instructions:

Like I said in the beginning - I don't understand enough about ssl to know whether this is the *right thing* or whether it will give you a false sense of security if connecting to imap and pop3 via ssl.

1.6 ezmlm-idx

If you want to enable your server for mailing lists, ezmlm-idx makes this very, very easy.

1.7 qmail-conf

From Tetsu Ushijima's website:

We will end up modifying the vanilla smtp run script so that we can enable smtp-auth. BUT, these -conf scripts have made the creation of run scripts for qmail extremely easy.

Chapter 2 - Configuration of start scripts, vpopmail, smtp-auth and courier-imap

2.1 vpopmail crontab script

vpopmail is automatically sets up a selective relay when a user authenticates via pop3. We need to make sure that the tcp.smtp under /usr/local/vpopmail/etc is "cleaned" every 40 minutes.
Add the following line to your favorite root, vpopmail OR system crontab:

if adding to the system crontab:

2.2 Make the courier-imap startup scripts readable

The courier-imap port sets up install scripts in /usr/local/etc/rc.d that aren't yet readable at boot time because they are appended by the name .sample. We are simply going to copy the default the same name while removing the .sample ending.

You should take a quick look at the authdaemonrc file and take note of the configuration changes you can make.

To increase the number of concurrent sessions per IP, change the default value of imapd under /usr/local/etc/courier-imap

Look through the file for MAXPERIP and change to a value you think will be reasonable for your needs. I need to be able to access 10 plus accounts at any given time, so I set mine to a high number of 20. You may not need to change this value, but it might become important if you find your clients timing out all the time.

2.3 Create the qmail control files and start scripts

We will start by creating the control files and dot postmaster files:

We also need to make sure that the rcpthosts file is created so that the smtp server does not act as an open relay:

The following commands are almost identical to Oliver's, but I do not want to go though the hack to set up the selective relaying through vpopmail because I want to use auth-smtp. Also, we are going to edit the qmail-smtp run file afterwards. I still go through this process because I am not a good script writer and I like to see what someone else thinks is going to be a good run script.

Now, go into /var/qmail/service/smtpd and we will change the run file:

Copy the following into run:

and replace VCHKPWUID and VCHKPWGID with the appropriate UID and GID for vchkpw. Also replace mail.yourdomain.tld with your domain or fdqn of the server.

Now, create symlinks to the /var/service/ directory for qmail, pop3d and smtpd

There are a number of ways you can proceed. Either start each of the start scripts you have installed under /usr/local/etc/rc.d manually or reboot the system. The next chapter covers the basics of adding and removing domains/users and mailing lists.

3. Administration of vpopmail and ezmlm-idx

3.1 Adding/Removing Domains/Users

vpopmail offers a very easy interface with which you can add and remove both domains and users on the fly. The commands to do this are found under /usr/local/vpopmail/bin. If you run the commands directly with no options, the command will list the available switches and options that must be used with that command. Inter7's documentation is also a good source for information.

Add a domain:

Add a user to that domain:

Create an alias domain:

Change a user's password:

Delete a user:

Delete domain:

There are also ways to specify directories other than the default (/usr/local/vpopmail/domains) to store the virtual domains and user files. I have found, however, that this isn't necessarily a better way to manage security because the domain has to still have vpopmail:vchkpw read/write priviledges, so until I can find a way to give the user AND vpopmail access to the domain information/users and not others, I see no reason to use this option.

3.2 ezmlm-idx mailing lists

These set of commands are taken directly from Oliver's howto - I have yet to use mailing lists although I have always had ezmlm-idx installed.
To set up a moderated list (TEST) whose moderator can be reached at user1@domain1.tld, do the following:

3.3 Using SMTP-AUTH

The whole purpose for rewriting this howto was that I wanted to include smtp-auth into the mix. I have constantly had troubles with the courier-imap hack that allowed for selective relaying, so I decided a better way to open up the smtp server for relaying for my clients was to set up smtp-auth. In order to use smtp-auth (especially for those using imap folders and cannot open the smtp relay via the old hack), you simply need to set your email client to use smtp-auth with one of the usernames and passwords of one of the domains you have installed. Remember, a valid username is user@domain.tld and not simply user. I use Mozilla Thunderbird as my main email client these days and have not had any problems using smtp-auth. The only issue I have is that it slows the sending of emails because the authentication adds a delay to being able to send out the email. I would rather have the security, however - and perhaps it is an issue that I will find some solution to at a later time.

4. Web Administration

This is the section that I have been long waiting to work on. Now that I have a need for it - i.e. my own startup ISP that needs some remote-management interfaces - I have worked to get vqadmin, qmailadmin and sqwebmail working (all from inter7, of course).

I installed apache13-modssl from the ports and did do anything special to get it running other than making sure the ssl service started so that I could test running all of the above services via ssl.

4.1 Install apache13-modssl

To start apache13 with ssl activated:

I have had problems getting apache and ssl to start with the installed start script (/usr/local/etc/rc.d/apache.sh) and have yet to take the time to look further into the fact that neither apache nor the ssl portion of apache will start automatically once the server has been rebooted.

4.2 Install vqadmin

Make sure the destination directory for the cgi's is a place where you want them. Otherwise, modify your options or the Makefile accordingly. I set everything to install to /usr/local/www/cgi-bin-dist. While I know this may not be the proper way to do things, I wanted to have all of the web administration tools installed to the same directory so that I could first make sure they work. I will eventually modify this to fit my needs. Then make sure you follow the instructions on setting up .htaccess password protection to the directory where you will access vqadmin (it will not work without password restriction). You also need to modify the vqadmin.acl file to set up access permissions for your users. In my case, I only added myself to the passwd file (which .htaccess points to) and gave myself full permissions under vqadmin.acl.

4.3 Install qmailadmin

4.4 Install sqwebmail

I modified this installation a little so that it always runs over the ssl server. Although email is inherently insecure and open in nature, at least the web clients cannot be directly monitored.

Add the following to your root or system crontab: Again, if you are adding it to the system crontab:

You need to use the run scripts installed in /usr/local/etc/rc.d to test sqwebmail or reboot the machine.

There you have it... You should have a fully functioning email server.

While I have spent a lot of time managing and tweaking different qmail implementations - the last of which dealing with smtp-auth and tls - I have also found a good number of sites who do a better job at providing some information that I either don't understand too well myself or I haven't had the time to digest. If you are using my site as a guide to installing qmail with imap and virtual domain support, PLEASE take a look at the original sites from which I have based quite a bit of my work:

- Oliver Lehmann's qmail HOWTO - this is the beginning of qmail for me - the genesis of it all, if you will. With Oli's help, I got my first well functioning qmail server going. Of course, you need to be able to read German, but even if you can't, it is likely that you could stumble through his command line examples. Anyone who has used my HOWTO owes Oli a lot of credit.
- Qmail Rocks dot org - this has been my latest weapon of choice as it is obvious that Eric Siegel has done his research in compiling an extensive and powerful qmail installation. His site goes quite a bit beyond what I would have expected and helps do a lot of the work for you.
- Shupp.Org - Bill Shupp is the source of my last modified howto. Unfortunately, this howto no longer works - actually, everything works except the smtp-auth portion. I have spent the past two days trying to figure out why and can't... Alas, this is likely NOT BILL'S FAULT, but something that I am not doing correctly. I am going to add a disclaimer to the top of my recent howto for that reason. Still, Bill has come up with a fantastic patch that worked on one of my machines and is the one having allowed me to do smtp-auth in the first place.
- Dan Bernstein's Page - of course, this list would be without merit if I excluded Dan's site. He is the reason we are even here. I honestly considered changing to Postfix recently because of my problems with smtp auth, but I would have thrown away years of work in understanding how qmail works. Plus, there is one fact that remains - I have NEVER had a qmail installation fail. NEVER. I have also NEVER had a qmail installation get hacked. Never is quite powerful considering I have been using qmail for 4 years on 10 different servers. Some of those servers have been heavily punished by my ineptitude... but qmail has continued to run as advertised. So, while I have my complaints (i.e. the necessity of patches upon patches in order to get certain services to work), I couldn't have done it better and am not about to take the chance that Postfix will set up the need to spend yet another 4 years learning another system.

A few days ago, I finally updated the qmail howto that I had been maintaining for a few years to notify people that they are better off going to lifewithqmail.org for more accurate information on how to install and/or maintain qmail. I have not done the job of a howto writer in keeping the document accurate or up-to-date. A recent post on the OpenBSD misc mailing list made me think more philosophically about providing howto's and their inherent problems. Software developers work hard to document their programs. Their documents are designed to guide you through the installation and configuration of the software. A howto allows you to skip all of this in the hopes of providing you an "easy out." That's what it comes down to - you are trying to get at the easy way out of learning a piece of software in order to install it as quickly as possible - and this has major caveats. While trudging my way through learning how to install and configure postfix (and the kitchen sink), I found a great number of howto's that were either out of date or inaccurate. It was unnerving the amount of bad information was available on configuring amavis-new in particular. If I had simply gone to the information provided in the software readme's and install guides, I would have found up-to-date information that I was looking for. If the software's documentation was poor, the worst thing I could do is go to the howto to see how to get it to work. Why? Because if the software programmer can't maintain or get accurate documentation written, how well do you expect the software to work? I'm not trying to slam the hard working software developer who struggled to finish a project but had a hard time writing documents (how many writers are good programmers and how many programmers are good writers?), but personally, if your documentation doesn't explain how things work so that I can install them properly, then how do I expect to maintain it such that my server doesn't get bombed with a bug or some other sort of problem.
There is a difference between a guide and a howto. The name howto implies that you will need everything to get started. Perhaps. You are missing one very important element: knowledge and understanding of how things work and why they work the way they do. I learned a very important lesson last year when I inadvertently updated perl on my email system and completely destroyed the SpamAssassin installation. For hours, my server was throwing mail around queues and didn't know which way to take things. I, because I had never taken the time to understand what the hell I was doing, almost had a heart-attack as customers started calling to complain because they couldn't receive email. I vowed after that to never install something without at least a precursory knowledge of the software I'm implementing and how it works. Yesterday I spent the better part of 8 hrs. trying to figure out why amavis-new and SpamAssassin weren't tagging obvious spam. It was because of one line @local_domains_acl = ( ".$mydomain" ); that was missing in every freaking howto that purported to explain how to install postfix and the kitchen sink (well, except one). I figured this out by shutting down the caffein flow to my veins and taking a long, hard look at the amavis-new documentation - and there it was... right in front of my eyes.
So, the next time you decide to use a howto to configure something to be used in production, be under advisement that you do so not only at your risk but the risk of all those you are supporting.*

*This doesn't mean I'm going to stop providing "notes" on this website, but please be forewarned - none of those are meant to provide a howto anymore <wink>.