Email not dead, just perspiring a lot

EMAIL MIGHT have problems, but in my opinion, despite what Tom Henderson said here, it is far from dead.

The big problem with email is that the structure of the header was designed so that anyone with half-reasonable software could become part of the message exchange system - and that is exactly what has happened.

The documentation, which supposedly defines the standards for all traffic on the web, is a set of RFCs which are listed here and can be found in full here. In my day RFC meant "Request For Comment" or "Request for Change" but have somehow become the official standards - and perhaps that is symptomatic of the problem.

The standards for email are number 822 and 2822, the latter of which tried to tighten the extremely loose standards that were described in the former.

With all email, any machine that performs any task for that email writes a header before forwarding it to the next machine. Mail headers are layered like an onion with the earliest header being the one which is closest to the actual body of the message. Importantly, each machine does not attempt to verify any previous headers but simply adds its own before sending the entire bundle of headers plus message.

The greatest problem is that the machine receiving the message asks the sending machine for its name, and the sending machine can tell it anything. The receiving machine does record the IP address of the sending machine in the header and in most, but not all cases, the receiving machine will do a reverse lookup of the IP address to find the name as it is recorded in one of the major IP registers around the world. This reverse name is usually recorded in the message header if it can be found.

The problem is that there is no verification that the named claimed by the sending the machine actually matches the name found by the reverse lookup.

It is this failure to verify and the lack of strict standards and methods, as highlighted two paragraphs above, that have caused a lot of the problems with spam.

If my email software was configured in a certain way, I could send email and tell my ISP's system that my machine was another one entirely, and it would accept that and put that in the new mail header without hesitation.

For example, one spam email from my current collection has the header Received: from 62.2.95.11 ([168.115.72.95] and in this header the sending machine is claiming to have the name - not address - of "62.2.95.11" but the receiving machine is accepting the message from IP address 168.115.72.95, for which it has no name.

Had validation been used in a reasonable manner on the receiving machine, it should have rejected this message on the basis of a failure to confirm the name.

One common problem that we have with any notion of validation is that many machine names will vary slightly from those names found by a lookup of IP address. Again as an example, a friend regular sends me emails with the following line in the header Received: from gate.abcxyz.com (fw.abcxyz.com[###.###.###.###]) Where the machine uses a different name to that recorded in the IP registers.

The above example uses the common convention of enclosing the IP address within square brackets and then enclosing that with the reverse lookup name in round brackets. If you read the RFCs carefully you will see that even this is not a mandatory requirement and, sure enough, some popular mail routing packages do not adopt this method.

Quite frankly, the Internet standards applying to email are a mess and they need to be radically overhauled - but enforcing changes on the millions of email packages and the mail router packages used by ISPs and others will be almost an impossible task.

We just might be able to adopt new standards but only if new software - or new versions of existing software - are made available in a very easy manner. Frankly I can't see it happening.

There are lessons to be learnt here about the need to make standards strict from the outset and perhaps relax them later rather than have them too lax to start with and later try to tighten them.

Despite all these problems, there are steps that can be taken against the spammers who often exploit the "flexibility" of the Internet standards and in fact some ISPs are already taking such steps

. The problem that we have is that spammers can be divided into three groups, those professional spammers with their own mailing packages, the individuals who spam via their own internet accounts and the individuals who exploit poor network security and send email via some other machine whose owner is unaware of what is happening.

The problem with legislating against spamming is that we need to be sure of the origin of the spam. This is easy with professional spammers and people using their own accounts, but it can get difficult where network security is being broken and unknown persons are responsible sending the spam.

For the same reason the notion of putting a monetary fee on each email has some advantages and the disadvantages that it is difficult to implement on a worldwide basis and it could punish those with poor security. (Okay, maybe they need some incentive to correct their problems!)

The solution to the issue of network security is often simple - use some kind of software or hardware firewall to keep the spammers out. My experience is that there is an urgent need for these tools in China and Korea because those countries are currently the source of a large proportion of spam but if memory serves, a recent report said that many PCs in those countries have poor security. Perhaps there is a market there for Kerio and ZoneAlert, the two common software firewalls, but versions in the local languages look like a must.

For the last four to six weeks I have been looking at the headers of most of the spam emails that I receive and where practical, I will report the spam to the ISPs or even the companies who are sending it.

Following my reports, one company in the USA discovered that a SMTP relay had been placed on their voice-mail system and a company in Taiwan appreciated being told about a problem with their network security. In both cases they plugged the holes and apologised for the trouble it had caused.

The reaction of ISPs is often an automatic email but there have been several instances of friendly and encouraging statements.

One Dutch ISP stated, "Depending on the severity of the case, we may or may not discontinue service to the customer. In cases where we do not discontinue the service, we may, at our discretion, decide to send the user an official warning." And they even had a list of IP addresses that had recently been "secured".

A Canadian ISP said, "Based on the information provided, we have identified the offending computer and will take appropriate action(s). These actions may be:

- Issue a warning by email indicating a complaint has been registered
- Issue a warning that service may be suspended if activity continues
- Suspend or terminate [our] connection to customer

Many ISPs, in many different countries, made it very clear that their contract contains clauses about "acceptable use" and that they intended to enforce those conditions.

An Italian ISP said rather ominously "…your report was registered, due disciplinary measures about our customer are on the way."

One in Hong Kong went so far as to say " We deeply regret that because the configuration of some of our clients' mail servers are still incomplete, these servers are being taken advantage by spammers as a tool of sending or relaying e-mails. Once the incidents are identified and verified, we would notify our clients right away."

A Polish ISP was quite happy to update the RIPE register (the IP register serving Europe, Russia, the Middle East and north Africa) when I told him that there was no contact address for reporting spam.

One very interesting approach that dealt with spam pro-actively was from an Australian ISP who stated " [Name] has measures in place to prevent spamming from taking place within the [name] network by preventing users from sending too many emails at any one time."

This is an approach that has my vote, but as before there is no one-size-fits-all when dealing with spammers and this would do nothing about the professional spammers with their own large computer systems and mail routing software.

It really seems that email as we know it will be with us for a long while yet because any change would be very difficult to implement. This doesn't mean that spam will continue to be with us, but reducing spam will require a multi-pronged attack so that it can be prevented from entering the system.

If we can identify and correct problems with networks security, and if the ISPs will continue to take action against customers who fail to comply with clauses in their contracts, the only real source of spam will be the professional spammers and these can be suppressed by legislation if the politicians have the will to do so.

Far from spam taking over the web, I believe that the battle against spam is only just beginning and that within six months it will be spam that is dead, not email. µ

John Mclean wrote Web Mail Tracker, and there is an updated version of this on our web site