E-mail is dead, long live Spam

E-MAIL WAS truly the killer app, along with the Web, that made the Internet what it is today. A barren wasteland of adverting, conspiracy theories, and trite urban legends passed on as fact. Anyway, e-mail truly revolutionised our society in a very short period of time. It allowed us to communicate in ways never imagined. Brief notes to friends, relatives, or business colleagues can be passed around the world almost instantaneously, essentially for free. Originally designed for scholarly types to keep abreast of research, it has become a huge and powerful business tool. I could go into detail, but there is really no need. We all know how beneficial e-mail is within the business world.

Spam has recently been on my mind because of a strange set of circumstances. My personal e-mail account is almost devoid of Spam, I've taken steps to keep that from happening, and my domain provider has a pretty decent setup for blocking it. Even when it does mark something as spam it still forwards it to me, encapsulated, thus saving me from false positives. My work e-mail I use for nothing other than mails to clients and colleagues, so I had little spam there as well. Then a co-worker left, and I was chosen to have his e-mail forwarded to me. Now this guy has apparently been asking spammers to be put on every mailing list there is, because his account is just inundated all day long. So this, along with some other problems that I will get into, has brought Spam to my attention anew.

When Spam was first introduced to the world is a matter up for debate.Ggoogling "History of Spam" will give you a multitude of interesting links. Two are here and here . Regardless of when or how it was introduced, it's here and that's all that really matters. Just like nuclear weapons, once invented, they can't be un-invented. But spam has now infected the system of e-mail, made it sick, and is in fact killing it. Strangely what is killing it is not so much the Spam, but rather the defences raised to protect us from it.

First, Spam was simply an annoyance. The fact that it continued to grow is based solely in the stupidity of people, a market that is infinite in its depth. Those of us that really weren't interested in buying Viagra online, or get a second mortgage at 93% interest or whatever, were annoyed. At first it was one or two or three a day, now it can easily reach into the thousands of spams a day. Then hackers took up using Spam as a virus vector. Now we had a serious reason to block Spam. It became a security risk to networks around the world, networks that were the backbone of companies, companies that if their network were weakened would go out of business.

Doctors say the case is terminal
Recently my Aunt sent me an e-mail, which, like a good nephew I replied to right away. A week later she called and asked why I wasn't replying to her mail. Turns out that AOL was killing mail coming from blocks of IPs that include my domain provider. But most importantly, they don't do any kind of notification, to either side. They simply kill the mail, poof, gone, vanished, never to be seen again. It becomes an ex-mail. The same thing has happened to me in a business role, but instead it was due to the fact that that I had sent a .doc attachment from outside the AOL domain, and their rules find this unacceptable. But it's not just AOL, all sorts of organisations are doing this.

This is the first serious symptom of e-mail's malaise. The doctors keep saying that treatment will fix it, and e-mail will be well again. But once the patient is out of the room the doctors talk about how long the patient has left. They know the case is terminal, they hope for a miracle, but sadly miracles don't usually come in time. All sorts of treatments are discussed - paying a very small amount of money per mail, having the e-mail client do some computation, meaning regular e-mail will take a little bit longer, but sending huge amounts of mail then takes a long time. All of this is the modern day equivalent of bleeding with leeches.

Communication systems rely on one thing and that's reliability. Once reliability is compromised, the system of communication becomes useless. What would you do if every once in awhile you tried to place a call, only to have your friends phone service say, "Based upon the time you're calling we suspect you're a telemarketer, so we aren't going to let the call go through". Or if the Post Office started shredding your mail because you had the same zip code in you address as some fraudster. I can't find any stats on the amounts of mails lost daily, but it's much higher than it should be. Beyond the ones that are lost, are the e-mails simply not sent. I've completely given up sending e-mail to anyone within the AOL domain. E-mail continues based on pure inertia but it is far from enough to keep it alive indefinitely. Let's examine some of the cases where Spam blockers are triggered, and see how we are destroying its usefulness.

Recently a friend sent me an e-mail where she was complaining about something her mother had done wrong with her mortgage application. What had triggered the Spam filter were the use of the word "Mortgage" and her use of capital letters (shouting or accidentally hitting the caps lock key). Now while I know that the word "Mortgage" is used in many Spams, it is also a fairly common word to use in discussions with other people, seeing as how there are millions of people that actually have mortgages. In addition, the use of caps, while not as common, is frequently used as a way to show irritation.

Yesterday my friend's mail server was going to be offline for a couple of days (don't ask), so she asks me to forward those e-mails that she hadn't received the day before. (probably related, and we'll get into that later). So I did what any user of Outlook would do; I selected those e-mails and hit forward. This creates a .eml attachment. Well two of her other mail servers denied them as being a potential virus vector, due to the .eml attachment. Another file type that is banned by some e-mail servers is .zip files. Again, a fairly common thing to be doing. Sending a friend or associate a series of files zipped up for convenience, or zipping them up to save bandwidth. Forwarding and attachments, of all kinds, are a very common usage of e-mail. It is quickly becoming un-usable.

Linking to certain types of websites is also verboten nowadays. Links to things like .asp sites can also lead to downloading forms of malware and viruses. While this is less likely to be something important, it's a bit unnerving anytime you basically get an e-mail that says "Go away you nasty spammer" because you sent a link to a friend. This is seriously a case of throwing the baby out with the bathwater. Surely many kinds of links, once clicked, can lead to all sorts of undesirable results. However, blocking mail because a link could be damaging is asinine. Oftentimes I need to send a link for a program to someone. While I understand the possible sinister uses of links like this, it is also an important way to pass things on to other people, particularly computer neophytes who need to be led by the hand to fix problems. Often when I send a link like that I am trying to solve a malware problem, not cause it. As such by blocking links we are not only not helping the problem, we are actually making it worse.

Kill everyone with brown hair
In the end, rules for identifying Spam are the equivalent of saying "Most criminals have brown hair, so we should kill everyone with brown hair". The result of all this is that higher and higher percentages of e-mail are quite simply disappearing. Most of this is Spam, but every day more and more legitimate e-mail disappears, e-mail that is completely valid. Most of it is business mail, since the verbiage of business mail is closer to that of Spam, the false positives hit where it is the most damaging.

There is also another area where damage is being done. My friend with the mortgage mail and I send e-mail to each other all day long. Sometimes hundreds of messages a day. Basically we're sending mail while we are at the computer, and we are at the computer whenever we are awake. When there is a hiccup in our mail servers we notice it almost immediately. More and more often we are seeing delays ranging from hours to days in our mail getting through. The delays are in both of our servers, both independent with different filtering rules. Those supporting the systems give different reasoning for the problems at different times. Always, they say it's fixed, and then days, weeks, or months later it happens again. The frequency of these outages is steadily increasing. The underlying reason for these outages and delays are really all the same. The load on these servers is already pretty heavy just moving the mail through. Nowadays that's not all these servers have to do. They also have to apply filters to the mail, checking for Spam and trying to block that Spam. This takes clock cycles, and when you consider the amount of e-mail and Spam floating around in the ether, it's a large amount of clock cycles indeed.

These filters are in essence trying to do the one thing computers aren't good at. Pattern Recognition. It's not good enough to just cut any mail that uses the word "Viagra" it has to try to figure out the context. However, computers can't do that. So people create rules based upon looking at the mail that is out there. Unfortunately as the filters become well tuned, the Spammers can see the traffic reduced, so they change the text, and more rules are required. Then more clock cycles are required to process the additional rules. Yahoo in particular can show this. The boffins at Yahoo are constantly trying to do something about the Spam their users are inundated with. During certain times I get 10 spams a week, at others I get 10 an hour. This shows the Spammers constantly fine tuning their crap to pass the filters. At times the Spammers go on binges. Having taken over more computers here and there, they can increase the amount of spams. Since their business is based entirely on the basis of volume, the more they send, the more dupes respond to them, as such they are perfectly happy to increase the volume of spams to the limits. When these spam waves happen, mail servers around the world are brought to their knees trying to process the filtering rules on the hundreds of millions of e-mails floating around the ether. It's a losing battle. Spam is so prevalent simply because it's cheap to the sender. Each spam is worthless in more ways than one. It costs nothing to send, however, there is a cost on the other end to block it. The small amount of spammers can create countless amount of spam with no added cost per e-mail, but those administrators of mail servers need to purchase more and faster equipment in attempt to block spam and still let the occasional legitimate mail through. It's a recipe for disaster.

Blacklists don't work
So why do I say this is predicating the end of e-mail? In the past six months these problems are on the rise, along with the amount of Spam. Mail servers are taking serious hits in performance; even those mail servers of large computer companies, particularly the mail servers of large computer companies. Filtering obviously doesn't work. A significant percentage of the spam I receive, for some odd reason, have the name "John Elway" in the subject line, not to mention "Viagra." By the very nature of e-mail these Spams are passing through multiple servers on the way to your mail client, and they still pass through. Yet at the same time, mails that are obvious, to any casual reader, not spam are filtered out. In the above examples with my friend, whom I pass 100's of mails a day with, and have for years, you would think the servers would know that mail between the two of us is not Spam. Yet many times my outgoing server, or her incoming server still mark it as spam. Blacklists obviously don't work, but white-lists will work, simply by the fact that white-lists are based upon human interaction. Yet few Mail server administrators use white-lists, preferring automating filtering. This is a strategy that will never work. Computers simply do not have the proper insight to effectively filter out Spam.

There are vast amounts of open computers in this new Internet, with the multitude of computers hooked to the Internet 24/7, with little or no security. Most running Micro$oft software that is almost begging Black hats to take over these systems. Overall the very nature of today's Internet is just a vast fertile plain for Spammers and Hackers. Yet the software manufacturers obviously don't want to do anything that could stem the tide. Simple things like OS software, or mail clients, that actually monitors outgoing traffic, so that the user can easily see if their computer has been taken over. Like not allowing programs being run from outside parties, without the users consent. Like not allowing outside users access to the file structure on a users machine.

People are dumber than you ever suspected
Have you ever tried to report suspicious activity to an ISP that is not your own? Currently my company's dial-up provider, Bellnet, has a serious problem. When I log in, my computer easily receives 20+ intrusion attempts logged (and unknown not logged), but do they do anything about this, no. Hell they don't even respond to mails reporting the problem. Port sniffing should not even exist, much less be a problem. Having your computer sniff a port on another computer should bring down the entire might of your ISP on your head. Instead they couldn't care less. OS manufacturers should consider security holes in their software their #1 priority. Most importantly, everyday users should never respond to spam, ever, under any circumstance. The simple fact of the matter is that if anyone is trying to offer you a deal too good to be true by a mass mailing, guess what. It's a scam. These reptiles would go out of business and stop sending Spam if they never got a response, yet still they get enough to make the expenditure of their time worthwhile.

Sadly all of this is for naught. Because the only human nature that can be counted on is that people are not only dumber than you think, they are dumber than you can imagine. The problems that we currently have will continue to increase over time. Interruptions in service, lost mails and the like will increase, until e-mail becomes so unreliable that its value disappears. Don't get me wrong, I love e-mail, and will continue to use it for along time. But it is becoming more and more frustrating. There will come a time when I say forget it, it's not worth the hassle. Sometime after that I suspect it will become impossible to use.

Already the value of the web is steadily decreasing. I suspect that the entire idea of the Internet as we know it will become entirely obsolete, and will be replaced by something new. What that is, I have no idea. Whether it will be a major shift, or a minor one I don't know. But I will say this, there is money to be made in making e-mail and the Web more useful. µ