News Stay informed about the latest enterprise technology news and product updates.

There's logic in moving beyond Bayesian filters

Bayesian filters are effective spam fighters, but they are best used as a last line of defense against unwanted e-mail, experts say. To get more aggressive, try DNS blacklists, whitelists and a relatively new method called URL-based filtering.

Experts say the best way to deal with the constant barrage of unwanted commercial e-mail, or spam, is to set up several layers of defense at various points in your company's messaging infrastructure.

Conventional thinking holds that the most effective way to avoid spam is to examine the body and subject line of incoming messages

Congressmen don't know how bad the problem is. They don't get the spam, their secretaries do.
Daniel V. Klein, consultant,
with e-mail filtering programs that employ Bayesian logic, which uses knowledge of past behavior to decide which e-mails to accept.

Now, however, experts and e-mail administrators are increasingly coming to the conclusion that Bayesian filters, while effective, should only be used as the last line in a multi-tiered defense against spam.

A company's overall antispam strategy, they say, should include filtering e-mail at the IP level, comparing incoming messages against real-time domain name system (DNS) blacklists, and a relatively new method called URL-based filtering.

Ex-Lotus CTO: Look to IP level

Nick Shelness, a research analyst with Ferris Research and a former chief technology officer for what was once Lotus Development Corp., explained that attacking spam on several fronts all but eliminates unwanted messages and significantly decreases the number of false positives, or valid e-mail that is inadvertently eliminated as spam.

"I think that when antispam approaches first arose, they were kind of naive and said that all we have to do is examine the message and we can figure out if it's spam or not," Shelness said.

For more information

Definition: Bayesian logic


Article: Market for antispam software red-hot
"But how do you make sure you find all the spam without misidentifying good mail?"

Shelness said new spam-fighting software is being introduced that actually filters e-mail at the router, or IP level. The software is deployed and gradually maps out the IP addresses in terms of where good e-mail and bad e-mail is coming from. Under this method, system administrators can identify high-volume sources of e-mail before it ever hits an e-mail server.

Admins can then block or slow the influx of e-mail from those sources at the SMPT level. Shelness said the nice thing about this level of defense is that once an e-mail is blocked, a notification message is sent to the receiver, who can check to see whether it was a false positive.

A URL-based approach

The second-to-last line of defense against spam, before Bayesian filters, should be URL-based filtering, Shelness said. Under this method, software looks to see if there are hyperlinks embedded in the e-mail. If there are, e-mail filtering software follows the links to find out if it will lead or redirect to the site of a known spammer.

Michael D. Osterman, principal analyst of Black Diamond, Wash.-based Osterman Research Inc., cautioned that DNS blacklists are only effective when the lists are constantly updated. He said the biggest problem associated with DNS blacklists is that often, legitimate e-mail senders are labeled spammers. Getting your company off a DNS blacklist can be a daunting task, he added.

"The best spam filters use a combination of techniques," Osterman said.

Osterman explained that companies have three options when it comes to deploying e-mail filtering programs. It can be deployed at the server level, on the desktop, or companies can outsource the task of filtering spam altogether. Outsourcing, he said, can be highly cost effective for smaller companies.

"If you get into really big companies it is probably cheaper to outsource," Osterman said. "But the real bang for the buck comes with the small company."

Empower your users

Finally, Osterman said that it's important to let end users have access to the mailboxes or files where quarantined or filtered e-mail is stored. This way, they can look for false positives if they wish.

"Give the end users control over their own whitelists and blacklists," he said.

Daniel V. Klein, an independent consultant, proprietor of a small Internet service provider (ISP) and a longtime antispam crusader, agrees that a multi-layered defense is the best method of attack.

He said his ISP employs four spam filtering methods, including Bayesian logic, DNS blacklists, URL-based filtering and his own personal DNS whitelists, which are databases of valid senders of e-mail.

Write your congressman

Klein said he believes the federal Can Spam Act, which went into effect Jan.1, is doing little to deter spammers from sending unwanted mass e-mails because the culprits are too difficult to track down. The law stipulates that people who send commercial e-mails with deceptive subject lines -- or those who fake sender addresses -- could be punished with prison terms.

The consultant urges e-mail administrators and anyone else concerned about spam to write their congressional representatives or the Federal Trade Commission and ask them to investigate new ways of combating junk e-mail.

"Congressmen don't know how bad the problem is," Klein said. "They don't get the spam, their secretaries do."

Dig Deeper on Exchange Server setup and troubleshooting

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.