Answers in this category:
ASSP WIKI
Requirements
Get a working Perl
Download and unpack the ASSP scripts.
Run the ASSP program and point your browser to http://127.0.0.1:55555/.
Rebuild your Bayesian database.
Possibly change your existing SMTP server’s port.
Test that it is working.
Schedule regular updates for the Bayesian database.
Instructions for use for your end users.
Periodic Maintenance
When I try to send mail I get the error "relaying not allowed". What do I do?
When people try to email me they get the message, "Relaying not allowed." What do I do?
How can I learn Regular Expressions?
How do I use the email interface?
What command-line options does ASSP support?
How do I use ASSP's Whitelist-only mode?
How do I help ASSP recognize mail from traveling users?
Switch Off Test Mode.
What can I do about all the spam I'm getting?

Periodically you’ll need to rebuild the Bayesian database based on your spam and non-spam databases. This is very CPU and memory intensive. It takes about 5 minutes to run on our server, but could be slower if you have less ram, slower disks, or a slower CPU.

ASSP is implemented in Perl, so you’ll need a working Perl on your ASSP server. See http://www.cpan.org/ports/index.html for Perl for your computer. ASSP uses only IO::Select and IO::Socket which come preinstalled on all Perls. If you plan to run it as a service in Windows you’ll want Win32::Daemon from http://www.roth.net/perl/Daemon/. Install Win32-Daemon this way:

   perl ppm.pl install http://www.roth.net/perl/packages/win32-daemon.ppd

To filter spam the proxy uses a modification of the Bayesian statistical approach outlined in articles referenced here: http://www.spamarchive.org/more_community_resources.htm, but in short it looks at words and phrases used in unsolicited email, and words and phrases used in your site’s normal email traffic. When a new message arrives it compares the words and phrases in it with those from the statistical comparison, combines the individual probabilities and decides if this message is more like unsolicited email or like normal email.

For that to work it needs collections of spam and non-spam messages. There’s folders [directories] (specified in the configuration) which contain these collections. It works best if there are between 5000 and 10000 unique messages in each collection.

This program also makes use of a whitelist – a list of email addresses which would generally never send you spam. It maintains this list automatically.

Because the ASSP server becomes the public interface to your email system, it also must enforce email relaying – an open email relay is a source of great joy to spammers, young and old. ASSP also recognizes authentication confirmation from your mail transport’s SMTP server.

If you plan to run it as a service in Windows you’ll want Win32::Daemon from http://www.roth.net/perl/Daemon/. Install Win32-Daemon this way:

   ppm install http://www.roth.net/perl/packages/win32-daemon.ppd
 

Security note: Because the email database is stored without encryption you’ll want to consult your local security expert to find the best way to protect your user’s confidential emails. It probably involves setting the access rights to the notspam directory to only allow access to administrators and the assp process.

In your assp directory you must create directories like this:

 assp/spam
 assp/notspam
 assp/errors
 assp/errors/spam
 assp/errors/notspam

						

At a minimum you’ll want to change values for "Web Admin Password," "Accept All Mail," "Local Domains," "Spam Error," and "Spam Addresses." The password for your server is posted in bold on the internet for EVERYONE to see, so if you don’t change it, you deserve what you get. Also remember to press Enter or click the button at the bottom to register your changes – simply clearing a checkbox doesn’t send the change to ASSP.

You want your mail to flow like this:

 Internet -> ASSP -> Mail server -> Clients

 Clients -> ASSP -> Mail server -> Internet

						

If you have an antivirus proxy, do it like this:

 Internet -> ASSP -> Anti-virus -> Mail server -> Clients

 Clients -> ASSP -> Anti-virus -> Mail server -> Internet
 

						

HOWTO: General instructions for any non-SMTP mail server (MS Exchange, Lotus Notes, etc)

 Clients -> Exchange -> ASSP -> Mail server -> Internet

In the config you’ll see "Spam Addresses" – these are addresses of recipients at your site that only receive spam. You can put spambait on your website, or in Usenet posts, but most email administrators find that they continue to receive email for users who have been gone for months (years sometimes). These addresses are useful for filling your spam database, and for mail addressed to multiple recipients, for recognizing spam.

In *nix you’ll want to add ASSP to your local rc scripts so ASSP starts on system boot. Check the "As a Daemon" configuration box.

In Windows 2000 you’ll want to install it as a service: 1)Check that you have "As A Service" checked in your config; 2)Run the

 perl addservice.pl –i c:\assp\assp.pl c:\assp 

						

Once you have at least 400 messages that are properly categorized, do this:

 perl rebuildspamdb.pl

This will build your own site's customized anti-spam dictionary. The program will print a bunch of dots and when it’s done make a file called spamdb. This is the file that ASSP uses to qualify spam or non-spam.

  telnet 127.0.0.1 25

						

Note that you can also test your configuration by setting ASSP to listen on a different port (like 125) and connect to your SMTP server on port 25 and then set your email client to connect to port 125 to send email.

If all is working properly you should see email connections logged in the maillog.txt file (also visible from the admin page). You should see also see whitelist additions in that log. You should also see mail files being saved in the spam and notspam directories.

  perl rebuildspamdb.pl

						

We are using one of the most advanced SPAM filters on the internet. It continually adapts itself to detect spam. For the most part it does a good job, but occasionally a spam slips through.

When you receive a spam, you have a choice to either just delete it and move on, or report it to the spam filter to help improve its performance. To report the spam select the message in your inbox, right-click and select "Forward as attachment." (If your email software doesn't support "Forward as attachment" simply forward the mail.) Address it to assp-spam@mydomain.com and click send.

You've done your duty and added the evil spam to the spam filter's collection.

Just to remind you of the basic operation of our spam filter:
1) Anyone you email will never have a message blocked.
2) Never email a spammer -- you validate their address authorizing them to spam us.
3) Don't forward spam -- it makes the spam filter think we like it.
4) Report spam by forwarding it to the assp-spam@mydomain.com address.
5) The spam filter keeps track of mail we send and spam we receive -- if an incoming message is not from someone we've emailed and it's more like the mail we send than the spam we receive then it gets through. Otherwise it's blocked and the sender gets the message, "Mail appears to be unsolicited -- report errors to postmaster@ourhost.com"
6) If you become aware that mail you want is being blocked send the sender an email so their mail isn't blocked any more.

Check to be sure that rebuildspamdb is running as scheduled.

After a few months you may want to clear out some of the files in the errors/spam and errors/notspam folder, especially if these folders are becoming quite large. At some point I'll write a utility to do this automatically, but I haven't written it yet. For now it's probably safe just to delete the oldest files.

That a mail claims to be from one of your local domains does not allow it to be relayed -- this is easily spoofed and not useful as a security measure. Spoofing an IP address is more complicated in this type of environment, and generally relaying is limited by IP address.

Note that you can add entire groups of addresses: 10. represents 10.1.2.3 or 10.3.2.1 or 10.anything. Similarly, 169.254. will match any IP that starts with those quads.

If your clients dialup or are dynamically assigned from an untrusted pool, then the only relaible way to allow relaying is through AUTHENTECATED smtp, and your mail handler must support this type of authentication, and you must enable it in your clients. ASSP recognizes authenticated connections and allows them to relay.

Not all ISPs will allow their customers to connect to your SMTP port. Many block connections to port 25 (except to their own mail server) to prevent spam.

What is a regular expression? A regular expression is simply a string that describes a pattern. Patterns are in common use these days; examples are the patterns typed into a search engine to find web pages and the patterns used to list files in a directory, e.g., ls *.txt or dir *.*. For ASSP, the patterns described by regular expressions are used to search strings, extract desired parts of strings, and to do search and replace operations.

In basic terms, asd2 will match jjasd2 or asd2jj or jjasd2jj. A . matches any single character, so if you want to match a period put a backslash in front of it: \. Almost all punctuation has special meaning, so you need to put a backslash in front of it, unless you want the special meaning:

 . -- match any single character besides \n
 .* -- match 0 or more characters (not \n)
 .+ -- match 1 or more characters (not \n)
 \d -- match any number
 \w -- match any alpha-numeric or _
 [0-9a-zA-Z_] -- same as \w
 this|that -- match this or that
 

 assp-white   -- for whitelist additions
 assp-spam    -- to report spam that got through
 assp-notspam -- to report miscategorized spam

Assuming that your local-domain is mydomain.com, to add addresses to the whitelist, create a message to assp-white@mydomain.com. You can either put the addresses in the body of the message, or as recipients of the message. For example, if you wanted to add all the addresses in your address book to the whitelist, create a message to assp-white@mydomain.com and then add your entire address book to the BCC part of the message and click send. Note that no mail will be delivered to any address except assp-white@mydomain.com (and that won't actually be passed to your mail transport). Within a short time (depends on your mail server) you'll receive a response from ASSP showing the results of your mail.

To report a spam that got through, simply forward the mail to assp-spam@mydomain.com. It's best to forward it as an attachment, but you can just forward it normally if you must. In a short time you will receive a confirmation.

The process is the same to report a miscategorized spam, but send it to assp-notspam@mydomain.com.

Of course all these addresses can be changed to suit your sites preferences.

Also see this note How do I use the email interface with Exchange, Notes, or a RelayHost / RelayPort setup? if you are using RelayHost and RelayPort.

ASSP treats all these as the same. Any connection blessed by one of these will (a) be able to relay mail, and (b) be able to access the email interface.
1) mail from a "allow all mail" host
2) mail with a SMTP AUTH affermative response
3) mail through the relay port
4) mail approved for relaying by popb4smtp

 perl assp.pl /path/to/base 55556

Where /path/to/base is the path that ASSP will use as its install base And 55556 is the port on which to run the admin interface, overriding the default (but not a configured option). To override the configured option you must edit (or delete) your assp.cfg file.

Note that if you leave off a path/to/base then ASSP will look in the following locations (in this order) to try to find an assp.cfg and make that base:

 '.','assp','/usr/local/assp','/home/assp','/usr/assp','/assp'
 

If you want no bayesian filtering, only whitelist, then:
1) Check the "Reject All But Whitelisted Mail."
2) Check the "Don't log mail."
3) Check "Disable Good Hosts Antispam."
4) Delete all files in:

 a) spam
 b) notspam
 c) errors/spam
 d) errors/notspam
 f) other (if you used it)

						

Unfortunately, the from-local-address-to-local-address strategy is VERY COMMONLY used by spammers for exactly this reason.

If your mail system allows users to authenticate over SMTP, ASSP recognizes this authentication and you can use this to allow them to relay.

If not, the best solution for now is to use the "Expression to recognize Non-spam" (or WhiteRE) to make "passwords" for your road-warrior users. An easy way would be to look at their email signature(s) (if they use them -- and instruct them to use them if not). For example if they have a phone number in their signature, add that to the WhiteRE -- \(123\) 456-7890|\(123\) 123-1234 for example. You could also look closely at their email header to try to identify something unique to their outgoing mails that could go in the WhiteRE. Or just tell them to put NoSpam4Me in their signature and put NoSpam4Me in the WhiteRE.

I'm still hoping to negotiate a local user allowed on whitelist option. If I can convince/badger you into it, is it a difficult change? ;-)

--------------------

It's not really difficult, but it is highly ill-advised. Spammers routinely forge the from address to match someone in your local domain. Adding local users to the whitelist will allow a significant portion of spam to those users.

To do that, put the following in your "expression to identify non-processing mail" setting:

 from: .*(user1@localdomain\.com|user2@localdomain\.com)

Apply that change and any mail claiming to be from the listed users will be passed through with no processing.

You are now ready to really begin filtering mail.
1) Either turn off "Test Mode" or put something like "[SPAM]" in the "Prepend to Subject" option.
2) Clear the "Use Subjects as File Names" option.
3) do:

 perl move2num.pl -r

						

However, there's more you can do.

As legeslation is developing there are opportunities to sue spammers. This is not easy, but is the one of the best methods to end spam. Keep on top of local legislation. It is easy to search your maillog and identify the ip addresses of spammers. You can correlate that through http://www.openrbl.org/ or other geoip tools to identify if they might be candidates for a suit.

To learn about ways to fight spam start here: http://spamlinks.openrbl.org/spamlinks.htm

To learn more about the address spamming you start here: http://www.openrbl.org/

Note that the ip address sending you spam might be a hapless bystander caught in the crossfire of this war.

Good luck, and never give up!

Answers in this category:
I don’t want to block email, but I want to give end users the ability to sort or filter their email. Can I use ASSP to do that?
Can I use ASSP to scan for viruses?
I have some users who don’t want spam blocked How can I do this with ASSP?
Mail is getting blocked that shouldn’t be. How do I fix that?
Why isn’t ASSP blocking any mail?
Why is ASSP blocking all mail?
How do I use the email interface with Exchange, Notes, or a RelayHost / RelayPort setup?
Why is my whitelist empty?
My email client uses /sbin/sendmail to deliver mail. Will this work with ASSP?
I think ASSP blocked an important message. What do I do?
I'm still getting spam. What's wrong?
How can I change the text of the messages ASSP uses to reply to email submissions?
Why does every person who posts on a maillist get added to the whitelist?
I host maillists -- how does ASSP work together with ezmlm or other maillist software?
How do I set up ASSP to run with redundant MX domains?
When I send a spam/notspam report I see this in the logfile: RMabort: rcpt Expected 250 -- what's wrong?
Why doesn't ASSP work with TLS or secure SMTP?
How can I manually download the greylist?
What is the greylist and what does it do?
After a few hours ASSP stops allowing connections. How do I fix that?
Can I run a secondary MX without ASSP?
Why is ASSP running so slowly on my Sun / Solaris system?
What should I do about strange addresses on my whitelist?
How can I troubleshoot my ASSP system?
What should I do to run ASSP on an old / slow computer without much ram?
I've set spamlover addresses and I'm running in test mode, but I never see a "spam lover" message in the log file. What's wrong?
ASSP first ran as root, now I want to run it as another user; what permissions do I need to change?
ASSP is rejecting all mail with Relaying denied.
How do I use the dnsbl setting?

Answers in this category:
Can ASSP allow certain users to receive executable attachments?
I have ASSP running and it accepts SMTP connections but POP3 and IMAP don’t work. What’s wrong?
How do I add / remove / verify an addresses on the whitelist / redlist?
How can I make rebuildspamdb.pl use less RAM?
How do I completely empty the whitelist or redlist?
When I put a file name into the Pop Before SMTP Filename field ASSP stops accepting mail connections. What's wrong?
How do I disable the web administration interface?
HOWTO: rebuild the whitelist manually from the maillog?
How do I unpack the sample spam database in .tgz format?
The last line of my localdomains or relayhosts file is being ignored. What do I do?

You can add an address to the "no processing" list and email soley to those addresses will receive all mail, including executable attachments.

There is also an option to block executable attachments from all senders.

In versions prior to 0.2.0 you can use the list.pl script.

You can run the rebuildspamdb.pl script over the network on a different host than ASSP is running on if you're having trouble running out of available memory on your ASSP machine. However this requires that all the files in the spam, notspam, and errors collections must be read over the network.

Note that as of version 0.3.2 rebuildspamdb requires less than half the RAM of previous versions.

If you still are running low, and don't mind making the process slow down by a factor of 10, there is a line (about line 19) that you can uncomment to limit the RAM used to about 50mb. Remove the # at the beginning of this line:

 #$spamObject=tie %spam,orderedtie,"spamtmp";

						

1) Identify the file in your installation, probably whitelist or redlist.

2) Delete or rename the file.

3) In the web interface click "Show Whitelist" (or redlist)

4) Delete (or rename) the file again if it came back.

If you rewrite the POPB4SMTP subroutine for your mailserver, please post it in the HOWTO's section, or email it to the author to post.

 grep addition maillog.txt | awk -F":" '{ print $5 }' | sort | uniq | tr -d ' '

Then you can do this to fix it:
1) Stop assp.
2) Delete the whitelist file. (Rename it if you prefer.)
3) Start assp.
4) Open the web admin interface, possibly http://127.0.0.1:55555/
5) Click on the "update whitelist" option
6) Paste the list you made above in the box and click submit. Note that if the list is larger than 100k you should split it into smaller chunks.

 tar xzf asspsmpl-0.1.tgz

						

As a side note, I know of one researcher who uses a program like ASSP as a virus filter. If you put at least five copies of a virus in the spam collection, ASSP will probably block that virus the same way it blocks spam. The researcher even had reasonalbe success with polymorphic viri. However, ASSP has been tuned to recognize spam, and I make no guarentees that it will recognize viri.

Second, put samples of the type of mail that is being blocked in your notspam or errors/notspam folders so that the filter becomes biased to accept that kind of mail -- you have to run rebuildspamdb.pl before the changes take effect.

Third, look for a unique string or phrase to the type of email that is being blocked (perhaps your street address, city name, phone number) and add that to the "expression to identify nonspam" configuration option.

You can also use "whitelisted domains" to help mail get through the filter.

1) You have TestMode checked,

2) Your spamdb is empty, too small or ASSP can’t find it (check your log messages in maillog.txt at the startup),

3) ASSP thinks every message is local or whitelisted (check your log). This often happens if you have || or a | at the start or end of the whitelisted domains or "Expression to Identify Non-Spam."

1) Your non-spam collection is too small in proportion to your spam collection – add messages to it, or delete some spam and rebuildspamdb.pl.

2) You have something in Blacklisted Domains or Expression to Identify Spam that is too general -- use the analyze page to let ASSP tell you why it is blocking a message. Check for || or a | at the start or end of your expression.

ASSP is looking for it's command addresses at one of its localdomains. However your mail server will not accept mail for ASSP's addresses (and if it does, it will never be passed to ASSP). The work-around is to find a domain you never expect to actually send mail to, tell ASSP it is local, and use that domain for ASSP commands. So you could add spamreport.gov to ASSP's localdomains, and direct whitelist additions to assp-white@spamreport.gov.

ASSP's default setting for "Email From" doesn't work with Exchange -- change it to be "postmaster <postmaster@yourdomain.com>" instead of "ASSP <>".

Or you could just ignore the email interface.

The key is, to convince Exchange to pass it on to ASSP which DOES know about this magic names. It can be done by the following procedure:

1) invent a "dummy" domain, like "assp-nospam.org" (seems to be an available domain name)
2) tell ASSP that this domain is one of your "local" domains
3) tell your users to send their spam to "assp-spam@assp-nospam.org"; instead of "assp-spam@yourdomain.com"; or tell your Exchange-server to forward the later to the former. If you prefer the "forwarding" approach, you can pick a dummy domain name that is more unlikely to be ever registered (like foo123blob.tac), as nobody has to remember the name.

If you have ASSP on the outbound channel - as you should for the auto-whitelist function to work - and your Exchange is using the ASSP-relay-Port to deliver external mails, then the server thinks it is "external" and hands it over to delivery. ASSP thinks it's "local" and processes the magic-names as expected.

Works here on two different Exchange setups (5.5 and 2k)

(Robert Orso, 11-17)

If ASSP is set up correctly in the maillog.txt you should see something like this:

 Jul-9-03 14:40:31 10.1.1.90 <@mydomain.com> to: mike@yahoo.com local or whitelisted
 Jul-9-03 14:40:31 10.1.1.90 <@mydomain.com> to: mike@yahoo.com whitelist addition: mike@yahoo.com

If you don't see the first line, then you don't have "Accept all mail" correctly set up. If you don't see the second line then you don't have "Local Domains" set correctly (note addresses in the local domain don't get added to the whitelist).

ASSP gets whitelist info from your email client. It should look like this (substitute your mail server for postfix):

 Your client -> ASSP -> postfix -> queue -> remote delivery;
   and
 Remote mail -> ASSP -> postfix -> local storage -> local clients

With Exchange/Notes/RelayHost it's a little different:

 Local Client -> Exchange -> Local Client  [assp never sees these mails]
 Local Client -> Exchange -> ASSP (relayPort) -> ISP relay host -> remote delivery
 Remote mail -> ASSP -> Exchange (SMTP) -> Local Client

The easiest solution is to tell your mail client to use SMTP instead of sendmail. Most provide this option. Some clients have configuration options for both "Path to Sendmail" and "SMTP Server" -- in this case, make sure the "path to sendmail" is always blank (ie blank for your personal settings, and blank for your global server settings) and the "SMTP Server" is always set to ASSP's smtp port.

If you must, you can use ASSP's RelayHost and RelayPort settings to allow work around this, but you must also configure your mail transport to use a Smart Host, rather than directly deliver mail.

If the message in question hasn't been sent yet, email the sender so they're on the whitelist. Then it won't be blocked. If you don't know the sender's address, you can add a "whitelisted domain" or a "expression to identify non-spam" to try to ensure its delivery.

If it was "Bayesian spam" then you may be able to find a copy in the base/spam directory. Sort the files by date and look at the ones closest to the date & time of your message. Any text editor should be able to open the files. Files in the spam directory randomly can be overwritten, so the older the message, the less likely it to be there. Also, only the first 10k are stored, so if you're looking for an attachment, you're out of luck.

To send the message to its intended recipient do this: Copy the file to your hard drive, rename it to have an .eml extension (or whatever form your email client likes), make a new mail to the person who should get it, and attach the .eml file to it. Then click send. Of course you could make some note along the lines of, "Found this in the spam bucket -- thought you'd want to have it. If you email this person their mail will never be blocked. By the way, you owe me a beer."

Why isn’t ASSP blocking any mail?

Look in the maillog.txt to see what it says about the message. If it says "local or whitelisted" you should verify your "hosts to allow all mail" setting, or check if the sender's address was on the whitelist.

If the maillog.txt says, "message ok" then take the message and paste it into the "Analyze mail message" screen and click analyze. (There's detailed instructions at the bottom of the screen.) This will tell you how your message related to the spam database.

You will get spam for one of the following reasons:

1) The spam message is unlike others in the spam database -- Solution: put a copy in the spam or errors/spam directory.

2) The spam message is like others in your nonspam database -- Solution: double check your nonspam database to make sure you didn't get some spam in there by accident.

3) The message was from a whitelisted sender -- Solution: remove the sender from the whitelist if they shouldn't be there, also put a copy of the message in the errors/spam directory.

4) The message matched your "expression to identify non-spam" setting -- Solution: double check to make sure what you have entered there is what you want; Perl "regular expressions" are not just string matches. Read the help page on Perl regular expressions.

This text is easy to find by searching the source code. Just edit it to suit your situation.

 ReturnMail($this->{mailfrom},"Spam report: $sub",<<EOT);

						

 ReturnMail($this->{mailfrom},"Spam report: $sub",<<EOT) if 0;

						

1) For most maillists it is not unusual for someone on the list to mail another person on the list -- whitelisting them is appropriate.

2) Since spammers rarely use valid "from" addresses these extra whitelist entries will not likely result in your receiving spam.

3) If you don't hear from them again in about 90 days, they'll probably be trimmed of the whitelist automatically.

4) If you really don't like this behavior, add the list's "envelope sender" (you can find it in the maillog.txt) to your "No Processing" addresses. This will (a) prevent whitelisting, (b) not put mail in the spam, notspam, or other directories from the list, and (c) pass the list's messages through (ie not block them).

 The ASSP / Qmail / Ezmlm machine at 64.0.0.1 and 10.0.0.1
 A local client who manages the maillist at 10.0.0.2
 A maillist participant at 222.0.0.1
 Many other maillist participants across the internet.

						

If you don't trust your list members to post ham (not spam) this is good behavior. Alternatively, you could put the list address on the "no processing" list so that no filtering (or influence of the spam / ham collections or whitelist) happens. However unless you can tell ezmlm to deliver by SMTP rather than by sbin/sendmail there's no way to get all list participants whitelisted.

This error comes when ASSP posts its address to say whom the report mail is from. The default is "ASSP <>", but some mail servers, Exchange for one, won't accept an address in this form.

The address with the problem is not your address, or the address in the spam you're submitting, but the address that ASSP is trying to use to send mail.

Go to the web configuration page, under "Email Interface" and find the "From Address for Email" setting. To clear this error, you need to find an address that is acceptable to your mail server for ASSP to send mail.

Try just postmaster@mydomain.com or spammaster <postmaster@mydomain.com> or a valid local address or other variations until it starts working.

In theory one could use STUNNEL to still allow TLS connections to ASSP and then on to your mail transport. Also in theory one could use a version of openssl to add this capability to ASSP. If anyone does either of these please write me and I'll include it with future releases of ASSP.

If you are running 1.0.2 or less I reccomend disabling TLS on your server. TLS has been reported to cause problems with ASSP.

The file is currently about 140k, but its size will change.

ASSP downloads it about every 12 hours. There's no point in downloading it more frequently than that.

You can use a tool like wget to download the file, but you'll need to work out the naming scheme. This is the process you should follow:
1) Download the greylist file to a temp file
2) If the download failed, remove the temp file and quit.
3) Delete the current greylist
4) Rename the temp file to the greylist file's name

When ASSP does this automatically, it also resets the greylist cache, but there's no way to do that externally. However the cache will eventually reset on its own.

The greylist associates IP addresses of mail sending hosts with their recent statistical probability of sending spam or not spam. It's not a whitelist, or a blacklist, but somewhere in-between -- a grey list. Of course it is rare to find a host that sends equal amounts of spam and not-spam, so very few entries are 50/50 or completely grey.

This type of information is of practically no value to traditional IP-based spam-blocking systems. However, it is ideal for a Bayesian descriminator: this probability is factored in with other probabilities associated with the mail and helps affect the outcome in the desired way -- better spam AND not-spam detection.

ASSP takes some care in how the user-supplied statistics are handled in order to reduce the probability that a spammer or misconfigured host will unduely influence the statistics. Because strong security is impractical in this situation I prefer to leave the exact method obscure. Nevertheless, as the ASSP community grows the effectiveness of the greylist will also increase.

Assp will stop forwarding new requests to Courier after about half a day (sometimes it can last 24 hours before resulting in this behaviour).

Upon manual telnet connection to port 25, I get an open connection and then an immediately closed connection. After killing and restart assp.pl, the problem goes away - for atleast 12 hours.

----------------------------------

By: ladylong ( Ruth Rogers )
10-15 01:25

I had the same problem with my setup (ASSP with Merak MailServer) and finally discovered that Merak was tarpitting ASSP because of the number of connections it was attempting when things got busy. If your mail server has a similar feature, you might just need to change settings - with Merak it's a matter of telling it to bypass tarpitting for 127.0.0.1 (or wherever you have ASSP running from).

-----------------------------------
By: clayne ( christopher layne )
10-29 20:31

ladylong,

Thanks for the information, this is what was happening. I adjusted courier's settings and ASSP never "hangs" now.

ASSP avoids bouncing NDRs because it rejects mail in the SMTP conversation. This only works if the SMTP conversation is with the source of the mail.

If a spam host connects to your primary site and tries to send a spam, at the end of their message they get a 550 error and the connection closes. They are both notified of non-delivery and the mail is not delivered.

If a spam host connects to your secondary without ASSP, the secondary will store the message. Then it will try to deliver it to the primary which will block it. The secondary will then generate a non-delivery report that will probably bounce.

This same situation can happen if you put a store-and-forward anti-virus service in front of ASSP.

You have two options:

1) Just ignore them. The odds that you'll receive a spam from one of the addresses that has been whitelisted is very low. After a while the whitelist entries will go away on their own. Having extra entries on the whitelist is not much of a performance hit, and you have better things to do with your time than mess with them.

2) Copy the section of the maillog that contains the erronious whitelist addition -- edit it to make sure there's no valid whitelisted addresses in it, then paste it into the "remove addresses" box in the ASSP config -- you don't have to clean out the other text from the maillog -- just make sure the only email addresses that appear in what you post are ones you want removed.

You choose. (I'd pick the first.)

telnet 127.0.0.1 125
(type quit)
Did you get the banner from your mail transport?

telnet 127.0.0.1 25
(type quit)
Did you get the banner from your mail transport again?
Did you see a message from ASSP on the screen or maillog.txt, "Connection from 127.0.0.1"?

Are your local users in the subnet(s) listed in the "Allow All Mail" setting?

When you configure your email client to connect to the ASSP box, port 25, to deliver SMTP mail, does it go through?
Does the entry appear in the maillog.log:
10.x.x.x ... Local or whitelisted

Is your mail MX pointing to ASSP's host?

Can you send a mail through ASSP to a local address? Do they receive it?

Can you send a mail through ASSP to an external address? Do they receive it?

Can an external address send you a mail? How does it show up in the maillog.txt file?

Is the greylist less than 24 hours old and > 50k in size?

Are you seeing some email in the maillog.txt marked as "Bayesian Spam" and "Message OK"?

Are files collecting in your spam and notspam folders?

Is your spamdb getting rebuilt automatically?

Also, files in the errors/* folders take extra RAM as well, so try to keep those folders down to about 100 each.

Rebuild the spamdb with the Ramsaver option. Start it running and come back in the morning.

 > If I have ASSP running a  user other than root, should all files within
 > the distribution be owned by that user?

						

 > The web interface/maillog indicates whitelist additions to but no
 > files are being created in the distribution for redlist or whitelist.

						

 > I'm also not able to use the web interface to view log files and
 > apparently assp is no longer writing to the log.  All of this I assume
 > to be permissions issues.

						

But I hear you say, "But I want to block mail from known-bad IP addresses." Can't ASSP do that?

ASSP could do that but that is not what the DNSBL setting was used for.

These are the factors involved with DNS black listing and how they relate to ASSP:

1) I used DNS black lists for a number of years before I wrote ASSP. I found that they rejected far too little spam and had far too many false positives. They change slowly, while spammers adjust quickly. There is no such thing as a "realtime" black-hole list. I also found that truely successful black-hole lists either get sued out of existance, become pay (ie for profit) services, or simply go bust too quickly. The bottom line is that an IP address alone does not give you enough information to correctly classify incoming mail. ASSP's greylist is an attempt to make use of what information is available about an IP address without creating false positives or negatives.

Perhaps you'll argue that you know of a truely fantastic black hole list, and maybe times have changed and such a thing really exists. If you have one that's > 99% effective, then use it and skip ASSP. If it's less than 99% effective, then just use ASSP and forget about the black hole list -- it's unnecessary and a distraction.

2) ASSP is a multiplexed server, not multi-process or multi-threaded. This allows ASSP to be truely cross platform and quite effecient in how it handles connections. Unfortunately it means that any process that blocks will cause a temporary SMTP outage. Perl's standard DNS functions block. This means that traditional DNSBL lookups via DNS are incompatible with ASSP's multiplexed design.

The alternative (and the approach in the original DNSBL and that continues in today's greylist) is to load all the DNSBL values into a file where lookups can be made in a timely fassion. However most DNSBL services only provide this option if you can prove that your load is quite high. Or you can use a tool like openrbl and update your file on a daily basis. This ends up being problematic.

3) "Spam filtering works best by combining a variety of spam-fighting technologies." And to the extent that that is true, ASSP incorporates a variety of spam-fighting technologies. However, each technology carries not just a benifit, but also a margin of error and a maintenance cost. You must be careful in combining technologies or you find that you increase your maintenance costs and increase your overall error margin without increasing your accuracy. I believe DNS blacklists fall in this category.

4) Bayesian content filtering is a fantastic tool. Generally the requests I've received from people who want DNSBL support are from those who have used it in the past and haven't used a good Bayesian content filter before. They're trying to keep doing what they've always done before. I'd encourage you to give ASSP a try. See how it performs. I expect that even without DNSBL support it will exceed your expectations in most cases.

Answers in this category:
What is rebuildspamdb.pl and how do I use it?
What are the updatednsbl and dnsbl.pl tools?
What is upgrade.pl?
What is repair.pl?
What is move2num.pl?
What is stat.pl?
What is goodhosts.pl?

The updatednsbl.sh and updatednsbl.bat script / batch file is used to download the current list of IP addresses of spammers from the openrbl.org probject website. It calls wget and bzip2 to retrieve this data and decompress it. The dnsbl.pl script reads the data from the openrbl format and converts it to an appropriate format for ASSP.

The upgrade.pl only appears in releases where it is required. If you are upgrading from 0.1.6 all the way to 0.3.4 (or beyond) you will need to download each of the intermediate releases and run upgrade.pl. Generally upgrade.pl is only required between major releases (ie 0.1.x to 0.2.0 and 0.2.x to 0.3.0), though there could always be exceptions.

Upgrade instructions are posted in the mailling list (search the archives) or usually in the release news.

Run the script and it will load, resort and write the sorted hash files.

To repair the spamdb file run rebuildspamdb.pl.

 perl move2num.pl -r

Note that ASSP reads all files in the directories irregardless of their name, so numbers or words for filenames is fine. However filenames that aren't numbers will remain eternally in the spam / nonspam folders and never be rotated out.

Note also that the errors/spam and errors/notspam folders are not processed in this way -- the files always remain in these folders indefinately.

 Perl stat.pl maillog.txt

Your results should look something like this:

 As of Thu Mar 27 10:48:54 2003 the mail logfile shows:
 78843 messages, 47637 were spam (60.4%) in 73 days
  for 1080.0 messages per day or 652.6 spams per day
 8303 additions to / verifications of the whitelist (113.7 per day)
 28273 were judged spam by the bayesian filter (59.4% of spam)
 18862 were to spam addresses (39.6% of spam)
 502 were rejected for executable attachments (1% of spam)
 12608 were sent from local clients (40.4% of nonspam)
 7838 were from whitelisted addresses (25.1% of nonspam)
 10760 were ok after a bayesian check (34.5% of nonspam)
 14467 addresses are on the whitelist

 15108 hits on the blacklist
 14890 resulted in spam (52.7% of Bayesian spam, 98.6% of blacklist hits)
 218 resulted in non-spam (1.443% of blacklist hits)

For an explanation of what goodhosts is see What is goodhosts and what does it do?

It is possible that your goodhosts file can become corrupted. To know if you're affected, look at the end of your goodhosts file -- the last two entries should be something like this:

 notspam?3000
 spam?52384

(Your numbers will be different). If either or both of these lines are missing, or if you're just paranoid, do this:

1) stop assp
2) do:

  perl goodhosts.pl maillog.txt

						

Answers in this category:
ASSP and OS/2
HOWTO: Post.Office
HOWTO Change the SMTP port for Lotus Notes
HOWTO: General instructions for any non-SMTP mail server (MS Exchange, Lotus Notes, etc)
HOWTO: ASSP and TLS or secure SMTP.
HOWTO: Change the SMTP port in Courier-MTA?
HOWTO: change the smtp port in sendmail
HOWTO: ASSP and Antivirus proxies
HOWTO: ASSP Mysql local domains
HOWTO: ASSP & VPopMail PopBeforeSmtp
HOWTO: ASSP & SAV on the same server
HOWTO: ASSP & Merak Mailserver
HOWTO: ASSP & Fetchmail
HOWTO: ASSP on Mitel SME (E-Smith)
HOWTO: automatically restart ASSP if it dies in *nix
HOWTO: NAV for Gateways, IIS SMTP -> Exchange
HOWTO Change the SMTP port for IMAIL
HOWTO: ASSP & Postfix
SMTP on Telstra Australia after April 2004
SIMS and ASSP on MacOS X HowTo
ASSP on Win32 Install

Answers in this category:
How can I get Exchange 5.5 to stop sending NDRs?
Can I use Exchnage 2000 Public folders to report spam / notspam?
HOWTO: Change the SMTP port in Exchange
HOWTO: Modification to make exchange public folder for whitelist additions?
How to trick Exchange 5.x to forward outgoing mails through ASSP
HOWTO: Use an Exchange 2000 Virtual Server instead of your ISP.
HOWTO: Exchange 2003 to check for valid recipients

This method results in NDR messages not being generated because the address DOES exist... it just doesn't GO anywhere. According to what I read while finding this tip, the incoming messages do not get stored anywhere... they just "disappear".

So that, combined with redlisting "Postmaster" in ASSP, has pretty much resulted in the elimination of NDR messages from Exchange... while leaving the Postmaster mailbox able to receive incoming mail addressed specifically to it. That way, if anyone on the outside has a problem, they can still reach you using the Postmaster address.
Unless you use this method, Exchange 5.5 will ALWAYS send out NDR messages for any e-mail sent to an invalid address. There is no known way to turn it off the "feature". (Apparently, Exchange 2000 DOES have a way to disable NDR messages.)

(From Matthias Behnke, Tuesday, June 24 10:20 PM)
In Exchange 5.5 (not 2000) you can enable the SMTP VRFY-command to reject email delivery at the SMTP-Level. Microsoft does not recommend to enable this feature as spammers will then be able to quickly check which accounts are hosted on your server, so spams will be more direct. In Exchange 2000 the VRFY-command is no longer supported.

Refer to http://support.microsoft.com/default.aspx?scid=kb;en-us;289521 Disabling NDRs at all is not really a good idea as senders, who accidently misspell the mailaddress will get no reply that their mail was not delivered.

As John said, redlisting should help to protect the whitelist from adding the NDR-receivers

Then on Exchange machine, on disk M: (virtual disk for Exchange mail storage) I can find .eml files in plain text format (header+body), just copy it to c:\assp\spam and nospam, and use move2num.pl to convert file names to numbers. This task easy to automatize with .bat files.

Users not need change it client's configuration (our organization use Outlook as MAPI client on most workstations)

Users, who connect to Exchange over SMTP/POP3/IMAP, can forward spam to Public Folder's mailbox adress.

I think it's useful methode for such configuration.

See also http://sourceforge.net/forum/forum.php?thread_id=897310&forum_id=235332

Look for the "services." file (usually at c:\winnt\system32\drivers\etc\)

load this in the editor and change the port number for SMTP to anything other than 25. Make sure Editor does not add any file extension to this file upon save or remove it afterwards.

done.

Exchange 5.5 reads this file on startup and uses whatever port there is for SMTP.

So here's my very first Perl hack. I'd love to hear from any of you Perl programmers (particularly JohnH, of course) on whether this is broken in any way or if there's simply a better way of doing it.

I added these two subroutines to ASSP.PL (doesn't matter much where they go - as long as it isn't in the middle of an existing routine...):

 ################ HACK
 sub FileWhiteCheck {
  local $_;
  mlog(0,"looking for file-based whitelist additions...");
  foreach $filename (glob("$base/whitelst/*")) {
    open(F, "<$filename") or next;
    while(<F>) {
      last if /^[\r\n]*$/;
      for $a ($_=~/^From:.+?([^<>,;"'\(\)\s]+\@[^<>,;"'\(\)\s\?]+)/g) {
        FileWhiteAdd($a);
      }
    }
    close(F);
    unlink ($filename);
  }
 }
 sub FileWhiteAdd {
  my($a)=@_;
  if ($a=~/\@($localDomains)$/) {
    # can't add local users to whitelist
  } elsif( $Whitelist{lc $a} ) {
    # already on whitleist
  } else {
    # ok, add it
     mlog(0,"file new whitelist addition: $a");
     $Whitelist{lc $a}=time;
  }
 }
 ################ HACK

Then in ASSP's existing MainLoop right before the first call to SaveWhitelist(line 575 of version 0.3.2) I added a call to FileWhiteCheck:

 if($time >= $saveWhite) {
  d(9);
  FileWhiteCheck();
  &SaveWhitelist;
  $saveWhite=$time+$UpdateWhitelist;
 }

						

Note that FileWhiteCheck looks for ANY files in the subdirectory "whitelst" (the 'i' is missing so that it doesn't conflict with the existing "whitelist" file) AND WILL DELETE THEM AFTER PROCESSING! You've been warned...

My search for whitelist additions in the messages is also a bit different from JohnH's: I only look for addresses in the first line of the message's "From:" header. Any other addresses are ignored.

-Oscar

Microsoft Exchange-Server is not easily tweaked to run flawlessly with ASSP.

To have all outgoing mails scanned by ASSP for automatic whitelist additions,
Exchange should forward all outgoing mail to the IP:Port address that is
configured on the ASSP machine (might be the same machine).

Unfortunately Exchange 5.x does NOT allow the specification of a port number
for the "forward all mail to ..." setting in the Internet-Connector.
We therefore have to set the ASSP-Relay-Port to "25" - which usually conflicts
with the port "25" that we use for INBOUND Mail.

My solution: Bind another IP address to the network card on the ASSP machine.

This might sound a little weird but works quite well:
Open "Control Panel / Network"
Pick "TCP/IP" and klick [Properties]
on the "IP Address" tab klick [options]
under "IP-Addresses" ckick [Add] and enter a new IP Address and Netmask
Save all

Example:
ASSP / Exchange on the same machine
local IP address: 192.168.1.1, Netmask 255.255.255.0

Add a second IP Address: 192.168.1.2 netmask 255.255.255.0

Change the SMTP port in \winnt\system32\drivers\etc\services as described in
the Faq-O-Matic to 125

Configure ASSP Ports:
Listen Port: 192.168.1.1:25
SMTP Destination: 192.168.1.1:125
RELAY port: 192.168.1.2:25   <-- bound to the other IP
RELAY host: mail.yourprovider.com:25

Mailflow:

sending SMTP-Server
   ----> (firewall mail.yourdomain.com:25)
   ----> ASSP 192.168.1.1:25
   ----> Exchange 192.168.1.1:125

Exchange-User
   ----> Exchange Server
   ----> ASSP(Relay) 192.168.1.2:25
   ----> (firewall)
   ----> mail.yourprovider.com:25

This works equaly well if ASSP is running on a different machine. Normally
there should be no problem to have the PC listen to another internal IP
address as long as it is in the same subnet and is not used otherwise.

If the ASSP-Machine uses an official IP address for which you would avoid to
add another OFFICIAL address you might add one of the 10.x.x.x/255.0.0.0 range
but that will probably work only if Exchange is on the SAME machine.

Incoming:
Internet->ASSP(25)->Default Virtual SMTP Server(225)

Outgoing:
Default Virtual SMTP Server(225)->ASSP(25)->ASSP Relay SMTP Virtual Server(125)->Internet

This worked flawlessly

Thought this might be of interest to you and your users
Cliff Williams

 Internet->ASSP->Default SMTP Virtual Server

						

 Default Virtual Server->ASSP->Relay SMTP Virtual Server->Internet

						

Overview:
This explains how to setup ASSP on an Exchange 2000 server in a single server implementation. This implementation utilizes 2 SMTP virtual servers setup in Exchange 2000.

Step 1:
Assign 3 IP addresses to your Exchange server (I'm assuming you know how to do this). In this example we will be using the following addresses:

10.5.200.2 This is the inside address that ASSP listens on. It is also the address that the firewall routes SMTP to and the address of the Default SMTP Virtual Server.
10.5.200.3 This is the address of the SMTP Virtual Server used by ASSP to route mail to the internet
10.5.200.4 This is the address ASSP uses to accept connections from your exchange server. This will be your smart host.

Step 2:
We will need to modify the Default SMTP Virtual Server and create a new virtual server. These servers will operate on ports other than the default 25. In this example we will be using 125 & 225. ASSP should not be running. If it is stop it.

Open Exchange System Manager and expand the Servers node.
Then expand the server node you are going to work on.
Expand the Protocols node and click on SMTP.
In the result pane, right click on the Default SMTP Virtual Server and select Properties.

In the IP address drop down, select the original IP address for your server. In our example this is 10.5.200.2.

Click the Advanced button, then click the IP address in the list and click Edit. Change the port to 125. Click OK, then Apply and OK.

Now, right click in the white space in the result pane and select New SMTP Virtual Server.

In the Name Field give this server a name. I used "ASSP Relay Host" click next. Now select an IP address from the drop down list. In our example we'll use 10.5.200.3. Click finished. (you will see a question mark on the new server)

Now right click on the new virtual server and choose Properties. Click Advanced and change the port to 225. Click Ok and OK.

Right click on both Virtual servers and stop them. This will prevent them from colliding with ASSP if there is an interface:port conflict.

Step 3:
Install ASSP if you have not already done so.

Start ASSP from the command line. (we will start the service later).
perl assp.pl

Open the Configuration web page. http://localhost:55555

Click "Show Advanced Configuration Options, scroll to the bottom and click Apply Changes.

Now, in the SMTP Destination enter the IP:port of your Default SMTP Virtual Server. In our example. 10.5.200.2:125

Click As a Service if it's not already checked.

Change the Listen Port to the IP:port that incoming SMTP connection will be made on. Example. 10.5.200.2:25

Enter an alternate if you like. Remember to change your password.

Under Relaying make these changes.

Add your local subnet to the Accept All Mail field. Example: 10.5.
Enter your local Domains

In Relay Host enter your ASSP Relay Virtual Server. Example 10.5.200.3:225
In Relay Port enter the 3rd address we created on port 25.
Example 10.5.200.4:25

Configure all other parameters as you see fit and click "apply changes".

Shut down ASSP.

Start the ASSP Service.

Step 4:
In your local DNS (local not public) enter a host name for your Relay Port IP address. I called mine smarthost. As of Service Pack 3 Exchange will no longer allow you to enter an IP address for the smart host.

In Exchange System Manager right click on the Default Virtual Server and choose properties. Click the Delivery tab, and then click the Advanced button.
In the Smart Host field enter the FQDN of your smart host virtual server created above. Example: smarthost.domain.com

Click OK, Click OK.

Start both Virtual SMTP servers.

If any of the servers fail to start you have an address:port conflict somewhere. Most likely ASSP is colliding with one of your virtual servers.

This configuration should work fine for small to mid size organizations. Larger organizations should separate these services on different boxes.

With Exchange 2003, there's now a Recipient Filtering option to be found in Exchange System Manager, under Global Settings / Message Delivery. This simply allows you to block messages that are sent to specific recipients.

There's also an interesting tick box titled 'Filter recipients who are not in the Directory'. You might wonder why you'd want to filter messages that are addressed to non-existant users - after all, the server will just send a non-delivery report (NDR) back to the sender.
advonet

 wget http://spfilter.openrbl.org/data/output/DEFAULT.octets.bz2 --output-document=dns.bz2
 if not exist dns goto cont
 if exist dns.sav del dns.sav
 ren dns dns.sav
 :cont
 bzip2 -d dns.bz2
 perl dnsbl.pl dns

That information is stored in the registry and can be modified for Post.Office.
1. As a local NT Administrator for that host, open the Registry Editor.
2. Within HKEY_LOCAL_MACHINE select: SOFTWARE -> Software.com -> post.office -> SMTP-Accept -> Config
3. Double-click on the "Socket" Key to edit it.
4. Change the value from 25 to 26 (for example) and click on OK.
5. If you do not have permission to save this value, you will need to: select the Security Menu Option -> Owner…-> Take Ownership; select the Security Menu Option again -> Permissions -> ; highlight your administrative self; select the Type of Access Pull Down and select Full Control; select OK; now try the edit the Socket Key.
6.You will now need to stop and start the Post.Office service in your Control Panel - Services Window.

For Release 5 of IBM/Lotus Domino you need to change the Internet Ports in the Server document. See attachment Domino_R5_Server_Document.jpg or Domino_R5_Server_Document.png After you have changed this value in the Server document, you only need to restart the Domino router.

For Release 6 of IBM/Lotus Domino you need to change the Internet Ports in the Server document. See attachment Domino_R6_Server_Document.jpg or Domino_R6_Server_Document.png After you have changed this value in the Server document, you only need to restart the Domino router.

Kind Regards from Switzerland
SteveB

You need to make some choices: Will ASSP run on the same box as Exchange? ASSP requires a second mail server to deliver internet mail; will you use your ISP’s mail relay host, or configure your own secondary mail server (in addition to Exchange)? If so, what server will you use?

I'll explain two ways to do this.

The first way works best with two servers. I’ll assume you have Exchange running on one server and that you run ASSP and qmail on a separate FreeBSD server. (You can use many other possibilities, but I’ll describe this setup. Actually Windows with Mercury will work equally well.) This setup looks like this:

Internet -> ASSP -> qmail -> exchange -> client
and
Client -> Exchange -> ASSP -> qmail -> Internet

Set up your FreeBSD server with perl and qmail. (see http://www.freebsd.org/ ) Make sure it’s working on the network. Edit /etc/services and change SMTP from 25 to 125. Create a group and user called assp. Install ASSP to the assp user’s home. (These are suggestions – do something else if you prefer.) Verify that qmail relays mail for 127.0.0.1. Configure any mail applications (web mail or local clients) to use SMTP for delivery and not sendmail.

Direct your DNS MX for Exchange’s domain to the ASSP / qmail box. Configure qmail to route mail for Exchange’s domain to Exchange’s IP. (Depending on your mail server this may be tricky. One option might be to put the Exchange's IP in the /etc/hosts or /winnt/services32/drivers/etc/hosts file associated with the mail host in your DNS MX.)

Configure ASSP this way: Listen Port = 25, SMTP Destination = 125, Accept All Mail = exchange server, 127.0.0.1 and whatever else you like.

Configure Exchange to route all Internet mail to the ASSP / qmail box.

That’s it. Please read the documentation to know how to set up your spam and notspam collections, prime the whitelist, rebuild your spamdb, customize ASSP’s configuration settings for your site, and more.

This is the more complicated, but more flexible way.

If you plan to run everything on one box and you’re not planning to use your ISP’s mail relay, you can use Mercury (see http://www.pmail.com) or even IIS’s SMTP service. Be sure you’re familiar with their configuration options and limitations.

Incoming mail will follow this path:

Internet Mail -> DNS MX record -> (port 25) ASSP -> Exchange -> Clients

Outgoing mail will follow this path:

Client -> Exchange (Smart host) -> (Relay port) ASSP -> ISP’s Mail Relay
or
Client -> Exchange (Smart host) -> (Relay port) ASSP -> local secondary SMTP

If ASSP and Exchange are on the same box, you need to move Exchange’s SMTP port. Different mail servers allow you to do this in different ways. You may find a configuration menu like Domino / Notes: Ports -> Internet Ports -> Mail -> SMTP Inbound. For Exchange you need to look in c:\winnt\system32\drivers\etc\services and edit the SMTP line so say 125 instead of 25. (You can use any port you want, but use ports consistently and be sure that your firewall protects ports appropriately.) Save the file (make sure your editor didn’t add a .txt). You may need to restart Exchange.

In the ASSP Configuration web page (http://127.0.0.1:55555/) put Exchange’s address in the SMTP Destination: 127.0.0.1:125 perhaps.

If ASSP and Exchange are on the same box put your public IP : 25 in the “Listen Port” box. For example, 220.1.2.3:25. Then put 127.0.0.2:25 in ASSP’s “Relay Port” and the address of your ISP’s mail relay or your secondary SMTP server in the “Relay Host.” For example: smtp.localhosting.com:25 or 127.0.0.1:325. Configure Exchange to route all Internet mail to 127.0.0.2.

If ASSP and Exchange are on separate boxes, you need to assign a second IP address (often called an alias) to the ASSP box. (If Exchange is on a private network and your ASSP box is on both the public and private networks, so it already has two addresses, you can skip this step.) Let’s assume you have given ASSP 220.1.2.3 and 220.1.2.4. Your Mail MX should point to 220.1.2.3, and ASSP should have 220.1.2.3:25 for its “Listen Port.” 220.1.2.4:25 should be the “Relay Port” and your firewall should block external connections to 220:1.2.4:25. Configure Exchange to route all Internet mail to 220:1.2.4.

Don’t forget to click submit on the ASSP configuration page to accept the changes.

If you are using a secondary SMTP server, configure it to listen on port 325. (You can use any port you want, but use ports consistently and be sure that your firewall protects ports appropriately.) Configure it to accept all connections/mail from ASSP for relaying, and disable any message size limitations.

See also the note on how to use the email interface with Exchange / Notes, etc at How do I use the email interface with Exchange, Notes, or a RelayHost / RelayPort setup? .

Start by downloading and installing stunnel as described on their website. Note that in Windows you can use srvany.exe and ntinstsrv.exe to run stunnel as a service. Your "Parameters/Application" registery entry will be something like this:

 c:\service\stunnel.exe -D 0 -d 465 -r 127.0.0.2:25 -p c:\service\stunnel.cer -n smtp

						

Set ASSP to "allow all mail" from 127.0.0.1 (if you like) but exclude 127.0.0.2. Stunnel will connect to ASSP on that port, and ASSP will treat these connections as non-local. Enable SMTP authentication on your email client. (Your mail transport must also support authentication.)

If your mail server doesn't support authentication you can run with ASSP "allow all mail" from the stunnel connection, but you create an open relay. Generally spammers haven't used TLS / port 465 / Secure SMTP as open relays, but they might start doing that at any time, so this configuration is not recommended.

There should be a line PORT=smtp

Change that to

 PORT=125

						

Restart courier.

We're using courier-mta here on a bunch of machines and will be consolidating the SPAM analysis to a single point of entry to simplify and unify the data. During the process of setting up a machine (that acts as a firewall, i.e., two (2) nics) we found that courier happily binds to each nic for smtp listening.

Actually, it's documented to work that way and it does.

We had to get very specific with courier's etc/esmtpd file entry, using the following line:

 PORT=127.0.0.1.125

..to convince it to really only listen to your ASSP proxy on the one nic (inside). Otherwise, courier would reject the socket bind attempt to 127.0.0.1:125 (Note: that courier's file uses 127.0.0.1.125 dotted only)

It took a short while to grok that courier was the culprit, and now it behaves way better.

We set up ASSP to be the real honest SMTP service for the machine; and it handles SMTP connections on both nics without incident.

Please, if you get a chance, update the FAQ thingy to add some words about telling courier to be very specific (if it makes sense to you) because this was really the only issue here - and it was very frustrating. Now, after the fact, it's pretty obvious - but, as you know, hindsight is..... and all that.

 O DaemonPortOptions=Port=smtp,Addr=127.0.0.1, Name=MTA 

						

 O DaemonPortOptions=Port=125,Addr=127.0.0.1, Name=MTA

If you use Norton Anti-virus for Exchange on your mail server and have recipient notification turned on, you will add the virus sender's email address to the whitelist every time you receive a virus, because NAV sends them a notification.

This is of course easily fixed by turning this feature off which is probably fine, with the types of viruses nowadays it probably isn't notifying the actual sender anyways. :-) Or you can add the NAVMSE email address to the redlist.

This caught me by surprise so I thought I would share it with the group. I am probably the only one who didn't think of this before now. But there you go anyways! :-)

Ged West

I am looking into using ASSP for our organization, and ran into a
situation where I'm using Postfix/Mysql/Courier-Imap - Rather than using
your config file to specify local domains, I needed to be able to query
a table in a MySQL database.

If you are interested in offering a similar feature for that config
entry or others, here is the diff output I used to patch the existing
codebase for 1.0.3.

14a15,19
 > #20031007 BLF
 > #Note: Added Mysql local database query for domains
 >
 > my %cachedomain;
 >
939a945,985
 > #Added by BLF
 > sub loaddatabase {
 >         use DBI;
 >         my $host = shift;
 >         my $result;
 >         my $count = keys %cachedomain;
 >         if ($count > 10000) {
 >                 %cachedomain = ();
 >         }
 >         if (!(exists $cachedomain{$host})) {
 >                 my (%data) = (
 >                         MySQL_Server => "localhost",
 >                         MySQL_Username => "myusername",
 >                         MySQL_Password => "mypassword",
 >                         MySQL_Transport => "transport",
 >                         MySQL_ServerDB => "postfix",
 >                 );
 >                 my $dbh =
DBI->connect("dbi:mysql:$data{MySQL_ServerDB}:$data{MySQL_Server}","$data{MySQL_Username}",
"$data{MySQL_Password}", { AutoCommit => 1 }) || { $errorcode=
$DBI::errstr };
 >                 if ($errorcode) {
 >                         return 0;
 >                 }
 >                 my $queryhost = $dbh->quote($host);
 >                 my $sth = $dbh->prepare("SELECT domain FROM
$data{MySQL_Transport} where domain = $queryhost");
 >                 $sth->execute;
 >                 ($result) = $sth->fetchrow_array;
 >         } else {
 >                 return $cachedomain{$host};
 >         }
 >         if ($result eq $host) {
 >                 $cachedomain{$host} = 1;
 >                 return 1;
 >         } else {
 >                 $cachedomain{$host} = 0;
 >                 return 0;
 >         }
 > }
 >
944a991,994
 >
 > #Added by BLF
 >  return 1 if &loaddatabase($h);
 >


--
Brian Fields

 sub ok2Relay {
  my $ip=shift;

							
  # If you want to allow pop before smtp uncomment this line and implement the PopB4SMTP sub below

							
  # This must be done first...

							
  return 1 if PopB4SMTP($ip);

							
  return 1 if $acceptAllMail && $ip=~/^($acceptAllMail)/i;

							
  if($relayHostFile) {
   check4update(relayHostFile);
   return 1 if $relayHostFile{$ip};
  }

							
  # failed all tests -- return 0
  0;

 # if you want to allow pop before smtp authentication then umcomment the line above
 # where this sub is called and edit this sub to be appropriate for your implementation

							
 # It sequentially test the text file /users/vpopmail/etc/open-smtp
 # maintained by vpopmail (Guy)

							
 sub PopB4SMTP {
  my $ip=shift; my $s; my $cip;
  open(PBSMTP,"<$PopB4SMTPFile") or return 0 ;
  while(<PBSMTP>) {
     $s=index($_,":");
     if ($s != 0) {
       $cip=substr($_,0,$s);
       if ($cip eq $ip) {
         mlog(0,"PopB4SMTP OK for $ip");
         close(PBSMTP);
         return 1;
       }
     }
  }
  close(PBSMTP);
  mlog(0,"PopB4SMTP NOT OK for $ip");
  return 0;
 }

						

 return 0 unless $PopB4SMTPFile;
 my $ip=shift;
 my $ind; my $ipMatch; my $preByte; my $postByte;

							
        open(MKPOPSMTP,"<$PopB4SMTPFile") or return 0 ;

							
        while ($line = <MKPOPSMTP>) {
                $ind = index($line,$ip);
                        if ($ind  != 0) {
                                #find the match, and get it and the char before and after
                                #if both of those aren't a number ord("0") & ord("9") 48 - 57 then I have the whole IP [ord is same as asc]
                                #if not, this is a subset of a larger IP address and thus no good
                                #ex: 66.35.250.203 I don't want to let 66.35.250.20 or 6.35.250.203 or 6.35.250.20 you get the idea

							
                                $preByte = ord(substr($line, ($ind - 1), 1));
                                $postByte = ord(substr($line, ($ind + length($ip)), 1)); #0 based, so no + 1

                                if ( (($preByte < 48) || ($preByte > 57)) & (($postByte < 48) || ($postByte > 57)) ) {
                                        close(MKPOPSMTP);
                                        mlog(0,"PopB4SMTP OK for $ip");
                                        return 1;
                                }
                        }
        }
        close(MKPOPSMTP);
        mlog(0,"PopB4SMTP NOT OK for $ip");
        return 0;

sub PopB4SMTP {
 return 0 unless $PopB4SMTPFile;
 my $ip=shift;
 my $ind; my $ipMatch; my $preByte; my $postByte;
        open(MKPOPSMTP,"<$PopB4SMTPFile") or return 0 ;
        while ($line = <MKPOPSMTP>) {
                $ind = index($line,$ip);
                        # Here was my bug. I had Not 0, which was true for -1;
                        # Well, that just goes to show you
                        if ($ind  > 0) {
                                #find the match, and get it and the char before and after
                                #if both of those aren't a number ord("0") & ord("9") 48 - 57 then I have the whole IP [ord is same as asc]
                                #if not, this is a subset of a larger IP address and thus no good
                                #ex: 66.35.250.203 I don't want to let 66.35.250.20 or 6.35.250.203 or 6.35.250.20 you get the idea
                                $preByte = ord(substr($line, ($ind - 1), 1));
                                $postByte = ord(substr($line, ($ind + length($ip)), 1)); #0 based, so no + 1

                                if ( (($preByte < 48) || ($preByte > 57)) & (($postByte < 48) || ($postByte > 57)) ) {
                                        close(MKPOPSMTP);
                                        mlog(0,"PopB4SMTP OK for $ip");
                                        return 1;
                                }
                        }
        }
        close(MKPOPSMTP);
        mlog(0,"PopB4SMTP NOT OK for $ip");
        return 0;
} 

 RE: can't send outbound mail   

When I changed the setup as discussed, I still was unable to deliver outbound mail. Strangely, the fix was to change the ASSP's "SMTP Destination" field from 127.0.0.1:125 to 10.100.10.54:125 (the assigned IP address). Everything is working fine now.

-----------------------
ASSP can't perform delivery to the destination mail servers because it's a proxy, not a mail transport. Please re-read the page discussion how to setup assp with exchange.

You must involve another mail transport to make this work, (note the new post in the online documentation that apparently with exchange 2000 or better you can create a "virtual server" to handle this -- I'm no exchange guru, so don't ask me how this works.)

If you're going to use SAV in the process on both incoming and outgoing mail you must set it up like this:

Internet -> assp -> sav -> SMTP2 -> exchange -> clients

Clients -> exhcnage -> assp -> sav -> SMTP2 -> internet

Note that assp -> sav -> smtp2 is the same in both setups.

SMTP2 is your second smtp server and can be IIS's smtp, or an exchange virtual server, or Mercury, but it can't be the same SMTP that receives your internet mail for exchange users. (I duno, some exchange guru is gona come up with some crazy way to get client mail out through a relay host throug assp and sav and back into exchange's smtp connection then out to the internet, but that sounds pretty twisted to me. I suppose that's what the virtual server does.)

You don't need to use ASSP's relay port / relay host to do this.

Not being an SMTP guru, what is the purpose of the "Send NOOP Info" option? I've searched the forums and docs and not found anything enlightening. I'm hoping it will fix an error I get a LOT of in my application error log:

 Event Type: Warning
 Event Source: Symantec AntiVirus for SMTP Gateways
 Event Category: None
 Event ID: 2
 Date: 10/31/2003
 Time: 12:05:42 PM
 User: N/A
 Computer: EMAILSERVER
 Description:
 Error 0 receiving data from remote host. (10.100.10.54) 

It seems to occur with each message tagged as spam, and it occurs with Test Mode on and off.

--------------

vreiner ( Victor Reiner ) 11-03 18:17

I checked with Symantec on this, and it is neither possible to reduce the level of verbosity nor even turn this warning off. I put it into them as a suggestion....

You might want to place a note in the SAV section of the docs to warn people that this is "normal" for an installation of ASSP with SAV and not changeable/fixable at least right now.

From someone who learned the hard way!

Add 127.0.0.1 to the "bypass tarpitting" list, or else when things get busy Merak will Tarpit ASSP and it won't be able to process mail for the next hour (or whatever your tarpitting timeout is set to).

Applicable to versions 1.0.6 onwards unless ASSP operation changes significantly!

I have spent some time trying to make ASSP work on a linux system that fetches mail from our ISP via fetchmail, and forwards it to our users via Sendmail. This is a fairly common configuration for a linux mailserver that is not permanently connected to the internet.

In our case this set-up is also used because our ISP has an excellent virus filtering system for email which we could not economically replicate in-house.

ASSP has a problem in this type of configuration though:

To filter email, ASSP has to be interposed between fetchmail and sendmail. The problem arises because, when running in it's full fitering mode (ie not testmode) ASSP tells fetchmail to send a rejection message back to the SMTP server before discarding the mail - unfortunately, ASSP then closes the socket which was opened for mail delivery in the first place, fetchmail sees that its socket has closed and promptly shuts down. In testmode, however, as all mails are delivered regardless, everything works as expected.

It seemed obvious to me that the way to make ASSP work in these circumstances was to leave it in test mode, and to get sendmail to deal with any mails marked as spam by ASSP. Combine this with the ASSP spamtrap facility, and you can have all emails delivered, but stop the spam reaching your end-users.

So:

Setup ASSP/Sendmail as describe in the ASSP documentation.

Configure ASSP to run in testmode, have it mark all spam email Subject lines with something that is almost certainly unique eg TRAPPEDSPAM.

If you want to retain copies of all email regardless, configure a spam trap account and have ASSP copy all identified SPAM to it - note that this happens before the header is changed and hence the emails will be accepted by the altered sendmail configuration below, and placed in the trap.

     Add the following to your sendmail configuration file (usually

     In the FORMAT OF HEADER area add (<tab> represents a tab character - tabs

 HSubject:<tab>$>75<tab>Used to sheck for spam mail marked by ASSP

     Add a new rule set 75 (if this number is already in use, pick one that

 S75
 RS*TRAPPEDSPAM$*<tab>$#discard $: discard

If you get this all correct, then emails identified by ASSP will have their headers marked with

 Subject: [TRAPPEDSPAM] ...

Sendmail will see that in the subject and discard the mail!

Note that this setup does mean that all email presented to your server will be downloaded - for busy systems this may be too expensive in bandwith/server load, but it is perfect for us (about 1000 email per day including spam).

Mike

#!/usr/pkg/bin/bash

#
# ASSP watchdog process.
# This should be run from ROOTs crontab.
# joehodge

#
# Start in the correct direcroty.
#
cd /usr/local/assp-1.0.7

#
# Check for a running ASSP process.
#
PROC=`pgrep -f -U 10000 assp`

#
# If ASSP is not running, start it.
#
if [ "${PROC}" = "" ]
then {
echo "Restarting ASSP."
ulimit -d unlimited
perl assp.pl `pwd`
}
fi

*************************************
The route my mail takes -
Incoming
Internet->ASSP(25)->Norton's Anti-Virus for Gateways(225)->SMTP Virtual Server(125)->Firewall->Exchange Server(25)->Clients

Outgoing
Clients->Exchange Server(25)->Firewall->ASSP(25-2nd IP)->Norton's Anti-Virus for Gateways(225)->SMTP Virtual Server->Internet

*****************************************
The SMTP Virtual Server -
Assign 2 IP addresses to your Windows 2000 mail server. As an example I will be use private addresses:

192.168.0.1 This will be the address that ASSP listens on. It is also the address of the Default SMTP Virtual Server.
192.168.0.2 This will be the address of the relay port used by Exchange as a "smart host" to forward all outgoing mail to.

Install IIS.
Open the Information Systems Manager.
Right click on the Default SMTP Virtual Server and select Properties.
Select the original IP address for your server. In our example this is 192.168.0.1.
Click the Advanced button, then click the IP address in the list and click Edit. Change the port to 125. Click OK, then Apply and OK.

Right click on the Default SMTP Virtual Server and click New, and select Domain.
In the "Create New Domain" Wizard choose "Remote" click next, type your local email domain domain.com (you don't need the @ symbol) then click ok.

In the left pane of the Information Systems Manager expand your Default SMTP Virtual Server, double click on the "Domains" folder, right click on your domain name and choose Properties. In the Properties choose "Allow all incoming mail to be relayed through this server." and select "route all incoming mail to this host" and type in the IP address of your Exchange Server(or the IP that answers for it on your firewall)and click ok.

Right click on the Virtual server and stop it.

Installing Norton anti-Virus for Gateways -

Double click on the NAVGW icon. Logon and click on the "Configure" button.
Configure notification - Do NOT SEND NOTIFICATION TO SENDER! That will add spammers to your whitelist! Click save.
Click on the SMTP Tab, set SMTP Port to 225 click save.
Click on Relay tab, add 127.0.0.1 port 125 as the default relay host to route all mail to. That will send everything to the SMTP Virtual server after being scanned. Click save.

Start ASSP.

Goto http://localhost:55555

Click "Show Advanced Configuration Options, scroll to the bottom and click
Apply Changes.
in the "SMTP Destination" enter the IP:port of NAVGW - 127.0.0.1:225

Add your local subnet to "Accept All Mail" - 192.168.
Add your local Domains - i.e. domain.com

In the "Relay Host" enter the NAVGW port - 127.0.0.1:225
In "Relay Port" enter the 2rd address we created - 192.168.0.2:25.

Configure all other parameters as you see fit and click "apply changes".

Shut down ASSP.

On your Exchange Server, in the "Connections" tab of the IMS, choose "Forward all messages to Host:" and put 192.168.0.2 in the box.

Start the Virtual SMTP server.

Start ASSP.

Ged West

In versions before 7.10 you have to change the listening port for SMTP in the Windows Registry.
1: Run regedit
2:Go to HKEY_LOCAL_MACHINES/System/CurrentControlSet/Services/SMTPD32/Parameters
3:If you do not have a Value named "Port" on the right side, then you will need to create one.
4:Click on edit | New | DWORD value , name it "Port" and hit enter(exclude the " marks).
5:Double click on this new Port key and change the value to whatever port you want SMTP on. Set the port number as decimal rather than hexadecimal.
6:Stop and restart the SMTP service.
Versions 7.10 and later you can change the SMTP port on the Advanced tab in the IMAIL Administrator | Localhost | Services | SMTP .

Hope this helps...
Bill Somers

In /etc/postfix/master.cf, there is a line that says:
smtp      inet  n       -       n       -       -       smtpd

Change that to be:
localhost:smtp   inet  n       -       n       -       -       smtpd

Then restart postfix.

Then, tell assp to listen to <youripaddress>:25 and forward to localhost:25.

, ml

Answers in this category:
Security Considerations
Theory of Operation
ASSP uses a content filter – won’t spammers disguise their content?
If everyone we email gets added to the ASSP whitelist, won’t spammers just use an address from the whitelist to spam us?
Will ASSP block messages I want to receive?
One man’s spam is another man’s ham – how does ASSP decide what to block?
Will ASSP work with non-English languages?
I want to mess with the mail collections. What format are they in?
I’ve heard content filtering is CPU intensive. Is ASSP a CPU hog?
I want to add per-user settings. How hard is that?
Is it required to take down (stop) assp to do rebuildspamdb & dnsbl?
How does ASSP compare to SpamAssassin?
What is the difference between the redlist, no-processing, and spamlover lists?
What is "cache reset" in the log file?
What is "helo rndhelo" on the analysis page?
I've seen discussion of configuration settings that aren't on my config page. What do I do?
How really does ASSP detect spam?
What is goodhosts and what does it do?
What is the http ://[\w\.]+@ default expression to identify spam?
Why does ASSP only show one recipient per message in the maillog.txt file?
Virus blocked -- what was blocked and why?
Can I delete files from the spam / notspam / other collections?

In a *nix environment you will want to use ASSP’s ability to run as a non-root user. You may also consider running it in a chroot jail. To do this set the ChangeRoot variable in the configuration to set to your ASSP directory and copy (or link) the /etc/protocols file into a etc/protocol file in the ASSP directory.

The collections of spam and non-spam email may represent a security risk, and access should be restricted to mail administrators. The non-spam email collection will certainly contain sensitive correspondence, and steps should be taken to protect it from those who don’t require access.

Your administration password is transmitted with basic authentication (ie no encryption). If you plan to use the web interface from a host where you feel sniffing is a possibility I’d recommend installing stunnel (www.stunnel.org) to create an encrypted tunnel for your web-admin sessions. The password is stored in plain text in the assp.cfg file -- make sure file permissions protect this file from read access for unauthorized users. You can also add ip addresses to the Allow Admin Connections From configuration entry to restrict access to the admin interface, although this type of packet is quite easy to spoof.

Every time a message passes through your SMTP server it has a from address and one or more to addresses. Your SMTP server also knows if the message is being sent from your local network (and to allow relaying for that message), or if it’s coming from outside (and must be delivered to a local address). Your local users don’t send unsolicited email (right?) and the people they correspond with would only send you solicited email. In fact the people they email would also be unlikely to send UCE. By monitoring these addresses ASSP builds a web of trust – local users are trusted, the addresses in their TO or CC fields are trusted, as are the addresses in their TO and CC fields. Any email from these people is considered not-spam without further checking. (Note this is not a good strategy for virus containment, but it is a good strategy for UCE.)

Users of the local mail domains are not added to the whitelist. They are identified by being a part of the local network. Many spammers forge a from addresses with the same domain as the to address, so it is important to avoid adding local addresses to the whitelist.

With only a few days of operation you should see your whitelist grow to more than 1000 addresses. The whitelist is not only helpful in identifying non-spam, but in building your database of non-spam emails. The whitelist is automatically saved every $UpdateWhitelist seconds (1 hour by default).

Spambuckets are addresses which receive only spam. They can be integrated on your web site, posted on Usenet, or come naturally by having employees leave your site; after a reasonable period of time bouncing their mail all mail received for these addresses can be considered unsolicited. Any email whose sender is not whitelisted and is addressed to a spambucket is classified as spam. Spambuckets are helpful both in identifying spam, and in building and maintaining your spam database.

Finally, if an email comes and is not addressed from someone not on your local network, nor on the whitelist, nor addressed to a spambucket, it is compared to the statistical profile generated by the Bayesian filter. The Bayesian filter works by looking for words and phrases (up to three words long) that occur significantly more often in either your non-spam collection, or your spam collection. For most organizations spam identifiers include things like “get rich quick” while non-spam identifiers are things like your organization’s full name or address, or personal names of people who work there. They also include considerably more subtle references like HTML tags which spammers prefer, or jargon specific to your line of business.

To classify a new email all the words and phrases in the first 10000 bytes of the email (including the header) are checked against the statistical model. The top 50 ranking words and phrases are combined according to Bayes theorem to predict how well the mail compares to spam / non-spam in your collections.

I have made the working assumption that only the first 10000 bytes of an email are significant for identifying spam. Spammers may change their profile, but historically spam has been relatively small, and keeping many large files in your collection is a waste of disk space and processing time.

After an email is classified as local or whitelisted, or as Bayesian spam or spam to a spambox its first 10000 bytes are are saved in the appropriate collection directory. It is given a random number between 0 and MaxFiles (12000 by default) and written to that file name. In this way older files will gradually (randomly) be replaced with newer files, thus keeping the collections both diverse and up-to-date. Files in the errors folders (correctedspam and correctednotspam) are never overwritten.

What follows is a sample statistical analysis of mail we received:

 As of Thu Mar 27 10:48:54 2003 the mail logfile shows:
 78843 messages, 47637 were spam (60.4%) in 73 days
  for 1080.0 messages per day or 652.6 spams per day
 8303 additions to / verifications of the whitelist (113.7 per day)
 28273 were judged spam by the bayesian filter (59.4% of spam)
 18862 were to spam addresses (39.6% of spam)
 502 were rejected for executable attachments (1% of spam)
 12608 were sent from local clients (40.4% of nonspam)
 7838 were from whitelisted addresses (25.1% of nonspam)
 10760 were ok after a bayesian check (34.5% of nonspam)
 14467 addresses are on the whitelist
 15108 hits on the blacklist
 14890 resulted in spam (52.7% of Bayesian spam, 98.6% of blacklist hits)
 218 resulted in non-spam (1.443% of blacklist hits)
 

The type of email that most often falls in this category is confirmation emails from web sites. Often these mails are only as personal as your email address and contain a lot of advertising – they look a lot more like spam than they look like the mail you send. If someone has a good idea how to recognize this type of email please let me know.

As of ASSP 0.3.4 we have active users working in Spanish, French, and German without problems.

As of version 0.3.4 ASSP also began to track helo phrases passed in the SMTP conversation -- see the format of the ASSP received header line to see how this should be formatted.

> 2. if not ... why

I used spamassassin (www.spamassassin.org) for some time prior to developing ASSP. I found SA difficult to install. It also had to be regularly upgraded. Finally, ASSP's Bayesian filter was more effective at stopping spam than SA. I understand that since then SA has developed a Bayesian component as well, but I'm not completly up-to-date on their development.

> 3. what are the pros of SpamAssassin compared to ASSP

SA has a great investment in hand-made regular expressions and header analysis to recognize spam.

> 4. what are the cons of SpamAssassin compared to ASSP

These same hand-crafted expressions are brittle as spammers adjust their strategies. ASSP relies on the flexibility (and customization) from your own site's Bayesian database. Furthermore, ASSP is a complete spam blocking solution, not just a filter that must be integrated to your mail transport.

I credit SA with some of the impetus for getting ASSP going -- it is a great tool with a lot of features. In fact SA's smtp proxy was part of the inspiration for ASSP. And I would cheer them on -- every effective anti-spam tool reduces spammer's success and makes spam less profitable

However, my goal was to have a system that was easy to install, worked unmodified with nearly every MTA on any OS, and I believe ASSP is achiving those goals. Yes, a competant Linux system administrator can probably achieve similar results with SA, but ASSP broadens that opportunity 100 fold.

I trust you will find the best tool for your situation.

[ filtered mail | unfiltered mail ] x [ contributes to whitelist | doesn't contribute ] =
filtered & contributes = normal
unfiltered & contributes = spamlover
filtered & doesn't contribute = redlist (does contribute to spam/nonspam collections)
unfiltered & doesn't contribute = no processing (also doesn't contribute to spam/nonspam collections)

If one of your cache is resetting more often then every 7 minutes, then change the line where it says, "if($this->{cnt}++ >5000" and change the 5000 to 20000. This will make ASSP use more RAM but give you better performance.

Note that after one of the databases has been updated (whitelist, redlist, spamdb, or dnsbl) an average of 255 hits on that database you'll get a "cache reset" because ASSP noticed that the file modification timestamp changed. However new data can be read from the file from the moment it's updated -- it's only cached data that won't be re-read.

As of version 1.0.0 the cache size is in the configuration options.

Unfortunately, a bug in versions prior to 0.3.5 meant that all messages without a header are interpreted as randomhelo greetings (or rndhelo).

Second, the wording may have changed, or an abreviation may have been used -- look for another setting with a similar use. For example, WhiteRE is actually, "Expression to identify Non-Spam."

The rebuildspamdb program is where I will start. It reads the files in your errors/spam, errors/notspam, spam and notspam directories. As it reads the files in the errors directory it also builds a hash of the mail body to be able to identify duplicate messages misfiled. This hash is used to delete messages from the notspam collection that were also in the errors/spam collection and from the spam collection that were also in the errors/notspam collection. Think of it like scrubbing bubbles – they do the work so you don’t have toooo!

As rebuildspamdb reads the files it also does two things. First it runs a filter (the subroutine “clean”) that prepares the message for statistical analysis. Second it walks through the file tallying word pairs in the spam or not-spam categories according to the collection. Files in the errors/spam collection count double; files in the errors/spam count x4.

The “clean” subroutine does a number of important operations. Primarily its function is to undo the things spammers do to trick filters. It cleans up base64 encoding. It cleans up many HTML obfuscation techniques. Look at the code of the “sub clean” for more details – it’s all commented. It also does two other things (and may do more in the future) to help the Bayesian analysis. First, it inserts a keyword after each word of the subject – this lets the Bayesian filter recognize words in the subject uniquely. For example the word “free” in the subject will have a different Bayesian rating than the word “free” in the body of the message. Second it does a couple of tricks to isolate the “HELO” greeting that was sent when the message was delivered. This has also proven to be a useful Bayesian factor in identifying spam.

Paul Graham’s “A Plan for Spam” recommends complete header analysis within the Bayesian filter. Because ASSP initially used three-keyword identifiers, and now (as of 0.3.4) two-keyword identifiers, I found this useless. However, header analysis will be a fruitful area of development for improving ASSP’s spam / ham recognition rate in the future. That will take place in the “clean” subroutine. There may be other pre-processing features that will be introduced there in the future.

Once each mail message is pre-processed (cleaned) each word pair is tallied (words being defined as [-\$A-Za-z0-9\'\.!\240-\377]+ – shorter than 2 or longer than 19 are ignored and are further cleaned in this way: s/[,.']+$//; s/!!!+/!!/g; s/--+/-/g;) [Sorry for the technical stuff for those allergic to it.] So that in the end you end up with a big database of word pairs and their counts: “in the”: spam=23210, total=46411; “order now”: spam=20001, total=20121. The rebuildspamdb program then steps through this database discarding identifiers with total less than 5 (i.e. if a word pair occurred 4 or fewer times in all the collections combined and with errors/spam x2, and errors/spam x4 then the pair can be ignored) and calculating the spaminess ratio this way:

If the spam count = 0 or the spam count = the total count then square both counts. (This amplifies factors which appear only in the spam or not-spam collection.)

Spaminess = (spam count + 1) / (total count + 2) (This should look familiar to anyone with a basic understanding of Bayesian filters. It also somewhat de-emphasizes rare identifiers and emphasizes common ones.)

Throw out the identifier if it’s between 0.41 and 0.59 – this identifier appears almost equally in both spam and non-spam there’s no point in keeping it.

Force the result between 0.999999 and 0.000001 – Bayesian classifiers croak if the value is too close to 0 or 1.

All of these results are sorted (by identifier) and stored in the spamdb for use by ASSP.

Rebuildspamdb also randomly (1 time in 20) prunes outdated entries in the whitelist and goodhosts databases.

Now you know how the spamdb is built, so let’s see how it is used.

Suppose a mailer in the internet connects to ASSP. ASSP makes a connection to your “SMTP Destination” and begins relaying their conversation. It notes the IP address of the connecting server. It notes their HELO string. It notes their MAIL FROM (envelope sender). It notes their RCPT TOs. It notes their DATA directive. (This is all in sub “getline”.) Relay attempts are blocked. The presence of spam bucket addresses is noted. Mail to the email interface is detected. Mail to no-processing or “spam lover” addresses is noted. Assuming none of that qualifies the message is passed on to “getheader.”

Getheader is looking for the mail header. When the header is complete getheader calls “onwhitelist” which determines if the message should be treated as whitelisted/local (it’s the same really) and if so to update the whitelist. If not processing goes on to “getbody.”

Getbody reads the rest of the message (or the first 10000 bytes including the header, which ever comes first), checks for attached executables (if that’s enabled) and calls “isspam” which is probably why you’re reading this document.

The isspam subroutine first checks WhiteRe and BlackRE, the expressions to identify non-spam and spam, respectively. Then it calls “clean” to clean up any spammer obfuscation, and calls them again with the “cleaned” version. Then it checks for a DNSBL hit, which adds 0.97 twice to the list of Bayesian factors for this message. Then it checks for a goodhost miss, which adds whatever your site’s goodhost factor is twice, provided it is > 0.65. Then it walks through the message’s word pairs, just like rebuildspamdb did, completing the list of Bayesian factors. Unlike rebuildspamdb, an identifier hit will only be counted a maximum of two times, so if the identifier “free money” rates 0.955 and “free money” occurs three or more times in the mail message, only the first two count.

The list of factors is sorted and the thirty factors closest to 0 or 1 (i.e. the 30 furthest from 0.5 or neutral) are combined as Bayes taught into a single probability. If this probability is greater than 0.6 the message is spam. (Mail is very rarely between 0.2 and 0.8 – it’s almost always > 0.9 or < 0.1.)

Spam is logged in the spam directory and local and whitelisted mail is logged in the notspam directory. Headers are updated as configured. If you’re not in test-mode the connection to your “SMTP Destination” is dropped if it is spam, and when the client stops spewing the mail body, it gets the “spam error” message, and it’s connection is dropped. (In test mode the connection is completed and ASSP sends updated headers.)

I noticed that we were getting a number of spams slip through the filter all with the same qualities: they were short, they were deliberately misspelled on many words, and they linked to some website.

I started doing some research on (a) why they got through, and (b) how to block them.

It turned out that because of the shortness and misspellings many passed through without any hits in the bayesian database, good or bad.

One solution would be to assume that all mail is just a little spammy and then force the content to justify itself before being allowed to pass. This would have the added effect of possibly raising the false positive ratio, although I didn't research it to be sure.

But further research revealed something more useful.

Because ASSP keeps a whitelist, it is a trivial addition to track what hosts send whitelisted mail. A site of any size will quickly get AOL, Hotmail, and a few others on that list -- they'll also get their organizational partners on it quickly. This is the goodhost database, and it represents a sort of social network for your email. You're likely to email them, and they're likely to email you. Doing the math for our site I found that less than 1% of mail from these goodhosts is spam. And 89% of spam was from a not-goodhost. Each site's ratio will be different, but I expect that the goodhost marker is a healthy sign that an email is not spam.

So the goodhost database is sort of like a inverse-dns-blacklist that you don't have to download. Hosts absent from the goodhost list will get your site's non-goodhost-spam ratio added to the Bayesian determination, once that ratio is higher than 65%.

Other benifits of the goodhost strategy:
1) requires no download (unlike the DNSBL)
2) totally self-maintaining & tuning
3) totally customized to your own site's traffic patterns
4) unspoofable by spammers
5) this is exactly the sort of push that these short & misspelled mails needed to correctly fall into the spam pit.

This is a good reason to tell your friends about ASSP -- it's only the best anti-spam tool in existance... And it's free.

It is most often used to trick the readers eye as http://www.mcafee.com@spamsite.com/securitypatch.exe "looks" as if it would connect to the trustworthy "www.mcafee.com" site where in reality it connects to "spamsite.com" with a "username" that is "www.mcafee.com". If this website does not need authentication (and they never do), then the username part is discarded.

By using this expression you will quickly sort out a bunch of Spams, that in turn automatically provide you with suitable spamwords. I found no need to manually add more expressions.

(Robert Orso: 11-17)

The log file gives you the time and sender (though the sender is often faked, but it IP address would be right). If you use the "other" folder "External mail that wasn't spam (mostly)" you can find a copy of what was blocked there, though it's only the first 10k. That might be enough to try to recognize what was sent, either by inspecting the file or by running a virus scanner. (You can identify the file by the creation date/time -- it will match the time in the log entry.) Files don't stay there forever, though.

The spam and notspam files are used by rebuildspamdb.pl to create your spamdb. Do not delete these files unless you become aware that your spam collection is hopelessly corrupted and want to start from scratch, categorizing spam and notspam by hand.

Answers in this category:
ASSP + Exchange = 99.86% success with no false positives.
From the Slovak Republic
Nonprofit & Exchange -- 100,000 spams per week no more.
Win2k3/ASSP + Win2k3/Exchange2k3

Between Oct 1 & 15, my company has received 182,055 e-mails. This team of products has provided us with a 99.86% spam kill rate. False positives...0, zero, zilch, nada, nothing. Since we don't have Outlook 2003 completely rolled out, this number is anticipated to improve another .1%.

Also, SPAM average has decreased by 2% since we implemented it in September. I'm not sure what caused this.

jasont80 ( Jason Thompson ) 11-19 23:58

I was really surprised and I' ve desided to try assp in production environment of our small company. Now it is one month of my testing and assp really solve our spam problems. I have only one!! spam reported. (But some good mail was lost in the beggining and version 1.0.6 has crashed two times.)

So I want to tell you big thank for this good job.

See you.

Patrik.

OUR SOFTWARE ENVIRONMENT
We use Exchange 2000 so our configuration is: Internet -> ASSP -> exchange 2000 -> Outlook Client Our server runs Windows XP Corp Edition. We are using ActivePerl with Win32::Daemon – everything was installed exactly as per the directions on the ASSP website. I also installed Norton AntiVirus Corporate Edition 8.0 and WinConnect Server XP so that I can manage the server from home.

OUR HARDWARE ENVIRONMENT & STATS
We are running it on an old Pentium II, 300 MHz with 256MB of RAM. CPU utilization is about 4%. During the first week of operation:
- about 100,000 spams were rejected
- about 6,000 good email messages were passed through to our exchange server.
- About 600 relay attempts were rejected.
- “netstat” reports that we received 700,000,000 bytes and sent 300,000,000 bytes (this works out to 2,000 bytes/second)
- 100% uptime was enjoyed

-grenfell

Answers in this category:
Linux, Qmail, Cyrix 233mz
Exchange 2000, 1066 messages / day
WinNT, Post Office, 500 mz
Redundant MX, Windows 2000, McAfee, Exchange
Postfix + Amavis, Redhat, 450 mz
Exchange + NAI Webshield + PIII-500
Win2k, GFI Mail security, k6/2 350
Mac OS/X and qmail
Dual Piii-800, Win2k, Imail 8
Win2Ksrv, Lotus Notes/Domino, ASSP
Assp - Exchange - Assp - Mercury/32

Incoming Network Path:
  internet -> (25) assp -> (55554) qmail -> (110) pop3 client

Outgoing Network Path:
  client -> (25) assp -> (55554) qmail -> internet

Operating System:             linux kernel 2.4.21,  slackware 9.0.0
CPU:                          Cyrix M II, 233MHz  (no, I am not joking)
RAM:                          256M
Messages per day:             263
Load: (best guess)            0.03 (not loaded at all)
Mail Transport:               qmail
smtpDestination:=             127.0.0.1:55554
listenPort:=                  25
Mail Transport's Listen Port: 55554



What was the hardest part of getting ASSP working for you?
  * To find out by trial and error what features that did not work for me.
  * To find out which announced new features are working for me when
    upgrading to newer versions.
    Examples: Complete email addresses in "spamaddresses" are ignored but
    username only is fine, greylist download starts every hour but no list
    is saved, "redRe" is ignored and such minor quirks.


What do you wish you knew before you started that you know now?
  * Exactly what files and directories that has to be owned and/or writeable
    by the assp user.
  * A rough estimate of how much memory "rebuildspamdb" would require
    without RamSaver, in my case around 130MB with 23000 messages total.


Any other comments regarding installing / configuring ASSP?
  Easy to install (compared to qmail, at least).
  It works very well and does not require much maintenance.

  I suspect that the files in "asspsmpl/notspam" in "asspsmpl-0.1.tgz"
  are helpful for installations that handle mostly english content.
  In my case it is still the major part of the files with english content
  in the "notspam" directory. Over time it will be replaced with mostly
  swedish content so I will have to copy it back to "notspam" from time
  to time. It is the other way around in the "spam" directory, almost only
  english and very little swedish content since most spam is written in
  english. The consequence is that assp has a small tendency to let
  swedish spam through and to classify english content as spam.
  This is not a big problem though, assp works very well even with
  mostly swedish notspam and english spam.


-----------------------------------------------------------------------
used assp versions:                all from 0.3.1 to 1.0.7
current assp version:              1.0.7
main language in filtered mails:   swedish
time to rebuild database:          2462 seconds, around 23000 messages total



Line in "/etc/rc.d/rc.firewall" to stop others from accessing qmail and
web interface directly:

  iptables -A INPUT -p tcp  --dport 55554:55555  -i ! lo  -j DROP

  (drop incoming packets that are destined to tcp port 55554 to 55555
  and originates from anywhere but this machines local interface)




Lines in "supervise/qmail-smtpd/run" to get qmail to listen on port 55554:

  exec /usr/local/bin/softlimit -m 2000000 \
    /usr/local/bin/tcpserver -v -R -l "$LOCAL" \
    -x /etc/tcp.smtp.cdb -c "$MAXSMTPD" \
    -u "$QMAILDUID" -g "$NOFILESGID" 0 55554 \
    /var/qmail/bin/qmail-smtpd your.servername.org \
    /bin/cmd5checkpw /bin/true  2>&1



-----------------------------------------------------------------------
Not used:
  relayPort:=
  relayHost:=

-Rutger E

To get EXIM MTA to listen on another port eg.10025 just change
the setting in /etc/services to read:

smtp             10025/tcp         mail         #Simple Mail Transfer
smtp             10025/udp         mail         #Simple Mail Transfer

then execute "killall -HUP exim"
Exim picks up the port settings here.


THanks for a super program

 Internet -> (25) ASSP -> (25) McAfee Webshield e250 -> (25) MS Exchange 2000 -> Outlook Client

						

 Client -> MS Exchange 2000 -> (25) ASSP -> (25) McAfee Webshield e250 -> Internet

						

-Morgan E

If you run a virus scanner, how does it fit in?
On access scanner running locally on the machine

What was the hardest part of getting ASSP working for you?
Getting Perl to run as a service.

What do you wish you knew before you started that you know now?
Not much now. The web interface has really taken care of most of my issues (how mail is scored, the maillog tail etc.)

Any other comments regarding installing / configuring ASSP?
For us ignorant win32 users, always use "/" for file paths instead of "\". Seems pretty obvious now, but for those of us with limited programing experience... The half gig of ram is also important for us. The rebuildspamdb used to tank our server everytime, that was back in the pre 1.0 days though...

AJ

Four servers just for mail may sound over the top, but it works for us, and our business is critically dependent on it (if something goes wrong, I get a complaint within about 10 minutes). Anyway, we like servers; we have 15 of them; we also host our own web site, ftp and forums.

If I could get a virus scanner running in a hook in Weasel, that would have the benefit of enabling me to use SMTP auth, and to reject email to invalid users immediately.

======================================

Incoming Network Paths:
Internet -> [first MX] (25) ASSP -> (125) McAfee VirusScan SMTP -> [distribution server] (25) Weasel (some pop3, and:) -> [office mail server] (25) Exchange
Internet -> [second MX] (25) NAPT router -> (125) ASSP -> (25) McAfee VirusScan SMTP -> [distribution server] (25) Weasel (some pop3, and:) -> [office mail server] (25) Exchange

Outgoing Network Paths:
[office] Outlook -> Exchange -> [first MX] (25) ASSP -> McAfee VirusScan SMTP -> [distribution server] (25) Weasel -> Internet
[other] Client (SMTP) -> [first MX] (25) ASSP -> McAfee VirusScan SMTP -> [distribution server] (25) Weasel -> Internet

Mailshots go out through the VirusScan on the second MX (which is why the ports are reversed there) rather than via ASSP, so that the mailing list addresses (which are not always trustworthy) don't get whitelisted.

Operating System:

 [first MX] Windows Server 2003
 [second MX] Windows XP
 [distribution server] OS/2

						

 [first MX] 2 x P-II/400
 [second MX] AMD K6-2/500
 [distribution server] P-III/600

						

 [first MX]  640MB
 [second MX]  256MB
 [distribution server]  256MB
 

 [first MX] 2200
 [second MX] 750

						

Note that these figures include the email virus scanning as well as the spam scanning.

Mail Transport:

 McAfee VirusScan SMTP - has rather flexible routing, but no auth :-(
 Weasel - an OS/2 MTU, with some features to my spec :-)
 Exchange - no comment

						

I started trying to run it on the OS/2 machine, and the spam rebuild crashed Perl 5.8.0 on OS/2 (I attempted to debug this with the person who did the port, but he felt in the end that it was a memory-handling issue in the compiler library). I see that others are now running it on OS/2, so that crash might have gone away with the redesign of the spam db. [note, perl 5.6 may also be a good choice.]

I had a pre-existing email corpus from running another client-based Bayesian scanner for a while, so I started essentially straight in. Suggestions of corpus size are very variable at different sources - the writers of the Bayesian filter in the Polarbar mailer (www.polarbar.org) recommend as small as possible (say 500/500), and others recommend around 5000/5000. I am using 32767/32767, as I feel that the statistics can only be improved by quantity; and I have noticed many identical spams, so the number of /distinct/ messages may be appreciably less than expected.

My boss (now ex-boss) has installed it elsewhere, and repeated gets confused about the different ports for relay; certainly I had it easy as I was already running my own relay server.

What do you wish you knew before you started that you know now?

That valid SMTP names containing a % would be miscategorised by ASSP as relay attempts :~) This is because I use a manufactured reply address for mailshots that happened to use a %, and I got a lot of flack when suddenly all the replies to our mailshot got bounced. I use another character now...

Any other comments regarding installing / configuring ASSP?

The documentation is still a bit geeky, but I guess that most people who run SMTP servers are geeks ;-)

Regards,
Paul Hodges

 internet - (25)assp - (225)postfix - (10024)amavis-new - (10025)postfix - (pop3)client
                                                                  |
                                                         (3310)clamd

							
 client - (25)assp - (225)postfix - (10024)amavis-new - (10025)postfix -internet
                                                                  |
                                                         (3310)clamd

						

If you run a virus scanner, how does it fit in?
called by postfix with the help of amavisd-new

What was the hardest part of getting ASSP working for you?
finding out that the files have to be run thru dos2unix and of course: On the older versions the problem with chroot/change user

Enrico

Outgoing Network Path:
Exchange-Cloud with Outlook and IMAP/POP/Web-Clients -> (25) Exchange2K-Frontend -> (25) ASSP -> (26) NAI Webshield -> (25) Exchange2K-Frontend -> Internet
Operating System: MS Windows 2000 Server
CPU: Pentium III 500
RAM: 640MB
Messages per day: 11000
Load: 10% (rebuild not included)
Mail Transport: NAI Webshield
SMTP Destination: 141.21.6.1:26
Listen Port: 141.21.6.1:25
Mail Transport's Listen Port: 26
Relay Port: 141.21.6.210:25
Relay Host: 141.21.6.1:26
What do you use for your relay host (Software or just ISP if you use your ISP's mail relay): NAI Webshield SMTP 4.5 (Windows)

Mail Transport's Smarthost / Relay Host: MS Exchange 2000 (Frontend)

If you run a virus scanner, how does it fit in?
NAI Webshield (behind ASSP) and On-Access-Scanners on Exchange-Server Attachment-Blocking on ASSP and On-Access-Scanners Server/Client-On-Access-File-Scanners on all PCs

What was the hardest part of getting ASSP working for you?
Planning the path for mail-delivery

Matthias

Outgoing Network Path:
Depending on the mailserver's setup:
Setup #1: Client -> (125) mailserver -> Internet
Setup #2: Client -> (125) mailserver -> (25) ASSP -> (25000) SMTP Virtual Server (GFI MailSecurity) -> Internet

Operating System: Windows 2000 Server
CPU: AMD K6/2 350Mhz (PII 333MHZ as backup computer)
RAM: 192mb (256mb in backup machine)
Messages per day: ~700
Load: it doesn't seem too busy except when it rebuild the spamdb which takes 40 min. Processing time for incoming mails when rebuilding the spamdb doesn't seem to slow down though.

SMTP Destination: localhost:25000
Listen Port: 25
Mail Transport's Listen Port: ?
Relay Port: none
Relay Host: none

If you run a virus scanner, how does it fit in? via SMTP virtual server on same machine as ASSP

What was the hardest part of getting ASSP working for you? Making automatic jobs for rebuilding spamdb, download greylist etc.

What do you wish you knew before you started that you know now? That the two hard drives I initially used would crash ;)

Any other comments regarding installing / configuring ASSP? I'm a totel novice when it comes to smtp, relaying etc. so I was quite surpriced to find how easy it was to get it to work. Still, most of the users don't send via ASSP which means that whitelist addition and a good portion of hammails are missed. The reasons why many users won't send via ASSP is, 1) their mailserver host many different domains and relaying is not per-domain but for all domains, 2) I've been told that message tracking stops at relay servers - so if we ever want to track a message that strand somewhere we can't if we send via ASSP.

Ulrik

  Internet -> (25) ASSP -> (24) Qmail -> (pop3)

						

  Client -> (SMTP 25) ASSP -> (24) Qmail -> Internet
  Client -> (OFMIPD 8025) -> (SMTP 25) ASSP -> (24) Qmail -> Internet

						

If you run a virus scanner, how does it fit in?
ClamAV with QMailScanner for all files not blocked by ASSP

What was the hardest part of getting ASSP working for you?
ASSP support for Non English languages is poor (no SPAM/NOSPAM DB's)

What do you wish you knew before you started that you know now?
The global percentage of SPAM (75%)

Any other comments regarding installing / configuring ASSP?
Must modify ASSP code to support Pop before SMTP with VPopMail.
Guy

 ----------------------------------------------------------------------
 T h i n k    d i f f e r e n t    -    u s e    a    M a c i n t o s h
 ----------------------------------------------------------------------

Incoming Path:
Internet -> (25) ASSP -> (255) Imail -> client
Outgoing Network Path:
client -> (25) ASSP -> (255) Imail -> Internet
Operating System: MS Windows 2000 Server
CPU: dual PIII 800's
RAM: 1gb
Messages per day: 11000
Load: 0-10%, seems to avg about 2% (rebuild not included). 7-11mb
Rebuild: About 50% of one processor and 75% ot the other.  Peak usage: 200mb

If you run a virus scanner, how does it fit in?
Declude A/V Pro plugs into Imail.  This in turn runs F-Prot A/V on every
received message.

What was the hardest part of getting ASSP working for you?
Setting up the program was easy, as was training it.  Introducing the system to
users is where the real work is.

Hot Tips?
1. Schedule a service restart of Perl as part of your daily rebuild process.  I
found that ASSP's memory usage slowly grew to about 60mb of RAM during normal
operations (using a spam db that grew to approximately 30,000 messages) and
no doubt this growth would have continued.  However usage dropped to 6-11mb
with the high message counts once I worked in a daily service restart.  It
works best in a batch file, where that same batch file handles the daily spam
db rebuild.  Be sure NOT to shut down ASSP during the rebuild.  Just cycle
the service quickly.

2. Use the scripting language of your choice to build a whitelist submitter.
Many users, for various reasons, send mail thru a different smtp server than
the one protected by ASSP.  Give them a *secure* form to input email address,
username, password and whitelisted address.  Use your scripting environment
to send an auth'd mail to the assp-white address.  Security for an operation
like this must be well thought-out.

Incoming path

Internet -> ASSP(25/TCP) -> Lotus(125/TCP)

network clients use ASSP too since the
machine sits on a DMZ and only port 25
is exposed to both LAN and internet

CPU barely used, around 2000 msg/day on
regular days (spam-worms may increase the
traffic) the Lotus mailservers run a copy
of Trend MailScan which takes care of any
beast escaped to the ASSP spam filtering

Due to various reasons, I avoided letting
ASSP automatically download the graylist
and I've setup a scheduled batch file as
follows

@echo off
cls
m:
cd\assp
:
:REPAIR
rem --- echo Repairing ASSP databases...
rem --- perl repair.pl
:
:NUMBER
rem --- echo Renumbering stored messages...
rem --- perl move2num.pl -r
:
:GREYLIST
echo Downloading grey list...
if exist greylist.txt del greylist.txt
wget -t 3 -nd http://assp.sourceforge.net/greylist.txt
if not exist greylist.txt goto NOGREY
if exist greylist copy greylist greylist.bak
copy greylist.txt greylist
del greylist.txt
:
:NOGREY
echo Rebuilding ASSP databases...
perl rebuildspamdb.pl
:
:RESTART
echo Stopping ASSP filter service...
net stop asspsmtp
ping -n 1 -w 3000 10.1.1.1 >NUL
echo Starting ASSP with new files...
net start asspsmtp
:
:QUIT
cls
exit

the above takes care to download and update the
graylist and to rebuild the spamdb, after those
operation it will restart the ASSP service (just
to freeup/clean memory) the "ping" to an unused
IP is only used to wait 3 seconds before starting
back the ASSP service, also, the commented code
at the top may be useful either for first runs or
to repair a damaged database (note: you will need
to get a copy of wget for windows for the above to
work correctly)

Martin

1. Most spam has an invalid bounce address, so notifying non-delivery simply bounces to Postmaster, creating even more wasted bandwidth. Failing to notify non-delivery is a problem for false-positives. The SMTP server is the only place where spam can be stopped before entering your system.
2. The only possible feedback for spammers to void an address is from the SMTP server.
3. Spam that passes through your SMTP server into mailboxes incurs cost to your organization: storage, transmission, backup, deletion – in all these ways spam costs you money. The only way to minimize cost is to reject it at the initial point.

However, mail transport systems are slow to adopt new technology, and spammers are quick and flexible, able to adopt new technology as quickly as it becomes available. Consequently, most SMTP servers are ill equipped to stop spam.

Furthermore, one spam-stopping solution could work with all existing SMTP servers if it was implemented on a second level – a transparent SMTP filtering proxy. This was my goal for this project.

I wanted a server that accepted connections on port 25, passing the transmission on to the official SMTP server, and relaying its replies back to the SMTP client. But when enough of the message had been transmitted to validate its legitimacy the ASSP server could either pipe the remainder of the message to the official SMTP server or close the connection to the official SMTP server and ignore the remainder of the message.

The Anti-Spam SMTP Proxy (ASSP) Server project aims to create an open source platform-independent SMTP Proxy server which implements whitelists and Bayesian filtering to rid the planet of the blight of unsolicited email (UCE). UCE must be stopped at the SMTP server. Anti-spam tools must be adaptive to new spam and customized for each site’s mail patterns. This free, easy-to-use tool works with any mail transport and achieves these goals requiring no operator intervention after the initial setup phase.

Who is it not for?

ASSP is not for the following:
1) Individual clients -- ASSP must be installed together with a SMTP server.
2) Domains which receive mail indirectly, fetchmail for example.

1. Easy browser-based setup (optional).
2. Uses your existing message transport and existing SMTP server.
3. Works with Sendmail, qmail, Postfix, Imail, Exchange, Courier, Mercury, Lotus Notes, and all other standard SMTP servers.
4. Runs on Linux, Unix, Windows, OS X, OS/2, and more.
5. Automatically customizes to your site’s unique email profile.
6. Automatic whitelist – noone you email will ever be blocked.
7. Senders receive immediate notification if mail is blocked but no erroneous bounces are ever generated.
8. Redlist keeps an address off the whitelist.
9. No-processing addresses pass through.
10. Makes use of honeypot type spambucket addresses to automatically recognize spam and update your spam database.
11. Bayesian filter intelligently classifies email into spam and non-spam.
12. Supports additional site-defined regular expressions to identify spam or non-spam email.
13. Can optionally block all non-whitelisted email (for anti-spam zelots).
14. Mime encoded and other camouflaged spam is also recognized.
15. Automatically maintains the spam and non-spam databases.
16. Accepts whitelist submissions and spam error reports by authorized email.
17. Optionally rejects executable attachments from non-whitelisted (or all) addresses.
18. Free (as in speech) software -- Licensed under the GPL.
19. Practically no maintenance required.
20. Active user community and email list for support questions.
21. Source code included if you need to customize your installation.
22. Runs as a service in Windows NT / 2000.
23. Optionally blocks no mail but adds an email header and/or updates the message subject.
24. Optionally uses community-based spam statistics to identify hosts that are likely to send spam or non-spam mail.
25. In *nix environments can switch to non-root user. Also supports chroot jail.
26. Individual users can be configured to receive all mail.
27. Shows detailed analysis of spam rating process for specific messages.
28. Option to forward a copy of every rejected mail to an address.
29. No perl modules to install -- runs complete "out of the box".
30. Can block spam-bombs (when spammers forge your domain in the from field).
31. Keeps spam statistics for your site.
32. Detailed up-to-date documentation.
33. Can listen on more than one smtp port.
34. And much more!

However, this is NOT TRUE with ASSP.

ASSP is the ONLY solution I am aware of (I haven't made an exhaustive search) that when spam is blocked the SENDER RECEIVES NOTIFICATION, and it does this without generating non-delivery reports that bounce and bounce again because spammers forge their from address.

Imagine three scenerios:

1) No filter. A client sends an order by email. Your orders' desk gets roughly 10 spam for every order. Their job satisfaction is low because they spend almost as much time deleting (possibly offensive and sexually explicit) spam as they do filling orders.

2) Client-based filter: A client sends an order by email. Their message is accidently flagged as spam. It is either (a) automatically moved to a "spam" folder, or (b) manually moved there because the clerk always does that with messages marked spam. The customer thinks their email got through, but how long will it sit there before either (a) they call and complain and the clerk finds it mislabeled as spam or (b) the clerk gets around to looking at all that spam and notices the 1 in 100 that was miscategorized.

3) ASSP: A client sends an order by email. Their message is accidently flagged as spam. They IMMEDIATELY receive notification from their email server that the message was not accepted. They either call or email again, saying "How come my message wasn't accepted?" The clerk replies back, saying, "Send it again, it will get through now." And it does because they are now on your whitelist.

But let's think about this more seriously. First it is unlikely (not impossible, but unusual) that you receive critical business correspondance from someone you HAVE NEVER EMAILED before. I don't have a percentage, but I know that business doesn't work that way. Business is built on relationship, and relationship doesn't come out of the blue. Secondly, what foolish mail admin set up ASSP without putting a number of orders or "critical business correspondance" in the notspam or errors/notspam folder? Come-on, folks, this is where we start (if you take a minute to even look at the documentation, and if not, then you get what you deserve). If ASSP is going to give false positives, they're NOT going to be on your "critical business correspondance" -- orders, RFQs, and the like. What gets blocked is Thelma's ebay notification (which is kinda spammy anyway, now that you mention it, it's just that she wants it), or Ralph's porn of the day that he probably shouldn't be getting at the office anyway, but since he's the VP's kid, nobody has stopped him.

Furthermore, you have the opportunity to use "expressions to identify non-spam" to make 100% sure that mail that follows certain patters (like an order would) gets through.

I agree that false postivies are a big deal. But I want to emphasize that ASSP has been designed from the very beginning to handle false positives better than any other solution I have ever seen.

You have to filter. Spam is only going to get worse. If you filter, you will have false-positives -- it's inevitable. What solution has the best means to deal with false positives and still does a reasonably good job filtering? That's the solution to use.