Pre and Post greylisting stats
Following on from my previous post about greylisting read the full version of this entry to see some stats before and after deploying it on my secondary MX.
SMTP Greylisting
A few days back Jeremy Zawodny mentioned his intention to test out greylisting on a mail server and mentioned greylistd. I had previously read about greylisting but was not aware of greylistd so moved it down on the TODO list.
During the past few days I fiddled with it and deployed it into my systems. Initially I had it only on my Primary MX which of course did not help much. I did block some spam but the vast majority of spam these days will rather go for the secondary MX. The next day just before 12 AM I installed it on my Secondary while leaving it on the Primary as well and the result was amazing.
Usually by midday my spamassassin and other stuff would have tagged about 50 mails as spam, today they tagged 4 by midday, 2 of those were false positives. I poked around a bit and was annoyed by the delay it introduced in mail arriving, also some large mail setups like those of Gmail and others use MX pools and the mail will not always come from the same MX introducing even more delays, this meant I had to add them to whitelists and also meant I had to be looking at my logs often which I hate doing.
In the end I settled for a setup with the secondary being the only box running greylistd and it works on the C-Class of the sending server rather than just the sending server ip alone. This should hopefully resolve most issues with people who have big MX pools.
I read a bit and came across this page on one of the MSN servers. They seem to have a very interesting take on greylisting which I think I will be implementing in time.
Servers contacting MSN TV mail relays must be able to follow MX chains. MSN TV uses multiple pools, and servers refused access to the initial pool (smtpinvite servers) must retry delivery to the secondary pool (smtpin servers).
Their MX records look like this:
webtv.net mail is handled (pri=10) by smtpinvite.mx.webtv.net
webtv.net mail is handled (pri=20) by smtpin.mx.webtv.net
At the moment their smtpinvite seems to be down, but smtpin is up and it is running postfix
220 smtpin-3308.bay.webtv.net ESMTP WebTV_Postfix+sws (2.1.1/in.gso.28Feb2003) ready to rumble
Anyways, so the way I understand it you would try to deliver the mail to the secondary directly which will not be allowed. If you try and deliver to the primary it will add you to the white list and if you then follow the SMTP chain to the secondary you get let through.
This would – for current generation spammers and virii – catch the same amount of spam as I do with the traditional greylisting setup but without the problem of delays in delivery.
Hopefully this will be effective for a while, I do not intend to stop 100% of my spam else I would put it on my secondary and primary, I am just hoping to make significant gains without imposing a penalty in the usability of my mail system to my users, this seems to be a good compromise.
Some other links to greylisting info:
http://greylisting.org – articles and list of available implementations
Greylisting with MySQL and Exim
Greylisting with PostgresSQL and Exim
Furl Firefox Plugin
I posted about Furl.net some time ago, I have been using it on and off but found it a schlep to use since I have to go and open the site all the time.
Today in their newsletter I noticed they mentioned a Firefox Plugin for their services, I downloaded and installed it and must say I am quite happy with it, will probably use it more often now.
Screenshots and stuff on the above url, way kewl.
mod_gzip
As I mentioned the other day I want to start fiddling with mod_gzip, I did just that and am quite pleased with the result.
More details on the full entry.
iCal, Mozilla Sunbird, PHP iCalendar and RSS
Till now I have been using Remind for a simple daily reminder system. It relies on simple text files and sends me daily emails notifying me about stuff.
This works fine really and I didn’t think I will need much more. Then comes the news that Mozilla has released a standalone calendar app called Sunbird. It is early days for this project but it is already very usable. It uses iCal format files which is compatible with all sorts of other applications and most importantly supports storing its files on WebDAV servers.
A quick install of mod_dav on my Apache web server and a bit of setup using authentication to keep things private got me going. Sunbird has some strange things related to blank files on WebDAV servers so you need to put at least 1 entry into a local calendar then publish that calendar to the WebDAV server to create your new calendar. Once that is done you can just subscribe to it.
This was nice, and I was happy, but I got thinking that I would need to sometimes access this stuff without being near my desktop computers – another great thing about WebDAV, multiple desktops all sharing the same calendar – so I thought it would be nice to get this stuff visible on the web.
A quick search around found PHP iCalendar that can view multiple iCal files in a very sexy looking web interface. So I had remote access to the data all sorted.
The problem with all this is of course were notifications, Sunbird seems to have issues sending mail – for me any case – and I don’t fancy keeping it open forever just to get notifications. I then noticed that PHP iCalendar can export your iCal files as RSS! For each iCal file you can get a day, week or month feed. Pop this into your aggregator and you have a nice place to get notifications as well. This leaves Sunbird only to edit the entries really and keeps things nicely integrated into what is quickly becoming the killer app on my desktop, my Newzcrawler.
Once everything was iCal enabled I had a look at iCalShare.com where there are 100’s of iCal files that you can just subscribe to or copy onto your own server. I got UK and SA holidays from there and will be investigating some other stuff.
Blogcatalog and RSS stuff
Scoble blogged about Blog Catalog which is a kind of yahoo/DMOZ directory effort for blogs, very nice.
On the general usefulness of RSS, reading Scoble’s site again made me realize just how much RSS has changed the way we use the web. Scoble says he is subscribed to more than 900 feeds and has some ideas about information overkill.
I remember my life before RSS aggregators, I managed to read a small amount of sites regularly, perhaps 10 at most, simply because going to each one was just a pain in the behind. I never read peoples personal pages and I never knew half of what is going on there since my main source of information ended up being slashdot. These days it is different, I keep up with around 90 feeds during the day they offer a good 2 minute distraction from work every hour or so. I am also finding myself much better informed about general happening because I can subscribe to a wide variety of feeds.
The fact that one person can have 900 feeds shows you just how far aggregators and the whole RSS technology has progressed and what it has enabled us to do. Yes RSS has flaws that is being worked on, but so far it has done a great job. I think it also says something interesting about the mental stability of someone who wants to subscribe to that many feeds, but that is another discussion ๐
