I’m in the middle of decomissioning some old sites and thought I’d post some info about our FreeBSD 4.x based firewalls that we were running.
Barry and Neil put these together when they were still with iTouch, they are FreeBSD machines running ipfw, modified natd, IPSec and jails for nameservers using bind. They’ve proven incredibly reliable more reliable than anything I’ve every seen before, first some uptimes:
4.3-RELEASE-p28 FreeBSD 4.3-RELEASE-p28 #0
8:56AM up 1175 days, 14:25, 1 user, load averages: 0.01, 0.00, 0.00 4.3-RELEASE FreeBSD 4.3-RELEASE #3: Thu Aug 9 08:24:10 SAST 2001 8:55AM up 1353 days, 13:07, 1 user, load averages: 0.07, 0.03, 0.00 4.3-RELEASE FreeBSD 4.3-RELEASE #3: Thu Aug 9 08:24:10 SAST 2001 8:57AM up 1636 days, 12:16, 2 users, load averages: 0.01, 0.02, 0.00
That last machine was put in the 2nd day I arrived in the UK almost 4.5 years ago now. There has been a few security issues since these were put in, the biggest were Bind issues and a IPSec issue, but none of them really huge deals for us due to the nature of these issues.
Some packet counts through their diverts:
11000 14873464727 9086343964578 divert 8668 ip from any to any via sf0
11010 2694675129 2230790516204 divert 8668 ip from any to any via sf2
11020 21332945704 16515209189995 divert 8668 ip from any to any via sf1
11030 2190579388 1838075424554 divert 8668 ip from any to any via em1
11040 31142270005 26337236597684 divert 8668 ip from any to any via sf3
11000 12363062208 6728197633745 divert 8668 ip from any to any via fxp0 11050 13585672383 7625773331834 divert 8668 ip from any to any via sf0 11075 1672241479 943217267415 divert 8668 ip from any to any via sf1 11000 9709855806 3616673887622 divert 8668 ip from any to any via fxp0 11010 15438460240 7026578427847 divert 8668 ip from any to any in recv sf0 11015 18623997883 6347362524481 divert 8668 ip from any to any out xmit sf0 11020 7574307452 2981257820300 divert 8668 ip from any to any in recv sf1 11025 6957613786 2361008898017 divert 8668 ip from any to any out xmit sf1 11030 5520959014 1551914815579 divert 8668 ip from any to any in recv sf2 11035 8724539029 2097991945468 divert 8668 ip from any to any out xmit sf2 11040 2988122935 604858451646 divert 8668 ip from any to any in recv sf3 11045 3930006137 632095496483 divert 8668 ip from any to any out xmit sf3 11050 3842161713 3177937890519 divert 8668 ip from any to any in recv fxp1 11055 4106903810 3282379599303 divert 8668 ip from any to any out xmit fxp1
These aren’t the bussiest machines by far, but they moved quite a bit of data, keep in mind these counters were probably reset quite a few times over the time to aid in debugging problems. One interface in the top bunch has done 23 TB.
I don’t really like these long uptime machines, they are a constant cause of worry for me, you dont know if all the configs were saved, you dont know if they’ll ever come up after a reboot etc, once you’ve gone over 500 days I think you’re pretty much at a point where rebooting machines becomes a bit of a worry to me, as these are/were firewalls the problem is much worse since the impact of them not booting or configs going missing would be massive, arranging downtime though isn’t always easy either, but I think worth the effort in hind sight.
