e-mail System Changes
Friday 1 August, 12:30 PM
This announcement is to inform clients of some work
we will be doing throughout next week (during off
peak times) on the email system on our servers.
1. Maildir conversion
At the current time the email storage used on our servers involves all emails for an inbox being stored in one file (which can get large if a lot of emails are stored). We will soon be changing this system so that each email is stored in an individual file within a directory.
Usage of email accounts will not change and clients will be able to access email accounts in the same way as normal. The new system brings some major improvements though:
a) improved support for Imap folders
b) new Imap and Pop software better uses less server resources
c) faster access to inbox via Imap and Webmail
d) no more lock file problems or issues with pop3 checks from multiple sources
Although no problems are expected, the conversion process involves making full backups of all user email accounts. After the work is complete, clients who use Imap and webmail regularly should notice a considerable speed improvement.
2. Spam/Virus Prevention
Spam problems have been on the increase with knock on effects to many different areas especially on some of our servers where spam email accounts for around 10,000-40,000 emails per day. A lot of spam being delivered to inboxes requires more bandwidth, all the server CPU/RAM to process the incoming emails, disk space used up for email storage, CPU/RAM required to download the emails, clients time wasted in dealing with the spam, etc which is something we have been working to improve.
With a lot of work and testing we have now got a better solution which should cut out a much larger proportion of spam emails received per day on each server.
a) Virus scanning and file extension checking will remain in place and these settings are unchanged from what we have been running for a long time.
b) Also remaining in place are the use of DNS RBL (email blacklist checking) on our servers whereby incoming email connections are checked against known blacklists and rejected if listed. The lists we have been using for a very long time are extremely reliable with near zero false positive rate. A new addition to this feature is that we will soon be allowing all clients to disable RBL checks on their incoming email as we are aware some clients prefer that RBL checking is not done. We strongly recommend however that it is left enabled at all times to prevent spam being received.
c) The default Spam Assassin install which has been running up to now is okay but misses a lot of spam email. To help remedy this, we have installed some major improvements to Spam Assassin in the form of addon applications
(SARE (http://www.rulesemporium.com/), DCC
(http://www.rhyolite.com/anti-spam/dcc/), Razor
(http://razor.sourceforge.net/), Rulesdujour
(http://www.exit0.us/index.php?pagename=RulesDuJour)) which are far more effective at detecting spam and auto learning new spam trends on a server-wide basis (not on a per account basis).
This installation of Spam Assassin + extras is intended to be used on all email accounts to replace the 'per user/account' Spam Assassin settings which is configurable via cPanel and has been for a very long time. The system will automatically discard any email messages which have an extremely high spam score and will rewrite the subject of those with a medium spam score (allowing for user filtering via Outlook/Thunderbird/etc). We have had this in testing for some time and it has proved extremely successful. Not only has it cut spam down by ~85% with zero false positives, it has also been seen to decrease server load and result in much better email handling all around.
Due to the far better Spam Assassin we now have ready, we will be disabling all 'per user' Spam Assassin settings when this is installed across our servers resulting in less spam and faster email processing.
d) We have done a lot of work on our Exim configuration to carry out checks for bogus SMTP conversations to cut out a lot of the confirmed spam at SMTP time. This includes non-RFC complient SMTP connections, bogus data being passed (eg fake sender hostname/IP), etc. Clients do not need to do anything to configure this and everything is done from our side.
e) Auto blocking has been in place when email addresses come under a dictionary attack (spammers sending to many random addresses on a domain hoping to get a match). This will remain in place and in addition, we now have auto blocking for servers which have sent a larger batch of confirmed spam within the last 5 minutes. This will halt spammers in their tracks when trying to send a batch of spam together to any email address we host and they will not be able to connect for a period of 5-10 minutes after the spam attack is confirmed.
As mentioned, all of the above has resulted in a massive decrease in spam email in our testing period along with a much more stable server environment as the CPU/RAM resources are nowhere near as stretched (due to not having as much email to process). We recommend that all clients leave everything untouched to acheive the best results in blocking spam.
Thank you for reading and we hope all clients enjoy their improved email experience with less spam to cope with
1. Maildir conversion
At the current time the email storage used on our servers involves all emails for an inbox being stored in one file (which can get large if a lot of emails are stored). We will soon be changing this system so that each email is stored in an individual file within a directory.
Usage of email accounts will not change and clients will be able to access email accounts in the same way as normal. The new system brings some major improvements though:
a) improved support for Imap folders
b) new Imap and Pop software better uses less server resources
c) faster access to inbox via Imap and Webmail
d) no more lock file problems or issues with pop3 checks from multiple sources
Although no problems are expected, the conversion process involves making full backups of all user email accounts. After the work is complete, clients who use Imap and webmail regularly should notice a considerable speed improvement.
2. Spam/Virus Prevention
Spam problems have been on the increase with knock on effects to many different areas especially on some of our servers where spam email accounts for around 10,000-40,000 emails per day. A lot of spam being delivered to inboxes requires more bandwidth, all the server CPU/RAM to process the incoming emails, disk space used up for email storage, CPU/RAM required to download the emails, clients time wasted in dealing with the spam, etc which is something we have been working to improve.
With a lot of work and testing we have now got a better solution which should cut out a much larger proportion of spam emails received per day on each server.
a) Virus scanning and file extension checking will remain in place and these settings are unchanged from what we have been running for a long time.
b) Also remaining in place are the use of DNS RBL (email blacklist checking) on our servers whereby incoming email connections are checked against known blacklists and rejected if listed. The lists we have been using for a very long time are extremely reliable with near zero false positive rate. A new addition to this feature is that we will soon be allowing all clients to disable RBL checks on their incoming email as we are aware some clients prefer that RBL checking is not done. We strongly recommend however that it is left enabled at all times to prevent spam being received.
c) The default Spam Assassin install which has been running up to now is okay but misses a lot of spam email. To help remedy this, we have installed some major improvements to Spam Assassin in the form of addon applications
(SARE (http://www.rulesemporium.com/), DCC
(http://www.rhyolite.com/anti-spam/dcc/), Razor
(http://razor.sourceforge.net/), Rulesdujour
(http://www.exit0.us/index.php?pagename=RulesDuJour)) which are far more effective at detecting spam and auto learning new spam trends on a server-wide basis (not on a per account basis).
This installation of Spam Assassin + extras is intended to be used on all email accounts to replace the 'per user/account' Spam Assassin settings which is configurable via cPanel and has been for a very long time. The system will automatically discard any email messages which have an extremely high spam score and will rewrite the subject of those with a medium spam score (allowing for user filtering via Outlook/Thunderbird/etc). We have had this in testing for some time and it has proved extremely successful. Not only has it cut spam down by ~85% with zero false positives, it has also been seen to decrease server load and result in much better email handling all around.
Due to the far better Spam Assassin we now have ready, we will be disabling all 'per user' Spam Assassin settings when this is installed across our servers resulting in less spam and faster email processing.
d) We have done a lot of work on our Exim configuration to carry out checks for bogus SMTP conversations to cut out a lot of the confirmed spam at SMTP time. This includes non-RFC complient SMTP connections, bogus data being passed (eg fake sender hostname/IP), etc. Clients do not need to do anything to configure this and everything is done from our side.
e) Auto blocking has been in place when email addresses come under a dictionary attack (spammers sending to many random addresses on a domain hoping to get a match). This will remain in place and in addition, we now have auto blocking for servers which have sent a larger batch of confirmed spam within the last 5 minutes. This will halt spammers in their tracks when trying to send a batch of spam together to any email address we host and they will not be able to connect for a period of 5-10 minutes after the spam attack is confirmed.
As mentioned, all of the above has resulted in a massive decrease in spam email in our testing period along with a much more stable server environment as the CPU/RAM resources are nowhere near as stretched (due to not having as much email to process). We recommend that all clients leave everything untouched to acheive the best results in blocking spam.
Thank you for reading and we hope all clients enjoy their improved email experience with less spam to cope with
