Late last week I added my own email to the previous two domains that I had running on Spam Free Email in the BETA stage. I had been getting between 100 to 150 spam messages a day, my email account along is not getting 300 to 500 messages a day and the system definitely needed some tweaking to get it working well again.
The problem turned out to be trying to get too much information out of my MySQL database and into the Erlang system. I had been using some ETS tables to do some caching, but the real solution has been Mnesia.
Mnesia is a distributed database written in Erlang that is in the OTP. After playing with it for a while I implemented it into the system to replace the existing ETS cache and it made a huge difference.
Now that I understand how Mnesia works I can really see how it will end up benefiting the system as a whole. At first the system was taking 1 to 5 minutes to process and email message, with the Mnesia doing the caching that is down to about 1 minute per message.
I am planning on further implementing Mnesia to take over some of the profiling data tasks and I am hoping that I will be able to start talking in seconds instead of minutes by the end of the week.