Spam Free Email

So the FETCH command in the IMAP client gave me some problems. Turns out this command is overly complicated and the syntax that the server return is extremely difficult to parse.

Maybe it was just me, but it did not make sense at first and while it still does not make sense I can now parse it into Erlang data structures.

I still have one or two sub-commands to iron out before I release anything and some bug testing to do as well, but the IMAP client is functional, so long as you know how to use it.

I’m going to start writing my web-mail client this week and hopefully that will find all the major bugs in the IMAP client and I can release it well before the first of the year.

For most of this week I have been working on an IMAP client written in Erlang to be part of the http://www.erlmail.com project and I’m happy to say that I am almost finished with it.

I have two or three commands to finish before I start doing some real world testing and two or three commands that I am not planning on implementing at this time. I haven’t decided if I need the search command for what I am planning on doing next.

I am planning on writing an Erlang based web-mail application on top of YAWS. to my knowledge this would be the first 100% Erlang web-mail application. This will be one of a series of web applications I am writing based on YAWS and the third time I have written a web-mail client. The previous two time I wrote them in Cold Fusion.

If the speed of the IMAP client that I have been working with is any indication of the performance of the web-mail application I will be working on (hopefully before the weekend) I expect it to be amazingly fast with moderate hardware.

Once I get the major features working and the bugs worked out I plan on releasing the very first build of http://www.erlmail.com. This will be Build 0.0.1 and the IMAP client will be the only component that will be working

For the past month I have been working on the http://www.netflixprize.com trying to develop a collaborative filter based completely in Erlang. It’s been great amount of fun and boy have I learned a LOT!

For instance, Mnesia crashes hard when you try to put 2GB of data into 1.5GB of RAM. More importantly I’ve been trying different approaches to store large amount of data into Mnesia in data structures that are easy to work with in Erlang. Coming from an SQL background I was designing the data structures completely wrong for use in Mnesia.

I’ve also been playing with ram_only and disc_only tables in Mnesia and the idea of having ram_only tables on one server and disc_only tables on another server for the same table. This approach finally gave me the room in the RAM that I needed and the fault tolerance I was looking for.

I am actually using 3 Mnesia server, one with ram_only tables and two with disc-only tables and I am doing this with three different tables. This way I can have any two of the server fail and still maintain my system, albeit slow, but when all three system are up it run each table is in RAM on at least on of them.

The experience I have gain so far have changed my ideas on how to design Erlang system and I will be redesigning http://www.spamfreeemail.com with this new knowledge in mind.

I will also be breaking SFE into small segments and using them as libraries, which will be the birth of ErlMail, which is the open source Erlang email server I am starting to write as of today. I’m going to break the SMTP client and server, a POP3 client and server, and an IMAP4 client and server into ErlMail and then use those libraries as part of SFE. Hopefully that will come into it’s own over the next few months.

I’ve also been working with YAWS and designing my new websites with Erlang only code in mind. (Okay, there is some SQL…) And collaborative filtering was going to be a very large part of my new website designs. So I’ve learned a whole lot about how I want to design those system as well.

While I am not giving up on the http://www.netflixprize.com I am quickly coming to the conclusion that I need to work on my other projects for a while and get them up to speed with my new ideas, once implemented they may give me more insight into how to win the http://www.netflixprize.com