Ebot is a opensource and scalable web crawler written on top of Erlang, a NOSQL database (Apache CouchDB or Riak), RabbitMQ, Mochiweb, Webmachine, .. See my Erlang info page for more details
The project is hosted at http://github.com/matteoredaelli/ebot
Thanks to this project I’ve been improving my knowledge about the AMQP protocol (RabbitMQ) and NOSQL databases (Apache CouchDB and Riak) with the distribuited map/reduce queries
