Monday, January 2, 2012

Storm is upon us!

A storm (from Proto-Germanic *sturmaz "noise, tumult") is any disturbed state of an astronomical body's atmosphere, especially affecting its surface, and strongly implying severe weather.
-- wikipedia:storm

Storm is open-source: distributed and fault-tolerant realtime computation
-- Nathan Marz via twitter
Twitter has open sourced Storm, a distributed real-time processing engine ( Announcement, Github ). Hadoop is to batch as Storm is to stream. This is huge. So huge that it seems to be mostly ignored?!

Storm's primary author, Nathan Marz, is the same guy who created cascalog for hadoop queries (a marriage of Clojure and Datalog running queries in hadoop) and ElephantDB. You'll remember he was working at BackType (their first non-founder hire. Good choice, guys!). Twitter bought BackType and now they are sharing storm with us. Thanks twitter!

Since Storm's release seems to have flown in under the radar and is a difficult generic search term, I've collected up links to the relevant resources.

Resources & Announcements:

StrangeLoop/Infoq presentation
Nathan opensourced storm in the middle of his StrangeLoop 2011 presentation.
The slides below the image update as the video plays.
Storm Slides
StrangeLoop 2011 video links
A Storm is Coming more details from twitter.
Storm Source on Github
Storm Wiki on Github
Wiki Front Page
Mailing List
0.6.1 released
Testing Storm Topologies(in Clojure)
Overview of Storm Presentation at BashoChats 001
BackType techtalks

No comments: