Monday, January 2, 2012

Storm is upon us!

A storm (from Proto-Germanic *sturmaz "noise, tumult") is any disturbed state of an astronomical body's atmosphere, especially affecting its surface, and strongly implying severe weather.
-- wikipedia:storm

Storm is open-source: distributed and fault-tolerant realtime computation
-- Nathan Marz via twitter
Twitter has open sourced Storm, a distributed real-time processing engine ( Announcement, Github ). Hadoop is to batch as Storm is to stream. This is huge. So huge that it seems to be mostly ignored?!

Storm's primary author, Nathan Marz, is the same guy who created cascalog for hadoop queries (a marriage of Clojure and Datalog running queries in hadoop) and ElephantDB. You'll remember he was working at BackType (their first non-founder hire. Good choice, guys!). Twitter bought BackType and now they are sharing storm with us. Thanks twitter!

Since Storm's release seems to have flown in under the radar and is a difficult generic search term, I've collected up links to the relevant resources.

Resources & Announcements:

StrangeLoop/Infoq presentation
http://www.infoq.com/presentations/Storm
Nathan opensourced storm in the middle of his StrangeLoop 2011 presentation.
The slides below the image update as the video plays.
Storm Slides
http://www.slideshare.net/nathanmarz/storm-distributed-and-faulttolerant-realtime-computation
StrangeLoop 2011 video links
https://thestrangeloop.com/news/strange-loop-2011-video-schedule
A Storm is Coming more details from twitter.
http://engineering.twitter.com/2011/08/storm-is-coming-more-details-and-plans.html
Storm Source on Github
https://github.com/nathanmarz/storm
Storm Wiki on Github
Wiki Front Page
Rationale
Tutorial
Mailing List
http://groups.google.com/group/storm-user
0.6.1 released
http://groups.google.com/group/storm-user/browse_thread/thread/72b3d4a4aebdebea
Testing Storm Topologies(in Clojure)
http://www.pixelmachine.org/2011/12/17/Testing-Storm-Topologies.html
Overview of Storm Presentation at BashoChats 001
http://basho.com/blog/technical/2011/12/20/Basho-Chats-001-Talk-Videos/
BackType techtalks
http://tech.backtype.com/pages/presentations-8

No comments: