Wednesday, July 25, 2012

Oscon 2012 resources, presentations and videos

OSCON 2012 recap: "Awesome, how could it not be?"

Huge! There were upwards 18 rooms in use for different talks on Wed and Thu. While that could be an opportunity to pick poorly, it really meant seeing something awesome but at the expense of other awesome things.

Unlike this year's YAPC, not all of the talks were recorded. Most of them were not. I've spent the past few hours crawling the internet to find slide decks and etc for you, so DO NOT FRET!

One stop shopping, visit the Oscon Speaker Slides page. This lists all the presentations where the speaker provided her slides to the organizers for publishing.

The OSCON 2012 YouTube Channel has the keynotes and a bunch of interviews. And it wouldn't be a conference without Piers singing beautifully. A four part video on "From Datacenter to the Cloud" is up on Vimeo.

Trawling through #oscon on twitter, I found some more slidedecks and videos.

My OSCON

View My personal Schedule to see which talks I attended. Only a few seem to have slides uploaded on the oscon/o'reilly page.

Some of my favorites:

"How to Write Compilers and Optimizers"
Holy awesome batman! Go read the slides(pdf) now.
Shevek's goal was a "thirty minute overview of a grad school course on compilers." (aka no math) And that's pretty much what we got. Bummed I missed the beginning with his SQL parser demo. Other big take away? "Use SableCC, a beautiful Java LR parser generator."

"Moose is Perl: A Guide to the New Revolution" (Ricardo Signes)
The slides aren't on the oscon page, but can be pulled directly from manxome.org: Moose is Perl slides, specifically the pdf
I attend the first half (90 minutes) before switching for Presentation Akido, toughest talk choice of the week. And the second 90 minutes was going to be about stuff I didn't know.

"Build a better team through Improv"
Got us out of our comfort zone doing improvisational comedy to improve work interactions. Fun stuff. The slides(pdf) weren't used during the activity, but have useful notes on our exercises. Take-away: "Yes, and" >> "Yes, BUT".

Profiling memory usage of Perl applications (Tim Bunce)
He posted the slides on slideshare, but not yet linked from oscon.
A lovely wander through Devel::Peek, Devel::Size and friends and what-could-be for profiling memory usage (and why it is hard).

Perl Lightning Talks and Ignite
The ignite talks were good, the lightning talks were better. No one is going to do a presentation about "phpsadness" at an ignite talk. Gong!
Tim Bunce did an enlightening lightning talk, A Space for Thought." He was kind enough to upload a copy to pastebin and send to me via twitter.

"Wrangling Logs with Logstash and ElasticSearch"
I really wanted to attend this one, by our local L.A. MediaTemple friends Nate Jones and David Castro, but it was packed beyond standing room only and the ushers weren't letting anyone in. We may get them to present again locally, for LAPM or LAdevops?

"One-man ops; deploying at scale in EC2",
A bummer that the slides don't have presenter notes. Take-away: automate it!

The Damian Conway collection:

"Instantly better VIM" (Damian Conway)
The slides aren't listed on the oscon site, but don't worry, I hooked you up: Instantly_better_Vim.tar.gz

"Presentation Akido"
I caught the first half of Moose and the second half of Presentation Akido. They were both awesome talks. Damian didn't provide a link for slides, we had nicely bound print outs in the room. Of course, there were way more people and we ran out. So they ran off extra copies. I found a stash of them on Friday during clean-up, so have a few spares.
I've missed this talk before, so was glad to be able to schedule it in.

"The Conway Channel"

"Taming Perl Regexes"
Regexp::Debugger
demo now! live interactive ncurses display showing regex application.

He also did "New Features of the Modern Perls (5.10 to 5.16)" and held an office hour.
Lots of Damian time. I was actually at the office hour, which also featured Ward Cunningham (mr XP), Shevek (compilers, parallel and distribute systems) and Christopher Clark(of sparkfun).

Hyperpolyglot:

"Inside Python"
Interesting example of digging into the python compiler/VM, with the idea of "adding a new keyword," which can't be done at the language level. That's a rough way to do meta-programming (harhar!). I left this with a better understanding of the what happens during python compile time.
"Data Science in R"
Great slide content available from the presenter's site
"Computing with Clojure"
Fun introduction to clojure and lispness. Tag teamed with two presenters. Highly entertaining.
"Storm: distributed and fault-tolerant realtime computing" (Nathan Marz)
I've been following storm and nathan_marz on twitter&blog for a long time. It was neat to see live presentation. It mostly covered the overview of storm, but did talk about a new technique for simpler, shorter code.

The Synacor Challenge:


The challenge ate up a huge number of person-hours at OSCON. I started it at 11pm Thursday and (coupled with the important lesson "tea == caffeine" ) forgot to sleep until 7am Friday. I hit an infinite loop and bailed on it -- later I found it was merely a highly bounded loop that would recover 700,000 cycles later.

What was it? Two files: a description of a VM and a binary to be run on that VM. There are 8 codes to find along the way and report back. There was a $1000 prize for the first to solve it during the convention, it was given away to one of the three people tied with 6 of 8 codes. I've gone as far as 5. I see that @dag completed all 8 sometime yesterday.

Thursday, July 12, 2012

ssh_agent + screen

Screen is awesome, ssh agent forwarding is wondrous. If you've been following along, you know this already.

You may be frustrated that when you reconnect to a screen session your agent stops working. ssh-agent sets environment variables to tell ssh where to find the agent, and your old shells get stuck with pointers to dead agent processes.

There are a few ways around this. A common method is to make a pair of scripts, one to dump the vars to a file in your login shell and another to read them back in from your screen shells. This works, but you have to do something manually after you reconnect - IN EVERY SHELL IN YOUR SCREEN SESSION.

Some will tell you to launch the agent from within the screen session, and then manually copy the env variables to each shell. This keeps a long running agent, which is nice but less secure. I like to run keep my keys local to my laptop and agent around from there. Also, when you spawn new shells, they still need to be updated with the ssh environment vars.

For years I've used a work around of using a symlink pointer for my agent. Before I launch screen, I set SSH_AUTH_SOCK to point to ~/.ssh/agent. All of my screen shells point to symlink, making it easy to update when my agent changes.

rm -f ~/.ssh/agent 
ln -s $SSH_AUTH_SOCK ~/.ssh/agent
export SSH_AUTH_SOCK=~/.ssh/agent

In practice it looks like this:
[vm53@vm53] 1003% rm -f ~/.ssh/agent; ln -s $SSH_AUTH_SOCK ~/.ssh/agent; export SSH_AUTH_SOCK=~/.ssh/agent

Meachum used to have an LD_PRELOAD hack that would pass certain ENV vars (SSH_AUTH_SOCK and DISPLAY) through to his running in-screen shells, but that was some crazy magic.

I've heard that tmux has support for updating certain env variables in the running shells ( maybe via 'update-environment'?) , specifically to handle cases like this. Anyone have details? I know that tmux is teh new h0tness but I'm not really ready to flush 20 years of screen familiarity. But maybe.

Tuesday, July 3, 2012

rabbitmq cluster problem : version_mismatch

Goal: a rabbitmq cluster ready for HA mirrored queues.
Problem: "Error: version_mismatch" during cluster command.

I went through the steps in the clustering guide and they worked fine on my freshly installed staging servers. I'm getting an error running through them on my production instance, trying to cluster a newly created EC2 instance in with my current production rabbitmq server.

Error: version_mismatch. Specifically one of them has 'topic_trie_node'.

I created the second server ( 10.7.203.145 ) from the first by creating a new volume from a snapshot of the disk from the first and then mounting the volume on a new EC2 instance.

Dear LazyWeb: Any ideas?

[andrew@ip-10-7-203-145]% sudo rabbitmqctl stop_app                               :) ~
Stopping node 'ip-10-7-203-145@ip-10-7-203-145' ...
...done.
[andrew@ip-10-7-203-145]% sudo rabbitmqctl reset                                  :) ~
Resetting node 'ip-10-7-203-145@ip-10-7-203-145' ...
...done.
[andrew@ip-10-7-203-145]% sudo rabbitmqctl cluster ip-10-7-203-145@ip-10-7-203-145 ip-10-7-203-85@ip-10-7-203-85Clustering node 'ip-10-7-203-145@ip-10-7-203-145' with ['ip-10-7-203-145@ip-10-7-203-145',
                                    'ip-10-7-203-85@ip-10-7-203-85'] ...
Error: {version_mismatch,[add_ip_to_listener,exchange_event_serial,
                          exchange_scratch,gm,ha_mirrors,mirrored_supervisor,
                          remove_user_scope,semi_durable_route,topic_trie,
                          user_admin_to_tags,add_queue_ttl,
                          multiple_routing_keys],
                         [add_queue_ttl,multiple_routing_keys,
                          add_ip_to_listener,exchange_event_serial,
                          exchange_scratch,gm,ha_mirrors,mirrored_supervisor,
                          remove_user_scope,semi_durable_route,topic_trie,
                          topic_trie_node,user_admin_to_tags]}
[andrew@ip-10-7-203-145]%                                                     2 :( ~

I agree with you prompt, sad-face indeed.