Hack day with Kenny: Fey::ORM, testing and screen. [lost draft from 1/12/10]

After sleeping through the LILAX users group meeting (sorry guys), I rolled up to Kenny's (Kenny Flegal), where he had invited me for a day of coding and authentic Salvadorian food. Win Win!

I showed him briefly the topic of my upcoming Monger's presentation, but mostly we looked at his current project. He is forking a GPL licensede project, to recreate part of the functionality and extend it in a different direction. Along the way he's rewriting the app layer in perl from command line php scripts.

We discussed the various clauses of the Gnu Affero GPL with regards to the hosting of the project during the initial revs. Can he have a public repository before he has finished changing all references to the old name to a new name and adding "prominent notices stating that you modified it, and giving a relevant date" as per Section 5, paragraph a? We decided that he probably could, but that it'd be easier to start with a private repo and not publish until that part is done. That seems sub-optimal from a "getting the source to the people" mindset, but it is more optimal in the "protect the good name of the original project and publishers."

Along with switching from php to perl, he's pulling out the hard coded sql from the scripts and moving to an ORM. He's picked Dave Rolsky's impressive Fey ORM. This project has a ridiculously complex set of schemas, with inconsistent table names and not explicit foreign key constraints. As such, it is extra work to get the fey schema situated.

Kenny started to give me a run through of some of the code, but it was awkward with both of us on laptops to see the code conveniently. I made him stop and set up a screen session for sharing, as described in my previous post on screen. This was more difficult than I expected, with the problem eventually being that ubuntu 9.4 and beyond has moved /usr/bin/screen to /usr/bin/screen.real and made screen a shell wrapper. The screen multiuser ACL system requires that the screen binary be setuid (chmod +s). With this setup we needed to make screen.real setuid. That took a while to notice.

Once we had a shared session open, it was much easier for him to give me a guided tour of the codebase and database/sql setup. Once that was clear it was time to get some code started. He showed me some of the Fey::ORM model code and how he was migrating over the individual sql statements to the ORM. He had been plugging away on the model code for a while, starting by creating a comment for every line of sql in the application including the file and line of the caller.

The next step was clear, we needed some tests. We set to work getting an initial test of the model code. First we installed Fey::ORM::Mock as a mock layer. This works at a higher level than a standard DBD::Mock interface to allow better testing of the Fey::ORM features. The test didn't pass at first due to missing data in the mock object, so we grabbed a list of the fields that mapped to DB fields and started adding values to pass constraint failures on the data. Once we had a minimal set of data then we started to see problems with the ORM schema description. The lack of well defined foreign key constraints meant we needed to explicitly define that structure for the ORM. More boilerplate code into the model. We repeated this test-update-repeat cycle a few more times adding more data linkage descriptions.

I took a brief break from our pairing and jumped to a different screen to install some goodies. I grabbed a copy of the configuration files from the December la.pm.org talk and started updating his config. He didn't have a .vimrc, .vim or .perltidyrc on this brand new dev box, so I pulled those in from the repo. I showed him how much time using ":make" in vim could slice off his build/test cycle, and he was super excited. (ok, not till the third or fourth try but he eventually got the hang of it).

To get around some issues in code placement, I modified the .vimrc and .vim/ftplugin/compiler code to add -MFindBin::libs to the calls to perl -c and prove. This allowed the parent libs/ directory to be found for these non-installed modules. This is a bit of a hack and I'll get it removed as we move closer to an initial release and pick a packaging tool, possibly Dist::Zilla.

An open question is the speed of Fey::ORM. It takes a big startup hit while building the models from the schema and interacting with the database. This is supposed to lead to a big speed gain during runtime from aggressive caching of that information. All I know for certain is that the compile-run-test cycle was really slow. This is my first time using Fey so I don't know how this plays out normally. It could just be that the number of crosslinked tables in the db config were causing additional slowdowns.

By this point we had already had two delicious meals of El Salvador cuisine and it was approaching midnight. The first meal was home cooked fried (skinless) chicken for lunch and the second was papoosas at a local, excellent place in Van Nuys. I was all coded out, which made for a perfect transition to the party at Andy Bandit's that night, conveniently just 6 miles from Kennys.

All in all, a fine Saturday.

Mojo::UserAgent dom parsing is FUN!

I'm about to roll out a new feature at work. I've added new data to the "schema" behind some of our pages and another team has implemented the template changes.

Now, How do I test that feature appears on the page? And by "on the page" I mean embedded attributes into a javascript call on the page.

I used Mojo::UserAgent and it's built in dom handling to make this easy-peasy! Load the page, look for script tags, find the one calling our Magic.Marker function and then use a regex to pull the args. Wrap it all up in Test::Most and throw some data into _DATA_!

ps. Writing his post took considerably longer than writing this test.

#!/usr/bin/perl
use v5.12;

use Mojo::UserAgent;
use List::Util qw(first);
use Test::Most;

my $ua = Mojo::UserAgent->new();
sub x_param_from_url
{
    # load URL and find the first script block that 
    # contains Magic.Marker.  Parse Magic.Marker args 
    # for items like "{ x: value }" and return all the 
    # values found.
    my $url     = shift;
    my @scripts = $ua->get($url)->res->dom->find('script')->each();
    my $script  = first { $_->all_text =~ /Magic\.Marker/ } @scripts;
    return unless $script;
    
    my $text = $script->all_text;
    my @matches = ( $text =~ m/\{ \s* x \s* : \s* (\S+) \s* \}/gimx );
    return @matches;
}

foreach my $data (<DATA>)
{
    chomp $data;
    my ( $url, @expected ) = split( /\s/, $data );
    # redirect to the internal-staging server
    my $url =~ s/www.example.com/internal-staging.example.com/;
    my @output = x_param_from_url($url);
    eq_or_diff( \@output, \@expected, "$url")
}
done_testing

__DATA__
www.example.com/how-to_123  '2' '3'
www.example.com/why-not-1234 '1' '2'
www.example.com/why-not-777 '3'

This produces a lovely TAP output for the site:

% ./verify.pl

not ok 1 - internal-staging.example.com/how-to_123
#   Failed test 'internal-staging.example.com/how-to_123'
#   at ./verify.pl line 35.
# +----+---------+----+----------+
# | Elt|Got      | Elt|Expected  |
# +----+---------+----+----------+
# |   0|'\'2\''  |   0|'\'2\''   |
# |    |         *   1|'\'3\''   *
# +----+---------+----+----------+
ok 2 - internal-staging.example.com/how-to_123
ok 3 - internal-staging.example.com/why-not-777

Hadoop::Streaming v0.122420 released

Hadoop::Streaming perl module v0.122420 released to CPAN today.

Yanick was kind enough to send me a patch (Pull Request #2) to clean up the formatting of the POD documentation. Thank you for the patch Yanick! I love patches and pull requests! Woot!

You know what else I love? Dist::Zilla! Thanks Ricardo! Here are the complete steps to release a new version and push it to CPAN after merging in the pull request:
git pull && dzil release

Steps accomplished by dzil release:

1. generate new version number
   
2. podweaver magic to generate structured POD
   
3. boilerplate: create LICENSE, MANIFEST, META.json, META.yml, Makefile, etc.
   
4. create tar file of distribution
   
5. extract tar file and run all tests
   
6. verify no dirty files in git
   
7. update Changes file to include the new version number.
   
8. git commit Changes file
   
9. upload tar file to cpan
   
10. git tag with version number
    
11. git push to origin
    

Dzil plugins: GitHub::Meta vs GithubMeta

"Dear lazyweb: how do I add a github repository link to my dist-zilla dist.ini?"

Curtis Poe (@ovidperl) via twitter

Ovid asked how to add git repo information to a dist.ini file. Both Dist::Zilla::Plugin::GitHub::Meta and Dist::Zilla::Plugin::GithubMeta were suggested, along with the old-school Dist::Zilla::Plugin::MetaResources.

I've used MetaResources to include repository information in my dist.ini files. Which plugin should I use now: GitHub::Meta GithubMeta?

My MetaResources usage is pretty simple: homepage + repository information. The GitHub::Meta and GithubMeta versions are nearly the same from the dist.ini side, both have the same format for manually overriding the default homepage:

#MetaResources Github example:
[MetaResources]
homepage        = http://lowlevelmanager.com/
repository.web  = http://github.com/spazm/app-pm-website
repository.url  = http://github.com/spazm/app-pm-website.git
repository.type = git

## GithubMeta version
[GithubMeta]
homepage = http://lowlevelmanager.com/

## GitHub::Meta version
[GitHub::Meta]
homepage = http://lowlevelmanager.com/

There are differences if we look inside the modules. GithubMeta makes a system call to git to get remote url(s). The plugin will only run if called from within a git repository and with git in $PATH. If it finds a github.com url, it uses that as the remote. This url is then parsed to get the github user name and project name.

Conversely, GitHub::Meta shells out to git to get the github.user entry from git config via git config github.user and then makes an API call to github to get the project name. It does have an interesting feature where it can check if the checkout is a fork and use the upstream information instead.

GitHub::Meta could be nice if you want to go all-in and auto-create the repository with GitHub::Create and push updates with GitHub::Update (actually, this just seems to change the homepage url?), all of them will require the github.user configuration set in git. The docs suggest using GitHub::Create with Git::Init. I must admit, I'm behind the DZil times, as I'm not even sure where to store the default plugin options to apply when running dzil new, to actually trigger these plugins.

GithubMeta does what you'd expect and does it with a minimum of fuss. No extra options to set in your git configuration, but you do have to have the github remote added to your repo by other means. This seems a much slimmer change for existing repositories.

Building a jabber bot with Bot::Backend

There are many ways to write a (jabber/xmpp) chat bot. A quick search for "Jabber Bot" turns up Net::Jabber::Bot, Bot::JabberBot, Bot::Backbone::Service::JabberChat, Bot::Jabbot, IM::Engine, AnyEvent::XMPP and more. You'll see even more if you search for XMPP instead.

How to choose?

I started with Net::Jabber::Bot, but ran into problems connecting to google-talk based chat. Jabber modules built on AnyEvent::XMPP (seem to) connect to google-auth better than those built on Net::XMPP. This is moot now that I have a normal jabber server, but it still tripped me up. The pod doc lays out funny, so I patched and filed a change-request at github.

Knowing that I would eventually want an event-loop based app, I focused on AnyEvent::XMPP packages. Bot::Backbone and IM::Engine are both interesting abstractions. IM::Engine is quote "currently alpha quality with serious features missing and is rife with horrible bugs." Perhaps I should be happy that he admits this up front? On to Bot::Backbone and sugary Moose!

Once I realized that the group_domain wouldn't be automatically filled in as "conference.$domain" in Bot::Backbone::Service::JabberChat, I was able to connect to a group chat room on my jabber server!

Follow along with my bot, App::Sulla on github. Thus far I have a working connection that responds to "!time" requests -- I've even fixed the part where it threw warnings on all other messages!

#this code in my dispatch table:
    also not_command not_to_me run_this {
            my ( $self, $message ) = @_;
            respond { "hello world" };
    };

# lead to the following error on every other chat message:
unhandled callback exception on event (message, AnyEvent::XMPP::Ext::MUC=HASH(0x3746818), AnyEvent::XMPP::Ext::MUC::Room=HASH(0x3a6d700) ANY_STRING ): Can't call method "add_predicate_or_return" on an undefined value at perl5/lib/perl5/Bot/Backbone/DispatchSugar.pm line 124.

I decided I wanted to pass the auth and channel information into App::Sulla as parameters to the constructor. This didn't play very well with the service sugar. service executes at compile time and squirrels away the configuration hash for the service into services. I added a before modifier to construct_services which is called during run just before initializing each service.

28c3: Effective Denial of Service attacks against web application platforms

Synopsis

Hour long presentation from the C3 security conference, on hash collision based attacks against web apps. The idea being that most web stacks automatically grab the query params and stick them into a hash of key-value pairs and by exploiting hash collisions an attacker can waste huge amounts of CPU time by a simple HTTP POST. In languages that don't randomly perturb their hashing functions (mostly DJBX33A or DJBX33X), collisions can be easily found and exploited.

This affects node.js/v8, php, python, ASP, ruby, java and etc. Only perl (5.8.1 ~2003) and cruby (1.9 ~2008) are patched for randomized hashing functions. The v8 devs are unconcerned "because v8 is a client side language," which caused a bit of alarm for the node.js folks.

The presenters started their investigation after reading a mention in the perl security faq (perldoc perlsec), in the section Algorithmic Complexity Attacks, that before 5.8.1 perl had a security flaw where hash collisions could be exploited. 5.8.1 was released in 2003.

           In Perls before 5.8.1 one could rather easily generate data that as
           hash keys would cause Perl to consume large amounts of time because
           internal structure of hashes would badly degenerate.  In Perl 5.8.1
           the hash function is randomly perturbed by a pseudorandom seed
           which makes generating such naughty hash keys harder.  See
           "PERL_HASH_SEED" in perlrun for more information.

Thread from the node.js mailing list: "HOLY CRAP. nearly all nodejs http servers are vulnerable to DoS and apparently, the V8 guys seem to not care much"

Related Node.js followup:

• https://plus.google.com/103939784105159993090/posts/Sh2Nxw8p3WB
• https://plus.google.com/116910304844117268718/posts/BfTdM25d23y

Perl Advent Calendars, 2011 edition.

It's Advent Calendar time in the perl ecosystem! Start each day with a delicious treat of knowledge.

I've found a half dozen english language perl advent calendars, starting with the original perl advent calendar. For extra fun I've included another half dozen Japanese language calendars -- I can still read the perl it's just the prose that is lost in translation.

Ricardo (RJBS) has taken over the Perl Advent calendar this year, which is awesome. Sadly, that means he won't be doing his own "month of rjbs" calendar. I've added a link to his 2010 calendar, in case you missed it the first time around. He's starting the month with Day 1: cpanm and local::lib.

For a second year, Miyagawa has skipped updating plack advent calendar. Check out the 2009 edition linked below. He has given us plenty of other presents this year: Carton, etc.

Perl Advent

http://perladvent.org/2011/

(Formerly the perladvent.pm.org calendar)

Perl Dancer -- the dancer mini web framework

http://advent.perldancer.org/2011/

Catalyst Advent Calendar -- The Catalyst Web Framework

http://www.catalystframework.org/calendar/

Perl 6

http://perl6advent.wordpress.com/

For the adventurous: Japanese Perl Advent Calendars, 8 different tracks!

http://perl-users.jp/articles/advent-calendar/2011/

AnySan Track

Casual Track

dbix Track

English Track

Hacker Track

Test Track

Acme Track

Teng Track

Amon2 Track

Ricardo's 2010 advent calendar -- a month of RJBS

http://advent.rjbs.manxome.org/2010/

2009 Plack calendar

http://advent.plackperl.org/

One bonus list, for the sysadmin in your life:

SysAdvent - The Sysadmin Advent Calendar.

http://sysadvent.blogspot.com/

Evil: If I were creating the world I wouldn't mess about with butterflies and daffodils. I would have started with lasers, eight o'clock, Day One!
-- Time Bandits

Git::CPAN::Patch

Just saw an announcement of Git::CPAN::Patch over at Yanick's blog.

This is an awesome tool to create patches for other people's modules (OPM(tm)). It will create a git repo from current sources so you can get straight to patching. New in v0.7.0, if the module has a public repo, it'll pull directly from that. Rockstar!

Git::CPAN::Patch could already seed a local repository with the latest distribution of a module, or its whole BackPAN history, or its GitPAN mirror. But with version 0.7.0, it can now go straight for the meat and clone the distribution's officil git repository, provided that it's specified in its META.json or META.yml.
-- Read more: http://babyl.dyndns.org/techblog/entry/new-and-improved-git-cpan-patch-0.7.0

Provides two tools git cpan-sources and git cpan-init

HackDay! 11/12/11

David (DDubs!) came over for a HACKDAY today. Fun times!

He's in a maze of twisty little passages, all alike. He's installing SVN on localhost, with the full webDAV thing. Why? So he can practice migrating SVN to git. Yes, crazy land. I'm more looking forward to the actual project, a Bayesian Classifier to practice with Moose.

I've updated my App::PM::Website tool for maintaining the Los Angeles Perl Mongers website. It now uses Lingua::EN::Numbers::Ordinate to render dates like "Wednesday the 7th" instead of "Wednesday the 7." That's one more piece of manual hard coding replaced with software. Woot! Come on out on December 7th for Mike's talk on VOIP!

I really do want to get this code into a state where it's releasable to CPAN and usable by other monger groups. Maybe by making my vapor App::PM::Toolbox into reality? Why am I so tentative on releasing that?

I also updated and released v0.113160 of Hadoop::Streaming perl module, mostly to use Any::Moose. I made the changes a few months ago at a user's request. I sent him a beta version for testing, and it got lost in the weeds. I merged the code over today from feature/mouse and pushed it out. While I was in there, I updated the documentation in the main package to show how to use the '-archive' flag to hadoop.

I love typing 'dzil release' and having my Changes file updated, checked into git, release git tagged, release built and bundle pushed to PAUSE for CPAN.

If I had checked github first, I would have seen this lovely change request waiting where a user had made the Any::Moose conversion for me. My first incoming github change request. ROCKSTAR!

This post brought to you by the number 30 and the letter D (eltron).

2011-11-13 00:52:18 $$5747 v1049: Info: Need to get uriid[S/SP/SPAZM/Hadoop-Streaming-0.113160.tar.gz] (paused:337)
2011-11-13 00:52:18 $$5747 v1049: Info: Going to fetch uriid[S/SP/SPAZM/Hadoop-Streaming-0.113160.tar.gz] (paused:625)
2011-11-13 00:52:18 $$5747 v1049: Info: Requesting a GET on uri [ftp://pause.perl.org/incoming/Hadoop-Streaming-0.113160.tar.gz] (paused:647)
2011-11-13 00:52:19 $$5747 v1049: Info: renamed '/home/ftp/tmp/S/SP/SPAZM/Hadoop-Streaming-0.113160.tar.gz' to '/home/ftp/pub/PAUSE/authors/id/S/SP/SPAZM/Hadoop-Streaming-0.113160.tar.gz' (paused:760)
2011-11-13 00:52:19 $$5747 v1049: Info: Got S/SP/SPAZM/Hadoop-Streaming-0.113160.tar.gz (size 28088) (paused:496)
2011-11-13 00:52:20 $$5747 v1049: Info: Sent 'has entered' email about uriid[S/SP/SPAZM/Hadoop-Streaming-0.113160.tar.gz] (paused:561)
2011-11-13 00:53:46 $$5747 v1049: Info: Verified S/SP/SPAZM/Hadoop-Streaming-0.113160.tar.gz (paused:308)
2011-11-13 00:53:46 $$5747 v1049: Info: Started mldistwatch for lpath[/home/ftp/pub/PAUSE/authors/id/S/SP/SPAZM/Hadoop-Streaming-0.113160.tar.gz] with pid[11168] (paused:313)

Perl for newbies

I've heard the tutorials at Perl-Begin.org are a good way to get started with perl. Current, modern perl. Not some 15-year-old script kiddie intro.

Speaking of, it might be interesting to make a learnperlthehardway book, ala Write your own LxTHW and the companion site LearnCodeTheHardWay.org.

Author dependencies in Dist::Zilla

Wanna edit/tweak/build/play-with a Dist::Zilla based perl module you've checked out? Seems daunting because "the module installer isn't included" or "what if I don't have the same helper modules that the author uses?" ? Worry Not!

It's easy to get the minimal pieces installed, so let's get to it.

Basic steps for building/using a Dist::Zilla based module from raw source:

1. Check out module source
2. install Dist::Zilla:
   cpanm Dist::Zilla
3. install author deps:
   dzil authordeps | cpanm
4. install module deps:
   dzil listdeps | cpanm
5. build module with dzil:
   dzil build

1. Check out module source

For my example, I'm migrating my own App::PM::Website sources from an old laptop to a new one.

Looking for example code to checkout? Try searching for dist.ini on github to find an interesting perl module. ;)

% git clone git@github.com:spazm/app-pm-website 
% cd app-pm-website

Check the code out of the repository and cd into the top level.

2. Install Dist::Zilla

On this relatively clean perl 5.12.3 install, cpanm Dist::Zilla brought in 81 packages.

% cpan Dist::Zilla
[andrew@fred]% cpanm Dist::Zilla           127 (git)-[dev] ~/src/app-pm-website--> Working on Dist::ZillaFetching http://search.cpan.org/CPAN/authors/id/R/RJ/RJBS/Dist-Zilla-4.300002.tar.gz ... OK
[... snip ...]
Building and testing Dist-Zilla-4.300002 ... OKSuccessfully installed Dist-Zilla-4.30000291 distributions installed

3. Install author dependencies

dzil authordeps will show the modules necessary for Dist::Zilla to build the module from raw source into a built module. Pipe this to cpanm to install the modules.

% dzil authordeps
Dist::Zilla::Plugin::MetaResources
Dist::Zilla::Plugin::AutoPrereqs
Dist::Zilla::Plugin::Repository
Dist::Zilla::Plugin::NextRelease
Dist::Zilla::PluginBundle::Basic
Dist::Zilla::Plugin::AutoVersion
Dist::Zilla::PluginBundle::Git
Dist::Zilla::Plugin::PkgVersion
Dist::Zilla::Plugin::MetaJSON
Dist::Zilla::Plugin::PodWeaver

% dzil authordeps | cpanm
Dist::Zilla::Plugin::MetaResources is up to date. (4.300002)
--> Working on Dist::Zilla::Plugin::Repository
[... snip ...]
Successfully installed Dist-Zilla-Plugin-PodWeaver-3.101641
11 distributions installed

4. Install module dependencies

Install the authordeps before module dependencies, in case authordeps are required for dzil to calculate the module dependencies. E.g. I needed PodWeaver installed via authordeps before I could run dzil listdeps to see the module dependencies.

% dzil listdeps
App::Cmd
App::Cmd::Command
App::Cmd::Tester
base
Config::YAML
Data::Dumper
Date::Parse
DateTime
DateTime::Format::Strptime
ExtUtils::MakeMaker
HTTP::DAV
Net::Netrc
POSIX
strict
Template
Test::Class
Test::Class::Load
Test::More
warnings

% dzil listdeps | cpanm
App::Cmd is up to date. (0.312)
base is up to date. (2.15)
--> Working on Config::YAML
[...snip...]
Successfully installed Test-Class-0.36
Test::More is up to date. (0.98)
10 distributions installed

5. Build module

dzil build will build the module into a directory and tar it up ready for cpan.

Similarly, dzil test will build the code and run the tests, you'll use this to verify your changes to the target module.

% dzil build
[DZ] beginning to build App-PM-Website
[DZ] guessing dist's main_module is lib/App/PM/Website.pm
[DZ] extracting distribution abstract from lib/App/PM/Website.pm
[DZ] writing App-PM-Website in App-PM-Website-0.112890
[DZ] building archive with Archive::Tar; install Archive::Tar::Wrapper for improved speed
[DZ] writing archive to App-PM-Website-0.112890.tar.gz

And now I can get back to the task at hand, improving this module. I'll let you know how that goes too.

Wedding Dance

My latest Dancer project is up at SweetieBeast.us, a website for my pending wedding. T-minus-13 days and counting!

The source is up at github. Look in the wedding-dancer directory for the dancer project.

The dancer code is very short, because I'm not really using any dancer bits. I'm using dancer to provide an interface around templating code and page layout wrapping. As such, the pages are all pure templates.

Version .1 was very close to the following code snippet. Auto_page turns my template files into corresponding routes. I then created a template for each of my pages, spent a day futzing with CSS and poof: wedding website!

package sweetiebeast;
use Dancer ':syntax';
our $VERSION = '0.1';
set auto_page => 1;
get '/' => sub { template 'index'; };
1

When I launched, I put the site behind an apache proxypass directive. So I added Dancer::Plugin::Proxy to create links relative to the external proxy. I query Dancer::Plugin::Proxy to explicitly create link targets in the model to simplify my views.

package sweetiebeast;
use Dancer ':syntax';
use Dancer::Plugin::ProxyPath;
our $VERSION = '0.2';

set auto_page => 1;
get '/' => sub { template 'index'; };

before_template sub {
    my $tokens = shift;
    $tokens->{uri_base}           = proxy->uri_for("/");
    $tokens->{uri_for_index}      = proxy->uri_for("/");
    $tokens->{uri_for_faq}        = proxy->uri_for("/faq");
    $tokens->{uri_for_location}   = proxy->uri_for("/location");
    $tokens->{uri_for_travel}     = proxy->uri_for("/travel");
    $tokens->{uri_for_guests}     = proxy->uri_for("/guests");
    $tokens->{uri_for_images}     = proxy->uri_for("/images");
};
true;

my first yapc

I'm looking forward to my trip to Asheville, NC for YAPC::na. June 26-30. Yet Another Perl Conference, North America.

I was supposed to go last year while I was at Rubicon Project, but my trip got canceled (some mix of busy, crunch time, and politics).

I haven't been to a focused perl conference since the short lived "O'Reilly University" circa 2000 in nyc. MJD was the bomb! It'll be interesting to compare with our <shameless_plugs> excellently run LA pm</shameless_plug> and all the other LA tech meet-ups.

what do I need to do to prep? Rest my brain and liver, I suppose.

More LWP SSL 500 issues! Now with HTTP::DAV crossover

LWP::UserAgent is returning a 500 level error in the case of a self-signed site key. This is similar to my previous post on this topic, ( Fixed 500 can't verify SSL peers ):

For https://... default to verified connections with require IO::Socket::SSL and Mozilla::CA modules to be installed. Old behaviour can be requested by setting the PERL_LWP_SSL_VERIFY_HOSTNAME environment variable to 0. The LWP::UserAgent got new ssl_opts method to control this as well.

I use HTTP::DAV to push changes to the Los Angeles Perl Mongers website. When I tried to push updates for tonight's meeting, HTTP::DAV threw an error " The URL "https://groups.pm.org/groups/losangeles/" is not DAV enabled or not accessible.".

This seemed familiar -- in fact I filed a bug against HTTP::DAV ( Bug 59674 ) to pass through SSL errors from LWP when SSL libraries were not installed. ( Cosimo, I am sorry that I didn't respond in a timely manner when the fixes were proposed. Thanks for fixing it!). The fix for 59674 included having a specific message for various classes of errors out of LWP::UserAgent.

## Error conditions
my %err = (
    'ERR_WRONG_ARGS'    => 'Wrong number of arguments supplied.',
    'ERR_UNAUTHORIZED'  => 'Unauthorized. ',
    'ERR_NULL_RESOURCE' => 'Not connected. Do an open first. ',
    'ERR_RESP_FAIL'     => 'Server response: ',
    'ERR_501'           => 'Server response: ',
    'ERR_405'           => 'Server response: ',
    'ERR_GENERIC'       => '',
);

LWP::UserAgent is returning a 500 level error in the case of a self-signed site key. Not 501 as in the prior case.

Example:

#!/usr/bin/perl
use strict;
use warnings;
use LWP::UserAgent;
use Data::Dumper;
my $url = "https://groups.pm.org/groups/losangeles";
my $ua=LWP::UserAgent->new();
my $resp = $ua->get( $url );
print Dumper $resp;

Output Snippet:
LWP::Protocol::https::Socket: SSL connect attempt failed with unknown errorerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed at /Library/Perl/5.10.0/LWP/Protocol/http.pm line 51.

One can get around this by setting an environment variable export PERL_LWP_SSL_VERIFY_HOSTNAME=0, or by using the ssl_opts option to UserAgent. A third, preferred solution would be import the key and mark it as "OK" on the client side.

I pushed my website changes by using the PERL_LWP_SSL_VERIFY_HOSTNAME=0 workaround. Now let's see if I can figure out workarounds 3 and 2.

[LA.pm] March Los Angeles Perl Mongers

LA.pm.org will be back at Rent.com this month. Thanks for hosting again!

What:   Los Angeles Perl Mongers Meeting
When:   7-9pm
Date:   Wednesday, March 23, 2011
Where:  Rent.com - 2425 Olympic Blvd Suite 400 E, Santa Monica, CA 90404
Theme:  Perl!
RSVP:   Responses always appreciated.

As always, I'm looking for presenters. What are you doing in the greater perl infrastructure?

Looking forward to seeing my fellow mongers in 11 days.

RSVP at facebook

Thrift!

Sweet, a new module popped onto the CPAN this weekend, Thrift::XS, an XS version of Thrift. This is doubly nice -- one, it's a faster XS version. two, it's available directly on cpan. The current module from the Apache Thrift project requires finding and downloading the package.

Thrift is a streaming serialization format. See also Protocol Buffers and Avro.

DESCRIPTION

Thrift::XS provides faster versions of Thrift::BinaryProtocol and Thrift::MemoryBuffer. On average it is about 4-6 times faster.

Thrift compact protocol support is also available, just replace Thrift::XS::BinaryProtocol with Thrift::XS::CompactProtocol.

To use, simply replace your Thrift initialization code with the appropriate Thrift::XS version.

Scale9x: Take Advantage of Modern Perl

Chromatic's talk on Modern Perl at Scale9x is in about an hour -- 11:30am, Sun Feb 27, 2011. If you can't make it, at least check out the live stream.

I really shouldn't have gone to scale yesterday, since I'm so sick and it wiped me out. Yet here I am contemplating going again today. I do want to get my copy of Modern Perl autographed, afterall.

Perl's recent renaissance has produced amazing tools that you too can use today.

This talk explains the philosophy of language design apparent in Perl 5 along the two fundamental axes of the language: lexical scoping and pervasive value and amount contexts. It also discusses several important pragmas and language extensions to improve Perl 5's defaults, to reduce the chance of errors, to allow better abstractions, and to encourage the writing of great code.

Speaker: chromatic x
-- http://www.socallinuxexpo.org/scale9x/presentations/take-advantage-modern-perl

Cpanm 1.1 -- now with mirror support!

There is a new version of cpanm (App-cpanminus) that supports --mirror and --mirror-only to allow offline usage.

Kick ass! Thanks again miyagawa

cpanm 1.1 is shipped, and with `--mirror-only` option, you can use it with your local minicpan mirror, or your own company's CPAN index (aka DarkPAN).

The only reason for a few experienced perl programmers who loves cpanm but can't use cpanm offline or at work was the lack of the proper mirror index querying support.

cpanm always has required an internet connection to resolve module name and dependencies, and always relies on CPAN Meta DB and search.cpan.org to query package index.

It's been a fair requirement for 95% of the usage, but again, for an experienced hacker who spends their most of airplane's time hacking code on their laptops, the offline support to fallback to local minicpan would be really nice. (Even though many airlines nowadays provide in-flight Wi-Fi :))

So I opened a bug to support `--mirror-only` option to bypass these internet queries and parse mirror's own 02packages.txt.gz file for module resolution a while ago, and a couple of people have tried implementing it in their own branches. (Thank you!)

Today I merged one of those implementations, and improved a little bit to make it run even faster and more network efficient. The way to use it is really simple, just run cpanm with options such as:

cpanm --mirror ~/minicpan --mirror-only Plack

and it will use your minicpan local mirror as the only place to resolve module names and download tarballs from. (TIP: you can alias this like `minicpanm` to save typing)

---- http://bulknews.typepad.com/blog/2010/11/cpanm-11-hearts-minicpan-and-darkpan.html

"DBIX::Class::Deployment handler is awesome" is an article on using DBIx::Class::DeploymentHandler (along with SQL::Abstract ) to automatically produce database version upgrade and downgrade scripts from DBIX::Class schema documents and schema layout diagrams.

awesome. This is why I follow the Perl Iron Man blogging feed. Great stuff in there!

SCALE presentation proposals : denied.

Sigh, Neither of my modern perl SCALE proposals were accepted -- dev track proposals for hands on demonstrations of using Hadoop Streaming with Big Data and quickly building web applications with Dancer. I hope we get an perl mongers booth/table.

I'm glad to hear there were so many presentation proposals. Sounds like we'll have some great talks!

Dear Speaker,

The SCALE committee has reviewed your proposal(s). Unfortunately, your proposal, while excellent, was not accepted. SCALE again had many high quality submissions, so we could only accept a small fraction of those submitted (47 out of 160 submissions).

We thank you for your interest in SCALE and we appreciate your submittal! We hope you'll participate in future SCALE events. The latest updates for the conference are available at http://www.socallinuxexpo.org