Perl 2011: Where are we now?

Piers Cawley wrote an excellent forward looking piece The Perl Future in January 2009. As we approach the two year mark, how have we fared? He talks about perl 6, perl 5.10.0, aka "perl5 version 10," Perl Enlightenment & the rise of Moose, and "on frameworks and the future."

Where are we now?

Perl 5.12 is out, as scheduled, on time -- two years of work representing 750,000 lines of changes over 3000 files and 200 authors. Deprecated features of perl4&5 are finally marked as such. More Unicode improvements. Features improvements for the Y2038 bug (is epoch time 64 bit now ?) Includes pluggable keywords and syntax. A new release schedule means stable releases come out in the spring, followed by a .1 fix release, then monthly releases (on the 20th) for new bug fixes.

Perl 6 released a real, honest to goodness release candidate. Rakudo Star, "A useable perl6 release" was released in June, aimed at "Perl 6 early adopters." Rakudo star has seen monthly updates, most recently Rakudo Star 2010.11 released in November 2011. Rakudo Perl is a specific implementation of Perl 6 the language, this Rakudo Star 2010.11 release includes "release #35 of the Rakudo Perl 6 compiler, version 2.10.1 of the Parrot Virtual Machine, and various modules, documentation, and other resources collected from the Perl 6 community."

A year-and-a-half of the Perl Iron Man blogging project has seen a flurry of posts from nearly 250 perl bloggers! We've seen advocacy, snippets, whining, and community. I've seen a lot more Japanese language perl posts -- folks happy to use perl, python and ruby and pull the best from each.

I now find it strange and unsettling to meet self proclaimed perl programmers who don't use Moose. If you haven't played with it (and it does feel like playing, it's liberatingly fun), go do so now. I'll wait.

I don't know about you, but I just switched from one startup using perl to another startup using perl. Awesome perl folks are hard to find, they're mostly already busy doing work they love. Why are we using perl? -- because perl works, it scales with developer time, and perl is beautiful.

Piers mentioned frameworks -- yes individual frameworks are important but the vast armada of options available at CPAN as a whole provide an immense multiplier on developer productivity. It's so massive, it is easy to overlook -- Doesn't everyone have a massive, distributed, user-written body of code with excellent testing methodology available at the touch of a button?

Merry Christmas to all!

[...snip...]
However, if you look at the good parts (O'Reilly haven't announced "Perl: The Good Parts", but it's a book that's crying out to be written), there's a really nice language in there. Arguably there's at least two. There's the language of the one-liner, the quick throwaway program written to achieve some sysadmin related task, and there's the more 'refined' language you use when you're writing something that is going to end up being maintained.

I think it's this split personality that can put people off the language. They see the line noise of the one liner school of programming, the games of Code Golf (originally called Perl golf, the idea spread), the obfuscated Perl contests, the terrible code that got written by cowboys and people who didn't know any better in the dotcom bubble (you can achieve an surprising amount with terrible Perl code, but you will hit the wall when you try and change it) and they think that's all there is.

But there is another Perl. It's a language that runs The Internet Movie Database, Slashdot, Booking.com, Vox.com, LiveJournal and HiveMinder. It's a language which enables people to write and maintain massive code-bases over years, supporting developers with excellent testing and documentation. It's a language you should be considering for your next project. It's also something of a blue sky research project - at least, that's how some people see Perl 6.

----http://www.h-online.com/open/features/Healthcheck-Perl-The-Perl-Future-746527.html

A story of one man's journey to Vim Nirvana (Vimvana). For those of you stuck on Monday, keep trying you'll make it.

I was watching a violinist bow intensely and I had this thought: I probably have as many brain cells devoted to my text editor as he does to playing his chosen instrument. Is it outlandish to imagine that an MRI of his brain during a difficult solo wouldn’t look much different than mine while manipulating code in vim?

Consider, if you will, the following montage from one vimmer’s journey.

---- http://kevinw.github.com/2010/12/15/this-is-your-brain-on-vim/

Perl Advent Calendars

It's Advent Calendar time in the perl ecosystem! Start each day with a delicious treat of knowledge.

I've found a half dozen english language perl advent calendars, starting with the original perl advent calendar. For extra fun I've included another half dozen Japanese language calendars -- I can still read the perl it's just the prose that is lost in translation.

Perl Mongers Perl Advent calendar

http://perladvent.pm.org/2010/

Catalyst Advent Calendar -- The Catalyst Web Framework

http://www.catalystframework.org/calendar/

Perl Dancer -- the Dancer mini web framework

http://advent.perldancer.org/2010

Ricardo's 2010 advent calendar -- a month of RJBS

http://advent.rjbs.manxome.org/2010/

UWE's advent calendar - a cpan module every day.

http://www.perl-uwe.com/

Perl 6

http://perl6advent.wordpress.com/

Last Year's Plack calendar

http://advent.plackperl.org/

For the adventurous: Japanese Perl Advent Calendars, 8 different tracks!

http://perl-users.jp/articles/advent-calendar/2010/

Hacker Track

Casual Track

English Track

Acme Track

Win32 Track

Meta Advent Calendar Track

Symbolic Programing Track

perl 6

One bonus list, for the sysadmin in your life:

SysAdvent - The Sysadmin Advent Calendar.

http://sysadvent.blogspot.com/

Vimana : cpan module to automate vim plugin installation

VIMANA! The Vim script manager. A cpan module for downloading and installing vim plugins! It works with .vim files, archive files (zip, rar), and vimball formats. By c9s / cornelius / Yo-An Lin. caveat: the "installed" command only recognizes plugins installed via vimana.

Cornelius's perl hacks on vim presentation has been on slideshare for two years. It covers "why you should improve your editor skills" -- ranging from "stop moving around with the arrow keys" to advanced commands and plugins. He's written quite a few vim plugins and quite a lot of cpan modules. Props!

Vimana Example:

% cpan Vimana
% vimana search nerd
nerd-tree-&-ack     [utility]      Adding search capability to NERD_Tree with ack
the-nerd-tree       [utility]      A tree explorer plugin for navigating the filesystem
nerd-tree-project   [utility]      It tries to find out root project directory, browse project file with NERD_tree.
the-nerd-commenter  [utility]      A plugin that allows for easy commenting of code for many filetypes.
findinnerdtree      [utility]      Expands NERDTree to file in current buffer

% vimana info the-nerd-tree
#... shows the install instructions ...

% vimana install the-nerd-tree
Plugin will be installed to runtime path: /home/andrew/.vim
Package the-nerd-tree is not installed.
Downloading plugin
.
 - Makefile : Check if makefile exists. ...not ok
 - Meta : Check if 'META' or 'VIMMETA' file exists. support for VIM::Packager. ... - Rakefile : Check if rakefile exists. ...not ok
Package doesn't contain META,VIMMETA,VIMMETA.yml or Makefile file
Copying files...
/tmp/yohBObI3iy/ => /home/andrew/.vim
Updating helptags
Done

There are quite a few plugins mentioned in the presentation. I've listed the ones I'm interested below. I'll be installing and reviewing them soon. Slide 120 begins a nice section on advanced movement keys. Slide 135 has a list of the variables controlling the perl syntax highlighter.

Perl folding variables:

" set :help folding for more information
:set foldmethod=syntax              " enable syntax based folding
let perl_include_pod=1              " fold POD documentation.
let perl_extended_vars=1            " for complex things like @{${"foo"}}
let perl_want_scope_in_variables=1  " for something like $pack::var1
let perl_fold=1                     " enable perl language based folding
let perl_fold_blocks=1              " enable folding based on {} blocks

---- slide 135

Vim Plugins:

Exciting vim plugins to check out from vim.org. Reviews and usage information to come in future posts. I'm excited to try Cornelius's updated OmniCompletion helpers.

perlprove.vim

http://www.vim.org/scripts/script.php?script_id=1319

How does this compare with efm_perl.pl, included with vim?

DBExt.vim

http://www.vim.org/scripts/script.php?script_id=356

Run database queries from inside vim. I skipped past this the first time around, but now I see that it will let you copy the query directly from your source language, apply that languages string mechanics to get the output string and then prompt for bound variables. interesting!

FuzzyFinder

http://www.vim.org/scripts/script.php?script_id=1984

Allows a handy shorthand mapping to search for files/buffers/etc.

the NERD tree

http://www.vim.org/scripts/script.php?script_id=1658

updated file listing explorer

You're already using this, right?

the NERD commentor

http://www.vim.org/scripts/script.php?script_id=1218

improved commenting, under current development.

taglist

http://www.vim.org/scripts/script.php?script_id=273

a ctag integration that shows tag information for the current file / source code browser

"The most downloaded and highest rated plugin on vim.org"

BufExplorer

http://www.vim.org/scripts/script.php?script_id=42

Buffer Explorer / Browser -- easily switch between buffers.

Git-Vim

https://github.com/motemen/git-vim

Git commands within vim.

HyperGit

http://www.vim.org/scripts/script.php?script_id=2954

a git plugin for vim ( with a git tree menu like NERDtree ), by c9s.

screenshot

Updates to vim omnicompletion for perl:

http://www.vim.org/scripts/script.php?script_id=2852

Demonstration video

https://github.com/c9s/perlomni.vim

AutoComplPop

http://www.vim.org/scripts/script.php?script_id=1879

Automatically opens popup menu for completions

TEDxCALTECH -- Friday, January 14, 2010

"Feynman's Vision -- The next 50 years"

TEDx, the independent TED event series is coming to Caltech in January. TEDx events are inspired by TED and use the same plans and speaking formats. I'm surprised I haven't heard more buzz about this event. I wasn't able to get into TEDxUSC, the first of the TEDx events. I'll be volunteering for the event and hope to see you there!

Will we see coverage of Feynman's vision of the "race to the bottom" -- his challenge to his engineering and scientist peers to work together and compete to see how small we can go in nanotech. Will anyone from Professor Tai's micro-machine lab be speaking? I'm sure we'll see interesting wetware - bio/cs/eng research, given the Institute's focus on Bio over the past decade.

You won't know until you go! (Or until you check the speakers tab)

Feynman's Vision -- The next 50 years.

TEDxCaltech is a community of people who are passionate about sharing "Feynman’s Vision: The Next 50 Years." If that sounds like something you want to be a part of, complete and submit the application below. Due to limited venue space, we cannot approve all applicants instantaneously. If you are approved, you will receive an email shortly either inviting you to register for the event, or letting you know that you are on the waiting list. The registration fee is $25 for Caltech students; $65 for Caltech faculty, staff, postdocs, alumni and JPL; and $85 for all others. The all-inclusive day will begin with breakfast and will be punctuated with generous breaks for food and conversation with fellow attendees. It promises to be an exciting and entertaining intellectual adventure—a time to unplug from the day-to-day routine. You won’t want to miss a minute!

Registration opens at 8:00 am, doors open at 9:30 am, talks conclude at 6:00 pm and will be followed by a reception. Parking is free.
-- http://www.tedxcaltech.com/apply

perl, tags, and vim : effective code browsing.

Adding a tags file to vim makes for an effective code browser for perl.

I've just started a new job, so I have a large new repository of perl code and modules to familiarize myself with. I've taken this as an opportunity to refresh my tag-fu in vim. After creating a tag file with ctags [exhuberant ctags], I can now jump around my whole perl repo from within my vim session.

The -t command-line flag to vim opens files by tag (module) name, e.g. vim -t My::Module::Name. Within a vim session, I jump to the definition of a function by hitting ctrl-] with the cursor over a usage of the function, even if that definition is another file! Ctrl-t and I'm back where I started.

Today I found the -q flag to ctags, which adds the fully qualified tag for package methods, e.g. My::Package::method_1, which aids with long package names and the ctrl-] key. FTW!
I have this set of ctag flags aliased as "ctagit":


ctags -f tags --recurse --totals \
--exclude=blib --exclude=.svn \
--exclude=.git --exclude='*~' \
--extra=q \
--languages=Perl \
--langmap=Perl:+.t

In my .vimrc file, I defined the tags search path as ./tags, tags,~/code/tags ;, this will look for a tags file in the directory of a loaded file, then the current directory, and then a hardcoded path in my code area.

" [ .vimrc file ] " set tag search path: directory of current file, current working directory, hard-path set tags=./tags,tags,~/code/tags

More info on using tags in vim is available in :help tags. I've found the following commands useful.

ctrl-]	jump to tag from under cursor
visual ctrl-]	jump to tag from visual mode
:tag tagname	jump to tag tagname
:stag tagname	split screen and open tagname
:tags	show the tag stack, '>' labels the current tag
:ctrl-t	jump to [count] older entry in the tag stack
:[count]pop	jump to [count] older entry in tag stack
:[count]tag	jump to [count] newer entry in tag stack

Update:
To configure vim to treat ':' (colon) as part of the keyword to match Long::Module::Sub::Module package names, add it to the iskeyword setting. I have multiple perl filetype hooks stored in files in .vim/ftplugin/perl/. These filetype hooks are enabled with the filetype plugin on directive in my main .vimrc file.

" [ .vimrc file]
"enable loading the ftplugin directories
filetype plugin on

" [ .vim/ftplugin/perl/keyword_for_perl file]
" Append : to the list of keyword chars to allow completion on Module::Names
set iskeyword+=:

The same effect could be conjured directly in the .vimrc file via autocmd:

" append colon(:) to the iskeyword list for perl files, to enable Module::Name completion.
autocmd FileType perl set iskeyword+=:

My Configuration files are available in the spazm/config git repository on github.

NoNaNoWriMo

Seems I have just said "no," to blogging in November.  I have not been off writing the great American novel.
This drops me back to  paperman status for perl ironman blog competition.  Time to start back up?

naptime

Siesta! Nap time! Plenty of studies show that people who nap are more productive and more creative. We all napped in kindergarten. Why is it that adult naps are dismissed by corporate and polite society? Rather, we feed the endless caffeine addiction that powers the modern factory mentality. Drucker shows us that knowledge workers are paid for their thought, insight and productivity not for their time.

It's time to let them nap and see if effectiveness improves.

"Knowledge worker productivity is the biggest of the 21st century management challenges. In the developed countries it is their first survival requirement. In no other way can the developed countries hope to maintain themselves, let alone to maintain their leadership and their standards of living."
---- Peter Drucker, Management Challenges for the 21st Century (1999)

"We know now that the source of wealth is something specifically human: knowledge. If we apply knowledge to tasks we already know how to do, we call it 'productivity'. If we apply knowledge to tasks that are new and different we call it 'innovation'. Only knowledge allows us to achieve these two goals."
---- Peter Drucker, Managing for the Future (1992)

Simba (my dog) is on his second nap of the day. I think I'll join him for this 2pm siesta and see how it affects the rest of my day and outlook. Night night!

• Life While article on napping
• powersleep.org
• Knowledge Worker via wikipedia

Announce: Net::HTTP::Factual

Factual hosts community editable data on a variety of topics, and hopes to be your one-stop-shop for wiki-data. They provide an element of "truthiness" for rows to show if the data is verified and some other interesting ideas.

They hosted a contest at Startup weekend LA last week, with the top team winning kindles for each member. I didn't win the contest[*], since I got pulled onto a different team and didn't finish my factual thingy (clearly I didn't define my factual thingie either). I did create a quick cpan module to wrap the factual ReST interface, Net::HTTP::Factual. I built it using Spore.

All I needed to do was translate the API documentation into json for spore to load. I also needed to work out the differences between Spore v0.2 spec format the v0.1 documentation. The demonstration spec files in the spore github are in the v0.1 format. These are the problems of using bleeding edge alphaware -- Spore v.01 came out Oct 12, v0.2 on Oct 14 and I was using them Oct 16.

* : I didn't win the contest, but it did remind that I wanted a kindle. So I went home and ordered one. It arrived two days ago, and I'm 80 pages into my first book. The screen is gorgeous. I'm reading a PDF of the Mythical Man-Month, and while the pdf handling isn't perfect it works well -- flows quite nicely reading from a rotated screen. Props!

LA.pm November meeting: Wed Nov 17 @Media Temple

Dearest Mongers,

See you in a few weeks at the November meeting, generously hosted by Media Temple. With our early October meeting it feels like it's such a long time in between, I'm missing you all.

I hope you can make it!

What:     Los Angeles Perl Mongers meeting
Date:     Wednesday, November 17
Time:     7-9pm.
Location: Media Temple.

topics and directions will be posted at la.pm.org

--Andrew

your data is your program.

Much more often, strategic breakthrough will come from redoing the representation of the data or tables. This is where the heart of a program lies. Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won't usually need your flowchargs; they'll be obvious.
--The Mythical Man-Month

In theory, I read The Mythical Man-Month almost 20 years ago for CS1. At the time, his historical examples seemed so dated, talking about monthy rental of kilobytes of ram and OS360 and PDP8 and PDP11 design. What could that teach me about cranking out modern C code?

As I reread it know, I realize how timeless are his examples of software as engineering discipline. Problems we still haven't solved -- how to maintain the vision of a small number of minds over a project that requires many, many hands.

I wish I could have understood his words back then, but wisdom is not so easily gained.

Experience is a dear teacher, but fools will learn at no other.
-- Benjamin Franklin

Dost thov love life? Then do not sqvander time
[ for that is what life is made of ]
-- Benjamin Franklin, as inscribed in Blacker Hovse

GRT : Guttman Rosler Transform

The Guttman-Rosler Transform is a technique from Uri Guttman and Larry Rosler for improving sort speed in perl.

The Guttman-Rosler transform runs faster than the orcish or Schwartzian transforms by avoiding the use of custom search subroutines. This is accomplished via pre and post transformation of the list. This allows the native optimizations of the perl sort function to shine.

Sort::External has a special affinity for the GRT or Guttman-Rosler Transform, also known as the "packed-default" sort. The algorithm is examined at length in "A Fresh Look at Efficient Perl Sorting", a paper by Uri Guttman and Larry Rosler, located as of this writing at http://www.sysarch.com/perl/sort_paper.html.

For many applications, the GRT is the most efficient Perl sorting transform. This document explores how to use it to solve common coding problems in conjunction with either Perl's built-in sort() function or Sort::External.

-- Sort::External::Cookbook

Synopsis:

1. prefix each element with a synthetic key (string or numeric).
2. sort.
3. remove the prefix key from each element.

Example:

    
    my %colors = (
        "Granny Smith"     => "green",
        "Golden Delicious" => "yellow",
        "Pink Lady"        => "pink",
    );

  #A) standard sort:
    my @sorted_by_color = sort { $colors{$a} cmp $colors{$b} } keys %colors;

  #B) GRT sort:
    # 1. Encode
    my @sortkeys;
    while ( my ( $variety, $color ) = each %colors ) {
        push @sortkeys, sprintf( "%-6s%-16s", $color, $variety );
    }
    # @sortkeys = (
    #     "green Granny Smith    ",
    #     "yellowGolden Delicious",
    #     "pink  Pink Lady       ",
    # );

    # 2. Sort
    my @sorted_by_color = sort @sortkeys;    # no sortsub!

    # 3. Decode
    @sorted_by_color = map { substr( $_, 6 ) } @sorted_by_color;

Links:

Research Paper from Uri Guttman and Larry Rosler:

http://www.sysarch.com/Perl/sort_paper.html

Sort Sort::Maker on cpan:

http://search.cpan.org/perldoc?Sort::Maker

Sort::External::Cookbook

Http://Search.cpan.org/perldoc?Sort::External::Cookbook

Aloha Rubicon!

From:    Andrew Grangaard
To:      Rubicon Project
Subject: Aloha Rubicon

Dearest Rubicon Coworkers,

I have moved on from the Rubicon Project, saddened that opening new doors requires closing old ones. A closing of two wonderful years at tRP -- challenges bested, bonds forged and friends earned.

I will miss you all and our shared sense of wonder and excitement at preparing to attack the sleeping google giant. I hope my two years of improvements to infrastructure and fundamentals will aid you in the days and years ahead.

Pacing, RTS, click tracking, proration, scheduling, scheduler v2, scheduler v3, aggregation, merging, hadoop, datacube, svn branch maintenance and svnmerge merging, TDD, unit testing, jira tickets and wiki pages, reproducability, deployment, quality assurance ... I'll miss all my initiatives and projects (I won't miss you, aggregator-multithreaded.pl !!)

Thanks and adieu to my wonderful teammates from my two years on the core team Masao, Bill, Peter, Ian, Subash, Damien, David, Jonathan, Willis, Yushik and my core team leads: Mark Douglas, Damien, Mon-Chaio,, Damien(x2), Mon-Chaio(x2), Karim, Duc(x2) and most recently Sam. Core work is challenging and difficult. I salute the dedication and devotion you have all shown to our team to battle the twin complexities of our problem domain.

James, thanks for working with me during my first week when I was pitching out to help the UI team. Subash, you were a wonderful first week buddy. I hope I was able to help other newRubies the way you helped me. Jonathan, you've been a great desk neighbor and programming pair, even when frustrated you're still unassailably upbeat.

RI team, thanks for fighting to put me your squad and then graciously loaning me back to core for the okanagan push.

Mallory, our Den Mother and Cruise Director, you've done an impressive job of adapting the culture over the two-and-a-half population doubles we've seen in my two years.

Frank and Craig, thanks for always telling it like it is and welcoming my questions. I've never seen this level of internal transparency. I know you'll keep the ship on a steady course.

To my many friends and acquaintances on the other teams: you are family. Look me up any time. Life is so different without you all around everyday. Thanks for always bringing a smile to my face.

Thanks to the many of you that I leaned over the past 13 months through my divorce. Especially to those who did not know the circumstances, yet supported me anyways because that's just what awesome people do. Your strength and support have been invaluable.

Aloha,
Andrew

• linked in
• facebook
• @spazm twitter
• blog
• Events Los Angeles Perl Mongers and Los Angeles NerdNite

PS. "woof :(" -- Simba

Reading is FUNdamental

I am currently reading three CS books. The differences in styles and approaches are striking, but they are all effective in teaching.

Structure and Interpretation of Computer Programs aka SICP.

Classic text computer science and functional programming using Scheme as the implementation language. Used in 6.001 at MIT.

Related links:

• official mitpress site with html edition
• pdf version found on citeseerx
• mit video lectures
• online tutor

Programming Collective Intelligence

Interesting algorithms for mining the huge datasets available in the web era. Uses python as the implementation language. Working through the first three chapters, the first thing I did was rewrite the examples in perl. Then I found I wasn't the only one to do so.

• O'Reilly catalogue page

Modern Perl

chromatic has written a lovely book on Modern Perl, and shared the review copy for comments/editing. It was pretty sweet to email in some typos and receive a link back to an updated git repository with my changes applied.

• Modern Perl Books blog
• chromatic's blog post with a review copy.

LA.PM hits the road for October meeting

Los Angeles Perl Mongers meeting for October will be held downtown hosted by Oversee.net. Randall Schwartz is gathering speakers and handling the Oversee details. We are meeting early too, second Wednesday. see you October 13th!

send email to the list if you are interested in carpooling from the westside.

la.pm.org

September Los Angeles Perl Mongers

Game on!

I'm too sick to be here, but I came into the office tonight to run the meeting. Mad props to Tommy for driving down from Westlake to present tonight. Aran is also sick, so he's bailing on presenting. His presentation "12 cpan modules in 12 penta minutes" may well be cursed.

Big crowd tonight, 20+.

convert to CPAN Testers 2.0 (CPANTS)

I woke up to find a bevy of "Mail Delivery Failure" messages in my inbox. Seems the cpan-test reports I emailed in bounced back because cpan tester 2.0 dropped support of incoming email reports in favor of http. I'm excited to hear about this http switch, as I hated not being able to send test reports from machines that lacked email configurations.

This message was created automatically by the mail system (ecelerity).

A message that you sent could not be delivered to one or more of its recipients. This is a permanent error. The following address(es) failed:

>>> cpan-testers@perl.org (after RCPT TO): 550 cpan-testers no longer accepts test submissions via email. Please convert to CPAN Testers 2.0 and the http submission method. Instructions are at:
   http://wiki.cpantesters.org/wiki/QuickHowToCT20.

Let's follow the wiki article: http://wiki.cpantesters.org/wiki/QuickHowToCT20 and get upgraded!

Upgrade steps for a current CPAN::Reporter user:

1. Upgrade CPAN::Reporter, add Test::Reporter::Transport::Metabase module
# check current version:
% perl -MCPAN::Reporter -l -e 'print $CPAN::Reporter::VERSION'
1.1711
#upgrade
% cpanm CPAN::Reporter
...
Successfully installed CPAN::Reporter
% cpanm Test::Reporter::Transport::Metabase
...
Successfully installed Test-Reporter-Transport-Metabase-1.999008
2. create a profile using 'metabase-profile', put it into location.
   % metabase-profile
Enter full name: ...
Enter email address: ...
Enter password/secret: ...
Writing profile to 'metabase_id.json'
% mkdir ~/.cpantesters
% mv metabase_id.json ~/.cpantesters
% chmod 400 ~/.cpantesters/metabase_id.json

3. upgrade my ~/.cpanreporter/config.ini file to add a transport line
   #add transport line to my ~/.cpanreporter/config.ini file
% echo 'transport = Metabase uri https://metabase.cpantesters.org/api/v1 id_file ~/.cpantesters/metabase_id.json' >> ~/.cpanreporter/config.ini

4. Test
cpan Hadoop::Streaming
...
CPAN::Reporter: Test result is 'pass', All tests successful.
CPAN::Reporter: preparing a CPAN Testers report for Hadoop-Streaming-0.102520
CPAN::Reporter: ssending test report with 'pass' via Metabase
...
5. Verify Test : Check metabase tail log for my entry.
% wget --quiet http://metabase.cpantesters.org/tail/log.txt -O- | grep -i grangaard
[2010-09-20T20:47:06Z] [Andrew Grangaard] [pass] [SPAZM/Hadoop-Streaming-0.102520.tar.gz] [i486-linux-gnu-thread-multi] [perl-v5.10.1] [3acd2e9e-c4f8-11df-b898-64160c3e84b1] [2010-09-20T20:47:06Z]


Dr. Frankenstein, It lives!

Hadoop::Streaming PAUSE registration submitted

Submitted a PAUSE (Perl Authors Upload SErver) request to register Hadoop::Streamingin the User Interface tree at CPAN. I wasn't really sure which top-level category to put it in, but settled on UI as it provides a simple adaption of the Streaming interface of Hadoop.

Woo, my first registered module space. Update: oooh, brian d foy!

On Wed, Sep 08, 2010 at 06:50:34AM +0200, Perl Authors Upload Server wrote:
> 
> The next version of the Module List will list the following module:
> 
>   modid:       Hadoop::Streaming
>   DSLIP:       RdpOp
>   description: simple interface to Hadoop Streaming
>   userid:      SPAZM (Andrew Grangaard)
>   chapterid:   8 (User_Interfaces)
>   enteredby:   BDFOY (brian d foy)
>   enteredon:   Wed Sep  8 04:50:33 2010 GMT
> 
> The resulting entry will be:
> 
> Hadoop::
> ::Streaming       RdpOp simple interface to Hadoop Streaming         SPAZM

Hadoop::Streaming 0.102490 pushed to CPAN

I've pushed a new release of Hadoop::Streaming to CPAN. It should be available in a couple of hours, depending on how long it takes your CPAN mirror to do the mirror update dance.

The release includes expanded documentation in the base Hadoop::Streaming placeholder file. Also included is a Hadoop::Streaming::Combiner role, for creating combiners. Combiners are like reducers that run post-map, per-merge. Once can reuse the reducer as combiner, if the reducer produces the same key/value format on output as input.

After writing my new documentation, test and code, I tested it with dzil test. After passing tests, it's a simple 1 step push to CPAN and github via dzil release. AWESOME! Dist::Zilla makes maintaining CPAN modules brilliantly easy.

Happy Labor Day!

Links

CPAN - Comprehensive Perl Archive Network

http://search.cpan.org

Hadoop::Streaming perl modules

http://search.cpan.org/perldoc?Hadoop::Streaming

Dist::Zilla

http://search.cpan.org/perldoc?Dist::Zilla

There's no Step Two!

[andrew@mini]% dzil release                                1 ~/src/hadoop-streaming-frontend
[DZ] beginning to build Hadoop-Streaming
[DZ] guessing dist's main_module is lib/Hadoop/Streaming.pm
[DZ] extracting distribution abstract from lib/Hadoop/Streaming.pm
[DZ] writing Hadoop-Streaming in Hadoop-Streaming-0.102490
[DZ] writing archive to Hadoop-Streaming-0.102490.tar.gz
[@Basic/TestRelease] Extracting /home/andrew/src/hadoop-streaming-frontend/Hadoo
p-Streaming-0.102490.tar.gz to .build/dVEDcaew44
Checking if your kit is complete...
Looks good
Writing Makefile for Hadoop::Streaming
cp lib/Hadoop/Streaming.pm blib/lib/Hadoop/Streaming.pm
cp lib/Hadoop/Streaming/Combiner.pm blib/lib/Hadoop/Streaming/Combiner.pm
cp lib/Hadoop/Streaming/Role/Emitter.pm blib/lib/Hadoop/Streaming/Role/Emitter.p
m
cp lib/Hadoop/Streaming/Reducer/Input/ValuesIterator.pm blib/lib/Hadoop/Streamin
g/Reducer/Input/ValuesIterator.pm
cp lib/Hadoop/Streaming/Reducer.pm blib/lib/Hadoop/Streaming/Reducer.pm
cp lib/Hadoop/Streaming/Reducer/Input/Iterator.pm blib/lib/Hadoop/Streaming/Redu
cer/Input/Iterator.pm
cp lib/Hadoop/Streaming/Role/Iterator.pm blib/lib/Hadoop/Streaming/Role/Iterator.pm
cp lib/Hadoop/Streaming/Reducer/Input.pm blib/lib/Hadoop/Streaming/Reducer/Input.pm
cp lib/Hadoop/Streaming/Mapper.pm blib/lib/Hadoop/Streaming/Mapper.pm
Manifying blib/man3/Hadoop::Streaming::Combiner.3pm
Manifying blib/man3/Hadoop::Streaming.3pm
Manifying blib/man3/Hadoop::Streaming::Role::Emitter.3pm
Manifying blib/man3/Hadoop::Streaming::Reducer::Input::ValuesIterator.3pm
Manifying blib/man3/Hadoop::Streaming::Reducer::Input::Iterator.3pm
Manifying blib/man3/Hadoop::Streaming::Reducer.3pm
Manifying blib/man3/Hadoop::Streaming::Role::Iterator.3pm
Manifying blib/man3/Hadoop::Streaming::Reducer::Input.3pm
Manifying blib/man3/Hadoop::Streaming::Mapper.3pm
PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/00-load.t ....... ok
t/01-wordcount.t .. 8/? # perl path -> /usr/bin/perl
t/01-wordcount.t .. ok
t/02-analog.t ..... ok
All tests successful.
Files=3, Tests=19,  3 wallclock secs ( 0.05 usr  0.01 sys +  2.34 cusr  0.20 csys =  2.60 CPU) 
Result: PASS   

[@Basic/TestRelease] all's well; removing .build/dVEDcaew44
*** Preparing to upload Hadoop-Streaming-0.102490.tar.gz to CPAN ***

Do you want to continue the release process? [y/N]: y
[@Git/Check] branch master is in a clean state
[@Basic/UploadToCPAN] registering upload with PAUSE web server
[@Basic/UploadToCPAN] POSTing upload for Hadoop-Streaming-0.102490.tar.gz
[@Basic/UploadToCPAN] PAUSE add message sent ok [200]
[@Git/Commit] Committed Changes
[@Git/Tag] Tagged v0.102490
[@Git/Push] pushing to origin

github + cpan = gitpan

Gitpan is a clone of all the modules on cpan in git form, nearly twenty-two thousand public repositories. This is not a place for development of modules. Instead it is a place to easily pull the current source for a module to make a patch and send to the maintainer, without having to find where she maintains her golden copy.

I read about gitpan a while ago, but then when I wanted to find it last week, I couldn't find the correct search terms. [github cpan] produces a list that doesn't include gitpan in the first page, as it is crowded out by the many perl modules developed on github for release to cpan and of course things like Net::GitHub and GitHub::Import, and an interesting discussion at perlmonks on (informal) perl naming convention for github projects.

Now that I know the name, it is still hard to find information! From the FAQ section of the readme:

What is gitPAN?
---------------
gitPAN is a project to import the entire history of CPAN (known as BackPAN) into a set of git repositories, one per distribution.

Why is gitPAN?
--------------
CPAN (and thus BackPAN) is a pile of tarballs organized by author. It is difficult to get the complete history of a distribution, especially one that has changed authors or is released by multiple authors (for example, Moose). Because releases are regularly deleted from CPAN even sites like search.cpan.org provide an incomplete history. Having the complete history of each distrubtion in its own repository makes the full distribution history easy to access.

gitPAN also hopes to make patching CPAN modules easier. Ideally you simply clone the gitPAN repository and work. New releases can be pulled and merged from gitPAN.

gitPAN hopes to showcase using a repository as an archive format, rather than a pile of tarballs. A repository is far more useful than a pile of tarballs, and contrary to many people's expectations, the repository is turning out smaller.

Finally, gitPAN is being created in the hope that "if you build it they will come". Getting data out of CPAN in an automated fashion has traditionally been difficult.

Where is gitPAN?
----------------
The repositories are on github.com at http://github.com/gitpan (watch out, it may overload your browser).

Code, discussion, and issues can be had at http://github.com/schwern/gitpan.

[...]

How can I contact gitPAN?
-------------------------

Email:   schwern+gitpan@gmail.com
Web:     http://github.com/gitpan/
Dev:     http://github.com/schwern/gitpan
Issues:  http://github.com/schwern/gitpan/issues
Twitter: #gitpan

Links:

google search for [github cpan]

http://google.com/search?q=github+cpan

google search for [gitpan]

http://google.com/search?q=gitpan

gitpan at github -- 21,976 public repositories and counting!

Schwern's announcement of gitpan on his use.perl blog.

http://use.perl.org/~schwern/journal/39972

http://github.com/gitpan

discussion of gitpan and code:

http://github.com/schwern/gitpan

gitpan issues:

http://github.com/schwern/gitpan/issues

a page with 4 links at integra.net

http://gitpan.integra.net/