Tuesday, September 22, 2009

Care and feeding of third party perl modules at work -- how do you do it?

How do you install and maintain third party perl module dependencies in your work perl code base? How about your own perl modules?

I'll be hosting an open discussion on this tomorrow at the Los Angeles Perl Mongers meeting tomorrow. (Wed 2009-09-23). Let's see what we can dig up.

I've been at a few perl companies now, and they all seem to have taken different approaches. Yes, TIMTOWTDI, but TIMTOWTDIWrong aka TIMTOWTFIU. This seem like it should already be a "solved problem," so what are the best practices in this arena?

Popular memes in this space:

  • System Perl
    • Install to system perl via cpan. Upgrade whenever.
    • Install to system perl via package manager, deb or rpm. Manually roll unpackaged cpan modules through a tool like cpanflute.
    • Sub part 1: install those rpms/dpkgs manually
    • Sub part 2: run a local yum/apt-get/dag repository internally to pull "trusted" internally packaged cpan modules
    • Install via system tools (puppet, etc) live on each box from a configuration file of needed modules.
  • (shared) Application Perl, compile your own single perl version for internal applications
    • Install into the application perl via cpan
    • check into local source control source for third party modules, install during application install.
    • check into local source control binary version of third party modules.
  • Install into local library, using Local::Lib with either application or system perl.
  • Install from a "mini cpan", maintained internally and populated with vetted, approved third party modules.
How do you do it?

If you're just installing a handful of modules, the ad-hoc approach may be sufficient. Right now I'm writing some new apps with Moose, and installing that module dependency chain seems crazy without figuring out a plan. With Moose and the MooseX namespace there are probably 20+ modules I want to install, and many of them are improving and releasing rapidly. Edit: Sorry Dave, you're right, not hundreds of modules. I didn't mean to spread Moose FUD, I love the Moose

Plan: flesh out this page with links to the current wisdom on these matters, from fellow iron mongers, perl monks, etc. I know I've ready half a dozen recent articles on this, time to find and organize them.


Ranguard said...

http://code.google.com/p/cpan-debian-package-builder/ - build debian packages of both CPAN and your own code.

Very much rough at the edges, and also could do with integration with Shipwright, but has worked happily for me for 4 or so years.

Unknown said...

My current process is to combine the minicpan + local::lib approaches. I have a minicpan of stuff I know works and then each application and developer has their own local lib, so I maintain application dependencies on an app level, not on a user level as the example code for Local::Lib shows. I then use a Makefile.PL to manage app level dependencies.

The only thing I do differently is I install DBD::mysql globally, since that's a big hassle to deal with installing. I usually use my OS package manager (such as apt) to install that one. All the rest I use cpan with a local lib.

Dave Rolsky said...

Hundreds deep? How does c.10-20 (depending on your Perl version) become hundreds? Why do people keep spreading this "Moose has a lot of deps" FUD?

It's getting kind of annoying.

Andrew Grangaard said...

Dave, sorry to inadvertently spread the moose FUD. There are lots of MooseX libraries that I want to install, on top of Moose. But probably not hundreds.

Updating the post.

Anonymous said...

s/perl flute/cpanflute/

Anonymous said...

We use .debs - either backports or quickly made with dh-make-perl.

Modules in applications are simply checked out from the source code repository.

(Using Ubuntu.)

Andrew Grangaard said...

Anonymous said...
s/perl flute/cpanflute/
Thanks! fixed.