Showing posts with label social coding. Show all posts
Showing posts with label social coding. Show all posts

Sunday, February 8, 2015

haskell on centos 6.5

Use justhub rather than version in epel repo.

Don't bother with the version of haskell-platform in epel repo. It is sufficiently out-of-date (circa 2010) that it can't update via cabal install cabal-install. Jump straight to using justhub.

Justhub example for centos 6.x:

# install the justhub yum repo:
sudo rpm -ivh http://sherkin.justhub.org/el6/RPMS/x86_64/justhub-release-2.0-4.0.el6.x86_64.rpm

# install single current haskell version into /usr/bin
yum install haskell

# update cabal
cabal update

# e.g. install some packages via cabal
cabal install haskell-src-exts
Now I can get back to coding for exercism.io. Come review my first haskell program.

Saturday, June 8, 2013

Hack day with Kenny: Fey::ORM, testing and screen. [lost draft from 1/12/10]

After sleeping through the LILAX users group meeting (sorry guys), I rolled up to Kenny's (Kenny Flegal), where he had invited me for a day of coding and authentic Salvadorian food. Win Win!

I showed him briefly the topic of my upcoming Monger's presentation, but mostly we looked at his current project. He is forking a GPL licensede project, to recreate part of the functionality and extend it in a different direction. Along the way he's rewriting the app layer in perl from command line php scripts.

We discussed the various clauses of the Gnu Affero GPL with regards to the hosting of the project during the initial revs. Can he have a public repository before he has finished changing all references to the old name to a new name and adding "prominent notices stating that you modified it, and giving a relevant date" as per Section 5, paragraph a? We decided that he probably could, but that it'd be easier to start with a private repo and not publish until that part is done. That seems sub-optimal from a "getting the source to the people" mindset, but it is more optimal in the "protect the good name of the original project and publishers."

Along with switching from php to perl, he's pulling out the hard coded sql from the scripts and moving to an ORM. He's picked Dave Rolsky's impressive Fey ORM. This project has a ridiculously complex set of schemas, with inconsistent table names and not explicit foreign key constraints. As such, it is extra work to get the fey schema situated.

Kenny started to give me a run through of some of the code, but it was awkward with both of us on laptops to see the code conveniently. I made him stop and set up a screen session for sharing, as described in my previous post on screen. This was more difficult than I expected, with the problem eventually being that ubuntu 9.4 and beyond has moved /usr/bin/screen to /usr/bin/screen.real and made screen a shell wrapper. The screen multiuser ACL system requires that the screen binary be setuid (chmod +s). With this setup we needed to make screen.real setuid. That took a while to notice.

Once we had a shared session open, it was much easier for him to give me a guided tour of the codebase and database/sql setup. Once that was clear it was time to get some code started. He showed me some of the Fey::ORM model code and how he was migrating over the individual sql statements to the ORM. He had been plugging away on the model code for a while, starting by creating a comment for every line of sql in the application including the file and line of the caller.

The next step was clear, we needed some tests. We set to work getting an initial test of the model code. First we installed Fey::ORM::Mock as a mock layer. This works at a higher level than a standard DBD::Mock interface to allow better testing of the Fey::ORM features. The test didn't pass at first due to missing data in the mock object, so we grabbed a list of the fields that mapped to DB fields and started adding values to pass constraint failures on the data. Once we had a minimal set of data then we started to see problems with the ORM schema description. The lack of well defined foreign key constraints meant we needed to explicitly define that structure for the ORM. More boilerplate code into the model. We repeated this test-update-repeat cycle a few more times adding more data linkage descriptions.

I took a brief break from our pairing and jumped to a different screen to install some goodies. I grabbed a copy of the configuration files from the December la.pm.org talk and started updating his config. He didn't have a .vimrc, .vim or .perltidyrc on this brand new dev box, so I pulled those in from the repo. I showed him how much time using ":make" in vim could slice off his build/test cycle, and he was super excited. (ok, not till the third or fourth try but he eventually got the hang of it).

To get around some issues in code placement, I modified the .vimrc and .vim/ftplugin/compiler code to add -MFindBin::libs to the calls to perl -c and prove. This allowed the parent libs/ directory to be found for these non-installed modules. This is a bit of a hack and I'll get it removed as we move closer to an initial release and pick a packaging tool, possibly Dist::Zilla.

An open question is the speed of Fey::ORM. It takes a big startup hit while building the models from the schema and interacting with the database. This is supposed to lead to a big speed gain during runtime from aggressive caching of that information. All I know for certain is that the compile-run-test cycle was really slow. This is my first time using Fey so I don't know how this plays out normally. It could just be that the number of crosslinked tables in the db config were causing additional slowdowns.

By this point we had already had two delicious meals of El Salvador cuisine and it was approaching midnight. The first meal was home cooked fried (skinless) chicken for lunch and the second was papoosas at a local, excellent place in Van Nuys. I was all coded out, which made for a perfect transition to the party at Andy Bandit's that night, conveniently just 6 miles from Kennys.

All in all, a fine Saturday.

Thursday, May 23, 2013

Code Review (part 1)

I love code review

What is code review? This wikipedia quote sounds ok, but who could love anything that includes "formal process" and "scrutinized"? Sounds like a lot of work, right? What's the upside?

Code review is systematic examination (often as peer review) of computer source code intended to find and fix mistakes
http://en.wikipedia.org/wiki/Code_review
A code review (sometimes called a program inspection) is a formal process where a software developer presents the code he or she has written to other software engineers who are familiar with the project. The code is scrutinized carefully to identify potential bugs, design problems, non-compliance with project standards, inconsistencies, and any other problems in the code.
http://sea.ucar.edu/best-practices/code-review

Code review allows developers to collaborate and improve code by reading it early and catching bugs during development. The earlier bugs are caught, the less impact and expense they cause. Code review is a lot more fun before the changes are live than retroactively trying to figure out what change broke everything in production.

I remember code reviews at my first software company. Someone must have heard they were a good idea, so we had to do code reviews of new features. We waited until the feature was done, then printed out all the code and took a few engineers into a room. We'd sit there for a few hours looking over the printouts before making a few token suggestions and calling it quits. We shared some small insights and caught a few bugs, but overall this heavy process was unstructured, late, disorganized and ineffective. We had the right motivations, but we were looking at code too late in too big of a chunk.

At the other end of the scale is the ad-hoc system of emailing around some diffs or code refs and asking for input. Here the lack of formal process is a pain -- emailing diffs around? Another process flowing through (stalling in) my mailbox? Where do I send my comments, how do I archive the results?

Somewhere in the happy middle are tools for "light weight code review." These tools take a diff and present it in a web interface providing the ability to view the diffs and make comments and enforce some sort of workflow. Gerrit (inspired by Rietveld inspired by Mondrian), Review-Board and BarKeep are some of the open source options, github reviews are free and pay software is available from SmartBear (Code Collaborator), Atlassian (Crucible) among others. These systems all make different trade-offs: pre-vs-post commit, forced-vs-optional reviews, VCS agnostic-vs-integrated, inline vs side-by-side diffs.

At work we've been using Gerrit for two years now after switching from Rietveld when we migrated from SVN to git. Gerrit is very opinionated: it is for mandatory, pre-commit reviews and only supports git. Gerrit integrates nicely with Jenkins continuous integration server for running unit-tests before the review. We originally picked Gerrit at [undisclosed startup] and managed to integrate it into Demand after we were acquired.

For open source projects, I'm happy with github pull-request discussions (I still wish I could see side-by-side diffs! I have this huge monitor for a reason) and couldn't see enforcing the gerrit model (even though the android project does) without discouraging drive-by patches from random developers. But at work, I want the small dollop of process that gerrit provides.

I've been super happy with gerrit and can't wait to tell you more about it in "Code Review, Part II. Gerrit FTW".

Monday, November 19, 2012

Fingerworks Touchstream on Mac OSX 10.7.4

Woohoo! "New" Fingerworks Touchstream keyboard arrived today. This is my first Touchstream. It's from before Fingerworks was bought by apple and shuttered, circa 2005.

It works out of the box, but for the full experience requires running the configuration software to change the chord/multitouch bindings. Getting this running on modern hardware is challenging.

  • the company website is down.
  • The installer is for powerpc only (no longer supported by apple).
  • the application itself is a java app that requires an old version of java (no longer supported by apple).
  • the java app uses opensource jusb, which has been mostly abandoned.
The awesome people at the fingerfans message board have done a lot to keep these beloved pieces of future-tech up and running. They have a copy of the original website, the original help forums, original software, third party software, manuals, pds, and instructions for repair. I've seen posts on replacing the fpga, which is just NUTS-slash-Awesomesauce.

My steps:

  • install 1.5.3 software.
    Download this custom installer for linux and ran it on my mac: 1.5.3 software
    wget http://fingerfans.dreamhosters.com/download/setupfw153_noJava.bin
    sh setupfw153_noJava.bin
  • update jusb
    download a patched jusb from github, build and install into /Applications/FingerWorks/
    git clone https://github.com/DanThiffault/jusb.git
    cd jusb
    make
    cp -r libjusbMacOSX.jnilib* /Applications/FingerWorks/lib/jusb/
    cp jusb.jar /Applications/FingerWorks/lib/jusb/
  • install an alternative run script mtu_run.sh into /Applications/Fingerworks
    wget -O /Applications/Fingerworks/mtu_run.sh https://raw.github.com/gist/1096642/9004f21e6697fa080bb1ddde95f8a2a9d2bccae5/mtu_run.sh
    chmod a+rx /Applications/Fingerworks/mtu_run.sh
Now launch from the command line:
/Applications/Fingerworks/mtu_run.sh

Success!

The multitouch tool aka fingerworks.firmup.UtilityLauncher launched and detected my "TouchStream ST/LP ver 1.6". [RUN Diagnostics...] reported:
All sensor array tests PASSED!
Loaded 1243 Key/Gesture Mappings SUCCESFULLY
    Keymatrix#: 34
Testing Complete.

Not so fast: Can't write to device

Doh. Seems I can run the diagnostics, but I can't push a new configuration onto the device. That's a major bummer. I'll have to look into the java errors and see what can be done.
Starting transfer...
        Writing MTS_config Binary to:  /Users/andrew/Documents/MyGestures/custom4f0040stealth34.byt
        Sending configuration to Gesture Processor...
          (Sending DeleteMsg w/ minfirmver 326, minsurfver 7, keymatrixver 34
          (Sending user options)
          (Sending 16 macro definitions)
          (Sending 100 tapareas)
          (Sending 0 switches)
          (sending -1 hand)
          (sending 1 hand)
          (sending 2 hand)
          (sending 0 hand)
          (Sent 642 total events!)
...finished merging /Users/andrew/Documents/MyGestures/custom4f0040stealth34.byt
S8 Terminated with FLASH image CRC32: 0x6f1f72da
new  idDevice: 0x160, idProduct: 0x90b,  idVendor: 0xe97
USB DFU suffix appended to: /Users/andrew/Documents/MyGestures/custom4f0040stealth34.U.byt
        MTS_config Binary /Users/andrew/Documents/MyGestures/custom4f0040stealth34.U.byt ready for transfer! 
        existing  idDevice: 0x160        idProduct: 0x90b        idVendor: 0xe97
Java computed firmware image CRC32 0x6f1f72da on 32870 bytes
Exception in thread "Thread-8" java.lang.IllegalAccessError: tried to access class usb.linux.DeviceImpl from class fingerworks.firmup.USBupgrader
        at fingerworks.firmup.USBupgrader.a(Unknown Source)
        at fingerworks.firmup.USBupgrader.downloadFirmwareFile(Unknown Source)
        at fingerworks.firmup.USBupgrader.send2GestureProcessor(Unknown Source)
        at fingerworks.firmup.a.run(Unknown Source)
        at java.lang.Thread.run(Thread.java:680)
Nov 19 16:57:06 femto.corp.dm.local java[59263] : CGContextGetCTM: invalid context 0x0
Nov 19 16:57:06 femto.corp.dm.local java[59263] : CGContextSetBaseCTM: invalid context 0x0
Nov 19 16:57:06 femto.corp.dm.local java[59263] : CGContextGetCTM: invalid context 0x0
Nov 19 16:57:06 femto.corp.dm.local java[59263] : CGContextSetBaseCTM: invalid context 0x0

References

Friday, August 31, 2012

Unexpected error report from cpantesters!?

After pushing my release of App::PM::Website, I received a cpantesters failure report. Opening it revealed an error from File::Path not exporting make_path. What-the-what? Of course File::Path exports make_path, that's like all it does!
# Failed test 'no eval errors'
# at t/app-pm-website.t line 12.
# got: '"make_path" is not exported by the File::Path module

Further inspection of the report shows that this test box is using File::Path v2.04 with perl 5.10. You remember: the File::Path version released in 2007.

PREREQUISITES:
Here is a list of prerequisites you specified and versions we
managed to load:

Module Name Have Want
App::Cmd 0.318 0
App::Cmd::Command 0.318 0
App::Cmd::Tester 0.318 0
Config::YAML 1.42 0
Data::Dumper 2.121_14 0
Date::Parse 2.30 0
DateTime 0.76 0
DateTime::Format::Strptime 1.52 0
File::Path 2.04 0
File::Spec 3.33 0
...

Ok, so this was me being lazy in importing my File::Path requirement, as I clearly needed 2.07 or greater (from 2008). wow. Patched version of App::PM::Website is now released and pushed to CPAN.

I have to wonder if this is a box set to be purposefully obtuse and install the lowest version it could find that matched my requirements, just to catch this sort of thing? If so, my hat is off to you fine perler!

Wednesday, August 29, 2012

Update your Perl Monger group page with App::PM::Website

Fresh CPAN Upload: my first public release of App::PM::Website!

App::PM::Website installs pm-website, a commandline tool for maintaining a perl monger website at pm.org. pm-website handles rendering the index page and updating it via WebDAV.

Monger groups get free hosting at $group.pm.org, but only static content is allowed and it can only be edited via WebDAV. I use pm-website to maintain the Los Angeles Perl Mongers website.

WebDAV is not complicated, but it can be a pain. Once I decided to automate the WebDAV upload, it was a quick hop, skip and a jump to automating the whole process of updating the front page with information about the next meeting and updating the list of past meetings.

A single yaml configuration file maintains configuration and data for the meetings, presenters and locations. Updating for a new meeting is a simple matter of adding an entry in the meetings array with the date, location and information about the presentations and presenters. pm-website build renders the page and pm-website install pushes it via WebDAV.

Get started by creating a stub configuration file with pm-website init. Check out the la.pm.org source repository on github for more inspiration.

Technical details:

App::PM::Website uses App::Cmd and is easily extensible by writing new commands in the App::PM::Website::Command::* namespace. HTTP::DAV provides WebDAV transport and Net::Netrc handles reading credentials from a standard .netrc file. Template::Toolkit is used for rendering the index template. App::PM::Website is a Dist::Zilla based distribution.

Check out the app-pm-website source at github.

Wednesday, July 25, 2012

Oscon 2012 resources, presentations and videos

OSCON 2012 recap: "Awesome, how could it not be?"

Huge! There were upwards 18 rooms in use for different talks on Wed and Thu. While that could be an opportunity to pick poorly, it really meant seeing something awesome but at the expense of other awesome things.

Unlike this year's YAPC, not all of the talks were recorded. Most of them were not. I've spent the past few hours crawling the internet to find slide decks and etc for you, so DO NOT FRET!

One stop shopping, visit the Oscon Speaker Slides page. This lists all the presentations where the speaker provided her slides to the organizers for publishing.

The OSCON 2012 YouTube Channel has the keynotes and a bunch of interviews. And it wouldn't be a conference without Piers singing beautifully. A four part video on "From Datacenter to the Cloud" is up on Vimeo.

Trawling through #oscon on twitter, I found some more slidedecks and videos.

My OSCON

View My personal Schedule to see which talks I attended. Only a few seem to have slides uploaded on the oscon/o'reilly page.

Some of my favorites:

"How to Write Compilers and Optimizers"
Holy awesome batman! Go read the slides(pdf) now.
Shevek's goal was a "thirty minute overview of a grad school course on compilers." (aka no math) And that's pretty much what we got. Bummed I missed the beginning with his SQL parser demo. Other big take away? "Use SableCC, a beautiful Java LR parser generator."

"Moose is Perl: A Guide to the New Revolution" (Ricardo Signes)
The slides aren't on the oscon page, but can be pulled directly from manxome.org: Moose is Perl slides, specifically the pdf
I attend the first half (90 minutes) before switching for Presentation Akido, toughest talk choice of the week. And the second 90 minutes was going to be about stuff I didn't know.

"Build a better team through Improv"
Got us out of our comfort zone doing improvisational comedy to improve work interactions. Fun stuff. The slides(pdf) weren't used during the activity, but have useful notes on our exercises. Take-away: "Yes, and" >> "Yes, BUT".

Profiling memory usage of Perl applications (Tim Bunce)
He posted the slides on slideshare, but not yet linked from oscon.
A lovely wander through Devel::Peek, Devel::Size and friends and what-could-be for profiling memory usage (and why it is hard).

Perl Lightning Talks and Ignite
The ignite talks were good, the lightning talks were better. No one is going to do a presentation about "phpsadness" at an ignite talk. Gong!
Tim Bunce did an enlightening lightning talk, A Space for Thought." He was kind enough to upload a copy to pastebin and send to me via twitter.

"Wrangling Logs with Logstash and ElasticSearch"
I really wanted to attend this one, by our local L.A. MediaTemple friends Nate Jones and David Castro, but it was packed beyond standing room only and the ushers weren't letting anyone in. We may get them to present again locally, for LAPM or LAdevops?

"One-man ops; deploying at scale in EC2",
A bummer that the slides don't have presenter notes. Take-away: automate it!

The Damian Conway collection:

"Instantly better VIM" (Damian Conway)
The slides aren't listed on the oscon site, but don't worry, I hooked you up: Instantly_better_Vim.tar.gz

"Presentation Akido"
I caught the first half of Moose and the second half of Presentation Akido. They were both awesome talks. Damian didn't provide a link for slides, we had nicely bound print outs in the room. Of course, there were way more people and we ran out. So they ran off extra copies. I found a stash of them on Friday during clean-up, so have a few spares.
I've missed this talk before, so was glad to be able to schedule it in.

"The Conway Channel"

"Taming Perl Regexes"
Regexp::Debugger
demo now! live interactive ncurses display showing regex application.

He also did "New Features of the Modern Perls (5.10 to 5.16)" and held an office hour.
Lots of Damian time. I was actually at the office hour, which also featured Ward Cunningham (mr XP), Shevek (compilers, parallel and distribute systems) and Christopher Clark(of sparkfun).

Hyperpolyglot:

"Inside Python"
Interesting example of digging into the python compiler/VM, with the idea of "adding a new keyword," which can't be done at the language level. That's a rough way to do meta-programming (harhar!). I left this with a better understanding of the what happens during python compile time.
"Data Science in R"
Great slide content available from the presenter's site
"Computing with Clojure"
Fun introduction to clojure and lispness. Tag teamed with two presenters. Highly entertaining.
"Storm: distributed and fault-tolerant realtime computing" (Nathan Marz)
I've been following storm and nathan_marz on twitter&blog for a long time. It was neat to see live presentation. It mostly covered the overview of storm, but did talk about a new technique for simpler, shorter code.

The Synacor Challenge:


The challenge ate up a huge number of person-hours at OSCON. I started it at 11pm Thursday and (coupled with the important lesson "tea == caffeine" ) forgot to sleep until 7am Friday. I hit an infinite loop and bailed on it -- later I found it was merely a highly bounded loop that would recover 700,000 cycles later.

What was it? Two files: a description of a VM and a binary to be run on that VM. There are 8 codes to find along the way and report back. There was a $1000 prize for the first to solve it during the convention, it was given away to one of the three people tied with 6 of 8 codes. I've gone as far as 5. I see that @dag completed all 8 sometime yesterday.

Thursday, May 31, 2012

Dzil plugins: GitHub::Meta vs GithubMeta

"Dear lazyweb: how do I add a github repository link to my dist-zilla dist.ini?"

Curtis Poe (@ovidperl) via twitter


Ovid asked how to add git repo information to a dist.ini file. Both Dist::Zilla::Plugin::GitHub::Meta and Dist::Zilla::Plugin::GithubMeta were suggested, along with the old-school Dist::Zilla::Plugin::MetaResources.


I've used MetaResources to include repository information in my dist.ini files. Which plugin should I use now: GitHub::Meta GithubMeta?

My MetaResources usage is pretty simple: homepage + repository information. The GitHub::Meta and GithubMeta versions are nearly the same from the dist.ini side, both have the same format for manually overriding the default homepage:

#MetaResources Github example:
[MetaResources]
homepage        = http://lowlevelmanager.com/
repository.web  = http://github.com/spazm/app-pm-website
repository.url  = http://github.com/spazm/app-pm-website.git
repository.type = git

## GithubMeta version
[GithubMeta]
homepage = http://lowlevelmanager.com/

## GitHub::Meta version
[GitHub::Meta]
homepage = http://lowlevelmanager.com/

There are differences if we look inside the modules. GithubMeta makes a system call to git to get remote url(s). The plugin will only run if called from within a git repository and with git in $PATH. If it finds a github.com url, it uses that as the remote. This url is then parsed to get the github user name and project name.

Conversely, GitHub::Meta shells out to git to get the github.user entry from git config via git config github.user and then makes an API call to github to get the project name. It does have an interesting feature where it can check if the checkout is a fork and use the upstream information instead.

GitHub::Meta could be nice if you want to go all-in and auto-create the repository with GitHub::Create and push updates with GitHub::Update (actually, this just seems to change the homepage url?), all of them will require the github.user configuration set in git. The docs suggest using GitHub::Create with Git::Init. I must admit, I'm behind the DZil times, as I'm not even sure where to store the default plugin options to apply when running dzil new, to actually trigger these plugins.

GithubMeta does what you'd expect and does it with a minimum of fuss. No extra options to set in your git configuration, but you do have to have the github remote added to your repo by other means. This seems a much slimmer change for existing repositories.

Wednesday, March 28, 2012

Smile! with a Zsh prompt happy/sad face

UPDATE: Now in color!
%(?,%F{green}:%),%F{yellow}%? %F{red}:()%f

% RPROMPT='%(?,%F{green}:%),%F{yellow}%? %F{red}:()%f'
% /bin/true                                                                     :)
% /bin/false                                                                    :)
%                                                                             1 :(
Wouldn't you like a more descriptive shell prompt, that would show you the return value of the last command in a visually intuitive way? Sure, you could use %? to get the return int, zero for success and non-zero for failure. But that's just going to make your prompt more bizarre to your (pair-programming) partner.

Add this expansion sequence %(?.:%).:() to your PROMPT or RPOMPT to add a smiley/frowny face to your zsh prompt based on the return status of the previous command.

Let's see it in action! "Before" vs "After" of putting this in my RPROMPT to set the right prompt, to replace the raw %? return value I had previously.

#BEFORE
% RPROMPT='%? %~'                                                0 ~
% /bin/true                                                      0 ~
% /bin/false                                                     0 ~
%                                                                1 ~

#AFTER
% RPROMPT='%(?,:%),:() %~'                                       1 ~
% /bin/true                                                     :) ~
% /bin/false                                                    :) ~
%                                                               :( ~
Right, you're wondering how you'll live not knowing the actual failure int return code, right? Well, you don't have to give that up. Let's modify the false-text to prepend the failure int %? to the sad face:
#Add this to your PROMPT or RPROMPT: %(?,:%),%? :()

[vm53@vm53]% RPROMPT='%(?,:%),%? :() %~'                                   :) ~
[vm53@vm53]% /bin/true                                                     :) ~
[vm53@vm53]% /bin/false                                                    :) ~
[vm53@vm53]%                                                             1 :( ~

How does this work? We use the built-in conditional form, %(x.true-text.false-text), as documented in the zsh man pages under "CONDITIONAL SUBSTRINGS IN PROMPTS". Closing parenthesis must be encoded to appear in the true-text or false-text. ? checks against the return code of the previous command. Our true text makes a happy face :) and our false-text makes a sad face :(. Our updated sad face prepends the return value%? :(. I chose to prepend as the is the first item in my right prompt, this makes the smiley/frowny faces align.

Inspired by Selena Deckelmann's Use This Interview. Thanks Clark for showing me the post and then putting up the challenge to do it in zsh (and do it more cleanly).

CONDITIONAL SUBSTRINGS IN PROMPTS:

%(x.true-text.false-text)
              Specifies a ternary expression.  The character following  the  x
              is  arbitrary;  the  same character is used to separate the text
              for the `true' result from that for the  `false'  result.   This
              separator  may  not appear in the true-text, except as part of a
              %-escape sequence.  A `)' may appear in the false-text as  `%)'.
              true-text  and  false-text  may  both contain arbitrarily-nested
              escape sequences, including further ternary expressions.

              The left parenthesis may be preceded or followed by  a  positive
              integer  n,  which defaults to zero.  A negative integer will be
              multiplied by -1.  The test character x may be any of  the  fol‐
              lowing:

              !      True if the shell is running with privileges.
              #      True if the effective uid of the current process is n.
              ?      True if the exit status of the last command was n.
              _      True if at least n shell constructs were started.
              C
              /      True if the current absolute path has at least n elements
                     relative to the root directory, hence / is counted  as  0
                     elements.
              c
              .
              ~      True if the current path, with prefix replacement, has at
                     least n elements relative to the root directory, hence  /
                     is counted as 0 elements.
              D      True if the month is equal to n (January = 0).
              d      True if the day of the month is equal to n.
              g      True if the effective gid of the current process is n.
              j      True if the number of jobs is at least n.
              L      True if the SHLVL parameter is at least n.
              l      True  if  at least n characters have already been printed
                     on the current line.
              S      True if the SECONDS parameter is at least n.
              T      True if the time in hours is equal to n.
              t      True if the time in minutes is equal to n.
              v      True if the array psvar has at least n elements.
              V      True  if  element  n  of  the  array  psvar  is  set  and
                     non-empty.
              w      True if the day of the week is equal to n (Sunday = 0).

Sunday, July 17, 2011

Wedding Dance

My latest Dancer project is up at SweetieBeast.us, a website for my pending wedding. T-minus-13 days and counting!

The source is up at github. Look in the wedding-dancer directory for the dancer project.

The dancer code is very short, because I'm not really using any dancer bits. I'm using dancer to provide an interface around templating code and page layout wrapping. As such, the pages are all pure templates.

Version .1 was very close to the following code snippet. Auto_page turns my template files into corresponding routes. I then created a template for each of my pages, spent a day futzing with CSS and poof: wedding website!

package sweetiebeast;
use Dancer ':syntax';
our $VERSION = '0.1';
set auto_page => 1;
get '/' => sub { template 'index'; };
1

When I launched, I put the site behind an apache proxypass directive. So I added Dancer::Plugin::Proxy to create links relative to the external proxy. I query Dancer::Plugin::Proxy to explicitly create link targets in the model to simplify my views.

package sweetiebeast;
use Dancer ':syntax';
use Dancer::Plugin::ProxyPath;
our $VERSION = '0.2';

set auto_page => 1;
get '/' => sub { template 'index'; };

before_template sub {
    my $tokens = shift;
    $tokens->{uri_base}           = proxy->uri_for("/");
    $tokens->{uri_for_index}      = proxy->uri_for("/");
    $tokens->{uri_for_faq}        = proxy->uri_for("/faq");
    $tokens->{uri_for_location}   = proxy->uri_for("/location");
    $tokens->{uri_for_travel}     = proxy->uri_for("/travel");
    $tokens->{uri_for_guests}     = proxy->uri_for("/guests");
    $tokens->{uri_for_images}     = proxy->uri_for("/images");
};
true;

Friday, April 29, 2011

Dude, what function am I in?

The ctags plugin for vim shows the function name containing the current line. This is helpful when you're refactoring down functions that are longer than a screenful.

I've been using this (exuberant) ctags based approach for years. It's quite handy and works across more languages than I use: 41 languages from ant to YACC. I've tested with perl, python, C, java and ruby.

The plugin has options to display the function name in the status bar or in the xterm menu bar. It takes care of generating the ctags files on the fly.

I've blogged before on using vim with tags file, recommending using ctags to create a tags file for your codebase. Using the tags browser to jump to subroutines defined in other files is just ... super fun.

ctags plugin for vim:
http://www.vim.org/scripts/script.php?script_id=610
exuberant ctags:
http://ctags.sf.net/
My blogpost on effective VIM with perl and tags. http://www.lowlevelmanager.com/2010/12/perl-tags-and-vim-effective-code.html

These are the options I use with the plugin:

  let g:ctags_statusline=1
  let g:ctags_title=0
  let g:generate_tags=1 
  let g:ctags_regenerate=1

PS. OSX comes with a different version of ctags, you'll need to install exuberant ctags for the ctag plugin to work, it's available in both fink and MacPorts.

Saturday, April 16, 2011

Moving at Scale : Etsy

I know the etsy engineers like to graph, graph, graph, but I hadn't heard too much about the rest of their release cycle. They have taken specific steps to attempt to "Stay a Startup as we grow larger." Watch as they expound on the benefits of "one button deployment", empowering releases, branching-in-code, A-B testing and configurable features.

Warning, the video is a couple of hours long.

Watch live streaming video from etsy at livestream.com
Source

  1. Make deployments easier
  2. No-fault post-mortems
  3. everyone works on head, no branches in source control, lots of if(0){ ... } blocks
  4. configuration file that determines which features are live, to which percentage of the population.
  5. public logging and metrics
  6. ...
  7. Change are quick, easy and encouraged.
  8. maintains the "can do" "default is yes" attitude of the early startup.
  9. mean time to notice problems: <5 minutes, mean time to fix problems: 4.5 minutes.
  10. ...
  11. profit

They have released two projects as openSource, available on github:

logster
https://github.com/etsy/logster
Forked from ganglia-logtailer, they removed the daemon mode to simplify the code.
Logster is a utility for reading log files and generating metrics in Graphite or Ganglia. It is ideal for visualizing trends of events that are occurring in your application/system/error logs. For example, you might use logster to graph the number of occurrences of HTTP response code that appears in your web server logs.
statsd
Github: https://github.com/etsy/statsd
Blogpost: measure anything measure everything/
A network daemon for aggregating statistics (counters and timers), rolling them up, then sending them to graphite.
They make heavy use of Graphite for displaying graphs. http://graphite.wikidot.com/

Saturday, April 2, 2011

Better programming reading list

How to become a better programmer in 90 days -- a lightning talk.
http://www.foraker.com/how-to-become-a-better-programmer-in-90-days/

I stumbled upon this lighting talk by Neal Enssle. In it he recommends three books to improve our craft. I admit to not having read any of them, but I have heard of some of the authors.

The Passionate Programmer: Creating a Remarkable Career in Software Development (Pragmatic Life) Clean Code: A Handbook of Agile Software Craftsmanship Refactoring: Improving the Design of Existing Code

The Passionate Programmer, Chad Fowler
Love what you do, your attitude comes through. Highlight a daily win.
Clean Code, Robert C. Martin
A language for describing our code.
Refactoring, Martin Fowler
60 refactoring patterns.

Wednesday, March 30, 2011

Startup Toolkit.

I'd like to see a project of a "startup kit," a collection of opensource tools that are known to work well-enough together: revision control + code review + continuous integration + issue tracking + deployment&configuration.

I don't need perfect, but I would like good-enough.

I suppose this is close to what Atlassian provides, but that's a little too much corporate proprietary synergistic kool-aid for me. Maybe I'm judging too harshly after the Rubicon Atlassian experience, which was soured more by the lack of system and sysadmin resources to upgrade/install/approve the tools than by the tools themselves.

Some options I'm considering now:

  • Issue Tracking
    • Trac
    • bugzilla
    • redmine
    • github issue tracker
  • Source control
    • git, shared via github private account
  • Continuous Integration
    • hudson/jenkins
    • buildbot
    • cruise-control / cruise-control-rb
  • Code Review
    • mondrian (hosted by google appengine)
  • Configuration management and bootstrapping onto ec2
    • chef (pretty far down the chef road at this point)
    • puppet
    • cfengine
  • Monigoring (using all of these for different aspects:
    • nagios
    • ylastic
    • aws console
  • statistics and graphs
    • ganglia
  • log aggregation
    • splunk -- hadn't considered this, but was suggested by friends at GumGum, to monitor logs across a cloud of app-servers
This should be enough to get a scalable environment for developing, testing and deploying an application. After that, it's time to pick the additional infrastructure pieces needed by the application (data persistance, caching, logging, ... )

TODO(andrew): add links for the projects.

using git-flow with github

We've decided to use nvie's "Git Branching Model" for a work project using github as a "central" repository (by convention). The gitflow documentation is aimed at people running their own local repositories and using the default git-flow settings for branch names.

I want to make some branch naming changes from the original model. First I was going to swap develop and master with master and production. I've gone a step further and done away with master. I want my branches to be develop, qa, and production.

First, how to initialize this in a new repo:

% mkdir fakerepo
% cd fakerepo
% git init fakerepo
Initialized empty Git repository in /Users/andrew/src/fakerepo/.git/

% git flow init
No branches exist yet. Base branches must be created now.
Branch name for production releases: [master] production
Branch name for "next release" development: [develop] 

How to name your supporting branch prefixes?
Feature branches? [feature/] 
Release branches? [release/] qa
Hotfix branches? [hotfix/] 
Support branches? [support/] 
Version tag prefix? [] 

% git branch
* develop
  production

#git flow saves elements to the local repo configuration, .git/config
% git config -l | grep gitflow
gitflow.branch.master=production
gitflow.branch.develop=develop
gitflow.prefix.feature=feature/
gitflow.prefix.release=qa
gitflow.prefix.hotfix=hotfix/
gitflow.prefix.support=support/
gitflow.prefix.versiontag=
Now, how to push to the matching github repository, spazm/fakerepo? Normally I'd use:
 git remote add origin git@github.com:spazm/fakerepo.git
 git push -u origin master
But now I don't have a local master. Should I be using the publish features? #Add manually, rather than via git flow * publish
%  git remote add origin git@github.com:spazm/fakerepo.git

% git push --set-upstream origin develop
Counting objects: 2, done.
Writing objects: 100% (2/2), 173 bytes, done.
Total 2 (delta 0), reused 0 (delta 0)
To git@github.com:/spazm/fakerepo.git
 * [new branch]      develop -> develop
Branch develop set up to track remote branch develop from origin.

% git push --set-upstream origin production
Total 0 (delta 0), reused 0 (delta 0)
To git@github.com:/spazm/fakerepo.git
 * [new branch]      production -> production
Branch production set up to track remote branch production from origin.

% git branch
* develop
  production

% git branch -r
  origin/develop
  origin/production

Now, what does the next user need to do after he clones the remote repo?

% git clone git@github.com:spazm/fakerepo.git
% cd fakerepo
% git status
# On branch develop
nothing to commit (working directory clean)

% git branch
* develop

% git branch -r 
  origin/HEAD -> origin/develop
  origin/develop
  origin/production

% git config -l | grep gitflow

#blank, the gitflow configuration items aren't present in the checkout.
% git flow init

Which branch should be used for bringing forth production releases?
   - develop
Branch name for production releases: [] production
Local branch 'production' does not exist.

#manually bring over tracking branches
% git branch --track production origin/production
Branch production set up to track remote branch production from origin.

% git flow init
Which branch should be used for bringing forth production releases?
   - develop
   - production
Branch name for production releases: [production] 

Which branch should be used for integration of the "next release"?
   - develop
Branch name for "next release" development: [develop] 

How to name your supporting branch prefixes?
Feature branches? [feature/] 
Release branches? [release/] qa
Hotfix branches? [hotfix/] 
Support branches? [support/] 
Version tag prefix? [] 

% git config -l |grep gitflow
gitflow.branch.master=production
gitflow.branch.develop=develop
gitflow.prefix.feature=feature/
gitflow.prefix.release=qa
gitflow.prefix.hotfix=hotfix/
gitflow.prefix.support=support/
gitflow.prefix.versiontag=
Question: Do I really have to manually set up the tracking branches when I clone, prior to running git-init? That seems messy. I'll keep digging and let you know.

Saturday, March 12, 2011

patching github projects

Wow, adding changes to projects on github is surprisingly easy, once you figure out the steps. I made a patch for vim-space earlier today, after I found an error in a comment that cost me an hour yesterday.

I nearly got the github flow correct on the first pass. Here's what I did:

  1. fork the project (via UI on github)
  2. clone locally
    git clone git@github.com/spazm/vim-space.git
  3. make change to plugin.vim in my working directory, test:
    vim plugin.vim ...
  4. create an issue for the original project (via UI on github)
  5. commit change locally:
    git add plugin.vim
    git commit plugin.vim -m "[issue 7] ..."
  6. push change to my cloned repo on github:
    git push origin master
  7. create the patch file:
    git format-patch HEAD^..HEAD
  8. upload the patch file to the issue. Nope, can't upload files to the issue tracker at github.
  9. create pull request:
    git request-pull HEAD^ origin HEAD
    The following changes since commit 9640d4d1ee980e352abd96e2c0ef13372d1c14cd:
    
      Merge remote branch 'remotes/origin/master' (2010-04-13 16:22:27 +0200)
    
    are available in the git repository at:
      git@github.com:spazm/vim-space.git master
    
    Andrew Grangaard (1):
          [issue 8] correct loaded_space to space_loaded in plugin
    
     plugin/space.vim |    4 ++--
     1 files changed, 2 insertions(+), 2 deletions(-)
  10. Copy and paste this pull request into issue 7.

This was close to the correct flow. While the issue tracker on github doesn't allow uploaded files in the comments, it does track pull requests. So the flow should look like:

Correct Flow

  1. Clone Repo (github UI)
  2. clone locally:
    git clone git@github.com/spazm/space-vim
  3. Make Changes, test
    vim plugin.vim ...
  4. commit locally:
    git add plugin.vim
    git commit -m "loaded_space fix"
  5. push to github:
    git push origin master
  6. issue pull request(github UI)
  7. done. bask in the warm glow of improving an open source project.

The issue pull request stage will create a tracking issue. Slickness. The Clone and Pull Request are handled in the github UI. Check the Pull Requests as Github UI documentation for pretty pictures.

Proper form would be to create a topic branch in the cloned repository, this becomes more useful if you plan to make any further changes or modifications to the project. Make the upstream girl's job as easy as possible to enhance the likelihood of your changes being accepted.

Monday, December 13, 2010

TEDxCALTECH -- Friday, January 14, 2010

"Feynman's Vision -- The next 50 years"

TEDx, the independent TED event series is coming to Caltech in January. TEDx events are inspired by TED and use the same plans and speaking formats. I'm surprised I haven't heard more buzz about this event. I wasn't able to get into TEDxUSC, the first of the TEDx events. I'll be volunteering for the event and hope to see you there!

Will we see coverage of Feynman's vision of the "race to the bottom" -- his challenge to his engineering and scientist peers to work together and compete to see how small we can go in nanotech. Will anyone from Professor Tai's micro-machine lab be speaking? I'm sure we'll see interesting wetware - bio/cs/eng research, given the Institute's focus on Bio over the past decade.

You won't know until you go! (Or until you check the speakers tab)

Feynman's Vision -- The next 50 years.

TEDxCaltech is a community of people who are passionate about sharing "Feynman’s Vision: The Next 50 Years." If that sounds like something you want to be a part of, complete and submit the application below. Due to limited venue space, we cannot approve all applicants instantaneously. If you are approved, you will receive an email shortly either inviting you to register for the event, or letting you know that you are on the waiting list. The registration fee is $25 for Caltech students; $65 for Caltech faculty, staff, postdocs, alumni and JPL; and $85 for all others. The all-inclusive day will begin with breakfast and will be punctuated with generous breaks for food and conversation with fellow attendees. It promises to be an exciting and entertaining intellectual adventure—a time to unplug from the day-to-day routine. You won’t want to miss a minute!

Registration opens at 8:00 am, doors open at 9:30 am, talks conclude at 6:00 pm and will be followed by a reception. Parking is free.
-- http://www.tedxcaltech.com/apply

Wednesday, October 27, 2010

Announce: Net::HTTP::Factual

Factual hosts community editable data on a variety of topics, and hopes to be your one-stop-shop for wiki-data. They provide an element of "truthiness" for rows to show if the data is verified and some other interesting ideas.

They hosted a contest at Startup weekend LA last week, with the top team winning kindles for each member. I didn't win the contest[*], since I got pulled onto a different team and didn't finish my factual thingy (clearly I didn't define my factual thingie either). I did create a quick cpan module to wrap the factual ReST interface, Net::HTTP::Factual. I built it using Spore.

All I needed to do was translate the API documentation into json for spore to load. I also needed to work out the differences between Spore v0.2 spec format the v0.1 documentation. The demonstration spec files in the spore github are in the v0.1 format. These are the problems of using bleeding edge alphaware -- Spore v.01 came out Oct 12, v0.2 on Oct 14 and I was using them Oct 16.

* : I didn't win the contest, but it did remind that I wanted a kindle. So I went home and ordered one. It arrived two days ago, and I'm 80 pages into my first book. The screen is gorgeous. I'm reading a PDF of the Mythical Man-Month, and while the pdf handling isn't perfect it works well -- flows quite nicely reading from a rotated screen. Props!

your data is your program.

Much more often, strategic breakthrough will come from redoing the representation of the data or tables. This is where the heart of a program lies. Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won't usually need your flowchargs; they'll be obvious.
--The Mythical Man-Month

In theory, I read The Mythical Man-Month almost 20 years ago for CS1. At the time, his historical examples seemed so dated, talking about monthy rental of kilobytes of ram and OS360 and PDP8 and PDP11 design. What could that teach me about cranking out modern C code?

As I reread it know, I realize how timeless are his examples of software as engineering discipline. Problems we still haven't solved -- how to maintain the vision of a small number of minds over a project that requires many, many hands.

I wish I could have understood his words back then, but wisdom is not so easily gained.

Experience is a dear teacher, but fools will learn at no other.
-- Benjamin Franklin

Dost thov love life? Then do not sqvander time
[ for that is what life is made of ]
-- Benjamin Franklin, as inscribed in Blacker Hovse

Thursday, October 7, 2010

Reading is FUNdamental

I am currently reading three CS books. The differences in styles and approaches are striking, but they are all effective in teaching.

Structure and Interpretation of Computer Programs aka SICP.
Classic text computer science and functional programming using Scheme as the implementation language. Used in 6.001 at MIT.
Related links:
Programming Collective Intelligence
Interesting algorithms for mining the huge datasets available in the web era. Uses python as the implementation language. Working through the first three chapters, the first thing I did was rewrite the examples in perl. Then I found I wasn't the only one to do so.
Modern Perl
chromatic has written a lovely book on Modern Perl, and shared the review copy for comments/editing. It was pretty sweet to email in some typos and receive a link back to an updated git repository with my changes applied.