このページは大阪弁化フィルタによって翻訳生成されたんですわ。

翻訳前ページへ


Perlsphere - the Perl blog aggregator
The Wayback Machine - http://web.archive.org/web/20090917210016/http://perlsphere.net:80/

dagolden: Applying git patch emails to subversion

I prefer git, but have been spending a lot with subversion working on Module::Build. To give myself local version control when offline (I was on an airplane), I layered a local git repository on top of the subversion checkout. (Yes, there might be better ways to do this with “git svn” but not as easily once already offline!)

Once I had taught git to ignore .svn directories and taught svn to ignore .git and .gitignore, I imported the whole project to git and proceeded with my normal workflow, creating separate branches for different things I was working on.

Once I was back online, I wanted to merge my work back to the subversion tree, while still preserving the commit history from when I was offline. A short bit of Perl programming later and I was done. For each of the git branches I had created, I generated a series of patch emails with “git format-patch”. Then, after having subversion revert all files back to the latest commit, I passed the patches on the command line to this program:

#!/usr/bin/env perl
use 5.010;
use strict;
use warnings;
use Email::Simple;
use Path::Class;
use File::Temp;

die "$0: This is not an svn repo\n"
  unless -d '.svn';

for my $f (sort @ARGV) {
  say "*** $f ***";
  # try to apply patch
  system("patch -p1 < $f");
  die "Patch failed!  Aborting!\n" if $?;

  # extract commit info
  my $email = Email::Simple->new( scalar file($f)->slurp );
  my $subject = $email->header("Subject");
  $subject =~ s{^\[PATCH[^\]]*\]\s+}{};
  my $body = $email->body;
  $body =~ s{\A(.*?)---.*$}{$1}ms;

  # write commit info to tempfile and commit it
  my $temp = File::Temp->new;
  say {$temp} $subject;
  print {$temp} "\n$body" if $body;

  system("svn commit -F $temp");
  die "Commit failed!  Aborting!\n" if $?;
}

I wouldn’t want this kind of hybrid VCS to be part of my regular workflow, but in a pinch when I wanted offline version control, it worked remarkably well.

Alias's Journal: Creating GUI code for Padre - A walk through a new feature

Alias's Journal

One of the most common problems people who have only recently arrived on the Padre team have is the creation of Wx code that, well, doesn't suck ass.

The API is rather opaque, badly documented (for now) and it's behaviour isn't always entirely obvious.

So when Sewi, who's been trying to make headway on a Padre::File API (for doing remote SSH/FTP/etc file operations), demonstrated quite possibly the worst textbox entry dialog ever seen, he was more than happy for someone else to produce a better one.

Since it's a relatively straight forward job, I thought that it might be a good idea to demonstrate how to do Wx GUI code in a reasonably efficient and decent looking way.

You can follow the set of commits, and see graphical diffs of each, at the following:

http://fisheye2.atlassian.com/changelog/~author=adamk,date=2009-09-18T00%3A00%3A 00/padre

The implementation starts at commit 7970.

I start by mocking up the layout in wxGlade. The Perl code generated by this Python application is... well... questionable. While it technically works (thankfully) you'd struggle to build a scalable application using the style of code it produces.

HOWEVER, once you get past the horrible boilerplate, what it does generate is pretty much perfect code for creating the various widgets and sizers. This is the most verbose and annoying part, so having something to copy and paste liberally from makes a huge difference.

In commit 7971 I drop in a basic class skeleton for a dialog. This is just a cut and paste job from the Find dialog, and does nothing more than create the dialog itself, sucking in a role that simplifies a few things for dialogs that are direct children of the main window.

Commit 7975 sucks in the code from wxGlade for the actual form components. We do this first because the layout doesn't matter until we have the parts that actually do work in place.

Unfortunately, the code doesn't actually work, because the wxGlade code uses wxCONSTANT values directly and Padre does not. It turns out exporting 5000 constants into 300 classes results in a rather ugly memory overhead, so in Padre we use Wx::wxCONSTANT directly.

So in commit 7976 I drop in a temporary "use Wx 'everything';" so the constants work.

Commit 7978 drops in the wxGlade layout code, which should pretty much just work as is.

Some of the objects use horrible auto-generated "sizer_1" variables. But oddly, I find for some of the stuff like the border sizers it doesn't really matter that much, since there's really not much better meaning you can use instead.

And now to the clean up.

First up, wxGlade likes to make more things accessors than are really necessary. So in 7980 we convert them over to regular lexicals that will just exist in the C heirachy of the application, not the Perl heirachy.

Next, in commit 7983 I'll separate, comment, group and layout the sizer code into something that's actually maintainable.

Note that I'm doing pretty much all of this without actually running anything, although I do bother to at least run the test script which checks that the code compiles. We'll get to the running of code later.

But in anticipation of that day (and now that all the cut and pasting is done) we do a standardisation pass.

In commit 7989 we finish converting all the wxCONSTANTS to Wx::wxCONSTANTS, and add in the standard Padre copyright fragments to the Debian people don't get annoyed with us. :)

Now to the actual functionality.

Copy/paste again from the Find dialog gives us a working cancel button in commit 7989 and a working OK button in commit 7999.

Having reached this point, we pause briefly to watch azawawi blatantly steal commit 8000 and quietly loath and envy him.

Anyway, onwards again.

So now we can start to actually run the code. And of course, as soon as I tried to do that I discovered that the dialog really should run modally. So in commit 8005 we do some refactoring to make the dialog modal (and to give it a top level easy to use method for starting and stopping the dialog).

Now, finally, we start running it for real. So the final commit 8011 is debugging, plus replacing the old dialog in the main window class.

The result? A great looking "Open URL" menu entry that does absolutely nothing at all other than spit out a "this doesn't work" message box.

But then that's the main point. By making a working chunk of UI ready for Sewi to step in and take over, I've removed a major obstacle from his completing the feature that he REALLY cares about.

And hopefully this entry can serve as something of a template for others needing to create simple to medium complexity gui code.

sartak: YAPC::Asia 2009 Moose course

Moose course

I have received a lot of good feedback about my YAPC::Asia 2009 Moose course. Several people on IRC and Twitter thanked me. I got second-hand reports of students taking the course then immediately being able to write useful Moose code. That is nothing short of ideal. It makes me very happy that people not only enjoyed the course, but that it was practical.

Several people also wrote blog posts about the course's content and my presentation.

As expected, the students loved the exercises. I feel very strongly that the exercises are the best part of the course. They force you to make sure you understand what you have just learned. It's handy to have a Moose man there to help students when they get stuck. My only regret about this iteration of the course is that we had to skip the last two exercises for time constraints. To the students' credit, every single one did stay well over an hour past the scheduled end time.

I was pleasantly surprised that I received such nice remarks about my live coding. As these are not my slides, I sometimes had to dive into vim to explain a tangential point about Moose. For example, demonstrating the use of has after __PACKAGE__->meta->make_immutable (which led to a bug report and then a bug fix ― all during the course!). I very much enjoy this style of presentation, but it is difficult to come up with real examples on the fly. It feels great when it all just works.

I had a whole lot of help in making this course happen. A very big thanks to Daisuke Maki and the Japan Perl Association for establishing and marketing the course (and of course running this excellent YAPC::Asia!). Thanks to Emerson Mills for working with me to make the course happen, and for translating the exercises. Thanks to Nathaniel for interpreting my presentation. I did not make it easy for him! Thanks to Kenichi Ishigaki (charsbar++) for helping out with translation, for buying (with his own money) my throat lots of tea, and for being generally awesome. Thanks to all of the students for asking very good, deep questions. It's very satisfying to know one's audience is attentive and thoughtful. Finally, thanks to Dave Rolsky for letting me use his course material yet again. Next week he at long last gets to teach his own course for the first time in Minneapolis.

I look forward to giving this course again. I really enjoy spreading the Moose love.

Ovid's Journal: Testing Remote Client Scheduling

Ovid's Journal

Imagine a client (Java)/server (Perl) set up. The client reads data from the server and based on the data it receives, it writes data back to the server. The client is running on a remote host and we can't run it locally. If the client receives data which indicates a major amount of writes, it schedules the writes for a time when server load is lowered.

What we were thinking of is having acceptance tests which conceptually do this:

  1. Load a bunch of fixture data into server
  2. Notify client that data is available via a particular host:port
  3. Block until client has processed data
  4. Verify that correct data is available in server

This didn't look to hard, but with the "schedule later in the day" part, acceptance tests could block for hours. That sucks. What's the best way to test this when viewing the client as a black box?

dukeleto.pl: Blizkost is passing 102 tests!

Blizkost, Perl 5 on Parrot, is really starting to pick up speed. I just imported the base Perl 5 tests (the ones that live in t/base) and Blizkost passes every file except lex.t and one TODO test about implementing the -l command line argument. If you can make lex.t work, you get super-cool ninja points. It is probably the use of the package keyword in lex.t that borks the test. If you feel like porting other tests from the Perl 5 Test Suite, they live in t/spec in the Blizkost repo.

Currently, Blizkost cannot load XS modules, such as Data::Dumper. You get an error like:

Can't load '/usr/lib/perl/5.10/auto/Data/Dumper/Dumper.so' for module Data::Dumper: /usr/lib/perl/5.10/auto/Data/Dumper/Dumper.so: undefined symbol: Perl_sv_cmp at /usr/lib/perl/5.10/XSLoader.pm line 64.
 at /usr/lib/perl/5.10/Data/Dumper.pm line 36


If you can help get enough XS-loading machinery to work so that Data::Dumper works, that would really help development and testing. If you can try out Blizkost on your system and make sure that it compiles and the test suite passes, we would greatly appreciate it. Feel free to open an issue on github if you find a bug.

Just another lambdabananacamel: The moose counts beans - managing my finances with Perl

I've recently been trying to improve my rather disorganized personal finances. While I previously "managed" by leaving the bulk of the money in my account and hoping that I'd calculated my expenses versus salary roughly correctly, this is suboptimal for answering various questions like "Do I have enough money to go on holiday?" or "Have I forgotten to pay the electricity bill for the last year so that I now have to pay it back?"

Clearly there is a better way. Checking off expected payments, knowing what's due in the next week/month/etc., and siphoning off the excess into a linked savings account for example. But this requires being alert and accurate... Failing that, I could always use a computer ;-)

Spreadsheets

Surprisingly perhaps, the humble spreadsheet is an instrument of frustration for this kind of task. I'm currently using Google Docs, which is underpowered, but not really much more hateful than Excel or OpenOffice.org.

It's particularly bad at being a working document. Moving an item (to reschedule the date, for example) involves:

  • inserting rows
  • cutting and pasting
  • deleting the original rows
  • changing the date
  • "filling" down the subtotal column (because spreadsheets don't have the concept of a calculated column).
Worse yet, because spreadsheets don't know about dates, adding regular payments (bills, weekly spends etc.) is an exercise in manually creating them, and then visually auditing the sheet to make sure you put them in the right places.

Other apps

Of course dedicated apps for personal finances do exist. I looked at the Linux ones, Gnucash, Kash, etc. Mostly they crashed. Sometimes they ran but utterly confused me. And then crashed.

There are also web based ones. My initial survey of these suggests that they are prettier and easier to use, but perhaps clunkier, or tied into specific bank systems.

Beans means...?

So, being a programmer, the obvious answer is to attempt to roll my own. As a learning exercise ;-) Though I'll attempt various sketches of this in Haskell, I'll start off by trying to implement it in Modern Perl: specifically the Moose OO framework, using various niceties such as the pretty declarative syntax of MooseX::Declare.

Every project needs a good name! But this is just a learning exercise, so for now I'll call it "Beans". I'm hoping this will end up an ongoing series of posts, please feel free to follow along at the Beans git repo if you like, or to suggest improvements, for example:

  • a better project name
  • shinier ways to do this in Moose and Modern Perl
  • how this would be more elegant in Haskell/Lisp/etc...
As always, these posts will be fairly rough drafts, I'm hoping to tidy this up into a more finished article soon, so any comments are really welcome to help me polish it up!

Declarative code

In this installment, I'll just define a data type for a line item in Beans. You can see the current code as of this evening, but let's look at it in detail here. We'll start off by using Moose, and declaring our class:
use MooseX::Declare;

class Beans::Item {
    ...
}

Now we'll want to declare some fields for our class. For example, every payment made or expected will have a value:
    has value     => (
                       isa      => Num,
                       is       => 'rw',
                       required => 1,
                     );

What's that? Perl has types?! Yes indeed, Moose gives us types, and automatically validates against them. So when we try to create our object, we'd find that:
    my $obj = Beans::Item->new( );                # fails, value is required!
    my $obj = Beans::Item->new( value => "foo" ); # fails, "foo" is a string
    my $obj = Beans::Item->new( value => 10.00 ); # OK! 10.00 is a number

I've cheekily missed out a few details though... Num isn't a Perl builtin keyword, so we'll need to import it. In fact, while we're at it, let's pull in Str too, so that we can add a couple of string fields at the same time:
    use MooseX::Types::Moose     qw/ Num Str /;

    has name      => ( isa      => Str, 
                       is       => 'rw', 
                       required => 1,
                     );

    has comment   => ( isa => Str,         
                       is  => 'rw', 
                     );

Now we also want to be able to add the date of the expected payment:
    use MooseX::Types::DateTime  qw/ DateTime /;

    has due_date  => ( isa      => DateTime, 
                       is       => 'rw', 
                       required => 1, 
                     );

This means we could do
    my $obj = Beans::Item->new(
        value    => 10.00,
        due_date => DateTime->new(
            year  => 2010,
            month => 6,
            day   => 30,
           ),
      );

        # but I'd prefer to be able to do
        due_date => '2010-06-30',

And we can indeed make this prettier using coercions!
    use DateTime::Format::Natural;
    my $dt = DateTime::Format::Natural->new( prefer_future => 1 );
    coerce 'DateTime'
        => from 'Str'
            => via { $dt->parse_datetime($_[0]) };

    # We'll need to tweak the due_date declaration too...
    # (coercions don't run unless you ask for them)

    has due_date  => ( isa      => DateTime, 
                       is       => 'rw', 
                       required => 1, 
                       coerce   => 1,
                     );

While every line item will have an expected due_date, we'll also want to keep track of what date the item was actually paid, if it has been paid. As this is uncertain, we use the Haskell-inspired Maybe type. (As far as I can see, this isn't actually as powerful as Haskell's Maybe monad, it's more like a nullable type).
    use MooseX::Types::Moose     qw/ Num Str Maybe /;

    has paid_date => ( isa => Maybe[DateTime], 
                       is  => 'rw',
                     );

Once we've got this type, we can calculate a boolean field based on it. The paid field will be true if paid_date contains anything, and false otherwise. Let's try defining it like this:
    use MooseX::Types::Moose     qw/ Num Str Maybe Bool/;
    has paid      => ( isa => Bool,        
                       is  => 'rw', 
                       lazy => 1,
                       default => sub { 
                            my $self = shift; 
                            defined $self->paid_date 
                          },
                     );

The default code block gets called the first time we ask for the field. There are a couple of problems with this of course - once it's set, if we change the paid_date, this field won't change. We'll come back to this later, but here are some ideas:
  • change it into a method
  • use immutable objects, so that setting this lazily on first request is always the right thing to do
  • use triggers to update paid when paid_date is set (and vice versa! setting the boolean flag could set the payment date to today's date, for example).
Now, though we specified that the name should be a string, nothing is preventing us from passing the empty string ""... but Moose allows us to define a better type constraint here too!
    use MooseX::Types -declare => [qw/ NonEmptyStr / ];
    subtype NonEmptyStr
        => as Str
            => where { length $_ };

    # so now we can modify this definition to:

    has name      => ( isa      => NonEmptyStr, 
                       is       => 'rw', 
                       required => 1,
                     );

> Finally, let's add a list of tags. Just for now, we'll define these as an array of non-empty strings:
    use MooseX::Types::Moose     qw/ Num Str Bool Maybe ArrayRef /;

    has tags      => ( isa      => ArrayRef[NonEmptyStr], 
                       is       => 'rw', 
                       default  => sub { [] },
                     );

Notice how we're using default again to set the default value to an empty array.
That's it for now! Next time around we'll maybe look at parsing and displaying line items, and start to do useful things with them.

Modern Perl Books, a Modern Perl Blog: Tiny, Void Context Core Optimizations

I'm writing about arrays in the Modern Perl book now. While writing about push and unshift yesterday, I looked in perlfunc to see if I'd missed anything subtle about push -- and I had:

Returns the number of elements in the array following the completed push.

I can't think of a time when I'd used this in the past decade. Every use of push I can think of is in void context:

push @some_array, qw( some list of items );

Curiosity convinced me to look at the bleadperl source code. The push op is in a file called pp.c in a function called pp_push:

PP(pp_push)
{
    dVAR; dSP; dMARK; dORIGMARK; dTARGET;
    register AV * const ary = MUTABLE_AV(*++MARK);
    const MAGIC * const mg  = SvTIED_mg((const SV *)ary, PERL_MAGIC_tied);

    if (mg) {
        *MARK-- = SvTIED_obj(MUTABLE_SV(ary), mg);
        PUSHMARK(MARK);
        PUTBACK;
        ENTER;
        call_method("PUSH",G_SCALAR|G_DISCARD);
        LEAVE;
        SPAGAIN;
        SP = ORIGMARK;
        if (GIMME_V != G_VOID) {
            PUSHi( AvFILL(ary) + 1 );
        }
    }
    else {
        PL_delaymagic = DM_DELAY;
        for (++MARK; MARK <= SP; MARK++) {
            SV * const sv = newSV(0);
            if (*MARK)
                sv_setsv(sv, *MARK);
            av_store(ary, AvFILLp(ary)+1, sv);
        }
        if (PL_delaymagic & DM_ARRAY)
            mg_set(MUTABLE_SV(ary));

        PL_delaymagic = 0;
        SP = ORIGMARK;
        PUSHi( AvFILL(ary) + 1 );
    }
    RETURN;
}

I know this is a big chunk of lots of macros, but it's not too difficult to understand. The first if branch handles the case where the array on which to push has magic -- if it's a tied array, for example. Ignore that. The second branch loops through every list item provided to the op and appends them to the array.

I've emboldened a line at the end of that branch. The PUSHi macro pushes an integer value (an IV, in core parlance) onto the stack. The AvFILL macro returns the index of the final element in the array. Adding one to that number gives the number of elements in the array.

Every execution of this branch retrieves that value and pushes it on the stack. Even if the opcode takes place in void context such that the compiler can determine that at compilation time, this push occurs.

I wrote a patch:


diff --git a/pp.c b/pp.c
index 9cedc3f..fbdc90c 100644
--- a/pp.c
+++ b/pp.c
@@ -4561,7 +4561,9 @@ PP(pp_push)
 
     PL_delaymagic = 0;
     SP = ORIGMARK;
-    PUSHi( AvFILLp(ary) + 1 );
+    if (GIMME_V != G_VOID) {
+        PUSHi( AvFILL(ary) + 1 );
+    }
     }
     RETURN;
 }

I've emboldened the important condition. The GIMME_V macro evaluates to the current context of the expression. Usually this context is statically determinable, but if this push is the final expression in a subroutine, the calling context matters. The G_VOID macro represents void context. In other words, don't push anything onto the stack to return a value from this expression unless something wants that return value.

Yitzchak Scott-Thoennes commented on my patch to say that GIMME_V may be more expensive than I intended. This is because looking up through calling scopes to find the runtime context is not always cheap. He suggested the simplification of:

OP_GIMME(PL_op, 0) != G_VOID

... to check only the compile-time context of the operator. You can see that this cheaper check is still correct in ambiguous cases:

$ perl -MO=Concise,check_push_context
sub check_push_context
{
    push @_, 'static void context';
    push @_, 'dynamic context';
}^D
d  <1> leavesub[1 ref] K/REFC,1 ->(end)
-     <@> lineseq KP ->d
1        <> nextstate(main 61 push_ctx.pl:6) v:%,*,&,$ ->2
6        <@> push[t3] vK/2 ->7
2           <0> pushmark s ->3
4           <1> rv2av[t2] lKRM/3 ->5
3              <#> gv[*_] s ->4
5           <$&gt const[PV "static void context"] s ->6
7        <> nextstate(main 61 push_ctx.pl:7) v:%,*,&,$ ->8
c        <@> push[t6] sK/2 ->d
8           <0> pushmark s ->9
a           <1> rv2av[t5] lKRM/3 ->b
9              <#> gv[*_] s ->a
b           <$> const[PV "dynamic context"] s ->c

I've emboldened the lines representing the push opcodes and I've emphasized the relevant context flags for these ops. The first opcode has a flag of v, which indicates that it occurs in void context. The second opcode has a flag of s, which indicates scalar context. Thus Yitzchak's suggestion will work for both cases without ruining any dynamic context call of this function.

As is the case with such optimizations, the question is whether the cost of checking such an optimization possibility is worth the cost of doing the work anyway. Measuring that, however -- well, you're not getting huge speed improvements out of this code. For a one-line patch and a very common use of this op, it may be worthwhile.

transfixed but not dead!: Easy Anonymous Objects

transfixed but not dead! ? perl

On use Perl; the other day revdiablo asked whether there were Anonymous objects as easily as hashrefs? any where within Moose or sitting (hiding!) on CPAN.

Well the answers posted gave a yes for both Moose & CPAN.

Here is my twist on things using something already on CPAN which uses Moose and its Class::MOP foundations: MooseX::SingletonMethod.

With this we can easily create anonymous singleton objects like so:


{
    package Anonyject;
    use MooseX::SingletonMethod;
}

my @dingfirsts;
for my $line ( 1..10 ) {
    push @dingfirsts, Anonyject->new->add_singleton_methods(
        ding => sub { 'ding' x $line },
        dong => sub { 'dong' x $line },
        line => sub { $line },
    )
}

for (@dingfirsts) {
    say "First ", $_->ding, " on line ", $_->line;
}

We can show that each of these objects created have unique anonymous classes using Moose introspection:


for (@dingfirsts) {
    say $_->line, ': ', ( $_->meta->class_precedence_list )[0];
}

# outputs
# => 1: Class::MOP::Class::__ANON__::SERIAL::2
# => 2: Class::MOP::Class::__ANON__::SERIAL::3
# => 3: Class::MOP::Class::__ANON__::SERIAL::4
# => 4: Class::MOP::Class::__ANON__::SERIAL::5
# => 5: Class::MOP::Class::__ANON__::SERIAL::6
# => 6: Class::MOP::Class::__ANON__::SERIAL::7
# => 7: Class::MOP::Class::__ANON__::SERIAL::8
# => 8: Class::MOP::Class::__ANON__::SERIAL::9
# => 9: Class::MOP::Class::__ANON__::SERIAL::10
# => 10: Class::MOP::Class::__ANON__::SERIAL::11

The rest of the class chain upwards for each of these objects is:
Anonyject >> Class::MOP::Class::__ANON__::SERIAL::1 >> Moose::Object

So you can add methods or attributes to Anonyject class and they will be available to all the anonymous objects created.

NB. Class::MOP::Class::__ANON__::SERIAL::1 is where the MooseX::Singleton methods live.

So for more proof in the singleton pudding:


my $obj1 = Anonyject->new->add_singleton_methods(
    ding => sub { "wing" },
    dong => sub { "wong" },
    line => sub { 23_939 },
);

my $obj2 = Anonyject->new->add_singleton_methods(
    something => sub { "else" },
);

say $obj1->ding;      # Says "wing"
say $obj2->something; # Says "else"

say $obj1->something; # Should break and now does
say $obj1->broken;    # Breaks properly
say $obj2->dong;      # And also breaks properly

And all these objects are easily extensible:


# add dingdong just to $obj2
$obj2->meta->add_method( dingdong => sub { 'dingdong!' } );
say $obj2->dingdong;   # "dingdong!"

# add bang to all anonymous Anonyject objects
Anonyject->meta->add_method(
    bang => sub { 'bang ' . ( $_[0]->meta->class_precedence_list )[0] }
);
say $obj1->bang;       # "bang Class::MOP::Class::__ANON__::SERIAL::12"
say $obj2->bang;       # "bang Class::MOP::Class::__ANON__::SERIAL::13"

/I3az/

Blog of G?bor Szab?: CPAN client for the beginners

Every time I go to run CPAN for some reason there's more to configure. Woe. and it asks you all kinds of questions. Yeah, some of which I don't know what the best answer is to. I just get choice overload.

The above is a quote from foxfirefey who seems to be a nice person as she helped me a lot with here comments that lead to my previous posts Improving the Moose experience and Improving the Padre experience. She just isn't a power user of Perl.

Actually I get comments with similar content but in a much ruder way from other people, some of them with many years of experience.

Strawberry Perl for Windows has a CPAN client that does not need any configuration. I hope someone with the right powers and tuits will make it sure that the cpan clients that come with the Linux distributions are also going to need 0 configuration.

The Cattle Grid: Italian Perl Workshop 2009 approaching

The 5th edition of the Italian Perl Workshop is approaching: on October 22 and 23 we'll be all in Pisa. Some of the latest news include: 108 registered people from 15 countries (most from Italy, of course) Call for papers will close on Sunday, September 20; the talk schedule will be available shortly after Dave Rolsky class was canceled as, for personal reasons, Dave won't be able to make it to Italy this October. Mike Whitaker (Penfold), another Moose developer, is going to speak at the workshop Other confirmed guests: Tim Bunce, Jonathan Worthington, Thomas Fuchs, Amy Hoy We wait for you, no matter whether you're a Perl developer or you're just interested in the language. As you can see from the guests list, it's not only Perl but also some JavaScript and some Ruby. La 5a edizione dell' Italian Perl Workshop si sta avvicinando: il 22 e 23 Ottobre saremo tutti a Pisa. Ecco alcune delle ultime news: 108 utenti registrati da 15 nazioni (la maggior parte dall'Italia, chiaramente) Il Call for Paper chiuder? Domenica 20 Settembre; il programma degli interventi sar? disponibile poco dopo Il corso di Dave Rolsky ? stato cancellato poich? Dave, per ragioni personali, non riuscir? ad essere in Italia in Ottobre. Mike Whitaker (Penfold), un altro sviluppatore di Moose, interverr? al workshop. Altri ospiti confermati: Tim Bunce, Jonathan Worthington, Thomas Fuchs, Amy Hoy Ti aspettiamo, sia che tu programmi in Perl sia che tu sia solo interessato al linguaggio. Come puoi notare dalla list degli ospiti, non c'? solo Perl ma anche un po' di JavaScript ed un po' di Ruby....

dagolden: Module::Build inc bundling is getting close

Wow. Is it time for another Ironman post already? The last week has just flown by.

Last time, I mentioned the roadmap for Module::Build and the work being done to support bundling Module::Build into an inc/ directory in CPAN distributions for maximum back-compatibility.

Thanks to some rapid design brainstorming with Eric Wilhelm, we can demonstrate a working prototype. In practice, it means that a Build.PL might look like this:

use inc::latest 'Module::Build';
use inc::latest 'Devel::AssertOS::Unix';

Module::Build->new(
  module_name => 'Local::Blah',
  license => 'perl',
)->create_build_script;

This example would also bundle Devel-CheckOS (which contains Devel::Assert::Unix) along with Module::Build. And both are automatically added to configure_requires in the META.yml, so that any perl with an updated CPAN or CPANPLUS will have these prerequisites satisfied automatically and the bundled versions will be ignored.

Download this sample Local-Blah distribution to see how it works.

There are some non-trivial kinks to iron out and some refactoring and cleanup, but the prototype proves the concept, so I think we’ll have inc bundling available at least in a development release very soon.

acme's bits: Perl/iX for HP e3000 MPE

I'm trying to update various URLs in the Perl source code. Regarding the Perl/iX for HP e3000 MPE README.

It's very out of date now and most of the URLs are broken. I've contacted Mark Bixby, but he no longer has any involvement with the MPE/iX OS or that particular Perl port.

Does anyone run Perl on this platform? Does anyone build Perl on this platform? If so, please contact me and we'll try and update the details. Cheers!


Read and post comments | Send to a friend

nothingmuch's perl blog: Hackathon Summary

During the YAPC::Asia::2009 hackathon I refactored a bunch of XS modules out of some other code, both rafl's and mine.

XS::Object::Magic

This module provides an alternative to the standard T_PTROBJ approach to creating C struct based objects in Perl.

The traditional way creates a blessed scalar reference, which contains an integer value that is cast to a pointer of the right type.

This is problematic for two reasons:

  • If the pointer value is modified (accidentally or maliciously) then this could easily corrupt memory or cause segfaults.
  • Scalar references can't be extended with more data without using inside out objects.

This module provides a C api which uses sv_magicext to create safer and more extensible objects associated with some pointer, that interoperates nicely with Moose amongst other things.

Magical::Hooker::Decorate

This module lets you decorate an arbitrary SV with any other arbitrary SV using the magic hook API.

This is similar to using a fieldhash to create inside out decorations, but is designed to be used primarily from XS code, and as such it provides a C api. The memory footprint is also slightly smaller since there is no auxillary storage.

B::Hooks::XSUB::CallAsOp

This module was refactored out of Continuation::Delimited and lets an XS routine trigger code that will be invoked in the context of its caller.

This allows a little more freedom in the stack manipulations you can do (which is of course very important for continuations).

These modules are all possible due to the awesome ExtUtils::Depends module. If you find yourself cargo culting XS consider refactoring it into a reusable module using ExtUtils::Depends instead.

Blog of G?bor Szab?: Padre Standalone for Windows 0.45 released

While we have already seen the release of 0.46 to CPAN Curtis Jewell has released the first version of Padre Standalone for Windows.

It contains Padre 0.45 and is built on a recent version of Strawberry Perl for Windows Once installed you can easily upgrade Padre to 0.46 directly from CPAN. (See the Howto page)

The objective of Padre Standalone for Windows is to provide an easily installable and easily upgradeable editor for people who already have perl installed or who are using perl on a remote machine (opening and saving files via ftp or ssh) or for people who use it for editing other languages.

While it will contain a full Strawberry Perl installation, it will happily live together with any other Perl installation (e.g. any version of ActivePerl ) and the users won't even need to know it is running on Perl. Actually it will probably require a separate perl installation if you really want to run perl code.

We are still a couple of months away from having all the features but we have started to build a wiki page outlining the plans and the feature we would like to include.

If you would like to help, you can look around our bug tracking system and/or you can get involved in other ways.

Ovid's Journal: My Least Favorite Word: "Should"

Ovid's Journal

In the class of verbs known as modal auxiliaries, I don't have many axes to grind. "Might" is OK. I can deal with "Could". "Must" is often annoying, but a sad reality. However, the word "should" is the one which sends shivers down my spine. Its use is often a code smell embedded in the foundation of how we communicate, leading people down the primrose path to a faux nirvana.

Rather than just rant without context, I should give you some examples. Here's a great one.

Your test suite should run faster.

Please note that I have never assaulted anyone for saying that. However, I think another quote is in order.

But I say unto you, That whosoever looketh on a programmer TO commit grievous bodily harm on him hath committed assault on him already in his heart.

Ovid 5:28

OK, I'm guilty of assault.

Yeah buddy, I know my test suite should run faster. I know that having a test suite taking two hours to run isn't optimal and breaks the tweak/test/tweak/test cycle. I also know that I can sometimes use tools like Devel::CoverX::Covered to narrow down what I really need to test for a particular change, but when those tests still take 10 minutes to run, that's still killing the tweak/test cycle. The reality is, sometimes test suites don't run in 15 seconds. Deal with it.

Here's another quote, this one another annoyance.

The programmer should just read the documentation.

And another quote to illuminate that.

Do not think that I came to update the documentation or the comments. I did not come to update but to create. For assuredly, I say to you, till heaven and earth pass away, one jot or one tittle will by no means be changed in the documentation till all is fulfilled.

Ovid 5:17-18

At the end of the day, relying on documentation is simultaneously mandatory and a fool's game. CPAN documentation is very hit-or-miss. It's often underdocumented, poorly documented, or incorrectly documented. DarkPAN code, on the other hand, tends to be undocumented. So I have to read the source, but the source is often a confusing mess which reflects a poor understanding of requirements or years of maintenance and subtle bug tweaks that are critical, but poorly understood.

Here's another fun quote.

Programmers should choose the appropriate language for the task at hand.

And the requisite illuminating quote.

And Larry answering said unto them, Render to Management the things that are Management's, and to Programmers the things that are Programmer's. And they marvelled at him.

Ovid 12:17

When you know that Mercury is the perfect programming language to solve that thorny scheduling problem, you might get very annoyed when management says you must write it in PHP. But let's think about this for just a moment. If management says "everything is a nail and PHP is our hammer", you have every right to be frustrated at this short-sightedness. However, consider the other extreme. What if you have 30 programmers using 30 different programming languages, 5 different source control systems, 6 different operating systems, 4 different database management systems, 7 different packaging systems, etc. See the problem? No? Ask a system administrator who has to deal with this. TIMTOWTDI is brilliant, but often doesn't scale. If management tends to go to the other extreme, it's often the programmers who are to blame, failing to demonstrate the virtues of flexibility.

Many times when I hear programmers talk about "ideals" (i.e., using the word "should"), their heaven and my reality are often at odds with one another. Sometimes I'm talking with very brilliant people who nonetheless seem to have very little experience working on large-scale code bases in business environments. My jaw drops when rebuttals to things I say are based on ideal worlds where everyone pair programs, test coverage approaches 100%, refactoring is commonplace and documentation is always perfect and up-to-date. Even if this was true for any given system, when it's approaching 600+ classes, it's not reasonable to expect programmers to know the ins-and-outs of every part of the system.

More importantly, it's not reasonable to expect that business needs, which are often a constant juggling of competing requirements and synchronization across multiple teams, should be subordinate to the programmer's desire to get to programming heaven. There must be balance. Throw away your programming Bible and start thinking about the Yin and Yang of business and technical requirements. If you can't balance them, you should be ashamed of yourself.

Next time you hear a programmer using the word "should" in a debate/argument, listen carefully to what they're saying. Are they seeking an unattainable Nirvana? Call them on it. "Should" is a dangerous word.

dukeleto.pl: Parrot Hacktivity Report


  • Applied many patches from flh++ and darbelo++
  • Translated lots of tests written in Perl to PIR. This makes the test suite faster! It also makes the code easier to debug. It has also uncovered lots of bugs. Fun!
  • Implemented throws_like() in test_more.pir, which allows for easily testing if a bit of PIR code throws an exception that matches a PGE pattern. This is inspired from warnings_like() in Test::Warn and uses the like() from test_more.pir under the hood, so it was reasonably simple to implement. Added lots of tests that use throws_like() to verify that proper errors are thrown.
  • Fixed a bug in FixedPMCArray where it would not check for negative lengths and core dump Parrot. Oops!
  • Wrote an interactive Parrot Shell for REPL/rapid prototyping. Type perl tools/dev/parrot_shell.pl in the Parrot source code to start it up. More about this soon!
  • Wrote many tests for the proper handling of Inf/NaN, which exposed some bugs in fdiv, cmod and mod when using PMCs .
  • Updated editor documentation regarding generating tags
  • Updated documentation about math functions in Ch04 of The PIR Book
  • I will be the release manager of Parrot 1.7, which is getting released October 20th. Exciting!

PJF's Journal: Facebook Privacy talk at BarCampMelbourne now on-line

Facebook Privacy talk at BarCampMelbourne
This weekend at BarCampMelbourne I gave a talk on Facebook privacy, and what information I was able to extract from the API using some reasonable simple Perl programs. Due to the incredibly fast efforts of Avi Miller, this talk is now available on-line. If you're reading this blog on my main blog, then you can also watch it below:

You can also watch the talk on the BarCampMelbourne channel on blip.tv.

As mentioned at the end of my talk, you can be kept up-to-date on my research by joining my facebook study privacy group, or the google group, as well as my blog.

Posted: 15th September 2009.

Tags:

Bookmark:

Digg this Digg this

Modern Perl Books, a Modern Perl Blog: Befriend a Novice

Around half of all attendees to a YAPC have never attended a YAPC before.

As of today, 7680 people have uploaded a distribution to the CPAN.

Several hundred people -- over a thousand -- have credits in the source code to Perl 5.

In any given yearly period, a few dozen people have contributed more than a single patch to Perl 5, Rakudo, or Parrot.

PerlMonks has a couple of hundred very active users.

It takes months of effort to get a dozen potential students for each year's Google Summer of Code for Perl.

A new edition of the book Programming Perl may sell hundreds of thousands of copies. A new edition of Learning Perl may sell tens of thousands of copies.

A controversial or active Perl blog most may get a dozen comments.

There may be a million people in the world who've wrote more than one line of Perl code for some purpose. Go find a novice, encourage him or her to continue learning Perl, and -- most importantly -- introduce him or her to the community. You don't have to file a bug on rt.perl.org or rt.cpan.org or fix the XS documentation in the Perl 5 core or help bootstrap the Parrot JIT or organize a conference or volunteer to mentor a TPF grant. Yet you'll get much, much more out of Perl if you add your little bit to the wider Perl community.

If we are to improve our community to welcome more participants (and we should do that), we should also actively recruit new participants. Don't wait for them to come to us. Let's go find them and invite them and make them feel welcome that way, too.

Damien Learns Perl: Installing Padre on Windows Vista

Last week, I (re-)installed Padre on my Windows Vista machine. Padre is a development environment geared towards Perl programmers, with a focus on Perl beginners.
I use the Strawberry Perl distribution (5.10.0.6) and read that it was a breeze to install Padre.
Well, it almost is.
I opened the cpan shell (typing cpan in a DOS shell is all it takes if C:\strawberry\perl\bin is in your PATH).
The regular install (install Padre) failed because of a dependency on Wx::Perl::ProcessStream not building. Just force install that module (force install Wx::Perl::ProcessStream) then install Padre (a ticket has already been written for the issue).
It works like a charm, no need to force install Padre anymore!
So if you are a Perl beginner and want to use an editor written in Perl + suited for Perl tasks, you have no excuse anymore.

If you already have Padre and want to update it to version 0.46 released today (September 14th 2009), type upgrade Padre in your cpan shell. Better yet, you can update directly from the Padre Plugins menu: Plugins>Module Tools>Install CPAN Module and type Padre.


transfixed but not dead!: Building a STS (Simple Template System)

transfixed but not dead! ? perl

Marek Foss recently posted a Simple Template System in Perl article on his f055: lazy design & development blog.

This got my brain buzzing and had a few hours to kill. So I decided to stretch my Perl foo & memory to build a STS from scratch to pass the time.

Here was the first prototype:


#!/usr/local/bin/perl -T

use strict;
use warnings;
use CGI ();

sub transform {
    my ($template, $vars) = @_;
    eval {
        no warnings;
        $template =~ s/<!--(.*?)-->/$vars->{$1}/g;
    };
    return $template;
}

sub internal_template {
    my $tmpl;
    $tmpl.= $_ for <DATA>;
    return $tmpl;
}

print CGI::header( -type => 'text/html' );

print transform internal_template() => {
    sidebar => 'This is a menu',
    content => 'This is the main content block',
};

__DATA__
<html>
<head>
<title>Simple Template System</title>
</head>
<body>
<div class=“menu”>
<!--sidebar-->
</div>
<div class=“main”>
<!--content-->
</div>
</body>
<!--baz-->
</head>

I’m using a hashref to pass the template variables. This is more flexible and allows me to use just the one substitution to transform all the template placeholders:

$template =~ s/<!--(.*?)-->/$vars->{$1}/g;

The eval & “no warnings” around this cover issues like calling an unknown template variable (ie. undefined hash key). For eg. see <!?baz?> silently disappears from output.

I am also using an internal_template() where the template HTML follows the __DATA__ line. Nice for simple CGI scripts but nothing stopping us having external_template() either.

If you require more than one internal template then:


#!/usr/local/bin/perl -T

use strict;
use warnings;
use CGI ();

sub transform {
    my ($template, $vars) = @_;
    eval {
        no warnings;
        $template =~ s/<!--(.*?)-->/$vars->{$1}/g;
    };
    return $template;
}

sub internal_template {
    my $show = shift;
    my %tmpl;
    my $template_name = 'main';  # default (1st) template

    for my $line ( <DATA> ) {
        if ( $line =~ m/^\*\*\*\s(\S+)\s\*/ ) {
            $template_name = $1;
            next;
        }
        $tmpl{ $template_name }.= $line;
    }
    return exists $tmpl{ $show } ? $tmpl{ $show } : $tmpl{ main };
}

print CGI::header( -type => 'text/html' );

print transform internal_template() => {
    sidebar => 'This is a menu',
    content => 'This is the main content block',
};

__DATA__
<html>
<head>
<title>Simple Template System</title>
</head>
<body>
<div class=“menu”>
<!--sidebar-->
</div>
<div class=“main”>
<!--content-->
</div>
</body>
</head>
*** nomenu *************************************************************
<html>
<head>
<title>Simple Template System (nomenu)</title>
</head>
<body>
<div class=“main”>
<!--content-->
</div>
</body>
</head>

Now there is a separator between each template (*** template_name ***). internal_template() now takes an optional template name (internal_template(’nomenu’) would produce the alternate template in above code).

So what would a STS need next?

Well having functions passed to the template that can be called could be handy. This can be achieved by preceding an additional text substitution in the transform() sub:

$template =~ s//@{[ $vars->{$1}->( split ',', $2) ]}/g;

This code picks up placeholders with parenthesises and passes the parameters given to the specified function name. The @{[ ... ]} does the extra bit of interpolation work needed for this to work.

Thus we can create function placeholders like this:


print transform internal_template('main') => {
    gbp       => sub { sprintf "£%.2f", $_[0] },
    li_helper => sub { map { qq{<li>$_</li>} } @_ },
};

using a template like this:
<p>Great British Pounds: <!--gbp(200)--></p>
<ul><!--li_helper(one,two,three)--></ul>

would produce the following:


<p>Great British Pounds: £200.00</p>
<ul><li>one</li> <li>two</li> <li>three</li></ul>

OK so far so good. But what if I wanted to call a Perl function without having to pre-define anything in script? Well we could precede the following substitution into transform() sub:

$template =~ s/<!--perl(.*?)-->/@{[ eval "$1" ]}/gs;

Which means you then could do something like this:
<p>Random number between 1 and 9: <!--perl int( rand(10) ) --></p>

Well this is all nice and dandy if all we want is the expression results returned. What if we wanted to use control statements that effect the output? ie. something like this in the template:


if ( int(rand(2)) ) {
    print '<ol>';
    for my $numb (1..5) {
        printf('<li>%d * 2 = %d</li>', $numb, $numb * 2);
    }
    print "</ol>\n";
}
else { print "</p>Nothing to show at this moment in time</p>" }

We’re starting to play with fire now! This can be achieved with a bit more control of the output stream so I created a process() subroutine to handle that.

Below is the full program with everything in bar the kitchen sink:


#!/usr/local/bin/perl -T

use strict;
use warnings;
use CGI ();

sub transform {
    my ($template, $vars) = @_;
    eval {
        no warnings;
        $template =~ s/<!--perl(.*?)-->/@{[ eval "$1" ]}/gs;
        $template =~ s/<!--(.*?)\((.*?)\)-->/@{[ $vars->{$1}->( split ',', $2) ]}/g;
        $template =~ s/<!--(.*?)-->/$vars->{$1}/g;
    };
    return $template;
}

sub process {
    my $tmpl = shift;
    my $vars = shift;
    my $buffer;

    my $start = qr{<\?perl>};
    my $end   = qr{</\?>};

    for (split /\n/, transform( $tmpl, $vars)) {
        if ( /$start/ .. /$end/ ) {
            if    ( /$start/ ) { $buffer = '' }
            elsif ( /$end/   ) {
                eval $buffer;
                $buffer = '';
            }
            else { $buffer.= $_ . "\n" }
            next;
        }
        print $_, "\n";
    }
}

sub internal_template {
    my $show = shift;
    my %tmpl;
    my $template_name = 'main';

    for my $line ( <DATA> ) {
        if ( $line =~ m/^\*\*\*\s(\S+)\s\*/ ) {
            $template_name = $1;
            next;
        }
        $tmpl{ $template_name }.= $line;
    }
    return exists $tmpl{ $show } ? $tmpl{ $show } : $tmpl{ main };
}

print CGI::header( -type => 'text/html' );

process internal_template('main') => {
    sidebar => 'This is a menu',
    content => 'This is the main content block',
    gbp     => sub { sprintf "£%.2f", $_[0] },
    li_helper => sub { map { qq{<li>$_</li>} } @_ },
};

__DATA__
<html>
<head>
<title>Simple Template System</title>
</head>
<body>
<div class=“menu”>
<!--sidebar-->
</div>
<div class=“main”>
<p>Great British Pounds: <!--gbp(200)--></p>
<ul><!--li_helper(one,two,three)--></ul>
<p>Random number between 1 and 9: <!--perl int( rand(10) ) --></p>
<?perl>
    if ( int(rand(2)) ) {
        print '<ol>';
        for my $numb (1..5) {
            printf('<li>%d * 2 = %d</li>', $numb, $numb * 2);
        }
        print "</ol>\n";
    }
    else { print "</p>Nothing to show at this moment in time</p>" }

</?>
<!--content-->
</div>
</body>
<!--baz-->
</head>

process() simply loops thru the fulfilled template printing out each line as it goes along except when hitting a <?perl> block which it evals instead.

Above isn’t perfect but it does exactly what it says on the tin. Well that is it use to before I bogged it down with features. I don’t think we can call it a Simple Template System anymore? How about “Super”? “Spectacular”? or even “Special”? Perhaps another word beginning with “S” maybe more appropriate ;-)

They say writing a template system is a rite of passage for every Perl programmer. Its been a while since I did my last one so perhaps its also some kind of recursive itch we have?

Anyway I’ll stop procrastinating and return to using Template Toolkit for my templating needs ;-)

/I3az/

J?r?me Quelin: moosex::poe ftw!

in the serie "why using moose and moosex" - today i present moosex::poe.

say you want to create a poe session. traditional way is to create a module handling spawn method that would be called as:


the module itself would be something like this:


now that's moutful. and it has an easy-to-miss error - found it? right, it's should be event3, not evnet3. you can curse poe quite some time trying to understand why your nice handler never gets called for event3.

here's the version using moosex::poe:


of course, it's still possible to mis-spell event3. however, you only have to spell it once, whereas you wrote it 3 times before - which is 2 times too much!

and i'm not talking about the easy accessors that could be added by moose - but i'm concentrating on moosex::poe here.

granted, there's one downside: subs are now anonymous, which can make it a bit harder to debug. but that's a small price to pay, imo.

so, moosex definitely makes your code more readable - i'm definitely not looking back, and i advise you to have a look too.

xsawyerx's Journal: git-svn FTW!!1

xsawyerx's Journal

Google Code uses Subversion. If you do not have a commit bit to a project, you can use the readonly mode and check it out.

However, since it's Subversion, you cannot commit anything under readonly. So, yeah, I guess you create a patch from a diff and send it to the developer. However, if you do a lot of changes, you have to create patches against the older versions of changes you've already made.

For example: I got A from Subversion readonly. I made changes and now have B. I can't commit B so I create a patch from the diff (B.patch) send send it to the developer. Meanwhile, I've made more changes (now it's C) and I want to send it to the developer. I need to create a diff now between B and C. I need to create a copy of the code at every stage to create patches between the different stages. Err.. isn't that why we have SCMs in the first place? I haven't even begun on other issues like making my changes on top of more changes the developer put in between my work.

Git allows me to create a local repo that has access to an SVN path to pull from or push changes to. I just use git-svn and then once I have a commit bit, I can push (or dcommit in git-svn-speak).

That way I fetch the readonly using git-svn, commit commit commit, then when I have a commit bit I can update the Subversion repo with all my commits, or even rebase my code to be the latest after pulling more recent changes from the Subversion repo.

Blog of G?bor Szab?: What is Perl used for?

Abstract

Recently I was asked What is Perl used for?. I typed in the question in Google but I did not get a satisfactory answer. So I decided to spend some time wring up a list of fields and some example application just to end up pointing to 3 much better lists.

For the full article visit What is Perl used for?

tsee's Journal: Users, speak up!

tsee's Journal

A few days ago, Gabor Szabo has put out a request for help packaging Padre. Since then, a few people stopped by the #padre channel on irc.perl.org willing to help with packaging Padre using PAR. After a short introduction, some stated something like "I can help with packaging Padre because we packaged a very large and complex $work application using PAR::Packer. It required a few hacks for the worst external dependencies, but apart from that it was as easy as pie."

This is great for several reasons. It means people are considering Padre important enough that they are willing to help out and share their expertise in packaging applications with PAR. But it is also one of the few occasions that I have seen folks publicly acknowledge the usefulness of the PAR toolkit. I regularly get private mail from people asking for help with packaging their big-ass $work applications because it's grown over their head. Unfortunately, those people generally also ask for a bit of confidentiality as well. So I know PAR is being used all over the place, in large and small applications alike, scaling up to tens of thousands of computers. But I mustn't point at a list of companies who use it for deployment and do the "this is enterprisey software, worship it" dance.

So dear PAR users. Please share with the world that you use it and maybe how you do. You're enjoying a good, free tool. The least you can do is give (public) feedback and share your experience with others. The same applies if you're not enjoying it at all. Let us know so we can improve it!

Cheers,
Steffen

PS: Most of this post would probably equally apply to many other successful modules. If you have stories about those, please share them as well!

chorny's Journal: Sci-Fi story that mentions Perl

chorny's Journal

If you know Russian, you should read this short story: О программистах.

ajt: O&#39;Reilly Discount Perl Books

Today has been an excellent LUG meeting at IBM Hursley. We had lots of talks and they were all very good. I gave a brief "Book Reviewing" talk and managed to give away all my APress books for review. I told people that I'm expecting more from APress and O'Reilly in the future but I suspect they publishers' would like to see reviews of the first wave first.

I mentioned that O'Reilly are doing a limited trial of cheap PDF Perl books, and I'm getting emails from people asking about the discount code. It's good to see people giving O'Reilly a go with their PDF books, and better still they are Perl books that people are interested in.

nothingmuch's perl blog: Git Hate

I like Git, but sometimes I also really hate it.

My friend Sartak asked if there is a better way of finding a merge base in a command than hard coding master as the merge point, and I stupidly said "of course, you just check what that branch tracks". That is the same logic that git pull would use, so in theory you can apply the same concise logic to merge or rebase your branch without running git fetch.

In principle that's correct, but in practice it's utterly worthless.

The first step is to figure out what branch you are on.

You could easily do this with the git-current-branch command. Except that it doesn't actually exist. Instead you need to resolve the symbolic ref in head and truncate that string:

$(git symbolic-ref -q HEAD | sed -e 's/^refs\/heads\///'

Except that that's actually broken if the symbolic ref points at something not under heads.

Ignoring that, we now have a string with which we can get our merge metadata:

branch="$( git current-branch )"

git config --get "branch.$branch.remote"
git config --get "branch.$branch.merge"

The problem lies in the fact that the tracking refers to the remote, and the ref on the remote:

[branch "master"]
 remote = origin
 merge = refs/heads/master

But we want to use the local ref. Based on git config, we can see that this value should be refs/remotes/origin/master:

[remote "origin"]
 url = git://example.com/example.git
 fetch = +refs/heads/*:refs/remotes/origin/*

If you read the documentation for git-fetch you can see the description of the refspec for humans, but it's pretty feature rich.

Fortunately Git already implements this, so it shouldn't be too hard. Too bad that it is.

In builtin-fetch.c are a number of static functions that you could easily copy and paste to do this refspec evaluation in your own code.

So in short, what should have been a simple helper command has degenerated into an epic yak shave involving cargo culting C and reimplementing commands that should have built in to begin with.

And for what?

git merge-base $( git-merge-branch $( git current-branch ) ) HEAD
instead of
git merge-base master HEAD

In short, it's completely non-viable to do the right thing, nobody has that much spare time. Instead people just kludge it. These kludges later come back with a vengeance when you assume something does the right thing, and it actually doesn't handle an edge case you didn't think about before you invoked that command.

I'm not trying to propose a solution, it's been obvious what the solution is for years (a proper libgit). What makes me sad is that this is still a problem today. I just can't fathom any reason that would justify not having a proper way to do this that outweighs the benefit of having one.

Shlomi Fish: Perl-Begin Updates: CSS, Page about Hashes, and More

Here are the new changes to the Perl Beginners site since last time.

  • A New Page about Hashes has been added to the site, with "Jerusalem Perl Monger"'s excellent presentation called "The Hash Cookbook".
  • There are many corrections to the "Perl for Newbies" presentations, in part thanks to input from several people who tried to learn Perl using them.
  • The CSS stylesheet of the page was compressed, minimised, transformed to a fluid design and converted to the compass CSS frameworks. Many thanks to Alan Haggai Alavi, who did the bulk of the work on the conversion of the page to compass.
  • The "Mailing Lists" page was enhanced with a new introduction and some link fixes.
  • The IDEs page now has some screenshots.
  • We've heard many complaints from experienced Perl users about a joke that has appeared on the top of the "Get a Job Doing Perl" page, so it was now removed.

Modern Perl Books, a Modern Perl Blog: Aspects of a Novice-Friendly Distribution

Scott Walters suggested a Perl optimized for the novice user experience. Adam Kennedy called it Perl::Tiny.

If someone were to build such a thing, what would it be? How would it look? What features would it have and why?

I'd start with Adam's Chocolate Perl, a full distribution of Perl 5 which includes the toolchain necessary to build and install CPAN modules as well as supplementary tools and external modules such as Task::Kensho. In short, it's a modern, enlightened Perl 5 distribution.

Next, I'd add Padre. It's not the only Perl IDE, but it's under active development with responsive maintainers, frequent releases, and a plugin mechanism that allows the bundling and use of several other interesting pieces of infrastructure.

I'd use the Perl::Critic plugin and make it active by default. It'd take some work to choose the right set of default rules, but I'm sure we could come up with something modern. This will help novices avoid common problems which the Perl 5 core doesn't warn about.

I'd set a default template which enabled Modern::Perl by default. There's no sense in giving novices good tools without asking the compiler to give them as much assistance as possible.

I'd include Moose, of course. I'd include other CPAN modules such as Try::Tiny, though I wouldn't include all of perl5i. The goal here is to provide better alternatives to core syntax which is difficult to get right.

I'd provide a plugin which can explain Perl 5 magic variables by extracting the appropriate documentation from perldoc perlvar. I don't know if I'd recommend the English equivalents.

I'd also provide a plugin which can use Perl::Tidy and even B::Deparse to canonicalize code -- and explain syntactic constructs, keywords, and operators with the appropriate documentation from perlsyn, perlop, and perlfunc.

I'd love to see -- though this may be an additional project -- a graphical CPAN client similar to Ubuntu's Syntaptic, which can browse and search the CPAN indexes.

It might be worthwhile to add a plugin to search PerlMonks and other sites for help and questions.

I can imagine that there's a small bit of income available from selling training material -- especially katas and interactive lessons, but that's a different story altogether.

Would you have used such a system when you were a novice? Would you recommend it to others now? Am I missing things?

nothingmuch's perl blog: YAPC::Tiny, YAPC::Asia

I'd like to thank to everyone that made my trip to Asia so great thus far.

To list but a few: thanks to the JPA for organizing YAPC::Asia, especially Daisuke for being so accommodating when it wasn't clear if I would make it till the end, gugod for organizing YAPC::Tiny, Kenichi Ishigaki and Kato Atsushi for translating my talks (though only one was presented in the end), clkao and audreyt for being such great hosts in Taiwan, Dan Kogai for hosting us at Hotel Dan, and the Japanese and Taiwanese Perl communities in general for being so much fun to be around.

I'm so indebted to everyone involved in this conference and all of the others, it makes every one of these trips fun and memorable, totally justifying the fact that Perl takes up such an unhealthily large part of my life and income.

Tomorrow is the hackathon, always my favourite part of conferences. On my agenda:

  • Hacking with gfx and rafl on Moose's XS stuff
  • Fixing a weird corruption that happens with Continuation::Delimited, so that finally it passes all of its tests. Once that happens I can focus on adding new failing tests, which is much more fun ;-)
  • Playing with PSGI/Plack, Stardust, and other cool things I learned about during the conference and will probably discover tomorrow

On Sunday, the last day of my trip, CL and I are going to try to not get killed on Mount Myogi.

Modern Perl Books, a Modern Perl Blog: The False Dilemma of Novice and Savant User Experiences

In the discussion of Perl Applications for Normal Users, the subject of installation came up again. User experience isn't always my primary concern, and I don't mind overgeneralizing that to other parts of the Perl 5 community.

For example, many Perl community websites are unattractive visually. There are exceptions. The London Perl Mongers site is very attractive. I also like the design of the Perl Moose site.

Yet a common refrain in many (more heat than light, sadly) discussions is that functional is more important than attractive. Yes, that's a false dichotomy.

I don't want to restart the debate over the attractiveness of Perl 5 websites. (I'd love to see variant templates that writers, developers, and projects could borrow and adapt for consistency of good design, but that's a different story.)

I want instead to talk about installation. This ties in to Who Benefits from the CPAN?. Consider, for example, the amount of work required to configure a CPAN client in 2000 versus the amount of work required to configure the same client in 2009. This has improved.

Yet compare that to installing a random PHP application.

Preemptive ad hominem: if at any time you feel the need to comment here or elsewhere saying "You fool! You foolish fool! You want to turn Perl into Java, you drooling maniac!", you are a masochist and your opinion does not matter.

One commenter on the previous post praised Ruby Gems for not failing tests and not spewing diagnostic output and asking confusing questions. Of course the thought of not running tests on installation horrifies me (how do you have any idea if your software works?), but the point is a good one. What information do users need to know to install your software? What questions do you really need to ask them?

We've optimized the Perl ecosystem for Perl developers. That's good. That helps us. That doesn't necessarily mean that what's good for Perl developers is good for Perl users. That also doesn't mean that we can only optimize the experience for one group at the expense of the other.

We can do better.

I argue often for better defaults -- not just at the language level but at every level users might see. Yes, I've overloaded the word "user". Deal with it.

One important question for me as I ponder visions for Perl 5 and its ecosystem is "How can we improve the default behavior for novices and for most uses by experienced programmers while allowing experienced programmers to customize behavior for advanced uses?" That's a language design principle. If you analyze the design of Perl 5, you'll see Larry's answers everywhere. The same goes for Perl 6.

I don't have complete, systemic answers for that yet. We have a lot to ponder. As food for thought, consider the (oblique) connection between two comments. The first comes from a critique of Python:

Also, its libraries require compilation. What a nonsense.

The second comes from Scott Walters, specifically his Language choice motivated by greed journal:

Moose is good and Moose is great, but the real win would be to automatically install modules, automatically call the OS's package manager to install libraries and other deps, to have created PHP before php did (or after) and have a libperl linking executable that outputs web pages (period), and most of all, to have a sandbox mode enabled by default (tired idea, I know) where users can write code without getting yelled at by humans, only by the computer, even if just because the code is labeled (all is fair if you predeclare) as "babytalk". Perhaps this would be a mix of strict, diagnostics/warnings, and a Perl::Critic policy that tries to help them in only the most immediate sense likely to be useful to novices in the short term.

The Cattle Grid: MojoMojo wiki software on Gentoo

MojoMojo is a great wiki software based on Catalyst. Installation, however, can be quite a nightmare if you want to do it through the package manager of your operating system because of the many Perl libraries it depends on. Using the package manager is actually a good idea, unless you decide to install all dependencies locally through local::lib, as it doesn't pollute your system and makes upgrade operations much more linear. Although almost no Linux distribution has packages for all all those libraries, Gentoo has a project which already provides ebuilds for them and for MojoMojo itself. They can be found in the Perl overlay: http://git.overlays.gentoo.org/gitweb/?p=proj/perl-overlay.git;a=summary The overlay can be easily synced using the layman Gentoo tool (see this wiki page for more information). You can then just add the overlay with: layman -a perl-experimental and you're nearly ready to emerge dev-perl/MojoMojo. Yeah, nearly. You have to perform some (potentially) boring operations before, which include the unmasking of all the ebuilds in the overlay (300 or so): autounmask or a combination awk, perl and/or some other tool will help you adding them to /etc/package.keywords. Hopefully, portage will allow us to use wildcards for unmasking soon or after. MojoMojo itself is currently hard masked because a dependency of it, dev-perl/Encode has conflicting files with perl 5.8.8. So, first of all you need to add MojoMojo and Encode to /etc/portage/package.unmask and then decide what to do regarding the conflict. You can either ignore it and install Encode with perl 5.8.8 or (best of all) unmask perl as well and upgrade it to 5.10.1 (it's in the same overlay). In my experience, perl 5.10.1 is very stable on Gentoo, and upgrading to it will also help testing it and speed its inclusion in mainstream Portage. Once you sorted out the perl thing (don't forget to run perl-cleaner if you upgrade... ;-)), take a look at the MojoMojo USE flags and enable what you need. Here's an excerpt from the metadata.xml file: <flag name="createdb">Create new Database automatically</flag> <flag name="docbook">DocBook formatter</flag> <flag name="tocgen">Table Of Contents Generator</flag> <flag name="podformatter">POD formatterr</flag> <flag name="syntaxhighlight">Syntax Highlightert</flag> <flag name="transclusion">Transclusion support</flag> <flag name="amazonboxes">Amazon boxes</flag> <flag name="rssformatter">RSS Formatter</flag> <flag name="emoticons">Emoticons</flag> <flag name="recaptcha">reCAPTCHA for anonymous edits</flag> There are two other entries in the file ( markdown and autodeploy ) which refer to older version of the software and are not to be used anymore. When emerging, portage will surely ask you to unmask some more packages: it's boring, bot once done... it's done. When everything is unmasked, just relax while MojoMojo installs. ;-) At this point you have everything you need, and you can just unwrap a MojoMojo package anywhere you like and customize it to create your wiki. At present time this is the only option, although ideally one should be able to use webapp-config to create a new blog in the future....

Ovid's Journal: Java Tutorial Fail

Ovid's Journal

So I came back from two weeks in the US to find that I'm now a Java programmer. Not having seriously programmed in Java since 1.4 (pre Tiger, or pre 5.0, depending on how you count), I decided to brush up my basics. Working through Sun's Java tutorial, I came across this delightful code:

public class Rectangle {
    private int x, y;
    private int width, height;

    public Rectangle() {
        this(0, 0, 0, 0);
    }
    public Rectangle(int width, int height) {
        this(0, 0, width, height);
    }
    public Rectangle(int x, int y, int width, int height) {
        this.x = x;
        this.y = y;
        this.width = width;
        this.height = height;
    }
    ...
}

OK, I can understand multiple constructors. I'm OK with method overloading (wish we had it in Perl), but switching the order of arguments?

Rectangle rec1 = new Rectangle(4,5,0,0);
Rectangle rec2 = new Rectangle(4,5);

It would have been trivial to make those equivalent, but since the tutorial doesn't show extending the arguments, they're not equivalent. Nice job showing new programmers bad code, Sun!

Side note: I'm going to be in Java-land for a while because we're creating a new service which requires the domain knowledge that I and another programmer have, but due to internal requirements, must be written in Java. Thus, we have two Java devs on loan who will build the system and I'll be pairing with them and our team will take over maintenance after they leave. Should be interesting to see how this works out.

Blog of G?bor Szab?: Help packaging Padre on Linux and Mac OSX

Riding on top of Strawberry Perl for Windows, now Windows is the easiest platform for Padre installation. Compared to that installing Padre on Linux or Mac OSX is still a headache.

We have several directions on how to improve the situation. Friendly people at Debian, Ubuntu, FreeBSD, Mandriva, Fedora and probably elsewhere are working on including Padre in the upcoming releases of those distributions but there are people who would like to try Padre now.

We would like to make their first time experience better by allowing them to download a single file, unzip it and have Padre up and running in minutes.

We have started to work on it and we have outlined our plans but we would love to see some people come along and help with either of these directions.

So if you are into PAR packaging, know Shipwright or is experienced in building code (perl, Gtk, etc) on Linux or Mac OSX we would love to get your help.

The best way to contact us is via IRC or on our mailinglist.

xsawyerx's Journal: Thinking Aloud

xsawyerx's Journal

Here are some ideas I have floating around. I really don't have any spare time to implement them, so I feel like I can't really ask for anything of the authors, but I feel like sharing:

  • A fork of Module::Starter that will use File::ShareDir to store the templates. That way I could access it and change it. I find inline POD very disruptive.
  • On that note, perhaps a script or something to tear the POD out of files, and perhaps create a link for an online tool that does that for modules on CPAN. I view the source often, and I would like to reach just the code, not the user documentation POD. Also, syntax highlighting in CPAN "Source" view would be quite helpful.
  • Put use.perl.org on Git or SVN and start a fund to help pay for some designer (preferably from the Perl community like Andy Wardley or Jon Allen) to redo the website. Please oh god please!
  • Fix use.perl.org so I when I login it returns to where I was so I don't have to crawl back there from the main page. It is so god damn annoying.
  • Patch POE::Component::Generic to encapsulate the internal object in the Session, so you won't have to put it in an attribute or stuff it in the data hashref.
  • Create the ::Simple interface to Devel::Declare. I have no idea what's going on there.
  • Conversely, learn Perl internals, become wizard, learn to use Devel::Declare!
  • Improve the design of Perl101.org.
  • Explain this to everyone searching for help without using strict or warnings.
  • Dear god, proper documentation for WxWidgets. Gabor: this is why I strayed away from contributing to Padre. I tried and tried and tried till I hit myself over the head with the keyboard, loss consciousness and forgot I ever wanted to write a plugin or fix bugs with anything that uses WxWidgets :)
  • Do a lecture on Moose in Israel.pm or ask/convince/threaten Yuval Kogman HaMagniv [some Hebrew there] to do one.
  • Telekinetically force at least 3 authors to apply at least 4 patches I wrote to at least 4 modules a while ago which went completely unnoticed.
  • Help out more with Module::Build, I really liked the guys there. Same goes for Perl::Critic.
  • Write a 100% Perl implementation of Wordpress to use Template::Toolkit. Dear god I hate the stupid "write templates using hardcoded php!" approach.

xsawyerx's Journal: Eating my (own) hat

xsawyerx's Journal

A long while ago I wrote of how Perl still lacks a lot when it comes to web. I was referring specifically to CRUD.

Recently I tried Catalyst::Plugin::AutoCRUD and found it to be truly amazing! One of the easiest and most beautiful CRUDs I've seen, all fully configurable and fully automated.

Now to find some salt for that hat...

PJF's Journal: Dark Stalking on Facebook

Dark Stalking on Facebook
For a while I've been using Facebook's API and Facebook Query Language (FQL) via Perl's WWW::Facebook::API module to run fairly innocent queries on my friends. If I visit a town, I'd like a reminder of who lives there. If I want to go rock-climbing, it helps if I can easily search to see which of my friends share that hobby. This is good, innocent stuff, and makes me glad to be a developer.

Last week I decided to play with event searches. If a large number of my friends are attending an event, there's a good chance I'll find it interesting, and I'd like to know about it. FQL makes this sort of thing really easy; in fact, finding all your friends' events is on their Sample FQL Queries page.

Using the example provided by Facebook, I dropped the query into my sandbox, and looked at the results which came back. The results were disturbing. I didn't just get back future events my friends were attending. I got everything they had been invited to: past and present, attending or not.

I didn't sleep well that night. I didn't expect Facebook to share past event info. I didn't expect it to share info when people had declined those events. I haven't found any way of retrieving friends' past events using Facebook's website, but using FQL made it easy. Somehow, implicitly, I thought old events would fade away, only viewable to those who already knew about them. I didn't expect them to stick around for my code to harvest, potentially years into the future.

Finding my friends' old events crossed a moral boundary I honestly didn't expect to encounter. Without intending, I really felt like I was snooping. It didn't matter that these friends had agreed to share this information under the Facebook terms and conditions. I would personally feel uncomfortable with this much information being so readily available, and assume my friends would feel the same.

However my accidental crossing of moral boundaries wasn't the only thing that kept me awake last night. I was also kept awake by wondering just how much information could I tease out of the Facebook API. What could I discover? What if I were evil?

However I'm not evil, so I put my code on hold for a while and made a call for volunteers. I'd be restricting myself to just using the Facebook API, and without them installing any additional applications. I wouldn't share their data in any way, but I'd be able to inspect and use it, and would try to provide them with a copy when I was done. To be honest, I was surprised by the response; I now have almost two dozen people who have agreed to participate, covering a wide range of lifestyles and privacy settings.

The results have been very interesting. I expected to be able to obtain personal information, including things like events, photographs, and friends; it doesn't take much imagination with the FQL tables to find those. What was most interesting are some of the more creative queries I was able to run.

Most recently, I've been able to obtain status feeds, even for users who have very tight privacy settings, although I had to tweak my own application's privileges to do so. I don't know how far into the past these go, but they also come with likes information, and comments. This gives me a wealth of information on the strength and types of relationships people have. A person who comments a lot on another user's posts probably finds that user interesting. If I descended into keyword and text analysis, I may even be able to determine how they find that user interesting.

But by far the most interesting part of all of this have been dark users. Like dark matter, these users are not directly observable, usually because they've completely disabled API access. In fact, some of these users are completely dark unless you're a friend. They don't show up in search results. They don't show up on friends' lists. You can't send them messages. If you try to navigate to their user page (assuming you know it exists), you get redirected back to your homepage. These users have their privacy settings turned up real high, and are supposed to be hard to find.

However like dark matter, dark users are observable due to their effects on the rest of the universe. If a dark user comments on a stream entry, I can see that comment. More importantly, I can see their user-ID, and I can generate a URL to a page that will contain their name. I can then watch for their activities elsewhere. Granted, I can't directly search for their activity, but I can observe their effects on my friends. For want of a better term, I've been calling this "dark stalking".

What makes this all rather chilling is that I'm doing all of this via the application API. If your friend has installed an application, then it can access quite a lot of information about you, unless you turn it off. If your friend has granted the application the read_stream privilege, then it can read your status stream. Even if a friend of a friend has done this, and you comment on your friend's status entries, it's possible to infer your existence and retrieve those discussions through dark stalking.

While I've always considered people's own carelessness to be the biggest threats to their own privacy, in the social 2.0 world it seems we need to be increasingly worried about our friends, too!

I'm preparing a detailed paper with the results of my research (which is still ongoing), but I will be presenting my preliminary findings at BarCampMelbourne, this weekend (11-12th September 2009), with a further update at the University of Tasmania Computing Society (TUCS) on the 2nd October. A conference talk will invariably follow.

If you want to keep track of my research, then you can join the facebook group, or the facebook privacy group. I prefer comments and questions to directly to the facebook privacy group, or to me directly.

Posted: 10th September 2009.

Tags:

Bookmark:

Digg this Digg this

John Napiorkowski’s Perl Development Blog: Higher Order Perl Book PDF Version, and other Free Perl Books

I know this has been reported elsewhere, but the very excellent, "Higher Order Perl", is available as a free download.

This is definitely worth buying if you can spare the cash, or can convince your boss to pay for it.  However if you can't, or if you just really want something you can read on your notebook or mobile phone while riding the train, this pdf is very high quality.

If you haven't heard about this book, it's a fantastic exploration of how powerful and beautiful Perl syntax can be.  I know people often charge that Perl is 'read only' or 'line noise'.  Taking a quick look here should silence that criticism.

If you are trying to learn Perl and looking for other free books, check this page out.  Or, check the general Learning Perl page.  Or just ask me what to do in the comments section below :)

Read and post comments | Send to a friend

Perlbuzz: Perlbuzz news roundup for 2009-09-09

These links are collected from the Perlbuzz Twitter feed. If you have suggestions for news bits, please mail me at andy@perlbuzz.com.

dagolden: Module::Build progress and roadmap

From my recent posts, you may have noticed that I’m doing a lot of work on Module::Build these days. I got back into it to get it ready for 5.10.1 and once I had re-learned the code base it was easy to start adding some frequently requested features and cleaning up other bugs.

Here’s a snapshot of new developments and current plans:

  • MYMETA.yml ― implemented in 0.35_01 as discussed previously
  • File::ShareDir support ― implemented in 0.35_02. Adding a ’share’ directory to your distribution will automatically configure it to be installed for use with File::ShareDir (and will add File::ShareDir to your ‘requires’ list of prerequisites). The ’share_dir’ property allows more custom control of shared directories if needed.
  • Build installdeps ― implemented in 0.35_02. This build action is a helper routine for authors and developers who check out a distribution from a repository and want an easy to way to install dependencies. It uses ‘cpan’ by default, but can be configured to use ‘cpanp -i’.
  • needs_compiler property ― implemented in 0.35_02. Module::Build now auto-detects XS or C files and checks for the presence of a compiler. It also adds ExtUtils::CBuilder automatically to ‘build_requires’.
  • bundling Module::Build in inc/ ― under development. Soon, it will be possible to automatically bundle Module::Build within the inc/ directory for use with perls that haven’t been upgraded to support configure_requires in META.yml. This will probably be released in experimental form before 0.36
  • declarative interface ― under discussion. Some people seem to really like the declarative interface of Module::Install. Once bundling works, it’s likely that a declarative interface for Module::Build will follow, perhaps for 0.37.
  • plugins ― conceived, but not discussed. Subclassing has limits and some people would like a way to create and combine plugins to customize the build process. This will take some rework of the Module::Build internals, so is probably unlikely to happen soon, but it’s on the list of long-term goals.

As you can see, Module::Build is moving along at a good clip. The next stable release will have some significant new features to address deficiencies that were attracting people to Module::Install and should help make Module::Build more easily backwards compatible in ways it hasn’t exactly been to date.

Blog of G?bor Szab?: Help! How to deal with madness?

Two encounters: On our Perl list someone asked for help and it turned out his manager set a policy not to use strict. On the local Linux list while trying to help someone I found out that they are using a home made module called DB for database access which I guess means they cannot use the debugger of Perl as that also lives in the same name-space.

The thread starts here, no strict policy mentioned here.

The thread with the DB starts here

I really don't know how to deal with such situation other than running away.

I mean, my little children already know they should put on the seatbelts when in the car and should not run around with scissors.

Perlbuzz: Mentoring in open source communities: What works? What doesn't?

By Esther Schindler

Open source offers amazing opportunities. There are almost no barriers to entry. If you want to try creating a new-to-you kind of application, or to learn how to write bright-shiny documentation, or to use the latest technology that your Day Job doesn't give you access to -- you can just barrel right in with an open source project and get involved. Once you become proficient (or demonstrate that you already are), you can apply those skills in the next phase of your career. Even better, you can choose which community you want to be a part of, and find a comfortable culture where your contributions matter.

However, because open source is so personally driven and self-motivated, there aren't always a lot of opportunities to consciously improve your skills -- except on your own. While that's certainly valuable, it relies on you recognizing what needs improvement and then knowing what to do about it. In a regular office, you might be lucky enough to work with someone who'll take you under her wing, and give you specific advice about how to improve your code. Or someone senior to you will let you talk his ear off about the hard choices you have to make, and suggest solutions you didn't think of. The distinction I'm making here is between "learn on your own" (such as examining the changes others make to the code you contributed) and somebody offering specific, individual advice (e.g. "It might run faster if you did THIS..."), particularly in an ongoing personal relationship.

Many open source communities do actual mentoring (even if they don't think of it with that label); others don't. Some make a concerted effort to connect newbies with more experienced people. They provide opportunities for people to work together in smaller teams (not just a gang hanging out in an IRC channel, however useful that is), such as in sprints and code-a-thons. (Tops on the list of "encourage mentorship" is, of course, the Google Summer of Code. But I know there are other less-public endeavors.)

For a feature article at ITWorld.com, I want to interview people from several open source communities about the mentoring experiences. I want to hear what they do right, and how they go about encouraging mentoring relationships. I'd also like to hear from open source participants who have yearned for a bit more one-on-one attention... and what (if anything) they've done about it.

My goal here is to explore what's involved in a successful mentoring effort, and also find out what doesn't work. I like to think that this can help all sorts of open source communities that want to attract more participants.

How you can help

Think you can help? Please email your thoughts on the topic to esther@bitranch.com. Here are some of the questions you could address:

  • What have been your mentoring experiences in open source communities? How well or how poorly have they worked? Why do you have that opinion?
  • If you developed mentoring relationships in an open source community, how did they come about? Was there a deliberate effort to connect people (how did that work?) or did it evolve on its own (how did it happen?)?
  • What did you learn? What did you hope to learn?
  • Knowing what you do now, what would you do differently?
  • What advice would you give to open source communities in regard to mentoring?
  • I'm also particularly interested in hearing from people in communities where mentoring doesn't exist or where it doesn't come as naturally -- opportunities may exist, but they're harder to find.

Be sure to identify:

  • the project(s) you're involved in. Include the URL for the project if you like, as well as how you contribute (I write code, or I've led locally-run code-a-thons, etc.)
  • your name, role/title, and company in the way you prefer me to refer to you ("Esther Schindler, a programmer at the Groovy Corporation, and also a frequent contributor to the Blahblah open source project").

I'll accept input on this topic until Monday, September 14th. After that I have to write the article. :-)

A long-time technology evangelist and community instigator, Esther Schindler has been in the computer press since 1992. Her primary journalistic focus for the last decade has been software development and open source, and she's contributed as writer or editor to Software Test & Performance, InformIT.com, DevSource.com, and dozens of other publications. She's currently on assignment for ITWorld.com -- where she writes the open source blog Great Wide Open

Modern Perl Books, a Modern Perl Blog: Applications for Normal Users

I spend most of my programming time writing tools for other programmers. My business (Onyx Neon Press) has a modest amount of code to produce books, but a fair amount of the code in our production process is Pod::Simple and Pod::PseudoPod -- tools for other programmers.

Most of my work on Parrot and Perl 6 is infrastructure for other programmers to use too.

I suspect -- but can't prove sufficiently -- that many of the most popular and most widely used projects on the CPAN are likewise tools for programmers. Moose is a tool for programmers. So are Catalyst, Scalar::Util, Method::Signatures, Devel::NYTProf, and perl5i.

That's not a bad thing. I'm not complaining. It's an observation.

Perhaps CPAN is a tool primarily for developers. That's fine too.

Even so, I wonder where are all of the wonderful applications Perl programmers can mention when people outside the Perl community ask "What's it good for? What can it do for me?"

I'm happy to talk about Padre and BioPerl or Movable Type and Melody. Frozen Bubble is a good story (especially with SDL Perl under fresh development again -- see Kartik Thakore's journal for more SDL Perl details). dotReader didn't get much attention, but it's an impressive project. BBC iPlayer is just on the edge of projects to mention.

I'm not sure bragging about websites implemented in Perl 5 is useful or interesting. For the most part, that's immaterial. A web site is a web site is a web site.

Maybe Perl 5 doesn't need this. Java does pretty well with its comfortable niche in enterprisey web development, Eclipse (a programmer tool for writing programmer tools, remember), and LimeWire. Python gets a boost from the original Bittorrent code (Mercurial is a developer tool, as is embedded Python for game developers and 3D modeling programs). Ruby... well, there's Rails.

Sometimes I read Planet Gnome and Planet KDE and marvel at all of the nice projects that non-developer end-users can use. I see some of these applications developed in Vala and JavaScript (yes, a developer tool -- but one used to write a web browser, a mail client, a music player, et cetera) and even sometimes Python.

I know I've used Perl 5 for system administration, web development, workflow automation, games, visualization, and more. Maybe I'm a very unrepresentative user, and that's fine. Maybe I don't have the mindset to create software for non-developer users and that's probably fine too.

Even so, sometimes I wonder if the Perl community spends so much time obviously and obsessively focused on the needs of other developers that we might neglect some easy and obvious problems where Perl can be part of a great solution.

... not that such users will use Perl directly, but that expanding our area of influence may be a very good idea.

John Napiorkowski’s Perl Development Blog: RFC: Catalyst::TraitFor::Model::DBIC::Schema::AutoDeploy?

One of my personal goals for this year is to reduce some of the boilerplate code I have to write in order to get working with my applications.  This effort to ease Perl Programming is inspired by great projects like Moose, which really allows me to get focused on the job at hand and not worry about framework and setup.  One area that I'd like to see improved relates to the effort involved in setting up development and testing databases.  My recent cpan release of Test::DBIx::Class is my swing at the latter; this blog is my outline and RFC for the former.

One of my goals for Test::DBIx::Class is to make it trivial to deploy, seed and cleanup testing databases.  Currently we have the ability to automatically deploy DBIC based Schemas to SQLite, MySQL and Postgresql.  This ability is built on top of DBICs deployment code as well as some other bits from CPAN.  This makes the job of deploying, setting up and testing as easy as:
    
    use Test::More;
    use Test::DBIx::Class qw(People);

    fixtures_ok 'basic',
        'basic fixtures installed';

    is_fields 'first_name', People, [qw/john vanessa vincent/],
        'Got expected first names';

    done_testing();

Which will automatically create a database (using SQLite by default, but you can override and deploy to MySql or Postgresql as well), install the 'basic' fixtures (from a configuration file, but you can 'inline' create statements easily as well, see the docs for more) and test the schema resultset called 'People' to see if that set contains the asked for first_names.  This reduces a lot of boilerplate code for deploying the database, checking it, etc (and we've all written 10 half baked version of that, right :) ) and offers some helpers for actually testing the data.  For example, the 'is_fields' will ignore sorting order by default, comparing just the actual set.  The above test assertion would probably have to be rewritten as:

    is_deeply [sort map { $_->first_name } $schema->resultset('People')->all],
        [qw/john vanessa vincent/],
        'Got expected first names';

If you wanted to duplicate most of it's functionality, but there might be edge cases missed, like date fields and columns that inflate, weird differences in null handling, etc.  And that's for a very simple case where you are testing a single field.  If your test is more involved the syntax becomes increasingly verbose.

One of the things I really wanted for this module is to make it very easy to create a test database not only for SQLite, but for at least Postgresql and Mysql.  Both these goals are helped along by the cpan distributions Test::mysqld and Test::postgresql which autodetect the presence of Mysql or Postgresql (they need to be installed, but don't need to be running) and creates a temporary database installation in /tmp or the directory of your choice.  These databases live until the final tests complete and are then automatically cleaned up, unless you want to keep them, using a configuration setting or an %ENV variable.

This grants you a lot of benefits when you are trying to speed development along.  You don't need to login and create the testing database, worry about setting up all the correct permissions, etc.  If you are working in a multi developer company (like most of us) you probably have to create testing databases for everyone, unless you are willing to deal with the chaos of everyone's tests banging into each other.  This method basically just creates a database for the programmer on the fly, reducing that effort considerable.  Finally, since each temporary database is given it's own area to live, this allows you to run prove in parallel using the --jobs (or -j) option.  Usually you can't run database tests in parallel, since each test will essentially be contending for a single testing database instance, using classic methods.  Running parallel test jobs can greatly speed completing your tests, even with the overhead needed to instantiate a test database.

Now that I feel pretty comfortable with the code functionality, I am looking to factor out some core bits and use it to build a similar tool for development databases.  This would be a trait loadable by newer versions of Catalyst::Model::DBIC::Schema that would autodetect your target database, instantiate and deploy one as needed.  This way it would make it much easier to just get going with development, particularly if you are following a methodology were each developer has a personal development database.

I'd like your feedback on the idea, and request comments on the features you'd like to see most.  For example, I am currently working on supporting Replication in Test::DBIx::Class, since if you are using replication in production then your test cases really should run against a replicated setup.  This may not be needed in development.  On the other hand we might want our development databases to support the versioning system built into DBIC.  Since in test I generally build the database from scratch each time, I don't care about versioning so much.

My imagination tells me we'd want something called  “Catalyst::TraitFor::Model::DBIC::Schema::AutoDeploy” which would add a configuration option to your Catalyst::Model::DBIC::Schema called 'target_driver' or similar so you could specify SQLite, MySQL or Postgresql, as well as whether you wanted to preserve the database between running the application server and so forth.  I also figure by default to built the database files in $home/share/data, or similar, (again looking for suggestions on the best default).

This would be intended primarily for development, but would probably also be useful for people deploying catalyst applications to shared hosting setups.  This would allow one to basically deploy everything for an application to a single directory root.

Thoughts?

Read and post comments | Send to a friend

ajt: Cheap Books

Good/cheap books are like buses, you wait for an age then several turn up at once.

At the moment I'm swamped with Apress books for reviewing. I'm hoping to farm most out next Saturday at a LUG meeting, but even so it's still a lot of reviewing...

Then brian d foy announces cheap Perl ebooks (no DRM) from O'Reilly!

Perlbuzz: ack 1.90 released

I just released ack version 1.90 to CPAN. If you don't know about ack, it's a text searching tool for programmers aimed specifically at searching large trees of code. Find out more at betterthangrep.com.

Here's the changelog for this version:

1.90        Mon Sep  7 23:24:24 CDT 2009
[ENHANCEMENTS]
Added Ada support.  Thanks to Shaun Patterson.

Added -r, -R and --recurse options as in grep.  They have no
effect because directory recursion is on by default.  Also added
--no-recurse for orthoganality. Thanks to Mark Stosberg and
Ryan Niebur.

Version in --version is prettier.  Thanks, Ori Avtalion.

Added an updated ack.bash_completion.sh from Adam James.

[FIXES]
Expanded --files-without-match to --files-without-matches.

Removed all the hi-bit characters, so we don't have any encoding
problems.  It's all entities now.

Fixed capture-stderr to localize some globals that were obscuring
errors.  Thanks very much to Christopher Madsen.

Fixed uninitialized errors in tickets #138 and #159.

[DOCUMENTATION]
Fixed an incorrect command line in the docs for -f.

Added notes on --pager.  Thanks to Mike Morearty.

[BUILD]
Made the squash program more robust when handling POD.  Thanks
to Kent Fredric.


1.89_02     Wed May 13 16:20:21 CDT 2009
[DISTRIBUTION]
Updated Makefile.PL to use new ExtUtils::MakeMaker features.
Thanks, Schwern.

[FEATURES]
--version now shows the version of Perl that ack is running
under, and the full path to the Perl executable.

Added new switches --color-match and --color-filename, which
let you define ACK_COLOR_MATCH and ACK_COLOR_FILENAME from the
command line.

Added new switch --column to display the column of the first
hit on the row.  Thanks to Eric Van Dewoestine.

Added .ss to --scheme.

[FIXES]
No longer die if you have a .tar.gz file in your tree.

More tweaks to get the detection of input and output pipes
working.

Fixed an amazingly bad call to cmp_ok() in t/ack-passthru.t.

[DOCUMENTATION]
Started an ack FAQ.

Dave's Free Press: Bryar security hole

Someone on IRC reported a bug in Bryar. Namely that a Naughty Person can exploit the feature that notifies you of blog-spam by email to execute arbitrary code on your machine, as the user you run Bryar under.

A patched release is on the way to the CPAN, and you are strongly urged to upgrade.

Perlgeek.de : Starry Perl 6 obfu

I decided to have some fun with Perl 6 and wrote my first obfu. So here it is, broken to 50 columns per line:

<(*+*)*(*+* +*/*) **** ** * * *+*+*/* **** ** * *
*+*-*/* **** ** * * *+* ****-* **** ** * * *-(*-*/
*) ** (*+*/*) **** ** * * *-* **** ** * * *-*+*/*
**** ** * * *+* **** ** * * *-* * * ***((* - */*)
** * * *-*/*) * * **** ** * * *+*/* ****-* ***(*-*
/*)**(*+*/*) * * ***((*-*/*)** * * *-*/*) * * ****
** * * *+*/* **** ** * * *+*+*/*-* * *>.split(<***
>).map: {eval("(* ** *+*+*-*/*+$_)(<* * *>)").chr\
.print}

(Not syntax-hilighted as I usually do here because the syntax hilighter does this one oh so wrong, it would only enhance confusion).

It works on today's Rakudo build, but I'm not aware of any feature that I use which wasn't available in the 2009-08 release.

Have fun figuring it out!

Update: Since it has caused a bit of confusion on #perl6, it does not print the usual obfu text, but rather a minor variation.

Dave's Free Press: Thanks, Yahoo!

[originally posted on Apr 3 2008]

I'd like to express my warm thanks to the lovely people at Yahoo and in particular to their bot-herders. Until quite recently, their web-crawling bots had most irritatingly obeyed robot exclusion rules in the robots.txt file that I have on CPANdeps. But in the last couple of weeks they've got rid of that niggling little exclusion so now they're indexing all of the CPAN's dependencies through my site! And for the benefit of their important customers, they're doing it nice and quickly - a request every few seconds instead of the pedestrian once every few minutes that gentler bots use.

Unfortunately, because generating a dependency tree takes more time than they were allowing between requests, they were filling up my process table, and all my memory, and eating all the CPU, and the only way to get back into the machine was by power-cycling it. So it is with the deepest of regrets that I have had to exclude them.

Cunts.

[update] For fuck's sake, they're doing it again from a different netblock!

Perlgeek.de : Timeline for a syntax change in Perl 6

August 10, 16:30 (UTC): Carl M?sak starts a thread about the syntax for embedded comments on p6l. Various different ideas are being discussed for about a day.

August 11, 17:28 (UTC): Larry Wall decides on the new syntax and commits his changes to the specification.

August 11, 17:43 (UTC): (that is, 15 minutes later) Larry Wall updates the official grammar to parse the new syntax.

August 11, 17:51 (UTC): (that is, another 8 minutes later): Moritz Lenz (me) updates the spectests to reflect the new syntax..

August 11, 18:11 (UTC): (that is, another 20 minutes later): Patrick Michaud updates Rakudo to the new syntax.

So from the time of the decision it took less than an hour to be adopted by the test suite and two major implementations.

Dave's Free Press: Module pre-requisites analyser

As a service to module authors, here is a tool to show a module's pre-requisites and the test results from the CPAN testers. So before you rely on something working as a pre-requisite for your code, have a look to see how reliable it and its dependencies are.

Perlgeek.de : Dissecting the "Starry obfu"

I've been asked a few times how the starry obfu works. Here it is again in all it's *cough* glory:

<(*+*)*(*+* +*/*) **** ** * * *+*+*/* **** ** * *
*+*-*/* **** ** * * *+* ****-* **** ** * * *-(*-*/
*) ** (*+*/*) **** ** * * *-* **** ** * * *-*+*/*
**** ** * * *+* **** ** * * *-* * * ***((* - */*)
** * * *-*/*) * * **** ** * * *+*/* ****-* ***(*-*
/*)**(*+*/*) * * ***((*-*/*)** * * *-*/*) * * ****
** * * *+*/* **** ** * * *+*+*/*-* * *>.split(<***
>).map: {eval("(* ** *+*+*-*/*+$_)(<* * *>)").chr\
.print}

This first character is a >, which introduces a list quote: <a b c> is the same as ("a", "b", "c"). This quote covers the first six and three quarter of the sevenths line.

It is later then used a string (by calling .split on it, and as such stringifies with spaces between the items. So it's roughly the same as "..." in this context.

Then comes a fairly trivial part: the string is split on ***, and via map a transformation is applied to to each item: it is interpolated into a string, which is eval'ed. The result is taken as a Unicode codepoint number, and printed to the screen.

To find out what exactly is evaled, you can replace the code inside the map to read something like this:

<(*+*)*(*+* +*/*) **** ** * * *+*+*/* **** ** * *
... 
>.map: { say("(* ** *+*+*-*/*+$_)(<* * *>)") };

The output begins with

(* ** *+*+*-*/*+(*+*)*(*+* +*/*) )(<* * *>)
(* ** *+*+*-*/*+* ** * * *+*+*/* )(<* * *>)

You might ask yourself "how on earth is this valid Perl code?"

The answer is quite simple: a single star can be either a term or an operator in Perl 6. As an infix operator it is simply multiplication. As a term it does magic: It constructs a closure which takes one positional argument, and puts that everywhere where a * appears as a term within that expression.

The reason is to make things like @array[*-1] work: it creates a closure { $_ - 1 } which is passed on to the array, which in turn passes the number of items in that array to closure, making @array[*-1] return the last array item.

So by the same token * ** *+*+*-*/*+(*+*)*(*+* +*/*) is internally transformed to

{ $_ ** $_ + $_ + $_ - $_ / $_ + ($_ + $_ ) * ($_ + $_ + $_/ $_) }

The (<* * *>) behind it invokes that closure with a list consisting of three items. In numerical context that becomes just the number 3, so the above translates to 3**3 + 3 + 3 - 3/3 + (3+3) * (3 + 3 + 3/3), evaluated 74 - the Unicode codepoint (and at the same time ASCII code) for the letter capital J.

It works similarly for all other letters; mystery lifted.

Dave's Free Press: cgit syntax highlighting

For the last few months I've been using git for my version control system. It's better than CVS because it can handle offline commits. So if I'm using my laptop on a train, I can still use version control without having to have a notwork connection.

And to give a pretty web front-end to it for other people to read code without having to check it out of the repository, I use cgit, which I mostly chose because it's a dead simple CGI and not a huge fancy application.

One problem with cgit is that by default it doesn't do code highlighting. But it has the ability to run blobs of code through any filter you care to name before displaying them, so to get something nice like this all you need to do is write a highlighter and add a single line to your cgitrc:

source-filter=/web/www.cantrell.org.uk/cgit/highlighter

My highlighter program is this:

   1 #!/usr/local/bin/perl
   2 
   3 use warnings;
   4 use strict;
   5 
   6 my $file = shift;
   7 
   8 if($file =~ /\.(p[ml]|t)$/i) {
   9     system "/usr/local/bin/perltidy -html -st -ntoc -npod -pre -nss -nnn"
  10 } else {
  11     system "cat -n";
  12 }

Perlgeek.de : Goodby Iron Man

<update> (from 2009-08-23) It turned out that my disappearance on the ironman blog feed was due to a broken RSS feed. Matt S. Trout tried to inform me by blog comment, my blog marked it as spam and swallowed it.

So now we talked on IRC, clarified things, and I'm back in the game. </update>

So I accepted the Iron Man blogging challenge a few month ago. And last week I discovered that my blog was gone from their feed. For the second time. Without any notification.

rusty iron man

Image: rusty iron man, by courtesy of artvixn, available under a create commons non-commerical by-attribution license.

The first time they had a good reason: the date tags in my RSS feed were goofed; still I'd thought it would be nice to at least notify me of such a removal. After some mails back and forth I was able to fix it; after the second removal without any notification I'm simply fed up and don't want to investigate any more energy into this.

Still I'll continue to follow the collected RSS feed, there are still many interesting blogs to be read there.

Perlgeek.de :

Dave's Free Press: Ill

I am ill. I've been ill since Thursday, with a cold. You're meant to be able to cure a cold with [insert old wives tale remedy here] in 5 days, or if you don't, it'll clear itself up in just under a week. So hopefully today is the last day.

So what have I done while ill?

On Friday I became old (see previous post), and went to the Byzantium exhibition at the Royal Academy. It was good. You should go.

Saturday was the London Perl Workshop. My talk on closures went down well, and people seemed to understand what I was talking about. Hurrah! I decided that rather than hang around nattering and going to a few talks, I'd rather hide under my duvet for the rest of the day.

I mostly hid on Sunday too, and spent most of the day asleep. In a brief moment of productivity, I got my laptop and my phone to talk to each other using magic interwebnet bluetooth stuff. I'd tried previously without success, but that was with the previous release of OS X. With version X.5 it seems to Just Work, so no Evil Hacks were necessary.

The cold means that I can't taste a damned thing, not even bacon. So now I know what it's like to be Jewish. Being Jewish sucks.

And today, I am still coughing up occasional lumps of lung and making odd bubbling noises in my chest, although my nasal demons seem to be Snotting less than they were, so hopefully I'll be back to normal tomorrow.

Dave's Free Press: Devel::CheckLib can now check libraries' contents

Devel::CheckLib has grown a new feature. As well as checking that libraries and headers exist, it can now also check that particular functions exist in a library and check their return values. This will be particularly useful if you need to check that a particular version of a library is available.

It works if you have a Unixish toolchain. I need to wait for the CPAN-testers reports to see if I managed not to break anything on Windows. Unfortunately, even though the lovely Mr. Alias has worked hard to make Windows machines available to developers, I found it to be just too hard to use. Even downloading my code to the Windows machine was hard, as Windows seemed to think it knew better and shouldn't download the file I told it to download. Then once I had downloaded it, Windows decided to hide it somewhere that I couldn't get to using the command line. So I gave up.

I might try again once there are some decent tools on the machines: wget, tar, and gzip at minimum, as given those I can quickly bootstrap anything else. Software development isn't just about having compilers available.

Dave's Free Press: POD includes

One of my CPAN distributions is CPAN-FindDependencies. It contains a module CPAN::FindDependencies, and a simple script that wraps around it so you can view dependencies easily from the command line. That script, naturally, has a man page. However, that manpage basically says "if you want to know what arguments this program takes, see the CPAN::FindDependencies docs". This is Bad from a usability point of view, good from a not-duplicating-stuff point of view, and good from a laziness point of view. Which means that it's Bad.

So, the solution.

=over

#include shared/parameters

=back

and some Magic that does the cpp-stylee substitution at make dist time. Note the 'dist' section in my call to WriteMakefile.

This is, of course, crying out to be made less horribly hacky, but it works for now, so I'm happy.

My original idea was to write some crazy shit that would do the #include at install-time, when the user was installing my code. But that has the disadvantage that tools like search.cpan wouldn't show it properly, as they simply look at the files in the distribution. So this does the #includes at the last moment just before I package up the code and upload to the PAUSE. You lovely people get the right documentation in all the right places, I only have to maintain it in one place so it stays in sync, and (in the interests of Laziness) I don't have to remember to run any extra scripts before releasing, make dist just Does The Right Thing.

Perlgeek.de : Perl 6 Tidings from August 2009

It's been quite some time since my last tidings post, so I don't think I can cover it all in great detail. I'll try anyway.

Specification

  • File test operations now look like $str.IO ~~ :e (r27503)
  • Some cool regex features were documented that previously only existed unofficially (r27692, r27695)
  • The want() function was removed as unimplementable (r27755).
  • Subroutines without signatures don't always default to (*@_, *%_), only when these variables are actually used (r27801).
  • Nil coercions (r27882)
  • Embedded comments now look like #`[...] (r27959)
  • infix:</>(Int, Int) now produces a Rat object, that is a fraction.
  • Changes to context, variables (r28151); ::= now does read-only binding (instead of compile time binding as before) (r28150)

Rakudo

Rakudo had two releases by people other than Patrick Michaud, namely by me and Kyle Hasselbacher.

Among the highlights are much improved introspection capabilities, user defined traits, user defined and overloadable operators.

Jonathan also worked on the prototype of perl 5 integration into parrot.

MP6

Flavio Glock updated mp6 and bootstrapped in on SBCL.

perl6.org

We now have a shiny perl6.org site.

Dave's Free Press: YAPC::Europe 2007 report: day 1

As is becoming normal, I used the times between talks to bugfix some of my modules - this time Tie::STDOUT and Data::Transactional. The former was failing on perl 5.6, the latter on 5.9.5. The former was a bug in perl (you can't localise tied filehandles and expect the tieing to go away in 5.6, so it now declares a dependency on 5.8), the latter was a bug in my code.

Philippe Bruhat's talk on Net::Proxy was great - you can tell it's great because I came away with ideas for at least four things that I need to write. First up will be a plugin for it to allow the user to specify minimum and maximum permitted data rates for proxied connections. This will permit bandwidth limits for maximum permitted rates, but will also help to defeat IDSes doing traffic analysis if you specify a minimum permitted data rate.

This will protect (eg) ssh sessions from being identified based on their very bursty traffic pattern, by "filling in the blanks" with junk data.

In the evening, the CPAN-testers BOF was productive.

Dave's Free Press: XML::Tiny released

I have released my XML::Tiny module. The parser at its core is less than twenty lines of code. Pretty easy to follow code too, I think, and that also includes error handling. One of my aims in writing it was to keep memory usage and code to the absolute minimum, so it doesn't handle all of XML. The documentation says that it supports "a useful subset of XML". Personally, I think it supports the useful subset. It's certainly enough to parse the data I get back from Amazon when I use their web services, and to parse an RSS feed.

Dave's Free Press: Palm Treo call db module

To make up for a disappointing gap in Palm's software for the Treo smartphone, I wrote a <a href=http://www.cantrell.org.uk/david/tech/treo/call-dumper/>small perl script</a> to parse the database that stores my call history. I then re-wrote it as <a href=http://search.cpan.org/search?query=Palm%3A%3ATreoPhoneCallDB>a re-useable module</a> which also figgers out whether the call was incoming or outgoing.

Perlgeek.de : The Happiness of Design Convergence

Recently I've been having fun writing a plotting module for Perl 6. It's been really fun because adding new shiny things (titles, new chart types, whatever) has been relatively easy, and there was fast visual feedback.

However it's really just a toy module, lacking proper design. When I added legend boxes to one of the chart types (the lines plots), I noticed how much hassle it was, and that I needed code changes in many places to incorporate that feature into other chart types too - that's what you call "design smell".

So I thought about how to improve it, and the answer was quite obvious. Currently each thing that I plot (the data representation, the axis, ticks, labels, title, legend box) know where to place itself, so if I need more space for the legend box every piece has to adapt. Which is ugly.

Instead, I should just make each of them plot itself on to coordinates (0, 0), record their width and height, and later move (and possibly scale) them to fit onto the canvas (Note that I only planned that so far, I implemented the boxing only for a single component so far).

Flashback to the past: Shortly after announcing my very early SVG::Plot module masak pointed me to Scruffy, a small ruby based SVG plotting library that produces beautiful graphs. I looked into the source and found that it was spread into about 50 classes, all of which were short and sweet. I didn't really understand the code, mostly due to lack of Ruby reading skills, and lack of effort.

So I and others decided to port scruffy to Perl 6, the Perl 6 version will be called Tufte. And after re-reading the Scruffy source code I noticed that it was doing just what I planned for SVG::Plot: it encapsulates each graphical element into components (called Scruffy::Components::*). On top of that it allows multiple plots per canvas (the plots are called Scruffy::Layers::*), and allows arranging of them too.

So I was happy to come up with the same idea as the Scruffy hackers, independently, reassuring me that I actually port a nicely designed library ;-)

Dave's Free Press: Number::Phone release

There's a new release, <a href=http://www.cantrell.org.uk/david/tech/perl-modules/Number-Phone-1.58.tar.gz>version 1.58</a>, of Number::Phone, my set of perl modules for picking information out of phone numbers. Changes from the previous release are that Mayotte, Reunion and Comoros can't decide which country is which, and there's the usual updates to the database of UK numbers, mostly to support the <a href=http://www.ofcom.org.uk/media/news/2007/02/nr_20070213b>new 03 numbers</a>.

Dave's Free Press: CPANdeps

<a href=http://cpandeps.cantrell.org.uk/>CPANdeps</a> now lets you filter test results by perl version number, and also knows what modules were in core in which versions of perl. Hurrah!

Perlgeek.de : When we reach 100% we did something wrong

<meta> This is not a marketing blurb as I originally intended to write. It is quite a lengthy and a bit philosophical brain dump. If my regular readers skip it that's fine; I promise that my next post will be shorter, more exciting and easier to digest.</meta>

A few times per week we (the Rakudo development team) update our spectest graph which shows how many tests Rakudo passed in the past and now, and how many tests there are in the official test suite.

This is a very useful information, but still its use is limited. Among other reactions we also received two kinds of responses that I want to talk about. One is "And when Rakudo passes all tests, Perl 6 is ready, right?", the other is "I don't care how many tests you pass until it's 100% (the latter seen in a reddit or Y-combinator comment, for example).

Implications

The first reaction points to a larger problem - how can we know if our tests actually cover the whole specification? The answer is we can't. Ever. We can identify parts of the spec that are very well covered by tests (for example multi dispatch), and parts that are covered badly or not at all (IO, and way too many other areas). But there is a huge number of ways the various parts of the specification interact (growing exponential in the number of specified facts), which makes it impossible to reach complete covering.

Still we can read each paragraph in the specs and see if we have an appropriate test, achieving some kind of shallow coverage.

But worse still there are many things implicit but not explicit in the spec. I write "worse" because for the test suite writer it's a pain, but on the other hand if everything was stated explicitly, we'd waste too much time on trivialities.

One example is a bug which Rakudo had in the beginning: recursion would screw up some lexical variables rather badly. When Patrick and Jonathan looked into fixing it, I wanted to contribute my part and provided some tests. And then I didn't know where to put them. The specs talk about subroutines, and about lexical variables. But it does not talk about the interaction between recursion and lexicals - because it's obvious that each instance of a function gets its own lexical pad on recursion. Perl 5 does this just fine, we carry that semantics over to Perl 6. Perfectly sane, but it's an implication nonetheless.

And, after much talking, I finally come to my first main point: We can't measure test coverage of such implications. We have no automatic way to turn implications into explicit statements either. We can't know if spec and the implications have a decent test coverage.

We won't ever reach 100% passing tests

Most people consider Perl 5 "done". Not in the sense that there's nothing left to do, but that there are stable releases, wide acceptance, on generally technical and social maturity. And yet Perl 5 does not pass all of its test.

It might come as a shock to you if you haven't looked at Perl 5's source tree, and never compiled it from source and run the tests (and actually looked at the results), but it's true.

$ ~/tmp/perl> ack -w 'skip(?:_all)?' t|wc -l
381
$ ~/tmp/perl> ack -w 'todo' t|wc -l
48

There are many valid reasons why some tests should not be run - for example the tested feature might not be available on the current platform, or maybe a previous failure would make their outcome inconclusive.

There are also valid reasons why some tests are marked as TODO (which means the are run, but don't pass) - for example to test a bug that is not yet fixed, or for behavior that is subject to change but has not yet been adapted.

So even with huge efforts our chances of ever unconditionally passing each and every test in the Perl 6 test suite are practically 0. When a run over the whole test suite shows that all tests passed, the first thing I'll do is to check our test module and test harness for errors - maybe some bug prevented them from successfully identifying errors.

If you compare how the tests for Perl 5 and Perl 6 were originally written, you'll see that in the case of Perl 5 a huge amount was written against a working implementation, while many Perl 6 tests were whipped up by reading the spec and transforming it into code, or even as feature requests for pugs. So it won't surprise you that in the Perl 6 test suite there is a larger amount of contradictory tests, written either against different versions of the specification, or by various people who understood the spec differently. We'll work hard to weed out such contradictions, but it would be an illusion to assume that we will fully succeed.

Are you moody?

When you read the previous paragraphs you might think I'm moody, and quite negative about Perl 6. I'm not. I just faced the reality that each software project of at least modest complexity faces, and brain-dumped it here.

In fact I'm quite optimistic. Many areas of Perl 6 improve at an impressive rate these days: the specification, Rakudo, smop+mildew, the websites, available libraries and applications, and last but not least: the community.

Having written quite some Perl 6 code (see my SVG related posts on this blog) over the previous weekend I think that developing Perl 6 has never been more fun - at least not to me ;-)

Dave's Free Press: YAPC::Europe 2007 report: day 2

A day of not many talks, but lots of cool stuff. Damian was his usual crazy self, and MJD's talk on building parsers was really good. Although I probably won't use those techniques at work as functional programming seems to scare people.

The conference dinner at a Heuriger on the outskirts of Vienna was great. The orga-punks had hired a small fleet of buses to get us there and back, and one of the sponsors laid on a great buffet. The local wine was pretty damned fine too, and then the evening de-generated into Schnapps, with toasts to Her Majesty, to her splendid navy, and to The Village People.

It wasn't all debauchery in the evening though - on the bus, I had a very useful chat with Philippe about Net::Proxy, and re-designing it to make it easier to create new connectors for it.

Dave's Free Press: YAPC::Europe 2006 report: day 3

There were quite a few interesting talks in the morning, especially Ivor's one on packaging perl applications. Oh, and mine about rsnapshot, of course, in which people laughed at the right places and I judged the length of it just right, finishing with a couple of minutes left for questions.

At the traditional end-of-YAPC auction, I avoided spending my usual stupid amounts of money on stupid things, which was nice. Obviously the hundred quid I put in to buying the hair style of next year's organisers wasn't stupid. Oh no. Definitely not.

An orange mohican will suit Domm beautifully.

Dave's Free Press: YAPC::Europe 2007 report: day 3

My Lightning Talk on cpandeps went down really well, although as Jos? pointed out, I need to fix it to take account of File::Copy being broken. I also need to talk to Domm after the conference is over to see if I can get dependency information from CPANTS as well as from META.yml files.

There were lots of other good lightning talks. Dmitri Karasik's regexes for doing OCR, Juerd Waalboer's Unicode::Semantics, and Ren?e B?cker's Win32::GuiTest were especially noteworthy.

Richard Foley's brief intro to the perl debugger was also useful. Unfortunately Hakim Cassimally's talk was about debugging web applications, which I'd not noticed on the schedule, so I didn't stay for that.

And finally, Mark Fowler's grumble about why perl sucks (and what to do about it) had a few interesting little things in it. I am having vaguely sick ideas about mixing some of that up with an MJD-stylee parser.

At the auction I paid ?250 to have the Danish organisers of next year's YAPC::Europe wear the Swedish flag on their foreheads. This, I should point out, was Greg's idea. I would never be so evil on my own.

Perlgeek.de : A shiny perl6.org site

I'm very pleased to announce that perl6.org now got a shiny new face - the one that Su-Shee developed for perl6-projects.org. I'd also like to thank Daniel Wright, owner of the perl6.org domain, for making that domain available to us.

The story behind it is as simple as amazing: I noticed that we had no good, up to date overview of what Perl 6 projects existed out there. So I bought the domain perl6-projects.org, put up a few lists of different projects - and gave it a rather bad design. Because I may be decent with programming, blogging and explaining things, but as a designer I simply suck.

But what I did was sticking the source code into a public repository (the pugs repository, to be exact) to which it is very easy to gain access, and made the server pull from that repository every 15 minutes. So basically everybody who wanted could update the site.

And lo and behold, four month later Su-Shee contributed a nice design, which we modified in small things based on feedback from #perl6.

Slowly it dawned to me (and others probably) that this site had the potential to be much larger in scope than a simple project list - it could evolve into the central web site for Perl 6 developers and users. With the small caveat that perl6.org would have been a much nicer domain name - but it was taken already.

But the whois entry gave us hope - it belongs to Daniel Wright, and a quick google search turned up that he's a perl monger, organizer and attendee of YAPC conferences. So we decided that pmichaud should contact him, being more of an authority in Perl 6 land than me.

After a few exchanged mails we found out that Daniel's and our goals matched, so about one day later he changed the DNS to the same IP that perl6-projects.org uses, I added another alias to the virtual host, and we were done.

It's a prime example both for attracting contributors, and for what you can reach by simply talking to people. Thanks to everyone involved.

Dave's Free Press: Perl isn't dieing

Perl isn't dieing, but it tells me that it wishes it was. Last night it went out on the piss with Python and Ruby (PHP was the designated driver) and it did rather too many cocktails. It isn't quite sure what happened, but it woke up in the gutter in a puddle of its own fluids and its head hurts a lot.

It asked me to ask you all to keep the volume down.

Dave's Free Press: YAPC::Europe 2007 travel plans

I'm going to Vienna by train for YAPC::Europe. If you want to join me you'll need to book in advance, and probably quite some way in advance as some of these trains apparently get fully booked.
arr dep date
Waterloo 1740 Fri 24 Aug
Paris Nord 2117
Paris Est 2245
Munich 0859 0928 Sat 25 Aug
Vienna 1335

The first two legs of that are second class, cos first wasn't available on Eurostar (being a Friday evening it's one of the commuter Eurostars and gets booked up months and months in advance) and was way too spendy on the sleeper to Munich. Upgrading to first class from Munich to Vienna is cheap, so I have.

Coming back it's first class all the way cos upgrading was nearly free ...

arr dep date
Vienna 0930 Fri 31 Aug
Zurich 1820
Zurich 1402 Sun 2 Sep
Paris Est 1834
Paris Nord 2013
Waterloo 2159

Don't even think about trying to book online or over the phone, or at the Eurostar ticket office at Waterloo. Your best bet is to go to the Rail Europe shop on Picadilly, opposite the Royal Academy and next to Fortnums.

Perlgeek.de : Let's build an object

Building an object in Perl 6 is rather easy. As the author of a class you don't really have to care (at least in the simplest case), you inherit a default constructor from class Object. As a consumer of that class you just write YourClass.new(attrib1 => $value1) to create an object of class YourClass, at the same time initializing an attribute.

Running initializations code

If you want to run some initialization code on object creation, you don't have to touch the new method at all. Something like this works:

class C {
    submethod BUILD {
        say "Created a new instance of C";
    }
}

C.new();

The BUILD submethod is called by the constructor automatically, and can do any initialization that's necessary. It also receives the named arguments that the user passes on to new().

(In case you wonder, a submethod is a public method that's not inherited to child classes).

Custom constructors

Suppose you're not a big fan of named arguments, and you want to write a constructor that takes one mandatory positional parameter. In that case you'd write a custom new method. To create an object, that method has to call self.bless:

class C {
    has $.size;
    method new($x) {
        self.bless(*, size => 2 * $x);
    }
}

say C.new(3).size;      # prints 6

The star * as the first argument to bless tells it to create an empty object itself.

If you want to enable additional named parameters, that's easily done:

class C {
    has $.size;
    method new($x, %n) {
        self.bless(*, size => 2 * $x, |%n);
    }
}

Note that these two concepts (custom new() and BUILD() (sub)methods) are orthogonal; you can use both at once, and both peacefully coexist.

Understanding object initialization

As demonstrated above you don't need to understand the full process of building and initializing objects to manipulate it. If you still want to know, read on.

Suppose you have a class C which inherits from another class B, then the process of building an object of class C looks like this:

Perl 6 object creation

The user calls C.new (which is inherited from class Object), which in turn calls self.bless(*, |%args). bless creates a new P6Opaque object which is the storage for the newly created object. This is the call to CREATE in the image above.

After the storage has been allocated and the attributes initialized, new passes control to BUILDALL (passing along all named parameters), which in turn calls BUILD in all classes in the inheritance hierarchy, starting at the top of the hierarchy and calling the BUILD method of class C at last.

This design allows you to substitute parts of the initialization with least effort, and especially writing custom new and BUILD methods very easily.

Instead of passing a * to bless, you can also pass along an object of different storage type, which means that you can instruct Perl 6 to store your objects by a different mechanism, for example by the gnome library GObject or similar (though that's not yet implemented in Rakudo).

Perlgeek.de : Perl 6 is optimized for fun

Audrey Tang coined term optimized for fun in the context of Perl 6, abbreviated -Ofun. (The -O compiler option instructs gcc to optimize code).

And indeed we have lots of fun: Perl 6 is our MMORPG, but we still get things done and avoid the planning trap.

We also have fun in our specification:

    1. Retreat to remote Himalayan monastery

    2. Learn the hidden mysteries of space and time

    ????

    3. Prophet!

We also have fun on IRC:

12:49 < pugs_svn> r28043 | moritz++ | [S02] small clarifications on Whatever 
                  comprehensions
12:50 <@wayland76> I experience a strange feeling when reading moritz_'s
                   commit message :)
[...]
14:11 < pugs_svn> r28045 | wayland++ | [S02,S03,S09,S11] Changed any example 
                  that had the animal "Cat" to have "Squirrel"
14:11 < pugs_svn> r28045 | wayland++ | instead (both have kittens), because we 
                  already have a type called Cat and a 
14:11 < pugs_svn> r28045 | wayland++ | method called "cat", and I could see 
                  some potential for confusion.  
[...]
14:14 <@moritz_> wayland76: +     %cats\   .{'fluffy'} = Squirrel.new;
14:15 <@jnthn> Squirrel is not a type of cat?
14:15 <@moritz_> well, it would be sensible to name the variable %squirrels in 
                 that case ;-)
14:18 < pugs_svn> r28046 | moritz++ | [S03] store Squirrels in %squirrels, not 
                  in %cats
14:19 <@moritz_> somehow my commit messages sound a bit surrealistic today
14:20  * jnthn likes surrealism.
14:20  * masak fish surrealism
14:37 <@frettled> moritz_: You know, the cats may disagree and think that 
                  squirrels _should_ be stored in cats.  Just mentioning it.
14:37 <@jnthn> wantfud!
14:38 <@wayland76> well, those cats can go clarify their Whatever 
                   comprehensions for all I care :)
14:38 <@masak> one would think that cats have been evolved to store rodents.
14:39 <@wayland76> I doubt the rodents think of it as "storage" :)

If you also like programming and having fun, join us on #perl6 on irc.freenode.net.

Perlgeek.de : Visualizing match trees

I think I mentioned once or twice that regexes and grammars in Perl 6 really rock. Most Perl 6 compilers use them to parse Perl 6, which demonstrates how powerful they are. Writing a full grammar for JSON took only about 70 generously spaced lines.

A successful match stores all kind of captures in a Match objects, which is actually a full match tree. If you're new to Perl 6 regexes, you might find the structure a bit surprising at first.

So I wrote a module which visualizes a match tree, annotating parts of the original string with the access path on the match object.

use SVG::MatchDumper;

token fruit     { banana | apple }
token currency  { 'USD' | 'dollar' | 'EUR' | '$' | '?' }

my $x = 'just 20,000 dollar per apple';

if $x ~~ m/:s ((\d+) ** ',') <currency> 'per' <fruit> $ / {
    svg-dump($/, $x);
} else {
    die "no match";
}

produces

match tree visualization

I appreciate any feedback and testing.

Perlgeek.de : Rakudo "star" announced

(Update:: Here is the "official" blog post from pmichaud - which pretty much makes my rambling superfluous.)

Yesterday Patrick Michaud announced that he'll release a special version of Rakudo called Rakudo star in Spring 2010. Sadly I was not present, but as minor minion I picked some things up on IRC and want to provide some backgrounds.

Alias bitched about the term "Spring", which is a rather regional thing and of course he is right. I'd like to clarify that we are talking about spring on the northern hemisphere, so the release will probably be between March and June.

The idea behind this release is to provide something more shiny than our usual development releases, get good press for it, and motivate lots of people to use it. We need that for several reasons:

  • A release is the only way to get good testing from a wide audience; we learned as much from the perl-5.10.0 release, for which many new bugs were only found after the release.
  • Many Perl 5 programmers are vaguely excited about Perl 6, but wait for a "real thing" before they spend more time on it. We want to show these people that Rakudo has become a "real thing" by now, and implements much of the language.
  • We want to encourage hackers to port their Perl 5 modules to Perl 6. For them it's a good way to learn Perl 6, for the rest of the community it provides a valuable ecosystem. Perl is a good example of how a great collection of libraries can greatly enhance a languages popularity - we'd like to have such a bonus for Perl 6 too.

Currently the Rakudo developers mostly focus on features that are either at the very heart of Perl 6, or that are needed for bootstrapping. Examples for the first category re the object model, multi dispatch and regexes/grammars. The second category covers features like contextual variables, proto regexes and longest token matching.

I expect the focus to (temporarily) shift towards other features, like improved module loading/importing and maybe speedups. But still there are areas that not yet actively pursued - the roadmap does not even mention concurrency/threads. So we hope for volunteers to step up and champion their pet projects.

In the last few months various pieces of PR helped to bring more people into the Perl 6 community. I hope that the Rakudo star release will boost that and will help to form a healthy Perl 6 community.

Dave's Free Press: Wikipedia handheld proxy

I got irritated at how hard it was to use Wikipedia on my Treo. There's so much rubbish splattered around their pages that it Just Doesn't Work on such a small screen. Given that no alternatives seemed to be available - at least, Google couldn't find any - I decided to write my own Wikipedia handheld proxy.

It strips away all the useless rubbish that normally surrounds Wikipedia pages, as well as things like the editing functions which are also hard to use on portable devices. Internally, it's implemented using perl, LWP, and mod_perl, and is hosted by Keyweb.de.

Perlgeek.de : A SVG plotting adventure

The Quest

It usually begins with Carl M?sak - this time was no exception.

First he sent us on big adventure, and then me on a particular quest by asking innocently if there were any vistor statics for the site http://perl6-projects.org. Being curious myself I copied the access.log, started a log analyzer and voil?, we had a HTML page with some statistics.

But a tiny nagging voice in my head asked Can't you do that with Perl 6?. Well, sure I can, with a bit of effort.

So I decided to spend some time of my weekend (namely a travel by train and the time when my girlfriend took a nap) on this, and wrote a web log analyzer that produced a bar chart of the daily number of visitors. Here it goes:

visitor stats for perl6-projects.org

If you use a browser that supports SVG, you can view the svg version directly..

That was the short story. If you are more of a geek, you might be more interested in some of technical details.

The Gory Details - SVG

The choice of SVG as output format was pretty obvious: It's text based, a vector format, and masak had already written a SVG module for Perl 6. Despite its name, it's actually a module that generally transforms data structures of a particular form to XML - the only things specific to SVG in there are the name and the documentation.

Since it doesn't know about SVG, I as the programmer had to learn some SVG. So I downloaded the specs, and went offline.

Coordinate transformations

Skimming through the specs revealed something interesting: you can apply coordinate transformations to SVG primitives or groups. Also the drawing area stretches from negative to positive infinity in both x and y direction. In combination that means that the plotting module can start drawing without looking at the whole input data. It just has to keep track of how much space it has written to, and then in the end it can apply a scaling to fit the canvas (ie the visible drawing area).

There's just one problem with this approach:

distorted text

The text is scaled with the same aspect ratio as the chart, leading to distorted text. Of course one could apply the reverse transformation to the text, but to avoid overlapping label text one has to know the size of the text anyway, so there's nothing gained by letting the SVG renderer do coordinate transformation.

So I ended up doing the scaling in my program code after all.

Text in SVG

There's one open problem so far: since the plotting program just emits SVG (or more accurately, a data structure which SVG.pm turns into SVG) and doesn't know about its rendering, it can't know how much space the text will take up, making it impossible to calculate the spaces appropriately. Inkscape and gqview render the text quite differently, I'm sure other user agents will have their own interpretation of text sizes and scaling.

The specs talks about precomputed text width according to which text might be scaled; I'll have to see if I can make use of that feature to solve this problem.

SVG != SVG

Speaking of different user agents: Firefox and gqview only show drawing on the canvas, silently (as per SVG spec) dropping elements outside the canvas. It was incredible helpful to have inkscape, which shows the border of the canvas, but also objects outside of it. When I got a coordinate transform wrong, that was a good way to debug it.

Mandatory Options

I made one observation not tied to SVG, but to plotting in general: without having planned for it, I ended up with having about ten configuration variables/attributes, more to come when I include captions, axis labels and so on. It just falls naturally out of process of writing the code, and thinking twice each time I used a numeric constant in code.

Perl 6 makes them very nice, the begin of the code looks like this:

class SVG::Plot {
    has $.height            = 200;
    has $.width             = 300;
    has $.fill-width        = 0.80;
    has $.label-font-size   = 14;
    has $.plot-width        = $.width  * 0.80;
    has $.plot-height       = $.height * 0.65;

    has &.y-tick-step       = -> $max_y {
        10 ** floor(log10($max_y)) / 5
    }

    has $.max-x-labels      = $.plot-width / (1.5 * $.label-font-size);

    has $.label-spacing     = ($.height - $.plot-height) / 20;
    ...
}

Basically I declare attributes with default values, which can be overridden by passing a named argument to the new constructor - and the subsequent declarations which depend on them simply pick up the user supplied value if present, or the default otherwise.

It seems easier to come up with a myriad of configuration options than choosing sane defaults.

The Result

... can be found on http://github.com/moritz/svg-plot/. The log parser is included in the examples/ directory.

It is far from complete, the interface is not yet stable etc, but it's still a nice piece of software, IMHO.

The Future

I want to pursue development in a few directions: more meta data (axis labels, captions, annotations, maybe clickable data points), other chart types (lines and points), documentation, and maybe more styling options.

I don't know yet how to write unit tests for it, and test driven development only works if you know in advance what to expect.

I don't know if I'll find the time to turn this into a full-blown charting module, but so far the work on it has been very rewarding, and I liked it.

Perlgeek.de : How to Plot a Segment of a Circle with SVG

circle with differently colored segments

(SVG version)

I wanted to generate some pie charts with SVG (for SVG::Plot), and a search on the Internet quickly showed that I need paths for that. But the exact usage remained unclear to me until I read the specs and did some experimenting.

The way to go is:

  1. Move to (M) the center of the circle
  2. Line to (l) the starting point of the arc
  3. Arc to (A) the end point of the arc
  4. Close the path (z)

    The start and end point of the arc can be calculated as

    x = center_x + radius * cos(angle)
    y = center_y + radius * sin(angle)
    

    The parameters to the A command in the path are rx, ry, axis-rotation large-arc-flag sweep-flag x y.

    For our purposes rx and ry need to be just the radius of the circle, large-arc-flag is 1 if the difference between start angle and end angle is larger than pi (or 180°). sweep-flag is 1 if we assume that the start angle is smaller than the end angle (and thus, since the SVG coordinate system has the positive y axis downwards, plot clockwise). x and y are the coordinates of the end point.

    The code I used to generate the SVG above is (in Perl 6, using Carl M?sak's SVG module):

    use v6;
    BEGIN { @*INC.push: 'src/svg/lib' }
    use SVG;
    
    sub arc($cx = 100, $cy = 100, $r = 50, :$start, :end($phi), :$color = 'red') {
        my @commands = 'M', $cx, $cy,
                'l', $r * cos($start), $r * sin($start),
                'A', $r, $r, 0,  + ($phi - $start > pi), 1,
                        $cx + $r * cos($phi), $cy + $r * sin($phi), "z";
    
        my $c = join ' ', @commands;
        return 'g' => [
            :stroke<none>,
            :fill($color),
            path => [ :d($c) ],
        ];
    }
    
    
    say SVG.serialize(
        'svg' => [
            :width(200), :height(200),
            'xmlns' => 'http://www.w3.org/2000/svg',
            'xmlns:svg' => 'http://www.w3.org/2000/svg',
            'xmlns:xlink' => 'http://www.w3.org/1999/xlink',
            arc(:color<blue>, :start(0),    :end(pi/3)),
            arc(:color<red>,  :start(pi/3), :end(3 * pi / 2)),
            arc(:color<yellow>, :start(3 * pi / 2), :end(0)),
        ]);
    

    The only syntactic feature here worth explaining is that + ( $thing > pi ) returns 1 if $thing is larger than pi and 0 otherwise. Everything else should be straight forward.

Header image by Tambako the Jaguar. Some rights reserved.