Perlsphere

There were nearly 150 active entries in the CGI.pm bug tracker when I was approved recently as a new co-maintainer. As I had time in the evenings after the baby was sleep, I went through and reviewed every one of these bug reports. Many had already been addressed by Lincoln some time ago. Those were simply closed. Still, I found about 20 fairly ready-to-go patches, and those have now been processed and released today as CGI.pm 3.45. Whenever code changes were made, I also strived to make sure new automated tests were added that covered those cases. You may be surprised how many methods in CGI.pm have no automated tests for them at all.

Now there are still about 50 open issues in the CGI.pm bug tracker. For these, I have tried to use the subject line some summary indication of what is needed to move it forward, like “Needs Test: “, or “Needs Peer Review: ” or “Needs Confirmation”. Generally, I plan to wait patiently for volunteers to help with these. If you use CGI.pm, consider helping to move one of these forward.

To make collaboration easier, CGI.pm is now on github. You are welcome to fork and send pull requests through there, although posting patches to the bug tracker continues to work file for small changes. The full CVS history has not been translated yet, but may be eventually.

CGI.pm has been in the Perl core since 5.4. With its maturity comes quirks. I’m not a fan of the HTML generation functions. I find the support for both OO and procedural styles awkward. As I have more time, I also hope to continue updating the CGI.pm documentation to promote more modern practices, and de-emphasize other parts of it, like the HTML generation functions and the procedural interface.

I would like to thank Lincoln Stein for building and releasing CGI.pm, and for maintaining it for over a decade. Many projects from CGI::Application to Movable Type depend on it. I also appreciate his willingness to allow for direct help on the project, and his receptiveness to the documentation overhaul idea. Lincoln is continuing his involvement. He completed the final review and release of the changes proposed for 3.45.

I understand that CGI.pm is broadly used, but like ExtUtils::MakeMaker, not always well loved. It’s true that some day it will be completely replaced by next-generation tools, and some reasonable candidates exist now. Until then, there are thousands of existing users will appreciate our collective maintenance of the module. Let’s get the bug tracker back down to zero!

Posted by Mark Stosberg on Perl, CGI.pm, perl at Sat, 15 Aug 2009 01:28:23 -0000 | Permalink

This is response to Laurent Dami's post about when to prefer direct access to the hash fields of an object over an encapsulated API.

The point of the article is that encapsulation is a tradeoff, especially in Perl where you could do some things more easily or efficiently if you decide to give it up.

While I agree that this tradeoff exists, I think that it's not the only tradeoff you're allowed to make, and that Laurent's reasoning in defense of hash access can be solved in ways that don't involve breaking encapsulation and that are in my opinion superior.

In short, I find most of his examples are either better solved by Moose, or fall short of being substantial in my opinion.

I will attempt to refute his arguments one by one.

Good Perl idioms are no longer available

Admittedly the fact that the results of object methods or subroutine application cannot be easily interpolated into strings is a big annoyance in Perl, but that's far from a good reason to avoid them.

I might be alone on this, but personally I find the rest of these examples quite troubling. I think they miss the point of OO programming.

By refactoring these examples into a real API we no longer need the comments to explain what they're actually doing. Maybe the code implementing these methods is not as idiomatic, but we don't need to know it's there (and since we don't know, it could be that it actually is as idiomatic as he likes under the hood.

The total lines of code may be longer for a single usage, but most applications that manipulate points in a 2D space would need to do these operations many times, reducing the overall verbosity:

my $opposite = $point->symmetry_transform;

my $zoomed = $point->zoom($factor);

# I'd actually use a wrapper object for this
# (DisplayPoint with a Menu and a Point delegate)
my $point = Point->new_with_traits(
    traits => [qw(Menu)],
    menu   => { color => "green" },
    ...
);

There is one final example:

# temporary push aside
{ local $point->{x} += $far_away;
    do_something_without_that_point_in_the_way();
} # point automatically comes back to its previous location

And though for this one there is no direct correspondence^[1], the problem is that instead of passing the set of relevant points to do_something_without_that_point_in_the_way(), the point is mutated to temporarily hide its effects.

I've posted on immutability before and why I think that this approach is wrong, but suffice it to say that I would write the above using an explicit collection of points, and make the operation of creating new collections easy instead of monkey patching the data.

No obvious distinction between "setter" methods and other methods

This problem can be solved by using an alternate method naming style. For instance MooseX::Policy::SemiAffordanceAccessor makes the default setter for an attribute named foo into set_foo.

Personally I feel this is overkill. If the relationship between the object and the method name is obviously that of a noun and an attribute, such as $thing->size(10), that is far from ambiguous.

For cases that are more involved, like a field manipulation that requires additional actions I tend to have a method name that has a verb in it and keep the setter completely private. Since the method would be doing something more than just set_ that is reflected in the API.

Hard to debug

This is a deficiency in the Perl debugger. I strongly believe that the maintainability of your code shouldn't suffer because of inferior tools. Someone might fix the Perl deubgger one day. Though unlikely, it's much more unlikely that they will also all of my code.

Since the inner workings of Catalyst is obviously not what Laurent wants to debug in the example, the problem lies in the Perl debugger's inability to handle complex (and idiomatic) perl expressions.

Furthermore, in this particular case it shouldn't be a problem, $self is known and you could simply set a breakpoint for that class's min_body method and continue execution instead of stepping through every operation.

Secondly, if Catalyst had supported lazy body parsing, in which case that line might be run before the body length is actually known (the body may not have been fully read), then this operation wouldn't be a simple accessor fetch but rather a blocking read and an update of the request object.

This polymorphism in the API is precisely what makes it possible to enable such optimizations in future versions of Catalyst without breaking existing applications. If we simply used length($c->{request}{body}) directly, there would be no room for future improvements, even on an HTTP abstraction that could support that easily.

Non-scalar attributes must either copy or reinvent an API

This problem in particular is solved using MooseX::AttributeHelpers (note that this will be in core Moose soon, after receiving a big facelift).

In the more general sense, Moose deeply supports the encapsulation and packaging of patterns using its meta trait system. By applying traits to attribute we can alter their behavior, providing additional functionality in a concise and declarative way.

There's no need to reinvent or copy because of Moose's extensibility, something that is missing from other class systems, and this applies not only to non scalar data, but to many other patterns for which MooseX modules were written.

No generic data traversal modules

Again, introspection is key here. KiokuDB would never be possible without generic data traversal for Moose objects. That's what sets it apart from Pixie.

I've been meaning to write a Data::Visitor::Moose for a long while but simply never got around to it. It's definitely possible, and even not that hard.

Here is a simple example demonstrating JSON integration:

method TO_JSON {
    return {
        map { $_->name => $_->get_value($self) }
            $self->meta->get_all_attributes
    };
}

But the really nice thing is that since all the attributes have extensible metadata we can be very particular about how we filter them. Unfortunately the JSON module's API doesn't allow us to specify parameters to the TO_JSON method, but if it would then we could very easily output different variants for different contexts.

From the point of view of the class being serialized (obviously the TO_JSON method could go in a role and get reused), we could decorate attributes with traits to determine their correct behavior. For instance an attribute that is used both by an ajax frontend displaying the data, and a REST Json feed for this object could be declared as:

has name => (
    traits => [qw(MyApp::View MyApp::REST)],
    ...
);

The actual implementation involves two roles (MyApp::View and MyApp::REST) with no implementation, and a simple $attr->does($role) call in the TO_JSON method.

This is not only cleaner, but also more powerful than violating encapsulation. The difference is that the encapsulation provided by e.g. Class::Accessor or Object::Tiny is just not introspectable or extensible.

Methods are slower than direct access to attributes

I won't argue about this, but in my experience even with large (some would say gratuitous) amounts of method calls I have never been able to demonstrate that accessors or other small methods were an actual performance issue (that is, using a profiler on real code with real inputs).

My dying laptop can make about 1,000,000 method calls per second. My $20 a month virtual private hosting image does 3,000,000. The difference between 5 method calls and 500 in a single dynamic web request is still too small to actually matter, at least for the kind of apps that I write. 100x as many method calls for the same amount of code will not be 100x slower, but will probably be 10x more maintainable ;-)

That aside, the XS branch for Moose has been performing almost as fast as direct hash access in microbenchmarks, while still providing full type validation, support for lazy attributes, access control, etc. This is because XSUBs are faster than pure perl subroutines (they require no context scope, and a single opcode dispatch). And of course these accessors can be subclassed or swapped out for more involved implementations should the need arise.

The XS branch is unmerged and unfinished, but there are a number of other XS accessor implementations on the CPAN if that is actually a problem in your code.

Conclusion

Laurent concluded his post saying:

In Perl, fully encapsulated objects are sometimes the best solution, sometimes not; weight these considerations before taking strong design decisions.

An interesting design is the one of DBI : objects (handles) are totally encapsulated, yet they exploit the power of tie to expose their attributes through a conventional hashref API, instead of OO getter and setter methods. This is a very clever compromise.

As far as I am concerned, I purposedly designed DBIx::DataModel to fully exploit the dual nature of Perl objects, having both the OO API for executing methods, and the hashref API for accessing column values. I wouldn't necessarily do that everywhere, but for row objects in a database, which are very open in nature, this just seemed an appropriate solution.

Specifically in the case of DBIx::DataModel, I think that hash access is a valid approach. But this is because DB records are not objects, but tuples. You can model tuples as objects, but not the other way around.

I couldn't agree more with his sentiments that this a tradeoff you should consider, but I think his cutoff point is very different from mine. I would need to be much more desperate to reach for "naughty" hash access when accessors would do.

Lastly, every time I've used the DBI api I really wished it didn't do that horrible hashref hack. Cleverness is bad, it's confusing to newbies, it's a red flag to experienced programs new to this API, and to reference his previous point about speed, tie is about 4 times as slow as normal method calls. To me that feature seems like a half hearted attempt to allow polymorphism and encapsulation in an API where methods would break compatibility.

This complexity makes understanding DBI internals a lot harder, and by proxy makes writing DBDs harder too. This is a fact of life that we deal with because DBI has been around for longer than I've known to program and because it's still robust and performant. It has flaws but it makes up for them. That doesn't mean the tied hash attributes were a good idea.

I feel that my decision to be an OO purist, especially in a language where impurities are so tempting has made my job as a maintenance programmer much easier for me and my coworkers.

To me polymorphism is a far more important idiom than concise hash manipulation. It may not be unique to Perl, but it's damned useful.

^[1] $point->localize( x => $new_value, body => sub { ... } )

Posted by nothingmuch on moose, object oriented, perl at Sat, 15 Aug 2009 00:43:00 -0000 | Permalink

For some reason, a PerlMonks post from last year, Perl Cannot Be Parsed, has resurfaced in online discussions.

As you might expect, most comments are ill informed and blatant in their wrongness.

At least half of the comments are the equivalent of "OH noes i cantn't read PERL LOLLERS now i is nos Y!!!", but that's what you get for reading Twitter, Digg, YouTube, and occasionally Reddit. (If no one's invented the phrase "The Eternal 4channing of the Internet", I claim it.)

Another significant portion of the comments come from people who've read only the headline and feel sufficiently enlightened that they can proclaim their wisdom to the world at large, despite comments and clarifications on that PerlMonks post from people who've actually worked on the Perl 5 internals and understand the implications of that statement and their limitations.

A few comments are from people who know what this means and what it implies and those (few) cases where it applies.

I suspect the rest of the programming world which might care persists in a limited state of confusion. What does this mean? Why does it matter? What happens now? This is especially true for people who use Perl but don't necessarily consider themselves programmers. They don't have computer science backgrounds, but they'd use good tools like Perl::Critic if they knew they were available.

On Static Parsing

The unparsability of Perl 5 is a static unparsability. "Static" is a specific term of art which means, basically, "You can't run any part of the program to determine what it means."

One of the most common points of confusion in the discussion is a misunderstanding of what "static" means. This doesn't mean that it's impossible to parse Perl 5. The files perly.y and perltoke.c in the Perl 5 source tree do just that. However, the existence of subroutine prototypes and bareword declarations combined with the possibility of running arbitrary code during compilation with BEGIN blocks introduces ambiguity to the process.

That's all the proof addresses.

I realize that's not obvious to everyone reading this so far. That's fine. Just remember that the proof addresses important but limited circumstances.

Why limited? It is possible to parse Perl 5 code statically in many cases -- most cases, even. PPI is an example of this. PPI can build a document object model representing Perl 5 programs sufficient to perform static analysis; this is why Perl::Critic works.

PPI isn't perfect, and it can't resolve all possible Perl 5 programs to unambiguous object models. Part of that is the malleability of Perl 5 syntax.

Changing the Parse

I mentioned subroutine prototypes and barewords; these are two cases where predeclaration changes how Perl 5 will parse existing code.

Subroutine prototypes can change the expected arity and parse and runtime behavior of subroutines. In general, you should avoid them unless you know exactly what this means and what it implies.

Changing Expectations

In specific circumstances, they're useful to change how Perl 5 passes arguments to the functions. My favorite prototype lets you pass blocks as anonymous subs:

sub wrap (&) { ... }

my $wrapped = wrap { do_this(); do_that() };

Without the & prototype character, you'd have to call wrap() with more verbosity:

my $wrapped = wrap( sub { ... } );

It's a small prettification, but it's occasionally useful.

There are two important points to note about this example. First, Perl 5 must encounter the sub declaration with its prototype prior to the point at which it parses the call to wrap. The prototyped declaration changes how Perl 5 parses that construct -- otherwise the curly braces would run through the "is it a hash reference or a block?" heuristic and would not get promoted to an anonymous subroutine declaration.

Second, a smart and stateful static parser could identify this type of declaration and parse the code the same way the Perl 5 parser does. This is not by itself sufficient to make parsing Perl 5 statically indeterminate.

(Some people might object that statefulness in a static parser violates some rule of computer theory somewhere, which is a true point but practically useless; you need at least some statef in your parser to find the end of a double-quoted string if you allow escaped embedded double quotes. The question is not if you allow lookahead but how much lookahead you need. Also parser people, remember that I reserve the right to tell small fibs to explain things to an audience that won't ever take a class about automata.)

Changing Arity

Arity can be more interesting; this is a fancy programming term which means "how many arguments a function or operator takes". Perl 5 subroutines are usually greedy; they'll happily slurp up all arguments in a big list. Certain subroutine prototypes allow you to change this. For example, you can define a foo subroutine which only takes two arguments:

sub foo ($$)
{
    return "<@_>";
}

say foo 1, 2;

The prototype changes the parsing of foo calls so that providing more than two arguments is a syntax error:

say 'Parsing done';
say foo 1, 2, 3;
$ perl protos.pl
Too many arguments for main::foo at protos.pl line 13, near "3;"
Execution of protos.pl aborted due to compilation errors.

Again, Perl 5 must have encountered the prototyped declaration before it can apply the parsing rule changes to any code. Again, a clever and stateful static parser could detect these parser changes and adapt to them.

Barewords

The Perl 5 parser has a complex heuristic for figuring out what a bareword actually means. It could be a filehandle, a class name, a function, or more. Several good Perl 5 programmers recommend against the indirect object invocation syntax (though dative is a description with more technical accuracy for linguists) because it can parse differently depending on code Perl 5 has already executed.

my $obj = new Class foo => 'bar'; # do not mimic this example

Perl 5 has heuristics to determine what new and Class mean in this context. It may be obvious to readers that this code really means:

my $obj = Class->new( foo => 'bar' );

... but if the parser has already seen a subroutine declaration for Class in this particular namespace (in certain circumstances, I believe it's possible to fool the parser with a specific way of declaring new), the heuristic may choose the wrong option.

The indirect pragma is an effective way to root out this ambiguity from your code. There are some seven cases in which you can run afoul of this bareword heuristic; you can tell who's a real Perl 5 guru by asking him or her to list at least five. See Matt S. Trout's Indirect but Still Fatal for a fuller discussion of the issues here.

A very clever, very careful static parser could still replicate the complex heuristic in the Perl 5 parser and parse code appropriately.

The Real Problem

What's the source of the ambiguous parsing issue?

If Perl 5 requires predeclaration of all parse modifications, and if they're declarative, why is writing a static parser which can parse all Perl 5 programs in the same way that the Perl 5 parser does impossible?

Perl 5's compilation phase parses Perl 5 code into a parse tree. The execution phase traverses this tree. They're two separate stages...

... except that BEGIN and implicit BEGIN blocks can run arbitrary code during the compilation phase.

That is, you can write code that will not compile successfully:

sub foo ($$)
{
    return "<@_>";
}

say foo 1, 2, 3;

... and insert a BEGIN block to change the way Perl 5 will parse subsequent code:

sub foo ($$)
{
    return "<@_>";
}

BEGIN
{
    no warnings qw( prototype redefine );
    eval 'sub foo { return "<@_>" }'
}

say foo 1, 2, 3;

Even though the eval code wouldn't normally run until after the compilation stage has finished and the execution stage has begin, the BEGIN block causes the Perl 5 parser to execute its contents immediately after its parsing has finished -- before parsing the very next line.

This is a contrived example -- why would anyone write this code? -- but you should start to see the problem.

Even though this syntax is declarative to the Perl 5 parser, a static analysis tool would have to analyze the semantic meaning of code within the BEGIN block to determine if it has any effects. That means peeking into double-quoted strings used as arguments to eval.

Even if a static parser could do that, it'd be easy to confuse it again:

BEGIN
{
    no warnings qw( prototype redefine );
    eval 'sub' . ' foo ' . '{ return "<@_>" }'
}

Even though Perl 5 can coalesce those three concatenations to a single constant string during its optimization phase, the parser must get more complex to detect this case.

You can see where this is going.

Ultimately, completely accurate static parsing of Perl 5 programs is impossible in certain cases because I could write:

BEGIN
{
    no warnings qw( prototype redefine );
    eval 'sub' . ' foo ' . '{ return "<@_>" }'
        if rand( 1 ) > 0.5;
}

The static analysis tool can't even tell if this program would parse successfully. About half the time, Perl 5 will report that it has a syntax error. Half of the time it will succeed.

Thus there exist pathological cases written by curious (and potentially malicious) people such as myself where a static Perl 5 parser cannot predict what the actual Perl 5 parser will do. If you're morbidly curious, consider changing the prototype of a symbol already parsed through raw typeglob access.

The Implications

Don't panic. Static analysis tools can't parse all valid Perl 5 programs, but they can parse many Perl 5 programs. While the Perl 5 parser is complex with many special cases, it's possible to write a clever tool such as PPI which is good enough for almost all code. Besides that, the language-bending manipulations that could trip up a static analysis tool are mostly declarative and mostly decidable, unless you're doing something beyond weird.

In most cases this doesn't matter. That example code is completely unreliable. There are ways to disambiguate potentially ambiguous code -- and good Perl 5 hackers recommend using those approaches anyway. Most languages without a very strict homoiconic representation have parsing ambiguities (and if you allow arbitrary code execution in templates or reader macros, even homoiconicity can't always save you).

Besides that, once you allow loading language extensions from languages with access to raw memory, all bets are off.

With a bit of work, I'm sure I could come up with ways to confuse static Ruby and Python parsers without resorting to C. Implicit variable declarations seems like one perilous approach....

Posted by chromatic on modern perl, perl 5, perl internals, perl parsing at Fri, 14 Aug 2009 19:52:10 -0000 | Permalink

Imagine that you need to store the following tree structure in your database:

    5---
   / \  \
  62 32  7
/  /   / \
9  3   29  1
          / \
         48  55

One way this is typically handled is by creating a table linking parents and children:

parent | child
-------+------
5      | 62
5      | 32
5      | 7
62     | 9
... and so on

Because SQL isn't brilliant at recursion (and because most of us aren't as brilliant as Joe Celko), we find ourselves writing several SQL calls to walk up the tree. However, this becomes prohibitively expensive when you have millions of objects in your tree and you need to repeatedly traverse it. Enter materialized paths.

A materialized path is a database denormalization where you flatted each branch of the tree. You join all of the parents, in order, with a separator character which is guaranteed not to be in the id. So the materialized path for 48 might be 5.7.1.48.

node | path
-----+---------
1    | 5.7.1
3    | 5.62.3
5    | 5
7    | 7.32
9    | 5.62.9
29   | 5.7.29
32   | 5.32
48   | 5.7.1.48
55   | 5.7.1.55
62   | 5.62

They can be tricky to maintain, but if you get them right, you can find all parents IDs in a single SQL statement.

Today I ran into a nasty problem where I needed the parents, but I had to guarantee that the order of those parents was preserved at the database level (via ORDER BY). The SQL looks sort of like this:

SELECT *
FROM  tree
WHERE id IN (5,7,1,48)
ORDER BY ...

Um, what goes in that order by? I need the items returned in the same order as the ids in the IN ids. I asked many people and they were stumped. They came up with Oracle specific answers. Some suggested writing the ids to a temp table with sort criteria and joining on that. Others through up their hands. One person suggested a complicated case statement.

Then one of my colleagues, Mark Morgan, said "can't you just ..."

The solution was rather counter-intuitive to me, but once I saw it, I was dumbfounded in its simplicity. Do you see it?

Update: I've posted the solution in the comments, so if you want to think about it a bit first, don't read 'em.

Posted by Ovid on journal at Fri, 14 Aug 2009 15:07:26 -0000 | Permalink

Posted by John Napiorkowski on perl mac cpan uri at Fri, 14 Aug 2009 13:48:24 -0000 | Permalink

Dear Lazyweb, I can't figure this out. I'd like to do something like this:

perl -MPIPs -de '$s = PIPs->new->model("PIPs")->schema; $DB::single = 1'

That would theoretically enter the debugger and I could just hit 'c' and continue to that breakpoint, have my variable set and start debugging my app. Instead, I have the following nasty alias:

alias debug_pips='echo "\$s = PIPs->new->model('PIPs')->schema";perl -MPIPs -de1'

And I just cut-n-paste the echoed line into the debugger. Is there a better way?

Posted by Ovid on journal at Fri, 14 Aug 2009 13:02:14 -0000 | Permalink

(a very off-topic SQL exercise follows...)

Apple's iPhoto 2009 has a very cool face recognition feature where it will show you faces and prompt you to enter names. When it finds resemblances, it will suggest names for you to confirm or reject.

When it's wrong, it makes for funny little screenshots you can send to friends: "Haha, cute girl! You look like an older heavier man!"

But once you reject the match, you can't undo or get it back in any other way. Luckily, iPhoto stores ALL of its metadata in a collection of SQLite databases. So, in the particular case that I linked above I was able to undo by telling the database not to ignore the marked mismatches from IMG_0095.JPG:

% osascript -e 'tell application "iPhoto" to quit'
% cd "Pictures/iPhoto Library"
% sqlite3 iPhotoMain.db
sqlite> select photoKey,relativePath from SqFileImage,SqFileInfo where sqFileInfo=SqFileInfo.primaryKey and relativePath like '%IMG_0095.JPG';
110|Data/2003/Roll 15/IMG_0095.jpg
110|Originals/2003/Roll 15/IMG_0095.JPG
3666|Data/2006/Apr 25, 2006/IMG_0095.jpg
3666|Originals/2006/Apr 25, 2006/IMG_0095.JPG
sqlite> .exit
% sqlite3 face.db
sqlite> update similar_faces set ignore=0 where image_key=110;
sqlite> .exit
% open -a iPhoto.app

Posted by ChrisDolan on journal at Fri, 14 Aug 2009 02:33:23 -0000 | Permalink

A few years ago I started perl101.org. My tag line was "What every Perl programmer should know," and the site would cover a wide range of basic topics.

The goal is not to cover the topics in depth, but give just enough of a taste to give the reader an idea of how to do something, and enough to go off of to investigate further. It's an FAQ of sorts, but of techniques.

The idea came from having the same questions asked over and over again in IRC.

• How do I remove elements from the middle of an array?
• How do write something to fetch web pages?
• How do I write a regex that doesn't match something?

I also want to show examples of the Right Way to do things, like using chomp instead of s/\n$//. I wanted to tell people about tools like perldoc that I know many programmers don't even know exist.

Here's where you come in

I haven't done much with perl101 for a while, but I'd like to get it going again, fully awesome. I've put the code on github and I'm asking for contributions. Please pull a fork, add content to it, and let me know when your fork is ready to merge back to the master and I can republish. I'm specifically not making it a wiki, because I want to maintain editorial control to help keep it focused.

The site is also pretty ugly, and if you have better ideas for getting the content out there in an attractive way, I'd love to hear it.

There's also a perl101-talk mailing list and I welcome your participation there as well.

Let's put our collective heads together to help out newcomers to Perl!

Posted by Andy Lester on Perl 5 at Thu, 13 Aug 2009 17:48:28 -0000 | Permalink

As I mentioned yesterday, I now have a new shiny analysis toy in the form of downstream dependency graphing.

So now we can tackle some issues that have been hard to get traction on until now.

The first one I'd like your help dealing with is some fallout from Kwalitee and Perl::Critic, unintended consequences of superficially positive metrics targeted by contestants in a game without access to clear best practices resulting in damage to the CPAN graph.

(Don't get me wrong, I'm a huge supported of CPANTS and Dom did a lot to help me with it, including creating the original version of the awesome red and green box author page basically to my spec. This is just about two particular metrics)

The two big problems are caused by Test::Pod and Test::Pod::Coverage.

It's not so much the use of them at all, since they are quite safe to use when applied with author test best practices.

It's that many (alas, very many) people used these modules in ways that made their use compulsory, imposing hard dependencies instead of soft dependencies (only running them if already installs).

Test::Pod imposes a Perl 5.8 dependency, blowing any chance at back-compatibility for anything that uses it.

And Test::Pod::Coverage has a history of failing to get to 100% PASS rates, the current release has 30 odd failures.

So, what's the shape of the problem we've got here?

Let's start with Test::Pod::Coverage.

http://svn.ali.as/graph/Test-Pod-Coverage

(Don't forget to use CTRL-minus to zoom out) :)

As you can see, there are a couple of large nasty tangles of dependencies, and a number of high-dependency explosions.

And even though Test::Pod::Coverage has 650 dependencies (resulting in 18,500 FAILure points) it's obvious from the graph that several hundred dependencies can be removed by fixing just three modules

    Test::Compile
    Module::Find
    WWW::Search

There's one more critical high-dependency node there (Test::Distribution) but as a Test:: module depending on another Test:: distribution, it's far more likely it is a legitimate dependency.

But even without that last one, we've got some very juicy low-hanging fruit to go after here. And once these three have been fixed, we can rerun the graphs and see if anything else stands out.

So now we move on to Test::Pod, which is bigger again.

http://svn.ali.as/graph/Test-Pod

As with Test::Pod::Coverage, the three previous modules (Test::Compile, Module::Find and WWW::Search) show up again.

And this graph also adds some more interesting targets.

    MooseX::Types::Structured
    MIME::Types
    MailTools
    Test::Strict

So that's no more than around 7 modules that we need to tweak and rerelease to unwind half or more of the dependencies, and reduce the dependency weight of many of the Heavy 100 by 5 dependencies each.

By doing this reduction of false dependencies, we reduce the time it takes to install CPAN software, and reduce false failures due to failing dependency modules for things that never should have been dependencies at all.

Posted by Alias on journal at Thu, 13 Aug 2009 17:47:17 -0000 | Permalink

One of the ways to ease the learning of Perl for those the none English speakers is to have translated documentations and tutorials for them. Even if they know English it is easier if they don't have to learn something in a foreign language. We already have some of the Perl documentation translated to a few languages but IMHO they have not received enough attention.

So let me announce the first version of perldoc.org.

I'd like to thank Tiago Stock, the owner of that domain who let me use it for the advancement of Perl.

My plan is to help the people who would like to have a translated version of the Perl documentation available on the web.

Currently there are two sites the French and the Italian version.

The site is built using the code that John Allen wrote for the previous version of perldoc.perl.org and released more than half a year ago.

Still many features of the original site are missing but I did not want to wait more with the announcement. I hope some people will be interesed and start to improve the code so we can have all the features back soon. The code is now on Github. Please fork it.

The documentations themselves are maintained by other people. I am just using the packages released in the POD2::* namespace.

Posted at Thu, 13 Aug 2009 09:30:25 -0700 | Permalink

Posted by draegtun on Programming, OOP, perl at Thu, 13 Aug 2009 14:40:32 -0000 | Permalink

The Portuguese team did a fantastic job with this year's Perl conference YAPC::EU::2009 in Lisbon just last week.

What I learnt

• Lots of companies -- Cisco, Opera, the canton of Geneva, among others -- are proud to use Perl.
• There is lots of work going on in the "Enlightened" Perl movement. Including ways to make deployment and packaging of Perl apps easier. You can join the EPI for a mere GBP 100/year (whether you're UK based or not) and can pay by regular direct debit.
• Perl is "Alive and kicking, and stronger than ever". (OK, that's marketing speak. But Barbie's talk on the statistics of Perl backs it up: for example there are between 30 and 50 new authors uploading their first module to the CPAN every month!)
• As always, the hallway track is one of the best parts of any conference - the chance to meet up with old friends and colleagues, and make new contacts with the very clever people in the Perl community and find out what they're doing.

Cool modules mentioned included:

• Moose, MooseX::Declare, etc.
• TryCatch
• XML::Pastor
• autodie
• autobox, and MooseX::Autobox
• Regexp::Grammars (very cool indeed, from the Damian - Perl6 like rules, in Perl 5)

Logistics

Joel++ booked a flat (via waytostay.com) for 5 of us. This was an excellent idea, reasonably priced, and a beautiful flat, great for socializing in. Highly recommended, especially for a group of friends/colleagues.

Organizers

The Lisbon team did a fantastic job! Among their several innovations, were such delights as:

• Coffee/lunch all the time: this was very bare bones, but none the worse for it. There was food any time you wanted it, which meant that you never had to worry about leaving a talk late etc. This was especially useful given that the campus wasn't right next to large restaurants. (But even in central venues, if 300 people go out to lunch, they will be gone for 2 hours, realistically).
• The Quizz show, hosted by Damian Conway at the conference dinner. (One of the orgas)++ had hacked some PS2 games controllers, and great fun was had by all. (Me and Polettix came 2nd, but the important thing is that we answered a Buffy question first, to the horror of davorg and Greg).
• Moderators in the auditorium (Wendy++ and Liz++).
• "Special" namebadges with an information booklet inside.
• sapo.pt sponsored a lovely chillout area with wonderfully comfortable beanbags.

They set a worryingly high standard especially as...

YAPC::EU::2010 will be in Pisa

I seem to have got myself involved in this... but it's going to be great. Look forward to seeing you all there!

Functional Pe(a)rls and Foose

I gave the fourth version of my Functional Pe(a)rls talk, which went down quite well. Thanks to everyone who showed up, especially for putting up with me being late...

I've been apologizing for some time for not having actually sat down and released modules for all the techniques I'm playing with here. But James Laver finally pestered me enough, and we've created a new project: Foose (working title, in homage to Moose) which will aim to bring neatly packaged functional goodness to Perl. (See also irc:irc.perl.org/#foose)

Moving on

I attended YAPC in the last week of employment with Thermeon Europe, who kindly paid for my time there. It's been a crazy 2 years (almost), and I'll miss the guys there, but I'm excited to be moving on to consulting, initially on some Perl development and documentation projects.

If you're interested by anything you've read in this blog and would like to hire me, then please do get in touch ;-) I'm mainly interested in teleworking or contracts in Northwest UK, but happy to discuss!

Posted by osfameron on perl, foose, yapc, yapceu2009 at Thu, 13 Aug 2009 15:32:51 +0100 | Permalink

xfce4 is a lightweight (compared to Gnome and KDE) which I happen to love, as it still provides nice theming, a well-done panel system and some interesting goodies. The window manager (xfwm4) is not handy as others. xmonad is, however, a great one, in particular: tiling, highly customizable and... written and extendable in Haskell!!! So, it makes up for a good replacement. I decided to change xfce's window manager and use xmonad. The thing required a few steps in order to begin working properly. Here's a tutorial which presumes you use the following software versions: xfce 4.6.1 xmonad 0.8.1 gmrun 0.9.2 dmenu 4.0 This is a screenshot of how it's going to look like. I also assume you have a working xfce4 installation and you're able to login to the xfce desktop with an account (by the way, I suggest slim as login manager). First of all, you need to emerge xmonad (after unmasking it in /etc/package.keywords): emerge -av xmonad xmonad-contrib xfce configuration The first thing is to properly configure xfce. My suggestion is to open the Desktop settings dialog box and, in the Icons tab, uncheck all default icons: it's a tiling window manager, and having icons around will only leave a box containing them which is going to look weird. You're not migrating to a tiling window manager to use desktop icons, aren't you? ;-) Switching from xfwm4 to xmonad is a bit tricky, as xfce4 doesn't support (like Gnome and KDE do) configuring the window manager at all, and xmonad doesn't support the session protocol (so we must ensure that xmonad is started every time we log in). So, just login with your account, open a terminal and then terminate xfvwm4 while launching xmonad: kill `pidof xfvm4` && xmonad & The window manage should be replacing (you'll notice your windows tiling). At this point exit xfce4 saving the session: this will ensure xmonad is started again when you log back in. Problem is xfce will also try to start xfvwm4: to avoid this you have to locate you session file in ~/.cache/sessions and remove the entry for xfvwm4. At this point you should be able to login again and xmonad will be your new default window manager. xmonad configuration xmonad needs a some tweaking to play well with xfce, and also to be more usable on its own. Looking into you ~/.xmonad directory you'll find a xmonad.hs file which is used to compile xmonad (the xmonad-x86_64-linux binary in the same directory, with the name changing depending on your arch) when starting it for the first time. The configuration file is in Haskell, so it might look a bit weird if you haven't seen the language before. The instruction given below might vary if you xmonad.hs is a bit different, or if you already customized it somehow. First of all, we need to ensure xmonad properly manages the xfce4 panels (I have one, at the bottom), by placing them where you expect them to be, by making them visible in all workspaces and by avoid application windows to overlap them. Add this line after the other import ones: import XMonad.Hooks.ManageDocks Locate the: myLayout = tiled ||| Mirror tiled ||| Full and change it to: myLayout = avoidStruts(tiled ||| Mirror tiled ||| Full) Then locate: myManageHook = composeAll [ className =? "MPlayer" --> doFloat , className =? "Gimp" --> doFloat , resource =? "desktop_window" --> doIgnore , resource =? "kdesktop" --> doIgnore ] and replace it with the following code. It's actually a small changes but I chose to create an extra variable for the sake of clarity. stockManageHook = composeAll [ className =? "MPlayer" --> doFloat , className =? "Gimp" --> doFloat , resource =? "desktop_window" --> doIgnore , resource =? "kdesktop" --> doIgnore ] myManageHook = manageDocks <+> stockManageHook Now that we ensured the panels get a proper placement, we'd better change the default xmonad mod key: the default left Alt conflicts with the shortcuts of many applications, including xfce terminal, firefox, and others. Luckily, we have an otherwise almost useless "Windows start" key on the left of the keyboard; to use that as mod key find this line: myModMask = mod1Mask and change it to: myModMask = mod4Mask I suggest you delete the binary inside the .xmonad directory before logging back in, in order to ensure it's compiled again. additional software In its default configuration, xmonad provides key binding for a couple of useful softwares. gmrun (mod+shift+p) opens a small text box where you can enter the name of a command to launch. You can also use xfce's own "Run program..." feature of course. dmenu(mod+p) opens a nice menu bar (at the top by default) which lists what you have in /usr/bin (by default) and can be used to launch programs by just typing a few letters and navigating the list. Very handy. Well, that's it: you should have everything working now. Exercise your hands and fingers for xmonad. ;-)...

Posted by Michele Beltrame on gentoo, linux, xfce, xfce4, xmonad at Thu, 13 Aug 2009 09:46:05 -0000 | Permalink

Posted by claudio on Perl, Solaris, UNIX, Solaris 10, Mysql, DBD::mysql at Thu, 13 Aug 2009 06:04:03 -0000 | Permalink

One of the suggested objectives in Gabor Szabo's Measurable objectives for the Perl ecosystem is "Increase the number of Perl book sales". I like most of the objectives in Gabor's post, but I must caution against taking the numbers presented too seriously. At best, they're incomplete. At worse, they're completely misleading.

Gabor refers to State of the Computer Book Market - Mid-Year 2009, written by Mike Hendrickson. Mike's company publishes similar analyses a few times a year, based on sales data from Nielsen Bookscan. (For more on the culture of Bookscan rankings in the publishing world, see Why writers never reveal how many books their buddies have sold.)

This data sounds wonderful and the pretty graphs and charts give you the impression that you're getting useful information. Yet this is only a picture of the market. As Mike writes later in the piece:

Many publishers report that more than 50% of their revenue is achieved as direct sales, and those numbers do not get reported into Bookscan. Sales at traditional college bookstores are typically not reported into Bookscan as well. Again this is US Retail Sales data recorded at the point of sale to a consumer.

These numbers reflect less than half of revenue. Throw out half of sales by dollar and hope that the results are stochastic.

Yet there's a deeper flaw behind these numbers.

How Book Sales Work

If you plot the sales curve of multiple books, you'll notice that they tend to follow the ubiquitous power law. A book sells as many copies in its first three months as it will the rest of the first year. A book sells half as many copies in the second year as it did in the first. This model is so accurate that the publishing industry calls titles "frontlist" titles if they're in their first year of publishing and "backlist" titles if they're not.

While a few titles have strong backlist sales, they're rare. They're the Bibles and Harry Potters and How to Win Friends and Influence Peoples. The publishing industry's san greal is to find a new strong backlist bestseller.

They tend to exhibit strong frontlist behavior as well.

The retailer's point of view is different. Limited shelf space means that new books still in their three-six-twelve month short snout sales levels often get priority over older books in the long tail sales levels. If you're going to see 3000 copies of a book in the first three months and 1000 copies in the next three years, stock up early.

This is especially true in technical book publishing, where I have trouble giving away Python 2.3 and Oracle 7 and Red Hat Linux 6 books. Publishing dates are expiration dates: best by a year after the copyright date.

Why does this matter? It's a simple matter of economics: people won't buy books you don't publish.

The Freshness Factor

2005 was a good year for Perl book sales. Why? Four strong Perl books came out in 2005. The Perl book sales numbers for that year reflected the short snout of Perl book sales.

Four years later, is PBP selling as many copies? Is the Perl Testing book? Is HOP? Is APP 2e?

Those are rhetorical questions. You already know the answer. You can even answer that question for the Camel 3e. A book published in 2000 may still be useful nine years later, but Camel 3e predates almost every part of the Perl Renaissance. Besides that, the 250k or 300k units already sold have reached a fair amount of the Perl 5 programming market.

Compare that with the Ruby book market in 2006, where you couldn't leave an editorial board meeting without an assignment to publish a new Ruby or Rails book. Initial sales numbers looked great; the growth in that market segment was huge!

Did any Ruby book sell 250k copies, though? That number's missing from the year-by-year analysis.

Look at this year's numbers. Objective-C is huge! It's 1999 all over again! Except that, yet again, the comparison is to an emerging market segment without analysis of historical trends.

The Missing Data

The biggest piece of data obviously missing from these State of the Computer Book Market entries is historical context. Six months or a year of appositional data comparing different market segment maturities is misleading, at beast. Should you go learn Objective-C just because Bookscan reported more Objective-C titles sold than SQL?

No -- but to be fair, Mike doesn't suggest this directly.

Other missing data is more subtle, and perhaps more meaningful. Where's the breakdown of frontlist/backlist for these sales figures? More than nine out of ten books follow the power law I described earlier. If the Objective-C books have all come out in the past year, they're in their short snout period. Of course they're selling more units now than books in the long tail period.

How many total units does the market represent? If the number of books sold in 2009 is half the number sold in 2008, it's difficult to compare the performance of books against each other year-over-year. There are too many other factors to consider. (You can still get interesting information, but you can't compare technologies against each other in meaningful appositive ways.)

How many books are in each category? Title efficiency (average number of unit sales per title and standard deviation) can tell other interesting stories. Is one language category hit driven (iPhone Programming, Ruby on Rails)? Are there niche subjects intended as modest sales targets and not bestsellers? Is every book a moderate success, with no breakout quintessential must-have tome? Is there a gold rush of publishing with 40 new titles produced in a year and each of them selling a dismal 1000 copies apiece?

How many new books are in a market segment this year compared to last year? This is the biggest question that matters to Perl books, especially with regard to Gabor's suggestion. Again, this should be obvious: no one can buy Camel 4e right now.

A Completely Hypothetical Fictional Example I Made Up Completely From Whole Cloth

If that didn't convince you, consider a short fable about oyster farming.

Suppose you own a publishing company. Suppose you discover a new topic area: oyster farming. No one's published on this topic before, but hundreds of thousands of people are doing it. There's a lot of institutional knowledge, but there's a ripe opportunity for documenting best practices and nuanced techniques -- especially given that you have found the person who invented modern oyster farming and convinced him to write a book about it.

You publish the book. It takes off. Its short snout is wide. (My metaphor is awkward.) You've discovered a new market segment; you've invented a new market segment. Life is grand.

You branch out. You publish More Oyster Farming and Learn to Farm Oysters and Pteriidae, Reefs, Bivalves, and Mollusks. You even write a cookbook for Oysters.

Then a catastrophic triploid spawning accident removes the long-beloved MSX resistance in most commercial oyster farms, ruining the market for a year -- maybe longer -- and in a panic you cancel all of your upcoming frontlist titles.

A few other publishers publish one- or two-off titles in the market segment. They sell a few copies. You had a corner on the market though. You were the publishing world's China of oyster farming. Over the next four years, you look at your sales numbers and congratulate yourself for getting out of the oyster farming publishing market segment when you did, because no one's buying oyster farming books anymore.

After all, publishing one frontlist title per year is obviously a sign you take the oyster farming market seriously and want to see it continue.

Posted by chromatic on modern perl, perl, perl books, perl marketing, perl programming at Wed, 12 Aug 2009 21:32:18 -0000 | Permalink

After less hacking than I'd originally thought, I've gotten a reasonable reverse of my dependency graphs working now in CPANDB.

So, I'm proud to present my favourite example.

The Universe of POE

Note that it is somewhat larger than new kid on the block The Universe of AnyEvent

Posted by Alias on journal at Wed, 12 Aug 2009 19:17:44 -0000 | Permalink

Lately my pet project has been implementing delimited continuations for Perl. In an effort to keep myself motivated (it's hard and frustrating), I'll try and explain what I'm actually trying to do.

Caveat lector, the repository is nasty. There is no coherent process in the versioned history, it's just git commit -a at the end of the day, so the repository will be breaking. Secondly, I don't really know what I'm doing. I do in principle, but the details are still vague and I am discovering them as I go along.

So, WTF are delimited continuations? As Perl programmers the easiest way to understand them is probably Tom Moertel's favourite explanation, by Oleg Kiselyov.

As core hackers, the easiest way to understand them is to see how they are implemented. This is what I will stick with.

But really, the best way to understand continuations is to use a language that actually has them, and play around a bit. It's a strange concept to get used to. Unlambda, the INTERCAL of functional programming uses continuations precisely because they can be difficult to understand (though technically they aren't delimited).

Continuations for Core Hackers

Delimited continuations are structured around two operations, reset and shift.

reset is the delimiter, it saves a position in the call chain for a future call to shift.

shift is what actually creates the continuation. It does that by walking up the call chain from the current position, up to the reset, and saving everything in between into the continuation. The continuation itself is reified as a function, and is passed as an argument to the block given to shift. Hold that thought, we'll get back to that in a second.

This interaction between shift and reset is much like the one between die and eval. eval delimits the stack, and die is what's called an escape continuation:

eval {
    warn "reached";
    die $value;
    warn "not reached";
};

return $@;

The die causes the code in the eval to be aborted, and sets the special variable $@ to $value. With shift and reset this could be expressed a little more elegantly as:

return cont_reset {
    warn "reached";
    cont_shift { return $value };
    warn "not reached";
};

What happens in shift is similar to die. The stack is unwound all the way up to the begining of reset. The difference is that the return value of the shift block becomes the return value of the reset block.

Scope::Upper makes this sort of abstraction possible already, as demonstrated by Continuation::Escape.

Capturing a Continuation

Semantically eval/die and delimtied continuations are actually very different. shift doesn't unwind the stack, but stashes the frames into a data structure.

Delimited.xs introduces two structs, delim_t and cont_t.

delim_t is very trivial, it contains a snapshot of the various state variables in the Perl interpreter, like the stack pointers, the current opcode, etc. When we call cont_reset a new delimiter is pushed onto a linked list.

Inside cont_shift the init_cont function will create another delimiter, and then destructively move all the stack frames between the two delimiters into a cont_t. When it's done current state of the interpreter has effectively been rewound to that of the start delimiter, but unlike die none of the data has been cleaned up, those stack frames are still "live", just not visible from the current state of the interpreter.

Reifying a Continuation

The restore_cont function takes a cont_t and appends copies of all the captured stack frames inside it to the current state of the interpreter. There are many things to fix up, and that is where the bulk of the implementation lies.

At the end of the restore_cont the next step for the interpreter is to resume execution right after the call to cont_shift, the end of the continuation capture.

The difference is when those stack frames are evaluated and we reach the end of the cont_reset block, instead of returning to the caller of cont_reset we return to the caller of restore_cont. To do this we overwrite the top level PERL_CONTEXT stack frame to use the current PL_op as the retop, instead of the one it was originally created with.

So how does restore_cont get called? We wrap the cont_t in a closure, that gets invoked as a function. Here is an example of no-op continuation usage:

my $value = cont_reset {
    return 3 + cont_shift {
        my $k = shift;
        $k->(7);
    };
};

is( $value, 10 );

This is no-op because cont_shift unwinds the stack, and gets the continuation in $k, and then immediately reinvokes it, restoring everything and causing cont_shift to return a value of 7 from cont_reset. This code is functionally equivalent to:

my $value = cont_reset {
    return 3 + 7;
};

Which is really the same as

my $value = sub { return 3 + 7 }->();

But $k is more than that, it's literally a function that appends a bunch of things to the interpreter's stacks. You can call it multiple times:

my $add_three = cont_reset {
    return 3 + cont_shift {
        my $k = shift;
        return $k;
    }
};

In this example cont_shift returns $k instead of invoking it. Since the stack has been unwound, this causes cont_reset to return $k.

So what happens when you invoke $add_three from the outside? The stack frames from reset until shift are appended, so the interpreter state is waiting for a value from shift to add 3 to.

This means that you can call $add_three->(7) and get back a value of 10, any number of times. In fact, this is what the basic sanity test does.

Gory Details

Unfortunately what goes on behind the scenes isn't that simple (almost nothing is when it comes to Perl internals).

delim_t is initialized by init_delim to capture relative offsets for PL_stack_sp, PL_markstack_ptr, and the values of cxstack_ix, PL_scopestack_ix, PL_savestack_ix and PL_tmps_ix (which are relative offsets anyway). This information captures all of the stack states.

In addition it keeps track of a number of variables (like the current opcode, PL_op, the current pad, PL_comppad, and another of other variables corresponding to the current state.

When we init_cont the start and end delim_ts are used to capture state in the cont_t using several functions.

init_cont_stack allocates buffers for the mark stack and value stack. All the stacked SVs are put into an AV, increasing their reference counts, and the marks are converted to 0 based offsets, based on the value of PL_stack_sp at the start delimiter.

init_cont_cxs captures Perl's context stack. This stack contains PERL_CONTEXT structures which contain all the information about the call chain as well as loops and other blocks requiring data.

In Perl the actual storage for lexical variables is not allocated on the context stack (though it probably should be), but instead its appended to CvPADLIST(cv), which is an array of arrays stored in the CV's data directly (this allows storage space to be cleared instead of freed, so repeated calls to the same subroutine are slightly faster).

The context stack is walked from the end to the begining, and any CXt_SUB, which denotes a subroutine call, causes the lexical pad instance of that subroutine to be popped off of CvPADLIST(cv) and stored in a pads AV in the continuation. CvDEPTH(cv) is adjusted appropriately.

The contexts also contain values like cx->blk_oldsp, previous offsets of the various stack pointers, and these, like the mark stack, are converted to be relative to the start delimiter.

init_cont_saves handles the save stack, and the associated scopestack (which is like the markstack of the savestack). The savestack is used to roll back the interpreter state when leaving a scope.

Any operation that would need to be undone is handled using the savestack, and dispatched using the ENTER and LEAVE macros.

our $foo = 1;
{ # calls ENTER
    local $foo = 3;
} # calls LEAVE

ENTER pushes PL_savestack_ix onto the scope stack. This marks the offset of the savestack.

local $foo pushes a save entry onto the savestack with the previous value of $foo, which is the IV 1.

When the scope that called local is exited the LEAVE macro pops an entry from the scope stack, and then calls leave_scope (from scope.c) to dispatch all the SAVEt_* entries between PL_savestack_ix and the value of PL_savestack_ix popped off the scope stack, causing $foo to be set back to 1.

There are many types of save entries which are crucial to properly managing Perl's data structures and runtime semantics. For instance SAVEDESTRUCTOR_X(function, pointer) can be used to automatically call function on pointer when the current scope is left, and is very useful for managing data from XS modules.

To properly handle the semantics of this stack unwinding we have to partition the savestack into entries that should be recreated every time restore_cont is called (and therefore called zero or more times), and those which should be recreated when the continuation is garbage collected (and therefore only called once).

The entries that should be recreated every time are those that keep track of the currently active lexical pad (SAVEt_COMPPAD), those that keep track of stack pointers (SAVEt_STACK_POS and SAVEt_STACK_CXPOS), and those that clear lexical variables (SAVEt_CLEARSV, pushed to the savestack by pad_sv opcodes when assigning the initial value to a lexical variable).

Localization entries will be handled in the future, but are currently deferred.

Lastly, init_cont_state resets the state of the interpreter to the start delimiter and detaches the two delimiters from the linked list.

restore_cont

Now that we have a detached continuation in a cont_t, appending its contents to the running interpreter is what remains.

The process of invoking a continuation is pretty much the inverse of the init_cont functions. All 0 based offsets are added to the current values of the stack counters, the cont->end delim_t is used to set the values of the interpreter variables, etc.

The main things that are done in addition is cloning of pads (and other state), and fixups at the top level context entry.

Pad cloning is done to ensure that each recreated instance of the continuation has its own copies of the lexical variables it uses. Without this the repeated invocations of the continuations will get in each others' way. Unfortunately this is pretty tricky, we have to walk the context stack top down and clone various values, keeping the addresses in a pointer table so that everything points back at the right thing. This is the most broken bit of code right now.

Similarly, SAVEt_COMPPAD entries have to be mapped as well, in order to reset the interpreter to the cloned pads, not the pads captured in cont_t.

Though not implemented, faked @_ arrays would need to be recreated with the duplicated value stack, and things like loop counters for foreach blocks also need more intricate copying.

Finally, the top level PERL_CONTEXT that was copied has to have its retop set to PL_op->op_next, where PL_op is the one making the call to the reified continuation. This causes the cont_reset block to return to where the caller invoked the continuation, properly connecting the call chain.

TODO

Tests are failing. Shit is segfaulting. Sanity is fading. But it works, sort of. This is a proof of concept, and now I have to finish off the various details to make it into a usable module.

If you'd like to help and have no core-fu, feel free to write failing tests. Anything you'd like to do with delimited continuations should be tests.

If you'd like to help and do have core-fu please review my horrible C code, point out obvious mistakes, or even implement some the missing bits.

I've been previously told that continuations aren't possible in Perl 5, but I think this has been demonstrated to be a false. Anyway, $title->().

Posted by nothingmuch on functional programming, internals, c, perl, continuations at Wed, 12 Aug 2009 19:15:00 -0000 | Permalink

I'm happy to report that the protection phase of security operations on the CPAN has been completed, taken in response to evidence of significant password sharing amongst the publicly exposed subset of the stolen passwords.

To prevent the need for resetting passwords for all users, Andreas Koenig has instead obtained the entire set of 51,105 PerlMonks passwords, hashed, against the entire set of all 4,096 possible salts.

After checking for all 209,326,080 possible colliding password hashes, we have locked out 767 CPAN authors who either shared their password with PerlMonks or co-incidentally shared a password with a different PerlMonks user.

An explanation of the situation and a password reset keys for these accounts have been generated and sent to both the public cpanid@cpan.org address and secret email address for these users. Where an address has been changed, we have also sent the password reset key to their secret email address as it existed at a time before the original break in.

With the exposed password database now rendered unusable in any future attacks, we can now move into an auditing phase.

This will include a number of steps including secondary checks on authors with suspicious changes in details during the potential attack window, and checks on any suspicious uploads during the attack window.

Because no suspicious uploads or reports of account hijacking have been received at this time (and because the auditing task is somewhat larger than the protection task) we will be doing this gradually over the next several weeks.

I will report in again once this task is completed.

I'd like to thank James Mastros and the other PerlMonks admins for producing and providing access to the (rather large) hash list for us.

Posted by Alias on journal at Wed, 12 Aug 2009 14:15:41 -0000 | Permalink

As I already wrote about it, there was this annoying question on the Marketing BOF on YAPC::EU on how can we measure success of the promotional or marketing project we are planning to do.

It is still unclear to me what are the objectives of others and I am sure some people will even say, that Perl does not need any promotion but the presence of those who came to the Marketing BOF confirmed that I am far from being alone in my thinking that Perl needs promotion.

I think I am also on the ambitious end of those who think we need more marketing. Some would say I am just a dreamer. So let me do just that here.

We need some measurable objectives and a time frame to achieve those. I mentioned some of the general objectives in my previous post, let me try to describe a few specific and mostly measurable objectives along with a target date. We can of course add more such objectives and discuss these too and set all of them to some reasonable value.

As we cannot really be sure which of the following is achievable at all IMHO we should say that by a certain date we would like to achive some of these objectives.

Let's say by end of 2012 we would like to achieve at least 3 out of the following targets:

Perl as an approved corporate programming language

• Turn Perl into an approved corporate programming language by 10% of the Fortune 1000 companies
  I wonder how many of those companies have a list of approved languages and which are those languages. Before we can actually set an objective like this we should try to find out this and also try to find out how we can check this. To make it clear, this means that Perl has entered in the short list of supported and recommended languages of the organization. To refine this we could say 10% of those that have any approved languages list.
• The same with the Fortune Global 500 so it will be clear our efforts are global.

Increase number of job posts

• Double the number of jobs as listed on JobStats.
  Currently I see the following data:

  Skill	30 days	90 days	1 year
  Java All Jobs	5,430	15,502	84,736
  PHP All Jobs	2,150	6,395	35,772
  Perl All Jobs	1,342	3,859	22,229
  Python All Jobs	666	1,841	9,134
  Ruby All Jobs	333	1,023	4,871

  Doubling these numbers for Perl would be a nice achievement.
• Increase the number of job posts on dice.com that have "perl programming" in their description. I could not find the data for the baseline.
• Increase the monthly number of job posts on jobs.perl.org to be 500 per month This is three times as many as the peak was and about 8-9 times of the current numbers. See the stats.
  The increase here can come from both an increased number of overall perl jobs or by more companies and recruiters knowing about this site. Someone could try to "game" the system by contacting all the companies looking for a Perl programmer and suggesting them to post on this site as well. I would say such a behavior is most welcome. That would be part of the promotion of Perl to make sure companies ar aware of the services the community can offer them.

Increase the number of Perl book sales

• Increase the number of Perl book sales to be the number 5 group in the O'Reilly report. The latest report from July 2009 indicated a decline in the whole market but I think I saw real numbers on in an earlier report from February 2009 Based on that I have this slightly rounded data on 1000 units sold for some of the languages I found relevant.

  Language	2008	2007
  C#	272	232
  Java	211	242
  PHP	173	159
  Visual Basic	72	100
  Ruby	61	96
  Python	60	46
  Perl	29	38
  Lua	11	2

  As an objective we can say that we would like to see the number of Perl books sold to reach 60,000.

Get some major web-sites proudly display a "using Perl" logo

• Have 15 of the top 500 websites (as listed by Alexa) proudly display a "using Perl" logo.
  For this first of all we need to figure out which companies are making heavy use of Perl that are listed among the top 500 websites. Then we should contact them and encourage them to display a logo. Of course they might want to have a nice logo to display and they might want to link to somewhere, so we have to make sure there is a web-site they would be ready to link to. We could use this opportunity to try to understand why are they reluctant to put up a link - if that is the case - and see if we can change that.

Increase traffic on major Perl web sites

• Looking at Alexa again I searched for perl and found the following numbers:

  Rank	Site	Comment
  9,605	cpan.org	92% search.cpan.org, 5.4% cpan.org, 2% kobesearch.cpan.org
  18,458	perl.org	(50% went to perldoc.perl.org 18% went to perl.org see)
  21,231	perlmonks.org
  38,831	perl.com
  273,104	perlfoundation.org

  In order to compare I searched for a couple of other keywords and found these:

  Rank	Site	Comment
  931	php.net
  975	java.com
  6,736	python.org
  7.095	php.de
  11,329	rubyonrails.org
  19,007	selfphp.de
  30,929	ruby-lang.org
  70,586	diveintopython.org
  74,125	php.ru

  I am sure we have to invest more energy in locating other representative web sites and in analysing this data. We then should set a target ranking for the perl related web sites. Getting two perl related web sites in the top 5000 sites should be a good target.

Increase traffic on IRC channels, web forums and mailing lists, blogging sites

• I have not looked for data on this, someone should see if we can have statistics on number of posts, number of (new) users, number of threads etc.

Publish at least 100 articles about Perl

• Publish at least 100 articles about Perl in important magazines and on important web sites. This might be more of a tool in our quest than an objective and I don't heave a ready made list of important journals but I think we can easily put together an initial list and we can measure the number of articles.

Improve the perception of Perl

• I guess if Ovid and others can manage to run a market research now that will help us analyse the state of the market and the perception of the world of Perl outside the community we could set some targets based on those data. The achievements then could me measuered later by running a similar research again a few years from now.

Increase the number of contributors to the Perl world

• During YAPC::EU we saw some encouraging numbers in terms of number of new CPAN uploads and number of new authors. (See the blog post of Mark Keating about Perl is alive, kicking and stronger than ever!
  That seems to mean that there is disconnect between what the people in the Perl community do and what happens outside of the community, in the more general user-base.
  Ohloh indicates that number of Perl projects and the number of commits are increasing but their percentages are sharply dropping.
  I wonder if we can set some objecteves in either of these metrics too?

No conclusion yet

We need to invest more energy in finding metrics that we think can be interesting and we should decide on some objectives that are reasonable to reach in a few years.

Posted at Wed, 12 Aug 2009 04:04:20 -0700 | Permalink

Clearly, with an inflammatory subject like that you're probably expecting a false question, and I quickly in my journal post say "Yes!" or "No!".

I am, clearly, not pro-git. But I'm actually asking a real question :)

Git still has problems being Windows friendly (most solutions still make you act in a very unix-like manner) and so it's still not on my radar to move any time soon.

But I keep seeing one particular pro-git (and pro-distributed in general) argument come up over and over again in git discussions, the argument that git helps encourage contribution.

And THIS question is one I'm very interested in, it's something that might actually push me towards switching.

I much preferred CVS for it's better branch logic, but because svn makes it so much easier to contribute switching to SVN resulted in improved contribution rates despite the shitty branching.

Has anyone actually measured rate of change and rate of contribution across a SVN to Git conversion?

For my repository, I have quite a good understanding of how many people are contributing.

http://www.ohloh.net/p/3299/contributors

Here's a better one showing contributors over time, compared to the (epic) contributor growth rate of Padre.

http://www.ohloh.net/p/compare?metric=Contributors&project_0=Adam+K+%28et+al%29+ CPAN+Modules&project_1=Padre

Is it possible to establish via actual metrics that the diversity of contributors and rate of change flowing into the final production CPAN releases is increased relative to the rate and diversity of change prior to the move?

Have you done it? Can I see the comparative results?

Does the theoretical improvement to contribution result in a real world improvement in contribution? Or does it suffer from the "You are not Linux" scaling problem, that our projects mostly just aren't big enough to benefit?

Posted by Alias on journal at Wed, 12 Aug 2009 05:15:28 -0000 | Permalink

These links are collected from the Perlbuzz Twitter feed. If you have suggestions for news bits, please mail me at andy@perlbuzz.com.

• Perl whipupitude to the rescue to notify Perlmonks users of security breach (dagolden.com)
• Skud debunks myths and answers questions about her OSCON keynote on women in open source (infotrope.net)
• Bail on Perl because of Perlmonks fiasco? Sounds silly to me (staynalive.com)
• Milestones in the Perl Renaissance (modernperlbooks.com)
• Help end licensing under "same terms as Perl itself" (perlbuzz.com)
• The awesome Peteris Krumins strikes again: Famous Perl one-liners explained, part II: (catonmat.net)
• Strawberry Perl releases its biggest update yet (csjewell.dreamwidth.org)
• Perl does error messages right (use.perl.org)
• Dave Rolsky: How I use Catalyst (blog.urth.org)
• Carl Masak: 3D graphing with Perl 6 (use.perl.org)
• PPI now supports Perl 5.10 (use.perl.org)
• Dear TPF - don't send money, send thank you letters (perlalchemy.blogspot.com)
• Perl::Metrics2 scalable enough to parse the GreyPAN (use.perl.org)
• First impressions of Padre (use.perl.org)
• The YAPC::EU Perl marketing BOF (use.perl.org)
• Rakudo project manager Patrick Michaud: Building a "useful release of Perl 6" (use.perl.org)
• Stumbling with Rakudo (lastofthecarelessmen.blogspot.com)
• RT @perloneliner perl -MTest::Valgrind -e 'leaky()'
• Help test the first release candidate for Perl 5.10.1 (use.perl.org)
• Brad Choate and I just moved Text::Textile over to github (github.com)
• Please comment on these seven grant proposals (use.perl.org)
• BooK on his White Camel award (use.perl.org)
• Rocco Caputo tries out git (use.perl.org)
• Why I stick with Perl (mechanicalrevolution.com)
• Data structures made easy with Data::AsObject (use.perl.org)
• Perl 6 has not missed the boat (perlbuzz.com)

Posted by Andy Lester on CPAN, Conferences, Perl 5, Perl 6, Perl Foundation, Rakudo at Wed, 12 Aug 2009 02:46:48 -0000 | Permalink

Too often I see user-facing text that includes far more detail than is necessary. My current example is this web app where I can upload files for my expense reports. The help screen tells me:

You can attach the following types of files:

• Portable Document Format (.pdf)
• Joint Photographic Experts Group (.jpg or .jpeg)
• Graphic Interchange Format (.gif)
• Tagged Image Format (.tif)
• Windows OS/2 Bitmap Graphics (.bmp)

Why why why do they bother giving those unnecessary explanations of what the file extensions mean? What they really mean is:

You can attach .pdf, .jpg or .jpeg, .gif, .tif and .bmp files.

Has anyone ever said "Hey, Bob, can you send me that Joint Photographic Experts Group file?" No? Then why do you think that people want to read it?

Posted by Andy Lester on Code craft at Wed, 12 Aug 2009 02:38:18 -0000 | Permalink

Since modules is taken, I was thinking about packages to do this:

#use Spreadsheet::ParseExcel::Workbook;
#use Spreadsheet::ParseExcel::Worksheet;
#use Spreadsheet::ParseExcel::Font;
#use Spreadsheet::ParseExcel::Format;
#use Spreadsheet::ParseExcel::Cell;
#use Spreadsheet::ParseExcel::FmtDefault;

use packages 'Spreadsheet::ParseExcel' => qw(
    Workbook
    Worksheet
    Font
    Format
    Cell
    FmtDefault
);

No idea if people would really care, but I like the cleaner syntax. Or maybe as a new feature with 'aliased':

use aliased 'Spreadsheet::ParseExcel', import => [qw(
    Workbook
    Worksheet
    Font
    Format
    Cell
    FmtDefault
)];

Seems an uncomfortable overloading of aliased, but it could be handy.

Posted by Ovid on journal at Tue, 11 Aug 2009 22:23:53 -0000 | Permalink

So I've been hacking on a pet project and thought that Freebase would be my answer. As far as I can tell, it's not. Not even close. Right now, Freebase is like a huge Wikipedia, but with a nice query language on top. I needed a list of all countries in the world along with basic stats like capital, population, GDP, official language, etc. Here's the script I hacked together:

use strict;
use warnings;

use WWW::Metaweb;

my $mh = WWW::Metaweb->connect(
    server      => 'www.freebase.com',
    read_uri    => '/api/service/mqlread',
    trans_uri   => '/api/trans',
    pretty_json => 1
);

my $countries = '[{"type":"/location/country","name":null}]';
my $result    = $mh->read( $countries, 'perl' ) or die $WWW::Metaweb::errstr;
my @countries = sort map { $_->{name} } @$result;

# http://www.freebase.com/app/queryeditor
my %country_stats;

for my $country (@countries) {
    my $country_info = sprintf <<'    END' => $country;
    [{
      "type": "/location/country",
      "name": "%s",
      "capital":null,
      "currency_used": [],
      "form_of_government": [],
      "gdp_nominal" : [{"timestamp":null,"currency":null,"amount":null}],
      "gdp_nominal_per_capita" : [{"timestamp":null,"currency":null,"amount":null}],
      "/location/statistical_region/population" : [{"number":null,"timestamp":null}],
      "official_language":[{"name":null}]
    }]
    END
    print "Reading the data for $country\n";
    my $result = $mh->read( $country_info, 'perl' )
      or die $WWW::Metaweb::errstr;
    use Data::Dumper;
    $Data::Dumper::Indent   = 1;
    $Data::Dumper::Sortkeys = 1;
    print Dumper($result);
}

Not only do I get only 100 countries returned -- including the Weimar Republic and West Germany (but not East Germany) -- most of whom have almost no data associated with them. The ones which do have data often have curious results which might be correct (see the official languages), but without context, who knows? Oh, and WWW::Metaweb needs a monkey patch to get around an incompatible API change in JSON::XS. One suggestion on the Freeweb message boards involved posting back the correct information. This sounds reasonable, but at the end of the day, it also sounds like a lot of work, particularly since I didn't want to base my project on Freebase. I just saw it as a useful source of information. Freebase looks awesome, but it's not quite there yet. Or I don't understand it. Who knows?

I'll have to figure out a better way of extracting this information (CIA World Factbook sounds good), but then figuring out the posting API for Freebase just sounds like more work that will distract me from my main project.

Back to the drawing board.

Posted by Ovid on journal at Tue, 11 Aug 2009 21:27:46 -0000 | Permalink

Marcus Ramberg responded to my post on How I Use Catalyst, and I'd like to respond to a few points he made.

Marcus wrote:

I disagree that $schema->resultset('Person') is a significant improvement on $c->model('DBIC::Person').

Me too! I don't think the former is a significant improvement over the latter. They are, after all, more or less the same. The one big problem is that the latter version uses a nonexisting DBIC::Person namespace. There are no DBIC classes anywhere in the app. I think the model version would be much better if it was just written as $c->model('Person').

Marcus also points out that the model layer lets you configure multiple models and access them in a unified way. That is indeed nice. Unfortunately, that has the problem of tying yourself to Catalyst's config, which is problematic for reasons I already described. Similarly, the unified layer only exists inside Catalyst, which is really only accessible during a web request. So now we're stuck with recreating all of this if we need to access our models outside of a web request.

The long-term Catalyst roadmap includes the much-talked-about application/context split. Once this is done, presumably you will be able to access the application, which I take to mean config and models, outside of the context (of a web request). Once that is in place, I think many of my objections will go away. Unfortunately, for now I have to write my own application/context splitting code.

Posted by Dave Rolsky on Programming at Tue, 11 Aug 2009 15:58:35 -0000 | Permalink

Carl Masak looks at Perl 6 development from a unique angle.

On the way to lunch today, I queried two collegues about Eve Online, their favourite way to spend evenings and weekends:

<masak> So, are people confused by all the detail when they first join?
<jonalv> Of course.
<masak> And how do you... handle that? How do you make it easy and fun for people to get up to speed?
<jonalv> Well, people tend to find the fun in it themselves. As to getting up to speed, they simply have to start in one end and unwind the whole thing.
<Gpox> It is complicated. There are occasional jokes about the learning curve involved.
<jonalv> Nowadays people have the wiki to help them get started as well. There wasn't one when I joined.


I think the parallels to the Perl 6 community are so clear that I don't even need to point them out.

I'm sure there are many differences between multiplayer online games, but the social dynamics seem very similar to what we have over at #perl6. Which brings me to today's thesis:

The Perl 6 community has the structure of a MMORPG.

We have just as much fun, and ― what's perhaps more important ― the bits we're putting together won't end up being owned by Blizzard Entertainment or CCP Games. Instead, we're building something that we ourselves and others will find directly useful during a large part of our lifetimes. I'm not out to devalue online multiplayer roleplaying games, just pointing out that there's perhaps an extra dimension to the acheivement we get to feel in the Perl 6 community.

Just for the heck of it, let's take the analogy one step further.

Ahem.

So, you want to join the Perl 6 community? If so, congratulations! You're about to embark on a quest that will seriously impact your day life, your habits and your sleep patterns. You will be building things, fighting unknown dangers, and communicating with your peers in order to make sure people are maximizing the benefit to the team and to the quest.

Here are the different classes you might want to occupy:

Warrior

One who actually fights the creatures out there, who braves new, possibly lethal dangers every day. In short: a Perl 6 application writer.

Being a warrior requires strength (you will have to hit the keyboard a lot when writing all the code, and the code will mercilessly hit you back... often), some dexterity (because working around known bugs requires adaptability) and perhaps a bit of charisma (should you want to collaborate with people, it's good to be pleasant).

We're currently on the lookout for warriors, so if you think you have it in you, be sure to stop by #perl6 and help us equip you for the perilous quest you have before you.

Priest

The priest is important for the group, but doesn't put himself in the way of direct danger like the warrior. Instead, they perform vital tasks with their hands at a safe distance. In the Perl 6 world, priests submit bug tickets, write tests and answer newbie questions on the #perl6 channel.

Being a priest requires intelligence (because their job is mainly to increase the collective knowledge of the group), constitution (because it's a lot of work) and perhaps a bit of charisma (so we don't bite the newbies).

I must say that I admire the priests. The impressing thing isn't in the incantations themselves, which are individually quite basic. The impressing thing is that their favour from the gods doesn't seem to run out. moritz++ and KyleHa++ especially come to mind. I'm sure there are slots for a few more priests too.

Mage

In the flurry of activity during a quest, the mages are the ones who wield forceful spells in the form of new features in our implementations. The mages know some pretty hefty incantations, but they speak in codes (like "Haskell", "Parrot" or "Lisp") so that us mere non-initiates can only stand by in admiration when they get going.

Being a mage requires wisdom (because, hey, you're implementing a compiler), intelligence (in figuring out how to fix the bugs and still pass the test suite), and constitution (because it's a lot of work).

Learning to be a mage looks nigh-impossible, but our mages assure us that there's actually not that much to it. And the rewards are of course enormous: you get to have an incantation of yours immortalized in millions of installations around the world.

Healer

Some people are in the group to make sure the group is doing well, and that no-one is critically low on hit-points. Discussions can sometimes get heated or sharp, by which point we're very glad to have the people around who are specially trained to see beyond the ego and help us focus on the important parts of the picture. We simply need to be reminded at times, that we're (as S01 expresses it) "a bunch of ants all cooperating (sort of) to haul food toward the nest (on average)". We don't need to agree always on everything, but keeping the group coherent is important, and healers do their magic in the background to help us with that.

Being a healer requires wisdom (to know where and when to intervene), intelligence (to know to pick the right soothing words), and plenty of charisma (to get it across that -Ofun is what we're optimizing for).

I think many people jump in as healers now and then during our quest, but we also have one permanently employed high-level healer among us. TimToady++.

Archer

As for long-distance influence, and reaching outside of our own circles, the archer fills the important role of blogging, tweeting, iron-manning and generally making a positive noise about Perl 6, which can reach people far away.

Being an archer requires strength (again, typing is hard, and commenters/other bloggers might shoot back at you), and dexterity (you need to be fluid in expressing yourself). Having intelligence, wisdom and charisma ― something I as a simple warrior often find myself wanting more of in my blogging ― is not a necessity, but sure helps in getting your point across.

In conclusion: LFA, n00bs?

The Perl 6 community is on a pretty awesome quest. If you'd like to enter as one of the above classes, you're sure to find your place as a productively contributing members. Filling several roles is perfectly fine too, but people tend to find their specialties and favourites.

As to when we'll be "done", who knows? When is an MMORPG "done"? All we know is the quest we're on until April is our most ambitious yet, and that it's guaranteed to be an interesting set of months on #perl6 until then.

(Careful postscript: I know that Eve Online might not be a role-playing game in any strict sense, so if you plan to write a comment that amounts to that, consider writing "Your post made me go download Rakudo" instead. The Eve Online bit was mostly how I got to thinking along these lines in the first place.)

Carl Masak has been programming Perl since 2001. He found Perl 6 somewhere around 2004, and fell in love. Now developing November (a Perl 6 wiki), Druid (a Perl 6 board game), and proto (a Perl 6 package installer), Carl is a heavy user of and irregular committer to Rakudo.

Posted by Andy Lester on Community, Perl 6 at Tue, 11 Aug 2009 15:46:20 -0000 | Permalink

I'll write more about this later on my personal blog, but I live in Dolphin Square, a place with a long, illustrious history. Now that I'm engaged, they've made it very clear that my fianc?e cannot live with me (she has a cat) and I cannot live with her (I have a lease). They've patiently explained that I signed the papers and they can put my flat on the market along with several other flats they already are having trouble renting, but no, they are not willing to budge one inch. The little human touch about actually caring about people has been lost somewhere, but given my experience with Dolphin Square, this is no surprise.

Had this been simply a contractual matter, I might have just accepted that things like this happen in life and my fianc?e and I will have to wait a while before we can live together. However, Dolphin Square is a shiny fa?ade with precious little behind it (aside from, say, tiny issues like that pile of feces they left festering in a stairwell for several days or refusing to clean my flat before I moved in), so I effectively feel like the quality of living they've promised and the actual abysmal reality has left me robbed. The lease is the last straw. Hopefully I'll get a long, good rant going on my LJ and push it up near the top of Google.

Posted by Ovid on journal at Tue, 11 Aug 2009 09:28:04 -0000 | Permalink

There's a fair bit of discussion about version numbers and people get confused by them. Get confused no more. Here's the way you should be setting version numbers in your modules:

our $VERSION = "0.001"; # or "0.001_001" for a dev release
$VERSION = eval $VERSION;

dagolden goes in extreme detail as to why you need this and what led to this state of affairs.

Posted by Ovid on journal at Tue, 11 Aug 2009 06:29:39 -0000 | Permalink

A thread on Ubuntu Forums included the absurd assertion that Perl 6 has missed the boat.

This cannot be true, simply because there is no boat.

Perl 6 is not, and never has been, about capturing market share, or fitting a given niche, or becoming the pre-eminent tool of choice for a given problem area.

Larry says "Perl 6 isn't about missing boats, but about inventing airplanes."

I'm finally working on my "handy responses for Perl 6 sniping" cheat sheet. I'll post it soon!

Posted by Andy Lester on Perl 6 at Tue, 11 Aug 2009 05:11:19 -0000 | Permalink

Thanks to Dave Cross and Ovid for organizing the marketing BOF during YAPC and the notes Ovid made later. The fact that there were more than 50 people in the room made it clear people are interested in the subject.

Unfortunately that number of people made it impossible to have a discussion but I think just the presence of so many people show we have to move on. I hope some of them will come forward and write about their opinions. I think the more people talking about this the better.

As I don't know the names of most of the people present let me just write down some of the objectives people mentioned. I think it ranged from

Make sure Perl is not banned in my organization
through
Bringing the excitement back to Perl
all the way to
Turn Perl into an accepted application development language such as Java or C#.
(this was me).

None of these are really measurable objectives so we will need to find a few of those as well but I think this is a good start. I'd be glad to see what other objectives people can write down. So go ahead, please add your comments.

Market research

Ovid kept mentioning the market research and Richard Dice suggested a matrix idea that sounded quite good. I am much more convinced now in the need of some such market research and I hope Ovid will be able to take upon himself and push it forward or hand it over to someone who will do it in a reasonably short timeframe.

Actions

Some people suggested all kinds of actions that seem to be totally reasonable regardless of any market research. Some of the ideas were

• Write more blog entries - which just strengthens the idea of the Iron Man
• Write articles and get them published in other places - such articles need more work than a blog entry but this should help us break out from the echo chamber
  Maybe the rules of IronMan could be ammended so any Perl related article in one of the magazines counts as to be worth 5 weeks of blogging. Can someone come up with a list of publications that might be interesting?
• Create really nice web sites - Many people have already written before me so I only repeat it but we have to make sure our central web sites (e.g. use.perl.org, perl.org) look live and modern without being distracting.
• Write exciting application in Perl and show them to everyone - These can be web applications or you could try and write a game using SDL such as the Space Invaders by Thomas Klausner.
  Or it could be something completely different.

Posted at Mon, 10 Aug 2009 15:40:14 -0700 | Permalink

"Localizing legacy code, automating the process, and how to work with your translators"

Jeff Goff (DrForr) will be talking about his experience with localizing a
large and hugely successful mission critical legacy perl application.

The meeting will take place at 7 pm PST on Tuesday August 25th at Six Apart World Headquarters.

http://search.cpan.org/~jgoff/

Announcement posted via App::PM::Announce

RSVP at Meetup - http://www.meetup.com/San-Francisco-Perl-Mongers/calendar/11080121/

Posted by Phred on journal at Mon, 10 Aug 2009 21:23:38 -0000 | Permalink

I created a new website as a fun little personal project, She Said What?!

It was a fun experiment both in minimal web design, and also in minimal code. I can update it from the command line just by typing:

ssw 'A quote goes here|and commentary goes here'

This adds a quote to the quote "database", which is just a directlory of timestamped flat files on my desktop. Then it regenerates the site as static HTML and pushes it to the live server.

The code is in my mercurial repository for anyone who might care.

Posted by Dave Rolsky on Personal, Programming at Mon, 10 Aug 2009 20:51:18 -0000 | Permalink

In a comment on A Modern Perl Success Story for the Internet CLI, Darian Patrick asked for an explanation of my file slurping one liner:

my $contents = do { local $/ = <$fh> }

While File::Slurp exists on the CPAN to encapsulate these sort of tricks and Perl 6 provides the slurp method on filehandles, this one-liner is in my toolbox because it's very simple (for me) to remember and type.

As you may remember from perldoc perlvar, $/ is Perl 5's input record separator variable. Its contents are a literal string used to identify the end of a record when using readline on a filehandle. Its default is the platform-default newline combination -- \n, whatever that translates to on your platform and filesystem.

To read a file with different record separators -- perhaps double newlines -- set $/ to a different value.

To read the whole file in one go, set $/ to the undefined value.

It's always good practice in Perl 5 to localize all changes to the magic global variables, especially in the smallest possible scope. This helps prevent confusing action at a distance. (I appreciate that Perl 6 moves these globals to attributes on filehandles.)

That explains how this code works:

my $contents = do { local $/; <$fh> };

(I may have first encountered this idiom in perldoc perlsub.)

How does my code work?

The localization has to occur before the assignment, for obvious reasons. As it happens, it occurs before the readline. As the readline uses the contents of $/ to determine how much to read, it sees the undefined value, reads the entire file, and assigns its contents to $/. Even though leaving the do block immediately restores the previous contents of $/, the assignment expression occurred in scalar context, thanks to the assignment of the block's result to $contents. An assignment expression evaluates to the value assigned: the slurped contents of the file.

As you may have determined already, the do block both limits the scope of the localization and makes all of the file manipulation into a single expression suitable for assignment to $contents.

Using slurp from a module is likely clearer. As well, localizing and copying file contents may be somewhat inefficient. In the case of my Identi.ca poster, files will rarely be larger than 140 characters, the program is short-lived, it blocks on user input, and it immediately produces and consumes network traffic, so this is unlikely to be a bottleneck in any sense.

I skimmed the relevant documentation and couldn't find a guarantee that the order of operation of localization and readline will remain as it is; I tried a few experiments with B::Concise to confirm my suspicions, but ran afoul of the Perl 5 optimizer. It may be safer to use two expressions in the block:

my $contents = do { local $/; <$fh> }

Even still, a silly little idiom like this demonstrates some of the interesting features of Perl 5.

Posted by chromatic on modern perl, perl, Perl 5, perl programming at Mon, 10 Aug 2009 20:48:17 -0000 | Permalink

I have tons of things to say about the conference, the talks and the people I met there but let me just say this:

Thank you! All of you! It was great!

There were more than 300 people in the conference and more than 100 talks.

I attended many good talks and many more I missed as they were overlapping with each other or because I was on the hallway track.

Thanks for the great talks!

In between the talks - and sometimes instead of them - on the hallway track I managed to talk to many people. I met several friends from the previous conferences but also many people who I did not know earlier or who I only new by e-mail address or by IRC nickname.

Lots of interesting discussions, lots of good ideas, I'll about those later.

See you next year in Pisa or at an earlier Perl workshop or conference.

Posted at Mon, 10 Aug 2009 13:04:57 -0700 | Permalink

My version number article exhausted me, so this will be a short post. My work to convert some more Perl module repositories from Subversion to git continues with two more things hosted on github:

• ExtUtils::CBuilder ― This is the complement to ExtUtils::ParseXS, which I described converting last week. Anyone interested in working on it, please see the bug queue.
• The CPAN META.yml spec ― this isn’t exactly a module, but extracting it from the Module::Build repository is a first step towards some needed improvements. I have some things in development on that front that I hope to announce in a week or so.

Posted by dagolden on cpan, git, perl, ironman at Mon, 10 Aug 2009 19:41:05 -0000 | Permalink

MRO::Magic is a system for writing your own method dispatcher in Perl. I have written about MRO::Magic before, but it's rushing toward being useful as 5.10.1 rushes toward release.

Right now, I have two weird issues. Neither is a real blocker, but both concern me because they really highlight the hairiness of doing this sort of thing.

The first is that in one case, the following line emits an "Uninitialized value in subroutine entry" warning:

$class->UNIVERSAL::isa('Class');

In all cases in the test, $class is the string "Class" and in all but one case, there is no warning. What? I don't know if this matters, but it's confusing, and makes me feel like my own code is voodoo, which is not a feeling I want.

The other case is more troublesome, as it is likely to drastically affect performance.

Because we're altering the way that method resolution works, we need to manage our own method caching, which we can do with the "mro" module. I tried to use mro::method_changed_in on the classes being magicked, but it failed. It looked like I'd need to alter superclasses, so I just called the method on UNIVERSAL and things worked. Of course, this means that any time you called a method on an MRO::Magic class, you'd clear you entire method cache. Oof!

Today I tried injecting a new, empty package into the @ISA of magicked classes and marking it as having changed. This got me neither the pre-UNIVERSAL-clearing behavior (of failing tests) nor the UNIVERSAL-clearing behavior (working). Instead, I got deep recursion.

Waah?!

If anyone is feeling brave and wants to have a look at MRO::Magic on 5.10.1 RC1, I would really appreciate it!

Posted by rjbs at Mon, 10 Aug 2009 11:02:07 -0500 | Permalink

Posted by John Napiorkowski on perl programming dns-hijacking uri cpan at Mon, 10 Aug 2009 15:52:24 -0000 | Permalink

I was working on the 5th part of my Perl for Newbies tutorials when I wrote some example Perl code like the following:

my $dir_path = shift;

opendir my $dir_handle, $dir_path
    or die "Cannot open $dir_path - $!.";

my @files = readdir($dir_path);

closedir($dir_handle);

(Can you spot the bug?)

Much to my surprise, this code segfaulted when I gave it "." as an argument. I realised that the problem was that I was reading from the path variable instead of from the directory handle, but it still shouldn't have segfaulted.

I asked a few people on IRC about it, and some of them were able to reproduce it, but only with perl-5.10.0 on Linux. Someone was able to golf it down to perl -e 'readdir($x=".")', which segfaults too.

So I opened a ticket for this bug, and Rafael (RGS) fixed it in blead, but without a regression test case. So I submitted a patch to add it, and also pushed it to my github clone of the perl repository. I hope it get applied.

On a different note, I'd like to note that I have completed the transition of the automated test suite of Quad-Pres from Bash to Perl 5 and Test::More TAP-emitting tests. Now they are much less verbose, and possibly run faster (though it might be just a feeling).

Now I'd like to continue its switch from GNU Autohell to something more decent. I still don't know whether I should use CMake or Module-Build as I depend on the wml executable from Website Meta Language. Any insights would be appreciated.

Posted by shlomif@iglu.org.il (Shlomi Fish) on test, automated, suite, tap, quadpres, cmake, perl, bug, readdir, quad-pres, test-more, module-build, segfault at Mon, 10 Aug 2009 09:28:02 -0000 | Permalink

She said "yes" :)

Posted by Ovid on journal at Sun, 09 Aug 2009 09:23:00 -0000 | Permalink

perl 5.10.1-rc1 has been released 3 days ago, so i packaged it for mandriva. the good thing is that a lot of patches that we were adding to our perl are now merged upstream.

the new perl (perl-5.10.1-0.rc1.2mdv at the time of writing) rpms are available in mandriva testing media. you can install them with:

# urpmi perl perl-base --media "Main Testing"

please report any problem to either mandriva cooker mailing list, or to perl 5 porters if that's a perl related bug.

Posted by jquelin@gmail.com (J?r?me Quelin) on rpm, mandriva, perl at Sun, 09 Aug 2009 08:41:00 -0000 | Permalink

This has been a good week for me to take things that look to big and break them into smaller, attainable goals. I did it for some work projects, and I think it's helping me feel like I can start making more progress. It also led to a huge update to Path::Resolver, including much-needed documentation.

Now I'm doing the same the same thing with some personal projects, both code and otherwise. I'm tired of feeling boxed in by huge projects that really just need a lot of small steps taken.

My initial list of goals was:

• a clear distinction laid out for ::Nester v. ::Document
• extensive tests for Pod::Elemental, and Document implemented
• Pod::Weaver operating on Document objects
• the "Allow" Weaver, which leaves specific stuff in place, if found
• the "Accordion" Weaver, which leaves generic stuff in place, if found
• Pod::Weaver::Config, for loading Pod::Weaver config from files

...with some optional stuff I'd like to get done.

The first one I did already, pretty early on. I also realize that I did the final one during OSCON by producing Config::MVP, which makes the Dist::Zilla configuration generic enough for anyone to use. (Maybe I should add more documentation, though.)

So, what's left? Tests, "operating," and two specific weavers. Tests are sort of uninteresting. They'll happen. The specific weavers are also sort of obvious. Those two, especially, are very hand-wavey. Everything interesting comes down to "Pod::Weaver should work."

What a stupid goal! What was I thinking?

Tomorrow, I will start sketching out the specifics of what that means, how Pod::Weaver will be used, and how I will get it into an operational state.

Posted by rjbs at Sat, 08 Aug 2009 22:01:37 -0500 | Permalink

YAPC::EU 2009 microreport
I'm still travelling, and so I don't have a reliable sources of connectivity, time, or caffeine. As such, this report is much briefer than I would like.

I made it to YAPC::EU 2009 and survived. As my first YAPC ever, it was great to be at a conference where I could assume that everyone knew Perl. It was also great to meet a number of the people who I'd been working with on-line for years, but never met in person.

It seems that lots of people use autodie. That's good, because it's hard to gauge feelings and reactions on-line, but it's easy when a small group grabs one after a talk and asks detailed questions about its internals. There seems to be a lot of demand for autodie to provide the ability for third-party code to see if it's enabled, and also a lot of demand (not least for me) for Perl to call a special method on exception objects if they're about to kill one's process, as opposed to be caught and handled.

The conference highlight for me were the people. They were very warm, very willing to participate, and heckled only about things I could easily answer. I suspect that means they were also being very nice, since they were a very knowledgeable audience. Privately, everyone was extremely accommodating. Apparently travelling from Australia makes one special, and I felt very special indeed from all the attention.

Looks like I'm out of time for this entry already. I've got yet another flight tomorrow, so I'll try to squeeze out some writing on the plane.

Posted: 9th August 2009.

Tags: europe perl portugal travel yapceu2009

Bookmark:

Posted at Sun, 09 Aug 2009 00:00:00 -0000 | Permalink

While testing 5.10.1 RC1, I've hit a show-stopper for Padre. Alien::wxWidgets makes use of a get_options "debug" option that collides with the new internal "debug" option in the latest Module::Build that comes with 5.10.1 RC1.

If you must have Padre on 5.10.1 (and lets face it, you must) you can work around this problem by downgrading Module::Build to 0.32.

On Strawberry, you can just run the following on the command line

    pip http://search.cpan.org/CPAN/authors/id/E/EW/EWILHELM/Module-Build-0.32.tar.gz

Posted by Alias on journal at Sat, 08 Aug 2009 19:37:12 -0000 | Permalink

After seeing the goodies in Tim Bunce's OSCON talk on NYTProf 3, I couldn't resist trying it out, even though half the tests fail in the repository at the moment (especially since half his demos inevitably involve Perl::Critic or PPI and I spotted some curiosities in his video).

I've happy to report that because of the increased detail in the area of xsubs and built ins, I was able to fairly quickly find a situation in which excessive calls via overloading to a centralised xsub boolean true was still consuming far more time than I'd expected.

A few defined() wrappers later to speed up the worst offenders, I managed to pick up somewhere in the vicinity of 1-2% speedup on my sample loads (which are pretty similar to the loads of Perl::Critic). And since I've spent far more time than I care to admit optimising PPI already, anything I can measure as an entire percent or higher is a pretty decent win for me.

PPI 1.206 with just these and no functional changes will be uploaded shortly, and I'm hoping to find some time to tackle some other promising-looking avenues relating to unexpectedly expensive regular expressions shortly.

I also ran the profiling on Perl 5.10.1 RC1 and I'm happy to report that the improvements to UNIVERSAL::isa do seem to make it less expensive (by maybe somewhere between 10-20%). This should provide for another percent or two total speed up on typical loads.

Posted by Alias on journal at Sat, 08 Aug 2009 17:59:23 -0000 | Permalink

games::pandemic 0.6.0 is now on cpan, with a lot of improvements:

• introducing epidemic events - fear! :-)
  
• closing current game is now supported
• player cards now displayed in a window of their own
• detecting game over condition when no more cards
• sorting cards by type then by disease then by city name
• introducing a new "action" menu, with shortcuts for actions
• actions now independant of buttons / menu for easier reuse

of course, there are still a lot of things to do, but it's shaping up nicely...

Posted by jquelin@gmail.com (J?r?me Quelin) on games, pandemic, perl at Sat, 08 Aug 2009 13:44:00 -0000 | Permalink

perl community places some emphasis on testing. which is great, since it enhances the overall quality of perl ecosystem.

the first, easiest test that one should provide is basic: making sure the modules compile. we have 2 options here:

• either create a test and add some use_ok('Foo::Bar'). but we then need to update it whenever we add more modules to our distribution - without forgetting to update test count.
• or, to get rid of this burden, ship a test that automatically finds the module and tries to compile them.

but shipping the exact same generic test is boring, and could be automated. which is now the case for dist::zilla with the new module dist::zilla::plugin::compiletests. just add the following in your dist.ini file:

[CompileTests]

and you will get a brand new t/00-compile.t that will test your modules and your scripts one by one. this my take some time if your dist ships a lot of modules having lots of prereqs, especially if they are a bit slow to compile such as Moose. but this will potentially find more compile errors in your modules.

Posted by jquelin@gmail.com (J?r?me Quelin) on plugin, dzill, dist, tests, perl at Fri, 07 Aug 2009 07:14:00 -0000 | Permalink

If you put a million monkeys at a million keyboards, one of them will eventually write a Java program. The rest of them will write Perl programs.

― anonymous Internet wag

I was a laser printer guru in 1998. I spent most of my time talking to customers about printing and imaging. I can probably find a blocked paper sensor on an HP LaserJet 5si ten years after my last experience troubleshooting the paper jam message. (I can probably close my eyes and walk someone through it from memory.)

I delivered two programs in that job. One was for the customer service department. Every time someone from level one support had to ask a question of level two support, level two support needed to record the relevant product line. These became daily, weekly, and monthly reports used to bill the subdepartment for each product for the service calls.

I was teaching myself Java at the time. I wrote a small AWT GUI application (in these days, that's what you could get with the Java GNU/Linux port on Red Hat 5.3) which ran on a spare PC we set up at the level two support desk. It logged all statistics to a flat file which another program -- I believe a Tcl program written by a co-worker -- summarized into the reports. This program took a couple of weeks to write. The program was still running when I last walked through that floor of that building in early 2000.

My second program was for the networking support group. In those days, internal support groups often managed the external-facing web site for their products. They wanted a program which could identify when a page or pages in the web site changed and email an opt-in list of customers.

I thought about the problem and wrote a small Bourne shell script (running on HP-UX 10.x, I believe) to do the job. The code must have been between ten and fifteen lines long. As it happens, the networking group used IIS and an early Java application server plugin to manage their web site, so they wanted a Java program which ran on Windows. They asked me to port my proof of concept to Java instead.

I never finished that program. I switched to a system administrator role and discovered that the Perl I'd dabbled with late in 1998 was a much better fit for the system administration and web development I needed to do.

The afternoon of my first day on the new job, the administrator I was replacing turned around and asked me if I knew Perl. I'd dabbled. I'd written a bit. I could read a program and have some sense of what was happening. I said "Some, yeah."

He was working on a program and kept getting a weird syntax error. I looked over his shoulder and said "That elseif should be an elsif."

As I went through the files he left, I found several printouts of Perl programs he'd downloaded from the Internet, changed the copyrights, and tweaked very slightly.

If the department I worked in back then still exists (and it might) and if someone still remembers me from then (doubtful), I'd have no surprise to learn that some of the code I wrote there still exists. (One evening I saved my group a million dollars in penalties by fixing a misbehaving FTP server right before a deadline to deliver new firmware images to a vendor. They gave me a plaque.)

I've toyed with Java some since then, but I haven't delivered any significant software in Java. I hear it's a passable language with some great libraries and effective tools.

I've spent the intervening years understanding software development, helping create a testing culture, figuring out what's wrong with object orientation and how to fix it, and learning how virtual machines work to help invent the future (or at least drag mainstream dynamic languages kicking and screaming into the '90s).

I delivered that first Java program because I was stubborn and had a lot of spare time over that fortnight.

I've delivered almost every Perl program since then because of Perl's whipupitude factor. That's how the administrator I replaced could download a likely program, tweak a few things, and pass off a working solution as his own. That's how a motivated, mumble-something fledgling music performance major like me could write a proof of concept mail notification service in an hour with the Bourne shell and port it to Perl 5 in an afternoon and still not be sure which Java SMTP package actually worked back in 1998.

I have no illusion that I wrote idiomatic Perl 5 code then. (The Perl Renaissance had yet to begin.) I doubt I'd want to maintain that code now. I wouldn't suggest it as a good example for novice developers to emulate.

Yet it did the job. The sound of that code is the sound of me and my users and co-workers not banging their heads against the wall. It's the sound of a 30 second task repeated ten times a day -- and its concomitant context switch -- vanishing.

Anonymous Internet critics can shake their heads and wag their beards and tear their outer garments and throw dust in the air as they wail and weep that ugly Perl code exists and persists.

I'm all for improving Perl code. I believe in tests and refactoring and idioms and reuse and clarity and good naming and succinctness and encapsulation and proper factorization and favoring simplicity over cleverness. I believe in teaching novices and guiding them and encouraging them to develop good habits and consider maintenance and understand what happens and how things work and to think not only about what they want to do but why and how to do so sustainably.

I want better defaults and clearer error messages and fewer rough patches where they can trip and stumble and skin their knees and elbows.

I want that very much.

Yet the whipupitude that lets novices -- and there are some six and a half billion people on the planet who've never even thought they could write a useful computer program -- do useful things even if sometimes they do it the ugly way is a positive aspect of Perl. It's a positive aspect of many tools.

By all means let's continue making Perl easier to understand and gentler to learn and more encouraging to write good code the right way. That's important. We're all novices sometimes. Let's help them get their work done and then help them get their jobs done well. Programming well is difficult; why not encourage them with small victories? Why make the barrier to accomplishment artificially high?

I don't understand that argument.

Then again, some people probably think that Pablo Neruda never existed after hearing me speak Spanish.

Posted by chromatic on modern perl, novices, perl 5, perl programming at Thu, 06 Aug 2009 20:26:47 -0000 | Permalink

During some test writing, I had to spawn a shell command that would run while /bin/true was true. Unfortunately, my test script figured the run time should be limited, so I had to kill that little brat myself.

Let's have a look at a simplified setup first:

my $pid = fork();
if($pid == 0) { print "I'm the child: $$\n"; system('while(/bin/true); do echo "Mwuahaha"; sleep 1; done');
} else { print "I'm the parent: $$\n";
} sleep 3;
kill 1, $pid;

Although this might look valid perl, the "Mwhuahaha" will continue to fill up you terminal. The forked system command will not be killed when your program ends.

There had to be a way to kill the child (grandchild, if you want), and there is. Using setpgrp and getpgrp to the rescue!

Let's have a look at the documentation first (always a good thing):

 setpgrp PID,PGRP Sets the current process group for the specified PID, 0 for the current process. Will produce a fatal error if used on a machine that doesn't implement POSIX setpgid(2) or BSD setpgrp(2). If the arguments are omitted, it defaults to "0,0". Note that the BSD 4.2 version of "setpgrp" does not accept any arguments, so only "setpgrp(0,0)" is portable. See also "POSIX::setsid()". getpgrp PID Returns the current process group for the specified PID. Use a PID of 0 to get the current process group for the current process. Will raise an exception if used on a machine that doesn' implement getpgrp(2). If PID is omitted, returns process group of current process. Note that the POSIX version of "getpgrp" does not accept a PID argument, so only "PID==0" is truly portable.

Almost there! Let's see the kill documentation as well, for it contains a fatal clue:

 Unlike in the shell, if SIGNAL is negative, it kills process groups instead of processes. (On System V, a negative PROCESS number will also kill process groups, but that's not portable.) That means you usually want to use positive not negative signals. You may also use a signal name in quotes.

Note that we need the negative signal!

So, if we change our test script to this, it'll all work, and those pesky (grand)children are killed when we want to:

my $pid = fork();
if($pid == 0) { setpgrp; print "I'm the child: $$\n"; system('while(/bin/true); do echo "Mwuahaha"; sleep 1; done');
} else { print "I'm the parent: $$\n";
} sleep 3;
kill -1, getpgrp($pid);

Posted at Thu, 06 Aug 2009 09:51:00 -0700 | Permalink

Posted by acme on perl, book, moose, yapceu2009 at Thu, 06 Aug 2009 15:44:12 -0000 | Permalink

Posted by acme on perl, book, moose, yapceu2009 at Thu, 06 Aug 2009 15:43:56 -0000 | Permalink

As reported elsewhere the first super easy Padre installer for Windows was released last week.

It is now called the Almost Six package as it contains a development version of Rakudo Perl 6.

Aloms six contains the following packages:

• Strawberry Perl 5.10 for Windows
• Rakudo Perl 6 release #19 ("Chicago")
• Padre version 0.41
• Padre::Plugin::Perl6 v0.55

It is very easy to download the msi package and then install it as you would install any other Windows application.

Then you can start writing either Perl 5 or Perl 6 code.

I was on YAPC::EU in the last 3 days but there almost everyone is using Linux or Mac OSX. So I wonder what do you people suggest? How can we attract Windows users who write Perl code to try Padre an give us feedback?

Posted at Thu, 06 Aug 2009 06:05:20 -0700 | Permalink

dist::zilla::plugin::autoprereq just gained more moose support: it can now extract the base module declared with:

extends 'Foo::Bar';

it's not perfect since it only extracts the first base class, but i think it will cover 90+% of the cases out there.

Posted by jquelin@gmail.com (J?r?me Quelin) on moose, plugin, dzill, dist, perl, prereqs at Thu, 06 Aug 2009 12:54:00 -0000 | Permalink

So YAPC is pretty much over, and it was awesome. My last minute talks went somewhere between "fine" and "well" depending on the talk. Unlike most other YAPCs I actually caught some of the other sessions too.

I'm really happy with the feedback from the attendees, people are responding better than ever to Moose and even KiokuDB despite the fact that it's such a young project.

This YAPC, more than any other, is reassuring me that the Perl community is moving in the right direction. We seem to be refocusing on what's really important (to me anyway). We are not all the way there, Perl still has problems with marketing, technical debt, flawed perceptions, bad practices and so on, but it's obvious that the trend is one of moving towards where we want to be, not some inevitable doom.

Posted by nothingmuch on yapc at Thu, 06 Aug 2009 10:30:00 -0000 | Permalink

After the plethora of commits and rush of features into the 0.42 Birthday Edition of Padre, the brief lull afterwards has given me the chance to do some refactoring in less exciting areas.

One of these areas is in the Plugin Manager, which currently has two interpretations of a "Plugin Name" (The one that the plugin tells us to call it, and the wrong one) :)

So I've spent quite a few hours last night digging all the remnants of those old names out (which were used everywhere, and deeply integrated) and just sticking to the class name for the plugin primary internal identifier.

One benefit of this is that the Plugin system can now (theoretically) support plugins outside the Padre::Plugin::FooBar flat namespace (although they won't be auto-detected and there's no way to tell the plugin manager to add it yet).

A secondary benefit of this is that I can now add auto-detection for plugins in the long-awaited Acme::Padre::FooBar namespace.

Padre is started to accumulate enough subsystems now that the idea of doing joke plugins is getting quite realistic.

For example, the SVN-only plugin Padre::Plugin::Media shows a proof of concept for making the editor asynchronously play sound. This Media code, combined with Hook::LexWrap and a File::ShareDir collection of wav files, would make it pretty easy to inject sound effects into arbitrary editor events.

Imagine a Simpsons or South Park themed editor! Or for something less copyright-breaky, perhaps something more like a Acme::Padre::TheCollectedSayingsOfRicardoSignes, with witty and biting interjections whenever the plugins spots you doing something rjbs doesn't like :)

"Oh god, you're using MIME-Tools. Just kill yourself now!"

"You're using SVN? Switch to Git you fool!"

"Oh good, you're not connected to the internet. We need to talk..."

The new Acme:: detection will is available in SVN now, and in the next 0.43 release on the CPAN.

Posted by Alias on journal at Thu, 06 Aug 2009 06:55:29 -0000 | Permalink

I've never believed the argument that web applications will completely replace native client-side applications. The last time someone said that, I held up my Android phone and said "This is a pervasive Internet device. I use several applications regularly. The only native web application I use is a web browser."

Maybe I'm an old dinosaur well before my time, but I'm happy writing these posts in Vim (with copious macros and customizations) through It's All Text!. I don't mind using Git from the command line, and I think ctags is still one of the most useful IDE inventions.

That doesn't mean I reject the billion or so devices that make up the Internet, nor all of the information they contain, nor the services they provide.

It means I can be clever about using them.

A Modern Perl Identi.ca/Twitter CLI

Twitter may be a morass of uselessness which all too often proves the existence of sewers upstream of the collective stream of consciousness, but in between all of the "lollers" and "RT RT RT lollers cat falling down video!!" sometimes it's a good way to pass on lightweight, transient information to people who might care.

I've used Identi.ca for a year now. As part of marketing my business, I set up a couple of accounts to which I publish notifications about the fiction and non-fiction publishing work we do. If I edit a chapter in an upcoming novel we've announced, I can send out a short notice to anyone following that novel's progress. The same goes for a book.

It's easy, it's quick, it's non-intrusive, and people really do seem to follow this information. (Wouldn't you like to know how far away is the next book from your favorite author? Still waiting for that seventh Zelazny Amber story....)

Setting all of this up for multiple books and multiple accounts could be tricky. I could log into Identi.ca manually every time I want to post an update (maybe a couple of times a day). I have to manage several user names and passwords. What could be a 15-second update when it's fresh in my mind could take several times that long, and I wouldn't do it.

That's why I have a decent shell and modern Perl.

I wrote a proof of concept in ten minutes using Net::Identica; all hail Perl and the author:

#!/usr/bin/perl

use Modern::Perl;
use Net::Identica;
use File::Temp 'tempfile';

sub main
{
    my ($username, $pass, $message) = @ARGV;

    do
    {
        $message = get_message( $message );
    } until ($message && length( $message ) <= 140);

    post( $username, $pass, $message );
}

sub get_message
{
    my $message = shift;

    my ($fh, $filename) = tempfile();
    print {$fh} $message if $message;

    system( $ENV{EDITOR}, $filename );

    seek $fh, 0, 0;
    exit if eof $fh;

    return do { local $/ = <$fh> };
}

sub post
{
    my ($username, $pass, $message) = @_;

    say "[$message]";

    my $ni      = Net::Identica->new(
        legacy   => 0,
        username => $username,
        password => $pass,
    );

    $ni->update( $message );
}

main();

I've highlighted the most interesting line. The program takes three arguments, a username, a password, and an optional message to post. The emboldened line launches (for me) Vim on a temporary file containing the message. If there's no message, I can write one. If there is a message, I can edit it. When I save the file, the program immediately posts it to Identi.ca, with the given username and password.

It's easy to create bash shell aliases to customize the behavior further. I have a file called ~/.alias, which can contain, for example:

alias gigdent="perl ~/bin/dentit gigapolis password"
alias mpdent="perl ~/bin/dentit modern_perl password"

For every new project I start, I can create a new account and shell alias. Then I only have to remember the name of the alias to write a status update.

50 generously-spaced lines of modern Perl code and a bit of shell glue later, I can get my work done with little fuss and even less cognitive overhead from task-switching. Maybe it's not jaw-droppingly amazing like rounded corners on a Web 2.0 startup written by skinny-jeans hipsters with deliberately messy hair and flipflops, but it helps gets my work done and it took longer to write this post than to write the code.

I'll call that a Perl success story any day.

Posted by chromatic on bash, CPAN, identica, modern perl, perl 5, perl programming, twitter at Wed, 05 Aug 2009 23:44:34 -0000 | Permalink

In a perfect world, I would never blog about version numbers in Perl.

Version numbers should be boring. No one should have to think about how they specify or check a version number. Perl programming makes things easy, not hard, right?

Unfortunately, version numbers in Perl aren’t boring and easy. Instead, they are complicated and confusing. Every Perl programmer needs to understand at least some of this complexity. Otherwise, you can make life difficult for yourself or others without realizing it.

In this article, I’m going to explain what I think are ‘good practices’ for dealing with version numbers in Perl. I’m going to point out what I think are ‘bad practices’.¹ In the end, I’m going to make some recommendations that I hope people will follow for their own sake and the sake of others who work with their code.

For the impatient, the disinterested or those who just want to follow a recipe, my advice for all modules is this:

our $VERSION = "0.001"; # or "0.001_001" for a dev release
$VERSION = eval $VERSION;

If you already use decimal versions that aren’t in the 3-digit decimal format, that’s fine, too. It’s not worth switching for code you’ve already published.

If you disagree with me or would like to see alternatives or want to know the ugly details behind this recommendation, then the rest of this article is for you.

Can we skip the history lesson?

It will be helpful to review at least a little history, then we can ignore how we got here and just focus on what to do. There are three big historical accidents worth noting:²

Accident #1: Perl expects a module to put its version number in a package variable called $VERSION

The first accident means that a module version number can be anything that can be represented by a Perl scalar: a number, a string, an object, etc. There are no constraints, only conventions. This is a blessing and a curse; it’s the main reason why version numbers in Perl aren’t boring.

Accident #2: The Perl 5.6 interpreter changed from a ‘decimal’ version number (5.005_03) to a ‘dotted-integer’ version number (5.6.0)

The second accident got people thinking that ‘version numbers’ for modules shouldn’t be decimals either. It also introduced a three-digit convention for translating between the two approaches: 5.6.2 is equivalent to 5.006002 and 5.005_03 is equivalent to 5.5.30. Finally, it introduced new syntax for ‘v-string’ literals of the form ‘v1.2.3.4′ to help represent dotted-integer version numbers.

A third historical accident was actually an attempt to fix the second:

Accident #3: The Perl 5.10 interpreter made $^V a version object instead of a v-string

Version objects are an attempt to overcome numerous limitations and challenges using v-strings for module version numbers. The UNIVERSAL::VERSION() method also changed, to better accomodate version objects and v-strings.

Here is a very contrived example of the change to VERSION(), where 120, 100, 103 are just the ASCII codes of the letters of my IRC nickname:

$ perl5.8.8 -e '$VERSION=v120.100.103; print main->VERSION'
xdg

$ perl5.10.0 -e '$VERSION=v120.100.103; print main->VERSION'
v120.100.103

The version.pm module provides an interface to version objects and, for older versions of Perl, overrides UNIVERSAL::VERSION() globally to match the behavior of Perl 5.10. The version.pm module is one more thing that keeps version numbers from being as boring as I’d like.

$ perl5.8.8 -Mversion -e '$VERSION=v120.100.103; print main->VERSION'
v120.100.103

Think about the implications of that for a module called ‘Foo’. Even if Foo doesn’t use version.pm, if version.pm is loaded anywhere then Foo->VERSION acts differently and gives a different value than is in $Foo::VERSION.

Version numbers are for machines, not people

For the most part, version numbers are used to answer just two questions:

• What version of Module X does Module Y require in order to work properly?
• Is there a more up-to-date release of Module Z available?

Neither question is one that we, as people, particularly want to waste our time on if a computer can answer the questions for us. That means that version numbers need to be specified in a way that makes it easy for a computer to answer these questions.

Recommendation: Never use alphabetical characters in a version

Don’t use ‘1.20alpha’ or ‘2.34beta2′ or ‘2.00R3′. There are no standard conventions for alphanumerics, and you just make life hard for the machines, which means no help for the humans.

Decimals and dotted-integers

After following the first recommendation and excluding alphabetical characters for module versions, we are left with two numeric conventions, just like Perl itself:

• decimal numbers
• dotted-integers (also called dotted-decimals or tuples)

A decimal version is just what it seems: an ordinary number that is either an integer (12) or decimal fraction (12.345). Decimal versions are compared in the ordinary, numerical way.

Dotted-integer versions are vectors of (positive) integers, so called because they are usually shown separated by full-stop characters (12.3.45). Dotted-integer versions are compared by pairwise numerical comparison of the first element of each version, then the second, the third, and so on.

There are two ways to represent dotted-integer versions in Perl:

• v-strings
• version objects

I already described v-strings, but you can read more about them in the perldata documentation page. Despite what it says, v-strings are not deprecated. (This warning will be removed in a future version of Perl.)

Version objects are created using the version.pm module. There are several ways to do this. Here is one example (and it must be kept on one line):

use version; our $VERSION = version->new("v1.2.3");

The version.pm documentation has been updated in 0.77 to better explain the options and potential pitfalls creating version objects, so I won’t repeat them here.

John Peacock, the author of version.pm, recently suggested that it is probably enough to use version.pm and give a quoted $VERSION string without the need to explictly construct an object, as UNIVERSAL::VERSION will convert it to a version object internally anyway.³

use version; our $VERSION = "v1.2.3";

This approach is new and untested, but may eventually be regarded as the best way to use version.pm.

Three-digit convention

Perl decided upon on a three-digit convention to convert between the older decimal style and the newer dotted-integer style so that older Perls could give a useful error message when checking the Perl version number.

# on Perls before 5.6.0
use 5.6.2; # syntax error
use 5.006002; # Perl version error

The three-digit convention takes each dotted-integer component after the first, pads them with leading zeroes if less than 3 digits, and concatenates them together. To go the other way, the fraction portion of a decimal version is padded with trailing zeroes until the number of characters is a multiple of three, then each group of three digits is turned into an integer.

This convention has a subtle complexity: note that the conversion does not round-trip if a dotted-integer element is greater than 999. This might happen if a version number were constructed from a VCS revision number or a timestamp.

v1.2.34567 -> 1.00234567
1.00234567 -> v1.2.345.670

The introduction of the three-digit convention confused people in two ways: how to convert modules from decimal versions to dotted-integers and how to specify version number limits to use().

Many CPAN modules don’t use three-digit decimals, partly because most module boilerplate tools create modules with a two-digit decimal starting version:

$VERSION = "0.01";

These modules usually increment their versions as 0.02, 0.03 and so on. Some authors have been surprised trying to convert their decimal versions into dotted-integers and running afoul of the three-digit convention. In the following example, a naive conversion results in a number that is less than the previous one:

0.01
0.02
0.03
v0.4.0 # WRONG: this is 0.004
v0.40.0 # RIGHT: this is 0.040

The other area of confusion is providing a version number requirement to the use() keyword. With the introduction of v-strings, but prior to Perl 5.10.0, use() internally converts “use Foo v1.2.3″ to “use Foo 1.002003″. So this works as expected:

# in Foo.pm
our $VERSION = "0.001002";

# in foo.pl
use Foo v0.1.2; # WORKS

However, since Perl recommends specifying “use v5.6.0″ as “use 5.006″, some people think the same should apply to loading modules with use(). But, prior to Perl 5.10.0 (and the change to UNIVERSAL::VERSION), the inverse case might not work at all! Consider this example:

# in Foo.pm
our $VERSION = v0.1.2;

# in foo.pl
use Foo 0.001002;

On a Perl compiled with support for long doubles, the extra precision in converting with the three-digit convention causes the comparison to fail with this incredibly confusing error message:

$ perl-5.8.9-64bit -e 'use Foo 0.001002'
Foo version 0.001002 required--this is only version 0.001002

Recommendation: always use() in the same form as $VERSION in a module

When $VERSION is a v-string, it should only be requested as a v-string (”use Foo v0.1.2″), except on Perl 5.10 or when using version.pm. There is a corollary:

Recommendation: don’t switch version number schemes for a published module

If you do switch, then users won’t know in advance the right format to request.

Distribution version numbers

Distributions on CPAN also have version numbers. These are specified as part of the filename. (See a prior article for a formal definition of modules and distributions.)

DAGOLDEN/File-Marker-0.13.tar.gz

Recommendation: Set the distribution version from a module

Usually, distribution versions are set automatically from a primary module within the distribution. This is good ― we let the machine take care of it for us. It’s one less thing for us to do, and helps to ensure machine-friendly distribution versions. That’s good for binary packages like .rpm and .deb.

For example, using Module::Build, the ‘module_name’ parameter in Build.PL specifies the name of a module to examine for the distribution name and version.

use Module::Build;
Module::Build->new(
 module_name => 'Foo::Bar',
 license => 'perl',
)->create_build_script;

Or, using Module::Install, you can do the same thing with a Makefile.PL like this:

use inc::Module::Install;

name 'Foo-Bar';
all_from 'lib/Foo/Bar.pm';

WriteAll;

The underscore convention

It has been a long-standing CPAN convention that distribution version numbers containing an underscore are ‘development’ or ‘alpha’ versions, and the corresponding distribution files do not get indexed by PAUSE as a ‘release’ version.

DAGOLDEN/Test-Reporter-1.53_03.tar.gz

The three main build tools, ExtUtils::MakeMaker, Module::Build and
Module::Install will all attempt to parse a version number for a
distribution from a primary module file using the MM->parse_version() function provided by ExtUtils::MakeMaker. The parse_version() method looks for the first line in the file that appears to set $VERSION, and then calls eval() on that entire line.

The following sections show how to specify an alpha version in each of the three version number styles and some things to consider for each.

Decimal alpha version

our $VERSION = "0.001_001;
$VERSION = eval $VERSION;

For a decimal alpha version, the definition is split into two parts. The
first part provides the version in quotes, which is what gets returned by
MM->parse_version(). It has to be in quotes so that the underscore is preserved in the eval() call within parse_version(). Without quotes, parse_version() returns it as an ordinary decimal.

our $VERSION = 0.001_001; # WRONG: parse_version() gives 0.001001

The second line is required to make $VERSION an ordinary number at runtime. Without it, Perl would convert $VERSION to a number by truncating at the underscore, resulting in the wrong version number.⁴

Dotted-integer alpha version with v-strings

our $VERSION = v0.1_1;

For some versions of Perl, it’s possible to specify an ‘alpha v-string’, with a final decimal point replaced with an underscore.⁵ Consider a ‘Foo module with a $VERSION line like the one above. Here’s how different versions of Perl handle a request for a higher version number:

$ perl5.10.0 -e 'use Foo v0.1.2'
Foo version v0.1.2 required--this is only version v0.1_1 at -e line 1.
BEGIN failed--compilation aborted at -e line 1.

$ perl5.8.9 -e 'use Foo v0.1.2'

$ perl5.8.0 -e 'use Foo v0.1.2'
Foo v0.1.2 required--this is only v0.1.1 at -e line 1.
BEGIN failed--compilation aborted at -e line 1.

$ perl5.6.2 -e 'use Foo v0.1.2'

Note how Perl 5.8.9 and Perl 5.6.2 both succeed, even though a higher version is requested. One potential solution is to require the version.pm module. This ‘fixes’ 5.8.9, but fails in a different way for 5.6.2.

perl5.8.9 -Mversion -e 'use Foo v0.1.2'
Foo version v0.1.2 required--this is only version v0.1_1 at -e line 1.
BEGIN failed--compilation aborted at -e line 1.

perl5.6.2 -Mversion -e 'use Foo v0.1.2'
Foo version v0.1.2 required--this is only version v0.0.0 at -e line 1.
BEGIN failed--compilation aborted at -e line 1.

Dotted-integer alpha version with version objects

use version; our $VERSION = version->new("v0.1_1");

This form is visually similar to an alpha v-string, but by putting it in quotes and by passing it the version object constructor, the version is protected from idiosyncracies of different versions of the Perl interpreter.

On the surface, it seems like this is a good approach, but it still has problems if one tries to use() the same version on 5.8.0 and 5.6.2.

$ perl5.8.0 -e 'use Foo v0.1_1'
Foo version v0.1.1 required--this is only version v0.1_1 at -e line 1.
BEGIN failed--compilation aborted at -e line 1.

$ perl5.6.2 -e 'use Foo v0.1_1'
Foo version 0.011 required--this is only version v0.1_1 at -e line 1.
BEGIN failed--compilation aborted at -e line 1.

Requiring an alpha version as a decimal

The counter-examples in the previous sections assume a situation where an alpha version of a module is a prerequisite and show how an alpha v-string argument to use() means different things depending on the version of Perl and whether version.pm has been loaded.

The other alternative is to specify the version as a decimal.

use Foo 0.001_001;

While this is in the form of a decimal alpha version, the Perl parser sees this numeric literal as 0.001001. This works perfectly with the decimal alpha version example above that has the “$VERSION = eval $VERSION” line. Both are just numbers and they compare as equal.

That isn’t the case for version objects. An alpha version object is not equal to a non-alpha version object, even if their numeric components are the same. If Foo has this version line:

use version; our $VERSION = version->new("v0.1_1");

then “use Foo 0.001_001″ fails, even on a recent Perl.

$ perl5.10.0 -e 'use Foo 0.001_001'
Foo version 0.001001 required--this is only version v0.1_1

Recommendation: Don’t use v-strings or version objects as alpha versions

There are just too many ways for alpha v-strings and version objects to be used incorrectly. Even if requiring Perl 5.10 or with version.pm, which a module author can control, if a user follows the three-digit convention and uses a decimal version in the call to use(), it can fail.

Other issues in the toolchain

There are several other ways in which version numbers refuse to be boring. Each could be an entire mini-essay, so I will only highlight a few of the issues I’ve found:

• Module::Build can’t handle v-string prerequisites unless they are quoted
• ExtUtils::MakeMaker and Module::Install have trouble with v-string module versions and prerequisites, even if quoted or given as version objects
• In at least one recent bug I’ve studied, an XS module failed to load with a version object created using the form “1.2.3″ (without a leading-v)

Conclusion

I think version numbers should be boring. I don’t want to have to think about how to write them and I don’t want to make users think about how to format a version prerequisite. And I don’t want to get bug reports about it when a user gets it wrong.

For me, a ‘boring’ version number must:

• be expressed the same way in its definition, in use() checks, in prerequisites, in *.PL files and in META.yml
• support the underscore convention in the same format
• have similar behavior across different Perl versions
• have similar behavior in use() when converted from X.YYYZZZ to vX.Y.Z format or vice-versa⁶

Given these criteria, my recommendation is to use decimal version numbers, put them in quotes, and add a string eval:

our $VERSION = "0.001";
$VERSION = eval $VERSION;

This is safe and effective and always works. By putting any $VERSION in quotes, even if it isn’t an alpha, you don’t have to remember to add them if you ever change to an alpha version. (And numbers with trailing zeroes are nicely formatted when parsed for distribution versions.)

If you really want to have a dotted-integer module version, then I strongly recommend that you limit your module to Perl 5.10 (or require version.pm and at least Perl 5.8.1) and that you never use an alpha version number. Always quote your dotted integer version when you define it and always use a leading-v to guide your users towards proper usage.

It’s unfortunate that version numbers are so complicated in Perl, but if you follow the recommendations in this article, your version numbers will be as boring as possible. And if you’ve read this all the way to the end, I hope I’ve convinced you that ‘boring’ version numbers are exactly what you want.

I would like to thank Curtis Poe, Ricardo Signes and Eric Wilhelm for reviewing a draft of this article and providing excellent suggestions for improving clarity and correctness. I would also like to thank the particpants in the #toolchain and #corehackers channels on IRC for being a sounding board as the article was developed. Thanks also go to John Peacock for his graceful acceptance of my criticisms and suggestions about version.pm.

1. N.B., the Perl Best Practices book is wrong
2. They were actually intentional, but had unforeseen consequences
3. The ‘use version;’ part isn’t even required if the code is limited to Perl 5.10 or later.
4. Perl converts strings to numbers differently at runtime than how it parses numeric literals during compilation.
5. This is an undocumented syntax feature/bug
6. Remember that users might write a dotted-integer as a decimal, just like they are taught for Perl itself.

Posted by dagolden on perl, ironman at Wed, 05 Aug 2009 23:25:44 -0000 | Permalink

We had a BOF ("Birds Of a Feather") for marketing Perl yesterday. Dave Cross organized it and I kicked things off. My agenda didn't quite go as planned. I started by laying out the goals:

1. No solutions were to be proposed. I stressed this several times and people apologized and then presented their solutions.
2. We needed to define a clear goal of what we were hoping to achieve.
3. Once a goal is defined, determine what information we needed to achieve that goal.
4. When we know the information, we could then start work on understanding how to collect this information.

The BOF didn't follow that. I didn't think we'd get all of these concerns addressed, but when we had 51 people showing up and pulling things into different directions, it became herding cats. I tried unsuccessfully to pull things back on track, but everyone was insisting on explaining the problems they had with Perl and the solutions they saw for the problems. After cutting a few people short to ensure that others got a chance to talk (frustrating because some people who went on at length had excellent lists of issues that they wanted to address), I went with the flow and steered things from there. Here's what came out of it.

We couldn't agree on a clear goal, so I just pulled rank and said "our goal is 'bringing the excitement back to Perl'" and we moved on from there. Then one person (I didn't catch his name, sorry), asked the annoyingly obvious question: how do we measure success? We have several ideas here, but we need more. Job postings, analyzing server logs from perldoc.perl.org (a lovely site, by the way) and other Perl sites, tracking articles and similar things. We don't yet know the most effective things that will be needed here, but we know we need this.

Richard Dice suggested that when companies are worked with (he had a lot of say about the selection of companies), we need to have a neutral way of getting information. Imagine a matrix with the vertical axis listing value drivers ("availability of programmers", "speed of development", "commercial support" and so on) and the horizontal axis listing various technologies and have them "score" their perceptions of these technologies. I've no idea if this is the way we'll go (yes, we know we need to talk to more than just companies), but it's much better than what we previously considered.

There was also general -- not universal -- agreement that we need to have market research and analysis professionals involved. How we pay for this is unknown at this time, but Gabor and others had plenty of ideas on how to make this happen.

Edmund von der Burg, the brains behind send-a-newbie.com, surveyed the newbies for their attitudes. It's not a comprehensive or formal survey, but that information was illuminating nonetheless.

Some of the various issues which were raised (no endorsement, just mentioning issues):

• No one hears about important Perl projects. Everyone knows about Rails, Django, WordPress, etc. Perl doesn't have a lot of visibility outside the community.
• Perl sites are ugly. If you here Perl is dead, our community will bear that out. Compare the first Google hit for Ruby and the first Google hit for Perl. It's embarrassing.
• TIMTOWTDI is a problem. Companies are finding it makes system administration a nightmare and newcomers complain about being lost in a sea of choices.
• People outside of Perl think that Perl 5 is waiting to die and be replaced by Perl 6.
• Programmers trying to learn Perl are having trouble finding resources online. The first Google hit for "Perl tutorial" is a tutorial for Perl 4.

There was quite a bit of information that I learned which I didn't previously know, but perhaps one of the most fascinating things was this: in many countries, Perl does not have a negative reputation because no one's heard of Perl. If that's not an opportunity, I don't know what is. That might be beyond our scope (I really don't know and it would be hard to quantify), but I highly recommend people think about how we could change that.

Obviously there's a lot more than that and we have more work to do, but this is a good start. Some potentials can't be discussed yet (there are some companies which might be willing to provide advice/money/assistance) and others aren't finalized, but we're starting to get things on track.

Posted by Ovid on journal at Wed, 05 Aug 2009 10:32:24 -0000 | Permalink

Someone on IRC reported a bug in Bryar. Namely that a Naughty Person can exploit the feature that notifies you of blog-spam by email to execute arbitrary code on your machine, as the user you run Bryar under.

A patched release is on the way to the CPAN, and you are strongly urged to upgrade.

Posted by david | Permalink

[originally posted on Apr 3 2008]

I'd like to express my warm thanks to the lovely people at Yahoo and in particular to their bot-herders. Until quite recently, their web-crawling bots had most irritatingly obeyed robot exclusion rules in the robots.txt file that I have on CPANdeps. But in the last couple of weeks they've got rid of that niggling little exclusion so now they're indexing all of the CPAN's dependencies through my site! And for the benefit of their important customers, they're doing it nice and quickly - a request every few seconds instead of the pedestrian once every few minutes that gentler bots use.

Unfortunately, because generating a dependency tree takes more time than they were allowing between requests, they were filling up my process table, and all my memory, and eating all the CPU, and the only way to get back into the machine was by power-cycling it. So it is with the deepest of regrets that I have had to exclude them.

Cunts.

[update] For fuck's sake, they're doing it again from a different netblock!

Posted by david | Permalink

As a service to module authors, here is a tool to show a module's pre-requisites and the test results from the CPAN testers. So before you rely on something working as a pre-requisite for your code, have a look to see how reliable it and its dependencies are.

Posted by david | Permalink

I am ill. I've been ill since Thursday, with a cold. You're meant to be able to cure a cold with [insert old wives tale remedy here] in 5 days, or if you don't, it'll clear itself up in just under a week. So hopefully today is the last day.

So what have I done while ill?

On Friday I became old (see previous post), and went to the Byzantium exhibition at the Royal Academy. It was good. You should go.

Saturday was the London Perl Workshop. My talk on closures went down well, and people seemed to understand what I was talking about. Hurrah! I decided that rather than hang around nattering and going to a few talks, I'd rather hide under my duvet for the rest of the day.

I mostly hid on Sunday too, and spent most of the day asleep. In a brief moment of productivity, I got my laptop and my phone to talk to each other using magic interwebnet bluetooth stuff. I'd tried previously without success, but that was with the previous release of OS X. With version X.5 it seems to Just Work, so no Evil Hacks were necessary.

The cold means that I can't taste a damned thing, not even bacon. So now I know what it's like to be Jewish. Being Jewish sucks.

And today, I am still coughing up occasional lumps of lung and making odd bubbling noises in my chest, although my nasal demons seem to be Snotting less than they were, so hopefully I'll be back to normal tomorrow.

Posted by david | Permalink

Devel::CheckLib has grown a new feature. As well as checking that libraries and headers exist, it can now also check that particular functions exist in a library and check their return values. This will be particularly useful if you need to check that a particular version of a library is available.

It works if you have a Unixish toolchain. I need to wait for the CPAN-testers reports to see if I managed not to break anything on Windows. Unfortunately, even though the lovely Mr. Alias has worked hard to make Windows machines available to developers, I found it to be just too hard to use. Even downloading my code to the Windows machine was hard, as Windows seemed to think it knew better and shouldn't download the file I told it to download. Then once I had downloaded it, Windows decided to hide it somewhere that I couldn't get to using the command line. So I gave up.

I might try again once there are some decent tools on the machines: wget, tar, and gzip at minimum, as given those I can quickly bootstrap anything else. Software development isn't just about having compilers available.

Posted by david | Permalink

One of my CPAN distributions is CPAN-FindDependencies. It contains a module CPAN::FindDependencies, and a simple script that wraps around it so you can view dependencies easily from the command line. That script, naturally, has a man page. However, that manpage basically says "if you want to know what arguments this program takes, see the CPAN::FindDependencies docs". This is Bad from a usability point of view, good from a not-duplicating-stuff point of view, and good from a laziness point of view. Which means that it's Bad.

So, the solution.

=over

#include shared/parameters

=back

and some Magic that does the cpp-stylee substitution at make dist time. Note the 'dist' section in my call to WriteMakefile.

This is, of course, crying out to be made less horribly hacky, but it works for now, so I'm happy.

My original idea was to write some crazy shit that would do the #include at install-time, when the user was installing my code. But that has the disadvantage that tools like search.cpan wouldn't show it properly, as they simply look at the files in the distribution. So this does the #includes at the last moment just before I package up the code and upload to the PAUSE. You lovely people get the right documentation in all the right places, I only have to maintain it in one place so it stays in sync, and (in the interests of Laziness) I don't have to remember to run any extra scripts before releasing, make dist just Does The Right Thing.

Posted by david | Permalink

I have released my XML::Tiny module. The parser at its core is less than twenty lines of code. Pretty easy to follow code too, I think, and that also includes error handling. One of my aims in writing it was to keep memory usage and code to the absolute minimum, so it doesn't handle all of XML. The documentation says that it supports "a useful subset of XML". Personally, I think it supports the useful subset. It's certainly enough to parse the data I get back from Amazon when I use their web services, and to parse an RSS feed.

Posted by david | Permalink

As is becoming normal, I used the times between talks to bugfix some of my modules - this time Tie::STDOUT and Data::Transactional. The former was failing on perl 5.6, the latter on 5.9.5. The former was a bug in perl (you can't localise tied filehandles and expect the tieing to go away in 5.6, so it now declares a dependency on 5.8), the latter was a bug in my code.

Philippe Bruhat's talk on Net::Proxy was great - you can tell it's great because I came away with ideas for at least four things that I need to write. First up will be a plugin for it to allow the user to specify minimum and maximum permitted data rates for proxied connections. This will permit bandwidth limits for maximum permitted rates, but will also help to defeat IDSes doing traffic analysis if you specify a minimum permitted data rate.

This will protect (eg) ssh sessions from being identified based on their very bursty traffic pattern, by "filling in the blanks" with junk data.

In the evening, the CPAN-testers BOF was productive.

Posted by david | Permalink

To make up for a disappointing gap in Palm's software for the Treo smartphone, I wrote a <a href=http://www.cantrell.org.uk/david/tech/treo/call-dumper/>small perl script</a> to parse the database that stores my call history. I then re-wrote it as <a href=http://search.cpan.org/search?query=Palm%3A%3ATreoPhoneCallDB>a re-useable module</a> which also figgers out whether the call was incoming or outgoing.

Posted by david | Permalink

There's a new release, <a href=http://www.cantrell.org.uk/david/tech/perl-modules/Number-Phone-1.58.tar.gz>version 1.58</a>, of Number::Phone, my set of perl modules for picking information out of phone numbers. Changes from the previous release are that Mayotte, Reunion and Comoros can't decide which country is which, and there's the usual updates to the database of UK numbers, mostly to support the <a href=http://www.ofcom.org.uk/media/news/2007/02/nr_20070213b>new 03 numbers</a>.

Posted by david | Permalink

<a href=http://cpandeps.cantrell.org.uk/>CPANdeps</a> now lets you filter test results by perl version number, and also knows what modules were in core in which versions of perl. Hurrah!

Posted by david | Permalink

A day of not many talks, but lots of cool stuff. Damian was his usual crazy self, and MJD's talk on building parsers was really good. Although I probably won't use those techniques at work as functional programming seems to scare people.

The conference dinner at a Heuriger on the outskirts of Vienna was great. The orga-punks had hired a small fleet of buses to get us there and back, and one of the sponsors laid on a great buffet. The local wine was pretty damned fine too, and then the evening de-generated into Schnapps, with toasts to Her Majesty, to her splendid navy, and to The Village People.

It wasn't all debauchery in the evening though - on the bus, I had a very useful chat with Philippe about Net::Proxy, and re-designing it to make it easier to create new connectors for it.

Posted by david | Permalink

There were quite a few interesting talks in the morning, especially Ivor's one on packaging perl applications. Oh, and mine about rsnapshot, of course, in which people laughed at the right places and I judged the length of it just right, finishing with a couple of minutes left for questions.

At the traditional end-of-YAPC auction, I avoided spending my usual stupid amounts of money on stupid things, which was nice. Obviously the hundred quid I put in to buying the hair style of next year's organisers wasn't stupid. Oh no. Definitely not.

An orange mohican will suit Domm beautifully.

Posted by david | Permalink

My Lightning Talk on cpandeps went down really well, although as Jos? pointed out, I need to fix it to take account of File::Copy being broken. I also need to talk to Domm after the conference is over to see if I can get dependency information from CPANTS as well as from META.yml files.

There were lots of other good lightning talks. Dmitri Karasik's regexes for doing OCR, Juerd Waalboer's Unicode::Semantics, and Ren?e B?cker's Win32::GuiTest were especially noteworthy.

Richard Foley's brief intro to the perl debugger was also useful. Unfortunately Hakim Cassimally's talk was about debugging web applications, which I'd not noticed on the schedule, so I didn't stay for that.

And finally, Mark Fowler's grumble about why perl sucks (and what to do about it) had a few interesting little things in it. I am having vaguely sick ideas about mixing some of that up with an MJD-stylee parser.

At the auction I paid ?250 to have the Danish organisers of next year's YAPC::Europe wear the Swedish flag on their foreheads. This, I should point out, was Greg's idea. I would never be so evil on my own.

Posted by david | Permalink

Perl isn't dieing, but it tells me that it wishes it was. Last night it went out on the piss with Python and Ruby (PHP was the designated driver) and it did rather too many cocktails. It isn't quite sure what happened, but it woke up in the gutter in a puddle of its own fluids and its head hurts a lot.

It asked me to ask you all to keep the volume down.

Posted by david | Permalink

I'm going to Vienna by train for YAPC::Europe. If you want to join me you'll need to book in advance, and probably quite some way in advance as some of these trains apparently get fully booked.

	arr	dep	date
Waterloo		1740	Fri 24 Aug
Paris Nord	2117
Paris Est		2245
Munich	0859	0928	Sat 25 Aug
Vienna	1335

The first two legs of that are second class, cos first wasn't available on Eurostar (being a Friday evening it's one of the commuter Eurostars and gets booked up months and months in advance) and was way too spendy on the sleeper to Munich. Upgrading to first class from Munich to Vienna is cheap, so I have.

Coming back it's first class all the way cos upgrading was nearly free ...

	arr	dep	date
Vienna		0930	Fri 31 Aug
Zurich	1820
Zurich		1402	Sun 2 Sep
Paris Est	1834
Paris Nord		2013
Waterloo	2159

Don't even think about trying to book online or over the phone, or at the Eurostar ticket office at Waterloo. Your best bet is to go to the Rail Europe shop on Picadilly, opposite the Royal Academy and next to Fortnums.

Posted by david | Permalink

I got irritated at how hard it was to use Wikipedia on my Treo. There's so much rubbish splattered around their pages that it Just Doesn't Work on such a small screen. Given that no alternatives seemed to be available - at least, Google couldn't find any - I decided to write my own Wikipedia handheld proxy.

It strips away all the useless rubbish that normally surrounds Wikipedia pages, as well as things like the editing functions which are also hard to use on portable devices. Internally, it's implemented using perl, LWP, and mod_perl, and is hosted by Keyweb.de.

Posted by david | Permalink