Perlsphere

I enjoyed Aldo Cortesi's rather interesting post about language statistics on github. He's done some good analysis, and there are some interesting nuggets of information to be had about Perl, Haskell (though fewer, as there are only 18 projects that made his criteria) as well as other languages.

Of course, there is some silliness there too: you can bash Perl for many reasons (and if you've read my blog before, you might know that I do too), but there are some gems of forced interpretation:

C and Perl projects show a marked decline in activity over their first year. I suspect that the Perl result is due to the fact that it becomes harder and harder to contribute to a Perl codebase, the bigger it gets. The C result is more of a mystery.

I'm not sure that the premise is true -- perhaps Perl projects are more limited in scope, for example. And Modern Perl is a quite different beast from the Matt's PERL script archive of yesteryear. But the punchline is priceless ;-)

Here's what I read with my biased interpretation of his results ;-)

• Far fewer Perl projects than Ruby/Python so far. Github is a Ruby community effort, so it's unsurprising that it would dominate here.
• Median contributors for Perl is above average. This is substantiated by the total contributors for long running projects being comparable with Ruby/Python.
• Perl projects seem to have many, small commits. This would seem to be a good thing, and rather in keeping with the Git Way. (/me shuffles embarrassedly at the sight of his own, rather monolithic git commits...)
• While Aldo suggests Perl projects are "significantly more "top-heavy" than those in other languages, with a smaller core of contributors doing more of the work," one could also hypothethize that Perl projects are good at attracting and retaining a strong core team. This certainly seems to be the case with long running, active projects such as Catalyst and Moose.

So, thanks Aldo for taking the time to do this fascinating analysis (though I'm sure you won't mind if I draw some slightly different conclusions than you ;-P)

Posted by osfameron on Uncategorized, perl, git, github, statistics at Tue, 15 Dec 2009 11:26:07 -0000 | Permalink

I can be a bit of a lazy open source author at times. I love it when someone improves my code and then just emails me patches. I love it even more now that I'm using Github so that people can just fork my code and send me pull requests.

That happened over the weekend. I got a pull request from cycles saying that he'd been refactoring Perlanet and asking if I'd be interested in merging his changes. These refactorings had come out of a mini-hackathon that had been run by the North West England Perl Mongers. I knew that they had been interested in using Perlanet to power the Iron Man aggregator, but that the monolithic nature of  the code made it hard for them to subclass the bits that they needed to change.

So what cycles has done is to take my code and break it down into a number of smaller functions (and a couple more classes) so that people can subclass it and override methods where they want different behaviour to the defaults that I use. I can see this leading to a little ecosystem of Perlanet subclasses where people release their favourite tweaks to CPAN. I see this as a good thing.

I liked cycles' changes so I merged them into my repository. They're available there now if you want to look. I haven't yet released a new version to CPAN as there are a few things I want to check out first.

Firstly, this new update seems to have increased the version of Moose required. With the version of Moose that I was running on my system (0.88), a lot of the tests were failing. Updating to 0.93 solved that problem. I need to work out exactly what the problem was and update the "requires" line in Build.PL appropriately.

Secondly, the newer version of Moose gave a deprecation warning when used with CHI. Updating to the latest version of CHI fixed that (but that, in turn, meant upgrading a few modules in the Log::Any family). This is all starting to get a bit too close to the bleeding edge of CPAN for a module that I want as many people as possible to use.

Finally, cycles had started to use TryCatch in the module. Not, of course, that I object to high quality exception handling in my code, but this is another module that isn't yet in general use. It's something that you won't find in a "standard" (whatever that means) Perl installation.

I'm in the process of building RPMs of all of the missing modules (or later versions for the modules where the Fedora/Centos build is just lagging CPAN a bit). They'll be available from rpm.mag-sol.com in the next few days.

Currently I'm leaning towards just releasing the new version and hoping that the people who want to use it will have enough enthusiasm that they won't complain about the updated and new modules that are required. But I thought it would be interesting to ask for your opinions too.

As a user of CPAN modules, how do you decide when a module is too cutting edge for you to use? Do you just install newer versions of modules automatically when an installation asks you too? Or are you a little more careful than that? Would the constraints in this latest version of Perlanet prevent you from using it?

And as a writer of CPAN modules, how cutting edge do you allow yourself to be? Are you happy to release stuff that only works with the very latest versions of Moose or other fast-moving modules? Or do you like to ensure that your stuff is usable by people who might be a little behind the curve?

I should make it clear that I'm very grateful for the work that cycles did and I'm not disparaging his efforts at all. I'm just dithering a bit about how cutting edge I want to be.

Posted by Dave Cross on CPAN, bleeding edge, cpan, github, modules, nwe.pm, perlanet at Tue, 15 Dec 2009 08:40:16 -0000 | Permalink

Here’s another short one this week…

I first learned about Linux::Inotify2 from a posting on acme’s blog. It’s a neat interface to the Linux Inotify system, which itself is basically a way of being informed whenever things happen in the file-system.

I was able to make use of this almost immediately– I took the sample script from acme’s post and made a few adjustments, then used it to look at the libraries that Image::Size loads during start-up. Turns out that there were a couple of them that didn’t need to be loaded until/unless needed, and once I saw this I was able to move them around and speed up the compile-time phase of the module.

This module has some great potential for monitoring, measuring, etc. I’m toying with a graphical version of acme’s script that I can have running during test suite runs, to get a better idea of other places I can improve things.

Check out the module, and check out acme’s original post on the subject, too!

Posted by rjray on CPAN, Perl, module-monday at Tue, 15 Dec 2009 07:43:20 -0000 | Permalink

I've written before about Australia's Superannuation private pension accounts, and how the trade unions have managed to turn their lack of conflict of interest into rather excellent $100 billion share of the $1 trillion total (this number does not include government super funds, which share a similar benefit).

While still less than 50% of the total, these funds share such a large performance advantage (between 0.5% and 1% per year better than commercial funds) that they are starting to become the new "normal" for what these funds "should" be like.

Assuming similar basic investment performance (and given the rather strict rules around what Super funds are allowed to invest in, this is likely) some quick back of the envelope math suggests that commercial management of the remaining half trillion or so is costing their members around $5 billion per year, while adding no value (something like $1,000 per person per year).

There's a good reason the industry refers to Super as the "river of gold".

I had been pondering whether or not an influx of Union Bankers might cause changes in the (still newish) Labor government. While I can't determine any kind of causative link, the government has decided to do something about this disparity between the two.

It appears that the rules around "default investment behaviour" are to be rewritten based on behavioural economics. It's been observed that the majority of Super members will just use whatever their employer recommends, and will NEVER touch their accounts to tweak them. Most famously, in the recent stock market crash it was observed that a mere 2% of Super accounts were altered to defend against the crash.

Under the new rules, the "defaulting" is likely to look a lot more like the industry funds. Less options, simpler investment profiles, and lower fee structures. Employees will need to specifically choose to leave this "Universal Option" in order to get access to more complex investment products.

Altering the rules to assume that people are lazy and forgetful is a change well worth while, and a long time coming. And with the Super pool on track to reach around $3.5 trillion, these changes should save the country (in aggregate) a hell of a lot of money.

Of course, the industry super funds still have more things to complain about. Their second big issue is the paying of "financial advisors" fairly hefty commissions based on Super account subscriptions. Industry funds generally refuse to pay commissions (after all, most of the members are send to them by the Trade Unions) and so it's no great surprise that despite their better performance few retail advisors will send customers to these funds (another conflict of interest example is almost every "consumer choice" website, and no-fee financial advisors).

But this problem is possibly a bit stickier than the behaviour economics refactoring, so I would expect this problem to be put to one side until after the "Universal Account" changes have been fully digested by the financial industry.

Posted by Alias on journal at Tue, 15 Dec 2009 00:52:48 -0000 | Permalink

What version of Perl 5 is necessary to run this example?

say $message;

It depends. The obvious answer is any version newer than 5.10, at least if you use feature ':say'; or use 5.010;.

It can also be Perl 5.6 or 5.8, if you use Perl6::Say.

What version of Perl 5 is necessary to run this example?

sub foo
{
    state $foo = shift;
    say $foo;
}

foo( 100 );
foo( 200 );
foo( 400 );
foo( 800 );

Again, the obvious answer is any version newer than 5.10, but I can imagine having written code something like this for Perl 5.6 or 5.8:

my $foo;
my @lexicals;

sub state :lvalue
{
    push @lexicals, \(my $foo);
    ${ $lexicals[-1] } = shift;
    return $lexicals[-1];
}

The semantics aren't exactly the same, but that's the point. The syntax is similar enough that you can't reliably read the code and determine to which Perl 5 version the original author wrote!

This problem comes up over and over when someone proposes adding a new keyword to Perl 5. The likelihood that someone wrote Perl 5 code with this syntax is vanishingly low:

class Foo extends Bar {
    ...
}

... but it's not entirely impossible.

There is no general purpose heuristic by which the Perl 5 parser can determine the appropriate version of the Perl 5 syntax to use to parse any given piece of code. (That's not a problem specific to Perl 5, but I'm only talking about the Perl language family now.)

No one wants to break existing code. Yet if there's value to adding new features, there are only a few possibilities to find a balance between adding new features which may conflict with existing programs and not adding new features.

The current Perl 5 approach prefers to maintain existing programs unchanged. It's possible to take code written for Perl 1 (and last modified in 1987) and run it unmodified on Perl 5.11.2, released last month. I repeat, unmodified.

In other words, given a Perl 5 program which does not specifically declare the intended version of the Perl 5 syntax and (more important) keyword set, the Perl 5 parser prefers the oldest available keyword set. (Of course, any syntax which was impossible in older versions and is now available in newer versions may be available... but that doesn't require modifications to existing programs.)

That's not likely to change soon, if ever in Perl 5, and I can understand that policy especially given its two-decade history.

That does suggest changing how the Perl 5 community thinks about writing new code and modifying existing code under active development. I'll write more about that next time.

Posted by chromatic on modern perl, perl 5, perl programming, version numbers at Mon, 14 Dec 2009 23:38:44 -0000 | Permalink

The gitPAN import is complete.

From BackPAN
------------
118,752 files
10,440,348,937 bytes (measured by adding individual file size)
21987 distributions (I skipped perl, parrot and parrot-cfg)

To git
------
21,766 repositories
4,495,204 bytes (measured by total disk usage
                                  after git gc with no checkout)
150 gigs on github (they have to index it)
12 days (lots of starts and stops)
1 laptop (1st gen Macbook)

I had to do it on a disk image because OS X's case-insensitive filesystem

I've written up a small FAQ. gitpan is reasonably stable, but you may have to rebase in the future.

Next, I take a break.

Then begins the second pass, mostly improving and adding tags. Here's the list of planned features. The second pass will be a rolling reimport of each distribution to bring everything up to the same standard, there was a lot of incremental improvements during the first pass. I expect this to be changes to commit logs and tags with very little content change.

The issue of PAUSE ownership I'm going to punt on. Its ugly and can be done entirely in parallel. If someone else makes available a historical distribution ownership database, gitPAN will use it.

Posted by schwern on journal at Mon, 14 Dec 2009 22:27:48 -0000 | Permalink

MySQL and Oracle. (Despite teething pains, blogs.perl.org is holding up quite well on the new server)

Posted by Ovid on journal at Mon, 14 Dec 2009 12:56:02 -0000 | Permalink

I'm happy to report that our Catalyst + PostGIS application Geo2gov took 2nd prize, plus one bonus prize, in the Australia government's MashupAustralia competition. Jeffery Candiloro (co-developer on the project) and myself will split $6,000 in prize money :)

"geo2gov serves as an excellent example of what can be possible with open government data. This entry provided an online service that will take a location description in a wide range of formats, and map that location to the government. The testament to its utility was demonstrated by the fact that several other entries used geo2gov. Contest judge Mark Pesce said that this app that was such an impressive prototype of what was possible with government data that it made his geeky pants wet."

Posted by Alias on journal at Mon, 14 Dec 2009 07:41:33 -0000 | Permalink

Posted by rjray on CPAN, Perl, Software, XML, RPC-XML at Mon, 14 Dec 2009 05:48:56 -0000 | Permalink

last time I used this topic, it was just about the line editing commands. of cource there is more. and the major idea is to pay attention to what you need right now, while you trying to achieve something another way that you do in your imagination. Its the Perl principles, brought into the realm of coding.

One of the first things we did extra to the basic edit functions was an idea of manuel. He brought this up, even before the perl workshop in dresden and I called it, lacking a better term, braceindent. You may know autoindent which let you start after pressing enter with a new line the has same indentation like the last one had. Kephra has that too but if you press enter after an open curly bracket ({), your next line is indented one tab or several spaces more (depends on your settings for that document). It also creates the matching brace below you cursor, already correctly indented. But many may not like that, so this partial behaviour is also optional.

Of course Kephra highlights matching brace, but you can also navigate through matching braces with very easy key combination, just alt+cursor not that crypting things you may found elsewhere. I did this because braces reflect the logical structure (blocks) of code and if you can easy navigate them you may easier get to the position in code you looking for than with a search. Folding commands which also mainly ease navigation and orientation in code lay on key commands that are very similar to allow fast navigation. Sometimes you want to select easily also whats inside a block/braces or string. Vim reminded me on that and one of the next Kephra versions (0.4.4) should bring that too.

0.4.3 is just too close. most features are written and im currently just cleaning the cage.

Posted by sir_lichtkind on journal at Sun, 13 Dec 2009 17:03:54 -0000 | Permalink

libnova is a Celestial Mechanics, Astrometry and Astrodynamics Library written in C. Just yesterday, I uploaded an inital, thin XS wrapper to CPAN as Astro::Nova, so we can use it from Perl.

Here's a simple example that calculates the current moon rise, transit and set times at my home in Karlsruhe using Astro::Nova. It's quite similar to the equivalent example of the C version.

use strict;
use warnings;
use Astro::Nova qw/:all/;

my $observer = Astro::Nova::LnLatPosn->new();
# observer location: Karlsruhe, for rst 
$observer->set_lat( Astro::Nova::DMS->from_string("49°00' N")->to_degrees );
$observer->set_lng( Astro::Nova::DMS->from_string("8°23' E")->to_degrees );

my $now = get_julian_from_sys(); # current julian date

print "Current Julian Day: $now\n";

my $moon_from_earth = get_lunar_geo_posn($now, 0);

my ($moonx, $moony, $moonz) = ($moon_from_earth->get_X, $moon_from_earth->get_Y, $moon_from_earth->get_Z);
printf("Moon is at (%.02fkm, %.02fkm, %.02fkm)\n", $moonx, $moony, $moonz);
printf("Moon distance: %.02fkm\n", get_lunar_earth_dist($now));


my $moon_lnglat = get_lunar_ecl_coords($now, 0);
print "Moon at:\n", $moon_lnglat->as_ascii();

my $moon_equatorial = get_lunar_equ_coords($now);

my $moon_fraction = get_lunar_disk($now);
print "Current moon fraction: $moon_fraction\n";
print "Moon phase " . get_lunar_phase($now) . "\n";

my ($status, $moon_rst) = get_lunar_rst($now, $observer);
if ($status == 1) {
  print "Moon is circumpolar\n";
}
else {
  print "Rise time:\n",    get_local_date( $moon_rst->get_rise() )->as_ascii(), "\n";
  print "Transit time:\n", get_local_date( $moon_rst->get_transit() )->as_ascii(), "\n";
  print "Set time:\n",     get_local_date( $moon_rst->get_set() )->as_ascii(), "\n";
}

The interface is mostly still very C-ish and that's unlikely to change entirely, but the various containers will probably gain a few more convenience and conversion methods.

Next piece of the puzzle: a module for reading a star catalog. Astronomy is fun.

Posted by Steffen Mueller on astronomy, libnova, perl, perl modules, perl astronomy libnova perl modules at Sun, 13 Dec 2009 16:57:41 -0000 | Permalink

this was originally posted on my new blogs.perl.org journal, which can be found here which is also the RSS feed for it.

Module::Starter 1.54 was recently released. I felt like blogging a bit about it.

Module::Starter is one of my favorite projects in the Perl community and on CPAN. I use it whenever I want to start a new module or program. It's very simple (read: beginner-friendly), modest and provides what I don't feel like doing myself. It has boilerplate files, text and tests and keeps updating those with habits that are considered "best practice", such as replacing the default licensing term we use.

Recently I've had the pleasure and honor of contributing to this project. I've made several changes mainly taking care of tickets and patches people submitted, making this release a fixer upper release. Following releases will contain some bigger and more exciting changes.

I want to take this chance to thank Andy Lester, Ricardo Signes, Shlomi Fish and anyone else involved with Module::Starter. Also, I want to thank some lesser-known contributors to Module::Starter, those that send bug reports, feature requests and patches (on my god, patches *drool*).

It seems like Ricardo's Dist::Zilla is the contender for Distribution Manager position, and with just cause. However, I rest assure that there are plenty of people (such as me) who prefer using small specific tools for each job they have. I prefer Module::Starter as a module starter, Module::Build as a Distribution Manager, etc.

That being said, you should still totally check out Dist::Zilla and see if it's the right tool for you. But don't worry, Module::Starter is not going away. :)

Posted by xsawyerx on journal at Sun, 13 Dec 2009 16:49:15 -0000 | Permalink

Posted by draegtun on Blogging, Programming, perl at Sun, 13 Dec 2009 14:28:05 -0000 | Permalink

After some time with livejournal.com, I have decided to move on. I have decided to use a domain name, which have been in my possession for ages (since 2002-04-18), however pretty unused.

So my new blog will be at: e-diot.dk

The blog is a Wordpress blog and it does give me a more fine grained control of the contents and it’s rendering and layout, using the numerous plugins.

I still have a lot of stuff to do, like customizing the twitter plugin, getting all settings running correctly with MacJournal and last but not least, get some more blog entries published.

It has been fun... over and out

jonasbn

Posted on e-diot.dk, blogging, wordpress, moving at Sun, 13 Dec 2009 13:44:41 -0000 | Permalink

ANDRE has a posse.

MSCHWERN has a PAUSEID.

Do you?

Posted by schwern on journal at Sun, 13 Dec 2009 07:36:42 -0000 | Permalink

The problem with my previous implementation of nested [% IF expr %] is that the nature of [% FOREACH item IN list %] forcefully requires shallow-to-deep resolution.

The negative-lookahead solution only allows for deep-to-shallow resolution, and there isn't really any good way to change that.

So it appears that I've gone as far as I can in memory-conservation mode, and it's time to spend some of my remaining budget.

The real problem here, as I mentioned earlier, is the [% END %] tag. By using the same closing tag for everything it's impossible to match an opening tag to the appropriate closing tag.

Changing to directive-specific closing tags doesn't really help either because it doesn't fix the nested scenario of the same tag. So switching to a reverse-style [% IF expr %]foo[% FI %] won't get us anywhere either (it will just create the appearance of doing so).

What would be REALLY nice would be to have every single tag pair different. All we need to do is invent our own TT derivative. We'd never expose this alternative TT language to the users, just use it as an intermediate format.

Since we know that the negative-lookahead method can identify all the tag pairs, this turns out to be quite feasible (if somewhat expensive in memory terms).

To start with, we redefine the lookahead search slightly to match the longer FOREACH tags as well.

my $PREPARSE = qr/
        $LEFT ( IF | UNLESS | FOREACH ) \s+
                (
                        (?: \S+ \s+ IN \s+ )?
                \S+ )
        $RIGHT
        (?! .*?
                $LEFT (?: IF | UNLESS | FOREACH ) \b
        )
        ( .*? )
        (?:
                $LEFT ELSE $RIGHT
                (?! .*?
                        $LEFT (?: IF | UNLESS | FOREACH ) \b
                )
                ( .+? )
        )?
        $LEFT END $RIGHT /xs;

We can use this to iterate over the string and create a set of custom tags for each different match. At the same time, we might as well make those custom tags based on a single letter (regex tricks get much easier when string elements are only one character long).

The only complicating factor is that if we use the string-chomping left and right brace matches now, we'll need to be careful not to chomp again on the second pass.

my $id = 0;
1 while $copy =~ s/
        $PREPARSE
/
        my $tag = substr($1, 0, 1) . ++$id;
        "\[\% $tag $2 \%\]$3\[\% $tag \%\]"
        . (defined($4) ? "$4\[\% $tag \%\]" : ''); /gsex;

The result of this is that we take a template that looks like this...

People:
[% FOREACH item IN list %]
[%- item.name %]
[% END -%]
Cool People:
[% FOREACH item IN list %]
[%- IF item.cool %]
[%- item.name %]
[% END %]
[%- END -%]

and turns it into this...

People [% foo %]:
[% F3 item IN list %]
[%- item.name %]
[% F3 %]Cool People:
[% F2 item IN list %][% I1 item.cool %]
[%- item.name %]
[% I1 %][% F2 %]Done!

The price for this preparsing step isn't cheap, it chews up a whopping 15k without adding any new features (which is more than we paid for UNLESS, ELSE and IF recursion combined).

But with this one, we can switch to top-to-bottom and shallow-to-deep processing. And we can also do recusive top down processing, with some simple stash localisation so that "item" in [% FOREACH item IN list %] is resolvable.

my $CONDITION = qr/
        \[\%\s
                ( ([IUF])\d+ ) \s+
                (?:
                        ([a-z]\w*) \s+ IN \s+
                )?
                ( $EXPR )
        \s\%\]
        ( .*? )
        (?:
                \[\%\s \1 \s\%\]
                ( .+? )
        )?
        \[\%\s \1 \s\%\] /xs;

1 while $text =~ s/
        $CONDITION
/
        ($2 eq 'F')
                ? $self->_foreach($stash, $3, $4, $5)
                : eval {
                        $2 eq 'U'
                        xor
                        !! # Force boolification
                        $self->_expression($stash, $4)
                }
                        ? $self->_process($stash, $5)
                        : $self->_process($stash, $6) /gsex;

This single match approach means that, bizarely, both the following are legal in T:Tiny.

[% FOREACH item IN list %]
foo
[% ELSE %]
bar
[% END %]

[% IF foo IN bar %]
[% END %]

Well, maybe not legal. But they won't throw an error, which is tolerable at least.

We'll also need to spend about another 5k to factor out the code into 3 or 4 different methods, so that everything runs properly when evaluated recursively.

After all this overhead, and about 5-10k on adding the actual support for FOREACH iteration, the result is we can now arbitrarily nest IF/UNLESS/ELSE/FOREACH inside each other.

This leaves me with 68k spent out of my original budget of 100k of memory.

And without support for files yet (and thus things like PROCESS or INSERT or INCLUDE) I still have a lot to do...

Posted by Alias on journal at Sun, 13 Dec 2009 05:08:36 -0000 | Permalink

It’s been a frustrating week. I really want to finish up some of my Perl projects and, between family and work obligations this week, I’ve had no chance to work on them. So if anyone has some round tuits, could you please send them my way?

At least I’m keeping up my ironman status.

Posted by dagolden on perl-programming, ironman at Sat, 12 Dec 2009 23:52:45 -0000 | Permalink

As you may or may not know, people on CPAN own modules (technically they own the namespace). Each Foo::Bar is owned by one or more CPAN accounts. Usually you gain ownership on a "first-come" basis, but it can also be transferred. Only the "official" tarball for a given namespace is indexed. So if the owner of Foo::Bar uploads Foo-Bar-1.23.tar.gz Foo::Bar will point at Foo-Bar-1.23.tar.gz. If I (presumably unauthorized) upload Foo-Bar-1.24.tar.gz the index will still point at Foo-Bar-1.23.tar.gz.

Here's the rub. Not owning a module doesn't stop you from uploading. It also says nothing about who owns the distribution. gitpan is by distribution. Now it gets a little more difficult to figure out who owns what. For example, look at MQSeries-1.30. All but two modules are unauthorized. BUT notice that MQSeries.pm is authorized. The CPAN index does point MQSeries at M/MQ/MQSERIES/MQSeries-1.30.tar.gz (everything else is at 1.29). Likely what we have here is a botched ownership transfer.

How do you mark that? search.cpan.org seems to take the strict approach, if anything's unauthorized its out. The CPAN uploads database I have available is the opposite, if anything is authorized its in. What to do?

Then there's stuff like lcwa. Looks like junk, but here's the thing. CPAN has a global module index to worry about, gitpan doesn't. Each distribution is its own distinct unit. So lcwa does no harm on gitpan, it can be recorded.

What does matter? The continuity of a distribution's releases, and this is precisely what CPAN does not track. It doesn't even have a concept of a distribution, just modules inside tarballs. CPAN authors playing nice with tarball naming conventions gives the illusion of a continuous distribution.

So... for a given release of a distribution (ie. a tarball), how does gitpan determine if the release should be included in the distribution's history? If we go strict, like search.cpan.org, we're going to lose legit releases and even entire distributions (like lcwa). If we let anything in gitpan is not showing an accurate history.

Add the complication that authorization changes. For example, the MQSeries module ownership will eventually be fixed. What then?

First pass through, gitpan is ignoring this problem. Its just chucking everything from BackPAN in. Second pass will rebuild individual repos with collected improvements. This is the first thing I'm not sure what to do about.

Suggestions?

Posted by schwern on journal at Sat, 12 Dec 2009 22:02:03 -0000 | Permalink

For an incremental cost of zero extra statements, and less than 1k of memory, I've managed to successfully implement support for nested IF/UNLESS/ELSE statements!

Starting from the premise that the regular expression engine already had support for recursion, I was somewhat dismayed to discover I didn't have access to recursion because it wasn't available in older Perls.

Suggestions from Google were that in a pinch you can do something similar using negative look-behind assertions (similar to tricks I've used before to do rather elegant and fast unescaped-quote scanning in PPI).

Alas, delving through the man pages I find that I don't have support for look-behind assertions either if I want 5.004 support (and that's always a nice-to-have in ::Tiny modules, although not essential).

So, how to do recursive conditions.

The slightly odd answer is that you can emulate the same kind of negative look-behind using a slightly crazier negative look-ahead assertion (which I DO have access to).

The new condition regex looks something like this (where $LEFT and $RIGHT are fairly complex whitespace-chomping expressions for [% and %])

my $CONDITION = qr/
        $LEFT (IF|UNLESS) \s+ ( $EXPR ) $RIGHT
        (?! .*?
                $LEFT (?: IF | UNLESS )
        )
        ( .*? )
        (?:
                $LEFT ELSE $RIGHT
                (?! .*?
                        $LEFT (?: IF | UNLESS )
                )
                ( .+? )
        )?
        $LEFT END $RIGHT /xs;


So basically, look for the IF block, which ISN'T followed by another IF block, then an option ELSE block (which isn't followed by an IF block), followed by an END block.

Superficially, this doesn't really make sense, because you only match things at the deepest level.

Actually, it's even worse, because it ONLY matches the last and/or deepest IF block. So lets first at least deal with all of them by doing multiple passes.

1 while $copy =~ s/
        $CONDITION
/
        eval {
                $1 eq 'UNLESS'
                xor
                !! # Force boolification
                $_[0]->expression($stash, $2)
        } ? $3 : $4; /gsex;

The key part there is the "1 while". This keeps passing over the string until we've run out of last and deepest conditions.

The only real problems here are that substiting in multiple passes runs the risk of an infinite loop, and we're processing the conditions from bottom to top and deepest to shallowest.

The infinite loop isn't a problem here (fortunately) because we never add new content to the file, only cut parts out. The expression evaluation stuff is done in a separate regex.

And the fact the conditions run in the opposite order isn't a problem either because we've already said that the stash MUST be immutable and side effects aren't allowed (although if anyone tries to cheat on this point, they're going to get weirded out).

So there's full (and infinitely deep) support for nested conditions, for pretty much no cost. This leaves the 60k of spare room largely free, which is good because FOREACH support is going to be much much harder than IF/UNLESS/ELSE.

The main problem here is that all the blocks end with the same terminator [% END %] (which means it's really hard to process different statements separately).

I do, however, have some interesting new ideas on how I might deal with this problem now that I have the negative-lookahead+while trick down pat.

Work continues...

Posted by Alias on journal at Sat, 12 Dec 2009 14:21:32 -0000 | Permalink

The actual entry is at blogs.perl.org. However, here's a few extra notes about that site:

It's much more stable than last time. Aaron moved it to a new server and Dave set up MT. The templates are on github along with an issue tracker.

Barbie talked with Dave and myself at the London Perl Workshop (lpw2009) about the site and I wrote down all of his suggestions and they've now been added to the issue list. We look forward to more people trying it out and Dave has a post asking for more feedback. Also, he's aware that several people only made it partway through the registration process. Drop him a line if you're affected and he'll clear it up for you.

I can't thank Dave, Aaron and SixApart for all they're doing to create a modern blogging platform for us.

Posted by Ovid on journal at Sat, 12 Dec 2009 12:24:54 -0000 | Permalink

In the last post I mentioned Kephra's menu definition files. These can only work, because we have a very important file namend commandlist. There is a binding of a chunk of data to each call. the internal call, a callback that says me which state this call has or if should be disabled. also events the will be triggered if that data or status changes, an association to an icon file and the key binding is also defined there. the label and hint texts are of course part of the localisation.

how to call such files? I put it under "config/interface/*" and "config/localisation" but i'm not shure if its the right term for that. there is also a config file, where all the setiings are saved, under "config/global". Just recently i throw some parts out that are not really settings. for instance the previously used search terms and bookmark position. I put it into the search_data.yml to clean things a bit up. Also the config validate routines have now a much easier job. also the list of closed files i move from the settings into an own file. it will be a regular Kephra file session file, that holds all properties so if you restore any file from the closed files menu, you have all you settings with the marked postitions back.

There is also an extra file for the notepad content. we have a file with the templates you can insert form the templates menu. and then you have the directory "config/syntaxstyles" where are the color definitions for the syntax hightlighters are lying. there is also an dir "syntaxmodes" because we soon do a transition from these syntaxstyles to a richer format that contain more information about that language. how to comment, which interpreter /compiler to use and so on.

the we have also some files we try to hide a bit from the use in some deeper nested directories. that are subconfigs that define the help menu for a particula language or some precompiled yaml informations, that help to speed up the boot time. for instance if you translate Kephra into spanish or russian (hint, hint!) and translate the config/localisation/english.conf into an spanish.conf and put it into the localisation folder. Kephra recognises during the next start that there is a new l18n file and recompiles the language selection menu and replaces the old definition in the "config/interface/cache" folder. The label and hints for that menu item will be taken out of the meta head of the l18n file.

but all that will grow and grow and soon (before kephra 0.5) we get plugins that have to be stored somewhere with their own config files that have to be also stored somewhere. so i currently just rethinking this whole system. We had 3 extensions so far, but they all went into the core, because the're small. I don't believe that making a plugin out of every tiny thing, that is not needed to boot an app is such a smart idea like purely rational people like me to believe. I'm here very anti eclipse (the nether Khephra is a sun bearer not a sun eclipser) but Kephra is anti eclipse in all means. But thats an topic of its own, for a new article.

Posted by sir_lichtkind on journal at Fri, 11 Dec 2009 21:44:29 -0000 | Permalink

Happy Chanukah, everyone! Lately, I've had little energy for blogging, and have been coding a lot. I suppose that's a good thing, but I'd like to keep up with my Planet Perl Iron Man statistics. So here we go - another recent hacktivity report.

As I said, I've went over the Moose::Manual and learned many nice new tricks to make my Moose-using Perl modules more Moosey. I've implemented some of them in XML-Grammar-Fiction and also on Test-Run, which is a much more massive codebase.

While I worked on Test-Run I noticed that some of the Test-Run-CmdLine tests failed, and realised the problem was a MooseX-Getopt bug which I was able to supply a test for, and which involved a very short fix. While I was in the neighbourhood, I also contributed a test for handling the Getopt::Long no_ignore_case which I used as well, and the results were released in MooseX-Getopt-0.26 by Tomas Doran.

I also ran into an undesirable behaviour with MooseX-StrictConstructor which I was able to workaround with some help from the #moose people: when using it inside a class which has a base class, one needs to do BEGIN { extends ("BaseClass"); } instead of just extends("BaseClass");. I didn't understand why, but it just works.

I'm now reading the chapters from the Moose Cookbook, which will take me some time.

After that, I finally did the work I promised on the list of Perl mailing lists. There's still a lot to fill in, but the YAML/DBI foundation is in place.

I worked on a Ruby program which I originally started writing in order to learn Ruby. I've added some tests, and refactored it. I've got bitten by a case where I overrided a variable called yx as a parameter to a closure and it got clobbered afterwards. This is one reason why I prefer Perl 5's (and Scheme's) explicit lexical scoping using "my", which provides better protection against such mishaps.

I continued to work on Mozbot, but my original patch was not applied yet, so I don't know if there's any interest in further work on the internals of the robot.

I've made some progress with reading the book xUnit Test Patterns: Refactoring Test Code. One of the insights I gained so far was that the failed tests should point what became broken without too much debugging. The book is off to a slow start with many overviews and introductions, which is kinda annoying, though.

Finally, I attended a meeting of PHP-Israel where there were a lot of Bourekas, and were someone gave a presentation about PHP Unit and writing unit tests. There was a discussion of whether we should only test the external behaviour of a class, or also its protected/private methods, and if the latter - how (there was some way to do it in PHP). One thing I noticed was that some of the PHPers used Eclipse for their development, and there was some discussion about how to get it to behave properly on Linux.

A few of the PHPers also thought that they would love to use JavaScript on the server. It's funny, but as a Perl 5 programmer (with experience in many other languages) I look at JavaScript and see many horrible aspects, and missing features. However, the PHPers seem to think it's better than PHP. Maybe it's the Blub Paradox.

Posted by shlomif@iglu.org.il (Shlomi Fish) on ruby, kiss, js, fc-solve, qa, manual, mozbot, freecell, javascript, php, irc, test, mozilla, recent, hacktivity, perl, solver, moose, run at Fri, 11 Dec 2009 21:15:23 -0000 | Permalink

A programming language is syntax and semantics.

Spend some time designing a programming language, implementing a parser and compiler, and working on a virtual machine, and you'll discover that both syntax and semantics influence each other. Even so, you can separate them. Myriad syntax forms can produce the same semantics. There are differences in expressivity, abstraction, and concision, but within the same language, you can support multiple syntactic forms (even with differing semantics) from the same set of low-level semantic primitives.

Even though the most effective way to develop a software project over the long term is to get feedback about the problems that users face in their own work and iterate through potential solutions to find the best approach, users tend to hate this kind of churn in the syntax of their programming languages. Even if the semantics don't change (much or at all), especially at the low levels, rewriting for the sake of rewriting is an expensive exercise.

In the long term, better expressivity, maintainability, abstraction, and correctness may make change worth it, but who practices long-term thinking?

There's a lot of pressure to get things right the first time, even if what's right doesn't stay right for long, because there's even more pressure to stick with what's there because it's there and it's good enough.

Consider this in the context of The Replaceable Batteries of Your Standard Library and Replacing the Standard Library with Distributions. Those posts discussed a language's core library. They could very well describe the core language itself. The difference between a library and a language (given a sufficiently expressive language) is one of convention alone.

That convention is important; it's easier to convince people of library changes than language changes. Of course, people believe it's easier to change a library (or provide an old version or a wrapper for backwards compatibility) than to change the syntax of a programming language.

They may be right.

However, Perl 6 separates the grammar of the language from the rest of the language. This is somewhat new for Perl, but nothing new in the world of programming languages.

Perl 6 also lets you create your own local grammar modifications. The most important attribute of these modifications is lexical scoping. You must declare which modifications you use, and they take effect only in the innermost lexical scope of that declaration.

Perl 6's grammar is a self-hosting grammar. It's written in itself. Combine that with lexical scoping and a Perl 6 specification which includes the self-hosting grammar and puts a version number on it, and you have something special.

You can have multiple Perl 6 grammars co-existing in the same program. You can have components written in Perl 6.0.0, 6.0.1, 6.1.0, and 6.2.0. Yes, that could become a recipe for a maintenance disaster if you don't encapsulate your code well. It can also allow you to refactor your code to take advantage of new features and remove deprecated syntactic features component by component.

In other words, rather than Perl 6 defaulting only to those features which are supremely unlikely to conflict with features existing in the earliest releases of Perl 6 (as is the case with Perl 5), the language can evolve and change without leaving existing programs in the dust.

Add to that the policy that you must declare the specific version of the parser you wish to use -- and if you don't declare a version, you get the most recent version available -- and Perl 6 can avoid some of the backwards compatibility concerns that have influenced Perl 5 development.

Posted by chromatic on modern perl, perl, perl 5, Perl 6 at Fri, 11 Dec 2009 18:59:31 -0000 | Permalink

Languages evolve.

Some people want to put closures in Java. (Set aside the fact that only 10% of the people I've seen talk about closures in Java know the difference between first-class functions and closures.)

C++ will eventually get a new specification.

COBOL has OO extensions.

Despite the fact that Microsoft can't seem to support C99 in Visual Studio (hey, give them a break -- they're still working on supporting PNG and CSS 1.x, two standards even older than C99), C# gets new features.

You don't have to use the new versions. You don't have to use the new features of the new versions. You don't have to like the new features of the new versions, and you don't even have to know they exist.

You have to delve deeply into delusional solipsism to deny that they exist... and that's merely language features. Consider all of the changes in libraries available for those languages.

In The Replaceable Batteries of Your Standard Library, I argued that change is even more constant when you consider the available libraries for a language. This is true of standard libraries blessed by the implementors (try using gets() in a modern C project and see the reactions you get) as well as libraries made popular by common use (remember Class::DBI?).

Change happens.

Language designers and developers and maintainers and patchers and porters and releasers ought to consider how to deal with the inevitability of change.

Should a new Perl project use CGI today? Plack seems like a better option. Should a new Perl project start with Perl 5's basic OO or use Moose? Is there any excuse to recommend the use of File::Find, when the basic API requires the use of features that 90% of Java programmers (in my unscientific and unrepresentative experience) do not understand?

The best argument for using CGI and bare-bones Perl OO and File::Find is that they're available on whichever version of Perl 5 you're likely to have installed (even if that's Perl 5.005, which has been obsolete for the whole of the 21st century and even if they have bugs which have been fixed in the past several years). The second best argument for using them is that there's plenty of example code which demonstrates how to use them (though if you write Perl 5 code based on examples you can find online... ouch).

The third best reason is that, once something is in a language's core library, only an act of the language designer can remove it. In theory, you'll get a lengthy period of deprecation warnings (if you pay attention to that sort of thing, but convince me that the people not using Moose and sticking with Perl 5.6.1 know how to get deprecation warnings from Perl 5). In practice, you're in trouble anyway, unless your language has a specification or a comprehensive test suite and you read it to figure out if specific behavior on which your code relies will always be present in the future.

In other words, change happens.

How can a language designer or release manager or maintainer or porter walk the thin line between providing a bare-bones language with few libraries and cramming everything into a core bursting with subtly-different alternatives and growing less relevant to real-world adept uses because of too-early blessings of approaches which time has proven inopportune for the common uses?

I've argued before that the right approach is a small core which contains only those libraries absolutely necessary to bootstrap an intelligent library loading system. This works for operating systems such as Gentoo GNU/Linux, Debian in its flavors, and many of the BSDs. Parrot is exploring a similar approach.

The Perl 5 core could do very well to include only a couple of libraries necessary to build and install Perl and to download (or otherwise build from a download or checkout) a bare-bones CPAN client. Perl 6 may do something very much like this.

That leaves one question unaddressed. What happens to those unfortunate novices who download the core distribution themselves and find out that they can't do much with it? What happens to people with cruel, cruel sysadmins who refuse to install anything other than the core?

The solution comes from realizing that the people who are best at maintaining and porting and releasing a new version of a language and its core are not necessarily the same people who are best at discovering what the language's community considers most useful, most usable, or most appropriate.

You can already see the solution in something like Strawberry Perl, which unabashedly includes a compiler and is on its way to including an IDE and a GUI toolkit -- not because you can't use Perl 5 effectively without any of those, but because Perl 5 is much more powerful and much more useful with them.

That line of reasoning is very similar to the one which includes any given library in the standard library.

Why not embrace it? Let distributions come up with their own lifecycles and deprecation and suppport policies. Let them identify which libraries people should have available by default for the next eighteen to twenty-four months. Let them perform the work of integration and testing and bug reporting and working with upstream authors.

Upstream would always remain the same. If you can access CPAN, you can ignore distributions or install your own additional packages or build your own distribution. (Strawberry makes its own tools available.)

A similar process works well for operating systems -- and, arguably, operating system users are less savvy than programmers. Many distributions already package Perl 5 this way.

Could you live with such a change?

Posted by chromatic on CPAN, modern perl, perl 5, perl 6, software development at Fri, 11 Dec 2009 01:47:40 -0000 | Permalink

I wrote this some time ago, and it sat in the queue and got stale.? I’ve mentioned one place I’m not using Config::JFDI, but I’m using it other places and I thought it desrved a mention.

I mentioned my old framework had a way to infer which configuration file to use by examining the environment it was running in. ?I’ve moved to the Catalyst Way of things (more or less) and that will make things easier. ?A bunch of my old code can go away.

I have some useful tools to do things which do like to read the config file, too. ?They used the same code. ?They now use Config::JFDI. ?It reads config files the same way Catalyst does, and gives you a very similar $config object you can use to look at things with.

I did have to dig in to the sources to get it to work right, even though it’s pretty well documented.

When I run my programs, I usually run them from a shell where I have environment variables set to get the right suffix for my config files. ?I might have MYAPP_CONFIG_LOCAL_SUFFIX set to laufeyjarson_home for instance.

There was a case where the utility program needed to force a particular value there, so it would get the right configurations. ?I called Config::JFDI->new with local_suffix=>’whatever’, and expected that to work.

It didn’t seem to. The documentation said it works!

And, it does work. ?But it only works if the environment variables are not set. ?If the environment is set, that overrides the values set with new().

I almost filed a bug on it, too. ?That can’t be right!

It can, though. ?I just had to think it through. ?You want your program to have a sensible default coded in. ?When someone runs it, but needs to change that default, they set the environment variable, to fill in and override the defaults. ?That’s exactly what it did!

Too bad it isn’t what I wanted.

Config::JFDI supports an option to new called “no_env”, which will tell it not to read the environment. ?That would fix it.

I actually went the other way; I changed ${ENV}. ?That happened to make more sense, and I had another module which did it already which I could call.

So, Config::JFDI works right, and lets you do whatever you need. ?It’s also really handy to write stand-alone tools with it and have it work like Catalyst does.

I was ?considering using it in myapp_server so I can set only MYAPP_CONFIG_LOCAL_SUFFIX or CATALYST_CONFIG_LOCAL_SUFFIX and then put the default values for all the command line options in my config file rather than in four more variables.? Decided not to.? I just set the environment and let it be.

Posted by Laufeyjarson on Perl, Catalyst, config, Config::JFDI at Thu, 10 Dec 2009 23:20:42 -0000 | Permalink

We'll be having a group dinner for the December meeting, and have
a few drinks after to wrap up the year. The date for this meeting
is December 22th at 7pm.

"Naan-N-Curry" at 336 O'Farrell Street, between Mason and Taylor.

  http://maps.google.com/maps?q=336+OFarrell+St,+San+Francisco,+CA+94102,+USA

This place has moved around a few times, and has many satellite
locations now, so look at that address carefully. This is across the
street from the Hilton, and next to the entrance to a large parking
garage.

From the Powell Street Bart station: walk two blocks north along Powell,
and 1.5 blocks west. Don't try to walk up Mason or Taylor, unless
you're in an adventurous mood.

The food is inexpensive, high quality Indian food. They have a buffet
these days, which makes things simpler. Free chai. The dining room
is double-sized, with large tables: there's no need to worry too much
about RSVPs.

  http://naancurry.com/branches.php?brn=5

This place used to be 24 hours, but I guess they've scaled back to
11:00 AM to 4:00 AM. But I don't think we'll need to rush out of
there.

Announcement posted via App::PM::Announce

RSVP at Meetup - http://www.meetup.com/San-Francisco-Perl-Mongers/calendar/12060331/

Posted by Phred on journal at Thu, 10 Dec 2009 19:44:38 -0000 | Permalink

this was originally posted on my new blogs.perl.org journal, which can be found here which is also the RSS feed for it.

It's a lot of fun to contribute to other people's code (especially code you're using) and it's very fulfilling. Some people say it's a downer when your code isn't accepted, and I can understand that. However, the serious downer is when your code gets ignored.

When I go over a module, I go over the ticket list. When I see a ticket from over a year and it's still "new", it disappoints me and when I contribute to a project and it takes six months to get a reply, it disappoints me.

I know there are also different types of patches. Some of them complex and take time to test, check, correct and adapt. However, some patches are simple errors, common typos, data that should be added (like another entry for a hash of static data). These shouldn't be stuck so long back there. And, above all, don't ignore the contributor. Go over the tickets once a month and see if you can comment on anything. Even saying "I can't address this in the near future" means a lot to the contributor.

If you feel you don't have the time to work on a project, write about it, and suggest someone (most likely someone you already trust) be able to have co-maintainer status in order to apply some patches and release a new version.

Often times I see a project and think "well, I could fix this or that" or "seems like these only need applying and testing" but I don't want to take over the leadership of the module. I've contacted a lot of authors over the years. I've gotten only a few replies.

Sometimes I think releasing Your::Module::Patched will be the only way to get some of the patches I or other authors write included in. Obviously, that isn't a smart move. CPAN isn't Github. It doesn't look kindly on forks because it still has to maintain some authoritative version of a project for users to be able to download. Users want authoritative versions, not "his/her version".

Other than trying to contact an author, writing an email (or another one) and maintaining a personal collection of patches, I don't know what else can be done.

Please, give your RT some love, give your contributors some love.

I think I'll personally try to contact some authors for co-maintainer status just to patch things up.

Posted by xsawyerx on journal at Thu, 10 Dec 2009 13:27:27 -0000 | Permalink

A couple of years ago we held the PerlHike::IT. Even though there were just 2 of us, it marked a "first time" for the Italian Perl Community, and we hope to repeat it in summer 2010. For the winter (not necessarily this winter), I am planning to probe around if there's interest in a PerlSki::IT. Joining Perl and skiing is not a new idea, but the plan is to make something more geek than a day on the slopes. The event should at least feature: a day or two skiing talks (at least some lightning ones), possibly renting a small room one afternoon a night out (hotel, friend's house, apartment, tent, ...) food and beer in the evening For my own personal convenience, the event could be held in the world-known Cortina d'Ampezzo town, in the heart of the dolomites. It's a fairly easy reach from Treviso and Venezia airports, and I can even host a couple of folks in a small apartment not far from Cortina. Other accomodation can be arranged in nearby towns. So, if there's any interest, or any suggestion for some other places (there should be at least one Perl folk there to help arrange things), just post. ;-)...

Posted by Michele Beltrame on cortina, dolomites, ironman, italia, italy, perl, ski, skiing at Thu, 10 Dec 2009 09:09:36 -0000 | Permalink

part four.

Kephra has a main menu, that is compiled from a yaml file you can open via the "config > interface" menu. So you can change it easily with the previously listed line edit commands. It serves as an overview, sorted by topic. Optimized for finding a function you looking for, not optimized for finding it after the first click, that's the task for context menus. This main menu I refactor from time to time, to keep it structure sense-making, balanced and beautiful.

On the other hand Kephra also has context menus, that are also compiled from yaml (easy changeable), but contain a selection of important commands, associated with this spot in the app. this is almost self evident and widely used in GUI software, but editors miss here large opportunities. Lets take for instance Gvim. true its not a GUI Editor of the first order, but believe me, most IDE don't do much better. GVim has only one context menu with undo, insert and some selecting commands. if you have already selected, the usual 4 edit commands will be added.

Nearly same thing with scite, padre or notepad++. Only notepads menu is extralarge because its polluted withe a shiny but fancy feature for highlighting selections of text with different colors. I currently can't do that because Wxwidgets wraps an 2 years old version of scintilla. but back to context menu. The Kommodo IDE has also an extra long context menu that you have to scroll with tiny menu buttons.

But how does it Kephra? We put also a lot of stuff into the context menu, but copied a very reasonable idea from firefox that anybody else don't seems to notice. Noone of the here named editors and most of the others don't do it, even if its the natural thing to do: don't pollute precious context menu space with calls that don't provide a useful function in this context. or lets put it more clearly: separate this menu into two. one general menu with functions about this current file (undo,redo|insert, select all|back to last edit,search dialog|save,open|close). another menu will popup if you rightclick and you selected some text. then you get a menu of actions you can do with that selected text like:copy, insert, replace, cut, delete|take as search term, take as replace term, put it into notepad,lc,uc). maybe the comment function we should put here too like padre does.

That was nice and menus don't get too long. But it doesn't stop there. if you rightclick on the margin, you get another menu with items that have to do what the margins are about. markers, bookmarks, folding and switching the visibilities of the editpanel margins. I mean its the most natural thing to do: i don't like this so i click on it and say it should go instead of searching the call in the main menu, even all these calls are just in the "view" menu. The searchbar has a context menu where you can set all the search option like in the search and replace dialog. It can also be found in main menu under "search > options". and many status bar cells have a context menu too. its a bit like kommodo does it, but only richer. in kommodo cells that react on left click have an small updown arrow that signal: here can you popup a menu by left click. In kephra left and right click do something different. i mean with left you do point an click or switch buttons. thats what leftclick also do in kephra status cells. the switch between the 2 (in one case 3) most common settings for that document property. but if you rightclick you get an context menu with a lot of more stuff associated with that property. to me its often a way to get my stuff done fast.

To sum this up: this series of articles is not about bashing other editor, but to showing why I started Kephra in the first place. because most editors do the usual stuff like all the other do it too. to innovate is one of the most misused words by microsoft but innovation is needed in a lot of places and frankly i can't change all the bits in kommodo or notepad++ i would like. beside notepad is windows only and kommodo is not free as i can get really hands on the UI design. So there is a real need for Kephra. many are happy with the things there used too but but some who are picky about details and wand to tweak even small issues by changing the configs, may find kephra fitting.

uh, one thing i forgot. there was one menu more, that is currently disabled, because the AUI::Notebook can't react on mouse events. so it maybe will appear as a regular menu in the main documents menu. Its about a list of all open docs in a way you can look up their full file path. many editors waste a full menu on that but frankly i use that never and beside that most uses are already covered by the automatically generated menu on the AUI::Notebook.

Posted by sir_lichtkind on journal at Thu, 10 Dec 2009 01:47:36 -0000 | Permalink

The last couple of days I spent building a distribution for Linux to make it very easy to start using a recent version of Padre, the Perl IDE.

On yesterdays entry of the Catalyst Advent Calendar Jonathan Yu explained the advantages and disadvantages of using Catalyst as packaged by Debian vs. installing it from CPAN. Padre has the same issues but as Padre is younger than Catalyst it lags even more in the package management systems of the various Linux distributions. While on CPAN we have v0.51 some distros still come with 0.12 I think and many people use older distros that did not have Padre at all. Padre is also more difficult to install using local::lib than Catalyst due to the need of wxWidgets.

It was also a bit emberassing that we already had a stand alone Padre package with super easy installation for both Windows and Mac OSX Snow Leopard and we don't have one for Linux.

So I built a portable perl (meaning you can put it anywhere on the system), installed Padre into it and you can already download it. (download file size 30,009,562, after unzip 119 Mb)

After downloading it do the following:

 tar xzf perl-5.10.1-xl-0.03.tar.gz ./perl-5.10.1-xl-0.03/perl/bin/padre.sh 

Note, you need to run the .sh file!

Not only that you can use Padre now for writing Perl 5 scripts, this package also comes with the Perl 6 plugin so you can use it to write Perl 6 code as well.

Perl 6 on Padre

In order to use the Perl 6 plugin, first you will have to enable it: Go to

Plugins/Plugin Manager - Select Perl 6 and click on Enable Due to a bug you will have to restart Padre for this to take effect.

Then go to

Edit/Preferences/Files and Colors Select Perl 6 as File type and Std.pm as Highlighter in order to get syntax highlighting.

Catalyst

As you could read on the Catalyst advent calendar you can use Padre for Catalyst development. In order to make that easier this version already contains the Catalyst plugin of Padre.

In order to use it you'll need to enable the plugin the same way as you did for the Perl 6 plugin and follow the instructions of Breno G. de Oliveira on Developing Catalyst Applications with Padre.

Warning and bugs

This is a very experimental version with a lot of known flaws. Please, report issues via the regular channels of Padre. (IRC, mailing list).

Known issues:

PERL5LIB

If you have PERLLIB or PERL5LIB configured (e.g. beacuse using local::lib) this installation will get confused. Of course this is not specific to this build. Any other perl would be confused. The simple work-around is to unset those variables in the shell where you launch padre.

The longer term solution is to make changes to perl itself so I'll be able to compile out the PERL5LIB and PERLLIB support from perl. This does not seem to be a hard project and I think if someone wants a bit core perl development this could be a very useful and not too big project. Nicholas Clark told me the code is in perl.c and that I should pay attention to sitecustomize.pl too.

sh-bang

All the scripts installed have an sh-bang that points to the directory where I built the package. Hences none of them will run alone. (That's the reason I had to add a separate shell script for Padre. I am not sure how can I achieve but I'd like to make sure that every script installed will know that the perl it should use is relative to where the script is located.

CPAN.pm

The CPAN client is probably not functional. It might work for pure Perl modules but if you need a compiler then there might be issues with compiler mismatch with the one I used. (I built it on Ubuntu 8.04.03)

post mortem Segmentation fault

When the Catalyst plugin is enabled, there is a segmentation fault after you shut down Padre on some of the machines we checked.

Help and feedback

Your feedback regarding this package would be really appreciated and your help with any of the above issues would be just a really nice present.

Posted at Wed, 09 Dec 2009 23:01:07 -0000 | Permalink

after some chat with parrot-porters, i finally undertood that what's get installed in /usr/src/parrot (and that i was trimming during installation) is not parrot full source, but intermediate forms of pmc, needed if high-level languages want to subclass them. they are thus now shipped in parrot-src package - and this is enough for rakudo to be compiled.

therefore, rakudo 2009-11 is now (finally) available in mandriva...

Posted by jquelin@gmail.com (J?r?me Quelin) on rakudo, rpm, mandriva, parrot at Wed, 09 Dec 2009 10:39:00 -0000 | Permalink

Update: This is now working, and you can see it in action on Padre trunk

The recent Padre 0.51 contains the second phase of "Save Intuition", a set of small tweaks designed to make saving quicker and less distracting.

In the first round I discussed earlier, Padre started to guess what it thinks the file name should be for the file based on the content.

In the second round, Padre will now also try to put the save dialog into the correct directory as well.

The logic goes like this.

"New files probably belong to the same project as the file that was open just before the user triggered a File -> New action. If this is a Perl file, in a Perl project, and we're saving 'package Foo::Bar', the user probably wants to save into $project/lib/Foo. Set dialog path to that directory, if it already exists."

My hope is that this feature is subtle enough that most people won't even notice it (or if they do, it's with momentary happy surprise and after that they take it for granted).

Of course, there's gaps in this logic. What if the directory does NOT exist? In the 0.51 release, it just bails out and so falls back to the normal choice for the save directory. Creating nested directory trees without the user asking for it is risky business.

But what if I specifically want that?

If I'm making a new class for an existing project, surely I should be able to just wave my hands at Padre and say "Save in the usual place" and have that just happen without showing a dialog at all (unless Padre gets confused and can't determine what the "usual place" is).

This third phase of Save Intuition is what I plan to be my main feature for 0.52 (assuming I can get it to work). I'm hoping to take Ctrl-Shift-S for this, so it acts just like a normal save, but adding the extra shift key adds more DWIM to the process.

Posted by Alias on journal at Tue, 08 Dec 2009 23:10:52 -0000 | Permalink

How much should a language's core library contain?

Perl 5 has included the CGI module as long as I can remember. (Module::CoreList suggests a first release in Perl 5.004, released in May 1997.) It has support for the API of a Perl 4 library called cgi-lib.pl, which came around even longer.

You can count on having the CGI module installed on any complete Perl 5 installation deployed in the past decade.

(Even so, many novices asking questions on sites such as PerlMonks still show off their own copied and pasted CGI parsing routines which have somewhere between four and ten bugs, misfeatures, and security holes.)

You can argue that a CGI parameter and HTML generation module was essential throughout the late '90s and early 2000s, but look at the growth of PHP (with much easier deployment than Perl 5) and the popularity of frameworks such as Ruby on Rails (which lets you go from nothing to a database-backed CRUD application in ten minutes) and tell me that CGI.pm as a core library meets the needs of the dominant set of likely users.

I don't mean that people don't use CGI or shouldn't, but that it's worth considering if its presence in the core meets the current needs of the current Perl 5 users and likely Perl 5 users. You can make the argument that there's little cost to including this module for novices and expecting adepts to be able to install something else, but every additional core library adds a maintenance cost.

The thesis behind this site--behind the idea of "Modern Perl"--is that writing Perl today the same way you did in 1997 misses most of the power of the language. Yet if you look at the internals of the CGI.pm module, you see that it reads like the author doesn't know how to write modern Perl.

There's a good reason for this; it's stayed essentially the same in design and implementation since its earliest conception. Fifteen years ago, no one knew the best way to use objects in Perl 5. No one knew the right way to delay loading frequently-unused code to speed up compilation time. It wasn't staggeringly obvious that separating the concerns of HTML generation and argument processing was necessary or good. Perl 5 developers hadn't gone through the pain of implementing, understanding, using, and maintaining an API that wants to be stateful and procedural and OO while reusing almost all of the same code.

This is not a slight on Lincoln or Perl 5 developers in the '90s or anyone. We (and notice that this is a collective noun) didn't know what code we would wish we had written back then. We know better now, and hopefully code we write now will survive better the next decade.

This task of evolution could have been easier, but tradition impedes all but the most serious changes to APIs shipped with Perl 5. Backwards compatibility means never having to say your first attempt wasn't perfect.

It's a tricky situation. A library that was the obvious best option at the time makes sense in a the core library. As time goes on, it becomes less obvious, but it's difficult to change because people are using it and it becomes difficult to remove because people are using it, and new people start using it because there's little friction to start using it, but the people in the know don't use it because it's not obviously the best choice... and you have a world in which people in the know can write really great Perl 5 code and people who start using Perl 5 write clunky Perl 5 code because the whole process makes it easiest for them to write clunky code.

Assume that decent programmers in the know will always be able to improve their ability to write great code. What if we focused on encouraging novices to write great code? (They won't write great code immediately, but we can encourage them to write less clunky code by discouraging them in ways they won't even notice from writing clunky code.)

How do you avoid the problem of effectively freezing good libraries forever by putting them in the core where they're easy for novices to use and too tempting for adepts to ignore?

I think you subvert the whole process and get rid of core libraries.

Next up, how this can work in practice. (For fun, ponder the existence of Perl 6 and its answer to this problem at the language level.)

Posted by chromatic on cpan, language design, modern perl, perl, perl 5, perl 6 at Tue, 08 Dec 2009 22:18:33 -0000 | Permalink

These links are collected from the Perlbuzz Twitter feed. If you have suggestions for news bits, please mail me at andy@perlbuzz.com.

• blogs.perl.org is a new, modern blogging platform for the Perl community (blogs.perl.org)
• Michael Peters gives thanks for some special folks in the Perl community (use.perl.org)
• Peteris Krumins explores secret Perl 5 operators (catonmat.net)
• Dave Rolsky talks about his Markdown parser, Markdent: (blog.urth.org)
• Using PUT and DELETE methods in web apps (blogs.perl.org)
• Tests are divine (blogs.perl.org)
• Perl advent calendars galore (perlbuzz.com)
• mirod releases a new version of xml_grep2 (blogs.perl.org)
• Introducing gitalist, a modern Git browser written in Perl (blogs.perl.org)
• Posting to CPAN from Github (blogs.perl.org)
• GitPAN's first success: Recreating the history for Pod::Simple (use.perl.org)
• Why use Kephra? (use.perl.org)
• Recap of Eric Whilhelm's Perl Foundation grant to overhaul learn.perl.org (news.perlfoundation.org)
• Using Perl to scrape the web (ssscripting.blogspot.com)
• Peteris Krumins publishes 30 projects on github (catonmat.net)
• cope, a command-line program colorizer in Perl (stuff.cytzol.org)
• The Perl way of avoiding repeated code: (blogs.perl.org)

Posted by Andy Lester on CPAN, Code craft, Community, Perl 5, Perl 6 at Tue, 08 Dec 2009 18:53:47 -0000 | Permalink

Second release, and I now have everything I need for doing basic file construction in Padre.

I've got whitespace control in tags via explicit chomping such as [%- foo -%].

The basic recursive expression resolver is done for things like [% hashkey.1.methodname %] and works for arrays, hashes and objects. The resolver also now has the same kind of exception-tolerant, warning-squashing behaviour that Template Toolkit does.

And finally I've added the three simplest and most basic control structures, IF, ELSE and UNLESS.

All this cost about an extra 15-20k of RAM, for around 40kb total so far (with maybe 5-10kb of recoverable fat). But note that I don't yet have support for files, so there's some fairly chunky costs still to come (especially because I can't use things like File::Spec).

Now this is where things get a little more interesting, because the real skill in creating most ::Tiny modules are about selecting the really critical compromises.

For Template::Tiny, the key compromise looks to be stash immutability. If you can be sure that the stash will never mutate (and if you don't allow things to collide with the stash names) you no longer need to process the template elements in order from top to bottom. Which means you don't need to compile the template into intermediate Perl code, the templates won't be turing complete, and you can just evaluate it directly from template to finished text via a series of regex passes.

Of course, now I have some other problems, such as doing a fairly complex and recursive regex matching exercise using only features available in Perl 5.005 (and in very little code, and without resorting to things like Text::Balanced that are 1000 lines of code).

So if you have some serious regex-fu, I'd love some help because my other option is trawling around in Text::Balanced-type Damiancode for clues :)

Posted by Alias on journal at Tue, 08 Dec 2009 14:17:27 -0000 | Permalink

Stumbled across this weird behavior today. Took a while to debug it. In the debugger, I'm not seeing the "dollar digit" regex capture variables set, even though the regex matches.

$ perl -de 1

Loading DB routines from perl5db.pl version 1.3
Editor support available.

Enter h or `h h' for help, or `man perldebug' for more help.

main::(-e:1):   1
  DB<1> ($foo) = ('abcd' =~ /(bc)/)

  DB<2> x $1
0  undef
  DB<3> x $foo
0  'bc'

Is this documented? I can't find it.

Update: Rafael explained it. The digit variables are lexically scoped. That's why you can assign to a package variable in a debugger, but not a lexical.

  DB<6>  'abcd' =~ /(bc)/ && print $1
bc

Posted by Ovid on journal at Tue, 08 Dec 2009 14:02:43 -0000 | Permalink

As promised yesterday there is now a mailing list for people who are interested in organizing Perl related content to other technical events. The idea is based on what people in Debian are doing and a suggestion that came from Salve J. Nilsen from Oslo.pm.

We will list a bunch of events all over the world where we think it is important to show the powers of Perl or Perl based projects. It can be an event QA showing Bugzilla or RT. It can be an event for web development where we can show Catalyst or Plack/PSGI. It can also be an event for System administrators where we can show tons of other things.

To subscribe to the list please send a message to events-subscribe@lists.perlfoundation.org .

Once you are registered you can send messages to events@lists.perlfoundation.org .

The list has a public archive.

In the coming days I'd like to make a list of potential conferences and events to attend so we can ask TPF for an appropriate budget to cover our expenses. Please also tell about this list to the various projecs that might be interested in showing themselves.

We also need to act quickly as during the coming holidays we won't be able to organize mutch and the FOSDEM - the first conference we are going to represent Perl in such a manner is less than 2 months away.

Posted at Tue, 08 Dec 2009 10:39:22 -0000 | Permalink

this was originally posted on my new blogs.perl.org journal, which can be found here which is also the RSS feed for it.

<rant>

Rapidshare deletes illegal files, so instead of sharing Rapidshare links, sites nowadays apparently started sharing .dlc files (or so I've heard). DLC is an encrypted container format. It is very very stupid. I'll elaborate.

Let's break it down:

• Only one server decrypts - single point of failure.
• Disregarding the necessity for internet, this binds you to specific programs that have the keys hardcoded in them to be able to access the server.
• The protocol is secret, the key is secret, the programs are closed source (at least the part that matters).
• One program is for Windows only. The main one is written in Java. Stupid Java. I've gone over the parts of it that are open source and it's horrible (really really horrible).
• I have not managed to get the program running on three different computers, and on Windows as well.
• Apparently they change the key every once in a while so you always have to stay updated with the program.

However, these are all implementation issues. The problem here is that a bunch of freelance programmers (I'm assuming) think that by hardcoding and source-closing the key to their wanna-be-open-source application, it will somehow deter people who make it their job to crack it.

Since you still want the average boob (that is, the average dumbbell) to be able to operate the program and download the stuff, you need to make it accessible enough for him/her. Once it's that accessible, it's accessible to any IRAA/MPAA/<enter agency name> agent to reach as well, since they are at least as smart as the average bear, err... boob.

You're telling me a well funded agency can't open the stupid program and click the link? They can't write a program that automatically opens the other program and clicks the link? Really? Oh, this is ultra super secure now? You can write a Visual Basic program that does that in 10 minutes.
(please don't write a visual basic program)

I've emailed at least one website that does that, trying to get a decent answer for this. Still no response.

As to the JDownloader developers: Your program doesn't work, and your super ultra secure technology is closed source i-wouldn't-touch-it-with-a-ten-foot-pole icky. Yes, icky. You've secured nothing!

Fox21.at [fox21.at] release various Perl programs. One of them is called dlc2txt (oh, here's the Google Translate for it). Using this program, given the key and host, you can just crack DLC files yourself.

Of course the developers keep the key in JDownloader only. They also issues two other keys for two other programs (one of them written in Python) and threatened them not to reveal the key. The other programs and JDownloader keep the key closed source by compiling it.

So you have to use JDownloader or the other programs.
Right? Wrong!

Here is a fully detailed post on how to crack DLC completely.

I will probably write a Perl module to use this new kickass webservice that cracks DLC for you.

Meanwhile, please quit trying to convince people that your new thinga-ma-gic protects them even though it really doesn't. It only adds complexity for the user (much more than for an agent) and pisses people (like me) off.

Thank you.

</rant>

Posted by xsawyerx on journal at Tue, 08 Dec 2009 09:45:19 -0000 | Permalink

We've been doing some (long overdue!) mailing list upgrades and in the process we managed to make the pm.org lists not work for a couple of hours and the perl.org lists for about 5 minutes earlier in the evening. All should be well now, but do let us know if you notice something amiss.

As an added bonus we now have lists on lists.perlfoundation.org. Subscribe to the first of them by emailing events-subscribe at that domain.

Posted by Ask Bj?rn Hansen at Tue, 08 Dec 2009 08:19:20 -0000 | Permalink

Today, a short one as I’m short on time. But I wasn’t able to do a PMM last week, and I’d hate to skip it two weeks in a row, so…

Say hello to File::Tail. A simple concept, but a very, very useful one.? Ever run the *NIX “tail?-f?<file>” command? Yeah, it’s pretty much just like that. Only in Perl, so it’s actually more flexible. Much more.

You can use it in a very simple manner, to read from files that are continuously updating. You can control how often and to what degree it adjusts the wait-interval it uses when polling the underlying file to check for changes. You can also tie a file-handle to it. You can do things like select-polling across several such handles or objects, to read in a non-blocking way from multiple tail’d files at once. It even comes bundled with two sample scripts showing it in action.

Check it out!

Posted by rjray on CPAN, Perl, module-monday at Tue, 08 Dec 2009 05:27:48 -0000 | Permalink

Posted by rjray on CPAN, Perl, Software, XML, RPC-XML at Tue, 08 Dec 2009 04:39:22 -0000 | Permalink

(or caring about details part three)

Kephra 0.4.3 will come soon and the beside the finally arriving utf support it will also have markers. its the same thing notepad++, kommodo and scite call bookmarks. Not that I want to be different, but its not the same thing as the bookmarks Kephra has for nearly 2 years and I also believe that these terms are more correct.

Nowadays that browser often set the standard, what UI means, a bookmark is a link to specific location, which marker's aren't. You make marks in your rifle or book and go just from one to the next to search the places you previously highlighted. Thats what Kephra marker do. Only with nicer icons and more TIMTOWTDI in the UI. Of course you have the known key binding, but you can also use them by mouse directly (toggle with left click on the marker margin) or by context menu (right click on the marker margin) or navigate them with icons in the searchbar.

The Kephra bookmarks have numbers and you press the key (ctrl+number) to jump directly to that spot. A fast way to navigate, if you have to switch between 2 or more places often. Marker highlight more places of interest that can also be set by search functions or an debugger. And marker can be saved in session files and file history. So if you open a file again all your marks are still there, but the bookmarks turned into marker because there could be a conflict. Every bookmark has an uniquely given number.

Nontheless: after a restart of the app all the bookmarks and marker will still be there. I even found a way to set and delete bookmarks by mouse (middle click on marker margin). Present bookmarks will deleted or the free bookmark with the lowest number will be set. because bookmarks got new nice white icons with its number in it you can see which bookmark is on the spot.

Yes that all are not the biggest IDE features, but we tend to make one well thought out and complete before we move on. And Kephra 0.4.4 will be about something completely different.

Posted by sir_lichtkind on journal at Mon, 07 Dec 2009 21:03:43 -0000 | Permalink

This is a post about Moose the post-modern Object-Oriented Programming system for Perl 5. Until a few days ago, I've learnt most of what I knew about Moose by reading some presentations about it and experimenting. However, a few days ago, I've decided that maybe there was a more-Moosey way of doing the following pattern that I've been doing in my classes:

# Super-base class.
sub new
{
    my $class = shift;

    my $self = {};
    bless $self, $class;

    $self->_init(@_);

    return $self;
}

# In derived classes:
sub _init
{
    my $self = shift;
    my $args = shift;

    $self->_init($args);
    # Initialise from $args.

    return;
}

Turns out there was - you can use the BUILD method to provide initialisation for each part of the class hierarchy (it gets calls in a walk-method style - not only for the most qualified sub-class). Only in my case in my en-Moosification of XML-Grammar-Fiction, I only needed a 'default' callback to the class members, and to make them 'lazy' by default.

This made me conclude that I have to learn Moose top-down to become more familiar with it. So I started reading the chapters of Moose::Manual and have already learned that I could encapsulate push() and shift() out of an array ref using its delegation, which also proved useful in XML-Grammar-Fiction. After I'm done with the manual, I'm planning to go over the Moose cookbook.

Hopefully, this will give me a lot of ways to avoid repetitive code in my CPAN modules. Of course, now, I'm facing the classic dilemma of whether to make something a base class or a role… decisions, decisions.

Posted by shlomif@iglu.org.il (Shlomi Fish) on object oriented prorgamming, fiction, grammar, perl, xml, oop, moose, manual, cpan at Mon, 07 Dec 2009 20:43:50 -0000 | Permalink

mscolly correctly identified the SQL bug I posted. Sadly, no one discussed the implications of this bug and I think they're the most interesting part of this. Essentially, it comes down to the following:

SELECT    first_name,
          last_name,
          order_date,
          SUM(price) AS total                  -- what if there's no price?
FROM      customer
LEFT JOIN orders     ON customer.id = orders.customer_id
LEFT JOIN order_item ON orders.id   = order_item.order_id
GROUP BY  first_name, last_name, order_date
HAVING    total < 15                           --  what does NULL < 15 evalute to?
ORDER     BY order_date ASC;

The proper solution (as mscolly pointed out) is to change the "SUM" line to this:

COALESCE( SUM(price), 0 ) AS total

The English language, as we know, is ambiguous. If you boss had come in and asked for all customers whose orders (more accurately, whose orders with order items) totaled less than £15 pounds, then the above query would have actually been correct, but another programmer coming along to maintain it could be forgiven for thinking it's in error. If you ever write SQL which is likely to produce NULLs (e.g., outer joins), you should explicitly handle that case if you actually do anything with those NULLs.

But in this case, "customers whose orders total less than £15" is significantly different in meaning than "customers who spent less than £15" and the latter is what we want, but the former is what we have. While the above code seems logical, it gives a logically incorrect answer because it omits customers without orders (or order items), even though they're clearly intended. However, NULLs make it very difficult to identify what you actually mean because the database can't know why something is NULL.

Now consider a simpler, yet silly, example:

SELECT first_name
FROM   employee
WHERE  salary > 50000;

What happens if the salary field is NULL? You'll get a list of employees whose known salary is NOT NULL. Why might they not have a salary? Maybe they're an hourly employee and the salary field is not applicable. Maybe they're the CEO and he doesn't think you need to know his salary. Maybe they're an ex-employee and they have no salary.

Taking this a bit further, imagine that all employees in the table are current and all have salaries (no hourly workers), but the salary field is still sometimes NULL because the board of directors doesn't want you to know their salaries. With me so far? In this scenario, it is the case that everyone has a salary; you just don't know what some of them are. So here's the kicker:

SELECT first_name
FROM   employee
WHERE  salary = salary;

That won't return anyone on the board of directors, even though you know they have a salary. Furthermore, most would think it's self-evident that p = p, but in three value logic of databases, this is sometimes true and sometimes false. Heck, because of this, the following does not always evaluate correctly, even though we would think it does:

SELECT service_id,
    CASE WHEN master_brand_id =  master_brand_id THEN '='
         WHEN master_brand_id != master_brand_id THEN '!='
    END AS 'comparison'
FROM service

Sure, you say, but you're comparing something to itself. You don't do that in the real world. No? So look at this:

SELECT s.service_id,
    CASE WHEN m.master_brand_id =  s.master_brand_id THEN '='
         WHEN m.master_brand_id != s.master_brand_id THEN '!='
    END AS 'comparison'
FROM service      s
     master_brand m

If the s.master_brand_id is allowed to be NULL, than the comparison field will always have a NULL value when s.master_brand_id is NULL. It's easy to debug in this simple example, but what if that was a subquery? It looks fine, but it all breaks down in the presence of NULL values.

I didn't start with that example because people would say it's silly, but starting with the "order" example shows how NULLs in databases can return logically incorrect data and the reduction down to the simple p = p case not holding shows why this happens.

At this point, I can see people saying "yeah, but we already know that about databases." And this is true. It's well-known that certain types of queries can generate NULLs even though there are no NULL values in the database. Regrettably, many people assume the database logic is, well, logical. The p = p failure is a strong rebuttal, but I suppose some people assume that hitting themselves in the head with a hammer is normal.

If you really want to have some fun, read this blog entry about NULL values. In the comments, the author even explains how to deal with NULLs in outer joins, but it requires a relational database (very few databases really are) and that people understand what first normal form is really about. (If you think you know, please define "atomic values" in the comments below).

I wonder how database design would look today if, instead of 3VL, databases threw an exception when you tried to apply an operator or aggregation ('=', '+', 'SUM', etc.) to NULL values?

Note: I've discussed the problem with NULL values before, but in realizing I had a better real-world example, I thought it would make more sense to readers.

Posted by Ovid on journal at Mon, 07 Dec 2009 15:38:13 -0000 | Permalink

Eric Wilhelm has finished his grant improving learn.perl.org". He has written
six tutorials - from "Getting Started with Perl" to "Your First GUI Desktop
Application".

The six tutorials are

• Getting Started with Perl
• Control, Structures, Scoping, and Subroutines
• Using and Creating Modules
• An Introduction to Objects
• Perl/CPAN Configuration Howto
• Your First GUI Desktop Application

The tutorials are currently hosted http://learnperl.scratchcomputing.com, but if
Eric finds a better permanent home for them, he will setup redirects for that.

In addition to the original plan, Eric has created the
Combust::Spontaneously module and published it on the CPAN. This
lowers the technical barrier for contributing to perl.org sites by
allowing editors to preview their changes without needing to be an
apache admin. Eric hopes that this tool ultimately leads to perl.org
improvements which exceed what he had planned in terms of site rework
(perhaps it already has).

Eric's grant proposal included a website redesign, but the whole *.perl.org family
was redesigned by Leo Lapworth and friends (see http://news.perlfoundation.org/2009/11/new_look_on_wwwperlorg.html).

The initial Grant proposal can be found at
http://www.perlfoundation.org/eric_and_tina_improving_learn_perl_org.

Posted by reneeb on Grants, grants, learn.perl.org at Mon, 07 Dec 2009 14:01:08 -0000 | Permalink

I have just received an e-mail from the organizers of FOSDEM that my request for a Perl stand was accepted. This means we are going to have a table on the two days of FOSDEM (Saturday and Sunday, 6-7 Feburary 2010) where we can present Perl and related projects.

We need to ensure there are always people at the stand, that we have material to hand out. We have to make sure projects that can be interesting to people who are not (yet) Perl developers are well represented. I hope The Perl Foundation will finance some interesting give-aways.

There were a couple if ideas we discussed with Karen Pauley and Salve J. Nilsen. For example the wooden round tuits we got on pervious YAPCs and the baseball cards the Nordic Perl Workshop organizers handed out.

We also talked about the possibility to prepare a CD or DVD with various Perl projects on it. It could hold a version of Strawberry Perl for Windows or a Virtual Box image holding installed versions of several projects or the same but in a bootable image format. Something like Perl Virtual Appliances.

Very soon we are going to have a mailing list to coordinate the Perl presence on this and other events. I'll announce the creation of the list. In the meantime if you are planning to come to FOSDEM and would like to represent a project or if you just would like to participate in the organization please let me know by sending and e-mail to me.

Posted at Mon, 07 Dec 2009 09:30:50 -0000 | Permalink

Update: You can ignore the order_date below. It's a red herring and I probably should have left it out, but I had liked the fact that by putting it in the query, I added more complexity, thus making the real bug more difficult to spot.

Assume you're a diligent programmer. You've designed your database carefully. Foreign constraints are correct, you have no null columns and you've kept a nice, simple design. Now your boss wants you to provide a list of all customers who've spent less than £15 on your Web site because you want to offer them a special promotion. Here's the SQL you've written:

SELECT    first_name,
          last_name,
          order_date,
          SUM(price) AS total
FROM      customer
LEFT JOIN orders     ON customer.id = orders.customer_id
LEFT JOIN order_item ON orders.id   = order_item.order_id
GROUP BY  first_name, last_name, order_date
HAVING    total < 15
ORDER     BY order_date ASC;

The two left joins are there because a customer may never have placed an order before. Heck, they may have started an order but not added any items to it. The 'having' statement is required because you generally can't use aggregates in where clauses.

You run the SQL and hand-check the results very carefully. Choosing a random sampling of customers returned, you verify that none of them have spent more than £15 on your site. Nonetheless, you have a bug. What is it? What are the implications of the bug?

Posted by Ovid on journal at Mon, 07 Dec 2009 07:43:40 -0000 | Permalink

Posted by rjray on CPAN, Perl, Software, XML, RPC-XML at Mon, 07 Dec 2009 06:54:50 -0000 | Permalink

For a few years now, I've been thinking about doing a ::Tiny job on the Template Toolkit. But it was never quite a big enough problem to be worthwhile, because Template.pm doesn't have any particularly horrendous dependencies and always seems like a reasonable way to spend 2.2meg of RAM.

But for Padre 2.2meg is a hell of a lot to spend, because of the multiplier effect of threading. Template.pm is geared much more towards being a load focal point in a website or a content generation system, it is just that much too complex when used as a utility class for simple code generation (which is what I want to use it for in Padre).

So Template::Tiny is happening, and I've just uploaded the first release which support pretty much nothing other than basic [% foo %] insertions from a single variable HASH.

Just adding support for tags, plus a tiny bit of boilerplate, has memory consumption at around 24k of RAM consumed out of my anticipated budget of around 100k.

Before I go any further, I'd like to ask you guys what you consider to be the essential TT features that you would want to retain even in a light weight ::Tiny context.

For me, this includes the three basic array, hash and object foo.bar usages, and IF blocks (and potentially simple | filters).

What do you want?

Posted by Alias on journal at Sun, 06 Dec 2009 20:54:37 -0000 | Permalink

Yesterday was the London Perl Workshop. As always it was a fabulous day packed full of great talks about Perl. Thanks to the organisers for all the work they put in.

I gave the keynote speech first thing in the morning. The talk was called The "M" Word and it was an overview of how the Perl community has started to get to grips with the problem of marketing over the last year.

Here are the slides:

Posted by Dave Cross on Conferences, conferences, keynote, london, lpw, lpw2009, marketing at Sun, 06 Dec 2009 18:52:08 -0000 | Permalink

Yesterday was the 2009 London Perl Workshop. It was a really good day. There were lots of extremely well presented talks on a wide range of topics. Hats off to the London.pm for yet again organising a fantastic event.

Posted by Adam Trickett (ajt) at Sun, 06 Dec 2009 10:59:00 -0000 | Permalink

Posted by draegtun on Programming, perl at Sun, 06 Dec 2009 10:12:52 -0000 | Permalink

Padre is getting better by every release, but many people have not heard of it yet. It needs some promotion so I am wondering which one of the social networks and other tools could be used for this?

Blogging about it certainly helps. Submitting those posts to Reddit also has its impact and there are number of other ways. In any case it is you who can make the difference.

If you are using Padre already or if you think the Perl world would be better off if people used a Perl aware editor instead of Notepad++ and other generic programmers editors or even if you just think that the whole idea of building an open source IDE for Perl in Perl is a good cause, consider spending a few minutes in helping promoting it.

Some of the things you can do:

Mark yourself as a user of Padre on Ohloh (there are 24 users currently).

Subscribe to Padre on Freshmeat (there are now 16 subscriptions).

Join the Padre group on LinkedIn (there are currently 17 members).

I recently setup a Padre group on Facebook (there is 1 member now :-). and Padre page of Facebook where you can become a fan. (thanks to Bogdan Lucaciu)

I even setup a Twitter account for Padre you can follow.

I tried to search for a Perl IDE on Google. Padre was in 8th place with the perlide.org site being number 5. It is not bad but moving up the ladder would be nice. So if you have a web site, consider linking to Padre or to perlide.org from there.

Just to give it a shot I tried to search for Perl IDE on Bing as well. It shows perlide.org on 3rd place and on the 9th place but padre.perlide.org is only number 15.

After doing all the above, let me know, what other ways do you think Padre could be promoted?

Posted at Sun, 06 Dec 2009 09:58:37 -0000 | Permalink

Someone on IRC reported a bug in Bryar. Namely that a Naughty Person can exploit the feature that notifies you of blog-spam by email to execute arbitrary code on your machine, as the user you run Bryar under.

A patched release is on the way to the CPAN, and you are strongly urged to upgrade.

Posted by david | Permalink

[originally posted on Apr 3 2008]

I'd like to express my warm thanks to the lovely people at Yahoo and in particular to their bot-herders. Until quite recently, their web-crawling bots had most irritatingly obeyed robot exclusion rules in the robots.txt file that I have on CPANdeps. But in the last couple of weeks they've got rid of that niggling little exclusion so now they're indexing all of the CPAN's dependencies through my site! And for the benefit of their important customers, they're doing it nice and quickly - a request every few seconds instead of the pedestrian once every few minutes that gentler bots use.

Unfortunately, because generating a dependency tree takes more time than they were allowing between requests, they were filling up my process table, and all my memory, and eating all the CPU, and the only way to get back into the machine was by power-cycling it. So it is with the deepest of regrets that I have had to exclude them.

Cunts.

[update] For fuck's sake, they're doing it again from a different netblock!

Posted by david | Permalink

As a service to module authors, here is a tool to show a module's pre-requisites and the test results from the CPAN testers. So before you rely on something working as a pre-requisite for your code, have a look to see how reliable it and its dependencies are.

Posted by david | Permalink

Did you ever wonder how BEGIN, CHECK, END and so on are called in Perl? Well, they didn't have a good name, until recently.

The Perl?6 spec listed them under closure traits, which is unwieldy, and not really exact either. Now they are called phasers, because they tell you which execution phase the block or statement runs in.

There are so many possible puns that I'll refrain from writing any.

Posted by Moritz Lenz | Permalink

Lately I've been playing a lot of with SVG, and all in all I greatly enjoyed it. I wrote some Perl 6 programs that generate graphical output, and being a new programming language it doesn't have many bindings to graphic libraries. Simply emitting a text description of a graphic and then viewing it in the browser is a nice and simple way out.

I also enjoy getting visual feedback from my programs. I'd even more enjoy it if the feedback was more consistent.

I generally test my svg images with three different viewers: Firefox 3.0.6-3, inkscape (or inkview) 0.46-2.lenny2 and Opera 10.00.4585.gcc4.qt3. Often they produce two or more different renderings of the same SVG file.

Consider this seemingly simple SVG file:

<svg
    width="400"
    height="250"
    xmlns="http://www.w3.org/2000/svg"
    xmlns:svg="http://www.w3.org/2000/svg"
    xmlns:xlink="http://www.w3.org/1999/xlink"
    style="background-color: white"
>

    <defs>
        <path id="curve"
              d="M 20 100
                 C 60 30
                 320 30
                 380 100"
              style="fill:none; stroke:black; stroke-width: 2"
        />
   </defs>
   <text font-size="40" textLength="390" >
        <use xlink:href="#curve" />
        <textPath xlink:href="#curve">SPRIXEL</textPath>
   </text>
</svg>

If your browser supports SVG, you can view it directly here.

This SVG file first defines a path, and then references it twice: once a text is placed on the path, the second time it is simply referenced and given some styling information.

Rendered by Firefox:

Rendered by Inkview:

Rendered by Opera:

Three renderers, three outputs. Neither Firefox nor Inkview support the textLength attribute, which is a real pity, because it's the only way you can make a program emit SVG files where text is guaranteed not to overlap.

If you scale text in Inkscape and then put it onto a path, the scaling is lost. I found no way to reproduce opera's output with inkscape without resorting to really evil trickery (like decomposing the text into paths, can then cutting the letters apart and placing them manually). (Equally useful is the dominant-baseline attribute, which Inkscape doesn't support either).

The second difference is that only Firefox shows the shape of the path. Firefox is correct here. The SVG specification clearly states about the use attribute:

For user agents that support Styling with CSS, the conceptual deep cloning of the referenced element into a non-exposed DOM tree also copies any property values resulting from the CSS cascade [CSS2-CASCADE] on the referenced element and its contents. CSS2 selectors can be applied to the original (i.e., referenced) elements because they are part of the formal document structure. CSS2 selectors cannot be applied to the (conceptually) cloned DOM tree because its contents are not part of the formal document structure.

Sadly it seems to be a coincidence that Firefox works correctly here. If the styling information is moved from the path to the use element the curve is still displayed - even though it should not be.

Using SVG feels like writing HTML and CSS for 15 year old browsers, which had their very own, idiosyncratic idea of how to render what, and what to support and what not.

Just like with HTML I have high hopes that the overall state will improve; Indeed I've been told that Firefox 3.5 now supports the textLength attribute. I'd also love to see wide-spread support for SVG animations, which could replace some inaccessible flash applications.

Posted by Moritz Lenz | Permalink

How hard is it to build a good search engine? Very hard. So far I thought that only one company has managed to build a search engine that's not only decent, but good.

Sadly, they seem to have overdone it. Today I searched for tagged dfa. I was looking for a technique used in regex engines. On the front page three out of ten results actually dealt with the subjects, the other uses of dfa meant dog friendly area, department of foreign affairs or other unrelated things.

That's neither bad nor unexpected. But I wanted more specific results, so I decided against using the abbreviation, and searched for the full form: tagged deterministic finite automaton. You'd think that would give better results, no?

No. It gave worse. On the first result page only one of the hits actually dealt with the DFAs I was looking for. Actually the first hit contained none of my search terms. None. It just contained a phrase, which is also sometimes abbreviated dfa.

WTF? Google seemed to have internally converted my query into an ambiguous, abbreviated form, and then used that to find matches, without filtering. So it attempted to be very smart, and came out very stupid.

I doubt that any Google engineer is ever going to read this rant. But if one is: Please, Google, keep it stupid, stupid.

I'm fine with getting automatic suggestions on how to improve my search query; but please don't automatically "improve" it for me. I want to find what I search for. I'm not interested in dog friendly areas.

Posted by Moritz Lenz | Permalink

Most programming languages handle failures with either of two paradigms: failing routines return special values, or they throw exceptions.

Either way has its severe problems: in languages like C it can be very simple to forget to catch such a return value, and very tedious to propagate them to the caller; on the other hand throwing exceptions often clutters the code with way too many try blocks, and it's generally unfriendly if you try to automatically parallelize expressions.

So Perl?6 offers a middle ground: soft or unthrown exceptions. If a routine calls fail("message"), a new Failure object is created and returned from the current routine. That object behaves as an undefined value, which stores the message, file and line information of the fail() location, a backtrace and so on.

When you ask such an object whether it's true or false, or defined or undefined, you'll get a correct answer, and the exception is marked as handled. However if you try to use it as an ordinary value, it turns into an (ordinary) fatal exception. So both of these work:

# Variant 1: no exception thrown

my $handle = open('nonexistingfile');
if $handle {
    .say for $handle.lines;
} else {
    # do something else
}


# Variant 2

my $handle = open('nonexistingfile');

# throws a fatal exception while calling $handle.lines
.say for $handle.lines;

Now if you do some automatically parallelized operations, a single failure doesn't have to abort the whole operation, and neither is information lost

# divide @a1 by @a2 element-wise, a division by zero might occur:
@a1 ?/? @a2;

The API for accessing the Failure objects isn't very mature yet, but the concept stands. See S04/Exceptions for the gory details, as they stand today.

Posted by Moritz Lenz | Permalink

For the last few months I've been using git for my version control system. It's better than CVS because it can handle offline commits. So if I'm using my laptop on a train, I can still use version control without having to have a notwork connection.

And to give a pretty web front-end to it for other people to read code without having to check it out of the repository, I use cgit, which I mostly chose because it's a dead simple CGI and not a huge fancy application.

One problem with cgit is that by default it doesn't do code highlighting. But it has the ability to run blobs of code through any filter you care to name before displaying them, so to get something nice like this all you need to do is write a highlighter and add a single line to your cgitrc:

source-filter=/web/www.cantrell.org.uk/cgit/highlighter

My highlighter program is this:

   1 #!/usr/local/bin/perl
   2 
   3 use warnings;
   4 use strict;
   5 
   6 my $file = shift;
   7 
   8 if($file =~ /\.(p[ml]|t)$/i) {
   9     system "/usr/local/bin/perltidy -html -st -ntoc -npod -pre -nss -nnn"
  10 } else {
  11     system "cat -n";
  12 }

Posted by david | Permalink

I made a shiny thing! It can plot arbitrary functions of the form x=f(y) or y=f(x). Under the skin, it just massages its arguments and passes them through to Gnuplot. Here's the source code.

Update: now 48.3% even shinier - see on the right

Posted by david | Permalink

I am ill. I've been ill since Thursday, with a cold. You're meant to be able to cure a cold with [insert old wives tale remedy here] in 5 days, or if you don't, it'll clear itself up in just under a week. So hopefully today is the last day.

So what have I done while ill?

On Friday I became old (see previous post), and went to the Byzantium exhibition at the Royal Academy. It was good. You should go.

Saturday was the London Perl Workshop. My talk on closures went down well, and people seemed to understand what I was talking about. Hurrah! I decided that rather than hang around nattering and going to a few talks, I'd rather hide under my duvet for the rest of the day.

I mostly hid on Sunday too, and spent most of the day asleep. In a brief moment of productivity, I got my laptop and my phone to talk to each other using magic interwebnet bluetooth stuff. I'd tried previously without success, but that was with the previous release of OS X. With version X.5 it seems to Just Work, so no Evil Hacks were necessary.

The cold means that I can't taste a damned thing, not even bacon. So now I know what it's like to be Jewish. Being Jewish sucks.

And today, I am still coughing up occasional lumps of lung and making odd bubbling noises in my chest, although my nasal demons seem to be Snotting less than they were, so hopefully I'll be back to normal tomorrow.

Posted by david | Permalink

Devel::CheckLib has grown a new feature. As well as checking that libraries and headers exist, it can now also check that particular functions exist in a library and check their return values. This will be particularly useful if you need to check that a particular version of a library is available.

It works if you have a Unixish toolchain. I need to wait for the CPAN-testers reports to see if I managed not to break anything on Windows. Unfortunately, even though the lovely Mr. Alias has worked hard to make Windows machines available to developers, I found it to be just too hard to use. Even downloading my code to the Windows machine was hard, as Windows seemed to think it knew better and shouldn't download the file I told it to download. Then once I had downloaded it, Windows decided to hide it somewhere that I couldn't get to using the command line. So I gave up.

I might try again once there are some decent tools on the machines: wget, tar, and gzip at minimum, as given those I can quickly bootstrap anything else. Software development isn't just about having compilers available.

Posted by david | Permalink

One of my CPAN distributions is CPAN-FindDependencies. It contains a module CPAN::FindDependencies, and a simple script that wraps around it so you can view dependencies easily from the command line. That script, naturally, has a man page. However, that manpage basically says "if you want to know what arguments this program takes, see the CPAN::FindDependencies docs". This is Bad from a usability point of view, good from a not-duplicating-stuff point of view, and good from a laziness point of view. Which means that it's Bad.

So, the solution.

=over

#include shared/parameters

=back

and some Magic that does the cpp-stylee substitution at make dist time. Note the 'dist' section in my call to WriteMakefile.

This is, of course, crying out to be made less horribly hacky, but it works for now, so I'm happy.

My original idea was to write some crazy shit that would do the #include at install-time, when the user was installing my code. But that has the disadvantage that tools like search.cpan wouldn't show it properly, as they simply look at the files in the distribution. So this does the #includes at the last moment just before I package up the code and upload to the PAUSE. You lovely people get the right documentation in all the right places, I only have to maintain it in one place so it stays in sync, and (in the interests of Laziness) I don't have to remember to run any extra scripts before releasing, make dist just Does The Right Thing.

Posted by david | Permalink

As is becoming normal, I used the times between talks to bugfix some of my modules - this time Tie::STDOUT and Data::Transactional. The former was failing on perl 5.6, the latter on 5.9.5. The former was a bug in perl (you can't localise tied filehandles and expect the tieing to go away in 5.6, so it now declares a dependency on 5.8), the latter was a bug in my code.

Philippe Bruhat's talk on Net::Proxy was great - you can tell it's great because I came away with ideas for at least four things that I need to write. First up will be a plugin for it to allow the user to specify minimum and maximum permitted data rates for proxied connections. This will permit bandwidth limits for maximum permitted rates, but will also help to defeat IDSes doing traffic analysis if you specify a minimum permitted data rate.

This will protect (eg) ssh sessions from being identified based on their very bursty traffic pattern, by "filling in the blanks" with junk data.

In the evening, the CPAN-testers BOF was productive.

Posted by david | Permalink

I have released my XML::Tiny module. The parser at its core is less than twenty lines of code. Pretty easy to follow code too, I think, and that also includes error handling. One of my aims in writing it was to keep memory usage and code to the absolute minimum, so it doesn't handle all of XML. The documentation says that it supports "a useful subset of XML". Personally, I think it supports the useful subset. It's certainly enough to parse the data I get back from Amazon when I use their web services, and to parse an RSS feed.

Posted by david | Permalink

To make up for a disappointing gap in Palm's software for the Treo smartphone, I wrote a <a href=http://www.cantrell.org.uk/david/tech/treo/call-dumper/>small perl script</a> to parse the database that stores my call history. I then re-wrote it as <a href=http://search.cpan.org/search?query=Palm%3A%3ATreoPhoneCallDB>a re-useable module</a> which also figgers out whether the call was incoming or outgoing.

Posted by david | Permalink

These tidings posts seem to become rather sparse, but I hope you get some news by reading the Planet Six feed aggregator anyway.

Specification

• Larry lifted up the dual nature of Ranges. They mostly serve as an interval now, for smart iteration the series operator has been pimped up. You can now write for 1, 3 ... *+2, 9 { .say } to print all the odd numbers between 1 and 9. (r28344, r28348, r28351).
• Rational and Complex types now have their own literals (r28173).
• Stubbed classes are now documented (r28196, r28197).
• The new S08 documents Parcels and Captures.
• The numeric types have been cleaned up a lot (r28502 and later commits up to r28597).
• New and improved signature introspection (r28664, r28665).

Compilers

Rakudo

As opposed to two months ago, Rakudo now

• supports the Rat type
• supports overloading of many built-in operators
• has contextual variables
• has a faster and much better signature binder
• supports all kind of trigonometric functions, including on complex numbers
• implements sophisticated signature introspection

Patrick Michaud is also working on a new tool named npq-rx, which combines a self-hosting NQP compiler and a new regex engine, which already supports proto regexes, NQP code assertions and closures, and is generally much faster and better than PGE.

Sprixel

Mathew Wilson aka diakopter started sprixel, a Perl?6 to Javascript compiler.

Mildew

Mildew now has an experimental javascript emitter.

Other matters

perl6.org is redesigned again, this time spanning multiple pages, thus allowing much more stuff to be linked there.

Four Perl 6 and Rakudo hackers announced that they are writing a Perl 6 book, the print release of which shall coincide with the release of Rakudo Star.

Posted by Moritz Lenz | Permalink

There's a new release, <a href=http://www.cantrell.org.uk/david/tech/perl-modules/Number-Phone-1.58.tar.gz>version 1.58</a>, of Number::Phone, my set of perl modules for picking information out of phone numbers. Changes from the previous release are that Mayotte, Reunion and Comoros can't decide which country is which, and there's the usual updates to the database of UK numbers, mostly to support the <a href=http://www.ofcom.org.uk/media/news/2007/02/nr_20070213b>new 03 numbers</a>.

Posted by david | Permalink

If you have an array @a and want to access the first element, in Perl 5 you write that as $a[0], in Perl 6 you write it as @a[0]. We call the former mutable sigil, and the latter immutable sigil.

You might think that's a small change, but the implications are rather deep, and we've had quite a few discussions about it in #perl6. In particular people often ask if it's possible to backport the Perl 6 behavior to Perl 5. The answer is "not easily".

In Perl 5 context propagates inwards, which means that in a statement like

... = func()

The compiler wants to know at compile time which context func() is in. If it doesn't, it complains:

2$ perl -ce '(rand() < 0.5 ? $a : @a) = func()'
Assignment to both a list and a scalar at -e line 1, at EOF
-e had compilation errors.

This also means that, in Perl 5, array slices and scalar array accesses have to be syntactically distinguished:

my @a;
$a{other_func()} = ...; # scalar context
@a{other_func()} = ...; # list context

So you can't just make sigils in Perl 5 immutable without also rewriting the whole context handling rules.

In Perl?6 that's not a problem at all, because functions don't know the context they're in, in fact can't know because of multi dispatch.

Instead functions return objects that behave appropriately in various contexts, and the context is determined at run time.

After getting used to it the immutable sigils are quite nice, and less complicated when references are involved. Anybody who objects without having tried it for at least three weeks, and is spoiled by Perl 5, will be shot.

Posted by Moritz Lenz | Permalink

<a href=http://cpandeps.cantrell.org.uk/>CPANdeps</a> now lets you filter test results by perl version number, and also knows what modules were in core in which versions of perl. Hurrah!

Posted by david | Permalink

A day of not many talks, but lots of cool stuff. Damian was his usual crazy self, and MJD's talk on building parsers was really good. Although I probably won't use those techniques at work as functional programming seems to scare people.

The conference dinner at a Heuriger on the outskirts of Vienna was great. The orga-punks had hired a small fleet of buses to get us there and back, and one of the sponsors laid on a great buffet. The local wine was pretty damned fine too, and then the evening de-generated into Schnapps, with toasts to Her Majesty, to her splendid navy, and to The Village People.

It wasn't all debauchery in the evening though - on the bus, I had a very useful chat with Philippe about Net::Proxy, and re-designing it to make it easier to create new connectors for it.

Posted by david | Permalink

There were quite a few interesting talks in the morning, especially Ivor's one on packaging perl applications. Oh, and mine about rsnapshot, of course, in which people laughed at the right places and I judged the length of it just right, finishing with a couple of minutes left for questions.

At the traditional end-of-YAPC auction, I avoided spending my usual stupid amounts of money on stupid things, which was nice. Obviously the hundred quid I put in to buying the hair style of next year's organisers wasn't stupid. Oh no. Definitely not.

An orange mohican will suit Domm beautifully.

Posted by david | Permalink

We want a Perl 6 book. We want it badly enough to write it ourselves. So that's what we're doing: writing one.

We, that is Patrick Michaud (architect of the Rakudo Perl compiler), Jonathan Worthington (prolific contributor to both Rakudo and Parrot), Carl M?sak (frenetic Rakudo user, and our number one bug finder) and Moritz Lenz (keeper of the Perl 6 test suite, and Perl 6 user and blogger). We are also open to contribution from others - already Jonathan Scott Duff has written an initial preface for us.

We don't have a name yet for our book. We want to cover the basics of Perl?6, enough to get your feet wet, and enough to make you want to use it. We want it to be based on useful examples. It is not going to be the definitive book, that task we leave to Larry Wall and Damian Conway.

Our vision is to present primarily the subset of Perl 6 that Rakudo understands, and have printed copies available by the time Rakudo Star is released, that is April or May 2010. chromatic and Allison Randal have kindly offered to published it via Onyx Neon Press.

Until then, monthly releases will be published under a Creative Commons license (noncommercial, attribution, share-alike).

Currently we have four chapters under construction, and the intention of writing the more introductory chapters later, when we know what we need to introduce for the later chapters. So far we have

• Multi dispatch
• Classes and Object
• Regexes
• Grammars

You can download the preliminary PDF version of the book here.

Interested? Check out the git repository, and join us in irc://freenode.net#perl6book.

Posted by Moritz Lenz | Permalink

My Lightning Talk on cpandeps went down really well, although as José pointed out, I need to fix it to take account of File::Copy being broken. I also need to talk to Domm after the conference is over to see if I can get dependency information from CPANTS as well as from META.yml files.

There were lots of other good lightning talks. Dmitri Karasik's regexes for doing OCR, Juerd Waalboer's Unicode::Semantics, and Renée Bäcker's Win32::GuiTest were especially noteworthy.

Richard Foley's brief intro to the perl debugger was also useful. Unfortunately Hakim Cassimally's talk was about debugging web applications, which I'd not noticed on the schedule, so I didn't stay for that.

And finally, Mark Fowler's grumble about why perl sucks (and what to do about it) had a few interesting little things in it. I am having vaguely sick ideas about mixing some of that up with an MJD-stylee parser.

At the auction I paid €250 to have the Danish organisers of next year's YAPC::Europe wear the Swedish flag on their foreheads. This, I should point out, was Greg's idea. I would never be so evil on my own.

Posted by david | Permalink

Perl isn't dieing, but it tells me that it wishes it was. Last night it went out on the piss with Python and Ruby (PHP was the designated driver) and it did rather too many cocktails. It isn't quite sure what happened, but it woke up in the gutter in a puddle of its own fluids and its head hurts a lot.

It asked me to ask you all to keep the volume down.

Posted by david | Permalink

I'm going to Vienna by train for YAPC::Europe. If you want to join me you'll need to book in advance, and probably quite some way in advance as some of these trains apparently get fully booked.

	arr	dep	date
Waterloo		1740	Fri 24 Aug
Paris Nord	2117
Paris Est		2245
Munich	0859	0928	Sat 25 Aug
Vienna	1335

The first two legs of that are second class, cos first wasn't available on Eurostar (being a Friday evening it's one of the commuter Eurostars and gets booked up months and months in advance) and was way too spendy on the sleeper to Munich. Upgrading to first class from Munich to Vienna is cheap, so I have.

Coming back it's first class all the way cos upgrading was nearly free ...

	arr	dep	date
Vienna		0930	Fri 31 Aug
Zurich	1820
Zurich		1402	Sun 2 Sep
Paris Est	1834
Paris Nord		2013
Waterloo	2159

Don't even think about trying to book online or over the phone, or at the Eurostar ticket office at Waterloo. Your best bet is to go to the Rail Europe shop on Picadilly, opposite the Royal Academy and next to Fortnums.

Posted by david | Permalink

In the great tradition of Perl Advent Calendars, colomon started and announced the 2009 Perl?6 Advent Calendar, with a post about Perl?6 each day.

After the first post many #perl6 regulars volunteered to contribute a post, so 20 of the 24 days are already allocated.

I'm looking forward to many nice posts, most of which will probably highlight a small Perl?6 feature.

Posted by Moritz Lenz | Permalink

In Perl?5 there is the undef value. Uninitialized variables contain undef, as well as non-existing hash values, reading from unopened or exhausted file handles and so on.

In Perl?6 the situation is a bit more complicated: variables can have a type constraint, and are initialized with the corresponding type object:

my Int $x;
say Int.WHAT();     # Int()

These type objects are also undefined, but in Perl?6 that doesn't mean they are a magical value named undef, but that they respond with False to the defined() subroutine and method.

In fact there is no undef anymore. Instead there are various values that can take its place:

Mu is the type object of the root type of the object hierarchy (or put differently, every object in Perl?6 conforms to Mu). It's the most general undefined value you can think of.

Nil is a "magic" value: in item (scalar) context it evaluates to Mu, in list context it evaluates to the empty list. It's the nothing to see here, move along value.

Each type has a type object; if you want to return a string, but can't decide which, just return a Str.

Other interesting undefined values are Exception (which usually contain a message and a back trace), Failure (unthrown exceptions), Whatever is a generic placeholder that can stand for "all", "infinitely many", "many" or as a placeholder for a real value.

Posted by Moritz Lenz | Permalink

A few days ago masak blogged about the social aspect of the is Perl?6 really Perl? question.

He presumes that the answer is yes, but doesn't tell us why. I'll try to give some reasons.

Perl?6 started as the successor to Perl?5

Perl?6 started off as the successor to Perl?5, at a Perl?5 meeting, by the Perl crowd. It was a plan to escape both the backwards compatibility trap (which meant that broken things couldn't be fixed without many people yelling), and the lack of momentum in the community.

Perl?6 embraces the Perl philosophy

What makes Perl Perl? In my opinion it's not the sigils on variables that make Perl Perl, or that writing a regex only need two characters and so on. It's mostly the philosophy that makes the difference.

There are some underlying principles like TIMTOWTDI, context sensitivity, convenience over consistency, making simple things easy and hard things possible, often used constructs short and less frequent things longer, and so on.

Perl?6 is founded on all those philosophies and ideals, and also shares some technical principles. For example sigils on variables (oh, I mentioned them already ...), easy access to powerful regexes, that fact that operations coerce their arguments (instead of the type of arguments determining the operation like in javascript, where a + can either mean addition or string concatenation).

So if you agree with my definition of what makes Perl Perl, Perl?6 is also Perl. If not, please tell me what's the essence of Perl!

Posted by Moritz Lenz | Permalink

<meta>From my useless musings series.</meta>

As a programmer you have to have confidence in your skills, to some extent, and at the same time you have to constantly doubt them. Weird, eh?

Confidence

You need some level of confidence to do anything efficiently. Planning ahead requires confidence that you can achieve the steps on your way.

As a programmer you also need some confidence with the language, libraries and other tools you're using.

If you program for money, you also have to assess what kind of programs you can write, and where you might have problems.

Doubt

In the process of programming you make a lot of assumptions, some of the explicit, some of them implicit. If you want to write a good program, it's essential that you are aware of as many assumptions as possible.

When you find a bug in your program, you have to challenge previous assumptions, and that's where doubt comes in. You not only suspect, but you know that at least one of the assumptions was false (or maybe just a bit too specific), and you know that you did something wrong.

Sometimes programmers make really stupid mistakes which are rather tricky to track down. That's when you have to question your own sanity.

One example (that luckily doesn't happen all that often to me) is when I edit my program, and nothing seems to change. Nothing at all. Depending on the setup it might be some cache, but something it is even more devious - for example I didn't notice that the console where I edit and the console where I test are on different hosts - and thus the edits actually have no effect at all.

After having done such a thing once or twice I adopted the habit of just adding a die('BOOM'); instruction to my code, to verify that the part I'm looking at is actually run.

These are moments when I question my own sanity, thinking "how could I have possibly done such a stupid thing?". Doubt.

The same phenomena applies when doing scientific research: since you usually do things that nobody has done before (or at nobody has published about it yet), you can't know the results beforehand -- if you could, your research would be rather boring. So you have no external reference for verification, only your intuition and discussion with peers.

Posted by Moritz Lenz | Permalink

The typical life cycle of a bug filed against Rakudo Perl 6 looks like this:

• Somebody tries out a new feature, finds a bug, and submits it to rakudobug@perl.org. (Typically that's masak)
• Somebody writes a failing test for that, and puts it into the test suite. (Typically that's kyle or me)
• Somebody fixes the bug in Rakudo. (Typically that's pmichaud or jnthn).
• The one who fixed the bug usually closes the ticket.

The second and third step are occasionally reversed - in that case either the implementor writes a test himself, or he assigns the ticket to me, asking for tests. I just act as a placeholder here for the "needs a test" stage. In any case a bug that can be tested with reasonable effort won't be closed unless there's a regression test present.

The tests are usually disabled, because often they cause Rakudo to die until it behaves at least mostly correct.

But sometimes it's a bit different: there is a subsystem that needs substantial refactoring or a rewrite, and lots of bug tickets queue up for that subsystem. Somebody invests much time and energy into that subsystem, starts a branch, develops the rewrite there, and finally merges the changes.

That happend this week when Jonathan implemented a new signature binder, which fixed both many problems with passing arguments to routines, and also enabled classes to see outer lexical variables.

We have a script called autounfudge which goes through all the test files, enables the disabled tests one by one, run them again, and if they pass, it writes a patch that enables them.

That's not only useful for enabling the tests, but also for finding the tickets which can be closed. A typical auto-generated patch looks like this:

--- t/spec/S02-builtin_data_types/anon_block.t	Wed Oct  7 13:54:31 2009
+++ RAKUDO815xU3/temps1BKu.t	Tue Oct 20 10:26:03 2009
@@ -44,7 +44,6 @@
 
         eval '$anon_block( foo => "RT #64844" )';
         ok $! ~~ Exception, 'too many parameters';
-        #?rakudo skip 'RT #64844'
         is ~$!, $errmsg, 'same error for named param as positional';
     }
 }

So after a big branch merge somebody will run the autounfudge script, and that spits out a list of closable RT tickets. In the case of the signature binder that were about 15 tickets, which would have taken ages to find without help in the 500+ open tickets.

All in all I'm rather happy with this infrastructure.

Posted by Moritz Lenz | Permalink

A few days ago a relative new contributor to the Perl?6 test suite apologized for the perceived bad quality of his commits to the test suite. (There was no reason, the commits were actually quite good). It made me think about the quality of what I contribute there.

It's not easy to assess the quality of tests. We want to keep the test suite as simple as possible, in the sense that every test should test one specific piece of Perl?6, and use simplest syntax otherwise. That means we have trade offs regarding code duplication: Not using fancy data driven testing often results in multiple test that look similar - something you'd avoid in normal programs, and even in normal test suites.

So extensive tests are bound to look a bit ugly, but that's OK. But it also means that our usual standards of assessing code quality don't necessarily hold. Leaving that aside, many parts of the test suite still look a bit weird or suspicious. I remembered Sturgeon's Law: 90% of everything is crud. Assuming it holds for the test suite too...

... and considering the fact that I contributed about 20% of the commits to the test suite (number from 2009-10-10), at least half of what I contributed was crud too.

I'm not ashamed of it - as long as I strive for good code and do my best, everything is fine. And I encourage those who think their tests are crud to commit them anyway - or at least ask for review. Those who are concerned about their code quality usually produce quite decent code.

See also: Matt S Trout - You are not good enough.

Posted by Moritz Lenz | Permalink

This is a direct reply to a blog post by VM Brasseur, telling about the difficulties to get started in the Perl 6 community. I started it as a comment on that blog, but it got too long pretty quickly.

The problem is known: far too many Perl 6 sites, far too many of which are out of date. I hoped to kill fire with fire by starting one more (which is perl6.org), and trying to make it easy and attractive to keep it up to date - by making it central, easy to contribute to, and by making it light on content and rich on links.

However being a "seasoned warrior" in the community, it is sometimes hard for me to assess what we have to present to the newcomer, and in what way.

I also like to keep in mind that Perl 6 has multiple implementations, and even a newcomer should have the choice. Being involved with some of the implementations I also feel it's unfair against the others to just point to one of them.

So, what would a newbie need? I'm guessing here, please do tell if you have different opinions.

• Introductory documents/tutorials
• A way to run a Perl 6 compiler to try it out
• Ways to contact the community
• A list of what needs to be done
• Further reading

The first one is maybe the toughest: frankly speaking, we don't have any. We have my 5-to-6 blog which I put up here in chronological form, but which assumes Perl 5 knowledge, and which talks about many features that Rakudo does not implement yet, and which mostly isn't practical. We have examples, but the rest is unorganized, out of date, or woefully incomplete. D'oh.

On the second point we're doing fairly well with a big download button - at least I hope we do.

We also advertise our IRC channel (which I consider the heart beat of Perl 6 development) and mailing lists, another score here.

The fourth point is a tough one again. Being a completely volunteer driven project, we are not used to handing out tasks. There is a huge variety of tasks that all have vastly different requirements in terms of interests, skills, time and commitment. The larger such a list becomes, the harder it becomes to maintain. And TODO lists are one of those things that go out of date rather quickly. Which is why I'd encourage newcomers to contact the community, tell about their interest and skills so that we can search for suitable tasks together.

However that's quite a hurdle for some people. I usually suggest to write tests and applications, which sometimes finds resonance. But any ideas how to perform better is greatly appreciated.

The last point, further reading, is mostly covered by the Synopsis/specification.

But all in all I think we need a better presentation, though I don't yet know how. Ideas?

Also somebody mentioned that our IRC channel is friendly, but still not very accessible to the newcomer, because we use lots of technical terms when talking among ourselves. That's something we can't really avoid, but it does scare off some people. So far we have tried to avoid splitting into many subchannels (with the possible exception of per-project dedicated channels), but maybe the time has come to fork off a newbie or users channel?

Anything else we can do right now to improve newcomer experience? And yes, we're working on improving introductory material. More on that next Friday.

Posted by Moritz Lenz | Permalink

When you learn a programming language, you not only learn about the syntax, semantics and core libraries, but also the coding style and common idioms.

Idioms are common usage patterns; learning and reusing them means you have to spend less time thinking on common things, and have more time working out the algorithms you deal with.

That's different if you learn Perl?6 - it's a largely unexplored field, and while there are loads of nice features, you might still feel a bit lost. At least I do. That's because I often think "There's got to be a much easier way to achieve $this, but it often takes time to find that easier solution - because nobody developed an idiom for it.

In those cases it helps to ask on the #perl6 IRC channel; many smart people read and write there, and are rather good in coming up with simpler solutions.

For example see masak's ROT13 implementation in Perl?6. In the right column you can see later revisions, and how they gradually improve, steady up to a one-liner.

I also made some simplifications to JSON::Tiny, which basically shows that when I wrote these reduction methods first I used Perl?6 baby talk language.

The nice things about exploring the Perl?6 wonderland of unexplored idioms is that it really pushes your ego if you find a nice simplification, and that you have something to blog about for the Planet Perl Iron man ;-)

Posted by Moritz Lenz | Permalink

I got irritated at how hard it was to use Wikipedia on my Treo. There's so much rubbish splattered around their pages that it Just Doesn't Work on such a small screen. Given that no alternatives seemed to be available - at least, Google couldn't find any - I decided to write my own Wikipedia handheld proxy.

It strips away all the useless rubbish that normally surrounds Wikipedia pages, as well as things like the editing functions which are also hard to use on portable devices. Internally, it's implemented using perl, LWP, and mod_perl, and is hosted by Keyweb.de.

Posted by david | Permalink