|
このページは大阪弁化フィルタによって翻訳生成されたんですわ。 |
Planet Perl is an aggregation of Perl blogs from around the world. Its an often interesting, occasionally amusing and usually Perl related view of a small part of the Perl community. Posts are filtered on perl related keywords. The list of contributors changes periodically. You may also enjoy Planet Parrot or Planet Perl Six for more focus on their respective topics.
Planet Perl provides its aggregated feeds in Atom, RSS 2.0, and RSS 1.0, and its blogroll in FOAF and OPML
There is life on other planets. A heck of a lot, considering the community of Planet sites forming. It's the Big Bang all over again!
This site is powered by Python (via planetplanet) and maintained by Robert and Ask.
Planet Perl is licensed under
a Creative
Commons Attribution-Noncommercial-Share Alike 3.0 United States
License. Individual blog posts and source feeds are the
property of their respsective authors, and licensed
indepdently.
I've posted before about the problems with "same terms as Perl" licensing, not the least of which is the question "which version of Perl are you referring to?" I bet that most people who use "same terms as Perl itself" used it for the same reason I did: because it was the easiest way to do it, and I was lazy.
So to help out my lazy sisters and brothers in Perl, here is a block of POD that you can go paste into your modules right now, all formatted and pretty.
=head1 COPYRIGHT & LICENSE Copyright 2005-2009 Andy Lester. This program is free software; you can redistribute it and/or modify it under the terms of either: =over 4 =item * the GNU General Public License as published by the Free Software Foundation; either version 1, or (at your option) any later version, or =item * the Artistic License version 2.0. =back
You'll want to change the "Andy Lester" part. I don't want to take credit for half the CPAN.
1.40 Fri 31 Jul 2009
- Dear Ruby Community.
Thank you so much for your bug report to the "Ruby Talk" mailing list
regarding the lack of true/false support in Perl YAML modules.
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/169943
This flaw has now been resolved, around 2 hours after we
were made aware of it. Next time, we recommend you report these bugs
to the people who can actually fix your problem, and they should be
resolved in hours instead of 4 years.
git rebase is subject to quite a few controversial opinions, but in all my experience working with Git it seems that most people who use Git, even those who use rebase, myself included, often don't really know exactly what it does. Apparently, many people are afraid of using it because it does evil magic like changing the past and eating babies. The truth is that git rebase actually does so much more!
The critics of rebasing generally contrast it to Subversion, in which history is never modified after being recorded. However, much like the baby eating, this is just delusional superstition when you understand what Git actually does (and what subversion doesn't do).
Personally, I think git rebase should be used all the time; that is, git pull --rebase should be used instead of plain git pull (this can actually be the default).
You probably shouldn't rebase published heads, that can confuse people, but that's not a reason not to use rebase correctly.. I don't like the anti rebasing dogma:
In my experience rebase always produced the most desirable results, creating a clean history instead of one riddled with meaningless nonsense. rebase is smart.
Explaining exactly what rebase does is a job for the documentation, but what I feel the docs are lacking on is explaining what rebase is actually for.
Forward-port local commits to the updated upstream head
Actually, I take that back. This is an excellent way of explaining what rebase is for. Assuming, of course, that you already know what that means ;-)
Broken down, we have a few fnords to sort out:
I don't really want to explain what these mean, just to clearly define them.
Git is distributed. Even if you work with a single central Git repository for your project, that's just a special, simplified case of a distributed workflow.
When you clone a repository you get your own local copies of the remote heads (branches).
When you commit your work only your own local copies of these heads are modified.
git push will overwrite the references on the remote side with the local ones, but by default only if the operation is safe (Git calls this "fast forward"), when the local revision is derived from the remote revision (the commit in the remote version is an ancestor of the local revision).
If this is confusing then you should read up on how to work with remote repositories in Git first.
Between the point in time when you started working on an up to date local copy of a head and the the time you want to push your changes, other people may have modified the remote version.
When this happens the development has diverged, there are two chains of commits in history, one is the updated head, and one is the local head.
The local commits is the chain of commits leading from git merge-base updated-head local-head to the local head.
These are commit objects that are not visible by walking the history of the upstream head (updated or not), but only by walking the history of the local head.
This is where the awesome of rebase comes into play.
You can run git merge updated-head in your local version to create a new merge commit, that will combine the changes two diverged histories leaving you with the results of both in both the commit log and the source tree.
This is what git pull does by default; it's the same as git fetch followed by git merge.
git rebase is much more clever.
The results of git merge and git rebase are the same in terms of the resulting trees: the files will end up containing the same merged changes, and you will similarly need to run git mergetool if there are any conflicts.
The difference is that git rebase will take your local commits and apply them one by one to the revision from the updated upstream head, effectively creating a brand new local head with new versions of the local commits, whereas git merge creates a single merge commit that is derived from both the local commits and the upstream ones.
A while ago I was unaware that rebase is actually very smart about how, and more importantly whether or not to apply every local commit. rebase supports patch idempotence, local changes which have identical changes upstream (even if the commit metadata is different) are simply skipped without error. This means that changes that were merged upstream, even if they were cherry picked, signed off, etc, will still be dealt with correctly.
Similarly, merge commits that are no longer meaningful will be omitted. If you run git pull followed by git pull --rebase the merge commit created by the first pull will be omitted from the results.
The new set of forward ported local commits is clean and minimal, and therefore easier to work with.
If you always use git pull --rebase your local changes will never get out of control, resulting in a mess of branches and merges. They will simply be the most minimal set of changes needed to bring the upstream head into the changed version that you are trying to create in your development.
Correctly using rebase to create a clean local history is simple using git pull --rebase.
Furthermore, (and therefore probably unpublished) you can even modify them in other ways, as you keep working on them (for a more managed approach see stacked git). This sort of history (and change) rewriting is fair game. This is what Linus means when he says Keep your own history readable and Don't expose your crap.
When you run svn commit but you aren't up to date, you need to run svn update to forward port local changes to the updated upstream head. Does that sound familiar?
The difference is that in SVN you're rebasing history that hasn't been saved anywhere yet. With git you can at least back out of it (git reset, git rebase --abort) if you run into hurdles like conflicts. Subversion is merciless, it will simply modify your working copy with conflict markers, and you have no choice but to clean up the mess.
So in a sense Subversion does just as much "evil history rewriting" as Git does, except that in Git every step of the process is recorded as explicit versioned data, that you can compare with and revert to freely, whereas in Subversion it's done as one monolithic operation to your uncomitted working changes.
Git is not omniscient.
There are a number of things that won't be automatically cleaned up by git pull --rebase (and rightly so):
Applying such changes to a published branch is not just reorganization of history. The resulting commit history will have different changes when the deltas are actually computed.
Trying to rebase on top of something that had these operations applied to it will cause conflicts to the person running rebase, leading to confusion, which brings us back to the reason people are so afraid of rebase.
As long as you're aware of the implications of such operations, and make sure never to publish them where people won't expect them to happen, you can use them as much as you like.
[1] as long as there are no conflicts or other things that actually modify the deltas in a way that rebase can't automatically pick up
Why don't I like YAML? Well, have you read the spec? It's so awful that most YAML parsers disagreed about what YAML was. If I recall correctly, it was Why's libsyck which effectively set the default YAML standard. That doesn't mean that having a standard is a good thing, though (Prolog has an ISO standard which is universally ignored because it breaks Prolog). For example, do you know what this is?
---
file: Yes
Let's see what the three most popular Perl modules do with this.
#!/usr/bin/env perl
use strict;
use warnings;
use YAML;
use YAML::Syck;
use YAML::Tiny;
use Data::Dumper::Names;
my $yaml_string = <<END;
---
file1: Yes
file2: off
END
my $yaml = YAML::Load($yaml_string);
my $syck = YAML::Syck::Load($yaml_string);
my $tiny = YAML::Tiny->read_string($yaml_string);
print Dumper $yaml, $syck, $tiny;
And they all output:
$yaml = {
'file2' => 'off',
'file1' => 'Yes'
};
$syck = {
'file2' => 'off',
'file1' => 'Yes'
};
$tiny = bless( [
{
'file2' => 'off',
'file1' => 'Yes'
}
], 'YAML::Tiny' );
It's a shame they do that because those are not strings, they're boolean values. In short, the most popular YAML parser all ignore the spec. I won't fault YAML::Tiny because it's supposed to be a subset of YAML. I also won't fault YAML::Syck because it's a wrapper around libsyck. YAML.pm, on the other hand ...
So big deal. Who cares if we're violating the spec? This Ruby programmer does. The parser he's using follows the spec, but the Perl and Python generators don't properly quote the boolean. And why should they have to build in a special case for yet another string? And if you read the spec for the booleans in YAML, it's almost case sensitive, but not quite. "False", "FALSE" and "false" are all false, but "FalSe" is a string, which ironically would be interpreted as true in Perl.
Just try and read the spec. You probably won't finish it. And do you want to see a grammar for YAML? Apparently there's one hidden in the spec and you can install Schwern's Greasemonkey script to see it.
Oh, and if you really want to have fun, trying playing around with anchors and see if you can make 'em recursive and see how various parsers fail to handle it. I don't want a mega-spec. I don't want something which has all sorts of special meanings which different implementations fail on that I have to keep track of. I don't want the One True Way which would be able to serialize everything if only there were parsers to handle it.
JSON. One Page. Done. Any questions?
I’m baffled at what the PerlMonks developers and admins were thinking storing their passwords in plain-text.... This is something that not only has been in Perl since version 1.0, but has also been integrated natively in almost every database environment on the planet.
I’m ashamed as a Perl developer, and this gives a huge black eye to the entire Perl community.
After this, I’m seriously considering switching to another language for my next project.
Jesse Stay, There’s More Than One Way to Store a Password - PerlMonks Hacked
Storing passwords in plain text is wrong, full stop, but if I'd had a server rooted, I'd worry about more than just bad guys reading database dumps. A little phishing code would be easy to insert into a lot of web sites.
Then again, I also worry about DNS hijacking.
If I had a gripping hand, I'd tell you that I once saw a neighbor store a spare house key under the mailbox, and that's why I'm seriously considering switching to a condo for my next dwelling. Guy gave all homeowners everywhere a black eye, even though pockets have been in clothing since almost Textiles version 1.0, not to mention integrated natively in almost every pair of jeans on the planet.
Kids are born knowing Perl but with the education system most of them lose their ability by the time they grow up. - szabgab@cpan.org
The Perl 6 design team met by phone on 03 June 2009. Larry, Allison, Patrick, and chromatic attended.
Larry:
Patrick:
Larry:
need and define, what use usesneed just pulls in modulesPatrick:
Larry:
$! use v6 now allows a version numberPatrick:
Allison:
Patrick:
postfix:++ in Rakudo being incredibly slowLarry:
Patrick:
isa checks are incredibly expensiveisa opcode always creates a new ProxyPMC on every callAllison:
Patrick:
new and other thingsc:
Patrick:
Perl and 5 Setting Setting $.method() through the accessor interface now works correctlyBEGIN ro versus rw Allison:
c:
Allison:
c:
Patrick:
Allison:
Patrick:
Allison:
Patrick:
Allison:
Patrick:
Allison:
Patrick:
Allison:
Patrick:
Allison:
Patrick:
Allison:
Patrick:
zip @a[1,2,3] versus @a[1;2;3] Larry:
Patrick:
Larry:
Patrick:
rand() now a function call?Larry:
rand is a term parsed like pi or e Patrick:
Larry:
Patrick:
:2 and didn't match, what'd I get back?Larry:
Patrick:
Larry:
Patrick:
Larry:
Patrick:
Larry:
Patrick:
123 Easy Street?Larry:
Patrick:
Larry:
Patrick:
Larry:
try Patrick:
Larry:
Patrick:
expr // 0 try is a little longLarry:
\ is availablec:
Larry:
Patrick:
Allison:
I’ve seen a bunch of the same questions coming up again and again in comments on my OSCON keynote, both on my own blog and on other sites, so I thought I’d take the time to answer some of them in a central place.
1.5%? 20%? Really? Where did you get those numbers from?
1.5% women in open source: FLOSSPOLS survey, 2006, funded by the EU. 1541 open source developers were surveyed, from a range of projects both large and small. The surveyed developers were taken from those who had answered the earlier FLOSS survey and had provided their email addresses for subsequent contact; the 2002 FLOSS survey was a widespread survey of open source developers, who were recruited by asking people to pass information about the survey out to open source development mailing lists and the like. The FLOSS survey published a report on their methodology and validation of results which describes possible selection biases, etc.
5% women in the Perl community (users and developers): my own survey of approx 5000 Perl users in 2007. The specific figures were 95% male, 4% female, 1% other. In putting together my slides, I was working from memory and conflated the 4% and 1% into 5% total, who I represented as female. This was incorrect, and I will fix it if/when I give my presentation again. The results of the Perl Survey are available as a summary PDF (Letter, A4) or as a raw data dump.
10% women in Drupal community: DrupalChix website. Addison Berry, a Drupal contributor, told me at OSCON that the number is based on attendance at Drupal events. I have not had this confirmed as yet.
20% women in technology industry: figures vary between about 15% and 30% when looking at overall proprotions of women in IT roles from country to country. For example, some of the reports I’ve read say things like 24.5% of non-administrative ICT roles in the US (source – PDF), 19% of ICT roles in Australia (source), and 20% of the IT workforce in France (source – PDF). These surveys are usually done by either industry bodies such as the Information Technology Association of America or government bodies like the Australian Department of Workplace Relations. Usually they include all IT roles, including corporate IT and the like. When surveying for eg. women in technical management, the numbers are generally lower. I’ve been collecting links to such reports here.
Dreamwidth and OTW are about journalling and fanfic. Of course they have lots of women! But women don’t join other open source projects because they don’t use/care about that kind of software.
Here are some examples of widely-used open source software:
Each of these projects is in an area that either has a majority of women participating (blogging, IM, graphic design), or an equal or nearly equal number as men (desktop computing, education, web browsing), at least in western countries. And yet they do not have proportional representation of women in their development teams.
It is not (solely) the female-friendliness of the underlying application that leads to women joining an open source project. If it were, the above projects would have a high level of female participation.
Nor are there many applications so inherently female-unfriendly as to automatically explain a 98.5% male development team; the Linux kernel, which I’ve heard cited as an example, is no such thing, since the proportion of Linux users — including desktop and handheld devices such as the Kindle, Android phone, etc — is no doubt higher than 1.5%.
Isn’t 75% or 100% women on a project just reverse sexism? How come that’s OK when 100% men isn’t?
100% of men in a single project would be just fine if it were an outlier, and if that were an accurate representation of the user base for that software and the necessary developer skillset. But that’s not what’s going on. The problem we’re facing here is one of widespread inequality: a pervasive situation affecting thousands of projects, regardless of the skills and interests of the wider community.
A healthy open source community would show a bell curve of female participation rates, with the hump somewhere around the percentage of women who use that kind of software and/or the percentage of women who have (or can acquire) the appropriate skills — let’s say anywhere between 20% and 50% for most software projects. (That 20% is based on the number of women in the tech industry with the right kind of skills, and the 50% represents the number of female users for applications like blogging, web browsing, graphic design, or teaching.)
As with any bell curve, there will be outliers. Fan fiction, admittedly, is a heavily female activity so a higher proportion of female participants is not surprising. On the other hand, a man asked me at OSCON what to do about his all-male model railway automation project, and that’s one I think is just fine with 100% male developers given the pool of model rail enthusiasts he’s drawing from.
(Of course, we can also work in parallel to address imbalances in user communities or in the pool of people who have appropriate skills, but that’s not where I’m focussing right now. If you’re interested in that, you might like to look into organisations working on encouraging girls in STEM (science, technology, engineering, maths) education or teaching computer skills to adult women. For all I know there are also groups working to encourage women in the model railway community.)
No really, not just because I've not posted anything in.. a while. I got bored reading mst's mbox of ironman stuff, converting blog urls into atom/rss feeds, and typing the results into Plaggers yaml configuration file.
So I replaced all those manual entries with:
- module: Subscription::DBI
config:
schema_class: 'IronMan::Schema'
connect_info: ['dbi:SQLite:/home/castaway/plagger/subscriptions.db']
And wrote one of these to convert the existing junk:
#!/usr/bin/perl
use strict;
use warnings;
use YAML 'LoadFile';
use Data::Dumper;
use Data::UUID;
use IronMan::Schema;
my $yamlfile = shift;
my $dsn = shift;
my $yaml_dump = LoadFile($yamlfile);
my $schema = IronMan::Schema->connect($dsn);
my ($sub_conf) = grep { $_->{module} eq 'Subscription::Config' }
@{ $yaml_dump->{plugins} };
print Dumper($sub_conf);
my $feeds = $sub_conf->{config}{feed};
my $uuid = Data::UUID->new();
foreach my $feed (@$feeds) {
print "Feed: $feed->{url}\n";
my $fdb = $schema->resultset('Feed')->find_or_new({ id => $uuid->create_str, %$feed}, { key => 'url' });
if($fdb->in_storage) {
print "... already exists\n";
next;
}
$fdb->insert;
}
The twiddly part was setting up an app (form) for people to sign up, which now exists in the ironman repo, and runs on my local box, at the moment.
Thoughts of things to add:
Someone also needs to make the thing prettier!
Btw to install, symlink the css and image dirs from "plagger/assets/plugins/Publish-PagedPlanet/default/static" to the catalyst "root" dir.
@public,perl,ironman
#!/usr/bin/env perl
use Mojolicious::Lite;
get '/' => 'index';
get '/:groovy' => sub {
my $self = shift;
$self->render(text => $self->stash('groovy'));
};
shagadelic;
__DATA__
@@ index.html.eplite
% my $self = shift;
% $self->stash(layout => 'funky');
Yea baby!
@@ layouts/funky.html.eplite
% my $self = shift;
<!html>
<head><title>Funky!</title></head>
<body>
<%= $self->render_inner %>
</body>
</html>
You will have full access to the Mojolicious script system automatically, so this small app can already run on every backend Mojo supports. (including CGI, FastCGI and the built in HTTP 1.1 servers)
% ./myapp.pl daemon Server available at http://127.0.0.1:3000.If you've been paying attention to web framework developments recently, you will have noticed much similarity to sinatra.rb, a Ruby framework that got a lot of buzz for it's simplicity. So after Viacheslav Tikhanovskii proofed that it was possible to build Mojolicious::Lite on top of Mojolicious in about 40 lines of code, it was really a no brainer to make it a part of the core. For me the best of all is that we can now provide newcomers with a subset of real Mojolicious features, so they can start out very quickly and have their apps later grow to full Mojolicious applications with minimal effort.
#!/usr/bin/env perl
use Mojolicious::Lite;
get '/' => sub {
my $self = shift;
$self->render(text => 'Yea baby!');
};
shagadelic;
There is also a helper script to generate a small example application.
% mojolicious generate lite_appAll the normal Mojolicious script options are available from the command line.
% ./myapp.pl daemon Server available at http://127.0.0.1:3000. % ./myapp.pl daemon --port 8080 Server available at http://127.0.0.1:8080. % ./myapp.pl daemon_prefork Server available at http://127.0.0.1:3000. % ./myapp.pl cgi ...CGI output... % ./myapp.pl fastcgi ...Blocking FastCGI main loop...The shagadelic call that starts the Mojolicious script system can be customized to override normal "@ARGV" use.
shagadelic('cgi');
Routes are basically just fancy paths that can contain different kinds of
placeholders.
# /foo
get '/foo' => sub {
my $self = shift;
$self->render(text => 'Yea baby!');
};
All routes can have a name associated with them, this allows automatic
template detection and back referencing with "url_for".
# /
get '/' => 'index';
# /foo
get '/foo' => 'foo';
__DATA__
@@ index.html.eplite
<a href="<%= shift->url_for('foo') %>">Foo</a>.
@@ foo.html.eplite
<a href="<%= shift->url_for('index') %>">Home</a>.
Templates can have layouts.
# GET /with_layout
get '/with_layout' => sub {
my $self = shift;
$self->render(template => 'with_layout', layout => 'green');
};
__DATA__
@@ with_layout.html.eplite
We've got content!
@@ layouts/green.html.eplite
<!html>
<head><title>Green!</title></head>
<body><%= shift->render_inner %></body>
</html>
Route placeholders allow capturing parts of a request path until a "/" or
"." separator occurs, results will be stored by name in the "stash".
# /foo/*
get '/foo/:bar' => sub {
my $self = shift;
my $bar = $self->stash('bar');
$self->render(text => "Our :bar placeholder matched $bar");
};
# /*something/foo
get '/(:bar)something/foo' => sub {
my $self = shift;
my $bar = $self->stash('bar');
$self->render(text => "Our :bar placeholder matched $bar");
};
Relaxed placeholders allow matching of everything until a "/" occurs.
# GET /hello/*
get '/hello/(.you)' => sub {
shift->render(template => 'groovy');
};
__DATA__
@@ groovy.html.eplite
Your name is <%= shift->stash('you') %>.
Wildcard placeholders allow matching absolutely everything, including
"/" and ".".
# /hello/*
get '/hello/(*you)' => sub {
shift->render(template => 'groovy');
};
__DATA__
@@ groovy.html.eplite
Your name is <%= shift->stash('you') %>.
Routes can be restricted to specific request methods.
# GET /bye
get '/bye' => sub { shift->render(text => 'Bye!') };
# POST /bye
post '/bye' => sub { shift->render(text => 'Bye!') };
# GET|POST|DELETE /bye
any [qw/get post delete/] => '/bye' => sub {
shift->render(text => 'Bye!');
};
# /baz
any '/baz' => sub {
my $self = shift;
my $method = $self->req->method;
$self->render(text => "You called /baz with $method");
};
All placeholders get compiled to a regex internally, with regex constraints
this process can be easily customized.
# /*
any '/:bar' => [bar => qr/\d+/] => sub {
my $self = shift;
my $bar = $self->stash('bar');
$self->render(text => "Our :bar placeholder matched $bar");
};
Routes allow default values to make placeholders optional.
# /hello/*
get '/hello/:name' => {name => 'Sebastian'} => sub {
my $self = shift;
$self->render(template => 'groovy', format => 'txt');
};
__DATA__
@@ groovy.txt.eplite
% my $self = shift;
My name is <%= $self->stash('name') %>.
All those features can be easily used together.
# /everything/*?name=*
get '/everything/:stuff' => [stuff => qr/\d+/] => {stuff => 23} => sub {
shift->render(template => 'welcome');
};
__DATA__
@@ welcome.html.eplite
% my $self = shift;
Stuff is <%= $self->stash('stuff') %>.
Query param name is <%= $self->req->param('name') %>.
Here's a fully functional example for a html form handling application using
multiple features at once.
#!/usr/bin/env perl
use Mojolicious::Lite;
get '/' => 'index';
post '/form' => 'form' => sub {
my $self = shift;
my $groovy = $self->req->param('groovy') || 'Austin Powers';
$groovy =~ s/[^\w\s]+//g;
$self->render(
template => 'welcome',
layout => 'funky',
groovy => $groovy
);
};
shagadelic;
__DATA__
@@ index.html.eplite
% my $self = shift;
% $self->stash(layout => 'funky');
Who is groovy?
<form action="<%= $self->url_for('form') %>" method="POST">
<input type="text" name="groovy" />
<input type="submit" value="Woosh!">
</form>
@@ welcome.html.eplite
% my $self = shift;
<%= $self->stash('groovy') %> is groovy!
<%= $self->render_partial(template => 'menu') %>
@@ menu.html.eplite
<a href="<%= shift->url_for('index') %>">Try again</a>
@@ layouts/funky.html.eplite
% my $self = shift;
<!html>
<head><title>Funky!</title></head>
<body>
<%= $self->render_inner %>
</body>
</html>
Formats can be automatically detected by looking at file extensions.
# /detection.html
# /detection.txt
get '/detection' => sub {
my $self = shift;
$self->render(template => 'detected');
};
__DATA__
@@ detected.html.eplite
<!html>
<head><title>Detected!</title></head>
<body>HTML was detected.</body>
</html>
@@ detected.txt.eplite
TXT was detected.
External templates will be searched by the renderer in a "templates"
directory.
# /external
any '/external' => sub {
my $self = shift;
$self->render(template => 'foo/bar.html.epl');
};
Static files will be automatically served from the "public" directory if it
exists.
% mkdir public % mv something.js public/something.jsTo disable debug messages later in a production setup you can change the Mojolicious mode, default will be "development".
% MOJO_MODE=production ./myapp.plLog messages will be automatically written to a "log/$mode.log" file if a "log" directory exists.
% mkdir logFor more control the Mojolicious instance can be accessed directly.
app->log->level('error');
app->routes->route('/foo/:bar')->via('get')->to(callback => sub {
my $self = shift;
$self->render(text => 'Hello Mojo!');
});
In case a lite app needs to grow, lite and real Mojolicous applications
can be easily mixed to make the transition process very smooth.
package MyApp::Foo;
use base 'Mojolicious::Controller';
sub index {
shift->render(text => 'It works!');
}
package main;
use Mojolicious::Lite;
get '/bar' => sub { shift->render(text => 'This too!') };
app->routes->namespace('MyApp');
app->routes->route('/foo/:action')->via('get')
->to(controller => 'foo', action => index);
shagadelic;% mojolicious usage: /usr/local/bin/mojolicious SCRIPT [OPTIONS] These scripts are currently available: generate Generate files and directories from templates. cgi Start application with CGI backend. daemon Start application with HTTP 1.1 backend. daemon_prefork Start application with preforking HTTP 1.1 backend. fastcgi Start application with FastCGI backend. test Run unit tests. See '/usr/local/bin/mojolicious help SCRIPT' for more information on a specific script.The list of available scripts is dynamically generated and will change when more get installed.
% mojolicious generate usage: /usr/local/bin/mojolicious generate GENERATOR [OPTIONS] These generators are currently available: app Generate application directory structure. lite_app Generate a minimalistic web application. makefile Generate Makefile.PL. See '/usr/local/bin/mojolicious generate help GENERATOR' for more information on a specific generator.It just works for Mojolicious::Lite applications.
% mojolicious generate lite_app
[exist] /Users/sri
[write] /Users/sri/myapp.pl
[chmod] myapp.pl 744
% ./myapp.pl help daemon_prefork
usage: ./myapp.pl daemon_prefork [OPTIONS]
These options are available:
--address <address> Set local host bind address.
--clients <number> Set maximum number of concurrent clients per child,
defaults to 1.
--daemonize Daemonize parent.
--group <name> Set group name for children.
--idle <seconds> Set time children can be idle without getting
killed, defaults to 30.
--interval <seconds> Set interval for process maintainance, defaults to
15.
--keepalive <seconds> Set keep-alive timeout, defaults to 15.
--maxspare <number> Set maximum amount of idle children, defaults to
10.
--minspare <number> Set minimum amount of idle children, defaults to 5.
--pid <path> Set path to pid file, defaults to a random
temporary file.
--port <port> Set port to start listening on, defaults to 3000.
--queue <size> Set listen queue size, defaults to SOMAXCONN.
--requests <number> Set maximum number of requests per keep-alive
connection, defaults to 100.
--servers <number> Set maximum number of children, defaults to 100.
--start <number> Set number of children to spawn at startup,
defaults to 5.
--user <name> Set user name for children.
Just like in most other web frameworks there are countless polished "big features" in Mojo, but what really sets us apart is how much we care about the small ones. ;) The experimental Top 100 website has been working well for the last month, tracking modules and failures in them.
I have, however, had to exclude the CPAN Testers reports of DCOLLINS. In his rush to WTFPWN the top contributors list on CPAN Testers, a number of fairly nasty runs of false FAIL and UNKNOWNs were generated, enough that it was distorting the league table.
However, whatever these were can be fixed, or someone can create a mechanism for tracking which report ids are considered bad, so these can be removed from my stats after the fact.
But now, onwards.
The thing that's been bothering me is that these numbers just feel wrong, or at least naive. It's one thing to say that something depends on 100 modules, but when 30 of these are the twisted internal dependencies of the toolchain, it doesn't necessarily convey the amount of meaning that it should.
So I've been working on a second-generation graph for CPANDB 0.05, and it has just been uploaded for your consuming pleasure.
Firstly, the new graph merges in a LOT more metadata about distribution, which should allow me to make new and interesting Top 100 lists.
CREATE TABLE distribution (
distribution TEXT NOT NULL PRIMARY KEY,
version TEXT NULL,
author TEXT NOT NULL,
meta INTEGER NOT NULL,
license TEXT NULL,
release TEXT NOT NULL,
uploaded TEXT NOT NULL,
pass INTEGER NOT NULL,
fail INTEGER NOT NULL,
unknown INTEGER NOT NULL,
na INTEGER NOT NULL,
rating TEXT NULL,
ratings INTEGER NOT NULL,
weight INTEGER NOT NULL,
volatility INTEGER NOT NULL,
FOREIGN KEY ( author ) REFERENCES author ( author )
)
You can see a "meta" column, which is a true/false flag for whether or not the distribution has a META.yml file, and a license key for the declared license, which matches the value used to create the "License: " distribution header on search.cpan.org.
The new "rating" and "ratings" columns represent a merge of the CPAN Ratings data. Both the average score, and the number of ratings.
What is more interesting, however, are the dependency tables.
CREATE TABLE requires (
distribution TEXT NOT NULL,
module TEXT NOT NULL,
version TEXT NULL,
phase TEXT NOT NULL,
PRIMARY KEY ( distribution, module, phase ),
FOREIGN KEY ( distribution ) REFERENCES distribution ( distribution ),
FOREIGN KEY ( module ) REFERENCES module ( module )
)
The requires table now tracks, correctly, both the phase ( "runtime", "build", or "configure" ) of the dependency, and the module version that is declared.
This allows the creation of a more nuanced distribution dependency graph edge table.
CREATE TABLE dependency (
distribution TEXT NOT NULL,
dependency TEXT NOT NULL,
phase TEXT NOT NULL,
core REAL NULL,
PRIMARY KEY ( distribution, dependency, phase ),
FOREIGN KEY ( distribution ) REFERENCES distribition ( distribution ),
FOREIGN KEY ( dependency ) REFERENCES distribution ( distribution )
)
In the new dependency table, the module-level requires values are mapped against Module::CoreList to identify which (if any) distribution dependencies are actually just dependencies on the core, and if so what Perl core they were available in.
More on that in a bit.
Finally, I've added direct integration for all three major graphing libraries. Graph.pm is useful for mathy type things, Graphiz.pm is useful for generation visualisation, and Graph::Easy is there... well because I could. Graph::Easy is unfortunately too slow to render anything useful for most graphs that I work with.
Lets take a look at the Graphviz.pm support in action, with the famously "Dependency Free" Mojo! :)
Lets start with a raw dependency graph.
CPANDB->distribution('Mojo')->dependency_graphviz->as_svg('Mojo.svg');
That code fragment will get you something like this.
http://svn.ali.as/graph/Mojo.svg
As you can see, the "dependency free" Mojo shows as having a whole damned lot of dependencies. But then so will just about anything, because most of it is that twisty mess down the bottom that is the toolchain.
This is the same kind of graph logic that is used for the Heavy 100 list, and shown in this context, it is clearly less accurate than it should be.
What this graph REALLY represents is the API level dependencies. It's helping to show, for example, what will break if a module in the chain does a backwards-incompatible API change, or forcefully adds a new Perl 5.010 restriction.
So seen in this context, the "Volatile 100" list is doing what it is intended to do, while the "Heavy 100" isn't really working properly.
But what if we filter down to just the runtime dependencies. The new graph structure lets us do this easily for the first time!
CPANDB->distribution('Mojo')->dependency_graphviz(
phase => 'runtime',
)->as_svg('Mojo-runtime.svg');
Now we get something like this.
http://svn.ali.as/graph/Mojo-runtime.svg
Now, this graph SHOULD represent the API-level dependencies needed at run-time. Which should also match the list of modules that Mojo needs to load into memory at runtime.
The problem with THAT is that there's no less than 4 different distributions that are declaring a run-time dependency on Test-Simple, which is CLEARLY wrong.
This issue stems from the limitation in ExtUtils::MakeMaker to only list runtime dependencies as dependencies. Module::Build has no such problem, and Module::Install works around the problem by generating the META.yml itself and just letting EU:MM do the Makefile.
This graph is also what your downstream packagers see the dependency graph like. Because debian and friends can't automatically distinguish run-time dependencies from testing dependencies, they end up installing way more Perl modules than are actually going to be used.
Now, this is still the API level dependencies. But what if we now introduce a dependency filter on a version of Perl. This will cull out from the graph anything that was already available in the core.
Because a lot of these depend on version "0" (i.e. anything) we should be able to use a very very old Perl and still see a lot of modules fall off.
CPANDB->distribution('Mojo')->dependency_graphviz(
perl => 5.005,
phase => 'runtime',
)->as_svg('Mojo-runtime-5.005.svg');
For convenience, all version dependencies in CPANDB are stored in canonical decimal format.
So now, we have the list of modules that will need to be installed on top of a base Perl 5.005 install. And that looks like this.
http://svn.ali.as/graph/Mojo-runtime-5.005.svg
That drops the dependency graph a little, removing Getopt::Long and File::Path, which have been in the core a looong time. It's also going to remove things like Exporter most of the time too.
But 5.005 is old an uninteresting, so lets move the version up to the current "Long term support" version for the toolchain.
CPANDB->distribution('Mojo')->dependency_graphviz(
perl => 5.006002,
phase => 'runtime',
)->as_svg('Mojo-runtime-5.006002.svg'); ... which gets us this
http://svn.ali.as/graph/Mojo-runtime-5.006002.svg
NOW we're looking a bit more controlled. This almost halves the number of things that need to be installed.
Of course, this is all mostly irrelevant right, because Mojo depends on Perl 5.8.
Well, except it doesn't. It's Makefile.PL, now that has a 5.008001 dependency in it. But nowhere in the Mojo.pm file or in it's META.yml does it actually declare or enforce a version lock.
Lets contrast this to the version graph in the latest release...
CPANDB->distribution('Mojo')->dependency_graphviz(
perl => 5.010,
phase => 'runtime',
)->as_svg('Mojo-runtime-5.010.svg'); ... which produces this ...
http://svn.ali.as/graph/Mojo-runtime-5.010.svg ... and as you can see, with the current 5.10.0 release there are indeed no dependencies!
Now all we need to do is step back to the oldest version they SAY that they support with no deps and confirm it...
CPANDB->distribution('Mojo')->dependency_graphviz(
perl => 5.008001,
phase => 'runtime',
)->as_svg('Mojo-runtime-5.008001.svg'); ...which makes this...
http://svn.ali.as/graph/Mojo-runtime-5.008001.svg.
Wait a second, what's this.
On Perl 5.008 Mojo is NOT dependency free. CPANDB says that it needs to upgrade Test-Simple. But surely that's wrong, because it only needs Test::More 0.
A quick look at the META.yml for Mojo shows the reason.
It's that dependency they have there to Test::Builder::Module that is causing the dependency on a more modern Test-Simple distribution.
Sebastian confirms that the Mojo team have indeed made a mistake and overlooked that dependency, as well as the lack of an actual Perl dependency declaration on Perl 5.8.1. And of course, now they have been revealed, they'll likely be fixed in a matter of days.
The Top 100 site should pick up some new lists in the next few weeks, but in the mean time you might like to play with the CPANDB graph generator yourself.
And fortunately you can do this in a way that doesn't involve writing code, via the new "cpangraph" tool that is installed with CPANDB.
Rather than with the code snippits I showed you, those graphs were ACTUALLY being produce by the equivalent command line call (because I'm lazy)
cpangraph Mojo
cpangraph --phase=runtime Mojo
cpangraph --phase=runtime --perl=5.005 Mojo
cpangraph --phase=runtime --perl=5.006002 Mojo
cpangraph --phase=runtime --perl=5.010 Mojo
cpangraph --phase=runtime --perl=5.008001 Mojo
For one final example of a large graph, here is the install graph for Padre on Perl 5.008008, shown with the "rankdir" option to get a left to right ordering.
cpangraph --rankdir --perl=5.008008 Padre
http://svn.ali.as/graph/Padre-5.008008.svg
The Padre team has been using this graph to look for places we might be able to remove wasteful dependencies. The results have been excellent. We've managed to remove around 10-15 dependencies, resulting in Padre moving OFF the Heavy 100 list.
These links are collected from the Perlbuzz Twitter feed. If you have suggestions for news bits, please mail me at andy@perlbuzz.com.
Just in case you missed my Tweets about it, or don't read any other Perl blogs, or didn't receive the mail sent out from Perlmonks, Perlmonks was compromised. Passwords for users were published to the net.
Carl Masak has this analysis of the passwords and their weakness. It's clear, too, that many of the passwords were standard passwords meant to be used on multiple sites.
It’s been widely posted by now that 50,000 or so cleartext user names, email addresses and passwords were compromised from Perl Monks. Yesterday, as the news broke, a discussion started on IRC about how to respond. As some were debating hashing the exposed passwords and checking against other Perl community sites and various other remedies, I suggested that at a minimum, CPAN authors be notified so that they could change their PAUSE passwords if needed.
As the discussion swirled, I grabbed two modules from CPAN, whipped up a short script to email a warning to all CPAN authors and emailed it to Andreas, who manages CPAN & PAUSE. The elapsed time from concept to the email to Andreas, using two modules I’d never used before, including a test to my own email account, was probably 15 minutes. Andreas picked it up a couple hours later and started emailing CPAN Authors.
Here is the entire script:
#!/usr/bin/env perl
use 5.010;
use strict;
use warnings;
use Parse::CPAN::Authors;
use Email::Stuff;
my $mailrc = shift @ARGV;
die "usage: $0 <mailrc>\n" unless $mailrc && -f $mailrc;
my $pca = Parse::CPAN::Authors->new($mailrc);
my $body = << 'END_MAIL';
Dear CPAN author,
This email is being sent to inform you that all passwords on the popular
Perl Monks website were compromised. Many CPAN authors have accounts
there and in some cases have used the same password for PAUSE.
If you have any reason to suspect that your PAUSE account password
is no longer secure, please visit http://pause.cpan.org/ and change it.
If your PAUSE account is not affected, please disregard this message and
accept apologies for the unsolicited email.
Regards,
PAUSE Administrators
END_MAIL
for my $author ( $pca->authors ) {
Email::Stuff->from ('module@perl.org' )
->to ($author->pauseid . '@cpan.org' )
->subject ("Perl Monks compromised, PAUSE accounts at risk")
->text_body($body )
->send;
}
(If you're a CPAN Author, the text should be familiar to you as you either got it yesterday, don't have a current email address on PAUSE, or it got caught in your spam folder.)
So even as the community appreciates the value of "modern" Perl, it's good to remember that Perl and CPAN still rock when you need a quick solution to a problem.
Ubuntu has recently offered a Perl upgrade to 5.10 and I accidentally accepted the upgrade. Unfortunately, since our development Perl is 5.8.8, I've encountered a few annoying problems. Since many people are confused about how to do this, I decided to slog through and figure it out. Not fun. The answers to the various problems are scattered hither and yon, making this a very unpleasant process.
First, you need to get Perl and apply a regex patch which closes a security hole (many thanks to Rufus Cable for alerting me to the patch):
wget http://www.cpan.org/src/perl-5.8.8.tar.gz
wget ftp://ftp.cpan.org/pub/CPAN/authors/id/N/NW/NWCLARK/regexp-5.8.8.patch
tar xzf perl-5.8.8.tar.gz
cd perl-5.8.8
patch -p1 < ../regexp-5.8.8.patch
rm -f {config,Policy}.sh
sh Configure -de
Be sure to read the INSTALL file for more information about the Configure options.
At this point, everything should be fine, but you can't run make yet. Actually, you can run make, but it will likely fail. Fortunately, you can keep fixing errors and rerunning make until all errors go away.
The first problem is a strange "You haven't done a make depend yet" error. As it turns out, this is because Ubuntu has decided to link /bin/sh to /bin/dash instead of /bin/bash. It's a faster shell, but not only does it not support everything bash does, it also is less tolerant of errors. The makedepend file has an unterminated quote string, so doing this gets you over the first hurdle:
sudo ln -s /bin/bash /bin/sh
Be sure to change the symlink back after you're done if you want dash instead of bash (note that using a correct shebang line at the top of your bash scripts makes this a non-issue for you).
If you've already tried to run make, you might see this error:
makedepend: Already running, exiting
That's because it previously exited abnormally (anyone remember "abend"?) and left a .depending directory lying around. Simply remove this directory and rerun.
If you run make now, you might hit the second error:
No rule to make target `<command-line>', needed by `miniperlmain.o'.
Apparently, we filter out some of the gcc output, but it's changed over time and we missed this. The following fixes it:
perl -i~ -nle 'print unless /<command-line>/' makefile x2p/makefile
That command tells perl to edit those files in place, reprinting all lines except those containing the string <command-line>. If for some bizarre reason you don't yet have Perl installed (or if it's broken), you can edit those files manually, but it's tedious and error prone.
You can rerun make and it might be fine, but you might hit this error:
SysV.xs:7:25: error: asm/page.h: No such file or directory
Arg! I don't know what this is, but apparently it's related to the linux headers. What I don't understand is that I have them installed (sudo apt-get install linux-headers-$(uname -r)), but page.h is not symlinked in /usr/include/asm/. A locate '/page.h' | grep asm returned many candidates, but the most likely fix seemed like this:
sudo ln -s /usr/src/linux-headers-2.6.28-13/arch/x86/include/asm/page.h /usr/include/asm/page.h
Obviously, the exact page.h you're linking to will vary depending on your architecture.
Assuming all is well and make completes successfully, you can now run make test && make install and you'll have successfully downgraded perl (I would recommend you set your prefix when you run Configure to ensure that you don't overwrite the system perl. You can always set up a symlink to ensure you have the proper Perl running).
A problem that many programmers are encountering these days is a high startup cost for their application. We love reuse, but unfortunately, reuse is not free -- big libraries have to be loaded from disk, code has to be generated (for runtime efficiency), and so on. This is a particular problem for Perl programmers; we are used to writing tiny scripts in Perl that glue our lives together, but as the collective learns more about programming, we are pushed in the direction of using bigger libraries. This is gradually making our tiny scripts start up more slowly. (But, they work better and go together even more quickly, so it's generally a worthwhile tradeoff.)
One of the biggest pain points, specifically, is Moose. We all love
using it, since it makes our code so easy to write and understand, but
it does trade startup time for runtime (and programming-time)
efficiency. This is irrelevant for most of the work we do, like web
applications, since we start them up maybe once a week. It is
relevant for command-line applications, though, since we start those
up frequently. If you've ever used Devel::REPL you know that Moose
apps can take a while to start up.
One solution to this problem is called Mouse. It is most of
Moose's syntax sugar, without a deep MOP or as much modularity. Using
Mouse means you can say has 'foo' = ( is => 'ro' )> without
impacting the startup time of your application. This was OK for a
while, but the Moose community is moving too quickly for Mouse to
track it. This makes Mouse a Moose-alike, not a drop-in replacement
for Moose -- once in a while, someone comes into the Moose irc channel
and complains of a bug that we know we fixed months ago; they don't
bother to mention that they're actually using Mouse. Mouse really
has nothing to do with Moose, it's a totally separate project with a
totally separate community now. So if you really like Moose and the
MooseX:: modules, you can't just start using Mouse; you need to
rewrite your app and possibly port a bunch of MooseX:: modules over to
Mouse. This means that Mouse isn't really a solution to startup
performance problems with Moose applications; it's simply a decision
you can make when you want something kind of like Moose, but really
want fast startup at the expense of everything else, including runtime
speed. (Sort of like writing your application in Haskell instead of
Perl. Both are programming languages, but Haskell is not a drop-in
replacement that makes Perl scripts faster; it's its own thing. Mouse
is like Moose in that they are both object systems for Perl, but they
are different enough for each to be "its own thing".)
So anyway, with this in mind, we're left without a solution to the
startup problem. I decided to look at other big applications that I
use to see how they handle their startup time problems. The apps I
spend most of my time using are emacs,
conkeror, rxvt-unicode, and xmms2. All of
these apps are somewhat big, but need to start up quickly. Emacs
needs to start quickly when I want to edit a file in a terminal,
conkeror needs to instantly display a URL I pass to it on the command
line, rxvt-unicode needs to give me a new terminal when I need one,
and xmms needs to skip to the next song as soon as I press enter after
typing xmms2 next. Waiting for an application to start up would
not be acceptable in any of these cases.
Emacs handles the startup time issue with a client-server
architecture. You start emacs --daemon when you log in, and then
you get an emacs window by running emacsclient -c or
emacsclient -t. This is great; emacs --daemon takes almost 10
seconds to startup while I'm logging in, but emacsclient windows
come up instantly. A perfect solution to the startup speed problem.
Conkeror and urxvt do something similar. They start up, and when they get a message from their command-line app, they perform the action in the original process. (urxvtd/urxvtc, anyway; plain urxvt doesn't do anything special, but who uses plain urxvt?)
The xmms2 command-line utility is just a client that talks to the xmms2 server, which is started the first time you use the client, or at login.
The common theme is a client-server architecture. You have a server that does the hard stuff, which might cause it to take a while to start up, and you have a client, which doesn't do anything hard, and can startup instantly. At the cost of some extra disk activity at login, the app starts quickly every time you use it.
I think this is a fine solution to the problem of slow-starting apps,
so I wrote a framework for writing apps like this in Perl. I know
there is already
pperl, but it
suffers from being too magical. It tries to make an arbitrary script
persistent without the script knowing that it's persistent. Like
Apache::Registry and friends, sometimes it works, but often it
doesn't quite work. Since it's a big ball of XS magic, you are
basically fucked if it doesn't work. You can't fine-tune it inside
your application; you are stuck with the same behavior for every perl
script you ever want to make persistent.
I don't like this (or being "basically fucked"), but I do like
programming. So I wrote
App-Persistent, which is
a library that makes writing persistent command-line applications
really easy. It has two parts, the server App::Persistent::Server,
and the client, pclient. The client does most of the interesting
stuff; it reads the environment, current working directory, program
name, and so on, and sends that to the server. The server passes this
information to your app code, along with pipes representing stdin,
stdout, and stderr. The app then runs normally; with the client and
the server passing the input and output around as appropriate. As a
user, you won't notice that you aren't using the real app directly.
As an app author, you won't have to write your own persistence
solution or debug any magic.
I've actually been working on this for some time, but yesterday I hit
the milestone I was waiting for; Moose apps are now faster than
Mouse apps. I used sd as my
example app. The Mouse version of sd, which lazy-loads modules for
each command sd can run as that command is run, runs sd help and
begins producing output after about 0.3 seconds on my machine. (I
wrote a script to see how long it takes for output to arrive; see
http://github.com/jrockway/app-persistent/blob/0271c8568cae4eacb0d973886195914fed2c9522/util/bench-io.pl.)
This is pretty good, you never really worry about sd starting up.
You type sd, and it does its thing.
I then wrote a frontend to sd called psd, which uses Moose and
loads all of sd's commands at startup time. This takes about 12
seconds. (The time is spent loading the various modules implementing
each command; not loading Moose. Moose is not that slow.) I made
this frontend use App::Persistent::Server to handle requests.
I then timed how long it took to run the help command via pclient help.
The result; for a fully-Moose, non-lazy-loaded request? 0.2
seconds.
Yes that's right, the Moose version consistently starts producing output sooner than the Mouse version. Moose is faster than Mouse.
The fact that it's faster doesn't really matter. The fact that you can get fast-starting command-line apps with without writing your own custom object system and creating your own object system community is the big deal.
(Did I mention that Devel::REPL is much faster? It used to take
about 3 seconds per invocation to start; now it is under 0.2 seconds!
For various reasons, the persistent version is not in the svn or git
repository yet. Ask me on irc if you care, it's a looong story.)
If you want to try App::Persistent yourself, it's pretty easy to
get started. Here's an example app that reads from stdin and echoes
it back to you:
use App::Persistent::Server; my $server = App::Persistent::Server->new( code => sub { while(1){ local $| = 1; print "> "; my $line = <>; print "You said: $line"; exit 0 if $line =~ /quit/; }; }, ); $server->start; exit $server->completion_condvar->wait;
(The server opens up a control channel that lets you cleanly stop it,
but if you don't care about that, you can just EV::loop() or
whatever.)
If you called this echo.pl, just run echo.pl. The server will
start, and you can connect to it with pclient:
$ pclient > Hello You said: Hello > quit You said: quit $
(The "0" exit code is of course relayed back to your shell.)
You can connect more than one client at the same time; each client gets its own forked process. (This will be configurable in the future, the goal is to have many interfaces; a code block with pipes you can read/write from/to with AnyEvent, a coro thread with things like @ARGV localized to the thread, a separate process, etc. Each is appropriate for different kinds of apps, and being able to pick the one that suits your use case is the point of having a framework instead of quick hack.)
If you want to do something more complicated, the code to cut-n-paste-n-customize is:
my $server = App::Persistent::Server->new( code => sub { my $info = shift; # fake environment local %ENV = $info->environment; local $0 = $info->program_name; chdir $info->working_directory; local @ARGV = $info->cmdline_args; # code goes here }, );
This will set up everything as though you were a new program executed from the command-line, rather than a block that the App::Persistent server is running. (All the information comes in the form of an object you can inspect; there is no magic. If you want magic, you can do it yourself in a library. It's up to you.)
So this is what I have right now. In the future, I'd like to make the protocol that the client and server speak a bit richer; specifically, I would like to add a command for prompting for input interactively (like Term::ReadLine that will work across the pclient/server connection), and an command for starting a text editor on a given file, in the pclient window. (sd needs both of these things.)
I also want to add a feature for sharing a virtual terminal, instead of communicating over sockets. I am not sure how to do this yet, so if you know how to do this, let me know. (screen does it, but I haven't read the code yet. I have read a lot about ptys, but I am not sure how to share them between processes that are in different process groups.)
There are also issues with interactive signals; right now these are ignored. (If you hit Control-c, you kill pclient, not the server process that pclient is acting as a surrogate for. This is not the correct behavior.)
There is the issue of using AnyEvent after the fork. AnyEvent doesn't
have a way to cancel the parent event loop, so it will continue
running in the child, which will lead to incorrect behavior. I have
hacked around this in AnyEvent::Subprocess for POE and Event, but
not for EV, the loop that I most frequently use. (The obvious
solutions seem to make EV segfault!) When I figure this out
generically, I will put it in its own module (AnyEventX::Cancel?).
Finally, someone should write a pure Perl pclient. The version in the
repository is written in Haskell. (pclient actually starts up faster
than perl -e 1, though.) The next client I plan to write is an
emacs client, so I can use any persistent app right from emacs,
without having to emulate a terminal.
Anyway, that's about it. Try it out, tell me what features you need, and then we can spend our time writing applications instead of reimplementing common libraries so they load slightly faster!
OSCON 2009 adventures
I think that I've found a new term to describe myself. Adventuretarian. I live off adventure.
OSCON
This was my second year at OSCON, and my first visit to San Jose. Unlike last year, where I was a self-described OSCON rockstar, this year I was happy to take a more relaxed approach. I wasn't giving as many talks, the talks I gave were all quite technical, and I didn't keynote. However, that doesn't mean I didn't have fun; far from it!
This OSCON I played around a bit with outfits. I'd picked up a pirate hat earlier in Portland, and used it in my tutorial when talking about PAR, the Perl Archiver. The hat ended up being one of my best fashion decisions ever, as it found its way into photo shoots, restaurants, and social events. In terms of getting noticed, or being popular with small children, or having random people say "Arrrr..." as they walk past, a pirate's hat is awesome.
My other outfit was my Star Trek uniform, used for my talk on The Art of Klingon Programming. It's not something I can ever imagining wearing for more than an hour or two at a time, as it's hot, and doesn't breathe. Of course, it's fantastic when you want to hang out with the cast of Trek in the Park.
Talking of The Art of Klingon Programming, it looked like it came across smashingly well, but I had forgotten to remind the audience to rate the talk if they liked it. So if you were there, and you enjoyed the session, go rate it now. ;)
I went to less parties than last year, and so met fewer people, but I was able to spend more time with people who I really enjoy as a result. One of the highlights was a beach trip down to Santa Cruz and around to Half Moon Bay, with some pretty spectacular beaches, cliffs, and even a light-house.
San Francisco
After the conference was a trip to San Francisco, staying with Julian (the most amazing photographer ever), and Jackie (the most amazing story-teller ever). Julian and Jackie's house was a hub of creativity and creative people. If I hadn't been so happily exhausted for OSCON I would have made more of it, but as it was I feel I was almost bordering on impolite by crashing and immersing myself in e-mail.
The next day involved a relocation to Skud's house, a home-cooked meal (my first since Schwern's excellent cooking in Portland), and discussions about San Francisco burrito etiquette, gender issues, booth babes, Australian history, pirates, musicals, and conferences. Skud, Schwern, Jacinta, myself, Valorie, and Andre, who I thought I had never met went out for lunch and ice-cream. Of course, in true small world fashion, Andre was Australian, and knew me from linux.conf.au. He's now working for Pixar, which sounds pretty sweet.
My last day in America involved Schwern, Jacinta, and myself going of a tour of the more touristy parts of San Francisco. Crabs and clam chowder seem to be a big deal in these parts, and I was given a "sample" of chocolate that I'm sure provided me with my daily intake of sugar in a single bite. Unfortunately we didn't have enough time for a big get-together of all the SF residents and visitors before I had to fly out. The flight home was good, with an unexpected exit row seat providing lots of legroom.
Home
The only downer of the whole experience is that Jacinta had managed to wrangle me a cool (first generation) Google Android phone, which I discovered that I loved dearly, but which seemed to have fallen from my pocket inside the taxi home. Attempts to recover it were without success, and without having first recorded all the handset details I can't remote-brick the phone, so it's unlikely I'll ever see it again. Jacinta's now given me her android phone, and while I feel incredibly special and grateful, I'm paranoid about losing it, too!
Today I'm preparing my new laptop, which is about twice as awesome as my old one, comes with a three year worldwide warranty, and costs only a third of the price. Moore's Law + USA = Laptop win. I'm also paying bills, sending out invoices and faxes, paying super, catching up on tax, and generally doing all the things that keep a small business running.
Tonight I'm on a flight to Europe for YAPC::EU, which possibly represents the first conference ever where I have all my talks prepared and ready before the conference starts. I'll be back in Australia in a couple of weeks time, no doubt exhausted from my trip and looking forward to the next one.
Posted: 30th July 2009.
Tags: oscon oscon2009 perl travel
Bookmark:
Jay Kuri recently made the distinction between external and internal dependencies. He makes the case that when you choose to internalize a dependency the implementation of that dependency usually suffers as a result.
Whenever you choose to internalize any part of a software component, in my opinion that's very similar to hard coding a value.
My code is often criticized for being too componentized or abstract. To use it you usually have to use an OO API (even for simple things), provide configuration values explicitly (even for obvious values), it will have many CPAN dependencies, etc, but there is a reason I code this way.
Today I was trying to generate a password string from a large integer. Specifically I wanted something I can paste into a password input box, that contains alphanumeric and punctuation characters, and is the direct result of the output of Digest::HMAC.
There are many modules on the CPAN that generate passwords (Crypt::RandPasswd, String::MkPasswd, Crypt::PassGen). However, none of them can be used to solve my problem.
Generating random passwords involves two steps:
All of these modules provide a new approach to the second step, implementing clever pronounceable passwords, interesting encoding schemes, and so on. This is the code I would like to reuse.
Unfortunately they all also internalize the dependency of generating random numbers, and the closest thing to an API to override that is this gem from Crypt::RandPasswd's documentation:
{
local $^W; # squelch sub redef warning.
*Crypt::RandPasswd::rng = \&my_rng;
}
So to adapt Crypt::RandPasswd to my requirements I'd have to recreate the rand API that takes an upper bound and returns a floating point number between 0 and that number.
This is better than all the other modules, at least it would let me use something like Crypt::Random::Source to make sure the passwords are truly random, and it's documented so it's unlikely to break if I rely on it, but this is hardly a solution.
I'm left with two choices, monkey patch or reinvent.
If Crypt::RandPasswd was an OO module I could perhaps subclass it instead of monkey patching, but that's not necessarily enough to be reusable.
If these modules had OO APIs that used two delegates, one to generate random data and one to encode random data into a string representation optimized for human consumption, I probably wouldn't have this problem.
Like Jay said, by externalizing the algorithmic dependencies here (even if they still end up in the same CPAN distribution), we're increasing the likelyhood that a clean, well thought API for each delegate would emerge. This is what makes polymorphism so powerful when used right.
My critics would argue that this would be bloated and overengineered, but if the problem of encoding passwords were separated into a standalone class, at this point the work is done.
There is no need for more features, or an extensible API. Another encoding scheme could be implemented using the same API and dropped into where this fits. it wouldn't need to worry about details like generating cryptographically strong random numbers, or providing additional customizability.
This scope reduction is in my opinion fundamental to writing reusable and maintainable code. It seems like a PITA until you have a working system, and it may seem like more work when you're writing such code or installing modules that are written in this way.
Old monolithic code can obviously be cannibalized or refactored into new code during the maintenance cycle, but if its scope was properly reduced to begin with the flawed component could be replaced a lot more easily, either by dropping in a replacement if the API can remain intact, or also adjusting the dependent code to use a better API.
An example of how to not write a proper solution to a self contained problem, take a look at Catalyst::Plugin::Session.
When I rewrote the system that predated it, I made the distinction between preserving state in HTTP and storing associated data on the server. However, I unnecessarily intertwined the implementations of these two sub problems, creating an overly complex hook based API that is still monolithic.
This code cannot be saved. At best, snippets can be copied and pasted into a new implementation, but since the design is flawed and the implementation is so large and complex, there is no hope of reuse or even a backwards compatible plugin API. At best the same API (or a subset) can be provided for applications, so that user code will not have to be rewritten.
Off the top of my head I can enumerate at least 6 different components of a session management system, with no overlap and very little interaction between them. This does not mean that you would have to configure at least 6 different Catalyst components/plugins just to get $c->session. Instead, a wrapper component that configures them for you (and does nothing but configure) would be written on top of them, to provide DWIM.
This is the role of a sugar layer. I've previously stated the advantage of Moose lies in its underlying componentization. However, Moose's success lies in its appearance of a monolithic system; the sugar is concise, friendly and works out of the box. When you look closer the real extensibility becomes apparent.
In this way Moose is also very future proof. We know that the sugar layer exported by Moose.pm is flawed, we've learned many new things since it was written, and we can provide a cleaner, friendlier and more powerful syntax in the future.
Some time ago I commented on a blog post asking how some of the more prolific CPAN authors manage to keep up. Scope reduction is the most fundamental part of doing that, they key is that I don't actually maintain a lot of that code, there is nothing left to do except fix minor bugs. Most of my modules will never get a new feature, instead that new feature would be written as a module that depends on the existing one.
It's a little more work up front, hard coding seems easy when you're doing it, but it's a very near sighted optimization. I suspect this is what makes the quality of code in Haskell's hackage generally higher than other languages, the type system makes it hard to be lazy that way on the small scale, serving as a constant reminder to keep things cleanly separated.
Catalyst::Plugin::Session makes me cringe. I am very grateful to t0m for maintaining that piece of shit. Thankfully at least it taught me a valuable lesson in software design.
Dear CPAN author,This email is being sent to inform you that all passwords on the popular Perl Monks website were compromised. Many CPAN authors have accounts there and in some cases have used the same password for PAUSE.
If you have any reason to suspect that your PAUSE account password is no longer secure, please visit https://pause.cpan.org/ and change it.
If your PAUSE account is not affected, please disregard this message and accept apologies for the unsolicited email.
Regards,
PAUSE Administrators
I won't post links, but you may have recently heard of PerlMonk's current security situation.
As part of our response to this situation the CPAN admins (or more specifically, Andreas) will at some point in the near future most-likely be doing a partial bulk-reset of PAUSE passwords.
This will not need to be a complete password reset, as there is a method by which we can limit the password reset to any rotten PAUSE accounts infected by the (50,000) rotten PerlMonks accounts. Either with a shared password, or a co-incidentally identical password.
If you'd like to have some control over the situation, I would strongly recommend that if you have both a PerlMonks and PAUSE login you audit the password you were using on PerlMonks and check for any other sites or hosts you may have shared that password with, including PAUSE.
If you update your PAUSE account to a more secure password, you should not get caught in the reset.
Now would also be a good time to update your PAUSE email address if it has gone stale.
Posted with comments off, for obvious reasons. Now go check your security.
A lot’s happened in the few days since my keynote at OSCON and I think it’s time I did a round-up of women-in-open-source-related stuff from the conference itself and the not-quite-a-week since.
Some wins for the conference:
And since the conference:
There are a bunch of discussion threads going on in various places. Here are a few that I know of:
If you join in on any of the above threads, please try to maintain civility.
Now, I have an actual dayjob I need to get to.
Eric Wilhelm and I have a friendly, but long running debate over the wisdom of using pure v-strings for Perl module version numbers.? We just discovered that behind our difference of opinion is actually a technical difference: I use a 64 bit perl and he doesn’t.
The test case I usually cite has a module with a v-string version like this:
package Foo; use strict; use warnings; our $VERSION = v0.1.1; 1;
Since people are often taught to require minimum versions in decimal format instead of v-string format (e.g. “use 5.006″ instead of “use v5.6.0″), I want to know that v-string version numbers work as expected, even if someone says “use Foo 0.001001″.
On Perl 5.10 or if version.pm has been loaded, all is well.? But on earlier Perls, compiling for 64-bits produces a very different result:
$ perl5.8.9-32bit -e 'use Foo 0.001001' $ perl5.8.9-64bit -e 'use Foo 0.001001' Foo version 0.001001 required--this is only version 0.001001 at -e line 1. BEGIN failed--compilation aborted at -e line 1.
Apparently, the comparison between mixed version types is numeric and, with 64-bit precision, the conversion between v-string and decimal format gives unequal values and the version test fails.
Lesson: always test a module’s version in use() with the same format it has in $VERSION.
Corollary: if you want people to follow the rule above consistently, never change version number styles once a module is published.
This blog post makes Mojolicious::Lite look very interesting. The fact that it has no dependencies makes it even more compelling. It's the sort of thing one could more easily attract new developers with. Can you imagine rewriting my old CGI course with something like this? Or maybe a way to introduce new users to Perl? There are many excellent Perl frameworks out there which I would, nonetheless, hesitate to start new users on. Just trying to set up CPAN can be difficult for them; working through even one test failure is often too much. For people who want to jump in and learn "Web programming", though, this might work.
See what happened? Sebastian was trying to communicate the benefits of his software to a potential consumer and it piqued my curiosity. You know what that's called? Marketing. It may have a different end goals and customers, but it's still marketing.
Yes, folks, blog posts are often marketing, whether deliberate or not. If you blog about your breakfast, it's probably not marketing, but if you're a chef, it might very well be. Marketing is not something you can discreetly identify. Heck, I've lived for quite some time with someone who has directed marketing at a number of companies around the world. One of my best friends used to do marketing for Waggener Edstrom, Microsoft's PR firm (or maybe former firm. Don't know). While I'm hardly an expert in marketing, I've been amazed at a lot of the clever stuff they've come up with and it's a far, far cry from the "waste of time" that some people are calling it.
Marketing isn't just ads. It's not just articles in newspapers. It's not just viral videos. It's not just guys walking around in crazy costumes handing out leaflets. It's also people touting their latest Web framework in their blogs. It's people teaching classes about Perl 5. It's people giving talks at users groups about software testing. It's word of mouth. It's anything you want it to be, so long as you connect producers and consumers and get your message out there. I find it hilarious that many of the people decrying marketing are, in fact, marketing themselves (even though they don't know it).
RT has gotten a lot better in the last couple years.? I used to find it horribly slow and unwieldy, but the folks at Best Practical have made several nice improvements that are making it a useful tool for me to manage issues across the growing list of modules I wrote or help maintain.? For example:
Here’s a quick overview of how I use it to manage bugs and other tickets.? I’m not perfect at this process yet, but this is what I’m aspiring to and it’s worked very well recently in triaging bugs in Module::Build, ExtUtils::CBuilder and ExtUtils::ParseXS.
For all new or stale tickets, here’s my triage process:
With a little up-front investment using this system, I can pretty quickly see where I might spend any “round tuits” I have and spend less time figuring out where to get started.
For me, getting the severity right is one of the most useful things in prioritizing my time.? Here are the severity criteria I used when triaging existing bugs in Module::Build:
Before releasing 0.34, I set a goal to eliminate the Critical and Important bugs and reduce the list of Normal bugs as much as I could without spending tons of time on any particular bug.? With the list set, I just worked top down until I ran out of time or got bogged down.? Easy!
So while I didn’t think I’d ever say it, thank you, Best Practical for making RT something that finally works (more or less) the way I want it to.
Read more of this story at use Perl.
Read more of this story at use Perl.
I just went over the list of BOFs at YAPC::EU. added myself to the Rakudo, the Marketing and the Perl Trainers BOF and created a Padre BOF.
If you are coming to YAPC::EU and you'd like to setup padre or have a discussion about it then please either find me in the corridors or better yet sign up to the BOF and we'll try to organize a meeting. Hopefully not at the same time as one of my other BOFs.
In my last internals post I mentioned that most of the data in Perl revolves around the SV struct.
The excellent PerlGuts Illustrated describes these structures in great detail, and of course there's the two main documents in Perl's own reference, perlguts which tells you about them and perlapi which tells you what you can do with them.
This post is sort of like PerlGuts for dummies, intended to lightly touch on the SV struct and then explain some typical usage but not much more.
Unfortunately this is going to a hard read. In early revisions I tried to show how to write a very simple XS function, but there are just too many details, so I will save that for a later time.
I've stolen this diagram of an SV from PerlGuts Illustrated, but just this is only the tip of the iceberg. Click the link if you haven't already. Really. JFDI. I'll wait.
SV stands for scalar value. This C struct represents every singular value available in Perl, numbers, strings, references. The SV struct is technically what's called a tagged union, it contains a field that denotes that value's type, and the meaning of its 4 fields can change depending on that type.
The first field is called ANY. This contains a pointer to the actual data. We'll get back to this one in a second.
The second field is the reference count. This is just a simple integer. You can increment it by calling SvREFCNT_inc, and decrement it using SvREFCNT_dec. New SVs have a reference count of 1, and if you SvREFCNT_dec to 0 then the SV is destroyed.
The third and forth fields are the flags and the type. Again, for details refer to illguts, but the gist of it is that the type and the flags tell the various macros what structure the ANY field points to.
The simplest value is an unintialized SV. In this case the type is SVt_NULL and the ANY field is a null pointer.
So what about an SV containing a number?
In the Perl source code the type IV is an alias to the native integer type. It's pretty much the same as long or long long depending on the architecture and the arguments to Configure.
If you call sv_setiv(sv, 42) then the SV's ANY field will be set up such that it points at an an IV whose value is 42, and the SvTYPE will be SVt_IV, and set the IOK flag saying that the IVX slot has a valid value (this is actually not entirely accurate, so again, see illguts for more details).
The ANY field actually is set by the SvUPGRADE macro. If necessary this macro will allocate additional structures require for the target type, and changes the SvTYPE. In this case it will ensure storage area for the IV is available.
When the IV is actually stored in that structure the IOK flag will be enabled, signifying that there is a valid integer stored in this SV.
To extract an IV value from SV you use the SvIV macro. This macro resolves to an expression that will return a native integer type by from the structure pointed to by the ANY field.
So what happens when you call SvIV(sv) but sv actually contains a string? What SvIV actually does is check the IOK flag, which means that the ANY field points to a structure with a valid IV in it. If the flag is not set, then an IV will be generated (and stored) by numifying the value.
In the case of a string like "42" (whose type is SVt_PV), the value will be parsed using the grok_number function. The SVt_PV will then be upgraded to an SVt_PVIV, which has slots for for a string as well as an integer.
The next time SvIV is called on this SV the IOK flag will be set from the previous invocation, so it will just return the IV without computing anything.
So in effect, there is an SvTYPE for every kind of value value, and also for every sensible combination of values (in our second example we make an SV that is both an integer and a string at the same time).
As is the case everywhere in Perl, there is a rich set of macros to manipulate the structures, so that you generally don't have to think about the various flags and types, they are an implementation detail. You tell the macros what you want to get out of an SV and it does the hard work for you.
If you haven't already, please take a few minutes to at least skim illguts. It's not important to know all the different variations and wha they mean, but you should know what a reference looks like in C land, and how things like arrays are represented.
Perl 5 is a stack based virtual machine. This means that in order to pass data around, pointers to SVs are pushed onto a stack by the caller, and taken off the stack by the code they are called. The called code does its thing, and then overwrites its parameters with pointers to the SVs that are the result of the computation.
The two main stacks that pass data around are "the" stack and the mark stack. SVs are pushed onto the stack, while the mark stack contains pointers to interesting places on the stack.
In order to push a value, you use the XPUSHs macro or a variant. X stands for "extend", it makes sure the stack has enough room for the value before adding it.
So what's the mark stack for? When you call a subroutine, like foo($x, $y) a new mark is pushed, then $x, then $y. The body of foo gets a variable sized list of parameters by looking from TOPMARK to the head of the stack.
To get values you can use the SP macro. $x is available as SP(0) (though there are many convenience macros that are usually quicker than SP).
As a side note, recall that values pushed to the stack are passed as aliases. This is because a pointer to the SV is copied to the head of stack (as opposed to a copy of the SV itself).
So what about reference counting? Pushing parameters is fine (there's still a counted reference in the call site), but when you're pushing a return value you've got a problem: if you SvREFCNT_dec the value then it will be destroyed prematurely, but if you don't then it will be leak.
Fortunately Perl has yet another stack to solve this, the tmps stack. When you call sv_2mortal(sv) a pointer to sv is saved on the tmps stack, and on the next call to the FREETMPS macro all of these mortal SVs will have their reference count decremented.
In order to assign the value of SV to another, use the SvSetMagicSV to copy the data over. It doesn't copy the pointer in ANY, but the values themselves.
One important thing to keep in mind is that when assign copy a reference, like:
my $x = \%hash; my $y = $x;
$y is actually copied, it doesn't share any data with $x. The two SVs are both pointing at the same HV (%hash).
This copying is also done when returning values from a subroutine.
It's important to keep in mind that most operations in Perl involve copying the SV structure, because assigning an SV pointer actually creates an alias. Accidentally creating aliases is typical issue I have as an XS n00b.
In XS we usually use newSVsv to create a copy, because often the target SV does not yet exist. This is the same as creating a new empty SV and calling SvSetMagicSV on it.
If you've ever seen the error Bizarre copy of %s? That's what happens when sv_setsv is called on a value that isn't a simple SV, such as a number, string or reference. Correctly copying arrays and hashes involves making copies of all the SVs inside them and reinserting those copies into a new AV or HV.
So as I just mentioned, there are 3 valid things to copy around using setsv: undef, simple values (strings and numbers), and references. Complex data structures are passed by reference, or by evaluating in list context in a way that simply puts every element SV on the stack.
A reference is an SV whose ANY field points to an SV * which poinsts at another SV.
Calling the SvRV macro on an SV with ROK set returns the target SV.
In the case of e.g. \@array the SV will return true for SvROK(sv), and SvRV returns the AV of @array.
When working with references casting becomes important. Structs like AV, HV etc have the same fields as SV but the operations you can perform on them are different. The various API functions operating on these types require pointers of the correct type, which means you often need to cast points from one type to another.
For intsance, if we've received a reference to an array in ref_sv, we could do something like this:
SV *ref_sv; /* assume ROK is set */ SV *array_sv = SvRV(ref_sv); assert( SvTYPE(array_sv) == SVt_PVAV ); AV *array = (AV *)array_sv; SV **elem = av_fetch(array, i, FALSE);
This post is getting very long so I will STFU now. In the next post I'll review the basics of XS boilerplate, and that plus this post should be enough to get started on reading/writing simple XS code.
Many in the Perl community don't agree with me that we need professional marketing. Some hope to solve the branding issue by gettting more designers enthusiastic about Perl. I think such grassroot efforts are really good and it will hopefully lead to Perl gain more new users frome the enthusiastic crowd.
Let's see if there is another world where a Marketing Director of Technology evangelist can be better than grass-root efforts? What about convincing CTOs and other IT managers that Perl or some of the technology built on Perl is the right choice for them?
The grass-root efforts and the more commercial efforts don't rule out each other. They can even strengthen each other.
In order that you won't feel I have a secret selfish agenda, let me first write down what I'd like to have.
I am offering training in Perl in general but I have a specific course called Test Automation using Perl that I'd like to promote more. I have a test automation newsletter and I blog about it but I am sure I could have more clients if there was someone promoting the testing tools of Perl directly to Test and QA managers. It would especially help if that person was also mentioning my classes but even without that, if the market grows, hopefully my part grows as well.
I can't afford such a person at a full-time job but I could participate in paying part of the wage of such a person.
There are companies behind these open source Perl Content Management Systems. I guess some of the companies are large ebough to have their own marketing directors but others do not. I am sure they would all have more clients if there was a person promoting their their technology. They might even be ready to pay part of the salary of such person.
There are several companies behind these projects too. Companies that make money by implementing web applications. They could too have more clients if there was someone promoting their technology. I guess they all have some form of marketing already too but maybe they would also be ready to pay part of the salary of a marketing person.
I'd like to organize the people behind these companies and setup budget for someone who will help promote our respective technologies.
We can do it either privately within a company which would be a plain busineess deal for co-marketing or we can do it under the umbrella of either TPF or EPO. I'd prefer to do it in the open. The main reason for that is that I hope that once our efforts start to pay off we will be able to reach more organizations that would like to participate in the promotion of this technology. Meaning companies that are intrested in promoting Perl based soulutions (such as Bugzilla, Webmin, etc.) and companies that want to recruit Perl programmers. (Is it really hard to find good Perl programmers?)
I'd be glad to discuss this with any of the interested parties via IRC or if you are coming to Lisbon, then there too.
Read more of this story at use Perl.
Back in 2007, I worked with David Maxwell of Coverity to get Parrot scanned by Coverity Prevent. Coverity Prevent is a static C analysis tool that looks for potential bugs in a codebase, and it's far more comprehensive than built-in GCC warnings or splint. It's also expensive, licensed per-seat, but Coverity offers a service to open source projects to analyze their source code.
I worked on the Coverity results for a while, fixing bugs and removing dead code here and there, but for whatever reason turned my attention elsewhere.
Last week, I ran into David during OSCON, and it sparked my memory of how useful the Coverity tool was. It had been so long since I'd checked, I couldn't even remember my login credentials. It didn't matter, as it turns out the scanner hadn't been running, and in fact was still pointing at the old Parrot Subversion repository. Now, he's got things going again, and I have a raft of new potential bugs to investigate and fix.
I want to thank Coverity for providing this service to the Perl and Parrot communities. There are plenty of ways to support open source without having to shell out cash. This is a very useful one indeed.
This week at OSCON, Kirrily Robert, a/k/a Skud, gave a great keynote (slides, video) about women in open source, and how projects can improve by welcoming these potential contributors.
I loved it, and not just because of the importance of equality in society in general. From the sake of the quality of our projects, we need to keep the welcoming not only to women, but to everyone who might feel excluded from contributing. Contributing could mean anything from submitting code to answering mailing list questions to writing docs. Most specifically, it does not only mean writing code.
Skud worked on two projects that were specifically aimed at and populated predominantly women. She surveyed the project members, the comments she received from project members are telling, such as "I didn't feel like I was wanted" and "I never got the impression that outsiders were welcome."
I hope that I've never given anyone that impression on my projects. While I've always welcomed contributions from everyone, I've never explicitly stated it. I think it's time for that to change.
I've adapted part of the Dreamwidth diversity statement into what I'm calling, for now, a "welcoming statement." My plan is to put this on every module I maintain:
The (foo) project welcomes people of any ability level, age, social stratus, gender, sexual orientation, race, ethnicity, nationality, religion, political opinion or physical aspect. The only requirements for participation in the project are basic civility and respect for others.
I also want to put that here on Perlbuzz as well, with additional wording about my editorial policies as far as what I'll run and won't.
I'm interested in suggestions you might have for improving my welcoming statement.
If one thing stands out from recent discussions on The Perl Renaissance (the whole set of modern/enlightened/marketing blog posts) it's that we don't properly recognise that we are so unskilled at promotion that we don't know what we don't know about the subject.
We throw around words like "Modern" and "Enlightened" and "Directory of Marketing" because they are the best we can do, because we don't know how to name things well.
We discuss the competition for mindshare as if our users are an Economist's mythical "Rational Consumer" and only care about release schedules, and we grasp at ideas like paying people to take a role, with no grounding for knowing if that type of role is what is needed.
I'm just as bad, but I at least try to remember how little I know about most thing. My websites are either ultra-minimalist, intentionally devoid of style, or aim for the simpler is better look (which is about the best I can do when I don't have a real designer to draw source material from).
Perhaps I'm fortunate to have spent some time working for one of the world's best design companies (or more specifically, the Australian "Amnesia Solutions" office before they were bought and aggregated into the larger entity). At the time I didn't even have enough knowledge about design to recognise that they were good, I've only come to realise that years later.
But, I can say that I've spent enough time working side by side with "real designers" and promotion people that I've gained at least the knowledge of how truly bad I am at design and promotion.
This isn't a problem that we (the mirror ball crew) can solve ourself, and the sooner we acknowledge that the better. I certain have nothing much to say on the subject, other than to say how little I know about the subject.
Like other giant Open Problems, often the solution is not to try to solve the problem at all. Especially when you can't throw money at the problem.
Often the solution is to acknowledge you have a problem, build a suitable collaborative space for solving the problem, be inviting to outsiders or fringe members of the community that have the knowledge you need, and then do the best you can within the collaborative space to partly solve the problem until you can recruit even better people.
And at each step, you (as the community leader with no domain knowledge) deal with the tools to make sure that the output of every hour of work from the People Who Know is captured, reused and recycled.
That the work for the community is held by the community, so when your rare and valued knowledge giants run out of the time to help you and move on, you can safely stand on their shoulders.
This is pretty much the exact process that was used to solve the Perl on Windows Problem, and it's the approach that is being used to solve the Perl IDE Problem.
So I propose the following. If your blog is not stunningly beautiful, you don't get a say in how we solve our PR issues. If you don't have a blog, or engage in other forms of promotional work, you don't get a say.
You don't contribute to the solution in any useful way, so you don't count.
What you can do though, is run infrastructure. Even if the web people left for PHP, Perl still holds on to it's core community of sysadmins.
The existence of the DOZENS of websites, mirror networks, databases, automated testing infrastructure, analysis sites, forums and search engines is testament to our ability to build and run infrastructure better than most of our competitors.
So lets stop this bumbling conversation and do what we do best. Lets build and run the infrastructure so that the real people able to solve the problem. So people like Phillip Smith (and others with competant design skills) can have the support they need to solve these issues with as little process overheads as possible. of effort.
I'm happy to step up and put my server where my mouth is.
As a starting point, I'll be creating a new sub-repository off of http://svn.ali.as/ specifically for the collective storage of design materials, should you be generous enough to donate any.
My little repository management tool is simple enough that it should serve as a useful basis for design-type people to run it themselves (even if it's ugly, but I'm willing to work on that and it is templatable...).
If anyone has design material they would like to contribute, just say the word and I'll set up an account for you. I'm happy to take any contributions you wish to give.
File formats for commercial tools are welcome. The design world is still largely run on commercial tools and we need to be willing to deal with and accept that.
When someone comes along with something better than my infrastructure (I'm fully aware svn is not entirely ideal for large amounts of binary data) I'll happily stand aside and let that better thing be the place to store material.
Update: As my first contribution to making Perl better looking, the Padre team has chosen the Blue Morpho Butterfly as it's logo, and I've added the initial 16 x 16 pixel application icon for it.
Around the world with Perl
I've just finished my trip to the USA, which included adventures in Portland and San Francisco/San Jose and surrounds. I had a blast at OSCON, and will post memoirs soon. Right now I'm about to board a plane, fly back to Melbourne, do a stack of paperwork, and then fly off to Europe for YAPC::EU.
A huge thanks to everyone who brought me goodies, showed me around, took me adventuring, let me crash on their couch, took photographs, brought me food, gave me hugs, listened to my talks, commented on my talks, cycled back from hiking, took me to ice-cream, or any of the above.
More blogging when I arrive back in Australia. ;)Posted: 28th July 2009.
Tags: oscon oscon2009 perl photos
Bookmark:
I have this one day Moose class I've developed. I was supposed to give it at YAPC, but I was sick and cancelled my trip.
Here's the class summary I wrote for YAPC:
This will be an interactive hands-on course all about Moose. Moose is an OO system for Perl 5 that provides a simple declarative layer of "sugar" on top of a powerful, extensible meta-model.
With Moose, simple classes can be created without writing any subroutines, and complex classes can be simplified. Moose's features include a powerful attribute declaration system, type constraints and coercions, method modifiers ("before", "after", and "around"), a role system (like mixins on steroids), and more. Moose also has a vibrant ecosystem of extensions as seen in the variety of MooseX:: modules on CPAN.
This course will cover Moose's core features, dip a toe into the meta-model, and if there's time, explore some of the more powerful MooseX:: modules available on CPAN
Students are expected to bring a laptop, as you will be writing code during the class. You will also be provided with a tarball a week or so before the class is scheduled, which will contain a directory tree skeleton and test files.
The class ended up being taught by Shawn Moore, and I got a lot of good feedback about the class slides and exercises.
I'd really like to give the class myself, and I am trying to figure out if there's interest in the Twin Cities, MN area. I'd reserve a conference room at a hotel or on the UMN campus in Minneapolis or St Paul for the class, and I'm hoping to do this some time in August or September.
Normally, a one-day training session like this would run about $500 per person, but since this would be my first time actually giving the class, the rate would be a mere $120 per person.
In return, I want each attendee to commit to answering a short survey I'll give them after the class so I can get feedback on the class and my teaching. So there is a small price for the discount ;)
In order to make this happen, I'd need at least 5 people to sign up.
If you have questions, feel free to ask in a comment or email me or email the Minneapolis Perl Mongers list.
If you're interested in attending, please let me know what days of the week are best. Would you prefer a weekday, Saturday, or Sunday? Or maybe it doesn't matter. Again, leave a comment or email me.
If there is enough interest, I'll schedule it and then announce it on this blog and Minneapolis Perl Mongers list.
use 5.10.1;
$obj = bless { foo => 42 };
say "ok" if 'foo' ~~ $obj;
'foo' ~~ { %$obj }Currently $foo ~~ $object will die with the message "Smart matching a non-overloaded object breaks encapsulation". It would be nice to allow to bypass this by using explictly the syntax $foo ~~ %$object or $foo ~~ @$object.
I've been working on web applications for a long time, and over the years I've developed a specific approach to structuring web application code. In the past few years, this has been strongly influenced by REST concepts.
My approach leads me to make use of some existing concepts, like cookies and sessions, in an idiosyncratic way. Sometimes in discussions about web application I'll refer to these idiosyncrasies, but seen in isolation they may make no sense to anyone but me.
This entry will attempt to explain both my philosophical approach, and the practical consequences of that approach. This will not be a start-to-finish guide to building a web app (not even close).
In this context, an action is a Catalyst action. In non-Catalyst speak, it's a method that handles the response for a request. With REST, a request is identified by both its URI and its method, so we distinguish between a GET and POST to the same URI.
So what is one thing? That one thing can be one or more related database changes, some other non-database task (sending an email), or generating output for the client (HTML or something else).
An importance consequence of this "one thing" rule is that an action can either change some state on the server or return data to the client. An action never does both (ignoring side effects like logging or stats tracking). For a browser, this means that all non-GET requests are followed by a redirect to a GET.
I cannot abide an application that receives a POST, updates the database, and then returns some HTML. Inevitably, this produces crufty code filled with conditionals.
The worst examples of this are applications that have a single URI that displays a form, and then on a successful form submission display a different page at the same URI. This is a huge violation of REST, and makes the code even more crufty with conditionals.
I think the main reason people like to combine POST/PUT/DELETE handling and HTML generation is to make it easier to give feedback to the user. If the submission is unsuccessful, we want to show the same form to the user with some error messages ("password is required"). When the submisson is successful we want to give them feedback about that ("your comment has been accepted and is pending review").
Of course, providing clear feedback to users is absolutely crucial in building a usable application, but we can have our RESTful, well-designed cake and eat it too.
Contrary to some of what you might read online, you can have sessions with REST. The key is that the URI must describe the whole item being requested, including the session. That means putting the session in the URI.
But putting sessions in the URI has all sorts of well-known problems. First, it makes for ugly URIs. Second, it means you have to carry that session URI throughout the application. Third, it makes session-jacking much easier.
The second problem is easy to address. Just don't carry the session URI throughout the application! A properly built web application does not need a session for every request. If you just use the session to carry a response to a POST/PUT/DELETE, you can limit the scope of the session. I only need to include a session in the URI as part of a redirect, so my HTML pages never include links with a session.
To that end, I keep my sessions extremely minimal. There are only three things I put in a session:
All of this data shares the quality of being transient. It only needs to be used on a single URI. Once the user leaves the page in question, we no longer need the session.
This helps address the ugly URI problem. Most URIs just don't have a session, so we limit the scope of our ugliness.
This also helps address the session-jacking problem. Again, the data is transient, and is only needed on one page. Since I don't store authentication information in the session, you still need to authenticate to see the relevant page. This means that if the user gives the URI with a session to someone else, it doesn't matter.
In my applications, sessions expire after five minutes. If a user tries to go back to an old URI with a session, then the application simply redirects them to the session-less URI, which is always valid, because my applications have Cool, RESTful URIs, except for the session piece.
Taken together, this lets me have RESTful sessions with no security risk and minimal ugliness.
As a bonus, keeping sessions minimal helps force me to think clearly about application workflow, rather than just jamming tons of data into a session and using it all over the application.
Much of the literature on REST suggests that in order to be properly RESTful, you need to use HTTP authentication. Of course, anyone who's ever worked on a real web application knows that HTTP authentication is a horrible, horrible thing to deal with. You have no control over the UI, it's impossible to logout except with insane hacks, and it makes for an overall horrible user experience.
Realistically, if your client is a web browser, you have to use a cookie for authentication. Many applications, however, use a session cookie. Every request involves a session lookup, and the user id is stored in the session. This is bad and wrong. First, it requires that we have a session on every page, which I don't like. Second, it egregiously violates RESTful principles by passing around an opaque token with unclear semantics.
For me, using an authentication-only cookie is close enough to REST. With an authentication cookie, we get an effect very similar to HTTP authentication. Every request includes the authentication information, and the server checks this on every request. There is no hidden state, and while this is still technically opaque, the opaque piece is as minimal as possible.
This still isn't really RESTful, but I just don't care. I'm all for doing things in an architecturally proper manner, but only if I can provide a great user experience too. Mostly, REST and a great experience are compatible, but when it comes to authentication, HTTP auth is simply not tenable.
However, I can at least minimize the degree to which I compromise, and using the cookie for authentication and authentication only is as minimal as I can get.
These approaches, taken together, have led me to write several Catalyst plugins and/or adopt others. First of all, I use Catalyst::Action::REST. I'm primarily interested in using it to let me dispatch GET, PUT, POST, and DELETE to different controller methods internally, and it does this very cleanly.
I also use my own Catalyst::Request::REST::ForBrowsers module. This lets me define all of my internal code in terms of the four HTTP methods mentioned above, but yet still work with browsers. Browsers have two problems. First, they don't accept arbitrary document types, so I can't return a well-defined document type of my choosing, I have to return HTML, and that HTML needs to include (by reference) information about styling (CSS) and client-side behavior (Javascript). This is fundamentally unlike a notional smart REST client. Second, browsers do not support PUT or DELETE, so I "tunnel" that through a POST.
I also use the Catalyst Session plugin, along with Catalyst::Plugin::Session::State::URI. I make sure to configure the URI state plugin to only rewrite redirects, as mentioned earlier.
Yesterday, I released Catalyst::Plugin::Session::AsObject. THis enforces session discipline by providing the session as an object, not a hash reference. Instead of writing $c->session()->{errors}, I write $c->session_object()->errors(). This makes it easy to limit what data can go in the session, and gives me a much nicer API for pulling data back out. Note that all the plugin does is let you configure the class name of the session object. You can provide your own class on a per-application basis, meaning you can use it for any session data you want, not just my three items.
For authentication, I use my own Catalyst::Plugin::AuthenCookie. When I wrote this, the dominant Catalyst Authentication plugin required that the user authentication be stored in a session. Since I did not want a session on every request, I ended up writing my own plugin. Since that time, the original Authentication plugin has made use of the session has optional, but I still find it dauntingly complex for common use, so I'll stick with my own plugin.
So that's a slice of my approach to web applications. Hopefully this was interesting, and less confusing than hearing about it piecemeal. It may also help explain why I've written some of my modules, in case anyone was wondering "what use is X".
Shlomi Fish wrote in to tell about updates on the site for Perl beginners with which he's involved:
After the last news item, a lot of work was invested into Perl-Begin.org, the Perl Beginners' Site, making it even better than it used to be. Here's a summary of the changes:
Several new topical pages were added:
There's a new IDEs and tools page, featuring some Integrated Development Environments.
A testimonials page was added, with some honest-to-God testimonial quotes which are now featured in the Testimonials side-bar.
There's a new page sporting links to collections of blogs.
A ShareThis button was added to the bottom of every page for easy bookmarking and sharing.
Many writing errors (spelling, grammar, syntax, phrasing, etc.) have been corrected by Alan Haggai Alavi. He seems to have a good eye for catching such problems, and I am indebted to him.
Corrected several broken links including those to Ovid's CGI course.
We hope you enjoy the new Perl Beginners' Site and please recommend it to your friends. All content on Perl-Begin is made available under the Creative Commons Attribution Licence which allows almost unlimited re-use.
Important caveat before you read: please don't read this blog post as Ovid saying "don't evangelize Catalyst, don't write tutorials, etc." I'm not saying that. These things are important and we should continue them. I'm saying that we need a core team to get together to really understand the nature of the problem we face rather than relying on anecdotal evidence. Now on with the show.
Gábor wrote the following in his blog:
Yesterday I wrote Perception is Reality - we need a director of marketing. After a few hours I was caught on IRC by some people who said the main problem why companies are not picking Perl or why they are leaving Perl is that they cannot find good Perl programmers. As reaching programmers to learn perl is not the job for a marketing director hence we don't need a marketing person.
Gábor doesn't buy this argument and he's right not to because this reflects a very fundamental and common error of identifying the solution as the problem. The reasoning works like this: "companies aren't finding enough Perl programmers so we need to teach more programmers Perl." The error can be illustrated by a slight change to that: "companies aren't finding enough COBOL programmers so we need to teach more programmers COBOL."
The problem is identified as "not enough Perl programmers", but that's actually a component of the solution because it completely fails to address the problem of why we don't have enough Perl programmers. Since this error has been made, I really have to attribute a large part of that to myself because I've failed to communicate as effectively as I can, so I'll try to take another swing at it. Let's look at a few proposed solutions to what I'm calling P3 ("Perl Perception Problem") and see what new (or old) perceptions might result. To be fair, many of these "solutions" might very well reflect a perception of a different problem.
(Note: read the next paragraph carefully. I've been accused of trashing Perl and I expect that's because people are skimming what I write.)
Notice that I am not saying I agree with any of those hypothetical responses. Nor am I saying that those are the responses we'd receive. Further, all of those proposed solutions have merit, but the one (or ones) which will gain us the most bang for our buck are unknown. Heck, parts of the solution might be things we've not even considered yet.
One danger of proposing solutions without understanding the underlying problem is that we may come off as self-serving and we don't want to do that. To avoid it, one thing we must realize is that not all of our critics are wrong. Consider PHP. They mopped the floor with us on what many considered our home turf. They've obviously done something right. We might make vague claims of "technical superiority", but tell that to SmallTalk and Scheme developers. They also think they have "technical superiority", but superiority for what? If we don't understand the problem, we can't say what we're superior for! If our solutions are truly self-serving, they deserve to be ignored as there's a good chance we're solving the wrong problem. Just look at the RIAA. Many people involved with that really believe they're doing the right thing just as we're convinced they're not. It's easy to hear what one wants to hear.
In regards to understanding the problem, chromatic wrote:
I think we can both agree that marketing activities should reflect reality. That's why I base my conclusions [regarding the Perl 5 development process] on (and refer to) publicly accessible raw data, such as timelines, release dates, bug reports, patch submissions, commit logs, documentation, and mailing lists are all public information.
Relying on raw data is important and I'm happy chromatic is doing that, but few, if any of those sources really tell us why we have a problem (the mailing lists sound tempting, but we're really in an echo chamber). For example, let's say that you're consulting your access logs and you find many searches for the term "email". You might conclude that you have a support problem and people are looking for contact information. It might be spambot email harvesters. We're the BBC. We might have people thinking they can find celebrities email addresses online. The thing is, you can't know by just looking at raw data. You can only guess. If we can gather better information about P3, we still won't know the answer, but we can make better guesses.
To understand P3, we need to do research. Some people have mentioned the financial aspect and that's a worthwhile concern, but we don't necessarily have to spend money -- though we will need to spend time. I doubt many of us are experts in market research but maybe someone is and is willing to volunteer services? Maybe we can team up with another open-source group to facilitate this? Maybe we can find a university teaching marketing and propose an interesting research project and domain expertise? Do we have contacts for any of this? If we want to understand this problem, we can. We just need to be sufficiently creative and motivated to get to the bottom of it.
Dave Cross and I (and Rozallin Thompson, I believe) will be meeting in Lisbon at YAPC::EU 2009 to discuss this issue. I encourage others to join us. Let's tackle this beast once and for all. Our careers depend on it.
Lots of stuff happening over the past week because of OSCON.
These links are collected from the Perlbuzz Twitter feed. If you have suggestions for news bits, please mail me at andy@perlbuzz.com.
A recent thread on the Mason users list reminded me of the problems I had grokking Catalyst when I first looked at it. Raymond Wan wrote "I'm skimming over the MVC part and as my system doesn't use an SQL database, I'm wondering if Catalyst is overkill??"
Those of us who know Catalyst know that this question is based on some wrong assumptions, but it's easy to see why Raymond has those assumptions. Look at the Catalyst tutorial. In the very first chapter, it's already talking about databases and DBIx::Class.
It's easy to look at this and assume that Catalyst is somehow tightly bound to DBIx::Class or SQL databases.
The problem is that the tutorial docs really need to serve two different audiences, though both audiences are Catalyst newbies.
On the one hand, you have people with relatively little web app experience. Presumably, they know some Perl, and they do a web search for "perl web application framework". Eventually, they'll get to Catalyst and start reading about it. For those people, being given a set of standards and a full walk through of Model and View (in addition to Controller) is very valuable. It gives them all the tools they need to get started on simple web apps without having to make too many choices.
The other audience is people who have some real web app development experience with something more primitive than Catalyst. These could be people coming from a Mason-based site without any real controllers, or people who used something like Apache::Template with mod_perl, or maybe they wrote their own controllers "by hand" using the mod_perl API.
Many of those folks will already have some experience with models. To refer back to Raymond, presumably his system already has some sort of model code, it just doesn't use a SQL DBMS. Those people will look at the existing Catalyst tutorial and get confused. Isn't Catalyst flexible? Then why does it look like RoR, with all my tool choices made for me?
It took me a while to realize that Catalyst, at its core, is smaller than you might think, and you can use just the pieces you like.
The very core of Catalyst is its dispatching system. Given a URI, it selects a piece of code to run. Its dispatcher is very powerful (Chained methods are great!), and with plugins like Catalyst::Action::REST, it's even better.
Along with the dispatching system, Catalyst also provides an abstraction over the typical web app request/response cycle. The Request makes it easy to look at incoming query arguments, POST data, file uploads, and headers. The Response lets you set headers and return output to the client, whether that be HTML or a downloadable file.
Catalyst also includes engines (think "environment adapters") for a number of common web application environments, including vanilla CGI, mod_perl, FastCGI, and more. These engines make sure that the Request/Response API works exactly the same in any environment where Catalyst can be deployed.
This is a huge win, since you can write your app without worrying about the deployment environment. If you're writing an app for public distribution, it gives the installing users a choice of how to deploy.
These core pieces are really the only parts of Catalyst you have to use when you use Catalyst. If you don't want dispatch and a request/response API, you don't want Catalyst.
Catalyst (really Catalyst-Devel) also includes a fantastic single-process development server. This server can be started straight from an application's checkout directory with one command. Even better, this dev server can be told to monitor all relevant files and restart itself when any of them change. Note that this is a proper restart, which avoids all the myriad problems that afflict Apache::Reload and its ilk, which attempt to reload modules in the same Perl interpreter.
Just these things - controllers, a request/response abstraction, deployment agnosticism, and a great dev environment - are enough to make Catalyst a great choice. Ignore everything else it does and you'll still have improved your development process and improved your code.
Catalyst also does some other things ...
It has a component system which has allowed people to release a whole host of useful plugins. If you look on CPAN, you'll find things like sessions, powerful authentication, dumb authentication, I18N, and much more. If a plugin does what you need, it'll save you a lot of development time.
Note that the old "Catalyst::Plugin" system is in the process of being deprecated, but the concept of pluggable components is still core to what Catalyst is. All that's changed is the way pluggability works.
Catalyst lets you have multiple views. While many apps will just output HTML via a templating system, this flexibility is great for RESTful apps that may want to output XML, JSON, and still fall back to HTML for browsers (see my REST-ForBrowsers for some help with that).
Catalyst also has configuration file handling built-in. Personally, I avoid it, because it only works within the context of the whole "Catalyst environment". That means it's awkward at best to load the configuration outside of the web environment. I always make sure that application wide setting are available for things like cron jobs. This falls into a category of Catalyst features which are not best practices, but are probably useful for people writing their first web app.
Catalyst gives you hooks for models. Again, this is something I never use, but it's another "useful for web app n00bs" feature.
There's probably many other things I've left out. The point is that Catalyst provides a very powerful set of core web app tools, and that core is actually small.
Relative to a "my way or the highway" type framework (RoR and Jifty, I'm looking at you), it's easy to port an existing application to Catalyst. In fact, using Catalyst::Controller::WrapCGI, you can wrap an existing CGI application with Catalyst, and then convert slowly over to "native" controllers.
And most importantly, you can move to Catalyst without touching your model at all! Since many applications have the bulk of their code in the models (at least, they do if you're doing it right), this is a huge win.
Next step is to turn some of this rambling into doc patches. I think a section of the Catalyst tutorial aimed at folks coming from an "old school" web app background would be great, and would really help people like Raymond (and would've helped me a few years back).
It occurred to me today that if you look at my history as a CPAN author, you'll see that I've either written or heavily participated in every popular target of re-invention on CPAN!
I started off by writing an ORM, Alzabo. Back when I released it, this was one of the very first ORMs for Perl. Amusingly, I re-invented the whole ORM concept pretty much independently. I don't think I even heard the term until after Alzabo was released.
(Class::DBI was also released around the same time, and became much more popular.)
Of course, if you do enough database programming in an OO language, an ORM is an obvious concept to "invent", and that probably explains why there's so damn many of them. People "invent" them before they realize that there are dozens already out there.
I've also worked on a templating system, Mason. In fact, before that, I wrote a mini-templating system at Digital River, where I worked with Ken Williams. Fortunately, Ken pointed me at Mason before I could get too far into my own, which was a good thing.
Mason is also a (primitive) webapp framework, so I've worked on one of those too, as well as providing an odd patch or two to Catalyst and various Catalyst plugins.
Then there's the Perl DateTime project, which was an explicit and clearly thought out wheel reinvention exercise. The existing wheels were all kind of broken, and you couldn't easily put any two wheels on the same car, so we came up with a plan for a whole set of interchangeable wheels and assorted other parts.
Nowadays I spend a lot of time on Moose, which is one of the most recent (and by far the best) in a long series of "make Perl 5 OO better" modules.
To top off my wheel-ish history, I have a new ORM, Fey::ORM, meaning I am the creator of both one of the oldest and one of the newest ORMs on CPAN.
Truly, I am the master of wheels, both re-invention and polishing those already in existence.
Every time I provide training somewhere I like to ask my students about their background and about their reasons to attend the class.
Unfortunatelly in corporate training I am usually shielded from the students up until the first day of the class so I cannot ask this question up front. Even more unfortunate that in many such cases the students actually don't really know why are they attending the class. The answers usually go along the lines of I have to maintain Perl scripts or I'll have to write Perl scripts or even that I was told to come to the class.
These people usually bring zero enthusiasm with themselves and will probably invest negative energy in actually making sense of what I am teaching.
Hence it is really refreshing to teach advanced classes such as the Test Automation using Perl where people usually already know what they want and why do they attend the class.
It is especially fun to teach classes around YAPCs or Perl workshops as then the students are also community aware. Many attend Perl Monger meetings, are using Perl Monks or chat on IRC.
That, along with the fact that this is a totally new class makes the Perl 6 training that I am going to give in Lisbon very promising. At least to me.
As an official version of Perl 6 has not yet been released and you cannot convince your boss to use it in production it was even more interesting to see why people take this class? I asked them my usual questions and most of them already answered.
As I think the answers can be interesting to others as well let me share some of them with you without any personal identification.
I'd really like to thank those who took the time to answer. I think this can provide a great feedback not only to me for the remaining preparations but to the Perl 6 developers as well.
Here some of them really got carried away. I think I'll have to reduce their expectation quickly. Before we start the two days training.
Yesterday I wrote Perception is Reality - we need a director of marketing. After a few hours I was caught on IRC by some people who said the main problem why companies are not picking Perl or why they are leaving Perl is that they cannot find good Perl programmers. As reaching programmers to learn perl is not the job for a marketing director hence we don't need a marketing person.
Let's set aside all the other areas where professional marketing could help the Perl world and let's focus on the issue of the immpossibility to find good Perl programmers.
I think if we compare it with Python, Ruby and to some extent even with PHP the situation isn't better there either. I don't think it is a problem specific to Perl. On the other hand if you look at Java, people don't have the same problem.
Java does not have a lack of programmers
There are several reasons for this. One of them is that most of the universities teach Java but not any of the dynamic languages. The other reason is that in Perl/Python/Ruby they are looking for good programmers while in Java they are looking for programmers. I am not sure I know what they mean by that but that's usually the phrasing I hear. The explanation I get sometimes is that in Java you need one expert and you can get by with many beginners. In dynamic languages you usally need a lot less people but they must be really good if you want maintainable code. This might be less so with Python and Ruby but the difference is neglectable.
So are Python and Ruby in better situation than Perl?. If a company or a team in a company decides to use one of the dynamic languages they need to overcome the lack of Perl/Python/Ruby programmers. One of the ways is to take some of the Java or C# programmers and teach them the language. Either by formal training or by letting them learn on their own. In either case it requires an investment on the side of the company and the willingness on the side of the programmer. Perl has a clear disadvantage here that boils down to three issues.
The first problem can be reduced or eliminated by improving Perl, the language or by the various projects such as Enlightened Perl, Modern Perl or by creating a really good IDE for Perl.
The second issue might be also changed by the above improvements but that is already mostly a question of perception.
The third issue is clearly something we programmers and system administrators don't know how to deal with. There is where we need the professional help.
Since my last use.perl post about Perl's perception issue seemed to generate a moderate amount of interest, I decided to do a bit more research about, well, research. In order for it to be effective, a clear goal needs to be established. I think a first pass at said goal would be "creating the perception of Perl as a modern, useful language." If that alone could be done, most of the other issues might take care of themselves. It will be a tough thing, though. I still talk to people today who don't know Perl has OO programming. We've had OO for fifteen years, but when people still don't know this, we've failed miserably (of course, this might not be a widespread perception problem).
Once a goal is defined, there are two questions which need to be asked before market research is feasible:
In other words, this is something which might need to be a joint effort between P5P and either TPF or Enlightened Perl. Polaris research has a handy guide explaining some market research basics. This is something which could be done. Whether we'd do anything with it or not is the real question.
At times I feel like some people in the Perl community are hoping Perl 5 dies a quiet, dignified death. Do we care about Perl 5, or are all eyes on Rakudo? I certainly hope people still feel there's merit to Perl 5, but if the Perl community gives up on it, why shouldn't the rest of the world?
This is my other presentation from OSCON. I gave this one with all-round lovely guy Yoz Grahame from Linden Lab, and Freebase developer platform czar Jason Douglas. We presented on Friday morning in a really tough slot — there were half a dozen other talks that I wanted to be at! — so attendance was a little light.
There were some upsides though:
So, for those of you who couldn’t make it for whatever reason, I hope this will be useful!
Quick notes:
Yoz first gave his “Folk Programming” talk at a Wiki Wednesday in 2007 and I was there and I thought it was great, so we’ve been talking a lot about the concepts on and off since then.
Basically our presentation here is the 15 minute version of his longer talk, followed by me talking about how the concepts apply to open source software, and then Jason talking about how we (i.e. Freebase.com) applied the concepts to our new app development platform to help encourage adoption and a strong developer ecosystem.
Yoz starts out talking about he first learnt to program (on a ZX Spectrum), and how beginning programmers often cargo cult as a way of learning. Cargo cult programming is often stigmatised, but it can be seen as a kind of folk process, like folk music or folk stories, and it’s a good way to get started.
He then gave a bunch of examples of folk programming in action, from an MIT study of children passing around little games on Dreamcast modules, to Yahoo Pipes and Second Life. He identifies five things necessary for a real folk programming environment:
I then took over and slightly reframed his list for the open source world:
I looked at some examples including github, Amazon EC2, and Google App Engine, and looked at them to see whether they provided folk programming environments for open source development. What I found is that each provides some elements, but none provides all of them.
When Jason and his team at Freebase were starting work on our hosted app development platform, Acre, we got Yoz in to give a lunchtime talk about Folk Programming, and used a lot of his ideas in developing our platform. Jason walks us through how Acre has BSD licensing, one-click cloning, and interacts with Freebase’s huge store of open data.
Some of the demos he shows include “data games” for Freebase contributors to more quickly add data to Freebase itself, which started off with a single app called Typewriter and quickly evolved into a cluster of apps and eventually a library and toolkit to allow people to quickly build more. (Some were built at our recent Hack Day, for example, including one to easily find CC-licensed images to attach to Freebase topics.)
Another set of apps he showed were clustered around the concept of lists in Freebase, starting with an app to create lists of anything and then showing a recommendation website and an app for TV fans all based on the same underlying data.
Jason closed by talking about some of the challenges we’d faced and future directions for folk computing in the open source world. Challenges included version control and collaboration (because there is a time when you’re no longer experimenting and need to work together on things rather than fork), and the state of in-browser development tools like syntax-highlighting code editors and Firebug-style console debugging for server-side apps.
Questions from the audience included “Have you seen people writing large apps in folk programming environments, in the thousands or tens of thousands of lines range?” to which the answer is “yes” — LambdaMOO and SecondLife being enormous, and some of the Freebase apps being in that linecount range already despite only having properly launched about two weeks ago.
“How do you handle security?” was the second question, which Jason answered at length. Some of the parts of the Acre security model included sandboxing on the server; having each app on its own subdomain to prevent cookie hacks; and everything being sanitised by default to protect against cross-site scripting attacks.
What Does the Outside of Perl Look Like? asked Ovid and listed some of the negative images Perl probably has from the outside. I commented on his blog but let me write here more in detail.
When I did my MBA I also took classes in marketing that I barely passed.
So I am an authority to say that
We don't know how to deal with this!
One of the things we learned there is that Perception is Reality.
I remember two interesting case studies. In one case study people were given two shopping lists and had to rate the women who had those shopping lists. The only difference was that in one list coffee was listed in the other one Nescafe. The group whose list contained Nescaf?, described the woman as lazy, unorganised, careless, bad wife and mother.
So the Nescafe addressed the perception and their message changed to women who buy instant coffee can spend more time with their family.
I even found a recent blog post about this: How Nescafe Convinced Women
The other one was I think about Exon, the oil company. Due to its heavy investement in telling about its green projects for it was preceived as one of the most environment friendly companies.
In this blog entry I found some comments that would indicate that they are far from really being environment friendly.
Of course I am not suggesting that Perl is either bad for the environment or that we need to be sexists but that changing the frequency of releases of perl or changing the language itself won't lead to a change in perception. In fact probably we don't need to make any changes in most of the areas where Perl is preceived negatively? (I love the questions of some people: Can perl access web sites?). What needs to be changed a lot more is the perception.
The other thing I wrote in in that response to Ovide is that because
perception is reality, you won't really be able to get an answer to
a straight question of Why do you think Perl is unreadable.
That's just how it is. That's reality. For the responding person.
In any case I think
I think we need someone who has marketing background and who is being paid to improve the marketing message of Perl. I don't think it can be done by volunteers of TPF of EPO. This is not perl coding we enjoy to do, nor do we know how to really do that.
This person might be employed by TPF or by EPO or by some organization that has vested interest in the well being of Perl. One of her first tasks will probably be a market research finding out what are the perceptions and what would be good to be as perceptions. Such research probably can be done relatively quickly but marketing is a long term process. So whoever sets up such a position needs to be able to commit to long term work both in supporting and in financing the marketing director.
The video from Tuesday night’s Ignite OSCON is up! Mine is the second talk, starting around eight minutes in. It’s on the subject of Five Geeky Things You Can Do With Textiles.
The five things, by the way, are:
If you weren’t at OSCON this morning, here is what I spoke about in my keynote, Standing Out in the Crowd. I’m including most of the key visuals, so my apologies for the image-heavy post. I’ll also be uploading to slideshare.net (with voiceover I hope) and I’m told there will be video up at the OSCON blip.tv channel in due course. (ETA: it’s up.)
Anyway, on with the show.
They asked me to speak about women in open source, and most specifically about two recent open source projects that have a majority of women developers. But first, I need to give a bit of context.
This is a normal sort of open source project. I’ll give you a minute to spot the women in the picture. Sorry, make that woman. She’s on the right. Can you see her?
This is normal for open source.
In 2006, the FLOSSPOLS survey (a broad survey of open source usage and development, funded by the EU) found that only 1.5% of open source contributors are women.
In 2007, my survey of the Perl community — both contributors to Perl and users of Perl — found about 5% women. The Drupal community is doing even better, around 10%. And in technical professions and in computer science in universities, reports vary: it can be anywhere from 10% to 30% depending on who you ask and how they’re slicing it. Let’s say 20%.
So in most technical communities, women are in a minority. But in open source communities, we’re in an even smaller minority — by a factor of about ten or more.
So what does it feel like to be a woman in open source? Jono Bacon, at the Community Leadership Summit on the weekend, said — addressing the guys in the room — that if you want to know what it’s like to be a woman in open source, go and get your nails done at a salon. He did this a week or so back, and when he walked into the salon he realised he was the only man there, and felt kind of out of place.
Another example someone suggested is walking into a sports bar on game night wearing the wrong team’s jersey. It can be the most friendly sports bar in the universe, but you’re still going to feel pretty awkward.
So as a woman in open source, it can be a bit like that. You walk into a space, and you feel like you stand out. And there’s enormous pressure to perform well, in case any mistake you make reflects on everyone of your gender.
And that’s just the subtle stuff. There’s also more blatant problems, like sexist jokes, pornographic presentations at conferences, harrassment, and even death threats against women in the open source community. I wish I was joking, but I’m not.
The FLOSSPOLS survey asked open source contributors whether they had witnessed sexism, harrassment, or discrimination in our community. Here’s what they found:
80% of women had noticed sexism in the open source community. 80% of men never noticed anything. That’s a pretty big gap.
Well, enough of this depressing stuff. Let’s talk about something more cheerful.
There are two new open source projects I’ve been involved with this last year, which are have a majority of women developers.
The first is the Organization for Transformative Works’ Archive Of Our Own (AO3 for short). The OTW supports creators of fan fiction, fan videos, and all other creative things that fans do, like this:
They’re lobbying against the DMCA, they have an academic journal called Transformative Works and Cultures, and they’re working on creating a fanfic archive, by fans and for fans, that won’t cave to over-enthusiastic takedown notices or pressure from nervous advertisers.
When the OTW decided to create the archive, they set up an open source project and they went out recruiting developers. But not necessarily experienced programmers — just anyone who was interested in taking part, and had a passion for making this project work. In fact, they made an effort to include non-programmers early on, and decided to choose a language based on what was easiest for non-programmers to learn.
You can see their process in this LiveJournal post. Basically they said flip a coin: heads is Ruby, tails is Python. Go learn that language to a basic level, install the development toolkit on your computer, and write a simple “Choose your own adventure” style program with a couple of conditional statements in it.
They got about 70 people to do this — all non-programmers, and almost all women from the fan community — and used their feedback to choose Ruby as their programming language of choice.
The AO3 project now has about 60k lines of Rails and Javascript and HTML and all that (the count is for non-comment non-whitespace LOC). There are over 20 developers who have submitted code, and every single one of them is female. (There are some men in other parts of the project, like sysadmin, but as far as I know none have submitted code to AO3 itself.) I’ve put together a Google spreadsheet with vital statistics about the project.
The second project I wanted to talk about is Dreamwidth, a fork of the LiveJournal codebase, which means it’s a blogging and community platform. It was founded by two ex-LJ employees, one male and one female. It’s currently in open beta and has about 40,000 users, 210,000 lines of Perl etc, and 40ish developers of whom 75% are female. You can check that same Google spreadsheet for more stats.
Like AO3, Dreamwidth makes a point of encouraging new developers, who they call “baby devs”. For instance, there is an IRC channel called #dw-kindergarten where they can go for non-judgemental help. Dreamwidth also provides hosted development environments called “Dreamhacks” for anyone who wants them.
From the very start, Dreamwidth has had a diversity statement that welcomes developers from every walk of life:
We welcome people of any gender identity or expression, race, ethnicity, size, nationality, sexual orientation, ability level, religion, culture, subculture, and political opinion.
Their disability/accessibility policy is also great, and has led to a high proportion of developers with disabilities:
We think accessibility for people with disabilities is a priority, not an afterthought. We think neurodiversity is a feature, not a bug.
The result is an open source developer community that looks like this:
Or, to be more exact, like this:
I surveyed women on the Dreamwidth and AO3 projects and asked them about their experiences. You can read a fuller report of their responses on my earlier blog post, Dispatches from the revolution.
One of the first things I asked them was whether they had previously been involved in open source projects. They gave answers like:
I’d never contributed to an open source project before, or even considered that I could.
I didn’t feel like I was wanted.
I never got the impression that outsiders were welcome.
I considered getting involved in Debian, but the barriers to entry seemed high.
Those who got a little further along still found it hard to become productive on those projects:
It’s kind of like being handed a box full of random bicycle parts: it doesn’t help when you don’t know how they go together and just want to learn how to ride a bike.
People without a ton of experience get shunted off to side areas like docs and support, and those areas end up as the ladies’ auxiliary.
But on Dreamwidth and AO3…
What I like most is that there isn’t any attitude of “stand aside and leave the code to the grown-ups”. If there’s something that I’m able to contribute, however small, then the contribution is welcome.
And this one, which is my favourite:
Deep down, I had always assumed coding required this kind of special aptitude, something that I just didn’t have and never would. It lost its forbidding mystique when I learned that people I had assumed to be super-coders (surely born with keyboard attached!) had only started training a year ago. People without any prior experience! Women! Like me! Jesus! It’s like a barrier broke down in my mind.
So, what can we learn from this? Well, one thing I’ve learnt is that if anyone says, “Women just aren’t interested in technology” or “Women aren’t interested in open source,” it’s just not true. Women are interested, willing, able, and competent. They’re just not contributing to existing, dare I say “mainstream”, open source projects.
And this is great news! It’s great news for new projects. If you are starting up a new open source project, you have the opportunity to recruit these women.
Here are some tips based on what I’ve seen on Dreamwidth and AO3.
From the very earliest days of your project, recruit the diversity you want to see. The first two or three members of the project will set the tone for what follows. People will look at the project and think, “Is this somewhere I can see myself fitting in?”
If you’re working on a desktop app, recruit desktop users. If you’re writing a music sharing toolkit, recruit music lovers. Don’t worry about their programming skills. You can teach programming; you can’t teach passion or diversity.
Get yourself a Code of Conduct or a Diversity Statement, or possibly both. Use it to set expectations around how your community should treat each other. And don’t just mouth the words. Stand by your policy, and uphold it.
Put up a wiki and make sure it contains plenty of information for new developers. Set up your bug tracking system to flag easy tasks for beginners to do. You could even provide a hosted development environment like Dreamwidth’s Dreamhack system, with all the tools and libraries and configuration already in place, to get people hacking as quickly as possible.
Communicate. Not just to existing developers and users, but to newcomers and outsiders as well. Take the time to show what’s going on inside your project — teams, sub-projects, internal processes and communication — so that people who are interested can envision what it would be like to be a part of it.
Of course, all these things are easy to do when you’re starting a new project. It can be harder to implement them in an existing project, maybe one that’s been around two or five or ten years, because you have a lot of inertia in place. Which is not to say you shouldn’t try, but I recognise that it can be difficult and frustrating.
So here are some things that you can do as individuals, no matter what project you’re on.
If a woman joins your project, don’t stare. Just welcome her (politely but not effusively, because that can be creepy), give her any information she needs to get things done, and thank her for her contributions.
Large or small, code or docs or bug reports or organisational tasks. All are valuable to your project. Say “thank you”. You don’t have to be the project leader to do this; anyone can do it, and it makes a big difference.
If someone’s being an asshole, call them on their crap. How do you tell if someone’s being an asshole? Well, if there’s a naked woman on the projector screen, that’s a good sign.
Let them know that their behaviour is making people feel unwelcome, and that you don’t like it.
Pay attention to your own behaviour and the behaviour of others. This is possibly the hardest piece of advice I’m going to give. You’re not used to noticing this stuff. 80% of you haven’t noticed the sexism in our community.
As men, you are able to glide through life ignoring these things. If you are white, and straight, and speak English, and are university educated, there are a bunch of other things you’ve been able to ignore so far, too. I’m asking you to try not to ignore them. Keep your eyes and ears open and be aware of how things feel to people who don’t share your privilege.
So, those are a few tips I’ve picked up from Dreamwidth and the Archive Of Our Own. They’re only two projects, and they’re both still new, but I think it’s a start.
I’d like to leave you with a parting thought.
What do you think would happen to this picture if we got more women into the open source community?
Do you think some of those little blue figures will turn pink? Do you think there will be less of the blue?
That’s not how it works. Any step you take to improve the diversity of your project will work to increase the developer pool overall.
We’re not far enough along in our plans for world domination that we can afford to turn anyone away.
Thank you.
If you attended OSCON and saw my keynote, please take a moment to rate it. You might also be interested in this write-up by Robert Kaye on the O’Reilly Radar blog.
ETA: Both Mark Smith and Denise Paolucci (the two Dreamwidth founders) have posted about their experiences at OSCON and conversations they had there about Dreamwidth and its development processes and community: Denise’s post, Mark’s post. Very much worth reading. Check the comments too.
ETA2: Comments are moderated. Anonymous comments are not permitted. Anything excessively argumentative, vicious, or personal will not be posted. I don’t mind constructive criticism or differences of opinion, but I won’t take abuse. Thanks.
Every year at OSCON I come home with a head full of ideas, and better yet, a huge list of new things to work on. Since the book is now done, and OSCON is now over, there's a chance I could work on them.
For those of you leaving OSCON, what tasks did you just assign yourself in the past week?
I'm sure most of you are familiar with the debates which have consumed the Perl community lately. I really don't want to get into that other than to say that supporters on all sides have valid reasons for their views and everyone has something to contribute. Rather than speculate on what some problems are -- oh, and believe me, I can! -- I think what's more important is to figure out how to obtain information relevant to these considerations. In fact, I have several "perceptions" I think need to be better understood.
How did these perceptions arise? What is the source of them? Are people merely parroting what they've heard or do they have first-hand experience? What would it take to change people's minds?
These perceptions are a few things I hear constantly from those outside of Perl. Inside of Perl, we have different perceptions, but outside of Perl, I hear wildly different stories. Many of those stories don't match my experiences (see item #6 above), but in order to understand where the perception problem is coming from, it would be useful to better understand why people have these ideas (rather than play "pin the tail on the scape goat").
What I am thinking would be useful is for someone with some expertise in creating surveys to figure out a way to start collecting this information. So when I created the testing survey, there was a lot of fascinating information, but given my lack of expertise in creating surveys, it was rather limited (brian d foy helped quite a bit in expanding that survey beyond my initial questions).
I'm very keen on understanding the perceptions that programmers and organizations have about Perl. If we identify strong, persistent currents in thought, that might give us a clue as to what a good way forward would be. A lot of people are making blanket assertions about what Perl needs for the future. Some are quite reasonable (how do we tighten up the P5P development process), but many are based on hunches (Perl definitely needs to do X!!!).
Anyone interested in picking up this gauntlet and trying to better understand what we need to do? (This would involve potentially surveying companies in addition to individuals).
Read more of this story at use Perl.
Frew Schmidt has read the first four chapter of the new Catalyst book and gives some first impressions:
So far though, I would say that the book is better than most programming books. Really, a lot of programming books need to be more like this, instead of focusing entirely on the arcana of one framework they should help you be a better programmer overall.
Read more of this story at use Perl.
Even though there's still 2 days to go until Padre's 1st Birthday Party Hackathon is due to begin, there's already something of a party atmosphere and the commit rate in the last week has skyrocketed.
In particular, Gabriel Vieria (who has only been contributing for a couple of weeks) has decided to take the directory browser as his own personal fiefdom.
Half a week and 1500 lines of code later, he's turned it into what Swarm hacker Andrew Bramble appropriately describes as "Double plus awesome fwoar!".
Not only is it orders of magnitude faster, he's added icons, allowed the expansion state to be cached and restored across changes in project context, and managed to clone Komodo's recursive nested filter search in a single night.
As with many previous new contributors, not only does the additional effort accelerate the rate of change generally, but his concentrated effort in one specific area drivers development in other areas. In this case, the awesome directory/project browser should drive improvements in Padre's "Implied Project" support, in particular adding things like zero-configuration detection of hidden directories that can be culled out of the directory tree.
With the wallclock time for the last 50 commits down to only 5 hours, it's looking like it will be an awesome weekend hackathon.
If you'd like to join us, you'll find us in #padre on irc.perl.org.
Found in an Ivan Krstic post about Sugar and the OLPC:
In truth, the XO ships a pretty shitty operating system, and this fact has very little to do with Sugar the GUI. It has a lot to do with the choice of incompetent hardware vendors that provided half-assedly built, unsupported and unsupportable components with broken closed-source firmware blobs that OLPC could neither examine nor fix. [...]
We had an embedded controller that blocks keyboard events and stops machine suspend, and to which we — after a long battle — received the source, under strict NDA, only to find a jungle of nested if statements, twelve levels deep, and no code history. (The company that wrote the code doesn’t use version control, see. They put dates into code comments when they make changes, and the developers mail each other zip files with new versions.)
Haha. Been there, done that. Sometimes it’s great not to have to work with custom hardware anymore…
I’ve noticed that many recent discussions about changes to Perl 5 core development spiral off into circular discussions about deprecation cycles, definitions of compatibility, source code branch strategies, timeboxing, volunteer motivation, and so on.? Consensus has been slow to converge and conflict is in the air.
I think the problem is that people have different goals for Perl 5.? Some of these goals mesh well, but others conflict.? Some of the remedies support many goals, some only one or two. And some remedies work against some goals.? Placing some goals above others has big implications for changes to the development process and for priorities for new development.
I’ve started brainstorming some of the goals that I’ve heard or read or imagine might be out there.? Maybe if we can find some common ground around some goals, getting to common ground about methods won’t be quite so difficult.
Read these over and tell me what you think, here or on a blog or site of your own.? Do? you agree with some?? All?? None?? If you had to pick just one or two, what would it be?? How would you know if they happened? What would it look like? What would be different?
What do you want Perl 5 to be?
My own view is a mix of several of these, and I’ll try to articulate that further in a future post.
I'm sitting in the communication lobby on the fringe of the p5p meeting discussing potential ways of doing releases for Perl 5. It's quite a brain-dump of Perl 5 names: Chip Salzenberg, David Adler, Patrick Michaud, David Wheeler, Robert Spier, Paul Fenwick, Jacinta Richardson, Tim Bunce, Michael Schwern, Ricardo Signes and Jesse Vincent.
Here are twelve brilliant programmers in the Perl world, and they're talking about a rancorous topic, but there's no anger, no animosity. The talk is honest and frank, but the benefit of having everyone present is clear. It makes me happy to see.
In sessions today, Jacinta's survey of Perl frameworks was great, in that it was pragmatic and aimed directly at the programmer wondering "What should I do my next talk in?" I skipped out early on Tim Bunce's Devel::NYTProf talk, but I've seen a couple of tweets being very impressed with it.
I have, in addition, introduced a new method of philosophizing on the basis of numbers. -- Pico della Mirandola, Oration on the Dignity of Man
Greetings all.
In an effort to get PPI off the FAIL 100 list, and to get a ton of Elliot Shank's improvements (including support for Perl 5.10) out to production, I've just produced what I expect to be the first serious release candidate.
http://svn.ali.as/cpan/releases/PPI-1.204_06.tar.gz
As well as a range of new object model classes (such as PPI::Statement::Given) this new release contains a change to the way that for ( .. ) braces are handled.
PPI will now distinguish between two types, either a PPI::Structure::List for ( @list ) and a PPI::Structure::For for ( $a = 0; $a++; $a < 10 ).
This, and a few other minor changes, have some potential to break existing PPI-based applications (or their test suites). Perl::Critic, for example, will be doing a synchronised release at the same time as PPI to maintain compatibility.
If you maintain any PPI-based modules you may want to install this release candidate and look for any potential breakages.
I'll do some downstream testing myself, but of course there's a limit to how in depth I can go.
I expect to release the new PPI in 1-2 weeks.
After a break of almost 20 years I recently started to use IRC again and found it to be a great tool for communication in a development team. I was surprised to see it is much more mainstream than what I thought. One of the drawbacks of IRC is that because of the line based communication you cannot show a piece of code to someone. This is nicely solved with pastebins.
That too has drawbacks as the pastebin shows a static version of the code on a public web site and if you make some changes you need to paste it again. It needs a bot on the IRC channel to send the URLs as they are pasted or you have to copy them manually to the channel. It is still a lot of clicks you have to make.
Before looking at a solution let's take a look at two vastly different working environments. I think I can safely say that there are two very separate worlds of software development. It is not black and white, this is just two end points of a spectrum but it is interesting to think about this a bit.
In one group there are the people who live in their IDE. It provides them with version control, integrated language help, compilation, execution and debugging environment etc. Everything and the kitchen sink. They only leave their IDE for reading e-mail and browsing the web.
On the other hand there are the commmand line people who have lots of relatively small tools for different tasks. These people usually make extensive use of the command line which implies they are mostly Unix/Linux/OSX users though I am sure there are similar people on Windows as well. In a very small minority.
I think we can safely say that there is some corellation between the development environment and the operating system they are using.
I am not sure but I think the command line people have a lot more power in their hands. They can go a lot deeper in their work environment. They have to understand more of the guts of the system. Both their operating system, their work environment and probably the application they are developing too. There are also a lot less of these than the IDE people.
The IDE people also have a huge advantage. They have everything integrated in one place - hopefully in a smooth way - so they don't have to waste their time on learning and understanding each one of the tools in use.
In the long run I think people who know how to use the command line can be more efficient than people using IDEs but in reality most people don't have the time to invest in the longer learning curve of the many tools they need to use.
Frankly also many of the command line people would like to see more integration between their tools. No surprising that they integrate tools into vim and even take features from Padre to integrate them into vim.
One of the things Padre could do to improve the development environment is to provide an integrated way to help each other write an application.
Padre already has a Nopaste plugin but that still reuses a pastebin web site. Work is being done to integrate IRC communication. Going further the Swarm project is aiming to allow you to share a whole document with others - initially for viewing only but later maybe for cooperative editing.
What we could have is to allow groups of people - either in an open source project or within a company - to setup an IRC channel, let the Padres communicate with each other somehow and let the people share documents. E.g. If I want to show a snippet of code to ask for advice, I could select it, click on "show to others" that will bring up a window in their Padre with the code snippet. They don't have to know about pastebins and bots and IRC channels. They don't need to open their browser. Right from their IDE they can communicate with others and share docments or parts of documents.
Another thing Padre should help is transition peple from pure IDE users to partially use the command line. This way they can slowly take advantage of the features available on the command-line.
E.g. In the Ecliptic plugin of Padre there is an open-resource window - the idea was take from Eclipse - that allows you to type in a filter and it will show you the list of files in your project that match that filter so you can open that file. No need to browse through the directory structure of the project.
My issue is that I often edit files in several projects at once. At a minimum I have my blog in POD format in a place totally separated from the other projects. So when I want to write a blog entry I have to travel all the way through the file system to find the blog directory and the file I need to edit. On the command line or in vim it would be much simpler both becaus of the tab-completition and because it can remember tons of files I opened in the past.
The "Recent files" menu option of Padre is no help as it can only remember and show a very limited number of files. What I'd like to have is the same history feature the command line or vim has. It would be nice if the open-resource window remembered the 1000 most recent files and showed them - after filtering - so I can easily open one of them.
This will bring a command-line type feature to the IDE.
In a similar way we might have a window to filter and show the most recent commands. E.g. if I commit a change using the SVN integration of Padre it should remember the actual svn command and in the most-recent-commands window it should list it.
ps. I think I need to mention Perl in the post in order to be displayed on the IronMan web site...
Perl has broken backward compatibility in the past where I judged very few people would be affected.
-- Larry Wall in P5P.
"Living hell" would accurately describe my development for the past two days. Multi-screen stack traces generally starting with this:
DBIx::Class::Schema::throw_exception(): DBIx::Class::Row::throw_exception(): DBIx::Class::Row::throw_exception(): DBIx::Class::Row::throw_exception(): DBIx::Class::Row::throw_exception(): DBIx::Class::Row::throw_exception(): DBIx::Class::Row::throw_exception(): DBIx::Class::Row::throw_exception(): Unknown error
Compilation failed in require at /home/ovid/pips_dev/work/Pips3/branches/rights_modeling2/deps/lib/perl5/Class/C3 /Componentised.pm line 126.
at /home/ovid/pips_dev/work/Pips3/branches/rights_modeling2/t/lib/PIPs/Test/More.pm line 136
PIPs::Test::More::__ANON__('Unknown error\x{a}Compilation failed in require at /home/ovid/pip...') called at /home/ovid/pips_dev/work/Pips3/branches/rights_modeling2/deps/lib/perl5/Class/C3 /Componentised.pm line 126
... and so on
Eleven in the morning and I already need a drink.
I just read this review on eWeek about NetBeans integration with Project Kenai. As I understand Project Kenai is some repository similar to Sourceforge and Google Code just by Sun. Obviously it has a web based interface but now, the latest version of the NetBeans IDE provides an integrated interface to it. So right from the IDE you can search for projects, check them out from their repository and start working on them.
This got me dreaming.
Padre will soon provide a way to search CPAN and install modules right from the GUI. As some of the modules already provide links to their version control system listed in the META.yml we could integrate that too to Padre. So when looking at a module Padre could already recognize if it has a link to a version control system and offer you to check it out.
If there is no version control system, Padre should be able to setup one for you and import the latest version of the module so you can start hacking on it right away. There was an article in the Winter 2008 edition of The Perl Review on how to do this with Git. We could take that approach and make it integrated into Padre. We could also implement a similar process for Subversion.
I think something like this would be very interesting. (Perl 6 pseudo-code, obviously)
<script
grammar="http://rakudo.org/grammar/mozilla/perl6/"
version="1.2"
authority="https://rakudo.org/grammar/authority/">
use Mozilla::Document;
subset SafePassword of String where {
$_ ~~ m/<:digit:>/
&& $_ ~~ m/<:punct:>/
&& $_.length >= 8
};
sub safe_login (Mozilla::Document::Form $form) {
my Mozilla::Document $document .= new();
# $_ defaults to last element
$form.password ~~ SafePassword or $_.error("Password Insecure!");
}
</script>
In other words, if the sandbox is secure enough, one could use the Parrot Compiler Toolkit to write a set of bindings to allow you to write your client-side in any language you like, include bespoke languages, if desired. I think this could be a very compelling feature. Don't like languages available for client side browsing? Fine, write your own with the Parrot Compiler Toolkit and if the browser sees an unknown grammar from a safe authority, it downloads it. Otherwise, it can prompt you with the authority information, depending on your security setup.
Read more of this story at use Perl.
On the off chance you're going to YAPC::EU, it's not too late to start learning a bit of Portuguese. Livemocha is a free social networking site based around learning new languages. Regrettably, their Portuguese (Brazil) course is more extensive than their Portuguese (Portugual), but hey, wouldn't it be nice to say a few words in the language? I'm brushing up on my French there (and hence won't have much time for Portuguese) and think they've done a great job.
Jim Brandt of the Perl Foundation writes for input from the community.
At OSCON this year, on Wednesday night at 7 PM in Exhibit Hall 3, I'm participating in a Language Roundtable Session with representatives from some of the other popular open source languages. We're going to talk about some of the best and worst features of our preferred languages and how well they perform with different types of application development.
http://en.oreilly.com/oscon2009/public/schedule/detail/9380
I know why I love Perl, and there's plenty of new activity in the "modern" Perl community to talk about. This is a great chance to let everyone know what great strides Perl has made. It's a chance to get people to take an up-to-date look at Perl. However I don't want to waste any time on "worst" features in other languages.
So what are the best features of Perl today? What do you want the OSCON audience to hear about?
I’ve recently seen or heard a lot of discussion about how to make Perl and CPAN even better, but I feel strongly that we need a more a formal definition of packages, modules and distributions so that there is common language and understanding for the current semantics.? Only then, do I think we can have meaningful discussions of potential changes.
With that in mind, here is my best understanding of the as-is state of Perl 5.
I plan to use these definitions going forward as I discuss the evolution of CPAN, so I would welcome any feedback on whether these definitions seem consistent with how Perl 5 and CPAN work today and whether the “well-formed” designations? are clear and appropriate.
After 6 months working through legal and technical hurdles with Microsoft Australia and their hosting partner Emantra, and a month of beta testing, I'm happy to report that the CPAN Testing Laboratory is now open for general access.
The initial incarnation consists of 7 virtual hosts with the following operating systems.
1. Windows XP Professional
2. Windows Server 2003 32-bit
3. Windows Server 2003 64-bit
4. Windows Vista Ultimate Edition
5. Windows Server 2008 32-bit
6. Windows Server 2006 64-bit
7. Windows 7 Beta
These machines are intended for ad-hoc testing and development by CPAN authors, and currently all of them are configured for exclusive access by a single user (to allow for complete flexibility in testing).
I've set up a specific #msopensource channel on irc.perl.org to help authors coordinate their usage of the different machines and for administration purposes. If you would like a login to the testing lab, you can either join the channel of (If I'm not there) email me via my adamk@cpan address.
For the time being, to create an account I'll need your CPAN id, full name, and a unique password that you don't mind me (or any other admins) seeing.
Because the current lab is quite experimental (both for us and for Microsoft), the current plan is to observe how people use it over the next 6 months and improve it incrementally as needed.
Finally, thank you all for your patience over the last 6 months, and thanks in particular to the guys at Microsoft that helped spearhead this effort through to completion. Of particular mention out of the dozen people involved:
Cristoph Otto, Software Test Engineer, Microsoft Open Source Software Lab
Anandeep Pannu, Senior Program Manager, Microsoft Open Source Software Lab
Sam Ramji, Senior Director, Microsoft Open Source Software Lab
Lawrence Crumpton, Platform Strategy Manager, Microsoft Australia
Jorke Odolphi, Web Platform Architect Evangelist, Microsoft Australia
Portland Adventures II
Today is my last day in Portland, and wow, what an adventure it's been. Friday was spent writing slides, relaxing in tea-houses (green mango bubble-tea with wifi rocks!), and a trip to Beer and Blog at the Green Dragon.
Beer and Blog I was particularly pleased with on many different levels. Ua had invited me to this fine establishment at last year's OSCON, and this represented me arriving, albeit a year late. I had a chance to socialise with cool new people, although I didn't realise just how cool some of them are until I did my research.
What made Beer and Blog really special was that during one of my conversations there was a comment that, "there's another Australian here, he's only just moved over". That other Australian was Mike McClure, with whom I went to University, but had not seen in about a decade!
Oh yes, Beer and Blog also had free beer. That also made it special. ;)
That evening was I was given a tour of Portland by Schwern, Kate, Ua, and Nick. That included beer, dinner, a walk along the river, and a trip to Voodoo Doughnuts. I'd been assured many times that my life would not be complete without having gone to Voodoo Doughnuts, and having been there, I can agree.
While I've been in Portland, Selena has been a wonderful host, and I'd felt that I'd been a terrible guest. Selena is a morning person, and I routinely came home late, slept in, and disappeared at odd times for ice-cream or doughnuts. On Saturday morning, I was determined to buck this trend.
With thanks to Jacinta and Schwern who went on a secret ninja grocery mission, I got up extra-early and prepared breakfast. Coffee, juice, amazing toast, and an omelette made with thinly sliced super-fresh swiss brown mushrooms. The look on Selena's face and the huge thank-you hug made it all worthwhile. ;)The rest of Sunday was amazing. After a snooze I made it to Trek in the Park. This is theatre at its absolute finest. Trek in the Park is brimming over with quality, humour, and superb acting. If you haven't seen it yet, then tonight and next weekend are your last chances to do so, and it won't cost you a cent.
I wore my starfleet uniform to Trek in the Park, which was a huge win. I had arrived a little late, but many members of the audience members must have assumed that I was part of the production, and as such I was able to get a rather nice seat. But the biggest win was the cast reaction; having a guy in uniform and an Australian accent seemed to be something special, which meant that I had no problems meeting the cast, learning about the production, and getting lots and lots of photographs. One amusing fact about the whole thing is that in true cosplay fashion, all the uniforms were made by Kirk's mum. ;)
In the evening was dinner with Stacy, one of my most favourite Portlandians. Stacy was my guide at OSCON 2008, where she gave up much of her time to show me around town, explain the local customs and delicacies, educate me regarding local mushrooms, and stop me from cycling on the wrong side of the road. Stacy was out and about bicycle-hiking this week, but cut short her trip and cycled all the way back to Portland in record time for dinner, making me feel incredibly special.
Today ends my Portland adventures, as I head to San Jose for OSCON 2009, where I'm presenting Doing Perl Right and The Art of Klingon Programming.
Posted: 20th July 2009.
Tags: BeerAndBlog cosplay oscon oscon2009 pdx perl portland startrek TrekInThePark
Bookmark:
|
When you need perl, think perl.org
|
|
