This week on Perl 6 (week ending 2002-08-11)

Aug 11, 2002 by Piers Cawley

Far out in the uncharted backwaters of the unfashionable end of the Western Spiral arm of the Galaxy lies a small unregarded yellow sun. Orbiting this at a distance of roughly ninety-eight million miles is an utterly insignificant little blue-green planet whose ape-descended life forms are so amazingly primitive that they still think inventing programming languages is a pretty neat idea.

So, as one amazingly primitive life form to another, here’s what’s been going on in Perl 6 development this week. As is becoming tediously traditional, we’ll start with the internals list…

Array vs. PerlArray

Way back in the mists of a fortnight ago (that’s ‘two weeks’ for the unenlightened Americans among us.) Melvin Smith announced that he was thinking about copying some of the fixes that had been made to the PerlArray PMC over to array.pmc as well. This week, Steve Fink sent in a patch for an alternative approach, which makes PerlArray a subclass of Array. Being appropriately lazy, Steve also added some rather useful looking keywords to the PMC parser, which added some shiny looking OO features. He then delivered an extended, 3 post monologue in which he kept replying to himself and refining his explanation of what the patch did. When he finally stopped talking to himself Dan told him to just commit it.

A side point in Steve’s discussion with himself (and Sean O’Rourke off-list, apparently) was the idea of dramatically simplifying default.pmc so it just throws an exception for operations it doesn’t know how to deal with, rather than trying manfully to satisfy the demands of an unreasonable programmer. Dan reckons he should go ahead and rip out the old, obsequious code.

http://groups.google.com/groups – Start here

Unifying PMCs and Buffers for GC

Mike Lambert had stepped up to this task and gave us a brain dump of his plans. This week Jerome Vouillon wondered about the long term plans for the Garbage collector, and wondered if we knew how to implement a generational garbage collector when one has several pools. Dan’s long term plan for the outside world is that ‘GC is a black box that just works.’ Internally, he doesn’t care, so long as it satisfies the ‘just works’ criterion. Mike Lambert thinks that a generational collector shouldn’t be too hard.

Josef ‘how do I type “ö” again?’ Höök wondered how/if this would affect his matrix implementation, and when/whether his code would be merged with the current tree. Dan answered that he was a little concerned about PMCs which add code to the core, and Josef explained that he wanted to be able to reuse some of the code in some future multiarray.pmc

http://groups.google.com/groups

http://groups.google.com/groups

Register allocation for the JIT

Nicholas Clark and Daniel Grunblatt held a learned discussion about this, with particular reference to the ARM and i386 architectures. There were diagrams. And discussions of hardware documentation. It was both scary and surprisingly easy to understand. And a consensus was reached, which is nice.

http://groups.google.com/groups

Stack mark ops & such

Dan announced that he was ‘about half a step from putting pushmark, popmark, stack marks, and suchlike things into the core.’ and that this was everyone’s opportunity to tell him what a bad idea it was. Jerome Vouillon wondered how they’d interact with continuations and coroutines. Answer: ‘Interestingly’. Jerome offered a code sample which may be surprising, and asked Dan a hard question. Which Dan hasn’t answered yet.

Melvin Smith wondered if there was a real issue with continuations, since each continuation got its own copy of the call stack. There is an issue, but it’ll likely be solved by documenting its existence.

http://groups.google.com/groups

http://groups.google.com/groups

http://groups.google.com/groups

Exceptions

Dan posted his road map to exception handling. Or rather, he described the two ops (push and throw) needed to do exception handling, but punted on what an exception object should look like.

Florian Weimer wondered if it wouldn’t be possible to handle exception objects in the throwing context. Dan reckoned that this could be problematic, which is why we’re not going to do it ‘at the moment’.

Jerome wondered if we really needed the pushx opcode at all. Tanton Gibbs disagreed and there was some back and forth about that, and about whether it made sense to think of a return as an exception (it does, but only rarely, when you’re writing your own control structures; Damian will explain more later).

Dan also thought that pushx would be necessary, and Jerome clarified his suggestion. Simon Glover wondered what happened when a program didn’t define an exception handler and was reassured that there would be some kind of default (probably language specific) handler in place, and if an exception got past that then Parrot would handle it itself with a suitably ‘big fit to STDERR’ before dying in a fit of pique.

http://groups.google.com/groups

http://groups.google.com/groups – Discussion of whether pushx is necessary starts here.

Regex speedup

Angel reposted his patch to the regex subsystem, this time with appropriate diff flags. Brent Dax needed more clarification, which Angel gave; Dan accepted that, given docs and tests, the patch could go in. Angel made another patch with docs and tests, although Mike Lambert then asked for more clarification, and pointed out some issues with the inline keyword not being standard across all C compilers. He also wondered if Brent’s old ‘Tutorial’ docs could be updated and reinstated. Angel responded with more clarification, and a promise that the tutorial would go in when he’d rewritten it, and commented that ‘The previous Rx version had excellent documentation, so it will take a bit of time to get there.’ I don’t think this patch has been applied yet.

http://groups.google.com/groups – Start here.

Lexicals and globals assembly question

Brian Wheeler has been looking at store_lex, find_lex, store_global and find_global and pointed out that the store_* ops seemed to be the wrong way round, since the parrot rule is that the destination is always the first operand. The response to this can be summed up in one word: “Oops.” Brian has patched it, and added tests. Leopold Toetsch fixed things up in Builtins.pm.

http://groups.google.com/groups

Questions about pdd03_calling_conventions.pod

Jonathan Sillito asked a few questions about calling conventions, and noticed that the callcc op appeared to be going the way of all flesh. Dan answered the questions and remarked that callcc was to be reinstated and shouldn’t have disappeared in the first place. Sean O’Rourke wondered why, since it could be replaced with calls to invoke and other cleverness. Dan agreed. So callcc will stay dead. (The only reason it was on the perch in the first place was because it’d been nailed there.)

http://groups.google.com/groups – Start here

Hash optimisations

Having had a patch to remove a function call from the hash algorithm accepted, Jason Greene offered a more comprehensive set of hash optimisations. Dan applied the patch. Nicholas Clark and Steve Fink had a few questions. Nicholas wondered why a constant had been changed, and Steve worried about memory use. Dan reckoned that worrying about memory to the tune of an unsigned int + alignment bytes at the expense of speed was the wrong trade-off to make. However, the other subthread had led to a different design which saved the space and only gave a small performance hit in the less common case. Dan agreed.

http://rt.perl.org/rt2/Ticket/Display.html

Various PMC issues

Stephen Rawls is still trying to write tuple.pmc. Which means he’s been nosing around the sources of all the other PMCs, looking at how they do things, so, he had a bunch of questions, mostly to do with PerlNums and PerlInts. The answer to many of the questions appears to be ‘the problem will go away when we get multimethod dispatch’. Dan is pouting because he had ‘wanted to go with a left-side-wins scheme, but alas correctness has trumped speed…’. Dan has promised details of Parrot multimethods ‘soonish’.

http://groups.google.com/groups

http://groups.google.com/groups

Never ending story Keys

Josef wondered where we are with multi keyed access. Tom Hughes, who has promised to have a go at fixing things wondered if he was waiting for hell to freeze over. At which point it started ‘snowing in Texas’ as Dan came through with the design. There was much rejoicing, and a snowball fight was threatened. Tom had some questions (no surprise there: when you’re the one doing the implementation, there’s almost never enough information). Clarification was provided, as were supplementary questions, and supplementary answers. As Blur put it so eloquently: “Woohoo!”

http://groups.google.com/groups

http://groups.google.com/groups

Hatchet job on assemble.pl / the fixup table

Brian Wheeler took an axe to the assembler and some associated files and asked for comments on his plans. Dan liked the sound of it, so Brian posted a patch. Leopold “How do you pronounce” Toetsch suggested making it a standalone library so stuff could just use Assembler instead of calling yet another external program. Sean O’Rourke asked for a summary of what had changed, and Brian gave him one. (Oo er).

http://groups.google.com/groups – proposal

http://groups.google.com/groups – patch

http://groups.google.com/groups – summary

PMC assignment stuff

Dan has been a positive goldmine of designs and specs this week. This time he offered the design for a new PMC assignment opcode. We already have SET, which copies the pointers and CLONE making a ‘full clone’, but we also need ASSIGN to ‘stuff a value from one PMC to another’. Peter Gibbs made a start on implementation and asked for some clarification, which was forthcoming, and then supplied a patch with his implementation, along with tests. Go Peter!

http://groups.google.com/groups

http://groups.google.com/groups

Status on matrix patch?

Josef is worried about what’s happening to the matrix patch. Simon Cozens used this as an opportunity to crack the worst joke yet seen on any perl6 mailing list, but Josef appears not to have seen the film and didn’t get the joke. Anyhoo, back on the main line of the thread, the problem seems to be that in order to have things sensibly arranged, it made sense for Josef to break the .pmc file up into a couple of .c and .h files, along with matrix.pmc, but there’s currently no scheme for multi-file PMC classes. Dan also thinks that putting code into the parrot core just to facilitate code sharing between PMCs is probably not the right answer. This appears to have opened a can of worms. Dan is searching for a bigger can – sorry – designing the infrastructure that this needs.

http://groups.google.com/groups – Thread starts here.

http://groups.google.com/groups – Terrible joke…

Constant & opcode swap ops

Dan offered designs to support multiple segment bytecodes, or at least, to swap in constant tables. Steve Fink wondered if it might be a good idea to wrap the interpreter in a PMC and use vtable methods to do constant table manipulations etc. Dan thinks that’s a great idea for introspection, but worries that it’ll slow things down if we make the interpreter itself go through those hoops all the time. Nicholas Clark also offered a few caveats (‘Are you sure you want to do it this way?‘, words to strike fear into the heart of any designer I think). Dan isn’t sure.

Faster assembler

Nicholas Clark offered a patch which makes Assembler::_generate_bytecode faster. By around 1.5%. Dan applied it. Sean O’Rourke offered more tweaks. Mattia Barbon had a patch which meant that the tests could just ‘use Parrot::Assembler’ rather than call assemble.pl, but the tweaks have outdated it. Dan and Nicholas asked for the patch anyway because it would make remaking it easier.

http://rt.perl.org/rt2/Ticket/Display.html

Perl 6 regexes…

Dan threw down the gauntlet of implementing Perl 6’s regexes from Apocalypse. Sean O’Rourke has apparently already taken it up. I think we’ll be seeing more of this thread next week…

http://groups.google.com/groups

Meanwhile, in perl6-language

The terribly named thread about different default values of true and false rumbled on. Damian averred that code which ‘relies on the poorly specified standard values of truth and falsehood deserves to break’. Damian also argues that Perl ought to have a proper boolean type. After all, it has proper numeric and string types, and that they should be used by all built ins. (Which does lead me to wonder about code like 0 but true). Chip reckons that the standard values of truth and falsehood aren’t that badly specified, it’s not like they’ve ever changed since perl 1, though he does agree that a ‘real’ boolean would be nice. Elsewhere in the thread, Chip worried about the scoping of operator definitions and Damian reassured him a bit.

http://groups.google.com/groups

Use of regular expression on non-strings

Threads in perl6-language seem to be longer lived than those over in internals. I wonder why? This was another thread that started last week and kept on going. David Whipp clarified his original question and various people responded. Err… I’m not entirely sure how to summarise this one, and following the threading is a tad tricky because Mr Whipp’s mailer doesn’t do the In-Reply-To: thing.

http://groups.google.com/groups is probably as good a place as any to start looking.

Autovivi

Luke Palmer wondered how autovivification was going to work with specific reference to print %foo{bar}{baz}. Would %foo{bar} still be created if it didn’t exist? Answer: “NO! Thank ghod!”. Unless Larry says otherwise.

http://groups.google.com/groups

Perl summary for week ending 2002-08-04

See recursion. Miko O’Sullivan queried the throw-away comment that regexes were now called ‘rules’ and wondered if the term ‘regex’ would be going away. It turns out that I wasn’t quite accurate, but that Damian still reckons that ‘regexes’ should be deprecated in favour of either ‘rules’ or ‘patterns’, at least in his own writing. For some reason this ended up spinning off into a discussion of regular, context-free and context-dependent languages and the nitty gritty details of when an expression was regular or not. Mark J. Reed ended up posting a fine essay to the list on the differences between the various types.

http://groups.google.com/groups

http://groups.google.com/groups

http://groups.google.com/groups – Mark’s essay.

http://groups.google.com/groups

http://groups.google.com/groups

---

In brief

Piers Cawley missed the point about doing copy on write of the entire stack when doing closuresque things. Various people explained, very politely that he was wrong. In a moment of selfish editorial wossname, I’ll spare Piers’s blushes by omitting the URL.

Josef added a couple of useful ops. Dan applied the patch, but wondered where the tests were. Josef has, of course, tested the ops, but Dan wants ‘Comprehensive tests in t/ for everything’ so we notice when we break things later. Nicholas Clark raised the spectre of ‘The Schwern with the big stick.’ if we failed to meet that testing goal.

Mike Lambert posted the first of his patches which will, eventually, unify PMCs and buffers.

Steve Fink posted a patch to convert the hashtable from pointer based to index based, making the GC system happier. Applied.

Angel Faus posted a ‘loop discovery’ patch for imcc which attempts to avoid array spilling whilst in an inner loop. This patch was applied. Then Leopold Toetsch pointed out a few bugs, and Angel posted another patch, which hasn’t been applied yet.

Andy Dougherty sent some patches to eliminate warnings under Solaris 8. Applied.

Jarkko sent in more warnings patches. Applied.

Sean O’Rourke offered a patch to hashes to do deep cloning. Dan asked him to hold on a sec, adding that if he hadn’t addressed the issue inside a day or to we should nudge him. Dan, consider yourself heartily nudged.

Peter Gibbs has removed the set_string_unicode and set_string_other vtable methods, which are, frankly, unnecessary. Leopold caught perlundef.pmc, which had been missed from the original purge.

Jonathan Sillito offered a patch to give scratchpads a pointer to their parent pads, and ended up with a new Scratchpad PMC. Warnock’s Dilemma applies.

Peter Gibbs offers perlscalar.pmc, Aldo Calpini asked a few questions, and Warnock currently applies.

Daniel Grunblatt announced that the PPC JIT is now working.

Dan noted that there are some size restriction – we need to make sure that our chosen INTVAL is at least as big as a pointer. Andy Dougherty pointed out some possible caveats and noted that his warnings patch gave some assistance in this area.

Nicholas Clark found a bug in the assembler under perl 5.005_03. Simon Glover agreed it was a bug, but nobody is quite sure how to fix it. Nicholas is working on it though.

Boris “Thank heavens for cut and paste” Tschirschwitz wondered if the headers of the various PDD (Parrot Design Documents) were kept up to date, and if he could, in general, trust the docs to be accurate. Dan reckons they should be mostly up to date. Patches are almost certainly welcome.

In perl6-language, Chris Dutton wondered if we’d be able to create anonymous classes ( my $foo = class {...} ) in the same way as we currently create anonymous subs. Personally, I hope so. Chris worries slightly about my $foo_class $foo_obj = $foo_class.new, but I think he’s getting his compile time and runtime constructs mixed up. But what do I know?

SpamAssassin marked last weeks summary as spam in some mailboxes. John Porter suggested that ‘Maybe people should add SpamAssassin rule that deducts 5 points if the message contains /leon brocard/ ?’

---

Who’s who in Perl 6

Who are you?
Dave Mitchell. I work for a small UK-based software company.

What do you do for/with Perl 6?
I wrote PDD7 (coding standards), then got sidetracked into fixing perl 5 instead. I intend to get more involved in Perl 6 post 5.8.0 release.

Where are you coming from?
Sheffield?

When do you think Perl 6 will be released?
In about 2 years, then ready for production use in a further year.

Why are you doing this?
Because I love Perl and want to contribute to OSS development

You have 5 words. Describe yourself.
Lazy, apathetic, enjoyerofbugfixing, unsufferinggladlyoffools, rabidsceptic.

Do you have anything to declare?
My lack of genius?

---

Acknowledgements, corrections, threats and funding drives

Nicholas Clark would just like to clarify that, as far as right shifts of signed integers go, we should offer both arithmetic and logical right shifts. He doesn’t appear to have any preference about which should be the default.

This summary was again prepared with the aid of GNER tea, supplemented this week by a ham and cheese toasted sandwich, the toasted sandwich of the gods (though their bacon and tomato toastie also has its adherents I remain faithful to good old ham and cheese.)

Thanks are also due to the wonderful Pete Sergeant, and to a certain person who I’ll not mention again (but his favourite colour is orange) for their excellent proofreading skills. Anything which may have snuck past them (especially in this paragraph) is, of course, entirely my fault.

If your name appears in this, or any previous summary and you’ve still not sent me your answers to the perl6 questionnaire, please consider doing so. My “Perl 6 Who’s who?” archive hasn’t run out yet, but it’d be good to know that I’d got a good supply to fall back on.

Still no T?iBook, but this week’s haul of egoboo is well up there. Many thanks to the flatterers in perl-golf@perl.org.

If you didn’t like this summary, write your own. Go on, I dare you. If you did like it, send money to the Perl Foundation at http://donate.perl-foundation.org/ and remember, suitably large donations will earn you or your company a plug in a future summary. (You have to let me know though). See last week for details. And you’ll be helping to fund the next generation of Perl, which will give you the warm fuzzies and a general feeling of virtue and well being. In the words of Mrs Doyle, “Go on, go on, go on, go on!”