Report on the Perl 6 Announcement

Jul 25, 2000 by Mark-Jason Dominus

“The Effort to Write Perl 6 Has Begin In Earnest”
What About Perl 6?
Other Resources
Editorial Conclusion

Last Monday, nobody knew that anything unusual was about to happen. On Tuesday, the Perl 6 project started. On Wednesday, Larry announced it at his “State of the Onion” address at the Perl conference.

Yes, it all really did happen that fast.

Several problems have seemed to beset Perl in the past year:

Infighting on p5p, the main Perl developers’ list, seemed out of control for a while, and, although the problems seem to have been solved by the addition of refereeing (See “the way forward” for details), several important developers had quit the list already.

But not a lot seems to have happened to Perl in the last year. The last really interesting Perl development was probably POE, which won the “Best New Module” award at last year’s Perl conference. There was a Perl 5.6 release, but most of the changes were not user-visible. There was better support for threads (including support for fork() on Windows systems, which is very impressive) and Unicode. But it’s hard to get excited about this, and it was really hard to put in, and incomplete.

Perl development was stuck.

Stuck How?

There are two big problems with Perl 5.

First, the internals are extremely convoluted. It’s hard to hack on Perl’s internals. There’s a lot of accumulated cruft, and Perl 5 has reached that stage of maturity in which it’s difficult to make a change or fix a bug without introducing some new bug. The excellent test suite prevents the new bugs from getting into the release, of course, but it also prevents the new feature from getting into the release.

The addition of threads, the compiler, bytecode generation, and Unicode support is still incomplete, because none of those things were designed into Perl 5; they are all being bolted on afterwards, and it shows.

The other big problem is thirteen years of backward compatibility history. The porters might like to rationalize the syntax a little, or clean up Perl’s semantics, but that might break old code, and Perl is committed to not breaking old code. We can’t add a new built-in function because it might break old code. It’s extremely difficult to remove even the most bizarre and little-used old features, because it might break old code. Every few months, someone suggests replacing Perl’s garbage collector with a more modern one, but there are always objections from people who have written code that assumes that the garbage collector will always be reference-count based and that it can control when objects will be destructed.

Technically, there’s not very much “wiggle room” in the source code, because the internals are so convoluted and difficult. And there’s not much “wiggle room” in the language itself, because of the need to support ten-year-old Perl 2 code.

Tuesday

I had to miss the Tuesday meetings, because I was teaching classes. So this part of the report is hearsay.

A meeting of the Perl 5 Porters was scheduled to take place in the afternoon. In the morning, Chip Salzenberg held a pre-meeting for discussion of “constitutional issues,” to change the way Perl development was run. But after a while, according to Larry, “Jon Orwant … pointed out that it was going to be useless to invent a new constitution if everyone was just going to say, ‘Ho, hum.’” Jon suggested that Perl try to go in a new direction, and do something new and big; he used Napster as an example of a new, big idea. This was the catalyst for the discussion to go in a new direction. At the end of the the morning, the decision was made to start working on Perl 6.

What About Perl 6?

Perl 6 will be a complete rewrite of Perl, starting from scratch. The internals will be redesigned. Threading, Unicode, and reliable signal handling will be designed in from the bottom up. The external API will be cleaned up; the core will be smaller and faster, and the XS system for extension modules will be scrapped in favor of something simpler and cleaner. One perennial problem has been that installing a new version of Perl has frequently broken binary compatibility with older versions and required that all of Perl’s unbundled extension modules be recompiled; this may be fixed in Perl 6.

At the same time, the social structure behind Perl will also be redesigned. Larry said that the perl5-porters model was not working well any more, and led to too much talk and too little action. Perl is too big for one person to manage, and the pumpkings burn out quickly. The new model will probably be several separate working groups, each charged with the design and implementation of one aspect of Perl. Larry said that Perl 6 would be designed by the community, and not by one person, and that with the new organization, Perl would be able to evolve into the language that we need in 2020.

Probably most amazing was that the Porters agreed to break backwards compatibility. As Larry said on Wednesday, the language itself needs revision. On Wednesday, Larry mentioned several small, specific changes:

The system() function will return true on success and false on failure.
The localtime() function will return the four-digit year.
The notion of “currently selected filehandle” will probably go away.
Typeglobs may be eliminated.

But these are just by way of example. Larry said “We’re really serious about reinventing everything that needs reinventing.” Everything is up for discussion, and the suggestions so far have even included giving Perl a pattern-matching system more like the one in SNOBOL. It seems that the big technical inspiration that came out of the Tuesday afternoon meeting was that it doesn’t have to be backward compatible. Perl 5, right now, can compile a Perl program into its internal syntax tree and then uncompile the tree back into source code, using the B::Deparse module. But you don’t necessarily have to produce the same source code you took in; you could instead rearrange the code to produce a Perl 6 program to do the same thing. So the upward migration path from Perl 5 to Perl 6 will probably be to run your code through a translator.

Larry promised not to abandon Perl 5. The 5.6 maintenance track will continue as planned, and the 5.7 development track will eventually yield Perl 5.8, as planned. 5.8 will be the final release of Perl 5, but it will continue to be maintained and stable. Larry pointed out that Perl 4 had not yet gone away, and there was no reason to expect Perl 5 to go away either. CPAN itself can serve as a test suite for Perl 6: When it can run most of the things in CPAN, either directly or post-translation, it’s ready.

Larry said that easy things should stay easy, hard things should get easier, and impossible things should get hard. He promised code in one year: alpha code, but well-designed alpha code, and he called on each person to play their part.

Questions

At the end of Larry’s announcement, Jon Orwant graciously offered to cancel his talk on “Rebuilding Postapocalyptic Civilization with Perl” so that Larry could have a question and answer session. These are the questions that people asked, with Larry’s answers.

Will Perl 6 be in C++?

“Maybe.”

When can we expect the grand plan?

Larry said that culturally, it has started already, and that we should expect a roadmap this month. His personal goal as language designer was to have something substantial to say by the time of his Linux Expo keynote speech in October. But the schedule is not yet nailed down.

Will Perl 6 have a spec?

Larry said he did not know how strict it would be, and that he was not big on specs. He likes the reference implementation approach. He said he wanted to particularly stress not a spec, but the development of acceptance tests into regression and validation tests that would say “This is Perl; this isn’t.” He said that this was more important than mere verbiage.

Will you look into technical infrastructure to support more constructive discussion than mailing lists can?

Larry said that the Porters were thinking about 2-tier mailing lists that each have a person in charge, and an RFC-like mechanism. He said that we would be paying less attention to the “Wouldn’t it be nice if…” type of proposals, and that proposals would need to be more formal and official.

Can you hint at some of the language changes?

Larry said that everything was negotiable. He specifically mentioned that Perl’s old “formats” feature should probably be in a module instead of in the core, and that he wanted to clean up ambiguities in the indirect object syntax. (This question concerns whether the notation

my $x = new Carrot (...);

parses as if you had written

my $x = new(Carrot(...));

or as if you had written

my $x = 'Carrot'->new(...);

instead. At present, this construction is rather fragile.) Larry said that since the Porters had decided to bite the bullet of backward incompatibility anyway and require translation of Perl 5 to Perl 6 code, they would consider any change at all, as long as it could be translated. He said that if 95% of all scripts could be translated 95% accurately, and if 80% of scripts could be translated 100% accurately, that was probably good enough.

He said that the Porters wanted Perl to have all the features that other languages say “nyah nyah” about, such as byte-compilation. He suggested that Perl’s garbage collector might be replaced with a more modern mark-sweep garbage collector, and that object finalization might be separated from destruction.

One problem of language translation is that you lose comments and formatting.

Larry replied that a2p, the AWK-to-Perl translator that is bundled with Perl, which he wrote, doesn’t have this problem, and that he didn’t think that the person asking the question would need to worry about this.

Is there a possibility of getting a switch construction?

(At this point Damian Conway shouted “I have a paper on that!”)

Larry said, “Certainly.” (Meaning that there certainly was a possibility, not that there certainly would be one.) Then he said that he would not promise one.

Will there be an ISO or ANSI standard for Perl?

“We will be our own standards body.”

How long have you been thinking about this?

“Since yesterday.”

What about POD?

Larry said it was all negotiable.

Which languages will you borrow from this time?

“All of them.”

(Then Larry said that they probably wouldn’t borrow COBOL’s IDENTIFICATION DIVISION syntax, and that part of the reason there was so little COBOL poetry is because it’s hard to write a poem that begins IDENTIFICATION DIVISION.)

He also mentioned the bytecode issue again, and said that it would be nice to have a cleaner object-oriented interface to I/O.

Some people using Perl to develop large software want features like strong type checking. Is there a possibility of providing this?

Larry said there was, and suggested a use strict 'typechecking' directive, or maybe use stricter. He also said that perl 6 should get rid of many of the stranger global variables, such as $#.

Other Resources

The official web site for Perl 6 right now is http://www.perl.org/perl6/. The site has a list of short-term goals, a list of the roles that have been assigned to various people, the notes from the Tuesday afternoon meeting, and press releases concerning Perl 5 maintenance and Perl 6 development.