All about Perl 6 – interview of Jonathan Worthington (Part 2 of 3)

NV: What does that say about the JVM’s project’s life expectancy? Will it run in parallel with MoarVM’s development? Does all this forecasts Parrot’s demise?

JW: The JVM is playing that difficult “second backend” role. Going from one target VM to two exposes all kinds of assumptions, such as places where things are not sufficiently well abstracted and so forth.  Such abstractions are not easy to design because you’re not only thinking about how to hide differences, but how to convey enough semantic information downwards in order to allow good code generation.

Having the second VM that  is already mature and well established is a huge help. OK, I did manage to segfault the JVM thanks to the odd invokedynamic bug I managed to hit upon. But by and large, if something is wrong, I can be pretty sure that the JVM itself will not be to blame. So, the JVM porting leads the way.

That naturally means we’ll get to a rather complete Rakudo on JVM some way ahead of getting there with MoarVM. The work on the two is going on in parallel. The JVM work has already blazed the “how to port Rakudo” trail, so really the MoarVM project complexity is much more tied up with the VM itself, not with working out how to get things ported to it.

As for Parrot: I think it’s worth remembering that Rakudo, along with the module ecosystem out there, runs on Parrot today. Of course, we aim to get there with the JVM and MoarVM in the coming months. But whatever happens with Parrot, those using and developing Rakudo will probably still be running it on Parrot for some  time to come. So that’s the short term. In the mid-term, the user base will decide what they prefer to run Rakudo on, and the answer will vary between users. We’ve no plans to go ripping out Parrot support once we run on the JVM and MoarVM. The path genuinely is multi-backend.

NV: Marketing wise, aren’t you concerned that announcing the development of yet another VM for Rakudo Perl will  reignite doubt as well as  sustain the commotion over Perl 6 and its future ?

JW: Yes, though to put it into context, the doubt around Parrot itself has not been especially helpful from a Rakudo marketing perspective either. The reason MoarVM was built in secret up to the point it was revealed, was partly a marketing consideration. Coming out saying, “Well, we have this idea to build yet another VM” would have been rightly met with a lot of scepticism. Saying, “Oh, we have this thing you can already translate and run over 80% of the NQP (a Perl 6 subset used to write Rakudo) test suite on” is a rather stronger position.

So yes, it concerned me. Deciding to work on MoarVM was not easy  for that reason and others. Frankly, I didn’t need a third full time job. Don’t get me wrong, I enjoy working on MoarVM, but it’s demanding in terms of concentration and brain cycles. Not to mention that debugging a GC is not exactly a fun evening in.Rather, I did it because, having spent a bunch of years growing an understanding of Perl 6 as a language and how to implement it, I reached the point where the costs of working on MoarVM looked like they could be outweighed by the wins from doing so. Plenty of toil remains to reach the goal, but if the Rakudo developers have shown anything, it’s endurance.

NV: In a previous interview, Moritz Lenz shared with us that Parrot’s main mistake was that  instead of focusing on satisfying Rakudo’s needs, it aimed to become a general platform for dynamic languages, something that ultimately made life hard for the Rakudo implementers. Has MoarVM learned from that, or is interoperating with other dynamic languages on the agenda?

JW: Well, there’s one dynamic language we’re very interested in interoperating with, and that’s Perl 5. For now, we’ll take the expedient path of making 5/6 interop work nicely rather than trying to generalize the whole thing into some beautifully abstract set of interop primitives. But it’s not just a case of running Perl 5 on MoarVM – though the v5 module work that’s going on may allow that to – but rather a case of integrating the existing Perl 5 interpreter.

On the question of being a general platform, though, if you look at the VMs that do successfully host multiple languages today, they’ve typically started by hosting one language really well. My interest is in making MoarVM do Perl 6 really well. If, some years down the line, that makes it an attractive target for other languages… well, that’s just fine. But I’m most certainly not thinking about the needs of other languages when taking design decisions for MoarVM.
All of this said, Perl 6 has a LOT of language features. And the NQP compiler tool-chain that we build Rakudo on is rather rich and powerful. So there’s nothing to really stop people using it to write a compiler for another language today.

Indeed, writing a compiler once that targets the JVM, MoarVM, and so forth for free could be an interesting proposition. Naturally, I’ll be happy if people pick up the tools and enjoy creating other language implementations with them. But my focus when designing and building MoarVM is quite firmly – “What do I need for Perl 6?”

NV: Integrating/embedding the existing Perl 5 interpreter into the platform means that Perl 5 will keep its blood ties to XS and won’t finally get a proper hardware independent VM ?

JW: Exactly. The goal of this bit of interop is to make CPAN modules and an organisation’s existing Perl 5 investment accessible from Perl 6. You don’t go that far before you hit an XS dependency, so if the goal is to make existing stuff accessible then XS is a non-negotiable. All that said, there is an active effort to implement Perl 5 on top of the same compiler tool-chain that Rakudo is built on top of. It’s early days yet for the work, but it is making good progress.

That bit of work is known as v5, by the way. And it wasn’t something coordinated top-down from those of us already working on Rakudo for years. It was a new contributor who said “Hey, let’s try to make this work!” The good news is that it should be fairly easy to get that effort running on the JVM and later MoarVM also.

NV: Can or will there be alternative tools that could emit bytecode for MoarVM?  For example could MoarVM be considered a possible backend to Perlito?

JW: The bytecode format is specified, and the goal is for it to be declared stable a little way down the line, so there’s no real reason why not. One thing I decided fairly early on is that MoarVM will not have is an official intermediate language or assembly language.

The code generation we do today for the NQP to MoarVM translator goes directly from a tree data structure down to bytecode, skipping the costly generating and reparsing of text. So if you want to do that from outside of MoarVM,  you’d actually need to generate bytecode. Or maybe somebody will build a little assembler.

NV: The I/O infrastructure is currently based on ARP.Why are you considering a move from ARP to libuv? Is it because of the node.js-like asynchronous execution capabilities? How will those be integrated into the platform?

JW: Yes, it’s for the asynchronous I/O. Rakudo’s lack of support for that has been a common and very legitimate complaint for a while, and I know it’s an adoption blocker for some, so clearly I’m interested to deliver something there. We’ll be able to do it on the JVM, and MoarVM should certainly handle that case too.

We could build what we need from scratch, but why? I mean, things like the object model and JIT compilation make sense to be done in a very custom way. They’re the core domain, the places that MoarVM can shine by doing what Perl 6 needs efficiently. For async IO, there’s no reason to recreate something when we simply want to do well there, not discriminate MoarVM from the market by somehow doing async IO better.

Of course, that doesn’t mean at the language level we shouldn’t have a better story.  But at the runtime level, taking something battled-hardened and integrating it seems the way to go.
As for how, we have a rather free hand there, since Rakudo doesn’t support async IO yet

The more interesting challenges will probably be in making async IO really nice to do at the Perl 6 language level, rather than the VM level stuff, which will certainly have it’s challenges but are more “hidden”.

NV: So how is the Rakudo-to-Java interoperation utilized on the JVM? Because of that relation can Perl 6 run on Android?

JW: The first of those is a currently work in progress, and I’m optimistic we may ship support for calling into Java code in the next Rakudo compiler monthly release. It will be interesting to see how that gets used; I know various people have expressed interest, so I’m mostly waiting to see what they will do with it.

As for Android, one current blocker is our use of the new invokedynamic instruction. Once we get a “don’t use indy” compilation option, or invokedynamic becomes available on Android, I suspect that with a little effort it should be possible.

Part 3 which concludes the interview will be published next week

 

Nikos Vaggalis holds a BSc in Computer Science and a MSc in Interactive Multimedia. Professionally, he is engaged in the development of database applications and anything related to the RDBMS. He is a SQL aficionado and codes in both Perl and C#. As a journalist, he writes articles, conducts interviews and reviews technical IT books