New Programming Language - Discussion

Hi Maks,

This is an interesting topic that I have spent a great deal of time thinking and working on myself.

I developed a transcompiled object oriented Mumps language called MOLE a number of years ago...

http://www.memcog.com/mole/index.html

I've also had a number of attempts at writing a JavaScript to Mumps transcompiler and can share my thoughts on why this is not a good idea.

Firstly COS is a perfectly good language that I am very happy to use on a day to day basis. There are times when it feels a little stale, particularly as I jump around a lot of languages, but on the whole, it does a very good job. Any peccadilloes or missing features seem trivial when you have all the other benefits from the entire Caché platform. That said, there is always room for improvement, and anything that helps us write better, smaller, and more maintainable code has to be a good thing.

A couple of things that I learned from MOLE was that when building a transcompiler you have to work with the underlying language. Fundamental aspects such as coercion should be first class citizens of the transcompiled language. 

As an example, JavaScript and Mumps differ greatly on data types and coercion. Mumps has one single data type whilst JavaScript has 6. Mumps has a fairly simple set of coercion rules whilst JavaScript is far more complex. As an exploration, I wrote a bank of 1600 coercion tests that Mumps would have to replicate, which highlights some of the complexities...

https://gist.github.com/SeanConnelly/4c25afc61113ee113ec14211f59076ec

The other consideration is that whilst JavaScript can have just in time compilers, it is fundamentally a run time language with a complex run time environment for managing things like dynamically changing prototypal inheritance and async call stacks, all of which would all have to be simulated in a Mumps compiled environment. On reflection, it's just a really complex idea that would be much less performant than its underlying language.

So any transcompiled language should essentially be a super set of its underlying language, not a wild shift in syntax or behaviour to JavaScript, C# or any other language.

We can learn lessons from other transcompiled languages such as CoffeeScript and TypeScript. For me CoffeeScript is marmite and I refused to even try and like it. Whilst there were some interesting features to CoffeeScript, it went too far by changing the syntax of the language. It's interesting however that features such as arrow functions influenced and made its way into the core JavaScript languages (ES6). This shows that a transcompiled language can be a good test bed for ideas that can influence and improve an underlying language.  

The gold standard of transcompilers is TypeScript. TypeScript didn't try to change the language, but rather provide a backwards compatible compiler for future features of JavaScript, whilst providing a small amount of syntactic sugar for injecting compile time type checking.

In COS we already have syntax for declaring things like argument types, but it's just a shame that the compiler does not enforce them, something I touched on here...

https://community.intersystems.com/post/compilation-gotchas-and-request-...

TypeScript actually shows that we don't need stronger types, just better compile time validation of using types. For me, I would borrow this aspect of TypeScript as the first feature of any transcompiled language. As an example...

set x = "Hello World"

can be marked as type string...

set x : string = "Hello World"

This would be removed at compile time and have no run time behaviour, but it would enable the compiler to stop this variable from being used incorrectly, e.g. passing it as an argument that should have been of type integer.

This would allow code to still be written loosely, but where someone is trying to enforce better protection, it will warn when it is implemented and implemented wrongly. Generics would also be a good fit.

The other aspect I would borrow from TypeScript would be to shim backwards compatible features.

An interesting example is the return command which was introduced in 2013. It's nice to see a new language feature, but then, of course, we have the problem of backwards compatibility. I think back to using the asterisk in $piece, e.g. $piece(x,2,*) which is a great feature but tripped me up on one particular deployment, and still makes me a little nervous to use it in library code.

This is where a transcompiler could be a big help. Being able to use these new features and know that they are transcompiled into backwards compatible code would be a really useful tool.

The next on my list would be the addition of the "in" command, to be combined with the "for" command. Whilst the $order function is mighty powerful, it's not the prettiest to look at. Instead of writing...

set name=$order(^names(""))
while name'="" {
    write !,name
    set name=$order(^names(name))
}

you end up with a more cleaner and easier to read solution...

for name in ^names {
    write !,name
}

which obviously gets cleaner when these $order loops become heavily nested.

Next on my list would be to promote simple types to fake objects. As in the above type declaration example, if a variable is declared as a string then it would automatically inherit a whole host of useful string functions, e.g.

set x : string = "Hello World"
write x.toUpper()

HELLO WORLD

This would compile down to...

set x = "Hello World"
write $zconvert(x,"U")

Next up would be lambda functions. Combined with an array type it would allow for things like map, reduce and filter methods on variables declared as an array, or implicitly referenced as a global. Arrow function syntax would sit well with cos...

//filter out all numbers
set bar : array = foo.filter( value, index => {
    if 'value.IsNumber() return value
})

From here you can then introduce method chaining which would make code much more readable, so instead of...

set x=$zconvert($extract($piece($piece(y,","),"~",2),1,10),"U")

you end up with

set x=y.piece(",").piece("~",2).extract(1,10).toUpper()

As a starter these would all be great additions to an already great language. There are probably more suggestions, such as built in async callback support, but this is where I would start from.

That said, there is one important aspect that also needs to be addressed. All of these additions would break both studio and atelier.

You could also argue that these types of additions would only be half of a solution, where improved tooling should and must go hand in hand with a transcompiler. As a minimum the IDE highlighting needs to support the new syntax, as well as adding value with improved real time type detection issues.

I spent a great deal of time working on IDE solutions for this very reason. I felt there was no point releasing anything unless it had its own IDE as well. I went through lots of experiments, extending Eclipse and various other IDE's, but I always came back to one important requirement. Any solution, should and must work remotely (as does studio) and that inevitably it would have to be a browser based solution to work as a modern day cloud supporting solution.

Here is a glimpse of what I have been up to here, it's 100% JavaScript and built on top of its own in-house widget library...

http://www.memcog.com/images/Nebula.png

It's far from finished as I am working on a better way to write API code with Caché, which is what I am currently working on with the Cogs library. If I eventually get there then the plan is to release all of this into the wild, along with the MOLE transcompiler which could then be used to extend COS, or experiment with other Mumps based ideas and innovations.

What I would say is it's very easy to underestimate how much effort is required to make a transcompiler robust, fully unit tested, tooled up and fully supported to the degree that anyone would consider using it in production. When you're doing this without a sponsor and part time it's a long hard road...

Sean