· Sep 13, 2017

New Programming Language - Discussion

(Photo by Jeremy Thomas on Unsplash)

Dear Developer Community!

My name is Maks.

I developed DeclarativeCOS framework for declarative programming on COS.

Today I want to ask you about New Programming Language.

I have an idea to create new programming language which could be translated to COS classes / routines or even to int-files.

So, lets suppose it is happened. What do you want to see in such language?

Please feel free to say your minds. It is open discussion.

Discussion (15)2
Log in or sign up to continue

I would not mind if it were statically typed, but I know there are lots of people who would prefer otherwise.

Anyway, I would like it to have some way of declaring strings like Ruby's heredoc and string interpolation, because my code is full of ""s, """"s and "_var_" (those all in combination decrease legibility significantly IMO).

    variable = <<-TERMINATOR
      Your free trial will expire in #{days_until_expiration} days.
      Please update your billing information.

Also, lambdas and closures would be nice.

Hi Maks,

This is an interesting topic that I have spent a great deal of time thinking and working on myself.

I developed a transcompiled object oriented Mumps language called MOLE a number of years ago...

I've also had a number of attempts at writing a JavaScript to Mumps transcompiler and can share my thoughts on why this is not a good idea.

Firstly COS is a perfectly good language that I am very happy to use on a day to day basis. There are times when it feels a little stale, particularly as I jump around a lot of languages, but on the whole, it does a very good job. Any peccadilloes or missing features seem trivial when you have all the other benefits from the entire Caché platform. That said, there is always room for improvement, and anything that helps us write better, smaller, and more maintainable code has to be a good thing.

A couple of things that I learned from MOLE was that when building a transcompiler you have to work with the underlying language. Fundamental aspects such as coercion should be first class citizens of the transcompiled language. 

As an example, JavaScript and Mumps differ greatly on data types and coercion. Mumps has one single data type whilst JavaScript has 6. Mumps has a fairly simple set of coercion rules whilst JavaScript is far more complex. As an exploration, I wrote a bank of 1600 coercion tests that Mumps would have to replicate, which highlights some of the complexities...

The other consideration is that whilst JavaScript can have just in time compilers, it is fundamentally a run time language with a complex run time environment for managing things like dynamically changing prototypal inheritance and async call stacks, all of which would all have to be simulated in a Mumps compiled environment. On reflection, it's just a really complex idea that would be much less performant than its underlying language.

So any transcompiled language should essentially be a super set of its underlying language, not a wild shift in syntax or behaviour to JavaScript, C# or any other language.

We can learn lessons from other transcompiled languages such as CoffeeScript and TypeScript. For me CoffeeScript is marmite and I refused to even try and like it. Whilst there were some interesting features to CoffeeScript, it went too far by changing the syntax of the language. It's interesting however that features such as arrow functions influenced and made its way into the core JavaScript languages (ES6). This shows that a transcompiled language can be a good test bed for ideas that can influence and improve an underlying language.  

The gold standard of transcompilers is TypeScript. TypeScript didn't try to change the language, but rather provide a backwards compatible compiler for future features of JavaScript, whilst providing a small amount of syntactic sugar for injecting compile time type checking.

In COS we already have syntax for declaring things like argument types, but it's just a shame that the compiler does not enforce them, something I touched on here...

TypeScript actually shows that we don't need stronger types, just better compile time validation of using types. For me, I would borrow this aspect of TypeScript as the first feature of any transcompiled language. As an example...

set x = "Hello World"

can be marked as type string...

set x : string = "Hello World"

This would be removed at compile time and have no run time behaviour, but it would enable the compiler to stop this variable from being used incorrectly, e.g. passing it as an argument that should have been of type integer.

This would allow code to still be written loosely, but where someone is trying to enforce better protection, it will warn when it is implemented and implemented wrongly. Generics would also be a good fit.

The other aspect I would borrow from TypeScript would be to shim backwards compatible features.

An interesting example is the return command which was introduced in 2013. It's nice to see a new language feature, but then, of course, we have the problem of backwards compatibility. I think back to using the asterisk in $piece, e.g. $piece(x,2,*) which is a great feature but tripped me up on one particular deployment, and still makes me a little nervous to use it in library code.

This is where a transcompiler could be a big help. Being able to use these new features and know that they are transcompiled into backwards compatible code would be a really useful tool.

The next on my list would be the addition of the "in" command, to be combined with the "for" command. Whilst the $order function is mighty powerful, it's not the prettiest to look at. Instead of writing...

set name=$order(^names(""))
while name'="" {
    write !,name
    set name=$order(^names(name))

you end up with a more cleaner and easier to read solution...

for name in ^names {
    write !,name

which obviously gets cleaner when these $order loops become heavily nested.

Next on my list would be to promote simple types to fake objects. As in the above type declaration example, if a variable is declared as a string then it would automatically inherit a whole host of useful string functions, e.g.

set x : string = "Hello World"
write x.toUpper()


This would compile down to...

set x = "Hello World"
write $zconvert(x,"U")

Next up would be lambda functions. Combined with an array type it would allow for things like map, reduce and filter methods on variables declared as an array, or implicitly referenced as a global. Arrow function syntax would sit well with cos...

//filter out all numbers
set bar : array = foo.filter( value, index => {
    if 'value.IsNumber() return value

From here you can then introduce method chaining which would make code much more readable, so instead of...

set x=$zconvert($extract($piece($piece(y,","),"~",2),1,10),"U")

you end up with

set x=y.piece(",").piece("~",2).extract(1,10).toUpper()

As a starter these would all be great additions to an already great language. There are probably more suggestions, such as built in async callback support, but this is where I would start from.

That said, there is one important aspect that also needs to be addressed. All of these additions would break both studio and atelier.

You could also argue that these types of additions would only be half of a solution, where improved tooling should and must go hand in hand with a transcompiler. As a minimum the IDE highlighting needs to support the new syntax, as well as adding value with improved real time type detection issues.

I spent a great deal of time working on IDE solutions for this very reason. I felt there was no point releasing anything unless it had its own IDE as well. I went through lots of experiments, extending Eclipse and various other IDE's, but I always came back to one important requirement. Any solution, should and must work remotely (as does studio) and that inevitably it would have to be a browser based solution to work as a modern day cloud supporting solution.

Here is a glimpse of what I have been up to here, it's 100% JavaScript and built on top of its own in-house widget library...

It's far from finished as I am working on a better way to write API code with Caché, which is what I am currently working on with the Cogs library. If I eventually get there then the plan is to release all of this into the wild, along with the MOLE transcompiler which could then be used to extend COS, or experiment with other Mumps based ideas and innovations.

What I would say is it's very easy to underestimate how much effort is required to make a transcompiler robust, fully unit tested, tooled up and fully supported to the degree that anyone would consider using it in production. When you're doing this without a sponsor and part time it's a long hard road...


Hi Evgeny,

I guess it's not as much about the size of the community as it is about its composition. COS is not necessarily the kind of programming language/stack a start up is going to choose over JS/Ruby/Python/whatever is hip today. I assume (and it really is just an assumption) many companies are featuring business oriented development focusing on effecient database access and maintaining pretty old systems. Leaving little time/opportunity to try out new things and therefore contribute to the open source community.

I think if Intersystems wants to maintain COS as a viable language and not have every developer rather focus on Node and using Caché solely as a database system they are better off supporting open source efforts and making the language more appealing to younger developers.

Regarding your bullet points

1. Seeing how much of a difference even small contributions make in the COS world (vscode integration, json-output way before itw as implemented in the core language) I think many developers would benefit from even small tools/routines being open sourced.

2. github makes it incredibly easy. Convincing your boss that spending time on something someone else might get for free eventually is not that easy in a corporate environment.

3. As long as it is not using external functionality a simple xml import or install script should be viable.

Personally I really am looking forward to some new features and would like to contribute to projects.


I think the community as a whole take some blame as well. Since Caché is robust enough to make us think  that we can only use features provided and audited by ISC themselves., you can see that when comparing the number of questions with articles/anouncements. 

So far, my impressions are that most of Caché developers prefer to take the safest route instead of trying what the community provides. There're exceptional projects as well, like WebTerminal but such projects still represent a minor part of us using it.

This subsequently results in open-source projects being abandoned due to the lack of interest as user and developer (maintainers). My proof when saying that is the amount of replies and attention a thread that presents a new project gets.

Also, if you don't want to lose time trying to convince your boss about open-sourcing a tool, do it outside and bring it to where you work. This way you'll be able to say that the project is your initiative and not something on behalf of the company you work. Just remember to design it in a way that won't use any business code.

Hi, Sebastian! What I like about your comment that you are not saying that it is impossible.

About small COS community: how large the community should be in numbers to support the project in your opinion?

In my opinion, 3rd party library for InterSystems data platform would be supported by the community if:

1. It provides a great added value for significant part of community

2. It is easy to collaborate in development

3. It is easy to install/deinstall the library