Cache annotations mini-framework

Hi all!

I'm developing a mini-framework to implement annotations in Cache. I want to support two kind of annotations: metadata and method decorators. I've got stuck trying to implement the second one.

Metadata

With metadata annotations I can add metadata to any kind of target. A target can be a method/classmethod, parameter, property and class.

For example:

Class cache.TestClass Extends lib.Annotations
{

/// @Author(UserName = "me")
ClassMethod Test()
{
Write "Test!",!
}

}

Then, after compiling the class "cache.TestClass"  I can obtain the Author metadata by calling the method "GetMethodAnnotation" of the class "cache.TestClass".

Method decorators

With this kind of annotations it's possible to modify the run-time of a method by just adding an annotation. For example I would like to be able to do:

Class cache.TestClassDecorator Extends lib.Annotations
{

/// @NiceTest
ClassMethod Test()
{
Write "Test!",!
}

}

And then, after compiling this class. The output of the method "Test" would be:

Init NiceTest!
Test!
End NiceTest!

So the code that should be executed if we call the method "Test" is:

ClassMethod Test()
{
  Write "Init NiceTest!",!
  Do ..MyOriginalMethodTest()
  Write "End NiceTest!",!
}

My ideal approach to solve this problem would be to add a new step in the compiler (preprocessor?), so I can first generate the modified code, and then compile it. I have no idea how to do this.

Another approach that I've investigated, and it's working (partially), is to generate a new method in the same class ("DecoratedTest"), and then redirect the original method to the new one (by modifying an internal Cache global... very dangerous, I know). So from the decorator I'm able to call to the original method by calling the routine. The problem with this solution is that I need to compile two times the same class. The first compilation is when I generate the modified method, and in the second compilation I redirect the original method to the modified one. In the first compilation, the redirection fails because the method doesn't exists yet.

The generated method looks like this:

ClassMethod DecoratedTest()
{
  Write "Init NiceTest!",!
  Do ztest^cache.TestClassDecorator.1()
  Write "End NiceTest!",!
}

Do you have any idea to implement this?

Thank you!

P.D.: For those who doesn't know what is an annotation, here is some lecture:

 

  • + 3
  • 0
  • 449
  • 11
  • 4

Answers

Hi Marc,

different languages have feature like the annotation you are describing. Most of these have one thing in common: they are quite young. Especially compared to M/COS.
That being said, some of the functionality you are envisioning is there already. Well, kinda, sorta ;)

You can use ZBREAK to add hooks to your existing code (even already compiled ones). This is more comparable to introspection than annotation as it happens at runtime. However, with the ZBREAK execute_code parameter it is easy to do anything from any point in your code.

To actually implement annotations the way you are thinking of here, you'd need to implement a wrapping solution around the compiler.
You might be able to hook into the source control utilities to do that: extending studio, however, modifying the sourcecode before passing it to the compiler would be difficult.

Using the wrapper approach, you could programmatically create shadow classes of your actual code and insert your annotation's code.
This will come with the drawback that you'd have to invoke your shadow classes instead of the direct code.
On the other hand it would allow you to have the annotations optional, so you're not slowing down your application once you're done developing.

You could possibly also use generator methods to move the wrapping into the class and generate new methods based on the code you put in and the annotations you added.

HTH,
Fab

Hi Fab,

I suppose that the first approach (using ZBREAK) would penalize the run-time performance. I'm trying to solve it at compile time, so there's no run-time penalization. Doing it at compile time it also feels more confident, just after compile I know how it will behave by just looking at the INT routine.

I like the idea of using the Studio hooks. I think that should be possible to have one namespace to do the first compilation, and another namespace to do the second (where the shadow classes will live). Having this two namespaces there is no need to change the method calls. The run-time should be done in the second namespace. I will investigate this solution.

Thanks,
Marc

Java has had annotations since version 5, which wikipedia says is 2004, and they are also heavily used in C#, where they are called attributes. They are heavily used in many places, notably in Hibernate, Spring and JUnit/TestNG. Introducing custom attributes/annotations is a very powerful way of extending a language.

It seems to me that Cache method keywords are the most similar thing to Java method annotations. Expanding them to match annotations would require a mechanism to introduce custom method keywords, to be able to fetch keywords for a method at runtime, and also to have custom keywords on properties, and for keywords to have arbitrary objects as values instead of being limited to strings. But I don't think it would be easy to do.

 

I thought the same, but method keywords have some limitations. For example (AFAIK) you cannot set multi-line strings.

You can modify method code during compilation (obviously dangerous) using generators. Here's the sample that annotates all methods, containing @ in description:

Class Utils.Annotations
{

/// @NiceTest1
ClassMethod Test()
{
    Write 1
}

ClassMethod OnCompile() As %Status [ CodeMode = objectgenerator ]
{
    #dim sc As %Status = $$$OK
    for i=1:1:%class.Methods.Count() {
        #dim method As %Dictionary.MethodDefinition = %class.Methods.GetAt(i)
        if method.Description [ "@" {
            set sc = ..annotate(method)
            quit:$$$ISERR(sc)
        }
    }
    quit:$$$ISERR(sc) sc
    set sc = %class.%Save()
    quit sc
}

ClassMethod annotate(method As %Dictionary.MethodDefinition) As %Status
{
    #dim sc As %Status = $$$OK
    set code = method.Implementation.Read($$$MaxCacheInt)
    set code = ..cleadOldAnnotation(code)
    set code = ..addNewAnnotation(code, $e(method.Description, 2,*))
    
    do method.Implementation.Clear()
    do method.Implementation.Write(code)
    
    quit sc
}

ClassMethod cleadOldAnnotation(code As %String) As %String
{
    if $find(code, "//init annotation end") {
        set code = $e(code, $find(code, "//init annotation end") + 2, $find(code, "//complete annotation start") - $l("//complete annotation start") - 2)
    }
    return code
}

ClassMethod addNewAnnotation(code As %String, annotation As %String) As %String
{
    #define Tab $c(9)
    set code =  $$$Tab _ "Write ""Init " _ annotation _ """" _ $$$NL _
                $$$Tab _ "//init annotation end" _ $$$NL _
                code _
                $$$Tab _ "//complete annotation start"_ $$$NL _
                $$$Tab _ "Write ""End " _ annotation _ """"
    return code
}

}

Compiles into:

/// @NiceTest1
ClassMethod Test()
{
    Write "Init NiceTest1"
    //init annotation end
    Write 1
    //complete annotation start
    Write "End NiceTest1"
}

GitHub

Another approach would be using dynamic dispatch:

Let's say you have this method:

/// @NiceTest1
ClassMethod Test()
{
    Write 1
}

Using Class Generators you generate 2 additional methods:

/// @NiceTest1
ClassMethod OnBeforeTest()
{
    Write "Before NiceTest1"
}

/// @NiceTest1
ClassMethod OnAfterTest()
{
    Write "After NiceTest1"
}

and you add dynamic dispatch method (and dynamic dispatch classmethod) to the class:

/// Is used to implement an unknown method call.  It is also used
/// to resolve an unknown multidimensional property reference (to get the value
/// of a property) because that syntax is identical to a method call.
Method %DispatchMethod(Method As %String, Args...) [ ServerOnly = 1 ]
{
    // Amethod -> method
    set Method = $e(Method, 2, *)
    do $method(, "OnBefore" _ Method, Args...)
    do $method(, Method, Args...)
    do $method(,"OnAfter" _ Method,Args...)
}

 

So, as a whole it can look like this:

Class Utils.Annotations1 Extends %RegisteredObject
{

Method OnBeforeTest()
{
    Write "Init NiceTest1",!
}

/// @NiceTest1
Method Test()
{
    Write 1,!
}

Method OnAfterTest()
{
    Write "End NiceTest1",!
}

/// Is used to implement an unknown method call. 
Method %DispatchMethod(Method As %String, Args...) [ ServerOnly = 1 ]
{
    set Method = ..getMethodName(Method)
    do $method(, "OnBefore" _ Method, Args...)
    do $method(, Method, Args...)
    do $method(,"OnAfter" _ Method,Args...)
}

/// Is used to implement an unknown class method call
ClassMethod %DispatchClassMethod(Class As %String, Method As %String, Args...) [ ServerOnly = 1 ]
{
    set Method = ..getMethodName(Method)
    do $classmethod(, "OnBefore" _ Method, Args...)
    do $classmethod(, Method, Args...)
    do $classmethod(,"OnAfter" _ Method,Args...)
}

ClassMethod getMethodName(method As %String) As %String
{
    // Amethod -> method
    set method = $e(method, 2, *)
    return method
}

}

Works like this:

USER>do a.ATest()
Init NiceTest1
1
End NiceTest1
 
USER>do a.Test()
1

1) The following:

USER>do a.Awtf()
 
    do $method(, "OnBefore" _ Method, Args...)
    ^
<METHOD DOES NOT EXIST>%DispatchMethod+2^Utils.Annotations1.1 *OnBeforenBeforenBeforenBeforenBeforenBeforenBeforenBeforenBeforenBeforenBeforenBefo,Utils.Annotations1
USER 58d1>

is not a great problem as can be easily corrected by adding a check whether a method named $e(method,2,*) really exists in the class.

2) The whole approach seems to be too complicated as you omitted an important step:
all calls of all app's methods should be switchable from "do a.methodName" to "do a.AmethodName" back and  forth. How to implement this?

Always call

do a.AmethodName()

And inside dynamic dispatch check global variable %debug (or global, or some IsDebug method etc.). If it exists call both annotation methods and target method, If it does not exist only call target method.

Always call "do a.AmethodName()"

I don't think it's a good idea as each extra nested call inevitably introduces extra performance drawback.

Anyway, imagine an existing large app with 100x classes and 1000x of methods calls, some of which can be done indirectly. In most common case an ability to do such transformation (all a.m() to a.Am()) sounds unrealistic, while your approach may fit small projects with simple calling conventions, especially when they are being started from scratch.

Macros for method names which compile into AmethodName or methodName depending on global var set for compilation then.

 

Dynamic dispatch is a hit on speed anyway.

Dynamic dispatch is a hit on speed anyway.

Sounds like yet another reason not to use it in production code.

Macros can help in many cases but using $method() and other flavors of indirect method calls.

Macros can eliminate using $method/dynamic dispatch here altogether.

Class methods calls "decorating" with macros seems to be possible using #def1arg macros.

But how to automate the substutitution of all flavors of an instance method calls to macros in a real world project? The method calls may use different <oref> variables in different parts of a project, or even use ".." or $this:
 set res=oref.Method1(x,y,.z)
 do Ref.Method1(x,y)
 if ..Method1(x,,.z) { ... }
 set a=$this.Method1(x,y)

So we need to define a macro with at least two arguments: 1) for oref, 2) for all other method's arguments. Having no variable macro arguments list, how to achieve it?

Note: we discussed direct calls only; taking in account $method() and indirection (@, Xecute), the situation would be even worse.