Agreed, I tend to use zpm for my own projects even if I don't intend to distribute. Between declaring dependencies, simpler running of unit tests, ability to script more things with my project than just "install" - it's just generally handy.

Hi Javier,

There are a few topics for running builds and unit tests via Jenkins (or really any CI tool):

• Calling in to Caché (or IRIS; the approaches are very similar)
• Reporting unit test results
• Test coverage measurement and reporting

Here's a quick intro; if you have questions on any details I can drill down further.

Calling in to Caché:

The most common approach I've seen is writing out to a file and then using that as input to csession / iris session. You can see some examples of this (for IRIS, with containers, but quite transferrable) here: https://github.com/timleavitt/ObjectScript-Math/blob/master/.travis.yml - I'm planning to write an article on this soon.

Some rules for this:

• Either enable OS authentication or put the username/password for the build user in the script or environment variables
• End the script with Halt (in case of success) or $System.Process.Terminate($Job,1) (to signal an OS-level error you can pick up from errorlevel/etc.); alternatively, always end with Halt and create a "flag file" in the case of error, the existence of which indicates that the build failed.
• Keep the script short - ideally, put the meat of the build logic in a class/routine that is loaded at the beginning, then run that.

Sample for Windows:

:: PREPARE OUTPUT FILE
set OUTFILE=%SRCDIR%\outFile
del "%OUTFILE%"

:: NOW, PREPARE TO CALL CACHE
::
:: Login with username and password
ECHO %CACHEUSERNAME%>inFile
echo %CACHEPASSWORD%>>inFile

:: MAKE SURE LATEST JENKINS BUILD CLASS HAS BEEN LOADED
echo do $system.OBJ.Load("","cb") >>inFile

:: RUN JENKINS BUILD METHOD
echo do ##class(Build.Class).JenkinsBuildAndTest("%WORKSPACE%") >>inFile

:: THAT'S IT
echo halt >>inFile

:: CALL CACHE
csession %INSTANCENAME% -U %NAMESPACE% <inFile

echo Build completed. Press enter to exit. :: PAUSE

pause > nul

:: TEST IF THERE WAS AN ERROR
IF EXIST "%OUTFILE%" EXIT 1

:: Clear the "errorlevel" variable that (it looks like) csession sets, causing successful builds to be marked as failure
(call )

Sample for Linux:

# PREPARE OUTPUT FILE
OUTFILE=${WORKSPACE}/outFile
rm -f $OUTFILE

# PREPARE TO CALL IRIS
# Login with username and password
echo $IRISUSERNAME > infile.txt
echo $IRISPASSWORD >> infile.txt

# MAKE SURE LATEST JENKINS BUILD CLASS HAS BEEN LOADED
echo 'do $system.OBJ.Load("'${WORKSPACE}'/path/to/build/class"),"cb")' >>infile.txt

# RUN JENKINS BUILD METHOD
echo 'do ##class(Build.Class).JenkinsBuildAndTest("'${WORKSPACE}'")' >>infile.txt

# THAT'S IT
echo halt >> infile.txt

# CALL IRIS
# csession is the equivalent for Caché
iris session $IRISINSTANCE -U $NAMESPACE < infile.txt

# TEST IF THERE WAS AN ERROR
if [ -f $OUTFILE ] ; then exit 1 ; fi

The next question is, what does Build.Class do? Given the Jenkins workspace root (WORKSPACE variable), it should load the code appropriately (likely after blowing away the code database to start with a clean slate; %Installer can help with this), then set ^UnitTestRoot based on the workspace directory, then run the tests, then report on results. Best to wrap the whole thing in a Try/Catch and throw/handle exceptions appropriately to ensure the error flag file / exit code is set.

Reporting Unit Test Results:

See https://github.com/intersystems-community/zpm/blob/master/src/cls/_ZPM/PackageManager/Developer/UnitTest/JUnitOutput.cls
(feel free to copy/rename this if you don't want the whole community package manager) for a sample of a jUnit export; Jenkins will pick this up and report on it quite easily. Just pass an output filename to the method, then add a post-build action in Jenkins to pick up the report. (You'll want to call this from your build script class.)

Measuring Test Coverage:

Seeing how much of your code is covered by unit tests helps to close the feedback loop and enable developers to write better tests - I presented on this at Global Summit a few years ago. See https://openexchange.intersystems.com/package/Test-Coverage-Tool - we've successfully used this with Jenkins for both HealthShare and internal applications at InterSystems. It can produce reports in the Cobertura format, which Jenkins will accept. Instead of using %UnitTest.Manager, call TestCoverage.Manager. The parameters detailed in the readme can be passed into the third argument of RunTest as subscripts of an array; to produce a Cobertura-style export (including reexporting all source in UDL for coverage reporting in the Jenkins UI), add a "CoverageReportFile" subscript pointing to an appropriate place in the Jenkins workspace, and set the "CoverageReportClass" subscript to "TestCoverage.Report.Cobertura.ReportGenerator".

If you want to use the Jenkins coverage/complexity scatter plot, use https://github.com/timleavitt/covcomplplot-plugin rather than the original; I've fixed some issues there and made it a bit more resilient to some oddities of our Cobertura-style export (relative to the data Cobertura actually produces).

Oof - by "newer tricks" you meant "objects." Yikes. Really, it'd be significantly lower risk to use the object-based approach than to roll your own without objects. (e.g., see my comment on automatic cleanup via %OnClose)

I don't have bandwidth to provide an object-free version, but you might look at the code for %IO.ServerSocket for inspiration.

How about (swapping in ^oddDEF if you don't care if it's been compiled):

ClassMethod PackageExists(package) As %Boolean
{
    Set prefix = package_"."
    Set firstClass = $Order(^oddCOM(prefix))
    Quit prefix = $Extract(firstClass,1,$Length(prefix))
}

I think the answers so far have missed the point. The number of arguments itself is variable. This is handy for things like building a complex SQL statement and set of arguments to pass to %SQL.Statement:%Execute, for example.

The data structure here is an integer-subscripted array with the top node set to the number of elements. (The top node is what's missing in the example above). Subscripts can be missing to leave the argument at that position undefined.

Here's a simple example:

Class DC.Demo.VarArgs
{

ClassMethod Driver()
{
    Do ..PrintN(1,2,3)
    
    Write !!
    For i=1:1:4 {
        Set arg($i(arg)) = i
    }
    Kill arg(3)
    ZWrite arg
    Write !
    Do ..PrintN(arg...)
}

ClassMethod PrintN(pArgs...)
{
    For i=1:1:$Get(pArgs) {
        Write $Get(pArgs(i),"<undefined>"),!
    }
}

}

Output is:

For bootstrap-table, I think the examples on their site are probably more useful than anything I could dig up. https://examples.bootstrap-table.com/#welcomes/large-data.html shows pretty good performance for a large dataset. Tabulator looks nice too though.

In any case it would probably be cleanest to load data via REST rather than rendering everything in the page in an HTML table and then using a library to make the table pretty.

Hi Steve,

We (Application Services - internal applications @ InterSystems) use %UnitTest with some extensions. We have a base Unit Test case that has a "run this test" helper method (among other things), a wrapper around %UnitTest.Manager that makes it easier to run all the tests without deleting them (and with some other features optionally enabled, like test coverage measurement - see a video of my 2018 Global Summit presentation on this here). Our wrapper also loads all of the unit tests before running any, which allows unit tests to extend classes within the unit test root in different packages from their own.

I agree with @Eduard Lebedyuk that %UnitTest meets our needs well aside from the lack of parallelization. A parallel %UnitTest runner would be an interesting project indeed...

We routinely run unit tests via Jenkins CI, report test results in the jUnit format, and also report on code coverage (Cobertura-style) and a complexity/coverage scatter plot.

For automated UI testing, Selenium/Cucumber has worked well for older Zen/CSP UIs. True unit testing of newer UIs (e.g,. Jasmine and Karma for Angular) is handy too.

Disclaimer: I know more about what John is doing than is covered in the post.

It looks like, for the prebuilt themes, Angular Material itself uses Bazel (see introduction at https://angular.io/guide/bazel). The relevant bits are here:

https://github.com/angular/components/blob/master/src/material/prebuilt-themes/BUILD.bazel
https://github.com/angular/components/blob/master/src/material/core/BUILD.bazel
https://github.com/angular/components/blob/master/src/material/BUILD.bazel

I think that's probably the place to start, in terms of Angular 8 best practices.

Fun fact, you can also order by the column by number.

In this case:

select top 5 Description,Category,* from Cinema.Film order by 2

With correct web server configuration to route everything through the CSPGateway, the above example should handle URLs for other resources like that without issue (as long as the content served by the dashboard server has relative links to those endpoints, not absolute) - that's the point of the <base> element.

In my sample use case, it also handles several requests for images and other assets.

I don't think it's totally stupid. We're looking at doing something similar with another technology (JReport).

Here's a code sample to help you get started - just change the port / page / filters / permission checks appropriately.

Class Demo.CSPProxy Extends %CSP.Page
{

Parameter HOST = "127.0.0.1";

Parameter PORT = 8888;

Parameter PAGE = "jinfonet/runReport.jsp";

/// Event handler for <b>PreHTTP</b> event: this is invoked before
/// the HTTP headers for a CSP page have been sent.  All changes to the
/// <class>%CSP.Response</class> class, such as adding cookies, HTTP headers,
/// setting the content type etc. must be made from within the OnPreHTTP() method.
/// Also changes to the state of the CSP application such as changing
/// %session.EndSession or %session.AppTimeout must be made within the OnPreHTTP() method.
/// It is prefered that changes to %session.Preserve are also made in the OnPreHTTP() method
/// as this is more efficient, although it is supported in any section of the page.
/// Return <b>0</b> to prevent <method>OnPage</method> from being called.
ClassMethod OnPreHTTP() As %Boolean [ ServerOnly = 1 ]
{
    Set request = ##class(%Net.HttpRequest).%New()
    Set request.Server = ..#HOST
    Set request.Port = ..#PORT
    
    // TODO: Add other stuff here, like authentication.
    
    Set page = $Piece(%request.CgiEnvs("REQUEST_URI"),"/"_$classname()_".cls/",2)
    If (page = "") {
        Set %base = $classname()_".cls/"_$Piece(..#PAGE,"/",1,*-1)_"/"
        
        // TODO: add query parameters from %request to the URL requested below.
        $$$ThrowOnError(request.Get(..#PAGE))
    } Else {
        Set fullPage = "http://"_..#HOST_":"_..#PORT_"/"_page
        Do ##class(%Net.URLParser).Parse(fullPage,.parts)
        
        // TODO: Better way of checking the requested resource.
        If $Piece($Piece(parts("path"),"/",*),".",2) = "jsp" {
            Set %response.Status = ##class(%CSP.REST).#HTTP403FORBIDDEN
            Quit 0
        }
        $$$ThrowOnError(request.Send(%request.Method,page))
    }
    Set %data = request.HttpResponse.Data
        
    // TODO: Do any other headers matter?
        
    Set %response.Status = request.HttpResponse.StatusCode
    Set %response.ContentType = request.HttpResponse.ContentType
    Quit 1
}

/// Event handler for <b>PAGE</b> event: this is invoked in order to  
/// generate the content of a csp page.
ClassMethod OnPage() As %Status [ ServerOnly = 1 ]
{
    If (%response.ContentType [ "html") && $Data(%base) {
        &html<<base href="#(..EscapeHTML(%base))#">>
        Do %data.OutputToDevice()
    } Else {
        Do %data.OutputToDevice()
    }
    Quit $$$OK
}

}

A few weird things with custom login pages:

CSPSystem needs read permission on the DB containing them. (You'll need to close CSPGateway connections after making this change so that it can reconnect with the right privileges.)

It's specified with the ".cls" extension in the web application configuration, not just the classname.

You should check the SQLCODE from running that statement. If SQLCODE is 100 (no record exists with the specified key), KeyID may not be defined.

See: https://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=GSQL_esql#GSQL_esql_hvars

"Why?" (a good question) and "Don't do that!" aside, here's a technical answer: override %OnDetermineClass to allow instances of the parent class to be treated as the subclass in question. For example:

Class DC.Demo.Extension.BaseRecord Extends %Persistent
{

Property Name As %String;

ClassMethod %OnDetermineClass(oid As %ObjectIdentity, ByRef class As %String) As %Status [ ServerOnly = 1 ]
{
  Set tSC = ##super(oid,.class)
  If (class = "DC.Demo.Extension.BaseRecord") {
    Set class = $classname()
  }
  Quit tSC
}

Storage Default
{
<Data name="BaseRecordDefaultData">
<Value name="1">
<Value>%%CLASSNAME</Value>
</Value>
<Value name="2">
<Value>Name</Value>
</Value>
</Data>
<DataLocation>^DC.Demo.Extension.BaseRecordD</DataLocation>
<DefaultData>BaseRecordDefaultData</DefaultData>
<IdLocation>^DC.Demo.Extension.BaseRecordD</IdLocation>
<IndexLocation>^DC.Demo.Extension.BaseRecordI</IndexLocation>
<StreamLocation>^DC.Demo.Extension.BaseRecordS</StreamLocation>
<Type>%Library.CacheStorage</Type>
}

}

Class DC.Demo.Extension.SubRecord Extends BaseRecord
{

Property Foo As %String;

Storage Default
{
<Data name="SubRecordDefaultData">
<Subscript>"SubRecord"</Subscript>
<Value name="1">
<Value>Foo</Value>
</Value>
</Data>
<DefaultData>SubRecordDefaultData</DefaultData>
<Type>%Library.CacheStorage</Type>
}

}

Class DC.Demo.Extension.Driver
{

ClassMethod Run()
{
  Do ##class(DC.Demo.Extension.BaseRecord).%KillExtent()

  Set tBaseRecord = ##class(DC.Demo.Extension.BaseRecord).%New()
  Set tBaseRecord.Name = "Fred"
  Write !,"Save base record: ",tBaseRecord.%Save()

  Write !,"Contents of global ^DC.Demo.Extension.BaseRecordD:",!
  zw ^DC.Demo.Extension.BaseRecordD

  Set tSubRecord = ##class(DC.Demo.Extension.SubRecord).%OpenId(1)
  Set tSubRecord.Foo = "Bar"
  Write !,"Open as sub record, name = ",tSubRecord.Name
  Write !,"Save sub record (converts to sub record): ",tSubRecord.%Save()

  Write !,"Contents of global ^DC.Demo.Extension.BaseRecordD:",!
  zw ^DC.Demo.Extension.BaseRecordD
}

}

Output is:

USER>d ##class(DC.Demo.Extension.Driver).Run()
 
Save base record: 1
Contents of global ^DC.Demo.Extension.BaseRecordD:
^DC.Demo.Extension.BaseRecordD=1
^DC.Demo.Extension.BaseRecordD(1)=$lb("","Fred")
 
Open as sub record, name = Fred
Save sub record (converts to sub record): 1
Contents of global ^DC.Demo.Extension.BaseRecordD:
^DC.Demo.Extension.BaseRecordD=1
^DC.Demo.Extension.BaseRecordD(1)=$lb("~DC.Demo.Extension.SubRecord~","Fred")
^DC.Demo.Extension.BaseRecordD(1,"SubRecord")=$lb("Bar")

On IRIS, you can call irisdb.exe the same way you used to call cache.exe.

Compare:

https://docs.intersystems.com/irislatest/csp/docbook/DocBook.UI.Page.cls... (IRIS)

https://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY... (Caché/Ensemble)

The %System/%System/RoutineChange audit event is the only thing other than a source control hook that comes to mind. That's handy for detecting source code changes after the fact (when a class is compiled), but not intercepting attempts to change source code via an editor. What's your specific use case?

After further review, I'm really not sure why the first/second queries don't use the index. The problem with the last query is that %Key is the index in the list, not anything about Tag itself.

Here's a solution that performs well in my testing:

Class DC.Demo.Tag Extends (%Persistent, %Populate)
{

Index Tag On Tag [ Unique ];

Property Tag As %String;

}

Class DC.Demo.Tagged Extends (%Persistent, %Populate)
{

Relationship HasTags As DC.Demo.HasTag [ Cardinality = children, Inverse = Tagged ];

ClassMethod Run()
{
    Do ..%KillExtent()
    Do ##class(DC.Demo.Tag).%KillExtent()
    
    Do ##class(DC.Demo.Tag).Populate(50)
    Do ..Populate(5000)
    Do ##class(DC.Demo.HasTag).Populate(10000)
}

}

Class DC.Demo.HasTag Extends (%Persistent, %Populate)
{

Relationship Tagged As DC.Demo.Tagged [ Cardinality = parent, Inverse = HasTags ];

Property Tag As DC.Demo.Tag [ Required ];

Index UniqueTag On Tag [ IdKey ];

Index TaggedByTag On (Tag, Tagged);

}

It's not the prettiest, but I think the simplest solution would be to avoid navigating to the parent object entirely:

• Add a Foobar property to EmbedObj.
• Via a row/object trigger in ContainerObj, propagate changes to Foobar to EmbedObj_Foobar.
  • As an initial step for data population in existing records, run SQL: update ContainerObj set EmbedObj_Foobar = Foobar
• Base your SQLComputeCode on the copy of Foobar in the serial class.

Re: extending method keywords, you can't do that at this time, but a useful approximation is structuring a comment - for example:

/// @MyKeyword MyValue
ClassMethod TestOne()
{
    // Implementation
}

And then looking at the %Dictionary.MethodDefinition / %Dictionary.CompiledMethod documentation in a generator method. (But it looks like you might already be on to that with @AutoGenerated.)

Re: making first compilation work, this works for me, by making the first compilation automatically trigger a second one when needed:

ClassMethod GenerateMethods() As %Status [ CodeMode = objectgenerator ]
{
    For i=1:1:%class.Methods.Count() {
        #dim method As %Dictionary.MethodDefinition = %class.Methods.GetAt(i)
        Continue:((method.Description["@AutoGenerated") || (method.CodeMode="objectgenerator"))
        Do %class.Methods.Insert(##class(util.TestGenerator).Generate(method.Name_"DoSomethingElse",%class.Name))
    }
    If (%class.Methods.%IsModified()) {
        Do ##class(%Projection.AbstractProjection).QueueClass(%class.Name)
    }
    Quit %class.%Save()
}

Test code:

Class util.Driver
{

ClassMethod Run()
{
    do ##class(%Dictionary.MethodDefinition).IDKEYDelete("util.Test","TestOneDoSomethingElse")
    do ##class(%Dictionary.MethodDefinition).IDKEYDelete("util.Test","TestTwoDoSomethingElse")
    do $system.OBJ.UnCompile("util.*")
    do $system.OBJ.Compile("util.*","ck")
}

}

If you want to count (or otherwise traverse) all the elements in a multidimensional array, you can use $Query - here's a sample with method that does that:

ClassMethod Run()
{
    Set a(1) = "blue"
    Set a(1,3,5,2) = "navy"
    Set a(2) = "red"
    Set a(2,2) = "crimson"
    Set a(3) = "yellow"
    
    Write ..Count(.a)
}

ClassMethod Count(ByRef pArray) As %Integer [ PublicList = pArray ]
{
    Set count = 0
    Set ref = $Query(pArray(""))
    While (ref '= "") {
        Set count = count + 1
        Set ref = $Query(@ref)
    }
    Quit count
}