· Apr 20, 2016

Experimental project for unit testing + coverage; and help needed

Hello community!

Heretofore is announced a new project which aims at providing a usable library for both running unit tests and collecting code coverage information at the same time:

I shamelessly admit that this is my first project written in ObjectScript; the only source file right now barely loads a %Studio.Project instance on a given namespace and fails ungracefully on failure (it HALTs; meh); and even on success it will not even list the items correctly... Well, that's part of the learning curve.

This project is of course not "innocent": for CachéQuality, we want that information (the result of unit tests and coverage information). But we believe that it is in the interest of a lot of people that a package exists which provides such information in as generic a way as possible.

While no license file is present yet, this project will be licensed under the MIT license (this seems to be the license of choice for all current open source COS projects I've seen so far).

As this is the first item on my TODO list, expect frequent code updates... And a flood of questions :)


Discussion (14)2
Log in or sign up to continue

Somewhat related to this, I think there's still a useful role for %Studio.Project projects as ad-hoc server-side collections of classes etc that can be assembled independently of your package hierarchy, which typically has to remain far more static. But it's not yet clear to me how these projects (which after all are more correctly called "Studio projects") will evolve/mutate/expire in the brave new world of Atelier.

Although I do foresee the need to run all unit-tests in current namespace or for the seelected package, but from developers prospective I will need to run only those unit-test which are connected to the current project I'm working on. And for that I use Studio project (or would use separate Atelier workspaces once and if I'd migrate to Atelier).

So it's a proper approach IMVHO to start from project items and then add wider collections (package(s) or namespace wide).

Since project is a collection of classes (and other items, yes, but we're talking about classes here) the  flow would be something like that:

  • Get project name
  • Project name to class list
  • Get unit tests for class list
  • Run unit tests

I'm just saying that 3rd step can have more callers (alternative routes to 3rd step if you will), for example:

  • Package to class list
  • Wildcard expression to class list
  • Namespace to class list

Ok, Eduard, let me elaborate using some fictional, ideal picture.

  • Let assume you are part of large team which is working on some pretty huge system, say multiple hundreds or even few thousands of classes in the single namespace. And whole your team are good citizens and have covered all classes with unit-tests pretty extensively.
  • Let assume you are working on a changes to classes which are part of pretty-large "system" package (i.e. ocnsisting of few hundreds of unit-tests)
  • And you have changed only 2 classes in the package, which is covered by dozen of unit-tests here and where.

And as a good citizen you always run all relevant unit-tests for changed classes before committing to VCS. So what would you choice before commit: to run unit-tests for whole package (which might take hours) or only for your project with relevant classes (which will take few minutes)?

Here is an example how you should code in ObjectScript:


ClassMethod listClasses(ns As %String = {$namespace}, projectName As %String) As %Status
    #dim tSC As %Status = $$$OK
    #dim tException As %Exception.StatusException = $$$NULLOREF
    #dim tOrigNS = $namespace
    try {
        // Switch namespaces to the new one
        set $namespace = ns

        // Grab our project, by name; fail otherwise
        // TODO: failing is CRUDE at this point...
        #dim project as %Studio.Project
        #dim status as %Status

        // TODO: note sure what the "concurrency" parameter is; leave the default which is -1
        set project = ##class(%Studio.Project).%OpenId(projectName,, .status)
        if ($$$ISERR(status)) {
            Throw ##class(%Exception.StatusException).CreateFromStatus(status)
        #dim item as %Studio.ProjectItem

        #dim key = ""

        while (project.Items.GetNext(key)'="") {
            set key = project.Items.GetNext(key)
            set item = project.Items.GetAt(key)
            if ($IsObject(item)) {
                write "Name: " _ item.Name _ ", type: " _ item.Type, !
    } catch (tException) {
        Set tSC = tException.AsStatus()
    set $namespace = tOrigNS
    quit tSC

It's to change the namespace later back to the namespace you're comming from.

ClassMethod DoIt() As %Status
    //Current Namespace = "A"
    Do ..MethodThatChangesNamespace()
    //Namespace is now "B"
    Do ##class(Class).AnotherMethod()
    //Results in "NOT FOUND"


Edit: I don't know if "new $namespace" results in changing back the namespace. I got some problems changing namespaces in methods. Maybe it's changed since Version 2008.