Debugging Unittests in InterSystems IRIS
Hi folks!
Those who actively use unittests with ObjectScript know that they are methods of instance but not classmethods.
Sometimes this is not very convenient. What I do now if I face that some test method fails I COPY(!) this method somewhere else as classmethod and run/debug it.
Is there a handy way to call the particular unittest method in terminal? And what is more important, a handy way to debug the test method?
Why do we have unittest methods as instance methods?
VSCode has a way to help with running tests, but it requires implementing from our side
Thanks, Dima! Is there a listed related issue?
Unit testing from within VS Code is now possible. Please see https://community.intersystems.com/post/intersystems-testing-manager-new...
thanks, @John Murray ! Will give it a try!
A handy way to call UnitTest Case method in a terminal
Do ##class(%UnitTest.Manager).DebugRunTestCase("", "[ClassName]", "", "[MethodName]")
Run all Test methods for a TestCase:
Do ##class(%UnitTest.Manager).DebugRunTestCase("", "[ClassName]", "", "")
Placing a "break" line within a method can be useful when iterating creating the test. See the variables. Run other code and then type "g"+ [Enter] to continue.
The instance gives context to current test run, when raising assertions and other functionality.
Thanks Alex.
See the following:
USER>Do ##class(%UnitTest.Manager).DebugRunTestCase("", "dc.irisbi.unittests.TestBI", "", "TestPivots")
(root) begins ...
LogStateStatus:0:Finding directories: ERROR #5007: Directory name '/usr/irissys/mgr/user/u:/internal/testing/unit_tests/' is invalid <<==== **FAILED** (root)::
Hi Evgeny,
The global ^UnitTestRoot needs to be set to a real directory to satisfy the CORE UnitTest Runner.
As the first argument "Suite" is not specified, then no sub-directories are needed.
Set ^UnitTestRoot="\tmp\"
... may be sufficient for your purposes.
In fact there is a handy way to run all the tests via:
zpm "test module-name"
But, I'd love to see debugging of it
It is common to need to run unit tests for other modules that have overlapping / breaking functionality.
This is where the value of loading and running multiple TestSuites comes into its own.
with zpm you can use additional parameter for it
zpm "test module-name -only -D UnitTest.Case=Test.PM.Unit.CLI:TestParser"Use class name only to run all tests there, or add Method name to test only that method in the class
Thanks! this is helpful.
@Alex Woodhead , do you know by a chance why unittest methods are instance methods but not classmethods? Could it be converted to classmethods or provided the option to do that?
There are no reasons for them to be ClassMethods, UnitTests is quite a complex thing, and it's there are use-cases where it needs to be this way.
I am utilizing properties on class methods to good effect.
I would not use a classmethod only approach for normal development.
There is nothing stopping a community parallel UnitTest.Manager re-implementation that follows a ClassMethod pattern.
Some have reimplemented UnitTest.Manager:
1) Without Deleteing the test classes at end of run (With Run instead of DebugRun)
2) Not needed ^UnitTestRoot to be defined.
@Evgeny Shvarov I have a detailed writeup here (although Dmitry already hit the important point re: IPM): https://community.intersystems.com/post/unit-tests-and-test-coverage-obj...
A few other notes:
Unit test class instances have a property (..Manager) that refers to the %UnitTest.Manager instance, and may be helpful for referencing the folder from which unit tests were loaded (e.g., to load additional supporting data or do file comparisons without assuming an absolute path) or "user parameters" / "user fields" that were passed in the call to run tests (e.g., to support running a subset of tests defined in unit test code). Sure, you could do the same thing with PPGs or % variables, but using OO features is much better.
I'll also often write unit tests that do setup in OnBeforeAllTests and cleanup in %OnClose, so that even if something goes very horribly wrong it'll have the best chance of actually running. Properties of the unit test are useful to store state relevant to this setup - the initial $TLevel (although that should always be 0), device settings, global configuration flags, etc.
Thanks Tim! How do you debug a particular unit test in VSCode?
@Evgeny Shvarov my hacky way of doing this is to create an untracked mac file with
ROUTINE debugdefined.I just swap in the suite or method I want to run per Alex's instructions, set my breakpoints in VS code and make sure the debug configuration is in my VSCode settings:
"launch": {"version": "0.2.0","configurations": [{"type": "objectscript","request": "launch","name": "debugRoutine","program": "^debug"}]}That's indeed hacky! Thanks @Michael Davidovich!
Why are unit test methods instance methods? Since a running unit test is an instantiated object (%RegisteredObject), the unit test class itself can have custom properties, and the instance methods can use those properties as a way to share information. For example, initialize the properties in %OnBeforeAllTests(), and then access/change the properties in the test methods.
Thanks @Joel Solon!
But all this could be achieved without instance methods, right? Anyway, I'm struggling to find an easy way to debug a failed unittest. @Michael Davidovich suggested the closest way to achieve it but I still want to find something really handy, e.g. an additional "clickable button" in VSCode above the test method that invites "debug this test method". Similar what we have for class methods now - debug this classmethod and copy invocation.
That'd be ideal.
That's one of the things the new extension is designed to achieve.
Looking forward!
Anything can be achieved without instance methods. The point here is that instance methods exist in object-oriented systems because they are considered a good, straightforward way to achieve certain things. In the case of unit tests sharing information using properties, that approach saves you from having to pass info around as method arguments, or declaring a list of public variables.
I understand the argumentation, makes sense. Just curious how do you debug those unittests that fail?
💡 This question is considered a Key Question. More details here.