Return Error Code After Unit Tests to CircleCI Job

Question

Question

Michael Davidovich · Aug 12, 2022

#DevOps #Testing #InterSystems IRIS for Health

Does anyone have experience with CircleCI or really any automation tool using %UnitTest?

I have built my app and ran my unit test in a Docker container automated by CircleCI. However, after many of the tests failed I see:

How do you halt out of the IRIS session and return an exit code to the shell?

I have a script in my repo that I pass into IRIS after CircleCI does the code checkout

iris session IRIS < inFile

Am I trying to approach this the wrong way? Should I be writing test output to a file or something? Even so, I still need a way to say to CirclCI "unit tests failed, so don't do the next step of building the code on the actual server" and to send out alerts so we know it failed.

Discussion (8)2

Log in or sign up to continue

Timothy Leavitt · Aug 12, 2022

@Michael Davidovich , try this:

Do $System.Process.Terminate($Job,<desired error code>)

See e.g. https://github.com/intersystems-community/zpm/blob/master/src/%25ZPM/Pac...

1 0

Timothy Leavitt · Aug 15, 2022

@Eduard Lebedyuk it depends on the caller. In a CI process I could imagine doing different error handling for failed compilation vs. failed unit tests, this would be a way to signal those different modes of failure.

I've taken/seen approaches that are more shell-centric vs. more ObjectScript-centric which would be a driver for this being useful. With the package manager it's generally simpler to wrap things in <Invoke> or resource processors and then call IRIS with individual zpm commands (i.e., load then test) from CI. For some of my pre-package manager CI work we've had a big ObjectScript class that wraps all the build logic, running different sorts of tests, etc. In this case it would be useful to indicate the stage at which the failure occurred.

Regardless, $System.Process.Terminate is simpler to manage than "flag files" for sure, which would be the next best alternative. (IIRC in old days, maybe just pre-IRIS, there were Windows/Linux differences in $System.Process.Terminate's behavior, and that led us to use flag files.)

0 0

score 0 · Answer 1 · 2022-08-12T15:49:59-04:00

Michael Davidovich · Aug 12, 2022

Thanks, Tim!

0 0

score 0 · Answer 2 · 2022-08-12T16:08:47-04:00

@Timothy Leavitt

Another question popping up. How do I access the test result on the command line to see if it failed? When I run DebugRunTestCase, the the URL spits out with the result index in it. I created an instance of Manager and then ran the method but I don't seem to get any properties or output that say "this failed."

Thoughts here?

score 0 · Answer 3 · 2022-08-12T16:15:31-04:00

It's not exactly straightforward. See https://github.com/intersystems-community/zpm/blob/master/src/%25ZPM/Pac... for an example of how to work around it / get the answers you need.

score 0 · Answer 4 · 2022-08-12T16:18:36-04:00

Ok thanks, I'll take a look.

This begs the question: if it's not straightforward, is this a typical implementation? The CircleCI examples give all these test examples that are run from the command line (e.g. python test, npm test), but of course the IRIS setup is a little different.

score 1 · Answer 5 · 2022-08-12T16:33:56-04:00

Timothy Leavitt · Aug 12, 2022

This is a typical implementation.

1 0

score 1 · Answer 6 · 2022-08-13T03:14:45-04:00

Can skip $Job, leaving just:

Do $System.Process.Terminate(,<desired error code>)

it would terminate the current job.

Question: are you using the error code? I'm always exiting with error code 1 in case of any errors, but is there an advantage in using some custom error codes here?