Unit test in InterSystems with Jenkins
Hi ,
I am stuck with unit test failure with intersystem . In case of unit test failure, the build in jenkins is succeding while the build in jenkins should fail in case unit test failure .
In cache programming i am using %UnitTest.Manager class and DebugRunTestCase method within it. I'm able to link studio with jenkins. I wanna fail my build in jenkins, if any of the test cases fails. Could anyone help?
Please, provide how you call Caché from Jenkins?
You can create a windows batch file, where you can write commands to call Cache terminal using cterm.exe
One gotcha in windows is making sure the encoding on your .scr file is in UTF-8. Windows thinks a .scr file is a screensaver file and can give it a weird encoding. Once I switched it to UTF-8 it worked as described in the documentation.
I don't know how you call Caché method from Jenkins, but anyway you can use $SYSTEM.Process.Terminate in Caché script to exit with an exit status. Something like this.
I suggest that you may use csession or cterm to call Caché code, then you should get exit code and send it to Jenkins, which will be recognized by Jenkins as an error and will fail the job.
Could you please help us in Clarifying the below Query.
Is there any methods/ways through which will get to know whether any of the Unit Test cases is/are failing (we are getting an url of the csp page with the report which has the passed failed status) as we need to send this failure status to Jenkins for the Build to Fail (where in we have acheive this part making the build failure/success based on harcoded boolean)
Here's an object and SQL approaches to get Unit Tests status.
Hi,
We use something like the below to output the unit test results to an xml file in JUnit format.
/// Extend %UnitTest manager to output unit test results in JUnit format. /// This relies on the fact that unit test results are stored in <b>^UnitTest.Result</b> global. Results displayed on CSP pages come from this global. Class com.isc.UnitTest.Manager Extends %UnitTest.Manager { ClassMethod OutputResultsXml(pFileName As %String) As %Status { set File=##class(%File).%New(pFileName) set i=$order(^UnitTest.Result(""),-1) if i="" quit $$$OK // no results kill ^||TMP // results global set suite="" for { set suite=$order(^UnitTest.Result(i,suite)) quit:suite="" set ^||TMP("S",suite,"time")=$listget(^UnitTest.Result(i,suite),2) set case="" for { set case=$order(^UnitTest.Result(i,suite,case)) quit:case="" if $increment(^||TMP("S",suite,"tests")) set ^||TMP("S",suite,"C",case,"time")=$listget(^UnitTest.Result(i,suite),2) set method="" for { set method=$order(^UnitTest.Result(i,suite,case,method)) quit:method="" set ^||TMP("S",suite,"C",case,"M",method,"time")=$listget(^UnitTest.Result(i,suite,case,method),2) set assert="" for { set assert=$order(^UnitTest.Result(i,suite,case,method,assert)) quit:assert="" if $increment(^||TMP("S",suite,"assertions")) if $increment(^||TMP("S",suite,"C",case,"assertions")) if $increment(^||TMP("S",suite,"C",case,"M",method,"assertions")) if $listget(^UnitTest.Result(i,suite,case,method,assert))=0 { if $increment(^||TMP("S",suite,"failures")) if $increment(^||TMP("S",suite,"C",case,"failures")) if $increment(^||TMP("S",suite,"C",case,"M",method,"failures")) set ^||TMP("S",suite,"C",case,"M",method,"failure")=$get(^||TMP("S",suite,"C",case,"M",method,"failure")) _$listget(^UnitTest.Result(i,suite,case,method,assert),2) _": "_$listget(^UnitTest.Result(i,suite,case,method,assert),3) _$char(13,10) } } if ($listget(^UnitTest.Result(i,suite,case,method))=0) && ('$data(^||TMP("S",suite,"C",case,"M",method,"failures"))) { if $increment(^||TMP("S",suite,"failures")) if $increment(^||TMP("S",suite,"C",case,"failures")) if $increment(^||TMP("S",suite,"C",case,"M",method,"failures")) set ^||TMP("S",suite,"C",case,"M",method,"failure")=$get(^||TMP("S",suite,"C",case,"M",method,"failure")) _$listget(^UnitTest.Result(i,suite,case,method),3) _": "_$listget(^UnitTest.Result(i,suite,case,method),4) _$char(13,10) } } if $listget(^UnitTest.Result(i,suite,case))=0 && ('$data(^||TMP("S",suite,"C",case,"failures"))) { if $increment(^||TMP("S",suite,"failures")) if $increment(^||TMP("S",suite,"C",case,"failures")) if $increment(^||TMP("S",suite,"C",case,"M",case,"failures")) set ^||TMP("S",suite,"C",case,"M",case,"failure")=$get(^||TMP("S",suite,"C",case,"M",case,"failure")) _$listget(^UnitTest.Result(i,suite,case),3) _": "_$listget(^UnitTest.Result(i,suite,case),4) _$char(13,10) } } } do File.Open("WSN") do File.WriteLine("<?xml version=""1.0"" encoding=""UTF-8"" ?>") do File.WriteLine("<testsuites>") set suite="" for { set suite=$order(^||TMP("S",suite)) quit:suite="" do File.Write("<testsuite") do File.Write(" name="""_$zconvert(suite,"O","XML")_"""") do File.Write(" assertions="""_$get(^||TMP("S",suite,"assertions"))_"""") do File.Write(" time="""_$get(^||TMP("S",suite,"time"))_"""") do File.Write(" tests="""_$get(^||TMP("S",suite,"tests"))_"""") do File.WriteLine(">") set case="" for { set case=$order(^||TMP("S",suite,"C",case)) quit:case="" do File.Write("<testsuite") do File.Write(" name="""_$zconvert(case,"O","XML")_"""") do File.Write(" assertions="""_$get(^||TMP("S",suite,"C",case,"assertions"))_"""") do File.Write(" time="""_$get(^||TMP("S",suite,"C",case,"time"))_"""") do File.Write(" tests="""_$get(^||TMP("S",suite,"C",case,"tests"))_"""") do File.WriteLine(">") set method="" for { set method=$order(^||TMP("S",suite,"C",case,"M",method)) quit:method="" do File.Write("<testcase") do File.Write(" name="""_$zconvert(method,"O","XML")_"""") do File.Write(" assertions="""_$get(^||TMP("S",suite,"C",case,"M",method,"assertions"))_"""") do File.Write(" time="""_$get(^||TMP("S",suite,"C",case,"M",method,"time"))_"""") do File.WriteLine(">") if $data(^||TMP("S",suite,"C",case,"M",method,"failure")) { do File.Write("<failure type=""cache-error"" message=""Cache Error"">") do File.Write($zconvert(^||TMP("S",suite,"C",case,"M",method,"failure"),"O","XML")) do File.WriteLine("</failure>") } do File.WriteLine("</testcase>") } do File.WriteLine("</testsuite>") } do File.WriteLine("</testsuite>") } do File.WriteLine("</testsuites>") do File.Close() kill ^||TMP quit $$$OK } }
Please tell us very clearly how to exit the cache terminal via command line and then that batch script should fail the jenkins job ? Please tell us if it is feassible in intersystem . We have tried with all methods from intersystem doc library but still we could not progress (Jenkins job is not getting triggered based on boolean value passed from cache). Please suggest
Hi Ankita,
Would you be able to provide some more details about how you currently run the unit tests from your batch script?
Regards,
Caza
Hi Caza ,
We are facing the following issues :-
1. We are getting the boolean value for the pass and fail unit test cases in cache Intersystem, but we are not able to assingn this boolean value into a variable in a batch script( r3 in this case)
2. @ECHO %FAILUREFLAG% is not giving any output to us, can you help with that also.
Please suggest us for the following problems or some alternative approach. The Batch script code is here :-
Hi Ankita,
I found a couple of issues in the script that might affect your end results:
- the folder C:\unittests doesn't exist (at least not on my computer); unless the value of the WORKSPACE env variable is C:\untittest then you have to ensure the folder exists (you can create it either using batch
mkdir
or using COS##class(%%File).CreateDirectoryChain()
method )- what is stored in the
^UnitTest.
Result(i,"
myunittest
")
global is not a status code but a numeric value; so I would suggest replacing Do $system.OBJ.DisplayError(r3) with a simple write command, like this:Regarding "@ECHO %FAILUREFLAG%" - make sure there are no spaces before or after the = character in the following two commands:
When I did copy/paste of the example script I ended up with a space character before the = character.
Can you try these changes and let me know how you go?
Cheers,
Caza
Thanks for your response it worked
Yay!
If you add to your batch script something like:
then you could pass on the unit_test_results.xml file to the JUnit plugin from Jenkins. This will give you useful reports, duration and individual results breakdown, etc.
For example: https://support.smartbear.com/testcomplete/docs/_images/working-with/integration/jenkins/trend-graph.png