Eduard Lebedyuk · Sep 22, 2022 go to post

1. I have a PDF file which I need to read  from a folder location as text and put data from PDF into HL7 message and send it to downstream system.

Do you mean OCR/text layer extraction?

2. I have a PDF file which I need to read  from a folder location encode it in base64 and put in OBX.5  of MDM message

Do it like this.

Eduard Lebedyuk · Sep 19, 2022 go to post

Have you tried executing the same code from the InterSystems terminal?

Also try:

text = self.db.run_class_method("%SYSTEM.Status", "GetErrorText", res)
print(text)
Eduard Lebedyuk · Sep 16, 2022 go to post

Try to reload like this:

set importlib = ##class(%SYS.Python).Import("importlib")
do importlib.reload(helloWorld)

Also, it not an IRIS-specific behavior, you'll get the same results in any python interpreter:

import helloWorld
helloWorld.helloWorld()
>'Hello world'
del helloWorld

# modify helloWorld.py in text editor

import helloWorld
helloWorld.helloWorld()
>'Hello world'
Eduard Lebedyuk · Sep 13, 2022 go to post

I'm sure pattern matching can do better but no idea how:

ClassMethod findShort(s) As %Integer [ ProcedureBlock = 0 ]
{
 s s=" "_s_" " for i=1:1 {x "s q=(s?.E1P"_i_"A1P.E)" q:q} q i
}
Eduard Lebedyuk · Sep 13, 2022 go to post

Check this example.

In short:

  1. Create class extending %SYS.Task.Definition
  2. Add properties - that's task settings
  3. Implement OnTask method, which returns %Status
  4. Set TaskName parameter

After that you can add a task of TaskName type from the SMP.

Eduard Lebedyuk · Sep 9, 2022 go to post

Will it run in the same Windows process?

Yes.

Will there be any issues with multitasking (considering python doesn't seem very good at this)?

GIL still exists and applies. If you write async code it would only be executed while control flow is on a python side of things. You can't spawn async task in python, go to InterSystems ObjectScript to do something else and then come to a completed python task.

Also, is there a performance penalty to pay for running embedded python vs "using IRIS APIs from Python". 

IRIS APIs from Python (Native SDK/Native API) can be invoked either in-shared-memory or over TCP. TCP comes with a performance penalty.

Another question is what python interpreter the embedded python is using? Is it an Intersystems one or the regular c.python?

CPython.

Version?

Use sys.version to check. Recently it was Python 3.9.5 on Windows and 3.8.10 on Linux.

Eduard Lebedyuk · Sep 1, 2022 go to post

Except that calling Python is about 10x slower, a

Not really, more like faster if you need to call it more than once:

 

Code

Class Utils.Time
{

/// do ##class(Utils.Time).Test()
ClassMethod Test()
{
	do ..Main(1)
	do ..Main(1)
	do ..Main(1)
	do ..Main(2)
	do ..Main(3)
	do ..Main(5)
	do ..Main(10)
	do ..Main(100)
	do ..Main(1000)
	do ..Main(10000)
}

/// do ##class(Utils.Time).Main(100)
ClassMethod Main(count = 100)
{
	set od=$io
	set nul="\\.\nul" // /dev/null/ - UNIX
	open nul
	use nul
	
	s startA = $NOW()
	do ..JobA(count)
	s endA = $NOW()
	s timeA = $p(endA,",",*) - $p(startA,",",*)
	
	s startB = $NOW()
	do ..JobB(count)
	s endB = $NOW()
	s timeB = $p(endB,",",*) - $p(startB,",",*)
	
	use od
	close nul
	
	w "Iterations: ",count,!
	w "Time JobA: ",timeA,!
	w "Time JobB: ",timeB,!
	w "JobA takes ",$FN(timeA/timeB*100,"",2) _ "% of JobB time",!,!
}

ClassMethod JobA(count = 100) As %Status
{
	for i=1:1:count {
		set now = $now()
		set ts = $zdt(now,-2) _ "." _ $p(now, ".", 2)
	}
}

ClassMethod JobB(count = 100) As %Status
{
	set time = ##class(%SYS.Python).Import("time")
	for i=1:1:count {	
		set ts = time.time()
	}
}
}

Results in:

 

Output

Iterations: 1
Time JobA: .0000257
Time JobB: .0000728
JobA takes 35.30% of JobB time
 
Iterations: 1
Time JobA: .0000154
Time JobB: .0000206
JobA takes 74.76% of JobB time
 
Iterations: 1
Time JobA: .0000157
Time JobB: .0000158
JobA takes 99.37% of JobB time
 
Iterations: 2
Time JobA: .0000214
Time JobB: .0000161
JobA takes 132.92% of JobB time
 
Iterations: 3
Time JobA: .0000264
Time JobB: .0000158
JobA takes 167.09% of JobB time
 
Iterations: 5
Time JobA: .0000528
Time JobB: .0000258
JobA takes 204.65% of JobB time
 
Iterations: 10
Time JobA: .0000987
Time JobB: .0000256
JobA takes 385.55% of JobB time
 
Iterations: 100
Time JobA: .0007513
Time JobB: .0000606
JobA takes 1239.77% of JobB time
 
Iterations: 1000
Time JobA: .0057593
Time JobB: .0002278
JobA takes 2528.23% of JobB time
 
Iterations: 10000
Time JobA: .0535617
Time JobB: .0036764
JobA takes 1456.91% of JobB time
Eduard Lebedyuk · Sep 1, 2022 go to post

1. Yes. Stopping primary makes backup a new primary. Before stopping you might want to validate that backup is caught up:

set sql = "SELECT CASE WHEN DatabaseLatency=? THEN 1 ELSE 0 END IsCaughtUp FROM SYS.Mirror_MemberStatusList() WHERE CurrentRole = ?"
set rs = ##class(%SQL.Statement).%ExecDirect(,sql, "Caught up", "Backup")
do rs.%Next()
if rs.IsCaughtUp {
	write "Caught up"
	halt
} else {
	write "Not caught up"
	do $system.Process.Terminate(,1)
}

Or at least is not too far behind (check this post).

2.  No. ^MIRROR does the same.

Eduard Lebedyuk · Sep 1, 2022 go to post

There's a distinction between Background Jobs and Background Tasks.

Backgound job is anything you run using JOB command. Background Task is a limited set of named actions (AuditCopy, AuditExport, AuditPurge, Backup, CompactDBSpace, CopyNamespaceMappings, CreateDatabase, Compile, DatabaseIntegrityCheck, DataMigration, DefragmentDB, Delete, EnableEnsNamespace, Export, ExternalLangServers, FileMan, Import, SQLExport, SQLImport, SQLExportStatement, SQLImportStatement, QueryExport, JournalIntegrityCheck, LinkTable, LinkProcedure, MirrorActivateCatchupDB, MirrorRemoveDB, MirrorMountDB, MirrorAddDatabases, ModifyDatabaseSize, RebuildIndices, TuneTable, TuneTables, PurgeAllCachedQueries, JobShowPlan, JobSaveQuery, JobPossiblePlans, JobComparePlans, ShardActivate, ShardAssign, ShardVerify, ShardRebalance) runnable through a special interface. Docs.

You can run a Background Task (but probably should not  - it's a system action) by calling:

set parms("ClassName") = "Sample.Person"
set tSC = ##class(%CSP.UI.System.BackgroundTask).RunTask("RebuildIndices", "SAMPLES", .parms, .job)

As HIHLib.Support.GetHL7MessageStat:ISBListingQuery is not on the list you can't run it as a Background Task, but you can run it as a Job.

Next - no output. Note that you do not specify these parameters:

principal-input Principal input device for the process. The default is the null device.
principal-output
Principal output device for the process. The default is the device you specify for principal-input or the null device if neither device is specified.
UNIX®: If you do not specify either device, the process uses the default principal device for processes started with the JOB command, which is /dev/null.

Further:

Only one process can own a device at a time. This means that a job executing in a JOB Server is unable to perform input or output to your principal I/O devices even though you may close device 0.

So by default you run your job with stdio set to /dev/null and that's why there's no output.

You can either pass a device to write output to or write a wrapper which would handle the output and job that.

Eduard Lebedyuk · Aug 27, 2022 go to post

Sounds like you're writing SELECT * queries. While fine for debugging or development, remember to always explicitly specify the list of columns in actual production queries.