Edit.. for some reason I didnt see the above reply.

Given that you are extending Ens.BusinessProcess you have access to Ensemble/Interoperability.  If you are going down this path you might want to review RecordMaps which is a general purpose way of consuming data from files via a business service based on EnsLib.RecordMap.Service.FileService.  This will open the file, for each line create an instance of your RecordMap object and then if you define in the service the TargetConfigNames sent the Record as a message to commonly a business process.  In the busines process you will receive each line as a message object.  From there you can do what you like with it.

This could be a bit overkill if all you have is a single column in the file(just a column for ID) but is a general way of handling the ingestion of files.

I'm assuming folks from InterSystems have a more formal way of saying "you have a partner with InterSystems".  When the occasional crisis occurs, the support of many folks from WRC rally around your issue and see it to the end.  Having the ability to contact someone on the other end and resolve crisis issues is not something you are likely to get from other vendors.

Looking at your example class, and maybe its not practical as we aren't seeing the entire context of your real world usage but might you see better resuilts if you were using instead of a composite index on prop1, prop2, prop3 but rahter bitmap indices on prop1 as it seems like it is the field name and would have a suffeceintly small number of values, a bitmap index on prop2, and a traditional index on prop3.  

If you were to show the query plan that would also provide some insight.

I would imagine if you have a license from Intersystems you could install Cache, mount the DB, and then the next question is "Was the application developed back in 2017" with an SQL interface, ie are there SQL tables that exist for the database that you have?"  If so you could connect to the db via ODBC and extract the data.

If not then that's a much large endeavor as you would need to have the tables defined and depending on how much meta-data/documentation you have for the data that could be quite involved.

some thoughts on using custom ZEN Components that I follow that might help.

  1. In my custom component I have 
    ///  If true, then the include files generated for this component, are placed in the
    ///  common /csp/broker directory and not the local /csp/*namespace* directory.<br/>
    ///  This is intended for use with packages that are mapped so as to be visible to every namespace.<br/>
    ///  All Zen classes within the same package must have the same value for this parameter.
    ///  It is the developer's responsibility to ensure this.
    Parameter USECOMMONDIRECTORY As BOOLEAN = 1;
    and
/// XML namespace for component.
Parameter NAMESPACE = "http://www.readycomputing.com/zen";

2. In my ZEN page in 

<page xmlns="http://www.intersystems.com/zenxmlns:RC="http://www.readycomputing.com/zen"

if you are using Package Mapping with a common database for your classes I found using USECOMMONDIRECTORY neccessary.

Still not fully able to grasp what you are doing but if are trying to address 

log.cls_id IN (61) AND log.q_id IN (19, 25, 27)

I often find myself doing something like

log.cls_id %INLIST $LISTFROMSTRING(':Param1,',')

AND log.q_id %INLIST $LISTFROMSTRING(':Param2,',')

and then set Param1=61 and Param2 = "19, 25, 27"

so long as you know the values of Param1 and Param2. 

%INLIST is documented https://docs.intersystems.com/iris20241/csp/docbook/Doc.View.cls?KEY=RSQ...

and

$LISTFROMSTRING is documented here https://docs.intersystems.com/iris20241/csp/docbook/Doc.View.cls?KEY=RSQ...

Regarding

If the first bullet item in the Query Plan is “Read master map”, or the Query Plan calls a module whose first bullet item is “Read master map”, the query first map is the master map rather than an index map. Because the master map reads the data itself, rather than an index to the data, this almost always indicates an inefficient Query Plan. Unless the table is relatively small, we should create an index so that when we rerun this query the Query Plan first map says “Read index map.”

I think it's more subtle than that.  My general plan of attack is to review the results of Show Plan and then search for Looping.  If the first bullet item in Show Plan is one of the following

  • Read master map Ens.MessageHeader.IDKEY, using the given idkey value.
  • Read master map Ens.MessageHeader.IDKEY, looping on ID (with a range condition).

I'm not immediately concerned.  In the first case this is going directly to the row which is perfectly fine.  

In the second case so long as the range condition is not going to read the entire extent I can accept that and not look for a better query plan.

Honestly, where its 

Read master map Ens.MessageHeader.IDKEY, looping on ID (with a range condition).

or

Read index map Ens.MessageHeader.TimeCreated, looping on TimeCreated (with a range condition) and ID.

I don't care if its the master map or the index map, what I'm interested in is Looping and does looping cause the engine to look at the entire extent or index.

You wrote

The module.xml has the one line which describes unit tests:

<UnitTest Name="src" Package="UnitTests" Phase="test"/>

This line means that  ZPM will look in /src folder of the repo and will run all the tests of all the classes which are included in UnitTests packageIs 

Is this to say the Name="src" means it will look on the machine where the module has been installed in the /src folder?  If this is so this is problematic for us I believe.  We are trying to protect our IP and as part of the Module.xml definition we have 2 Invokes that help with this effort

1. <Invoke Class="RCIPM.Utils" Method="MakeModuleDeployed" Phase="Compile" When="After" CheckStatus="false">  We do this because there are some classes which cannot be marked as Deployed as they need to be compiled additional times on the application namespaces after we have run the install command in HSCUSTOM

2.  <Invoke Class="RCIPM.Utils" Method="DeleteModulesFolder" Phase="Activate" When="After" CheckStatus="true">

If we don't delete the Module folder then our classes are there in plain text and could be lifted off of the server.

so if Name="src" mean look in the /src folder on the target machine(not the registry) then this wouldn't work for us.

That's not to say that DeleteModulesFolder doesn't cause other issues but its currently what we feel we need to do to protect our code.