Not entirely dogmatic but I've always found https://github.com/jnguyen095/clean-code/blob/master/Clean.Code.A.Handb… to be a good guide.
this has sections on veriticle and horizontal formatting/density/distance. Again, not a specific answer but I use it as a guide.
I'm struggling to understand your original question "We are attempting to "Repoint" old class data to new class data to save disk space and data redundancy across multiple tables. "
Will "Repoint"ing acutall save disk space? At first glance it would be hard to see how that would make a significant difference. It might be true that the old class has a number of properties that are no longer used so in the physical storage you have a number of unused storage element but I wouldnt think that would make a significant difference.
One area that can make a difference is if your classes utilize bitmap indices and if you have rapidly changing data with fields are indexed. IRIS delivers %SYS.Maint.Bitmap which is described as
This utility is used to compact bitmap/bitslice indices. Over time in a volatile table (think lots of INSERTs and DELETEs) the storage for a bitmap index may become less efficient. To a lesser extent index value changes, i.e. UPDATES, can also degrade bitmap performance.
This utility can by run on a live system. It will loop over all bitmap/bitslice indices in a class or namespace and compact them by either removing a chunk that is all zeros or by using $BITLOGIC() to compact the bits.
Consider Ens.MessageHeader which has a bitmap index on Status where the Status values are
1=Crerated
2=Queued
3=Delivered
4=Discarded
5=Suspended
6=Deferred
7=Aborted
8=Error
9=Completed
You can imagine that a Message Header is save with
Event 1 Status=Created
Event 2 Status=Queued
Event 3 Status=Delivered
Event 4 Status = Completed
in the normal course of processing
This would have the side effect of
1. Event 1 setting the Status =Created bit on
2. Event 2 Setting the Status=Created bit Off, Setting the Status=Queued On
3. Event 3 Setting the Status=Queued bit Off, Setting the Status=Delivered On
4. Event 4 Setting the Status=Delivered bit Off, Setting the Status=Completed On
all of this means that the bitmap index for
Status=Created, Queued, Delivered over time is just a series of off bits and in many cases the entire bitchunk is just a bunch of off bits(although there is some compression here).
I have utilized %SYS.Maint.Bitmap on systems that have existed for many years and saw some noticable space savings. However, once run it doesnt seem to make a lot of sense to run very often.
https://github.com/synthetichealth/synthea can be used to create synthetic data, FHIR being one of the formats
Regarding
<Invoke>in module.xml behaves more intuitively by always checking the %Status return value if and only if the method signature declares %Status is returned. This means an error will be thrown if nothing is returned, the return value is not a %Status, or it is not $$$OK.
If we have
specifically
<Invoke Class="RCIPM.Utils" Method="MakeModuleDeployed" Phase="Compile" When="After" CheckStatus="false">
<Arg>{$root}</Arg>
<Arg>{$verbose}</Arg>
</Invoke>
CheckStatus="false", is it still going to check the status. sorry if I'm reading this incorrectly.
I guess then reading further I see
CheckStatus flag for <Invoke> action has been deprecated. Default behavior is now to always check the status of the method if and only if the method signature returns %Library.Statusso it would appear, yes, the CheckStatus attribute is now going to be ignored.
I have 109 examples across 31 modules that use CheckStatus="false"
some of these are during the Phase="Clean" where I dont care if the Invoke succeeds completely or not during a Clean.
With the behavior of modules being all part of a TSTART if any of the Invokes return an error %Status then the module is not installed and everything is TROLLBACK'd. in the case of Invoke I dont have a super string feeling about some of the <Invoke>s, if they fail I still want the module installed.
Ben,
Can you provide some clarity around why some of your SQL statements which are SELECT statements do not use
%NOLOCK
while one of them does use %NOLOCK.
I always thought a by default a SELECT statement does not attempt to lock any of the records of any type of lock.
If I study the ObjectScript code generated to satisfy a SELECT statement like
select id from ens.messageHeader
it's not clear that any of this does any form of a lock(whether exclusive, shared etc)
%0Afirst
; asl MOD# 2
set i%Vsb23=""
%0AmBk1 s i%Vsb23=$o(^Ens.MessageHeaderI("Extent",i%Vsb23),1,i%Vch24) i i%Vsb23="" { g %0AmBdun }
s i%Vhi25=i%Vsb23-1*64000
s i%Vvecvar17=$vop("+",$vop("positions",i%Vch24),i%Vhi25-1)
; asl MOD# 3
set i%Vlo26=""
%0AmCk1 s i%Vlo26=$bitfind(i%Vch24,1,i%Vlo26+1) i 'i%Vlo26 g %0AmCdun
s i%IDO1=i%Vhi25+i%Vlo26-1
s i%IDO1=$ve(i%Vvecvar17,i%Vlo26)
s:$g(SQLCODE)'<0 SQLCODE=0 s %ROWCOUNT=$i(i%rowcnt),%ROWID=i%IDO1,i%CursorState=10
//-- FETCH Output
set:$l($g(i%querystats)) i%time=$g(i%time)+$zh-$lg(i%querystats),i%commands=$g(i%commands)+$zu(61,8,$zu(61))-$lg(i%querystats,2),i%querystats=""
quit
%QRS0f //-- FETCH tag
i '$g(i%CursorState) { s SQLCODE=-102 q } i i%CursorState=100 { s SQLCODE=100 q } s SQLCODE=0
i i%rowlimit,i%rowcnt'<i%rowlimit { s SQLCODE=100,%ROWCOUNT=i%rowcnt,i%CursorState=100 q }
Do $system.Process.AppFrameInfo(i%StackInfo)
set i%querystats=$lb($zh,$zu(61,8,$zu(61)))
g %0Afirst:i%CursorState=1
g %0AmCk1
%0AmCdun g %0AmBk1
%0AmBdun
%0AmAdun
s %ROWCOUNT=i%rowcnt,SQLCODE=100,i%CursorState=100
set:$l($g(i%querystats)) i%time=$g(i%time)+$zh-$lg(i%querystats),i%commands=$g(i%commands)+$zu(61,8,$zu(61))-$lg(i%querystats,2),i%querystats=""
//-- no [more] data
quitWhile the above is for a simple query and is completely index satisfiable, i.e. it doesnt have to read from the mastermap I dont think the question of lock/nolock changes for a SELECT statement.
Question:
Why would the usage of %NOLOCK ever come into play?
you can quickly access the SQL shell witj
USER>:SQL SQL Command Line Shell ---------------------------------------------------- The command prefix is currently set to: <<nothing>>. Enter <command>, 'q' to quit, '?' for help.
You might find some success looking at the tables/sqlprocs found in the %SQL* schemas
.png)
This isnt an answer to your specific question but this looks like a measurement of the business hosts in a production, the number of Txns and the average time. Might this be already satisified by the Activity Monitor Dashboard. This dashboard is supported by the data in the tables Ens_Activity.*
for the example you provided given that the methods invoked by the <routes> are classmethod I commonly do something along the lines of
<Route Url="/error/list" Method="GET" Call="GetErrors" />
ClassMethod GetErrors(pStartTime As %UTC = "", pEndTime As %UTC = "") As %Status
{
#DIM tSC As %Status = $$$OK
#DIM eException As %Exception.AbstractException
#DIM %request As %CSP.Request
Try {
Set:$Get(%response)'="" %response.ContentType="application/json"
If $Get(%request) {
Set tStartDate = $Get(%request.Data("StartTime",1))
Set tStopDate = $Get(%request.Data("EndTime",1))
}
Else {
#;If they are defined in the parameter list
If pStartTime'="" Set tStartDate=pStartTime
If pEndTime'="" Set tStopDate=pEndTime
}
removeEmpty is defined as
The <table> removeEmpty attribute controls whether or not the empty nodes that Zen encounters in the XML data for this report display in the XHTML or PDF output generated by this <table> in the report. If removeEmpty is:
The group attribute must be set for removeEmpty to work.
This attribute has the underlying data type %ZEN.Datatype.booleanOpens in a new tab. See “Zen Reports Attribute Data Types.”
so I dont think removeEmpty addresses what you have asked. I would consider using an
ifxpath expression that would control whether or not to display the table
Interesting.. but if the user understands SQL and understands the db schema why dont they connect via ODBC/JDBC. If they dont understand SQL and the database schema why not make a specific REST class that has endpoints for each type of query they might want to run. Allow for arbitrary SQL statement here including INSERT, UPDATE, DELETE is probably not a good idea.. although it looks like /query may only allow Statement Type = 1 SELECT and Type=45 CALL.
I take the approach of adding my .DFI items to a class that extends %DeepSee.UserLibrary.Container following
as this allows me to treat like any other class.
This means I can
This capability has been around since 2018 +/-.
If we look to what InterSystems provides as an application like Ensemble(Interoperability) or HealthShare we will find the class defintions exist in the ENSLIB or HSLIB namespace. Then via Package mapping you will find that Interoperability enabled namespaces or HS enabled namespaces map the package definitions. For example in the namespace HSEDGE1 we see
.png)
which says the classes from these packages are visible to the HSEDGE1 namespace.
The data for the extent of these classes by default would live in the associated db for the namespace HSEDGE1.
In most cases the data for the extent is not mapped so as to live in a single univeral database but generally speaking it could be, although in the case iof the Ensemble/Interoperability and HealthShare data you shoud not do this.
The error you are seeing is a compile error, its not a run time error so the DC AI Bot isnt telling the correct answer. The field in the RecordMap UI is a bit wonky. If you hover over the label you get a tooltip which indicates you need to separate the parameter values by ;
So something like this works
MAXLEN=80;PATTERN=14N
which gets stored in the RecordMap definition class as
<Field name="NPI" required="0" ignored="0" datatype="%String" index="1" params="MAXLEN=80;PATTERN=14N" sqlColumn="2" repeating="0">
</Field>
and the generated RecordMap class now has
Property NPI As %String(MAXLEN = 80, PATTERN = "14N") [ SqlColumnNumber = 2 ];
so
1. parameters should be separated by ;
2. for pattern you do not need to surround the pattern value by "s
3. If something like ResearchId is actually a pure numeric and number it might be better defined as %Integer
4. For something like FirstName you might want PATTERN=1.A which means 1 to N alpha characters
You may have an issue with the format of the date.
What is the datatype of
Are they %Date, %TimeStamp, %UTC, or PosixTime
I'm not certain but does MAX(somedate) cause the value to no longer be in a format that would support
WHERE dateField >= anotherDateField
When in Analyzer you can see the actual query being run aftet the query runs by clicking on the Show Query button in the toolbar
.png)
which it seems like you have done so as you have
WHERE source.%ID IN (SELECT _DSsourceId FROM MyTable.Listing WHERE _DSqueryKey = 'en3460403753')
you can copy the query SQL Query Listing and paste into (albeit remove the portion that has
WHERE source.%ID IN (SELECT _DSsourceId FROM MyTable.Listing WHERE _DSqueryKey = 'en3460403753')
)
System Explorer->SQL and take special note of the Runtime mode of the query.
.png)
The Runtime mode has the greatest impact on columns that are
%Date
%UTC
as these columns will have different values based on the runtime mode(Logical/Display/ODBC)
Given that the routes in your enabled %CSP.REST class point to class methods you should be able to debug/step thru line by line by calling the class method. In the case where you might have a POST/PUT for your route that calls the class method you can always define the formal specificaiton of the class method to accept parameters and then have the code in your class method look at either your parameter values or %request.Data to determine if the class method is being called by a Http call or your simple debugging.
When building data for a cube, either a full build or a synchronization, think about it this way. The code generated to support this essentially does a SELECT AllOfYourDimensionsMeasuresRelationships FROM SourceTable.
when you have non primary field references these expressions are evauluated as ObjectScript expressions.
These are documented but not on a single page
%sourceID although this is typically thought of being used in a detail listing specifically if you chose the option for whereas the other's are when defining the cube dimension, measures etc
There were several questions asked during the pressentation. One of them was
How can we restrict usage of the cubes so that we only allow users to specific data?
If this were SQL one way of approaching this is to define a View and then grant access to the View but not the table to users. Fortunately Analytics has the same concept. Just as SQL tables are the resource that has the data and a view is defined representation of the data, Analytics has Cubes which has the data and Subject Areas.
A subject is a virtual view of the cube but does not require the additional storage/build/sync time. Within the Subject Area you can
1. Define a filter, consider we want to define a subject on Senior Citizens, we would define a filter on Age > X. Then when we make the Senior Citizen Subject area to a user they will only ever see patients > X,
2. Define what dimensions are available
3. Define listings
Answers to your questions
1. Think of the classes that describe the globals as being a meta data layer to the globals. The existing application will continue to run and you will now have classes that expose the data so that you can write Object and SQL code. If you create new indices in the SQL mapped classes the global representing the new indices would only be updated if something calls the Object.%Save/Delete or SQL Insert/Update/Delete or the legacy filers are updated to manage the new indices. The legacy application might be unlikely to do this so it would mean your new indices will never be populated which would be bad as the SQL engine would not "know" this and would attempt to read data from the new index and there would be no data.
2. You are correct... if you add new indices to the class and the legacy application is not maintaining the index then it would cause issues.
Hopefully you have a common filer for the legacy application, meaning you have one common filer to save a subject area(global). If this is the case then it's a matter of updating the common filer. If the legacy application has a number of places where the data is updated then all of those places would need to be updated or consider adopting an approach of a common filer.
On projects I have been involved in where we have created classes to map the existing globals to enable Object and SQL access we have added to the class
Parameter READONLY=1
so as to ensure no one could accidentially perform an Object Save/Delete or SQL Insert/Update/Delete operation.
You could consider using SemaphoreSpec and allow the folks placing the file in the FTP directory to also place the semaphore file in the directory on their external instruction.
One important note about QuickStream. If you decide to go down this path you must make certain to Clear the QuickStream as part of your pipeline. The Message Purge that is set up as a task will purge the message header and the associated message body but when it looks at the message body all it sees is a property called QuickStreamId and it doesnt know it should also Clear the associated QuickStream object.
A while back I read Clean Architecture: A Craftsman's Guide to Software Structure and Design (Robert C. Martin Series) which I thought had a ton of useful recommendarions....not all were entirely applicable to ObjectScript but still very useful.
https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture…
The Temp file often times uses a process private global
Process-private globals are written to the IRISTEMP database. In contrast to global variables, InterSystems IRIS does not treat a SET or KILL of a local variable or a process-private global as a journaled transaction event; rolling back the transaction has no effect on these operations.
That being said as Peter mentioned if you had an index on code_1_text you could greatly improve performance. In fact I suspect this type of query would be completely index satisfiable, ie the query would only examine the index and would not have to go to the mastermap for even better performance.
Depending on the legth of values for code_1_text if you do chose to add an index you might want to define the property as
Property code_!_text as %String(COLLATION="SQLUPPER(113)",MAXLEN=32000);
by setting the collation and length. If the values exceed the maximum global subscript length and you do not do this you could encounter <subscript> errors.
A subscript has an illegal value or a global reference is too long. For further details, refer to $ZERROR. For more information on maximum length of global references, see “Determining the Maximum Length of a Subscript”.
I ran into issue a long time ago with using the ToQuickXML methods and reported in https://wrc.intersystems.com/wrc/eucustprob2.csp?OBJID=946743.
My specific issues is with large tSDA objects, ie with many many relationships I would run into <store> errors. The generated code was not calling %UnSwizzleAt for the relationship properties.
The REST API is for
The InterSystems IRIS® SQL Search tool integrates with the InterSystems IRIS Natural Language Processor (NLP) to perform context-aware text search operations on unstructured data, with capabilities such as fuzzy search, regular expressions, stemming, and decompounding. Searching is performed using a standard SQL query with a WHERE clause containing special InterSystems SQL Search syntax.
while it wouldnt be difficult to build a REST API to accept an arbitrary SQL statement and return results there are several things to consider.
1. a single SQL SELECT statement might return 10s of thousands of rows making the REST service problematic with respect to timeouts and payload response size
2. you will want to make sure SQL security so that someone isnt allowed to
- perform a delete if they dont have access to do so
- select data from tables the dont have access to.
etc.
It looks like you are using the
/api/iKnow/latest/
endpoint which is for iKnow. Is ths for an iKnow domain?
the language=tsql feature allows you to write SQL statements that are supported by transact SQL.
https://docs.intersystems.com/iris20252/csp/docbook/Doc.View.cls?KEY=GT…
InterSystems TSQL is an implementation of the Transact-SQL procedural language which supports many of the features of both the Sybase and Microsoft implementations. Transact-SQL is used with Sybase Adaptive Server, Microsoft SQL Server (MSSQL), and other platforms.
On the one hand it means if you are familiar with tsql then you can continue to write using it's supported syntax and
When you run TSQL code on the InterSystems IRIS platform, the code is compiled into the corresponding InterSystems SQL and ObjectScript code. ObjectScript is InterSystems object-oriented programming language. The compiled code is run on InterSystems IRIS and available for debugging purposes, if needed. This compilation step is fully automated and not a one-time migration task. The migration effort consists of loading the schema, data and TSQL code from the old environment to InterSystems IRIS. From then on you can continue to use and modify the TSQL application code, simply recompiling it after making any changes.
On the other hand if you use language =tsql you may not be able to use the extensions that InterSystems SQL supports such as implicit join syntax.
InterSystems SQL provides a special –> operator as a shorthand for getting values from a related table without the complexity of specifying explicit JOINs in certain common cases. This arrow syntax can be used instead of explicit join syntax, or in combination with explicit join syntax. Arrow syntax performs a left outer join.
While you can write a method or trigger using language=tsql I do not believe you can write a class query with language = tsql. In this case you are kind of mixing apples and oranages.
A class Query is a defined/callable query statement that can optionally be defined as a SQLProc. When a Query is defined it can be defined as %Query in which case you write the Execute/Fetch/Close and within there I imagine you could use language = tsql but this is the less common approach and is a special case. If Query is based on %SQLQuery you would simply provide an InterSystems SQL statement and not a tsql statement.
A class method can define language =tsql and this means the sql statements in the method are using the tsql dialect. While a classmethod can also be defined as a SQLProc that can either act as a function returning a value or can use ReturnResultSets to
Specifies whether this method returns result sets (so that ODBC and JDBC clients can retrieve them).
The Data keyword means to capture/store the values of City and State in the index but not as a key. if your sql query needs to query for the ZipCode, City, and State with Data=(City,State) the query can be index satisfiable, meaning it will read data from the index and get the City and State from the index vs. having to go to the master map/Table to get the City and State.