#Performance

Hi, colleagues!

Consider you need to put some (less  than 1GB) data to an indexed array and do some data manipulations and calculations with it.

Should I prefer global or local for it?

Is there any general rule when local is faster than global and vice-versa?

Hello,
I have created this script that does lot of writes to a single global. DB write performance is much slower than expected (compared to another similar systems).

set rec = "..." //fill it with something
set time = $piece($horolog,",",2)
while(($piece($horolog,",",2)-time) < 30) //30 seconds
    set ^A($System.Util.CreateGUID()) = rec
}

I have notified the following : 

• CPU usage does not reach 100% on a single core (eg: 25% of total CPU usage should be seen on a 4 cores system).

We are experimenting with IIS, as the PWS will be gone in newer versions.

The code which is executed, takes 15ms to run. If we execute it through PWS (REST), there is some overhead and the total execution time is 40ms, which is acceptable. However, if we go through IIS, it takes 150ms or sometimes even more.

Both PWS and IIS are running on the same server as IRIS in this case. No optimisations have been done on IIS.

Any suggestions on where to look/what to optimize on IIS?

Question: can a business process with pool size > 1 invoke external programs in parallel via $ZF(-100)?

We have a production that takes HL7 (ORU_R01) messages in and produces a PDF from each one for onward transmission to a document store.

The PDF creation is implemented in a business process which transforms the HL7 to XML, saves the XML to the filesystem, then calls Apache FOP (the version distributed with Ensemble) via $ZF(-100) to produce the PDF, which is stored on the filesystem.

Please excuse my ignorance. I am trying to identify what areas would be best to review in the System Dashboard (for Cache 2010.2) for performance issues with the database. It seems to be running slower than usual, but I am trying to find out the best way to go about identifying what the issue is.

The following are captures from the System Dashboard.

As always, thanks a lot for your help.

Could anyone please explain why Caché has a %STARTSWITH function while it supports ANSI SQL "LIKE"?

thank you very much.

Hi Community! I need your advice.

The method below is 6 times slower on production server than on developing machine:

ClassMethod runme4()
{
    s cnt=615210
    s st=$zh
    for i=1:1:cnt {
        s p=##class(digi.packet).%OpenId("packet||5237")
        w:p="" "not found",!
        k p
    }
    w $zh-st,!
}

BUT the server is 6 times faster if OpenId replaced with simple read of a large global (s p=^someLargeGlobal). Any ideas what makes OpenId so slow only on the server?

And another question: I wanted to profile the code with  %Monitor.System.

I use the following code to calculate the SHA1 of a file :

set stream = ##class(%Stream.FileBinary).%New()
do stream.LinkToFile(filename)
write $SYSTEM.Encryption.Base64Encode($SYSTEM.Encryption.SHA1HashStream(stream))

This code is called thousands of time and performance is critical. I have tried to code same logic in another language (which is lower level) and it's almost twice as fast. It's unclear why so I started investigating.

Using Process Monitor, it shows that files are read in chunks of 1024 bytes (1K) which is suboptimal. Reading a file of 1MB while require 1024 file system calls.

Running cache 5.0.21 64 bit on Windows server 2016 in virtual environment. Trying to understand why every single process disk read speed (simple sql data walks) caps  around ~20MB/s, however 2 paralell such tasks on different data areas can reach 19MB/s each, four - 17MB/s each, that is 70MB/s total, etc. Also simple copy file to nul on that system reach ~400MB/s.

What can keep single query on idle system from reaching for example 200MB/s? Virtualization? Windows? Cache? Processors are below 1-3%

I work in a small development company that uses Caché as a database. In some support cases I have doubts about whether the client's infrastructure environment is not affecting Caché's response time. Reading a bit about comparing installations in different environments, both in production as testing and homologation environments , I understood that the TPC-E is a benchmarking method accepted in the market.
I'm reading the specifications of the TPC-E and I wonder if anyone has implemented or seen something similar in Caché.

Cache version: Cache for Windows (x86-64) 2017.2.1 (Build 801_3U)

Good Afternoon,
I have a co-worker who is trying to run the below query via ODBC. The issue is that the query appears to be running extremely slow (nearly 2 hours).

While I can see the benefits that $ZSTORAGE could have if used properly, I have not seen it used in the environments I have worked in. I was wondering if there are any developers that promote its usage.

If used properly, I would imagine it could be highly effective in maximizing free memory since some processes will never go over X amount, while others may very well need much more.

I deployed the IRIS container on my Mac M1 Docker Desktop Kubernetes cluster:

I limited the container 1.5Gb memory:

In the "merge.cpf" file I constrained IRIS memory usage aspects:

Now I load-test the container by multiple installing  and uninstalling the %ZPM package:

• install ZPM (zpm-installer.routine and execution):

Currently, we are running 2014.1 on two different servers (OpenVMS, RHEL). The plan is to transition from OpenVMS to RHEL, but our Write Daemon is in a Troubled state on both servers.

On the OpenVMS server, we have a WIJ file that's 26G and can grow to 40G (size of database cache). Since it hasn't grown to 40G, we don't believe the size of the WIJ file to be the issue.

What else should we be looking at regarding the performance of the Write Daemon?

Want to perform SNMP performance monitoring of cache2010 on AIX 5.3. Since the SNMP service that comes with AIX does not support agentX, it cannot extend the support for cache database. Therefore, I plan to deploy net-snmp on AIX first, then enable agentX, and finally configure cache's subagent. Is this workable? Any documents? Thx!

Hello Community,

I recently encountered a issue with Caché and I can't figure out where the problem is coming from.

I noticed that the license limit (200)  was reached whenever I was opening my Studio (so it seems). When this occurs, I restart Caché (with the Cube in the Taskbar), and the number of license used is back to 1%, but grows back after.  The time taken before the number of license  grows back again looks pretty random.

Here is a couple of screenshots :

Does anybody know where the problem might come from ?

I need to build an integration solution that reads messages from a Kafka topic. The topic has 3 partitions and contains several million messages.

For certain reasons, I can only use the standard EnsLib.Kafka.Service class and cannot use either KafkaClient or Python.

To measure performance and collect statistics I created a simple key + timestamp table with no indexes (so it is unlikely to be a bottleneck). Next, I started an instance of EnsLib.Kafka.Service. In the OnProcessInput method, I receive a message, extract the key from it, get the current time, and write the row to the table.

Hi,

I have a large XML zen report file that I want to convert to a CSV, what is the most efficient way to do it.

XML file:

<Details>
<Col1> aa1 </Col1>
<Col2> bb1</Col2>
<Col3> cc1 </Col3>
</Details>
<Details>
<Col1> aa2 </Col1>
<Col2> bb2</Col2>
<Col3> cc3 </Col3>
</Details>

Expected Output:

Col1,Col2,col3
aa1,bb1,bb1
aa2,bb2,bb2

Thank you.

Any recommendations, to this VMware disk type debate, with Cache installations in particular?

Thick Provision Lazy Zeroed

Thick Provision Eager Zeroed

Thin Provision

Currently, we are receiving an alert that states, "Write Daemon still on pass 31". It's been that way for a few hours.

I was wondering if it is possible to identify what the WD has left to work on so that we can see how we can reduce this and possibly identify if there are issues with the way something is written.

Thanks in advance for any insight.

We are seeing more and more customers being lured with latest infrastructure technologies, particularly Composable Infrastructure. Coming with all sorts of data center consolidations and costs savings.

Question is: are there any concerns for HealthShare/TrakCare being run on these platforms or things to look out for? Anyone out there, already on these platforms?

To be more specific this is HPe Synergy with 480 Compute blades booting as bare metal.

Regards;

Anzelem.

We have 1lakh records in table and while using sql select statement , it is taking more than 9mins to 12 mins to get the records. could you please how to optimize this performance issue if we have more records. how to optimize it.

Is there anyway to improve the performance of the display of DeepSee dashboards?

It takes minutes to do the cell calculations of computing the axis members.

Any direction or help would be appreciated.

Thanks,
LG

I have been asked to assist in the planning of the a new server for our database, which we will be changing operating systems from OpenVMS to Linux (RedHat distribution). However, its difficult to find material regarding what would be recommended, which is likely due to the database being proprietary.

In looking at the information provided below and hoping to decrease processing time, would anyone be able to recommend type of configuration we should have for the new Linux server? Please feel free to ask any clarifying questions.

Hello Community,

As per the Build index documentation "If you use BUILD INDEX on a live system, the index is temporarily labeled as not selectable, meaning that queries cannot use the index while it is being built. Note that this will impact the performance of queries that use the index." Is this  hiding/not selectable is only applicable for BUILD INDEX or it supports class level %BuildIndices as well. as far as my analysis both syntax setting this setting  SetMapSelectability. 

Thanks!

Hi,

I'm trying to find the faster way to get the data from a class, and I find it very slow compared to traditional globals. So, I hope some of you can bring some light to me :-)

I have thousands of registers in a class, and to access it quickly I'm going with $o at the index. From there, I get the values using $listget().

Hello everyone!

Some time ago, I changed the configuration in SQL Runtime Statistic to "Turn on Stats code generation to gather stats at the Open and Close of a query". With this change, the CACHE base (cache/mgr/cache/) has grown a lot to reach 198GB.

Yesterday, I returned the configuration of SQL Runtime Statistic to the default which is "Turn off Stats code generation" and the cache base is no longer growing.

My question is?

How to delete this data that was created in the CACHE base (cache/mgr/cache/) and what data I can delete without affecting the system.

Thanks,
Fábio Louly.

Hello,
I want to process more requests per second in Ensemble 2015 (soap service). My problem is in a business process that makes a great transformation. I thought that I can put its group size to 4 (the current value is 1), or put 4 business processes and apply, for example, the round-robin algorithm.
Which alternative is better?

Hi,

Has anyone got any experience of using the Microsoft diskspd utility to test the storage infrastructure in Healthshare/Ensemble environment.

I am interested in getting some figures to highlight any issues with different approaches to provisioning the disks on our new environment.

I am at a loss as to what parameters I should use to give a reasonable synthetic load that will give me any indication of potential issues. Any pointers would be greatly appreciated!

Kind Regards,

Jason

I would like to know if an encrypted caché database can run significantly slower than a normal "unencrypted" database, in a way that is noticeable to the end user (e.g. slower response time for most pages, especially the ones that rely on read/writing to globals).

I searched in Intersystems knowledge base and couldn't find anything related. I'm looking for possible before/after benchmarks.