#Performance

I need to build an integration solution that reads messages from a Kafka topic. The topic has 3 partitions and contains several million messages.

For certain reasons, I can only use the standard EnsLib.Kafka.Service class and cannot use either KafkaClient or Python.

To measure performance and collect statistics I created a simple key + timestamp table with no indexes (so it is unlikely to be a bottleneck). Next, I started an instance of EnsLib.Kafka.Service. In the OnProcessInput method, I receive a message, extract the key from it, get the current time, and write the row to the table.

Hello,
I have created this script that does lot of writes to a single global. DB write performance is much slower than expected (compared to another similar systems).

set"..."//fill it with something

I have notified the following : 

• CPU usage does not reach 100% on a single core (eg: 25% of total CPU usage should be seen on a 4 cores system). Instead, much lower CPU usage is shown (with some drops to 0% from time to time). It looks like process has to wait for I/O completion before proceeding. Removing the set

Hello Community,

The subroutine ^routine is not executed while the queue is being processed in WorkMgr. However, it works when defined as a function. Is it mandatory to define subroutine^routine as a function for it to execute properly?

testwqm.mac
 set##class

Thanks!

I use the following code to calculate the SHA1 of a file :

set##class%Stream.FileBinary

This code is called thousands of time and performance is critical. I have tried to code same logic in another language (which is lower level) and it's almost twice as fast. It's unclear why so I started investigating.

Using Process Monitor, it shows that files are read in chunks of 1024 bytes (1K) which is suboptimal. Reading a file of 1MB while require 1024 file system calls. Usually bigger buffer is used (eg : 4096 or 81920).

The SHA1HashStream() function is implemented this way : 

160

stream.

Question: can a business process with pool size > 1 invoke external programs in parallel via $ZF(-100)?

We have a production that takes HL7 (ORU_R01) messages in and produces a PDF from each one for onward transmission to a document store.

The PDF creation is implemented in a business process which transforms the HL7 to XML, saves the XML to the filesystem, then calls Apache FOP (the version distributed with Ensemble) via $ZF(-100) to produce the PDF, which is stored on the filesystem.

Hello Community,

As per the Build index documentation "If you use BUILD INDEX on a live system, the index is temporarily labeled as not selectable, meaning that queries cannot use the index while it is being built. Note that this will impact the performance of queries that use the index." Is this  hiding/not selectable is only applicable for BUILD INDEX or it supports class level %BuildIndices as well. as far as my analysis both syntax setting this setting  SetMapSelectability. 

Thanks!

We are experimenting with IIS, as the PWS will be gone in newer versions.

The code which is executed, takes 15ms to run. If we execute it through PWS (REST), there is some overhead and the total execution time is 40ms, which is acceptable. However, if we go through IIS, it takes 150ms or sometimes even more.

Both PWS and IIS are running on the same server as IRIS in this case. No optimisations have been done on IIS.

Any suggestions on where to look/what to optimize on IIS?

I deployed the IRIS container on my Mac M1 Docker Desktop Kubernetes cluster:

I limited the container 1.5Gb memory:

In the "merge.cpf" file I constrained IRIS memory usage aspects:

Now I load-test the container by multiple installing  and uninstalling the %ZPM package:

• install ZPM (zpm-installer.routine and execution):

Could anyone please explain why Caché has a %STARTSWITH function while it supports ANSI SQL "LIKE"?

thank you very much.

Hello,

I'm using the InterSystems Iris odbc driver (DriverODBCVer=02.10) to get data from Iris table but it is slow.

Is there any options to improve performance ? Increase the packet size ?

Change to do in odbcinst.ini / odbc.ini to make it quicker ?

Thanks

I use the following code to start a start a job :

Class MyClass Extends (...)
{
    ClassMethod Foo()
    {
       job $CLASSMETHOD("MyClass","MyMethod") //take forever depending hardware
    }

    ClassMethod MyMethod()
    {
       //do database related stuff
    }
}

On local environment, calling Foo() is instantaneous (a few ms). On production/test servers (which have much better hardware than local) calling this function is slow and take between 200 ms to 800 ms.

I would like to know if an encrypted caché database can run significantly slower than a normal "unencrypted" database, in a way that is noticeable to the end user (e.g. slower response time for most pages, especially the ones that rely on read/writing to globals).

I searched in Intersystems knowledge base and couldn't find anything related. I'm looking for possible before/after benchmarks.

Hello everyone!

Some time ago, I changed the configuration in SQL Runtime Statistic to "Turn on Stats code generation to gather stats at the Open and Close of a query". With this change, the CACHE base (cache/mgr/cache/) has grown a lot to reach 198GB.

Yesterday, I returned the configuration of SQL Runtime Statistic to the default which is "Turn off Stats code generation" and the cache base is no longer growing.

My question is?

How to delete this data that was created in the CACHE base (cache/mgr/cache/) and what data I can delete without affecting the system.

Thanks,
Fábio Louly.

We are seeing more and more customers being lured with latest infrastructure technologies, particularly Composable Infrastructure. Coming with all sorts of data center consolidations and costs savings.

Question is: are there any concerns for HealthShare/TrakCare being run on these platforms or things to look out for? Anyone out there, already on these platforms?

To be more specific this is HPe Synergy with 480 Compute blades booting as bare metal.

Regards;

Anzelem.

Want to perform SNMP performance monitoring of cache2010 on AIX 5.3. Since the SNMP service that comes with AIX does not support agentX, it cannot extend the support for cache database. Therefore, I plan to deploy net-snmp on AIX first, then enable agentX, and finally configure cache's subagent. Is this workable? Any documents? Thx!

Running cache 5.0.21 64 bit on Windows server 2016 in virtual environment. Trying to understand why every single process disk read speed (simple sql data walks) caps  around ~20MB/s, however 2 paralell such tasks on different data areas can reach 19MB/s each, four - 17MB/s each, that is 70MB/s total, etc. Also simple copy file to nul on that system reach ~400MB/s.

What can keep single query on idle system from reaching for example 200MB/s? Virtualization? Windows? Cache? Processors are below 1-3%

Hi Community! I need your advice.

The method below is 6 times slower on production server than on developing machine:

ClassMethod runme4()
{
    s cnt=615210
    s st=$zh
    for i=1:1:cnt {
        s p=##class(digi.packet).%OpenId("packet||5237")
        w:p="" "not found",!
        k p
    }
    w $zh-st,!
}

BUT the server is 6 times faster if OpenId replaced with simple read of a large global (s p=^someLargeGlobal). Any ideas what makes OpenId so slow only on the server?

And another question: I wanted to profile the code with  %Monitor.System.

Hi, 

iris version: IRIS for UNIX (Ubuntu Server LTS for x86-64 Containers) 2019.4 (Build 383U) Fri Dec 6 2019 08:49:54 EST

os: Ubuntu 18.04 LTS

memory: 16G

Disk: 256G SSD

Iris config: default.

soft/hard limit: 65535

I build follow simplest flow to do performance test.

1. create A EnsLib.HTTP.GenericService name it IncomingHTTPService listen on 9980

checked create connection per request

Keepalived = 0

Qsize = 1000

2. create A EnsLib.HTTP.GenericOperation name it OutgoingHTTPOperator to connect localhost:8080(it's a nginx server serve a static html page)

3.

Hi,

I have a large XML zen report file that I want to convert to a CSV, what is the most efficient way to do it.

XML file:

<Details>
<Col1> aa1 </Col1>
<Col2> bb1</Col2>
<Col3> cc1 </Col3>
</Details>
<Details>
<Col1> aa2 </Col1>
<Col2> bb2</Col2>
<Col3> cc3 </Col3>
</Details>

Expected Output:

Col1,Col2,col3
aa1,bb1,bb1
aa2,bb2,bb2

Thank you.

Is there anyway to improve the performance of the display of DeepSee dashboards?

It takes minutes to do the cell calculations of computing the axis members.

Any direction or help would be appreciated.

Thanks,
LG

Cache version: Cache for Windows (x86-64) 2017.2.1 (Build 801_3U)

Good Afternoon,
I have a co-worker who is trying to run the below query via ODBC. The issue is that the query appears to be running extremely slow (nearly 2 hours).

Suppose we need to store millions of values temporarily, that means, we don't care about them if we lose them but our application use them to get realtime information. Should I use Cachetemp or whatever other DB without journaling enabled? If answer is Cachetemp, shouldn't be a problem if we decide to scale using App Server + ECP? I'm not sure what would happen with the app logic in such architecture as I guess I couldn't map and share cachetemp...

Any idea/suggestion?

I have a ZEN page with nine tablepanes. Each tablepane queries a table in the same SQLServer db. I have a single SQLGateWay(odbc) to this SQLServer db. I need to get better performance when I query all nine table at the same time. Would my performance improve if I had nine SQLGateWays(nine odbc configurations/connections), one for each query? I would appreciate any and all suggestions for getting the very best performance when using SQLGateWays. Thank you.

Hi,

Has anyone got any experience of using the Microsoft diskspd utility to test the storage infrastructure in Healthshare/Ensemble environment.

I am interested in getting some figures to highlight any issues with different approaches to provisioning the disks on our new environment.

I am at a loss as to what parameters I should use to give a reasonable synthetic load that will give me any indication of potential issues. Any pointers would be greatly appreciated!

Kind Regards,

Jason

Hi,

I'm trying to find the faster way to get the data from a class, and I find it very slow compared to traditional globals. So, I hope some of you can bring some light to me :-)

I have thousands of registers in a class, and to access it quickly I'm going with $o at the index. From there, I get the values using $listget().

Currently, we are receiving an alert that states, "Write Daemon still on pass 31". It's been that way for a few hours.

I was wondering if it is possible to identify what the WD has left to work on so that we can see how we can reduce this and possibly identify if there are issues with the way something is written.

Thanks in advance for any insight.

Currently, we are running 2014.1 on two different servers (OpenVMS, RHEL). The plan is to transition from OpenVMS to RHEL, but our Write Daemon is in a Troubled state on both servers.

On the OpenVMS server, we have a WIJ file that's 26G and can grow to 40G (size of database cache). Since it hasn't grown to 40G, we don't believe the size of the WIJ file to be the issue.

What else should we be looking at regarding the performance of the Write Daemon?

Hi Guys,

Can you please advise on the below queries. 

Query 1:

Example 1:

 S a="345",b="arun",c="kumar",d="hi",e="yello",f="orange"

Example 2:

S a="345"

S b="arun"

S c="kumar"

S d="hi"

S e="yello"

S f="orange"

Can you please advise me, which one is performance wise is better. 

Query 2:

Example 1:

Example 2:

Please advise me, which one is giving better performance in this. 

Any lead would be appreciated. 

Thanks,

Arun Kumar Durairaj. 

Intersytems  documentation says not to hold entire SDA  as object in In-memory. 

 http://localhost:57772/csp/documatic/%25CSP.Documatic.cls?LIBRARY=registry&CLASSNAME=HS.SDA3.Container&MEMBER=&CSPCHD=001000000000x1zyO3kkajy0iIjjpi2dxUJUnwFSY_pz7jtbNV&CSPSHARE=1

 // Serialize Object to XML
  
  Set object = ##class(User.NewClass1).%New()
  s object.Property1="one"
  s object.Property2="two"
  s object.Property3="three"
  s object.Property4="four"
  Set w=##class(%XML.Writer).%New()
  set w.Charset="UTF-8"
  Set w.Indent=1
  set status=w.OutputToString()
  set status=w.StartDocument()
  Set status=w.