New post

#Performance

6

Performance tag groups posts regarding software performance issues and the best practices on solving and monitoring performance issues.

Here at InterSystems, we often deal with massive datasets of structured data. It’s not uncommon to see customers with tables spanning >100 fields and >1 billion rows, each table totaling hundred of GB of data. Now imagine joining two or three of these tables together, with a schema that wasn’t optimized for this specific use case. Just for fun, let’s say you have 10 years worth of EMR data from 20 different hospitals across your state, and you’ve been tasked with finding….

6 1
2 32
Article
· Jan 11, 2019 4m read
SQL Performance Resources

There are three things most important to any SQL performance conversation: Indices, TuneTable, and Show Plan. The attached PDFs includes historical presentations on these topics that cover the basics of these 3 things in one place. Our documentation provides more detail on these and other SQL Performance topics in the links below. The eLearning options reinforces several of these topics. In addition, there are several Developer Community articles which touch on SQL performance, and those relevant links are also listed.

There is a fair amount of repetition in the information listed below. The most important aspects of SQL performance to consider are:

  1. The types of indices available
  2. Using one index type over another
  3. The information TuneTable gathers for a table and what it means to the Optimizer
  4. How to read a Show Plan to better understand if a query is good or bad
13 3
8 1.1K

The rise of Big Data projects, real-time self-service analytics, online query services, and social networks, among others, have enabled scenarios for massive and high-performance data queries. In response to this challenge, MPP (massively parallel processing database) technology was created, and it quickly established itself. Among the open-source MPP options, Presto (https://prestodb.io/) is the best-known option. It originated in Facebook and was utilized for data analytics, but later became open-sourced.

Open Exchange app
5 0
3 173

We've recently made available a new version of InterSystems IRIS in the Vector Search Early Access Program, featuring a new Approximate Nearest Neighbor index based upon the Hierarchical Navigable Small World (HNSW) indexing algorithm.

7 1
1 163

sql-embedding cover

InterSystems IRIS 2024 recently introduced the vector types.
This addition empowers developers to work with vector search, enabling efficient similarity searches, clustering, and a range of other applications.
In this article, we will delve into the intricacies of vector types, explore their applications, and provide practical examples to guide your implementation.

Open Exchange app
5 0
1 81

A customer recently asked if IRIS supported OpenTelemetry as they where seeking to measure the time that IRIS implemented SOAP Services take to complete. The customer already has several other technologies that support OpenTelemetry for process tracing. At this time, InterSystems IRIS (IRIS) do not natively support OpenTelemetry.

Open Exchange app
12 5
1 568

Is there a difference in outcome between the two screengrabs below?

In both cases, when certain conditions are met, a transformation is called and the output sent on to two targets. In the first case we surmise the transformation is called twice, and the output of the first run sent to the first target, the output of the second run to the second target. In the second case we surmise the transformation is called once, and the output duplicated and sent to the two targets.

1 2
0 216

So if you are following from the previous post or dropping in now, let's segway to the world of eBPF applications and take a look at Parca, which builds on our brief investigation of performance bottlenecks using eBPF, but puts a killer app on top of your cluster to monitor all your iris workloads, continually, cluster wide!

Continous Profiling with Parca, IRIS Workloads Cluster Wide

1 0
0 121

I attended Cloud Native Security Con in Seattle with full intention of crushing OTEL day, then perusing the subject of security applied to Cloud Native workloads the following days leading up to CTF as a professional excercise. This was happily upended by a new understanding of eBPF, which got my screens, career, workloads, and atitude a much needed upgrade with new approaches to solving workload problems.

So I made it to the eBPF party and have been attending clinic after clinic on the subject ever since, here I would like to "unbox" eBPF as a technical solution, mapped directly to what we do in practice (even if its a bit off), and step through eBPF through my experimentation on supporting InterSystems IRIS Workloads, particularly on Kubernetes, but not necessarily void on standalone workloads.

eBee Steps with eBPF and InterSystems IRIS Workloads

3 0
1 141

It's been a long time since I didn't write an update post on IoP.

image

So what's new since IoP command line interface was released?

Two new big features were added to IoP:
- Rebranding: the grongier.pex module was renamed to iop to reflect the new name of the project.
- Async support: IoP now supports async functions and coroutines.

Open Exchange app
3 5
0 180

When there's a performance issue, whether for all users on the system or a single process, the shortest path to understanding the root cause is usually to understand what the processes in question are spending their time doing. Are they mostly using CPU to dutifully march through their algorithm (for better or worse); or are they mostly reading database blocks from disk; or mostly waiting for something else, like LOCKs, ECP or database block collisions?

15 1
4 427

There are often questions surrounding the ideal Apache HTTPD Web Server configuration for HealthShare. The contents of this article will outline the initial recommended web server configuration for any HealthShare product.

As a starting point, Apache HTTPD version 2.4.x (64-bit) is recommended. Earlier versions such as 2.2.x are available, however version 2.2 is not recommended for performance and scalability of HealthShare.

19 1
15 11.3K

Hello everyone,

Recently, I've been working on a Business Process that processes a large JSON FHIR message containing up to 50k requests in an array within the JSON.

Currently, the code imports the JSON as a dynamic object from the original message stream, obtains an iterator from it, and processes each request one at a time in a loop.

0 2
0 542

We are experimenting with IIS, as the PWS will be gone in newer versions.

The code which is executed, takes 15ms to run. If we execute it through PWS (REST), there is some overhead and the total execution time is 40ms, which is acceptable. However, if we go through IIS, it takes 150ms or sometimes even more.

Both PWS and IIS are running on the same server as IRIS in this case. No optimisations have been done on IIS.

Any suggestions on where to look/what to optimize on IIS?

2 4
2 187

[Background]

InterSystems IRIS family has a nice utility ^SystemPerformance (as known as ^pButtons in Caché and Ensemble) which outputs the database performance information into a readable HTML file. When you run ^SystemPerformance on IRIS for Windows, a HTML file is created where both our own performance log mgstat and Windows performance log are included.

Open Exchange app
12 2
4 666

You may have heard about our mg-dbx-napi interface for IRIS which provides insanely fast access from Node.js. If you've been following recent developments in the server-side JavaScript world, you'll be excited to know that mg-dbx-napi also works with Bun.js, the latter proving to be significantly faster than Node.js for many/most purposes.

Of course, if you're a Node.js user, you'll probably wonder how mg-dbx-napi compares with the Native API for Node.js that is included with IRIS.

5 8
2 220

It seems like yesterday when we did a small project in Java to test the performance of IRIS, PostgreSQL and MySQL (you can review the article we wrote back in June at the end of this article). If you remember, IRIS was superior to PostgreSQL and clearly superior to MySQL in insertions, with no big difference in queries.

Open Exchange app
7 6
3 787

Windows Subsystem for Linux (WSL) is a feature of Windows that allows you to run a Linux environment on your Windows machine, without the need for a separate virtual machine or dual booting.

WSL is designed to provide a seamless and productive experience for developers who want to use both Windows and Linux at the same time**.

2 0
1 345

Most transactional applications have a 70:30 RW profile. However, some special cases have extremely high write IO profiles.

I ran storage IO tests in the ap-southeast-2 (Sydney) AWS region to simulate IRIS database IO patterns and throughput similar to a very high write rate application.

The test aimed to determine whether the EC2 instance types and EBS volume types available in the AWS Australian regions will support the high IO rates and throughput required.

5 0
0 1.3K

Hi Community,

Watch this video to learn how InterSystems software enables both horizontal and vertical scaling:

⏯ Scaling for User and Data Volume in InterSystems IRIS

https://www.youtube.com/embed/A477wcJL2LA
[This is an embedded link, but you cannot view embedded content directly on the site because you have declined the cookies necessary to access it. To view embedded content, you would need to accept all cookies in your Cookies Settings]

1 0
0 176

InterSystems IRIS offers various ways how to profile your code, in most cases it produces enough information to find the places where the most time is spent or where the most global sets. But sometimes it's difficult to understand the execution flow and how it ended at that point.

To solve this, I've decided to implement a way to build a report in a way, so, it's possible to dive by stack down

Open Exchange app
3 4
2 322

Hello,

I would like to get a list of all globals that have been read or written during a given context. In Portal, there are counters in dashboard that give the number of read/write to globals in general.

What I am looking for :

- some handler (eg: like $ZTRAP) that will be called everytime something is read/written to a global.

- to activate a "global log mode" in Portal that will dump some information to a file (like ^ISCSOAP for SOAP requests).

1 2
0 276