Performance

Syndicate content 11 

In the previous parts (1, 2) we talked about globals as trees. In this article, we will look at them as sparse arrays.

A sparse array - is a type of array where most values assume an identical value.

In practice, you will often see sparse arrays so huge that there is no point in occupying memory with identical elements. Therefore, it makes sense to organize sparse arrays in such a way that memory is not wasted on storing duplicate values.

In some programming languages, sparse arrays are part of the language - for example, in J, MATLAB. In other languages, there are special libraries that let you use them. For C++, those would be Eigen and the like.

Globals are good candidates for implementing sparse arrays for the following reasons:

Last comment 3 days ago
+ 6   0 5
587

views

+ 6

rating

Hi Community!

Please welcome a new video on Developer Community YouTube Channel:

InterSystems IRIS from Spark to Finish

 

0   0 1
0

comments

26

views

0

rating

Hi Community!

Please welcome a new video on Developer Community YouTube Channel:

Implementing vSAN for InterSystems IRIS

 

0   0 1
0

comments

34

views

0

rating

IRIS and Ensemble are designed to act as an ESB/EAI. This mean they are build to process lots of small messages.

But some times, in real life we have to use them as ETL. The down side is not that they can't do so, but it can take a long time to process millions of row at once.

To improve performance, I have created a new SQLOutboundAdaptor who only works with JDBC.

BatchSqlOutboundAdapter

Extend EnsLib.SQL.OutboundAdapter to add batch batch and fetch support on JDBC connection.

Benchmark

Benchmarks released on Postgres 11.2 with 1 000 000 rows fetched and 100 000 rows inserted on 2 columns.

alt text

Prerequisites

Can be used on IRIS or Ensemble 2017.2+.

Installing

Clone this repositor

0   0 1
0

comments

76

views

0

rating

ˮ This is one of my articles which was never published in English. Let's fix it!

Hello! This article is about quite a practical way of developing InterSystems solutions without using the integrated tools like Studio or Atelier. All the code of the project can be stored in the form of "traditional" source code files, edited in your favorite development environment (for example, Visual Studio Code), indexed by any version control system and arbitrarily combined with many external tools for code analysis, preprocessing, packaging and so on

Last comment 2 March 2019
+ 6   5 3
821

views

+ 6

rating

Suppose we need to store millions of values temporarily, that means, we don't care about them if we lose them but our application use them to get realtime information. Should I use Cachetemp or whatever other DB without journaling enabled? If answer is Cachetemp, shouldn't be a problem if we decide to scale using App Server + ECP? I'm not sure what would happen with the app logic in such architecture as I guess I couldn't map and share cachetemp...

Any idea/suggestion?

Last answer 13 February 2019 Last comment 13 February 2019
0   0 3
111

views

0

rating

There are often questions surrounding the ideal Apache HTTPD Web Server configuration for HealthShare.  The contents of this article will outline the initial recommended web server configuration for any HealthShare product. 

As a starting point, Apache HTTPD version 2.4.x (64-bit) is recommended.  Earlier versions such as 2.2.x are available, however version 2.2 is not recommended for performance and scalability of HealthShare.

+ 13   5 3
0

comments

508

views

+ 13

rating

There are three things most important to any SQL performance conversation:  Indices, TuneTable, and Show Plan.  The attached PDFs includes historical presentations on these topics that cover the basics of these 3 things in one place.  Our documentation provides more detail on these and other SQL Performance topics in the links below.  The eLearning options reinforces several of these topics.  In addition, there are several Developer Community articles which touch on SQL performance, and those relevant links are also listed.

There is a fair amount of repetition in the information listed below.  The most important aspects of SQL performance to consider are:

  1. The types of indices available
  2. Using one index type over another
  3. The information TuneTable gathers for a table and what it means to the Optimizer
  4. How to read a Show Plan to better understand if a query is good or bad
Last comment 18 January 2019
+ 7   3 2
337

views

+ 7

rating

Hi Community!

New session recording from Global Summit 2018 is available on Developer Community YouTube Channel:

An Outlook on Scaling Out

 

0   0 1
0

comments

45

views

0

rating

Hi Community!

Please welcome a new video on Developer Community YouTube Channel:

Alexa: Connect Me with the World of IoT

 

+ 1   1 1
0

comments

145

views

+ 1

rating

Not everyone knows that InterSystems Caché has a built-in tool for code profiling called Caché Monitor.

Its main purpose (obviously) is the collection of statistics for programs running in Caché. It can provide statistics by program, as well as detailed Line-by-Line statistics for each program.

Using Caché Monitor

Let’s take a look at a potential use case for Caché Monitor and its key features. So, in order to start the profiler, you need to go to the terminal and switch to the namespace that you want to monitor, then launch the %SYS.MONLBL system routine:

zn "<namespace>"
do ^%SYS.MONLBL

As the result, you will see the following:

Last comment 14 December 2018
+ 3   7 2
376

views

+ 3

rating

Continuing on with providing some examples of various storage technologies and their performance profiles, this time we looked at the growing trend of leveraging internal commodity-based server storage, specifically the new HPE Cloudline 3150 Gen10 AMD processor-based single socket servers with two 3.2TB Samsung  PM1725a NVMe drives.  

Last comment 29 October 2018
+ 4   0 2
364

views

+ 4

rating

I have a ZEN page with nine tablepanes. Each tablepane queries a table in the same SQLServer db. I have a single SQLGateWay(odbc) to this SQLServer db. I need to get better performance when I query all nine table at the same time. Would my performance improve if I had nine SQLGateWays(nine odbc configurations/connections), one for each query? I would appreciate any and all suggestions for getting the very best performance when using SQLGateWays. Thank you.

Last answer 18 October 2018
0   0 1
0

comments

70

views

0

rating

Hi,

Has anyone got any experience of using the Microsoft diskspd utility to test the storage infrastructure in Healthshare/Ensemble environment.

I am interested in getting some figures to highlight any issues with different approaches to provisioning the disks on our new environment.

I am at a loss as to what parameters I should use to give a reasonable synthetic load that will give me any indication of potential issues. Any pointers would be greatly appreciated!

Kind Regards,

Jason

Last answer 15 August 2018 Last comment 15 August 2018
0   0 2
126

views

0

rating

Hi,

I'm trying to find the faster way to get the data from a class, and I find it very slow compared to traditional globals. So, I hope some of you can bring some light to me :-)

I have thousands of registers in a class, and to access it quickly I'm going with $o at the index. From there, I get the values using $listget(). Something like that:


    s FromDateH = (+$h-1)
    for {
        set id=$order(^TestI("StartDateIDX",FromDateH,id))
        quit:id=""
        set dat=$lg(^TestD(id))    //dat=$lb("a","b","c","d","e")
    }
    


I find it quite slow, so I've tried the same code but changing the conents of ^TEST from a List to a String

Last answer 7 August 2018 Last comment 9 August 2018
0   0 6
371

views

0

rating

Often InterSystems technology architect team is asked about recommended storage arrays or storage technologies.  To provide this information to a wider audience as reference, a new series is started to provide some of the results we have encountered with various storage technologies.  As a general recommendation, all-flash storage is highly recommended with all InterSystems products to provide the lowest latency and predictable IOPS capabilities.

The first in the series was the most recently tested Netapp AFF A300 storage array.  This is middle-tier type storage array with several higher models above it.  This specific A300 model is capable of supporting a minimal configuration of only a few drives to hundreds of drives per HA pair, and also capable of being clustered with multiple controller pairs for tens of PB's of disk capacity and hundreds of thousands of IOPS or higher. 

+ 3   0 1
0

comments

1153

views

+ 3

rating

This document describes considerations around SDDC and HCI solutions for InterSystems Clients.

Software Defined Data Centers (SDDC) and Hyper-Converged Infrastructure (HCI) – Important Considerations for InterSystems Clients

A growing number of IT organizations are exploring the potential use of SDDC and HCI solutions. These solutions appear attractive and are marketed as simplification of IT management and potential cost reductions across heterogeneous data centers and cloud infrastructure options. The potential benefits to IT organizations are significant, and many InterSystems clients are embracing SDDC, HCI, or both

+ 2   0 1
0

comments

547

views

+ 2

rating

Currently, we are receiving an alert that states, "Write Daemon still on pass 31". It's been that way for a few hours.

I was wondering if it is possible to identify what the WD has left to work on so that we can see how we can reduce this and possibly identify if there are issues with the way something is written.

Thanks in advance for any insight.

Last answer 2 May 2018 Last comment 3 May 2018
0   0 4
158

views

0

rating

In the last post we scheduled 24-hour collections of performance metrics using pButtons. In this post we are going to be looking at a few of the key metrics that are being collected and how they relate to the underlying system hardware. We will also start to explore the relationship between Caché (or any of the InterSystems Data Platforms) metrics and system metrics. And how you can use these metrics to understand the daily beat rate of your systems and diagnose performance problems.

Last comment 24 March 2018
+ 16   1 16
2070

views

+ 16

rating

Currently, we are running 2014.1 on two different servers (OpenVMS, RHEL). The plan is to transition from OpenVMS to RHEL, but our Write Daemon is in a Troubled state on both servers.

On the OpenVMS server, we have a WIJ file that's 26G and can grow to 40G (size of database cache). Since it hasn't grown to 40G, we don't believe the size of the WIJ file to be the issue.

What else should we be looking at regarding the performance of the Write Daemon?

Last answer 20 March 2018
0   0 2
0

comments

223

views

0

rating

Hi, Community! 

Check a new session recording from Global Summit 2017:

Pros and Cons of Virtual Environments

 

0   0 1
0

comments

79

views

0

rating

Hi Guys,

Can you please advise on the below queries. 

Query 1:

Example 1:

 S a="345",b="arun",c="kumar",d="hi",e="yello",f="orange"

Example 2:

S a="345"

S b="arun"

S c="kumar"

S d="hi"

S e="yello"

S f="orange"

Can you please advise me, which one is performance wise is better. 

Query 2:

Example 1:

S:a=1 R="Arun"

Example 2:

I a=2 R="Arun"

Please advise me, which one is giving better performance in this. 

Any lead would be appreciated. 

Thanks,

Arun Kumar Durairaj. 

Last answer 1 March 2018 Last comment 3 March 2018
0   0 6
628

views

0

rating