Performance

Syndicate content 11 

I have been asked to assist in the planning of the a new server for our database, which we will be changing operating systems from OpenVMS to Linux (RedHat distribution). However, its difficult to find material regarding what would be recommended, which is likely due to the database being proprietary.

In looking at the information provided below and hoping to decrease processing time, would anyone be able to recommend type of configuration we should have for the new Linux server? Please feel free to ask any clarifying questions.

The current server is running with the following hardware configuration

Last answer 22 March 2017
0   0 4
0

comments

480

views

0

rating

In previous posts I have shown how it is possible to collect historical performance metrics using pButtons. I go to pButtons first because I know it is installed with every Data Platforms instance (Ensemble, Caché, …). However there are other ways to collect, process and display Caché performance metrics in real time either for simple monitoring or more importantly for much more sophisticated operational analytics and capacity planning. One of the most common methods of data collection is to use SNMP (Simple Network Management Protocol). SNMP a standard way for Caché to provide management and monitoring information to a wide variety of management tools. The Caché online documentation includes details of the interface between Caché and SNMP. While SNMP should 'just work' with Caché there are some configuration tricks and traps.

Last comment 17 March 2017
+ 9   1 2
2113

views

+ 9

rating

While I can see the benefits that $ZSTORAGE could have if used properly, I have not seen it used in the environments I have worked in. I was wondering if there are any developers that promote its usage.

If used properly, I would imagine it could be highly effective in maximizing free memory since some processes will never go over X amount, while others may very well need much more.

Last answer 23 February 2017
0   0 2
0

comments

172

views

0

rating

In last week's discussion we created a simple graph based on the data input from one file. Now, as we all know, sometimes we have multiple different datafiles to parse and correlate. So this week we are going to load additional perfmon data and learn how to plot that into the same graph.
Since we might want to use our generated graphs in reports or on a webpage, we'll also look into ways to export the generated graphs

+ 5   0 3
0

comments

494

views

+ 5

rating

If I were trying to access an index of a global variable, what time complexity would this operation have? My understanding of languages like Java/C++ is that arrays are stored as blocks of memory so that x[15] would have a lookup time complexity of O(1) because it just goes to (address of the array + 15) and retrieves the value stored there.

How does this work in Cache where the index of a variable isn't necessarily an integer value? If I were to have a variable like the following:

x("Adam") = "Red"

x("George") = "Blue"

x("Bryan") = "Green"

etc...

Would the lookup operation scale with the size of the array such that the variable is iterated through until the given index meets the current index, yielding a O(n) lookup? Or is this done a different way?

Last answer 6 February 2017 Last comment 7 February 2017
0   0 2
236

views

0

rating

Can someone direct me to where in the documentation we can find how consumption may be calculated for global storage?

Caché Version2010.1
Operating SystemHP OpenVMS 8.4

 

EDIT: After receiving some responses, it seems I was unclear in my initial inquiry. I am looking to determine our rate of consumption of storage; however, I am having some difficulty in doing that.

While utilizing ^%GSIZE, which is used by the %GlobalEdit class, the results appeared odd. I have provided my results below, which illustrate the global structure on the left and the usage indicated by ^%GSIZE on the right

Last answer 23 January 2017 Last comment 23 January 2017
0   0 1
272

views

0

rating

I’ve a server that is running Windows Server 2003 R2 Enterprise Edition SP2 x86.

I just noted that you cannot allocate shared memory beyond 1.6GB.

 

Is this a known problem between Cache and this OS architecture, and has anyone configured it beyond this?

 

Cache gives the below errors (Version: Cache for Windows (x86-32) 2012.2.5 (Build 962_1) Wed Jun 11 2014 13:58:32 EDT).

 

Last answer 18 January 2017
0   0 2
0

comments

602

views

0

rating

Currently, namespace Alpha is configured to use database AlphaDB as its global database. How would we go about having namespace Alpha configured to use database AlphaDB for its global database except where global ^Customers(CustomerId) has a CustomerId greater than 10M, which we would like to have it redirected to database BetaDB.

In other words, ^|"AlphaDB"|Customers contains all customers between 1 and 10,000,000; and ^|"BetaDB"|Customers contains all customers greater than 10,000,000. Any help would be appreciated.

Last answer 28 December 2016 Last comment 28 December 2016
0   0 0
213

views

0

rating

I was benchmarking the populate utils when I noticed something strange. Consider this simple benchmarking method:

ClassMethod runBench(count As %Integer = 300000)
{
        s types=["name","ssn","company","title","phone","city","street","zip","mission","state","color","product","string","integer","float"]
        //s types=["name","ssn"]
        s iter=types.%GetIterator()
        while iter.%GetNext(.key,.value){
            s start=$P($ZTS,",",2)
            s opt={}
            s opt.type=value
            for i=1:1:count{
                s res=##class(DataGen.Generator).genVal(opt)            }
            s end=$P($ZTS,",",2)
            s total=end-start
            w value_$C(9)_" total time:"_total_$C(9)_" rate:  "_$NORMALIZE(count/total,2)_" per second",!
        }
}

With my genVal method being this

+ 4   0 2
0

comments

256

views

+ 4

rating

We don't often use SQL within our org, which is mostly due to the performance issue we experience due to the quantity of data we are reviewing.

Aside from the standard performance measures for non-Caché databases, are there any recommended approaches when querying large tables?

The table would have roughly 50M records, but there are not a finite amount of sub-nodes.

Last answer 2 December 2016 Last comment 6 December 2016
0   0 4
425

views

0

rating

Myself and the other Technology Architects often have to explain to customers and vendors Caché IO requirements and the way that Caché applications will use storage systems. The following tables are useful when explaining typical Caché IO profile and requirements for a transactional database application with customers and vendors.  The original tables were created by Mark Bolinsky.

In future posts I will be discussing more about storage IO so am also posting these tables now as a reference for those articles. 



A list of other posts in this series is here
 


It is vital to have well set up storage such as a storage array to provide predictable disk IO performance, support high availability features and provide storage redundancy, scalability, and reliability for your applications

Last comment 13 November 2016
+ 8   1 2
1290

views

+ 8

rating

A short post for now to answer a question that came up. In post two of this series I included graphs of performance data extracted from pButtons. I was asked off-line if there is a quicker way than cut/paste to extract metrics for mgstat etc from a pButtons .html file for easy charting in Excel.

See: - Part 2 - Looking at the metrics we collected

pButtons compiles data it collects into a single html file to make it easier to send to WRC and review the collated data. However, especially for pButtons run over long collection times like 24 hours, some of the time based data like mgstat, vmstat etc is easier to review graphically to look for trends or patterns

Last comment 4 November 2016
+ 7   0 5
837

views

+ 7

rating

This is a follow-up to Murray's article for scripting pButtons data extraction on Unix/MacOS:
    Extracting pButtons data to CSV for UNIX/MacOS

Extracting pButtons data to a CSV file on Windows

PowerShell scripting was used in this article because it is available by default on Windows platforms.

The script, Extract_pButtons.ps1, will search the given pButtons file until it finds the marker for the beginning of the section. The section will be printed line-by-line, from the header to the line before the section end marker

+ 4   0 4
0

comments

401

views

+ 4

rating

It has been noticed that some customers running JAVA programs (for example, FOP) on AIX would see the server eventually running low then out of memory. Customer would notice the system pages heavily and user experience becomes bad. And the server would crash when out of memory.

 

When the problem happens, we can see in ipcs a lot of shared memory segment marked for deletion (Capital D at the beginning of MODE section). This means they will not disappear until the last process attached to the segment detaches it.

 

The problem can be easily recreated using the following command from UNIX prompt on a server running AIX 7, for example:

 

/usr/java6_64/jre/bin/java -Xrs -classpath /scratch3/gcooper/ggc170refp750/dev/java/lib/JDK16/cachegateway.jar:/scratch3/gcooper/ggc170refp750/dev/java/lib/JDK16/cachejdbc.jar:/usr/java6_64/jre/bin com.intersys.gateway.JavaGateway 62001 2>&1

+ 5   1 1
0

comments

874

views

+ 5

rating

Your application is deployed and everything is running fine. Great, hi-five! Then out of the blue the phone starts to ring off the hook – it’s users complaining that the application is sometimes ‘slow’. But what does that mean? Sometimes? What tools do you have and what statistics should you be looking at to find and resolve this slowness? Is your system infrastructure up to the task of the user load? What infrastructure design questions should you have asked before you went into production? How can you capacity plan for new hardware with confidence and without over-spec'ing? How can you stop the phone ringing? How could you have stopped it ringing in the first place?

Last comment 30 September 2016
+ 17   1 18
2149

views

+ 17

rating

Previously I showed you how to run pButtons to start collecting performance metrics that we are looking at in this series of posts.

pButtons is compatible with Caché version 5 and later and is included with recent distributions of InterSystems data platforms (HealthShare, Ensemble and Caché). This post is a reminder that you should download and install the latest version of pButttons.

The latest version is always available for download via ftp.

ftp.intersystems.com/pub/performance

You need to download the file

+ 2   0 1
0

comments

636

views

+ 2

rating

Often times support and sales engineers are asked about recent benchmark results on various platforms and large scale configurations.  These will be made available here in the Developer Community in the "Documentation" section, and as an example here's a link to a recent Intel E7 v2 series processor benchmark.

https://community.intersystems.com/documentation/data-scalability-intersystems-caché-and-intel-processors-0

There are several reports available and more will be made available on an on-going basis.

0   0 2
0

comments

115

views

0

rating