Question
Fabio Care · Sep 28

Increase amount of read operations

Hello, 

First the question then the background:

Can you increase the amount of read operations that are being performed simulatiously to the hard drive where the cache.dat file is located.

Alternatively: Can you increase the amount that is being read with every operation? 

 

Background:
The database is setup with 8k blocks of data that are being read. I'm currently struggeling with a bottleneck concerning the read operations from a disk.

I'm assuming this based on the fact that the same routines when run twice in row perform vastly different. With the first being extremely slow up to multiple seconds and the second one being around half a second or less. Aswell as every subsequent one from there on.

As far as I understand, Caché uses memory to store recently accessed data, therefore this would explain why the second run of the same routine outperforms the first one so much since it's run from RAM

Looking at the windows ressource monitor it is visible that the cache.exe process responsible for reading the most data, at most uses around 20 MB/s of a possible 160 MB/s the disk provides. When performing tasks like moving a big file from A to B with the windows explorer, the ressource monitor shows that the peaks of 160 MB/s can be achived, so it's not a matter of other processes taking the rest of the performance.

Maybe there is a setting in the Management Portal to increase the read performance?
Thanks 

Product version: Caché 2018.1
0
0 120
Discussion (3)1
Log in or sign up to continue

There are no settings that would limit it.

There are various reasons, which may cause the low reading speed. It may also depend on the kind of disk HDD or SSD. Multiple processes can read simultaneously. If you manage to split one read to multiple processes, you may achieve better speed. Copy file, not a right way to measure the speed, because with Cache your data can be spread among disk wider than you think, and it will take time to read it. Defragmentation can be the reason of slow speed.

You may ask InterSystems WRC to help you to investigate this. Or may contact me, directly I can help with it as well.

On a given machine any process can run "as fast" as the CPU clock rate (higher = faster = more operations/sec.)

It is true that a single process can do approx. 15-20 MB/sec. (depends on the CPU clock rate & the disk type: SSD, Premium SSD, Nvme)

The best way to overcome this limitation is do to a "heavy I/O" processes in parallel using the queue manager.
On machines with 16/32 cores, you may achieve your "infrastructure limits" (160MB/sec) easily and even more (we managed to go to a 1000MB/sec of Nvme disks)

I wasn't aware of the queue manager, I will look into it. Maybe the IO Operations can be split.