#Sharding

5

Sharding is a type of database partitioning that separates very large databases into smaller, faster, more easily managed parts called data shards. The word shard means a small part of a whole.

Learn more.

In the good old days (tm) determining the size of the data, streams, and indices for a class/table was easy - you just ran %GSIZE and check D, S, and I globals respectively.

However, nowadays sharding, optimized global names, and indices in separate globals produce %GSIZE output looking like this:

30
1 0 93

In this article, we will run an InterSystems IRIS cluster using docker and Merge CPF files - a new feature allowing you to configure servers with ease.

On UNIX® and Linux, you can modify the default iris.cpf using a declarative CPF merge file. A merge file is a partial CPF that sets the desired values for any number of parameters upon instance startup. The CPF merge operation works only once for each instance.

Our cluster architecture is very simple, it would consist of one Node1 (master node) and two Data Nodes (check all available roles). Unfortunately, docker-compose cannot deploy to several servers (although it can deploy to remote hosts), so this is useful for local development of sharding-aware data models,  tests, and such. For a productive InterSystems IRIS Cluster deployment, you should use either ICM or IKO.

40
0 1 90
Question
David Foard · Nov 15, 2019
Question on sharding

Prior to IRIS, using ECP to share databases under heavy I/O load has known latency issues (in our environment), which pretty much restricted using shared database with relatively static or slow-changing data. In IRIS, will sharding mitigate the latency issue and allow any table to be shared?

 

David

00
0 2 129

InterSystems and Intel recently conducted a series of benchmarks combining InterSystems IRIS with 2nd Generation Intel® Xeon® Scalable Processors, also known as “Cascade Lake”, and Intel® Optane™ DC Persistent Memory (DCPMM). The goals of these benchmarks are to demonstrate the performance and scalability capabilities of InterSystems IRIS with Intel’s latest server technologies in various workload settings and server configurations. Along with various benchmark results, three different use-cases of Intel DCPMM with InterSystems IRIS are provided in this report.

50
0 5 440

InterSystems Online Learning has updated courses to include the latest information about InterSystems IRIS. See descriptions below, and try a course today!

Troubleshooting Productions
Get an introduction to troubleshooting productions, with a focus on locating and understanding some of the key Management Portal pages when troubleshooting.

00
0 0 144
Article
Robert Cemper · Sep 22, 2018 3m read
Sharding evaluation #1

IRIS brought us a new  WOW feature - SHARDING !
Definitely a great thing!
But how can I find out if it suits my actual applications?
Is there a practical advantage to go for it with my well cooked transactional application?
Or is it just for new still to be designed applications?

130
1 8 524

With the release of InterSystems IRIS, we're also making available a nifty bit of software that allows you to get the best out of your InterSystems IRIS cluster when working with Apache Spark for data processing, machine learning and other data-heavy fun. Let's take a closer look at how we're making your life as a Data Scientist easier, as you're probably already facing tough big data challenges already, just from the influx of job offers in your inbox!

20
0 2 891

Last week, we announced the InterSystems IRIS Data Platform, our new and comprehensive platform for all your data endeavours, whether transactional, analytics or both. We've included many of the features our customers know and loved from Caché and Ensemble, but in this article we'll shed a little more light on one of the new capabilities of the platform: SQL Sharding, a powerful new feature in our scalability story.

130
1 11 1,167