Data Quality Manager rebuild Cubes
Hi guys,
Does rebuilding of Data Quality Manager cubes affect patient index linkage definition data?
What I want to understand is, does DQM have any negative impact on the data each time you rebuild the cubes.
Additionally, if I make changes to the cube and rebuild it..will this have any impact on the original data?
Product version: IRIS 2020.2
$ZV: HealthShare 2020.2 [HealthShare Modules: Active Analytics:20.0.8620 + Core:20.0.8620 + Patient Index:20.0.8620] - IRIS for UNIX (Red Hat Enterprise Linux for x86-64) 2020.1 (Build 217_1_20418U) Tue Nov 17 2020 15:48:44 EST
Discussion (6)1
Comments
Absolutely not.
During SYNC or REBUILD events Data Quality Manager cubes would only read patient index linkage definition data.
There's no way for cubes to modify patient index linkage definition data.
That said, reads, cube data writes, and additional cpu load during SYNC or REBUILD events might impact the running system, since there's a limited amount of resources available. You might want to schedule SYNCs and especially REBUILDs for low-load times (at nights or weekends depending on your usage patterns).
Using async reporting mirror for cubes would completely remove the impact on the patient index.
Thanks Eduard,
From your response, even if I extend a cube with additional property and rebuild it won't affect the linkage definition data right?
Yes. Cube builds don't modify source data.
Cubes copy all source data into an entirely separate set of tables.
Hi Eduard,
I have enabled Data Quality Manager in our production environment and I am picking up a massive increase in memory utilization and CPU utilization. Database size keeps increasing,is it normal?
Logs
07/26/22-07:49:41:266 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 10 MB requested. 07/26/22-07:49:41:290 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 10 MB. 07/26/22-07:49:42:428 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 10 MB requested. 07/26/22-07:49:42:453 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 10 MB. 07/26/22-07:49:45:053 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 10 MB requested. 07/26/22-07:49:45:083 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 10 MB. 07/26/22-07:49:47:293 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 10 MB requested. 07/26/22-07:49:47:317 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 10 MB. 07/26/22-07:49:48:411 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 10 MB requested. 07/26/22-07:49:48:437 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 10 MB. 07/26/22-07:49:50:336 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 10 MB requested. 07/26/22-07:49:50:363 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 10 MB. 07/26/22-07:49:52:277 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 10 MB requested. 07/26/22-07:49:52:304 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 10 MB. 07/26/22-07:49:54:158 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 10 MB requested. 07/26/22-07:49:54:187 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 10 MB. 07/26/22-07:49:56:072 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 10 MB requested. 07/26/22-07:49:56:100 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 10 MB. 07/26/22-07:49:57:910 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 11 MB requested. 07/26/22-07:49:57:940 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 11 MB. 07/26/22-07:50:00:318 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 12 MB requested. 07/26/22-07:50:00:351 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 12 MB. 07/26/22-07:50:03:179 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 14 MB requested. 07/26/22-07:50:03:217 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 14 MB. 07/26/22-07:50:06:363 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 16 MB requested. 07/26/22-07:50:06:402 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 16 MB. 07/26/22-07:50:42:564 (2206) 0 [Database.SparseDBExpansion] Expanding capacity of sparse database /hs-patient/sys/healthshare/mgr/iristemp/ by 30 MB. 07/26/22-07:50:48:428 (2206) 0 [Database.SparseDBExpansion] Expanding capacity of sparse database /hs-patient/sys/healthshare/mgr/iristemp/ by 33 MB. 07/26/22-07:52:22:796 (2206) 0 [Database.SparseDBExpansion] Expanding capacity of sparse database /hs-patient/sys/healthshare/mgr/iristemp/ by 37 MB. 07/26/22-07:52:58:860 (2206) 0 [Database.SparseDBExpansion] Expanding capacity of sparse database /hs-patient/sys/healthshare/mgr/iristemp/ by 42 MB. 07/26/22-07:54:57:944 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 18 MB requested. 07/26/22-07:54:58:011 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 18 MB. 07/26/22-07:55:21:270 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 20 MB requested. 07/26/22-07:55:21:339 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 20 MB. 07/26/22-07:55:45:101 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 22 MB requested. 07/26/22-07:55:45:201 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 22 MB. 07/26/22-07:56:04:751 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 25 MB requested. 07/26/22-07:56:04:877 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 25 MB. 07/26/22-07:56:25:116 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 28 MB requested. 07/26/22-07:56:25:227 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 28 MB. 07/26/22-07:56:49:734 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 32 MB requested. 07/26/22-07:56:49:838 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 32 MB. 07/26/22-07:57:17:084 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 36 MB requested. 07/26/22-07:57:17:210 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 36 MB. 07/26/22-07:57:53:727 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 40 MB requested. 07/26/22-07:57:53:910 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 40 MB. 07/26/22-07:58:26:561 (2577) 2 [Utility.Event] [SYSTEM MONITOR] CPUusage Alert: CPUusage = 99, 99, 99 (Max value is 85). 07/26/22-07:58:35:865 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 45 MB requested. 07/26/22-07:58:36:028 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 45 MB. 07/26/22-07:59:19:415 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 51 MB requested. 07/26/22-07:59:19:639 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 51 MB. 07/26/22-07:59:19:976 (2198) 0 [Generic.Event] Journal log has been changed from edition 391 to 392 07/26/22-07:59:20:108 (2198) 0 [Generic.Event] INTERSYSTEMS IRIS JOURNALING SYSTEM MESSAGE Automatic journal file roll to: /hs-patient/jrn/pri/MIRROR-CCMIRROR-20220726.002 07/26/22-08:00:09:356 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 57 MB requested. 07/26/22-08:00:09:775 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 57 MB. 07/26/22-08:00:58:380 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 64 MB requested. 07/26/22-08:00:58:601 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 64 MB. 07/26/22-08:01:55:805 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 72 MB requested. 07/26/22-08:01:56:059 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 72 MB. 07/26/22-08:02:58:897 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 81 MB requested. 07/26/22-08:02:59:235 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 81 MB. 07/26/22-08:04:10:325 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 91 MB requested. 07/26/22-08:04:10:650 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 91 MB. 07/26/22-08:05:27:012 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 103 MB requested. 07/26/22-08:05:27:366 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 103 MB. 07/26/22-08:06:52:593 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 116 MB requested. 07/26/22-08:06:53:083 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 116 MB. 07/26/22-08:08:27:578 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 130 MB requested. 07/26/22-08:08:28:020 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 130 MB. 07/26/22-08:10:13:484 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 146 MB requested. 07/26/22-08:10:13:962 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 146 MB. 07/26/22-08:11:59:242 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 165 MB requested. 07/26/22-08:11:59:812 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 165 MB. 07/26/22-08:15:46:891 (2206) 0 [Database.StartExpansion] Starting Expansion for database /hs-patient/db/HSPIPIDQ/. 185 MB requested. 07/26/22-08:15:47:500 (2206) 0 [Database.FullExpansion] Expansion completed for database /hs-patient/db/HSPIPIDQ/. Expanded by 185 MB.
Yes? Cubes need space to store thier copy of the facts and CPU is needed to build it all.
As I said above, using async reporting mirror for cubes would completely remove the impact on the patient index.
Thanks Eduard,
I will look into implementing an async reporting mirror for cubes.
Regards,