Continuous FHIR conformance with IRIS/HealthShare: regression testing + drift tracking patterns
Hello everyone,
I am André from MedVertical. We are exploring InterSystems-native ways to operationalize continuous FHIR conformance: repeatable regression runs, baseline/delta comparisons, and evidence-style reporting to detect drift after releases and IG changes.
In many FHIR implementations, validation is done “point-in-time” in pre-prod, but conformance degrades in production due to IG/profile updates, terminology changes, mapping evolution, upstream releases, and configuration drift.
Environment context (to anchor recommendations):
- InterSystems: IRIS for Health and/or HealthShare (both relevant)
- FHIR: R4 and R5 depending on customer setup
- Goal: repeatable regression runs after deployments/upgrades; deterministic results and reproducibility metadata captured per run
If helpful, this is the reference context for what I mean by baseline/delta + evidence reporting: https://medvertical.com/records
I’m looking for proven InterSystems-native patterns (or community tooling) for:
- Regression testing as a gate:
What do you use to run automated regression tests repeatedly (e.g., %UnitTest, IPM-based test runs, CI pipelines)? Any reference implementations? - Test data + scenarios:
How do you manage stable but meaningful FHIR fixtures (Bundles/resources/search scenarios) for regression testing? - Baseline/delta + evidence reporting:
If you track changes over time, which metrics or report formats have worked best to make deltas actionable (beyond pass/fail)?
If you can share links to Open Exchange packages, repos, or CI examples—and a short note on what worked vs. what didn’t—that would be very helpful.
Comments
Hi @André Sheydin, for standardized FHIR conformance testing, we recommend specialist tools like Inferno or Touchstone. You can take a look at open-source FHIR tests available here: https://github.com/inferno-framework or https://inferno.healthit.gov/test-kits/
For example, we regularly test the IRIS for Health FHIR Server against the ONC G10 standardized APIs required for US health modules, plus a handful of other test plans. We typically use Synthea to generate sample bundles conformant to a particular profile (e.g, US Core 6.1.0) with some manual curation. These tools do provide detailed reports and case-handling which you can dashboard or diff, but lots of opportunity to customize further for your needs.