Consolidating Risk Management and Compliance Silos in Financial Services

For software engineers and data architects evaluating data platforms for risk and compliance consolidation in financial services.

Overview

Financial institutions typically run separate applications for risk management and regulatory compliance, creating fragmented data landscapes that are difficult to query and expensive to maintain. InterSystems IRIS is a distributed HTAP data platform that allows organizations to consolidate these systems onto a single, unified data layer. It provides cross-system querying, real-time and batch processing, NLP for unstructured data, and enterprise-grade security — without requiring organizations to replace their existing infrastructure.

The Problem: Fragmented Risk and Compliance Systems

For years, financial institutions have been implementing disparate risk management and compliance applications to meet regulatory requirements and reduce operational risk.

Recently, however, many organizations have begun unifying these separate applications via a single, comprehensive, enterprisewide platform. Doing so allows them to gain a more accurate and complete view of their enterprise data, identify operational and compliance risks and suspicious activity faster and more accurately, comply with new and changing regulations, and simplify overall management and maintenance.

InterSystems IRIS Data Platform for Consolidated Risk Management and Regulatory Compliance

InterSystems IRIS can help firms consolidate their risk management and regulatory compliance applications onto a single platform.

It complements an organization's existing data management infrastructure, including legacy applications, data warehouses, big-data technologies, and data lakes. Organizations can obtain a unified, panoramic, secure view of data from multiple sources across the organization via a real-time distributed caching layer.

InterSystems IRIS provides a scale-out processing layer to perform complex batch and real-time processing tasks on large, distributed data sets. It can perform complex multi-table joins on sharded data without requiring co-sharding, replicating data, or performing network broadcasts. This enables organizations to reliably and quickly process queries that would otherwise take hours or time out before completion, at a lower operational cost.

HTAP Multi-Model Database

At the core of InterSystems IRIS is a distributed, multi-model HTAP (hybrid transaction/analytical processing) database designed to work with large sets of heterogeneous data. It ingests, stores, and indexes large volumes of transactional data at very high rates to support real-time analytical use cases.
 

Key characteristics:

• Data is stored once and can be represented as relational data, objects, multi-dimensional arrays, key-value pairs, documents, and other data types;
• Eliminates the need to duplicate data or map between different representations (e.g., object-relational mapping);
• Natively supports sharded, distributed scale-out architectures using commodity resources;
• Provides Kerberos and LDAP integration, role-based access control, and encryption for data in transit and at rest.

InterSystems IRIS lets organizations gain a consistent view across all their enterprise data, gain more accurate and timely business intelligence, ensure better compliance with regulations, and respond more quickly to unanticipated questions from financial regulators and compliance analysts. Its horizontally scalable technology supports scale-out, sharded architectures to manage and analyze very large data sets using low-cost distributed processing and storage nodes.

Enterprise Cache Protocol

InterSystems provides high-performance, distributed, multi-workload (transaction and analytic) processing at very high scale via Enterprise Cache Protocol (ECP), which has been implemented and used in thousands of production environments.

ECP is an integral capability of InterSystems IRIS. It quickly and reliably returns responses to queries on distributed data sets regardless of how the data is organized. It natively supports distributed, sharded architectures. Complex joins are processed locally rather than broadcast across the network, eliminating the latencies and time-outs typically associated with broadcast joins while increasing performance and reducing operational costs. ECP makes it possible for regulators and compliance analysts to quickly obtain accurate results from ad hoc queries without expensive and time-consuming pre-processing or replication of data.

Using ECP requires no application changes or specialized techniques: applications and processing tasks simply treat the data as if it were local. The performance and scalability benefits enable organizations to answer questions, correlate information, and identify patterns in distributed data sets that are not co-sharded, with performance and reliability, and at a significantly lower cost.

Integration With Apache Spark

Many of the business and regulatory drivers for risk management and compliance now require intraday and near-real-time reporting and visibility, driving organizations to increasingly employ higher-performance computing techniques to reduce latency. As a result, Apache Spark, with its in-memory architecture, is being evaluated and adopted by some organizations.

InterSystems IRIS provides parallel connectivity with Spark and works with architectures already in use at many financial institutions. The shard-aware integration and enrichment layer supports batch inserts and individual inserts, allowing very large data sets to be ingested into InterSystems IRIS data shards quickly.

InterSystems technology integrates directly with Spark via a shard-aware native Spark connector. The connector presents InterSystems IRIS data shards as native partitions to optimize performance. This allows organizations to use InterSystems IRIS alongside Spark and optimize Spark queries for improved performance.

Concurrent Transactional and Analytic Processing Using Real-Time Data

Whereas some risk and compliance use cases call for data to be analyzed in batches, others require real-time data to be analyzed with historical or reference data. For example, they may require transaction monitoring and filtering, intraday or pre-trade liquidity calculations, and other real-time and low-latency applications.

InterSystems IRIS handles both real-time and batch requirements. It is optimized to process both very large transactional workloads and a high volume of analytic queries on transactional data and historical or other batch data simultaneously, without compromising performance.

Interoperability

InterSystems IRIS provides connectivity to a wide range of applications and data sources. It includes a built-in adapter library covering:

• REST, SOAP, HTTP/S, Financial Information Exchange, Kafka, Java Message Service
• Oracle, Sybase, DB2, Hive, Impala (via SQL gateway, presented as native tables)
• Object, OLAP, and JSON/REST access methods in addition to ANSI SQL
• Workflow engine, encryption, entity resolution, and transformation capabilities

Security

InterSystems IRIS provides a unified security architecture based on four controls:

• Authentication — verifies the identity of all users
• Authorization — role-based and row-level SQL access control, ensuring users access only what they need
• Auditing — event log for predefined system and application-specific events
• Encryption — managed key encryption for data at rest and in transit

Additional: SSL/TLS support, Kerberos and LDAP integration, PKI tooling, and certification standard compliance. The security infrastructure places a minimal burden on performance and operations.

Integrated Analysis of Unstructured Text

Unstructured data, including free text in emails, documents, text messages, master agreements, Suspicious Activity Reports, and external data from blogs and tweets, can provide valuable insight to help banks reduce risk and identify suspicious or fraudulent behavior.

Its NLP technology uses a bottom-up approach to analyze text based on what is contained within the text itself. It supports customer-defined dictionaries and ontologies, and provides embedded semantic analysis for pattern and correlation detection. Supported capabilities:

• Data exploration and signal detection;
• Trend analysis and content-based clustering;
• Information extraction, categorization, and mapping.

These capabilities can be useful in summarizing and contextualizing large amounts of free text for various compliance and surveillance initiatives.

Data Lineage and Data Governance

Effective data lineage — the ability to describe the source of the data and how it changes as it moves through the data pipeline — and data governance are critical for risk and compliance initiatives. For example, the Securities and Exchange Commission's Rule 613 (Consolidated Audit Trail) requires organizations to collect and accurately identify every order, cancellation, modification, and trade execution for all exchange-listed equities and options across all U.S. markets.

Different applications that perform different functions may store different representations of the data or modify the data (e.g., breaking a large initial order into smaller child orders for execution). Compliance analysts and regulators must have confidence in the original data sources and the processes and transformations that are applied. InterSystems IRIS provides support for multiple data types, including both object and SQL schemas, and its flexible metadata capabilities allow the application of proper data lineage and governance.

Frequently Asked Questions

What is InterSystems IRIS used for in financial services? InterSystems IRIS is used as a distributed data platform for consolidating risk management and regulatory compliance applications. It provides a unified query layer across existing data sources and supports real-time and batch processing, built-in NLP, security, and data lineage capabilities.

Does InterSystems IRIS require replacing existing infrastructure? No. InterSystems IRIS complements existing infrastructure including legacy applications, data warehouses, data lakes, and relational databases. It acts as a caching and processing layer on top of existing systems.

What is Enterprise Cache Protocol (ECP)? ECP is a distributed data coordination technology built into InterSystems IRIS. It allows complex multi-table joins to be processed locally on the node holding the relevant data, rather than broadcasting queries across the network. Applications require no code changes to use it.

Can InterSystems IRIS handle both real-time and batch workloads? Yes. InterSystems IRIS is an HTAP database, meaning it processes transactional and analytical workloads on the same dataset simultaneously without compromising performance on either.

How does InterSystems IRIS support regulatory compliance? It provides data lineage tracking, flexible metadata capabilities, role-based and row-level access control, full audit logging, and support for both object and SQL schemas — enabling organizations to respond to regulatory queries and meet requirements such as SEC Rule 613.

Conclusion

InterSystems IRIS complements financial institutions' existing infrastructures to deliver a secure, unified view of enterprise data. It provides a distributed scale-out processing layer for handling complex batch and real-time tasks required for consolidating risk management and regulatory compliance solutions onto a single platform. By using InterSystems IRIS, financial institutions can query their enterprise data more efficiently, respond faster to regulatory questions, and reduce both operational and regulatory risk.

More articles on the subject:

• Massive Scalability with InterSystems IRIS Data Platform
• A Superior Alternative to In-Memory Databases and Key-Value Stores
• InterSystems IRIS Data Platform For IoT Applications

Source: Consolidating Risk Management and Compliance Silos in Financial Services