#Python

Data privacy regulations such as GDPR, LGPD, and HIPAA demand that organizations know exactly where Personally Identifiable Information (PII) lives inside their databases. Yet in practice, most teams rely on manual inventories, tribal knowledge, or external scanning tools that require data to leave the database engine — a process that itself creates privacy and security risks.

This article presents an MVP that takes a different approach: it runs PII detection inside InterSystems IRIS using Embedded Python, analyzing data where it lives and never exporting it to an external process.

Abstract

Solar irradiance forecasting is critical for grid stability in photovoltaic (PV) power plants. This article replicates and extends the methodology of Lara-Benítez et al. (2023) "Short-term solar irradiance forecasting in streaming with deep learning" replacing the original offline simulation with a fully operational streaming pipeline built on InterSystems IRIS. We leverage IRIS Interoperability Productions as the streaming backbone, Embedded Python to run MLP, LSTM, and CNN deep learning models, and IntegratedML as an AutoML baseline.

This article introduces SHAP explainability methods as an approach to understand the reasons behind predictions in machine learning black-box models. It also includes a simple Jupyter notebook that you can use and modify to gain hands-on experience with these concepts:

https://www.kaggle.com/code/jorgeivnjh/explainability-in-ml-models

https://github.com/JorgeIvanJH/Explainability-in-ML-models

We will leverage these concepts for a future implementation in our Continuous Training Pipeline: https://community.intersystems.com/post/complementing-iris-mlflow-continuous-training-ct-pipeline