Challenges of real-time AI/ML computations

We will start from the examples that we faced as Data Science practice at InterSystems:

  • A “high-load” customer portal is integrated with an online recommendation system. The plan is to reconfigure promo campaigns at the level of the entire retail network (we will assume that instead of a “flat” promo campaign master there will be used a “segment-tactic” matrix). What will happen to the recommender mechanisms? What will happen to data feeds and updates into the recommender mechanisms (the volume of input data having increased 25000 times)? What will happen to recommendation rule generation setup (the need to reduce 1000 times the recommendation rule filtering threshold due to a thousandfold increase of the volume and “assortment” of the rules generated)?
  • An equipment health monitoring system uses “manual” data sample feeds. Now it is connected to a SCADA system that transmits thousands of process parameter readings each second. What will happen to the monitoring system (will it be able to handle equipment health monitoring on a second-by-second basis)? What will happen once the input data receives a new bloc of several hundreds of columns with data sensor readings recently implemented in the SCADA system (will it be necessary, and for how long, to shut down the monitoring system to integrate the new sensor data in the analysis)?
  • A complex of AI/ML mechanisms (recommendation, monitoring, forecasting) depend on each other’s results. How many man-hours will it take every month to adapt those AI/ML mechanisms’ functioning to changes in the input data? What is the overall “delay” in supporting business decision making by the AI/ML mechanisms (the refresh frequency of supporting information against the feed frequency of new input data)?

4 0
1 726

The first installment of this article series discussed how to read a big chunk of data from the raw body of an HTTP POST method and save it to a database as a stream property of a class. The second installment discussed how to send files and their names wrapped in a JSON format.

Now let’s look closer at the idea of sending large files in parts to the server. There are several approaches we can use to do this. This article discusses using the Transfer-Encoding header to indicate chunked transfer. The HTTP/1.1 specification introduced the Transfer-Encoding header, and the RFC 7230 section 4.1 described it, but it’s absent from the HTTP/2 specification.

4 0
0 879

Who has never been through in the following situation:

I have an application/global configuration that is not and cannot be mapped to a class, but it is necessary to provide a stored procedure so that a report tool using ODBC/JDBC or even using the resultset of IRIS itself, can access the data and generate the report.

In IRIS we have a functionality that allows us to create a query that can be accessed internally and also be exposed as a stored procedure, with our own logic. That functionality is Custom Class Query.

4 1
0 274

Continuing with the series of articles on voice file management, we are going to see how we can convert text into audio and receive the file with the chosen voice.
We will also explore how a service from OpenAI can help us analyze a text and determine the mood expressed in it.
Let's analyze how you can create your own voice file and how it can “read” your feelings.

4 0
2 379

Hey, community! 👋

We are a team of Stanford students applying technology to make sense of climate action. AI excites us because we know we can quickly analyze huge amounts of text.

As we require more reports on sustainability, such as responsibility reports and financial statements, it can be challenging to cut through the noise of aspirations and get to the real action: what are companies doing

That’s why we built a tool to match companies with climate actions scraped from company sustainability reports.

4 0
1 59

Hello.

The idea of this post is to introduce Frontier: An abstraction layer that allows Rapid REST development.

REQUIREMENTS:

Why?

Have you ever found yourself dealing with repetitive tasks like mounting objects, serializing them and eventually handling multiple errors for multiple cases? Frontier can boost your development by making you focus on what really matters: your application.

Frontier is made to stop you from WRITE'ing by instead forcing your methods to return values.
It's designed to make you code clean, and you'll see the why pretty soon.

This is the Part 1, where you'll learn he basics about how to work with Frontier. That means at the end of this part you should be capable of

creating GET requests without difficulties. Since this also serves as a way to introduce the framework, I'll be calling this part: Core concepts.

4 8
0 1.7K

This is the third post of a series explaining how to create an end-to-end Machine Learning system.

Training a Machine Learning Model

When you work with machine learning is common to hear this work: training. Do you what training mean in a ML Pipeline?
Training could mean all the development process of a machine learning model OR the specific point in all development process
that uses training data and results in a machine learning model.

4 10
2 365
Article
· Jul 21, 2022 11m read
ECP With Docker

Hi community,

This is the third article in the series about initializing IRIS instances with Docker. This time, we will focus on Enterprise Cache Protocol (ECP).

In a very simplified way, ECP allows configuring some IRIS instances as application servers and others as data servers. Detailed technical information can be found in the official documentation.

This article aims to describe:

4 0
2 931
Article
· Jun 16, 2023 10m read
Creating a REST service in IRIS

One of the most common needs of our clients is the creation of REST services that allow access to the information present in IRIS / HealthConnect. The advantage of these REST services is that it allows the development of custom user interfaces with the most current technologies taking advantage of the reliability and performance of IRIS in the back-end.

4 0
5 1.1K

I recently had the need to monitor from HealthConnect the records present in a NoSQL database in the Cloud, more specifically Cloud Firestore, deployed in Firebase. With a quick glance I could see how easy it would be to create an ad-hoc Adapter to make the connection taking advantage of the capabilities of Embedded Python, so I got to work.

4 2
1 431
Article
· Feb 7, 2024 2m read
Splitting access by WebServer port

Recently, I needed to run WebGateway on an additional port but with a twist - this port should publish only one web application.
At first, I thought about configuring Web Gateway to allow only specific web applications (~urls), but Web Gateway configuration is per Apache configuration:

4 3
1 283

InterSystems FAQ rubric

If you do not specify the option to remove the mirror attribute of the mirror database when deleting the mirror configuration, the database cannot be restored to a normal state and will be mounted read-only the next time it is mounted. To restore the database to a read-write state, you must remove the mirror attribute using the system routine ^MIRROR.

The procedure is as follows (execute in the %SYS namespace):

4 0
2 247

Terminal scripts can be used to run pre-designed commands on the terminal, like a batch file. You can write anything that can be executed on terminal, like for loop, if else and so on, inside Terminal scripts. In this article, I will show you how to call Terminal scripts, how to use parameters in Terminal scripts and how to avoid session disconnected when running Terminal scripts. If you have any information about how to use Terminal scripts or you have any feedback, please feel free to leave a comment.

4 6
2 3.7K
Article
· Nov 18, 2020 7m read
Creating a PEX Business Operation

PEX is a new InterSystems IRIS feature to allows extends IRIS with existent features from Java or .NET.

It is possible create interoperability inbound and outbound adapters, business services (integrate from external to internal) and operations (integrate internal to external).

To create a PEX component it is necessary import .NET (InterSystems.EnsLib.PEX.*) or Java (com.intersystems.enslib.pex.*) packages and extends or implements the properly class.

4 2
2 485

Maternal Risk can be measured from some parameters well known to the medical community. In this way, in order to help the medical community and computerized systems, especially AI, the scientist Yasir Hussein Shakir published a very useful dataset for training ML algorithms in the detection/prediction of Maternal Risk.

4 0
1 285

Sometimes we need to know for sure if the current environment has sufficient cores, memory, and bandwidth to support the planned number of users and such SLAs as latency, response time, and availability. This is true for databases and backends. This is why it is mandatory for critical applications and databases to simulate the users simultaneous/concurrent requests and collect metrics about performance and availability.

4 0
2 625
Article
· Jun 8, 2023 1m read
How to write SELECT ... FOR UPDATE

InterSystems FAQ rubric

Since SELECT ... FOR UPDATE is implemented in many RDBMS as a method of explicit row lock acquisition, I think there are many cases where this function is used.

This syntax is not an error for InterSystems products, but it does not acquire row locks as expected.

This article will show you how to achieve equivalent functionality.

4 0
0 341

I implemented a Python Flask application for the 2024 Python Contest with a page that provides common form fields for an outgoing email such as the To and CC fields. And it lets you input a message as well as uploading text based attachments.

Then using LlamaIndex in Python, the app analyzes the content you put in and returns to you in a result box if there is anything that should stop you from sending that email.

Take a look at the Github repo here.

4 3
0 126
Article
· Nov 11, 2024 4m read
IrisGoogleChat with AI

Hi Community,

I want to share with you the lastest app that I have uploaded to the Open Exchange "IrisGoogleChat".

IrisGoogleChat is a utility for InterSystems IRIS that allows seamless message integration with Google Chat using Cache ObjectScript. This application provides a set of tools to configure Google Chat Channels, create messages powered by moods generated with AI and send them to a Google Chat Channel.

4 0
1 114
Article
· Jan 16, 2020 2m read
Python Gateway VI: Jupyter Notebook

This series of articles would cover Python Gateway for InterSystems Data Platforms. Execute Python code and more from InterSystems IRIS. This project brings you the power of Python right into your InterSystems IRIS environment:

  • Execute arbitrary Python code
  • Seamlessly transfer data from InterSystems IRIS into Python
  • Build intelligent Interoperability business processes with Python Interoperability Adapter
  • Save, examine, modify and restore Python context from InterSystems IRIS

Other articles

The plan for the series so far (subject to change).

Intro

The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text.

This extension allows you to browse and edit InterSystems IRIS BPL processes as jupyter notebooks.

4 0
0 664

If you work with interoperability productions of InterSystems IRIS or Ensemble, no doubt you are familiar with the Message Viewer page. The page supports filtering messages according to filter criteria you enter in the Basic and/or Extended Criteria sections. Extended Criteria conditions are specified as property-operator-value triples. Once you click Search button, such triples become WHERE clause conditions of a generated SQL query executed against message header/body tables.

4 0
2 610
Article
· Nov 19, 2020 6m read
OCR and NLP together into InterSystems IRIS

According to IDC, more than 80% of information it is NoSQL, especially text into documents. When the digital services or applications not process all this information, the business lose. To face this challenge, it is possible use OCR technology. OCR uses machine learning and/or trained image patterns to transform image pixels into text. This is important, because many documents are scanned into images inside PDF, or many documents contains images with text inside. So OCR are an important step to get all possible data from a document.

4 3
2 578

The pandemic that struck the world in 2020 made everyone follow the news and the numbers that involve the COVID-19.

Why don’t you take that opportunity to create something simple and pleasant, to follow the number of vaccinations worldwide?

To face this challenge, I'm using the data provided by Our World in Data - Research and data to make progress against the world’s largest problems.

They have a dedicated repository on Github with the data of COVID-19, and I took the vaccination data to help me with my tracker.

4 4
0 394