What is Web Scraping:

In simple terms, Web scraping, web harvesting, or web data extraction is an automated process of collecting large data(unstructured) from websites. The user can extract all the data on particular sites or the specific data as per the requirement. The data collected can be stored in a structured format for further analysis.

17 12
7 657

Python has become the most used programming language in the world (source: https://www.tiobe.com/tiobe-index/) and SQL continues to lead the way as a database language. Wouldn't it be great for Python and SQL to work together to deliver new functionality that SQL alone cannot? After all, Python has more than 380,000 published libraries (source: https://pypi.org/) with very interesting capabilities to extend your SQL queries within Python.

17 3
0 535

Overview

We started to use Azure Service Bus (ASB) as an enterprise messaging solution 3 years ago. It is being used to publish and consume data between many applications in the organization. Since the data flow is complex, and one application’s data is usually needed in multi applications the “publisher” ---> ”multiple subscribers” model was a great fit. The ASB usage in the organization is dozens of millions of messages per day, while IRIS platform is having around 2-3 million messages/day.

24 3
1 441



This formation, accessible on my GitHub, will cover, in half a hour, how to read and write in csv and txt files, insert and get inside the IRIS database and a distant database using Postgres or how to use a FLASK API, all of that using the Interoperability framework using ONLY Python following the PEP8 convention.

12 1
1 457

Adding VSCode into your IRIS container

One of the easiest ways to setup repeatable development environments is to spin up containers for them. I find that when iterating quickly, it was very convenient to host a vscode instance within my development container. Thus, I have created a quick container script to add a browser-based vscode into an IRIS container. This should work for most 2021.1+ containers. My code repository can be found here

10 5
0 485
Article
Robert Cemper · Jan 2, 2022 3m read
DB Migration using SQLgateway

Thanks to @Yuri Marx we have seen a very nice example for DB migration from Postgres to IRIS.
My personal problem is the use of DBeaver as a migration tool.
Especially as one of the strengths of IRIS ( and also Caché) before is the availability of the
SQLgateways that allow access to any external Db as long as for them an access usinig
JDBC or ODBC is available. So I extended the package to demonstrate this.

10 3
2 381

The last days I've work with the great new feature: LOAD DATA With this post I would like to share my first experiences with you. The following points do not contain any order or other evaluation. These are only things that I noticed when using the LOAD DATA command. It should also be noted that these points are based on the IRIS Version 2021.2.0.617 which is a preview release. So it may be that my observations do not apply to newer IRIS versions.

6 5
2 661
Article
Dmitry Maslennikov · Dec 3, 2021 1m read
VSCode-ObjectScript on GitHub

Not so while ago GitHub introduced, ability to very quickly run VSCode in the browser for any repository hosted there. Press the . key on any repository or pull request, or swap .com with .dev in the URL, to go directly to a VS Code environment in your browser.

github dev

This VSCode is a light version of the Desktop version but works entirely in Browser. And due to this, it has a limitation for extensions which was allowed to work this way. And let me introduce the new version 1.2.1 of VSCode-ObjectScript extension which now supports running in Browser mode.

22 4
3 724
Article
Dmitry Maslennikov · Aug 20, 2021 6m read
GitHub Codespaces with IRIS

Some time ago GitHub, has announced the new feature, GitHub Codespaces. It gives an ability to run VSCode in the browser, with almost the same power as it would run locally on your machine, but also with a power of clouds, so, you are able to choose the machine type with up to 32 CPU cores and 64 GB of RAM.

Looks impressive, is not it? But how it could help us, to work with projects driven by InterSystems IRIS? Let's have a look, how to configure it for us.

17 3
7 783

We are ridiculously good at mastering data. The data is clean, multi-sourced, related and we only publish it with resulting levels of decay that guarantee the data is current. We chose the HL7 Reference Information Model (RIM) to land the data, and enable exchange of the data through Fast Healthcare Interoperability Resources (FHIR®).

13 2
1 490

InterSystems SAM is a great tool to monitor your InterSystems IRIS and InterSystems IRIS For Health clusters on prem or in a cloud environment. This article describes how you can implement a customized alert handler. This is currently an undocumented and most likely an unknown feature of InterSystems SAM. With future releases it will be probably made easier to leverage this useful concept.

7 2
1 515

Like hardware hosts, virtual hosts in public and private clouds can develop resource bottlenecks as workloads increase. If you are using and managing InterSystems IRIS instances deployed in public or private clouds, you may have encountered a situation in which addressing performance or other issues requires increasing the capacity of an instance's host (that is, vertically scaling).

5 1
0 283

cAdvisor (short for container Advisor) analyzes and exposes resource usage and performance data from running containers. cAdvisor exposes Prometheus metrics out of the box.

https://prometheus.io/docs/guides/cadvisor/

Prometheus is integrated in SAM. This makes it possible to leverage the cAdvisor metrics and expose them via Prometheus and Grafana.

5 2
0 1,020
Article
Timothy Leavitt · Mar 17, 2021 3m read
Making the most of $Query

I ran into an interesting ObjectScript use case today with a general solution that I wanted to share.

Use case:

I have a JSON array (specifically, in my case, an array of issues from Jira) that I want to aggregate over a few fields - say, category, priority, and issue type. I then want to flatten the aggregates into a simple list with the total for each of the groups. Of course, for the aggregation, it makes sense to use a local array in the form:

agg(category, priority, type) = total

Such that for each record in the input array I can just:

14 10
4 725

Introduction

InterSystems IRIS 2020.1 includes PEX (Production EXtension Framework) to facilitate the development of IRIS Interoperability productions with components written in Java or .NET.

Thanks to PEX, an integration developer with knowledge of Java or .NET can benefit from the power, scalability, and robustness of the InterSystems IRIS Interoperability framework and be productive in no time.

5 4
1 567

In this article, we’ll build a highly available IRIS configuration using Kubernetes Deployments with distributed persistent storage instead of the “traditional” IRIS mirror pair. This deployment would be able to tolerate infrastructure-related failures, such as node, storage and Availability Zone failures. The described approach greatly reduces the complexity of the deployment at the expense of slightly extended RTO.

23 15
5 2,297

In this article, we will run an InterSystems IRIS cluster using docker and Merge CPF files - a new feature allowing you to configure servers with ease.

On UNIX® and Linux, you can modify the default iris.cpf using a declarative CPF merge file. A merge file is a partial CPF that sets the desired values for any number of parameters upon instance startup. The CPF merge operation works only once for each instance.

Our cluster architecture is very simple, it would consist of one Node1 (master node) and two Data Nodes (check all available roles). Unfortunately, docker-compose cannot deploy to several servers (although it can deploy to remote hosts), so this is useful for local development of sharding-aware data models, tests, and such. For a productive InterSystems IRIS Cluster deployment, you should use either ICM or IKO.

6 3
1 447

While the integrity of Caché and InterSystems IRIS databases is completely protected from the consequences of system failure, physical storage devices do fail in ways that corrupt the data they store. For that reason, many sites choose to run regular database integrity checks, particularly in coordination with backups to validate that a given backup could be relied upon in a disaster.

16 8
9 1,349

Updated Jan 19th, 2023.

Hi all,

I want to share a quick little method you can use to enable ssl with a self signed certificate on your local development instance of IRIS/HealthShare. This enables you to test https-specific features such as OAuth without a huge lift.

1. Install OpenSSL

Windows     : Download from https://www.openssl.org or other built OpenSSL Binary. 

Debian Linux: $ sudo apt-get -y install openssl

RHEL        : $ sudo yum install openssl
9 6
4 1,202