Apache Spark has rapidly become one of the most exciting technologies for big data analytics and machine learning. Spark is a general data processing engine created for use in clustered computing environments. Its heart is the Resilient Distributed Dataset (RDD) which represents a distributed, fault tolerant, collection of data that can be operated on in parallel across the nodes of a cluster. Spark is implemented using a combination of Java and Scala and so comes as a library that can run on any JVM.

11 5
0 2,419

Motivation

This project was thought of when I was thinking of how to let Python code deal naturally with the scalable storage and efficient retrieving mechanism given by IRIS globals, through Embedded Python.

My initial idea was to create a kind of Python dictionary implementation using globals, but soon I realized that I should deal with object abstraction first.

So, I started creating some Python classes that could wrap Python objects, storing and retrieving their data in globals, i.e., serializing and deserializing Python objects in IRIS globals.

6 1
1 2,076

Hi Community,

This post is a introduction of my openexchange iris-python-apps application. Build by using Embedded Python and Python Flask Web Framework.
Application also demonstrates some of the Python functionalities like Data Science, Data Plotting, Data Visualization and QR Code generation.

image

Features

  • Responsive bootstrap IRIS Dashboard

  • View dashboard details along with interoperability events log and messages.

  • Use of Python plotting from IRIS

  • Use of Jupyter Notebook

  • Introduction to Data Science, Data Plotting and Data Visualization.

  • QR Code generator from python.

4 1
0 1,707

This is the first article of a series diving into visualization tools and analysis of time series data. Obviously we are most interested in looking at performance related data we can gather from the Caché family of products. However, as we'll see down the road, we are absolutely not limited to that. For now we are exploring python and the libraries/tools available within that ecosystem.

9 4
1 1,395

If a picture is worth a thousand words, what's a video worth? Certainly more than typing a post.

Please check out my "Coding talks" on InterSystems Developers YouTube:

1. Analysing InterSystems IRIS System Performance with Yape. Part 1: Installing Yape

Running Yape in a container.

2. Yape Container SQLite iostat InterSystems

Extracting and plotting pButtons data including timeframes and iostat.

13 3
2 1,335

1. interoperability-embedded-python

This proof of concept aims to show how the iris interoperability framework can be use with embedded python.

1.1. Table of Contents

1.2. Example

import grongier.pex
import iris
import MyResponse

class MyBusinessOperation(grongier.pex.BusinessOperation):

    def OnInit(self):
        print("[Python] ...MyBusinessOperation:OnInit() is called")
        self.LOGINFO("Operation OnInit")
        return

    def OnTeardown(self):
        print("[Python] ...MyBusinessOperation:OnTeardown() is called")
        return

    def OnMessage(self, messageInput):
        if hasattr(messageInput,"_IsA"):
            if messageInput._IsA("Ens.StringRequest"):
                self.LOGINFO(f"[Python] ...This iris class is a Ens.StringRequest with this message {messageInput.StringValue}")
        self.LOGINFO("Operation OnMessage")
        response = MyResponse.MyResponse("...MyBusinessOperation:OnMessage() echos")
        return response

1.3. Regsiter a component

No ObjectScript code is needed.

8 6
2 1,331
Article
Niyaz Khafizov · Jul 27, 2018 4m read
Load a ML model into InterSystems IRIS

Hi all. Today we are going to upload a ML model into IRIS Manager and test it.

Note: I have done the following on Ubuntu 18.04, Apache Zeppelin 0.8.0, Python 3.6.5.

Introduction

These days many available different tools for Data Mining enable you to develop predictive models and analyze the data you have with unprecedented ease. InterSystems IRIS Data Platform provide a stable foundation for your big data and fast data applications, providing interoperability with modern DataMining tools.

6 2
2 1,136

Keywords: Anaconda, Jupyter Notebook, Tensorflow GPU, Deep Learning, Python 3 and HealthShare

1. Purpose and Objectives

This "Part I" is a quick record on how to set up a "simple" but popular deep learning demo environment step-by-step with a Python 3 binding to a HealthShare 2017.2.1 instance . I used a Win10 laptop at hand, but the approach works the same on MacOS and Linux.

4 0
2 1,000

Hi,

this is a public announcement for the first release of Intersystems Cache Object-Relational Mapper in Python 3. Project's main repository is located at Github (healiseu/IntersystemsCacheORM).

About the project

CacheORM module is an enhanced OOP porting of Intersystems Cache-Python binding. There are three classes implemented:

The intersys.pythonbind package is a Python C extension that provides Python application with transparent connectivity to the objects stored in the Caché database.

2 2
1 895

In last week's discussion we created a simple graph based on the data input from one file. Now, as we all know, sometimes we have multiple different datafiles to parse and correlate. So this week we are going to load additional perfmon data and learn how to plot that into the same graph.
Since we might want to use our generated graphs in reports or on a webpage, we'll also look into ways to export the generated graphs.

5 0
0 852

Keywords: Jupyter Notebook, Tensorflow GPU, Keras, Deep Learning, MLP, and HealthShare

1. Purpose and Objectives

In previous"Part I" we have set up a deep learning demo environment. In this "Part II" we will test what we could do with it.

Many people at my age had started with the classic MLP (Multi-Layer Perceptron) model. It is intuitive hence conceptually easier to start with.

1 2
3 767

Headache-free stored objects: a simple example of working with InterSystems Caché objects in ObjectScript and Python

Neuschwanstein Castle

Tabular data storages based on what is formally known as the relational data model will be celebrating their 50th anniversary in June 2020. Here is an official document – that very famous article. Many thanks for it to Doctor Edgar Frank Codd. By the way, the relational data model is on the list of the most important global innovations of the past 100 years published by Forbes.

On the other hand, oddly enough, Codd viewed relational databases and SQL as a distorted implementation of his theory. For general guidance, he created 12 rules that any relational database management system must comply with (there are actually 13 rules). Honestly speaking, there is zero DBMS's on the market that observes at least Rule 0. Therefore, no one can call their DBMS 100% relational :) If you know any exceptions, please let me know.

4 0
3 662

Hi developers!

Recently we announced the preview of Embedded Python technology in InterSystems IRIS.

Check the Sneak Peak video by @Bob Kuszewski.

Embedded python gives the option to load and run python code in the InterSystems IRIS server. You can either use library modules from Python pip, like numpy, pandas, etc, or you can write your own python modules in the form of standalone py files.

So once you are happy with the development phase of the IRIS Embedded Python solution there is another very important question of how the solution could be deployed.

One of the options you can consider is using the ZPM Package manager which is described in this article.

2 5
0 655

ObjectScript Kernel Logo
Jupyter Notebook is an interactive environment consisting of cells that allow executing code in a great number of different markup and programming languages.

To do this Jupyter has to connect to an appropriate kernel. There was no ObjectScript Kernel, that is why I decided to create one.

You can try it out here.

Here's a sneak peek of the results:

10 4
3 629

Python has become the most used programming language in the world (source: https://www.tiobe.com/tiobe-index/) and SQL continues to lead the way as a database language. Wouldn't it be great for Python and SQL to work together to deliver new functionality that SQL alone cannot? After all, Python has more than 380,000 published libraries (source: https://pypi.org/) with very interesting capabilities to extend your SQL queries within Python.

17 3
0 615