This tag relates to the discussions on the development of analytics and business intelligence solutions, visualization, KPI and other business metrics management.
The rise of Big Data projects, real-time self-service analytics, online query services, and social networks, among others, have enabled scenarios for massive and high-performance data queries. In response to this challenge, MPP (massively parallel processing database) technology was created, and it quickly established itself. Among the open-source MPP options, Presto (https://prestodb.io/) is the best-known option. It originated in Facebook and was utilized for data analytics, but later became open-sourced.
In today's data landscape, businesses encounter a number of different challenges. One of them is to do analytics on top of unified and harmonized data layer available to all the consumers. A layer that can deliver the same answers to the same questions irrelative to the dialect or tool being used.
The invention and popularization of Large Language Models (such as OpenAI's GPT-4) has launched a wave of innovative solutions that can leverage large volumes of unstructured data that was impractical or even impossible to process manually until recently.
A few months ago, I faced a significant challenge: streamlining the handling of business logic in our application. My goal was to extract the business logic from the code and hand it over to analysts. Dealing with a multitude of rules could easily result in a code littered with countless "if" statements, especially if the coder lacked an understanding of cyclomatic complexity. Such code becomes a source of pain for those working with it—difficult to write, test, and develop.
Creating information dashboards, pivot tables, and widgets is an important step in analysis that provides valuable sources of information for informed decision-making. The IRIS BI platform offers many opportunities to create and customize these elements. In this article, we will take a closer look at the basic techniques for developing them and the importance of using them.
When analyzing data, there is often a need to look at specific indicators more thoroughly and to highlight sections of information of particular interest to a user.
For instance, examining the data dynamics for specific regions or dates can help us uncover some hidden trends and patterns that will allow us to make an informed decision about our project in the future.
As said in the previous article about the iris-fhir-generative-ai experiment, the project logs all events for analysis. Here we are going to discuss two types of analysis covered by analytics embedded in the project:
A simple data analysis example created in IntegratedML and Dashboard
Based on InterSystems' Integrated ML technology and Dashboard, automatically generate relevant predictions and BI pages based on uploaded CSV files. The front and back ends are completed in Vue and Iris, allowing users to generate their desired data prediction and analysis pages with simple operations and make decisions based on them.
When I started this project I had set myself limits: Though there is a wide range of almost ready-to-use modules in various languages and though IRIS has excellent facilities and interfaces to make use of them I decided to solve the challenge "totally internal" just with embedded Python, SQL, ObjectScript Neither Java, nor Nodes, nor Angular, PEX, ... you name it. The combination of embedded Python and SQL is preferred. ObjectScript is just my last chance.
Apache Superset is a modern data exploration and data visualization platform. Superset can replace or augment proprietary business intelligence tools for many teams. Superset integrates well with a variety of data sources.
And now it is possible to use with InterSystems IRIS as well.
An online demo is available and it uses IRIS Cloud SQL as a data source.
With the improvement of living standards, people pay more and more attention to physical health. And the healthy development of children has become more and more a topic of concern for parents. The child's physical development can be reflected from the child's height and weight. Therefore, it is of great significance to predict the height and weight in a timely manner.
With InterSystems IRIS 2022.2, we introduced Columnar Storage as a new option for persisting your IRIS SQL tables that can boost your analytical queries by an order of magnitude. The capability is marked as experimental in 2022.2 and 2022.3, but will "graduate" to a fully supported production capability in the upcoming 2023.1 release.
The product documentation and this introductory video, already describe the differences between row storage, still the default on IRIS and used throughout our customer base, and columnar table storage and provide high-level guidance on choosing the appropriate storage layout for your use case. In this article, we'll elaborate on this subject and share some recommendations based on industry-practice modelling principles, internal testing, and feedback from Early Access Program participants.
We offer you to embed business intelligence into your applications in order to give your users an opportunity to ask and answer sophisticated questions about their data. Typically, your application will include customizable dashboards that can provide insight into data from Business Intelligence models known as cubes.
In contrast with traditional BI systems that use static data warehouses, Business Intelligence keeps being constantly synchronized with the live transactional data.
Today we will talk about InterSystems Reports. This is a BI system that provides you with tools to create static reports and export them to different file formats. We will see how it works using the DC Analytics public analytical sample as an example. In this article, we will examine how to familiarize yourself with the reports available in the repository, how to make a new report based on a ready-made data structure, and how to prepare a data structure from scratch.
Today we will talk about Adaptive Analytics. This is a system that allows you to receive data from various sources with a relativistic data structure and create OLAP cubes based on this data. This system also provides the ability to filter and aggregate data and has mechanisms to speed up the work of analytical queries.
This set of tools (RanRead, RanWrite, and the combined RanIO) is used to generate random read and write events within a database (or pair of databases) to test the IO speed of IRIS running on a specified hardware setup. While Read operations can be measured in the usual Input/Output operations per second (IOPS) since they're direct disk reads, write events are sent to the database and thus their physical writes are managed by IRIS's write daemon.
Results gathered from the IO tests will vary from configuration to configuration based on the IO sub-system. Before running these tests, ensure corresponding operating system and storage level monitoring are configured to capture IO performance metrics for later analysis. The suggested method is by running the System Performance tool that comes bundled within IRIS. Please note that this is an update to a previous release, which can be found here.
The capacity of taking numerous records every second while also facilitating real-time queries simultaneously in real time is called Hybrid Transactional Analytical Processing (HTAP). It is also called Transactional analytics or Transanalytics or Translytics and is a very useful element in scenarios where there is constant flow of real time data coming from IIOT sensors or data on fluctuations in stock market, and supporting the need for querying these data sets in real-time or near real-time.
I just wrote up a quick sample to help a colleague load data into IRIS from R using RJDBC, and figured it's worth sharing here for future reference.
Ultimately it was pretty simple, aside from IRIS not liking "." in column names; the workaround is to just rename the columns. Someone better at R than me could probably provide some generic approach.
When we are at the starting stage of BI project development, we must remember that it is crucial to select the right tool for its implementation. Today we want to show you how one of the principal functionality of dashboards is implemented in different BI systems. Let's talk about drill down from both points of view: the dashboard development, and the convenience and clarity for the end user. We will touch on the applications of this technology in IRIS BI, Power BI and Tableau.
InterSystems Developer Community analytics. Project made with InterSystems IRIS BI (DeepSee), Power BI and Logi Report Designer to visualize and analyze members, articles, questions, answers, views and other pieces of content and activity on InterSystems Developer Community.
You can see your own activity, articles and questions. Track how your contribution changes developer community.
One of the most important tasks of dashboards is, on the one hand, to help you perceive data in an aggregated form, and, on the other hand, not to lose the depth of immersion in the data if you need this. One of the tools that help us achieve this result is drill down. It enables us to display several hierarchical levels of data, from aggregated to detailed.
In this article you will have access to the curated base of articles from the InterSystems Developer Community of the most relevant topics to learning InterSystems IRIS. Find top published articles ranked by Machine Learning, Embedded Python, JSON, API and REST Applications, Manage and Configure InterSystems Environments, Docker and Cloud, VSCode, SQL, Analytics/BI, Globals, Security, DevOps, Interoperability, Native API. Learn and Enjoy!
When we collect temporary data (the number of purchases in the store, the number of comments on the post), it may happen that there is no data for a certain period of time. In this case, this time period (hour, day, month) is not represented in the database, that is, there is not a single row for this period. In other words, there are no rows in the database for this period.
How to include IRIS Data into your Google Big Query Data Warehouse and in your Data Studio data explorations. In this article we will be using Google Cloud Dataflow to connect to our InterSystems Cloud SQL Service and build a job to persist the results of an IRIS query in Big Query on an interval.
I was using PowerBI to create regular display data obtained from one popular web sourse with hundreds of thousands of visitors per month and a big number of users.
At the beginning of that visualisation development, I was using direct connection from Power BI to Adaptive Analytics powered by AtScale. Adaptive Analytics is useful for cached data, aggregates and fast data sources switching between development and stage phases. The “AtScale cubes'' connection method was used:
I'm happy to share with the community a web server log dataset from our longtime customer, an operating company.
Their webserver operates on Apache webserver and contains data which can be useful to analyse a load and search engines activity.
After installing the project, you will get the data for a few months that can show a typical load and activity of clients, robots and also you can see how it depends on day of week, holidays and time of a day.
A full cycle project, from initial data initialization, daily update products from the official website, a service for sending a course on request and receiving a schedule for any period.
By date
Open Exchange app.png)
.png)
.png)