Many organisations implement centralised log management systems to separate and centralise the log data in order to e.g. automate threat detection (and response) and to comply with regulatory requirements. The primary systems of interest are the various user facing applications, but increasingly also other kinds of systems including integration platforms.
JDBC
The Caché System Management Portal includes a robust web-based SQL query tool, but for some applications it’s more convenient to use a dedicated SQL client installed on a user’s PC.
SQuirreL SQL is a well known open source SQL client built in Java, which uses JDBC to connect to a DBMS. As such, we can configure SQuirreL to connect to Caché using the Caché JDBC driver.
Last reply 21 May 2020
Business Operation, Databases, Interoperability, JDBC, Performance, Ensemble, InterSystems IRIS, InterSystems IRIS for Health
IRIS and Ensemble are designed to act as an ESB/EAI. This mean they are build to process lots of small messages.
But some times, in real life we have to use them as ETL. The down side is not that they can't do so, but it can take a long time to process millions of row at once.
To improve performance, I have created a new SQLOutboundAdaptor who only works with JDBC.
BatchSqlOutboundAdapter
Extend EnsLib.SQL.OutboundAdapter to add batch batch and fetch support on JDBC connection.
Last reply 7 April 2020
Keywords: Python, JDBC, SQL, IRIS, Jupyter Notebook, Pandas, Numpy, and Machine Learning
1. Purpose
This is another 5-minute simple note on invoking the IRIS JDBC driver via Python 3 within i.e. a Jupyter Notebook, to read from and write data into an IRIS database instance via SQL syntax, for demo purpose.
Astronomers’ tools
5 years ago, on December 19, 2013, the ESA launched an orbital telescope called Gaia. Learn more about the Gaia mission on the official website of the European Space Agency or in the article by Vitaly Egorov (Billion pixels for a billion stars).
However, few people know what technology the agency chose for storing and processing the data collected by Gaia. Two years before the launch, in 2011, the developers were considering a number of candidates (see “Astrostatistics and Data Mining” by Luis Manuel Sarro, Laurent Eyer, William O’Mullane, Joris De Ridder, pp. 111-112):
- IBM DB2,
- PostgreSQL,
- Apache Hadoop,
- Apache Cassandra and
- InterSystems Caché (to be more precise, the Caché eXTreme Event Persistence technology).
Comparing the technologies side-by-side produced the following results (source):
| Technology | Time |
|---|---|
| DB2 | 13min55s |
| PostgreSQL 8 | 14min50s |
| PostgreSQL 9 | 6min50s |
| Hadoop | 3min37s |
| Cassandra | 3min37s |
| Caché | 2min25s |
The first four will probably sound familiar even to schoolchildren. But what is Caché XEP?
Last reply 9 January 2019
Best Practices, Caché, Terminal, Business Service, Java, JDBC, ODBC, SQL, InterSystems Data Platform Blog
As we all know, Caché is a great database that accomplishes lots of tasks within itself. However, what do you do when you need to access an external database? One way is to use the Caché SQL Gateway via JDBC. In this article, my goal is to answer the following questions to help you familiarize yourself with the technology and debug some common problems.
Last reply 28 December 2018
This is a sample Ensemble/Health Connect production which demonstrates how to receive an HL7 order (ORM) inbound from a file, extract fields (in this case, basic demographic information), and insert those into a table in an external SQL database via ODBC.
Included in the zip file:
- Exported code
- Sample ORM message
- 'How to configure' doc
Last reply 19 April 2018
Apache Spark has rapidly become one of the most exciting technologies for big data analytics and machine learning. Spark is a general data processing engine created for use in clustered computing environments. Its heart is the Resilient Distributed Dataset (RDD) which represents a distributed, fault tolerant, collection of data that can be operated on in parallel across the nodes of a cluster. Spark is implemented using a combination of Java and Scala and so comes as a library that can run on any JVM.
I've asked a lot of questions leading up to this, so I wanted to share some of my progress.
The blue line represents the number of messages processed. The background color represents the average response time. You can see ticks for each hour (and bigger ticks for each day). Hovering over any point in the graph will show you the numbers for that period in time.
This is super useful for "at a glance" performance monitoring as well as establishing patterns in our utilization.
Last reply 4 November 2016