New post

查找

Article
· Mar 24, 2024 3m read

What more can be done with lists in SQL (%DLIST, %INLIST, FOR SOME)

What I find really useful about IRIS when teaching my subject of Postrelational databases is the fact that it is a multi model database. Which means that I can actually go into architecture and structure and all that only once but then show the usage of different models (like object, document, hierarchy) using the same language and approach. And it is not a huge leap to go from an object oriented programming language (like C#, Java etc) to an object oriented database.

However, along with advantages (which are many) come some drawbacks when we switch from object oriented model to relational. When I say that you can get access to the same data using different models I need to also explain how it is possible to work with lists and arrays from object model in relational table. With arrays it is very simple - by default they are represented as separate tables and that's the end of it. With lists - it's harder because by default it's a string. But one still wants to do something about it without damaging the structure and making this list unreadable in the object model.

So in this article I will showcase a couple of predicates and a function that are useful when working with lists, and not just as fields.

Let's say that we have a class Restaurant.Dish which will have all our dishes:

Class Restaurant.Dish Extends %Persistent
{
Property Name As %String;
Property Description As %String(MAXLEN = 1000);
Property Category As %String;
Property Price As %Float;
Property Currency As %String;
Property Calories As %Integer;
}

Which we somehow populated with data:

And there is a class Restaurant.Account which contains the favourite dishes of the patrons:

Class Restaurant.Account Extends %Persistent
{
Property Name As %Name;
Property FavouriteFood As list Of %String;
}

And it also contains data:

First, let's look at the aggregate function %DLIST. It returns an ObjectScript %List structure containing the values in the specified column as list elements. In general, the syntax is as follows:

%DLIST([ALL | DISTINCT [BY(col-list)]] 
  string-expr 
  [%FOREACH(col-list)] [%AFTERHAVING])

Now, let's say we need to group all the dishes by categories and get a list of all dishes in those categories:

select Category,
       %DLIST(Name) AS AllDishes,
       %DLIST(Distinct (Name)) AS AllDistinctDishes
  from Restaurant.Dish
GROUP BY Category
ORDER BY Category

Another one is a predicate condition %INLIST. It matches a value to the elements in a %List structured list. It resembles the predicate IN but expects to get a $LB as an argument, instead of a values, separated by comas. Its syntax is as follows:

scalar-expression %INLIST list [SIZE ((nn))]

For example, now we want to see what dishes out of favourites of our patrons we have on a menu:

select Name, Description, Price
  from Restaurant.Dish
WHERE name %INLIST (select FavouriteFood 
                      from Restaurant.Account 
                     where ID = 1)  SIZE ((10))

And the last predicate condition I want to highlight here is FOR SOME %ELEMENT. It matches the list elements in field with the specified predicate. The SOME keyword specifies that at least one of the elements in the field must satisfy the specified predicate clause. The predicate clause must contain either the %VALUE or the %KEY keyword, followed by a predicate condition. These keywords are not case-sensitive. The syntax is as follows:

FOR SOME %ELEMENT(field) [[AS] e-alias] (predicate)

As for the last example, let's say we got a deal for Sprite and Diet Coke and we wish to see if it will have traction with our patrons. 

select *
  from Restaurant.Account 
 where FOR SOME %ELEMENT(FavouriteFood) f
     (f.%VALUE IN ('Sprite','Diet Coke') and %KEY IS NOT NULL)

So this is just to attract your attention to these 3 possibilities of working with lists in SQL. For further details, please visit the documentation pages I referenced in the article. And of course, you can always use built-in functions when working with $lb in SQL.

2 Comments
Discussion (2)1
Log in or sign up to continue
Question
· Mar 23, 2024

FHIR Data not return content

Hi,

I found an issue while fetching records from FHIR DB, I am getting below error thou FHIR repository have the records with the corresponding id

{

    "resourceType": "OperationOutcome",

    "issue": [

        {

            "severity": "error",

            "code": "not-found",

            "diagnostics": "<HSFHIRErr>ResourceNotFound",

            "details": {

                "text": "No resource with type 'Appointment' and id '21'"

            }

        }

    ]

}

5 Comments
Discussion (5)2
Log in or sign up to continue
Article
· Mar 21, 2024 2m read

VECTOR inside IRIS

This is an attempt to run a vector search demo completely in IRIS
There are no external tools and all you need is a Terminal / Console and the management portal.
Special thanks to Alvin Ryanputra as his package iris-vector-search that was the base
of inspiration and the source for test data.
My package is based on IRIS 2024.1 release and requires attention to your processor capabilities.

I attempted to write the demo in pure ObjectScript.
Only the calculation of the description_vectoris done in embedded Python

Calculation of a vector with 384 dimensions over 2247 records takes time.
In my Docker container, it was running 01:53:14 to generate them completely.

You have been warned!
So I adjusted this step to be reentrant to allow pausing vector calculation.
Every 50 records you get an offer to have a stop. 

The demo looks like this:

USER>do ^A.DemoV

      Test Vector Search
=============================
     1 - Initialize Tables
     2 - Generate Data
     3 - VECTOR_COSINE
     4 - VECTOR_DOT_PRODUCT
     5 - Create Scotch
     6 - Load Scotch.csv
     7 - generate VECTORs
     8 - VECTOR search
Select Function or * to exit : 8

      Default search:
Let's look for TOP 3 scotch that costs less than $100,
 and has an earthy and creamy taste
     change price limit [100]: 50
     change phrase [earthy and creamy taste]: earthy 

 calculating search vector
  
     Total below $50: 222 

ID      price   name
1990    40      Wemyss Vintage Malts 'The Peat Chimney,' 8 year old, 40%
1785    39      The Famous Jubilee, 40%
1868    40      Tomatin, 15 year old, 43%
2038    45      Glen Grant, 10 year old, 43%
1733    29      Isle of Skye, 8 year old, 43% 5 Rows(s) Affected


- You see the basic functionalities of Vectors in steps 1..4
- Steps 5..8 are related to the search example I borrowed from Alvin
- Step 6 (import of test data) is straight ObjectScript
  SQL LOAD DATA was far too sensible for irregularities in the input CSV

I suggest following the examples also in MGMT portal to watch how Vectors operate.

GitHub
 

11 Comments
Discussion (11)5
Log in or sign up to continue
Article
· Mar 18, 2024 6m read

コンソールログに"There exists a MISMATCH.WIJ file" が記録され、インスタンスの開始ができない時の対処法

これは InterSystems FAQ サイトの記事です。
 

インスタンスの開始が失敗し、コンソールログに"There exists a MISMATCH.WIJ file"が記録されている場合、何らかのシステム障害の影響でデータベースの整合性に関して問題が生じていることを示しています。

このような状況が発生した際にインスタンスの開始ができるようにするためには、以下の手順を実施します。

(1) a. インスタンスをNOSTUモードで起動       注1:
(2) b. データベースの整合性チェック

◆(2)の整合性チェックでエラーを検出しなかった場合、
 (3) d. MISMATCH.WIJ ファイルのリネーム
 (4) e. インスタンスの再起動
を実施します。

◆(2)の整合性チェックでエラーが検出された場合は、
 (3) c. MISMATCH.WIJファイルの適用
 (4) b. データベースの整合性チェック
 (5) d. MISMATCH.WIJファイルのリネーム
 (6) e. インスタンスの再起動
を実施します。 

以下に各手順の詳細を説明します。 

a. インスタンスをNOSTUモードで起動します。

以下に記載の手順の内、1)および2)の手順まで実行します。

3)以降は実施する必要はありません。

(Cachéの場合は、右括弧なしの1,2,3)

NOSTUで開始する(Caché)

NOSTUで開始する(IRIS)

b. データベースの整合性チェック

STURECOV ルーチンを実行してデータベースの整合性チェックを行います。

%SYS>do ^STURECOV
Logins are not disabled.
This routine is designed to run when Cache' is in single user mode due to a problem running the STU startup routine.
Do you want to continue ? No => yes
Warning: Misuse of this utility can harm your system
There is no record of any errors during the prior startup
This could be because there was a problem writing the data to disk or because the system failed to start for some otherreason.
Do you want to continue ? No => yes
Enter error type (? for list) [^] => MISMATCHWIJ

1) List Affected Databases and View Blocks
2) Apply mismatched blocks from WIJ to databases 
3) Rename MISMATCH.WIJ         
4) Dismount a database
5) Mount a database
6) Database Repair Utility
7) Check Database Integrity
8) Bring down the system prior to a normal startup
--------------------------------------------------------------
H) Display Help
E) Exit this utility
--------------------------------------------------------------
Enter choice (1-8) or [Q]uit/[H]elp? 7
This utility is used to check the integrity of a database and the pointer structure of one or more globals.
Output results on
Device: Integrity_STURECOV.log        整合性チェックの結果をログファイルに出力します
Parameters? "WNS" =>
Stop after any error?  No=>
Do you want to check all databases?  No=> Yes
Checking c:\intersystems\irishealth\mgr\ at 23:09:20
Checking c:\intersystems\irishealth\mgr\enslib\ at 23:09:21
Checking c:\intersystems\irishealth\mgr\hscustom\ at 23:09:22
Checking c:\intersystems\irishealth\mgr\hslib\ at 23:09:23
Checking c:\intersystems\irishealth\mgr\hssys\ at 23:09:31
Checking c:\intersystems\irishealth\mgr\irisaudit\ at 23:09:31
Checking c:\intersystems\irishealth\mgr\irislib\ at 23:09:31
Checking c:\intersystems\irishealth\mgr\irislocaldata\ at 23:09:33
Checking c:\intersystems\irishealth\mgr\iristemp\ at 23:09:33
Checking c:\intersystems\irishealth\mgr\test\ at 23:09:33
Checking c:\intersystems\irishealth\mgr\user\ at 23:09:34
--------------------------------------------------------------
1) List Affected Databases and View Blocks
2) Apply mismatched blocks from WIJ to databases
3) Rename MISMATCH.WIJ
4) Dismount a database
5) Mount a database
6) Database Repair Utility
7) Check Database Integrity
8) Bring down the system prior to a normal startup
--------------------------------------------------------------
H) Display Help
E) Exit this utility
--------------------------------------------------------------
Enter choice (1-8) or [Q]uit/[H]elp?

 

出力されたログファイルの末尾に "No Errors were found." と記録されていれば整合性チェックでエラーは検出していません。

 

c. MISMATCH.WIJファイルの適用

STURECOV ルーチンよりMISMATCH.WIJファイルを適用します。

--------------------------------------------------------------
1) List Affected Databases and View Blocks
2) Apply mismatched blocks from WIJ to databases 
3) Rename MISMATCH.WIJ         
4) Dismount a database
5) Mount a database
6) Database Repair Utility
7) Check Database Integrity
8) Bring down the system prior to a normal startup
--------------------------------------------------------------
H) Display Help
E) Exit this utility
--------------------------------------------------------------
Enter choice (1-8) or [Q]uit/[H]elp? 2

d. MISMATCH.WIJ ファイルのリネーム
 

STURECOV ルーチンよりMISMATCH.WIJファイルをリネームします。

 

--------------------------------------------------------------
1) List Affected Databases and View Blocks
2) Apply mismatched blocks from WIJ to databases 
3) Rename MISMATCH.WIJ         
4) Dismount a database
5) Mount a database
6) Database Repair Utility
7) Check Database Integrity
8) Bring down the system prior to a normal startup
--------------------------------------------------------------
H) Display Help
E) Exit this utility
--------------------------------------------------------------
Enter choice (1-8) or [Q]uit/[H]elp? 3

e. インスタンスの再起動

STURECOV ルーチンよりインスタンスを停止後、インスタンスをランチャー等より通常起動します。

--------------------------------------------------------------
1) List Affected Databases and View Blocks
2) Apply mismatched blocks from WIJ to databases
3) Rename MISMATCH.WIJ
4) Dismount a database
5) Mount a database
6) Database Repair Utility
7) Check Database Integrity
8) Bring down the system prior to a normal startup
--------------------------------------------------------------
H) Display Help
E) Exit this utility
--------------------------------------------------------------
Enter choice (1-8) or [Q]uit/[H]elp? 8

MISMATCH.WIJの適用処理が正常終了しない、またはインスタンスの再開始後の整合性チェックでエラーがある場合は、カスタマーサポートセンターまでお問い合わせください 

WIJ ブロック比較
 

注1:

以下のようにシングルユーザーモードで開始されている旨のメッセージが含まれている場合は、この手順は必要ありません。

Cache is started in single user mode.
To log into Cache, type:
    ccontrol session CACHE -B
and D ^STURECOV for help recovering from this error.

以下のように直接シングルユーザーモードでログインし、b.から始めます。

(これらのコマンドのPATHが設定されているか、コマンドの実行イメージの存在する場所で実行する必要があります。)

Windows
 IRIS
 >irisdb -s ..\mgr -B
 Caché
 >cache -s..\mgr -B  
UNIX
 IRIS
 >iris session iris -B
 Caché
 >ccontrol session cache -B
Discussion (0)1
Log in or sign up to continue
Article
· Mar 18, 2024 2m read

Getting data from InterSystems IRIS CloudSQL using xDBC

Recently, the question came up while discussing the access to the data stored in IRIS from different languages with my students if it was possible to initiate the connection and get data from Cloud solution (InterSystems IRIS CloudSQL) from Microsoft Excel, not the other way around. Considering the many varied ways one can get data in Excel (import data from external sources, connecting to databases using ODBC drivers, using power queries and web queries etc.) the obvious choice was to try ODBC driver. The only task left was to try to connect to the database in the cloud using the ODBC driver.

Quite unsurprisingly, it worked!

If you're new to CloudSQL, I would highly suggest you read the documentation. From it you will be able to get access to the InterSystems Cloud Services Portal (or you can go directly to AWS and subscribe with your AWS account) and download necessary drivers. BTW, if you encounter problems with CloudSQL, you may try to look for answers in this document first.

Now that these preliminaries are out of the way, let's try the simplest approach - setting up the ODBC data source with just a password and getting data from it in MS Excel.

Step 0. Set up your InterSystems IRIS CloudSQL. You will see the necessary settings on the Overview page of your Deployment:

Step 00. Download and install ODBC driver for your OS.

Step 1. Open your ODBC Data Sources and switch to System DSN tab. In it, click on Add...

and fill in the settings from your Overview page (first screenshot of the article).

Step 2. Connect Excel to IRIS. Open Excel, navigate to the Data tab, and select "Get Data" or "From Other Sources," depending on your Excel version. Choose "From ODBC" as the data source

select the ODBC data source you configured earlier

enter authentication credentials if asked

and choose the table you wish to load into Excel

Step 3. You're good to go and do whatever with your data

PS. Some other interesting articles/videos that I would suggest you read/watch regarding ODBC and CloudSQL:

  • a whole series of videos that show how to connect to CloudSQL from solutions written in different languages
  • an interesting article on how to set up roles and give permissions to xDBC connections
  • another article on how to use SSL/TLS to establish secure connections over JDBC and other driver technologies
  • and the last one about switching between ODBC drivers in an environment with multiple versions installed
4 Comments
Discussion (4)2
Log in or sign up to continue