Question
Megumi Kakechi · Apr 21

Insert IRIS globals including Japanese into pandas dataframe

I tried to convert IRIS globals to pandas  dataframe.

I can do it as follows if there are no Japanese included in globals,

USER>zw ^ISJ2
^ISJ2=4
^ISJ2(1)=$lb("Name","Age","Address")
^ISJ2(2)=$lb("Sato","50","Tokyo")
^ISJ2(3)=$lb("Kato","40","Osaka")
^ISJ2(4)=$lb("Ito","30","Kyoto")
 
USER>do $system.Python.Shell()
 
Python 3.9.5 (default, Jan 31 2022, 17:55:36) [MSC v.1927 64 bit (AMD64)] on win32
Type quit() or Ctrl-D to exit this shell.
>>> mysql = "select name,value from %library.global_get('user','^ISJ2',,2,2)"
>>> resultset = iris.sql.exec(mysql)
>>> dataframe = resultset.dataframe()
>>> print (dataframe)
       name                        value
0     ^ISJ2                            4
1  ^ISJ2(1)  $lb("Name","Age","Address")
2  ^ISJ2(2)     $lb("Sato","50","Tokyo")
3  ^ISJ2(3)     $lb("Kato","40","Osaka")
4  ^ISJ2(4)      $lb("Ito","30","Kyoto")
>>>


However,
I got the following error if the globals include Japanese.  

Error #5002: Cache Error:%DispatchClassMethod+2^%SYSTEM.Python.1^1^ *<clas      s 'UnicodeDecodeError'>: 'utf-8' codec can't decode byte 0x8d in position 5: invalid start byte - BuildPythonElement failed

It seems that I need to set encoding. How can I set encoding and avoid the error?

USER>zw ^ISJ
^ISJ=4
^ISJ(1)=$lb("Name","Age","Address")
^ISJ(2)=$lb("佐藤","50","東京")
^ISJ(3)=$lb("加藤","40","大阪")
^ISJ(4)=$lb("伊藤","30","京都")
 
USER>do $system.Python.Shell()
 
Python 3.9.5 (default, Jan 31 2022, 17:55:36) [MSC v.1927 64 bit (AMD64)] on win32
Type quit() or Ctrl-D to exit this shell.
>>> mysql = "select name,value from %library.global_get('user','^ISJ',,2,2)"
>>> resultset = iris.sql.exec(mysql)
>>> dataframe = resultset.dataframe()
Traceback (most recent call last):
  File "<input>", line 1, in <module>
RuntimeError: エラー #2802: %ML cannot create dataframe.
  > エラー #5002: Cache エラー:<SYNTAX>%DispatchClassMethod+2^%SYSTEM.Python.1^1      ^ *<class 'UnicodeDecodeError'>: 'utf-8' codec can't decode byte 0x8d in position 5: invalid start byte - BuildPythonElement failed
>>>
Product version: IRIS 2021.2
$ZV: IRIS for Windows (x86-64) 2021.2 (Build 651U) Mon Jan 31 2022 17:39:05 EST
0
0 149
Discussion (2)0
Log in or sign up to continue

@Megumi Kakechi 
I can get correct result on 2022.1! laugh

I use iris 2022.1 container. (image lists :https://community.intersystems.com/node/511986)

USER>write $ZV
IRIS for UNIX (Ubuntu Server LTS for x86-64 Containers) 2022.1 (Build 164U) Tue Apr 5 2022 01:30:47 EDT

USER>do $system.Python.Shell()

Python 3.8.10 (default, Nov 26 2021, 20:14:08)
[GCC 9.3.0] on linux
Type quit() or Ctrl-D to exit this shell.
>>> sql="select name,value from %Library.Global_get('user','^ISJ',,2,2)"
>>> rset=iris.sql.exec(sql)
>>> dataframe=rset.dataframe()
>>> dataframe
      name                value
0  ^ISJ(1)  $lb("名前","年齢","住所")
1  ^ISJ(2)  $lb("佐藤","50","東京")
2  ^ISJ(3)  $lb("加藤","40","大阪")
3  ^ISJ(4)  $lb("伊藤","30","京都")