Robert Cemper · Jul 8, 2018 go to post

OK. You found 801 entries  with ~200 k global  access.
and the query works as expected

So 

SELECT COUNT(*) 
FROM SQ.CBPhoneResult_View
WHERE PhoneDateODBC = '2018-04-10'

for Tuesday Apr.10 it should return 801  

What is the count for Monday  2018-04-09 ?  I 'd expect more

SELECT
 COUNT(*)
FROM SQ.CBPhoneResult_View
WHERE PhoneDateODBC = '2018-04-09' 
Robert Cemper · Jul 8, 2018 go to post

As you see the query plan is significantly shorter.
Your original refers to 3 other tables.
To find out why and how to improve would require all table/class  & view definitions

  • SQ.CBPhoneResult_View
  • SQ.CBPH_Phone
  • SQ. CBPH_Result      your 2nd query ends here  
  • SQ. CB_Test      access to  CallbackComment starts here
  • SQ. CB_Order
  • SQ. CB_Contact

Without the related class definition it is hard to say how these tables link to each other

But you really should run it from the terminal prompt to see your result at least once:

Do you have developer access to your Caché  at all? 
Can you see the class definitions in Studio ? 

Robert Cemper · Jul 8, 2018 go to post

You may also reformulate your SELECT to read directly from the tables used
starting with the one holding  PhoneDateODBC and forget all the other burden.

Robert Cemper · Jul 8, 2018 go to post

OK,now your timeout is clear.
with a Relative Cost of  ~2 millions, your query requires some support to speed up.
You just see the first 6 empty results.

First reason: you run over ALL records in  ...PhoneMaster
with an inner loop in  ...Testmaster on IndexCall,OrdeCode, TestCode,...Result_Index

So your timeout is not surprising.

I'd suggest creating an Index on PhoneDateODBC to speed up your query. >>  lower  Relative Cost
with 2 million I'm quite sure that you even timeout over ODBC.

So I'm back to my earlier suggestion:
Let your query run from terminal prompt,
have a coffee or two and maybe it is completed then.

This is not your fault.
Blame the designer of that ugly VIEW

Robert Cemper · Jul 8, 2018 go to post

OK.
You use a Caché version before 2015.1 that doesn't know $TRANSLATE

It's then 

SELECT
 REPLACE(CallbackComment  ,'''','?')
FROM SQ.CBPhoneResult_View
WHERE PhoneDateODBC = '2018-04-09' 

I expect you will run into a timeout again.
So please click to "Show Query Plan" and let us see what's going on.

Additionally, on left border click to Views;
select  SQ.CBPhoneResult_View and get an Image similar to this

important part:  VIEW TEXT

Robert Cemper · Jul 8, 2018 go to post

OK. now the picture gets clear.

 the key problem looks as if a single ' is interpreted as a String delimiter.
so instead of naked CallbackComment 
You may try

$TRANSLATE(CallbackComment  ,'''','?')

(it's really 4single quotes, no typo)
So you replace the single quote by a non-conflicting character BEFORE the string is passed to ODBC   
This would prove that the single quote is the cause of trouble.

Robert Cemper · Jul 8, 2018 go to post

what product/component are you interested in?

  • IRIS
  • ENSEMBLE
  • HEALTHSHARE
  • CACHÉ
  • TRAK
  • iKNOW
  • DEEPSEE
  • ..
  • ...

and others is as wide as the whole software industry

please be more specific

Robert Cemper · Jul 7, 2018 go to post

in addition, it would be worth to see what your SQ.CBPhoneResult_View does.

By the ending VIEW I assume it is a View and not a Table:
So the complexity of the executed query is hidden and could be the real performance problem. 

Robert Cemper · Jul 7, 2018 go to post
  • referring to CSP.... you  let me assume you are working from Mgmt Portal
  • next you say:  I am using the Intersystem ODBC driver

This is a contradiction  as CSP doesn't use ODBC driver
So what are you really assuming to do ??

Anyhow in both cases a SQL String delimiter (') entered in a %String property aka VARCHAR is always presented as it was entered.
I verified it over an external ODBC viewer +  Intersystem ODBC driver as well as over JDBC:  no issue.

For ODBC on Windows just use 64bit Version on 64bit platforms.

in addition, you may verify your query also from the terminal prompt without any eventual timeout:

USER>do $system.SQL.Shell()
SQL Command Line Shell
----------------------------------------------------
 
The command prefix is currently set to: <<nothing>>.
Enter q to quit, ? for help.
USER>>  << entering multiline statement mode >>
        1>>SELECT
        2>>CallbackComment
        3>>FROM SQ.CBPhoneResult_View Where PhoneDateODBC = '2018-04-09'
        4>>go

 
Robert Cemper · Jul 7, 2018 go to post

if you define a method   mymethod() as something

You are expected to terminate ist either by QUIT anyvalue  or  RETURN ayvalue

in your example you announce a %Status. so Quit $$$OK would be fine like this

ClassMethod getFile() as %Status
{
    set filename = "/home/test.json"
    set newArray={}.%FromJSON(filename)
    write "New entity:"_newArray.%ToJSON()
    quit $$$OK
    }

if you ommit  as %Status you can allso forget the ending QUIT

Robert Cemper · Jul 6, 2018 go to post

You are wrong:   getFile+8 is 

    set i2 = resultSet.rowSet.%GetIterator()


so either resultSet. or resultSet.rowSet is not what you expect it to be.

first you do 

set sc = stream.LinkToFile(filename)

 but you don't check the success code

And with "path/to/file.json" the file name looks more than suspicious to be correct.

next you do set obj=...   but this obj isn't used at all.

next 

set jsonObj = [].%FromJSON(filename)

and there is the fundamental mistake as your variable filename is

set filename = "path/to/file.json"


which is anything else than the expected JSON array

 you may have lost some important lines during cut/paste from your example

 Deserializing from JSON to a dynamic object may have important information for you

Robert Cemper · Jul 5, 2018 go to post

It means in row 8 of method getFile you try to use some object reference that doesn't exist yet or is not initialized yet.
posting some of your code could enable more precise diagnostic

Robert Cemper · Jul 2, 2018 go to post

Hi Scott,

It's indeed surprising.
But digging into docs tells me>>>> it's not an ERROR code but a RESULT code and 0 = Success. (like SQL)

more LDAP Result Codes 
Err2String is defintely a misleading naming.
 

Robert Cemper · Jun 30, 2018 go to post

It could be sufficient to  set ID=""
at the beginning of your code to avoid the <UNDEFINED> later down.
 

Robert Cemper · Jun 29, 2018 go to post

I try a simple explanation and you should try to find some docs on HTTP:
So I leave out several steps in between to illustrate the basic actions.

your browser sends a request

GET /index.html HTTP/1.1
Host: www.example.com

and if it is successful the server replies

HTTP/1.1 200 OK
Date: Mon, 23 May 2005 22:38:34 GMT
Content-Type: text/html; charset=UTF-8
Content-Length: 138
Last-Modified: Wed, 08 Jan 2003 23:11:55 GMT
Server: Apache/1.3.3.7 (Unix) (Red-Hat/Linux)
ETag: "3f80f-1b6-3e1cb03b"
Accept-Ranges: bytes
Connection: close

<html>
<head>
  <title>An Example Page</title>
</head>
<body>
  Hello World, this is a very simple HTML document.
</body>
</html>


The highlighted part between HTTP and your first <html> is prepared or modified  if not default in OnPreHTTP().
It has to be ready BEFORE sending the reply. 

The delimiter between HTTP part and the content transmitted is just an empty line.
The closing is 2 empty lines.

For the items you can influence see class %CSP.Response

Robert Cemper · Jun 29, 2018 go to post

Analysing your code I  found @ line 24

  $$$ISWINDOWS,$$$UseSecureConnection{

ending @ line 42   that contains  LD=##Class(%SYS.LDAP).Init($$$LDAPServer)

but i didn't see an ELSE if the first IF fails.
Then LD is undefined.  

I'm a little bit surprised how it could go up to line 104

Robert Cemper · Jun 29, 2018 go to post

you do .ReadLineIntoStream()  
doc says: https://docs.intersystems.com/latest/csp/documatic/%25CSP.Documatic.cls

This reads from the stream until it find the LineTerminator and returns this as a stream. If the stream does not contain the line terminator this can potentially be the entire stream.

and your error message:  Premature end of data 12 Line 1 Offset 1     

indicates that you have hit some character interpreted as line terminator after 12 characters. (rather short for  JSON)
My guess: your JSON input is a multiline input with enough line terminators inside

As a consequence your JSON input is incomplete.

Suggested workaround

  • get length of your stream method SizeGet()
  • read full stream ignoring line terminators using method Read(ByRef len As %Integer, ByRef sc As %Status) as %CacheString

eventually, it might be necessary to remove the line terminators before  %FromJSON

Robert Cemper · Jun 28, 2018 go to post

with large numbers exceeding 64bit integers the logic with integer division \ and modulo #
was causing wrong results. So I changed it to pure string interpretation.

Recommendation:
pass all numbers as strings  to escape from numeric normalization

NOT  write $$^zahl(102100900002103201200301.6123100)
einhundertzwei Trilliarden einhundert Trillionen neunhundert Billiarden zwei Billionen einhundertdrei Milliarden zweihundertein Millionen zweihundert Tausend

BUT  write $$^zahl("102100900002103201200301.6123100")
einhundertzwei Trilliarden einhundert Trillionen neunhundert Billiarden zwei Billionen einhundertdrei Milliarden zweihundertein Millionen zweihundert Tausend dreihunderteins Komma sechs eins zwei drei eins null null

If you feel think this is exaggerated think about banking calculations for countries within low rated currencies.

Robert Cemper · Jun 28, 2018 go to post

You are right. 
Though due to the internal limits, next time I would avoid \ and # operations in favor of $E() for the next version
As there are some strange effects in handling numerics due to normalization

write $$^zahl(1.3400) >>>> eins Komma drei vier          
write $$^zahl("1.3400") >> 
eins Komma drei vier null null
Robert Cemper · Jun 28, 2018 go to post

Hola Francisco,
You motivated me to do something similar for German.
It's is straightforward .int routine and you are welcome to add the code to your project.
GermanNumberToText 
I did it up to 10e21, negatives and unlimited decimals.  (except what is cut down due to internal limits)
I tried to catch all the irregular structures of the language like singular/plural, varying genders,  upper/lower case
and tried to keep the output readable:

 w $$^zahl(-190000103201101.3903)
Minus 
einhundertneunzig Billionen einhundertdrei Millionen zweihundertein Tausend einhunderteins Komma drei neun null drei 

For quick copy:
Updated to avoid failover  from integer to floating format  for large numbers (2018-06-28 16:34 UTC)

zahl(num="",gen="") Public {
 ;;; convert number as German text
 ;;; w $$^zahl(-1123.505) >>>> Minus ein Tausend einhundertdreiundzwanzig Komma fünf null fünf
 set dec=$p(num,".",2),dec=$s(dec?1.N:$$dec(dec),1:"")
 if num=0 quit "null"_dec
 set neg=$S(num<0:"Minus ",1:"")

  if $l(neg) set num=$tr(num,"-")   
 if num=1 set gen=$zcvt(gen,"U") quit neg_"ein"_$case(gen,"W":"e","S":"es","M":"",:"s")
 if num<10e23 quit neg_$$trd($p(num,"."))_dec
 quit "*** Zahl zu groß ***"
}
 ;
dec(num) {
 set dec=" Komma"
 for p=1:1:$l(num) set dec=dec_" "_$$zig($e(num,p))
 quit dec
}
zig(num) {
 if num<10 quit $li($lb("null","eins","zwei","drei","vier","fünf","sechs","sieben","acht","neun"),num+1)
 if num<20 quit $li($lb("zehn","elf","zwölf","dreizehn","vierzehn","fünfzehn","sechzehn","siebzehn","achtzehn","neunzehn"),num-9)
 set zig=$e(num,*-1),zn=$e(num,*)
 set res=$s(zig=3:"dreißig"
           ,1:$li($lb(,"zwan",,"vier","fünf","sech","sieb","acht","neun"),zig)_"zig")
 if zn set res=$s(zn=1:"ein",1:$$zig(zn))_"und"_res
 quit res 
}
hun(num) {
 set hun=$e(num,*-2),zig=$e(num,*-1,*),res="",m="hundert"
 set res=$s(hun=1:"ein"_m
           ,hun>1:$$zig(hun)_m
           ,1:"" )
 quit $replace(res_$$zig(zig),"null","")
}
ein(res) {
 if $e(res,*-3,*)="eins" set res=$e(res,1,*-1)
 quit $replace(res,"null","")

tsd(num) ;1,000 10e3
 set tsd=$e(num,*-5,*-3),hun=$e(num,*-2,*),res=""
 if tsd set res=$$ein($$hun(tsd))_" Tausend "
 quit res_$$hun(hun)
}
mio(num) ;1,000,000 10e6
 set mio=$e(num,*-8,*-6),tsd=$e(num,*-5,*),m=" Million"
 set res=$s(mio=1:"eine"_m_" "
           ,mio>1:$$ein($$hun(mio))_m_"en "
           ,1:"")
 quit res_$$tsd(tsd)
}
mrd(num) ;1,000,000,000 10e9
 set mrd=$e(num,*-11,*-9),mio=$e(num,*-8,*),m=" Milliarde"
 set res=$s(mrd=1:"eine"_m_" "
           ,mrd>1:$$ein($$hun(mrd))_m_"n "
           ,1:"" )
 quit res_$$mio(mio)
}
bio(num) ;1,000,000,000,000 10e12
 set bio=$e(num,*-14,*-12),mrd=$e(num,*-11,*),m=" Billion"
 set res=$s(bio=1:"eine"_m_" "
           ,bio>1:$$ein($$hun(bio))_m_"en "
           ,1:"" )
 quit res_$$mrd(mrd)
}
brd(num) ;1,000,000,000,000,000 10e15
 set brd=$e(num,*-17,*-15),bio=$e(num,*-14,*),res="",m=" Billiarde"
 set res=$s(brd=1:"eine"_m_" "
           ,brd>1:$$ein($$hun(brd))_m_"n"_" "
           ,1:"" )
 quit res_$$bio(bio)
}
tri(num) {;1,000,000,000,000,000,000 10e18
 set tri=$e(num,*-20,*-18),brd=$e(num,*-17,*),m=" Trillion"
 set res=$s(tri=1:"eine"_m_" "
           ,tri>1:$$ein($$hun(tri))_m_"en"_" "
           ,1:"" )
 quit res_$$brd(brd)
}   
trd(num) ;1,000,000,000,000,000,000,000 10e21
 set trd=$e(num,*-23,*-21),tri=$e(num,*-20,*),m=" Trilliarde"
 set res=$s(trd=1:"eine"_m_" "
           ,trd>1:$$ein($$hun(trd))_m_"n"_" "
           ,1:"" )
 quit res_$$tri(tri)
}
Robert Cemper · Jun 28, 2018 go to post

Before dealing with adapters you should check if your UDP reaches your server at all.
You always get it if your server is on the same LAN segment as the sender.
But you depend on the setup of the router in the case that it is not in your LAN "neighborhood".

2nd your firewall should be willing to let pass this UDP.

Then you may check the traffic with some external tool to make it visible.

Or you check it directly from terminal following the instructions in IO device guide
chapter UDP Client/Server Communication

UDP is supported through the %Net.UDP class. This class provides methods
to Send() a packet to a specified destination and port,
to Recv() a packet from the socket,
and to Reply() to the transmitter of the last received packet.
Robert Cemper · Jun 27, 2018 go to post

you have the option to read by character  using

READ *var

Then you read exactly 1 byte in its  decimal representation 

set file="C:\ZZ\myfile.bin"
open file:"RS":0  else  write "file not open", ! quit
for  use file read *byte use 0 write byte,?5,"x\"_$ZHEX(byte),?12,"char:",$C(byte),!

Docs  on READ

You may also use %File Class and use a Read method with length 1

Robert Cemper · Jun 26, 2018 go to post

Use $ZCVT() function for decoding,

set y="<html><body><h1>Greater = &gt; lower 0= &lt; Ampersand = &amp; OK</h1></body></html>"

write $zcvt(y,"I","HTML")
<html><body><h1>greater = > lower 0= < Ampersand = & OK</h1></body></html>

Attention all &..; have to be in lower case 

See doc $ZCONVERT()

Robert Cemper · Jun 25, 2018 go to post

2 possible solutions:

TEST3(STATUS=1) [STATUS,MSG] PUBLIC {
 "S MSG=$S(STATUS:""HELLO"", 1:""GOODBYE"")"
 !,MSG
}
TEST4(%STATUS=1) PUBLIC {
 "S %MSG=$S(%STATUS:""HELLO"", 1:""GOODBYE"")"
 !,%MSG 
}

see also my article Summary on Local Variable Scoping

Robert Cemper · Jun 22, 2018 go to post

Vitaliy,

That's excellent. I had the vague feeling that it should be somewhere, somehow, ... ass these are all supported collations.

Now I have the correct solution!

Many thanks !

yesyesyesyesyes

Robert Cemper · Jun 21, 2018 go to post

You can also create your own conversion function like this:

Class SQL.ext [ Abstract ]
{
ClassMethod fromUTC(ts As %TimeStamp) As %TimeStamp [ SqlName = fromUTC, SqlProc ]
{
 quit $ZDT($ZDTH($ZDTH(ts,3),-3),3,1,2)
}
}

Then you simply use

 SELECT SQL.fromUTC(CreationTime) FROM Table

to get the conversion​​​​​​​

Robert Cemper · Jun 21, 2018 go to post

You try to upload a file and JS Code a the same time during your SUBMIT action.
so your &js< ...> lands every 32000 characters inside your file.

&js<..> is just a hidden WRITE and allows javascript syntax checking inside the <.. >

example:

&js<alert('Unable to create user.');> 

is identic to 

Write "alert('Unable to create user.');",!

So your concept doesn't work that way
You would require a second independent JS routine in a browser to call for progress using CSP hyperevent.