All
Accepted answers
Fabian Haupt · Nov 9, 2016 go to post

Hi David, you can actually debug a csp page with the studio debugger. However, this requires a bit more preparation, as you need to identify the csp server process running your page, which is not always straightforward.

An easier approach would be to use the command line debugger in combination with the CSP shell.

SAMPLES>ZBREAK zListFiles+1^csp.menu.1
 
SAMPLES>d $SYSTEM.CSP.Shell()
CSP Shell
 
Command shell for debugging CSP pages. It looks and acts like a Cache programmer prompt, but you can use the GET or HEAD command to fetch a CSP page. You can set breakpoints, step into the code etc. You may pass query parameters to the page as well, eg.:CSP>>> GET /csp/samples/request.csp?A=1&B=2
 
The output you see is what would be sent to the browser, including any HTTP headers. You can also interact with the session, request and response objects via the special variables %session, %request and %response.
 
CSP:SAMPLES>>> get /csp/samples/menu.csp
 200 OK
Content-Type: text/html; charset=utf-8
Set-Cookie: CSPSESSIONID-UP-csp-samples-=5864              5864; path=/csp/samples/;  httpOnly;
Cache-Control: no-cache
Date: Wed, 09 Nov 2016 17:49:45 GMT
Expires: Thu, 29 Oct 1998 17:04:19 GMT
Pragma: no-cache
 
<!-- Copyright (c) 2001 InterSystems Inc. ALL RIGHTS RESERVED. -->
 
<html>
<head>
<title>CSP Samples Menu</title>
<style type="text/css">
        body { color: black; background: #CCCCFF; font-size: 12pt; font-family: Verdana,Arial,Helvetica,sans-serif; }
        .Small { font-size: 10pt;}
        .DarkRow { background: #DDDDDD; }
        .LightRow { background: #FFFFFF; }
</style>
</head>
 
<body bgcolor="#CCCCFF">
 
<!-- display standard sample template using a custom tag -->
 
<table bgcolor="#000088" cols="2" width="100%" border="1" cellpadding="3">
<tr>
<td width="50%">
<nobr><font face="Verdana,Arial,Helvetica,sans-serif" color="white" size="+3">
<b>&nbsp;CSP Samples Menu</b>
</font></nobr>
</td>
</tr>
</table>
<br>
 
<table width="100%">
<tr>
<td width="66%"><font face="Verdana,Arial,Helvetica,sans-serif">
This page displays a list of available CSP pages within the same
directory.<br/>
Click this link to see a demonstration of the the <a href="ZENDemo.Home.cls">Zen Web Development Framework</a>.
 
</font></td>
<td align="right"><font face="Verdana,Arial,Helvetica,sans-serif"><a href="menu.csp">Samples Menu</a></font></td>
<td align="right"><font face="Verdana,Arial,Helvetica,sans-serif"><a href="showsource.csp?PAGE=/csp/samples/menu.csp">Source</a></font></td>
</tr>
</table>
 
 
   New file,rs,dir,menupath,subdir,list,page,description,url
   ^
<BREAK>zListFiles+2^csp.menu.1
SAMPLES 15d6>

In this example I am setting the breakpoint to

ZB zListFiles+1^csp.menu.1

and then request the menu page within the CSP shell:

SAMPLES>d $SYSTEM.CSP.Shell()
CSP Shell
 
Command shell for debugging CSP pages. It looks and acts like a Cache programmer prompt, but you can use the GET or HEAD command to fetch a CSP page. You can set breakpoints, step into the code etc. You may pass query parameters to the page as well, eg.:CSP>>> GET /csp/samples/request.csp?A=1&B=2
 
The output you see is what would be sent to the browser, including any HTTP headers. You can also interact with the session, request and response objects via the special variables %session, %request and %response.
 
CSP:SAMPLES>>> get /csp/samples/menu.csp

This drops us into the regular command line debugger.

I hope this helps. -Fab

Fabian Haupt · Nov 7, 2016 go to post

No. Any direct binary data in json would need to extend the base grammar, see: http://json.org/

That being said, there are initiatives to allow for that, like BSON. Caché doesn't have support for it as of this moment. So your base64 approach actually is very solid!

Fabian Haupt · Nov 4, 2016 go to post

Here is a script I've been using recently. It can easily be adapted to different instances on one machine. As Jose already pointed out, one should stick to one method of managing the instance state and not mix them.


[Unit]
Description=Intersystems Cache

[Service]
Type=forking
ExecStart=/usr/local/etc/cachesys/ccontrol start cache
ExecStop=/usr/local/etc/cachesys/ccontrol stop cache quietly
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target
Fabian Haupt · Nov 4, 2016 go to post

Hi Nikita,

especially on a 'development mode' installation you need to be careful with messing with UnknownUser. UnknownUser is being used for many different things, and setting a startup routine like this would for example lock you out of the Management Portal. This is because in the default installation the CSP Gateway is using the UnknownUser to log into the instance. So you would have to set it up to use CSPSystem instead.  George has already pointed out how to get the user into programmer mode after running any of your code. The more supported approach is what Tim told you.

Cheers,

Fab

Fabian Haupt · Nov 3, 2016 go to post

Hi Uri,

this comes down to the way computers are representing numbers. Since computers are based on a binary system, you have to approximate (some) numbers. This leads to things like .3 actually being .29999999999999998889 (in IEEE binary double precision floating point representation). This actually comes up from time to time, so I took the liberty to use some of the examples I gathered over the time: 

Various languages use different limits for rounding and or display of numbers and sometimes moving between languages leads to these artifacts coming up. One common example for that is moving from java's double to Cache's decimal types.

Cache's decimal representation has almost 19 digits of decimal precision and it has a range where its largest positive number is exactly 9.223372036854775807E+145 and its smallest non-zero positive number is exactly 1E-128.

Cache's binary representation, as well as Java's "double" type representation (just as an example, since you asked about other languages as well), uses the 64-bit double precision representation defined by IEEE Std 754-1985 (the IEEE Standard for Binary Floating-Point Arithmetic). This representation has a precision of 53 binary bits, which is almost 16 digits of decimal precision. It has a range where its largest positive number is approximately 1.797693134862315708E+308 and its smallest non-zero positive number is approximately 2.22507385850720138E-308. The set of decimal fractions that can be stored exactly in binary floating- point representation (i.e., without using an approximation) is very limited. The set includes 0.5, 0.25, 0.75, 0.125, 0.375, 0.625, 0.875. The list I just wrote down shows all the 1 digit, 2 digit and 3 digit decimal fractions that are represented exactly in binary. The remaining three- digit decimal fractions (992 decimal fractions out of a total of 999 fractions) must be approximated.

So looking at another example: 

The decimal value 4.2 is approximated in IEEE double precision binary floating-point representation by the decimal value 4.2000000000000001776356839400250464678... . The internal hex value is 4010CCCCCCCCCCCD. The next smaller binary value has hex value 4010CCCCCCCCCCCC and it is approximately 4.1999999999999992894572642398998141289... . Since the larger value is closest to 4.2, that is the IEEE double precision binary floating-point value we use to approximate the decimal value of 4.2. Both Cache and Java will use the same "double" value to approximate 4.2. The Cache decimal floating-point representation can exactly represent the decimal value 4.2 with no approximation necessary. The COS function $DOUBLE(x) can be used to force the value "x" into the IEEE binary floating-point representation by generating the best approximation. The COS function $DECIMAL(x) can be used to force the value "x" into the Cache decimal floating-point representation. The COS function $DECIMAL(x,n) can be used to convert the numeric value "x" into a string representation using "n" significant digits (but "n" is limited to be between 1 and 38.) Consider, the following:

USER>set x=4.2

USER>set xd=$DOUBLE(x)

USER>write x 4.2

USER>write xd 4.2000000000000001776

USER>write $DECIMAL(xd,30) 4.20000000000000017763568394003

USER>write $DECIMAL(xd,16) 4.2

USER>write $DECIMAL(xd,17) 4.2000000000000002

Cache converts the numeric value $DOUBLE(4.2) to a default string representation with 20 significant digits. Asking for 30 significant digits shows that $DOUBLE(4.2) has more than 20 digits in its decimal representation. Asking for 16 and 17 decimal digits shows that $DOUBLE(4.2) does approximate the decimal value 4.2 with an accuracy of about 16 decimal digits. Consider, the following computation which removes the leading 4 and 2. (It also multiplies by 10 which does cause a little round off error but you would get a lot more round off error if you computed xd-4.2. Computing xd- 4.2 causes so much round off error that all significance is lost and the answer is 0.0.)

USER>WRITE ((xd-4)*10)-2 .0000000000000017763568394002504646

You can try the above computation in Java using its "double" type and you should get the same answer (with a few less decimal digits printed.) Doing this in Java will give you another way (besides using the BigDecimal package) to demonstrate that "double xd=4.2:" does not produce an exact representation of 4.2 in Java. Another question is why Cache prints $DOUBLE(4.2) using as many digits as 4.2000000000000001776. The reason is that Cache can exactly represent other nearby decimal values such as 4.200000000000000177 and 4.200000000000000178. These are adjacent values in the Cache decimal floating-point representation and the value $DOUBLE(4.2) falls in between these two values. If we convert $DOUBLE(4.2) to decimal then we will get the larger of these values because that is the closest Cache decimal floating-point value.

USER>WRITE $DECIMAL($DOUBLE(4.2))

4.200000000000000178

This conversion from binary to decimal involves an approximation as the following comparisons show:

USER>write $DECIMAL($DOUBLE(4.2))=$DOUBLE(4.2)

0

USER>write $DECIMAL($DOUBLE(4.2))>$DOUBLE(4.2)

1

and the default conversions of these values to string representation shows why the comparisons give these results.

USER>write $DECIMAL($DOUBLE(4.2)),!,$DOUBLE(4.2)

4.200000000000000178

4.2000000000000001776

You should note that 4.2 and $DOUBLE(4.2) are not really close to each other. Cache can also represent an additional 177 decimal floating-point values between 4.2 and $DOUBLE(4.2). The default conversion of a $DOUBLE value to a string will usually have 20 significant digits. If the default representation has less than 20 significant digits then that $DOUBLE value exactly equals the corresponding decimal value represented by the string. If you want to format Cache $DOUBLE values as strings so that look they like Java conversions to strings then you can consider using $DECIMAL(xd,15) or $FNUMBER(xd,"G",14) which are two examples of formatting functions that only print 15 significant digits. If you take a Java "double" type value, move it into a Cache data base, and later extract the value to send it back to a Java "double" variable then the value originally sent to Cache will be identical to value that is returned. As long as the value is between 9.22E+145 and 1E-110 in magnitude then it will not matter whether the value stored internal to Cache uses decimal or binary representation. The approximations involved in converting between binary and decimal will not be large enough to change the "double" value. If the Java "double" data involves values that are outside this range then you must be careful to use the %Double type in Cache in order to eliminate conversions that might cause overflow or underflow. We usually recommend that customers use the default decimal representations in Cache and avoid the %Double type and avoid the COS $DOUBLE function. The default decimal representation rarely involves unexpected approximations. The only time to use $DOUBLE(), %Double and binary floating-point is when the values are not directly entered and read by humans but instead involve a machine-to-machine transfer of data stored using the representation defined by IEEE Std 754. One additional note: when you write 1.9f in Java, the suffix "f" means you are asking to use the 32-bit single precision binary floating-point representation defined by IEEE Std 754-1985. It has a much smaller range and only 24 bits of precision. It's approximation of decimal fractions is only good to about 7 decimal digits. This smaller precision explains the results you are seeing when using this value.

So all in all, you are not asking to get more accurate calculations, but you are rather asking for less accuracy, as you only want to see the rounded values. As shown in the above example you can use $double for some of these calculations. 

-Fab

Fabian Haupt · Oct 21, 2016 go to post

I have to disagree with your first point. Postconditionals actually increase the readability and add much less clutter to the code than having if/then/else blocks everywhere. 

Fabian Haupt · Oct 13, 2016 go to post

With lines being very flexible in COS (you can put pretty much everything into one line), wouldn't size be a more useful metric? Or you could also output an average like size/#lines ?

Fabian Haupt · Oct 11, 2016 go to post

It looks quite pretty. From a cursory glance it require REST/http endpoints to gather the data? It should be fairly trivial to expose any metrics that way so it can be consumed there? Have you looked into that yet?

Fabian Haupt · Oct 4, 2016 go to post

Could you add a little bit of documentation to the code? While I think it's a useful example, documenting the code will make it easier to understand. Not everyone is knows about the %Dictionary classes. And it's generally a good idea to add explaining comments. Thanks!

Fabian Haupt · Oct 4, 2016 go to post

I am using https://hackmd.io/. It is open source and allows for collaborative editing of articles (like Etherpad), but with MD support. You can install it locally in your own network or even your own machine (docker images are available). 

Fabian Haupt · Oct 3, 2016 go to post

Thanks! I fixed both:) (and updated the git repo). 

Please also keep in mind that the code is using the old json syntax, so you'll need to update those too. 

Cheers,

Fab

Fabian Haupt · Sep 30, 2016 go to post

$TR removes all the spaces from the complete string, which is not what was asked for. $ZSTRIP is the way to go.

Fabian Haupt · Sep 2, 2016 go to post

Just for clarification, this is known behaviour:

info coreutils 'chown invocation':

   The `chown' command sometimes clears the set-user-ID or set-group-ID
permission bits.  This behavior depends on the policy and functionality
of the underlying `chown' system call, which may make system-dependent
file mode modifications outside the control of the `chown' command.
For example, the `chown' command might not affect those bits when
invoked by a user with appropriate privileges, or when the bits signify
some function other than executable permission (e.g., mandatory
locking).  When in doubt, check the underlying system behavior.
Fabian Haupt · Sep 2, 2016 go to post

Hi,

our implementations of encryption and hash algorithms can be accessed through the %SYSTEM.Encryption class. Unfortunately we currently don't have a bcrypt implementation.

HTH

-Fab

Fabian Haupt · Sep 1, 2016 go to post

Please note that this example uses the old (deprecated) dot-syntax for the loop. It also loops through your data twice. Nowadays you would write the same more like this:

#include %occInclude
ListDir
 set statement=##class(%SQL.Statement).%New()
 set status=statement.%PrepareClassQuery("%File","FileSet")
 if $$$ISERR(status) { do $system.OBJ.DisplayError(status) }
 set resultset=statement.%Execute("c:\temp","*","",1)
 while resultset.%Next() {
    write:resultset.%Get("Type")="D" !, resultset.%Get("Name")
 }
Fabian Haupt · Aug 28, 2016 go to post

Hi Alexander,

The problem was: with just removing the directory, the processes are still hanging around. So the 1972 port was still in use by the original cache instance. 

Restarting removed that process, that's why it solved the issue. Part of Dmitry's answer was useful in such as you can use lsof to determine which process holds a handle on a port. 

Please let me know if you have any further questions about this!

HTH,

Fab

Fabian Haupt · Aug 16, 2016 go to post

Preserve mode should be avoided if at all possible. It has a negative impact on performance, and tempts people into very bad design choices!

Fabian Haupt · Aug 16, 2016 go to post

Can you elaborate what you mean by that? Every incoming request automatically gets assigned a new session, unless it carries cookies with it that tie it to an existing one. 

Fabian Haupt · Aug 5, 2016 go to post

Hi Marc,

you can use PERFMON [1] to collect statistics about which globals are being accessed.  This will allow you to monitor a process and then distinguish between your globals being accessed and system globals. 

That being said, I don't quite see why would would need that distinction. Where do you draw the line? When you write a sql query and it gets compiled into a routine, would you consider that being your code, or Caché's code? 

There are a couple of global accesses that will happen when you run your code, such as loading a routine (classes get compiled into routines). However, as soon as that has happened once, the routine will be in the routine buffer, so no further disc access will happen. Overall, if your application handles any amount of data that goes beyond a couple of blocks, the 'system accesses' will be negligible. 

In your example you are seeing the first time the routine is being loaded, hence the 4 references. A more realistic example would be to access 100000 random global nodes in your random class. There will still be a more accesses but 3/100000 really doesn't matter by any metric you might employ. 

I hope this addresses your concerns. 

Best,Fab

[1]http://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=GCM_perfmon#GCM_perfmon_using

Fabian Haupt · Aug 1, 2016 go to post

You can also run LOG^%ETN() at any point, which will create an entry in the application error log, with a lot more information than just the stacktrace.  It's very useful as it also gives you the parameters passed into the functions along the stack. 

Fabian Haupt · Jul 14, 2016 go to post

There is ^%GSIZE which will give you an overview of the usage per global. However, if you have a normal size database (>1TB) this is going to take a while. 

I have written statistical approaches in the past. I'll clean up one of them and post it on the community if there is enough interest. (that one actually has a graphical display of the results) 

Best,

Fabian