New post
AllAccepted answers
Sean Connelly · Feb 22, 2018

Hi Jacques,

A word of caution.

The server() method requires a little finesse to ensure a good user experience.

What you might not realise is that the server() method bakes a call to the browsers XMLHttpRequest object with a synchronous flag. This will block the main browser process until the response message is returned.

You can simulate a 1 second delay with a hang in the class method...

ClassMethod comboboxOptions(chbox = 0)
{
    hang 1
    set options = ""


When you now click on the check box notice that it does not change its state, not until the response comes back. To the end user this is really bad. They will think it did not work and will click it again. This will normally result in a second request and the checkbox going back to its original state. The user now starts clicking maniacally causing one long UI freeze.

Ideally you want the checkbox to immediately show as checked, to do this you need to debounce the server() call and let the browser do a much needed re-paint. You can do this with a window setTimeout...

function checkboxChanged(chbox) {
    window.setTimeout(function(){
      var options = #server(..comboboxOptions(chbox.checked))#;
      document.getElementById("combo").innerHTML = options;      
    },0)
}


You will now see the check box change immediately.

Thats gives the user some instant feedback, but it wont fully stop them from still clicking on the checkbox. Ideally you want to prevent them from doing it again until the first request has finished.

We can half solve this by disabling the checkbox, like so...

function checkboxChanged(chbox) {
    document.getElementById("chbox").disabled = true;
    window.setTimeout(function(){
      var options = #server(..comboboxOptions(chbox.checked))#;
          document.getElementById("combo").innerHTML = options;
        document.getElementById("chbox").disabled = false;
    },0)
}


However, you will notice that click events are still bubbling up to the re-enabled check box. This is because the event is pending and bubbles up after the check box has been re-enabled, hence why it then gets processed. It means that we also need to debounce this re-enablement to allow the click event to be processed and ignored.

function checkboxChanged(chbox) {
    document.getElementById("chbox").disabled = true;
    window.setTimeout(function(){
      var options = #server(..comboboxOptions(chbox.checked))#;
          document.getElementById("combo").innerHTML = options;
          window.setTimeout(function(){        
            document.getElementById("chbox").disabled = false;
          },0)
    },0)
}

    
This is almost done, the last thing I would do would be to set a fixed width on select box so it doesn't cause a UI re-layout / flicker with different sized contents. And then for good measure have a temp loading message in the first option...

function checkboxChanged(chbox) {
    document.getElementById("combo").innerHTML = "<select id='combo' style='width:150px;'><option>Loading...</option></select>";
    document.getElementById("chbox").disabled = true;
    window.setTimeout(function(){
      var options = #server(..comboboxOptions(chbox.checked))#;
          document.getElementById("combo").innerHTML = options;
          window.setTimeout(function(){        
            document.getElementById("chbox").disabled = false;
          },0)
    },0)
}


There are alternatives that would avoid needing to do this, I will put something together...

Sean.

Sean Connelly · Feb 19, 2018

Hi George / Gordon,

Feels like too many potential trip hazards getting this right, and the possibility of creating false positives.

I would recommend using an HTTP client instead.

Here is a quick and dirty example, it uses %Net.HttpRequest to make the REST calls. Take a look at the documentation here http://docs.intersystems.com/latest/csp/documatic/%25CSP.Documatic.cls as there are many other settings (e.g. credentials) that you might also want to set.

The example has both a GET and a POST...

Class Foo.Rest1 Extends %CSP.REST
{

XData UrlMap [ XMLNamespace = "http://www.intersystems.com/urlmap" ]
{
    <Routes>
    <Route Url="/book" Method="POST" Call="PostBook" />
    <Route Url="/book/:ibn" Method="GET" Call="GetBook" />
    </Routes>
}

ClassMethod PostBook() As %Status
{
    set obj=##class(%DynamicObject).%FromJSON(%request.Content)
    set ^Foo.Books(obj.ibn,"name")=obj.name
    set ^Foo.Books(obj.ibn,"author")=obj.author
    quit 1
}

ClassMethod GetBook(pIBN) As %Status
{
    if '$Data(^Foo.Books(pIBN)) set %response.Status = 404 return 1
    set book = {
        "ibn" : (pIBN),
        "name" : (^Foo.Books(pIBN,"name")),
        "author" : (^Foo.Books(pIBN,"author"))
    }
    write book.%ToJSON()
    return 1
}

ClassMethod GetNewRestHelper() As %Net.HttpRequest
{
    set httpRequest=##class(%Net.HttpRequest).%New()
    set httpRequest.Server="localhost"
    set httpRequest.Port=57772
    set httpRequest.Timeout=2
    quit httpRequest
}

ClassMethod Tests() As %Boolean
{
    //basic assert, replace with actual unit tester implementation
    #define assert(%a,%b) write !,"Test ",$increment(count)," ",$select(%a=%b:"PASSED",1:"FAILED, expected """_%b_""" recieved"""_%a_"""")

    //TEST 1 : should save ok
    set rest=..GetNewRestHelper()
    set book = {
        "ibn" : 1234,
        "name" : "Lord Of The Rings",
        "author" : "J.R.R. Tolkien"
    }
    set json = book.%ToJSON()
    do rest.EntityBody.Write(json)
    do rest.Post("/foo/book")
    $$$assert(200,rest.HttpResponse.StatusCode)

    //TEST 2 : should find book 1234
    set rest=..GetNewRestHelper()
    do rest.Get("/foo/book/1234")
    $$$assert(200,rest.HttpResponse.StatusCode)
    set json=rest.HttpResponse.Data.Read(32000)
    set book=##class(%DynamicObject).%FromJSON(json)
    $$$assert(book.ibn,1234)
    $$$assert(book.name,"Lord Of The Rings")
    $$$assert(book.author,"J.R.R. Tolkien")

    //TEST 3 : should NOT find book 5678, test should fail
    set rest=..GetNewRestHelper()
    do rest.Get("/foo/book/5678")
    $$$assert(200,rest.HttpResponse.StatusCode)
}

}

Sean Connelly · Jan 17, 2018

Hi Tom,

The HTTP error looks like a red herring. I suspect its just a generic CSP broker issue, probably a secondary side effect to the SQL connection failing.

It's difficult to see anything specific from your post that might be causeing the SQL connection to fail. I normally keep tweaking settings and trying different drivers at this stage until it works (sorry, I know its obvious advice).

I assume from port 1433 that you are trying to connect to SQL Server. The only thing that I am suspicious of is if you really need a kerberos connection to SQL Server. If you have not specifically configured this on SQL Server then I think the 459 error is confusing matters, in which case I would raise a WRC.

As an alternative, you could try using a JDBC driver instead. I've had better luck in the past using JDBC on Linux.

Sean.

Sean Connelly · Dec 10, 2017

Hi Ponnumani,

I agree with Robert and Eduard, it's very difficult to answer such a broad question in any detail without duplicating in a single post (at length and time) what has already been written many times over.

However, tree traversal and its methods are very pertinent to mastering the fundamentals of Caché and worthy of a reference answer to help anyone else starting out on this path.

It's important to understand that Caché is essentially a modern implementation of the M programming language / database.

M was first developed back in the late 60's. It was mainly implemented on PDP servers which at that time where limited to a few hundred K of memory. These limitations greatly influenced the design and efficiency of M's data globals which would go on to be its core strength.

Caché was an evolutionary step in this technology time line which added a new high level object oriented programming language as well as a comprehensive set of libraries and tools.

Despite being an evolutionary step, Caché is still considered an implementation of M, adhering to much of the 1995 ANSI language standards.

Whilst some people distance themselves from this heritage, many of us are embrace these roots and like to combine both the modern aspects of Caché with much of what we mastered as M developers, the combination of which makes for a very powerful platform that you won't find in any other single technology.

It's important to understand this history as much of the information and teachings to master the raw fundamentals of Caché can be found by using information and teachings that still exist for the M programming language.

To help on-ramp any new Caché developers, I always buy them "M Programming: A comprehensive Guide" by Richard Walters as a short staple learning tool. There are also many other good books out there and a simple search on Mumps books will bring up a good selection.

All of these books will dive into globals, the hierarchical tree data storage which is very relevant to your question. They will also explain the $order and $query functions which are key to traversing global's.

Most of the time you will have shallow global data and traversing this data will be implemented with an equal number of nested $order functions. When the data is deeper, or the traversal needs to be more generic then you can either use the $query function for a brute force traversal, or implement a combination of $order and a recursive loop.

So the short answer to your question would be to seek out everything that you can find on Caché / M globals and the $order and $query functions. Take the time to read and practice using globals directly and you will master tree traversal. In addition, if you haven't already done so, I would also seek out generic books and teachings on programming algorithms that cover topics such as recursion and trees.

I hope this helps, and we are more than happy to help when you get stuck with more specific (and hopefully more detailed) questions.

Sean Connelly · Dec 9, 2017

Hi Motti,

Sounds like you could get in a tangle trying to do this.

My approach would be to throw more disk space at it and keep everything at 60 days, lowest TCO solution with minimal effort.

If you were to move some of the interfaces, then you could use your web servers config to reverse proxy specific URL paths to different namespaces. If your using the default apache web server then you might look to implement a dedicated installation.

Alternative suggestions.

1. Write your own message purge utility.

2. Use an enterprise message bank (latest versions of Ensemble recommended)...

http://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=EGDV_message_bank

Sean Connelly · Dec 3, 2017

Darn it Bert, these things are addictive!

There's no way I'm getting up at 5am to even possibly get on the main leader board, but its fun to see how we all approach it.

I'll post mine up here....

https://gist.github.com/SeanConnelly/065cb51fc6572c6bf3dac4a9973fb54f

And maybe a few JavaScript ones as well if I get time...

https://gist.github.com/SeanConnelly/e0623e13241fd94fbf8df96b09755f95

Just for fun, one liners get an extra point - everyone rolls eyes :)

Day three part 2...

s (x,y)=0,d=7,g(0,0)=1,a="+1  -1" f i=1:1 {s e=$s(d=7:1,1:d+2),d=$s($g(g(x+$e(a,e,e+2),y+$e(a,e-2,e)))="":e,1:d),x=x+$e(a,d,d+2),y=y+$e(a,d-2,d),g(x,y)=$g(g(x,y+1))+$g(g(x,y-1))+$g(g(x+1,y))+$g(g(x+1,y+1))+$g(g(x+1,y-1))+$g(g(x-1,y))+$g(g(x-1,y+1))+$g(g(x-1,y-1)) i g(x,y)>t ret g(x,y)}
Sean Connelly · Nov 8, 2017

RawContent is a calculated property that is truncated to 10,000 characters. Something to do with playing nice with result-sets.

Perhaps RawContent was not the best name for this property, maybe PotentiallyTruncatedRawContentSoBeCarefull would have been better.

The actual raw content is stored as a string in a global, so presumably it is limited to either 32k or around 3.6GB depending if long string support is enabled or not.

Sean Connelly · Nov 8, 2017

A new persistent object will have its ID created in %SaveData which is called after %OnBeforeSave(), if you look in the code behind you will see this...

If 'partial { Set sc=..%OnBeforeSave(%objTX(1,+$this,4)) Throw:('sc) ##class(%Exception.StatusException).ThrowIfInterrupt(sc) }
Set id=$listget(serial),sc=..%SaveData(.id) If ('sc) { Throw ##class(%Exception.StatusException).ThrowIfInterrupt(sc) }

So this shouldn't work for you, unless the object has already been persisted.

Sean Connelly · Nov 7, 2017

Hi Sergey,

> 1. No client caching

You will obviously know this already, but JSON can of course be cached (with its image) on the client if required.

However, images served in JSON are typically private images served via a secure API and should never be cached. As such, all caching benefits go out of the window anyway, whether your using JSON or not.

> 2. No lazy-loading

Fetching images dynamically in JSON does not stop it from being a lazy-loading technique, if anything it is perfectly suited to loading images just when they are needed.

> There are also other downsides possible.

There is really only one technical downside, images posted as Base64 are inflated by 37%. If you are paying for network bandwidth by the $ then you might think twice, even if web server compression will bring this back down.

Of course continue to serve static unsecure images as binary over HTTP, particluarly if you want to leverage on automatic image caching.

If you are building a rich API (REST/RPC) with a JSON payload, then don't be afraid of embedding images. This will make for consistent API's.

When we build SOAP interfaces, we don't expect a SOAP client to drop out of the SOAP protocol to raw HTTP to make a seperate request for an image. How would we describe this via a WSDL to the soap client? How should the authentication for this be implemented? Some kind of token that needs its own seperate emergency hatch in the server side API?

The same will eventually be true for REST and JSON-RPC. It doesn't matter if this is server to server, or client to server, what we really want is a consistent API that can be described and authenticated in a consistent way (e.g SWAGGER + OAuth 2.0).

Bottom line, to say there are "big problems" is a little FUD, and I hope will not deter other developers from considering this approach before they have done their own research.

Sean.

Sean Connelly · Oct 12, 2017

The %Net.HttpRequest class provides an HTTP client for interacting with HTTP servers.

Underneath it essentially opens up a TCP port to the given IP address (DNS lookup), sends the required HTTP headers, request body / MIME parts, and waits for the response which is unpacked into a response object.

I'm not sure if you are asking if its safe to use or how to use it, so here is a reply to both.

The class is very robust and adheres to the HTTP/1.1 specification.

You will find that this class underpins many other classes in Caché / Ensemble and should be considered battle tested code that you can fully rely on.

To use it, the best way to start out would be to play around with the code in caché terminal. You can cut and paste the snippet you provided directly into the terminal (without the tabs), and this is what you will see...
 

USER>Set httprequest=##class(%Net.HttpRequest).%New()
 
USER>Set httprequest.Server="www.intersystems.com"
 
USER>Do httprequest.Get("/")
 
USER>Do httprequest.HttpResponse.OutputToDevice()
HTTP/1.1 301 Moved Permanently
CONNECTION: keep-alive
CONTENT-LENGTH: 178
CONTENT-TYPE: text/html
DATE: Thu, 12 Oct 2017 14:14:02 GMT
LOCATION: https://www.intersystems.com/
SERVER: nginx
X-TYPE: default
 
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx</center>
</body>
</html>

As you can see, it returns a 301 status code because the intersystems web site is only running over HTTP on port 443, there are settings on the request object that support HTTPS.

Better to play around with a non HTTP site to begin with, here is an example with my website...

(a small trick, the command zw will dump the contents of an object to screen, but not if that object is a property of another object, so assign it to a variable and zw that variable, here you can see the status code and other response properties that you can use)
 

USER>set httprequest=##class(%Net.HttpRequest).%New()
 
USER>set httprequest.Server="www.memcog.com"
 
USER>set sc=httprequest.Get("/")
 
USER>set response = httprequest.HttpResponse
 
USER>zw response
response=<OBJECT REFERENCE>[4@%Net.HttpResponse]
+----------------- general information ---------------
|      oref value: 4
|      class name: %Net.HttpResponse
| reference count: 3
+----------------- attribute values ------------------
|    ContentBoundary = ""
|        ContentInfo = ""
|      ContentLength = 4660
|        ContentType = "text/html"
|               Data = "5@%Stream.FileCharacterGzip"
|Headers("ACCEPT-RANGES") = "bytes"
|Headers("CONTENT-ENCODING") = "gzip"
|Headers("CONTENT-LENGTH") = 4660
|Headers("CONTENT-TYPE") = "text/html"
|    Headers("DATE") = "Thu, 12 Oct 2017 14:29:58 GMT"
|    Headers("ETAG") = """66a36a6a-3f38-53cedab4fb6b1"""
|Headers("LAST-MODIFIED") = "Tue, 20 Sep 2016 10:12:42 GMT"
|  Headers("SERVER") = "Apache"
|    Headers("VARY") = "Accept-Encoding,User-Agent"
|        HttpVersion = "HTTP/1.1"
|       ReasonPhrase = "OK"
|         StatusCode = 200
|         StatusLine = "HTTP/1.1 200 OK"
+-----------------------------------------------------

USER>write response.Data.Read(1500)​

<!--
                                                         DEVELOPED BY...
 

__/\\\\____________/\\\\__/\\\\\\\\\\\\\\\__/\\\\____________/\\\\________/\\\\\\\\\_______/\\\\\__________/\\\\\\\\\\\\_
 _\/\\\\\\________/\\\\\\_\/\\\///////////__\/\\\\\\________/\\\\\\_____/\\\////////______/\\\///\\\______/\\\//////////__
  _\/\\\\///\\\/\\\/_\/\\\_\/\\\\\\\\\\\_____\/\\\\///\\\/\\\/_\/\\\__/\\\______________/\\\______\//\\\_\/\\\____/\\\\\\\_
   _\/\\\__\///\\\/___\/\\\_\/\\\///////______\/\\\__\///\\\/___\/\\\_\/\\\_____________\/\\\_______\/\\\_\/\\\___\/////\\\_
    _\/\\\____\///_____\/\\\_\/\\\_____________\/\\\____\///_____\/\\\_\//\\\____________\//\\\______/\\\__\/\\\_______\/\\\_
     _\/\\\_____________\/\\\_\/\\\_____________\/\\\_____________\/\\\__\///\\\___________\///\\\__/\\\____\/\\\_______\/\\\_
      _\/\\\_____________\/\\\_\/\\\\\\\\\\\\\\\_\/\\\_____________\/\\\____\////\\\\\\\\\____\///\\\\\/_____\//\\\\\\\\\\\\/__
       _\///______________\///__\///////////////__\///______________\///________\/////////_______\/////________\////////////____

-->

<html>
    <head>
    <title>MEMCOG: Web Development, Integration, Ensemble, Healthshare, Mirth</title>
    <link href="main.css" rel="stylesheet">
    <link href="https://fonts.googleapis.com/css?family=PT+Sans" rel="stylesheet">
    <script src="https://use.fontawesome.com/
Sean Connelly · Oct 12, 2017

When you created your router, you selected VDocRoutingEngine, which expects a doc type in the request message.

For your requirements you should use the standard RoutingEngine.

I would remove the process you have created and start again, open the business process wizards, and select...

EnsLib.MsgRouter.RoutingEngine

from the business process class drop down list (within the "all processes" tab), then add the rules as you have done previously.

Sean Connelly · Oct 11, 2017

Hi Tom,

There is no default rename method inside studio.

The workaround is to do the following...

1. File > Open Project , then select the project

2. File > Save As , provide a name for the new project

As its the default you wont need to clean up, but if you want to delete a previously named project then click on file open project, right click on the project name and select delete from the context menu.

Sean Connelly · Oct 10, 2017

Hi Thembelani,

Assuming that you have got as far as serialising the objects to an XML stream, and you just want to prettify the XML, then you can use the following solution.

Add the following to a new / existing class...
 

ClassMethod Format(pXmlStream As %CharacterStream, Output pXmlStreamFormatted As %CharacterStream) As %Status
{
    set xslt=##class(%Dictionary.XDataDefinition).%OpenId(..%ClassName(1)_"||XSLT",-1,.sc)
    quit ##class(%XML.XSLT.Transformer).TransformStream(pXmlStream,xslt.Data,.pXmlStreamFormatted)
}

XData XSLT
{
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
 <xsl:output omit-xml-declaration="yes" indent="yes"/>
 <xsl:strip-space elements="*"/>
 <xsl:template match="node()|@*">
  <xsl:copy>
   <xsl:apply-templates select="node()|@*"/>
  </xsl:copy>
 </xsl:template>
</xsl:stylesheet>
}

To use it...

set sc=##class(Your.Class).Format(.xmlStream,.xmlStreamFormatted)


Pass in your XML stream as the first argument (by ref) and get back the formatted stream as the second argument (by ref).