New post
AllAccepted answers
John Murray · Feb 27, 2017

After Michael's post originally appeared on the intersystems-public-cache Google Group (as a result of the automatic crossposting being done by a username intersystems.dc) my colleague George James responded in the Google Group (GG). However, whatever mechanism the user intersystems.dc has set up for crossposting only handles the initial DC post, and nor does it feed GG responses back to DC. So I'm re-posting George's response here where I think it will get a wider audience.

George wrote:

It seems to me that your ixdLastName index might be usable as some kind of rainbow table to attack the data contained in the AES encrypted field.
 

If I were able to perform a chosen-plaintext attack then querying with like 'J%', then 'Ja%', then 'Jam%' would trivially discover where my name was in the database. 

 
Have you carried out a cryptographic analysis of the strength of this approach?  Logically it must be weaker than just AES on its own.  My question is how much weaker?
John Murray · Feb 27, 2017

Amplifying what Dmitry wrote, here's the web app I defined to make your example class work:

I also had to change your classmethods so they Quit $$$OK instead of simply quitting.

And to test from the browser I used http://localhost:57772/csp/user/testing/print because that's the route you have defined as accepting the GET method from the browser.

There's a REST sample (REST.DocServer) in the SAMPLES namespace. To use it you need to enable the /csp/samples/docserver web application by setting this checkbox:

Then this URL will return the source of the Cinema.Review class from the SAMPLES namespace:

http://localhost:57772/csp/samples/docserver/class/samples/Cinema.Review

Given that this REST sample will return the source of any class from any namespace, it's understandable that the /csp/samples/docserver application is disabled by default on a new installation.

John Murray · Feb 10, 2017

I had assumed that if the "Serve Files" setting isn't "Always and cached" then the Gateway wouldn't cache them.

Doc at http://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=... seems to imply this.

That doc also states that the "Serve Files Timeout" is to do with caching by the browser.

Anyway, it'll be interesting to hear what your tests show.

John Murray · Feb 7, 2017

Based on my reading of 2014.1 code the Exists method in Ens.Util.FunctionSet doesn't check locks when testing for the existence of the entry.

I think the Save button from Portal leads to the SaveLookupTable method of EnsPortal.LookupSettings. That method obtains an exclusive lock on the subtree of ^Ens.LookupTable where the LUT gets stored. It then kills the entire LUT and re-files each entry one by one. Once finished it releases the lock.

All of that is protected by a TSTART / TCOMMIT but this is not sufficient to prevent another process momentarily detecting the absence of an entry that would normally exist.

John Murray · Feb 2, 2017

I assume you want the subscripts of the global always to collate as strings. If so, here's how you can specify the collation type of a global when you create it. Note that the global must not exist to start with:

USER>w $D(^a)
0
USER>s sc=##class(%Library.GlobalEdit).Create(,"a",133)
 
USER>w sc
1
USER>s ^a("1.0012")=""
 
USER>s ^a("1.0011")=""
 
USER>s ^a("1.0010")=""
 
USER>zw ^a
^a("1.0010")=""
^a(1.0011)=""
^a(1.0012)=""
 
USER>

To get a list of collation codes (which is where I found 133):

%SYS>d ^COLLATE
 
Status       Number   Abbrev   Name
----------   ------   ------   ----------------------
Built-in        0     OANS     ISM Pre-6.2
Built-in        1     ANSI     ISM 6.2->6.4
Built-in        2     COBR     Ipsum/Cobra
Built-in        3     DTMC     DTM-compatible
Built-in        4     CBR2     Ipsum/Cobra-2
Built-in        5     UNIC     Cache standard
Not loaded     10     GER1     German1
Not loaded     11     POR1     Portuguese1
Not loaded     12     POL1     Polish1
Not loaded     13     GER2     German2
Not loaded     14     SPA1     Spanish1
Not loaded     15     DAN1     Danish1
Not loaded     16     CYR1     Cyrillic1
Not loaded     17     GRE1     Greek1
Not loaded     18     CZE1     Czech1
Not loaded     19     CZE2     Czech2
Not loaded     20     POR2     Portuguese2
Not loaded     21     FIN1     Finnish1
Not loaded     22     JAP1     Japanese1
Not loaded     24     POL2     Polish2
Not loaded     27     FRE1     French1
Not loaded     28     FIN2     Finnish2
Not loaded     29     HUN1     Hungarian1
Not loaded     30     GER3     German3
Not loaded     31     POL3     Polish3
Not loaded     32     SPA2     Spanish2
Not loaded     33     DAN2     Danish2
Not loaded     34     GRE2     Greek2
Not loaded     35     FIN3     Finnish3
Not loaded     36     LIT1     Lithuanian1
Not loaded     37     CYR3     Cyrillic3
Not loaded     38     SLO1     Slovenian1
Not loaded     39     SLO2     Slovenian2
Not loaded     40     TUR1     Turkish1
Not loaded     41     DAN3     Danish3
Not loaded     42     UKR1     Ukrainian1
Not loaded     43     CYR4     Cyrillic4
Not loaded     44     CZE3     Czech3
Not loaded     46     MAL1     Maltese1
Not loaded     48     MAL2     Maltese2
Not loaded     49     SPA4     Spanish4
Not loaded     50     SLO1     Slovak1
Not loaded     51     SPA5     Spanish5
Not loaded     52     FIN4     Finnish4
Built-in      128     OSTR     ISM Pre-6.2 string
Built-in      129     NSTR     ISM 6.2->6.4 string
Built-in      133     USTR     Cache standard string
 
%SYS>

If you don't want to have to bother with setting the global to use String collation rather than the default (which will be what's sometimes called Numeric collation), then prefix all your subscripts with a character that will force them to be interpreted as a string. Cache SQL uses this trick internally, adding a leading space (" ") to the subscripts it creates.

John Murray · Feb 1, 2017

It's now been confirmed that the relevant parsers will not be ported to OpenVMS. So it'll never be possible to roundtrip all classes in UDL from that platform.

Also worth noting that GetTextAsFile appears to pre-delete the class it's about to import before it encounters the parse error and aborts the import. So you're left without a copy of the class in your namespace. Oops!

John Murray · Feb 1, 2017

I think you could simplify your first approach a little by reverting to calling OpenStream on your reader object rather than using OpenFile:

  // 1st approach - it succeeds, so the file errorOpenFile.xml is NOT generated
  //    
  d msg.Value.Rewind()
  set fs=##class(%Stream.FileCharacter).%New()
  set fs.Filename="D:\DATABASES\OVPATH\temp\test.xml"
  set fs.TranslateTable = "UTF8"
  set tSC=fs.CopyFrom(msg.Value)
  set tSC=fs.%Save()
    
  s reader=##class(%XML.Reader).%New()

  d fs.Rewind() // might not be necessary, but won't hurt
  Set sc = reader.OpenStream(fs)

 

Anyhow, the fact that you don't get an error confirms my hypothesis that the original stream (msg.Value) contains Unicode data but the reader treats it as though it is UTF8-encoded.

In your code above I think you can also omit the line where you set fs.Filename and instead allow the stream to generate its own temporary file. Explicitly naming the file may be handy when debugging, but it will cause problems if more than one process runs this code concurrently.