Eduard Lebedyuk · Mar 7, 2019 go to post

Link to file doesn't need for a file to exist, but the containing directory must exist and should be writable by a OS user (cacheusr in uyour case probably).

I'd try to write into a temp dir first, where you're sure you have access:

set file = ##class(%File).TempFilename("pdf")
set sc = stream2.LinkToFile(file)
quit:$$$ISERR(sc) sc

record value of file somewhere (output to display or store in global) and check if the file was created).

%Save method also returns status, you should return it instead of $$$OK:

set sc = stream2.%Save()
quit sc
Eduard Lebedyuk · Mar 7, 2019 go to post

The code looks good. What error are you getting?

Try replacing stream2 %Stream.FileCharacter with %Stream.FileBinary.

Try replacing stream1 %Stream.FileCharacter with %Stream.TmpBinary.

If you have message sample less than 3,5 mb in size try to write a test without intermediate stream.

You probably should write to a pdf file and not a txt one.

If you can, get a sample original/decoded file. Compare original file and your stream2 file using hex editor to spot differences.

Eduard Lebedyuk · Mar 7, 2019 go to post

I try to separate SQL from ObjectScript code. So mainly, I'm using queries, for example:

/// Some report. To display in terminal call:
/// do ##class(class).reportFunc().%Display()
Query report(date As %String = {$zd($h-1,8)}) As %SQLQuery(SELECTMODE = "ODBC")
{
SELECT
  ID,
  Value,
  EventDate
FROM myTable
WHERE EventDate>=TO_POSIXTIME(:date,'YYYYMMDD') AND EventDate<TO_POSIXTIME(:date+1,'YYYYMMDD')
ORDER BY EventDate
}

Queries can be calles from ObjectScript using autogenerated Func method:

/// Really %sqlcq.<NAMESPACE>.cls<NUMBER>
#dim rs As %SQL.ISelectResult
set rs = ..reportFunc(date)
//do rs.%Display()

while rs.%Next() {
    write rs.ID,!
}

I found this approach improves readability of the codebase. More about queries in this article.

Special case - one value.

Sometimes you don't need a resultset, but one value. In that case:

If you know ID it's possible to use GetStored method:

set value = ##class(test).<PropertyName>GetStored(ID)

If you know unique indexed value but don't know ID, it's possible to get id with Exists method:

ClassMethod <IndexName>Exists(val, Output id) As %Boolean

And after that use GetStored method.

More on auto-generated methods, such as GetStored and Exists in this article.

Finally, if you can't use above methods or you need one value but it's an aggregate, use embedded SQL if it's a short SQL and Query if it's long.

Eduard Lebedyuk · Mar 7, 2019 go to post

Use methods from %CSP.Portal.Utils class:

set pageID = "" //page url, URL encoded via $zconvert(url,"O","URL")

set currentResource = ##class(%CSP.Portal.Utils).%GetCustomResource(pageID)

set sc = ##class(%CSP.Portal.Utils).%SetCustomResource(pageID, newResource)

Eduard Lebedyuk · Mar 7, 2019 go to post

I' do it in 2 steps.

  1. Iterate over data array and build a temp local structure to hold all additional items you need
  2. Iterate over this new structure and add these items to data array.

You can add to array using %Push method:

do Obj.data.%Push(newItem)

If you want to push at a specific posiiton, use %Set

do Obj.data.%Set(position, newItem)

That said your later structure contains data which does not exist in the original structure (text values for projects and sub-projects) so you need to get it from somewhere.

Also is project - subproject hierarchy one-level, or there could be an arbitrary number of sub-project levels (i.e. 10  10-1 → 10-1-1 → 10-1-1-1)?

Eduard Lebedyuk · Mar 7, 2019 go to post

Enable ODBC log. Maybe there would be errors pointing to the root of the issue?

Maybe something to do with INFORMATION_SCHEMA.

Eduard Lebedyuk · Mar 5, 2019 go to post

Okay, even if the classes/globals are the same there's a solution. Let's say you have Sample.Person class in namespaces SAMPLES and USER, each with their own data:

Class Sample.Person Extends %Persistent
{

Property Name;

Storage Default
{
<Data name="PersonDefaultData">
<Value name="1">
<Value>%%CLASSNAME</Value>
</Value>
<Value name="2">
<Value>Name</Value>
</Value>
</Data>
<DataLocation>^Sample.PersonD</DataLocation>
<DefaultData>PersonDefaultData</DefaultData>
<IdLocation>^Sample.PersonD</IdLocation>
<IndexLocation>^Sample.PersonI</IndexLocation>
<StreamLocation>^Sample.PersonS</StreamLocation>
<Type>%Library.CacheStorage</Type>
}

}


And you want to query both from the USER namespace. In that case create a new class extending Sample.Person in the USER namespace and modify storage like this:

Class Utils.SamplesPerson Extends (%Persistent, Sample.Person)
{

Storage Default
{
<Data name="PersonDefaultData">
<Value name="1">
<Value>%%CLASSNAME</Value>
</Value>
<Value name="2">
<Value>Name</Value>
</Value>
</Data>
<DataLocation>^["SAMPLES"]Sample.PersonD</DataLocation>
<DefaultData>PersonDefaultData</DefaultData>
<ExtentSize>200</ExtentSize>
<IdLocation>^["SAMPLES"]Sample.PersonD</IdLocation>
<IndexLocation>^["SAMPLES"]Sample.PersonI</IndexLocation>
<StreamLocation>^["SAMPLES"]Sample.PersonS</StreamLocation>
<Type>%Library.CacheStorage</Type>
}

Note that DataLocation, IndexLocation, IdLocation and StreamLocation point to the SAMPLES namespace.

Now query:

SELECT *
FROM Utils.SamplePerson

Would fetch data from SAMPLES namespace.

Eduard Lebedyuk · Mar 5, 2019 go to post

Some ideas.

1. The problem is isc.rabbitmq.API  class was imported with an error. Delete this class and try this code (post output):

Set class = "isc.rabbitmq.API"
Set classPath = ##class(%ListOfDataTypes).%New()
Do classPath.Insert(PATH-TO-JAR)
Set gateway = ##class(%Net.Remote.Gateway).%New() 
Set sc = gateway.%Connect("localhost", PORT, $Namespace, 2, classPath)
Zwrite sc
Set sc = gateway.%Import(class)
Zwrite sc
Write ##class(%Dictionary.CompiledClass).%ExistsId(class)

2. Try recompiling isc.rabbitmq.API  class.

3. Maybe have 2 amqp jars with com.rabbitmq.tools.jsonrpc.JsonRpcException class.  The only class you should import is isc.rabbitmq.API  class. It shouldn't pull many additional ones.

4. What version of AMQP are you using? I'm using amqp-5.0.0. Try the same verison?

Eduard Lebedyuk · Mar 4, 2019 go to post

Great to see more AoC participants!

Also please note, that $ListValid returns 1/0 (%Boolean) and not a %Status. So $system.Status methods won't work.

set status = $LISTVALID(list)

you can write something like this:

set isValid = $listValid(list)
if isValid {
  // main code
} else {
  write "List is invalid:",!
  zwrite list
}
Eduard Lebedyuk · Mar 2, 2019 go to post

Abstract class could have this method:

ClassMethod GetNew() As test.Order {
  set class = "test.Order"
  if ..GetCompileDate($classname())'=..GetCompileDate(class) {
    set sc = $system.OBJ.Compile($classname(),"cukb /display=none")
  }
  quit $classmethod(class, "%New")
}

ClassMethod GetCompileDate(class As %Dictionary.CacheClassname) [CodeMode = expression]
{
$$$comClassKeyGet(class,$$$cCLASStimechanged)
}

Eduard Lebedyuk · Mar 1, 2019 go to post

Eduard, that's a good suggestion, but one issue I see is that if you are processing millions of documents, that's a lot of database overhead. Using a simple global reference with locks would work better as you would only have n files persisted up to the Pool Size. 

The solution I outlined at all times contains exactly up to PoolSize records, so I don't think it's a very big overhead. You can lock filenames I suppose, why not?

My next challenge is to determine a better way for the file service to pick off files from the OS folder without loading all the files into a potentially massive result set which can blow out local memory and in extreme cases quickly blow out CacheTemp DB size.

Ensemble File inbound adapter uses FileSet from %File class. This query uses $zsearch to iterate over files in a directory and populates a ppg with results at once. Calling Next on that result set only moves the ppg key. You can rewrite FileSet query to advance $zsearch  on a Next call. Don't know how it would affect performance though.

Eduard Lebedyuk · Mar 1, 2019 go to post

I suppose you can solve this problem by separating your persistent class into abstract definition and persistent "storage only" class.

It could look like this:

Class test.Abstract.Order [ Abstract ]

{

Property a;

Method doStuff()

{

}

}

and persistent class:

Class test.Order Extends (test.Abstract.Order, %Persistent)

{

/*Generated Storage */

}

And only map test.Abstract package. This way you'll need to:

  • Copy test.Order class manually (but only once, as it does not change).
  • Recompile test.Order class in each namespace if Abstract realization changes.

But in this setup persistent class could be tuned, etc.. Also you can automate deployment steps with CI/CD tools.

Eduard Lebedyuk · Feb 27, 2019 go to post

What does this command return?

dumpbin /headers c:/InterSystems/IRIS_2019/CSP/bin/CSPa24.dll

For me (on IRIS for Windows (x86-64) 2018.2 (Build 241U) Fri Jan 26 2018 01:06:37 EST) it shows:

Dump of file ./CSPa24.dll

PE signature found

File Type: DLL

FILE HEADER VALUES
             14C machine (x86)
               5 number of sections
        5BC558F7 time date stamp Tue Oct 16 06:20:23 2018
               0 file pointer to symbol table
               0 number of symbols
              E0 size of optional header
            2102 characteristics
                   Executable
                   32 bit word machine
                   DLL

Note 14C machine (x86) value, it means that it's a 32bit dll. If you have similar output, I recommend installing 32 bit apache.

Eduard Lebedyuk · Feb 26, 2019 go to post

I have not tested the code on Windows, but here's my idea.

As you can see in the code for test method in cause of exceptions I end all my processes with

do $system.Process.Terminate(, 1)

it seems this path is getting hit.

How to fix this exception:

  1. Check that test method actually gets called. Write to a global in a first line.
  2. In exception handler add do ex.Log() and check application error log to see exception details.
Eduard Lebedyuk · Feb 26, 2019 go to post

I recommend locking as a solution to that problem.

Create a table/class/global which holds {filename, lock, ownerJobId}.

All service jobs execute the same file search, take first filename, check the lock table. If it's empty - write into it and start processing the file.

If lock table has the entry take the next file, till you find one without lock.

After file is processed delete/move it and remove the entry from lock table.

On job shutdown purge the table records associated with job id.

This way you can scale jobs easily.

Eduard Lebedyuk · Feb 25, 2019 go to post

Here's how you can do it:

  1. Download CSPGateway kit from WRC  for 2017.2.1
  2. Install it on a new server
  3. Connect it to IIS
  4. Open csp/bin/Systems/Module.cxw and add required application(s)

I think this documentation page describes exactly what you want to do.

Eduard Lebedyuk · Feb 24, 2019 go to post

You get this error (Datatype value '2019-02-01' is not a valid number) because %Date stores data in horolog format, so you need to do one of:

  • (Recommended approach) Convert date value from external format (2019-02-01) into internal format (horolog) using $zdh function:
Set consumerRecord.ActivePeriod.StartDate = $zdh(obj.activePeriod.startDate, 3)
  • Use datatype for which this value is valid, i.e. %TimeStamp. To check, all datatype classes offer IsValid method, you can use it to check value validity
zw ##class(%TimeStamp).IsValid("2019-02-01")
  • If you're using InterSystems IRIS you can store dates as %PosixTime.

Regarding Atelier, I use it to debug REST services and it shows variable values. You can try to reinstall it. If you're on Windows you can also use Studio.

Eduard Lebedyuk · Feb 23, 2019 go to post
  1. Go to SMP - Menu - Manage Web Applications.
  2. Open /csp/healthshare/hspc/  application settings.
  3. Check that Inbound Web Services are enabled for the application.
  4. Press Save button.

Eduard Lebedyuk · Feb 22, 2019 go to post

Small addition: objects are always passed by reference, so usually you don't need to pass them with dot.

Eduard Lebedyuk · Feb 22, 2019 go to post

What does this output in a terminal:

set respText="{""access_token"":""4SDFDSFDSF-aSDASDASD"",""expires_in"":""3300"",""refresh_token"":"""",""scope"":""sms_send"",""token_type"":""Bearer""}"  
set sc = ##class(%ZEN.Auxiliary.jsonProvider).%ParseJSON(respText,,.pObject,1)
zw pObject

I tried it on a later verison and got:

pObject=<OBJECT REFERENCE>[1@%ZEN.proxyObject]
+----------------- general information ---------------
|      oref value: 1
|      class name: %ZEN.proxyObject
| reference count: 2
+----------------- attribute values ------------------
|           %changed = 1
|%data("access_token") = "4SDFDSFDSF-aSDASDASD"
|%data("expires_in") = 3300
|%data("refresh_token") = ""
|     %data("scope") = "sms_send"
|%data("token_type") = "Bearer"
|             %index = ""
+-----------------------------------------------------
Eduard Lebedyuk · Feb 22, 2019 go to post

That is a fairly common problem - the need to compare two different codebases. It could be a test and prod server or just different production servers. Anyway I usually compare them like this:

  1. Install source control hook (I prefer cache-tort-git udl fork but any udl based will do) on all affected servers.
  2. Export everything from the base server (with oldest sources) using source control hook into a new repostory.
  3. Commit this state as an initial state.
  4. Clone the repository with initial state to the other server.
  5. Export everything from the other server into the repository.
  6. Commit again (if you have more than two parallel codebases you may need to branch out).
  7. Use any commit viewer to see the difference (I prefer GitHub/GitLab).