Eduard Lebedyuk · Jun 1, 2018 go to post

What risk?

There are two distinct cases - where we have users/passwords in place or we don't.

In the first case container should use these passwords (via Durable %SYS) and they shouldn't be be superseded by anything happening with a container change.

In the second case we have some empty application - no users, no data and so specifying random password only adds unnecessary steps down the line.

There is one case where we need to force our container user to create new password - when we are:

  • supplying a complete application
  • don't have control over how it is deployed
  • passwords are stored inside the container

in this case yes (when all conditions are met), password should be scrambled, but this situation is wrong on itself (mainly in storing passwords inside the container), and should be resolved by other means.

Eduard Lebedyuk · Jun 1, 2018 go to post

On non-prod servers we don't need that - developer/ci server should be able to pull the image and work with it.

On prod Durable %SYS should be used with secret password and not changed with each new app version, but rather through a separate process (once a year, etc).

Eduard Lebedyuk · May 30, 2018 go to post

This feels more like a DevOps process, where coding is more acceptable. Anyway, maybe you need to ask a separate question on configuration synchronization best practices?

Eduard Lebedyuk · May 30, 2018 go to post

1. Import task has these two qualifiers:

displayerror
displaylog

They do not affect import.

2. No. If you check exported xml, you can see that exported tasks do not contain IDs, so, on import they could not overwrite existing tasks. Not sure about how correlate/next work on ID fields btw.

Exporting and importing what is currently present is the best way. 

It's the easiest way. The best way would be to write and document  a script (ObjectScript code) that transforms base instance into what they need. This way changes are documented and adding another server is easy.

Eduard Lebedyuk · May 30, 2018 go to post

1. Execute this to get a list of available flags and qualifiers:

Do $System.OBJ.ShowQualifiers()
Do $System.OBJ.ShowFlags()

Qualifiers are preferable to use.

2. I'd rather go with programmatic access. Write code that's:

  • Opens system task object
  • Modifies and saves it

What system task did you modify?

Eduard Lebedyuk · May 27, 2018 go to post

Sure.

  1. Open Chrome
  2. Go to Settings | Manage search engines...
  3. Scroll to the bottom of the window
  4. In Add a new search engine, enter InterSystems
  5. For Keyword, enter i
  6. For URL, enter one of:
  7. Click Done

After that you'll be able to search InterSystems documentation in Chrome address  bar by typing:

i keyword

and pressing Enter.

Eduard Lebedyuk · May 25, 2018 go to post

Do you want to pull all the rows each time?

If so,  just clear “key Field Name”  setting and BS would process all rows every time it's run.

Eduard Lebedyuk · May 25, 2018 go to post

Compilation is usually multi-process, I'd try single-process compilation and see how it works:

$SYSTEM.OBJ.Compile(.classes, "/multicompile=0")

Multicompile qualifier:

           Name: /multicompile
    Description: Enable use of work queue manager (optionally specify the number of workers if value >1)
           Type: string

UPD. Who throws a RESJOB? Do you want to kill compile process from external process (that was my idea)? Or do you want to terminate abnormally?  If it's the second you need to check errolog from Compile method and terminate the process explicitly:

do $system.Process.Terminate(, 1)

Also I wrote a series of articles on managing Continuous Delivery process. Check it out.

Eduard Lebedyuk · May 25, 2018 go to post

Use INSERT OR UPDATE:

INSERT OR UPDATE INTO  ocsGUMCHECDINFO
(CODE,DODATE,NAME,DAY,PURPOSE,REQUIREMENT,BEFORENOTE,AFTERNOTE,CNT,REPORT,CONSENT,CONTACT)
VALUES (#{code},#{doDate},#{name},#{day},#{purpose},#{requirement},#{beforeNote},#{afterNote},#{cnt},#{report},#{consent},#{contact})
Eduard Lebedyuk · May 25, 2018 go to post

About XLSX files, they are not one XML but a ZIP archive with arbitrary large number of xml files referencing each other.

For example there is a separate XML file for cells' text values.

It's possible to extract information from it but it seems a rather daunting task.

Here's a example of a direct manipulation of a Docx file.

Eduard Lebedyuk · May 25, 2018 go to post

There are several approaches available:

1. Automatic conversion of XSL(X) to CSV is possible using LibreOffice. Here's an article on how to do that.

2. Use Java library, such as Apache POI to traverse the cells and send results back. To send results back quickly and efficiently you can use String[] type and fill it with $lb built on Java side. String[] would become %ListOfDataTypes in Cache and $lb would remain $lb.

I've done some preliminary work but didn't get around to finishing or publishing it so here's a sample code that outputs list to console:

package isc.poi;

import org.apache.poi.hssf.usermodel.HSSFRow;
import org.apache.poi.ss.usermodel.*;
import org.apache.poi.ss.usermodel.Row;

import java.util.ArrayList;
import java.util.Iterator;
import java.io.File;

import static org.apache.poi.ss.usermodel.CellType.*;

public class Main {

    public static String ROWSEPARATOR = "\t\t\t";

    public static void main(String[] args) {
        try {
            Test1();
        } catch (Exception ex) {
        }
    }

    public static String[] Test1 () throws Exception{
        ArrayList<String> list = new ArrayList<String>();

        File file = GetFile();
        Workbook workbook = WorkbookFactory.create(file);
        Iterator<Sheet> sheetIterator = workbook.sheetIterator();
        while(sheetIterator.hasNext()){
            Sheet sheet = sheetIterator.next();
            String name  = sheet.getSheetName();
            String value = null;

            Iterator rows = sheet.rowIterator();
            while (rows.hasNext()) {
                Row row = (Row) rows.next();

                for(int i=0; i<row.getLastCellNum(); i++) {
                    Cell cell = row.getCell(i, Row.MissingCellPolicy.CREATE_NULL_AS_BLANK);
                    if (cell.getCellTypeEnum() == FORMULA) {
                        switch(cell.getCachedFormulaResultTypeEnum()) {
                            case NUMERIC:
                                value = String.valueOf(cell.getNumericCellValue());
                                break;
                            case STRING:
                                value = cell.getRichStringCellValue().getString();
                                break;
                        }
                    } else {
                        value = cell.toString();
                    }
                    list.add(value);
                    ///System.out.print("'" + cell.toString() + "'"+" ");
                }
                list.add(ROWSEPARATOR);
                System.out.println();
            }


            /*for (Row row : sheet) {
                for (Cell cell : row) {
                    System.out.print(cell.toString()+" ");
                    //int i=1;
                }
                System.out.println();
            }*/
        }
        String[] result = list.toArray(new String[list.size()]);
        return result;
    }

    public static File GetFile () {
        File file = new File("D:\\Cache\\POI\\Book1.xlsx");

        return file;
    }

    public static Object Test(Object in)
    {
        String[] ret = new String[1];
        ret[0] = "144";
        return ret;
    }
}
Eduard Lebedyuk · May 23, 2018 go to post

I'd run a BS with SQL inbound adapter called every 86400 seconds. Query is some SELECT Count(*) ...

To get optimal WHERE condition you'll probably need to answer these questions:

  • Are records immutable?
  • Can they be updated?
  • Can records be deleted?
  • Can records be inserted for a past/future dates?

If it's a local table you can run BS without adapter and just use embedded SQL.

Eduard Lebedyuk · May 23, 2018 go to post

Redirect is easy to do in REST:

set %response.Redirect = "url"

And that's it. The problem is forcing a client to redirect to a POST verb and also providing the body.

Eduard Lebedyuk · May 23, 2018 go to post

There are two use cases here:

  • File is tied to a specific object (for example you have "Document" class and it has "scan" file). In that case you can use %FileBinaryStream property - as before getting the file you would probably open "Document" object first
  • File is not tied to a specific object. In that case you can create a separate table "Files" that stores
    • link to file as a FileBinaryStream
    • hash
    • displayed file name
    • file path
    • user who uploaded the file
    • extension
    • size
    • any other attributes you need

It would always work faster compared to OS search.

Other notes:

  • Files are immutable - if you're building an application where user can edit files, it's usually preferable to have immutable "files" objects and just create new file versions.
  • File size limits - always define and check for maximum size.
  • Extension limits - limit extensions user can upload.
  • Storage - if it's a low volume inserts (<1000/day) store files in a folder = date, otherwise generate a new folder for each new thousand of files. These approaches can be combined: date/1, date/2 ...
  • Hash name - I often store files where their OS name is their hash. This way I can quickly validate that file is valid and also it  solves the problems with non-latin characters.
  • Never store files under names supplied by user. Acceptable filenames are: guid, hash (integer id should also be avoided).
  • GZIP - in some cases using GZIP streams can save on space, especially if it's a text file. For example XML envelopes and such.
Eduard Lebedyuk · May 22, 2018 go to post

You can concatenate statuses with:

set sc = $$$ADDSC(sc1, sc2)

Or append statuses with:

set sc = $$$ADDSC(sc, sc2)
Eduard Lebedyuk · May 22, 2018 go to post

That's orefs currently in memory.

I need the number of instantiations, preferably by class.

Objects are created and destroyed on a second stage.

Eduard Lebedyuk · May 22, 2018 go to post

Try to check who calls GetBatchHeader and what it returns.

At a glance, I think it's called from GetBatchObject method of EnsLib.RecordMap.Service.BatchStandard class:

Set tStatus = $classmethod(..BatchClass, "GetBatchHeader", pIOStream, pTimeout, .pBatch, .pLookAhead)
If $$$ISERR(tStatus) Quit

Check that tStatus is not an error, fix GetBatchHeader it if it's an error.

Eduard Lebedyuk · May 21, 2018 go to post

As I originally said

Note that changing value back would still mark property as modified.

Any change (except immediate same value) would set m%PropName to 1.

I think that checking m%PropName and only if it equals one additionally checking GetStored would work faster.

Eduard Lebedyuk · May 21, 2018 go to post

Thanks. I've read Config.Mirrors docs, got the memo that it shouldn't be used to edit, and only recalled ^MIRROR routine. SYS.Mirror and %SYSTEM.Mirror would be way better.

Eduard Lebedyuk · May 21, 2018 go to post

You can determine that in run time with:

  • $system.CLS.GetModified(oref) to ger if oref was modified
  • m%property to get if property changed

Here's a simple example of how it all works

 Class User.Person Extends (%Persistent)
{

Property Name As %String(DISPLAYNAME = "Name");

Property Age As %Integer(DISPLAYNAME = "Age");

/// Create one user
ClassMethod Recreate() As %Status
{
  do ..%KillExtent()
  set person = ..%New()
  set person.Age = $random(100)
  set person.Name = $random(100) 
  quit person.%Save()
}

/// do ##class(User.Person).Test()
ClassMethod Test()
{
  $$$QuitOnError(..Recreate())

  set person = ..%OpenId(1)
  do person.ModificationState()

  set person.Age = $random(100)
  do person.ModificationState()

  set person.Name = $random(100)
  do person.ModificationState()
}

/// Show current object state
Method ModificationState()
{
  write "Object modified: ", $system.CLS.GetModified(), !
  write "Name prop modified: ", ..IsNameModified(), !, !
}

/// Get name property modification status
Method IsNameModified() As %Boolean
{
  quit m%Name
}

}

Executing in terminal:

>do ##class(User.Person).Test()
Object modified: 0
Name prop modified: 0
 
Object modified: 1
Name prop modified: 0
 
Object modified: 1
Name prop modified: 1

Note that changing value back would still mark property as modified.

Eduard Lebedyuk · May 21, 2018 go to post

It's a url not a file so you need to use OpenURL for that:

set reader=##class(%XML.Reader).%New()
set httprequest=##class(%Net.HttpRequest).%New()
set httprequest.SSLConfiguration="...."
set status=reader.OpenURL("https://.....",,httprequest)

But as @Robert Cemper pointed out it's a dtd file and so would not be parsed by XML reader.

If you want to generate classes automatically from XML schema, you'll need XSD files. Documentation.

Eduard Lebedyuk · May 20, 2018 go to post

What's your current locale? If it's not Chinese you may want to change it. Note that it significantly affects system behavior and you should understand the implications in such a change.

???? often indicates the data loss during encoding conversions consider this Cyrillic  example:

set t = "Привет"
 
USER>zzdump t
 
0000: 041F 0440 0438 0432 0435 0442                           Привет
USER>zzdump $zcvt(t, "I", "UTF8")
 
0000: 3F 3F 3F 3F 3F 3F                                       ??????
USER>zzdump $zcvt(t, "O", "UTF8")
 
0000: D0 9F D1 80 D0 B8 D0 B2 D0 B5 D1 82                     Ð.Ñ.ивеÑ.

Note the difference between question marks and the last output. Last output is just encoding translation, but the first conversion - into questions completely lost the information, so you need to look deeper.

You can try to modify TCP/IP IO table via ^NLS utility in %SYS namespace

do ^NLS
 
 
1) Display current locale
2) Select defaults
3) Change locale
4) Display loaded settings
5) Advanced

NLS option? 2
 
1) Internal tables
2) I/O tables
3) CSP files
4) Date, time and number formats
 
Category of defaults? 2
 
Items marked with (*) represent the locale's original default
 
I/O table              Current default
---------------------  --------------------
 
1) Process             RAW (*)
2) Cache Terminal      UTF8 (*)
3) Other terminal      UTF8 (*)
4) File                UTF8 (*)
5) Magtape             UTF8 (*)
6) TCP/IP              RAW (*)
7) System call         RAW (*)
8) Printer             CP1251 (*)
 
I/O table: 6
 
 1) RAW (*)
 2) UTF8
 3) UnicodeLittle
 4) UnicodeBig
 5) CP1250
 6) CP1251
 7) CP1252
 8) CP1253
 9) CP1255
10) CP866
11) CP874
12) EBCDIC
13) KOI8R
14) Latin2
15) Latin9
16) LatinC
17) LatinG
18) LatinH
19) LatinT
 
Selection for TCP/IP: 1 =>