Alexey Maslov · May 31, 2017 go to post

90 and 100 lines for serializers is quite the achievement.

At the moment I have a serializer (<glvn> to json string) and a deserializer (json string to <glvn>) of 40-60 lines of pure M COS code. Should run on any Caché version, was tested on 2012.2 and higher. Some proof that they really exist:

USER>d $system.CPU.Dump()
 
-- CPU Info for node maslov --------------------------------------------------
          Architecture: x86
                 Model: Intel(R) Core(TM) i5-4460  CPU @ 3.20GHz
< ... >
USER>d j2gSmall^zmawr

JSON2G^Wmgr Cache for Windows (x86-64) 2017.1
total time = 1.05795 avg time = .0000105795

G2JSON^Wmgr Cache for Windows (x86-64) 2017.1
total time = 1.898275 avg time = .00001898275

sJson <=> arr()? yes ; the result of conversion reversibility check

USER>zw sJson
sJson="{""BirthDate"":""1970-03-25"",""FirstName"":""Sean"",""Hobbies"":[""Photography"",""Walking"",""Football""],""LastName"":""Connelly""}"
 
USER>zw arr
arr("BirthDate")="1970-03-25"
arr("FirstName")="Sean"
arr("Hobbies",0)="Photography"
arr("Hobbies",1)="Walking"
arr("Hobbies",2)="Football"
arr("LastName")="Connelly"

This code is not my own development (while I contributed a bit), so if anybody wants it to be published I should redirect this request to main contributor(s).

BTW: Sean, was this JSON string

TestRawJson = "{""TestAllAsciiChars"":"" !\""#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]"_$c(142,143,144,145,146,147,148,149,150,151,152,153,154,157,158,159)_"...

 from your sample wrong escaped deliberately?

Alexey Maslov · May 27, 2017 go to post

I don't understand the difference between this two kinds of voting :) Which solution is the best, depends on many factors: if we need turbo performance, we'd take your approach, if not - %ResultSet based one. BTW, I guess that file dirs scanning is a small part of a bigger task and those files are processed after they have been searched and the processing takes much longer than the directory search.

Last 2c for cross-platform approach: the main place where COS developer faces problems is interfacing with 3d party software. As I was told by one German colleague, "Cache is great for seamless integration".

E.g., I've recently found that forcely resetting of LD_LIBRARY_PATH by Cache for Linux may cause problems for some utilities on some Linux versions. It's better to stop here, maybe I'll write about it separately.

Alexey Maslov · May 26, 2017 go to post

I'm voting for Rubens's solution as it is OS independent. Caché as a great sandbox in many cases, why not use its "middleware" capabilities? Developer's time costs much more than CPU cycles, and every piece of OS dependent code should be written and debugged separately for each OS that should be supported.

As to performance, in this very case I doubt if the recursion costs much compared to system calls. Anyway, it's not a great problem to replace it with the iteration.

Alexey Maslov · May 24, 2017 go to post

If one simply gets an older storage def from VCS, how Cache internal migration facility can help? The history of schema changes will be lost. It seems that only some ad hoc global level migration code can help in this case.

So this kind of "forced" schema migration (having some populated data which you don'want to loose) would usually be a pain.

Alexey Maslov · May 24, 2017 go to post

And what will happen, if one decides to revert his class def to a previous version with previous storage def state, having some data already populated using new schema?

It seems that there is no "good" choice between TS's two options, only some "bad" vs "worse", unless business logic is accurately separated from data and kept in different classes. In this case the probability of reverting data class def should be lower.

Alexey Maslov · May 24, 2017 go to post
  • Вопросы есть? Вопросов нет!
  • И называйте меня просто — товарищ Сухов!
  • Have you any questions? No questions!
  • And call me simply - comrade Sukhov

Quotes from the cult film "White Sun of the Desert" / "Белое солнце пустыни", 1970.

This film is traditionally watched by Russian cosmonauts before every space flight.

PS. Sorry for off-topic, Jon provoked me :)

Alexey Maslov · May 24, 2017 go to post

Thank you, Jon, you've answered my question.

Frankly, I've noticed that CPUPct object still exists in 2017.1.0, but didn't notice that it's no longer being tracked.

Alexey Maslov · May 24, 2017 go to post

Thank you, Jon, you answered my question.

Frankly, I've noticed that CPUPct object still exists in 2017.1.0, but didn't notice that it's no longer being tracked.

Alexey Maslov · May 19, 2017 go to post

Do you think that strange name is from MultiValue stuff?
If so, not sure if it's possible to define macros in MultiValue .

Alexey Maslov · May 19, 2017 go to post

Routines can't be named as "%zu2f%zt" because only the first letter of routine name can be "%". Maybe such "name" occured due to some encoding, having in mind that $char($zhex("2F"))="/"? If so, the original could look like:

$$zt^%zu/%zt

If so, a macro can be defined as:

#define YourMacro(%yourparam)   $$zt^%zu/%yourparam

Alexey Maslov · May 19, 2017 go to post

Looking at the console log I can see the following which I presume implies it is using large pages?

Definetely yes.

Graham, if you feel that your question is answered please don't hesitate to mark it answered smiley

Alexey Maslov · May 17, 2017 go to post

During Cache backup it appears that all the available memory on the server is being used.

It's usually not a problem as the more memory is used for buffering the quicker file i/o operates. Only free memory is used for this purpose, so memory allocated by users' or system processes should not be swapped. Please add more details, why it turned to be a problem in your case?

PS. Double check in console log whether Cache allocates its shared memory segment using large pages as it guaranties that it is totally allocated at Cache startup and will neither be expanded nor swapped afterwards.

Alexey Maslov · Apr 28, 2017 go to post

Question may have more than one right answer.
Several answers may add some useful info to each other.
Complicated question may even collect several competiting answers. Who will be a judge?

Alexey Maslov · Apr 28, 2017 go to post

If you are curious, you may try to do it following John's hint, but if you didn't catch the idea, better do not.

Alexey Maslov · Apr 27, 2017 go to post

Raghuram,

I wrote my answer before reading yoyr comment. In this very case it seems that it's nothing should be done with GUID.

Happy testing!

Alexey Maslov · Apr 27, 2017 go to post

While it's easy to change GUID following John's hint (if you understood the code, you already know which global node to kill, and it will be recreated on the next call), I wonder why do you want it? It seems that your instanse have migrated to another host, hense you may want to save all its requisites (including GUID) untouched. This GUID is not used by Cache itself, but may be used in some app data fields, so you may loose app level integrity.

PS. If you didn't understand the code, or are cautious to change system globals, you'd better deinstall Cache and re-install it into the same folder. System DBs would be totally rewritten while your app DBs as well as cache.cpf would be left as is.

Alexey Maslov · Apr 26, 2017 go to post

Neither MERGE nor ^GBLOCKCOPY move the global: they are only copy it, unless somebody KILL it's old instance after copying.

Alexey Maslov · Apr 25, 2017 go to post

John, it seems that inquiring folks rarely accept answers, maybe because of their rare visits to this site.

With voting feature (like some sites have) it would be possible to "auto-accept" the most voted answer(s) after some reasonable time, e.g. after two weeks.

Alexey Maslov · Apr 25, 2017 go to post

Hi, Richard.

...to have cache run as a different user to access that macro...

Running Caché for Windows using a dedicated user account (so called service account) may have some other advantages:

  • Caché processes get ability to use MS network resources (shared folders), e.g. Caché Backup can be performed directly to remote folder; 
  • Kerberos authentication can be used.

So long story short just going down the libre office route.

Why do you expect that libre office route would be shorter? Do you plan to use another (not $zf(-1,...)) approach?

Alexey Maslov · Apr 24, 2017 go to post

Thank you, John, for this reminder, while I prefer to deliberately enable auditing of all events for the simple reason: a few added-on disk IOPS, as well as 1GB+ of disk space for audit database (even on high loaded Caché instances) do not seem to be a great price for an ability to trace not predicted cases in production.

We even met a prospect's requirement that any audit database write failure should be dealt as a critical error with a notification of admins by all means, even with shutting the Caché down!

Alexey Maslov · Apr 21, 2017 go to post

Sagun, this code would fail with <WRITE> error as after opening the file for reading:

 ..o resfile:"RSK\CP874\"

it tries to write into it:

 ...w $g(^ztonEDI("TXT","OUT-DATA",ind,fileno,"SEQ",seq))_$c(13),$c(10)

What this code needs as the first remedy is a change of the Open command to:

..o resfile:"NWK\CP874\"
Alexey Maslov · Apr 21, 2017 go to post

... and second thing is to insert a Close command before the correspondent Open, e.g.

..c resfile o resfile:"NWK\CP874\"

even if you are quite sure that the file is closed at the moment of opening. The reason is to avoid the cases when your program have failed with error without closing the file, so its open parameters (at least, the translation table setting) keep unchanged despite of subsequent Open command. Such cases often happen during development / debugging, when the error trap code is completely switched off or greatly simplified.

Alexey Maslov · Apr 20, 2017 go to post

It's better to use stream wrappers instead of open/use directly

Just because of higher abstraction level of streams; nothing special inside: just the same Open/Use/etc COS commands. So, choosing one or another way of working with files can't be the source of errors itself.

By the way, `open file:("NWK\table\")` is a proven and documented way of translation table setting. AFAIK, its only limitation is that it works with Open command only rather than with Use command.

Alexey Maslov · Apr 20, 2017 go to post

Eduard, can you explain the main difference between Sagun's method of translation table setting:

 open file:"RSK\CP874\"

and yours in this very case? The latter is a piece of code of  %Stream.FileCharacter.cls which actually sets a table:

 If (table'=0) && (i%CurrFile'="") {
   Set io=$io
   Use i%CurrFile
   $$$SETIO(table) ; -> Do $zutil(96,18,2,table)
   Use io
 }

It seems that it's some other problem, perhaps a bug.

Sagun, if you provide us with a small piece of your code where you open the file, use it, and write it, it would be easier to say something.

Alexey Maslov · Apr 19, 2017 go to post

Hello Eduard,

Following your 1st link I've failed to find any info on 7-zip usage. It was about Libre Office stuff.

No problems with 7-zip for Windows (its native OS), not worth to mention that 7z format compression method(s) of its last versions can be unsupported (and uncompressed) with old ones. If using zip format (even with 7-zip), your are quite safe.

p7zip, its Linux branch, which I tried several years ago, performed very poor as it was not multi-threaded (as 7-zip was for years). So we decided to drop it mostly for this reason.

One little hint about 7-zip for Windows: we use its 7za.exe build in our deployment and update procedures as it doesn't need installation and can be just dropped in any place before usage.

Alexey Maslov · Apr 14, 2017 go to post

Looks like, do you use integration in Studio?

Yes, I do.

...maybe it working so long, and you did not wait for the result. 

Not too long, just a few seconds. It provided an empty report, and AFAIR there was some record in its internal log that no problems were found.

Alexey Maslov · Apr 14, 2017 go to post

I've tried it with Cache 2017.1.0. The first impressions are:
- It completely ignored all "blockers" in the class with 300+ methods, while there was a many of them it should not pass by: 'not procedureblock', legacy flow control, etc; maybe some internal limit was exceeded.

- On the smaller class it popped rather funny messages, e.g. 

  1 blocker
        1 Method declared as "ProcedureBlock = 0"
User.test.cls(+6): ClassMethod RestVars(zzzzzzsource As %String, bOref As %Boolean) As %Status [ ProcedureBlock = 0 ]
  2 major
       2 Usage of QUIT to exit a method
User.test.cls(+24): quit $$$OK
User.test.cls(+28): quit $$$ERROR($$$CacheError, $ze)

Well, 'ProcedureBlock = 0' can be considered not a good style, but what if I need it in some special case? This very method restores local variables stored in a global, so it would loose all its sense be it 'procedureblock'. IMHO, there should be a facility to protect such special case methods.

'Usage of QUIT to exit a method': QUIT is standardized command to exit methods/routines/etc. IMHO it's better than RETURN as it prevents coders to exit their methods from inside the loops and try/catches constructs, so it encourages the modular coding: each module (method) should have one enter and one exit.

So, many of such rules seem to be subjective. Without a facility to customize the rules this linter seems to be no more than a demo of some commercial product.

P.S. As I noticed later, it never clears 'mgr/Temp/' sub-folders it creates.

Alexey Maslov · Apr 14, 2017 go to post

I just wanted to add that some occurrences of `quit:$quit value  quit` idiom was probably inherited from legacy coding practices, as Caché allows calling methods/functions using DO command (thus ignoring the value being returned).

From the other hand, using of `quit:$quit ""  quit` construct looks like an attempt to amend erroneous $$ calls of methods/routines, which did not return any value by their initial design. One might say that such design was initially wrong, as any method should return something, at least %Status which is always $$$OK.

Alexey Maslov · Apr 13, 2017 go to post

It may depend on local coding traditions. Just for example, quick search using Studio's <Ctrl-F> through one of our apps (comprised 1000+ classes) showed: 

 q:$q <lvn> q            ; Found 4501 occurrence/s in 90 file/s
 quit:$quit <lvn> quit   ; Found 126 occurrence/s in 28 file/s.
 if $quit                ; Not found.

So, it turned that about 10% of our classes used $QUIT. Not too many, but not negligible few.