All
Accepted answers
Dmitry Maslennikov · Sep 30, 2022 go to post

Protocol and its implementation

I've looked at the realization in IRIS, and it does not support any streams, So, any call of Write, is like a complete message and it accepts only string.

Dmitry Maslennikov · Sep 30, 2022 go to post

Why do not pass a stream to %ToJSON, and then Write the stream content?

Set stream = ##class(%Stream.TmpCharacter).%New()
Do message.Msg.%ToJSON(stream)

But, I'm sure that you will not be able to send the whole big stream as a whole message. You still have to split it and receive it knowing that it's not complete.

Dmitry Maslennikov · Sep 28, 2022 go to post

There are no settings that would limit it.

There are various reasons, which may cause the low reading speed. It may also depend on the kind of disk HDD or SSD. Multiple processes can read simultaneously. If you manage to split one read to multiple processes, you may achieve better speed. Copy file, not a right way to measure the speed, because with Cache your data can be spread among disk wider than you think, and it will take time to read it. Defragmentation can be the reason of slow speed.

You may ask InterSystems WRC to help you to investigate this. Or may contact me, directly I can help with it as well.

Dmitry Maslennikov · Sep 27, 2022 go to post

I'm sure that error should point to the particular class, where it found the issue with storage

btw, you may report your issue here.

Dmitry Maslennikov · Sep 27, 2022 go to post

So, probably you missed some of the classes, if you use VSCode, import all the code and compile manually and export with VSCode. So, all classes will get storage correctly.

Dmitry Maslennikov · Sep 27, 2022 go to post

Why did you remove storage from the classes?

It's an important part of the code. And ZPM mentioned this. All classes have to have storage defined in the code. It will not allow installing without storage.

Dmitry Maslennikov · Sep 27, 2022 go to post

it would be a huge waste of resources with higher complexity of the development process. I would not recommend it this way at all.

A much better way is to use named volumes, where you could munt data, which you would like to keep between restarts of containers, but not easy way as well requires a lot of attention to many staff in IRIS, or use with Durable %SYS

Dmitry Maslennikov · Sep 25, 2022 go to post

It's essential to keep in mind, that ObjectScript code has to be compiled. And actually, there are two ways of doing this.

  • Build stage, mostly with using Dockerfile
  • Runtime stage, start vanilla IRIS and build application there.

Yes, docker build can be slow, but the issue here is that IRIS starts in fully functional mode, and it takes too much time to start. While it would be enough if IRIS would be started with very minimal functionality (no spare wrtitedaemons, no hot jobs for Interoperability, and so on)

Building an application in runtime does make not so much sense. First of all, it's not so simple to make this happen. And second, the logging of this is quite complicated as well. So, it makes the debugging of the build process much more difficult. And in the case of a significantly big project, it still will take a lot of time to be built, but with less understanding of the progress.

The ability to fail fast is quite important, and it's a sign to a developer, that something is odd. Building images with BuildKit, have a different way of logging, and it may not show the real reason for failure, but you can turn on the full logging this way

docker-compose build --progress plain

If your project uses ZPM, so, most probably you would have lines like this

RUN --mount=type=bind,src=.,dst=. \
  iris start iris && \
  iris session iris "##class(%ZPM.PackageManager).Shell(\"load /home/irisowner/$MODULE -v\",1,1)" && \
  ([ $TESTS -eq 0 ] || iris session iris "##class(%ZPM.PackageManager).Shell(\"test $MODULE -v -only\",1,1)") && \
  iris stop iris quietly

ZPM's Shell accepts three parameters

  • executed command itself
  • Fail on error
  • Halt on exit

So, you can just change the second parameter (the first 1) to 0, and the build will be successful in any case

Dmitry Maslennikov · Sep 23, 2022 go to post

And unfortunately some (almost all) of the images there are already outdated and replaced with new ones

Dmitry Maslennikov · Sep 23, 2022 go to post

An idempotent operation can be performed multiple times without changing the result beyond the initial application. Put more simply, an operation is idempotent if you can repeat it over and over again without causing any unwanted side effect or harm to the system, always producing the same result.

I agree with these, and, that's not what actually happens with MergeCPF.

We have two different options

  • CreateNamespace
  • ModifyNamespace

But, to make MergeCPF truly follow idempotent, it should be only one like CreateOrModifyNamespace or to make it declarative way just Namespace. Where in this case, I will get what I defined in any case

I have this file

[Actions]
CreateDatabase:Name=DEMO,Directory=demo
ModifyNamespace:Name=DEMO,Globals=DEMO

I have no NameSpace Demo, and after merge, I will get the new Database, But No Namespace at all. And no failure, merge said all good.

Yeah, probably Namespace not a good example, not so variants to change it on the fly, but what about User, I would probably want to change password with the next deploy, or changes Roles. Or CSP Application , and so on. 

In the current state of work, the only correct way is to always define Create and Modify, so, it will always create the desired resources, and Modify, if I would need to update already created previously. This does not look like an obvious way to go, at all.

Dmitry Maslennikov · Sep 23, 2022 go to post

Right %Installer is still in place, and we have it in some of the templates.

So, what would I like to see in with iris merge

  • error code 0, for success
  • --dry-run flag, to have an ability to somehow validate the merge process, what will happen if I do merge
  • --verbose|-v flag, or by default, and --no-verbose to get the current behavior. Verbose output should show every single change made, similar to %Installer output, including plain changes not covered by [Actions]
  • --fail-fast by default, fail on any error that happens, CreateNamespace on existing namespace should fail, and so on.
  • --ignore-errors for the current behavior, when some issues are ignored completely. But it still has to log in verbose output.
Dmitry Maslennikov · Sep 22, 2022 go to post

Do you know that you can just do ZWrite in objectgenerator method, and see the all variables?

So, you would find %qstruct variable, where you could check all the compile flags

Dmitry Maslennikov · Sep 22, 2022 go to post

When I use docker in any project, I expect that any Docker build, will do the job as I expected, or fail-fast. We already have a bunch of templates with docker and zpm. And we have lines like this in Dockerfile

RUN --mount=type=bind,src=.,dst=. \
    iris start IRIS && \
	iris session IRIS < iris.script && \
    iris stop IRIS quietly

So, I expected that I would just add merge there, so I could get what I want

RUN --mount=type=bind,src=.,dst=. \
    iris start IRIS && \
    iris merge IRIS ./merge.cpf && \
	iris session IRIS < iris.script && \
    iris stop IRIS quietly

And what I found, is that even though I use start, session, and stop from the same iris command, always had no issues, and suddenly, merge failed my build, even though the output says, it's a success. I did not expect that the error code would be just a random magic number, while the whole Linux world uses 0 as a success, and even Windows uses 0 as success, but for unknown reasons, InterSystems uses 3 as success in this case, and did not think it's kind of important and worth to at least mention in the Documentation. 

So, I have no idea how you debug the issues, there, probably you can type with no typos, you can't do any mistakes, but people do mistakes, and it should cover these situations, and indicate that you doing something unexpected. 

As soon as iris merge does not return 0, does not have any verbose output, and ignores issues, I strictly do not recommend this in development and in production for sure at all, to anyone. 

Dmitry Maslennikov · Sep 22, 2022 go to post

You have to keep in mind, that nothing is persistent on containers.intersystems.com at all, InterSystems can delete anything from it at any time. So, if docker says no manifest, it means, the image is already deleted, and you have to find which version is the latest, and yes, when you omit the tag, docker tries to find the tag latest, and InterSystems don't use it as well. 

And unfortunately, InterSystems did not offer any useful way to find the versions available there. So, I made an extension to Docker Desktop, where you can easily find it (refresh manually), copy an image name with tag and past to where you need it

Dmitry Maslennikov · Sep 19, 2022 go to post 
ClassMethod makeComplement(dna As %String) As %String [language = python]
{
  return dna.translate(dna.maketrans("CGAT","GCTA"))
}
Dmitry Maslennikov · Sep 19, 2022 go to post

The oldest version I found is 2007, I checked on it, works well. If you have a version older, could you say the version?

Dmitry Maslennikov · Sep 16, 2022 go to post

Yeah, node-red was my first attempt. And now, I only found n8n somehow, but did not find zapier and make, which looks like a bit better.

And to do it for make will be even more challenging for me, because it requires .Net for plugins, while node-red, n8n, and zapier are with NodeJS

Dmitry Maslennikov · Sep 14, 2022 go to post

Yeah, I know, It's not going to be a popular idea, but, forget about ObjectScript.

You can still do any projects with IRIS, but try to implement it in any different supported language. Python is the latest trend, select any project and try to implement it. Do not use Embedded Python, you will still be stuck with ObjectScript. Use DB-API for instance, probably in Django with my Django driver, or find another way.

Nodejs, I've published a few projects on it already, with my own driver as well. the latest project for the contest it's a NodeJS based application, where I added support for IRIS.

I've also had projects on Golang, and Rust languages.

InterSystems also supports Java and .Net, you may try to use the.

ObjectScript as a language is quite simple and limited, when you will use other languages, you will get wider knowledge about programming, when much stuff possible to do simpler than with ObjectScript. Applications are not only about fast access to the data, it's running the application itself too. And in some cases, it could much faster development to it with other languages than with ObjectScript.

Dmitry Maslennikov · Sep 5, 2022 go to post

You marked wrong lines, which would alert you in this case, any mentioning of Write Daemon is

And in your case, many alerts that Write Daemon completed a pass. And most it means, that your disk is too slow. So, check the disk queue and how fast it works.

And WIJ file more than 8 GB, is bad sign, for sure.

Dmitry Maslennikov · Sep 1, 2022 go to post

Background tasks are an internal feature mostly for System Management Portal, and in most cases also not supposed to be called by you.

JOB, is a complete different story. with this command, you have control over many aspects of how to run this process in the background. 

You can check $Test, (be careful, and do right after JOB) which says if your process even started in the background

$ZChild returns that job ID, you can check it in SMP if it's running

$Data(^$JOB(child)) will say if your child process is still alive.

You may have up to 25 (can be less) background jobs per process. So, store the child process ID after each call of JOB command

You may redirect output from that process to some file, by passing principal-output parameter

With ^$JOB and $ZChild, you at least can wait until the process is finished its work. With an endless loop and reasonable HANG 

Dmitry Maslennikov · Aug 29, 2022 go to post

So, it looks like you doing it a wrong way. 

If you need to build your application, then you have to do it with Dockerfile during the build stage, not when it is just up and running. And you still build some container before it.

In any case you may use this way, to wait until it's started

sleep 5; docker exec $CONTAINER /usr/irissys/dev/Cloud/ICM/waitISC.sh

Teah, not documented, internally used script, but nothing offered instead.

Contact me directly, I may help to review the build process, and give particular recommendations.

Dmitry Maslennikov · Aug 28, 2022 go to post

Nowadays I would not recommend plain CSP files at all, and for sure, and totally no ZEN. 

I see that you use Python in some cases, I may recommend you use Django in that case, I wrote a few articles about it recently.

Or just do REST with %CSP.REST class, and do plain web application on plain HTML and JS, or may use ReactDOM (my application as an example).

Dmitry Maslennikov · Aug 26, 2022 go to post

10 seconds may not be enough, in my case starting IRIS may take more than one minute.

Where do you use the sleep command, and what for?