Clear filter
Article
Evgeny Shvarov · Oct 27, 2019
Hi Developers!
Those who use Dockerfile to work with InterSystems IRIS often need to execute several lines of ObjectScript. For me, this was a game of "escaping this and that" every time just to shoot a few commands on ObjectScript to IRIS. Ideally, I'd prefer to code ObjectScript without any quotes and escaping.
Recently I found a nice "hack" on how this could be improved to exactly this state. I got this from @Dmitry.Maslennikov's repo and this lets you use Objectscript in a way as you would type it in IRIS terminal.
Here is what you have in dockerfile:
///
COPY irissession.sh /
SHELL ["/irissession.sh"]
RUN \
do $SYSTEM.OBJ.Load("Installer.cls", "ck") \
set sc = ##class(App.Installer).setup()
# bringing the standard shell back
SHELL ["/bin/bash", "-c"]
CMD [ "-l", "/usr/irissys/mgr/messages.log" ]
///
The trick is that before executing ObjectScript we call irisession.sh which does the following:
1. starts IRIS
2. Calls some "standard" ObjectScript commands and brings control to Dockerfile to let you introduce lines in ObjectScript - "$@" instruction.
3. stops IRIS
Check the content of the irissession.sh:
#!/bin/bash
iris start $ISC_PACKAGE_INSTANCENAME quietly
cat << EOF | iris session $ISC_PACKAGE_INSTANCENAME -U %SYS
do ##class(%SYSTEM.Process).CurrentDirectory("$PWD")
$@
if '\$Get(sc) do ##class(%SYSTEM.Process).Process.Terminate(, 1)
do ##class(SYS.Container).QuiesceForBundling()
do ##class(SYS.Container).SetMonitorStateOK("irisowner")
Do ##class(Security.Users).UnExpireUserPasswords("*")
halt
EOF
exit=$?
iris stop $ISC_PACKAGE_INSTANCENAME quietly
exit $exit
This $@ expects one RUN command in Dockerfile which will contain only ObjectScript. If you need to use some more commands in Dockerfile bring the control back with standard shell:
# bringing the standard shell back
SHELL ["/bin/bash", "-c"]
CMD [ "-l", "/usr/irissys/mgr/messages.log" ]
This code is introduced into IRIS community template to let you start using ObjectScript in your IRIS solutions with accurate and readable code.
Learn how to use Github templates with IRIS in this video.
This looks like a convenient way to handle things. It would be good, though, if someone could explain what the various ObjectScript commands in irissession.sh are needed for. I also see no credentials being passed? Well, here are some comments:
do ##class(%SYSTEM.Process).CurrentDirectory("$PWD")
This is to make WORKDIR /opt/irisapp current for IRIS.
$@
Here we run the arbitrary ObjectScript in Dockerfile
if '\$Get(sc) do ##class(%SYSTEM.Process).Process.Terminate(, 1)
Here we check the status of the sc variable changed with status from ObjectScript in Dockerfile if it has an error and terminate IRIS in this case and fail the build.
do ##class(SYS.Container).QuiesceForBundling()
do ##class(SYS.Container).SetMonitorStateOK("irisowner")
Two methods to prepare IRIS operate in a container mode properly. I'm pinging @Luca.Ravazzolo to provide more details on it.
Do ##class(Security.Users).UnExpireUserPasswords("*")
This removes the password expiration because it's very annoying to change the password on every build. This line could be used for DEVELOPMENT MODE only. Please remove the line if you build the image for PRODUCTION. Thanks for that, Evgeny, that does clear up things a bit. I hope Luca responds as well about the Quiesce... and SetMonitorState calls. I get the impression that things relating to creating containers are still changing rapidly. I do find the backslash-escaped multiline commands annoyingly ugly; your post above made that a bit better, compared to what was there before. (I'm hoping some of the ugly boilerplate will be delt with by ISC "inside" the container eventually.) I'm going to play around with your template a bit when I get the time! Hi Evgeny,
As clever as this hack is, I strongly recommend against this approach. This is a fun tool to write but a dangerous tool to hand to someone who doesn't understand all the factors at play well enough that they could have written our own. There are a few problems lurking here just beneath the surface, and when taken together they mean that the SHELL directive in Dockerfiles is not our friend.
Error handling
irissession is a tool intended for human, interactive use. In containers we often make use of it to "bootstrap" some code into IRIS, but there are caveats here. The Docker build daemon interprets exit status 0 as success, and anything else as failure. irissession reports errors by printing them:
USER>write fooWRITE foo^<UNDEFINED> *foo
This error doesn't change irissession's exit status, so it isn't caught by an image build daemon. Nor are bad status values, though I see that @Dmitry.Maslennikov 's caught that by checking "sc". :) Additionally, while these errors end an ObjectScript routine or method when uncaught, they do not end execution in irissession, which increases the risk that these errors get silently swallowed.
No multi-line blocks
The tool here is similar to pasting your text into irissession, which takes line-oriented input. This means that if someone looks at the format and uses blocks, they can get caught off-guard, when blocks like this get read as their component lines:
for i=1:1:10 { w !, "i: ", i}
That gets executed like so:
USER>for i=1:1:10 {
FOR i=1:1:10 { ^<SYNTAX>USER> w !, "i: ", i
i: 1USER>}
}^<SYNTAX>USER>
In simple cases, this is no problem to cram onto one line: "for i=1:1:10 w !, "i: ", i" In complex real-world cases, this paradigm breaks down quickly.
Backslashes
All this and we don't quite get the ability to paste in ObjectScript code from elsewhere. Because of the Dockerfile's syntax re: line endings, each line of ObjectScript that we'd put in still needs an extra backslash, except for the last one.
It's okay, though, because there's a better way.
A better way
Every time you cross the ObjectScript/shell boundary, you have to be careful. Since you have to be careful there, why not cross it as few times as possible? Your existing payload is good:
do $SYSTEM.OBJ.Load("Installer.cls", "ck")
set sc = ##class(App.Installer).setup()
But it could be better. If you're writing Installer.cls, why not add all the other calls there? All of these:
if '$Get(sc) do ##class(%SYSTEM.Process).Process.Terminate(, 1)
do ##class(SYS.Container).QuiesceForBundling()
do ##class(SYS.Container).SetMonitorStateOK("irisowner")
Do ##class(Security.Users).UnExpireUserPasswords("*")
Can be added into your installer.cls. There, you can engage in as-sophisticated-as-you-want error handling, with no need for backslashes and no need to keep a tight leash on your use of whitespace.
You'll still need to have a minimum of one pipe into irissession, until we add other features to streamline this process, but your code will be more readable, and your daily builds will be more robust.
InterSystems IRIS 2019.3 includes a lot of new features for containers, several of which are aimed at streamlining this kind of image building. Take a look at what we've committed to https://github.com/intersystems/container-tools, and what the 2019.3 Class Reference has to say about the SYS.Container class. These things are intended to make the ObjectScript-in-a-Dockerfile process less frustrating, and to demystify some of the things we do like "kill ^SYS("NODE")".
I also see no credentials being passed?
As of 2019.3, the IRIS images we provide have OS Authentication enabled. The net effect of this is that login to irissession is automatic for the right Unix user. This does not change anything else, as the new user can't log in in any other way in those images.
Customers who build their own images from our kits - available on the WRC! - can choose to use this feature or not. I have enabled and used that feature in older versions as well; it is really useful. I have a PuTTY setup with a key pair that starts the IRIS terminal in docker on a remote machine, without any password, but still safe. (I wanted to use my regular account name as well, so I had to do some additional setup in the container, but the principle remains the same.)
Anyway, that explains why there's no need for a password anymore, thanks!
By the way, I have not een an announcement that 2019.3 became an official release, did I miss something? (I also see no Studio for 2019.3, and the 2019.4 preview Studio download doesn't work.) Hi Conor!
Thanks for really wise suggestions. Agreed with everything. Two cents/comments:
1. I think we never want to code a lot of ObjectScript inside Dockerfile. Only a few lines to make some necessary configurations and the intention is to make the code more readable paying with some coding conditions (like sc hard-coded sc for error handling).
2. I am very supportive about putting all the possible setup activity into Installer.cls. But the issue is that not all the actively used setup tweaks are supported by %Installer format, e.g. RESTFul Web app. If you suggest to use <Invoke> - it's better to call method directly. So, here I'm putting some wishes for %Installer format approvements.
Always open for improvements.
I would rather expect that some kind of irissession.sh was implemented by InterSystems. It is a docker build process, InterSystems already added ##class(SYS.Container).QuiesceForBundling() which in fact I would expect it should be called automatically while IRIS stopping during docker build.
I'm against putting all of those container's only lines to Installer manifest, just because of Installer manifests can be used in non-container's environment as well. And those lines need only during docker build.
PS. With 2019.4 this line
do ##class(SYS.Container).SetMonitorStateOK("irisowner")
should not be used (even will give an error), as it is already as part of
do ##class(SYS.Container).QuiesceForBundling() Multi-line blocks
It's actually working. And it's because of code from RUN command goes to chosen SHELL as one line without line endings, \ at the end of the line, is not line ending it is line continuation. So, you can write code in Dockerfile visually like multi-line but it is no so.
Error handling
Any error in iris session will change exit code, so, any such error will fail docker build anyway.
So, with, such lines in my Dockerfile
SHELL ["/irissession.sh"]
RUN \
write !,"Start",! \
for i=1:1:10 { \
write !,"Test Loop",i \
}\
write !,"Finish",! \
write !, error
I will get this
Starting IRIS
Node: 24032930b1e3, Instance: IRIS
%SYS>
%SYS>
Start
Test Loop1
Test Loop2
Test Loop3
Test Loop4
Test Loop5
Test Loop6
Test Loop7
Test Loop8
Test Loop9
Test Loop10
Finish
WRITE !,"Start",! FOR i=1:1:10 { WRITE !,"Test Loop",i } WRITE !,"Finis
h",! WRITE !, error
^
<UNDEFINED> *error
%SYS>
ERROR: Service 'iris' failed to build: The command '/irissession.sh write !,"Start",! for i=1:1:10 { write !,"Test Loop",i } write !,"Finish",! write !, error' returned a non-zero code: 1
As you can see, for loop working well, and UNDEFINED error fails to build That kind of whitespace transformation has serious tradeoffs in ObjectScript. While they're not commonly used, ObjectScript does have cases where two spaces are significantly different than one (argumentless commands), or where a newline and a space are significantly different (legacy versions of no-block FOR, DO, etc). The way that docker parses these things in a Dockerfile, as part of SHELL, mangles that whitespace.
I would rather expect that some kind of irissession.sh was implemented by InterSystems.
The reasons I've listed here are why we have chosen not to implement a feature like this. It is not possible to embed exact ObjectScript syntax inside a Dockerfile. I recommend, wherever possible, piping as few lines as you can into an irissession, and letting the routines or classmethods handle things from there. This is part of why QuiesceForBundling exists.
See this done in imageBuildSteps.sh on GitHub: https://github.com/intersystems/container-tools/blob/master/2019.3/official/iris/imageBuildSteps.sh#L53 Unfortunately, InterSystems officially does not offer how to deal with docker at all. Where articles from, let's say from you or somebody else, with the ways how to do it the Right way.
Comments to the repository https://github.com/intersystems/container-tools:
2019.4 already exists and published, but examples still only for 2019.3
ZSTART.XML in XML format. Why so, when it should be MAC file
quite arguing the decision, to load code in this place
endless loop with goto in the code
Dockerfile, with a line longer than screen size.
where QuiesceForBundling?
InterSystems say how to run InterSystems products with docker, while nothing about running customer's applications on IRIS with Docker.
What's happens here is, it is how the community works. I have a problem, I don't want to write to much in my Dockerfile. I wrote a lot of them, for many different projects. I need some universal way, for me. I did it, and I update my own template from time to time. Hi @Conor.Walsh! Thanks for sharing Container tools! Cool stuff!
Announcement
Anastasia Dyubaylo · May 8, 2019
Hi Everyone!New session recording from Global Summit 2018 is available on InterSystems Developers YouTube Channel:InterSystems Healthcare Showcase In this video several of our clients talk about how they are innovating with our healthcare technology.Takeaway: I can create innovative healthcare solutions with InterSystems' software.Presenters: @Jeff.Fried, @Todd.Winey, and Guest Speakers Diane Steidler from Partners Healthcare, George Rosello from Jackson Memorial Hospital, and Eric Widen from HBI SolutionsBig applause for these speakers, thank you guys! Additional materials to the video you can find in this InterSystems Online Learning Course.Feel free to ask any questions about the video in the comments of this post.Enjoy watching the video!
Announcement
Anastasia Dyubaylo · Jun 12, 2019
Hi Everyone!One more session recording from Global Summit 2018 is available on InterSystems Developers YouTube Channel:InterSystems IRIS in a Container This video covers all aspects of deploying InterSystems IRIS in a container. We will discuss capabilities, benefits, best practices, and our support for containers. To get the most from this session, attendees should be familiar with Docker containers.Takeaway: InterSystems IRIS is a first-class citizen of the container world.Presenters: @Saurav.Gupta, Louise ParberryAdditional materials to the video you can find in this InterSystems Online Learning Course.Enjoy watching the video!
Announcement
Jeff Fried · Jun 10, 2019
The 2019.2 version of InterSystems IRIS Data Platform is now Generally Available! Container images for InterSystems IRIS are available via the WRC's download site.The build number for these releases is 2019.2.0.107.0. InterSystems IRIS Data Platform 2019.2 is the first CD (continuous delivery) release of InterSystems IRIS. It has many new capabilities including:Addition of the IRIS Native API for Python and Node.js and relational access for Node.jsSimplified sharding architecture and flexible sharded schema designSupport for the new PowerBI connector for InterSystems IRISNew look in the Management PortalSystem security, performance, and efficiency enhancementsEnhancements to the InterSystems Cloud Manager These are detailed in the InterSystems IRIS 2019.2 documentation and release notes. As this is a CD release, it is only available in OCI (Open Container Initiative) a.k.a. Docker container format. The platforms on which this is supported for production and development are detailed in the Supported Platforms document. For more information on what a CD release is, review the new release cadence post on InterSystems Developer Community. Thanks, Jeff!When the Community version of InterSystems IRIS 2019.2 will be available on Docker Hub too? Will there be a Caché 2019-version released this year or will it stop at version 2018 ? Is it just Docker container now? will there be a classic Full Kits? The CD releases are only in containers. review the new release cadence post IRIS 2019.2 is available on Docker Hub.The line for Dockerfile is:
ARG IMAGE=store/intersystems/iris:2019.2.0.107.0-community
Announcement
Anastasia Dyubaylo · Jul 25, 2019
Hey Developers,
We bring you good tidings!
You have a great opportunity to promote your business with InterSystems! This is exclusively for Global Masters Advocacy Hub Advocates.
Be first to redeem our new prizes! Please welcome:
➜ "$1,000 Google Adwords Voucher for your Open Exchange application promotion"
Redeem this prize to promote your OEX application on Google Adwords!
We'll spend $1,000 on Google Adwords for your Open Exchange application with your keywords, description, and will set up the audience for you.
Requirements: The application should work on InterSystems IRIS/IRIS for Health or be a tool to manage/develop with IRIS.
Redeem this reward for 3,000 GM points.
➜ "Advertise your services, events or vacancies on Developer Community"
Redeem this prize to promote your services, events or vacancies on Developer Community!
Recently, we launched a new block on DC Site called "NEWS" (you can see it through all website pages on the right). You can suggest your services, invite to your events and promote your vacancies on the "NEWS" block for a week.
Note: Our designer will help you to prepare graphics for the announcement (if needed).
Requirements: Development services, events or vacancies should be related to InterSystems technology.
Redeem this reward for 2,000 GM points.
➜ "Open Exchange project promotion on Developer Community"
Redeem this prize to promote your OEX project on the Developer Community!
A banner with a clickable link to your project will be shown for all DC visitors. More details please find in this post.
Redeem this reward for 1,000 GM points.
So!
InterSystems Developer Community has an audience of more than 30K people visiting every month. Let the world know about your applications, solutions, and services on InterSystems Data Platform!
Do not miss your chance! Earn more points and get all the prizes on Global Masters Advocacy Hub Hi Community,
Another prize on Global Masters that will help you promote your business!
➜ Your Company's Tag on InterSystems Developers
With this reward, you can get a tag for your Company on Developers Community and thus a description, a section with posts and your own subscribers.
Redeem this reward for 5,000 GM points.
Be first to redeem our new prize! In addition, we already have 2 companies that are part of the InterSystems Partner group.
Please welcome 2 new tags from InterSystems partner companies:
CaretDev
ITvisor
So! Who's next? Dear Developers,
Global Masters Advocate Hub gives you one more opportunity to promote your apps and solutions on InterSystems Data Platform!
➜ Publish Your Video on InterSystems Developers YouTube
Do you have the YouTube video which describes the tool, solution or experience related to InterSystems Data Platforms?
Increase the traffic to your YouTube videos ordering the Video Boost Pack for InterSystems Open Exchange Apps:
Promotion of the video on the InterSystems Developers YouTube Channel;
Listing in the monthly "InterSystems Developers Videos" digest. Here is an example;
Promotion on Global Masters and InterSystems Developers Social Media.
Redeem this reward for 1,500 GM points.
Let the world know about your solutions! 👍🏼 Hey developers,
One more Global Masters reward to promote your business:
⚡️ Your Webinar supported by InterSystems ⚡️
Would you like to hold a professional webinar for developers and tell about your solution/tool and your company services?
Redeem this prize for 3,000 points and we will help you to organize it!
➡️ What will you get? More details here.
Article
Evgeny Shvarov · Jul 18, 2019
## Our Pledge
In the interest of fostering an open and welcoming environment, we as contributors and manager pledge to making participation in InterSystems Developer Community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a professional setting
* We accept any claims and assessments regarding the behavior and features of InterSystems Data platforms, but we don't accept and don't discuss the behavior and level of service of any InterSystems departments and employees.
* We reserve the right to remove any posts (without giving a reason) that we believe do not provide useful information to members of the community.
* We reserve the right to restrict access to the community (without giving any reason) to individuals whose posts have been deleted multiple times.
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
## Scope
This Code of Conduct applies to any public contribution to InterSystems Developer Community which could be found on community.intersystems.com and openexchange.intersystems.com. Representation of community may be further defined and clarified by InterSystems Community management.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the [project team](https://community.intersystems.com/about#intsys-team). The Community team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The community team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
InterSystems Developer Community members who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the Community Management.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
[homepage]: http://contributor-covenant.org
[version]: http://contributor-covenant.org/version/1/4/
Announcement
Anastasia Dyubaylo · Mar 3, 2020
Hi Community,
On 8-9 June 2020 InterSystems UK & Ireland will be hosting the annual events for its customers, offering both technical and strategic insight content on the different ways to achieve successful digital innovation:
➡️ InterSystems UKI Developer MeetUp
➡️ InterSystems UKI Innovation Day
InterSystems is committed to your success, and to ensure we make the best use of your valuable time, we would like to understand more about the technology drivers and challenges your business is facing now, and how you are preparing beyond 2020.
With your help, we can ensure that the conference agenda features topics relevant to your business, with information that will maximize your IT investment and partnership with InterSystems.
The survey will only take an estimated 3 minutes to complete. Thank you in advance for your feedback!
SAVE THE DATE
Keep your eyes peeled for an official event invitation in coming weeks, but you can save the date and register your interest here.
Announcement
Anastasia Dyubaylo · Nov 29, 2019
Hi Developers,
The new video from Global Summit 2019 is already on InterSystems Developers YouTube:
⏯ Showcase: InterSystems IRIS Directions
This video provides additional information about the new and future directions for InterSystems IRIS and InterSystems IRIS for Health.
Takeaway: InterSystems IRIS and IRIS for Health have a compelling roadmap, with real meat behind it.
Presenter: @Jeffrey.Fried, Director of Product Management, Data Platforms
Additional materials to this video you can find in this InterSystems Online Learning Course.
Enjoy watching this video! 👍🏼
Announcement
Jamie Kantor · Dec 2, 2019
Hello, Community!
After beta testing the new exam InterSystems IRIS Core Solutions Developer Specialist at Global Summit 2019, the Certification Team of InterSystems Learning Services has performed the necessary calibration and adjustments to release it to our community. It is now ready for purchase and scheduling in InterSystems exam catalog.
This is InterSystems first certification exam to feature mainly code samples as question topics. To give potential candidates a sample of the exam's content, practice questions have been published to help orient to exam question approaches and content. Take a look here.
Passing the exam allows you to claim the electronic badge that can be embedded in social media accounts to show the world that your InterSystems IRIS development skills are first rate. And if you aren't successful in your first attempt, we now offer a new service, Exam Retake Support, that can help you prepare for a retake. Read all about it in our Policies.
If you are new to InterSystems Certification, please review our new program pages that include information on taking exams, exam policies, FAQ and more. Also, check out our Organizational Certification that can help your organization access valuable business opportunities and establish your organization as a solid provider of InterSystems solutions in our marketplace.
The Certification Team at InterSystems Learning Services are pretty excited about this new exam and we are also looking forward to working with you to create new certifications that can help you advance your career. We are always open to ideas and suggestions at certification@intersystems.com.
Looking forward to celebrating your success,
James Kantor - Certification Manager, InterSystems And I already have it.
Hi,
I think for all those who are certified, there should be an option to add the badge next to their profile here.Even better, Intersystems themselves should add it on their own as they already have the data.
That would not only make the certification more desirable but also will make those who have done the hard work to get certified stand out, can be referred by Intersystems as their preferred consultants and employers also save the effort of scrutinize from dozens of resumes they get.
This platform is the best and may be the only place ideal for my above suggestion. Neerav!
This is a very good idea and of course, we can do that.
But not automatically - not all the members want to expose their certificates.
What is the appropriate way you think? What if we add the extra field in the member profile for the certificate link?
Also, we can add some signs to the avatar image.
Hey,
A sign/star on the avatar and yes ability to add multiple certificates should be there.
The profile should be a brief summary of the person's profession somewhat like Linkedin (Obviously optional if someone wants to fill the values in )
Btw is there a way to update the avatar picture ?? I appreciate if you add a task here - you'll be able to get the update on progress.
>Btw is there a way to update the avatar picture ??
I bet. Same way possibly you added the first one? Hi,
The link is not working. Can you please check?
Thanks fixed, thanks Hi, everyone,
Evgeny is absolutely right - our certified staff must be the ones to decide how they use their credentials. InterSystems cannot automatically publish any credential for a person, they have to do it themselves.
That said, I think it's a great idea to be able to link the credentials to profiles of this community.
Thanks! J. Congrats, Dmitry! It was great to meet you at Global Summit 2019 and hope to see you again, J. For those who participate in ObjectScript Advent of Code 2019: there is an opportunity to receive an attempt for the certification exam for free!
Be one of the top 3 performers in the competition and get a voucher for the certification exam!
The current leaderboard is the following:
And you still have 2 weeks to change the leaderboard!
*The option is not eligible for InterSystems employees so your chances to win are higher! ;) This is great news, Evgeny! Our Certification Team is excited and proud to offer free exam attempts to winners of our Community's initiatives.
So good luck to everyone! We in Certification will be happy to redeem the winners' vouchers for an exam of their choice - even future exams!
Best regards, Jamie And you are able to redeem the voucher for certification attempt with GM points! See the announcement.
Announcement
Anastasia Dyubaylo · Jan 17, 2020
Hi Developers,
New video, recorded by @Benjamin.DeBoe, is already on InterSystems Developers YouTube:
⏯ Python and InterSystems IRIS
InterSystems Product Manager @Benjamin.DeBoe talks about using Python, InterSystems IRIS Data Platform, and the native Python API's.
Try InterSystems IRIS: https://www.intersystems.com/try
Enjoy watching the video! 👍🏼
Article
Robert Cemper · Apr 28, 2023
I decided to write this down before time wiped out my memoryIt's a very personal story as a partner, as a competitor, as an employee,as a customer and finally as an external observer of InterSystems.
After superfast graduation from Technical University in Vienna,some years of development for a virtual machine at SIEMENS,some hard-core networking experience and core OS development at OLIVETTII joined Digital Equipment (DEC) in 1978 as a support and sales engineer of their brand new DSM-11.
The core of DSM-11 - the Global module - was written by Terry Ragon.I had the chance to meet him during the kick-off training in Maynard, MA.InterSystems was rather fresh after its foundation.
It was a challenging time to get in all the (to me) new hardware technologyand all the details of that big cake of operating system, database, and interpreter.It was fascinating. It was great to see how I could win any benchmark againsttraditional relational DBs.As the provider of the core engine, I saw InterSytems as a Partner
Though based on my background in virtualization. I just couldn't resist modifying and adjusting drivers and disk modules to transform DSM-11into a "Layered Product" [an App] running with almost the sameperformance on top of RSX-11M.
You might imagine DEC was not so pleased with this experiment.Especially as short time later, they started the same exercise on VMS.The result was poor and not attractive to customers at that time.DEC was just not able to understand what jewel they had in their hands.Later, under the guidance of InterSystems, this gap was closed.
But one of my customers invited me to write his own "DSM-like" OSdirectly on bare-bone VAX without any bit of VMS.How often did you hear from such an engineering opportunity?It was one of those"IMPOSSIBLE - CAN'T BE DONE" triggers that I met during my life.We were 2 experienced engineers dedicated to the project:Bought a VAX-750 in 1981, started to read the processor handbook and after passing page 35 we started typing on the console.Realistic:- There was a design beyond the limits of PDP- There was a stack of new functionalities- Already supporting diskless machines over Ethernet,- I created the bootstraps, my own protocol on Ethernet,my own hardware format on disk, that no one else could read......
A huge effort. But after 18 months only it was doneand we could run the first installation at a customer.This was the time when InterSystems became a Competitor.A very remote one since there was no overlap in the market.We watched suspiciously each other from a far distance.
It was a commercial success for the company.For me, it became a lot of routine work with MicroVAX as a main box.Bug fixing and a fresh release every now and then was no real challenge.Some years later DEC offered a new "MISSION IMPOSSIBLE" to me and I followedIt was a technology challenge - but far from the old DSM that was goneout of scope. And it converted into an organizational and management challengeSo I sold my "Engineer Soul" to climb up international management in DEC.But that made me aware of the "Quality" and "Customer first" mindset.
When Compaq bought DEC, the local management structures were destroyed.Looking for new opportunities I was contacted by a headhunter.Big surprise: his customer was InterSystems:
Within a few weeks, I joined as Employee and felt at home immediately.Now I realized what I had missed for a long time - Creative Working -and what I had sold just for a few bucks more.I just joined when objects became invisible in the local partition.Caché Studio was fresh and new. And 2 decades of development had gone.But the basic structures were the same that I had built into my own variant.And of top importance to me "Quality first" + "Customer dedication" I spent 12 very exciting years at InterSystems seeing a lot of new featuresraising and some less attractive ones fading away.But it was never a routine job. Every day a different challenge to fight.I'm especially proud that I never lost any benchmark against other DBs:Topped by winning against Oracle Spacial by a significant factor.After my retirement 9 years ago I changed into the role of aCustomer of InterSystems. This experience completes the image.A very personal pleasure to me is this Developer Community that I joined in 2017Now I see myself in the role of Observer of InterSystems. understand I proposed it already back in 2005. But it was just too early to be accepted My dedication to Customers + Quality is unbroken and my message tomy support engineers at DEC is still valid for me:
the customer is always right - he has the problem
our products may have a bug - so find it
the description of our products might be wrong - explain it better
don't assume the customer has your technical background - explain in detail
we didn't understand what the bug is - ask twice and try to listen deeper
Those are also the 5 principles I apply when I do my reviews on OEXand that makes the quality of an OEX package.
Now in 2023 I became a critical Observer and have my surprises,have some dejà-vue recognizing old ideas fresh dressed. My applausefor news is filtered. Some are great some less convincing.
I always try to pass on my personal experience to those that ask for it.And sometimes I try to remind contributors, that their articles orpackages are a service to the other members and they deserve Quality.
I hope it wasn't boring for you. Thanks for sharing, @Robert.Cemper1003. Always interesting to hear about the early days! Thanks for this great article @Robert.Cemper1003 ! This brings back memories ... I did some benchmark testing as a student on a DEC MicroVAX I in 1987 in VMS (using a VT100 terminal). I didn't work with M at that time yet but our retail business run by older family members started using it in +/-1986 on a PC AT (Micronetics MSM running on top of DOS) with an ISA bus terminal multiplexer card - RS232/422 cables were installed for multiple Falco VT420 terminals. Soon after, our other retail business started using Digital Unix workstations with these VT terminals too. I started programming in M in 1997 using DTM, soon followed by Open M when we grounded our current retail business and now - 26 years later - I'm writing modern apps and api's in JavaScript/ObjectScript using a Caché/IRIS + Node.js back-end running on the same globals structure! I don't think many technologies can stay relevant/on top for that long ... 😉
Announcement
Anastasia Dyubaylo · Feb 18, 2020
Hi Community,
The new video from Global Summit 2019 is already on InterSystems Developers YouTube:
⏯ InterSystems IRIS on HCI
HCI, or hyper-converged infrastructure, is a software-defined IT infrastructure that virtualizes all the elements of a conventional system. This video will show the benefits and limitations of deploying an InterSystems IRIS-based application on HCI.
Takeaway: You will gain an understanding of HCI, and learn where and when it may be useful.
Presenter: @Murray.Oldfield, Technology Architect, InterSystems
Additional materials to this video you can find in this InterSystems Online Learning Course.
Enjoy watching this video! 👍🏼
Article
Anton Umnikov · Oct 17, 2019
October 17, 2019
Anton UmnikovSr. Cloud Solutions Architect at InterSystemsAWS CSAA, GCP CACE
AWS Glue is a fully managed ETL (extract, transform, and load) service that makes it simple and cost-effective to categorize your data, clean it, enrich it, and move it reliably between various data stores.
In the case of InterSystems IRIS, AWS Glue allows moving large amounts of data from both Cloud and on-Prem data sources into IRIS. Potential data sources include, but not limited to on-Pem databases, CSV, JSON, Parquet and Avro files residing in S3 buckets, Cloud-native databases such as AWS Redshift and Aurora and many others.
This article assumes that you have the basic familiarity with AWS Glue, at least at the level of completing AWS Glue Getting Started tutorials. We will concentrate on the technical aspects of configuring AWS Glue Jobs to use InterSystems IRIS as a Data Target, or in other terms - "data sink".
Image from https://docs.aws.amazon.com/glue/latest/dg/components-key-concepts.html
AWS Glue Jobs are run in a "Serverless" manner. All the resources, required for performing the job are dynamically provisioned by AWS only for the time the job is actually running and immediately destroyed the moment the job is completed, so instead of provisioning, managing and incurring ongoing costs for the required infrastructure you are being billed only for the time job is actually running and spend you efforts only on writing the Job code. At the "idle" time - no resources are consumed, other than S3 buckets, storing the job code and configuration.
While multiple options are available, typically Glue Jobs are executed on dynamically provisioned Apache Spark and written in PySpark code. Glue Job contains the "Extract" portion, where data is being extracted from data sources, series of "Transformations", build using Glue API and finally the "Load" or "sink" part, where after final transformation data is being written to the target system.
In order to enable AWS Glue to interact with IRIS we need to ensure the following:
Glue has network access to the IRIS instances involved
IRIS JDBC driver JAR file is accessible to the Glue Job
Glue Job is using API, compatible with InterSystems IRIS JDBC
Let's examine each of the required steps.
Create IRIS Connection
In AWS Console select AWS Glue->Connections->Add Connection
Enter the name of your connection and select "JDBC" for Connection Type.
In JDBC URL enter JDBC connection string for your IRIS instance, username and password information.
The next step is crucial - you need to make sure that Glue places its endpoints into the same VPC as your IRIS instance. Select VPC and Subnet of your IRIS instance. Any security group with a self-referencing inbound rule for all TCP ports would do here. For instance - your IRIS Instance security group.
IAM Role with access to JDBC driver
If you haven't done it already - upload IRIS JDBC driver JAR file intersystems-jdbc-3.0.0.jar into S3 bucket. In this example, I'm using s3://irisdistr bucket. It would be different for your account.
You need to create IAM Role for your Glue Job, that can access that file, along with other S3 buckets that Glue would use to store scripts, logs etc.
Make sure it has the read access to the bucket of your JDBC driver. In this case, we granting this role (GlueJobRole) ReadOnly access to all buckets, along with predefined AWSGlueServiceRole. You might choose to further limit this role permissions.
Create and configure Glue Job
Create a new Glue Job. Select IAM Role, we created in the previous step. Leave everything else default.
Under "Security configuration, script libraries, and job parameters (optional)" set "Dependent jars path" to the location of the intersystems-jdbc-3.0.0.jar in the S3 bucket.
For Source - use one of your pre-existing data sources. If you followed tutorials, referenced above you'll have at least one already.
Use "Create tables in your data target" option and select IRIS connection you've created in the previos step. Leave everything else default.
If you followed us up until now you should arrive at the screen similar to this:
Almost there!!! We just need to make one simple change to the script in order to load data into IRIS.
Adjusting the script
The script that Glue generated for us uses AWS Glue Dynamic Frame - AWS proprietary extension to Spark. While it provides some benefits for ETL jobs it also ensures that you can't write data to any database that AWS don't have managed service offering for.
Good news - at the point of writing the data to the database all the benefits of Dynamic Dataframe such as no schema enforcement for "dirty" data are not required anymore (at the point of writing data is presumed to be "clean") and we can easily convert Dynamic Dataframe to Spark native Dataframe that is not limited to AWS managed targets and can work with IRIS.
So the line we need to change is line #40 on the picture above. One before last.
Here is the change we need to make:
#datasink4 = glueContext.write_dynamic_frame.from_jdbc_conf(frame = dropnullfields3, catalog_connection = "IRIS1", connection_options = {"dbtable": "elb_logs", "database": "USER"}, transformation_ctx = "datasink4")
dropnullfields3.toDF().write \
.format("jdbc") \
.option("url", "jdbc:IRIS://172.30.0.196:51773/USER/") \
.option("dbtable", "orders") \
.option("user", irisUsername) \
.option("password", irisPassword) \
.option("isolationlevel","NONE") \
.save()
Where irisUsername and irisPassword are the username and passwords for your IRIS JDBC connection.
Note: storing passwords in the source code is a big No No! We'll encourage you to use tools like AWS Secrets Manager for that, but going into this level of the security details is beyond the scope of this article. Here is the good article on using AWS Secrets Manager with AWS Glue.
Now hit "Run Job" button, sit back and relax while AWS Glue is doing ETL for you.
Well... more than likely you'll hit some errors at first... We all know how it works. A typo here, a wrong port in security group there... AWS Glue uses CloudWhatch to store all the execution and error logs. Browse /aws-glue/jobs/error and /aws-glue/jobs/output log groups to identify what went wrong.
Happy ETLing in the cloud!
-Anton
💡 This article is considered as InterSystems Data Platform Best Practice.
Announcement
Paul Lilienfeld · Nov 18, 2019
Good Morning-
I am currently looking for a Cache ObjectScript Developer. Please see job description below. If you are interested please send an email to paul.lilienfeld@libertyits.com
Required Qualifications:Bachelor s degree in Computer Science, Business Administration, Management Information Systems, Computer Engineering or equivalent experience/trainingWorking knowledge of programming languages3+ years of experience developing and supporting mainframe applications in a collaborative environment that includes business experts and other software developersDemonstrated desire and ability to make meaningful contributions in a team environmentPreferred Qualifications:Experience with MUMPS/Cache Object Script, which is the primary coding language for our Cache database Experience with MySQL would also be a plusShadow server configuration knowledgeECP, DCP, Cache database configuration and troubleshooting
Announcement
Olga Zavrazhnova · Dec 29, 2019
Hi Community!
Thank you so much for being with InterSystems Developer Community yet another year!
We want to know how helpful the Developer Community is for you today.
Could you please go through this short survey which will let us know what do you think and what could be improved.➡️ Developer Community Survey 2019 (4 min)
We wish you a Merry Christmas and a Happy New Year! ✨
Sincerely,
Your InterSystems Developer Community Team