Search

InterSystems IRIS supports a unique data structure, called globals, for information storage. Essentially, globals are persistent arrays with multi-level indices, having several extra capabilities—transactions, quick traversal of tree structures, and a programming language known as ObjectScript. I'd note that for the remainder of the article, or at least the code samples, we'll assume you have familiarised yourself with the basics of globals: Globals Are Magic Swords For Managing Data. Part 1.Globals - Magic swords for storing data. Trees. Part 2.Globals - Magic swords for storing data. Sparse arrays. Part 3. Globals are completely different structures for storing data than the usual tables, and operate at a much lower level. And that begs the question, how would transactions look when working with globals, and what peculiarities might you encounter in the effort? We know from relational database theory that a good transaction implementation needs to pass the ACID test (see ACID in Wikipedia). Atomicity: All changes made in the transaction are recorded, or none at all. See Atomicity (database systems) in Wikipedia. Consistency: After the transaction is completed, the logical state of the database should be internally consistent. In many ways, this requirement applies to the programmer, but in the case of SQL databases, it also applies to foreign keys. Isolation: Transactions running in parallel shouldn’t affect one another. Durability: After successful completion of the transaction, low-level problems (such as a power failure) should not affect the data changed by the transaction. Globals are non-relational data structures. They were designed to support ultra-fast work on hardware with a minimal footprint. Let's look at how transactions are implemented in globals using the IRIS/docker-image. 1. Atomicity Consider the situation when 3 values must be saved in database together, or none of them should be recorded. The easiest way to check atomicity is to enter the following code in terminal: Kill ^a TSTART Set ^a(1) = 1 Set ^a(2) = 2 Set ^a(3) = 3 TCOMMIT Then conclude with: ZWRITE ^a The result should be this: ^a(1)=1 ^a(2)=2 ^a(3)=3 As expected, Atomicity is observed. But now let's complicate the task by introducing an error and see how the transaction is saved—partially, or not at all. We’ll start checking atomicity as we did before, like so: Kill ^a TSTART Set ^a(1) = 1 Set ^a(2) = 2 Set ^a(3) = 3 But this time we’ll forcibly stop the container using the command docker kill my-iris, which is almost equivalent to a forced power off as it sends a SIGKILL (halt process immediately) signal. After restarting the container, we check the contents of our global to see what happened. Maybe the transaction has been partially saved? ZWRITE ^a Nothing got out No, nothing has been saved. So, in the case of accidental server stop, the IRIS database will guarantee the atomicity of your transactions. But what if we want to cancel changes intentionally? So now let's try this with the rollback command, as follows: Kill ^a TSTART Set ^a(1) = 1 Set ^a(2) = 2 Set ^a(3) = 3 TROLLBACK 1 ZWRITE ^a Nothing got out Once again, nothing has been saved. 2. Consistency Recall that globals are lower-level structures for storing data than relational tables, and with a globals database, indices are also stored as globals. Thus, to meet the requirement of consistency, you need to include an index change in the same transaction as a global node value change. Say, for example, we have a global ^person, in which we store personal data using the social security number (SSN) as the key: ^person(1234567, 'firstname') = 'Sergey' ^person(1234567, ‘lastname’) = ‘Kamenev’ ^person(1234567, ‘phone’) = ‘+74995555555 ... We’ve created an ^index key to enable rapid search by last or last and first names, as follows: ^index(‘Kamenev’, ‘Sergey’, 1234567) = 1 To keep the database consistent, we need to add persons like this: TSTART ^person(1234567, ‘firstname’) = ‘Sergey’ ^person(1234567, ‘lastname’) = ‘Kamenev’ ^person(1234567, ‘phone’) = ‘+74995555555 ^index(‘Kamenev’, ‘Sergey’, 1234567) = 1 TCOMMIT Accordingly, when deleting a person, we must use the transaction: TSTART Kill ^person(1234567) Kill ^index(‘Kamenev’, ‘Sergey’, 1234567) TCOMMIT In other words, fulfilling the consistency requirement for your application logic is entirely up to the programmer when working with a low-level storage format such as globals. Luckily, IRIS offers the commands to organise your transactions and deliver Consistency guarantees for your applications. When using SQL, IRIS will use these commands under the hood to ensure consistency of its underlying globals data structures when performing INSERT, UPDATE, and DELETE statements. Of course, IRIS SQL also offers corresponding SQL commands for starting and stopping transactions to leverage in your (SQL) application logic. 3. Isolation Here’s where things get wild. Suppose many users are working on the same database at the same time, changing the same data. The situation is comparable to when many developers are working with the same code repository and trying to commit changes to many files at the same time. The database needs to keep up with everything in real time. Given that serious companies typically have a person responsible for version control—merging branches, managing conflict resolution, and so forth—and that the database needs to take care of this in real time, the complexity of the problem and the importance of correctly designing the database and the code that serves it both become self-evident. The database can’t understand the meaning of actions performed by users and try to prevent conflicts when they’re working on the same data. It can only cancel one transaction that contradicts another or execute them sequentially. Moreover, as a transaction is executing (before the commit), the state of the database may be inconsistent. Other transactions should not have access to the inconsistent database state. In relational databases, this is achieved in many ways, such as by creating snapshots or using multi-versioned rows. When transactions execute in parallel, it’s important that they not interfere with each other. This is what isolation is all about. SQL defines four levels of isolation, in order of increasing rigor. They are: READ UNCOMMITTED READ COMMITTED REPEATABLE READ SERIALIZABLE Let's consider each level separately. Note that the cost of implementing each level grows almost exponentially as you move up the stack. READ UNCOMMITTED is the lowest level of isolation, but it’s also the fastest. Transactions can read the changes made by other transactions. READ COMMITTED is the next level of isolation and represents a compromise. Transactions can’t read each other's changes before a commit, but can read any changes after a commit. Say we have a long-running transaction (T1), during which commits have happened in transactions T2, T3... Tn while working on the same data as T1. In such cases, each time we request data in T1, we may well obtain a different result. This is called a non-repeatable read. REPEATABLE READ is the next level of isolation, in which we no longer have non-repeatable reads because a snapshot of the result is taken each time we request to read data. The snapshot is used if the same data is requested again during the same transaction. However, at this isolation level, it’s possible that what will be read is phantom data—new strings that were added by transactions committed in parallel. SERIALIZABLE is the highest level of isolation. It’s characterized by the fact that any data used in a transaction (whether read or changed) becomes accessible to other transactions only after the first transaction has finished. First, let’s see whether there’s isolation of operations between threads with transactions and threads without transactions. Open two terminal windows and enter the following: Kill ^t Write ^t(1) 2 TSTART Set ^t(1)=2 There’s no isolation. One thread sees what the second one does when it opens a transaction. Now let's see whether transactions in different threads can see what’s happening inside. Open two terminal windows and start two transactions in parallel. Kill ^t TSTART Write ^t(1) 2​​​ TSTART Set ^t(1)=2 A 3 appears on the screen. What we have here is the simplest (but also the fastest) isolation level: READ UNCOMMITTED. In principle, this is what we expect from a low-level data representation such as globals, which always prioritize speed. IRIS SQL provides different transaction isolation levels to choose from, but what if we need a higher level of isolation when working with globals directly? Here we need to think about what isolation levels are actually for and how they work. For instance, lower levels of isolation are compromises designed to speed up database operations. The highest isolation level, SERIALIZABLE, ensures that the result of transactions executed in parallel is equivalent to the result of executing them serially. This guarantees there will be no collisions. We can achieve this with properly used locks in ObjectScript, which can be applied in multiple ways. This means you can create regular, incremental, or multiple locks using the LOCK command. Let's see how to use locks to achieve different levels of isolation. In ObjectScript, you use the LOCK operator. This operator permits not just exclusive locks, which are necessary for changing data, but also what are called shared locks. These shared locks can be accessed by several threads at once to read data that won’t be changed by other processes during the reading process. For more details about locking, see the article “Locking and Concurrency Control”. To learn about two-phase locking, see the article "Two-phase locking" on Wikipedia. The difficulty is that the state of the database may be inconsistent during the transaction, with the inconsistent data visible to other processes. How can this be avoided? For this example, we’ll use locks to create visibility windows within which the state of the database can be consistent. Access to any of these visibility windows will be through a lock. Shared locks on the same data are reusable—several processes can take them. These locks prevent other processes from changing data. That is, they’re used to form windows of a consistent database state. Exclusive locks, on the other hand, are used when you’re modifying data—only one process can take such a lock. Exclusive locking can be employed in two scenarios. First, it can take any process if the data doesn’t have locks. Second, it can take only the process that has a shared lock on the data and the first one that requested an exclusive lock.  The narrower the visibility window, the longer the wait for other processes becomes—but the more consistent the state of the database in it will be. READ COMMITTED ensures that we see only committed data from other threads. If data in another transaction hasn't yet been committed, we see the old version. This lets us parallelize the work instead of waiting for a lock to be released. In IRIS, you can't see an old version of the data without using special tricks, so we'll have to make do with locks. We need to use shared locks to permit data to be read only at points where it’s consistent. Let's say we have a database of users, ^person, who transfer money from one person to another. Here’s the point at which money is transferred from person 123 to person 242: LOCK +^person(123), +^person(242) TSTART Set ^person(123, amount) = ^person(123, amount) - amount Set ^person(242, amount) = ^person(242, amount) + amount TCOMMIT LOCK -^person(123), -^person(242) The point where the amount is requested for person 123 before the deduction should have an exclusive lock (by default): LOCK +^person(123) Write ^person(123) But if we need to display the account status in the user's personal account, we can use a shared lock, or none at all: LOCK +^person(123)#”S” Write ^person(123) LOCK -^person(123)#”S” However, if we accept that database operations are carried out virtually instantaneously (remember that globals are a much lower-level structure than a relational table), then this level is no longer so necessary in favor higher isolation levels. Full example, for READ COMMITTED: LOCK +^person(123)#”S”, +^person(242)#”S” Read data (сoncurrent committed transactions can change the data) LOCK +^person(123), +^person(242) TSTART Set ^person(123, amount) = ^person(123, amount) - amount Set ^person(242, amount) = ^person(242, amount) + amount TCOMMIT LOCK -^person(123), -^person(242) Read data (сoncurrent committed transactions can change the data) LOCK -^person(123)#”S”, -^person(242)#”S” REPEATABLE READ is the second-highest level of isolation. At this level we accept that data may be read several times with the same results in one transaction, but may be changed by parallel transactions. The easiest way to ensure a REPEATABLE READ is to take an exclusive lock on the data, which automatically turns this isolation level into a SERIALIZABLE one. LOCK +^person(123, amount) read ^person(123, amount) other operations (parallel streams try to change ^person(123, amount), but can't) change ^person(123, amount) read ^person(123, amount) LOCK -^person(123, amount) If locks are separated by commas, they are taken in sequence. But they will be taken atomically, all at once, if they’re listed like this: LOCK +(^person(123),^person(242)) SERIALIZABLE is the highest level of isolation and the most costly. When working with classic locks like we did in the above examples, we have to set the locks in such a way that all transactions with data in common will end up being performed serially. For this approach, most of the locks should be exclusive and taken to the smallest fields of the global, for performance. If we’re talking about deducting funds from a ^person global, then SERIALIZABLE is the only acceptable level. Money spent needs to be strictly serial, otherwise it’s possible to spend the same amount several times. 4. Durable I conducted tests with a hard cut-off of the container using the docker kill my-iris command. The database stood up well to these tests. No problems were identified. Tools to manage globals and locks You may find useful the following tools in IRIS Management portal: View and manage locks. View and manage globals. Conclusion InterSystems IRIS has support for transactions using globals, which are atomic and durable. To ensure database consistency with globals, some programming effort and the use of transactions are necessary, since there are no complex built-in constructions like foreign keys. Globals without locks are equivalent to the READ UNCOMMITTED level of isolation, but this can be raised to the SERIALIZABLE level using locks. The correctness and transaction speed achievable with globals depend considerably on the programmer's skill and intent. The more widely that shared locks are used when reading data, the higher the isolation level. And the more narrowly exclusive locks are used, the greater the speed. Sergey, it's great that you are writing articles for newbies, nevertheless you don't explicitly mark it. Just a quick note on your samples: ZWRITE command never returns <UNDEFINED> in IRIS, so to check the global existence one should use something like if '$data(^A) { write "Global ^A is UNDEFINED",! } I'm sure that you are aware of it; just thinking of novices that should not be confused. In the example for READ UNCOMMITTED, after the second (right-hand) Terminal session sets ^t(1)=2, when the first (left-hand) Terminal session writes ^t(1), the example shows/states that a "3" appears, but that's wrong; it should be a "2". Since transactions can be nested, and TROLLBACK rolls back all transactions, it's best practice to pair each TSTART with TROLLBACK 1, which will rollback only the currenct transaction. Thanks! I fixed the error Thanks! You're right Thanks! I fixed the error As you are talking about locks and transactions, and as others have noted, you can nest transactions it might be worth warning people about locks inside transactions and the fact that the unlock will not take place until the tcommit. These can cause issues, especially where this is a method using transactions and locks which calls another that does the same.

Dear Community, We're pleased to invite you to the InterSystems Benelux Symposium 2020, which will take place from February 11th to 12th in Antwerp, Belgium! At the Symposium, both InterSystems experts and external thought leaders will discuss what it takes to make your IT innovation work. You're more than welcome to join us in the Radisson Blu Astrid Hotel in Antwerp. Fastest Path to Possible - With Digital Innovation "The future lies in the hands of those who use data the best", said InterSystems CEO & Founder Terry Ragon at the InterSystems Global Summit 2019. "However, this data is spread across an enormous set of databases and hardware devices, meaning we need to find ways to connect these systems together." There is, of course, a way: interoperability. But how do you approach interoperability? And most importantly: how can you use it to create better applications and be a better partner to your customers? Be part of the discussions! Don’t miss out and register for the event by clicking right here. Can’t wait to see you in Antwerp! So, remember: ⏱ Time: February 11-12, 2020 📍Venue: Radisson Blu Astrid Hotel, Antwerp, Belgium ✅ Registration: SAVE YOUR SEAT TODAY I'm finally going to Antwerp this year. So, you have a chance to meet me there, to see VSCode-ObjectScript in action, get quick help with the migration process, and give your feedback. See you there. Will have two sessions on Feb 12th on during InterSystems Benelux Summit on behalf of CUG meetup. See you there! So, today, I'm presenting a demo of VSCode-ObjectScript at CUG meetup here in Antwerp ObjectScript Package Manager presentation is attached Please welcome a brief video overview of the InterSystems Benelux Symposium 2020: Enjoy! 👍🏼

Hi Community, We are pleased to invite you to the InterSystems Meetup in Moscow on December 10, 2019! InterSystems Moscow Meetup is a pre-New Year meeting for users and developers on InterSystems technologies. The meetup will be devoted to the InterSystems IRIS Data Platform. Please check out the agenda: 📌 18:40 Registration. Welcome coffee 📌 19:00Review presentation on the InterSystems Russia news for 2019 📌 19:15 Technology News on InterSystems IRIS by @Eduard.Lebedyuk, InterSystems Sales Engineer REST, JSON, IAM PEX, Native API, etc. 📌 20:00 Coffee Break 📌 20:20 Migration to InterSystems IRIS 📌 21:00 ObjectScript Package Manager Introduction — Package Manager Client for InterSystems IRIS by @Evgeny.Shvarov, Startups and Community Manager 📌 21:15 Open Exchange and other resources & services for InterSystems Developers by @Evgeny.Shvarov, Startups and Community Manager 📌 21:30 Free time. Drinks and snacks The agenda is full of interesting stuff. We look forward to seeing you! So, remember: 🗓 Date: December 10, 2019 ⏱ Time: 18:40-22:00 📍 Venue: Loft-Ministerstvo, Stoleshnikov Lane 6/3, Moscow ✅ Registration: Register for FREE here Space is limited, so register today to secure your place. Admission free, registration is mandatory for attendees. Save your seat today! Hey! The agenda is updated - I'll do the following sessions: 📌 21:00 ObjectScript Package Manager Introduction — Package Manager Client for InterSystems IRIS 📌 21:15 Open Exchange and other resources & services for InterSystems Developers Come to chat!

Temporary tables are tables available for a current process only (and destroyed when process ends). What are you approaches to creating temporary tables? Here's the two I know: Process-private Globals storage can be used as a data global in storage definition. That way, each process can have its own objects for the class with ppg storage. Here's how. Here's how 2. InterSystems TSQL supports #tablename temporary tables. A #tablename temporary table is visible to the current procedure of the current process. It is also visible to any procedure called from the current procedure. #tablename syntax is only supported in TSQL procedures (class methods projected as procedures with language tsql). Any other ways to achieve similar functionality? CREATE GLOBAL TEMPORARY TABLE <mytable> {} has similar effects as process-private globals... Indeed, that's the recommended SQL way of achieving what Eduard described about PPGs. Drawbacks are that queries on these tables cannot be parallelized (as that implies multiple processes, of course). Our TSQL support is meant for Sybase customers wishing to redeploy their TSQL applications on IRIS (especially now that SAP/Sybase is terminating support for those platforms). Just temporary table support by itself wouldn't be a reason to start building TSQL applications and abandon IRIS SQL/ObjectScript, of course :-). However, for a recent TSQL migration we did some work on our TSQL temp table support and were considering to roll that out to regular IRIS SQL, so this thread is a good place to share your experiences and requirements so we can make sure to do that properly, as needed. Hi, I recently needed a temporary class (not just a table) to store data while it was manipulated (imported, queried, modified, etc. and eventually exported) . I eventually set up all the storage locations with PPG refs as noted above, which works, but did wonder if there was a class parameter I had missed that just indicated the extent was temporary. Or maybe an alternative to extending %Persistent? Mike I'm not really sure what do you mean by if there was a class parameter I had missed that just indicated the extent was temporary what do you want to achieve with this? Hi, I don't want to achieve anything else, it's just that "cache being cache" there's often another way to do the same thing and it might be easier. :-) Mike

As you might have heard, we just introduced the InterSystems API Manager (IAM); a new feature of the InterSystems IRIS Data Platform™, enabling you to monitor, control and govern traffic to and from web-based APIs within your IT infrastructure. In case you missed it, here is the link to the announcement. In this article, I will show you how to set up IAM and highlight some of the many capabilities IAM allows you to leverage. InterSystems API Manager brings everything you need to monitor your HTTP-based API traffic and understand who is using your APIs; what are your most popular APIs and which could require a rework. to control who is using your APIs and restrict usage in various ways. From simple access restrictions to throttling API traffic and fine-tuning request payloads, you have fine-grained control and can react quickly. to protect your APIs with central security mechanisms like OAuth2.0 or Key Token Authentication. to onboard third-party developers and provide them with a superb developer experience right from the start by providing a dedicated Developer Portal for their needs. to scale your API demands and deliver low-latency responses I am excited to give you a first look at IAM, so let's get started right away. Getting started IAM is available as a download from the WRC Software Distribution site and is deployed as a docker container of its own. So, make sure to meet the following minimum requirements: The Docker engine is available. Minimum supported version is 17.04.0+. The docker-compose CLI tool is available. Minimum supported version is 1.12.0+. The first step requires you to load the docker image via docker load -i iam_image.tar This makes the IAM image available for subsequent use on your machine. IAM runs as a separate container so that you can scale it independently from your InterSystems IRIS backend. To start IAM requires access to your IRIS instance to load the required license information. The following configuration changes have to happen: Enable the /api/IAM web application Enable the IAM user Change the password of the IAM user Now we can configure our IAM container. In the distribution tarball, you will find a script for Windows and Unix-based systems named "iam-setup". This script helps you to set the environment variables correctly, enabling the IAM container to establish a connection with your InterSystems IRIS instance. This is an exemplary run from my terminal session on my Mac: source ./iam-setup.sh Welcome to the InterSystems IRIS and InterSystems API Manager (IAM) setup script.This script sets the ISC_IRIS_URL environment variable that is used by the IAM container to get the IAM license key from InterSystems IRIS.Enter the full image repository, name and tag for your IAM docker image: intersystems/iam:0.34-1-1Enter the IP address for your InterSystems IRIS instance. The IP address has to be accessible from within the IAM container, therefore, do not use "localhost" or "127.0.0.1" if IRIS is running on your local machine. Instead use the public IP address of your local machine. If IRIS is running in a container, use the public IP address of the host environment, not the IP address of the IRIS container. xxx.xxx.xxx.xxx Enter the web server port for your InterSystems IRIS instance: 52773Enter the password for the IAM user for your InterSystems IRIS instance: Re-enter your password: Your inputs are:Full image repository, name and tag for your IAM docker image: intersystems/iam:0.34-1-1IP address for your InterSystems IRIS instance: xxx.xxx.xxx.xxxWeb server port for your InterSystems IRIS instance: 52773Would you like to continue with these inputs (y/n)? yGetting IAM license using your inputs...Successfully got IAM license!The ISC_IRIS_URL environment variable was set to: http://IAM:****************@xxx.xxx.xxx.xxx:52773/api/iam/licenseWARNING: The environment variable is set for this shell only!To start the services, run the following command in the top level directory: docker-compose up -dTo stop the services, run the following command in the top level directory: docker-compose downURL for the IAM Manager portal: http://localhost:8002 I obfuscated the IP address and you can't see the password I used, but this should give you an idea how simple the configuration is. The full image name, IP address and port of your InterSystems IRIS instance and the password for your IAM user, that's everything you need to get started. Now you can start your IAM container by executing docker-compose up -d This orchestrates the IAM containers and ensures everything is started in the correct order. You can check the status of your containers with the following command: docker ps Opening localhost:8002 in my browser brings up the web-based UI: The global report does not show any throughput yet, as this is a brand new node. We will change that shortly. You can see that IAM supports a concept of workspaces to separate your work into modules and/or teams. Scrolling down and selecting the "default" workspace brings us to the Dashboard for, well, the "default" workspace we will use for our first experiments. Again, the number of requests for this workspace is still zero, but you get a first look at the major concepts of the API Gateway in the menu on the left side. The first two elements are the most important ones: Services and Routes. A service is an API we want to expose to consumers. Therefore, a REST API in your IRIS instance is considered a service, as is a Google API you might want to leverage. A route decides to which service incoming requests should be routed to. Every route has a certain set of conditions and if the conditions are fulfilled the request is routed to the associated service. To give you an idea, a route can match the IP or domain of the sender, HTTP methods, parts of the URI or a combination of the mentioned examples. Let's create a service targeting our IRIS instance, with the following values: field value description name test-iris the logical name of this service host xxx.xxx.xxx.xxx public IP-address of your IRIS instance port 52773 the port used for HTTP requests protocol http the protocols you want to support Keep the default for everything else. Now let's create a route: field value description paths / api /atelier requests with this path will be forwarded to our IRIS instance protocols http the protocols you want to support service test-iris requests matching this route will be forwarded to this service Again, keep the default for everything else. IAM is listening on port 8000 for incoming requests by default. From now on requests that are sent to http://localhost:8000 and start with the path /api/atelier will be routed to our IRIS instance. Let's give this a try in a REST client (I am using Postman). Sending a GET request to http://localhost:8000/api/atelier/ indeed returns a response from our IRIS instance. Every request goes through IAM and metrics like HTTP status code, latency, and consumer (if configured) are monitored. I went ahead and issues a couple more requests (including two requests to non-existing endpoints, like /api/atelier/test/) and you can see them all aggregated in the dashboard: Working with plugins Now that we have a basic route in place, we can start to manage the API traffic. Now we can start to add behavior that complements our service. Now the magic happens. The most common way to enforce a certain behavior is by adding a plugin. Plugins isolate a certain functionality and can usually be attached to certain parts of IAM. Either they are supposed to affect the global runtime or just parts like a single user (group), a service or a route. We will start by adding a Rate Limiting plugin to our route. What we need to establish the link between the plugin and the route is the unique ID of the route. You can look it up by viewing the details of the route. If you are following this article step by step, the ID of your route will be different. Copy the ID for the next step. Click on Plugins on the left sidebar menu. Usually, you see all active plugins on this screen, but as this node is relatively new, there are no active plugins yet. So, move on by selecting "Add New Plugin". The plugin we are after is in the category "Traffic Control" and is named "Rate Limiting". Select it. There are quite a few fields that you can define here as plugins are very flexible, but we only care about two fields: field value description route_id d6a97e5b-7da6-4c98-bc69-6a09263039a8 paste the ID of your route here config.minute 5 number of calls allowed per minute That's it. The plugin is configured and active. You probably have seen that we can pick from a variety of time intervals, like minutes, hours or days but I deliberately used minutes as this allows us to easily understand the impact of this plugin. If you send the same request again in Postman you will realize that the response comes back with 2 additional headers. XRateLimit-Limit-minute (value 5) and XRateLimit-Remaining-minute (value 4). This tells the client that he can make up to 5 calls per minute and has 4 more requests available in the current time interval. If you keep making the same request over and over again, you will eventually run out of your available quota and instead get back an HTTP status code 429 with the following payload: Wait until the minute is over and you will be able to get through again. This is a pretty handy mechanism allowing you to achieve a couple of things: Protect your backend from spikes Set an expectation for the client how many calls he is allowed to make in a transparent way for your services Potentially monetize based on API traffic by introducing tiers (e.g. 100 calls per hour at the bronze level and unlimited with gold) You can set values for different time intervals and hereby smoothing out API traffic over a certain period. Let's say you allow 600 calls per hour for a certain route. That's 10 calls per minute on average. But you are not preventing clients from using up all of their 600 calls in the very first minute of their hour. Maybe that's what you want. Maybe you would like to ensure that the load is distributed more equally over the hour. By setting the config_minute field to 20 you ensure that your users are not making more than 20 calls per minute AND 600 per hour. This would allow some spikes on the minute-level interval as they can only make 10 calls per minute on average, but users can't use up the hourly quota in a single minute. Now it will take them at least 30 minutes if they hit your system with full capacity. Clients will receive additional headers for each configured time interval, e.g.: header value X-RateLimit-Limit-hour 600 X-RateLimit-Remaining-hour 595 X-RateLimit-Limit-minute 20 X-RateLimit-Remaining-minute 16 Of course, there are many different ways to configure your rate-limits depending on what you want to achieve. I will stop at this point as this is probably enough for the first article about InterSystems API Manager. There are plenty more things you can do with IAM, we've just used one out for more than 40 plugins and haven't even used all of the core concepts yet! Here are a couple of things you can do as well and I might cover in future articles: Add a central authentication mechanism for all your services Scale-out by load-balancing requests to multiple targets that support the same set of APIs Introduce new features or bugfixes to a smaller audience and monitor how it goes before you release it to everyone Onboard internal and external developers by providing them a dedicated and customizable developer portal documenting all APIs they have access to Cache commonly requested responses to reduce response latency and the load on the service systems So, let's give IAM a try and let me know what you think in the comments below. We worked hard to bring this feature to you and are eager to learn what challenges you overcome with this technology. Stay tuned... More resources The official Press Release can be found here: InterSystems IRIS Data Platform 2019.2 introduces API Management capabilities A short animated overview video: What is InterSystems API Manager An 8-minute video walking you through some of the key highlights: Introducing InterSystems API Manager The documentation is part of the regular IRIS documentation: InterSystems API Manager Documentation Nice to have a clear step-by-step example to follow! Hi @Stefan.Wittmann ! If IRIS API Manager is published on InterSystems Docker Hub? `docker load -I iam_image.tar` did not work for me. I used `docker import aim_image.tar iam` instead and it worked. Have you unpacked the IAM-0.34-1-1.tar.gz? Originally I was trying to import the whole archive, which failed. After unpacking container it was imported successfully. No, it is not for multiple reasons. We have plans to publish the InterSystems API Manager on Docker Repositories at a later point, but I can't give you an ETA. Hi, I didn't have any problems loading the image, but when I run the setup script, I get the following error: Your inputs are:Full image repository, name and tag for your IAM docker image: intersystems/iam:0.34-1-1IP address for your InterSystems IRIS instance: xxx.xxx.xxx.xxxWeb server port for your InterSystems IRIS instance: 52773Would you like to continue with these inputs (y/n)? yGetting IAM license using your inputs...No content. Either your InterSystems IRIS instance is unlicensed or your license key does not contain an IAM license. Which license is required for IAM? We have installed Intersystems IRIS for Health in 2019.3 version on this server. $ iris list Configuration 'IRISDEV01' (default) directory: /InterSystems versionid: 2019.3.0.304.0 datadir: /InterSystems conf file: iris.cpf (SuperServer port = 51773, WebServer = 52773) status: running, since Tue Aug 13 08:27:34 2019 state: warn product: InterSystems IRISHealth Thanks for your help! Please write to InterSystems to receive your license. While trying to do this I've used both my public IP address and my VPN IP address and I get the error Couldn't reach InterSystems IRIS at xxx.xx.x.xx:52773. One or both of your IP and Port are incorrect. Strangely I got this to work last week without having the IAM application or user enabled. That said I was able to access the IAM portal and set some stuff up but I'm not sure it was actually truly working. Any troubleshooting advice? Michael, please create a separate question for this comment.

Hey Community, We're very pleased to present to you our brand new About Us pages! Please welcome: 📍 Our Team 📍 InterSystems Developer Ecosystem Overview So... Let's meet the Team! These are the people that make the magic happen ✨ Please feel free to contact us – we are here to make the Community a better place for developers! In addition, don't miss your chance to get to know all our regional teams: ES DC team PT DC team JP DC team CN DC team FR DC team And... 🤩 Let's meet our InterSystems Developer Ecosystem overview page! Here you can see a clear and simple overview of our related portals and resources where you can find lots of exciting and helpful stuff: How to find these pages? Go to the "About Us" section from the top menu: Hope you enjoy our brand new pages! Feel free to share your feedback in the comments ;)

Hi Developers, Enjoy watching the new video on InterSystems Developers YouTube: ⏯ InterSystems Deployment Options @ Global Summit 2022 Choosing the right deployment environment for your application built with InterSystems technologies means making informed decisions on a range of topics: Container, virtual machine, or bare metal? On-premise, hosted, private cloud, or public cloud? Linux or Windows Horizontal scaling, vertical scaling, or both? Platform high availability or synchronous mirroring? Disaster recovery required or not. In this this session, you'll hear real-world feedback about current best practices, the thinking behind it, and how InterSystems can help you make deployment decisions that mean your system will be effective, scalable, and resilient. 🗣 Presenter: @Ray.Wright, Principal Technology Architect, InterSystems Enjoy it and stay tuned! 👍

Hey Developers, Watch this video to learn about observability of your InterSystems IRIS application with InterSystems System Alerting and Monitoring (SAM) and modern DevOps tooling: ⏯ Observability with InterSystems IRIS @ Global Summit 2022 🗣 Presenter: @Robert.Kuszewski, Product Manager, Developer Experience, InterSystems Subscribe to InterSystems Developers YouTube to stay tuned!

Hi developers! As you know InterSystems IRIS besides globals, object, document and XML data-models also support relational where SQL is expected as a language to deal with the data. And as in other relational DBMS InterSystems IRIS has its own dialect. I start this post to support an SQL cheatsheet and invite you to share your favorites - I'll update the content upon incoming comments. Here we go! List all the tables in database: SELECT TABLE_SCHEMA, TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE='BASE TABLE' Credit to @Dmitry.Maslennikov Create table with current date and current time as defaults: CREATE TABLE CUSTOMER ( ID INTEGER PRIMARY KEY NOT NULL, NAME VARCHAR(100) NOT NULL, PASSWORD VARCHAR(20) NOT NULL, LAST_UPDATED TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL, REGISTRATION_DATE DATE DEFAULT CURRENT_DATE NOT NULL ); notice functions CURRENT_TIMESTAMP and CURRENT_DATE are without parenthesis. Create stored procedure/function that uses ObjectScript inside: CREATE OR REPLACE FUNCTION IRIS_VERSION() RETURNS VARCHAR LANGUAGE OBJECTSCRIPT { Return $ZV } Call Stored procedure/function: select IRIS_VERSION() as "Version" Call internal functions. Return IRIS Version: Select $version Return current namespace: Select $namespace Run SQL from file (credit @Raj.Singh5479 ): DO $SYSTEM.SQL.Schema.ImportDDL("c:\InterSystems\mysqlcode.txt",,"IRIS") Run SQL from file using python terminal(credit @Dmitry.Maslennikov): irissqlcli iris://_SYSTEM:SYS@localhost:1972/USER < c:\InterSystems\mysqlcode.txt Open SQL shell in terminal(credit @Chad.Severtson): USER>:sql Open SQL shell in webterminal (credit @Nikita.Savchenko7047 ): SAMPLES > /sql SAMPLES:SQL > select top 5 * from Cinema.Film ID Category Description Length PlayingNow Rating TicketsSold Title 1 1 A post-modern excursion into family dynamics and Thai cuisine. 130 1 PG-13 47000 Her Spicy Brothers 2 1 A gripping true story of honor and discovery 121 1 R 50000 Einstein's Geisha 3 1 A Jungian analysis of pirates and honor 101 1 PG 5000 A Kung Fu Hangman 4 1 A charming diorama about sibling rivalry 124 1 G 7000 Holy Cooking 5 2 An exciting diorama of struggle in Silicon Valley 100 1 PG 48000 The Low Calorie Guide to the Internet SAMPLES: SQL > /sql SAMPLES > write "COS!" cos! Add yours? nice! Thanks ... I didn't know you could do things like Select $zversion :) here it s in practice for those interested: USER>d $system.SQL.Shell() SQL Command Line Shell ---------------------------------------------------- The command prefix is currently set to: <<nothing>>. Enter <command>, 'q' to quit, '?' for help. [SQL]USER>>select $zversion 1. select $zversion Expression_1 IRIS for Windows (x86-64) 2022.1.2 (Build 574U) Fri Jan 13 2023 15:08:27 EST 1 Rows(s) Affected statement prepare time(s)/globals/cmds/disk: 0.1884s/34,356/143,526/0ms execute time(s)/globals/cmds/disk: 0.0007s/0/394/0ms cached query class: %sqlcq.USER.cls1 is there no quicker way to list all database tables? Like the postgresql \t? in irissqlcli, you can use \dt or .tables Try ":sql" instead of d $system.SQL.Shell() Holy cow @Chad.Severtson, that's really cool! Do you know when that was added / where I can find more information? I don't see it in the docs on SQL Shell (https://docs.intersystems.com/iris20221/csp/docbook/Doc.View.cls?KEY=GSQL_shell). Also, what is the ":" shorthand for and where else can it be used? Would be nice to have Run SQL from a file: DO $SYSTEM.SQL.Schema.ImportDDL("c:\InterSystems\mysqlcode.txt",,"IRIS") or irissqlcli iris://_SYSTEM:SYS@localhost:1972/USER < c:\InterSystems\mysqlcode.txt :py for embedded python @Dmitry.Maslennikov - thank you! Docs reference? Do you know what the ":" syntax is? Hey @Benjamin.Spead, You can find out more about the alias capability of iris session here: Using the Terminal Interactively | Using the Terminal | InterSystems IRIS Data Platform 2022.3 It's quite useful. You can even provide parameter substitution with $1, $2, etc. USER>:? :<number> Recall command # <number> :? Display help :py Do $system.Python.Shell() :mdx Do $system.DeepSee.Shell() :sql Do $system.SQL.Shell() :tsql Do $system.SQL.TSQLShell() :alias Create/display aliases :clear Clear history buffer :history Display command history :unalias Remove aliases HTH wow. Didn't know we have that! And aliases can be setup and transferred too? E.g. like a package (IPM)? Creating Aliases for Commonly Used Commands from docs:If you are using a UNIX® or Linux system, you can provide a list of alias definitions which the Terminal will set automatically at the start of every session. Define these aliases (one per line) in a file named .iris_init in the home directory.never tried this. I rarely work on *X or [SQL]USER>> run filename.sql You can also save the current query [SQL]USER>>save filename.sql Another very cool aspect of the feature :) So it looks like this is just a built-in alias which ships with InterSystems IRIS. Nice work @Evgeny.Shvarov those typical things you want to do quickly and never remember and have to read the docs :-) holy cow #2 :-) so much precious info on this thread! seriously! it's amazing how much knowledge there is to be gleaned within this community :) It has been around for a few years.Personally, I hesitate to use the alias capability lest I find myself in an environment without them! Thanks, Luca! Just a note... LAST_UPDATED TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL, This field will only define the LAST_UPDATED field at INSERT, not for an UPDATE. You probably want something like: LAST_UPDATED TIMESTAMP DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6) NOT NULL, This will cause LAST_UPDATED to be set at INSERT and UPDATE to the current timestamp to microsend precision. Thank you @Jeffrey.Drumm !!!

Hey Community! Welcome to the 6th edition of the InterSystems Ideas news bulletin! This time you can read about: ​​​​✓ Ideas implemented last month by Community members ✓ How to create a new idea on InterSystems Ideas ✓ New ideas created recently The Hall of Fame was updated with the names of Community Members who implemented Community Opportunity ideas: Add IRIS as a supported database for Apache Superset by @Herman.Slagman7521 was implemented by @Dmitry.Maslennikov Please add google oauth authorization to login to the management portal by @Aleksandr.Kolesov was implemented by @Yuri.Gomes 👏Thank you for implementing these ideas👏 The new article "How to create a new idea on InterSystems Ideas" describes the idea creation process step by step. Read it when adding new idea to the portal. And to round up this newsletter, here is the list of new ideas posted since the previous news bulletin 1. Dump SQL tables by @Evgeny.Shvarov 2. Java Hibernate support for IRIS by @Dmitry.Maslennikov 3. Add legitimate special symbol-separator for ObjectScript variable names by @Evgeny.Shvarov 4. allow cancel "Pending approval" by @Robert.Cemper1003 5. provide a switch to Hide/Unhide posts. or fix Re-Publish by @Robert.Cemper1003 6. Traceroutes to Java Gateway Display by @Scott.Roth 7. On Developer Community put animated GIFs behind a play/pause button by @John.Murray 8. Community in numbers on a GEO Dashboard by @Yuri.Gomes 9. Unit testing in VSCode by @Robert.Barbiaux 10. Saving Searched Criteria within Message Search by @Scott.Roth 11. Featured Article or Question by @Yuri.Gomes 👏Thank you for posting new ideas👏 Stay tuned for the next InterSystems Ideas news bulletin! In the meantime post new ideas for public discussion, vote for existing ideas, and comment on them on our InterSystems Ideas Portal! Hi Developers! 👋The idea "IRIS classes for OpenAI API" was implemented recently. @Francisco.López1549 thank you for implementing this idea and welcome to "Hall of Fame" page of Ideas Portal! 👏

IoT (Internet of Things) is a network of interconnected things, including vehicles, machines, buildings, domestic devices or any other thing with embedded TCP/IP remote connection available, allowing it to receive and send execution instructions and data. Each thing provides one or more services to the IoT network. For instance, smart light bulbs provide services of turning off and turning on the lights; smart air conditioners maintain the environment temperature; smart cameras send notifications when capturing movement. Yet, all those devices can operate together to provide building security, for example.When the camera detects movement, the lights turn on. The orchestration of those processes that can deliver business and integrated services, such as building security, is done by a server software controller. It can receive and send data and instructions from a dozen smart devices to a TCP/IP network (like the Internet) to operate these things as major services. There are two options of protocols used by things and server software to interoperate data and instructions: MQTT (Message Queuing Telemetry Transport) and REST (Representational State Transfer) API. The first one is more common because it has a simpler structure. Some devices have limited memory and processor features and don’t have enough resources to use REST. Thus, this article will detail how to implement the server-side software to interoperate the IoT using MQTT. About MQTT MQTT is a lightweight, publish-subscribe, machine to machine network protocol for message queue/message queuing service. It is designed for connections with remote locations that have devices with resource constraints or limited network bandwidth. It must run over a transport protocol that provides ordered, lossless, bi-directional connections—typically, TCP/IP. It is an open OASIS standard and an ISO recommendation (ISO/IEC 20922). See more on https://en.wikipedia.org/wiki/MQTT. The MQTT protocol has two actors to interoperate data between things: the message broker and the clients. The first one is a server software used to receive and publish messages to be produced or consumed by one or more clients. The last ones are the smart devices and server softwares used to provide major services using the thing’s services, such as a server software to provide a building automation service, or a software to control the traffic in the city using smart semaphores and public cameras services. MQTT Broker There are various MQTT message brokers on the market, but the most popular one is the Eclipse Mosquitto (https://mosquitto.org/), so we will use it in our example.The MQTT Broker is similar to JMS or MQ brokers. It is an intermediate employed to receive messages and post them into topics. Thus, the connected clients can subscribe to one or more topics to produce or consume messages from those topics. The topics allow the clients (software and devices) to do their job in an uncoupled and asynchronous way. It means that each client observes a message and takes action if necessary. They can also post new messages themselves. For instance, if a thermostat sends a message indicating high temperature in the topic “building A”, the air conditioning installed in that building and listed in the topic “building A” can take this message from there and turn itself on to reduce the temperature. However, it is common to see server-side software with business rules automated to turn on the air conditioning when the temperature reaches a specific value. The MQTT broker can also guarantee message delivery if it is configured to retain the messages when down and restore them when on again. Other advantages of the MQTT Broker include (from Wikipedia): Eliminating vulnerable and insecure client connections, if configured to do so. Being able to easily scale from a single device to thousands of gadgets. Managing and tracking all client connection states, including security credentials and certificates, if configured to do so. Reducing network strain without compromising the security, if configured to do so (when working with a cellular or satellite network). MQTT messages There are 3 types of messages: Connect: establish a connection between the broker and clients in a TCP/IP session. Disconnect: disconnect the server and the client from a TCP/IP session. Publish: put a message into a topic and send a copy of that message to each client that subscribed to the topic. Check out an example of messages in action: InterSystems IRIS support to IoT and MQTT protocol According to the InterSystems IRIS documentation, the currently supported version is MQTT 3.1. This MQTT specification is defined as an OASIS standard, MQTT Version 3.1.1. The Interoperability module from IRIS defines an Inbound (to consume MQTT messages) and an Outbound (to produce MQTT messages) MQTT Adapter. It is possible to develop custom business services and operations using these adapters, or you can use the built-in business service and operation, EnsLib.MQTT.Service.Passthrough and EnsLib.MQTT.Operation.Passthrough, respectively. If you want to use the MQTT protocol outside of an interoperability production, you can use the lower-level %Net.MQTT classes. The MQTT classes use the Eclipse Paho MQTT C Client Library. To create a Business Service to consume MQTT messages Class packagename.NewService1 Extends Ens.BusinessService { Parameter ADAPTER = "EnsLib.MQTT.Adapter.Inbound"; Method OnProcessInput(pInput As EnsLib.MQTT.Message, pOutput As %RegisteredObject) As %Status { set tsc=$$$OK set messageContent = pInput.StringValue …. Quit tsc } } To create a Business Operation to produce/send MQTT messages Class packagename.NewOperation1 Extends Ens.BusinessOperation { Parameter ADAPTER = "EnsLib.MQTT.Adapter.Outbound"; Parameter SETTINGS = "-SendSuperSession"; Method OnMessage(pRequest As packagename.Request, Output pResponse As packagename.Response) As %Status { #dim tSC As %Status = $$$OK #dim e As %Exception.AbstractException Try { Set message = ##class(EnsLib.MQTT.Message).%New() Set message.Topic = ..Adapter.Topic Set jsonValue = {} Set jsonValue.message = "Response message” Set message.StringValue = jsonValue.%ToJSON() Set tSC=..Adapter.Send(message.Topic,message.StringValue) Set pResponse = ##class(packagename.Response).%New() Set pResponse.message = “Message response” } Catch e { Set tSC=e.AsStatus() } Quit tSC } Settings for the Inbound and Outbound MQTT Adapter Both Business Service and Business Operation can be configured with the following parameters (source: https://docs.intersystems.com/irisforhealthlatest/csp/docbook/DocBook.UI.Page.cls?KEY=EMQTT_reference): Client ID: This is the string that identifies this client to the broker. It must be ASCII-encoded and contain between 1 and 23 characters. Connect Timeout: This is the connection timeout. Connecting to a busy server may take some time, so this timeout can be used to avoid a premature connection failure. It specifies the number of seconds to wait before a connection attempt fails. Credentials Name: This is the ID name of the set of credentials values used to access the MQTT broker. The username and password defined in your Credentials item must be ASCII-encoded. It is not needed if the broker does not require login credentials. Keep Alive: Specifies the maximum number of seconds to pass between messages sent by the client to the broker. QOS: This determines the quality of service required. It can have either of these two values: 0 - QOSFireAndForget: Do not wait for a response from the broker and 1 - QOSWaitForDelivery: Wait for a response from the broker and issue an error if the broker does not respond. Retained: This is the flag that indicates to the broker whether the message should be retained. Topic: This is the name of the topic to which you wish to publish or subscribe. The topic must be ASCII-encoded. The topic is typically a hierarchical string with levels of subtopics separated by a / (forward slash). In a subscription, a topic can have wildcards as a topic level. URL: This is the URL of the broker with which you wish to communicate. The scheme is either “tcp” or “ssl” followed by the domain name and the port delimited by a “:”, for example, “tcp://BIGBADAPPLE.local:1883”. Typically TLS-enabled endpoints are configured with a port of 8883, but this is not mandatory. Download and Install the Sample application The IRIS IoT Sample is a simple application to show you how to consume (receive) and produce (send) MQTT messages using an Interoperability Production. To get it, go to https://openexchange.intersystems.com/package/IoT-Sample. Now, execute the next steps. 1. Clone/git pull the repo into any local directory $ git clone https://github.com/yurimarx/iris-iot-sample.git 2. Open the terminal in this directory and run: $ docker-compose build 3. Run the IRIS container with your project: $ docker-compose up -d If you want to install it using ZPM, follow the next steps: 1. Open IRIS Namespace with Interoperability Enabled.2. Open Terminal and call: USER>zpm "install iris-iot-sample" Run the Sample Application. 1. Open the production and start it. It will begin observing the MQTT topic /DeviceStatusInputTopic and produce responses to the MQTT topic /DeviceStatusOutputTopic. Check out: 2. Use an MQTT client to send a message and test the production. To do it, go to https://chrome.google.com/webstore/detail/mqttbox/kaajoficamnjijhkeomgfljpicifbkaf on your Google Chrome browser. It is the Chrome Plugin MQTTBox. Just click Add to Chrome and then Add App. 3. In your Google Chrome browser go to chrome://apps/ and select MQTTBox (if required, click Open Anyway). 4. Click Create MQTT Client button. 5. Configure the MQTT connection with these settings: Client name: Local Protocol: mqtt / tcp Host: localhost:1883 Username: admin Password: admin All other settings stay the default values. 6. Configure the MQTT topics to send and receive MQTT messages: Topic to publish: /DeviceStatusInputTopic Topic to subscribe: /DeviceStatusOutputTopic Payload: { "deviceId":"Air Conditioner Level 1", "statusDate":"2023-01-07 14:03:00", "status": 0 } 7. Click the button Subscribe to check the messages on the topic /DeviceStatusOutputTopic. 8. Click the button Publish to send a message to /DeviceStatusInputTopic and see the results produced by IRIS production on /DeviceStatusOutputTopic. 9. Check out the message processing session on IRIS Management Portal Visual Trace The Sample Application source code The Dockerfile ARG IMAGE=intersystemsdc/irishealth-community ARG IMAGE=intersystemsdc/iris-community FROM $IMAGE WORKDIR /home/irisowner/irisbuild ARG TESTS=0 ARG MODULE="iris-iot-sample" ARG NAMESPACE="USER" RUN --mount=type=bind,src=.,dst=. \ iris start IRIS && \ iris session IRIS < iris.script && \ ([ $TESTS -eq 0 ] || iris session iris -U $NAMESPACE "##class(%ZPM.PackageManager).Shell(\"test $MODULE -v -only\",1,1)") && \ iris stop IRIS quietly The latest version of InterSystems IRIS Community is used to create a docker instance of InterSystems IRIS with a namespace called USER. The docker-compose file version: '3.6' services: mosquitto: image: eclipse-mosquitto:2 container_name: mosquitto user: root volumes: - ./mosquitto/config/:/mosquitto/config/ - ./mosquitto/log/:/mosquitto/log/ - ./mosquitto/data/:/mosquitto/data/ ports: - 1883:1883 - 9001:9001 iris: build: context: . dockerfile: Dockerfile restart: always command: --check-caps false ports: - 1972 - 52795:52773 - 53773 volumes: - ./:/irisdev/app Two docker container instances are now created, and up and running. The first one is for the MQTT broker (mosquitto service), based on the Eclipse Mosquitto product, and the second one is for the MQTT server-side software, responsible for consuming and producing MQTT messages in order to provide Device Monitoring services.The mosquitto docker instance is configured in the file /mosquitto/config/mosquitto.conf. The port, message persistence, security and log questions are defined there. The file /mosquitto/config/password.txt determines the user as admin with the password admin too. However, it is encrypted by the command mosquitto_passwd -U password.txt (you can read more about this at https://mosquitto.org/man/mosquitto_passwd-1.html). The iris.script file zn "%SYS" Do ##class(Security.Users).UnExpireUserPasswords("*") zn "USER" Do ##class(EnsPortal.Credentials).SaveItem(0, "mosquitto_cred","mosquitto_cred","admin","admin","") zpm "load /home/irisowner/irisbuild/ -v":1:1 halt This file creates the credentials for the business service and business operation. Login to the MQTT Broker and run the ZPM file module.xml. The module.xml This file is used to compile the source code on the server and to install the sample application when using ZPM. <?xml version="1.0" encoding="UTF-8"?> <Export generator="Cache" version="25"> <Document name="iris-iot-sample.ZPM"> <Module> <Name>iris-iot-sample</Name> <Description>A simple IRIS interoperability application - for IoT using MQTT.</Description> <Version>1.0.8</Version> <Packaging>module</Packaging> <Dependencies> <ModuleReference> <Name>sslclient</Name> <Version>1.0.1</Version> </ModuleReference> </Dependencies> <SourcesRoot>src</SourcesRoot> <Resource Name="dc.irisiotsample.PKG"/> <SystemRequirements Version=">=2020.1" Interoperability="enabled" /> </Module> </Document> </Export> On the SourceRoot tag there are source packages to be compiled. The DeviceStatus persistent class Class dc.irisiotsample.DeviceStatus Extends %Persistent { Property deviceId As %String; Property statusDate As %TimeStamp; Property status As %Boolean; Storage Default { <Data name="DeviceStatusDefaultData"> <Value name="1"> <Value>%%CLASSNAME</Value> </Value> <Value name="2"> <Value>deviceId</Value> </Value> <Value name="3"> <Value>statusDate</Value> </Value> <Value name="4"> <Value>status</Value> </Value> </Data> <DataLocation>^dc.irisiotsample.DeviceStatusD</DataLocation> <DefaultData>DeviceStatusDefaultData</DefaultData> <IdLocation>^dc.irisiotsample.DeviceStatusD</IdLocation> <IndexLocation>^dc.irisiotsample.DeviceStatusI</IndexLocation> <StreamLocation>^dc.irisiotsample.DeviceStatusS</StreamLocation> <Type>%Storage.Persistent</Type> } } This class is used to persist the DeviceStatus in an SQL table. The DeviceStatusRequest class Class dc.irisiotsample.DeviceStatusRequest Extends Ens.Request { Property deviceId As %String; Property statusDate As %TimeStamp; Property status As %Boolean; Storage Default { <Data name="DeviceStatusRequestDefaultData"> <Subscript>"DeviceStatusRequest"</Subscript> <Value name="1"> <Value>deviceId</Value> </Value> <Value name="2"> <Value>statusDate</Value> </Value> <Value name="3"> <Value>status</Value> </Value> </Data> <DefaultData>DeviceStatusRequestDefaultData</DefaultData> <Type>%Storage.Persistent</Type> } } This class is employed to exchange data between interoperability components. The DeviceStatusService Class dc.irisiotsample.DeviceStatusService Extends Ens.BusinessService { Parameter ADAPTER = "EnsLib.MQTT.Adapter.Inbound"; Method OnProcessInput(pInput As EnsLib.MQTT.Message, pOutput As Ens.StringResponse) As %Status { set tsc=$$$OK set DeviceStatusValue = ##class(%DynamicAbstractObject).%FromJSON(pInput.StringValue) set DeviceStatus = ##class(dc.irisiotsample.DeviceStatusRequest).%New() set DeviceStatus.deviceId = DeviceStatusValue.deviceId set DeviceStatus.statusDate = DeviceStatusValue.statusDate set DeviceStatus.status = DeviceStatusValue.status set tsc =..SendRequestSync("DeviceStatusProcess", DeviceStatus, .Response, -1, "Device Status Process") set pOutput = Response quit tsc } } The parameter adapter is indicating the use of the MQTT adapter when it comes to receiving MQTT messages. The method OnProcessInput receives the MQTT message on pInput and sends it to the Business Process, calling the method SendRequestSync as a synchronous message. The DeviceStatusProcess Class dc.irisiotsample.DeviceStatusProcess Extends Ens.BusinessProcess { Method OnRequest(request As dc.irisiotsample.DeviceStatusRequest, Output response As Ens.StringResponse) As %Status { Set tsc = 1 Set response = ##class(Ens.StringResponse).%New() Set DeviceStatus = ##class(dc.irisiotsample.DeviceStatus).%New() Set DeviceStatus.deviceId = request.deviceId Set DeviceStatus.status = request.status Set DeviceStatus.statusDate = request.statusDate Set tsc = DeviceStatus.%Save() If $$$ISOK(tsc) { Set tsc =..SendRequestSync("DeviceStatusOperation", request, .pResponse, -1, "Device Status Operation") Set response.StringValue = "Device id "_pResponse.deviceId_" has the status "_pResponse.status } Else { Set response.StringValue = "Error on save the device status" Set SuspendMessage = 1 } quit tsc } Storage Default { <Type>%Storage.Persistent</Type> } } This class receives the message from the business service, saves it to the database, and calls the DeviceStatusOperation to send (produce) a message with the results. The DeviceStatusOperation Class dc.irisiotsample.DeviceStatusOperation Extends Ens.BusinessOperation { Parameter ADAPTER = "EnsLib.MQTT.Adapter.Outbound"; Parameter SETTINGS = "-SendSuperSession"; Method NotifyDeviceStatus(pRequest As dc.irisiotsample.DeviceStatusRequest, Output pResponse As dc.irisiotsample.DeviceStatusResponse) As %Status { #dim tSC As %Status = $$$OK #dim e As %Exception.AbstractException Try { Set message = ##class(EnsLib.MQTT.Message).%New() Set message.Topic = ..Adapter.Topic Set jsonValue = {} Set jsonValue.message = "Device "_pRequest.deviceId_" has status "_pRequest.status Set message.StringValue = jsonValue.%ToJSON() Set tSC=..Adapter.Send(message.Topic,message.StringValue) Set pResponse = ##class(dc.irisiotsample.DeviceStatusResponse).%New() Set pResponse.deviceId = pRequest.deviceId Set pResponse.status = pRequest.status } Catch e { Set tSC=e.AsStatus() } Quit tSC } XData MessageMap { <MapItems> <MapItem MessageType="dc.irisiotsample.DeviceStatusRequest"> <Method>NotifyDeviceStatus</Method> </MapItem> </MapItems> } } This class receives a message from the Business Process and produces an MQTT message to a topic with the process response. The DeviceStatusProduction Class dc.irisiotsample.DeviceStatusProduction Extends Ens.Production { XData ProductionDefinition { <Production Name="dc.irisiotsample.DeviceStatusProduction" LogGeneralTraceEvents="false"> <Description></Description> <ActorPoolSize>2</ActorPoolSize> <Item Name="DeviceStatusService" Category="" ClassName="dc.irisiotsample.DeviceStatusService" PoolSize="1" Enabled="true" Foreground="false" Comment="" LogTraceEvents="false" Schedule=""> <Setting Target="Adapter" Name="ClientID">InterSystemsIRIS</Setting> <Setting Target="Adapter" Name="Topic">/DeviceStatusInputTopic</Setting> <Setting Target="Adapter" Name="Url">tcp://mosquitto:1883</Setting> <Setting Target="Adapter" Name="CredentialsName">mosquitto_cred</Setting> </Item> <Item Name="DeviceStatusProcess" Category="" ClassName="dc.irisiotsample.DeviceStatusProcess" PoolSize="1" Enabled="true" Foreground="false" Comment="" LogTraceEvents="false" Schedule=""> </Item> <Item Name="DeviceStatusOperation" Category="" ClassName="dc.irisiotsample.DeviceStatusOperation" PoolSize="1" Enabled="true" Foreground="false" Comment="" LogTraceEvents="false" Schedule=""> <Setting Target="Adapter" Name="ClientID">InterSystemsIRIS</Setting> <Setting Target="Adapter" Name="Topic">/DeviceStatusOutputTopic</Setting> <Setting Target="Adapter" Name="Url">tcp://mosquitto:1883</Setting> <Setting Target="Adapter" Name="CredentialsName">mosquitto_cred</Setting> </Item> </Production> } } This class configures the Business Service, the Business Process, and the Business Operation with the required parameters, and puts these components to work together. The main configured parameters are: URL: to set the MQTT Broker address. Topic: the queue to receive and send MQTT messages. DeviceStatusInputTopic for the business service and DeviceStatusOutputTopic for the Business Operation. CredentialsName: set the credentials with the username and password to connect with the MQTT broker. ClientID: it is a logical name assigned for the InterSystems IRIS and required by the broker to identify MQTT clients. As you can see, the IRIS interoperability adapter for MQTT is a very easy and powerful tool for consuming and producing MQTT messages and automating the business flow when it comes to IoT devices. So, enjoy it. Very good article, Yuri. Wonder when Intersystems will support AMQP protocol, to integrate seamlessly with ASB (Azure Service Bus) Thanks! Yaron, I'd appreciate if you submit it as an idea This is cool. Thank you very much. This gives you the opportunity to do a "Smart Home"" server in COS :-) - funny. Many thanks Thanks Yuri for this article and app. As MQTTBox doesn't work for me (no longer compatible), I'm using MQTT Explorer app to send messages :https://apps.apple.com/app/apple-store/id1455214828 Thanks! Of course MQTT Explorer also works on Windows, Ubuntu and any Linux platforms. http://mqtt-explorer.com/ https://github.com/thomasnordquist/MQTT-Explorer/releases

Hello Community! Welcome to the new edition of the InterSystems Ideas News! Learn what we've been up to this past couple of weeks. Curious about what is going on with all the great ideas you've been submitting to our InterSystems Ideas Portal? Here is the current status breakdown: ✓ 58 ideas are being triaged by InterSystems Product Managers. ✓ 43 ideas can be implemented by Developer Community members. ✓ 11 ideas are being implemented by InterSystems. ✓ 2 ideas are already implemented by InterSystems. ✓ 9 ideas are implemented by Developer Community members. To make it clearer what stages your ideas are going through here is the diagram: And to round up this newsletter, here is a list of ideas posted after Idea-A-Thon Improve Ukrainian translation in IRIS Code example Full Code Debugger Promote video contest Create a tool for IRIS BI to test all the pivots and dashboards if they still work after changes made Improve Spanish translation in IRIS Add IRIS as a supported database for Apache Superset For community articles, let admins (and possibly article authors) pin particular comments to the top Add address standardization to Normalization (using Project US@ standards) A tiny reminder, you can filter ideas by status, post new ideas for public discussion, vote for existing ideas, and comment on them on our InterSystems Ideas Portal! Stay tuned for the next InterSystems Ideas news bulletin and get creative in the meantime! Hello Community! Several users have asked me why the number of ideas in some statuses they see in the InterSystems Ideas is different from the number I mentioned in the news bulletin.I will try to explain it in this comment. Experts analyze your ideas daily. Thus, the status of analyzed or implemented ideas changes. In general, the number of ideas in the "Needs review" status is decreasing. And the number of ideas implemented by InterSystems and ideas that can be implemented by members of the Developer Community is increasing. Have a nice day. Vadim. Hello Community! I want to inform you that InterSystems Ideas now has a Portal Guide. The Portal Guide contains: information about the Ideas Portal goal, a complete list of ideas statuses, some links related to the portal. Please share your thoughts here or @Vadim.Aniskin about what else we can add there. Have a nice day, Vadim.

Hey Community! Welcome to the 4th edition of the InterSystems Ideas News bulletin! Here's what you can expect from it: Support your teammates with their ideas and votes. Users implemented ideas and gained tech bonuses for a contest. Dark version for InterSystems Developer Community: Yay or Nay? Recently added ideas. The most important piece of news is that in the last programming contest, 5 participants got extra points for implementing ideas from the InterSystems Ideas portal! Here they are: Developer(s) Application name Implemented idea @Lorenzo.Scalese OpenAPI-Suite Add a wizard similar to the SOAP wizard to generate a REST client from OpenAPI specification @José.Pereira @Henrique.GonçalvesDias @Henry.HamonPereira iris-tripleslash Add a project that helps to generate unittests for an ObjectScript class @MikhailenkoSergey gateway-sql, apptools-admin Create a UI for convenient and easy transfer of projects (classes, globals, applications, users, roles, privileges, grants, namespace mapping, SQLgateways, libraries, etc.) to other system instances for fast deployment. We added a new filter "My organization". From now on, people from the same company can filter the ideas and votes of people from the same organization. The other day, @Guillaume.Rongier7183 posted an idea concerning adding a dark theme for the Developer Community. We would appreciate your feedback on this idea via "polls" on the Ideas Portal home page, or by voting and commenting on this idea. Recently added ideas Dark version of InterSystems Community Add job location for job opportunities section on DC Searchable Index of New Features Set password through environment variable Schedule the article publication on DC site Support JSON on DC text editor Support linux bash language on DC text editor Change data capture from IRIS to kafka using SQL commands Cross-production Interoperability Messages, Service and Operation Additional Data Types for ISC Products Automatic XML Formatting of XData in DTL, BPL and Rulesets Assert CodeAssist prompting IRIS classes for OpenAI API Display UserValues contents on message property tab IRIS as a service Allow graphical editing of Interoperability components BPL, DTL and Business Rules in VS Code Add Source control for all the IRIS interoperability components Audit Lookuptables Add a "watch list" to WRC tickets (problems, issues) Connect InterSystems Documentation and Developer Community Publish the InterSystems IRIS Native SDK for Node.js on npm As usual, post your ideas on InterSystems Ideas, vote and comment on existing ideas, and implement Community Opportunity ideas. And stay tuned for our next News bulletin! 

Hi Developers! Welcome to the 5th issue of the InterSystems Ideas News! This time you can read about: ​​​​✓ Hall of Fame - a new page on the Ideas Portal ✓ Integration with Global Masters - get points for your ideas ✓ List of ideas that are planned for implementation 11 developers have already implemented ideas from the Ideas Portal. We created a new dedicated page on InterSystems Ideas to pay tribute to these heroes. The Hall of Fame lists: names of implemented ideas, developers who implemented ideas, implementation names with links to more information. You can implement one of Community Opportunity ideas and your name will be in the Hall of Fame! About a month ago, developers who submitted product ideas started getting points for these ideas. We would like to share that since 22 February, the authors have received a total of 18,200 Global Masters points for the following ideas 15 product ideas that were posted, promoted, or implemented: Cross-production Interoperability Messages, Service and Operation by @Stefan.Cronje1399 Additional Data Types for ISC Products by @Stefan.Cronje1399 Change data capture from IRIS to kafka using SQL commands by @Yuri.Gomes Allow graphical editing of Interoperability components BPL, DTL and Business Rules in VS Code by @Steve.Pisani Examples to work with IRIS from Django by @Evgeny.Shvarov Install python and java libraries from ZPM and Installation Manifest (%Installer) by @Yuri.Gomes Set password through environment variable by @Dmitry.Maslennikov Add a project that helps to generate unittests for an ObjectScript class by @Evgeny.Shvarov Create a UI for convenient and easy transfer of projects (classes, globals, applications, users, roles, privileges, grants, namespace mapping, SQLgateways, libraries, etc.) to other system instances for fast deployment. by @MikhailenkoSergey Add a wizard similar to the SOAP wizard to generate a REST client from OpenAPI specification by @Jaime.Lerga Public API for access to shared memory by @Alexey.Maslov Fold code on loops and If's on studio by @Heloisa.Paiva Chat bot to help with TrakCare customization/settings by Sumana Gopinath Iterative build of TrakCare configuration/code tables utilising FHIR and HL7 Messaging. by Linda McKay BPL, DTL, Business Rule Editor in VSCode by @Cristiano.Silva Post your great ideas and get points for them! And to round up this newsletter, here is the list of ideas that are planned for implementation Publish the InterSystems IRIS Native SDK for Node.js on npm by @John.Murray Move users, roles, resources, user tasks, Mappings (etc) to a seperate Database, other than %SYS, so these items can be mirrored by @Sean.O'Connor1391 Please add google oauth authorization to login to the management portal by @Aleksandr.Kolesov InterSystems Ideas - Long Term by @Vinay.Purohit3109 BPL, DTL, Business Rule Editor in VSCode by @Sawyer.Butterfield Add Favorites in GM by @Irène.Mykhailova LIMIT OFFSET support for IRIS SQL by @Dmitry.Maslennikov Introduce WITH into IRIS SQL engine by @Evgeny.Shvarov Security settings for mirror configurations by @Evgeny.Shvarov A modern management portal to manage InterSystems IRIS by @Evgeny.Shvarov copy/sync system configurations and user accounts between IRIS instances by @Evgeny.Shvarov Jupyter Notebook by Guest Stay creative, post your great ideas on InterSystems Ideas, vote and comment on existing ideas! Hi Community! I want to say special thanks to the developers who implemented the ideas from InterSystems Ideas: @Lorenzo.Scalese@Robert.Cemper1003 @MikhailenkoSergey @Dmitry.Maslennikov @Evgeniy.Potapov @Henry.HamonPereira @Henrique.GonçalvesDias @José.Pereira @Yuri.Gomes @Evgeny.Shvarov @Guillaume.Rongier7183 Your names are on the special Hall of Fame page!  Many thanks

Hi! Recently GitHub introduced topics for the projects. So you can change your InterSystems related projects introducing the topics to let it be categorized, more visible and searchable. Here is the list of good examples for your projects (some of them are clickable already): intersystems, intersystems-cache, intersystems-ensemble, intersystems-healthshare, healthshare, intersystems-iknow, iknow, intersystems-deepsee, deepsee, cache-objectscript, csp, intersystems-zen, zen. If you have any good ideas for topics or already using something, please introduce it here in the comments? Better with a working link. Thank you in advance! It's called topics. Tag is a git pointer to a commit (usually to mark release point). Thanks Eduard! Fixed and changed ) If you are looking for the projects with Caché ObjectScript with sources in UDL you can find it with cacheobjectscript-udl topic on Github. And if you have some open projects on Github with sources in UDL please mark it with cacheobjectscript-udl topic? Thank you in advance!