go to post Rob Tweed · May 18, 2021 It's not a good idea to store files in the DB that you'll simply be reading back in full. The main issue you'll suffer from if you do hold them in the database (which nobody else seems to have picked up on) is that you'll needlessly flush/replace global buffers every time you read them back (the bigger the files, the worse this will be). Global buffers are one of the keys to performance. Save the files and files and use the database to store their filepaths as data and indices.
go to post Rob Tweed · May 18, 2021 Yes, I think that everyone who uses Cache or IRIS should take the time to discover what lies beneath! The power and flexibility of Global Storage is way beyond anything else out there - something that has been true ever since I first started working with such databases way back in the early 1980s. In all that time, I've never come across any other database architecture that is as quick and simple to grasp and yet as devastatingly powerful. I'm hoping our efforts in putting together these resources will help a new generation of developers discover and harness that unique magic.
go to post Rob Tweed · Feb 3, 2021 No disrespect to what you've created, but I have to say I'm always amazed and dismayed at how complex the Java community always seem to manage to make even the simplest of tasks - Spring/Hibernate is a classic example of making a crazy mountain out of what should technically be a very simple mole-hill. By comparison, take a look at mgweb-server: https://github.com/robtweed/mgweb-server and its underlying mg_web interface (https://github.com/chrisemunt/mg_web) for probably the thinnest (and therefore the most performant) and simplest way possible of delivering REST services using IRIS.
go to post Rob Tweed · Jan 26, 2021 The simplest way is to use QEWD. It will save you a lot of time trying to sort out all the moving parts required. Start here: https://github.com/robtweed/qewd and check this out: https://groups.google.com/g/enterprise-web-developer-community/c/HQK-THx... Rob
go to post Rob Tweed · Jan 20, 2021 Awaiting my app/repo to be approved and then I'll submit it to the competition... Rob
go to post Rob Tweed · Jan 18, 2021 We have two Open Source products that will look after JWTs for you in the ways you are asking about (ie REST services with IRIS): - QEWD, if you want to implement everything at the back-end in Node.js / JavaScript - mgweb-server if you want to use ObjectScript logic for your back-end logic For QEWD and IRIS, see: https://github.com/robtweed/qewd-starter-kit-iris-networked In particular for REST services, see: https://github.com/robtweed/qewd-starter-kit-iris-networked/blob/master/... and specifically this section: https://github.com/robtweed/qewd-starter-kit-iris-networked/blob/master/... For mgweb-server, see: https://github.com/robtweed/mgweb-server specifically using with IRIS: https://github.com/robtweed/mgweb-server/blob/master/IRIS.md and within that document, this section on JWTs: https://github.com/robtweed/mgweb-server/blob/master/IRIS.md#using-json-... Rob
go to post Rob Tweed · Jan 1, 2021 if you're a JavaScript/Node.js developer, you can use the QEWD-JSdb abstractions of IRIS: - Document database - Persistent JavaScript Objects - Redis-like Lists - Redis-like key/object store - Persistent XML DOM (with XPath querying) See: https://github.com/robtweed/qewd-starter-kit-iris-networked Take the introductory tutorial: https://github.com/robtweed/qewd-starter-kit-iris-networked/blob/master/... and then delve into the other database models: - https://github.com/robtweed/qewd-starter-kit-iris-networked/blob/master/... https://github.com/robtweed/qewd-starter-kit-iris-networked/blob/master/... https://github.com/robtweed/qewd-starter-kit-iris-networked/blob/master/... A whole world of multi-model opportunities to explore for this competition!
go to post Rob Tweed · Dec 3, 2020 If you're interested in trying out the mgweb-server Docker Container (aka mg_web Server Appliance) with an IRIS back-end (eg using the IRIS Community Edition Docker Container), I've put together a detailed user guide and tutorial that takes you through the entire process, step by step, showing you how to create REST APIs, along with showing you how to get it working with the pre-built mgweb-conduit back-end Demonstrator/Example on IRIS, and how to add the wc-conduit front-end to the mg_web Server Appliance. It's all very quick and simple to get it up and working, with lots of utility functions included in the kit that make it very simple to create your REST APIs, use JWTs, secure passwords etc. It's a different and alternative approach to using IRIS for REST APIs that you might find interesting. Why not take a look at: https://github.com/robtweed/mgweb-server/blob/master/IRIS.md
go to post Rob Tweed · Dec 1, 2020 Notice, by the way, the hefty performance penalty incurred by the SQL/Class option compared with the low-level ObjectScript functions ($extract and $translate).
go to post Rob Tweed · Nov 30, 2020 Results running IRIS Container on Raspberry Pi 4 (4Gb RAM): $e() only 1.279893 $tr() 1.281999 $e()+$zd() 1.470656 SQL/class 66.405927 I didn't include the SQL/static in my test on the RPi. Comparison with IRIS on a Windows 10 i7 based Intel NUC machine: $e() only .14742$tr() .185998$e()+$zd() .182579SQL/class 11.305143 Mind you, the NUC cost about 15 times that of the RPi :-) Anyway, based on a combination of the performance and coolness of technique, the $tr() one gets my vote!
go to post Rob Tweed · Nov 30, 2020 If it was me I'd use the persistent DOM of QEWD-JSdb https://github.com/robtweed/qewd-jsdb/blob/master/DOM.md QEWD includes access to this. Rob
go to post Rob Tweed · Nov 27, 2020 ..and, of course, you could now be running both the mgweb-server container (Apache + mg_web) and the IRIS container on Raspberry Pi 4's (with 0.5 million global node sets/second performance), to create a very cool, low-cost REST platform. BTW did you know that Raspberry Pi (the company) run their public web site on a cloud-based farm of Raspberry Pi 4s? These are the guys that run it for them: https://www.mythic-beasts.com/blog/2020/06/17/raspberry-pi-4-now-availab...
go to post Rob Tweed · Nov 27, 2020 I would add that it is worth remembering that you're not limited to using the technologies built-in to IRIS (or Cache for that matter). There are also proven Open Source alternatives available for building web applications that use IRIS/Cache as a back-end environment (or simply an ultra-high-performance database). So, if you want to develop applications in JavaScript/Node.js and use IRIS as a persistent JavaScript Object store, QEWD should be on your list. Alternatively, check out mg_web and mgweb-server as an alternative to the built-in REST solution. For more info see: https://github.com/chrisemunt?tab=repositories https://github.com/robtweed?tab=repositories https://github.com/robtweed/qewd-conduit/blob/master/IRIS-windows.md
go to post Rob Tweed · Nov 26, 2020 It's pretty fast two - I do have a pretty fast micro SD in the RPi4 but it looks like you can set 0.5m global nodes/sec in a very simple test: USER>s s=$p($zts,",",2) f i=1:1:1000000 s ^rob(i)=$r(1000) i i=1000000 w $p($zts,",",2)-s2.072394 Not bad for a $35 computer :-)
go to post Rob Tweed · Nov 26, 2020 Bingo - nice find, Dmitry Here we go on Ubuntu Server 20.04 on my RPi4: ubuntu@ubuntu:~$ sudo docker run --name my-iris -it --rm -p 9091:1972 -p 9092:52772 -p 9093:52773 -p 9094:7041 store/intersystems/iris-community-arm64:2020.3.0.221.0[WARN] No init process detected! This container may accumulate zombie processes if run for a long time. Consider using "docker create --init ..." or equivalent.[INFO] Starting InterSystems IRIS instance IRIS...[INFO] This copy of InterSystems IRIS has been licensed for use exclusively by:InterSystems IRIS CommunityCopyright (c) 1986-2020 by InterSystems CorporationAny other use is a violation of your license agreementStarting IRIS 11/26/20-11:02:27:145 (361) 0 [Generic.Event] Global buffer setting requires attention. Auto-selected 25% of total memory.11/26/20-11:02:27:436 (361) 0 [Generic.Event] Allocated 449MB shared memory: 227MB global buffers, 80MB routine buffers11/26/20-11:02:27:588 (361) 0 [WriteDaemon.UsingWIJFile] Using WIJ file: /usr/irissys/mgr/IRIS.WIJ11/26/20-11:02:27:588 (361) 0 [WriteDaemon.CreatingNewWIJ] Creating a new WIJ file11/26/20-11:02:34:614 (361) 0 [WriteDaemon.CreatedNewWIJ] New WIJ file created11/26/20-11:02:34:629 (361) 0 [Generic.Event]Startup of InterSystems IRIS [IRIS for UNIX (Ubuntu Server LTS for ARM64 Containers) 2020.3 (Build 221U) Thu Oct 1 2020 15:11:45 UTC] in /usr/irissys/bin/ with mgr: /usr/irissys/mgr with wij: /usr/irissys/mgr/IRIS.WIJ from: /usr/irissys/mgr/ OS=[Linux], version=[#25-Ubuntu SMP PREEMPT Thu Oct 15 13:31:49 UTC 2020], release=[5.4.0-1022-raspi], machine=[aarch64] nodename=[b6deee189a6f]. numasyncwijbuf: 8, swdwrtmax: 64, wijdirectio: on, synctype: 3 System Initialized.11/26/20-11:02:34:642 (362) 0 [WriteDaemon.Started] Write daemon started.11/26/20-11:02:45:083 (373) 0 [Database.MountedRW] Mounted database /usr/irissys/mgr/ (SFN 0) read-write.11/26/20-11:02:45:216 (373) 0 [Utility.Event] Instance 'IRIS' starting on node b6deee189a6f by user irisuser11/26/20-11:02:45:216 (373) 0 [Utility.Event] Using parameters from file '/usr/irissys/iris.cpf'11/26/20-11:02:45:221 (373) 0 [Utility.Event] Loading DLLs11/26/20-11:03:31:951 (373) 0 [Database.MountedRO] Mounted database /usr/irissys/mgr/irislib/ (SFN 1) read-only. Database label is marked read-only.11/26/20-11:03:32:147 (373) 0 [Utility.Event] Switching to temporary %SYS Namespace11/26/20-11:03:32:340 (373) 0 [Utility.Event] Loading Locale enuw (English, United States, Unicode) from objects11/26/20-11:03:34:020 (373) 0 [Database.MountedRW] Mounted database /usr/irissys/mgr/iristemp/ (SFN 2) read-write.11/26/20-11:03:34:073 (373) 0 [Utility.Event] /usr/irissys/mgr/iristemp/ initialized as IRISTEMP11/26/20-11:03:34:080 (373) 0 [Utility.Event] Switching to default %SYS Namespace11/26/20-11:03:34:199 (390) 0 [Utility.Event] Log Monitor Started11/26/20-11:03:34:334 (391) 0 [Utility.Event] Clean Daemon Started11/26/20-11:03:34:382 (373) 1 [Utility.Event] Configuration file /usr/irissys/iris.cpf is not the same as when last shut down11/26/20-11:03:35:657 (373) 0 [Utility.Event] Updating configuration information from /usr/irissys/iris.cpf11/26/20-11:03:35:759 (373) 0 [Utility.Event] Performing Journal Recovery11/26/20-11:03:35:764 (373) 0 [Utility.Event] Graceful system shutdown, journal restore not required11/26/20-11:03:35:765 (373) 0 [Utility.Event] Graceful system shutdown, transaction rollback not required11/26/20-11:03:35:790 (373) 0 [Utility.Event] START: /usr/irissys/mgr/journal/20201126.00111/26/20-11:03:35:853 (373) 0 [Generic.Event] INTERSYSTEMS IRIS JOURNALING SYSTEM MESSAGEJournaling started to: /usr/irissys/mgr/journal/20201126.00111/26/20-11:03:35:855 (373) 0 [Utility.Event] Journaling to /usr/irissys/mgr/journal/20201126.001 started.11/26/20-11:03:35:979 (373) 0 [Utility.Event] Processing Startup section11/26/20-11:03:36:001 (392) 0 [Utility.Event] Purging old application errors11/26/20-11:03:36:246 (373) 0 [Utility.Event] Initializing Security system11/26/20-11:03:37:325 (392) 0 [Database.MountedRW] Mounted database /usr/irissys/mgr/irislocaldata/ (SFN 3) read-write.11/26/20-11:03:37:495 (373) 0 [Utility.Event] Processing Network section11/26/20-11:03:37:502 (373) 0 [Utility.Event] Activating Network11/26/20-11:03:37:756 (373) 0 [Utility.Event] Processing Databases section11/26/20-11:03:37:959 (373) 0 [Database.MountedRW] Mounted database /usr/irissys/mgr/irisaudit/ (SFN 4) read-write.11/26/20-11:03:37:978 (373) 0 [Utility.Event] Processing Namespaces section11/26/20-11:03:37:979 (373) 0 [Utility.Event] Namespaces are up to date11/26/20-11:03:37:979 (373) 0 [Utility.Event] Activating namespaces11/26/20-11:03:37:991 (373) 0 [Utility.Event] Activating new namespace map11/26/20-11:03:38:157 (373) 0 [Utility.Event] Namespace changes have been activated11/26/20-11:03:38:363 (373) 0 [Utility.Event] Starting Super Server on port 197211/26/20-11:03:38:406 (373) 0 [Utility.Event] Network Lock Upload Phase Starting11/26/20-11:03:38:410 (373) 0 [Utility.Event] Lock Upload Phase Complete11/26/20-11:03:38:410 (373) 0 [Utility.Event] Processing Miscellaneous section11/26/20-11:03:38:632 (397) 0 [Utility.Event] LMF Info: Licensed for 5 users.11/26/20-11:03:38:644 (398) 0 [Utility.Event] Starting Servers11/26/20-11:03:38:644 (373) 0 [Generic.Event] init_gcr_seed: gen_crypt_rand seeded from /dev/urandom: 64 bytes.11/26/20-11:03:38:644 (373) 0 [Utility.Event] Processing Devices section11/26/20-11:03:38:663 (373) 0 [Utility.Event] Processing DeviceSubTypes section11/26/20-11:03:38:664 (373) 0 [Utility.Event] Processing MagTape section11/26/20-11:03:38:684 (373) 0 [Utility.Event] Processing IO section11/26/20-11:03:38:691 (373) 0 [Utility.Event] Processing SQL section11/26/20-11:03:38:806 (373) 0 [Generic.Event] Auditing to /usr/irissys/mgr/irisaudit/11/26/20-11:03:38:978 (373) 0 [Utility.Event] Executing ^ZSTU routine11/26/20-11:03:38:984 (373) 0 [Utility.Event] Executing ^%ZSTART routine11/26/20-11:03:38:985 (373) 0 [Utility.Event] Enabling logons11/26/20-11:03:40:649 (398) 0 [Utility.Event] Private webserver started on 5277311/26/20-11:03:40:650 (398) 0 [Utility.Event] Processing Shadows section (this system as shadow)11/26/20-11:03:40:806 (373) 0 [Database.MountedRW] Mounted database /usr/irissys/mgr/user/ (SFN 5) read-write.11/26/20-11:03:40:812 (398) 0 [Utility.Event] Processing Monitor section11/26/20-11:03:41:242 (547) 0 [Utility.Event] Starting TASKMGR11/26/20-11:03:41:404 (548) 0 [Utility.Event] [SYSTEM MONITOR] System Monitor started in %SYS11/26/20-11:03:41:451 (398) 0 [Utility.Event] Shard license: 011/26/20-11:04:00:517 (373) 0 [Database.MountedRO] Mounted database /usr/irissys/mgr/enslib/ (SFN 6) read-only. Database label is marked read-only.11/26/20-11:04:00:527 (373) 0 [Utility.Event] Initializing Interoperability during system startup[INFO] ...started InterSystems IRIS instance IRIS and then in another process I can shell in and run iris terminal IRIS and access the IRIS shell and database! Amazing!
go to post Rob Tweed · Nov 26, 2020 Actually I think that might be the answer - uname -m on the RPi4 I've been using says armv7l which means 32 bit - it should say aarch64. The version of Raspbian is pretty old on that particular RPi4 and I'm not sure if they have now put out a 64-bit version of Raspbian? Now I have another RPi4 (I did say I had loads of them, right!) running Ubuntu Server 20.04 which I believe is a 64 bit version - I'll give it a try on that one and report back Rob
go to post Rob Tweed · Nov 26, 2020 The latest model RPi 4 running the standard Raspian OS. I have other Docker containers running fine on it - eg my rtweed/qewd-server-rpi and rtweed/mgweb-rpi ones (available on Docker Hub) I've not tried the 3, but really there shouldn't be any difference as far as Docker is concerned - all my RPi containers work fine on the 3 and 4.
go to post Rob Tweed · Nov 26, 2020 Dmitry What I see on the Raspberry Pi is this if I try to start it (in foreground mode so I can see the error): $ docker run --name my-iris -it --rm -p 9091:1972 -p 9092:52772 -p 9093:52773 -p 9094:7041 store/intersystems/iris-community-arm64:2020.3.0.221.0standard_init_linux.go:211: exec user process caused "exec format error"