Docker vs. Durability

Question

Robert Cemper · May 7, 2020

#Containerization #Development Environment #Docker #Field Tests #Microsoft Windows Server #Tips & Tricks #Tools #InterSystems IRIS

Working from home during these Corona-days I'm short on resources.
- no Linux machine available
- limited disk space
So I decided to give Docker in Windows 10 (named Docker Desktop) a try.
This worked nicely after some reboot (it's Windows ! ) and killing my VMware to have Hyper-V .

Next, I tried to implement Durable %SYS
And this was rather disappointing.
Independent of what IRIS container I used (released or preview or community edition) it crashed at start.
Using docker run -it ... (instead of -d) to see what's happening uncovered this strange error.

05/05/20-21:59:12:977 (364) 0 [Utility.Event] Executing rm -f /usr/irissys/bin/irisinstall 05/05/20-21:59:12:981 (364) 0 [Utility.Event] Executing rm -f /usr/irissys/bin/irisuxsinst 05/05/20-21:59:12:986 (364) 0 [Utility.Event] Executing rm -f /external/durable/CloneIncomplete 05/05/20-21:59:12:992 (364) 0 [Utility.Event] Instance IRIS data directory moved to /external/durable 05/05/20-21:59:12:993 (364) 0 [Utility.Event] Executing find /usr/irissys/ -type d -exec chmod ugo-w {} + 05/05/20-21:59:13:028 (364) 0 [Utility.Event] Executing chmod ugo-w /usr/irissys/ [ERROR] Possible causes: [ERROR] -Incorrect user/group ownership of IRIS binaries
[ERROR] - Insufficient write permissions to ISC_DATA_DIRECTORY [FATAL] Error starting InterSystems IRIS

The reasons are the high lighted lines
ISC_DATA_DIRECTORY had no protection at all and was not affected.
More detailed interactive tests showed a mismatch between file ownership and user in docker.

AS it happened with several IRIS containers I assume the problem in Desktop Docker and/or it'S underlying Windows 10.

As a workaround for me, I developed a customized lightweight durability solution that avoids this step.

1 563

Rich Taylor · May 7, 2020

Robert,

The cause is that fact that Desktop Docker (Docker for Windows) is a bit of a miss-leading. This is not really using Windows containers at all by default, though it can. From what I understand, while it is getting better, true Windows containers are still a bit problematic. Also all our images are Ubuntu based anyway. What is really happening is that there is a small Linux (Moby) vm running under Desktop Docker. So when you share volumes from the windows file system to the container you are going through two transitions. One from the Container to the Host Linux then from the Host Linux out to windows. While this works for just Docker volumes, there are issues when trying to do Durable %SYS around permissions as you surmised. I have heard of people getting into the Moby Linux environment and messing with the permissions, but this seems too much trouble and too fragile for my tastes.

You do have some options though.

A better option might be to use WSL2 (Windows Sub-system for Linux). This will enable you to run an Ubuntu environment within Windows. You do need to be on WSL2 and not the first version which as too limited. Here are a couple of links: https://code.visualstudio.com/blogs/2020/03/02/docker-in-wsl2 https://www.hanselman.com/blog/HowToSetUpDockerWithinWindowsSystemForLinuxWSL2OnWindows10.aspx
you could just run a Linux VM
You could go with the nuclear option and just switch to Linux as a Desktop. I took this route over a year ago, before WSL2 came out, and have not looked back.

Hope this helps.

1 0

Thank you @Rich Taylor !
I was sure not to be the first. My story was more a warning not to spend to much time on it.
My workaround was an after-midnight-hack. Nobody around and I want this beast to go.
You know: "Once engineer, ever engineer! "
But I'm already working on suggestion #2. I have just to clean the virtual disk from old junk.

Best wishes, stay save!

@Robert Cemper

Whatever gets the process to work right!

I would consider WSL2 also. Its probably just me, but conceptually I had a problem with running a virtual environment (VM) so I could run a virtual environment (Container). WSL was not functioning well enough at the time so I went all in on Ubuntu as my native OS.

Good Luck ans

0 0

Tani Frankel · May 7, 2020

Re #2 you might find this related article by @Jose Tomas Salvador useful:

https://community.intersystems.com/post/configure-environment-use-docker-hyper-v-ubuntu-virtual-machines-running-windows-10-host

Hi Tani, ( @Netanel Frankel )

I've seen Salva's article and tried it. But I always ended with some graphic version.
Downloading an Ubuntu Server kit looked good in the beginning.
But when it asked me to initialize all 127GB of my only Harddrive I got scared and stopped it.

Erik Hemdal · May 7, 2020

Robert, although we only support containers on Linux, we have some information concerning Docker for Windows since many people find themselves in just this situation.

https://community.intersystems.com/post/using-intersystems-iris-containers-docker-windows

For your Durable %SYS directory, try a named volume instead of a bind mount (use docker volume create ...).

Thanks for your post and for all your contributions in the DC. Erik

Quoting Robert

But I'm already working on suggestion #2.

Much better approach! I think you'll be much more successful that way. Erik

Erik,

Nice, I was not aware that the create volume option would allow Durable %SYS to function.

Thanks for that

Conor Walsh · May 7, 2020

This is an interesting hack, but I can't recommend using Docker for Windows or Mac. There's some detail on this in Shakespeare on Linux Containers, and @Rich Taylor did a great job of laying out the basics. If you have the ability to run your own Linux VM, I recommend it - and it looks like you're already working on this, which is great!

The file ownership problem on Windows bind mounts can't be fixed, Windows doesn't have user/group/world permissions like Unix does, and it doesn't understand your Linux uids. Even when we avoid that, the CIFS mechanism used to expose Windows folders into your container doesn't support all the different fsync() options needed to guarantee journal durability. I don't know for sure, but I think the fsync problem is the reason that every written-to-disk DBMS that runs in containers has the advice of "when running on Docker for Windows, use a named volume and not a bind mount."

Robert Cemper · May 8, 2020

Just to complete this issue.
My final solution is a VM with Ubuntu 18.04 + Docker (as suggested by @Rich Taylor ) .
After some troubles, I moved to a Docker VOLUME as advised by @J. Hemdal .

Docker Volumes are typically buried in a multilevel hard to remember directory tree.
An easy to remember symlink solved that.
And it works fine in all variations. Full durability and lightweight.
Thanks for your contributions.

4 0

Docker vs. Durability

Social networks

InterSystems resources