New post
Question
· May 7, 2020

Docker vs. Durability

#Containerization #Docker #Field Tests #Microsoft Windows Server #Tips & Tricks #Tools #InterSystems IRIS

Working from home during these Corona-days I'm short on resources.
- no Linux machine available
- limited disk space
So I decided to give Docker in Windows 10 (named Docker Desktop) a try. 
This worked nicely after some reboot (it's Windows !  sad )   and killing my VMware  sad to have Hyper-V .

Next, I tried to implement  Durable %SYS
And this was rather disappointing. 
Independent of what IRIS container I used (released or preview or community edition) it crashed at start.
Using docker run -it  ...   (instead of -d) to see what's happening uncovered this strange error.
 

05/05/20-21:59:12:977 (364) 0 [Utility.Event] Executing rm -f /usr/irissys/bin/irisinstall
05/05/20-21:59:12:981 (364) 0 [Utility.Event] Executing rm -f /usr/irissys/bin/irisuxsinst
05/05/20-21:59:12:986 (364) 0 [Utility.Event] Executing rm -f /external/durable/CloneIncomplete
05/05/20-21:59:12:992 (364) 0 [Utility.Event] Instance IRIS data directory moved to /external/durable
05/05/20-21:59:12:993 (364) 0 [Utility.Event] Executing find /usr/irissys/ -type d -exec chmod ugo-w {} +
05/05/20-21:59:13:028 (364) 0 [Utility.Event] Executing chmod ugo-w /usr/irissys/
[ERROR] Possible causes:
[ERROR]  - Incorrect user/group ownership of IRIS binaries
[ERROR]  - Insufficient write permissions to ISC_DATA_DIRECTORY
[FATAL] Error starting InterSystems IRIS


The reasons are the high lighted lines
ISC_DATA_DIRECTORY  had no protection at all and was not affected.
More detailed interactive tests showed a mismatch between file ownership and user in docker.

AS it happened with several IRIS containers I assume the problem in Desktop Docker and/or it'S underlying Windows 10.

As a workaround for me, I developed a customized lightweight durability solution that avoids this step. ​


 

Discussion (10)3
Log in or sign up to continue
Rich Taylor · May 7, 2020

Robert,

The cause is that fact that Desktop Docker (Docker for Windows) is a bit of a miss-leading.  This is not really using Windows containers at all by default, though it can.  From what I understand, while it is getting better, true Windows containers are still a bit problematic.  Also all our images are Ubuntu based anyway.  What is really happening is that there is a small Linux (Moby) vm running under Desktop Docker.  So when you share volumes from the windows file system to the container you are going through two transitions.  One from the Container to the Host Linux then from the Host Linux out to windows.   While this works for just Docker volumes, there are issues when trying to do Durable %SYS around permissions as you surmised.   I have heard of people getting into the Moby Linux environment and messing with the permissions, but this seems too much trouble and too fragile for my tastes.

You do have some options though.

  1. A better option might be to use WSL2 (Windows Sub-system for Linux).  This will enable you to run an Ubuntu environment within Windows.  You do need to be on WSL2 and not the first version which as too limited.  Here are a couple of links: https://code.visualstudio.com/blogs/2020/03/02/docker-in-wsl2https://www.hanselman.com/blog/HowToSetUpDockerWithinWindowsSystemForLinuxWSL2OnWindows10.aspx  
  2. you could just run a Linux VM  
  3. You could go with the nuclear option and just switch to Linux as a Desktop.  I took this route over a year ago, before WSL2 came out, and have not looked back.

Hope this helps.

1 0
Erik Hemdal · May 7, 2020

Robert, although we only support containers on Linux, we have some information concerning Docker for Windows since many people find themselves in just this situation.

https://community.intersystems.com/post/using-intersystems-iris-containers-docker-windows
 

For your Durable %SYS directory, try a named volume instead of a bind mount (use docker volume create ...).  

Thanks for your post and for all your contributions in the DC.   Erik

1 0
Conor Walsh · May 7, 2020

This is an interesting hack, but I can't recommend using Docker for Windows or Mac.  There's some detail on this in Shakespeare on Linux Containers, and @Rich Taylor did a great job of laying out the basics.  If you have the ability to run your own Linux VM, I recommend it - and it looks like you're already working on this, which is great!

The file ownership problem on Windows bind mounts can't be fixed, Windows doesn't have user/group/world permissions like Unix does, and it doesn't understand your Linux uids.  Even when we avoid that, the CIFS mechanism used to expose Windows folders into your container doesn't support all the different fsync() options needed to guarantee journal durability.  I don't know for sure, but I think the fsync problem is the reason that every written-to-disk DBMS that runs in containers has the advice of "when running on Docker for Windows, use a named volume and not a bind mount."

1 0
Robert Cemper · May 8, 2020

Just to complete this issue.
My final solution is a VM with Ubuntu 18.04 + Docker  (as suggested by @Rich Taylor ) .
After some troubles, I moved to a Docker VOLUME as advised by  @J. Hemdal  .

Docker Volumes are typically buried in a multilevel hard to remember directory tree. 
An easy to remember symlink solved that.
And it works fine in all variations. Full durability and lightweight.
Thanks for your contributions.

4 0