Configure an environment to use Docker on Hyper-V Ubuntu virtual machines running on a Windows 10 host

Primary tabs

This time I want to talk about something not specific to InterSystems IRIS, but that I think is important if you want to work with Docker and your server at work is a PC or laptop with Windows 10 Pro or Enterprise.

As you likely know, containers technology comes basically from Linux world and, nowadays, is on Linux hosts were it shows maximum potential. Those who use Windows on a normal basis see that both, Microsoft and Docker, have done important efforts during these last years that allow us to run containers based on Linux images on our Windows system in a really easy way... but it's something not supported for production systems and, this is the big problem, is not reliable if we want to keep persistent data outside of containers, in the host system,... mostly due to the big differences between Windows and Linux file systems. In the end, Docker for Windows itself uses a small linux virtual machine (MobiLinux) to run the containers... it does it transparently for the windows user... and it works perfectly well if, as I said, you don't require that your databases survive longer than the container...

Well,...let's get to the point,... the point is that many times, to avoid issues and simplify, we need a full Linux system and, if our server is based on Windows, the only way of having it is through a virtual machine. At least till WSL2 in Windows is released, but that will be another story and sure it'll take a bit of time to become robust enough.

In this article, I'll tell you, step by step, how to install an environment where you'll be able to work, if you need it, with Docker containers on an Ubuntu system in your Windows server. Let's go...

1. Enable Hyper-V

If you don't have it enabled yet, go to add Windows Features and enable Hyper-V. You'll need to restart (text images is in Spanish, but that's my current locale. I hope that altogether with instructions will help to "decrypt" it if you don't know the Don Quixote's lingua 😉 ) 

 

2. Create an Ubuntu virtual machine on Hyper-V

I don't think there is any easier way of creating a virtual machine (VM). Just open the window of the Hyper-V Manager,  and go to option  Quick Create... (right up in your screen) and create your virtual machine using any of Ubuntu versions already offered (you could download an iso file of whatever other Linux and so create the VM with a different distro). In my case, I've chosen the last Ubuntu release available: 19.10. Anyway, everything you'll see here is valid for 18.04. In 15 or 20 minutes, depends on what the image takes to download for you, you'll have your new VM created and ready.

Important: Leave the option of Default Switch as it's offered . This will guarantee you have access to internet from both, the host and the virtual machine.

3. Create a local subnet

One of the problems of using vitual machines that I've found very often has to do with network configuration... sometimes works, others don't, or it works if I'm connected with Wi-fi but not by cable or the opposite, or if I stablish a VPN in the windows host, then I lose internet access in the VM, or communication between the VM (Linux) and the host (Windows) gets broken... in short... it's crazy!  Makes me not to trust in my environment when I use my laptop for development, small and quick demos or for presentations, where most likely access to internet is not as important as being sure that communications between my host and my VM(s) work in a reliable way.

With an ad-hoc local subnet, shared between your Windows host and your virtual machines, you solve it. To let them communicate among each other, you use that subnet and that's it. You just have to assign specific IPs to your host and your VMs and all set.

It's really easy to do it with these steps. Just go to Virtual Switch Manager...  that you'll find in your  Hyper-V Manager:

Once there, go to option New  Virtual Switch  (it would be like a new network card for the VM afterwards):

Be sure you define it as Internal Network , choose the name we want and leave the other options by default

Now, if we go to Windows Control Panel --> Network and Sharing Center, we'll see that we already have there the switch we just created:

 

4. Configure the local Subnet shared by the host and the Virtual Machines

At this point you can finish the configuration of your new local network. To do that, set your cursor over the connection  Mi Nuevo Conmutador LOCAL,  click and go to Properties, and from there to IPv4 protocol so to assign a fixed IP address:

 

Important: The IP that you assign here will be your host (Windows) IP in this local subnet.

 

5. Link and configure your new local network to your virtual machine

Now come back to your  Hyper-V Manager. If your VM is running, stop it. Once stopped, go to its configuration and add the new internal virtul switch:

(Note.- In the image you can see another switch, the Hyper-V Conmutador INTERNO . It's for another subnet that I have. But it's not necessary for you in this configuration) 

Once you click on Add, you will just have to select the switch that you previously created:

Well, once this is done, click Apply, Accept, ... and you're ready! You just can start and login again in your virtual machine to finish the configuration of the internal connection. To do that, once the VM starts, clicnk on the network icon (right up) and you'll see that you have 2 networks: eth0 y eth1. The eth1 appears as disconnected... by now:

Go into the configuration of the Ethernet (eht1) and assign a fixed IP for this local subnet, for example: 155.100.101.1, and the subnet mask: 255.255.255.0

and that's all. Here you have your virtual machine, identified with IP 155.100.101.1 sharing the same subnet with your host.

7. Allow access to Windows 10 from your virtual machine

You will likely discover that Windows 10 doesn't allow by default connection from other server and, for your windows system, the VM you've just created is precisely that, an external and potentially dangerous server... so you'll have to add a rule in the Firewall to be able to connect to your host from those virtual machines. How? Very easy, just look for Windows Defender Firewall in your Windows Control Panel, go to Advance Configuration and  create a new Entry Rule:

You can set one port or one or many range of them... (also you can set the rule for all the ports)...

The action we want is Allow Connection...

For all network types...

Give a name to your rule...

And important, inmediately after that, open again the properties of your newley created rule and limit the application scope, so to apply only for connections within your local Subnet...

8. READY. Install Docker and whatever other application in your new Ubuntu Virtual Machine

Once you go through the entire installation process and you have your new VM ready and up-to-date, with internet access, etc.. you can install the apps you want... Docker as a minimum, that was the idea from the beginning, also you can install your VPN client if you need it for connection with your company network, VS Code, Eclipse+Atelier,...

Specifically, to install Docker, within your VM, you can follow the instructions you'll find here: https://docs.docker.com/install/linux/docker-ce/ubuntu/

Be sure that your Docker runtime is working, download some test image, etc... and that's it.

With this... ¡You're all set!, now you'll be able to have containers running with no limitations (apart from your hardware capacity) in your Ubuntu VM, to which you'll be able to connect from your Windows 10 host, from a browser or an app and the other way around, from your Ubuntu VM  to your Windows 10 host. All of that using  your IP addresses set in your shared local subnet, that will work no matter if you have a VPN stablished or not, if you access to internet through your Wi-fi adapter or trhough your ethernet cable.

Ah... one last advice. If you want to exchange files between Windows 10 and your virtual machines, a very useful and simple option is using WinSCP. It's free and it works really well.

Well, surely there are other configurations... but this is the one that I use and has proved to be the more reliable. I hope you find it useful also. If I've avoid you any headache, this article will have been worthy.

¡Happy Coding! 

 

 

Replies

@Luca Ravazzolo , @Jose Tomas Salvador 

I succeeded to force an upgrade of my Win 10 to  2004 and to activate WSL2   (installation blocked until VMware was de-installed !)
and Docker Desktop For WSL2.
Quite interesting to have a Windows-Installer for a Linux program. Take care to use version >2.3.0 2  for WSL2
And it was running as expected with no troubles. No issues with Images, Containers, Volumes, Networks.

It's an interesting experience, especially as I'm not forced to use PowerShell.
Communication with Win File System goes Docker ==> WSL2 ==> Windows.
Next positive surprise: my local C:\..  was mounted byWSL2 to /mnt/c/....  (I just tried read access)

Bad surprises:
- the guide to install / activate was  just of limited help, partially confusing / contradictionary   
Google helped with useful and just commercial advice.
- Reboots: after #5 I stopped counting 

Still a challenge: I couldn't find out where WSL2 file system was homed on my C:\ drive.
 

I'm running also with last Windows 10 version and WSL2... it goes really well till now. I found that WSL fights for resources with Windows and although it releases RAM and processors when they're not used in a while, it's not as agile as I would like... so, the only tweak I had to do was to limit the resources that the WSL2 could take... to do that it's just required to add a new file in your user profile folder: c:\Users\<youruser>, call it .wslconfig and add something like:

[wsl2]
memory=6GB # Limits VM memory in WSL 2 to 6 GB
processors=2 # Makes the WSL 2 VM use two virtual processors

Terminate WSL2 and launch it again and that's all...  from then on, your WSL2 won't take more resources than specified... if, for some tests for example, you need more, you can change the file and increment the resources.... this is something that likely will be addressed in a more unmanaged way in the future... at least is what Microsoft WSL teams is saying in the forums.

BTW, you can find WSL2 well under c:\Users\<youruser>\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu.....  in the case of my Ubuntu distribution... but you can change where is stored: https://github.com/MicrosoftDocs/WSL/issues/412

to complete this.
Docker has its storage here.

C:\Users\<youruser>\AppData\Local\Docker\wsl\data\ext4.vhdx
C:\Users\<youruser>\AppData\Local\Docker\wsl\distro\ext4.vhdx

Another issue I had is the requirement of Hyper-V.

The latest VMware Player 15.5.6  has no problem, with Hyper-V enabled anymore.
So they can be installed  in parallel (if your disk space allows  devil)

¡Hola Salva! ( @Jose Tomas Salvador) 

A great contribution. Having this experienced myself I know how much work it was.
A few remarks: 

  • enabling Hyper-V killed my well-grown VMware environment
  • my attempts to install a pure Ubuntu Server distibution failed with various releases.   so I had to accept the pre-selected kits with the graphic interface.
  • the effort to get the network clean in a multi-machine environment was significant   and I had just a week feeling what action were expected from me.
  • finally mapping external volumes for durability wasn't easy and not successful.

Thanks to your description I know that my expectations were just higher than reality.

I finally dropped Docker desktop and returned to my VMware that I used already
before it was supported platform for Caché, wink .  
Especially networking was much easier (to me). 

Maybe a future version might be more convincing.
But as I developed a predecessor of VMware already in 1972.. I'm quite demanding on that subject.  

¡ Saludos desde Viena !
Robert


 

Hallo Robert!

Glad to hear it helped you! Now it's time to test with native support to Linux that comes with last version of Windows... let's see how it goes with WSL2!

Best regards.