I’m a retired Unix admin. It was my job from the early '90s until the mid '10s. I’ve kept somewhat current ever since by running various machines at home. So far I’ve managed to avoid using Docker at home even though I have a decent understanding of how it works - I stopped being a sysadmin in the mid '10s, I still worked for a technology company and did plenty of “interesting” reading and training.
It seems that more and more stuff that I want to run at home is being delivered as Docker-first and I have to really go out of my way to find a non-Docker install.
I’m thinking it’s no longer a fad and I should invest some time getting comfortable with it?
Nine. How much ram do they use? How much disk space? Try running 90, or 900. Currently, on my personal hobby kubernetes cluster, there’s 83 different instances running. Because of the low overhead, I can run even small tools in their own container, completely separate from the rest. If I run say… a postgresql server… spinning one up takes 90mb disk space for the image, and about 15 mb ram.
I worked at a company that did - among other things - hosting, and was using VM’s for easier management and separation between customers. I wasn’t directly involved in that part day to day, but was friend with the main guy there. It was tough to manage. He was experimenting with automatic creating and setting up new VM’s, stripping them for unused services and files, and having different sub-scripts for different services. This was way before docker, but already then admins were looking in that direction.
So aschually, docker is kinda made for people who runs things in VM’s, because that is exactly what they were looking for and duct taping things together for before docker came along.
Yeah I can see the advantage if you’re running a huge number of instances. In my case it’s all pretty small scale. At work we only have a single server that runs a web site and database so my home setup puts that to shame, and even so I have a limited number of services I’m working with.
Yeah, it also has the effect that when starting up say a new postgres or web server is one simple command, a few seconds and a few mb of disk and ram, you do it more for smaller stuff.
Instead of setting up one nginx for multiple sites you run one nginx per site and have the settings for that as part of the site repository. Or when a service needs a DB, just start a new one just for that. And if that file analyzer ran in it’s own image instead of being part of the web service, you could scale that separately… oh, and it needs a redis instance and a rabbitmq server, that’s two more containers, that serves just that web service. And so on…
Things that were a huge hassle before, like separate mini vm’s for each sub-service, and unique sub-services for each service doesn’t just become practical but easy. You can define all the services and their relations in one file and docker will recreate the whole stack with all services with one command.
And then it also gets super easy to start more than one of them, for example for testing or if you have a different client. … which is how you easily reach a hundred instances running.
So instead of a service you have a service blueprint, which can be used in service stack blueprints, which allows you to set up complex systems relatively easily. With a granularity that would traditionally be insanity for anything other than huge, serious big-company deployments.
Well congrats, you are the first person who has finally convinced me that it might actually be worth looking at even for my small setup. Nobody else has been able to even provide a convincing argument that docker might improve on my VM setup, and I’ve been asking about it for a few years now.
It’s a great tool to have in the toolbox. Might take some time to wrap your head around, but coming from vm’s you already have most of the base understanding.
From a VM user’s perspective, some translations:
- Dockerfile = script to set up a VM from a base distro, and create a checkpoint that is used as a base image for starting up vm’s
- A container is roughly similar to a running VM. It runs inside the host os, jailed, which account for it’s low overhead.
- When a container is killed, every file system change gets thrown out. Certain paths and files can be mapped to host folders / storage to keep data between restarts.
- Containers run on their own internal network. You can specify ports to nat in from host interface to containers.
- Most service setup is done by specifying environment variables for the container, or mapping in a config file or folder.
- Since the base image is static, and config is per container, one image can be used to run multiple containers. So if you have a postgres image, you can run many containers on that image. And specify different config for each instance.
- Docker compose is used for multiple containers, and their relationship. For example a web service with a DB, static file server, and redis cache. Docker compose also handles things like setting up a unique network for the containers, storage volumes, logs, internal name resolution, unique names for the containers and so on.
A small tip: you can “exec” into a running container, which will run a command inside that container. Combined with interactive (-i) and terminal (-t) flags, it’s a good way to get a shell into a running container and have a look around or poke things. Sort of like getting a shell on a VM.
One thing that’s often confusing for new people are image tags. Partially because it can mean two things. For example “postgres” is a tag. That is attached to an image. The actual “name” of an image is it’s Sha sum. An image can have multiple tags attached. So far so good, right?
Now, let’s get complicated. The actual tag, the full tag for “postgres” is actually “docker.io/postgres:latest”. You see, every tag is a URL, and if it doesn’t have a domain name, docker uses it’s own. And then we get to the “: latest” part. Which is called a tag. Yup. All tags have a tag. If one isn’t given, it’s automatically set to “latest”. This is used for versioning and different builds.
For example postgres have tags like “16.1” which points to latest 16.1.x version image, built on postgres maintainers’ preferred distro. “16.1-alpine” that point to latest Alpine based 16.1.x version. “16” for latest 16.x.x version, “alpine” for latest alpine based version, be it 16 or 17 or 18… and so on. You can find more details here.
The images on docker hub are made by … well, other people. Often the developers of that software themselves, sometimes by docker, sometimes by random people. You can make your own account there, it’s free. If you do, make an image and pushes it, it will be available as shdwdrgn/name - if it doesn’t have a user component it’s maintained / sanctioned by docker.
You can also run your own image repository service, as long as it has https with valid cert. Then it will be yourdomain.tld/something
So that was a brief introduction to the strange World of docker. Docker is a for profit company, btw. But the image format is standardized, and there’s fully open source ways to make and run images too. At the top of my head, podman and Kubernetes.
One thing I’m not following in all the discussions about how self-contained docker is… nearly all of my images make use of NFS shares and common databases. For example, I have three separate smtp servers which need to put incoming emails into the proper home folders, but also database connections to track detected spam and other things. So how would all these processes talk to each other if they’re all locked within their container?
The other thing I keep coming back to, again using my smtp servers as an example… It is highly unlikely that anyone else has exactly the same setup that I do, let alone that they’ve taken the time to build a docker image for it. So would I essentially have to rebuild the entire system from scratch, then learn how to create a docker script to launch it, just to get the service back online again?
Instead of setting up one nginx for multiple sites you run one nginx per site and have the settings for that as part of the site repository.
Doesn’t that require a lot of resources since you’re running (mysql, nginx, etc.) numerous times (once for each container), instead of once globally?
Or, per your comment below:
Since the base image is static, and config is per container, one image can be used to run multiple containers. So if you have a postgres image, you can run many containers on that image. And specify different config for each instance.
You’d only have two instances of postgres, for example, one for all docker containers and one global/server-wide? Still, that doubles the resources used no?