Fork me on GitHub
class: title Docker
a lot for **Dev**
just a bit for **Ops** .nav[*Massimo Santini*] --- class: title Introduction --- ## Dependency hell Issues when developing complex software, *many*: * languages (*polyglot*), * libraries (multiple versions) * services (dbms, httpd, cache, log…), * people (*onboarding*). -- * testing (*fixtures*) -- Why not let the developers take care of (some) operations aspects? * the DevOps approach! --- ## Desiderata From the **Dev** point of view: * predictability, * replicability, * versioning, * lightweight. -- From the **Ops** point of view: * fault tolerance, * portability, * scalability, * composability, * scheduling & orchestration. -- And security? --- ## Solutions Configuration management: * [Ansible](http://www.ansible.com/), [Chef](https://www.chef.io/chef/), [Puppet](https://puppetlabs.com/), [Saltstack](http://saltstack.com/), [Terraform](https://www.terraform.io/). Virtualization: * [LXC](https://linuxcontainers.org/), [KVM](http://www.linux-kvm.org/), [Vagrant](https://www.vagrantup.com/). Or… containers: * [Docker](https://www.docker.com/), [Singularity](https://sylabs.io/docs/). --- ## Virtualization à la Docker Process & memory * LXC = kernel cgroups + namespaces. Filesystem * AUFS, * Device Mapper, * BTRFS * … Networking * Overlay, * MacVlan, * … --- class: title .small[*Images* and *containers*] --- ## A filesystem… (*image*) * a DAG of copy-on-write **read-only** filesystems, * obtained by: **pull** (from a *hub*, also **push**), **build**, or **commit**. ```bash $ docker pull hello-world ... Digest: sha256:d58e752213a51785838f9eed2b7a498ffa1cb3aa7f946dda11af39286c3db9a9 Status: Downloaded newer image for hello-world:latest $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE hello-world latest bf756fb1ae65 10 months ago 13.3kB ``` --- ## … and a process (*container*) * an **image** plus a **writable** *copy-on-write* layer, * (explicitly) obtained by: **create**. ```bash $ docker create --name dt-image hello-world 6ee4391b6c2c6b5ce07e2c162e691de5481b35083973ec5bcfc03a1c6c52b7b4 $ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES f65d9f282056 hello-world "/hello" 5 seconds ago Created dt-image ``` * (implicitly) obtained by: **run** = (**create** + **start**), ```bash $ docker run hello-world Hello from Docker. ... $ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 8c17068f0c80 hello-world "/hello" 25 seconds ago Created dt-image 7f937e7c88dc hello-world "/hello" 5 seconds ago Exited (0) silly_yalow ``` --- ## Bookkeeping Take care of your *images* by * list (**images**), inspect (**inspect**), remove (**rmi**), * **pull**, **push**, **export**, **import**. -- Take care of your *containers* by * list (**ps**), inspect (**inspect**), remove (**rm**) — *filesystem*, * **start**, **stop**, **kill**, **pause**, **unpause** — *processes*. --- class: title .small[Producing *images*] --- ## Worst practice (*commit*) * run (and *modify*) a container… ```bash $ docker run -it --name dt-busybox busybox:musl Unable to find image 'busybox:musl' locally ... Status: Downloaded newer image for busybox:musl / # touch file.txt / # exit ``` * and **commit**! ```bash $ docker commit dt-busybox dt-busybox:0.1 sha256:044aeb8cc3d70516f6a2b387ed9cd17fed276b0ac4065d479deaa53688b01f66 $ docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE dt-busybox 0.1 b57102c04ff0 15 seconds ago 1.213 MB busybox musl 733eb3059dce 5 weeks ago 1.213 MB ``` --- ## In a reproducible way (*build*) * write a `Dockerfile` to describe build steps and configurations… ```Dockerfile FROM python:3.8-alpine LABEL maintainer="santini@di.unimi.it" WORKDIR /app ADD . /app RUN pip install --trusted-host pypi.python.org -r requirements.txt EXPOSE 80 CMD ["python", "app.py"] ``` * and **build**! ```bash $ docker build -t mapio/dt-app . ... $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE mapio/dt-app latest 11a0c2a15c01 12 minutes ago 213.4 MB python 3.7-alpine 8922d588eec6 4 weeks ago 98.4MB hello-world latest c54a2cc56cbb 4 months ago 1.848 kB ``` * give a look at [BuildKit](https://docs.docker.com/develop/develop-images/build_enhancements/) --- ## Publishing images (*push*) * choose a (public) registry, create an account and authenticate ```bash $ docker login Login with your Docker ID to push and pull images from Docker Hub. If you dont have a Docker ID, head over to https://hub.docker.com to create one. Username: mapio Password: ********** Login Succeeded ``` * **push** your image to make it available to other devs (or in production): ```bash $ docker push mapio/dt-app The push refers to a repository [docker.io/mapio/dt-app] 455deb6fe734: Pushed b4bac53253fd: Pushed b3fec61839f1: Pushed latest: digest: sha256:75d28a5d629f7f1d2e2983839aad size: 1785 ``` --- class: title .small[Running *containers*] --- ## Many *flavors* * *ephemeral* ```bash $ docker run --rm busybox date Sun Jul 5 14:34:41 UTC 2020 ``` * *attached* ```bash $ docker run -it busybox / # date Sun Jul 5 14:34:59 UTC 2020 / # exit ``` * *daemonized* ```bash $ docker run --name dt-daemon -d busybox \ sh -c 'while true; do date; sleep 10; done' e2f98a4ab4b4de37b533a36d2ff310b1220d2f1facfec4b4b93c8635994b34a1 ``` --- ## Keeping in touch * obtain **logs** of *running* and *terminated* instances, ```bash $ docker logs dt-daemon Thu Nov 26 11:07:46 UTC 2020 Thu Nov 26 11:07:56 UTC 2020 Thu Nov 26 11:08:06 UTC 2020 ``` * **attach** to the stdin/out of the *running* (process in a) container. ```bash $ docker attach --sig-proxy=false dt-daemon Thu Nov 26 11:08:26 UTC 2020 ^C ``` * **exec** another process in a *running* container ```bash $ docker exec dt-daemon ps -a PID USER TIME COMMAND 1 root 0:00 sh -c while true; do date; sleep 10; done 29 root 0:00 sleep 10 30 root 0:00 ps -a ``` --- class: title .small[*Persistence* and *communication*] --- ## Named volumes (*guest* space) A road to persistence: just **create** a *volume* with a *name* ```bash $ docker volume create dt-volume ``` As before, you can (even with *ephemeral* runs), read/write ```bash $ docker run --rm --volume dt-volume:/data busybox sh -c 'date > /data/file.txt' $ docker run --rm --volume dt-volume:/data busybox sh -c 'cat /data/file.txt' Thu Nov 26 11:12:55 UTC 2020 ``` You can *manage* them with **volume** and subcommands: * list (**ls**), remove (**rm**), or inspect (**inspect**). --- ## Volumes (*host* space) * **map** a volume to a *host filesystem* (but may be a mess) ```bash $ docker run -it --rm -v $(pwd):/data busybox sh -c 'date > /data/file.txt' $ cat file.txt Thu Nov 26 11:12:27 UTC 2020 ``` * copy things *in*/*out* using **cp** (needs a *container*) ```bash $ CID=$(docker create --volume dt-volume:/data busybox) $ docker cp $CID:/data/file.txt file2.txt $ cat file2.txt Thu Nov 26 11:12:55 UTC 2020 $ docker cp file2.txt $CID:/data/file3.txt $ docker rm $CID $ docker run --volume dt-volume:/data busybox cat /data/file3.txt Thu Nov 26 11:12:55 UTC 2020 ``` --- ## Communication (*guest* space) A road to *micro services*: **create** a *network* and put containers on it ```bash $ docker network create dt-network $ docker run --network dt-network --name dt-nc --rm -it busybox nc -l -p 80 ``` you cant talk to it from containers on the same network: ```bash $ docker run -it --rm --network dt-network busybox sh -c 'date | nc dt-nc 80' ``` this will make `date` output appear in the stdout of the first container. You can manage networks with **network** and subcommands: * list (**ls**), remove (**rm**), or inspect (**inspect**). --- ## Communication (*host* space) To make a network service in a container public, you can *publish* a port (`-p`) ```bash $ docker run --name dt-nc --rm -it -p 80 busybox nc -l -p 80 ``` You can determine the public `PORT` for example as ```bash $ PORT=$(docker port dt-nc 80 | cut -d : -f 2) ``` and connect to such public address ```bash $ date | nc 127.0.0.1 $PORT ``` --- class: title The **Dev** point of view --- ## Using docker to compile/run code * see [examples](https://github.com/mapio/docker-Italiaonline/tree/master/examples/compilation) for: C, Java, Python and… PHP. --- ## Even better, your IDE in a container * Using [Remote Development](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack) in [Visual Studio Code](https://code.visualstudio.com/). --- class: title The **Ops** point of view --- ## Describe your services * In declarative form using a [compose file](https://docs.docker.com/compose/compose-file/): ```yaml services: web: image: mapio/dt-app ports: - target: 80 published: 80 networks: - dt-network redis: image: redis command: redis-server --appendonly yes networks: - dt-network volumes: - type: volume source: dt-data target: /data ``` --- ## Deploy locally * Start it using [docker-compose](https://docs.docker.com/compose/) ```bash docker-compose up -d ``` * Inspect it: ```bash docker-compose logs ``` * Stop it: ```bash docker-compose down ``` * You get persitence, somehow… --- ## Setup a swarm * Setup some iron with a docker and ssh daemon… we'll use [Play with Docker](https://labs.play-with-docker.com/). * Start a **master** node: ```bash node1$ docker swarm init To add a worker to this swarm, run the following command: docker swarm join --token SWMTKN-1-5ur3q84en… ``` * Start one (or more) **worker** node(s) (cut and paste the `init` output): ```bash node2$ docker swarm join --token SWMTKN-1-5ur3q84en… ``` * Check that it worked: ```bash node1$ docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION ecqr03l3 * node1 Ready Active Leader 19.03.11 jt7we9zl node2 Ready Active 19.03.11 ``` --- ## Deploy on the swarm * Start a *stack* ```bash $ docker stack deploy -c docker-swarm.yml app ``` * Show its *services* of a stack ```bash docker stack services app ``` * Show the *tasks* of a stack ```bash docker stack ps app ``` * Monitor a service ```bash docker service logs app_web ``` * Shut down a stack ```bash $ docker stack rm app ``` --- ## UI Monitoring tools * [Docker Swarm Visualizer](https://github.com/dockersamples/docker-swarm-visualizer) (beware: *toy app*). * [Portainer](https://www.portainer.io/). --- ## Scaling and updating * Scaling (if the app architecture allows for it) is easy: ```bash $ docker service scale app_web=5 ``` * Updating (to a newer image) is also straightforward: ```bash docker service update --image mapio/dt-app:latest app_web ``` * Both operation are available using Portainer… --- class: title A few resources for the voyager --- Really good starting points: * [Container training](http://container.training/) by Jérôme Petazzoni [@jpetazzo](https://twitter.com/jpetazzo), Looking for images? * [Docker Hub](https://hub.docker.com/explore/) the official *registry*, * [Alpine](https://github.com/gliderlabs/docker-alpine) base image, * [Phusion](https://github.com/phusion/baseimage-docker) base image, * Roll your own [registry](https://docs.docker.com/registry/deploying/). --- class: pic ![back](back.png)