Working with Docker

  • Last Updated 10/20/2021, 2:52:50 PM UTC
  • About 6 min read

# Installation

# docker-compose Installation

# SELinux

If enabled and enforcing (sestatus) AND NO OTHER CONTAINERS EXIST then also enable in docker /etc/docker/daemon.json

{
  "selinux-enabled": true
}

Restart docker for this to take effect

sudo systemctl restart docker

# Docker data directory

By default docker persists its state (images, volumes etc) under the /var/lib/docker mount point. This location should have a minimum of 50G of available space on a filesystem that supports the overlay2 storage driver. You should mount /var/lib/docker to an appropriate filesystem or change the data directory location to a different path in /etc/docker/daemon.json:

{
  "data-root": "/your/docker/data-root/mount"
}

Before proceeding stop docker

sudo systemctl stop docker

If you already have data inside the default mount then copy it to the custom mount point:

sudo rsync -aP /var/lib/docker/ /your/docker/data-root/mount

If running with SELinux enabled and enforcing, you will need to set the changed data-root to the proper SELinux context:

semanage fcontext -a -e /var/lib/docker /your/docker/data-root/mount
restorecon -R -v /your/docker/data-root/mount

Also modify the dockerroot user home directory to point to the new data mount:

sudo usermod -d /your/docker/data-root/mount dockerroot

Finally start docker

sudo systemctl start docker

# OverlayFS filesystem requirements

Must be created on a filesystem that supports d_type entries, e.g. ext4, xfs.

For xfs you can verify that d_type is enabled by running xfs_info on your filesystem and looking for parameter f_type set to 1:








 




$ sudo xfs_info /your/docker/data-root/mount
meta-data=/dev/mapper/ol-root    isize=256    agcount=4, agsize=3276800 blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=0        finobt=0, sparse=0, rmapbt=0
         =                       reflink=0
data     =                       bsize=4096   blocks=13107200, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=6400, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

To verify that docker engine is using the overlay2 storage driver with d_type enabled:



 

 


$ docker info

Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: false

# HTTP Proxies

Set in /etc/systemd/system/docker.service.d/http-proxy.conf

[Service]
Environment="HTTP_PROXY=http://your.proxy:port"
Environment="HTTPS_PROXY=http://your.proxy:port"
Environment="NO_PROXY=localhost,127.0.0.1,.mydomain.com"

Then reload and restart

sudo systemctl daemon-reload
sudo systemctl restart docker

# NOEXEC flag on /tmp and docker-compose

When /tmp is mounted with the noexec flag, docker-compose will fail with an error similar to failed to map segment from shared object. To work around this problem create a different temporary directory and set environment variable TMPDIR.

mkdir -p /opt/polaris/tmp
export TMPDIR=/opt/polaris/tmp

# Setup docker for polaris services

Add polaris user to docker group

sudo usermod -a -G docker polaris

Create a docker bridge network for myrmex containers

docker network create --driver bridge --opt com.docker.network.bridge.name=br-myrmex br-myrmex

# Firewall

If your firewall filters packets from bridge devices, you must configure the firewall to allow incoming connections from required docker bridges to any required host ports.

Filtering is enforced if kernel parameters net.bridge.bridge-nf-call-iptables and net.bridge.bridge-nf-call-ip6tables are set to 1:

sudo sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables

You may disable filtering at your own discretion as follows:

sudo sysctl -w net.bridge.bridge-nf-call-iptables=0
sudo sysctl -w net.bridge.bridge-nf-call-ip6tables=0

sudo bash -c 'cat <<'SYSCTL' >>  /etc/sysctl.d/999-custom.conf
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-ip6tables = 0
SYSCTL'

Preferably, if filtering is enabled, you can allow incoming connections as follows, where br-myrmex is the bridge interface you created earlier and ports 9855, 9856 are the myrmex-sd ports that polaris containers will open connections to. For other bridges/ports change the commands accordingly.

Check what type of firewall you are running:

# iptables service
sudo systemctl status iptables

# firewalld
sudo firewall-cmd --state
sudo systemctl status firewalld

# uncomplicated firewall 
sudo ufw status
sudo systemctl status ufw

    # SELinux Troubleshooting

    For more in depth information refer to https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html-single/selinux_users_and_administrators_guide/index#chap-Security-Enhanced_Linux-Troubleshooting (opens new window)

    # install seinfo
    yum install setools-console
    
    # get port labels
    semanage port -l | grep [YOUR-PORT]
    
    # get security context for user
    id -Z
    
    # check security context for a process
    ps -auZ | grep some-proc
    
    # check security context for file/dir
    ls -lZ /path/to/file-or-dir
    
    # replicate context from one directory to another
    semanage fcontext -a -e [FROM DIRECTORY] [TO DIRECTORY]
    restorecon -R -v [TO DIRECTORY]
    
    # search raw audit messages related to selinux access denials
    cat /var/log/audit/audit.log | grep AVC | grep denied
    
    # search raw audit messages related to selinux access denials for a specific path
    cat /var/log/audit/audit.log | grep AVC | grep denied | grep [/some/path]
    
    # silent AVC denials will not be logged. Temporarily disable don't audit rules, run the process again and then 
    # check in /var/log/audit/audit.log
    semodule -DB
    
    # re-enable dontaudit
    semodule -B
    
    # to see which don't audit rule prevented a message in /var/log/audit/audit.log
    sesearch --dontaudit -s container_t
    
    # check why access was denied
    audit2allow -w -a
    
    # check if docker is running with selinux enabled
    docker info | grep -i selinux
    
    # inspect a docker container's MCS label and check if it matches file/dir label
    docker inspect [CONTAINER NAME|ID] | grep ProcessLabel
    
    # ask docker to relabel bind volumes with correct context
    #  - use :z to relabel and SHARED between all containers
    docker run --rm -v /host/path/to/vol:/container/path/to/vol:z [CONTAINER NAME]
    
    # ask docker to relabel bind volumes with correct context
    #  - use :Z to relabel but PRIVATE to this container. USE WITH CAUTION
    docker run --rm -v /host/path/to/vol:/container/path/to/vol:Z [CONTAINER NAME]
    
    Last Updated: 10/20/2021, 2:52:50 PM