Lou Stack Base
Lou Stack Base

Docker Hands-on Observations: Layer Cache, Restart Behavior, Image Selection, and Resource Limits

Lou Chang Lou Chang included in DevOps  Docker
 About 1300 words 6 minutes 

DevOps Learning Notes

Hands-on observation notes on Docker fundamentals

Running through several core Docker behaviors, including layer cache hit logic, the difference between container restart and rebuild, image size comparison, and how to verify cgroup resource limits

Legacy Builder vs BuildKit

Modern Docker uses BuildKit as the build engine. The legacy builder has been marked as deprecated.

1
2
3
4
5

# legacy (deprecated)
docker build -t go-api .

# modern approach
docker buildx build -t go-api .

Important: The two builders store their caches separately and do not share them. After switching from legacy to BuildKit, the first build will have no cache to use.

BuildKit’s output format is clearer than the old version. Each step indicates whether it hit cache:

1
2

=> CACHED [builder 2/6] WORKDIR /app          0.0s   โ† cache hit
=> [builder 6/6] RUN go build ...             4.5s   โ† actually executed

Instruction-Order Effects on Tier Reuse

How Script Sequence Determines Cache Hits

Docker’s cache rule: if the content of a layer changes, all layers below it become invalid and must be re-executed.

This is why you put “things that rarely change” first:

1
2
3
4
5

# correct order
COPY go.mod go.sum ./      โ† go.mod rarely changes, easy cache hit
RUN go mod download        โ† stays CACHED as long as go.mod is unchanged
COPY . .                   โ† source code changes often, everything after re-runs
RUN go build ...
1
2
3

# wrong order
COPY . .                   โ† any code change invalidates this layer
RUN go mod download        โ† re-downloads packages every time, wastes minutes

Observation Results Across Three Runs

Experiment 1: First build

All layers are actually executed. No CACHED anywhere. Build time is the longest.

Experiment 2: Build again without changing anything

All layers show CACHED. Build time drops from seconds to under 1 second. This is the value of cache.

Experiment 3: Only change source code, don’t touch go.mod

1
2
3
4
5

=> CACHED [builder 1/6] FROM golang:1.25-alpine     โ† CACHED
=> CACHED [builder 3/6] COPY go.mod go.sum ./       โ† CACHED
=> CACHED [builder 4/6] RUN go mod download          โ† CACHED (packages not re-downloaded)
=> [builder 5/6] COPY . .                            โ† re-run (source code changed)
=> [builder 6/6] RUN go build ...                    โ† re-run

go mod download hitting cache means packages were not re-downloaded. This is the core reason for separating the COPY steps.

Restarting Containers vs Rebuilding Images

Reloading Does Not Recompile

This is an easily misunderstood concept. Experiment steps:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# 1. run with existing image
docker run -d -p 8080:8080 --name test-restart go-api

# 2. check current response
curl http://localhost:8080/healthz
# {"status":"ok","version":"0.0.1"}

# 3. change version to "0.0.2" in source code
# 4. only restart, no rebuild
docker restart test-restart
curl http://localhost:8080/healthz
# {"status":"ok","version":"0.0.1"}  โ† still 0.0.1, unchanged

Conclusion: restart only stops and re-starts the same process. The binary running inside the container is still the version compiled into the image at build time.

For code changes to take effect, you must go through the full flow:

1
2
3
4

docker buildx build -t go-api .    # rebuild image
docker stop test-restart
docker rm test-restart
docker run -d -p 8080:8080 --name test-restart go-api

Rationale Behind Frozen-at-Start Semantics

At the moment a container starts, it is frozen to that image version. No matter how the image is updated afterwards, running containers are not affected.

This is an intentional design. It makes production environment updates controllable โ€” things won’t change automatically without your knowledge.

Artifact Footprint: Empty Base Compared to Minimal Runtime

Measured Sizes 

1
2
3
4

docker images | grep go-api

go-api:distroless    22MB
go-api:scratch       15MB

Difference is about 7MB.

Progressive Runtime Additions 

1
2
3
4
5
6
7
8
9

scratch                   โ† completely empty, only your binary
    โ†“ +7MB
distroless/static         โ† adds CA certs, timezone data, basic user info
    โ†“ larger
distroless/base           โ† adds glibc (dynamic C library)
    โ†“ larger
distroless/cc             โ† adds C++ runtime
    โ†“ larger
alpine / debian / ubuntu  โ† full OS

Note: distroless/base is larger than distroless/static, not smaller.

Selection Criteria

The key question is: will your server make outbound HTTPS requests?

“Making outbound requests” means your server calls other services, for example:

  • Calling Stripe API for payment
  • Calling SendGrid to send email
  • Calling Slack to send notifications
  • Calling any third-party service

Receiving external requests and returning responses does not count โ€” that is normal server behavior and does not require CA certificates.

Scenario Choice
Server only receives requests, no outbound calls scratch
Server needs to make outbound HTTPS requests distroless/static
Language needs runtime (Java, Python) distroless/java / distroless/python

Base Image Line Change 

1
2
3
4
5
6
7

# scratch
FROM scratch
COPY --from=builder /app/server /server

# distroless
FROM gcr.io/distroless/static-debian12
COPY --from=builder /app/server /server

Only the base image is different. Everything else is identical.

Kernel-Enforced Resource Boundaries

Declaring CPU and Memory Ceilings 

1
2
3
4
5

docker run -d -p 8080:8080 \
  --cpus="0.5" \
  --memory="64m" \
  --name test-cgroup \
  go-api:scratch
  • --cpus="0.5": use at most 0.5 CPU cores
  • --memory="64m": use at most 64MB of memory

Confirming Constraints Are Active

Method 1: docker inspect

1
2

docker inspect test-cgroup | grep NanoCpus
# "NanoCpus": 500000000

NanoCpus conversion:

1
2

1 CPU = 1,000,000,000 nanocores
0.5 CPU = 500,000,000 nanocores

This number directly corresponds to the Linux kernel’s cgroup setting. Confirming this value confirms the limit is actually applied.

Method 2: docker stats

1

docker stats test-cgroup --no-stream
1
2

CONTAINER ID   NAME          CPU %   MEM USAGE / LIMIT   MEM %
2c9d821fbb02   test-cgroup   0.00%   2.84MiB / 64MiB     4.44%
  • CPU %: current actual CPU usage percentage (near 0% when idle)
  • MEM USAGE / LIMIT: how much memory is currently used / what the limit is
  • MEM %: memory usage percentage

--no-stream purpose: docker stats defaults to a continuously refreshing display (similar to top). Adding --no-stream prints a single snapshot and exits.

Control Groups as a Safety Boundary

Even if the program inside the container goes rogue (infinite loop, memory leak), it can only use up to the configured limit. It will not affect the host or other containers.

This is the Linux kernel’s cgroup enforcement at work. Docker just wraps the configuration into simple parameters.

Checking Published Ports

When running docker run, always verify the PORTS column has a value:

1
2
3
4
5
6
7
8
9

docker ps

# correct: has port mapping
PORTS                    NAMES
0.0.0.0:8080->8080/tcp   test-cgroup

# wrong: empty, cannot connect
PORTS     NAMES
          test-cgroup

If PORTS is empty, it means -p 8080:8080 was not applied. You need to stop and recreate the container.

Key Takeaways

  1. Layer Cache:

    • Dockerfile order determines cache efficiency
    • Put things that rarely change first โ†’ package downloads can be cached

  2. Container Restart:

    • restart = restart the same process, binary unchanged
    • rebuild = produce new image, code changes take effect

  3. Image Selection:

    • scratch โ†’ Go static binary, smallest, no outbound HTTPS needed
    • distroless โ†’ needs CA certificates or runtime
    • alpine/ubuntu โ†’ for development and debugging, not for production

  4. cgroup:

    • --cpus / --memory set limits
    • docker inspect check NanoCpus to confirm settings
    • docker stats view real-time resource usage

References

Updated on 2026-02-19 
CC BY-NC-SA 4.0
Read Markdown
LINE
DockerDevOpsContainerLinuxCgroupBuildKit
Back | Home

Contents