Debugging Failing Containers

Imagine a detective investigating a crime scene.

They gather evidence:

• What happened? (logs, events)
• Who was involved? (processes, users)
• What state are things in? (files, configuration)
• What led to this? (history, timeline)

Debugging containers is crime scene investigation.

You gather evidence from logs, inspect state, trace what happened, and form hypotheses.

1. Check if the container is running:

docker ps -a

The -a shows ALL containers (running and stopped).

Look at the STATUS column:

• Up 2 minutes — Running fine
• Exited (0) 5 seconds ago — Stopped normally
• Exited (1) 5 seconds ago — Crashed (non-zero exit code)
• Restarting — Crash loop

2. View logs:

docker logs <container>
docker logs --timestamps <container>

For containers that exited:

docker logs <stopped-container-name>

Logs persist even after the container stops.

3. Inspect the container:

docker inspect <container>

Returns JSON with full container details:

• Configuration
• State (running/stopped, exit code)
• Network settings
• Mounts
• Environment variables

Extract specific info:

docker inspect <container> --format='{{.State.ExitCode}}'
docker inspect <container> --format='{{.NetworkSettings.IPAddress}}'

4. Check resource usage:

docker stats <container>

Shows CPU, memory, network I/O. Helps identify if the container is resource-starved.

5. Exec into a running container:

docker exec -it <container> sh

Opens a shell inside the container. Check files, run commands, investigate.

6. Start a stopped container with a different command:

If your container exits immediately, override the command:

docker run -it --entrypoint sh myapp

This starts the container with a shell instead of the normal command. Now you can investigate inside.

7. View container events:

docker events --since 30m

Shows what Docker has been doing: containers starting, stopping, health checks failing, etc.

Scenario 1: Container exits immediately

docker run myapp
# Container starts and exits immediately

Debug:

# Check exit code
docker ps -a | grep myapp
# EXITED (127)

# View logs
docker logs myapp
# /bin/sh: mycommand: not found

Fix: The CMD in Dockerfile references a command that doesn’t exist.

Scenario 2: Container stuck in restart loop

docker ps
# Restarting (1) 5 seconds ago

Debug:

docker logs myapp
# Shows repeated errors on startup

docker inspect myapp --format='{{.State.ExitCode}}'
# 1

Fix: Application crashes on startup. Check logs for the error. Common causes:

• Missing environment variable
• Can’t connect to database
• Missing dependency

Scenario 3: Container running but not responding

docker ps
# Up 5 minutes

curl http://localhost:8000
# Connection refused

Debug:

# Check if port is published
docker ps
# If PORTS column is empty, port isn't published

# Check if app is listening
docker exec myapp netstat -tuln
# Is the app listening on 0.0.0.0:8000 or 127.0.0.1:8000?

Fix: App might be listening on 127.0.0.1 (localhost inside container) instead of 0.0.0.0 (all interfaces).

Change app to listen on 0.0.0.0:

app.run(host='0.0.0.0', port=8000)

Scenario 4: Out of memory

docker logs myapp
# Killed

Debug:

docker inspect myapp --format='{{.State.OOMKilled}}'
# true

Fix: Container ran out of memory. Increase memory limit or fix memory leak in app.

services:
  app:
    image: myapp
    mem_limit: 512m    # Increase memory limit

Attach to container output:

docker attach <container>

Attaches to container’s stdout. Useful for seeing live output.

Copy files out of a container:

docker cp <container>:/path/to/file ./local-path

Extract config files, logs, or data for inspection.

Run commands in stopped containers:

docker commit <stopped-container> debug-image
docker run -it debug-image sh

This creates an image from the stopped container’s state, then starts a shell.

Check network connectivity:

docker exec myapp ping db
docker exec myapp nslookup db
docker exec myapp curl http://api:8000

Test if the container can reach other services.

View all service statuses:

docker compose ps

View logs from all services:

docker compose logs

Check a specific service:

docker compose logs -f app

Restart a service:

docker compose restart app

Rebuild and restart:

docker compose up --build app

Stop all and remove containers:

docker compose down

---