# Docker pull Hangs on macOS? Check credential helpers and DOCKER_CONFIG

> Canonical: https://vxi.me/en/posts/docker-pull-credential-helper-docker-config/
> Locale: en-us
> Published: 2026-10-07
> Updated: 2026-10-07
> Markdown: https://vxi.me/en/posts/docker-pull-credential-helper-docker-config.md

One of the hardest unattended-runner failures to debug is not an explicit error. It is a command such as `docker pull` or `docker build` that simply stops making progress, with no obvious Registry response in the log.

The first things I normally check are network routing, DNS, proxies, and the Docker daemon. This time, following a real incident led to a less obvious path: before Docker actually pulls an image, the CLI may look up Registry credentials. If the background runner inherited the same credential helper as a desktop session, the automation may be coupled to an interactive Keychain without anyone realizing it.

The lead came from Fishhawk [issue #4046](https://github.com/kuhlman-labs/fishhawk/issues/4046). The project documented an unattended macOS runner where the Docker credential-helper chain stalled while the screen was locked and continued after unlock. I did not independently reproduce that real Keychain behavior, so I am not treating it as something every macOS machine will do.

What I did want to verify was the mechanism underneath it.

My initial assumption was simple: give the runner a separate `DOCKER_CONFIG` containing only:

```json
{"auths":{}}
```

and it should no longer touch the user's Keychain.

The current Docker CLI source and the macOS experiment below both disproved that assumption. **An empty `auths` object is not the same as an explicit anonymous posture. With the Docker CLI version I tested, it can still trigger platform-default credential-helper discovery.**

That is the edge case worth knowing.

## Why Docker CLI can touch a credential helper even for a public image

Docker's current documentation still describes `~/.docker/config.json` as the client config and documents `DOCKER_CONFIG` or `--config` as ways to select another config directory. It also defines `credsStore` as the default credential store and `credHelpers` as per-Registry overrides. The relevant references are [Docker CLI configuration](https://docs.docker.com/reference/cli/docker/) and [docker login / credential stores](https://docs.docker.com/reference/cli/docker/login/).

The important part is what the CLI does when the loaded config contains no authentication information.

In the Docker CLI commit I pinned for this investigation, `3d2650ce76053393872e8389c2b77033146cb356`, [`LoadDefaultConfigFile()`](https://github.com/docker/cli/blob/3d2650ce76053393872e8389c2b77033146cb356/cli/config/config.go) contains:

```go
if !configFile.ContainsAuth() {
    configFile.CredentialsStore =
        credentials.DetectDefaultStore(configFile.CredentialsStore)
}
```

And [`ContainsAuth()`](https://github.com/docker/cli/blob/3d2650ce76053393872e8389c2b77033146cb356/cli/config/configfile/file.go) is based on three things:

```go
return c.CredentialsStore != "" ||
    len(c.CredentialHelpers) > 0 ||
    len(c.AuthConfigs) > 0
```

That is why `{"auths":{}}` is not enough. `AuthConfigs` still has length zero, so `ContainsAuth()` is false and the CLI enters `DetectDefaultStore()`.

On Darwin, the platform default in this pinned source is [`osxkeychain`](https://github.com/docker/cli/blob/3d2650ce76053393872e8389c2b77033146cb356/cli/config/credentials/default_store_darwin.go). If `docker-credential-osxkeychain` is discoverable on `PATH`, the CLI can select it as the default store.

So an apparently empty runner config can turn into this:

```text
no credsStore
no credHelpers
no auth entries
        ↓
ContainsAuth() == false
        ↓
discover the platform default helper
        ↓
macOS: docker-credential-osxkeychain
```

Source inspection explains the path, but I also wanted to know whether a real Darwin process would actually take it.

## I ran four config shapes on the same macOS runner

I deliberately kept real accounts and the real Keychain out of the experiment. I did not execute Fishhawk code, did not run `docker login`, and did not modify the GitHub runner's system Keychain.

The fixture used a hosted macOS ARM64 runner with Docker CLI source pinned to the commit above. I placed two self-authored fake helper executables at the front of `PATH`:

```text
docker-credential-osxkeychain
docker-credential-probe
```

They never read real credentials. They only recorded which helper was invoked, which action was requested, and which Registry key was queried.

The exact command was:

```bash
GOFLAGS=-mod=vendor go test \
  -run '^TestAITokenCredentialExperiment$' \
  -count=1 -v ./cli/config
```

GitHub Actions run `37620686435` executed on macOS ARM64, `macos26`, with Go `1.26.8`. The command exited **0**. The raw terminal output was short:

```text
=== RUN   TestAITokenCredentialExperiment
AITOKEN_RESULT .../data/experiments/docker-credentials/raw-go-test.json
--- PASS: TestAITokenCredentialExperiment (0.95s)
PASS
ok   github.com/docker/cli/cli/config   0.956s
```

The marker output from the four arms is the useful part:

| Config | `ContainsAuth()` | Store after default load | Observed helper call |
| --- | --- | --- | --- |
| `{"auths":{}}` | false | `osxkeychain` | `docker-credential-osxkeychain get https://index.docker.io/v1/` |
| `{"auths":{"fishhawk.invalid":{}}}` | true | file store | **no helper call** |
| `{"credsStore":"probe","auths":{}}` | true | `probe` | `docker-credential-probe get https://index.docker.io/v1/` |
| `{"credHelpers":{"example.invalid":"probe"}}` | true | file store | `probe` called only for `example.invalid` |

The first row is the surprising one. A bare `{"auths":{}}` did not create an explicitly helper-free state. Once the fake `docker-credential-osxkeychain` executable was on `PATH`, this Docker CLI selected it and actually invoked it while resolving Docker Hub authentication.

The second row explains an odd-looking detail in Fishhawk's merged implementation. Instead of leaving `auths` empty, [`NewAnonymousDockerConfig()`](https://github.com/kuhlman-labs/fishhawk/blob/de153d47a86a839822c656c0308b58fe97729c06/runner/internal/gateiso/dockerconfig.go) writes a no-secret placeholder:

```json
{
  "auths": {
    "fishhawk.invalid": {}
  }
}
```

There is no username, password, or token in that object. Its purpose is control flow: make the auth map non-empty, which makes `ContainsAuth()` true and skips platform-default helper discovery.

At first glance, “add an auth entry so that the runner does not authenticate” looks backwards. The source plus the runtime result make the reason clear.

There is an important version boundary here. I verified this against a **pinned Docker CLI commit**. I would not treat the placeholder trick as a permanent protocol contract. If your automation depends on it, keep a regression test around the behavior when Docker CLI versions change.

## Keep the runner's Docker config separate from the desktop user's config

After this experiment, I would not fix an unattended job by deleting `credsStore` from the user's normal `~/.docker/config.json`.

Credential stores exist for a good reason. Docker's own documentation recommends external stores because keeping Registry credentials in a system Keychain or credential helper is generally safer than writing them directly into `config.json`. The problematic design is the background runner silently inheriting an interactive user's credential state.

For a runner that only pulls public images, I prefer a runner-owned `DOCKER_CONFIG`:

```bash
runner_docker_config="$(mktemp -d)"

cat > "$runner_docker_config/config.json" <<'JSON'
{
  "auths": {
    "runner.invalid": {}
  }
}
JSON

DOCKER_CONFIG="$runner_docker_config" docker pull alpine:latest
```

The placeholder key contains no real credential. The point is also not that it must be named `runner.invalid`. The important boundary is that the runner no longer reads the desktop user's `~/.docker/config.json`, while this tested Docker CLI version also avoids auto-detecting the platform helper.

Private images are a different case. An anonymous config cannot authenticate to a private Registry, so the runner needs an explicit automation credential path.

When possible, I prefer a Registry-scoped `credHelpers` mapping over one broad default `credsStore` because the scope is visible in the config:

```json
{
  "credHelpers": {
    "registry.example.com": "your-noninteractive-helper"
  }
}
```

In the fixture I queried both `example.invalid` and `other.invalid`. The fake helper was invoked exactly once, for `example.invalid`. That matches Docker's documented per-Registry helper behavior.

What `your-noninteractive-helper` should actually be depends on the Registry provider and the lifetime of the credential. I would not turn one helper into a universal recommendation. For unattended automation, the useful properties are machine-oriented credentials, narrow Registry scope, preferably short-lived access, and predictable behavior without an interactive desktop session.

There is one operational wrinkle that is easy to miss: `DOCKER_CONFIG` is the whole Docker CLI config directory, not just an auth file. If your desktop setup also relies on Docker Desktop's `buildx`, Compose, or other CLI plugins, pointing the runner at a brand-new directory can affect plugin discovery too.

Fishhawk's merged implementation explicitly carries forward plugin directories into the runner-owned config. The idea is worth copying; the exact paths are not. If your runner only does `docker pull`, this is usually simple. If it also uses `docker buildx`, Compose, or custom plugins, verify plugin discovery when you isolate credentials.

This is similar to the boundary I ran into in [using CLIProxyAPI as a Docker-hosted Codex proxy](/posts/cliproxyapi-codex-docker-proxy/): reusing one person's state is convenient at first, but once a service has to run unattended for months, the hidden coupling becomes harder to reason about than a little extra explicit configuration.

## If a helper blocks, do not wait for the whole docker pull timeout

I ran one more small experiment. The fake `docker-credential-probe get` command was changed to deliberately never return, then the Docker CLI credential lookup was sent through it.

The outer test harness allowed only **900 ms**.

The marker first recorded:

```text
docker-credential-probe|get|https://index.docker.io/v1/
```

The child process then stayed blocked until the outer deadline terminated it at **933 ms**.

This proves a narrow but useful point: once the credential lookup is inside an external helper, an unresponsive helper can hold that lookup path open. The current [`docker-credential-helpers` client `Get()`](https://github.com/docker/docker-credential-helpers/blob/df2fce1a0dad8cec1a85ead31f268ab416b7a6cb/client/client.go) runs an external helper and waits for its output; that layer does not choose your runner's business-level timeout for you.

For unattended jobs, I want the timeout at a place that explains what failed.

A single 30-minute job timeout that eventually reports “docker pull failed” is much less useful than a bounded probe before a private-Registry pull, or at minimum a dedicated timeout around the Docker step with a failure reason that says the credential helper did not answer.

Fishhawk's merged implementation follows that direction. When an operator explicitly supplies a Docker config, the runner identifies the helpers that the requested images would use and probes them under a bound before the pull. Fishhawk currently uses 20 seconds, but **20 seconds is Fishhawk's engineering policy, not a Docker default and not a universal recommendation from this article**.

A local machine-identity helper may deserve only a few seconds. A network-backed vault may need longer. The reusable idea is that credential lookup gets its own observable bound.

## What the macOS lock-screen evidence actually proves

This is where the article can most easily overclaim.

Fishhawk #4046 describes a real macOS Docker Desktop host whose interactive config used `credsStore: desktop`. According to that project's incident evidence, the helper chain reached `docker-credential-osxkeychain`, blocked in the Keychain path while the screen was locked, and resumed after unlock. The project later merged its hardening in [PR #4059](https://github.com/kuhlman-labs/fishhawk/pull/4059).

My first-party experiment did **not** lock a real macOS session and did **not** ask the real `docker-credential-osxkeychain` to read the system Keychain. I used a same-named fake helper to verify helper selection and a deliberately blocking fake helper to verify the wait behavior.

So the evidence supports these statements:

- `DOCKER_CONFIG`, `credsStore`, and `credHelpers` directly determine the credential path an unattended Docker CLI can take.
- With the pinned CLI version I tested, `{"auths":{}}` on macOS can still auto-select `osxkeychain` when that helper is discoverable.
- A no-secret placeholder auth entry prevented that default-helper discovery in the same fixture.
- An external credential helper can keep credential lookup waiting until the caller imposes an outer bound.
- Fishhawk's real locked-screen Keychain incident is consistent with this mechanism, but the actual `SecItemCopyMatching` lock-screen behavior remains Fishhawk's third-party field evidence.

If an unattended runner works while someone is using the Mac but hangs after lock, logout, or during an overnight task, this is the order I would inspect things:

1. Print the runner's actual `DOCKER_CONFIG`, `HOME`, and config path. Do not assume it is reading the file you expect.
2. Inspect `config.json` for `credsStore` and Registry-specific `credHelpers`.
3. Check which `docker-credential-*` executables are actually visible on `PATH`.
4. Compare against a dedicated runner config before changing the user's global `~/.docker/config.json`.
5. If a helper is required, observe its lookup separately from the full image operation and give that step an explainable timeout.
6. After changing `DOCKER_CONFIG`, verify `buildx`, Compose, and any required CLI plugins still resolve.

These failures are easy to misdiagnose as Docker networking because the last visible command is often just `docker pull`. The real stall may happen earlier, while the CLI is deciding whether the Registry has credentials and where to retrieve them.

I now treat Docker credential configuration as part of the runner's runtime state, not as a harmless user preference that background automation can inherit. The desktop user can keep using the Keychain. The unattended runner gets its own explicit, testable, bounded configuration. Once those two states are separated, the failure path becomes much easier to reason about.

If this runner is going to live on a server or development machine for the long term, the same principle also applies to recovery. My earlier guide on [backing up Docker volumes to GitHub](/posts/docker-volume-backup-github-sync/) is about a different failure mode, but the operational lesson is the same: do not stop at “it works today.” Know where state lives, how to diagnose it, and how to rebuild it on another machine.