Two of my favorite pieces of software are written in Go: Syncthing, which does continuous file synchronization, and restic, which does encrypted, deduplicated backups. Both are truly, truly excellent — some of the best software ever written: reliable, well documented, and maintained with obvious care. This post is not about a flaw in either of them. It is about what happens when you try to run Go programs over a proxy — and about a mess that sits in Go's libraries, beneath both.

Why you would proxy Syncthing at all

It can be a good idea to run Syncthing over a proxy such as Tor. Your Syncthing device ID is not exactly a secret: you hand it to everyone you sync with. Anyone who ever knew your device ID can query Syncthing's global discovery servers for the addresses your device announces and can therefore permanently track your IP address and also know where you are. Routing Syncthing through Tor decouples your device ID from your physical location. I currently use Syncthing only with my own devices, but for you, that may matter (e.g., if you use Syncthing to securely collaborate with coauthors and share data, which I have done in the past).

Syncthing gets proxying right

Syncthing supports this out of the box via environment variables. In a systemd unit:

[Service]
Environment="ALL_PROXY=socks5://127.0.0.1:9050"
Environment="ALL_PROXY_NO_FALLBACK=1"
Environment="NO_PROXY=192.168.0.0/16,10.0.0.0/8,172.16.0.0/12"

Three things are going on here:

• ALL_PROXY points at the local Tor SOCKS5 port.
• ALL_PROXY_NO_FALLBACK is essential. Without it, Syncthing races the proxied connection against a direct one and falls back to connecting directly if the proxy is down or slow — which would silently deanonymize you.
• NO_PROXY exempts the RFC 1918 private ranges, so devices on your own LAN are still reached directly instead of being pointlessly (and unsuccessfully) routed through Tor. CIDR ranges, single IPs, hostnames and domain suffixes are all supported.

Under the hood, Syncthing builds its dialing on golang.org/x/net/proxy, and credit where credit is due: this is a clean, well-behaved little library. It reads ALL_PROXY and NO_PROXY in both spellings, and — best of all — when given a hostname, its SOCKS5 implementation sends that hostname to the proxy (SOCKS5 address type 0x03) instead of resolving it locally, so name resolution happens on the far side of the proxy. That is exactly the right default: DNS leaks are the classic way to ruin an otherwise proxied setup. In curl's terminology this is socks5h:// behavior; the library treats socks5:// and socks5h:// identically, and in the safe direction.

So far, so pleasant. Now suppose that, out of curiosity, you try the same trick with another Go program you trust. Say, restic.

The world-historical mess

Set ALL_PROXY=socks5://127.0.0.1:9050 for restic and run a backup to an S3 bucket. It completes flawlessly. There is just one small problem: none of that traffic went through your proxy. No error, no warning — your backup traffic, and your DNS lookups, went out over the plain network.

This is not restic's fault. Restic does the textbook-correct thing: it builds its HTTP transport with Proxy: http.ProxyFromEnvironment, the standard mechanism that virtually every Go HTTP client uses. The problem is what that mechanism actually reads.

Go has two entirely separate libraries for reading proxy configuration from the environment, and they disagree about which variables exist:

• net/http.ProxyFromEnvironment, wired into http.DefaultTransport and copied into countless custom transports, calls httpproxy.FromEnvironment. Its source is unambiguous: it reads HTTP_PROXY, HTTPS_PROXY and NO_PROXY (plus lowercase variants) and nothing else. The variable that curl and much of the Unix world have honored for decades is not rejected and not warned about — it is simply invisible.
• golang.org/x/net/proxy — the nice library from above — does read ALL_PROXY. But net/http does not use it. Unless a program's author explicitly wired it into their dialing path, as Syncthing's did, it might as well not exist.

So whether ALL_PROXY does anything depends entirely on which Go program you happen to be running, and you cannot tell without reading its source. The Go team is aware: issue #31813, filed in 2019 by Brad Fitzpatrick himself, proposes that net/http.DefaultTransport honor ALL_PROXY. As of this writing it is still open. Related warts go back even longer: #16715 (for an unregistered non-SOCKS scheme, FromURL returns unknown scheme, while FromEnvironment swallows that error and silently returns the direct dialer) and #13456 (for a while, only the lowercase all_proxy spelling was read).

For restic specifically, the fix is to use the variables that net/http does read:

HTTPS_PROXY=socks5h://127.0.0.1:9050

(plus HTTP_PROXY if your backend is reached over plain HTTP).

Takeaways

First: for Go programs that use the standard HTTP transport, set HTTP_PROXY and HTTPS_PROXY, not ALL_PROXY.

Second: never just assume that setting a proxy variable did anything. The simplest test is to point the variable at a port where nothing is listening and check that the program now fails.

Third: if the proxy is security-critical, look up how the specific program dials out. Even a program that honors ALL_PROXY may, like Syncthing without ALL_PROXY_NO_FALLBACK, treat it as a suggestion rather than a rule.

None of this diminishes restic or Syncthing — they remain some of the finest software in existence, and Syncthing in particular went out of its way to get proxying right. The mess lives a layer below them, in a standard library and an extension library that have disagreed about the environment for a decade.