Two CPU cores, 100% busy, zero work: making a local LLM server event-triggered

For about a month, two of the sixteen CPU cores on my workstation ran at 100% continuously. Not under load. At idle, overnight, with both GPUs at zero utilisation and no requests in flight.

The cause was one line of default behaviour in the inference server and the fix was one flag. The flag is not the useful part of this story. The useful part is that those two cores had been lying to me about how much CPU the inference lane needed, and I had sized the whole machine around the lie.

Here is the plain-language version before any of the detail. The server was checking for work in a tight loop instead of waiting to be told. Asking "anything for me?" a few million times a second costs exactly as much CPU as doing real work, because to the processor it is real work. Nothing was broken. The server was doing precisely what it was configured to do, as fast as it could, forever.

I wrote up the full workstation tuning pass separately, with the lane layout and the thermal numbers. This post is the long version of one paragraph in it.

Two cores at 100% CPU at idle, and the GPUs doing nothing

The box is a Ryzen 9 9950X, 16 physical cores and 32 threads, with two RTX 5090s. It runs three tenants: Forgejo CI, a production web stack, and a local SGLang server holding Qwen3-Coder-Next across both GPUs with --tp 2.

In top, two threads named sglang::scheduler sat at 100.0% each. One note on that number, because it trips people up: in top, 100% means one fully busy thread, not the whole machine. This box all-out reads 3200%. So 200% is two threads out of thirty-two, which is small enough to wave away.

Two things stopped me waving it away.

The first is that the number never moved. Real work fluctuates. These two threads read 100.0% with the decimal place pinned, hour after hour, in a way that no genuine workload does.

The second is that nvidia-smi showed both GPUs at 0% utilisation the entire time. An inference server that is actually busy is busy on the GPU. CPU at the ceiling with the accelerators asleep is not a busy server. It is a server doing something that is not inference.

What it was not

Four candidates, and how each one dies.

A stuck request. The obvious first guess, and the easiest to rule out. The server reports its own running-request count, and a request still in flight would be holding an active batch on the GPU. Zero running requests, zero GPU utilisation. Nothing was stuck because nothing was there.

A leaked batch. Same evidence, opposite failure mode. A leaked batch that was still being processed would put the GPUs to work, and they were idle. A leaked batch that was not being processed would sit in memory doing nothing, which does not spin a CPU core. Neither shape fits a thread at a hard 100%.

Thermal throttling. This one is worth stating plainly because it gets reached for a lot. Throttling is a response to load, not a source of it. Its signature is high temperature with reduced clocks, and utilisation that is flat or falling. What I had was full utilisation on a cool, idle box. That is the opposite reading.

Pinning it somewhere it cannot bother anyone. This is the tempting one, and it deserves a section.

A pinned core is still a hot core

This machine already uses cpusets. Every tenant is confined to a lane, and that machinery was sitting right there, which made it the natural reach: pin the scheduler threads onto two cores nobody else wants and stop thinking about it.

That would have worked, in the sense that the symptom would have left the part of the screen I was looking at.

A cpuset changes where work runs, not whether it happens. nice changes who wins when two things want the same core, and against an otherwise idle core the spin loop wins every time no matter what priority it has, because there is nothing to lose to. Both tools take a thread that is consuming a core for no reason and make it consume a specific core for no reason.

The cost does not go away either. A core held at the top of its boost curve draws power and produces heat, and heat scales roughly with voltage squared times frequency, so the top of the curve is the expensive end. This box sits in a room where people work. Two cores boosting around the clock is a fan-speed decision, and confining them to a named pair does not change the wattage by a single watt.

The worse cost is the one I would not have noticed. Pinning preserves the reading. The measurement still says the inference lane consumes two cores, and I will keep believing it. That turned out to matter more than the heat.

The cause: a non-blocking receive inside a poll loop

SGLang's scheduler receives work over ZeroMQ, a message-passing library. The relevant detail is which kind of receive it uses.

A blocking receive parks the thread until a message arrives. The thread consumes nothing while parked and the operating system wakes it when there is something to read. A non-blocking receive returns immediately whether or not a message is there, so the calling code has to ask again. Put that inside a loop with no sleep, and you get a thread that runs flat out indefinitely, whether the request rate is ten thousand a second or zero.

One rank, one loop, one core. --tp 2 splits the model across both GPUs and runs two scheduler ranks, so it costs two cores. At --tp 8 it costs eight. The price is set by how you shard the model, not by how much work anyone sends it.

Calling this a bug misses what is going on. Busy-polling is the standard always-on server pattern and it is the right default for the deployment SGLang is mostly aimed at. In a datacenter fleet the server is never idle, a blocking wait costs a wake on every request, and at high request rates those wakes add up. Spinning removes them. You spend a core you were going to spend anyway and you buy latency back on every request. At high utilisation that trade is clearly correct.

It is clearly wrong on my machine. The inference lane is idle most of the day and answers a handful of requests when I ask it something. I was paying the fleet price for fleet-scale traffic I do not have.

The fix: --sleep-on-idle

SGLang ships a flag for exactly this. --sleep-on-idle swaps the non-blocking receive for an event-driven zmq.Poller that blocks until the socket has something to say. The thread stops asking and starts waiting. Upstream this is PR sgl-project/sglang#6026, against issues #1730 and #9863.

I added it to the launch script on 2026-07-01, restarted the container, and measured the same idle state as before:

Not reduced. Zero, to the decimal place top prints. One flag, two cores back, and nothing else about the deployment changed.

The number that mattered arrived three weeks later

Two cores back was the headline, and at the time it was the whole write-up. The result worth having showed up on 2026-07-28, when I finally measured what this machine's tenants actually demand instead of assuming it.

Steady state, on a loaded box, everything except CI:

Roughly two tenths of one core, in total, for every non-CI tenant on the machine.

The inference lane, the thing the box mostly exists to run, needed 6.7% of one core. Not 6.7% of the machine. 6.7% of a single core, because the work happens on the GPUs and the scheduler threads mostly move bytes around.

Before the flag, those same two threads read 200%. That is about thirty times the real figure, and it was the figure I had been designing against.

The original allocation reserved twelve physical cores for inference and production and left CI with four. That looked defensible while the inference lane appeared to burn two cores doing nothing. Against 6.7% it was obviously backwards, so I swapped the two blocks: CI took the twelve-core lane, inference and production took the four. CI throughput went from 205 to 441 jobs an hour and queue depth went from 39 to zero. Four physical cores is still roughly twenty times the measured demand.

I cannot claim the busy-poll caused that original split. The reservation was set when CI was small and was simply never re-derived from measurement. What I can say is that the busy-poll would have defended it. Any measurement taken while those threads were pinned at 100% would have agreed that the inference lane needed real cores.

The spin loop burned two cores. It also produced the evidence that justified them.

How to tell a busy-poll from real work

Three signals, in the order that settles the question fastest.

1. Per-thread CPU that does not move. Run top -H and watch the decimal place. Real work varies from sample to sample. A thread sitting at exactly 100.0% across several minutes is spinning.
2. Saturated CPU with an idle accelerator. For any GPU-backed server, nvidia-smi at 0% while a CPU thread is at the ceiling means that thread is not feeding the GPU. Whatever it is doing is not the job.
3. Cost that scales with parallelism rather than traffic. If moving from --tp 2 to --tp 4 doubles idle CPU, you are paying per rank, not per request. Per-request work does not care how you sharded the model.

The pattern is not specific to SGLang. Any serving stack built around a receive loop can be configured into it, so the thing to check in vLLM or anything else is not the engine's reputation but its event loop: does the idle path block, or does it poll, and is there a flag. The symptom is identical everywhere, and so is the test, which is signal 1 above.

What it costs, honestly

Event-triggered serving is not free, and this post would be dishonest without the other side of it.

When the scheduler is blocked on a poll instead of spinning, the first request after an idle period has to wake the thread. That is a scheduler wake and a context switch, and it happens before any inference starts. A spinning thread is already running and skips it.

The wake cost was negligible at this deployment's scale, and I want to be exact about how weak that claim is. I did not isolate and benchmark wake latency. I compared time-to-first-token before and after the change and could not tell them apart, which on a box configured for four concurrent requests and a 256K context is not surprising: time-to-first-token here is dominated by prefill, and a thread wake sits far below the noise floor of that comparison. If you need a defensible wake-latency figure, measure it on your own hardware rather than borrowing mine.

The trade reverses when the server is never idle. At sustained high request rates the poll loop is not burning a core for nothing, it is burning a core to keep a wake off the critical path of every single request, and you are getting something real for it. --sleep-on-idle is a bad idea on a saturated fleet. It is close to free on a machine that spends most of its day waiting.

Which is the rule underneath, and it is not really about inference. The right default depends on the ratio of idle time to busy time, and server software ships with the defaults of the environment its authors run it in. That environment is almost always larger than yours. The smaller your deployment, the more of those defaults are wrong for you, and they are wrong in a consistent direction: they assume you are never idle.

Two cores is not much hardware. What made it worth the attention was the second-order effect. An idle cost that looks like demand will get designed around, and once it is in the design it stops reading as a bug and starts reading as a requirement. The flag returned two cores. Measuring afterwards returned eight more.