Demote stage-1 per-file logging to @debug; add stage-1 decomposition benchmark

bin/bench_stage1.jl takes handle_classify_job apart — filesize, read_features,
Lux.apply, classify, move_to, enqueue_blocking!, and the log lines — times each
in isolation, then times the real handler end to end under four loggers so the
parts can be checked against the whole.

It found that logging was stage 1's dominant cost: as @info the two per-file
lines cost ~71 us of the handler's ~118 us, roughly 6x the classifier (10.6 us)
and 6x the rename (11.6 us). Nearly all of it is ConsoleLogger formatting
(~64 us), not the FlushLogger's per-message flush (~8 us).

Demoting them to @debug takes stage 1 from 8.5k files/s to 35.3k files/s on one
worker (4.2x). The messages are still available with JULIA_DEBUG=FileServer,
which the benchmark also prices (133 us/file). What remains splits evenly
between the rename (11.7 us) and classify (10.7 us, itself 74% feature read),
so stage 1 is now filesystem-bound; its thread sweep peaks at ~4 workers.

README.md

@@ -510,11 +510,12 @@ the moment it is wired up, and never on the read path.
 
 ## Benchmarking (throughput + memory)
 
-There are three harnesses. Only the first needs a running server:
+There are four harnesses. Only the first needs a running server:
 
 | script | measures | server? |
 |---|---|---|
 | `bin/bench.jl` (below) | intake, end-to-end and per-stage throughput; server RSS | **yes** |
+| [`bin/bench_stage1.jl`](#stage-1-component-benchmark-binbench_stage1jl) | stage 1 taken apart: classify vs. rename vs. enqueue vs. logging | no |
 | [`bin/bench_model.jl`](#model-microbenchmark-binbench_modeljl) | the classifier alone: inference, feature reads, thread scaling | no |
 | [`bin/cluster_calibrate.jl`](#unknown-format-discovery-stage-5-offline) | stage-5 clustering quality vs. an NCD baseline | no |
 
@@ -526,6 +527,9 @@ julia --project=. -e 'using Pkg; Pkg.instantiate()'   # once
 # 1. the model, on its own — no server involved
 julia --project=. -t auto bin/bench_model.jl
 
+# 1b. stage 1 taken apart — also no server
+julia --project=. -t auto bin/bench_stage1.jl
+
 # 2. the pipeline. Start the server in one terminal…
 julia --project=. -t auto bin/server.jl
 
@@ -631,12 +635,70 @@ resets the kernel's peak-RSS counter (`/proc/<pid>/clear_refs`) per run and flag
 a drifted baseline, but for a clean growth figure restart the server between
 memory runs.
 
+### Stage-1 component benchmark (`bin/bench_stage1.jl`)
+
+`bin/bench.jl` reports stage 1 as one number and `bin/bench_model.jl` takes the
+*classifier* apart — but stage 1 is more than the model. Per file it also
+renames the file into its stage directory, pushes a reference onto the
+downstream queue, and logs. `bin/bench_stage1.jl` times each of those in
+isolation, then times the real `handle_classify_job` end to end so the parts can
+be checked against the whole:
+
+```bash
+julia --project=. -t auto bin/bench_stage1.jl
+```
+
+Measured on this machine (Ryzen 7 2700X, 8 cores/16 threads, Julia 1.12; 2000 ×
+64 KiB files, minimum of 5 trials):
+
+| component | per file | share of the handler |
+|---|---|---|
+| `classify()` | 10.7 µs | 38% |
+| ↳ `read_features` | 7.9 µs | 28% |
+| ↳ `Lux.apply` | 2.3 µs | 8% |
+| `move_to` (rename) | 11.7 µs | 41% |
+| `enqueue_blocking!` | 0.12 µs | 0.4% |
+| per-file logging (disabled `@debug`) | 0.29 µs | 1% |
+| **`handle_classify_job`** | **28.3 µs** | 100% |
+
+**This benchmark is why stage 1's per-file log lines are `@debug` rather than
+`@info`.** As `@info` they cost ~71 µs of the handler's ~118 µs — about 6× the
+classifier and 6× the rename — and nearly all of it was `ConsoleLogger`
+*formatting* (~64 µs), not the `FlushLogger`'s per-message flush (~8 µs on top).
+Demoting them took stage 1 from 8.5k files/s to 35.3k files/s on a single worker,
+a 4.2× speedup for no algorithmic change. The script still prices a formatted
+line, so the cost of turning them back on is visible: running the handler under
+`JULIA_DEBUG=FileServer` measures 133 µs per file, a 4.7× slowdown. That is the
+trade — per-file tracing is available when you want it, and off by default,
+with `GET /stats` giving per-file observability that is counted rather than
+formatted.
+
+What's left is evenly split between the rename and the classifier, and neither
+has an easy 2×. Two things worth knowing:
+
+- **The rename, not the model, is the single largest component** (11.7 µs), and
+  it's a plain `mv` within one filesystem. Inside `classify`, the same pattern
+  holds: 7.9 µs of the 10.7 µs is `read_features` — the `open`, the two reads
+  and the `seek` — against 2.3 µs of actual inference. Stage 1 is now a
+  filesystem-bound stage with a neural network attached, not the reverse.
+- **Stage 1 now peaks at ~4 workers.** With the logger removed from the hot path
+  the sweep reads 35.0k/s at 1 worker, 66.6k/s at 2, **73.3k/s at 4**, then
+  *falls back* to 60.1k/s at 8 and 53.3k/s at 16 — every worker renaming into the
+  same two directories contends on the same directory inode. That ceiling
+  coincides with the one `bin/bench_model.jl` finds for inference, so ~4 is the
+  number from both directions: raising `FS_WORKERS` past it costs throughput.
+
+Reported times are the **minimum** over trials. Flags: `--files`, `--reps`,
+`--trials`, `--size`, `--dir`, `--model`, `--threads`, `--no-threads`,
+`--json PATH`.
+
 ### Model microbenchmark (`bin/bench_model.jl`)
 
 `bin/bench.jl` reports stage 1 as a single number — the wall time of
-`handle_classify_job`, which is a feature read, an inference, a rename, a log
-line, and whatever contention the other three pools create. That's the right
-number for capacity planning and the wrong one for "is the model slow?".
+`handle_classify_job`, which is a feature read, an inference, a rename, a
+(disabled) debug line, and whatever contention the other three pools create.
+That's the right number for capacity planning and the wrong one for "is the
+model slow?".
 `bin/bench_model.jl` answers that separately, with no server, queue, or HTTP
 involved:
 
@@ -654,7 +716,12 @@ Measured on this machine (Ryzen 7 2700X, 8 cores/16 threads, Julia 1.12):
 
 So the model is **not** the pipeline's problem, by three orders of magnitude: the
 same run measured stage 1 at 38.7 ms per file, ~4,500× the 8.4 µs `classify()`
-costs. Whatever stage 1 spends its time on, it isn't the network.
+costs. Whatever stage 1 spends its time on, it isn't the network. (That 38.7 ms
+predates the `@debug` demotion above and is a whole-pipeline figure — it includes
+time the stage-1 worker spends *blocked* on a full downstream queue, which is why
+it is three orders of magnitude above the 28.3 µs the handler costs in
+isolation. For the uncontended split, see
+[the stage-1 decomposition](#stage-1-component-benchmark-binbench_stage1jl).)
 
 Two findings worth acting on if stage 1 ever *does* become the constraint:
 
@@ -702,6 +769,7 @@ bin/
   server.jl           entry point
   bench.jl            throughput + memory harness against a running server
   bench_model.jl      classifier microbenchmark (inference, feature reads, scaling)
+  bench_stage1.jl     stage-1 decomposition (classify vs. rename vs. enqueue vs. logging)
   train.jl            offline training script → model/classifier.jld2
   cluster_calibrate.jl offline stage-5 hyperparameter calibration + NCD baseline
   cluster_sweep.jl    stage-5 phase-B runner: sweep binary/, update catalog, write nominations