Add Lux.jl file classifier (known/unknown) with offline trainer

Each uploaded file is scored by a fixed-structure neural net that labels it
known (resembling the training set) or unknown — novelty detection over the
first 16 + last 16 bytes (scaled to [0,1]), Dense(32->64->16->2), argmax.

- src/model.jl: shared architecture + byte->feature mapping (trainer + server)
- src/classify.jl: load committed artifact, classify a file at inference
- bin/train.jl: offline trainer, 1:1 blended negatives (random + grab-bag),
  seeded 80/20 split, writes model/classifier.jld2
- worker: classify (annotate-only) and log classification=known|unknown
- config: FS_MODEL_PATH; server fails fast if the artifact is missing
- deps: Lux, JLD2, Optimisers, Zygote
This commit is contained in:
2026-07-02 14:13:57 -04:00
parent 6d685cfcbb
commit e55129e3a4
10 changed files with 1145 additions and 12 deletions

View File

@@ -10,9 +10,11 @@ For now the "work" is just logging the received filename to prove the flow —
this is the seam where real heavy-lifting will go later.
"""
function handle_job(job::Job, cfg::Config, worker_id::Int)
# --- placeholder for real heavy-lifting work -------------------------
@info "received file" worker=worker_id id=job.id name=job.original_name size=job.size
# --------------------------------------------------------------------
# Classify the spooled file (annotate-only for now: the result is logged but
# every file still moves to done/ regardless of known/unknown). Sub-32-byte
# files short-circuit to :unknown inside classify without touching the model.
classification = classify(CLASSIFIER[], job.path)
@info "received file" worker=worker_id id=job.id name=job.original_name size=job.size classification=classification
dest = move_to(cfg.done_dir, job)
@info "completed" worker=worker_id id=job.id dest=dest
return nothing