Add Lux.jl file classifier (known/unknown) with offline trainer
Each uploaded file is scored by a fixed-structure neural net that labels it known (resembling the training set) or unknown — novelty detection over the first 16 + last 16 bytes (scaled to [0,1]), Dense(32->64->16->2), argmax. - src/model.jl: shared architecture + byte->feature mapping (trainer + server) - src/classify.jl: load committed artifact, classify a file at inference - bin/train.jl: offline trainer, 1:1 blended negatives (random + grab-bag), seeded 80/20 split, writes model/classifier.jld2 - worker: classify (annotate-only) and log classification=known|unknown - config: FS_MODEL_PATH; server fails fast if the artifact is missing - deps: Lux, JLD2, Optimisers, Zygote
This commit is contained in:
@@ -10,9 +10,11 @@ For now the "work" is just logging the received filename to prove the flow —
|
||||
this is the seam where real heavy-lifting will go later.
|
||||
"""
|
||||
function handle_job(job::Job, cfg::Config, worker_id::Int)
|
||||
# --- placeholder for real heavy-lifting work -------------------------
|
||||
@info "received file" worker=worker_id id=job.id name=job.original_name size=job.size
|
||||
# --------------------------------------------------------------------
|
||||
# Classify the spooled file (annotate-only for now: the result is logged but
|
||||
# every file still moves to done/ regardless of known/unknown). Sub-32-byte
|
||||
# files short-circuit to :unknown inside classify without touching the model.
|
||||
classification = classify(CLASSIFIER[], job.path)
|
||||
@info "received file" worker=worker_id id=job.id name=job.original_name size=job.size classification=classification
|
||||
dest = move_to(cfg.done_dir, job)
|
||||
@info "completed" worker=worker_id id=job.id dest=dest
|
||||
return nothing
|
||||
|
||||
Reference in New Issue
Block a user