Each uploaded file is scored by a fixed-structure neural net that labels it
known (resembling the training set) or unknown — novelty detection over the
first 16 + last 16 bytes (scaled to [0,1]), Dense(32->64->16->2), argmax.
- src/model.jl: shared architecture + byte->feature mapping (trainer + server)
- src/classify.jl: load committed artifact, classify a file at inference
- bin/train.jl: offline trainer, 1:1 blended negatives (random + grab-bag),
seeded 80/20 split, writes model/classifier.jld2
- worker: classify (annotate-only) and log classification=known|unknown
- config: FS_MODEL_PATH; server fails fast if the artifact is missing
- deps: Lux, JLD2, Optimisers, Zygote
Julia block-buffers stderr when it isn't a TTY, so a long-running
server's logs stayed trapped in the buffer until exit whenever output
was redirected to a file/pipe (log file, tee, journald, container log
driver). This made it look like workers never ran, when in fact the
"received file" lines were only being flushed at shutdown.
Install a FlushLogger wrapper as the global logger in run(), flushing
after every message so app logs, Oxygen request logs, and startup lines
all appear in real time regardless of where stderr points.
Track the resolved dependency versions so deployments build byte-for-byte
identically. Un-ignore Manifest.toml (kept only data/ ignored).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
REST endpoint (Oxygen.jl POST /upload, multipart) that spools uploaded
files to disk, enqueues lightweight references onto a bounded thread-safe
work queue, and hands off immediately (202 + job IDs; 503 when full). A
configurable pool of worker threads pulls jobs off the queue, logs the
received filename (placeholder for real processing), and moves files to
done/ on success or failed/ on error.
- Queue behind an enqueue!/dequeue!/close! seam for a future RabbitMQ swap
- Startup recovery: re-enqueues leftover files in spool/
- Graceful drain on SIGINT and SIGTERM (via atexit)
- Env-var config; filenames sanitized + UUID-prefixed on disk
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
