jeff/file-server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Renamed to DarkStruct

Browse Source
This commit is contained in:
Jeffrey Ward
2026-08-26 10:11:29 -04:00
parent 630bf5f9a7
commit 4aed713939
17 changed files with 62 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
README.md
View File

@@ -1,4 +1,4 @@
# FileServer
# DarkStruct
A minimal Julia service that receives files over HTTP and hands them off to a
pool of worker threads for processing. The HTTP endpoint does no real work: it
@@ -367,11 +367,11 @@ docker compose -f docker-compose.yml -f docker-compose.rabbitmq.yml up
# 15672 are published for exactly this)
docker compose -f docker-compose.yml -f docker-compose.rabbitmq.yml up -d rabbitmq
FS_QUEUE_BACKEND=rabbitmq \
FS_AMQP_URL=amqp://fileserver:fileserver@localhost:5672/ \
FS_AMQP_URL=amqp://darkstruct:darkstruct@localhost:5672/ \
julia --project=. -t auto bin/server.jl
```
Queues are named `<FS_AMQP_PREFIX>.<stage>` — `fileserver.classify`,
Queues are named `<FS_AMQP_PREFIX>.<stage>` — `darkstruct.classify`,
`.enrich`, `.triage`, `.language` — so the broker's queue list reads like the
pipeline, and `/stats` reports each one's depth from a once-a-second poll.
@@ -419,7 +419,7 @@ branch) run in the normal suite. The round trip against a real broker is gated:
```bash
docker compose -f docker-compose.yml -f docker-compose.rabbitmq.yml up -d rabbitmq
FS_TEST_AMQP_URL=amqp://fileserver:fileserver@localhost:5672/ \
FS_TEST_AMQP_URL=amqp://darkstruct:darkstruct@localhost:5672/ \
julia --project=. -e 'using Pkg; Pkg.test()'
```
@@ -555,7 +555,7 @@ init, so the artifact is exactly regenerable from the same inputs.
| `FS_NOMINATED_DIR` | `data/nominated` | One JSON per self-nominated cluster (human promote gate) |
| `FS_QUEUE_BACKEND` | `channel` | `channel` (in-process) or `rabbitmq` (durable); see "Queue backends" |
| `FS_AMQP_URL` | `amqp://guest:guest@localhost:5672/` | Broker connection, credentials included |
| `FS_AMQP_PREFIX` | `fileserver` | Queue names are `<prefix>.<stage>` |
| `FS_AMQP_PREFIX` | `darkstruct` | Queue names are `<prefix>.<stage>` |
| `FS_AMQP_PREFETCH` | worker count | Unacked messages the broker hands one stage at a time |
| `FS_AMQP_CONFIRMS` | `true` | Wait for a publisher confirm before a `202` (intake only) |
| `FS_RECOVER_SPOOL` | `false` | Re-drive `spool/` at startup even on the broker backend (use after a purged broker) |
@@ -800,7 +800,7 @@ classifier and 6× the rename, and nearly all of it was `ConsoleLogger`
Demoting them took stage 1 from 8.5k files/s to 35.3k files/s on a single worker,
a 4.2× speedup for no algorithmic change. The script still prices a formatted
line, so the cost of turning them back on is visible: running the handler under
`JULIA_DEBUG=FileServer` measures 133 µs per file, a 4.7× slowdown. That is the
`JULIA_DEBUG=DarkStruct` measures 133 µs per file, a 4.7× slowdown. That is the
trade. Per-file tracing is available when you want it and off by default, with
`GET /stats` giving per-file observability that is counted, not formatted.
@@ -970,7 +970,7 @@ Flags: `--model`, `--reps`, `--trials`, `--batches`, `--sizes`, `--no-threads`,
```
src/
FileServer.jl module + run() (startup, recovery, workers, serve, shutdown)
DarkStruct.jl module + run() (startup, recovery, workers, serve, shutdown)
config.jl Config struct + env parsing
job.jl Job (the queue reference)
queue.jl JobQueue seam + in-process ChannelQueue