Stream multipart intake; add throughput/memory benchmark harness
Adds a benchmark harness, which showed that intake buffered each upload whole, then makes intake streaming so the service's flat-memory property holds end to end rather than only for the queue and workers. The measurement problem first: /upload returns 202 once bytes are spooled and a reference is enqueued, so HTTP latency measures intake, not the pipeline. bin/bench.jl instead uploads a corpus and polls the terminal sinks until the count stops moving, reporting intake rate and end-to-end rate separately, sampling server RSS (kernel VmHWM, reset per run) and the intermediate stage depths so the bottleneck stage names itself. That exposed the buffering: HTTP.jl read the body into req.body, parse_multipart_form materialized each part, and read(p.data) copied again before spool_file wrote it — a 256 MiB upload grew RSS ~700 MiB, and 4 concurrent ones pushed a 950 MiB baseline past 2 GiB. - src/multipart.jl: incremental multipart/form-data reader. Pulls fixed chunks off the socket and hands each part's bytes straight to a sink, so memory is bounded by FS_UPLOAD_CHUNK_BYTES (64 KiB), not file size. Interface is two calls in a loop (next_part! then write_part_body! / skip_part_body!) so the handler keeps ordinary control flow. Retains the last length(delimiter)-1 bytes so a delimiter split across chunks still parses; part headers are bounded by policy, not by chunking. - src/server.jl: /upload is served by a stream handler. Oxygen's root handler wraps HTTP.streamhandler, which does request.body = read(stream) before dispatching — so no Oxygen route, not even a @stream route, can stream a body. root_stream_handler intercepts POST /upload at the stream level and delegates the rest to Oxygen unchanged; /upload is therefore absent from Oxygen's metrics and docs. A client hangup is classified as routine (info, not error) and answered best-effort; every exit path drains the body so keep-alive still works. - src/spool.jl: spool_file(bytes) -> spool_stream(write_body!, ...), which removes a partial file on a failed or abandoned write, so restart recovery can never pick up a truncated upload as if it were complete. - config.jl: FS_UPLOAD_CHUNK_BYTES, the intake memory dial. Streaming changes the 503 contract: a buffered handler knew up front how many files a request held, this one discovers them as they arrive. When the queue fills mid-request it no longer abandons the connection — it stops spooling (discarding remaining parts rather than writing files it cannot queue), drains, and answers 503 with the accepted list. Files already queued stay queued. Measured after (fresh server, 64 KiB chunk): 256 MiB +21.8 MiB, 1 GiB +20.8 MiB, 2 GiB +17.0 MiB at concurrency 1 — flat across a 32x size range; 4 concurrent 256 MiB uploads +86.9 MiB, linear in concurrency. A 2 GiB upload sustains 334 MiB/s. Small files did not regress (intake 107 -> 133 files/s, end-to-end 27.6 -> 32.9 files/s, p95 1930 -> 776 ms). A --size sweep that slopes upward is now the regression signal. - Tests (235 pass, 55 new): byte-exact round-trip of 10 files in one request, sizes straddling the chunk boundary (0/1/63/65535/65536/65537/ 131072/196615/1e6) plus a payload stuffed with near-boundary sequences; the same body parsed at chunk sizes 1..10000 to put the delimiter split at every offset; bounded allocation on a 16 MiB part; malformed and truncated bodies; spool_stream cleanup on a failed write. - Known cosmetic caveat, documented: when a body is cut short, HTTP.jl's own closeread logs an EOFError after the handler returns, because Content-Length promised more than arrived. Not reachable from a handler; the old code logged the same thing without replying.
This commit is contained in:
@@ -10,6 +10,7 @@ using Lux
|
||||
using JLD2
|
||||
using Languages
|
||||
|
||||
include("multipart.jl") # streaming multipart reader (defines UPLOAD_CHUNK_BYTES, used by config.jl)
|
||||
include("config.jl")
|
||||
include("job.jl")
|
||||
include("queue.jl")
|
||||
@@ -137,7 +138,11 @@ function run(; overrides...)
|
||||
for i in 1:cfg.text_worker_count]
|
||||
|
||||
register_routes()
|
||||
serve(; host = cfg.host, port = cfg.port, async = true, show_banner = false)
|
||||
# `handler` replaces Oxygen's root stream handler so POST /upload can read its
|
||||
# body incrementally instead of having it buffered into memory first; every
|
||||
# other route still goes through Oxygen (see `root_stream_handler`).
|
||||
serve(; host = cfg.host, port = cfg.port, async = true, show_banner = false,
|
||||
handler = root_stream_handler)
|
||||
|
||||
# Idempotent graceful drain: stop accepting uploads, let workers finish the
|
||||
# buffered jobs, then exit. Called from two places:
|
||||
|
||||
Reference in New Issue
Block a user