Jeffrey Ward
d9f32d9aaf
Implements phase A of the DESIGN_clustering.md design: a Dirichlet-process mixture of per-position categoricals over the first 32 header bytes (257-symbol alphabet) that clusters the binary/ pile by file format, plus signature extraction and promotion nomination. All base-Julia (a Lanczos loggamma keeps the Dirichlet-multinomial marginal dependency-free). - src/cluster.jl: header_symbols feature extraction, collapsed Gibbs sampler (phase A), sequential CRP-predictive assignment (phase B core), signatures/ promotion, and ARI/V-measure calibration metrics. - bin/cluster_calibrate.jl: grid-tunes hyperparameters against magic-collapsed ground truth and cross-checks a model-free NCD (gzip) baseline. - FS_CLUSTER_*/FS_PROMOTE_* config knobs; wire cluster.jl into the module. - Tests for the three DESIGN §10 assertions plus the model primitives. Calibrated defaults (n=32, alpha=1.0, beta=0.1) recover known formats at ARI 0.77 (0.885 excl. tar); docx+zip and the ELF family merge correctly and the NCD baseline agrees. DESIGN §11 records the results and three assumptions the data corrected (tar/ELF header-zero merge, the cold-start seeding deadlock, and the Bernoulli signature / Occam-penalized restart scoring).
38 lines
993 B
TOML
38 lines
993 B
TOML
name = "FileServer"
|
|
uuid = "b3f1c2d4-5e6a-4b7c-8d9e-0f1a2b3c4d5e"
|
|
version = "0.1.0"
|
|
authors = ["wardjm@gmail.com"]
|
|
|
|
[deps]
|
|
HTTP = "cd3eb016-35fb-5094-929b-558a96fad6f3"
|
|
JLD2 = "033835bb-8acc-5ee8-8aae-3f567f8a3819"
|
|
JSON3 = "0f8b85d8-7281-11e9-16c2-39a750bddbf1"
|
|
Languages = "8ef0a80b-9436-5d2c-a485-80b904378c43"
|
|
Logging = "56ddb016-857b-54e1-b83d-db4d58db5568"
|
|
Lux = "b2108857-7c20-44ae-9111-449ecde12c47"
|
|
Optimisers = "3bd65402-5787-11e9-1adc-39752487f4e2"
|
|
Oxygen = "df9a0d86-3283-4920-82dc-4555fc0d1d8b"
|
|
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
|
|
UUIDs = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"
|
|
Zygote = "e88e6eb3-aa80-5325-afca-941959d7151f"
|
|
|
|
[compat]
|
|
HTTP = "1.11.0"
|
|
JLD2 = "0.6.4"
|
|
JSON3 = "1.14.3"
|
|
Languages = "0.4.7"
|
|
Logging = "1.11.0"
|
|
Lux = "1.31.4"
|
|
Optimisers = "0.4.7"
|
|
Oxygen = "1.10.2"
|
|
Random = "1.11.0"
|
|
UUIDs = "1.11.0"
|
|
Zygote = "0.7.11"
|
|
|
|
[extras]
|
|
JSON3 = "0f8b85d8-7281-11e9-16c2-39a750bddbf1"
|
|
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
|
|
|
|
[targets]
|
|
test = ["Test", "JSON3"]
|