Default raised from 500 to 50_000 so legitimate long-form content
(novels, scholarly editions) is not silently dropped on a typical
literary corpus. Cap configurable via MEMPALACE_MAX_CHUNKS_PER_FILE
env var or --max-chunks-per-file CLI flag; sentinel 0 disables.
Negative or non-int values from either source emit a stderr warning
and fall back to the default.
Separate counter (files_skipped_chunk_cap) and summary line surface
chunk-cap drops independently of the residual already-filed bucket;
counter fires under --dry-run too so a corpus audit shows the same
signal without writing. Skip notice routed to stderr alongside the
existing symlink-skip line.
ONNX bad_alloc protection preserved architecturally:
DRAWER_UPSERT_BATCH_SIZE=1000 bounds per-ONNX-call exposure
regardless of per-file cap. A 50_000-chunk file produces 50 forward
passes of 1_000 chunks each, identical to what 500-chunk files
already produced.
process_file return extended to a 3-tuple (drawers, room,
skip_reason); skip_reason is None on non-chunk-cap paths and
"chunk_cap" when the per-file cap triggered. Internal callers
(_mine_impl, benchmarks/mine_bench.py, tests) updated.
Closes#1455
Co-Authored-By: David Glidden <d@davidglidden.eu>