The NUL-byte heuristic misfiled any non-ASCII UTF-8 text (accents, CJK,
emoji) as binary and let non-NUL control bytes through as text. is_binary
now calls a file text when its 8000-byte sniff window is valid UTF-8 with
no control bytes outside the text-safe set (tab/newline/CR/ESC/etc).
- trim_truncated_utf8 drops a multi-byte char split by the window edge so
it isn't mistaken for malformed bytes.
- NUL still classifies as binary (valid UTF-8 scalar, non-text control).
- Expanded tests: Unicode, ANSI logs, stray control byte, malformed UTF-8,
boundary-split char; updated README stage-3 description.
Jeffrey Ward
9fd1bf385b