Added
- Immutable
CodecCatalogand builder APIs for built-in and namespaced third-party persistent formats (498b0b147, 1e447c3aa) - Canonical self-identifying CodecKit Frame v1 with a separate body-format version, bounded bodies and xxHash64 integrity (030c9d868, 1e447c3aa)
- Logical loose and compound file sources for common inspection, compatibility and validation (c4503b33f)
- Structured codec frame, format, checksum and corruption diagnostics (030c9d868, c4503b33f)
- Frame, catalogue, historical-index, legacy-frame, corruption and compound-member coverage (183634d7e, 1e447c3aa, e1cc141fb)
- Codec frame write, open and checksum benchmarks, streamed migration allocation benchmarks, compound inspection benchmarks, Windows storage-path benchmarks, and refreshed published benchmark results (183634d7e, e1cc141fb, 489622546, 647da435a)
- A four-way Windows filesystem benchmark with file-sync, retry and physical-file diagnostics (cfc883c7e, #59)
- Property-based state machines and reusable metamorphic tests for deletion, commits, recovery, reader lifetimes, backup, migration, flushing and suggestions, with contributor guidance for the test infrastructure (edf5f2471, 1df63049c, 27c7f16d3, 936ff5380, 1f6aa2bdf, dc7a42ea8, 8d18771de, f35775092, 17d6fa599, 16e4ba7b7)
./devops test affectedfor selecting tests from changed code areas, plus test reports, Native AOT coverage, cancellation propagation and progress heartbeats in the main test workflow (32f2b85f9, 646cca9a5, 9fec82f5c, a79378779)- Reworked contributor, component and example documentation, with compact feature lists, clearer migration access, display settings and switchable API navigation (c97bc5825, c66289ae4, 0a650f7bf, 7d39092e5)
Changed
IndexInput.ReadSpannow returns stable copied data, while internal codec paths use lifetime-scoped borrowed spans for zero-copy reads (e9de9fac2, #60)- Current binary codec writers now use catalogue descriptors and canonical framing across direct, flush, merge and migration paths (fef419d2c, c57ead65f, bf4b7d7ee)
- DocValues, stored fields, term vectors, postings, vectors, HNSW, BKD, norms, field lengths, term dictionaries and live docs use their normal current writers during migration (777f978c4, bf4b7d7ee, 1e447c3aa)
- Random-access postings, vector, HNSW and BKD formats retain bounded
IndexInputaccess without ordinary-open checksum scans or whole-file materialisation (fef419d2c, 1e447c3aa) - Materialised codec readers validate canonical checksums before returning data (1e447c3aa)
- Inspector and CLI output now distinguish frame version from body-format version and report checksum plus physical location (c4503b33f)
- Migration planning uses staged validation and coordinated rewrites for paired and compound formats (777f978c4, 1e447c3aa)
- Recovery validates all recognised files with the configured catalogue before accepting a commit, while corrupt BKD data can still use the intended numeric-index fallback (1e447c3aa, bf6ca3951)
- Index compatibility includes explicit unknown-format handling (c4503b33f)
- Integrity-protected search cursors reject non-canonical Base64URL encodings (1e447c3aa)
- Token graphs now retain position lengths through analysis; shingles, synonyms and quoted queries preserve connected paths, while index-time graph output requires explicit flattening (6b702a035)
- Platform-specific file and directory synchronisation now sits behind the Store boundary, with precise Windows retry classification and versioned dirty-file tracking that avoids whole-directory scans during durable commits (cfc883c7e, 7a2a3ac8f, #59)
- Durable tracking is isolated per index directory, reopened indexes establish a one-time file baseline, and unsupported Windows directory flushes are cached per volume (7a2a3ac8f, #59)
- Compound readers share one physical mapping across member slices, and file-lifetime operations avoid global-lock filesystem work on Windows storage paths (52cf2bb0c, #61)
- Cancellable top-N search now follows the standard search path for validation, query rewriting, result caching, metrics, tracing, slow-query logging and analytics (66ba12003)
- Phrase, span, term-set and facet hot paths avoid repeated disposal guards, allocations and duplicate tracking while retaining their containing reader lifetime (326dbb185)
- DevOps commands now use PowerShell modules consistently, prefer the repository's configured .NET SDK, and expose the final codec benchmark suites through the release workflow (5769448c3, fa4ba26fb, f0bccb781, 0d93f3a6b)
Fixed
- Durable commits now retry transient Windows file-sync failures, synchronise only completed files in the commit snapshot, and cannot omit inherited segment files after a process restart (b6536c7a8, 7a2a3ac8f, #59)
- IndexWriter now releases its shutdown resources and write lock even when a pending flush fails during disposal (70a124c67, Identified from Lucene.NET #1284)
- Memory-mapped inputs, resident segment state and retained postings cursors now drain active operations before input, directory or searcher disposal can unmap their backing views (e9de9fac2, #60)
- Two-phase rollback restores the last published segment metadata and content token, while reopened writers allocate segment ordinals above the highest existing ordinal (6fd81fa9c)
- Commit publication failures after the atomic rename now leave the writer unusable instead of rolling memory state back over a visible commit (563b42866)
- Commits no longer wait for unrelated background merges before publishing (e7d2cf25e)
- Background merges and delete-then-add flushes cannot publish stale live-document state that resurrects deleted documents (7c8ce5dd3, e3fc24e26)
- Compound background merges read codec members through their segment reader, and token graph emission remains Native AOT compatible (da8301733)
- Merge policies retain committed segment order when assigning merged document IDs, and background merge failures preserve their original exception for diagnosis (6b6f39c4c, #61)
- Windows compound-file writes remain incremental instead of eagerly preallocating, and Windows CI propagates managed test failures correctly (6b6f39c4c, #61)
- Search and segment snapshots exclude temporary files that only resemble committed segment data (4a86755eb)
Compatibility
- Supported indexes from 2.0 onwards remain readable and can be migrated to the canonical format (fef419d2c, 777f978c4, bf4b7d7ee, 1e447c3aa)
- Indexes written or migrated by 3.0 cannot be opened by 2.x; retain a verified pre-migration backup when rollback may be required (6da7b5338)