What's Changed Bug Fixes π fix(namespace): handle legacy auto-cleanup policy on directory manifest by @Xuanwo in #9600 fix: start the commit retry budget after catching up by @Xuanwo in #9631 fix(io): count HEAD requests without object body bytes by @geruh in #9648 fix(mem_wal): rank a dot-product memtable index by distance, not by product by @xuanyu-z in #9689 Full Changelog: v13.1.0-beta.1...v13β¦
What's Changed New Features π feat(mem_wal): maintain every index by intent, replaceable on a live writer by @xuanyu-z in #9335 Full Changelog: release-root/13.1.0-beta.N...v13.1.0-beta.1
What's Changed Breaking Changes π perf(encoding)!: initialize only the page metadata a read will touch by @Ali2Arslan in #7465 feat(fts): add BM25F cross-field search by @sbrunk in #7905 feat(format): separate index keys from covering fields by @Ali2Arslan in #9159 feat!: expose file writer page options in dataset APIs by @ddupg in #9192 refactor: read JSON through one accessor in the JSON semantiβ¦
Release root for 13.1.0-beta.N series
What's Changed New Features π feat: enable the fragment reuse index on stable row id datasets by @brendanclement in #9120 feat: prewrite MemWAL blob payloads by @jackye1995 in #9589 Full Changelog: v13.0.0-beta.19...v13.0.0-beta.20
What's Changed Breaking Changes π feat(format): separate index keys from covering fields by @Ali2Arslan in #9159 fix!: derive merge_columns field ids from the dataset manifest by @zhangyue19921010 in #9547 New Features π feat(python): add combined_fields (BM25F) full-text query by @sbrunk in #8550 feat(table): load spilled row lineage ahead of a commit by @BubbleCal in #9338 feat(dataset): spill rβ¦
What's Changed Bug Fixes π fix(index): preserve 4-bit PQ scores in bulk search by @Gabriel39 in #9537 fix: bound raw vector reads when splitting IVF partitions by @Xuanwo in #9574 fix(namespace): build the mTLS REST client with rustls by @Xuanwo in #9582 Full Changelog: v13.0.0-beta.17...v13.0.0-beta.18
What's Changed New Features π feat(format): add a fragment metadata tree by @geruh in #9060 feat(mem_wal): resolve a schema change through one plan by @xuanyu-z in #9143 Performance Improvements π perf(index): coalesce btree page loads into one read_ranges request by @westonpace in #9310 Full Changelog: v13.0.0-beta.16...v13.0.0-beta.17
What's Changed Breaking Changes π feat(fts): add BM25F cross-field search by @sbrunk in #7905 New Features π feat(dataset): spill row lineage at compaction by @BubbleCal in #9337 Bug Fixes π fix(namespace): reload rotated mTLS certificates in the REST client by @maswin in #8456 fix: read schema-only nested columns added under a non-nullable parent by @LuQQiu in #9562 Full Changelog: v13.0.0-beta.1β¦
What's Changed New Features π feat(index): apply shared FRI remapping to scalar and vector queries by @LuQQiu in #9107 Bug Fixes π fix(fts): validate JSON index queries and targets by @app/lance-gatefixer in #8813 fix: tolerate temporary AWS STS errors in AwsCredentialAdapter by @cmccabe in #9561 Performance Improvements π perf(vector): score duplicate pairs with SIMD group kernels by @BubbleCal iβ¦
What's Changed Breaking Changes π perf(vector)!: score duplicate pairs in cache-blocked native tiles by @BubbleCal in #9551 New Features π feat(mem-wal): let a backpressure controller hold back memtable freezes by @hamersaw in #8994 feat(index): accept a vector index a table is too small to train by @xuanyu-z in #9134 feat(dataset): read and write spilled row lineage columns by @BubbleCal in #9336β¦
What's Changed Bug Fixes π fix(index): merge segments across a compaction without emptying the index by @xuanyu-z in #9421 Full Changelog: v13.0.0-beta.12...v13.0.0-beta.13
What's Changed Performance Improvements π perf(fts): track posting validation with one byte per token by @LuQQiu in #9457 Full Changelog: v13.0.0-beta.11...v13.0.0-beta.12
What's Changed New Features π feat(vector): add streaming duplicate pair enumeration by @BubbleCal in #9493 Full Changelog: v13.0.0-beta.10...v13.0.0-beta.11
What's Changed Breaking Changes π refactor: read JSON through one accessor in the JSON semantic module by @Xuanwo in #9479 New Features π feat(index): support MinHash LSH scalar index by @zhangyue19921010 in #9114 feat(format): define hidden row lineage columns by @BubbleCal in #9253 feat(blob): support mixed v1 and v2 files by @Xuanwo in #9459 feat(search): calibrate auto ivf probing for dot prodβ¦
What's Changed Bug Fixes π fix(encoding): split legacy decode batches before i32 offset overflow by @app/lance-gatefixer in #9225 Performance Improvements π perf: share one ScanScheduler per base within a scan by @LuQQiu in #9417 Full Changelog: v13.0.0-beta.8...v13.0.0-beta.9
What's Changed New Features π feat(format): add the MinHash LSH scalar index specification (experimental) by @zhangyue19921010 in #9076 feat(io): compile GooseFS metadata and page caches by @XuQianJin-Stars in #9263 Bug Fixes π fix(scanner): materialize list columns late as a unit by @zhangyue19921010 in #9148 fix: conflict an in-place column rewrite with a projection that dropped its field by @wkβ¦
What's Changed New Features π feat(index): configure IVF shuffle offset preload budget by @jackye1995 in #9434 Bug Fixes π fix(format): move FLAG_UNKNOWN off the tagged FRI bit by @LuQQiu in #9420 Full Changelog: v13.0.0-beta.6...v13.0.0-beta.7
What's Changed New Features π feat(knn): let a caller rebuild a batch KNN node with a new k by @hamersaw in #9334 feat(mem_wal): serve single-column and nested multi-match on the fresh tier by @hamersaw in #9363 Documentation π docs(knn): drop private intra-doc link that fails rustdoc on main by @hamersaw in #9410 Full Changelog: v13.0.0-beta.5...v13.0.0-beta.6
What's Changed New Features π feat(format): feature flag for a fragment reuse index on stable row id tables by @brendanclement in #9119 feat(io): add dataset label mode for object store metrics by @dshepelev15 in #9350 feat(io): accept block_size as a storage option by @dshepelev15 in #9380 Bug Fixes π fix(encoding): emit NIL control words when rep/def levels are non-empty but zero-width by @lichuβ¦
What's Changed Breaking Changes π feat(io)!: paginated directory listing on ObjectStore by @wjones127 in #8606 perf(mem_wal)!: serve the shard manifest from the store that wrote it by @hamersaw in #8640 feat(table): predecessor-conditioned publication for external manifest stores by @wkalt in #8800 feat(namespace)!: accept multiple columns for the merge insert on key by @wjones127 in #8915 feat: mβ¦
What's Changed New Features π feat(io): apply byte-sized batch budget to blob materialization by @lichuang in #8927 feat(index): implement stable partition mapping readers by @LuQQiu in #9064 feat(knn): expose ANNIvfBatchExec's query, width and inputs by @hamersaw in #9262 feat(mem_wal): evaluate cross-column predicates on the fresh tier by @hamersaw in #9292 Bug Fixes π fix(index): narrow f16/f64β¦
What's Changed Breaking Changes π perf(encoding)!: initialize only the page metadata a read will touch by @Ali2Arslan in #7465 New Features π feat(format): specify carried-column storage and allow carrying a keyed column by @vivek-bharathan in #8856 feat(core): define mapping readers with ordered compaction support by @LuQQiu in #9106 feat(mem_wal): cross-column FTS on the fresh tier by @hamersawβ¦
What's Changed New Features π feat(format): define a unified tagged fragment reuse history by @LuQQiu in #9136 feat(io): expose Hugging Face resolve cache option by @Xuanwo in #9236 Bug Fixes π fix(label_list): bound index build memory with a sorted stream by @vivek-bharathan in #8840 fix(datafusion): coerce numeric literals to and from Float16 by @LuciferYang in #8847 fix(linalg): point length-coβ¦
What's Changed Breaking Changes π feat!: expose file writer page options in dataset APIs by @ddupg in #9192 New Features π feat: expose exact V2 versions in bindings by @Xuanwo in #8585 feat(overlay): add a writer for data overlay files by @wjones127 in #8761 feat(encoding): implement plan_decoded_bytes for fixed-width structural decoder by @westonpace in #8792 feat(mem_wal): evaluate compound FTSβ¦
Release root for 13.0.0-beta.N series (same base as 12.1.0-beta.N)
What's Changed Breaking Changes π feat(io)!: paginated directory listing on ObjectStore by @wjones127 in #8606 perf(mem_wal)!: serve the shard manifest from the store that wrote it by @hamersaw in #8640 feat(table): predecessor-conditioned publication for external manifest stores by @wkalt in #8800 feat(namespace)!: accept multiple columns for the merge insert on key by @wjones127 in #8915 feat: mβ¦
Release root for 12.1.0-beta.N series
What's Changed New Features π feat(python): return matched row offsets from fragment update_columns by @zhangyue19921010 in #8891 feat(index): split IVF partitions to target size and join undersized ones in one optimize pass by @BubbleCal in #9051 Bug Fixes π fix(encoding): prevent bits_per_value=0 in FullZipLayout for FSL with AllNull child by @westonpace in #9130 fix: load external fragment reusβ¦
What's Changed New Features π feat(java): support segment-based distributed vector search by @summaryzb in #7169 Bug Fixes π fix: honor zero scanner limits by @app/lance-gatefixer in #9111 Performance Improvements π perf(fts): accelerate wide and queries with bulk intersection by @BubbleCal in #9030 perf(fts): bound classic conjunction scoring in dense windows by @BubbleCal in #9033 Full Changelogβ¦