* remove commented-out/obselete AwaitsFix
All of these issues are fixed, but the AwaitsFix annotation is still there, just commented out. This causes confusion and makes it harder to keep an eye/review the AwaitsFix tests, e.g. false positives when running 'git grep AwaitsFix'
* Remove @AwaitsFix from TestMatchRegionRetriever. The problem has been fixed in LUCENE-10229.
Co-authored-by: Dawid Weiss <dawid.weiss@carrotsearch.com>
This is very noisy, can contain gradle status updates, various other tests.verbose prints from other threads, you name it.
It causes the check to be flaky, and randomly "miss" seeing a test that executed.
Instead, let's look at the zip files. We can still preserve the essence of what the test wants to do, but without any flakiness.
- MergeOnFlushMergePolicy doesn't try to avoid O(n^2) merges, so I'm disabling
the test on it for now.
- TestUpgradeIndexMergePolicy would sometimes wrap with a non-standard merge
policy like the alcoholic merge policy, I forced it to wrap a
TieredMergePolicy.
LogDocMergePolicy would previously always force-merge an index that has 10
segments of size 1 to 10, due to the min doc count. This is not the case
anymore, but the test was assuming that such an index would get merged, so I
fixed the test's expectations.
Also changed the merge policy to keep working when RAM buffers are flushed in
such a way that segments do not appear in decreasing size order using the same
logic as LogMergePolicy.
If a LogByteSizeMergePolicy is used, then it might decide to not merge the two
one-document segments if their on-disk sizes are too different. Using a
LogDocMergePolicy addresses the issue as both segments are always considered
the same size.
This updates TieredMergePolicy and Log(Doc|Size)MergePolicy to only ever
consider merges where the resulting segment would be at least 50% bigger than
the biggest input segment. While a merge that only grows the biggest segment by
50% is still quite inefficient, this constraint is good enough to prevent
pathological O(N^2) merging.
- Removed dependency on LineFileDocs to improve reproducibility.
- Relaxed the expected exception type: any exception is ok.
- Ignore rare cases when a file still appears to have a well-formed footer
after truncation.
The original HNSW paper (https://arxiv.org/pdf/1603.09320.pdf) suggests
to use a different maxConn for the upper layers vs. the bottom one
(which contains the full neighborhood graph). Specifically, they
suggest using maxConn=M for upper layers and maxConn=2*M for the bottom.
This patch ensures that we follow this recommendation and use
maxConn=2*M for the bottom layer.
Doc values terms dictionaries keep the first term of each block uncompressed so
that they can somewhat efficiently perform binary searches across blocks.
Suffixes of the other 63 terms are compressed together using LZ4 to leverage
redundancy across suffixes. This change improves compression a bit by using the
first (uncompressed) term of each block as a dictionary when compressing
suffixes of the 63 other terms. This helps with compressing the first few
suffixes when there's not much context yet that can be leveraged to find
duplicates.
Currently vector's all docs of all fields are fully loaded into memory (for sparse cases).
This happens not only when we do vector search, but also when we open an index to
load meta info for vector readers.
This patch instead uses IndexedDISI to store docIds and DirectMonotonicWriter/Reader to
handle ordToDoc mapping. Benefits are reduced memory usage, and faster loading of
meta info for vector readers.
The skipping logic relies on the points index telling us by how much we can
reduce the candidate set by applying a filter that only matches documents that
compare better than the bottom value.
Some randomized points formats have large numbers of points per leaf, and
produce estimates of point counts for range queries that are way above the
actual value, which in-turn doesn't enable skipping when we think it should. To
avoid running into this corner case, this change forces the default codec on
this test.
Alan Woodward
Tomoko Uchida
Robert Muir
Dawid Weiss
Shai Erera
Rushabh Shah
Adrien Grand
sunwq
Uwe Schindler
Greg Miller
Yuting Gan
Julie Tibshirani
Mayya Sharipova
zacharymorn
Lu Xugang
xiaoping
Ramin ALirezaee