some things being cleaned up here are simplifications of changes that were 1-to-1 ported from ant, others are left over from when we had a 'released' PDF and are no longer needed
The problem of tracking dirtiness via numbers of chunks is that larger
chunks make stored fields readers more likely to be considered dirty, so
I'm trying to work around it by tracking numbers of docs instead.
The increase of the maximum number of chunks per doc done in previous
issues was mostly random. I'd like to provide users with a similar
trade-off with what the old versions of BEST_SPEED and BEST_COMPRESSION
used to do. So since BEST_SPEED used to compress at most 128 docs at
once, I think we should roughly make it 128*10 now since there are 10
sub blocks. I made it 1024 to account for the fact that there is a preset
dict as well that need decompressing. And similarly BEST_COMPRESSION used
to allow 4x more docs than BEST_SPEED, so I made it 4096.
With such larger numbers of docs per chunk, the decoding of metadata
became a bottleneck for stored field access so I made it a bit faster by
doing bulk decoding of the packed longs.
Allow using placement plugins to compute replica placement on the cluster for Collection API calls.
This is the first code drop for the replacement of the Autoscaling feature.
Javadoc of sample plugin org.apache.solr.cluster.placement.plugins.SamplePluginAffinityReplicaPlacement details how to enable this replica placement strategy.
PR's #1684 then #1845
Instead of configuring a dictionary size and a block size, the format
now tries to have 10 sub blocks per bigger block, and adapts the size of
the dictionary and of the sub blocks to this overall block size.
When index sorting is enabled, stored fields and term vectors can't be
written on the fly like in the normal case, so they are written into
temporary files that then get resorted. For these temporary files,
disabling compression speeds up indexing significantly.
On a synthetic test that indexes stored fields and a doc value field
populated with random values that is used for index sorting, this
resulted in a 3x indexing speedup.
* Restore lucene/version.properties
* Switch release wizard commands from ant to gradle equivalents
* Remove remaining checks for ant
* Remove checks for Java 8
* Update Copyright year
* Minor bug fixes around determining next version for a major release
This removes the ability to replace the IndexingChain / DocConsumer
in Lucenes IndexWriter. The interface is not sufficient to efficiently
replace the functionality with reasonable efforts. It also seems it's completely
unused at this point and hasn't been maintained in years.
Added mentions for BBoxField, NestPathField, RankField (and placehold for QParser, from SOLR-14590), RptWithGeometrySpatialField
Moved Deprecated types into separate table to improve reading comprehension
Added some cross-references for more in-depth reading.
With recent changes to stored fields that split blocks into several sub
blocks, the merge instance has become much slower at random access since
it would decompress all sub blocks when accessing a document. Since
stored fields likely get accessed in random order at flush time when
index sorting is enabled, it's better not to use the merge instance.
On a synthetic benchmark that has one stored field and one numeric
doc-value field that is used for sorting and fed with random values,
this made indexing more than 4x faster.
TermVectorsWriter might consume some heap space memory that
can have a significant impact on decisions made in the IW if
writers should be stalled or DWPTs should be flushed if memory
settings are small in IWC and flushes are frequent. This change adds
RAM accounting to the TermVectorsWriter since it's part of the
DWPT lifecycle and not just present during flush.
Erick Erickson
Chris Hostetter
Houston Putman
Cassandra Targett
Simon Willnauer
noblepaul
Dawid Weiss
Ignacio Vera
Noble Paul
Adrien Grand
Ilan Ginzburg
Erik Hatcher
Mike Drob
Munendra S N
Alexandre Rafalovitch
Julie Tibshirani
Eric Pugh
Moses Schwartz