SOLR-12804: Remove static modifier from Overseer queue access.
SOLR-12896: Introduce more checks for shutdown and closed to improve clean close and shutdown. (Partial)
SOLR-12897: Introduce AlreadyClosedException to clean up silly close / shutdown logging. (Partial)
SOLR-12898: Replace cluster state polling with ZkStateReader#waitFor. (Partial)
SOLR-12923: The new AutoScaling tests are way too flaky and need special attention. (Partial)
SOLR-12932: ant test (without badapples=false) should pass easily for developers. (Partial)
SOLR-12933: Fix SolrCloud distributed commit.
Recent JIRA's (SOLR-12947, SOLR-12965) have added support making it
easier to compose JSON query/faceting requests using SolrJ. But neither
made parsing the responses to these queries any easier.
This commit introduces NestableJsonFacet (along with several companion
types) which are Java representations of the JSON faceting response.
They can be accessed via the new QueryResponse method:
`getJsonFacetingResponse()`.
JsonQueryRequest had `setQuery` methods that took in a query either as a
String or as a Map. But no such overload for MapWriter, a SolrJ
interface used to transmit Maps via "push writing" over the wire. This
commit adds an overload taking this type, so that users can specify
their queries this way as well.
This commit also changes JsonQueryRequest writes out the request, to
ensure it uses "push writing" in non-MapWriter cases as well.
The JSON request API is great, but it's hard to use from SolrJ. This
commit adds 'JsonQueryRequest', which makes it much easier to write
JSON API requests in SolrJ applications.
This test too makes assumptions about how replicas are placed. In the legacy assignment strategy, the replica of a given collection are spread equally across all nodes but with the new policy based strategy, all cores across collections are spread out. Therefore the assumptions in this test were wrong. I've changed this test to use the legacy assignment policy because testing the autoAddReplicas feature doesn't have to depend on new replica assignment strategies. This change also fixes a bug in Assign which used "collection" key instead of "cluster" to figure out which strategy to use.
The testNonRetryableRequests test makes an assumption that a collection's replicas are equally distributed among all nodes but with the policy engine it is not true. Instead the policy engine spreads out the cores belonging to all collections equally among all nodes. This is fixed by only creating the collection needed by tests in this class just-in-time.
Previously, the maxShardsPerNode parameter was not allowed on collections when autoscaling policy was configured. Also if an autoscaling policy was configured then the default was to set an unlimited maxShardsPerNode automatically. Now the maxShardsPerNode parameter is always allowed during collection creation and maxShardsPerNode should be set correctly (if required) regardless of whether autoscaling policies are in effect or not. The default value of maxShardsPerNode continues to be 1 as before. It can be set to -1 during collection creation to fall back to the old behavior of unlimited maxShardsPerNode when using autoscaling policy. This patch also fixes PolicyHelper to find the free disk space requirements of a new replica from the leader only if said leader node is alive.
ConcurrentUpdateSolrClient can batch together many documents when making
an indexing request to Solr. When adding an update request to the
current batch being made, it checks that the query-parameters of the
docs being added match those already in the batch. But prior to this
commit it never checked that the collections/cores were the same.
This could result in documents being sent to the wrong collection if the
same client is used to index documents to two different
cores/collections simultaneously.
This commit addresses this problem, ensuring that documents aren't added
to a batch directed at a different core/collection.
The cluster wide defaults structure has changed from {collectionDefaults: {nrtReplicas : 2}} to {defaults : {collection : {nrtReplicas : 2}}}. The old format continues to be supported and can be read from ZK as well as written using the V2 set-obj-property syntax but it is deprecated and will be removed in Solr 9. We recommend that users change their API calls to use the new format going forward.
This commit deprecates the min_rf parameter. Solr now always includes the achieved replication
factor in the update requests (as if min_rf was always specified). Also, reverts the changes
introduced in SOLR-8034, replicas that don't ack an update will have to recover to prevent
inconsistent shards.
Now, assignment is done with the help of a builder class instead of calling a method with large number of arguments. The number of special cases that had to be handled have been cut down as well.
The API now supports 'nrtReplicas', 'tlogReplicas', 'pullReplicas' parameters as well 'createNodeSet' parameter. As part of this change, the CREATESHARD API now delegates placing replicas entirely to the ADDREPLICA command and uses the new parameters to add all the replicas in one API call.
Simplified test utility TrackingUpdateProcessorFactory.
Reverted some attempts the TRA used to make in avoiding overseer communication (too complicated).
Closes#433
Noble Paul
Jan Høydahl
markrmiller
Joel Bernstein
Jason Gerlowski
markrmiller
Moshe
Erick Erickson
Chris Hostetter
Gus Heck
Andrzej Bialecki
Marvin Bredal Lillehaug
Christine Poerschke
David Smiley
Anshum Gupta
Andrew Lundgren
Shalin Shekhar Mangar
Steve Rowe
Tomas Fernandez Lobbe
Alan Woodward
Varun Thacker