This test too makes assumptions about how replicas are placed. In the legacy assignment strategy, the replica of a given collection are spread equally across all nodes but with the new policy based strategy, all cores across collections are spread out. Therefore the assumptions in this test were wrong. I've changed this test to use the legacy assignment policy because testing the autoAddReplicas feature doesn't have to depend on new replica assignment strategies. This change also fixes a bug in Assign which used "collection" key instead of "cluster" to figure out which strategy to use.
The testNonRetryableRequests test makes an assumption that a collection's replicas are equally distributed among all nodes but with the policy engine it is not true. Instead the policy engine spreads out the cores belonging to all collections equally among all nodes. This is fixed by only creating the collection needed by tests in this class just-in-time.
Previously, the maxShardsPerNode parameter was not allowed on collections when autoscaling policy was configured. Also if an autoscaling policy was configured then the default was to set an unlimited maxShardsPerNode automatically. Now the maxShardsPerNode parameter is always allowed during collection creation and maxShardsPerNode should be set correctly (if required) regardless of whether autoscaling policies are in effect or not. The default value of maxShardsPerNode continues to be 1 as before. It can be set to -1 during collection creation to fall back to the old behavior of unlimited maxShardsPerNode when using autoscaling policy. This patch also fixes PolicyHelper to find the free disk space requirements of a new replica from the leader only if said leader node is alive.
ConcurrentUpdateSolrClient can batch together many documents when making
an indexing request to Solr. When adding an update request to the
current batch being made, it checks that the query-parameters of the
docs being added match those already in the batch. But prior to this
commit it never checked that the collections/cores were the same.
This could result in documents being sent to the wrong collection if the
same client is used to index documents to two different
cores/collections simultaneously.
This commit addresses this problem, ensuring that documents aren't added
to a batch directed at a different core/collection.
The cluster wide defaults structure has changed from {collectionDefaults: {nrtReplicas : 2}} to {defaults : {collection : {nrtReplicas : 2}}}. The old format continues to be supported and can be read from ZK as well as written using the V2 set-obj-property syntax but it is deprecated and will be removed in Solr 9. We recommend that users change their API calls to use the new format going forward.
This commit deprecates the min_rf parameter. Solr now always includes the achieved replication
factor in the update requests (as if min_rf was always specified). Also, reverts the changes
introduced in SOLR-8034, replicas that don't ack an update will have to recover to prevent
inconsistent shards.
Now, assignment is done with the help of a builder class instead of calling a method with large number of arguments. The number of special cases that had to be handled have been cut down as well.
The API now supports 'nrtReplicas', 'tlogReplicas', 'pullReplicas' parameters as well 'createNodeSet' parameter. As part of this change, the CREATESHARD API now delegates placing replicas entirely to the ADDREPLICA command and uses the new parameters to add all the replicas in one API call.
Simplified test utility TrackingUpdateProcessorFactory.
Reverted some attempts the TRA used to make in avoiding overseer communication (too complicated).
Closes#433
Cluster properties restriction of known keys only is relaxed, and now unknown properties starting with "ext."
will be allowed. This allows custom to plugins set their own cluster properties.
This commit adds support for preferredOperation configuration parameter which defaults to movereplica. Changes ComputePlanAction to add all (collection,shard) pair as hints to AddReplicaSuggester when addreplica is selected as the preferred operation.
Shalin Shekhar Mangar
Noble Paul
Steve Rowe
Joel Bernstein
Jason Gerlowski
Jan Høydahl
Tomas Fernandez Lobbe
Andrzej Bialecki
Erick Erickson
Alan Woodward
Varun Thacker
David Smiley