mirror of
https://github.com/honeymoose/OpenSearch.git
synced 2025-02-06 21:18:31 +00:00
c62f6f8177
The distribution of watches now happens on the node which holds the watches index, instead of on the master node. This requires several changes to the current implementation. 1. Running on shards and replicas In order to run watches on the nodes with the watches index on its primaries and replicas. To ensure that watches do not run twice, there is a logic which checks the local shards, runs a murmurhash on the id and runs modulo against the number of shards and replicas, this is the way to find out, if a watch should run local. Reloading happens 2. Several master node actions moved to a HandledTransportAction, as they are basically just aliases for indexing actions, among them the put/delete/get watch actions, the acknowledgement action, the de/activate actions 3. Stats action moved to a broadcast node action, because we potentially have to query every node to get watcher statistics 4. Starting/Stopping watcher now is a master node action, which updates the cluster state and then listeners acts on those. Because of this watches can be running on two systems, if you those have different cluster state versions, until the new watcher state is propagated 5. Watcher is started on all nodes now. With the exception of the ticker schedule engine most classes do not need a lot of resources while running. However they have to run, because of the execute watch API, which can hit any node - it does not make sense to find the right shard for this watch and only then execute (as this also has to work with a watch, that has not been stored before) 6. By using a indexing operation listener, each storing of a watch now parses the watch first and only stores on successful parsing 7. Execute watch API now uses the watcher threadpool for execution 8. Getting the number of watches for the stats now simply queries the different execution engines, how many watches are scheduled, so this is not doing a search anymore There will be follow up commits on this one, mainly to ensure BWC compatibility. Original commit: elastic/x-pack-elasticsearch@0adb46e658