Previously we used to normalize records with their buckets. This required
nested scrolls: an outer scroll over buckets, then a nested scroll for
records in each bucket. This was fragile.
The new approach is to simply scroll first through buckets, then through
records. This is made possible because we no longer store max_record_score
on buckets nor bucket anomaly_score on records.
While making these changes I noticed that the PerPartitionMaxProbabilities
class was redundant (because it was storing max_record_score in the case of
per-partition normalization), so I removed it. I also removed a redundant
Map from the Bucket class and fixed its equals() and hashCode() methods.
relates elastic/x-pack-elasticsearch#1115
Original commit: elastic/x-pack-elasticsearch@efbee63573
Deleting a job issues 2 cluster state updates.
The first marks the job as deleted.
The second actually removes the job.
Both check that there is no datafeed referring to the job.
If a put datafeed request arrives between those 2 cluster
state updates, the datafeed gets created and the final
job cluster state update fails. This means we end up with
both the job and the datafeed, but the job's results and
state have been deleted.
This commit changes the behaviour so that the put
datafeed request fails for a job that is marked as deleted
as this scenario is avoiding partially executing an action.
relates elastic/x-pack-elasticsearch#1510
Original commit: elastic/x-pack-elasticsearch@76fa0f0b1a
This commit changes all results to use the single doc type.
All searches were adjusted to work without the need to specify
type which ensures BWC with 5.4 results.
Additional work is needed to put the new doc mapping in indices
created with 5.4 but it will be done in separate PR.
Relates elastic/x-pack-elasticsearch#668
Original commit: elastic/x-pack-elasticsearch@041c88ac2d
* Reduced a longish timeout to a shorter one, as a watch should be
executed in a HTTP test.
* Ensured that the TimeThrottleIntegration tests only query for own
watches in the watch history, also use random names for watch ids
* HipChatServiceTests configured deprecated logging package, so it was
not possible to follow the HTTP calls to the hipchat service endpoint.
relates elastic/x-pack-elasticsearch#1514
Relates elastic/x-pack-elasticsearch#1515
Original commit: elastic/x-pack-elasticsearch@adb492e4e9
* Remove sequenceNum from anomaly records and influencers
* Generate unqiue IDs without sequence numbers
* Remove more instances of sequence_num
* Handle parsing sequnce_num from v5.4
Original commit: elastic/x-pack-elasticsearch@e60b206daf
The rolling upgrade nodes need a keystore for SSL configuration but
there was no dependency on the task that copies the keystore into the
output directory for the nodes to pick up as an extra configuration
file. This commit addresses this by adding such a dependency. To do
this, we need to break the dependency of the keystore copy task on the
REST spec copy task; this is not an issue since the dependency was for
convenience of ordering the task and not actually needed.
Original commit: elastic/x-pack-elasticsearch@fddbc06e9f
* [DOCS] Add ML aggregations configuration scenario
* [DOCS] Refine ML configuration page
* [DOCS] Add ML aggregation details
* [DOCS] Add links to aggregations in Configuring ML
* [DOCS] Address feedback about ML aggregations
Original commit: elastic/x-pack-elasticsearch@8474144093
This commit simplifies the WatchStatsTests to use a MockScriptPlugin.
The latch script engine previously depended on a static instance of the
engine to contain the latches. These are now moved to statics of the
test class itself.
Original commit: elastic/x-pack-elasticsearch@4170cd1bd3
* Updated docs for cross-cluster search
Wildcards are now supported in both cluster alias and index name,
and the "view_index_metadata" privilege is automatically conferred
by the "read_cross_cluster" role.
* For now, best to allow users permissions to query all indices on all remote clusters
Original commit: elastic/x-pack-elasticsearch@31959f78f4
This commit marks the default password migration test as awaits fix as
the test is reliabily failling.
Original commit: elastic/x-pack-elasticsearch@4377480d98
This commit enables security to work with an index named .security (as
it could before) OR an alias named .security that points to a concrete
index by a different name that has the security index. This prepares
the ability to migrate from a 5.x to 6.x security index that allows
changing and re-indexing the underlying security index while maintaining
a .security alias that points to the underlying updated index.
relates elastic/x-pack-elasticsearch#1216
Original commit: elastic/x-pack-elasticsearch@9fee12e5a0
Suppress many job/datafeed errors if a node is known to be shutting down. Also, ensure started datafeeds and open jobs don't end up stopped/failed due to errors as the shutdown progresses, as this would prevent them automatically relocating to a different node.
relates elastic/x-pack-elasticsearch#1114
Original commit: elastic/x-pack-elasticsearch@e56a7dbea1
This is to allow bwcTest to be run by ci. This commit also fixes the
existing packagingTest target, which was never updated after the xpack
repo split.
Original commit: elastic/x-pack-elasticsearch@f4696c1899
Set job to failed if autodetect manager fails closing, fix force closing of jobs that hang in closing
state, set timeout when waiting for clusterstate update, disallow closing of failed jobs with normal
close
relates elastic/x-pack-elasticsearch#1453
Original commit: elastic/x-pack-elasticsearch@493cf85e22
Aliases might be contained in requests that check index permissions
to disable caches etc. This commit preserves permissions for
aliases as well.
Original commit: elastic/x-pack-elasticsearch@233195aeba
* [DOCS] Add ML geographic functions
* [DOCS] Add script_fields info to ML geo functions
* [DOCS] Remove summary count from ML geographic functions
* [DOCS] Added example title to geographic functions
* [DOCS] Remove list from ML geographic functions
Original commit: elastic/x-pack-elasticsearch@a8e495657f
The authentication object was changed in 5.4.0 in that it was conditionally signed depending on
the version and other factors. A bug was introduced however that causes the authentication to
actually get written with the version of the node it is being sent to even if that version is
greater than the version of the current node, which causes rolling upgrades to fail.
Original commit: elastic/x-pack-elasticsearch@a718ff8a52
David Kyle
David Roberts
Ryan Ernst
Alexander Reelsen
Dimitris Athanasiou
Jason Tedor
Lisa Cawley
Clinton Gormley
Andrew Cholakian
Ali Beyad
Hendrik Muhs
Simon Willnauer
Tim Vernum
Jack Conradson
Jay Modi