Currently numDocs() is computed lazily, but this doesn't help since
BaseCompositeReader calls numDocs() on its sub readers eagerly. This may cause
performance issues since every time we wrap a reader with DocumentSubSetReader
(which means for every query when DLS is enabled) we need to recompute the
number of live documents, which runs in linear time with the number of matches
of the role query.
Not computing numDocs() eagerly in DocumentSubSetReader might help, but it
would also be fragile since callers of this method still usually assume that
it runs in constant time. So I am proposing that we add a cache of the number
of live docs in order to decrease the performance hit of document-level
security. I would expect this cache to be efficient as it will not only reuse
entries in-between refreshes, but also across refreshes for segments that
haven't received any new updates.
Original commit: elastic/x-pack-elasticsearch@5a3af1b174
Renaming should hopefully make it more clear that this is the size
of pages to process during rolling up, nothing to do with the size
of the various groups, metrics, etc.
Original commit: elastic/x-pack-elasticsearch@8a0a44f04b
This constructor was actually never used, other than in tests, and even then,
there is no need for a custom period type as the human-readable toString value
will suffice.
Original commit: elastic/x-pack-elasticsearch@fc666a04b9
It is common for users to wish to adjust the verification_mode in SSL
settings, usually with the intention of skipping hostname
verification. This has been supported for a long time, but the
relevant configuration setting was not clearly documented, which would
sometimes lead users to set `verification_mode` to `none`, and disable
more checks than they intended.
This commit adds clearer documentation regarding the options available
for `verification_mode` and actively discourages the use of `none`.
Original commit: elastic/x-pack-elasticsearch@2fdf53b42f
For the user cache, the crypt option rounds are actually the log2 of the number
of rounds. This commits updates the documentation to reflect this.
Original commit: elastic/x-pack-elasticsearch@d3cc2b7f29
With the change of requiring to configure account settings properly by
using affix settings, we forgot another special snowflake, namely the
configuration of mail properties, which can be arbitrary in the
configuration. Those properties are used when an email is sent.
This commit adds a few (but not all of those) options back and removes
the link in the documentation to refer to all of those settings.
Some settings are useless, as they only change the execution
expectations when a mail is sent, which the watch has control over.
The following settings are supported
* smtp.{host,port,user,password}
* smtp.auth
* smtp.starttls.{enable,required}
* smtp.{timeout,connection_timeout,write_timeout}
* smtp.{local_address,local_port}
* smtp.send_partial
* smtp.wait_on_quit
relates elastic/x-pack-elasticsearch#4048
Original commit: elastic/x-pack-elasticsearch@39d5624710
Adds `value_count` as one of the accepted metrics. The caveat is that
it only accepts numeric values for two reasons:
- Job validation at creation makes sure all metrics are numeric fields.
Changing this would require new syntax (or disallowing anything but
value_count on mixed fields)
- when `toBuilders()` is called, we have to supply a ValueSource to
the ValueCountBuilder, and we don't know what the field type is at that
time.
These are both fixable, but relatively more involved. I think numeric-only
is a reasonable limitation to start with
Original commit: elastic/x-pack-elasticsearch@270f24c8bf
Add a check to the migration assistant to warn about the renaming of
`delimited_payload_filter` to `delimited_payload`. This should still
word for old indices from 7.0 on but will throw an error for newly
created indices and the user should be warned about it when running the
migration checks.
Original commit: elastic/x-pack-elasticsearch@5d55e4e499
We specify an alias for signing key, but when we just have
a single key in key store this is an additional setting which
is annoying. This PR addresses this issue by making it optional.
- Changes in SamlRealmSettings to make signing/encryption
key alias optional
- Checks if none of the keys are useful for given operation
signing or encryption throws an error.
- Checks for no of aliases in key-store, if more than one and alias
is not specified throws error.
- If an alias is not specified and there is just one alias in
keystore then use it as the credential.
- Unit Tests
Note: A side effect of this change the above-mentioned behavior is
it's also applicable for encryption keys currently, but it is going
to change when fixing elastic/x-pack-elasticsearch#3980 for supporting multiple encryption keys.
relates elastic/x-pack-elasticsearch#3981
Original commit: elastic/x-pack-elasticsearch@2b5af1d8a8
The computed field contained a list of all aggs that were computed
for this particular rollup doc. It was used to help filter to the
correct rollup job/set of jobs.
But this functionality was never perfect, and has been obsoleted by
validating the rollup caps while searching. So we can remove the
computed field and save a bunch of space (since they were quite bulky)
Original commit: elastic/x-pack-elasticsearch@455644488f
This commit changes the combination of multiple automatons representing
a pattern so that the result of each step is minimal. Previously, the
code unioned the automata and performed the minimization operation
after all of the automata had been combined. This resulted in patterns
with lots of overlap causing a TooComplexToDeterminizeException even
though the end result could be a automaton that is total. Minimizing
the automata as we go, allows us to build an automata that could not
previously be built at the cost of additional operations. Automata are
typically cached in the security code, so the net performance impact
should be minimal.
Original commit: elastic/x-pack-elasticsearch@b59fe8d690
This commit fixes several issues with the current implementation of
starting & stopping watcher
1. The WatcherServiceResponse was always returning a message, that the
request was acknowledged, completely independent from the fact if it was
or not.
2. A new cluster state instance was always returned, regardless if the
state had changed or not (which is explicitely mentioned in the
javadocs to check for this)
3. The AckedClusterStateUpdateTask now returns a proper WatcherServiceResponse
4. A failure now gets logged
Relates elastic/x-pack-elasticsearch#4225 (this is just a hunch for now)
Original commit: elastic/x-pack-elasticsearch@f4c1749f95
This PR adds logic to ensure that the fields (and field types) configured
in the Rollup Job are present in the index/indices specified by the job's
index pattern. If a field is missing, or is not aggregatable, it
will throw an exception before the job is created.
This is important for user-friendliness, because otherwise the user
only discovers an issue with mapping when the job is started and
fails to rollup correctly (and only really noticeable by looking at logs,
since it's a runtime failure).
Original commit: elastic/x-pack-elasticsearch@686cd03072
This avoid setting the payload to `null` when sending it to the remote
monitoring cluster. The payload will be GCed when this overall object is
GCed, which should be very soon anyway.
Original commit: elastic/x-pack-elasticsearch@04f63c4150
When a watch is acknowledged, while it is also being executed, the
acknowledgment information can get lost. The reason for this is the
fact, that the execution writes the watch status inside of the watch
regardless, if other writes happened inbetween to make sure the
execution state is caught.
This commit checks the current executions in the execution service and
aborts the API call, if the specified watch ID can be found in those.
Note, this does not prevent this issue fully, as a watch could be
triggered, while the acknowledgement update is running, but it does
reduce the surface area of this problem. In order to properly solve
this, indexing the watch status as part of a watch would need to be
changed.
relates elastic/x-pack-elasticsearch#4003
Original commit: elastic/x-pack-elasticsearch@d7e218b2ac
Many users take the JSON from an PUT watch API and put it into the
execute watch API and then start to wonder why there is an error, as
they forget to wrap the watch inside a 'watch' field.
This commit adds a better error message in this case hinting at the user
to add a 'watch' field.
Original commit: elastic/x-pack-elasticsearch@5b56b4abad
This removes the `assert` that has been causing some very unexpected test
failures due to unexpected threading issues.
Some of the code changed and became async, so it is no longer guaranteed
that the same thread calls `doFlush` and `doClose`. We could similarly
make the field `volatile`, but since this `assert` is not really helping
anything it's easier to remove it.
Original commit: elastic/x-pack-elasticsearch@ba39de605f
If the license key specified by the system property license.key does not
exist, Gradle does not care. Gradle should care, so this commit makes it
care.
Original commit: elastic/x-pack-elasticsearch@afc0a1443c
This commit fixes an issue in the security nio transport tests where
renegotiation was not actually being tested. We were not waiting on the
handshake complete latches. This commit fixes this issue.
Original commit: elastic/x-pack-elasticsearch@47bebc5c13
* Add test matrix axis files for periodic java testing
* Add properties file defining java versions to use
Original commit: elastic/x-pack-elasticsearch@d679b9ab3e
Serialization assertions in ElasticsearchAssertions, a transport
interceptor that used them, and a plugin that added that interceptor
were removed from the test framework. This test case no longer needs to
exclude them from its plugins.
Original commit: elastic/x-pack-elasticsearch@07e5c58983
Adrien Grand
Hendrik Muhs
Lee Hinman
Zachary Tong
Zachary Tong
Dimitris Athanasiou
David Kyle
Tim Vernum
Igor Motov
Lisa Cawley
Alexander Reelsen
Christoph Büscher
Yogesh Gaikwad
Jay Modi
Alan Woodward
Albert Zaharovits
Chris Earle
Jason Tedor
Tim Brooks
David Leatherman
Aaron Bull Schaefer
Andy Bristol