In order to deal with the most anticipated scenario, when datafeed
frequency is greater than the query_delay, we add the query_delay
to the frequency in order to determine the next time we will trigger
a real-time run. For example, if frequency is 10s and query_delay 1s,
we make sure to trigger the real-time run at a 10s + 1s = 11s offset.
However, this is not correct in the case the frequency is less or
equal to the query_delay. For example, if frequency is 1s and
query_delay is 10s. we would also end up triggering at 11s offset.
But the right behaviour would be to trigger every second while
ensuring we are searching for up to 10seconds ago.
This commit fixes this issue.
relates elastic/x-pack-elasticsearch#4167
Original commit: elastic/x-pack-elasticsearch@f605885167
If a user specifies an 'Authorization' header in an HTTPRequest we,
which might be needed instead of using HTTP Basic Auth due to using
Bearer Authentication, then in case of an failure, the request gets
logged including that Authorization header.
In addition, each implementation of a sent message for jira/hipchat/slack filters
out special fields when a HTTP request is written in case of a failed
response in order to not leak secret data.
Relates elastic/x-pack-elasticsearch#3800
Original commit: elastic/x-pack-elasticsearch@66efdd9b36
This change disables security for trial licenses unless security is
explicitly enabled in the settings. This is done to facilitate users
getting started and not having to deal with some of the complexities
involved in getting security configured. In order to do this and avoid
disabling security for existing users that have gold or platinum
licenses, we have to disable security after cluster formation so that
the license can be retrieved.
relates elastic/x-pack-elasticsearch#4078
Original commit: elastic/x-pack-elasticsearch@96bdb889fc
This commit moves the dev key into core and renames to make it clear it
is for snapshots, and makes the production key a required parameter of
release builds.
Original commit: elastic/x-pack-elasticsearch@ea299bd5a2
Changes are done in SetupPasswordTool to load the keystore
and set security settings to load password protected keys in SSL configuration.
Check for the verification mode and appropriately use hostname verifier.
Close the keystore after setup-password is complete.
Unit test for CommandLineHttpClient
TODO: TLS documentation needs to be fixed, which will be taken up as a separate
fix due to documentation refactoring in progress.
relates elastic/x-pack-elasticsearch#3760
Original commit: elastic/x-pack-elasticsearch@72e8666e21
This commit adds the setting to use a trial license in the core rest
tests with security project. This is needed so that security is
actually enabled during these tests.
Original commit: elastic/x-pack-elasticsearch@b08bdaeb6f
Aggregations may return `NaN`, and the comparison code would return `true` if this result was passed to a `gte` or `lte` condition.
Original commit: elastic/x-pack-elasticsearch@3b16ae6675
SAML is only available on a Platinum license. If you try and use SAML
on a Gold license, then the error message is misleading - it gives the
equivalent of "cannot find saml realm".
This change adds a standard license error of
"current license is non-compliant for [saml]"
if SAML rest actions are used when SAML is not licensed.
Original commit: elastic/x-pack-elasticsearch@7c0e26d58e
Trying to setup passwords on a red cluster (or a cluster that cannot
reach a quorum) is generally not a good idea.
This commit:
- Adds a check for RED cluster status
- Prompts to confirm execution if the cluster is red
- Prints out the reason/type is an error response is received
- Increases the HTTP read timeout so that master election failures are
reported correctly.
Original commit: elastic/x-pack-elasticsearch@4ffbda23db
This creates a new "beats_system" user and role with the same
privileges as the existing "logstash_system" user/role.
The "beat_system" user is also added as a managed user within
the "setup-passwords" command.
Users who upgrade from an earlier version of Elasticsearch/X-Pack
will need to manually set a password for the beats_system user via
the change password API (or Kibana UI)
Original commit: elastic/x-pack-elasticsearch@6087d3a18e
If a watch is not active, it should still be executed, if it is called
via the execute watch API.
This commit adds an additional method to the execution context to check
for this, which returns true for a manual execution context but checks
the watch status for the triggered one.
Original commit: elastic/x-pack-elasticsearch@18f3f9e84b
This commit fixes the Javadoc build for MonitoringTemplateUtils after
changes to core removed the string and bytes methods from
XContentBuilder.
Original commit: elastic/x-pack-elasticsearch@49f3b95b22
These tests were configured to run every second, some of the watch
history entries could have an execution state throttled, when two
executions happened while waiting. The tests who were affected by
this are now explicitely searching for the watch history entry with
the correct state.
relates elastic/x-pack-elasticsearch#4137
Original commit: elastic/x-pack-elasticsearch@e463cc4770
This ensures that the per watch stats in the trigger service are
properly cleared, when execution is paused.
Original commit: elastic/x-pack-elasticsearch@f5119e4072
In some places in the packaging tests we use heredocs to run some
scripts, for example, under the root user. However, we were running
these heredocs under sh instead of bash. This is confusing since we use
bash everywhere and we end up with syntax in the herdocs that expects
bash but will not necessarily be run under bash. For example, one
circumstance where this arises is the bash syntax [[. This leads to
packaging test failures on some systems (e.g., the debian-8 box) that
are solely due to this sh/bash distinction. We should really be using
bash everywhere, it is the shell that we settled on for all of our
scripts everywhere awhile ago and then we can avoid this confusion. This
commit replaces the usage of sh in the X-Pack packaging tests with bash.
Original commit: elastic/x-pack-elasticsearch@7555bb32c8
Add realm name to all authz audit events: accessDenied, accessGranted,
runAsDenied and runAsGranted.
These event types receive the following attributes: realm,
run_by_realm and run_as_realm to go along with with the existing
attributes: principal, run_by_principal and run_as_principal. The
'effective realm name' (run_as_realm or run_by_realm) is certainly
filterable by ignore policies.
Original commit: elastic/x-pack-elasticsearch@cb3801e197
Audit trail filter policies can now be updated by the cluster update
settings API. Previously, policies were static inside the conf file.
This is helpful because, in practice, coming up with a reasonable
policy set is an iterative process, adding and changing policies to
shave off yet another pesky audit event.
Original commit: elastic/x-pack-elasticsearch@e8a670c427
Looks like sometimes it catches a throttled watch.
```
18:23:19 FAILURE 7.93s | SmokeTestWatcherWithSecurityIT.testSearchTransformHasPermissions <<< FAILURES!
18:23:19 > Throwable elastic/x-pack-elasticsearch#1: java.lang.AssertionError:
18:23:19 > Expected: is "executed"
18:23:19 > but: was "throttled"
18:23:19 > at __randomizedtesting.SeedInfo.seed([C40A591DC5A7785E:ED7ABE031CA86AB3]:0)
18:23:19 > at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
18:23:19 > at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.testSearchTransformHasPermissions(SmokeTestWatcherWithSecurityIT.java:189)
18:23:19 > at java.lang.Thread.run(Thread.java:748)
```
Original commit: elastic/x-pack-elasticsearch@b96b6438d3
This commit adds a Samba4 test fixture that acts as a domain controller
and has the same contents as the cloud active directory instance that
we previously used for tests.
The tests also support reading information from environment variables
so that they can be run against a real active directory instance in our
CI builds.
In addition, this commit also fixes a few issues that surfaced when
making this change. The first is a change in the base DN that is
searched when performing down-level authentication. The base DN is
now the configuration object instead of the domain DN. This change was
required due to the original producing unnecessary referrals, which we
cannot easily follow when running against this test figure. Referrals
cannot easily be followed as they are returned by the ldap server with
an unresolvable DNS name unless the host points to the samba4 instance
for DNS. The port returned in the referral url is the one samba is bound
to, which differs from the port that is forwarded to the host by the
test fixture.
The other issue that is resolved by this change is the addition of
settings that allow specifying non-standard ports for active directory.
This is needed for down-level authentication as we may need to query
the regular port of active directory instead of the global catalog
port as the configuration object is not replicated to the global
catalog.
relates elastic/x-pack-elasticsearch#185
Relates elastic/x-pack-elasticsearch#3800
Original commit: elastic/x-pack-elasticsearch@883c742fba
This stops providing the "hkey" (hash) of the license now that the
Monitoring UI no longer uses it (6.3+).
Original commit: elastic/x-pack-elasticsearch@235402fa92
This adds an indicator to Monitoring's portion of X-Pack usage whether or
not collection is actually enabled. It's no longer enough to have an
exporter defined by default to know if monitoring is actually running.
Original commit: elastic/x-pack-elasticsearch@b2eb881d61
This is the last YAML test, that waits for a watch execution by
specifying some timeout value. This one also gets replaced with a java
test that uses `assertBusy()` and thus is much more likely to succeed.
relates elastic/x-pack-elasticsearch#1513
Original commit: elastic/x-pack-elasticsearch@c2ab8777f4
This adds a minimum compatible version to the model snapshot.
Nodes with a version earlier than that version cannot read
that model snapshot. Thus, such jobs are not assigned to
incompatible nodes.
relates elastic/x-pack-elasticsearch#4077
Original commit: elastic/x-pack-elasticsearch@2ffa6adce0
This adds back usage stats by pickybacking on the watcher stats, which
are already running distributed in order to collect and merge watcher
statistics.
In order to be able to track statistics, we need to add information for
each watch in an in-memory data structure that is processed whenever a
usage request is coming in. This processing creates a number of counters
for each node, which then are merged together in the usage stats.
relates elastic/x-pack-elasticsearch#4071
Original commit: elastic/x-pack-elasticsearch@c8bfed288f
David Kyle
Lee Hinman
Dimitris Athanasiou
Christoph Büscher
Alexander Reelsen
Jay Modi
Tim Brooks
Ryan Ernst
Yogesh Gaikwad
Lee Hinman
Lisa Cawley
jaymode
Alan Woodward
Tim Vernum
Jason Tedor
Tanguy Leroux
Albert Zaharovits
Albert Zaharovits
Nik Everett
Chris Earle
Chris Earle