Commit Graph

6313 Commits

Author SHA1 Message Date
Nik Everett 1af9f56834 Remove `IndexResponse.created`
Core removed this in elastic/elasticsearch#25516.

Original commit: elastic/x-pack-elasticsearch@b1e7040fa2
2017-07-07 13:57:43 -04:00
Lisa Cawley 2e58164b96 [DOCS] Add ML limitation (elastic/x-pack-elasticsearch#1916)
Original commit: elastic/x-pack-elasticsearch@7730810f98
2017-07-07 10:17:55 -07:00
Lisa Cawley 2aa53f253f [DOCS] Update doc links for graph API (elastic/x-pack-elasticsearch#1884)
Original commit: elastic/x-pack-elasticsearch@9eebb6c9a6
2017-07-07 10:11:06 -07:00
Tim Sullivan 66ebdff447 [Monitoring] Email actions for Cluster Alerts (elastic/x-pack-elasticsearch#1879)
* [Monitoring] Email actions for Cluster Alerts

* fix quotations in email fields

* move email vars to transform, and rename for snake_case

* add state to email subject for cluster status alert

* remove types field in kibana_settings search

* simplify email action condition script

* uppercase the state for the email subject

* only append state to email subject if alert is new

* show state in email subject even when alert is resolved

Original commit: elastic/x-pack-elasticsearch@e6fdd8d620
2017-07-07 09:55:45 -07:00
Simon Willnauer c87d9278a6 Add validation for all `transport.profile.*` settings (elastic/x-pack-elasticsearch#1909)
Follow-up from elasticsearch/elastic#25508

Original commit: elastic/x-pack-elasticsearch@fe08e74ccc
2017-07-07 09:41:34 +02:00
Tim Vernum c5012ac6e8 [DOC] Miscellaneous security doc updates (elastic/x-pack-elasticsearch#1908)
- Document refresh interval for role mapping files
- Fix obsolete shield reference in transport profile example 
- Clarify that AD & PKI don't support run_as
- Fix logstash conf examples
- Clarify interaction of SSL settings and PKI realm settings
- Document PKI DN format, and recommend use of pki_dn metadata
- Provide more details about action.auto_create_index during setup

Original commit: elastic/x-pack-elasticsearch@49ddb12a7e
2017-07-07 13:33:35 +10:00
Jason Tedor b636dcc366 Upgrade to Netty 4.1.13.Final
This commit updates the SHAs here as core upgraded the Netty dependency
from 4.1.11.Final to 4.1.13.Final.

Relates elastic/x-pack-elasticsearch#1936

Original commit: elastic/x-pack-elasticsearch@d1803fc331
2017-07-06 15:37:17 -04:00
Lisa Cawley 0e322b525d [DOCS] Update doc links for security APIs (elastic/x-pack-elasticsearch#1883)
Original commit: elastic/x-pack-elasticsearch@660410008c
2017-07-06 11:06:03 -07:00
Tim Brooks d95c365e64 Loosen setup mode restrictions for upgrade tests (elastic/x-pack-elasticsearch#1927)
This commit is related to elastic/x-pack-elasticsearch#1896. Currently setup mode means that the
password must be set post 6.0 for using x-pack. This interferes with
upgrade tests as setting the password fails without a properly
upgraded security index.

This commit loosens two aspects of the security.

1. The old default password will be accept in setup mode (requests
from localhost).
2. All request types can be submitted in setup mode.

Original commit: elastic/x-pack-elasticsearch@8a2a577038
2017-07-06 10:37:48 -05:00
Dimitris Athanasiou 4e03c657a3 [ML] Write model_memory_limit with units into the cluster state (elastic/x-pack-elasticsearch#1934)
This is step 2 of elastic/x-pack-elasticsearch#1604

This change stores `model_memory_limit` as a string with `mb` unit.
I considered using the `toString` method of `ByteSizeValue` but it
can lead to accuracy loss. Adding the fixed `mb` unit maintains
the accuracy, while making clear what unit the value is in.

Original commit: elastic/x-pack-elasticsearch@4dc48f0ce8
2017-07-06 15:40:19 +01:00
David Roberts 984d2ca2ba [ML] Ignore unknown fields when parsing ML cluster state (elastic/x-pack-elasticsearch#1924)
ML has two types of custom cluster state:

1. jobs
2. datafeeds

These need to be parsed from JSON in two situations:

1. Create/update of the job/datafeed
2. Restoring cluster state on startup

Previously we used exactly the same parser in both situations, but this
severely limits our ability to add new features.  This is because the
parser was very strict.  This was good when accepting create/update
requests from users, but when restoring cluster state from disk it meant
that we could not add new fields, as that would prevent reloading in
mixed version clusters.

This commit introduces a second parser that tolerates unknown fields for
each object that is stored in cluster state.  Then we use this more
tolerant parser when parsing cluster state, but still use the strict
parser when parsing REST requests.

relates elastic/x-pack-elasticsearch#1732

Original commit: elastic/x-pack-elasticsearch@754e51d1ec
2017-07-06 13:36:11 +01:00
Dimitris Athanasiou adc6fd5a0f [ML] Parse model_memory_limit both as number and string with units (elastic/x-pack-elasticsearch#1921)
This is the first step for elastic/x-pack-elasticsearch#1604.

Original commit: elastic/x-pack-elasticsearch@70010a216d
2017-07-06 12:29:12 +01:00
Lee Hinman b66560fa85 [TEST] Use better REST endpoints for testing watch actions
This is the x-pack side of https://github.com/elastic/elasticsearch/pull/24437

It changes two things, for the disable tests, it uses a valid endpoint instead
of a previously invalid endpoint that happened to return a 400 because the
endpoint was bad, regardless of if watcher was disabled.

The other change is to create the watches index by putting a watch using the
correct API, rather than manually creating the index. This is because
`RestHijackOperationAction` hijacks operations like this and stops accessing the
endpoint in a regular manner.

Original commit: elastic/x-pack-elasticsearch@3be78d9aea
2017-07-05 10:55:25 -06:00
Igor Motov c035adb568 Rename upgrade.* client methods to migration.* (elastic/x-pack-elasticsearch#1881)
This makes client names consistent with REST APIs and makes it simplifies client development.

Original commit: elastic/x-pack-elasticsearch@90913f485b
2017-07-05 09:47:58 -04:00
Dimitrios Athanasiou 2e0560528f [TEST] Fix MlBasicMultiNodeIT after changing flush response
Relates elastic/x-pack-elasticsearch#1914

Original commit: elastic/x-pack-elasticsearch@5175bf64d9
2017-07-05 13:30:25 +01:00
Dimitrios Athanasiou 3057f7f4b6 [TEST] Fix post data request in post_data.yml
Original commit: elastic/x-pack-elasticsearch@14e9083c02
2017-07-05 12:06:53 +01:00
Dimitris Athanasiou 15f9b1ed9c [ML] Impove mechanism for ignoring maintenance windows (elastic/x-pack-elasticsearch#1914)
Currently, the autodetect process has an `ignoreDowntime`
parameter which, when set to true, results to time being
skipped over to the end of the bucket of the first data
point received. After that, skipping time requires closing
and opening the job. With regard to datafeeds, this does not
work well with real-time requests which use the advance-time
API in order to ensure results are created for data gaps.

This commit improves this functionality by making it more
flexible and less ambiguous.

- flush API now supports skip_time parameter which
sends a control message to the autodetect process
telling it to skip time to a given value
- the flush API now also returns the last_finalized_bucket_end
time which allows clients to resume data searches correctly
- the datafeed start API issues a skip_time request when the
given start time is after the resume point. It then resumes
the search from the last_finalized_bucket_end time.

relates elastic/x-pack-elasticsearch#1913


Original commit: elastic/x-pack-elasticsearch@caa5fe8016
2017-07-05 11:33:42 +01:00
Lisa Cawley 595468a505 [DOCS] Update doc links for watcher APIs (elastic/x-pack-elasticsearch#1880)
Original commit: elastic/x-pack-elasticsearch@47521f1261
2017-07-04 11:16:47 -07:00
Lisa Cawley 04ff94a180 [DOCS] Update doc links for ML APIs (elastic/x-pack-elasticsearch#1882)
Original commit: elastic/x-pack-elasticsearch@cdc45f282d
2017-07-04 11:10:34 -07:00
Sophie Chang 0ff26e7566 [DOCS] Added limitation for time-based index patterns (elastic/x-pack-elasticsearch#1910)
* [DOCS] Added limitation for time-based index patterns

* [DOCS] Updated limitations

Original commit: elastic/x-pack-elasticsearch@9927105cc0
2017-07-04 08:40:42 -07:00
David Roberts b93890b0df [ML] Add license downgrade warning (elastic/x-pack-elasticsearch#1912)
When applying a license that will cause ML to be disabled, a warning
to this effect is now included in the acknowledgement messages.

relates elastic/x-pack-elasticsearch#1888

Original commit: elastic/x-pack-elasticsearch@e453f20f58
2017-07-04 15:10:18 +01:00
Dimitrios Athanasiou 05a73a32bd [TEST] Correctly verify that aliases were deleted in MlJobIT
This fixes `testDeleteJobAfterMissingAliases` to not fail randomly.
The reason the test was failing is that at some point some aliases
are deleted and the cat-aliases API is called to verify they were
indeed deleted. This was checked by asserting an
index_not_found_exception was thrown by the cat-aliases request.
This was some times working as there were no other aliases. However,
that depends on whether other x-pack features had time to create their
infrastructure. For example, security creates an alias. When other
aliases had the time to be created, the cat-aliases request does not
fail and the test fails.

This commit simply changes the verification that the read/write
aliases were deleted by replacing the cat-aliases request with
two single get-alias requests.

Original commit: elastic/x-pack-elasticsearch@fe2c7b0cb4
2017-07-04 13:14:38 +01:00
Alexander Reelsen fff72256a5 Watcher: Fix wrong logging in reporting attachment parser (elastic/x-pack-elasticsearch#1900)
The logging shows a wrong HTTP response status code from a previous
request. In addition the body now also gets logged, as debugging
is impossible otherwise.

Original commit: elastic/x-pack-elasticsearch@cc998cd587
2017-07-04 13:01:14 +02:00
Clinton Gormley fd518ea020 Include shared/attributes.asciidoc directly from docs master
Original commit: elastic/x-pack-elasticsearch@f3a0828c5d
2017-07-03 18:23:19 +02:00
Christoph Büscher 8d26996afd Remove QueryParseContext (elastic/x-pack-elasticsearch#1895)
This is the x-pack side of elastic/elasticsearch#25486.

Original commit: elastic/x-pack-elasticsearch@c90a3e096b
2017-07-03 17:31:18 +02:00
Simon Willnauer 20f6d66294 Adopt to network settings cleanup in elastic/elasticsearch#25489
Original commit: elastic/x-pack-elasticsearch@364bb260ee
2017-07-02 10:17:30 +02:00
Chris Earle 02c0ad2aad [Monitoring] Reduce NodeStats Collection to required Data (elastic/x-pack-elasticsearch#1240)
This changes from collecting every index statistic to only what we actually want. This should help to reduce the performance impact of the lookup.

Original commit: elastic/x-pack-elasticsearch@80ae20f382
2017-06-30 19:42:44 -04:00
Tim Brooks 7b3b2d5f02 Localhost check: check if addr bound to interface (elastic/x-pack-elasticsearch#1901)
This is related to elastic/x-pack-elasticsearch#1217 and elastic/x-pack-elasticsearch#1896. Right now we are checking if an
incoming address is the loopback address or a special local addres. It
appears that we also need to check if that address is bound to a
network interface to be thorough in our localhost check.

This change mimicks how we check if localhost in `PatternRule`.

Original commit: elastic/x-pack-elasticsearch@a8947d6174
2017-06-30 14:19:49 -05:00
Dimitrios Athanasiou 8264cbf72f [TEST] Stabilise UpdateInterimResultsIT
Depending on the random numbers fed to the analytics,
it is possible that the first planted anomaly ends up
in a different bucket due to the overlapping buckets feature.
Then that may result to a single interim bucket being available
due to overlapping buckets blocking the other interim bucket
from being considered.

I am removing the initial anomaly from the test as it is not useful
and it makes the test unstable.

relates elastic/x-pack-elasticsearch#1897

Original commit: elastic/x-pack-elasticsearch@aca7870708
2017-06-30 17:11:54 +01:00
Christoph Büscher e7e24c453c Reenable SecurityIndexSearcherWrapperIntegrationTests (elastic/x-pack-elasticsearch#1894)
Original commit: elastic/x-pack-elasticsearch@03ff1bf9a5
2017-06-30 16:52:47 +02:00
David Roberts 10c37f0fa4 [TEST] Improve diagnostics for ML interim results test failure
Original commit: elastic/x-pack-elasticsearch@2ccc9d71ae
2017-06-30 11:47:53 +01:00
Tim Brooks 76bf3ba767 Bring back disabling-default-password docs section
There are multiple references to this section in different areas of the
documentation. This commit brings back this section to fix the build.

A more extensive PR updating the documentation for "no default
password" work will follow up.

Original commit: elastic/x-pack-elasticsearch@0378e78c8a
2017-06-29 16:23:58 -05:00
Tim Brooks c773115c39 Remove doc reference disabling-default-password
This section has been removed from setting-up-authentication. This
commit removes a reference to this section that no longer exists.

Original commit: elastic/x-pack-elasticsearch@43aa0077f9
2017-06-29 16:01:44 -05:00
Jay Modi a9707a461d Use a secure setting for the watcher encryption key (elastic/x-pack-elasticsearch#1831)
This commit removes the system key from master and changes watcher to use a secure setting instead
for the encryption key.

Original commit: elastic/x-pack-elasticsearch@5ac95c60ef
2017-06-29 14:58:35 -06:00
Tim Brooks f2cbe20ea0 Remove default passwords from reserved users (elastic/x-pack-elasticsearch#1665)
This is related to elastic/x-pack-elasticsearch#1217. This PR removes the default password of
"changeme" from the reserved users.

This PR adds special behavior for authenticating the reserved users. No
ReservedRealm user can be authenticated until its password is set. The
one exception to this is the elastic user. The elastic user can be
authenticated with an empty password if the action is a rest request
originating from localhost. In this scenario where an elastic user is
authenticated with a default password, it will have metadata indicating
that it is in setup mode. An elastic user in setup mode is only
authorized to execute a change password request.

Original commit: elastic/x-pack-elasticsearch@e1e101a237
2017-06-29 15:27:57 -05:00
Christoph Büscher 075eda4fc1 Temporarily disable SecurityIndexSearcherWrapperIntegrationTests
Original commit: elastic/x-pack-elasticsearch@bcef6ae8c6
2017-06-29 20:46:30 +02:00
Christoph Büscher 7c6b8ffa36 Adapting to changes in https://github.com/elastic/elasticsearch/pull/25448 (elastic/x-pack-elasticsearch#1887)
Original commit: elastic/x-pack-elasticsearch@5cdf5a2372
2017-06-29 17:10:34 +02:00
Jay Modi f60c0f893c Test: add a basic rest test for CCS with non-matching remote index patterns (elastic/x-pack-elasticsearch#1866)
This commit adds a basic rest test to verify that security works with cross cluster search when a
remote pattern is provided and no remote indices match.

Relates elastic/elasticsearch#25436
relates elastic/x-pack-elasticsearch#1854

Original commit: elastic/x-pack-elasticsearch@e804d0bb12
2017-06-29 07:14:11 -06:00
Christoph Büscher 3ff5ee3f47 Adapting to merging GetField and SearchHitField to DocumentField (elastic/x-pack-elasticsearch#1860)
Follow up to changes in https://github.com/elastic/elasticsearch/pull/25361

Original commit: elastic/x-pack-elasticsearch@5b1ca009f6
2017-06-29 11:36:20 +02:00
Deb Adair 5dc9fed9da Reverting broken change to skip testing in info.asciidoc."
This reverts commit elastic/x-pack-elasticsearch@5e4d77f4ca.

Original commit: elastic/x-pack-elasticsearch@6dca6d7e9f
2017-06-28 13:26:00 -07:00
Simon Willnauer 2f10afa40e Use explicit settings for simple bool settings (elastic/x-pack-elasticsearch#1871)
We made the mistake to generate way to many settings in xpack which makes
finding out the right string and where it's defined super difficult. If
we use constants we can just use commandline tools to find where the settings
are defined. This also removes 1.x and 2.x BWC from the enabled settings which should
be removed in 6.x

Original commit: elastic/x-pack-elasticsearch@ec25e6c40c
2017-06-28 22:07:08 +02:00
Deb Adair 2c7e39155d [DOCS] Fixed cross-doc link to Graph Troubleshooting.
Original commit: elastic/x-pack-elasticsearch@6d0bcf29c7
2017-06-28 13:01:35 -07:00
Deb Adair 8e3238f3b5 [DOCS] Temporarily added info.asciidoc to the test exclude list. Failing due to missing LS output from _xpack.
Original commit: elastic/x-pack-elasticsearch@5e4d77f4ca
2017-06-28 12:36:09 -07:00
lcawley cbf7c32b88 [DOCS] Fix broken link to security API
Original commit: elastic/x-pack-elasticsearch@85fa16e160
2017-06-28 12:00:28 -07:00
Lisa Cawley 08fdac5a93 [DOCS] Move security APIs to Elasticsearch Ref (elastic/x-pack-elasticsearch#1877)
* [DOCS] Move security APIs to Elasticsearch Ref

* [DOCS] Update links to security APIs

* [DOCS] Fix link to security APIs

Original commit: elastic/x-pack-elasticsearch@d7a9d3f1ab
2017-06-28 11:02:40 -07:00
Tim Brooks f275a3f07b Support bootstrap password when in container (elastic/x-pack-elasticsearch#1832)
This is related to elastic/x-pack-elasticsearch#1217. This commit reads two environment variables on
startup: BOOTSTRAP_PWD and ELASTIC_CONTAINER. If BOOTSTRAP_PWD is
present, ELASTIC_CONTAINER must be set to true. Otherwise a new
bootstrap check will fail.

If ELASTIC_CONTAINER is set to true, the elastic user can be
authenticated with the BOOTSTRAP_PWD variable when its password
has not been explicitly set.

Original commit: elastic/x-pack-elasticsearch@78f53fd232
2017-06-28 12:48:49 -05:00
Lisa Cawley 2f1693c0fd [DOCS] Update ML open job API (elastic/x-pack-elasticsearch#1875)
Original commit: elastic/x-pack-elasticsearch@00a76d79ae
2017-06-28 10:27:05 -07:00
lcawley c42a0844bc [DOCS] Re-add include-xpack attribute
Original commit: elastic/x-pack-elasticsearch@8312e2856f
2017-06-28 10:15:28 -07:00
Deb Adair 90e3007228 [DOCS] Fixed link order.
Original commit: elastic/x-pack-elasticsearch@7aa7ce5621
2017-06-28 09:50:13 -07:00
Deb Adair 3b87e67ec4 [DOCS] Updated links to the ES Ref for the X-Pack ref.
Original commit: elastic/x-pack-elasticsearch@d47f276678
2017-06-28 09:43:04 -07:00