128 Commits

Author SHA1 Message Date
David Turner
77789a733d Merge branch 'master' into 2018-11-08-merge-master 2018-11-08 13:38:18 +00:00
Alpar Torok
8a85b2eada
Remove build qualifier from server's Version (#35172)
With this change, `Version` no longer carries information about the qualifier,
we still need a way to show the "display version" that does have both
qualifier and snapshot. This is now stored  by the build and red from `META-INF`.
2018-11-07 14:01:05 +02:00
Shaunak Kashyap
00e66bab36
[Elasticsearch Monitoring] Collect only display_name (for now) (#35265)
This is a forward port of some of the changes made in #8445, specifically the change mentioned in https://github.com/elastic/elasticsearch/pull/34023#issuecomment-433212636.

Currently, in master, the `cluster_stats` collector collects _all_ cluster metadata and indexes it into `.monitoring-es-*`. However, per the discussion linked to above, we decided to collect _only_ the `display_name` cluster metadata setting for now. This PR makes this change.
2018-11-05 16:27:05 -08:00
Alexander Reelsen
409050e8de
Refactor: Remove settings from transport action CTOR (#35208)
As settings are not used in the transport action constructor, this
removes the passing of the settings in all the transport actions.
2018-11-05 13:08:18 +01:00
Nik Everett
e28509fbfe
Core: Less settings to AbstractComponent (#35140)
Stop passing `Settings` to `AbstractComponent`'s ctor. This allows us to
stop passing around `Settings` in a *ton* of places. While this change
touches many files, it touches them all in fairly small, mechanical
ways, doing a few things per file:
1. Drop the `super(settings);` line on everything that extends
`AbstractComponent`.
2. Drop the `settings` argument to the ctor if it is no longer used.
3. If the file doesn't use `logger` then drop `extends
AbstractComponent` from it.
4. Clean up all compilation failure caused by the `settings` removal
and drop any now unused `settings` isntances and method arguments.

I've intentionally *not* removed the `settings` argument from a few
files:
1. TransportAction
2. AbstractLifecycleComponent
3. BaseRestHandler

These files don't *need* `settings` either, but this change is large
enough as is.

Relates to #34488
2018-10-31 21:23:20 -04:00
Nik Everett
086ada4c08
Core: Drop settings member from AbstractComponent (#35083)
Drops the `Settings` member from `AbstractComponent`, moving it from the
base class on to the classes that use it. For the most part this is a
mechanical change that doesn't drop `Settings` accesses. The one
exception to this is naming threads where it switches from an invocation
that passes `Settings` and extracts the node name to one that explicitly
passes the node name.

This change doesn't drop the `Settings` argument from
`AbstractComponent`'s ctor because this change is big enough as is.
We'll do that in a follow up change.
2018-10-30 16:10:38 -04:00
Andrey Ershov
97f74c5a38 Merge branch 'master' into 'zen2'
Conflicts during the merge:
1. >=140 chars line length fixed for a lot of project files and warnings
for those files are no longer suppressed
2. Node name is removed from AbstractComponent, it’s no longer taken
from settings, but is explicitly passed as constructor argument and
there were quite a few new classes on zen2 branch that require this
change
3. TransportResponseHandler interface changed (new method added) and
Zen2 makes a lot of subclasses in tests
4. Deprecated way of obtaining logger was changed
2018-10-30 14:39:48 +03:00
Pratik Sanglikar
f1135ef0ce Core: Replace deprecated Loggers calls with LogManager. (#34691)
Replace deprecated Loggers calls with LogManager.

Relates to #32174
2018-10-29 15:52:30 -04:00
Jason Tedor
26f5c509af
Fix CCR API specification (#34963)
This commit fixes two issues with the CCR API specification:
 - remove the CCR stats endpoint, it is not currently implemented
 - fix the documentation links
2018-10-29 09:37:13 -04:00
Martijn van Groningen
1801518527
[CCR] Refactor stats APIs (#34912)
* Changed the auto follow stats to also include follow stats.
* Renamed the auto follow stats api to stats api and changed its url path
  from `/_ccr/auto_follow/stats` `/_ccr/stats`.
* Removed `/_ccr/stats` url path for the follow stats api, which makes
  the index parameter a required parameter.
* Fixed docs.
2018-10-29 07:45:27 +01:00
Alpar Torok
795d57b4f9
Auto configure all test tasks (#34666)
With this change, we apply the common test config automatically to all
newly created tasks instead of opting in specifically.

For plugin authors using the plugin externally this means that the
configuration will be applied to their RandomizedTestingTasks as well.

The purpose of the task is to simplify setup and make it easier to
change projects that use the `test` task but actually run integration
tests to use a task called `integTest` for clarity, but also because
we may want to configure and run them differently.
E.x. using different levels of concurrency.
2018-10-24 16:05:50 +03:00
David Turner
950ca3adda Merge branch 'master' into zen2 2018-10-16 14:41:14 +01:00
Jason Tedor
7478167d60
Rename CCR stats implementation (#34300)
In the CCR docs we want to refer to the endpoint that returns following
stats as the follow stats API. This commit renames the internal
implementation of this endpoint to reflect this usage.
2018-10-05 06:25:24 -04:00
David Turner
a9eae1d068 Merge branch 'master' into zen2 2018-10-03 08:36:34 +01:00
Shaunak Kashyap
3eed873dde
Updating test assertion (#34040) 2018-10-02 03:19:12 -07:00
Martijn van Groningen
17b3b97899
Fixed CCR stats api serialization issues and (#33983)
always use `IndicesOptions.strictExpand()` for indices options.

The follow index may be closed and we still want to get stats from
shard follow task and the whether the provided index name matches with
follow index name is checked when locating the task itself in the ccr
stats transport action.
2018-09-28 07:45:32 +02:00
Martijn van Groningen
793b2a94b4
[CCR] Expose auto follow stats to monitoring (#33886) 2018-09-25 07:19:46 +02:00
Shaunak Kashyap
9b86e9aed8
[Monitoring] Add cluster metadata to cluster_stats docs (#33860)
* WIP

* Adding cluster metadata to cluster stats monitoring doc

* Fixing checkstyle errors

* Adding missing license header

* Updating tests

* Getting cluster settings from cluster state

* Removing more unnecessary changes

* Adding cluster metadata settings to cluster_stats docs

* Updating test to include cluster metadata

* Fixing checkstyle

* Guarding against NPE

* Updating test fixture
2018-09-24 18:52:26 -07:00
David Turner
0b4a6ae97c Merge commit '3522b9084b611c89ec4f06c1863542883840ed0e' into zen2 2018-09-20 15:17:47 +01:00
Shaunak Kashyap
d78966c4c7
Fixing assertions in integration test (#33833) 2018-09-19 04:49:54 -07:00
Yannick Welsch
10009434bf Merge remote-tracking branch 'elastic/master' into zen2 2018-09-19 11:18:01 +02:00
Shaunak Kashyap
2aba52de8f
Implement xpack.monitoring.elasticsearch.collection.enabled setting (#33474)
* Implement xpack.monitoring.elasticsearch.collection.enabled setting

* Fixing line lengths

* Updating constructor calls in test

* Removing unused import

* Fixing line lengths in test classes

* Make monitoringService.isElasticsearchCollectionEnabled() return true for tests

* Remove wrong expectation

* Adding unit tests for new flag to be false

* Fixing line wrapping/indentation for better readability

* Adding docs

* Fixing logic in ClusterStatsCollector::shouldCollect

* Rebasing with master and resolving conflicts

* Simplifying implementation by gating scheduling

* Doc fixes / improvements

* Making methods package private

* Fixing wording

* Fixing method access
2018-09-17 18:33:43 -07:00
Shaunak Kashyap
a95226bdae
[Monitoring] Removing unused version.* fields (#33584)
This PR removes fields that are not actually used by the Monitoring UI. This will greatly simplify the eventual migration to using Metricbeat for monitoring Elasticsearch (see https://github.com/elastic/beats/pull/8260#discussion_r215885868 for more context and discussion around removing these fields from ES collection).
2018-09-17 18:29:30 -07:00
David Turner
7f6c13037c Mark testMonitoringService as @AwaitsFix 2018-09-17 16:23:55 +01:00
David Turner
c79fbea923
[Zen2] Implement basic cluster formation (#33668)
This PR integrates the following pieces of machinery in the Coordinator:

- discovery
- pre-voting
- randomised election scheduling
- joining (of a new master)
- publication of cluster state updates

Together, these things are everything needed to form a cluster. We therefore
also add the start of a test suite that allows us to assert higher-level
properties of the interactions between all these pieces of machinery, with as
little fake behaviour as possible. We assert one such property: "a cluster
successfully forms".
2018-09-17 15:00:30 +02:00
Yannick Welsch
01b3be917a Merge remote-tracking branch 'elastic/master' into zen2 2018-09-17 09:59:37 +02:00
Jason Tedor
f037edb8e3
Move CCR monitoring tests to ccr sub-project (#33730)
This commit moves the CCR monitoring tests from the monitoring
sub-project to the ccr sub-project.
2018-09-15 09:16:33 -04:00
Jason Tedor
2282150f34
Expose retries for CCR fetch failures (#33694)
This commit exposes the number of times that a fetch has been tried to
the CCR stats endpoint, and to CCR monitoring.
2018-09-14 08:52:46 -04:00
David Turner
31e8781eaa Merge branch 'master' into zen2 2018-09-14 14:28:28 +02:00
Martijn van Groningen
b9d0c8f25c
[CCR] Add monitoring mapping verification test (#33662)
Added test that verifies that all fields in ShardFollowNodeTaskStatus are
mapped in monitoring-es.json
2018-09-13 20:47:24 +02:00
Shaunak Kashyap
b2f78e5a47
[CCR Monitoring] Only collect stats for specified indices (#33646)
Follow up to #33617. Relates to #30086.

As with all other per-index Monitoring collectors, the `CcrStatsCollector` should only collect stats for the indices the user wants to monitor. This list is controlled by the `xpack.monitoring.collection.indices` setting and defaults to all indices.
2018-09-12 15:56:13 -07:00
Jason Tedor
eb715d5290
Add follower index to CCR monitoring and status (#33645)
This commit adds the follower index to CCR shard follow task status, and
to monitoring.
2018-09-12 17:35:06 -04:00
Jason Tedor
23f12e42c1
Expose CCR stats to monitoring (#33617)
This commit exposes the CCR stats endpoint to monitoring collection.

Co-authored-by: Martijn van Groningen <martijn.v.groningen@gmail.com>
2018-09-12 09:13:07 -04:00
David Turner
a2cd8f731e Merge branch 'master' into zen2 2018-09-11 09:38:10 +02:00
Nhat Nguyen
94e4cb64c2
Bootstrap a new history_uuid when force allocating a stale primary (#33432)
This commit ensures that we bootstrap a new history_uuid when force
allocating a stale primary. A stale primary should never be the source
of an operation-based recovery to another shard which exists before the
forced-allocation.

Closes #26712
2018-09-08 19:29:31 -04:00
Jim Ferenczi
79cd6385fe
Collapse package structure for metrics aggs (#33463)
This change collapses all metrics aggregations classes into a single package `org.elasticsearch.aggregations.metrics`.
It also restricts the visibility of some classes (aggregators and factories) that should not be used outside of the package.

Relates #22868
2018-09-07 10:58:06 +02:00
Adrien Grand
7eef7f441b Add info about the running threads in case of failure.
Relates #29880
2018-09-05 18:41:26 +02:00
Armin Braun
46774098d9
INGEST: Implement Drop Processor (#32278)
* INGEST: Implement Drop Processor
* Adjust Processor API
* Implement Drop Processor
* Closes #23726
2018-09-05 14:25:29 +02:00
Nik Everett
6c8f568808
Switch remaining LLREST usage to new style Requests (#33171)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. In a
long series of PRs I've changed all of the old style requests that I
could find with `grep`. In this PR I change all requests that I could
find by *removing* the deprecated methods. Since this is a non-trivial
change I do not include actually removing the deprecated requests. I'll
do that in a follow up. But this should be the last set of usage
removals before the actual deprecated method removal. Yay!
2018-08-28 14:20:14 -04:00
Nik Everett
1e9144d8e6
Switch remaining x-pack tests to new style Requests (#33108)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. This
changes all calls in the `x-pack/qa/saml-idp-tests` and
`x-pack/qa/security-setup-password-tests` projects to use the new
versions.
2018-08-24 16:39:08 -04:00
Jim Ferenczi
f4e9729d64
Remove unsupported Version.V_5_* (#32937)
This change removes the es 5x version constants and their usages.
2018-08-24 09:51:21 +02:00
Nik Everett
2c81d7f77e
Build: Rework shadow plugin configuration (#32409)
This reworks how we configure the `shadow` plugin in the build. The major
change is that we no longer bundle dependencies in the `compile` configuration,
instead we bundle dependencies in the new `bundle` configuration. This feels
more right because it is a little more "opt in" rather than "opt out" and the
name of the `bundle` configuration is a little more obvious.

As an neat side effect of this, the `runtimeElements` configuration used when
one project depends on another now contains exactly the dependencies needed
to run the project so you no longer need to reference projects that use the
shadow plugin like this:

```
testCompile project(path: ':client:rest-high-level', configuration: 'shadow')
```

You can instead use the much more normal:

```
testCompile "org.elasticsearch.client:elasticsearch-rest-high-level-client:${version}"
```
2018-08-21 20:03:28 -04:00
Nik Everett
f311680176 Monitoring: Clean up MonitoringIT
We recently reenabled MonitoringIT to hunt down #29880 but some of its
assertions were out of date. This updates the assertions.
2018-08-21 16:04:16 -04:00
Adrien Grand
5b446b81ef Reenable MonitoringIT#testMonitoringService.
When discussing this test, it made little sense that testMonitoringService
would fail but not testMonitoringBulk given their similarity. So we argeed to
enable it again.

Relates #29880
2018-08-21 16:53:59 +02:00
David Turner
f317562c82 Merge branch 'master' into zen2 2018-08-20 08:33:55 +01:00
Jason Tedor
0876630b30
Guard against null in email admin watches (#32923)
The Kibana settings docs that these watches rely on can sometimes
contain no xpack settings. When this is the case, we will end up with a
null pointer exception in the script. We need to guard against in these
scripts so this commit does that.
2018-08-16 16:25:34 -04:00
Lee Hinman
48281ac5bc
Use generic AcknowledgedResponse instead of extended classes (#32859)
This removes custom Response classes that extend `AcknowledgedResponse` and do nothing, these classes are not needed and we can directly use the non-abstract super-class instead.

While this appears to be a large PR, no code has actually changed, only class names have been changed and entire classes removed.
2018-08-15 08:06:14 -06:00
Albert Zaharovits
1dcf80795a
[TEST] Certificate NONE not allowed in FIPS JVM (#32753)
Certificate NONE not allowed when running in a FIPS JVM
2018-08-10 19:37:05 +03:00
Lee Hinman
7af28c48c3
Switch WritePipelineResponse to AcknowledgedResponse (#32722)
We previously discussed moving the classes extending `AcknowledgedResponse` to
simply use `AcknowledgedResponse`, making the class non-abstract.

This moves the first class to do this, removing `WritePipelineResponse` in the
process.

If we like the way this looks, I will switch the remaining classes over to using
`AcknowledgedResponse`.
2018-08-08 16:21:58 -06:00
Albert Zaharovits
1f50950099 Add @AwaitsFix for #32673 2018-08-07 13:22:12 +03:00