Commit Graph

61 Commits

Author SHA1 Message Date
Tanguy Leroux d3dff6659b Returns empty set of filters in MonitoringIndexNameResolver and fix tests
Original commit: elastic/x-pack-elasticsearch@e901347de9
2016-08-30 10:10:08 +02:00
Tanguy Leroux 203ad03d26 Use sets of String in XContentBuilders
With elastic/elasticsearchelastic/elasticsearch#19865 the XContentBuilder has changed to support both inclusive and exclusive filters and now uses Set<String> instead of arrays of Strings. This change updates the various places in x-plugins where string arrays were used.

Original commit: elastic/x-pack-elasticsearch@1f8d4485f4
2016-08-30 09:08:49 +02:00
Yannick Welsch cdc41f6082 Add recovery source to ShardRouting
Relates to elastic/elasticsearchelastic/elasticsearch#19516

Original commit: elastic/x-pack-elasticsearch@051fb25bfc
2016-08-27 15:19:49 +02:00
jaymode 334aa94946 security: simplify lookup of files inside the config dir
This commit simplifies the code used for resolving the files used for security to always
resolve against the `config` file. Elasticsearch no longer offers a way to disable the security
manager, so the files read by x-pack should not really be configurable and only exist in their
default locations since that is what can be read by the process.

As part of this, the documentation was updated to indicate that these files should always be in
the default location and the settings to change the locations have been removed. Also, a bug
was fixed in a few places where settings were still using `shield.` instead of `xpack.security.`.
Finally, some outdated and unused files were deleted from the repository.

Closes elastic/elasticsearch#305

Original commit: elastic/x-pack-elasticsearch@3884f080a0
2016-08-23 14:54:18 -04:00
Nik Everett 8ee190d741 Handle moving rest support
Original commit: elastic/x-pack-elasticsearch@99076b2936
2016-08-15 22:41:43 -04:00
Chris Earle af28d05515 Disabling local test while it awaits a fix
Original commit: elastic/x-pack-elasticsearch@3a8afb331f
2016-08-12 23:44:28 -04:00
Chris Earle e7ae3fa1f5 Give a longer timeout for test
Original commit: elastic/x-pack-elasticsearch@461d7eee8f
2016-08-12 17:18:44 -04:00
Chris Earle 3c9749b2b8 [Monitoring] Add Empty Ingest Pipeline to Future Proof Monitoring Documents
This adds an empty _ingest/pipeline named after the _current_ version of the Monitoring API (currently 2) from both exporters.

This allows us to eventually change the API version (to 3, 4, etc.) and go _back_ and modify the pipeline that exists. The modified pipeline would then "fix" the documents as best as possible and rename the index. As a result, different versions (starting with 5.0) will be able to report to the same monitoring cluster regardless of the running API version.

Note: This has no impact on stale data (e.g., the day before the upgrade) _and_ it implies that the monitoring cluster should always be updated first. A simple reindexing script can be supplied for old data, which can be done at the discretion of the user.

Original commit: elastic/x-pack-elasticsearch@45df5ee87b
2016-08-12 11:50:39 -04:00
jaymode a3e7536205 fix serialization of XPackFeatureSet.Usage
The XPackFeatureSet.Usage writeTo method failed to write the name to the
StreamOutput whereas the deserialization code expected it to be the first value
present in the StreamInput. This causes xpack usage requests made to nodes
other than the master to fail.

This change fixes the serialization and adds tests to all of the feature sets to ensure
that the usage is serialized properly.

Closes elastic/elasticsearch#3072

Original commit: elastic/x-pack-elasticsearch@eccab616ca
2016-08-12 07:48:55 -04:00
Lee Hinman 52f23afb4b Fix checkstyle violations in HttpExporter*.java
Two files had out-of-order @Nullable clauses and one had an invalid
license header

Original commit: elastic/x-pack-elasticsearch@8057871b18
2016-08-10 16:46:31 -06:00
Chris Earle c41550d36a [Monitoring] Allow HTTP Exporter to be given custom HTTP Headers for every request
This allows X-Pack Monitoring 5.0 to be given custom headers for any request.

Original commit: elastic/x-pack-elasticsearch@04050181bf
2016-08-10 16:33:32 -04:00
Ryan Ernst ce496938cd Consolidate settings for enabling xpack features
Currently there are a number of helper methods, all used at various
stages of initializing xpack, for reading "enabled" settings. Almost all
of these read the setting directly rather than use the new settings
infra. This change adds an XPackSettings class to contain these settings
constants, and adds all the enabled settings to it.

Note: While there is an existing MonitoringSettings, I think we do not
have enough settings (especially for all features) to warrant a separate
class per "feature" (eg graph would have nothing). We should decide
if/when this is necessary on a case by case basis, but I think the
enabled settings should all go here in XPackSettings.

Original commit: elastic/x-pack-elasticsearch@086152d5a5
2016-08-08 14:01:35 -07:00
Jason Tedor 7b7411f78d Remove mock Netty plugins
These mock Netty plugins were needed to assist in setting
sun.nio.ch.bugLevel. Yet, settings this property should no longer be
needed on JDK 8 and the missing privileged block when writing this
property can be fixed upstream.

Relates elastic/elasticsearch#3047

Original commit: elastic/x-pack-elasticsearch@e2aa4f6fb5
2016-08-05 14:58:50 -04:00
Boaz Leskes a2cca18c49 allow joining nodes, conflicting with existing nodes to elect a master (elastic/elasticsearch#2996)
this is a companion PR to elastic/elasticsearchelastic/elasticsearch#19743

Original commit: elastic/x-pack-elasticsearch@193c92d99a
2016-08-05 09:07:29 +02:00
Jason Tedor 4874d84f82 Merge branch 'master' into netty4_transport
* master: (25 commits)
  docs: remove unused file and add link to invalid role examples
  Remove interfaces for notification services
  Redirect to URL specified by next parameter, if it is set
  Fix fixture and tests so they pass again
  Update error message to be more actionable
  Switch to NamedWriteable pull based extension in xpack
  Fixing issue with infinite redirect loop
  Toggle display of security nav controls more dynamically
  Pass in xpackMainPlugin instead of xpackMainPlugin.info
  Wrap the return in a Promise
  Only unset the cookie if it is currently set
  Clarifying intent of code
  Updating tests fixtures + adding assertion for client cookie deletion
  If security is disabled, do not attempt to call the authenticate ES API
  Disambiguate between resolve function names
  Revert to not using xpackMainPlugin.info until the xpackMainPlugin is ready
  Redirect /login => / if security is disabled in ES
  Register/deregister security management items depending on whether there's an auth'd user
  Show/hide the username + logout button depending on whether there is an auth'd user
  If security is disabled, continue without auth + delete client cookie
  ...

Original commit: elastic/x-pack-elasticsearch@16b92a1a59
2016-08-03 09:18:42 -04:00
jaymode 55ccd27acf ensure the transport type is a security transport
Original commit: elastic/x-pack-elasticsearch@6c7e46e103
2016-08-03 07:23:55 -04:00
Jason Tedor dd181e3e13 Cleanup HTTP and transport security settings
After this commit, we have "security3" and "security4" as possible
transport and HTTP settings, we default to "security4" if it is not set,
and we randomize the selection in some of the integration tests.

Original commit: elastic/x-pack-elasticsearch@e56718911a
2016-08-03 00:23:58 -04:00
Ryan Ernst cdae14a5b9 Switch to NamedWriteable pull based extension in xpack
This is the xplugins side of elastic/elasticsearchelastic/elasticsearch#19764. It converts
the one use of registering custom NamedWriteable readers in xpack for
xpack feature sets to register them up front with the new pull based
registration.

Original commit: elastic/x-pack-elasticsearch@48e2020816
2016-08-02 16:00:06 -07:00
Jason Tedor 8fa06fbab7 Security Netty 4 HTTP server implementation
This commit is a work-in-progress commit on a Netty 4-based HTTP server
implementation.

Original commit: elastic/x-pack-elasticsearch@705a202574
2016-08-02 14:33:42 -04:00
Ryan Ernst 8514b1422e Move License into xpack
We currently have the license:base project, as well as license:licensor.
Xpack depends on license base. This change moves all the base license
code into xpack, and makes the license-tools (previously called
licensor) depenend on x-pack.

Original commit: elastic/x-pack-elasticsearch@7fd462ad6a
2016-07-28 00:16:18 -07:00
Chris Earle ac43e72aec [Monitoring] Change xpack.monitoring.collection.exporters to xpack.monitoring.exporters
Removes the exporters from the collection settings.

Original commit: elastic/x-pack-elasticsearch@e721e5cb6d
2016-07-27 14:05:36 -04:00
Ryan Ernst d53315b613 Merge branch 'master' into deguice15
Original commit: elastic/x-pack-elasticsearch@38f0a6f641
2016-07-26 15:07:20 -07:00
Ryan Ernst 059bf323c4 Cleanup: Move all license related classes into a single package
This change moves all license related code in xpack under
org.elasticsearch.license. Some classes become package private (more can
become package private later with further deguicing). It also adds
package level javadocs. Note, I did not move anything from the actual
licensing jar.  This should be cleaned up in a followup.

Original commit: elastic/x-pack-elasticsearch@2413c4f288
2016-07-26 13:26:29 -07:00
Ryan Ernst b02b30ee0a Internal: Remove guice from monitoring
This change removes guice from construction of monitoring classes.
Additionally, it removes disk threshold watermark and enabled setting
from node stats collector. These were not node stats, just cluster
settings. If we want to add back actual percentage of disk threshold
used, it should be in node stats directly.

Original commit: elastic/x-pack-elasticsearch@4cd49557cf
2016-07-26 11:25:22 -07:00
Ryan Ernst 0ae6e53173 Internal: Collapse Licensee per feature into single XPackLicenseState
The license code currently has a Licensee implementation per feature,
which is updated by the license service. This meant maintaining a
listener type feature for the license service, and having an abstract
listener and such. The licensee class also mixed in stuff only needed by
the license service (acknowledgement messages).

This change collapses all the methods from licensees into
XPackLicenseState. The naming was inconsistent across licensee
implementations, so here it is standardized on `is*Allowed()`. There are
also a number of tests which should be consolidated for testing the
license service but that is left for a future change. I also removed
collector tests that were testing license: that is better left for the
direct tests of the license state in XPackLicenseStateTests.

Original commit: elastic/x-pack-elasticsearch@734871e870
2016-07-23 16:55:04 -07:00
Ryan Ernst d734d483c5 Merge pull request elastic/elasticsearch#2902 from rjernst/license_state_removal
Internal: Removed LicenseState enum

Original commit: elastic/x-pack-elasticsearch@13abacbb78
2016-07-22 14:25:19 -07:00
javanna 9b73b26b7d Merge branch 'master' into feature/async_rest_client
Original commit: elastic/x-pack-elasticsearch@e58a8d9484
2016-07-22 22:22:14 +02:00
Ryan Ernst 7fcf05dcb1 Internal: Removed LicenseState enum
The LicenseState class exists to distinguish when a license is enabled,
vs being inside its grace period. However, the consumers of this state
do not care whether the license is in the grace period, they view that
and an active license as the same thing. The only part that cares about
the grace period is in the license service which logs a warning when a
license begins its grace period.

This change removes the LicenseState enum in favor of a simple boolean
indicating whether the license is active.

Original commit: elastic/x-pack-elasticsearch@5a90a0e3d4
2016-07-22 11:30:56 -07:00
Ryan Ernst a93a5fcd5b Rename LicensesService to LicenseService
We only have one license in 5.x. This change renames the service that
mantains the license state on each node to reflect that fact.

Original commit: elastic/x-pack-elasticsearch@bb241b30cb
2016-07-21 11:59:52 -07:00
javanna ca557af48c Merge branch 'master' into feature/async_rest_client
Original commit: elastic/x-pack-elasticsearch@693e281d0d
2016-07-19 16:29:50 +02:00
Chris Earle 1311935122 [Monitoring] Add Request Cache to returned stats for tests
Original commit: elastic/x-pack-elasticsearch@9bc34609c5
2016-07-15 12:51:43 -04:00
Areek Zillur 0db0e2f0c9 Implements cloud_internal license type
"cloud_internal" license type enables dynamically updating license operation mode via a config file.

When the installed license is "cloud_internal", the node level operation mode can be updated by writing
a `license_mode` file in the x-pack config directory (config/x-pack/license_mode). The file is expected
to have a string representing the desired license mode (e.g. "gold", "basic"). In case of a failure to
read a valid license mode from the `license_mode` file, the operation mode for "cloud_internal" license
defaults to PLATINUM.
This change also ensures that the correct operation mode is reported via the _xpack endpoint.

closes elastic/elasticsearch#2042

Original commit: elastic/x-pack-elasticsearch@6a2d788e45
2016-07-15 12:08:34 -04:00
Ryan Ernst f388ef01ed Merge branch 'master' into deguice2
Original commit: elastic/x-pack-elasticsearch@2d7264c2cf
2016-07-14 19:51:58 -07:00
Jason Tedor 496e112c5e Rename transport-netty to transport-netty-3
This commit renames the Netty 3 transport module from transport-netty to
transport-netty3. This is to make room for a Netty 4 transport module,
transport-netty4.

Relates elastic/elasticsearch#2827

Original commit: elastic/x-pack-elasticsearch@e6487cefa2
2016-07-14 22:03:29 -04:00
Jason Tedor 29387eab21 Monitoring index name resolver test awaits fix
This commit moves an awaits fix from
MonitoringIndexNameResolverTestCase#testResolver to
MonitoringIndexNameResolverTestCase#testSource as the previous commit
elastic/x-pack@98e76642ea marked the incorrect test as
awaits fix.

Original commit: elastic/x-pack-elasticsearch@62e9aaa628
2016-07-14 19:52:00 -04:00
Jason Tedor 3c0e83990b Monitoring index name resolver test awaits fix
This commit adds an awaits fix to
MonitoringIndexNameResolverTestCase#testResolver as awaits fix.

Original commit: elastic/x-pack-elasticsearch@586eb37f43
2016-07-14 19:46:05 -04:00
Chris Earle 4224d70986 [Monitoring UI] Use same Index Memory chart for Index page as Node page
This provides the same -- more useful -- index memory chart to the index page, instead of just "Lucene Memory", it now breaks down what it shows just like the Node page as as more generalized "Index Memory" chart. It also rolls Fielddata into the Index Memory. With Fielddata now in the Index Memory chart, I decided to add the Segment Count chart to the Index page.

Original commit: elastic/x-pack-elasticsearch@4e3490ce9c
2016-07-14 13:26:43 -04:00
Ryan Ernst f82fa65d7d Internal: Remove guice construction of most license classes
This change removes some of the complexity around licensing classes in
xpack. It removes the interfaces for registration and management so the
remaining LicensesService class is the thing that components wanting to
interact with the license should use. It also removes complexity around
the Licensee interface, removing generics and the registration at
construction time, as well as making the licensees no longer have a
lifecycle. There is still more to be done with simplification of license
classes construction, but this is a step towards a simpler world.

Original commit: elastic/x-pack-elasticsearch@5307d67b5b
2016-07-13 14:23:23 -07:00
Karel Minarik 63add2c959 Monitoring: Added `timeout: 60s` to the setup section of "Bulk indexing of monitoring data" test
When the monitoring tests are run in isolation, they succeed. However, when the whole suite of
REST tests is being run at the same time, the "Bulk indexing of monitoring data" intermittently
fails with a timeout. Therefore, a timeout of 60 seconds has been added.

Closes elastic/elasticsearch#2809

Original commit: elastic/x-pack-elasticsearch@d11dc7a2be
2016-07-13 17:40:20 +02:00
javanna c86c433aab Rest client: move to async client under the hood
Some configuration changes are needed due to the move to the async http client

Original commit: elastic/x-pack-elasticsearch@2f28dec0a0
2016-07-12 18:25:27 +02:00
Simon Willnauer 6522f49d83 Add MockNettyPlugin to sidestep permissions assertion in pseudo integ tests
Original commit: elastic/x-pack-elasticsearch@9e7f6532fb
2016-07-12 17:56:06 +02:00
Simon Willnauer 9204920d9a fix tests to use the transport-netty plugin
Original commit: elastic/x-pack-elasticsearch@65c0cd0cf2
2016-07-12 14:35:07 +02:00
Ryan Ernst 7438177313 Internal: Create CryptoService without guice
This change creates the CryptoService during xpack plugin construction,
and also collapses the only implementation of CryptoService into a
class instead of an interface.

Original commit: elastic/x-pack-elasticsearch@7f00a5d6ef
2016-07-11 12:18:39 -07:00
Chris Earle 16b3056b1f [Monitoring] Change xpack.monitoring.agent to xpack.monitoring.collection
This changes the xpack.monitoring.agent.* settings so that they fall under xpack.monitoring.collection.*.

- This also changes Kibana's equivalent data_collection to collection.

Original commit: elastic/x-pack-elasticsearch@a69bc18055
2016-07-11 12:03:01 -04:00
Areek Zillur 005304afa9 Merge branch 'master' into simplify_license_service_scheduling
Original commit: elastic/x-pack-elasticsearch@91258f1a3c
2016-07-07 17:54:46 -04:00
Nik Everett 87e84bf9e1 Handle core's cleanup of terms aggregation
Original commit: elastic/x-pack-elasticsearch@c764d11867
2016-07-07 17:06:07 -04:00
Chris Earle dbe189b064 [Monitoring] Decouple Kibana Stats Publishing from Application Version
This changes the way that Kibana (and future applications) send their monitoring stats to Elasticsearch.

Instead of sending their payloads with the System ID (e.g., "kibana") and System Version (e.g., "5.0.0-alpha4"), it now expects the System ID and System _API_ Version (e.g., "2"). This means a few things:

- Future releases are automatically compatible with previous releases as long as the API version doesn't change.
- Users don't have to update Kibana at the exact same time as their cluster (which technically means rolling updates were temporarily blockers of Kibana monitoring before).
- We can accept old API versions (if we need to make a breaking change) and automatically up-convert them to the latest API version. (We are in full control of how far back we choose to accept)

In general, this change implies that users should be updating their Monitoring cluster before their _monitored_ cluster(s) to get the best opportunity of monitoring backwards compatibility. That way if any API change does occur, then it can up-convert as needed. Then, any ES node should be updated, and only then should Kibana be updated. This is not required in any way, but it will give the smoothest experience.

Original commit: elastic/x-pack-elasticsearch@d3c24936e1
2016-07-06 11:24:22 -04:00
Jason Tedor 3577389f95 Rename writeThrowable to writeException
This commit is a response to upstream change
elastic/elasticsearch@96f283c195.

Original commit: elastic/x-pack-elasticsearch@2e485d4e76
2016-07-05 14:46:50 -04:00
Areek Zillur d88e1ddb27 Merge branch 'master' into simplify_license_service_scheduling
Original commit: elastic/x-pack-elasticsearch@faa20465e4
2016-07-05 10:51:40 -04:00
Boaz Leskes 8cc49b5b30 Remove DummyTransportAddress (elastic/elasticsearch#2048)
this is a partner PR to elastic/elasticsearchelastic/elasticsearch#17811

Original commit: elastic/x-pack-elasticsearch@1db11e4340
2016-07-04 21:09:53 +02:00