Commit Graph

462 Commits

Author SHA1 Message Date
Alexander Reelsen 6d0a2f642a Watcher: HttpResponse serialization may not contain dots in field names
The HTTP response toXContent() method contains the http response headers, which
are used as field names in Elasticsearch in the watch history.
These can contain dots, like `es.index` being returned when Elasticsearch
encounters an exception - which results in an index error.

This patch changes the dots to an underscore when calling toXContent()

Closes elastic/elasticsearch#1803

Original commit: elastic/x-pack-elasticsearch@e4070f8b70
2016-04-13 15:07:22 +02:00
Alexander Reelsen 847287278b Tests: Adapting to Version changes in core
Original commit: elastic/x-pack-elasticsearch@89e9cf427d
2016-04-13 11:43:54 +02:00
Daniel Mitterdorfer 3fd3adef4c Limit request size on HTTP level
With this commit we limit the size of all in-flight requests on
HTTP level. The size is guarded by the same circuit breaker that
is also used on transport level. Similarly, the size that is used
is HTTP content length.

Relates elastic/elasticsearchelastic/elasticsearch#16011

Original commit: elastic/x-pack-elasticsearch@318b7a4a8a
2016-04-13 10:39:49 +02:00
Daniel Mitterdorfer 0d0e2b432c Limit request size on transport level
With this commit we limit the size of all in-flight requests on
transport level. The size is guarded by a circuit breaker and is
based on the content size of each request.

By default we use 100% of available heap meaning that the parent
circuit breaker will limit the maximum available size. This value
can be changed by adjusting the setting

network.breaker.inflight_requests.limit

Relates elastic/elasticsearchelastic/elasticsearch#16011

Original commit: elastic/x-pack-elasticsearch@d1c43fe8d9
2016-04-13 10:39:04 +02:00
Nik Everett 120e13148b Handle core search refactoring
Original commit: elastic/x-pack-elasticsearch@fb512063ca
2016-04-12 15:24:19 -04:00
Alexander Reelsen 61fdd0ac3c Fix compilation error from core change
Relates 2c487110b2

Original commit: elastic/x-pack-elasticsearch@b3661a5c3e
2016-04-12 18:42:28 +02:00
Alexander Reelsen a1f7fff901 Watcher: Cut settings over to xpack.watcher (elastic/elasticsearch#1909)
This cuts over all settings from `watcher.` to `xpack.watcher` as
part of the settings cleanup for 5.0.

Relates elastic/elasticsearch#1441

Original commit: elastic/x-pack-elasticsearch@c82483bf25
2016-04-12 10:34:23 +02:00
Adrien Grand 26e1535eee Fix compilation as a result of elastic/elasticsearchelastic/elasticsearch#16268.
Original commit: elastic/x-pack-elasticsearch@4a334d7f7d
2016-04-11 18:06:48 +02:00
Adrien Grand 5b57727b34 Replace usage of settingsBuilder with just builder.
Original commit: elastic/x-pack-elasticsearch@fe038bbc49
2016-04-08 18:09:02 +02:00
Tanguy Leroux 703a88f95f Monitoring: Fix MarvelTemplateUtilsTests.java on Windows (attempt elastic/elasticsearch#2)
Original commit: elastic/x-pack-elasticsearch@10df1b26b9
2016-04-08 15:56:39 +02:00
Tanguy Leroux bc744e27c3 Monitoring: Fix MarvelTemplateUtilsTests.java on Windows
This is due to line ending differences.

Original commit: elastic/x-pack-elasticsearch@a5327cd3e6
2016-04-08 14:24:02 +02:00
Tanguy Leroux 98fc26c614 Monitoring: Manage multiple index templates
Original commit: elastic/x-pack-elasticsearch@fef9dcc5d1
2016-04-08 11:44:03 +02:00
Nik Everett 29263637c0 React to registration changes in core
Original commit: elastic/x-pack-elasticsearch@f5961dc410
2016-04-07 18:48:00 -04:00
Chris Earle daa875db11 Remove hostname from NetworkAddress.format (x-plugins side)
This removes the old usage of NetworkAddress.formatAddress in favor of the updated version, which is just
the method renamed to NetworkAddress.format (replacing the old version of that method).

There is no impact to x-plugins beyond making the build work because all places were currently using that
method variant already.

Original commit: elastic/x-pack-elasticsearch@05f0dcfa90
2016-04-07 17:29:14 -04:00
jaymode 52b6fc54b8 test: explicitly initialize anonymous user in ReservedRealmTests
Original commit: elastic/x-pack-elasticsearch@46ce5c03a1
2016-04-07 10:54:47 -04:00
jaymode 8049a82953 security: add support for main action
This commit adds support for the change in elasticsearch where the `/` rest
endpoint now delegates to an action and can be authorized.

Original commit: elastic/x-pack-elasticsearch@8ef38ce50f
2016-04-07 09:25:21 -04:00
jaymode b56e2f3bca test: reset anonymous after test to avoid messing with other tests
Closes elastic/elasticsearch#1956

Original commit: elastic/x-pack-elasticsearch@9b57d295c8
2016-04-07 06:12:02 -04:00
jaymode 931c67b49c security: add type argument to fix compile
Original commit: elastic/x-pack-elasticsearch@81acbd2e29
2016-04-06 19:09:29 -04:00
jaymode d08446e221 security: add reserved roles and users
This commit adds reserved or built-in user and role support to x-pack. The reserved roles
cannot be modified by users. The reserved users also cannot be modified with the exception
of changing the password for a user.

In order to change the password for a user, a new API has been added. This API only supports
changing passwords for native and reserved users.

To support allowing a user to change their own password, a default role has been added to grant
access. This default role only grants access to user operations that pertain to the user that is
being authorized. In other words, the default role grants `joe` the ability to change their own password
but does not allow them to change the password of a different user.

Additionally, the authenticate API was made a transport action and is granted by the default role.

Closes elastic/elasticsearch#1727
Closes elastic/elasticsearch#1185
Closes elastic/elasticsearch#1158

Original commit: elastic/x-pack-elasticsearch@1a6689d90f
2016-04-06 18:23:18 -04:00
jaymode f888082ce6 security: remove the use of shield in settings
This commit migrates all of the `shield.` settings to `xpack.security.` and makes changes to
use the new Settings infrastructure in core.

As a cleanup, this commit also renames Shield to Security since this class is only in master
and will not affect 2.x.

See elastic/elasticsearch#1441

Original commit: elastic/x-pack-elasticsearch@a5a9798b1b
2016-04-06 14:00:46 -04:00
jaymode 9031cee432 security: allow indices monitor actions to access the security index
This commit allows authorized users to monitor the security index. This fixes an issue
with the _cat/indices api, which resolves the concrete indices using the cluster state
and then makes a indices stats request. Without this change, the api fails with an
authorization exception because it is specifically requesting the security index and the
user is not the internal user.

Closes elastic/elasticsearch#1895

Original commit: elastic/x-pack-elasticsearch@070a389833
2016-04-06 12:59:15 -04:00
Tanguy Leroux 3c65f38fbe Monitoring: Update exporter & bulk in ExportersTests
Since elastic/elasticsearch#1832 exporters are created once, but the inner exporting bulks must be instanciated for each export. The CountingExporter and CountingBulk have not been updated to reflect this change.

Original commit: elastic/x-pack-elasticsearch@bbbde22363
2016-04-06 11:35:39 +02:00
Martijn van Groningen 1e3c56ce97 test: wait only for the index test_1
(there are other indices too, montoring indices and waiting for green fails there because these indices have replica shards)

Original commit: elastic/x-pack-elasticsearch@63dd3e6ebb
2016-04-06 10:52:50 +02:00
Tanguy Leroux 4df6f0f701 Monitoring: Change ExportBulk so that it has states
Original commit: elastic/x-pack-elasticsearch@8dc55dc0d2
2016-04-06 10:21:55 +02:00
Tanguy Leroux b65787f7dd Monitoring: Add segments stats to Index Stats
Original commit: elastic/x-pack-elasticsearch@e42dab0971
2016-04-06 10:05:40 +02:00
javanna 0a9b72233e Adapt to "Switch to ParseField for query names"
A ParseField object is now required to register queries against the SearchModule rather than the QueryParser#names method. ParseField handles camel case automatically. Also this allows us to log deprecation warnings (or fail in strict mode) when deprecated names are used for queries (e.g. "in", "mlt", "fuzzy_match" etc.)

Original commit: elastic/x-pack-elasticsearch@b0146e6e3d
2016-04-05 15:39:22 +02:00
Alexander Reelsen 9a5e60b58f Watcher: Add SMTP default timeouts
This adds default timeouts to the SMTP configuration to prevent infinite timeouts, that can lead to stuck watches.
This also requires to use time values instead of just milliseconds.

Closes elastic/elasticsearch#1830

Original commit: elastic/x-pack-elasticsearch@c886da7bff
2016-04-05 14:28:15 +02:00
Alexander Reelsen 366498eca4 Tests: Fix systemkeytool tests
One test was missing the check if posix permissions are supported by the file system.
As it does not make sense to not run 50% of the tests in 50% of the cases, the
logic to configure which capabilities a in-memory FS should have has been moved
into each test.

Original commit: elastic/x-pack-elasticsearch@59a32ea26d
2016-04-05 14:08:03 +02:00
Alexander Reelsen 323f80216d Security: Fix systemkey CLI tool
When called without arguments, systemkey tool returned with an AIOOE.
This fixes the issue, but also ports over the tests to jimfs, so they
can actually run, as the security manager is always enabled and thus the
tests never ran before.

Closes elastic/elasticsearch#1926

Original commit: elastic/x-pack-elasticsearch@887b681607
2016-04-05 11:46:20 +02:00
Chris Earle aa9f516655 Fix deserializing license response
Original commit: elastic/x-pack-elasticsearch@dae5e6f545
2016-04-04 18:45:15 -04:00
Tanguy Leroux 192e0cd582 Cancel cleaner future in doClose() rather than doStop()
Original commit: elastic/x-pack-elasticsearch@39ca253b31
2016-04-04 19:11:12 +02:00
Tanguy Leroux 3672e06343 Merge pull request elastic/elasticsearch#1902 from tlrx/catch-rejected-exception
Monitoring: Catch EsRejectedExecutionException when rescheduling the cleaner

Original commit: elastic/x-pack-elasticsearch@6ba20d5b8a
2016-04-04 09:24:14 -07:00
Tanguy Leroux 2ae6dec8e1 Rename RestExecuteWatchActionTest to RestExecuteWatchActionTests
Original commit: elastic/x-pack-elasticsearch@c9d8de10b2
2016-04-04 18:22:28 +02:00
Tanguy Leroux 115b037f06 Monitoring: Catch EsRejectedExecutionException when rescheduling the cleaner
closes elastic/elasticsearch#1900

Original commit: elastic/x-pack-elasticsearch@87efb135b4
2016-04-04 18:19:08 +02:00
Tanguy Leroux 652b69ad7f Monitoring: Ship segment stats with node stats
This commit adds all the following segments stats to the current node_stats document:
 - memory_in_bytes
- terms_memory_in_bytes
- stored_fields_memory_in_bytes
- term_vectors_memory_in_bytes
- norms_memory_in_bytes
- doc_values_memory_in_bytes
- index_writer_memory_in_bytes
- version_map_memory_in_bytes
- fixed_bit_set_memory_in_bytes

Original commit: elastic/x-pack-elasticsearch@ea4b8034ba
2016-04-04 18:13:24 +02:00
Alexander Reelsen 25f06bb5c1 Build: Fix packaging to not include test in artifact
A rest test was accidentally moved into the src/main/plugin-metadata
directory, which resulted the test being put into the plugin distribution
zip.

Closes elastic/elasticsearch#1907

Original commit: elastic/x-pack-elasticsearch@fbdf62b1d8
2016-04-04 17:10:07 +02:00
jaymode 4036ce97c1 shield: do not use ThreadPool#scheduleWithFixedDelay for pollers
This commit makes the user and roles poller use a self rescheduling runnable to schedule the
next run of the poller rather than using scheduleWithFixedDelay. This is done because the
pollers perform blocking I/O operations and everything using that thread pool method runs on
the schedule thread and because of this, in certain situations this can lead to a deadlock which
will prevent the cluster from forming.

Original commit: elastic/x-pack-elasticsearch@9fd0748c8c
2016-04-01 21:25:16 -04:00
Chris Earle 3126fcb856 Improved tests with better error message
Original commit: elastic/x-pack-elasticsearch@cb79988dc3
2016-04-01 14:20:03 -04:00
Chris Earle 05811c5167 Adding Locale.ROOT to String.format for forbidden API usage
Original commit: elastic/x-pack-elasticsearch@4d78705e12
2016-04-01 13:10:25 -04:00
Chris Earle 9f41a99e37 Modifying Monitoring cleanup acknowledgement message.
Original commit: elastic/x-pack-elasticsearch@1c5e1a3175
2016-04-01 12:49:05 -04:00
Chris Earle 86ed96b83b Adding support for STANDARD license
Original commit: elastic/x-pack-elasticsearch@1671d8ade3
2016-04-01 12:49:05 -04:00
Chris Earle 55b9569f7b Removing isPaid, allFeaturesEnabled, and isActive methods from enums.
Original commit: elastic/x-pack-elasticsearch@8b8c7792c7
2016-04-01 12:49:05 -04:00
Chris Earle 5e81beabf9 Simplifying License Checks
Too many places are checking for enumerations when they're really more interested in a "higher" level of
information. This will help with the forthcoming addition of the STANDARD operation mode as well.

Original commit: elastic/x-pack-elasticsearch@2799c27e19
2016-04-01 12:49:05 -04:00
jaymode d6cab8b9f1 security: read correct file when listing users
Original commit: elastic/x-pack-elasticsearch@dca906abba
2016-04-01 06:30:34 -04:00
Tanguy Leroux c9392183d2 Monitoring: Add smoke test for Monitoring with Security
Original commit: elastic/x-pack-elasticsearch@9dc800ebcc
2016-04-01 11:17:39 +02:00
Tanguy Leroux 1d72eb2b61 Monitoring: Check for source_node only for assigned shard in test
Original commit: elastic/x-pack-elasticsearch@f0d5bccecd
2016-04-01 10:34:40 +02:00
Tanguy Leroux 2c1dbf3eb6 Monitoring: Clean log messages in exporters
This commit adds the node name and the exporter name as log message prefixes.

Original commit: elastic/x-pack-elasticsearch@085b2ecf3c
2016-04-01 09:48:22 +02:00
Chris Earle 6c8ec7be28 Removing test logger now that fix is in
Original commit: elastic/x-pack-elasticsearch@8d80b59ddd
2016-03-31 12:53:20 -04:00
Tanguy Leroux 8a15a17442 Monitoring: Simplify bulk REST test
Original commit: elastic/x-pack-elasticsearch@0a02d3f3be
2016-03-31 17:49:27 +02:00
Tanguy Leroux 4007ff44b7 Monitoring: Fix synchronization in Exporters
This commit fixes an issue in synchronization in Exporters class. The export() method is synchronized and when used with LocalExport can provoke a deadlock. LocalExporter exports data locally using bulk requests that can trigger cluster state updates for mapping updates. If a exporters settings update sneaks in, the settings update waits for the export to terminate but the export waits for the settings to be updated... and boom.

This commit removes the synchronized and refactor Exporters/LocalExporter to use state and dedicated instance of LocalBulk for each export so that synchronizing methods is not necessary anymore.

It also lower down some random settings in MonitoringBulkTests because the previous settings almost always fill the bulk thread pool.

closes elastic/elasticsearch#1769

Original commit: elastic/x-pack-elasticsearch@f50c916f8b
2016-03-31 13:47:53 +02:00