Commit Graph

631 Commits

Author SHA1 Message Date
Tanguy Leroux 3672e06343 Merge pull request elastic/elasticsearch#1902 from tlrx/catch-rejected-exception
Monitoring: Catch EsRejectedExecutionException when rescheduling the cleaner

Original commit: elastic/x-pack-elasticsearch@6ba20d5b8a
2016-04-04 09:24:14 -07:00
Tanguy Leroux 2ae6dec8e1 Rename RestExecuteWatchActionTest to RestExecuteWatchActionTests
Original commit: elastic/x-pack-elasticsearch@c9d8de10b2
2016-04-04 18:22:28 +02:00
Tanguy Leroux 115b037f06 Monitoring: Catch EsRejectedExecutionException when rescheduling the cleaner
closes elastic/elasticsearch#1900

Original commit: elastic/x-pack-elasticsearch@87efb135b4
2016-04-04 18:19:08 +02:00
Tanguy Leroux 652b69ad7f Monitoring: Ship segment stats with node stats
This commit adds all the following segments stats to the current node_stats document:
 - memory_in_bytes
- terms_memory_in_bytes
- stored_fields_memory_in_bytes
- term_vectors_memory_in_bytes
- norms_memory_in_bytes
- doc_values_memory_in_bytes
- index_writer_memory_in_bytes
- version_map_memory_in_bytes
- fixed_bit_set_memory_in_bytes

Original commit: elastic/x-pack-elasticsearch@ea4b8034ba
2016-04-04 18:13:24 +02:00
Alexander Reelsen 25f06bb5c1 Build: Fix packaging to not include test in artifact
A rest test was accidentally moved into the src/main/plugin-metadata
directory, which resulted the test being put into the plugin distribution
zip.

Closes elastic/elasticsearch#1907

Original commit: elastic/x-pack-elasticsearch@fbdf62b1d8
2016-04-04 17:10:07 +02:00
jaymode 4036ce97c1 shield: do not use ThreadPool#scheduleWithFixedDelay for pollers
This commit makes the user and roles poller use a self rescheduling runnable to schedule the
next run of the poller rather than using scheduleWithFixedDelay. This is done because the
pollers perform blocking I/O operations and everything using that thread pool method runs on
the schedule thread and because of this, in certain situations this can lead to a deadlock which
will prevent the cluster from forming.

Original commit: elastic/x-pack-elasticsearch@9fd0748c8c
2016-04-01 21:25:16 -04:00
Chris Earle 3126fcb856 Improved tests with better error message
Original commit: elastic/x-pack-elasticsearch@cb79988dc3
2016-04-01 14:20:03 -04:00
Chris Earle 05811c5167 Adding Locale.ROOT to String.format for forbidden API usage
Original commit: elastic/x-pack-elasticsearch@4d78705e12
2016-04-01 13:10:25 -04:00
Chris Earle 9f41a99e37 Modifying Monitoring cleanup acknowledgement message.
Original commit: elastic/x-pack-elasticsearch@1c5e1a3175
2016-04-01 12:49:05 -04:00
Chris Earle 86ed96b83b Adding support for STANDARD license
Original commit: elastic/x-pack-elasticsearch@1671d8ade3
2016-04-01 12:49:05 -04:00
Chris Earle 55b9569f7b Removing isPaid, allFeaturesEnabled, and isActive methods from enums.
Original commit: elastic/x-pack-elasticsearch@8b8c7792c7
2016-04-01 12:49:05 -04:00
Chris Earle 5e81beabf9 Simplifying License Checks
Too many places are checking for enumerations when they're really more interested in a "higher" level of
information. This will help with the forthcoming addition of the STANDARD operation mode as well.

Original commit: elastic/x-pack-elasticsearch@2799c27e19
2016-04-01 12:49:05 -04:00
jaymode d6cab8b9f1 security: read correct file when listing users
Original commit: elastic/x-pack-elasticsearch@dca906abba
2016-04-01 06:30:34 -04:00
Tanguy Leroux c9392183d2 Monitoring: Add smoke test for Monitoring with Security
Original commit: elastic/x-pack-elasticsearch@9dc800ebcc
2016-04-01 11:17:39 +02:00
Tanguy Leroux 1d72eb2b61 Monitoring: Check for source_node only for assigned shard in test
Original commit: elastic/x-pack-elasticsearch@f0d5bccecd
2016-04-01 10:34:40 +02:00
Tanguy Leroux 2c1dbf3eb6 Monitoring: Clean log messages in exporters
This commit adds the node name and the exporter name as log message prefixes.

Original commit: elastic/x-pack-elasticsearch@085b2ecf3c
2016-04-01 09:48:22 +02:00
Chris Earle 6c8ec7be28 Removing test logger now that fix is in
Original commit: elastic/x-pack-elasticsearch@8d80b59ddd
2016-03-31 12:53:20 -04:00
Tanguy Leroux 8a15a17442 Monitoring: Simplify bulk REST test
Original commit: elastic/x-pack-elasticsearch@0a02d3f3be
2016-03-31 17:49:27 +02:00
Tanguy Leroux 4007ff44b7 Monitoring: Fix synchronization in Exporters
This commit fixes an issue in synchronization in Exporters class. The export() method is synchronized and when used with LocalExport can provoke a deadlock. LocalExporter exports data locally using bulk requests that can trigger cluster state updates for mapping updates. If a exporters settings update sneaks in, the settings update waits for the export to terminate but the export waits for the settings to be updated... and boom.

This commit removes the synchronized and refactor Exporters/LocalExporter to use state and dedicated instance of LocalBulk for each export so that synchronizing methods is not necessary anymore.

It also lower down some random settings in MonitoringBulkTests because the previous settings almost always fill the bulk thread pool.

closes elastic/elasticsearch#1769

Original commit: elastic/x-pack-elasticsearch@f50c916f8b
2016-03-31 13:47:53 +02:00
javanna dc998764e8 Merge branch 'master' into enhancement/discovery_node_one_getter
Original commit: elastic/x-pack-elasticsearch@5a7ed8aafd
2016-03-31 10:50:21 +02:00
javanna 770de79a92 Remove DiscoveryNode#id in favour of existing DiscoveryNode#getId
Original commit: elastic/x-pack-elasticsearch@69bd4a9640
2016-03-31 10:49:45 +02:00
Tanguy Leroux b056fed38b Monitoring: Add REST test for monitoring bulk endpoint
Original commit: elastic/x-pack-elasticsearch@52166aec1f
2016-03-31 10:44:08 +02:00
javanna 02751ffff8 Merge branch 'master' into enhancement/discovery_node_one_getter
Original commit: elastic/x-pack-elasticsearch@cf4c5bc630
2016-03-30 17:26:02 +02:00
javanna 52ad574827 Remove DiscoveryNode#id in favour of existing DiscoveryNode#getId
Original commit: elastic/x-pack-elasticsearch@64951de2f9
2016-03-30 17:22:59 +02:00
javanna 83bf15494b fix checkstyle line lenght issue
Original commit: elastic/x-pack-elasticsearch@3ddbde1922
2016-03-30 16:20:22 +02:00
javanna 126383439a Rename DiscoveryNodes#localNodeMaster to isLocalNodeElectedMaster
Original commit: elastic/x-pack-elasticsearch@ccb685fe9a
2016-03-30 15:40:32 +02:00
javanna 9461dde896 Remove DiscoveryNodes#masterNode in favour of existing DiscoveryNodes#getMasterNode
Original commit: elastic/x-pack-elasticsearch@070850c49f
2016-03-30 15:40:23 +02:00
javanna 9f87fd5fc7 Remove DiscoveryNodes#localNode in favour of existing DiscoveryNodes#getLocalNode
Original commit: elastic/x-pack-elasticsearch@fd85aa2325
2016-03-30 15:40:14 +02:00
javanna 716a3e743e Remove DiscoveryNodes#masterNodeId in favour of existing DiscoveryNodes#getMasterNodeId
Original commit: elastic/x-pack-elasticsearch@acbedb87fd
2016-03-30 15:40:03 +02:00
javanna be01a18b35 Rename static DiscoveryNode#masterNode(Settings) to isMasterNode
Original commit: elastic/x-pack-elasticsearch@7b9ec10675
2016-03-30 15:39:39 +02:00
Adrien Grand ffb70f3011 IndexActionTests: Always map `foo` as a keyword.
Original commit: elastic/x-pack-elasticsearch@de2ad22c57
2016-03-30 15:21:22 +02:00
Nik Everett 0531dd8b88 Switch from getRandom to random
This is a reaction to
https://github.com/elastic/elasticsearch/pull/17394
which handled a long standing TODO in core.

Original commit: elastic/x-pack-elasticsearch@76425300a2
2016-03-30 08:58:31 -04:00
Simon Willnauer a39433ab48 prefix node attribute with node.attr
Original commit: elastic/x-pack-elasticsearch@44a0ef8fc6
2016-03-30 14:55:13 +02:00
javanna bd6775e0da Remove DiscoveryNode#masterNode in favour of existing DiscoveryNode#isMasterNode
Original commit: elastic/x-pack-elasticsearch@0bd29df7ea
2016-03-30 14:52:53 +02:00
javanna 9842e649f7 Remove DiscoveryNode#name in favour of existing DiscoveryNode#getName
Original commit: elastic/x-pack-elasticsearch@5907a80818
2016-03-30 14:47:50 +02:00
javanna 7689141909 Remove DiscoveryNode#id in favour of existing DiscoveryNode#getId
Original commit: elastic/x-pack-elasticsearch@a498f45e4a
2016-03-30 14:43:03 +02:00
javanna c8ea0758e9 Remove DiscoveryNode#id in favour of existing DiscoveryNode#getId
Original commit: elastic/x-pack-elasticsearch@44b835ae38
2016-03-30 14:42:13 +02:00
Adrien Grand 216874881f Don't rely on fielddata being enabled by default.
See elastic/elasticsearchelastic/elasticsearch#17386.

Original commit: elastic/x-pack-elasticsearch@361af3931a
2016-03-30 14:34:54 +02:00
Jim Ferenczi 2f627f6a50 Merge pull request elastic/elasticsearch#1847 from jimferenczi/drop_conf_file
Remove CONF_FILE from scripts

Original commit: elastic/x-pack-elasticsearch@4cfdc339de
2016-03-30 14:34:43 +02:00
Boaz Leskes 15e9edc2f4 Make AgentService.stopCollection wait till things are stopped
Closes elastic/elasticsearch#1848

Original commit: elastic/x-pack-elasticsearch@bc1f9b203f
2016-03-30 14:25:48 +02:00
jaymode c41fc7dc1d change xpack -> x-pack in in.bat
Leftover from elastic/elasticsearch#1799

Original commit: elastic/x-pack-elasticsearch@00d8bfea4e
2016-03-30 07:34:54 -04:00
Boaz Leskes 32dfa07dd1 MonitoringBulkTests to log in DEBUG
Original commit: elastic/x-pack-elasticsearch@04aca6c654
2016-03-30 11:39:23 +02:00
Jim Ferenczi d0c0a9efc4 Remove CONF_FILE from scripts
Support for it has been dropped in es

Original commit: elastic/x-pack-elasticsearch@d5a17a61d5
2016-03-30 11:17:15 +02:00
Boaz Leskes ec34163b01 Marvel integ test should stop and start exporters between tests. The ongoing activity messes with ESIntegTest level clean ups
Original commit: elastic/x-pack-elasticsearch@474ed7080a
2016-03-30 10:21:13 +02:00
Alexander Reelsen e0fcbcbb51 Elasticsearch: Rename plugin from 'xpack' to 'x-pack'
This is just to be consistent with out naming, which is
supposed to be `x-pack`.

Closes elastic/elasticsearch#1759

Original commit: elastic/x-pack-elasticsearch@0697f70855
2016-03-30 09:48:46 +02:00
Chris Earle 793a6138a3 Adding logger to catch test failure. See elastic/elasticsearch#1769
Original commit: elastic/x-pack-elasticsearch@ab47c05739
2016-03-29 17:59:48 -04:00
Chris Earle 9b7feb25ca Adding type to generic call
Original commit: elastic/x-pack-elasticsearch@ec1cb8be55
2016-03-29 17:42:25 -04:00
javanna 99af2d60d3 Merge branch 'enhancement/node_client_setting_removal'
Original commit: elastic/x-pack-elasticsearch@31af38c4c9
2016-03-29 21:56:04 +02:00
jaymode fcbbb43425 shield: index metadata privilege allows shard actions
The view index metadata privilege did not grant access to the shard level field mapping
action or the shard level validate query action. This caused the apis to restrict access to
the data when it should have been allowed.

Closes elastic/elasticsearch#1827

Original commit: elastic/x-pack-elasticsearch@7832699cb6
2016-03-29 15:19:43 -04:00
jaymode 0a7b4257f5 add type parameters to fix compilation
Original commit: elastic/x-pack-elasticsearch@0a8a16f9a0
2016-03-29 15:00:53 -04:00
javanna ac1ec748a6 use TransportClient.CLIENT_TYPE constants for comparisons
Original commit: elastic/x-pack-elasticsearch@d2556e8d3d
2016-03-29 18:36:59 +02:00
jaymode c34598a3cd test: wait until threads are ready in MonitoringBulkTests#testConcurrentRequests
This commit synchronizes the start of the threads that are executing monitoring bulk requests concurrently
to ensure all threads are ready before starting. Without this some threads will execute requests while
other threads are still being constructed.

Original commit: elastic/x-pack-elasticsearch@e777fb5c28
2016-03-29 07:18:47 -04:00
javanna a5ed623251 Merge branch 'master' into enhancement/node_client_setting_removal
Original commit: elastic/x-pack-elasticsearch@af74045e0c
2016-03-29 12:34:31 +02:00
javanna d31983d6b6 adapt to additional changes, attributes is now a regular map in DiscoveryNode
Original commit: elastic/x-pack-elasticsearch@0ba590ed9b
2016-03-29 12:34:14 +02:00
jaymode 5ab407b2e0 test: handle update-by-query to update_by_query rename
Original commit: elastic/x-pack-elasticsearch@72ba860f3f
2016-03-29 06:34:04 -04:00
Tanguy Leroux b8e8d7d246 Rename bin/xpack/esusers to /bin/xpack/users
Original commit: elastic/x-pack-elasticsearch@388eda9f24
2016-03-29 10:36:38 +02:00
Lee Hinman c62ba37759 Add API endpoints for the cluster allocation explain API
Relates to https://github.com/elastic/elasticsearch/pull/17305

Original commit: elastic/x-pack-elasticsearch@839d8dc53c
2016-03-28 17:29:01 -06:00
jaymode 2550548a44 shield: handle merging granted and non-granted indices acls
This commit changes the handling in the merge method of the IndexAccessControl class to
properly handle merging IndexAccessControl objects with differing values for the granted
flag. Prior to this commit, in a scenario where the flag differed, one IndexAccessControl granted
no access to an index, and the other granted access with DLS/FLS resulted in full access
being granted to the index.

Closes elastic/elasticsearch#1821

Original commit: elastic/x-pack-elasticsearch@e403e43689
2016-03-28 12:27:50 -04:00
jaymode 77e6622179 shield: remove the ability to specify username in request body
This commit removes the parsing of the username in the request body of a put user
request. Additionally, we use the name passed into the put role request builder rather
than the name from the parsed role descriptor.

Original commit: elastic/x-pack-elasticsearch@0a085d5844
2016-03-28 12:08:27 -04:00
jaymode 0d1f3da353 security: rename ESUsersRealm to FileRealm
This commit is the forward port of renaming the type for esusers to file. There is no
backwards compatibility maintained here. Additionally, a few other renames and
cleanups have been made:

* `esusers` commands is now `users`
* org.elasticsearch.shield.authc.esusers -> org.elasticsearch.shield.authc.file
* Validation.ESUsers -> Validation.Users
* ESUsersTool -> UsersTool
* ESUsersToolTests -> UsersToolTests
* ESNativeUsersStore -> NativeUsersStore
* ESNativeRolesStore -> NativeRolesStore.
* org.elasticsearch.shield.authz.esnative collapsed to org.elasticsearch.shield.authz.store
*  ESNativeTests -> NativeRealmIntegTests

Closes elastic/elasticsearch#1793

Original commit: elastic/x-pack-elasticsearch@d2a0c136f3
2016-03-28 06:18:57 -04:00
Chris Earle 7d481aab94 Making Watcher disabled by default for Monitoring Integration tests
Some tests [reasonably] fail due to unexpected indices appearing in the cluster due to Watcher.

- Also had to reset shieldEnabled as a static field, which makes no sense, but tests were failing unpredictably without it
    - Now they're passing unpredictably with it... will investigate

Original commit: elastic/x-pack-elasticsearch@9b6ce681d8
2016-03-25 18:45:24 -04:00
javanna 257ae2cb44 Merge branch 'master' into enhancement/node_client_setting_removal
Original commit: elastic/x-pack-elasticsearch@d3522628d4
2016-03-25 22:28:40 +01:00
Chris Earle 5d3a608786 Fix checkstyle overrun
Original commit: elastic/x-pack-elasticsearch@c965dde2e1
2016-03-25 16:54:00 -04:00
Chris Earle aacbeb2a81 Randomly Enable Watcher while running Monitoring Tests
This is required to make sure that the integration for monitoring the Watcher Threadpool is actually working.

- Also added the full property name when the assertContains check fails
- Made shieldEnabled an instance level field rather than a static one
- Added watcherEnabled field in the same fashion (including enableWatcher method that by default randomly enables it)
- Added method to locally filter the expected field names based on watcher being enabled for the failing test

Original commit: elastic/x-pack-elasticsearch@2c56e2f26f
2016-03-25 16:17:02 -04:00
javanna 622193ca40 separated attributes from node roles in DiscoveryNode
Node roles are now serialized as well, they are not part of the node attributes anymore. DiscoveryNodeService takes care of dividing settings into attributes and roles. DiscoveryNode always requires to pass in attributes and roles separately.

Original commit: elastic/x-pack-elasticsearch@32a4eb0fb4
2016-03-25 20:15:07 +01:00
Tanguy Leroux ea2be5d4d9 Merge pull request elastic/elasticsearch#1807 from tlrx/add-more-threadpool-stats
Monitoring: Add more thread pool stats

Original commit: elastic/x-pack-elasticsearch@b9e533b25d
2016-03-25 16:43:05 +01:00
Tanguy Leroux 2397158d20 Fix ClusterStateTests
Original commit: elastic/x-pack-elasticsearch@7bca8abe67
2016-03-25 16:40:38 +01:00
jaymode 1bf3a93e4f test: fix IndexPrivilegeTests after removal of predefined privileges
Original commit: elastic/x-pack-elasticsearch@6b913449b3
2016-03-25 10:43:30 -04:00
Tanguy Leroux a3807b078d Monitoring: Add more thread pool stats
This commit adds stats for generic/get/management/watcher thread pools.

Related to elastic/elasticsearch#1750

Original commit: elastic/x-pack-elasticsearch@8b001b50c6
2016-03-25 15:40:06 +01:00
javanna fc2ece87bd Merge branch 'master' into enhancement/node_client_setting_removal
Original commit: elastic/x-pack-elasticsearch@4276ae3192
2016-03-25 15:21:11 +01:00
javanna c1414b9f86 adapt to upstream changes
Original commit: elastic/x-pack-elasticsearch@7f281d6f29
2016-03-25 15:20:49 +01:00
jaymode 6fab4680a2 security: roles store poller should only update existing entries
Original commit: elastic/x-pack-elasticsearch@6573f4d689
2016-03-25 07:24:26 -04:00
jaymode 929e179150 shield: put user should validate password length
This changes the put user request builder to validate password length when a
password is provided. The validation is the same as what we use in the file
based realm.

Closes elastic/elasticsearch#1800

Original commit: elastic/x-pack-elasticsearch@fde1d6c685
2016-03-24 15:25:22 -04:00
Alexander Reelsen cf6cadf19f Build: Move xpack to plugin group
This is needed in order to make `bin/elasticsearch-plugin install xpack`
work, as it expects the plugin in a certain path.

Original commit: elastic/x-pack-elasticsearch@252c55e5a8
2016-03-24 18:38:58 +01:00
jaymode ca9ebf5351 security: refresh before searching in pollers
This commit is the forward port of fixes made in 2.3 for the roles and users
pollers. The pollers now refresh since not all operations are guaranteed to
refresh.

The clear roles tests are also made more evil since the poller runs at different
intervals on each node and can sometimes run almost continuously. The
modification requests now randomize if they refresh or not as well.

Original commit: elastic/x-pack-elasticsearch@f61159c40a
2016-03-24 12:55:35 -04:00
Chris Earle ac6b5b7c25 Modifying based on review comments
Original commit: elastic/x-pack-elasticsearch@8e3b5f4c17
2016-03-24 11:47:49 -04:00
Chris Earle 87c3730244 Removing unnecessary JavaDoc
Original commit: elastic/x-pack-elasticsearch@083f5529ac
2016-03-24 11:47:49 -04:00
Chris Earle 43de1ff8da Modify the CleanerService to use a minimum
Users running the `CleanerService` should not be able to disable it (via a `-1` as the time setting) because they'll just shoot themselves in the foot. This PR changes the behavior to allow extensive amounts via the setting (e.g., they could set it to 2 years). By doing this via the `Setting`, we can avoid a lot of boilerplate code for verification as well. If we decide to allow it to be disabled, then the setting should be explicit. I've found that users tend to not understand setting times to `-1`.

With the internal `IndicesCleaner` runnable, I have also moved the rescheduling code to `onAfter` so that it always happens, even if the license makes it temporarily invalid.

I also think that we should allow the user to dynamically set the setting regardless of it being allowed -- and warn on it. This way they can set it when it's expired or during the trial, but it will take effect when they apply the paid license. I think that this will provide a better user experience so that they do not have to remember to re-set it later.

This also removes the `LocalExporter`-specific setting that allowed it to override the global retention. If we ever add another listener, then we should add exporter-specific settings to support this kind of functionality.

Adds some tests for the settings as well as for the service, while also removing now unneeded ones.

Original commit: elastic/x-pack-elasticsearch@3abd41807e
2016-03-24 11:47:49 -04:00
Nik Everett 8e3e19d8c6 Handle method rename in core
Original commit: elastic/x-pack-elasticsearch@43b5edbff0
2016-03-24 10:42:41 -04:00
Alexander Reelsen 2f267530a6 Set version 5.0.0-alpha1
Helping commit for changes in the core

Original commit: elastic/x-pack-elasticsearch@73c8e19a29
2016-03-24 08:36:43 +01:00
Areek Zillur 84d27b52be fix for removing suggest transport action in core elasticsearchelastic/elasticsearch#17198
Original commit: elastic/x-pack-elasticsearch@c8a742c9e4
2016-03-23 16:19:36 -04:00
javanna 57114ae4f0 Merge branch 'master' into enhancement/node_client_setting_removal
Original commit: elastic/x-pack-elasticsearch@c8a896b7e3
2016-03-23 17:51:54 +01:00
Adrien Grand 71542594e6 ShieldIndexSearcherWrapper should create the scorer only once. elastic/elasticsearch#1725
Currently it first creates a scorer, then checks if the role bits are sparse,
and falls back to the bulk scorer if they are dense. The issue is that creating
scorers and bulk scorers is very expensive on some queries such as ranges,
prefix and terms queries. So it should rather check whether bits are sparse
first in order to decide whether to use the scorer or bulk scorer.

Original commit: elastic/x-pack-elasticsearch@067d630099
2016-03-23 11:38:30 +01:00
Adrien Grand 52a91d7c6f Fix compilation.
Original commit: elastic/x-pack-elasticsearch@83e6882b10
2016-03-23 11:10:45 +01:00
Tanguy Leroux fe97d2ba51 Monitoring: Add REST endpoint to allow external systems to index monitoring data
Original commit: elastic/x-pack-elasticsearch@04aa96a228
2016-03-23 10:12:00 +01:00
Tanguy Leroux 40dc747968 Monitoring: Add MonitoringClientProxy
Similar to WatcherClientProxy, the elasticsearch client used in exporters must be proxied to avoid circular dependencies at Guice's injection time. This commit add a MonitoringClientProxy as well as a MonitoringClient to be used later in monitoring's transport actions.
(cherry picked from commit b70c095)

Original commit: elastic/x-pack-elasticsearch@17327cffe5
2016-03-23 09:21:33 +01:00
Jim Ferenczi a56a5dd193 Merge pull request elastic/elasticsearch#1766 from jimferenczi/xpack_isolation
xpack plugin can be isolated now that we have xpack extension support

Original commit: elastic/x-pack-elasticsearch@933f4acf23
2016-03-22 15:58:17 +01:00
Jim Ferenczi 1fa22c921a xpack plugin can be isolated now that we have xpack extension support
Original commit: elastic/x-pack-elasticsearch@9f742c754f
2016-03-22 15:27:30 +01:00
Jim Ferenczi cc152a867a Remove sigar from the x-pack windows script classpath
Original commit: elastic/x-pack-elasticsearch@247e945ff5
2016-03-22 15:07:03 +01:00
Jim Ferenczi 0f8f70a404 Fix windows build
Avoid empty elements in x-pack bat script classpath to make JarHell happy

Original commit: elastic/x-pack-elasticsearch@06dd95b8ca
2016-03-22 14:13:33 +01:00
Jim Ferenczi 9c6aa6353e Make xpack extensible:
* Add XPackExtension: an api class (like Plugin in core) for what a x-pack extension can do.
* Add XPackExtensionCli: a cli tool for adding, removing and listing extensions for x-pack.
* Add XPackExtensionService: loading of jars from pluginsdir/x-pack/extensions, into child classloader.
* Add bin/x-pack/extension script, similar to plugin cli, which installs an extension into pluginsdir/x-pack/extensions.
* Add XPack extension integration test.

Fixed elastic/elasticsearch#1515

Original commit: elastic/x-pack-elasticsearch@130ba03270
2016-03-22 11:41:38 +01:00
javanna 25847038ee Merge branch 'master' into enhancement/node_client_setting_removal
Original commit: elastic/x-pack-elasticsearch@b36411e98f
2016-03-21 17:22:47 +01:00
jaymode c5d155efe9 security: always serialize the version with the user
This change always serializes the version with the user so that we have this information
for times when we need to make changes and deal with serialization changes. We do this
in the authentication service because the user object is also serialized as part of the get
users response and the StreamInput there will have the appropriate version set on it already
and we do not need to add it in that case.

Closes elastic/elasticsearch#1747

Original commit: elastic/x-pack-elasticsearch@a7ceece09c
2016-03-21 09:50:35 -04:00
Boaz Leskes d939289825 Change ClusterService package
As a result of ESelastic/elasticsearch#17183

Closes elastic/elasticsearch#1751

Original commit: elastic/x-pack-elasticsearch@1e553855f0
2016-03-21 13:55:48 +01:00
Martijn van Groningen 51a69c5814 Fixes for upstream percolator changes
Original commit: elastic/x-pack-elasticsearch@b31f9ff62b
2016-03-21 12:37:16 +01:00
markharwood d6df27a3ac Graph rest test fix - can time out waiting for green so trying with only 1 shard
Original commit: elastic/x-pack-elasticsearch@0e7391eb92
2016-03-21 09:33:19 +00:00
Alexander Reelsen fc99174b48 Testing: Removing AwaitsFix annotation as tests pass again
Tests had been muted in elastic/x-pack@0ba4d7ead8,
but several local runs passed.

Original commit: elastic/x-pack-elasticsearch@6bf0dfa4e3
2016-03-18 15:28:50 +01:00
jaymode 67c6cf4055 test: add debug logging and cleanup stopping of servers
Adds debug logging to try to get more information about random failures in these
tests. Also cleans up some potential issues with the code that handled the stopping
of random ldap servers to test failure cases.

See elastic/elasticsearch#1542

Original commit: elastic/x-pack-elasticsearch@573b4161dd
2016-03-18 07:34:32 -04:00
jaymode 2872acd742 remove watcher and graph privileges. manage does not include security
Original commit: elastic/x-pack-elasticsearch@da250ed842
2016-03-17 14:31:03 -04:00
jaymode 833bf726e6 define graph and watcher privileges statically
These privileges no longer need to be defined as a custom privilege since the
code is now consolidated into a single plugin. This also changes the manage
cluster privilege to be an alias to the all privilege.

Original commit: elastic/x-pack-elasticsearch@a7f444c898
2016-03-17 14:31:03 -04:00
jaymode cf0fd986e1 rename graph actions
Original commit: elastic/x-pack-elasticsearch@70a71d6bd6
2016-03-17 14:31:03 -04:00
jaymode 135742823e rename watcher actions
Original commit: elastic/x-pack-elasticsearch@5c67344595
2016-03-17 14:31:03 -04:00
jaymode 5bc3c0c1f8 security: rename actions to not use shield
This commit renames the security actions to not use shield in their action names. This
also includes updating the privileges as well.

Original commit: elastic/x-pack-elasticsearch@10460dffdb
2016-03-17 14:31:00 -04:00
jaymode a22539aca0 shield: add support for new privilege naming
This commit adds support for the privilege naming defined in elastic/elasticsearch#1342 and removes the
support for the privileges that were deprecated in 2.3. This change also includes
updates to the documentation to account for the new roles format.

Original commit: elastic/x-pack-elasticsearch@98e9afd409
2016-03-17 14:29:26 -04:00
Lukas Olson 03f8452d70 Update API spec for Shield
Original commit: elastic/x-pack-elasticsearch@6dcdfebc0a
2016-03-17 10:27:44 -07:00
Simon Willnauer 214b4f269a Use IndexModule#forceQueryCacheType instead of overriding configrations
This is a follow up from elasticsearchelastic/elasticsearch#16799 which prevents setting index
level settings on a node level.

Original commit: elastic/x-pack-elasticsearch@80d1819ab3
2016-03-17 14:25:43 +01:00
Areek Zillur 8817d2a3c0 rename license API actions
GetLicenseAction: cluster:admin/plugin/license/get --> cluster:monitor/xpack/license/get
PutLicenseAction: cluster:admin/plugin/license/put --> cluster:admin/xpack/license/put
DeleteLicenseAction: cluster:admin/plugin/license/delete --> cluster:admin/xpack/license/delete

closes elastic/elasticsearch#1717

Original commit: elastic/x-pack-elasticsearch@fe3f07cd69
2016-03-16 14:21:14 -04:00
jaymode 9e08579d4f security: file parsing only supports the new format
This commit remove the pre-existing file parsing code and replaces it with the updated
code in the RoleDescriptor class. This unifies the parsing for the files and API for roles.

Closes elastic/elasticsearch#1596

Original commit: elastic/x-pack-elasticsearch@9e0b58fcf1
2016-03-16 12:33:05 -04:00
Tanguy Leroux 1161edca2c Monitoring: Do not expose sensitive settings
Original commit: elastic/x-pack-elasticsearch@25d81bb7b6
2016-03-16 17:14:40 +01:00
Ali Beyad 8e79737b06 Merge pull request elastic/elasticsearch#1696 from abeyad/suggester-wiring
Suggester refactoring requires a Suggester to parse X-Content to Builder

Original commit: elastic/x-pack-elasticsearch@26f4d84068
2016-03-16 12:01:37 -04:00
jaymode 60500ec6af security: cleanup the logging in the native stores
A lot of messages were being logged at the info level in the native user and roles
stores. This changes the logging to be more selective in the cases where the index
does not exist or the error is really an error and the user should be notified.

Closes elastic/elasticsearch#1339

Original commit: elastic/x-pack-elasticsearch@0bc0d9bf7a
2016-03-16 10:36:12 -04:00
Ali Beyad 7d8149cb86 Suggester refactoring requires a Suggester to parse X-Content to Builder
As part of the search refactoring effort, we need to pass a Suggester
to the methods that parse X-Content to a SuggestBuilder in every
instance where we are parsing search/suggest requests.

Original commit: elastic/x-pack-elasticsearch@7f815c617a
2016-03-16 10:27:29 -04:00
Colin Goodheart-Smithe e3551a7570 [TEST] fix timeout test so it checks message but not the reported time waiting
The reported time waiting for watches can be slightly different from the actual timeout (e.g. 2.1 seconds instead of 2 seconds) so checking the time waited in the message makes the test sometimes fail

Original commit: elastic/x-pack-elasticsearch@c2cd9da486
2016-03-16 13:44:33 +00:00
Nik Everett 787ebd5850 Handle task status registration cleanup
Core reworked how it registered tasks status's with NamedWriteableRegistry
so it was more pluggable. It changed a few signatures and x-plugins needs
these small changes to keep compiling.

Original commit: elastic/x-pack-elasticsearch@3dcf1df152
2016-03-16 08:20:40 -04:00
Jason Tedor 69b69f7af1 Use setting in integration test cluster config
This commit modifies using system properties to configure an integration
test cluster and instead use settings in the generated Elasticsearch
config file.

Original commit: elastic/x-pack-elasticsearch@65211b93d0
2016-03-15 20:01:01 -04:00
jaymode edc9580f66 security: validate that security and audit indices can be auto created
Adds a check to the settings at startup to ensure that the security and audit indices are
allowed to be auto created if a user has disabled auto create explicitly.

Additionally fixes a small issue with the error message for watcher passing the incorrect
value.

Closes elastic/elasticsearch#1453

Original commit: elastic/x-pack-elasticsearch@2b0698ff19
2016-03-15 13:15:00 -04:00
jaymode 03336912bb security: native realm is added when defined realms are unlicensed
If a user configures only custom realms and they are not licensed to use the custom realms then
we need to return our default realms. The default realms should be the esusers and esnative realms.
We were only returning the esusers realm previously.

Closes elastic/elasticsearch#1491

Original commit: elastic/x-pack-elasticsearch@3dc2b5d3a8
2016-03-15 13:11:19 -04:00
Yannick Welsch 0136f16ce7 [TEST] Suppress exception in Watcher benchmark
Original commit: elastic/x-pack-elasticsearch@099cad0ac1
2016-03-15 16:49:21 +01:00
Yannick Welsch 61123bb107 Remove System.out.println and Throwable.printStackTrace from tests
Relates to elastic/elasticsearchelastic/elasticsearch#17112

Original commit: elastic/x-pack-elasticsearch@404e40a4be
2016-03-15 16:30:23 +01:00
Simon Willnauer 4ec4b0d7e1 Watcher should try to load trust/keystore from `config` directory
Today Watcher tries to load stuff from the bin's parent directory which
is not readable since the shared data directory has been moved out of
the nodes parent in elasticsearchelastic/elasticsearch#17072 which causes security exception
now. The test copies trust stores into the config dir and that's where
we should read it from by default or even better explicitly configure the path?!

Original commit: elastic/x-pack-elasticsearch@1d32a595cf
2016-03-15 10:07:10 +01:00
Areek Zillur 4c4eac692a fix for elasticsearchelastic/elasticsearch#16442
Use index uuid as index folder name to decouple index name from being used as index folder name

Original commit: elastic/x-pack-elasticsearch@23193bcd6e
2016-03-14 23:28:05 -04:00
Ryan Ernst 6cc3b553a7 Merge pull request elastic/elasticsearch#1699 from rjernst/lazy_san
Lazily generate san string for certificates

Original commit: elastic/x-pack-elasticsearch@3f1dc7a2b4
2016-03-14 19:23:26 -07:00
Ryan Ernst 1b21f20e5b Lazily generate san string for certificates
The san string used by certificate generation for ssl tests currently
runs at gradle configuration time. This takes several seconds, and
significantly slows down gradle configuration on every invocation.
This change wraps the code into a lazy evaluator that will be invoked at
runtime, and cache the string once it is generated.

Original commit: elastic/x-pack-elasticsearch@812036f416
2016-03-14 17:55:22 -07:00
Simon Willnauer 6b7dadce43 remove unused imports
Original commit: elastic/x-pack-elasticsearch@4a8f4285f8
2016-03-14 20:52:49 +01:00
Simon Willnauer 67efe608d1 add assertion DefaultIndicesAndAliasesResolver that PutMapping special case holds
Original commit: elastic/x-pack-elasticsearch@417123150a
2016-03-14 20:51:54 +01:00
Tanguy Leroux 7d377a5b7f Fix LocalIndicesCleanerTests
Original commit: elastic/x-pack-elasticsearch@3e08c84e80
2016-03-14 16:25:00 +01:00
Simon Willnauer 3d1f1814a4 Wrap lines after 140 chars
Original commit: elastic/x-pack-elasticsearch@dce9c019ae
2016-03-14 15:40:42 +01:00
Simon Willnauer 3f9508d525 Merge pull request elastic/elasticsearch#1687 from s1monw/fix_put_mapping
Don't override indices when concreteIndex is set on PutMappingRequest

PutMappingRequest has a special case since it can come with one and only
one concrete index. In such a case we can't replace the indices list
with all authorized indices but should rather only check if the index
is authorized and otherwise fail the request.

Original commit: elastic/x-pack-elasticsearch@8949b16f16
2016-03-14 15:39:46 +01:00
Simon Willnauer d37bf240fe Don't override indices when concreteIndex is set on PutMappingRequest
PutMappingRequest has a special case since it can come with one and only
one concrete index. In such a case we can't replace the indices list
with all authorized indices but should rather only check if the index
is authorized and otherwise fail the request.

Original commit: elastic/x-pack-elasticsearch@4ee20029e1
2016-03-14 15:27:42 +01:00
Tanguy Leroux 9443086655 Monitoring: Declare resolvers id & version in registry
This commit changes the location where the ID and Version of the resolvers are defined. It was in each constructor, now it is in the ResolversRegistry class. It also rename MonitoringIds to MonitoredSystem.

Original commit: elastic/x-pack-elasticsearch@81d7711c40
2016-03-14 15:22:50 +01:00
Simon Willnauer 13b7bd884a wrap lines to match 140 chars
Original commit: elastic/x-pack-elasticsearch@108a982dc1
2016-03-14 12:52:24 +01:00
Simon Willnauer cd63903665 Fix compile errors
Original commit: elastic/x-pack-elasticsearch@9e95c31e9d
2016-03-14 12:42:34 +01:00
Tanguy Leroux 45dc717b92 Monitoring: Make MonitoringDoc implements Writeable
Update after Chris & Uri comments

Original commit: elastic/x-pack-elasticsearch@74e4420b2f
2016-03-14 10:07:52 +01:00
David Pilato 5a1fbe6d62 Update Setting according to changes in master
We changed Setting signatures in master branch of elasticsearch.
We need to adapt x-plugins to the new code.

See https://github.com/elastic/elasticsearch/pull/16629.

Closes elastic/elasticsearch#1684.

Original commit: elastic/x-pack-elasticsearch@c911aaca69
2016-03-13 20:34:15 +01:00
Ryan Ernst c739e9b61f Merge pull request elastic/elasticsearch#1654 from rjernst/cli-parsing
Switch to jopt-simple

Original commit: elastic/x-pack-elasticsearch@577af5af6f
2016-03-11 12:55:22 -08:00
jaymode 8e816bdfb9 test: add tests for scroll requests with document and field level security
Original commit: elastic/x-pack-elasticsearch@107b24e785
2016-03-11 11:50:38 -05:00
Tanguy Leroux e51aa21575 Monitoring: Add export() method to Exporters class
This commit adds a synchronized "export()" method to the Exporters so that the Exporters class can be used as an export service for exporting monitoring docs.

Original commit: elastic/x-pack-elasticsearch@22bda986c5
2016-03-11 17:33:16 +01:00
markharwood 925afa3cab Graph - port of 2.x graph API and kibana UI plugin
Closes X-plugins issue 518

Original commit: elastic/x-pack-elasticsearch@6c6371ed74
2016-03-11 14:22:31 +00:00
Yannick Welsch 0abe314bb6 Merge pull request elastic/elasticsearch#1674 from ywelsch/enhance/fix-logger-usages
Fix wrong placeholder usage in logging statements

Original commit: elastic/x-pack-elasticsearch@101f043fac
2016-03-11 10:30:28 +01:00
Tanguy Leroux be0e4255ce Mute EmailAttachmentTests and HistoryTemplateEmailMappingsTests
Original commit: elastic/x-pack-elasticsearch@5d62b2af11
2016-03-11 09:33:36 +01:00
Yannick Welsch 970efba3a3 Fix wrong placeholder usage in logging statements
Also make logging message String constant to allow static checks

Relates to elastic/elasticsearchelastic/elasticsearch#16707

Original commit: elastic/x-pack-elasticsearch@b5bd423de4
2016-03-10 20:18:07 +01:00
Alexander Reelsen f417ec7e8c CLI: Remove cygwin support
Cygwin support has been removed from master, so we should be
consistent with x-plugins as well.

See elastic/elasticsearchelastic/elasticsearch#16871

Closes elastic/elasticsearch#1635

Original commit: elastic/x-pack-elasticsearch@8dd93209c7
2016-03-10 17:29:35 +01:00
Nik Everett 016b9d017c Add reindex tests to shield
Original commit: elastic/x-pack-elasticsearch@fe00f317b3
2016-03-10 10:17:46 -05:00
Martijn van Groningen a3a7acd934 tests: mute ingest/70_bulk test suite for now
Original commit: elastic/x-pack-elasticsearch@5fa15106f3
2016-03-10 15:00:14 +01:00
Martijn van Groningen fbe0aefa0c fix compile erors due to upstream changes in node stats
Original commit: elastic/x-pack-elasticsearch@a0a600cb68
2016-03-10 13:22:59 +01:00
Simon Willnauer a033f95072 Use ShardId#getIndexName() since index name has been removed from ShardSearchRequest
Original commit: elastic/x-pack-elasticsearch@ba997d0ae3
2016-03-09 19:54:27 +01:00
Ryan Ernst 8c5d8653e0 Fix forbidden apis usages, and convert more tests to CommandTestCase
Original commit: elastic/x-pack-elasticsearch@f5400388eb
2016-03-09 00:18:23 -08:00
Ryan Ernst d880803c2d Cutover more tests to CommandTestCase
Original commit: elastic/x-pack-elasticsearch@19c168a712
2016-03-08 17:28:11 -08:00
Ryan Ernst 64419c0856 Merge branch 'master' into cli-parsing
Original commit: elastic/x-pack-elasticsearch@bfe0cdc477
2016-03-08 14:17:44 -08:00
Ryan Ernst 53d87d158f Remove old help files and references to old cli tool stuff
Original commit: elastic/x-pack-elasticsearch@b02faa5251
2016-03-08 14:16:47 -08:00
Alexander Reelsen e728a49853 Watcher: Fix home path for croneval CLI tool
Drive-by fixes putting the $JAVA binary into quotes to support spaces, like
we do in other scripts as well.

Closes elastic/elasticsearch#1642

Original commit: elastic/x-pack-elasticsearch@f40fba32cc
2016-03-08 17:48:42 +01:00
Tanguy Leroux 62ad9f4f0d Move lazy initialization classes from Watcher to XPack
This commit moves the InitializingModule and InitializingService classes in the common XPack package so that it can be used by any plugin. It also renames the module and service from Initializing* to LazyInitializing* and add a ClientProxy class.

Original commit: elastic/x-pack-elasticsearch@fbdf9d1614
2016-03-08 16:25:11 +01:00
Robert Muir 0f905e9b00 CLI: Don't let ubuntu try to install its crazy jayatana agent.
See https://github.com/elastic/elasticsearch/pull/13813

This is the corresponding workaround for x-plugins commandline tools.

Closes elastic/elasticsearch#719

Original commit: elastic/x-pack-elasticsearch@c607fd2197
2016-03-08 16:23:47 +01:00
Tanguy Leroux 146f91f730 Watcher: Rename ClientProxy to WatcherClientProxy
Original commit: elastic/x-pack-elasticsearch@84c17d1bc0
2016-03-08 14:28:39 +01:00
Alexander Reelsen f4eb0e7c7c Docs: Mention option to enable scripting for watcher-only
Deep down buried there is an option to not only allow global
script execution, but also allow a single scripting language for
watcher only. It is time to document it as well.

Renamed this option to `script.engine.groovy.inline.xpack_watch`
to align with xpack renaming.

Closes elastic/elasticsearch#1422

Original commit: elastic/x-pack-elasticsearch@845eb5a0c0
2016-03-08 12:04:28 +01:00
Alexander Reelsen 10644a2784 Watcher: Fix correct setting of email attachment names
Fix to ensure that the email attachment has a correctly set filename, which is
also now explained in the documentation.

In addition there is a check now for email attachments, that a filename can only
be specified once, otherwise an exception is thrown.

Closes elastic/elasticsearch#1503

Original commit: elastic/x-pack-elasticsearch@2a399058b3
2016-03-07 21:57:42 +01:00
Ryan Ernst bafbcd9ed3 Moved MockTerminal and cleaned up some tests
Original commit: elastic/x-pack-elasticsearch@ffb873c826
2016-03-07 12:42:44 -08:00
jaymode 03dcc5ea67 shield: copy settings to tribe nodes
The shield settings need to be copied down to the tribe nodes so that they are
aware of the shield configuration. Otherwise there will be issues such as SSL
not carrying over or authentication realms not being available.

Closes elastic/elasticsearch#702

Original commit: elastic/x-pack-elasticsearch@7bd7674f3e
2016-03-07 12:31:40 -05:00
jaymode 2612edf4cb test: blacklist the ingest default processors rest test
This test assumes no modules are installed but the shield rest tests run with the modules
installed.

Original commit: elastic/x-pack-elasticsearch@2ba47fcd0f
2016-03-07 12:15:06 -05:00
jaymode 101ff22546 fix compile after removal of versions < 2.0.0
Original commit: elastic/x-pack-elasticsearch@61e2814aac
2016-03-07 10:53:07 -05:00
jaymode 98e904deef fix compile due to core change in NodeInfo
Original commit: elastic/x-pack-elasticsearch@3ff3fa63e6
2016-03-07 09:34:53 -05:00
Robert Muir 2a9ba9e934 lucene 6 api changes (tests only)
Original commit: elastic/x-pack-elasticsearch@8120c29cd8
2016-03-07 04:14:09 -05:00
Ryan Ernst b54e6a7ae6 Merge branch 'master' into cli-parsing
Original commit: elastic/x-pack-elasticsearch@ff525e0e00
2016-03-06 13:33:08 -08:00
Ryan Ernst 59ec9302c8 Switch cli tests to unified MockTerminal
This is the xplugins side of elastic/elasticsearchelastic/elasticsearch#16966

Original commit: elastic/x-pack-elasticsearch@a2e2faf20a
2016-03-06 13:18:40 -08:00
jaymode 46cae1b2b1 test: remove check for bound addresses that fails on mac
Original commit: elastic/x-pack-elasticsearch@85c7f158ff
2016-03-04 17:47:38 -05:00
jaymode 186dbf547a security: protect the user and roles index
This commit adds the logic to protect the user and roles index that we store locally
by restricting access to the internal XPack user. We need to do this in two places;
the first is when resolving wildcards and the other is when authorizing requests
made against specific indices.

Original commit: elastic/x-pack-elasticsearch@8ee0ce02db
2016-03-04 17:16:03 -05:00
Ryan Ernst 6fa9c1631d Merge branch 'master' into cli-parsing
Original commit: elastic/x-pack-elasticsearch@83f7f8139d
2016-03-04 12:15:11 -08:00
Ryan Ernst 706216844b Changed esusers tool to use jopt-simple
Original commit: elastic/x-pack-elasticsearch@1f8763fcd6
2016-03-04 12:14:34 -08:00
javanna 30a7ff1daa Adapt to node.client setting removal
We would previosly check if a node was a client node, we can now check it by just verifying that it is not a transport client through client_type setting.

Original commit: elastic/x-pack-elasticsearch@bddd44866e
2016-03-04 20:41:13 +01:00
Tanguy Leroux 452e729a02 Monitoring: Fix NodeStatsResolverTests on Windows platforms
Because load_average is not available on Windows, it must be excluded from test assertions.

Original commit: elastic/x-pack-elasticsearch@f67f9bb5e7
2016-03-04 17:28:49 +01:00
Tanguy Leroux 66e49a0546 Marvel: Add integration test for Marvel+Shield with SSL
closes elastic/elasticsearch#1467

Original commit: elastic/x-pack-elasticsearch@9dd6bf9629
2016-03-04 16:55:35 +01:00
Tanguy Leroux a8e52eb520 Monitoring: Clean up and refactoring
This commit removes various constructors in monitoring documents and add a single constructeur that accepts a monitoring id and version. It also renames *Renderer classes to *Resolver and centralizes the logic of resolving the index name, type name and id in 1 place. It changes Exporter so that they use these resolvers to know in which index a given document must be indexed.

Original commit: elastic/x-pack-elasticsearch@c2349a95a6
2016-03-04 16:31:14 +01:00
Ryan Ernst fe377cfda2 Converted cron eval tool to use jopt-simple
Original commit: elastic/x-pack-elasticsearch@fde96657d5
2016-03-03 00:35:39 -08:00
Adrien Grand c16ca2c779 string has been split into text and keyword.
Original commit: elastic/x-pack-elasticsearch@b98100f8b5
2016-03-03 09:17:47 +01:00
Ryan Ernst ee2749365f Add tests for FileAttributesChecker
Original commit: elastic/x-pack-elasticsearch@eb78087e64
2016-03-03 00:03:30 -08:00
Ryan Ernst 9864ae05a2 Switch system key tool to use jopt-simple
Original commit: elastic/x-pack-elasticsearch@c5c459c77a
2016-03-02 23:16:50 -08:00
jaymode d8617556cf shield: do not require password for user update operations
When thinking about applications and the need to update a user, we should not need to
update the password of the user when making changes to things like roles, email, full
name, or metadata. This commit changes how we handle operations where the password
field is missing.

When the password field is missing, we try to execute an update. If the user exists, all
values for the user are updated except for the password field. If the user does not exist
and the password field is missing then a ValidationException is returned.

When the password field is present, we always issue an index request.

Closes elastic/elasticsearch#1492

Original commit: elastic/x-pack-elasticsearch@3d8a5f2db6
2016-03-02 10:26:55 -05:00
jaymode d46f465ddb shield: refresh on user and role modifications by default
This commit introduces the default refresh on user and role update and delete
operations. The behavior can be controlled via the `refresh` parameter on the
REST API and the refresh option in the Java API.

Closes elastic/elasticsearch#1494

Original commit: elastic/x-pack-elasticsearch@aff4d13886
2016-03-02 09:04:41 -05:00
Tanguy Leroux ab3ee46104 Fix checkstyle violation
Original commit: elastic/x-pack-elasticsearch@7730c96d7c
2016-03-02 11:14:13 +01:00
Martijn van Groningen ceaed02f38 Added `manage_pipeline` privilege and `ingest_admin` default role for the ingest feature.
Closes elastic/elasticsearch#1367

Original commit: elastic/x-pack-elasticsearch@a4c9e22203
2016-03-02 10:53:10 +01:00
Tanguy Leroux edd993077b Marvel: Only clean timestamped indices with the current template version
Only current timestamped indices, like .marvel-es-1-* indices should be deleted. Other indices like the ones created by pre v2.3.0 plugin versions should be kept (like .marvel-es-YYYY.MM.dd)

Original commit: elastic/x-pack-elasticsearch@b2aff31875
2016-03-02 10:47:30 +01:00
Tanguy Leroux b39f4dcc37 Monitoring: Index node attributes and remove default mappings in data index
Original commit: elastic/x-pack-elasticsearch@c1581ecc1b
2016-03-02 10:06:27 +01:00
Ryan Ernst 626bdbe7bf Convert license verifier tool to jopt-simple
Original commit: elastic/x-pack-elasticsearch@56ecc6333c
2016-03-01 17:13:17 -08:00
Jason Tedor 4c089cf33d Bump Elasticsearch version to 5.0.0-SNAPSHOT
This commit bumps the Elasticsearch version to 5.0.0-SNAPSHOT in line
with the alignment of versions across the stack.

Relates elastic/elasticsearchelastic/elasticsearch#16862

Original commit: elastic/x-pack-elasticsearch@155641c5e4
2016-03-01 17:18:13 -05:00
Ryan Ernst bd64bd6ff3 Switched licence gen and keypair gen tools to use jopt simple
Original commit: elastic/x-pack-elasticsearch@310229b4a5
2016-03-01 12:21:05 -08:00
jaymode c8ee64d0cb test: sort by _uid to get consistent ordering
Original commit: elastic/x-pack-elasticsearch@73b5c49ea5
2016-03-01 09:30:12 -05:00
Nik Everett 507e9ede59 Rename unit test so it looks like a unit test
Original commit: elastic/x-pack-elasticsearch@d2d39ad50b
2016-03-01 08:09:25 -05:00
jaymode de72f4aeee security: change DLS behavior to OR queries together
This commit changes the behavior of combining multiple document level security queries
from an AND operation to an OR operation.

Additionally, the behavior is also changed when evaluating the combination of roles that
have document level security and roles that do not have document level security. Previously
when the permissions for these roles were combined, the queries from the roles with document
level security were still being applied, even though the user had access to all the documents.
This change now grants the user access to all documents in this scenario and the same applies
for field level security.

Closes elastic/elasticsearch#1074

Original commit: elastic/x-pack-elasticsearch@291107ec27
2016-03-01 07:03:38 -05:00
javanna 0be2b6cbbc Adapt to SearchServiceTransportAction rename
Original commit: elastic/x-pack-elasticsearch@b154325787
2016-03-01 12:58:53 +01:00
uboness 2a1b3250db Cleanup Security Roles
- Renamed `AddRoleAction/Request/Response` to `PutRoleAction/Request/Response`
- also renamed the user/roles rest actions
- Changed the returned format for `RestGetRoleAction`. Previously this endpoint returned an array of role descriptor. Now it returns an object where the role names serve as the keys for the role objects. This is aligned with other APIs in ES (e.g. index templates).
- When `RestGetRoleAction` cannot find all the requested roles, it'll return an empty object and a 404 response status
- Also cleaned up `RoleDescriptor`

Original commit: elastic/x-pack-elasticsearch@742f6e0020
2016-03-01 05:47:22 -05:00
Boaz Leskes 3ddbd77090 Remove DiscoveryService and reduce guice to just Discovery elastic/elasticsearch#1571
DiscoveryService was a bridge into the discovery universe. This is unneeded and we can just access discovery directly or do things in a different way.

This is a complement to elastic/elasticsearchelastic/elasticsearch#16821

Closes elastic/elasticsearch#1571

Original commit: elastic/x-pack-elasticsearch@496f0c4081
2016-02-29 20:26:38 +01:00
jaymode 03be6e3a62 change shield in log messages to security
Original commit: elastic/x-pack-elasticsearch@9c5acc488a
2016-02-29 10:26:48 -05:00
uboness 759d99de9c changed the User API
- Now it's more aligned with other APIs in ES (e.g. index template API)
- the "get user" API now returns an object as a response. The users are keyed by their username. If none of the requested users is found, an empty object will be returned with a 404 response status.
- the body of "put user" request doesn't require "username" anymore (as it's defined as part of the URL)

Original commit: elastic/x-pack-elasticsearch@f7c12648b1
2016-02-29 09:47:39 -05:00
Alexander Reelsen 1f113e07f4 Watcher: Fail email action on attachment download issues
In case that a single email attachment cannot be downloaded, this ensures
that the whole action fails with a correct Action.Failure.

This also fixes an NPE that would occur otherwise.

Original commit: elastic/x-pack-elasticsearch@7bb042a719
2016-02-28 21:07:23 -08:00
Alexander Reelsen cc8109bc87 Watcher: Fix naming of data attachments to use id in email attachments
This is a small fix to use specified id when sending data attachments.
The current solution always used "data".

Also a minor refactoring was made to include get the different parser impls
from the EmailAttachmentsParser instead of specifying them twice in the
EmailAction.

Closes elastic/elasticsearch#1503

Original commit: elastic/x-pack-elasticsearch@9354e83c8b
2016-02-28 20:22:45 -08:00
Nik Everett d7170197f6 Handle core's log refactoring
Original commit: elastic/x-pack-elasticsearch@9e2e41db90
2016-02-26 16:06:31 -05:00
jaymode 06fc60c2f6 shield: handle null tokens when parsing roles
The roles parsing does not currently handle null tokens since the YAML parser
was not emitting them. With the upgrade to Jackson 2.7.1, the parser is now
emitting the null token value.

Original commit: elastic/x-pack-elasticsearch@abcad633ad
2016-02-26 15:03:56 -05:00
Alexander Reelsen 47f1c2daa5 Watcher: Throw exception when empty URL is handed in http requests
This ensures that invalid watches are not even added and rejected on
index time.

Closes elastic/elasticsearch#1510

Original commit: elastic/x-pack-elasticsearch@d18e0c8ef6
2016-02-25 17:31:11 -08:00
Alexander Reelsen b97fea44d7 Watcher: Fix SSL default port when using request.fromUrl
If no port was specified, port 80 was assumed, even if https was specified
was the protocol. This lead to weird failures in the logs and trying to use
SSL on port 80.

Relates elastic/elasticsearch#1567

Original commit: elastic/x-pack-elasticsearch@0ea11d612e
2016-02-25 16:26:53 -08:00
Alexander Reelsen 47ef39037b Watcher: Fix latch await in timeout tests
The awaiting latch was not waiting as long as the sleep in the code
causing the latch to fail and the test to fail.

This code aligns the time to wait for the latch and the sleep code
in the mock http server.

Original commit: elastic/x-pack-elasticsearch@8a2cc61204
2016-02-25 15:56:30 -08:00
uboness eb8dbfb998 Renamed `.shield` index to `.security`
Going forward (from 5.0 on) we'll remove all occurrences of the "shield" name/word from the code base. For this reason we want to already start using `.security` index in 2.3 such that we won't need to migrate it to a `.security` index later on.

Original commit: elastic/x-pack-elasticsearch@74a1cbfcf2
2016-02-25 15:10:22 -08:00
Alexander Reelsen 4eef709d2e Watcher: Fix timeout tests by increasing wait timeout
The request timeout and the real time the webserver slept was 5000ms.
In case of loaded systems, there might be cases, where the request was
still received in time.

This commit increases the server side sleep time to 10 seconds, to ensure
that the client aborts the request early

Original commit: elastic/x-pack-elasticsearch@718c05519f
2016-02-25 14:23:34 -08:00
Alexander Reelsen 2daef601d4 Watcher: Fix timeout tests
The current HTTP timeout tests had two problems.

* Binding to port 9200-9300
* The first request to hit was having a delay, the other ones had not,
  so if any other component hit the test inbetween (likely in a CI env),
  the HTTP request from the test itself will not be delayed.

Both cases are fixed in this commit.

Original commit: elastic/x-pack-elasticsearch@d696e020cc
2016-02-25 12:23:46 -08:00
Nik Everett c7796d5cb7 Rename more tests to match naming conventions
Original commit: elastic/x-pack-elasticsearch@d76c217bd9
2016-02-25 15:20:41 -05:00
jaymode 0522127924 Test: remove use of network.host in smoke test ssl plugins
This removes the use of a specific address in smoke test ssl plugins and instead generates
the certificate with all of the IP addresses and DNS names of the system as subject
alternative names. This required duplication and modification of some code from core's
NetworkUtils.

Original commit: elastic/x-pack-elasticsearch@576824376f
2016-02-25 13:56:46 -05:00
Alexander Reelsen 2f088a60bc Watcher: Always get HTTP response body independent from error code
When an HTTP input returns an error body, right now we check if the
error code is below 400 and only then we include the body.

However using another method from URLConnection, the body can be
access always.

Closes elastic/elasticsearch#1550

Original commit: elastic/x-pack-elasticsearch@1743fd0a77
2016-02-25 10:25:34 -08:00
Nik Everett 08e0717f6b Make tests follow naming conventions
One test wasn't running because it didn't match!

Original commit: elastic/x-pack-elasticsearch@081c6b09e2
2016-02-25 13:14:01 -05:00
uboness 7fbf5645e2 fixed checkstyle error
Original commit: elastic/x-pack-elasticsearch@7676e988a8
2016-02-25 01:50:19 -08:00
uboness 266bf09437 Fixed build failure related to security roles APIs
- roles are now reliably parsed
- in `Put Role` API, added a double check to verify that the role name in the URL matches the role name if the body. Also, if the body doesn't have a role name, the role name in the URL will be used.

Original commit: elastic/x-pack-elasticsearch@5054ce8567
2016-02-25 01:38:04 -08:00
uboness 8ff6b93a3c Cleanup Security Roles
- Renamed `AddRoleAction/Request/Response` to `PutRoleAction/Request/Response`
- also renamed the user/roles rest actions

Original commit: elastic/x-pack-elasticsearch@ae0ccd61e5
2016-02-24 13:46:32 -08:00
Chris Earle 7e334a5e4b Renaming interval variable to include units and reordering constructor field values to ensure listener is added last
Original commit: elastic/x-pack-elasticsearch@60983f4190
2016-02-23 13:17:42 -05:00
Chris Earle ef81157c47 Add Javadocs
Also a minor fix to the phrasing in `MarvelLicensee#expirationMessages()`.

Original commit: elastic/x-pack-elasticsearch@9366c07930
2016-02-22 15:56:19 -05:00
Chris Earle 0b0ca8f2a6 Removing unused imports
Original commit: elastic/x-pack-elasticsearch@40c094af91
2016-02-22 15:56:19 -05:00
Chris Earle df99174122 Removing duplicated import
Original commit: elastic/x-pack-elasticsearch@1618ec79d4
2016-02-22 15:56:19 -05:00
uboness 18b08c82ca Introducing user full name, email and metadata.
- `full_name` and `email` are optional user fields
- `metadata` is an optional arbitrary meta data that can be associated with the user
- cleaned up the user actions - consistent naming (e.g. `PutUserAction` vs. `AddUserAction`)
- moved source parsing from the `PutUserRequest` to the `PutUserRequestBuilder`
- renamed`WatcherXContentUtils` to `XContentUtils` and moved it to sit under `o.e.xpack.commons.xcontent`

Closes elastic/elasticsearch#412

Original commit: elastic/x-pack-elasticsearch@5460e3caf7
2016-02-22 10:22:36 -08:00
Alexander Reelsen 6d0d09468b Watcher/Shield: Ensure only one .in.bat file exists
This was a leftover from watcher/shield being different plugins.

Closes elastic/elasticsearch#1530

Original commit: elastic/x-pack-elasticsearch@521b4bad14
2016-02-21 15:20:24 -08:00
Tanguy Leroux a27d2bcc50 Fix line length
Original commit: elastic/x-pack-elasticsearch@bbf883437f
2016-02-21 15:19:52 -08:00
Tanguy Leroux b5f40adb12 Marvel: Add stats for primary shards
closes elastic/elasticsearch#1198

Original commit: elastic/x-pack-elasticsearch@e823d01397
2016-02-21 14:39:52 -08:00
jaymode e3f53be3ef test: disable marvel for watcher disabled tests
We shouldn't have marvel enabled for these tests because we get false test failures
due to marvel indices existing and failing to lock the shard.

Original commit: elastic/x-pack-elasticsearch@11123bb660
2016-02-21 14:11:43 -08:00
jaymode d9ca4e0ce3 fix shield settings to not rely on iteration order
This removes the use of group setting for `shield.` and introduces some individual settings
and some group settings that should not overlap and cause issues when iteration order
changes.

See elastic/elasticsearch#1520

Original commit: elastic/x-pack-elasticsearch@193e937193
2016-02-21 10:10:52 -08:00
Simon Willnauer 64e4ccf9a0 Update x-pack to elastic/elasticsearchelastic/elasticsearch#16740
Original commit: elastic/x-pack-elasticsearch@63a3f49730
2016-02-20 17:21:47 -08:00
Adrien Grand 7b2fae3982 Unmute DynamicIndexNameIntegrationTests.
Closes elastic/elasticsearch#1527

Original commit: elastic/x-pack-elasticsearch@4ba9fe5f08
2016-02-15 16:12:57 +01:00
Colin Goodheart-Smithe 77ffdbcbb4 Merge pull request elastic/elasticsearch#1519 from colings86/refactor/aggRefactoringChanges
X-Plugin changes due to the changes in the Aggregations Java API

Original commit: elastic/x-pack-elasticsearch@524be093de
2016-02-15 11:33:42 +00:00
Adrien Grand 026c26db54 Mute DynamicIndexNameIntegrationTests.
Original commit: elastic/x-pack-elasticsearch@1795d24800
2016-02-15 12:07:40 +01:00
javanna 4482cd4f6c Adapt to removal of unused generics type from TransportMessage
followup of elastic/elasticsearch#15776, the type is not needed anymore.

Original commit: elastic/x-pack-elasticsearch@3f96dc552d
2016-02-12 17:21:28 +01:00
jaymode 8337832405 test: skip discovery ec2 in smoke-test-plugins*
Until we can fix the shield settings, we have bugs where we depend on the iteration
order of a map and discovery ec2 settings provoke this (most likely through a map
resize).

See elastic/elasticsearch#1520

Original commit: elastic/x-pack-elasticsearch@fbc32cf069
2016-02-12 10:40:27 -05:00
Colin Goodheart-Smithe 197b8fe56f X-Plugin changes due to the changes in the Aggregations Java API
Original commit: elastic/x-pack-elasticsearch@b983d0a00f
2016-02-12 12:06:06 +00:00
Simon Willnauer ec76d3bce0 Fix imports
Original commit: elastic/x-pack-elasticsearch@79e4535040
2016-02-12 10:52:48 +01:00
uboness ffe339ae31 Refactoring for 5.0 - phase 5
- Moved all settings in Marvel from `marvel.*` to `xpack.monitoring.*`
- Cleaned up marvel settings in general - they're all now under `MarvelSettings` class
- fixed some integration tests along the way (they were configured wrong and never actually tested anything)
- Updated the docs accordingly
- Added `migration-5_0.asciidoc` under the Marvel docs to explain how to migrate from Marvel 2.x to XPack 5.0.
- Replaced all `marvel` mentions in the logs to `monitoring`
- Removed the `xpack.monitoring.template.version` setting from the templates
- renamed the templates to `monitoring-es-data.json` and `monitoring-es.json`
- monitoring indices are now `.monitoring-es-<version>-data` and `.monitoring-es-<version>-<timestamp>`

Original commit: elastic/x-pack-elasticsearch@17f2abe17d
2016-02-11 21:34:38 +01:00
jaymode 95a8f77146 shield: do not throw exception if authorization header is not a basic token
Custom realms may enable the use of other authorization schemes than just basic authentication
and these schemes should work in addition to our built in realms. However, our built in realms use
the UsernamePasswordToken class to parse the Authorization header, which had a check to ensure
the token was for basic authentication and if not, an exception was thrown. The throwing of the
exception stops the authentication process and prevents custom realms from evaluating the header
if they come later in the ordering of realms.

This change removes the throwing of the exception unless the header starts with 'Basic ' and is invalid.

Original commit: elastic/x-pack-elasticsearch@fd438ded95
2016-02-11 09:59:35 -05:00
uboness 42c9eead60 Refactoring for 5.0 - phase 4
- renaming `ShieldPlugin` to `Shield` (it's no longer a plugin)
 - renaming `WatcherPlugin` to `Watcher` (it's no longer a plugin)
 - renaming `MarvelPlugin` to `Marvel` (it's no longer a plugin)
 - renaming `LicensePlugin` to `Licensing` (it's no longer a plugin)
 - renamed setting:`watcher.enabled` -> `xpack.watcher.enabled`
 - renamed setting:`marvel.enabled` -> `xpack.marvel.enabled`

Original commit: elastic/x-pack-elasticsearch@35a6540b11
2016-02-10 11:15:35 +01:00
Igor Motov dbff0e1144 Add task cancellation mechanism
See elastic/elasticsearchelastic/elasticsearch#16320 for more information

Original commit: elastic/x-pack-elasticsearch@4f8a9b1258
2016-02-09 22:31:08 -05:00
Nik Everett 97e8cdc5f0 Remove suppression and implement hashCode
Original commit: elastic/x-pack-elasticsearch@0505f28e78
2016-02-09 21:49:13 -05:00
Nik Everett 390bbecf4b Suppress EqualsHashCode check where it fails
Original commit: elastic/x-pack-elasticsearch@eb5243a652
2016-02-09 21:32:23 -05:00
Jason Tedor 602f67d7c6 Use MessageDigests abstraction in core
This commit removes the message digest providers in x-plugins by using
the MessageDigests abstraction in core. In particular, this permits the
removal of the use of MessageDigest#clone in x-plugins.

Closes elastic/elasticsearch#1489

Original commit: elastic/x-pack-elasticsearch@6868e6e8ed
2016-02-09 10:18:00 -05:00
uboness 3a6a1d5dc2 Shield refactoring for 5.0 - phase 3
- Consolidated the `bin` and `config` directories of watcher, shield and marvel under a single `config/xpack` and `bin/xpack` directories.

 - updated docs accordingly

Original commit: elastic/x-pack-elasticsearch@c2aa6132fa
2016-02-09 16:06:49 +01:00
uboness 92f027159a Shield refactoring for 5.0 - phase 2
- Started to move configuration under the `xpack` name
 - Cleaned up `ShieldPlugin`
 - renamed `ShieldClient` to `SecurityClient`
 - Introduced `XPackClient` that wraps security and watcher clients

Original commit: elastic/x-pack-elasticsearch@f05be0c180
2016-02-09 14:32:33 +01:00
jaymode 50452e403f test: filter out unsupported ciphers when checking default socket factory
Closes elastic/elasticsearch#2

Original commit: elastic/x-pack-elasticsearch@6510f65dc4
2016-02-09 08:14:05 -05:00
Alexander Reelsen e8ad8cbb36 Watcher: Load versioned index template for watch history
This loads an index template for the watch history to make sure,
that field changes are taken into account.

Also, the dynamic mapping for the watch history template has been
changed from strict to false.

This means that new fields can be included in a document, but they
will not indexed and are not searchable.

In addition the index names have been changed from .watch_history-$date to
.watcher-history-$template-$date - using dashes to be more consistent.

Closes elastic/elasticsearch#1299

Original commit: elastic/x-pack-elasticsearch@794f982234
2016-02-09 09:39:07 +01:00
Simon Willnauer 25c3a66502 Fix compile error after core change
Original commit: elastic/x-pack-elasticsearch@ba170bbc63
2016-02-08 21:57:12 +01:00
jaymode aa2eb15f31 fixes to allow bad apple tests to pass
This commit fixes the bad apple tests that failed when running them. The
IndexAuditTrailEnabledTest was removed and the test was folded into the
IndexAuditIT. Some watcher tests that relied on mustache were moved
into the QA tests with the mustache plugin.

Additionally, fixing these tests uncovered a issue with the privileges needed
for writing data into an index. If the mappings need to be updated because
of a write, then the update mapping action gets executed. In 2.x this was
handled by the system user, but now is executed under the user's context,
which is the correct thing to do. The update mapping action is now added to
the read, index, crud, and write privileges for an index.

Original commit: elastic/x-pack-elasticsearch@30711f9625
2016-02-08 09:48:10 -05:00
Jason Tedor 7f5349db57 Avoid cloning Mac instances
This commit modifies the HmacSHA1Provider to return a thread local
instance of Mac instead of using clone since some providers do not
support clone.

Closes elastic/elasticsearch#1468

Original commit: elastic/x-pack-elasticsearch@cb38f5f9e8
2016-02-06 09:12:39 -05:00
Alexander Reelsen e6784d5c7d Checkstyle: Adhere to checkstyle in all xpack .java files
In elastic/elasticsearch#1442 checkstyle checks were added, but also some files were freed from this.
If we have support for checkstyle, we should check this for all files and not allow
exceptions. This commit removes the file list to ignore any files and fixes all the
java files.

Original commit: elastic/x-pack-elasticsearch@99e6cbc5be
2016-02-05 16:57:41 +01:00
Simon Willnauer 878d244a96 Fix compilation failures
Original commit: elastic/x-pack-elasticsearch@4afbf84c20
2016-02-05 16:07:46 +01:00
Tanguy Leroux 4488cacdd8 Marvel: Add permission for HTTPExporter + SSL support
See elastic/elasticsearch#1451

Original commit: elastic/x-pack-elasticsearch@5e3d87149e
2016-02-05 15:21:27 +01:00
Alexander Reelsen 93a3d3f570 Watcher: Ensure that HTTP headers are case insensitive in response
According to RFC 2616 HTTP headers are case insensitive.
But `HttpResponse#contentType()` only looks up for Content-Type.

This stores all header responses lower cased in the HTTP response.

Closes elastic/elasticsearch#1357

Original commit: elastic/x-pack-elasticsearch@c009be8365
2016-02-05 14:59:27 +01:00
Jason Tedor 166e8786ea Fix compilation from upstream XContentType changes
This commit fixes a compilation issue that arose from upstream changes
in core where some method names on XContentType were changed.

Closes elastic/elasticsearch#1463

Original commit: elastic/x-pack-elasticsearch@0a3763dbb5
2016-02-05 07:28:47 -05:00
Jim Ferenczi cc8ae273c9 Merge pull request elastic/elasticsearch#1462 from jimferenczi/rename_plugin_cmd
bin/plugin is now bin/elasticsearch-plugin

Original commit: elastic/x-pack-elasticsearch@8ecf75af66
2016-02-05 12:52:54 +01:00
Yannick Welsch 3dfb38b99e Change path separator for Checkstyle suppressions to be Windows compatible
Original commit: elastic/x-pack-elasticsearch@8850f753c7
2016-02-05 12:15:47 +01:00
Tanguy Leroux d70b49f42d Marvel: Add source_node information to marvel documents
This commit adds a new `source_node` field to all marvel documents that holds various information (node's name/ip/host/id/transport address) about the node that emitted the document.

(cherry picked from commit elastic/x-pack@29a411a931)

Original commit: elastic/x-pack-elasticsearch@66e057d334
2016-02-05 11:50:07 +01:00