Commit Graph

238 Commits

Author SHA1 Message Date
Ryan Ernst 8b26c260d1 Plugins: Enforce plugin zip does not contain zip entries outside of the unzip dir
When unzipping a plugin zip, the zip entries are resolved relative to
the directory being unzipped into. However, there are currently no
checks that the entry name was not absolute, or relatively points
outside of the plugin dir. This change adds a check for those two cases.
2016-03-11 14:53:14 -08:00
Ryan Ernst 5bd7da5659 Addressed PR feedback
* Fix tests still referring to -E
* add comment about missing classes
* rename writer constant
2016-03-11 11:46:23 -08:00
Ryan Ernst 591fb8f028 Merge branch 'master' into cli-parsing 2016-03-11 10:45:05 -08:00
Nik Everett b2eec96045 [reindex] Make search failure cause rest failure
Indexing failures have caused the reindex http request to fail for a while
now. Both search and indexing failures cause it to abort. But search
failures didn't cause a non-200 response code from the http api. This
fixes that.

Also slips in a fix to some infrequently failing rest tests.

Closes #16037
2016-03-10 13:47:49 -05:00
Nik Everett 94798a3672 [reindex][test] Fix timeout value 2016-03-10 13:12:29 -05:00
Nik Everett 378e0d2c0c [reindex][test] Script changes destination index 2016-03-10 13:06:54 -05:00
Nik Everett b8d931d23c [reindex] Timeout if sub-requests timeout
Sadly, it isn't easy to simulate a timeout during an integration test, you
just have to cause one. Groovy's sleep should do the job.
2016-03-10 13:05:23 -05:00
Boaz Leskes cd12241e9f Decouple the TransportService and ClusterService #16872
Currently, the cluster service is tightly coupled to the transport service by both managing node connections and requiring the bound address in order to create the local disco node. This commit introduces a new NodeConnectionsService which is in charge of node connection management and makes it possible to remove all network related calls from the cluster service. The local DiscoNode is now created by DiscoveryNodeService and is set both the cluster service and the transport service during node start up.

Closes #16788
Closes #16872
2016-03-10 11:45:15 +01:00
Ryan Ernst 80198accc1 Removed old cli stuff, and add tests for new Command behavior 2016-03-08 14:13:55 -08:00
Ryan Ernst e5c852f767 Convert bootstrapcli parser to jopt-simple 2016-03-08 13:39:37 -08:00
Ryan Ernst 45b5ab24fe Moved MockTerminal and created a base test case for cli commands. 2016-03-07 12:42:15 -08:00
Ryan Ernst 7a49cd1287 Merge branch 'master' into cli-parsing 2016-03-06 13:29:56 -08:00
Ryan Ernst 5a94f8b3e6 Remove outdated nocommit and tweak assert to output bad exception message on failure. 2016-03-06 11:34:10 -08:00
Ryan Ernst ed7934ee79 Cli: Simplify test terminals
This commit simplifies and consolidates the two different
implementations of terminals used in tests. There is now a single
MockTerminal which captures output, and allows accessing as one large
string (with unix style \n as newlines), as well as configuring
input.
2016-03-06 11:30:30 -08:00
Ryan Ernst 687aadef00 Merge branch 'master' into cli-parsing 2016-03-04 12:13:17 -08:00
Martijn van Groningen 8ee17d0a86 test: replace test with rest test 2016-03-04 14:28:36 +01:00
Ryan Ernst 209da28bb2 Removed check file command tests, check file command is going away 2016-03-03 09:37:33 -08:00
Jason Tedor e75a0da4d5 Add max number of processes check
This commit adds a bootstrap check on Linux for the max number of
processes available to the user running the Elasticsearch process.

Closes #16919
2016-03-03 11:40:10 -05:00
Ryan Ernst 354ede717b Removed old help files and improved plugin cli tests 2016-03-01 11:48:52 -08:00
Nik Everett c7c8bb357a Merge pull request #16861 from nik9000/reindex_is_ready
Reindex required some parsing changes for search requests to support
differing defaults from the regular search api.
2016-03-01 10:02:48 -05:00
Ryan Ernst 8cd919c687 Added jopt simple option parser and switched plugin cli to use it 2016-02-29 19:52:42 -08:00
Nik Everett 95cc3e38fc Check test naming conventions on all modules
The big win here is catching tests that are incorrectly named and will
be skipped by gradle, providing a false sense of security.

The whole thing takes about 10 seconds on my Macbook Air, not counting
compiling the test classes, which seems worth it. Because this runs as
a gradle task with propery UP-TO-DATE handling it can be skipped if the
tests haven't been changed which should save some time.

I chose to keep this in test:framework rather than a new subproject of
buildSrc because ESIntegTestCase and doesn't inroduce any additional
dependencies.
2016-02-29 16:31:49 -05:00
Boaz Leskes 195b43d66e Remove DiscoveryService and reduce guice to just Discovery #16821
DiscoveryService was a bridge into the discovery universe. This is unneeded and we can just access discovery directly or do things in a different way.

One of those different ways, is not having a dedicated discovery implementation for each our dicovery plugins but rather reuse ZenDiscovery.

UnicastHostProviders are now classified by discovery type, removing unneeded checks on plugins.

Closes #16821
2016-02-29 20:23:38 +01:00
Nik Everett d4113d77cf Don't try to install reindex plugin in bats tests 2016-02-29 10:28:30 -05:00
Nik Everett c38119bae9 Merge branch 'master' into feature/reindex 2016-02-26 16:59:54 -05:00
Nik Everett 7c4420bf3d Merge pull request #16807 from nik9000/not_mvn
Remove mention of mvn from java source files
2016-02-25 11:05:46 -05:00
Nik Everett 49f704fd6a Remove mention of mvn from ramaining java 2016-02-24 19:54:47 -08:00
Simon Willnauer d3b2ccb49d fix line length 2016-02-22 10:51:30 -08:00
Simon Willnauer 387f0473dc Assert that we can write in all data-path on startup
Today we might start a node and some of the paths might not have the
required permissions. This commit goes through all data directories as
well as index, shard and state directories and ensures we have write access.
To make this work across all OS etc. we are trying to write a real file
and remove it again in each of those directories
2016-02-22 10:32:29 -08:00
Nik Everett 821a20f582 Merge branch 'master' into feature/reindex 2016-02-11 17:41:05 -05:00
Nik Everett 18808b7576 Move reindex from a plugin to a module 2016-02-11 17:39:49 -05:00
Nik Everett 30107b4a74 Teach reindex to stop when cancelled
All we do is check the cancelled flag and stop the request at a few key
points.

Adds the cancellation cause to the status so any request that is cancelled
but doesn't die can be seen in the task list.
2016-02-11 15:21:22 -05:00
Jim Ferenczi cb50e73f7c Add missing try with resources in InstallPluginCommandTest, this should fix the build on windows. 2016-02-11 15:13:25 +01:00
Nik Everett 0da30d5eae Merge branch 'master' into feature/reindex 2016-02-10 14:07:41 -05:00
Jim Ferenczi a520e7c7ad Fix InstallPluginCommandTests failures on windows build 2016-02-10 13:02:29 +01:00
Jim Ferenczi b146f3ecb3 Pack all the plugin files into a single folder named elasticsearch at the root of the plugin zip. 2016-02-10 10:13:05 +01:00
Simon Willnauer 9b2f40f627 Merge pull request #16471 from s1monw/no_sys_props_leaked
Apply system properties after all arguemnts are parsed in BootstrapCLIParser
2016-02-05 14:23:00 +01:00
Simon Willnauer 1e18c9a7b6 Apply system properties after all arguemnts are parsed in BootstrapCLIParser
One of our tests leaked a system property here since we failed after appling some
system properties in BootstrapCLIParser. This is not a huge deal in production since
we exit the JVM if we fail on that. Yet for correctnes we should only apply them if
we manage to parse them all.
This also caused a test failure lately on CI but on an unrelated test:
  https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+periodic/314/console
2016-02-05 11:21:00 +01:00
Jim Ferenczi 7d0181b5d4 Rename bin/plugin in bin/elasticsearch-plugin 2016-02-05 10:09:14 +01:00
Martijn van Groningen 7a6adfd93a ingest: Added foreach processor.
This processor is useful when all elements of a json array need to be processed in the same way.
This avoids that a processor needs to be defined for each element in an array.
Also it is very likely that it is unknown how many elements are inside an json array.
2016-02-04 23:44:01 +01:00
Nik Everett bb4d8b79fe Merge branch 'master' into feature/reindex 2016-02-04 09:23:49 -05:00
Ryan Ernst e97345984d Remove format args from readText and readSecret 2016-02-04 02:03:17 -08:00
Ryan Ernst 2265c5d9e9 Tests: Fix bootstrap cli tests to not clear all properties previous tests set 2016-02-03 15:09:17 -08:00
Ryan Ernst 14f03a9750 Tests: Force tribe node to use the same "node.mode" as the tribe client nodes. 2016-02-03 14:37:53 -08:00
Simon Willnauer 4a4e523357 Merge branch 'master' into make_settings_strict 2016-02-03 11:34:12 +01:00
Martijn van Groningen 64ac037a82 Added an ingest qa that tests processor real world like configurations.
Renamed `ingest-with-mustache` to `smoke-test-ingest-with-all-dependencies`
Also renamed `ingest-disabled` to `smoke-test-ingest-disabled` so that the name is more inline with other qa smoke test modules.
2016-02-02 22:37:24 +01:00
Ryan Ernst 9f47b376da Plugin cli: Improve maven coordinates detection
Identifying when a plugin id is maven coordinates is currently done by
checking if the plugin id contains 2 colons. However, a valid url could
have 2 colons, for example when a port is specified. This change adds
another check, ensuring the plugin id with maven coordinates does not
contain a slash, which only a url would have.

closes #16376
2016-02-02 12:17:56 -08:00
Simon Willnauer 818a9eefb2 Make settings validation strict
This commit enableds strict settings validation on node startup. All settings
passed to elasticsearch either through system properties, yaml files or any other
way to pass settings must be registered and valid. Settings that are unknown ie. due to
typos or due to deprecation or removal will cause the node to NOT start up. Plugins
have to declare all their settings on the `SettingsModule#registerSetting` and settings for
plugins that are not installed must be removed.

This commit also removes the ability to specify the nodes name via `-Des.name` or just `name` in the
configuration files. The node name must be prefixed with the node prexif like `node.name: Boom`. Left over
usage of `name` will also cause startup to fail.
2016-02-02 11:32:44 +01:00
Ryan Ernst a2c37c0989 CliTool: Allow unexpected exceptions to propagate
Cli tools currently catch all exceptions, and only print the exception
message, except when a special system property is set. Even with this
flag set, certain exceptions, like IOException, are captured and their
stack trace is always lost.

This change adds a UserError class, which can be used a cli tools to
specify a message to the user, as well as an exit status. All other
exceptions are propagated out of main, so java will exit with non-zero
and print the stack trace.
2016-02-01 16:35:22 -08:00
Nik Everett c1d4478067 Merge branch 'master' into feature/reindex 2016-02-01 18:33:09 -05:00