Commit Graph

120 Commits

Author SHA1 Message Date
Nik Everett 06d26635de Packaging tests: ingest-grok is now ingest-common 2016-06-08 14:03:48 -04:00
Jason Tedor f16f65741e Fix when plugins directory is symlink
This commit fixes an issue with the plugins directory being a symbolic
link. Namely, the install plugins command attempts to always create the
plugins directory just in case it does not exist. The JDK method used
here guarantees that the directory is created, and an exception is not
thrown if the directory could not be created because it already
exists. The problem is that this JDK method does not respect symlinks so
its internal existence checks fails, it proceeds to attempt to create
the directory, but the directory creation fails because the symlink
exists. This is documented as being not an issue. We work around this by
checking if there is a symlink where we expect the plugins directory to
be, and only attempt to create if not. We add a unit test that plugin
installation to a symlinked plugins directory works as expected.
2016-05-26 14:10:32 -04:00
Jason Tedor d29844e597 Remove custom plugins path
This commit removes the ability to specify a custom plugins
path. Instead, the plugins path will always be a subdirectory called
"plugins" off of the home directory.
2016-05-26 10:16:25 -04:00
Jason Tedor 84dfa360b1 Fix list of poi dependencies for ingest-attachment
This commit fixes the list of poi dependencies for the ingest-attachment
plugin after they were upgraded in
1d40c4bbc1.
2016-05-24 17:03:25 -04:00
Tanguy Leroux 1f011f9dea Remove Delete-By-Query plugin
closes #18469
2016-05-24 13:28:20 +02:00
Jason Tedor c257e2c51f Remove settings and system properties entanglement
Today when parsing settings during bootstrap, we add a system property
for every Elasticsearch setting. Additionally, settings can be set via
system properties. This commit simplifies this situation.
 - settings are no longer propogated to system properties
 - system properties can not be used to set settings
 - the "es." prefix on settings is no longer required (nor permitted)
 - test logging has a dedicated system property (tests.logger.level)

Relates #18198
2016-05-19 14:08:08 -04:00
Tanguy Leroux 35d3bdab84 Add Google Cloud Storage repository plugin
Closes #12880
2016-05-19 13:26:23 +02:00
Jason Tedor db4809d906 Remove last vestigates of /bin/sh shebangs
This commit removes the remaining /bin/sh shebangs in favor of
/bin/bash.

Relates #18448
2016-05-18 11:03:00 -04:00
Lee Hinman ff5c7965ed Don't run `mkdir` when $DATA_DIR contains a comma-separated list
Resolves #16992
Resolves https://github.com/elastic/cookbook-elasticsearch/issues/441
2016-05-17 13:42:24 -06:00
Jason Tedor 9ce96f5792 Add tests that packages depend on bash
This commit adds bats tests that the RPM and Debian packages depend on
bash.

Relates #18292
2016-05-12 09:04:24 -04:00
Jason Tedor 7d1fd17172 Remove plugin script parsing of system properties
The plugin script parses command-line options looking for Java system
properties and extracts these arguments to pass to the java command when
starting the JVM. Since elasticsearch-plugin allows arbitrary user
arguments to the JVM via ES_JAVA_OPTS, this parsing is unnecessary. This
commit removes this unnecessary 

Relates #18207
2016-05-09 13:06:18 -04:00
Jason Tedor 3f3fa59406 Account for rpm behavior difference on directories
This commit modifies the packaging tests to account for the fact that
rpm behaves differently with respect to preserving directories marked as
"CONFIG | NOREPLACE" on older versions versus newer versions. Older
versions will leave the directory as-is while newer versions will append
the suffix ".rpmsave" to the directory name.

Relates #18216
2016-05-09 12:25:52 -04:00
Jason Tedor 0eaa831f48 Preserve config files from RPM install
This commit modifies the packaging for the RPM package so that edits to
config files will not get lost during removal and upgrade.

Relates #18188
2016-05-06 13:24:54 -04:00
Ryan Ernst 8fc51380de Tests: improve logging for vagrant to emit entire output on failure
This change makes the vagrant tasks extend LoggedExec, so that the
entire vagrant output can be dumped on failure (and completely logged
when using --info). It should help for debugging issues like #18122.
2016-05-04 17:29:23 -07:00
Jason Tedor 4c6cf7ee88 Pass ES_JAVA_OPTS to JVM for plugins script
This commit adds support for ES_JAVA_OPTS to the elasticsearch-plugin
script.

Relates #18140
2016-05-04 12:48:39 -04:00
Lee Hinman 9da88a99da Fix exit code
Exit with proper exit code (1) and an error message if elasticsearch
executable binary does not exists or has insufficient permissions to
execute.

Squashed commit of the following:

commit 9768d316303418ba4f9c96d3f87c376048a1b1bc
Author: Puru <tuladharpuru@gmail.com>
Date:   Fri Apr 22 23:26:47 2016 +0545

    Fixed ES_HOME typo

commit 79a2b0394297f8b02b6f71b71ba35ff79f1a684e
Author: Puru <tuladharpuru@gmail.com>
Date:   Sun Apr 10 11:00:24 2016 +0545

    Improve elasticsearch startup script test

    Added improvement as per conversation in https://github.com/elastic/elasticsearch/pull/17082#issuecomment-206459613

commit 7be38e1fefd4baa6ccdbdc14745c00f6dc052e0c
Author: Puru <tuladharpuru@gmail.com>
Date:   Wed Mar 23 13:23:52 2016 +0545

    Add elasticsearch startup script test

    The test ensures that elasticsearch startup script exists and is executable.

commit d10eed5c08260fa9c158a4487bbb3103a8d867ed
Author: Puru <tuladharpuru@gmail.com>
Date:   Wed Mar 23 12:30:25 2016 +0545

    Fixed IF syntax and failure message

commit 6dc66f616545572485b4d43bee05a4cbbf1bed72
Author: Puru <tuladharpuru@gmail.com>
Date:   Sat Mar 12 11:08:11 2016 +0545

    Fix exit code

    Exit with proper exit code (1) and an error message if elasticsearch executable binary does not exists or has insufficient permissions to execute.
2016-04-22 11:48:05 -06:00
Lee Hinman 4fca5f734a Explicitly set packaging permissions
This changes our packaging to be explicit about the permissions of files
and directories in the tar.gz, rpm, and deb packages. This is to protect
against a user having an incorrectly set umask when installing.

Additionally, plugins that are installed now have their permissions set
by the plugin installation so that plugins that may have been packaged
with incorrect permissions are secured.

Resolves #17634
2016-04-21 12:30:56 -06:00
Lee Hinman a1e8fb794c Allow JSON with unquoted field names by enabling system property
In Elasticsearch 5.0.0, by default unquoted field names in JSON will be
rejected. This can cause issues, however, for documents that were
already indexed with unquoted field names. To alleviate this, a system
property has been added that can be enabled so migration can occur.

This system property will be removed in Elasticsearch 6.0.0

Resolves #17674
2016-04-19 09:14:13 -06:00
Lee Hinman 66adcb2962 [TEST] Fix extra backslash causing sed to hang
This caused sed to hang and the output never to show for the smoke test
command
2016-04-13 10:24:39 -06:00
Jason Tedor a581d7cca4 Merge pull request #17675 from jasontedor/java-opts
Add JVM options configuration file
2016-04-12 23:07:40 -04:00
Jason Tedor 3879aa2a98 Add JVM options configuration file
This commit adds a new configuration file jvm.options to centralize and
simplify management of JVM options. This separates the configuration of
the JVM from the packaging scripts (bin/elasticsearch*, bin/service.bat,
and init.d/elasticsearch) simplifying end-user operational management of
custom JVM options.
2016-04-12 11:19:16 -04:00
Lee Hinman dd8afc9567 [TEST] Add extra logging for vagrant test output
The $user, $group, and $privileges are echoed when they are being
compared.
2016-04-11 09:36:50 -06:00
Ryan Ernst 19567fb794 Add stop task for vagrant which wraps all halt tasks, instead of using clean 2016-04-06 14:15:23 -07:00
Ryan Ernst bfc708ee58 Merge pull request #17573 from rjernst/vagrant_boxes_param
Tests: Add vagrant.boxes gradle property
2016-04-06 13:33:45 -07:00
Ryan Ernst dc376fb21b Tweak packagingTest description formatting 2016-04-06 13:08:38 -07:00
Ryan Ernst 9794d371ef Always add halt tasks for all available boxes 2016-04-06 13:07:11 -07:00
Ryan Ernst 9f24de7ca8 Tests: Add vagrant.boxes gradle property
This change makes specifying which boxes to run vagrant tests on a
little easier. Previously there were two tasks, checkPackages and
checkPackagesAllDistros. With this change, there is a single
packagingTest task. The boxes to run on are specified using the
gradle property vagrant.boxes, which can be easily specified on the
command line, or in a gradle properties file. There are also two
alias names, 'sample' for a yum and apt box, and 'all' for all boxes.
2016-04-06 12:36:35 -07:00
Jason Tedor eb7ecb7296 Add test of plugin script if JAVA_HOME has a space 2016-04-06 13:28:56 -04:00
Jason Tedor 85b06f48cf Revert "Add debug logging for Vagrant upgrade test"
This reverts commit 7ecfa6e2ad.
2016-03-24 21:59:32 -04:00
Jason Tedor a0634b2442 Merge pull request #17197 from jasontedor/chown
Elasticsearch ownership for data, logs, and configs
2016-03-24 21:21:11 -04:00
Jason Tedor d22b5977de Ownership for data, logs, and configs for packages
This commit ensures that the data, logs, and config directories have the
proper ownership after the packages are installed. Additionally, this
commit ensures that the configs in /etc/elasticsearch are preserved
after removal of the RPM package.
2016-03-24 20:56:02 -04:00
Alexander Reelsen baec7bc23c Tests: Fix vagrant test on debian
Debian asks during installation, if the configuration file should be updated.
This is asked via a prompt and thus hangs.

This adds an option to always update to the newer config file, so automated
installation keeps working.
2016-03-24 14:21:33 +01:00
Simon Willnauer e4e08b28ff [TEST] Use to wait for the node to start - it will wait for state recovery and not return 503 2016-03-24 11:06:40 +01:00
Jason Tedor 7ecfa6e2ad Add debug logging for Vagrant upgrade test 2016-03-23 23:12:16 -04:00
Jason Tedor 04e6e6c3e0 Fix cluster health API call 2016-03-23 17:06:45 -04:00
Jason Tedor 033167de5b Wait for yellow indices when running upgrade test
This commit makes the Vagrant upgrade test wait for yellow indices
before attempting to get documents from the upgraded Elasticsearch node.
2016-03-23 15:57:21 -04:00
Jason Tedor a7793f7271 Cutover to elastic Vagrant boxes
This commit cuts the Vagrant tests over to the elastic Vagrant boxes.
2016-03-20 22:11:30 -04:00
Jason Tedor c3f6ad89ef Simplify module or plugin file checks 2016-03-18 11:50:49 -04:00
Jason Tedor d4abfb2a87 Centralize umask utilities in bats tests 2016-03-18 08:25:21 -04:00
Jason Tedor 6bd19cff67 Centralize existence check for modules and plugins 2016-03-17 22:34:33 -04:00
Jason Tedor 6ecfc7adbe Permissions tests for modules and plugins 2016-03-17 22:16:09 -04:00
Jason Tedor 9f73152940 Fix plugins permissions 2016-03-17 15:35:54 -04:00
Jason Tedor dacb96ba61 Fix plugin installation logging 2016-03-17 15:35:54 -04:00
Jason Tedor ba6820caeb Fix plugin file permissions tests 2016-03-17 15:35:54 -04:00
Jason Tedor ffc6ba9e20 Fix installed plugins check 2016-03-17 15:35:53 -04:00
Jason Tedor 9ed4646296 Plugin site-example no longer exists 2016-03-17 15:35:53 -04:00
Jason Tedor 9e739da1d9 Add new plugins 2016-03-17 15:35:53 -04:00
Jason Tedor ace45e8bda Add module tests 2016-03-17 15:35:53 -04:00
Jason Tedor b46dd3f01c es.path.conf can not be empty 2016-03-16 23:38:04 -04:00
Jason Tedor 8a05c2a2be Bootstrap does not set system properties
Today, certain bootstrap properties are set and read via system
properties. This action-at-distance way of managing these properties is
rather confusing, and completely unnecessary. But another problem exists
with setting these as system properties. Namely, these system properties
are interpreted as Elasticsearch settings, not all of which are
registered. This leads to Elasticsearch failing to startup if any of
these special properties are set. Instead, these properties should be
kept as local as possible, and passed around as method parameters where
needed. This eliminates the action-at-distance way of handling these
properties, and eliminates the need to register these non-setting
properties. This commit does exactly that.

Additionally, today we use the "-D" command line flag to set the
properties, but this is confusing because "-D" is a special flag to the
JVM for setting system properties. This creates confusion because some
"-D" properties should be passed via arguments to the JVM (so via
ES_JAVA_OPTS), and some should be passed as arguments to
Elasticsearch. This commit changes the "-D" flag for Elasticsearch
settings to "-E".
2016-03-13 20:09:15 -04:00