Commit Graph

119 Commits

Author SHA1 Message Date
Tanguy Leroux 127b4a8efc Change permissions on config files (#20966)
This commit changes some default file permissions on configuration files.
2016-10-24 09:42:03 +02:00
Tanguy Leroux 268d5ba97a Add JVM option "-XX:-AssumeMP" in SystemD tests (#21063)
Since with j`ava-1.8.0-openjdk-1.8.0.111-1.b15.el7_2.x86_64`, the OpenJDK packaged for CentOS and OEL override the default value (`false`) for the JVM option `AssumeMP` and force it to `true` (see [this patch](https://git.centos.org/blob/rpms!!java-1.8.0-openjdk.git/ab03fcc7a277355a837dd4c8500f8f90201ea353/SOURCES!always_assumemp.patch))

Because it is forced to true by default for these packages, the following warning message is printed to the standard output when the Vagrant box has only 1 CPU:
> OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N

This message will then fail the test introduced in #20422  where we check if no entries have been added to the journal after the service has been started.

This commit restore the default value for the `AssumeMP` option for CentOS and OracleServer.
2016-10-24 09:30:55 +02:00
Tanguy Leroux 5495b66797 Mute journalctl output check in packaging tests
This commit mutes a check on the output of journalctl after the Elasticsearch's systemd service has been started. It expected no entries in the journal but since OpenJDK build 1.8.0_111-b15 the following warning message is printed:
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
2016-10-21 10:52:11 +02:00
Alexander Reelsen 2e447479c0 Tests: Fix ingest-attachment vagrant tests
The vagrant tests are checking for a certain jar version.

Relates #20710
2016-10-10 14:54:35 +02:00
David Pilato dfd1eebdd0 Remove mapper attachments plugin
We now have in 5.0.0 `ingest-attachment` plugin.
We can remove `mapper-attachments` plugin for 6.0.

Closes #18837.
2016-09-19 09:01:16 +02:00
Tanguy Leroux 5d373e7f94 [TEST] Fix SystemD test on Fedora 24
The journalctl command on Fedora24 returns an exit code 0 if no log entry exist in the journal, whereas other distributions return 1.
2016-09-14 21:14:30 +02:00
Tanguy Leroux f3f9b499ad [Packaging] Run BATS upgrade tests against the current version (#20453)
BATS upgrade tests fails on master branch because it tries to install 2.x versions to upgrade from instead of 5.x versions. And since #18554 we should only test upgrades from 5.0.0-alpha4 versions.

This commit changes the vagrant tests so that it tries to list all the previous releases from version N-1. If nothing is found, it will fetch the current version and will run the upgrade tests with it. It works nicely with the current master 6.0.0-alpha1-SNAPSHOT. Once 5.0.0 is released it should run the test with it.
2016-09-14 18:26:03 +02:00
Tanguy Leroux 3caaafa4bd [Package] Remove bin/lib/modules directories on RPM uninstall/upgrade (#20448)
When uninstalling or upgrading elasticsearch using the RPM package some empty directories remain on the filesystem:

    /usr/share/elasticsearch/bin
    /usr/share/elasticsearch/lib
    /usr/share/elasticsearch/modules
    /usr/share/elasticsearch/modules/foo

Having empty directories in modules can prevent elasticsearch to start after an upgrade: the plugins service expects to find a plugin-descriptor.properties file in every sub directory of modules.

This PR cleans things a bit so that these empty directories are removed on upgrade/removal like it was in 2.x.
2016-09-14 18:18:37 +02:00
Tanguy Leroux 26dc6f1306 [Packaging] Do not remove scripts directory on upgrade (#20452)
When upgrading elasticsearch using the RPM package, the scripts directory is removed if it's empty but it won't be recreated by the upgraded package. But after that the service won't start because the scripts dir is missing.
2016-09-14 10:11:27 +02:00
Ali Beyad 4431720c3d File-based discovery plugin (#20394)
This commit introduces a new plugin for file-based unicast hosts
discovery. This allows specifying the unicast hosts participating
in discovery through a `unicast_hosts.txt` file located in the
`config/discovery-file` directory. The plugin will use the hosts 
specified in this file as the set of hosts to ping during discovery.

The format of the `unicast_hosts.txt` file is to have one host/port
entry per line. The hosts file is read and parsed every time
discovery makes ping requests, thus a new version of the file that
is published to the config directory will automatically be picked
up.

Closes #20323
2016-09-13 20:52:39 -04:00
Tanguy Leroux 6090c51fc5 Add quiet option to disable console logging (#20422)
This commit adds a -q/--quiet option to Elasticsearch so that it does not log anything in the console and closes stdout & stderr streams. This is useful for SystemD to avoid duplicate logs in both journalctl and /var/log/elasticsearch/elasticsearch.log while still allows the JVM to print error messages in stdout/stderr if needed.

closes #17220
2016-09-13 14:08:24 +02:00
Lee Hinman 3c10f90b47 [TEST] Fix plugin list comparison
The plugin command now displays the version of the plugin, which is
compared to a string without the version. This removes the version from
the string.
2016-09-12 17:02:44 -06:00
Jason Tedor 27ff4f327c Remove allow unquoted JSON
Previous versions of Elasticsearch permitted unquoted JSON field names even though this is against the JSON spec. This leniency was disabled by default in the 5.x series of Elasticsearch but a backwards compatibility layer was added via a system property with the intention of removing this layer in 6.0.0. This commit removes this backwards compatibility layer.

Relates #20388
2016-09-08 13:36:31 -04:00
Jason Tedor 07d1a72395 Update packaging tests for Log4j 2
This commit updates the packaging tests for Log4j 2. Namely, these tests
make assertions about logging.yml that should now be about
log4j2.properties.
2016-08-31 15:52:28 -04:00
Nik Everett 3b8701199f Fix packaging tests to test upgrading on disk groovy script 2016-08-23 11:55:11 -04:00
Nik Everett f084469c27 Fix syntax for template tests 2016-07-13 11:34:06 -04:00
Nik Everett e9d292b450 Fix names of tar tests 2016-07-13 11:33:55 -04:00
Jason Tedor a13717633c Fix typo in name of remove ingest-user-agent test
This commit fixes a simple typo in the name of the test for testing the
removal of the ingest-user-agent plugin.
2016-07-07 09:09:11 -04:00
Martijn van Groningen b4defafcb2 ingest: Renamed from `ingest-useragent` to `ingest-user-agent` and processor from `useragent` to `user_agent`
and on some other places did similar renaming. This is consistent with ES naming. Also made sure that the docs are navigable from the reference guide.
2016-07-07 09:43:43 +02:00
Nik Everett b66fc308fc Add ingest-useragent to vagrant tests 2016-07-01 11:29:48 -04:00
Jason Tedor a24d302e73 Fix discovery-azure-classic plugin packaging test
This commit fixes the discovery-azure-classing packaging test by fixing
the expected name of the installed plugin.
2016-06-30 14:47:38 -04:00
David Pilato 527a9c7f48 Deprecate discovery-azure and rename it to discovery-azure-classic
As discussed at https://github.com/elastic/elasticsearch-cloud-azure/issues/91#issuecomment-229113595, we know that the current `discovery-azure` plugin only works with Azure Classic VMs / Services (which is somehow Legacy now).

The proposal here is to rename `discovery-azure` to `discovery-azure-classic` in case some users are using it.
And deprecate it for 5.0.

Closes #19144.
2016-06-30 14:42:40 +02:00
Alexander Reelsen 8b43480b94 Tests: Fix vagrant tests to ignore progress bar in assertions 2016-06-30 09:08:41 +02:00
Jason Tedor 30b8a51eb9 Fix lang-painless packaging test
This commit fixes a failing lang-painless packaging test after a change
to the dependencies was made in commit
0bfb166eeb.
2016-06-13 10:37:21 -04:00
Nik Everett 06d26635de Packaging tests: ingest-grok is now ingest-common 2016-06-08 14:03:48 -04:00
Jason Tedor f16f65741e Fix when plugins directory is symlink
This commit fixes an issue with the plugins directory being a symbolic
link. Namely, the install plugins command attempts to always create the
plugins directory just in case it does not exist. The JDK method used
here guarantees that the directory is created, and an exception is not
thrown if the directory could not be created because it already
exists. The problem is that this JDK method does not respect symlinks so
its internal existence checks fails, it proceeds to attempt to create
the directory, but the directory creation fails because the symlink
exists. This is documented as being not an issue. We work around this by
checking if there is a symlink where we expect the plugins directory to
be, and only attempt to create if not. We add a unit test that plugin
installation to a symlinked plugins directory works as expected.
2016-05-26 14:10:32 -04:00
Jason Tedor d29844e597 Remove custom plugins path
This commit removes the ability to specify a custom plugins
path. Instead, the plugins path will always be a subdirectory called
"plugins" off of the home directory.
2016-05-26 10:16:25 -04:00
Jason Tedor 84dfa360b1 Fix list of poi dependencies for ingest-attachment
This commit fixes the list of poi dependencies for the ingest-attachment
plugin after they were upgraded in
1d40c4bbc1.
2016-05-24 17:03:25 -04:00
Tanguy Leroux 1f011f9dea Remove Delete-By-Query plugin
closes #18469
2016-05-24 13:28:20 +02:00
Jason Tedor c257e2c51f Remove settings and system properties entanglement
Today when parsing settings during bootstrap, we add a system property
for every Elasticsearch setting. Additionally, settings can be set via
system properties. This commit simplifies this situation.
 - settings are no longer propogated to system properties
 - system properties can not be used to set settings
 - the "es." prefix on settings is no longer required (nor permitted)
 - test logging has a dedicated system property (tests.logger.level)

Relates #18198
2016-05-19 14:08:08 -04:00
Tanguy Leroux 35d3bdab84 Add Google Cloud Storage repository plugin
Closes #12880
2016-05-19 13:26:23 +02:00
Jason Tedor db4809d906 Remove last vestigates of /bin/sh shebangs
This commit removes the remaining /bin/sh shebangs in favor of
/bin/bash.

Relates #18448
2016-05-18 11:03:00 -04:00
Lee Hinman ff5c7965ed Don't run `mkdir` when $DATA_DIR contains a comma-separated list
Resolves #16992
Resolves https://github.com/elastic/cookbook-elasticsearch/issues/441
2016-05-17 13:42:24 -06:00
Jason Tedor 9ce96f5792 Add tests that packages depend on bash
This commit adds bats tests that the RPM and Debian packages depend on
bash.

Relates #18292
2016-05-12 09:04:24 -04:00
Jason Tedor 7d1fd17172 Remove plugin script parsing of system properties
The plugin script parses command-line options looking for Java system
properties and extracts these arguments to pass to the java command when
starting the JVM. Since elasticsearch-plugin allows arbitrary user
arguments to the JVM via ES_JAVA_OPTS, this parsing is unnecessary. This
commit removes this unnecessary 

Relates #18207
2016-05-09 13:06:18 -04:00
Jason Tedor 3f3fa59406 Account for rpm behavior difference on directories
This commit modifies the packaging tests to account for the fact that
rpm behaves differently with respect to preserving directories marked as
"CONFIG | NOREPLACE" on older versions versus newer versions. Older
versions will leave the directory as-is while newer versions will append
the suffix ".rpmsave" to the directory name.

Relates #18216
2016-05-09 12:25:52 -04:00
Jason Tedor 0eaa831f48 Preserve config files from RPM install
This commit modifies the packaging for the RPM package so that edits to
config files will not get lost during removal and upgrade.

Relates #18188
2016-05-06 13:24:54 -04:00
Jason Tedor 4c6cf7ee88 Pass ES_JAVA_OPTS to JVM for plugins script
This commit adds support for ES_JAVA_OPTS to the elasticsearch-plugin
script.

Relates #18140
2016-05-04 12:48:39 -04:00
Lee Hinman 9da88a99da Fix exit code
Exit with proper exit code (1) and an error message if elasticsearch
executable binary does not exists or has insufficient permissions to
execute.

Squashed commit of the following:

commit 9768d316303418ba4f9c96d3f87c376048a1b1bc
Author: Puru <tuladharpuru@gmail.com>
Date:   Fri Apr 22 23:26:47 2016 +0545

    Fixed ES_HOME typo

commit 79a2b0394297f8b02b6f71b71ba35ff79f1a684e
Author: Puru <tuladharpuru@gmail.com>
Date:   Sun Apr 10 11:00:24 2016 +0545

    Improve elasticsearch startup script test

    Added improvement as per conversation in https://github.com/elastic/elasticsearch/pull/17082#issuecomment-206459613

commit 7be38e1fefd4baa6ccdbdc14745c00f6dc052e0c
Author: Puru <tuladharpuru@gmail.com>
Date:   Wed Mar 23 13:23:52 2016 +0545

    Add elasticsearch startup script test

    The test ensures that elasticsearch startup script exists and is executable.

commit d10eed5c08260fa9c158a4487bbb3103a8d867ed
Author: Puru <tuladharpuru@gmail.com>
Date:   Wed Mar 23 12:30:25 2016 +0545

    Fixed IF syntax and failure message

commit 6dc66f616545572485b4d43bee05a4cbbf1bed72
Author: Puru <tuladharpuru@gmail.com>
Date:   Sat Mar 12 11:08:11 2016 +0545

    Fix exit code

    Exit with proper exit code (1) and an error message if elasticsearch executable binary does not exists or has insufficient permissions to execute.
2016-04-22 11:48:05 -06:00
Lee Hinman 4fca5f734a Explicitly set packaging permissions
This changes our packaging to be explicit about the permissions of files
and directories in the tar.gz, rpm, and deb packages. This is to protect
against a user having an incorrectly set umask when installing.

Additionally, plugins that are installed now have their permissions set
by the plugin installation so that plugins that may have been packaged
with incorrect permissions are secured.

Resolves #17634
2016-04-21 12:30:56 -06:00
Lee Hinman a1e8fb794c Allow JSON with unquoted field names by enabling system property
In Elasticsearch 5.0.0, by default unquoted field names in JSON will be
rejected. This can cause issues, however, for documents that were
already indexed with unquoted field names. To alleviate this, a system
property has been added that can be enabled so migration can occur.

This system property will be removed in Elasticsearch 6.0.0

Resolves #17674
2016-04-19 09:14:13 -06:00
Jason Tedor a581d7cca4 Merge pull request #17675 from jasontedor/java-opts
Add JVM options configuration file
2016-04-12 23:07:40 -04:00
Jason Tedor 3879aa2a98 Add JVM options configuration file
This commit adds a new configuration file jvm.options to centralize and
simplify management of JVM options. This separates the configuration of
the JVM from the packaging scripts (bin/elasticsearch*, bin/service.bat,
and init.d/elasticsearch) simplifying end-user operational management of
custom JVM options.
2016-04-12 11:19:16 -04:00
Lee Hinman dd8afc9567 [TEST] Add extra logging for vagrant test output
The $user, $group, and $privileges are echoed when they are being
compared.
2016-04-11 09:36:50 -06:00
Jason Tedor eb7ecb7296 Add test of plugin script if JAVA_HOME has a space 2016-04-06 13:28:56 -04:00
Jason Tedor 85b06f48cf Revert "Add debug logging for Vagrant upgrade test"
This reverts commit 7ecfa6e2ad.
2016-03-24 21:59:32 -04:00
Jason Tedor a0634b2442 Merge pull request #17197 from jasontedor/chown
Elasticsearch ownership for data, logs, and configs
2016-03-24 21:21:11 -04:00
Jason Tedor d22b5977de Ownership for data, logs, and configs for packages
This commit ensures that the data, logs, and config directories have the
proper ownership after the packages are installed. Additionally, this
commit ensures that the configs in /etc/elasticsearch are preserved
after removal of the RPM package.
2016-03-24 20:56:02 -04:00
Alexander Reelsen baec7bc23c Tests: Fix vagrant test on debian
Debian asks during installation, if the configuration file should be updated.
This is asked via a prompt and thus hangs.

This adds an option to always update to the newer config file, so automated
installation keeps working.
2016-03-24 14:21:33 +01:00
Simon Willnauer e4e08b28ff [TEST] Use to wait for the node to start - it will wait for state recovery and not return 503 2016-03-24 11:06:40 +01:00