Commit Graph

254 Commits

Author SHA1 Message Date
Ryan Ernst 4735b5a5fa Build: Use license, notice and readme files from root for distributions
The distributions had their own copies of these extra files, which was a
carry over from maven. This change removes the duplicate files and
copies them from the root of the project.

closes #18597
2016-05-26 09:19:38 -07:00
Ryan Ernst d505c3d5ba Build: Use dash for deb package version and underscore for rpm
This change tweaks the metadata version associated with the deb and rpm
packages. For rpm, dashes are allowed in versions, so we can use the
version as it already exists. For rpm, dashes are not allowed, so this
uses underscores instead. This only affects prerelease versions (eg
alpha/beta/rc), and usually someone doesn't specify the version, so the
inconsistency doesn't matter that much.
2016-05-24 12:11:13 -07:00
Lee Hinman 688254de41 Don't mkdir directly in deb init script
Elasticsearch will create the log and data path directories when it
starts, there is no need to do this in the init script

Resolves #18307
2016-05-20 15:52:16 -06:00
Jason Tedor 452faa220c Filter client/server VM options from jvm.options
The default jvm.options file ships with the -server flag to force the
server VM on systems where the server VM is not the default. However,
the method of starting the JVM via the Windows service does not support
the command-line flags for selecting the VM because it starts from a
DLL that is specific to the server or client VM. Thus, we need to
filter these options from the jvm.options configuration file when
installing the Windows service.

Relates #18473
2016-05-20 15:25:45 -04:00
Jason Tedor c257e2c51f Remove settings and system properties entanglement
Today when parsing settings during bootstrap, we add a system property
for every Elasticsearch setting. Additionally, settings can be set via
system properties. This commit simplifies this situation.
 - settings are no longer propogated to system properties
 - system properties can not be used to set settings
 - the "es." prefix on settings is no longer required (nor permitted)
 - test logging has a dedicated system property (tests.logger.level)

Relates #18198
2016-05-19 14:08:08 -04:00
Jason Tedor db4809d906 Remove last vestigates of /bin/sh shebangs
This commit removes the remaining /bin/sh shebangs in favor of
/bin/bash.

Relates #18448
2016-05-18 11:03:00 -04:00
Lee Hinman 4003d3f077 Merge remote-tracking branch 'dakrone/dont-mkdir-data-with-comma' 2016-05-17 13:56:59 -06:00
Lee Hinman ff5c7965ed Don't run `mkdir` when $DATA_DIR contains a comma-separated list
Resolves #16992
Resolves https://github.com/elastic/cookbook-elasticsearch/issues/441
2016-05-17 13:42:24 -06:00
Jason Tedor 10a881d860 Increase default heap size to 2g
Increase default heap size to 2g
2016-05-14 07:52:27 -04:00
Ryan Ernst 17de294460 Merge pull request #18344 from rjernst/run_distro
Build: Make run task you full zip distribution
2016-05-13 17:00:41 -07:00
Ryan Ernst d6f3d85925 Build: Make run task you full zip distribution
This change makes `gradle run` use the full zip distribution. Arguably
we could make it do the same when run inside plugins, but I started out
with this simple change. I am open to moving it into the RunTask itself
so that plugins will do this as well.
2016-05-13 16:36:26 -07:00
Ryan Ernst 5aea24137c Build: More pom generation improvements
This adds a non empty description to all generated poms, as well as
fixing distributions so they actually have a generated pom.
2016-05-13 16:32:35 -07:00
Jason Tedor bc46517f7e Restore unequal min and max default heap
After considerable discussion, we have elected to set the default min
heap to 256m and the default max heap to 2g. This is to balance the
desire for a good out-of-the-box performance experience (default max
heap of 2g) with a good out-of-the-box experience running on machines
with limited resources or running multiple instances on a single modern
developer laptop (default min heap of 1g).
2016-05-13 07:07:56 -04:00
Jason Tedor 52b09bc803 Increase default heap size to 2g
Benchmarking with the geonames data set shows that Elasticsearch truly
needs 2g of heap or the heap is a bottleneck for indexing rates. If our
goal is to satisfy the out-of-the-box-experience, we should prioritize
the out-of-the-box performance experience. Thus, the default heap should
be 2g until smaller heaps are not the bottleneck for indexing small data
sets.
2016-05-12 21:04:31 -04:00
Jason Tedor f9667dbe33 Switch init.d scripts to use bash
This commit modifies the init.d scripts to use bash now that bash is a
required dependency.

Relates #18308
2016-05-12 18:13:04 -04:00
Jason Tedor 8adfd60079 Fix Debian package bash requires clause
This commit fixes the Debian package requires clause for bash. For the
RPM it is okay to specify the binary, but for the Debian package the
package name must be specified.
2016-05-11 20:05:09 -04:00
Jason Tedor e4edee5d9a Equal default min and max heap settings
Today we encourage users to set their minimum and maximum heap settings
equal to each other to prevent the heap from resizing. Yet, the default
heap settings do not start Elasticsearch in this fashion. This commit
addresses this discrepancy by setting the default min heap to '512m' and
the default max heap to the default min heap.

Relates #16334
2016-05-11 10:48:08 -04:00
Jason Tedor 2bf585e642 Require /bin/bash in packaging
This commit adds a hard requirement to the RPM and Debian packages for
/bin/bash to be present, and adds a note regarding this to the migration
docs.

Relates #18259
2016-05-10 21:17:09 -04:00
Jason Tedor 1dfbe8b0f1 Fix handling of default path conf in plugin script
In the plugin script we set the setting default.path.conf to control the
path to the configuration file. But the default directory might not
exist in which case we can run into exceptions. This commit restores a
guard against this directory not existing.
2016-05-10 15:32:23 -04:00
Jason Tedor ef6a7631cf Switch scripts to use bash
This commit switches the command-line scripts to use bash instead of sh
so that we can take advantage of features that bash provides like
arrays.

Relates #18251
2016-05-10 15:06:51 -04:00
Jason Tedor 557fc8b437 Further simplifications of plugin script
In 7d1fd17172 the parsing of command-line
properties in the plugin script was removed. That commit missed
additional parsing of the properties for es.default.path.conf. This
commit removes that parsing and also replaces the use of eval with exec.

Relates #18239
2016-05-10 10:08:58 -04:00
Jason Tedor 7d1fd17172 Remove plugin script parsing of system properties
The plugin script parses command-line options looking for Java system
properties and extracts these arguments to pass to the java command when
starting the JVM. Since elasticsearch-plugin allows arbitrary user
arguments to the JVM via ES_JAVA_OPTS, this parsing is unnecessary. This
commit removes this unnecessary 

Relates #18207
2016-05-09 13:06:18 -04:00
Ryan Ernst 3d1be071c9 Merge branch 'master' into pom_gen 2016-05-06 12:56:51 -07:00
Jason Tedor 0eaa831f48 Preserve config files from RPM install
This commit modifies the packaging for the RPM package so that edits to
config files will not get lost during removal and upgrade.

Relates #18188
2016-05-06 13:24:54 -04:00
Ryan Ernst e16af604bf Build: Add pom generation to assemble task
In preparation for a unified release process, we need to be able to
generate the pom files independently of trying to actually publish. This
change adds back the maven-publish plugin just for that purpose. The
nexus plugin still exists for now, so that we do not break snapshots,
but that can be removed at a later time once snapshots are happenign
through the unified tools. Note I also changed the dir jars are written
into so that all our artifacts are under build/distributions.
2016-05-05 17:57:44 -07:00
Radovan Ondas 443976d03c Fix typo in message for variable setup ES_MAX_MEM
Small typo fix in startup script.

Relates #18168
2016-05-05 19:17:31 -04:00
Jason Tedor f1fb6a37c0 Fix error message on ES_MAX_MEM set
This commit fixes a typo in the error message when the now unsupported
environment variable ES_MAX_MEM is set on Windows.
2016-05-05 19:16:57 -04:00
Ryan Ernst 8ff699e41a Packaging: Make rpm not include parent dirs
closes #18162
2016-05-05 14:22:14 -07:00
Jason Tedor e11b96ca9c Default to server VM and add client VM check
Today we softly warn about running with the client VM. However, we
should really refuse to start in production mode if running with the
client VM as the performance of the client VM is too devastating for a
server application. This commit adds an option to jvm.options to ensure
that we are starting with the server VM (on all 32-bit non-Windows
platforms on server-class machines (2+ CPUs, 2+ GB physical RAM) this is
the default and on all 64-bit platforms this is the only option) and
adds a bootstrap check for the client VM.

Relates #18155
2016-05-05 10:36:21 -04:00
Jason Tedor 124e8e5a6d Remove client option for JVM for plugins script
Today we specify the client option for the JVM when executing plugin
commands. Yet, this option does nothing on a 64-bit capable JDK as such
JDKs always select the Hotspot server VM. And for 32-bit JDKs, running
plugin commands with the server VM is okay. Thus, we should just remove
this unnecessary flag and just let the default VM be selected.

Relates #18142
2016-05-04 14:11:29 -04:00
Jason Tedor 4c6cf7ee88 Pass ES_JAVA_OPTS to JVM for plugins script
This commit adds support for ES_JAVA_OPTS to the elasticsearch-plugin
script.

Relates #18140
2016-05-04 12:48:39 -04:00
Alexander Reelsen 9cadc1f40b Packaging: Fix vagrant tests on centos 6 (#17979)
The RPM init script did not include the check for `bin/elasticsearch` being
executable. This fix adds this checks and makes the tests pass.

This fixes the init script, so the tests pass on centos-6 again.
2016-04-26 11:55:50 +02:00
Jason Tedor 8a5032fae5 Remove unnecessary sleep from init script restart
Today when restarting Elasticsearch using the start-stop-daemon on
Debian-based systems using System V init, we sleep for one second
between the process successfully stopping and starting the process
again. This sleep is unnecessary as the stop function retries forever
until the previous instance successfully terminates. This commit removes
that unncessary sleep.

Relates #17966
2016-04-25 22:50:18 -04:00
Lee Hinman 9da88a99da Fix exit code
Exit with proper exit code (1) and an error message if elasticsearch
executable binary does not exists or has insufficient permissions to
execute.

Squashed commit of the following:

commit 9768d316303418ba4f9c96d3f87c376048a1b1bc
Author: Puru <tuladharpuru@gmail.com>
Date:   Fri Apr 22 23:26:47 2016 +0545

    Fixed ES_HOME typo

commit 79a2b0394297f8b02b6f71b71ba35ff79f1a684e
Author: Puru <tuladharpuru@gmail.com>
Date:   Sun Apr 10 11:00:24 2016 +0545

    Improve elasticsearch startup script test

    Added improvement as per conversation in https://github.com/elastic/elasticsearch/pull/17082#issuecomment-206459613

commit 7be38e1fefd4baa6ccdbdc14745c00f6dc052e0c
Author: Puru <tuladharpuru@gmail.com>
Date:   Wed Mar 23 13:23:52 2016 +0545

    Add elasticsearch startup script test

    The test ensures that elasticsearch startup script exists and is executable.

commit d10eed5c08260fa9c158a4487bbb3103a8d867ed
Author: Puru <tuladharpuru@gmail.com>
Date:   Wed Mar 23 12:30:25 2016 +0545

    Fixed IF syntax and failure message

commit 6dc66f616545572485b4d43bee05a4cbbf1bed72
Author: Puru <tuladharpuru@gmail.com>
Date:   Sat Mar 12 11:08:11 2016 +0545

    Fix exit code

    Exit with proper exit code (1) and an error message if elasticsearch executable binary does not exists or has insufficient permissions to execute.
2016-04-22 11:48:05 -06:00
Lee Hinman 4fca5f734a Explicitly set packaging permissions
This changes our packaging to be explicit about the permissions of files
and directories in the tar.gz, rpm, and deb packages. This is to protect
against a user having an incorrectly set umask when installing.

Additionally, plugins that are installed now have their permissions set
by the plugin installation so that plugins that may have been packaged
with incorrect permissions are secured.

Resolves #17634
2016-04-21 12:30:56 -06:00
Lee Hinman b8899cdb78 Merge remote-tracking branch 'dakrone/allow-bad-json' 2016-04-19 10:02:53 -06:00
Lee Hinman a1e8fb794c Allow JSON with unquoted field names by enabling system property
In Elasticsearch 5.0.0, by default unquoted field names in JSON will be
rejected. This can cause issues, however, for documents that were
already indexed with unquoted field names. To alleviate this, a system
property has been added that can be enabled so migration can occur.

This system property will be removed in Elasticsearch 6.0.0

Resolves #17674
2016-04-19 09:14:13 -06:00
Danilo Vaz 2e2d8c1442 Updated copyright years to include 2016 (#17808) 2016-04-18 12:39:23 +02:00
Alexander Reelsen 95579ca95a Build: Allow for file based deploy, sign packages
This allows for a local file based deploy without needed nexus
auth information.

Also signing of packages has been added, either via gradle.properties
or using system properties as a fallback.

The property build.repository allows to configure another endpoint if no
snapshot build is done.

Fix creation of .asc file for tar.gz distribution

Closes #17405
2016-04-15 17:14:05 +02:00
Jason Tedor 7297580170 Simplify JVM options
This commit simplifies the default JVM options that ship with
Elasticsearch. In particular, expert settings that were previously
configurable via environment variables have been removed from the
default configuration file. Further, the heap size settings have been
moved to the top of the file with a clearer message that is in
concordance with their importance.

Closes #17714
2016-04-13 17:59:59 -04:00
Jason Tedor f4ccf9bd4f Set variable in Windows scripts to default value
This commit sets the bad_env_var to a default value of 0 so that in case
no bad environment variables are encountered the variable is still
defined and does not later cause a parse error in the execution of these
batch files.

Relates #17675
2016-04-13 12:17:36 -04:00
Nik Everett bf2483e4b3 Setup jvm opts correctly for deb tests
Also stops warning about JAVA_OPTS when it is an empty string.
2016-04-13 11:18:19 -04:00
Jason Tedor a581d7cca4 Merge pull request #17675 from jasontedor/java-opts
Add JVM options configuration file
2016-04-12 23:07:40 -04:00
Jason Tedor 5662a14d22 Rewrite if-statement for shell compatibility
This commit rewrites an if-statement in bin/elasticsearch for
compatibility with some shells (e.g., dash on Ubuntu).
2016-04-12 22:27:08 -04:00
Jason Tedor 27867c729d Additional handling of ES_GC_LOG_FILE
This commit adds handling of the old ES_GC_LOG_FILE environment variable
to treat it like the other removed environment variables.
2016-04-12 22:26:43 -04:00
Jason Tedor bdfa063946 Add migration messages when rejecting env. vars
This commit adds migration messages to the startup scripts when
rejecting the old unsupported environment variables.
2016-04-12 15:32:31 -04:00
Jason Tedor d4cf0f021a Fail if unsupported environment variables are set
This commit adds a hard failure on startup if any unsupported
environment variables from older versions of Elasticsearch are still
set.
2016-04-12 13:29:09 -04:00
Jason Tedor 2b1c191b12 Add comments to JVM options parsing for Windows
This commit adds some comments to the portion of bin/elasticsearch.bat
that is responsible for parsing the JVM options.
2016-04-12 12:37:15 -04:00
Jason Tedor 7743c422db Warn in bin/elasticsearch if JAVA_OPTS is set
JAVA_OPTS is not a built-in mechanism for passing options to the JVM but
many people think that it is. We do not respect JAVA_OPTS, but this
commit adds a warning if it is set in case the end-user thinks that it
will affect Elasticsearch.
2016-04-12 11:51:30 -04:00
Jason Tedor 9460289bee Rename function that parses JVM options
This commit renames the function that parses JVM options in
bin/elasticsearch from jvm_options to parse_jvm_options. The reason for
the rename is because a for-loop variable was shadowing the name of this
function and changing the function name further clarifies the purpose of
the function.
2016-04-12 11:42:06 -04:00