Commit Graph

330 Commits

Author SHA1 Message Date
Ryan Ernst e16af604bf Build: Add pom generation to assemble task
In preparation for a unified release process, we need to be able to
generate the pom files independently of trying to actually publish. This
change adds back the maven-publish plugin just for that purpose. The
nexus plugin still exists for now, so that we do not break snapshots,
but that can be removed at a later time once snapshots are happenign
through the unified tools. Note I also changed the dir jars are written
into so that all our artifacts are under build/distributions.
2016-05-05 17:57:44 -07:00
Radovan Ondas 443976d03c Fix typo in message for variable setup ES_MAX_MEM
Small typo fix in startup script.

Relates #18168
2016-05-05 19:17:31 -04:00
Jason Tedor f1fb6a37c0 Fix error message on ES_MAX_MEM set
This commit fixes a typo in the error message when the now unsupported
environment variable ES_MAX_MEM is set on Windows.
2016-05-05 19:16:57 -04:00
Ryan Ernst 8ff699e41a Packaging: Make rpm not include parent dirs
closes #18162
2016-05-05 14:22:14 -07:00
Jason Tedor e11b96ca9c Default to server VM and add client VM check
Today we softly warn about running with the client VM. However, we
should really refuse to start in production mode if running with the
client VM as the performance of the client VM is too devastating for a
server application. This commit adds an option to jvm.options to ensure
that we are starting with the server VM (on all 32-bit non-Windows
platforms on server-class machines (2+ CPUs, 2+ GB physical RAM) this is
the default and on all 64-bit platforms this is the only option) and
adds a bootstrap check for the client VM.

Relates #18155
2016-05-05 10:36:21 -04:00
Jason Tedor 124e8e5a6d Remove client option for JVM for plugins script
Today we specify the client option for the JVM when executing plugin
commands. Yet, this option does nothing on a 64-bit capable JDK as such
JDKs always select the Hotspot server VM. And for 32-bit JDKs, running
plugin commands with the server VM is okay. Thus, we should just remove
this unnecessary flag and just let the default VM be selected.

Relates #18142
2016-05-04 14:11:29 -04:00
Jason Tedor 4c6cf7ee88 Pass ES_JAVA_OPTS to JVM for plugins script
This commit adds support for ES_JAVA_OPTS to the elasticsearch-plugin
script.

Relates #18140
2016-05-04 12:48:39 -04:00
Alexander Reelsen 9cadc1f40b Packaging: Fix vagrant tests on centos 6 (#17979)
The RPM init script did not include the check for `bin/elasticsearch` being
executable. This fix adds this checks and makes the tests pass.

This fixes the init script, so the tests pass on centos-6 again.
2016-04-26 11:55:50 +02:00
Jason Tedor 8a5032fae5 Remove unnecessary sleep from init script restart
Today when restarting Elasticsearch using the start-stop-daemon on
Debian-based systems using System V init, we sleep for one second
between the process successfully stopping and starting the process
again. This sleep is unnecessary as the stop function retries forever
until the previous instance successfully terminates. This commit removes
that unncessary sleep.

Relates #17966
2016-04-25 22:50:18 -04:00
Lee Hinman 9da88a99da Fix exit code
Exit with proper exit code (1) and an error message if elasticsearch
executable binary does not exists or has insufficient permissions to
execute.

Squashed commit of the following:

commit 9768d316303418ba4f9c96d3f87c376048a1b1bc
Author: Puru <tuladharpuru@gmail.com>
Date:   Fri Apr 22 23:26:47 2016 +0545

    Fixed ES_HOME typo

commit 79a2b0394297f8b02b6f71b71ba35ff79f1a684e
Author: Puru <tuladharpuru@gmail.com>
Date:   Sun Apr 10 11:00:24 2016 +0545

    Improve elasticsearch startup script test

    Added improvement as per conversation in https://github.com/elastic/elasticsearch/pull/17082#issuecomment-206459613

commit 7be38e1fefd4baa6ccdbdc14745c00f6dc052e0c
Author: Puru <tuladharpuru@gmail.com>
Date:   Wed Mar 23 13:23:52 2016 +0545

    Add elasticsearch startup script test

    The test ensures that elasticsearch startup script exists and is executable.

commit d10eed5c08260fa9c158a4487bbb3103a8d867ed
Author: Puru <tuladharpuru@gmail.com>
Date:   Wed Mar 23 12:30:25 2016 +0545

    Fixed IF syntax and failure message

commit 6dc66f616545572485b4d43bee05a4cbbf1bed72
Author: Puru <tuladharpuru@gmail.com>
Date:   Sat Mar 12 11:08:11 2016 +0545

    Fix exit code

    Exit with proper exit code (1) and an error message if elasticsearch executable binary does not exists or has insufficient permissions to execute.
2016-04-22 11:48:05 -06:00
Lee Hinman 4fca5f734a Explicitly set packaging permissions
This changes our packaging to be explicit about the permissions of files
and directories in the tar.gz, rpm, and deb packages. This is to protect
against a user having an incorrectly set umask when installing.

Additionally, plugins that are installed now have their permissions set
by the plugin installation so that plugins that may have been packaged
with incorrect permissions are secured.

Resolves #17634
2016-04-21 12:30:56 -06:00
Lee Hinman b8899cdb78 Merge remote-tracking branch 'dakrone/allow-bad-json' 2016-04-19 10:02:53 -06:00
Lee Hinman a1e8fb794c Allow JSON with unquoted field names by enabling system property
In Elasticsearch 5.0.0, by default unquoted field names in JSON will be
rejected. This can cause issues, however, for documents that were
already indexed with unquoted field names. To alleviate this, a system
property has been added that can be enabled so migration can occur.

This system property will be removed in Elasticsearch 6.0.0

Resolves #17674
2016-04-19 09:14:13 -06:00
Danilo Vaz 2e2d8c1442 Updated copyright years to include 2016 (#17808) 2016-04-18 12:39:23 +02:00
Alexander Reelsen 95579ca95a Build: Allow for file based deploy, sign packages
This allows for a local file based deploy without needed nexus
auth information.

Also signing of packages has been added, either via gradle.properties
or using system properties as a fallback.

The property build.repository allows to configure another endpoint if no
snapshot build is done.

Fix creation of .asc file for tar.gz distribution

Closes #17405
2016-04-15 17:14:05 +02:00
Jason Tedor 7297580170 Simplify JVM options
This commit simplifies the default JVM options that ship with
Elasticsearch. In particular, expert settings that were previously
configurable via environment variables have been removed from the
default configuration file. Further, the heap size settings have been
moved to the top of the file with a clearer message that is in
concordance with their importance.

Closes #17714
2016-04-13 17:59:59 -04:00
Jason Tedor f4ccf9bd4f Set variable in Windows scripts to default value
This commit sets the bad_env_var to a default value of 0 so that in case
no bad environment variables are encountered the variable is still
defined and does not later cause a parse error in the execution of these
batch files.

Relates #17675
2016-04-13 12:17:36 -04:00
Nik Everett bf2483e4b3 Setup jvm opts correctly for deb tests
Also stops warning about JAVA_OPTS when it is an empty string.
2016-04-13 11:18:19 -04:00
Jason Tedor a581d7cca4 Merge pull request #17675 from jasontedor/java-opts
Add JVM options configuration file
2016-04-12 23:07:40 -04:00
Jason Tedor 5662a14d22 Rewrite if-statement for shell compatibility
This commit rewrites an if-statement in bin/elasticsearch for
compatibility with some shells (e.g., dash on Ubuntu).
2016-04-12 22:27:08 -04:00
Jason Tedor 27867c729d Additional handling of ES_GC_LOG_FILE
This commit adds handling of the old ES_GC_LOG_FILE environment variable
to treat it like the other removed environment variables.
2016-04-12 22:26:43 -04:00
Jason Tedor bdfa063946 Add migration messages when rejecting env. vars
This commit adds migration messages to the startup scripts when
rejecting the old unsupported environment variables.
2016-04-12 15:32:31 -04:00
Jason Tedor d4cf0f021a Fail if unsupported environment variables are set
This commit adds a hard failure on startup if any unsupported
environment variables from older versions of Elasticsearch are still
set.
2016-04-12 13:29:09 -04:00
Jason Tedor 2b1c191b12 Add comments to JVM options parsing for Windows
This commit adds some comments to the portion of bin/elasticsearch.bat
that is responsible for parsing the JVM options.
2016-04-12 12:37:15 -04:00
Jason Tedor 7743c422db Warn in bin/elasticsearch if JAVA_OPTS is set
JAVA_OPTS is not a built-in mechanism for passing options to the JVM but
many people think that it is. We do not respect JAVA_OPTS, but this
commit adds a warning if it is set in case the end-user thinks that it
will affect Elasticsearch.
2016-04-12 11:51:30 -04:00
Jason Tedor 9460289bee Rename function that parses JVM options
This commit renames the function that parses JVM options in
bin/elasticsearch from jvm_options to parse_jvm_options. The reason for
the rename is because a for-loop variable was shadowing the name of this
function and changing the function name further clarifies the purpose of
the function.
2016-04-12 11:42:06 -04:00
Jason Tedor 00033b3da3 Fix placement of comment in bin/elasticsearch
This commit fixes the placement of a comment in bin/elasticsearch. The
comment was made out of place by the addition of a function definition
but order is restored.
2016-04-12 11:39:33 -04:00
Jason Tedor 3879aa2a98 Add JVM options configuration file
This commit adds a new configuration file jvm.options to centralize and
simplify management of JVM options. This separates the configuration of
the JVM from the packaging scripts (bin/elasticsearch*, bin/service.bat,
and init.d/elasticsearch) simplifying end-user operational management of
custom JVM options.
2016-04-12 11:19:16 -04:00
Adrien Grand 496c7fbd84 Upgrade Lucene 6 Release
* upgrades numerics to new Point format
* updates geo api changes
  * adds GeoPointDistanceRangeQuery as XGeoPointDistanceRangeQuery
  * cuts over to ES GeoHashUtils
2016-04-11 16:50:04 -05:00
Daniil Penkin 696fa18c61 Support for configuring Elasticsearch service display name, description and user on Windows 2016-04-07 10:39:57 +10:00
Lee Hinman 6e44ddaf0e Merge branch 'pr/11338' 2016-04-06 15:06:30 -06:00
Lee Hinman 42745c8c53 Merge pull request #14596 from girirajsharma/ES-RPM
rpm uses non-portable `--system` flag to `useradd`
2016-04-06 14:46:08 -06:00
Jason Tedor cea3c3c92f Quote path to java binary
This commit quotes the variable that contains the path to the java
binary. Without these quotes, when the arguments to eval are evaluated
the existing quotes will be removed leading to unquoted use of the path
to the java binary. If this path contains spaces, evaluation will fail.
2016-04-03 23:23:25 -04:00
Clinton Gormley 410147c754 Added RPM metadata 2016-04-02 10:38:34 +02:00
Joe Hillenbrand 81b6326891 Set MAX_OPEN_FILES to 65536
Relates to #17430
2016-03-30 16:44:37 -07:00
Mpdreamz e8776799cc Service command still had positional start command
Which caused the service to fail to start
2016-03-29 23:25:03 +02:00
Jason Tedor a0634b2442 Merge pull request #17197 from jasontedor/chown
Elasticsearch ownership for data, logs, and configs
2016-03-24 21:21:11 -04:00
Jason Tedor d22b5977de Ownership for data, logs, and configs for packages
This commit ensures that the data, logs, and config directories have the
proper ownership after the packages are installed. Additionally, this
commit ensures that the configs in /etc/elasticsearch are preserved
after removal of the RPM package.
2016-03-24 20:56:02 -04:00
Simon Willnauer 63b754aa22 Remove positional parameter from bat file 2016-03-24 11:19:25 +01:00
Alexander Reelsen b2573858b6 Version: Set version to 5.0.0-alpha1
Change version, required a minor fix in the RPM building.
In case of a alpha/beta version, the release will contain alpha/beta
as the RPM version cannot contains dashes/tildes.
2016-03-24 08:36:08 +01:00
Alexander Reelsen 4ac4f3c8bc Build: Update ospackage gradle plugin
The older version did not support signing of the packages.
2016-03-21 15:38:01 +01:00
Ryan Ernst f71f0d6010 Revert "Build: Switch to maven-publish plugin"
This reverts commit a90a2b34fc.
2016-03-18 17:22:25 -07:00
Jason Tedor 96c6ac2d3f Merge pull request #17176 from jasontedor/its-not-easy-being-green
Vagrant tests should be green
2016-03-18 15:28:37 -04:00
Jason Tedor 9f73152940 Fix plugins permissions 2016-03-17 15:35:54 -04:00
Ryan Ernst 6af4c43c4f Merge pull request #17128 from rjernst/maven_publish
Build: Switch to maven-publish plugin
2016-03-17 11:53:50 -07:00
Yannick Welsch e32da555aa Fix Windows start script to pass parameters in last position
Relates to #15320
2016-03-16 13:43:32 +01:00
Jason Tedor d2db9cf95f Fix es.path.home on Windows 2016-03-16 04:46:25 -04:00
Ryan Ernst a90a2b34fc Build: Switch to maven-publish plugin
The build currently uses the old maven support in gradle. This commit
switches to use the newer maven-publish plugin. This will allow future
changes, for example, easily publishing to artifactory.

An additional part of this change makes publishing of build-tools part
of the normal publishing, instead of requiring a separate upload step
from within buildSrc. That also sets us up for a follow up to enable
precomit checks on the buildSrc code itself.
2016-03-15 19:16:37 -07:00
Jason Tedor 618441aea3 Merge pull request #17088 from jasontedor/simplify-bootstrap-settings
Bootstrap does not set system properties
2016-03-15 19:25:16 -04:00
Jason Tedor 4ee90db13d Remove path.home command-line setting 2016-03-15 16:29:29 -04:00
Jason Tedor 5994e91b08 Fix systemd pidfile setting
This commit fixes the pidfile setting on systems that used systemd. The
issue is that the pidfile can only be set via the command line arguments
-p or --pidfile, and is no longer settable via a setting.
2016-03-15 15:57:37 -04:00
Adrien Grand 5596e31068 Upgrade to lucene-6.0.0-f0aa4fc. #17075 2016-03-14 07:58:52 +01:00
Jason Tedor 8a05c2a2be Bootstrap does not set system properties
Today, certain bootstrap properties are set and read via system
properties. This action-at-distance way of managing these properties is
rather confusing, and completely unnecessary. But another problem exists
with setting these as system properties. Namely, these system properties
are interpreted as Elasticsearch settings, not all of which are
registered. This leads to Elasticsearch failing to startup if any of
these special properties are set. Instead, these properties should be
kept as local as possible, and passed around as method parameters where
needed. This eliminates the action-at-distance way of handling these
properties, and eliminates the need to register these non-setting
properties. This commit does exactly that.

Additionally, today we use the "-D" command line flag to set the
properties, but this is confusing because "-D" is a special flag to the
JVM for setting system properties. This creates confusion because some
"-D" properties should be passed via arguments to the JVM (so via
ES_JAVA_OPTS), and some should be passed as arguments to
Elasticsearch. This commit changes the "-D" flag for Elasticsearch
settings to "-E".
2016-03-13 20:09:15 -04:00
Jason Tedor 4353b2e024 Do not pass double-dash arguments on startup
This commit addresses an issue in the init scripts which are passing
invalid command line arguments to the startup script.

Closes #17087
2016-03-13 18:58:08 -04:00
Ryan Ernst 591fb8f028 Merge branch 'master' into cli-parsing 2016-03-11 10:45:05 -08:00
Nik Everett 48191a44ab Merge pull request #15173 from rhoml/enable_es_include_on_init
Enable es_include at init
2016-03-10 14:31:50 -05:00
Lee Hinman 22e716551b Add -XX+AlwaysPreTouch JVM flag
Enables the touching of all memory pages used by the JVM heap spaces
during initialization of the HotSpot VM, which commits all memory pages
at initialization time. By default, pages are committed only as they are
needed.
2016-03-10 10:11:32 -07:00
Ryan Ernst 80ae2b0002 Fix more licenses 2016-03-09 00:10:59 -08:00
Ryan Ernst cb607a8fae Remove commons-cli sha and add jopt-simple sha 2016-03-08 19:18:14 -08:00
Robert Muir 54018a5d37 upgrade to lucene 6.0.0-snapshot-bea235f
Closes #16964

Squashed commit of the following:

commit a23f9d2d29220991aa498214530753d7a5a148c6
Merge: eec9c4e 0b0a251
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Mar 7 04:12:02 2016 -0500

    Merge branch 'master' into lucene6

commit eec9c4e5cd11e9c3e0b426f04894bb2a6dae4f21
Merge: bc67205 675d940
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 13:45:00 2016 -0500

    Merge branch 'master' into lucene6

commit bc67205bdfe1526eae277ab7856fc050ecbdb7b2
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 09:56:31 2016 -0500

    fix test bug

commit a60723b007ff12d97b1810cef473bd7b553a0327
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Mar 4 15:35:35 2016 +0100

    Fix SimpleValidateQueryIT to put braces around boosted terms

commit ae3a49d7ba7ced448d2a5262e5d8ec98671a9090
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Mar 4 15:27:25 2016 +0100

    fix multimatchquery

commit ae23fdb88a8f6d3fb7ba60fd1aaf3fd72d899aa5
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Mar 4 15:20:49 2016 +0100

    Rewrite DecayFunctionScoreIT to be independent of the similarity used

    This test relied a lot on the term scoring and compared scores
    that are dependent on the similarity. This commit changes the base query
    to be a predictable constant score query.

commit 366c2d518c35d31251033f1b6f6a93f6e2ae327d
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Mar 4 14:06:14 2016 +0100

    Fix scoring in tests due to changes to idf calculation.

    Lucene 6 uses a different default similarity as well as a different
    way to calculate IDF. In contrast to older version lucene 6 uses docCount per field
    to calculate the IDF not the # of docs in the index to overcome the sparse field
    cases.

commit dac99fd64ac2fa71b8d8d106fe68825e574c49f8
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 08:21:57 2016 -0500

    don't hardcoded expected termquery score

commit 6e9f340ba49ab10eed512df86d52a121aa775b0f
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 08:04:45 2016 -0500

    suppress deprecation warning until migrated to points

commit 3ac8908424b3fdad44a90a4f7bdb3eff7efd077d
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 07:21:43 2016 -0500

    Remove invalid test: all commits have IDs, and its illegal to do this.

commit c12976288124ad1a26467e7e848fb810548e7eab
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 07:06:14 2016 -0500

    don't test with unsupported back compat

commit 18bbfe76128570bc70883bf91ff4c44c82d27817
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 07:02:18 2016 -0500

    remove now invalid lucene 4 backcompat test

commit 7e730e572886f0ef2d3faba712e4256216ff01ec
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 06:58:52 2016 -0500

    remove now invalid lucene 4 backwards test

commit 244d2ab6868ba5ac9e0bcde3c2833743751a25ec
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 06:47:23 2016 -0500

    use 6.0 codec

commit 5f64d4a431a6fdaa1234adca23f154c2a1de8284
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 06:43:08 2016 -0500

    compile, javadocs, forbidden-apis, etc

commit 1f273cd62a7fe9ca8f8944acbbfc5cbdd3d81ccb
Merge: cd33921 29e3443
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Mar 4 10:45:29 2016 +0100

    Merge branch 'master' into lucene6

commit cd33921ac742ef9fb351012eff35f3c7dbda7264
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Mar 3 23:58:37 2016 -0500

    fix hunspell dictionary loading

commit c7fdbd837b01f7defe9cb1c24e2ec65604b0dc96
Merge: 4d4190f d8948ba
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Mar 3 23:41:53 2016 -0500

    Merge branch 'master' into lucene6

commit 4d4190fd82601aaafac6b8254ccb3edf218faa34
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Mar 3 23:39:14 2016 -0500

    remove nocommit

commit 77ca69e288b1a41aa9595c921ed166c272a00ea8
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Mar 3 23:38:24 2016 -0500

    clean up numericutils vs legacynumericutils

commit a466d696fbaad04b647ffbc0857a9439b583d0bf
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Mar 3 23:32:43 2016 -0500

    upgrade spatial4j

commit 5412c747a8cfe638bacedbc8233163cb75cc3dc5
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Mar 3 23:19:28 2016 -0500

    move to 6.0.0-snapshot-8eada27

commit b32bfe924626b87e540692375ece09e7c2edb189
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 11:30:09 2016 +0100

    Fix some test compile errors.

commit 6ccde35e9840b03c68d1a2cd47c7923a06edf64a
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 11:25:51 2016 +0100

    Current Lucene version is 6.0.0.

commit f62e1015d931b4cc04c778298a8fa1ba65e97ad9
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 11:20:48 2016 +0100

    Fix compile errors in NGramTokenFilterFactory.

commit 6837c6eabf96075f743649da9b9b52dd39611c58
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 10:50:59 2016 +0100

    Fix the edge ngram tokenizer/filter.

commit ccd7f070de5efcdfbeb34b9555c65c4990bf1ba6
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 10:42:44 2016 +0100

    The missing value is now accessible through a getter.

commit bd3b77f9b28e5b05daa3d49683a9922a6baf2963
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 10:41:51 2016 +0100

    Remove IndexCacheableQuery.

commit 05f3091c347aeae80eeb16349ac51d2b53cf86f7
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 10:39:43 2016 +0100

    Fix compilation of function_score queries.

commit 81cda79a2431ac78f56b0cc5a5765387f662d801
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 10:35:02 2016 +0100

    Fix compile errors in BlendedTermQuery.

commit 70994ce8dd1eca0b995870974a38e20f26f96a7b
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 23:33:03 2016 -0500

    add bug ID

commit 29d4f1a71f36f646b5a6060bed3db019564a279d
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 21:02:32 2016 -0500

    easy .store changes

commit 5e1a1e6fd665fa455e88d3a8987362fad5f44bb1
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 20:47:24 2016 -0500

    cleanups mostly around boosting

commit 333a669ec6c305ada5645d13ed1da0e19ec1d053
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 20:27:56 2016 -0500

    more simple fixes

commit bd5cd98a1e089c866b6b4a5e159400b110140ce6
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 19:49:38 2016 -0500

    more easy fixes and removal of ancient cruft

commit a68f419ee47da5f9c9ce5b372f01d707e902474c
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 19:35:02 2016 -0500

    cutover numerics

commit 4ca5dc1fa47dd5892db00899032133318fff3116
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 18:34:18 2016 -0500

    fix some constants

commit 88710a17817086e477c6c021ec346d0534b7fb88
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 18:14:25 2016 -0500

    Add spatial-extras jar as a core dependency

commit c8cd6726583e5ce3f546ed355d4eca037164a30d
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 18:03:33 2016 -0500

    update to lucene 6 jars
2016-03-07 04:12:23 -05:00
Nik Everett 97c4dd3b92 Merge pull request #16718 from biolds/master
Fix waiting for pidfile
2016-02-29 15:02:29 -05:00
Jason Tedor 79d8e225c4 Remove implicit support for Cygwin
This commit makes explicit that we do not support Cygwin.

Closes #16871
2016-02-29 14:59:42 -05:00
Nik Everett ba5be0332d Remove optional logger wrappers
Removes all our logger wrappers except the wrapper for log4j1.2. If you
depend on Elasticsearch's jar in your application you'll need to declare
log4j 1.2 and/or some bridge to your favorite logger.

We did this to simplify our builds and code. No more commons-logging like
log implementation sniffing. No more optional dependency hacks in gradle.

We might one day want to use j.u.l instead of log4j. If we do want that
we can recover its wrapper by studying this commit. We didn't go directly
to j.u.l in this commit because that is a bigger change. Our logging
configuration is based on log4j1.2 and people are used to it. So it'd
be a much more fraught breaking change to do that conversion.
2016-02-26 16:41:07 -05:00
Lee Hinman 5d359f737f Merge branch 'pr/16801' 2016-02-26 13:22:14 -07:00
James Bertouch f7ef8390db Upgrade to Jackson 2.7.1 2016-02-26 12:09:41 -05:00
Nik Everett 74754952db Remove mention of mvn from RestIT and Security 2016-02-24 19:43:56 -08:00
Mike McCandless 5fffede2b0 Upgrade to Lucene 5.5.0 official release 2016-02-20 17:34:16 -05:00
Laurent Defert 6661c4c37f Fix waiting for pidfile
Closes #16717
2016-02-18 02:00:10 +01:00
Nicholas Knize 52ee4c7027 upgrade to lucene 5.5.0-snapshot-850c6c2 2016-02-11 14:28:50 -06:00
gmarz 7a969c0bbe Windows service: Use JAVA_HOME environment variable in registry
This allows for updating Java without having to re-install the service.

Closes #13521
2016-02-11 12:31:56 -05:00
Rhommel f759604783
Adds ES_INCLUDE on rpm 2016-02-11 13:29:10 +11:00
Jim Ferenczi 7d0181b5d4 Rename bin/plugin in bin/elasticsearch-plugin 2016-02-05 10:09:14 +01:00
Robert Muir d5dc05f69e Upgrade to lucene 5.5.0-snapshot-1725675 2016-02-02 22:53:39 -05:00
Ryan Ernst a052dfeb38 Plugins: Reduce complexity of plugin cli
The plugin cli currently is extremely lenient, allowing most errors to
simply be logged. This can lead to either corrupt installations (eg
partially installed plugins), or confused users.

This change rewrites the plugin cli to have almost no leniency.
Unfortunately it was not possible to remove all leniency, due in
particular to how config files are handled.

The following functionality was simplified:
* The format of the name argument to install a plugin is now an official
  plugin name, maven coordinates, or a URL.
* Checksum files are required, and only checked, for official plugins
  and maven plugins. Checksums are also only SHA1.
* Downloading no longer uses a separate thread, and no longer has a timeout.
* Installation, and removal, attempts to be atomic. This only truly works
  when no config or bin files exist.
* config and bin directories are verified before copying is attempted.
* Permissions and user/group are no longer set on config and bin files.
  We rely on the users umask.
* config and bin directories must only contain files, no subdirectories.
* The code is reorganized so each command is a separate class. These
  classes already existed, but were embedded in the plugin cli class, as
  an extra layer between the cli code and the code running for each command.
2016-01-31 20:30:45 -08:00
Jason Tedor 26e5cd651e Default standard output to the journal in systemd
This commit modifies the default setting for standard output in the
systemd configuration to the journal instead of /dev/null. This is to
address a user pain point where Elasticsearch would fail to start but
the error message would be sent to standard output and therefore
/dev/null leading to difficult-to-debug situations.
2016-01-21 15:09:57 -05:00
Greg Marzouka 8fffc474dd Merge pull request #15549 from gmarz/master
Fix Windows service installation failure
2016-01-21 09:33:47 -05:00
Robert Muir 6e7e3a2274 Update lucene to r1725675
Adds DFI (divergence from independence) provider.
Fixes test bugs passing invalid values for BM25 parameters.
2016-01-20 03:32:51 -05:00
David Roberts febc2f08d6 Use egrep instead of grep -E for Solaris
Closes #15628
2016-01-04 17:07:43 +00:00
Adrien Grand cf52e96c42 Upgrade to lucene-5.5.0-snapshot-1721183.
Some files that implement or use the Scorer API had to be changed because of
https://issues.apache.org/jira/browse/LUCENE-6919.
2015-12-21 17:02:08 +01:00
Ryan Ernst 4ea19995cf Remove wildcard imports 2015-12-18 12:43:47 -08:00
gmarz 8004827522 Fix Windows service installation failure when ES_HOME contains parantheses
Closes #15349
2015-12-18 15:04:54 -05:00
Ryan Ernst 634c39ccfd Fix breakage of distribution integ tests 2015-12-10 19:05:47 -08:00
Ryan Ernst e29c9b9dfd Change task for building modules directory in distributions to sync.
This handles when dependencies have changed, so old ones will be
removed.
2015-12-10 17:30:58 -08:00
Ryan Ernst 2758099996 Merge pull request #15382 from rjernst/integ_test_deps
Fix module/integ-test cluster shutdown task ordering
2015-12-10 17:20:11 -08:00
Ryan Ernst f78fc6901f Build: Fix module/integ-test cluster shutdown task ordering
Both modules and integ-test-zip have integration tests (the latter being
the base rest tests). We can currently get odd behavior where
integ-test-zip's integ test does not shutdown its cluster before running
mdoule integ tests (and it then tries to shutdown all those clusters at
once after modules integ tests have run).

The underlying issue can be attributed to a bug in gradle with how cross project
mustRunAfter work with finalizers. This change works around this bug by
setting up mustRunAfter on the shutdown task itself.
2015-12-10 16:22:16 -08:00
Robert Muir 2741888498 Remove RuntimePermission("accessDeclaredMembers")
Upgrades lucene to 5.5.0-1719088, randomizedtesting to 2.3.2, and securemock to 1.2
2015-12-10 14:26:55 -05:00
Robert Muir e454fadc22 Merge branch 'master' into shave_mustache 2015-12-10 07:58:24 -05:00
Robert Muir 7595c4a3c8 Improve network docs
This makes some minor improvements (does not fix all problems!)

It reorders unicast disco in elasticsearch.yml to be right after the network host,
for better locality.

It removes the warning (unreleased) about publish addresses, lets try to really discourage setting
that unless you need to (behind a proxy server). Most people should be fine with `network.host`

Finally it reorganizes the network docs page a bit:

We add a table of 4 "basic" settings at the very beginning:

* network.host
* discovery.zen.ping.unicast.hosts
* http.port
* transport.tcp.port

The first two being the most important, which addresses to bind and talk to, and the other two
being the port numbers.

The rest of the stuff I tried to simplify and reorder under "advanced" headers.

This is just a quick stab, I still think we need more effort into this thing, but we gotta start somewhere.
2015-12-10 12:01:18 +01:00
Robert Muir 17436d4332 get gradle check passing for lang-mustache 2015-12-09 00:10:37 -05:00
Ryan Ernst 3d9d8bd45a Build: Remove duplicate runs of packaged rest tests
We currently use the full suite of packaged rest tests for each
distribution. We also used to run rest tests within core integ tests,
but this stopped working when we split out the test-framework, since the
test files are in there.

This change simplifies the code to run packaged rest tests just once,
for the integ-test-zip, and removes the unused rest tests from
test-framework. Distributions rest tests now check that all modules
were loaded.
2015-12-04 13:43:40 -08:00
Ryan Ernst 2dbd93d3b0 Tweaked module error message against non integ-test-zip and made checks
for limiting configuration to certain distributions prettier
2015-12-04 11:40:08 -08:00
Ryan Ernst 01d48e2062 Merge branch 'master' into jigsaw 2015-12-04 11:29:49 -08:00
Ryan Ernst f5b6b40a92 Fix distros to put modules dir int he right place and qa rest tests to
use the real zip
2015-12-03 23:03:14 -08:00
Ryan Ernst afedd24877 force integ tests for modules to run before distribution integ tests 2015-12-03 22:07:22 -08:00
Ryan Ernst caf77f7eea Add integ-test-zip distribution 2015-12-03 14:52:51 -08:00
Ryan Ernst 0a4a81afaf Added modules, distributions now include them (just plugins installed in
a diff dir)
2015-12-03 14:18:26 -08:00
Ryan Ernst b00c59398b Build: Cleanup test classes in buildSrc to be less groovyish 2015-12-03 01:49:54 -08:00
Robert Muir b7f497627f Merge pull request #15172 from rmuir/nuke_unused_deps
remove unused core dependencies
2015-12-02 07:20:11 -05:00
Ryan Ernst d68c6673a2 Build: Cleanup precommit task gradle code
This change attempts to simplify the gradle tasks for precommit. One
major part of that is using a "less groovy style", as well as being more
consistent about how tasks are created and where they are configured. It
also allows the things creating the tasks to set up inter task
dependencies, instead of assuming them (ie decoupling from tasks
eleswhere in the build).
2015-12-01 22:36:54 -08:00
Rhommel Lamas f623c88d99 Enable ES_INCLUDE 2015-12-02 16:37:30 +11:00
Robert Muir 6e5c301739 remove unused dependencies 2015-12-01 22:22:11 -05:00
Nik Everett 45e71329ee [build] Fix deb and rpm tests
Gradle is causing so much trouble here! Its too cute for its own good.
2015-11-28 19:07:06 -05:00
Ryan Ernst 7788c4cd69 Build: Fix rpm and deb generated poms to have correct artifactId
For all our distributions, we use "elasticsearch" as the artifactId.
This fixes the id for the generated poms of rpm and deb distributions.
2015-11-28 11:30:49 -08:00
Nik Everett efe27df096 Disable rpm and deb integ tests
They are broken and we didn't know it.
2015-11-25 12:35:30 -05:00
Michael McCandless e13b0d4bde upgrade lucene to 5.4.0-snapshot-1715952 2015-11-23 17:13:49 -05:00
Ryan Ernst b8e462b934 Build: Change project attachment into special extra-plugins dir
Currently we use the "gradle project attachment plugin" to support
building elasticsearch as part of another project. However, this plugin
has a number of issues, a large part of which is requiring consistent
use of the projectsPrefix.

This change removes projectsPrefix, and adds support for a special
extra-plugins directory in the root of elasticsearch. Any projects
checked out within this directory will be automatically added to
elasticsearch.
2015-11-22 08:44:33 -08:00
Nik Everett 452d85e582 [build] Skip package tests on windows
They aren't going to work because the packages don't include the windows
files.
2015-11-20 14:54:30 -05:00
Nik Everett 9b0a47d8e3 Fix rpm and deb distributions
and test them with vagrant
2015-11-18 14:16:42 -05:00
Ryan Ernst 50b48e7d8c Build: Make run command work within plugins
We recently got a run command with gradle, but it is sometimes useful to
run ES with a specific plugin. This is a start, by making each esplugin
have a run command which installs the plugin and runs elasticsearch in
the foreground.
2015-11-16 17:29:01 -08:00
Michael McCandless a0bf253d16 upgrade lucene 5.4 snapshot 2015-11-16 14:38:05 -05:00
Michael McCandless 9d7ca53022 upgrade lucene 5.4 snapshot 2015-11-16 14:35:17 -05:00
Adrien Grand d0853e9253 Merge pull request #14619 from jpountz/upgrade/lucene-5.4.0-snapshot-1712973
Upgrade to lucene-5.4.0-snapshot-1712973.
2015-11-09 22:38:44 +01:00
Adrien Grand d6d7af0a6c Upgrade to lucene-5.4.0-snapshot-1712973. 2015-11-09 15:53:27 +01:00
andrejserafim 2bd353d72d replacing run.bat and run.sh with gradle run
run.sh and run.bat were calling out to the old maven build system.
This is no longer in place, so we've created new gradle tasks to
start an elasticsearch node from the current codebase.

fixed #14423
2015-11-08 17:07:19 +00:00
girirajsharma 7ac3fb2c0c rpm uses non-portable `--system` flag to `useradd` 2015-11-07 11:35:37 +05:30
Ryan Ernst 95f885e11f Build: Change tar extension back to tar.gz instead of tgz
Gradle defaults to tgz extension when tar is compressed. This changes
the tar distribution back to tar.gz. Note that this also means the maven
packaging type is now tar.gz.
2015-11-03 11:24:27 -08:00
Ryan Ernst 3e4f87d4d6 Fix tar issues and build distro dependency for integ tests 2015-11-02 22:19:29 -08:00
Ryan Ernst f5556224f5 Build: Add back integ tests to distributions
rpm and deb are still skipped, but this configures rest tests to run for
zip and tgz distributions

closes #14361
2015-11-02 17:51:08 -08:00
Ryan Ernst 0dca49fae4 fix eclipse config for qa projects 2015-10-30 21:32:16 -07:00
Ryan Ernst b6dee6bd43 Merge pull request #14375 from rjernst/sweep_up_maven
Remove maven pom files and supporting ant files
2015-10-30 18:59:11 -07:00
Areek Zillur 13b60e1b92 update to lucene-5.4.x-snapshot-1711508 2015-10-30 15:42:02 -04:00
Clinton Gormley 551d255295 Merge pull request #14351 from jmferrer/master
typo error: adress -> address
2015-10-30 10:44:26 +01:00
Ryan Ernst 542522531a Build: Remove maven pom files and supporting ant files
This change removes the leftover pom files. A couple files were left for
reference, namely in qa tests that have not yet been migrated (vagrant
and multinode). The deb and rpm assemblies also still exist for
reference when finishing their setup in gradle.

See #13930
2015-10-29 23:53:49 -07:00
Ryan Ernst c86100f636 Switch build system to Gradle
See #13930
2015-10-29 11:40:19 -07:00
Robert Muir 1194cd0bdc Add threadgroup isolation.
Closes #14353

Squashed commit of the following:

commit edae0729f71ea3d3f9fa9c0d27c9effc042eb5a9
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Oct 29 14:13:42 2015 -0400

    update sha1 and simplify test

commit 635c4f245d66ad353a16267c810e02b725553fad
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Oct 29 07:01:26 2015 -0400

    Add threadgroup isolation.

    Code with `modifyThread` and `modifyThreadGroup` may only modify
    its own threadgroup (or an ancestor of that). This enforces
    what is intended by the ThreadGroup class.

    This has two immediate implications:
    1. Code without these permissions (scripts) may not create or mess with threads
    2. ES application threads cannot mess with Java system threads

    ES puts all application threads in one single group today, but in the future
    this can be organized better, and we will have more isolation in the system.
2015-10-29 14:14:27 -04:00
jmferrer 8880474f11 typo error: adress -> address 2015-10-29 11:00:21 +01:00
Adrien Grand 43958db10b Upgrade to lucene-5.4-snapshot-1710880. 2015-10-28 09:34:54 +01:00
Nicholas Knize 3d970f17f9 Fix Multi-geometry bbox
A long time coming this Upgrades to Spatial4J 0.5 which includes the fix for calculating a Multi-geometry bounding box.
2015-10-27 21:34:07 -05:00
Jason Tedor f1694e6663 Startup script exit status should catch daemonized startup failures
This commit fixes an issue where when starting Elasticsearch in
daemonized mode, a failed startup would not cause a non-zero exit code
to be returned. This can prevent the SysV init system from detecting
startup failures.

Closes #14163
2015-10-16 17:04:37 -04:00
Adrien Grand 5ae810991c Upgrade to lucene-5.4-snapshot-1708254. 2015-10-16 09:41:36 +02:00
Simon Willnauer f4eab500aa Upgrade Netty to 3.10.5.final
This fixes a critical bug in SSL handling.

See http://netty.io/news/2015/10/13/3-10-5-Final.html
2015-10-14 10:56:38 +02:00
Yannick Welsch 63366add7d Parse Java system properties in plugin.bat
Closes #13616
2015-10-13 13:33:34 +02:00
Jason Tedor 9a9a6a4b3b Remove Guava as a dependency
This commit removes Guava as a dependency. Note that Guava will remain
as a test-only dependency (transitively through Jimfs).

Closes #13224
2015-10-09 14:19:22 -04:00
Jason Tedor 81b9932ee8 Update mustache.java to version 0.9.1
This commit upgrades mustache.java to version 0.9.1. The primary motive
for this is because version 0.8.13 depends on Guava, but version 0.9.1
does not.

Relates #13224
2015-10-09 13:56:44 -04:00
javanna 648cc6defa Packaging: change permissions/ownership of config dir
When generating the rpm and dep package we now set proper group (elasticsearch) and permissions (750) to the conf dir (default /etc/elasticsearch). Same for the scripts subdirectory.

Expanded the assert_file bash function to also optionally check the group of files, so we can actually test that the group was set correctly.

Relates to #11016
Closes #14017
2015-10-08 15:40:12 +02:00
Britta Weber 71aefd5a06 remove option to configure custom config file via CONF_FILE or -Des.default.conf
It is rarely used and was not consistently handled by different distributions anyway.
This commit also adds a test for specifying CONF_DIR when installing plugins and
starting elasticsearch.

relates to #12712 and #12954
closes #5329
closes #13715
2015-10-06 19:02:43 +02:00
Robert Muir 8c4bc7d10b Nuke ES_CLASSPATH appending, JarHell fail on empty classpath elements
Closes #13880

Squashed commit of the following:

commit 316a328e5032e580ba840db993d907631334aac0
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 30 16:57:47 2015 -0400

    windows is terrible

commit 0406b560c58bf833f8d77af9c7cf3386771dd9c5
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 30 16:43:09 2015 -0400

    Nuke ES_CLASSPATH appending

    Out of box, ES expects its stuff to be in particular places. We should not be appending to ES_CLASSPATH, allowing users to specify stuff there, like we do in elasticsearch.bin.sh

    If the user sets it, its not going to work out of box.

    Closes #13812

commit 415d8972df28eddec322bb6d70100a1993fa95f6
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 30 16:26:35 2015 -0400

    Fail hard on empty classpath elements.

    This can happen easily, if somehow old 1.x shellscripts survive and try to launch 2.x code.
    I have the feeling this happens maybe because of packaging upgrades or something.
    Either way: we can just fail hard and clear in this situation, rather than the current situation
    where CWD might be /, and we might traverse the entire filesystem until we hit an error...

    Relates to #13864
2015-09-30 18:59:27 -04:00
Robert Muir 20a203b1a2 Merge pull request #13834 from rmuir/groovy_factor_out
Factor groovy out of core into lang-groovy
2015-09-29 05:58:21 -04:00
Tanguy Leroux 9cb27ad068 Update to Jackson 2.6.2 2015-09-29 10:20:54 +02:00
Robert Muir e0d42739dd Factor groovy out of core into lang-groovy 2015-09-28 20:17:45 -04:00
Robert Muir c0d098b233 Don't let ubuntu try to install its crazy jayatana agent.
By default, our security stuff will reject this (as do other apps).
See https://bugs.launchpad.net/ubuntu/+source/jayatana/+bug/1441487

However its not really the user's fault, ubuntu screws up here by
installing this agent by default. We don't want any agents.

So instead, we drop it like this:

```
$ bin/elasticsearch
Warning: Ignoring JAVA_TOOL_OPTIONS=-Bogus1 -Bogus2
Please pass JVM parameters via JAVA_OPTS instead
[2015-09-25 23:34:39,777][INFO ][node                     ] [Doctor Bong] version[3.0.0-SNAPSHOT], pid[19044], build[2f5b6ea/2015-09-26T03:18:16Z]
...
```

Closes #13785
2015-09-25 23:32:52 -04:00
Robert Muir 689af1a6d6 Factor expressions scripts out to lang-expression plugin 2015-09-22 20:33:47 -04:00
Nik Everett 20d8e79801 Merge pull request #13514 from nik9000/dont_detect_java_7
Remove JAVA_HOME detection from the debian init script
2015-09-22 08:59:10 -04:00
Nik Everett 2a5bb4e648 [packaging] remove JAVA_HOME detection
Only the debian init script did JAVA_HOME detection. Everything else just
relied on `bin/elasticsearch`'s `which java` style detection. This strips
the detection from the debian init script so its like the rpm init script.

Closes #13403
2015-09-22 08:56:45 -04:00
André Carvalho 9ab6e01f4d Adds a validation for plugins script to check if java is set
This does the same validation as in elasticearch bin script
2015-09-17 10:34:36 -03:00
Robert Muir 01e6d8e3dc Remove java.lang.reflect.ReflectPermission "suppressAccessChecks"
Closes #13603

Squashed commit of the following:

commit 8799fb42d80297a79285beaf407b1bbecdb5854d
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 16 03:32:29 2015 -0400

    Add randomizedtesting snapshot note

commit 0d874d9f0f5fddaeab8f48f9816a052dcaa691be
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 16 03:11:01 2015 -0400

    Add a mechanism for insecure plugins and get all tests passing

commit 80540aeb9a264f6f299aaa3bc89df7f9b7923a60
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Sep 15 22:59:29 2015 -0400

    Really remove, we are killing this

commit 884818c1ad44ca2e7572a6998c086580be919657
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Sep 15 22:57:22 2015 -0400

    fill in TODOs

commit 34f4cb81f249edfec4d8d211da892f8c987e5948
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Sep 15 22:31:43 2015 -0400

    Publish snapshots of RR and lucene and cutover

commit d68eb9d66ce059761805c64d67e41a29098c9afa
Merge: f27e208 f62da59
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Sep 15 12:32:41 2015 -0400

    Merge branch 'master' into kill-setaccessible

commit f27e20855216dab6a6ad035d41018d8c67f3144c
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Sep 15 12:32:21 2015 -0400

    make a real lucene snapshot
2015-09-16 04:08:31 -04:00
Alexander Reelsen 1e209e3802 Plugins: Remove java opts/es java opts from plugin manager
... and run as client VM.

Reasoning: When calling the plugin manager on java 7 with additional JAVA_OPTS
that change heap configuration compared to what is set at the plugin
manager shell script. This resulted in errors.

This commit removes the JAVA_OPTS and ES_JAVA_OPTS from the plugin
manager call to prevent those settings.

Closes #12479
2015-09-15 17:04:42 +02:00
Robert Muir c1f2fc76c2 Upgrade lucene to r1702090
The semantics of the `boost` parameter for `function_score` changed. This is
due to the fact that Lucene now requires that query boosts and top-level boosts
are applied the same way.
2015-09-10 23:36:43 +02:00
gmarz 9e6115b066 Packaging: Fix Windows service start/stop issues
This commit addresses several bugs that prevented the Windows
service from being started or stopped:

- Extra white space in the concatenation of java options in
  elasticsearch.in.bat which tripped up Apache Commons Daemon
  and caused ES to startup without any params, eventually leading
  to the "path.home is not configured" exception.

- service.bat was not passing the start argument to ES

- The service could not be stopped gracefully via the stop command
  because there wasn't a method for procrun to call.

Closes #13247
Closes #13401
2015-09-09 13:31:46 -04:00
Robert Muir f216d92d19 Upgrade to lucene 5.4-snapshot r1701068 2015-09-03 15:13:33 -04:00