Commit Graph

401 Commits

Author SHA1 Message Date
Simon Willnauer c13e604697 [BUILD] Restrict read permission to project.basedir
This prevents reads from anywhere outside of the elasticsearch
clone when running tests with security manager enabled.
2015-04-13 16:44:31 +02:00
Simon Willnauer d738b3d1e2 [BUILD] Use SuppressFrobidden annotation instead of class level excludes
Forbidden APIs 1.8 allows excludes based on annotations which can now be on methods etc. for more find grained control.

Closes #10560
2015-04-13 10:08:50 +02:00
Uwe Schindler df118214c4 Update forbiddenapis to version 1.8 2015-04-13 01:27:56 +02:00
David Pursehouse 97a9b4ec2f Fix m2e Eclipse Integration
Update Eclipse core prefs

Eclipse Luna overwrites the prefs file, putting all the settings
in alphabetical order and removing comments. This causes the prefs
files to be modified in the git workspace.

Update the file with the version generated by Eclipse to prevent it
from being modified every time.

No settings values are modified by this change.

This also  adds another plugin to the lifecycle mapping in the pom.xml which was missed in https://github.com/elastic/elasticsearch/pull/10524.
2015-04-10 16:48:04 +01:00
Simon Willnauer bdc4c7f84c [BUILD] Add sigar binaries when running unittests
the sigar binaries are not available when running tests today. This
commit adds the path to the test run.
2015-04-10 14:37:18 +02:00
Colin Goodheart-Smithe 7e353e0768 removed erroneous tab character 2015-04-10 10:37:26 +01:00
Colin Goodheart-Smithe 4a09a98561 Fix to pom.xml to allow eclipse maven integration using m2e 2015-04-10 08:34:01 +01:00
Tanguy Leroux e60f61d0a9 Fix maven-resources-plugin warnings
Commit 168238dab6 declared multiple maven-resources-plugin usages instead of declaring multiple executions for the same plugin... resulting to Maven warnings.

Closes #10433
2015-04-09 10:05:10 +02:00
Adrien Grand 89b97c1c8d Upgrade to lucene-5.1.0-snapshot-1671894.
This includes a fix for LUCENE-6406.
2015-04-07 20:04:54 +02:00
Robert Muir 1d56502e52 Upgrade to Lucene 5.1 r1671277.
Closes #10435.

Squashed commit of the following:

commit aa1935c790b2731fc2bbc7de6142b09e3fe8bd4a
Author: Ryan Ernst <ryan@iernst.net>
Date:   Mon Apr 6 13:44:40 2015 -0700

    fix index lookup

commit bb6373595ff62ffc56fdf0cba3ac9c0ebe679946
Merge: 916962b eb3a170
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Apr 6 14:24:38 2015 -0400

    Merge branch 'lucene_r1671277' of github.com:elasticsearch/elasticsearch into lucene_r1671277

commit 916962b82d192a53add471b4cc4a1396bc30eb0e
Merge: 197b3a2 21f72fe
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Apr 6 07:09:41 2015 -0400

    Merge branch 'master' into lucene_r1671277

commit eb3a1703f7932ddd0cf3e83bec0e86131d255407
Author: Ryan Ernst <ryan@iernst.net>
Date:   Sat Apr 4 11:06:03 2015 -0700

    re-enable index lookup tests

commit 80d65d5eab39062dd8364687da74ddbb87ebcb76
Author: Ryan Ernst <ryan@iernst.net>
Date:   Sat Apr 4 10:39:52 2015 -0700

    update pom to point to new snapshot repo

commit 197b3a21ac2c2d70c9f740fe53e58632a22d1aad
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Apr 4 12:51:22 2015 -0400

    fix postingsenum usage

commit 0e2b7a00cd07d068f755c51185ac521aa1eb0326
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Apr 4 12:21:23 2015 -0400

    upgrade to lucene r1671277 (have not yet run tests or looked at postings changes)
2015-04-07 06:09:35 -04:00
Tanguy Leroux 168238dab6 Add properties files to configure startup and installation scripts
Many scripts are used to start/stop and install/uninstall elasticsearch. These scripts share a lot of configuration properties like directory paths, max value for a setting, default user etc. Most of the values are identical but some of them are different depending of the platform (Debian-based or Redhat-based OS), depending of the way elasticsearch is started (shell script, systemd, sysv-init...) or the way it is installed (zip, rpm, deb...). Today the values are duplicated in multiple places, making it difficult to maintain the scripts or to update a value.

This pull request make this more uniform: values used in scripts must be defined in a common packaging.properties file. Each value can be overridden in another specific packaging.properties file for Debian or Redhat. All startup and installation scripts are filtered with the common then the custom packaging.properties files before being packaged as a zip/tar.gz/rpm/dpkf archive.
2015-04-02 18:09:43 +02:00
Lee Hinman 1cb169afb4 Add ability to launch bootstrapped ES using maven
This adds the exec-maven-plugin that allows a developer to run:

```
mvn exec:exec
```

To launch the `Bootstrap` process similar to the way that a Java IDE
would. All the logs go to logs/elasticsearch.log (or wherever
configured)
2015-03-30 11:07:21 -06:00
Simon Willnauer c87e1b41a4 [TEST] only use perm gen space on java 1.7.x 2015-03-24 10:04:32 +01:00
Simon Willnauer c017a400d5 [TEST] Add back permGen space tuning for now 2015-03-24 09:38:34 +01:00
Robert Muir 10e6fa1963 fix build with java 9
* add compiler workarounds for JDK bug JI-9019884
* remove permgen specification during tests (this results in an error on java 9)
* fix threadpool grow/shrink to call methods in the right order (this results in IAE with java 9)
2015-03-24 00:29:24 -04:00
Shay Banon 9f9ada4b7b Upgrade to Jackson 2.5.1
Note, Jackson 2.5 is less lenient when it comes to not starting an object before starting to add fields on a fresh builder, fixed where applicable.
closes #10210
2015-03-23 09:03:27 +01:00
Shay Banon 31b63b1a84 Better detection of CBOR
CBOR has a special header that is optional, if exists, allows for exact detection. Also, since we know which formats we support in ES, we can support the object major type case.
closes #7640
2015-03-20 22:06:03 +01:00
Simon Willnauer 73953142a7 [BUILD] fix snapshot URL 2015-03-16 22:59:49 -07:00
Clinton Gormley 4d78026ef1 Updated copyright years to include 2015 2015-02-28 03:18:45 +01:00
Robert Muir 4ca57ab10a Upgrade to Lucene r1662607
Closes #9915.

Squashed commit of the following:

commit cfa59f5a3f03d9d1b432980dcee6495447c1e7ea
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Feb 27 12:10:16 2015 -0500

    add missing null check

commit 62fe5403068c730c0e0b6fd1ab1a0246eeef6220
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Feb 27 11:31:53 2015 -0500

    Disable ExtrasFS for now, until we hook all this in properly in a separate issue.

commit 822795c57c5cf846423fad443c2327c4ed0094ac
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Feb 27 10:12:02 2015 +0100

    Fix PercolatorTests.

commit 98b2a0a7d8298648125c9a367cb7e31b3ec7d51b
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Feb 27 09:27:11 2015 +0100

    Fix ChildrenQueryTests.

commit 9b99656fc56bbd01c9afe22baffae3c37bb48a71
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Feb 26 20:50:02 2015 -0500

    cutover apis, no work on test failures yet.
2015-02-27 12:51:36 -05:00
Robert Muir 30a4294a6a Upgrade to lucene r1660560
Squashed commit of the following:

commit 07391388715ed1f737e8acc391cea0bce5d79db9
Merge: a71cc45 b61b021
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Feb 20 06:58:11 2015 -0500

    Git really sucks

    Merge branch 'lucene_r1660560' of github.com:elasticsearch/elasticsearch into lucene_r1660560

commit b61b02163f62ad8ddd9906cedb3d57fed75eb52d
Author: Adrien Grand <jpountz@gmail.com>
Date:   Wed Feb 18 19:03:49 2015 +0100

    Try to improve TopDocs.merge usage.

commit bf8e4ac46d7fdaf9ae128606d96328a59784f126
Author: Ryan Ernst <ryan@iernst.net>
Date:   Wed Feb 18 07:43:37 2015 -0800

    reenable scripting test for accessing postings pieces.  commented out
    parts that fail because of bad assumptions

commit 6d4d635b1a23b33c437a6bae70beea70ad52d91c
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Feb 18 09:41:46 2015 -0500

    add some protection against broken asserts, but, also disable crappy test

commit c735bbb11f38782dfea9c4200fcf732564126bf5
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Feb 18 02:21:30 2015 -0500

    cutover remaining stuff from old postings api

commit 11c9c2bea3db3ff1cd2807bd43e77b500b167aed
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Feb 18 01:46:04 2015 -0500

    cut over most DocsEnum usage

commit bc18017662f6abddf3f074078f74e582494c88e2
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Feb 18 01:19:35 2015 -0500

    upgrade to lucene_r1660560, modulo one test fail
2015-02-20 06:58:43 -05:00
David Pilato 3c2414128c Rename Maven repository id
We are using repository ids with spaces in our `pom.xml`. Although it's not forbidden, a common practice is to avoid space in id.

This commit changes codehaus snapshots and lucene snapshots to a consistent naming (using a dash, all lowercase).
We also add a name which is used by Maven when displaying some information about the repository.

This naming is also consistent with [elasticsearch-parent project](https://github.com/elasticsearch/elasticsearch-parent) which will be used in the next future in 1.x and master branch.

**Important note**: If you have trouble to compile elasticsearch or a plugin using `mvn compile` and hit a `Access denied to: [URL_HERE], ReasonPhrase: Forbidden. -> [Help 1]`, you can remove related maven files:

```sh
find ~/.m2/repository -name _remote.repositories -exec rm -v {} \;
find ~/.m2/repository -name _maven.repositories -exec rm -v {} \;
```

Another option is to tell Maven not using those files with `--llr`:

```sh
mvn compile --llr
```
2015-02-17 20:39:04 +01:00
Alexander Reelsen 5092904885 Packaging: Add antlr and asm dependencies 2015-02-13 23:55:35 +01:00
Victor Buldakov f500293ffb Dependencies: Update joda-time to v2.7
Closes #9338
Closes #9610
2015-02-12 15:06:18 +01:00
Alexander Reelsen 9cd14a5c29 CliTool: Add command to warn on permission/owner change
When using the CLI tool infrastructure, a command can potentially write
a new file. In case it overwrites an existing one, you may want to ensure
that the permissions, the owner and the group are kept the same and do not
accidentally change when overwriting those files.

This PR introduces a command that allows you to execute this check per path.

It also adds a new testing dependency, namely jimfs, which allows you to create
in-memory filesystems with certain properties (like supporting or not posix permissions
on this filesystem), so that you can test those features, without executing
tests on a certain operating system.
2015-02-12 10:10:11 +01:00
Robert Muir 9c9b5c27d3 Upgrade to Lucene r1657571.
Closes #9587

Squashed commit of the following:

commit 23ac91dca4b949638ca1d3842fd6db2e00ee1d36
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Feb 5 18:42:28 2015 +0100

    Do not compute scores if aggregations do not need it (like top_hits) or use a script (which might compute scores).

commit 51262fe2681c067337ca41ab88096ef80a2e8ebb
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Feb 5 15:58:38 2015 +0100

    Fix more compile errors.

commit a074895d55b8b3c898d23f7f5334e564d5271a56
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Feb 5 09:31:22 2015 -0500

    fix a few more obvious ones

commit 399c41186cb3c9be70107f6c25b51fc4844f8fde
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Feb 5 09:28:32 2015 -0500

    fix some collectors and queries

commit 5f46c2f846c5020d5749233b71cbe66ae534ba51
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Feb 5 09:24:24 2015 -0500

    upgrade to lucene r1657571
2015-02-06 08:53:20 -05:00
Michael McCandless e29cf903c8 Core: upgrade to Lucene snapshot r1656366
* IndexWriter deadlock and DV update concurrency fix
  * BytesRef reuse bug with SortedSetDVTermsEnum
  * Int overflow skip data corruption bug
  * Compound file API cleanups
  * IndexWriter doesn't accept per-doc Analyzer anymore

Closes #9524
2015-02-02 13:37:45 -05:00
Lee Hinman 6c27f1242a Update Groovy dependency to 2.4.0 2015-02-02 09:27:41 -07:00
Robert Muir be3e60efc8 Upgrade to lucene r1654549 snapshot.
Closes #9402.

Squashed commit of the following:

commit 85c71b6478441a73738c81f02257193f9837f3ba
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Jan 24 11:24:36 2015 -0500

    upgrade to lucene r1654549 snapshot
2015-01-25 15:01:45 -05:00
Alexander Reelsen ff48c649b1 Dependencies: Upgrade netty to 3.10.0.Final
Changelog is available at
http://netty.io/news/2014/12/17/3-9-6-Final-and-3-10-0-Final.html

Closes #9132
2015-01-19 15:47:09 +01:00
Adrien Grand 1fc24a8837 Upgrade to lucene-5.1.0-snapshot-1652032.
This new Lucene snapshot does not have out-of-order scoring anymore.

Close #9318
2015-01-16 09:37:29 +01:00
Robert Muir d226a973f7 core: upgrade to lucene 5 r1650327.
refactor _version docvalues migration to be more efficient.

closes #9206
2015-01-09 12:12:31 -05:00
Simon Willnauer 09b4d0e097 [EXEC] Remove reduced stack size
We used to reduces the stack size to 256kb. This commit removes
this optimization and relies on the JVMs default.

Closes #9135
2015-01-06 17:35:45 +01:00
Adrien Grand 90f98579a2 Upgrade to Lucene 5.0.0-snapshot-1649544.
A couple of changes that triggerred a refactoring in Elasticsearch:

 - LUCENE-6148: Accountable.getChildResources returns a collection instead of
   a list.

 - LUCENE-6121: CachingTokenFilter now propagates reset(), as a result
   SimpleQueryParser.newPossiblyAnalyzedQuery has been fixed to not reset both
   the underlying stream and the wrapper (otherwise lucene would barf because of
   a doubl reset).

 - LUCENE-6119: The auto-throttle issue changed a couple of method
   names/parameters. It also made
   `UpdateSettingsTests.testUpdateMergeMaxThreadCount` dead slow so I muted this
   test until we clea up merge throttling to use LUCENE-6119.

Close #9145
2015-01-06 09:24:18 +01:00
Simon Willnauer 3e37c89932 [INTERNAL] Remove OperationRouting abstraction
This commit removes the unneeded OperationRouting interface and flattens
the package structure inside cluster.routing
2015-01-02 12:17:35 +01:00
javanna d17db85794 [TEST] upgrade randomized runner to 2.1.11
2.1.11 contains the fix for this issue: https://github.com/carrotsearch/randomizedtesting/issues/179

Closes #8930
2014-12-18 10:40:05 +01:00
Adrien Grand 6d253aba08 Upgrade to lucene-5.0.0-snapshot-1646179. 2014-12-18 09:51:20 +01:00
Robert Muir a2ffe494ae [core] add best_compression option for Lucene 5.0
Upgrades lucene to latest, and supports the BEST_COMPRESSION parameter
now supported (with backwards compatibility, etc) in Lucene.
This option uses deflate, tuned for highly compressible data.

index.codec::
The default value compresses stored data with LZ4 compression, but
this can be set to best_compression for a higher compression ratio,
at the expense of slower stored fields performance.

IMO its safest to implement as a named codec here, because ES already
has logic to handle this correctly, and because its unrealistic to have
a plethora of options to Lucene's default codec... we are practically
limited in Lucene to what we can support with back compat, so I don't
think we should overengineer this and add additional unnecessary plumbing.

See also:
https://issues.apache.org/jira/browse/LUCENE-5914
https://issues.apache.org/jira/browse/LUCENE-6089
https://issues.apache.org/jira/browse/LUCENE-6090
https://issues.apache.org/jira/browse/LUCENE-6100

Closes #8863
2014-12-10 22:13:09 -05:00
Robert Muir 39186edc86 Ban java.io.File in tests.
Restrict use of java.io.File to 5 methods (excluded), but otherwise ban.
This is a prerequisite to do any mocking here.

I don't try to do any heavy cleanup on these tests, I am not familiar with them.
So this is mostly a rote straightforward conversion.

Closes #8836
2014-12-09 05:57:48 -05:00
Thilo Fromm 98d18c4bd9 deb: add systemd service config for upcoming Jessie
This change adds a systemd service configuration file, and adds systemd logic
to installation and de-installation scripts. The upcoming Debian 8 "Jessie"
release will use systemd.

fixes #8943

Signed-off-by: Thilo Fromm <github@thilo-fromm.de>
2014-12-05 12:25:48 +01:00
Simon Willnauer a6510f9245 Add File.java to forbidden APIs
This commit cuts over all of core (not quite all tests) to java.nio.Path
It also adds the file class to the core forbidden APIs to prevent its usage.

This commit also resolves #8254 since we now consistently useing the NIO Path
API. The Changes in this commit allow for more information if IO operations fail
since the NIO API throws exceptions instead of boolean return values. The build-in
methods used in this commit are also more resillient to encodeing errors like
unmappable characters and throw exceptions if those chars are present in a file.

Closes #8254
Closes #8666
2014-12-02 21:29:26 +01:00
Adrien Grand eba2bd51c1 Core: Upgrade to lucene-5.0.0-snapshot-1642891.
The only required change was to add a description to our index outputs.

Close #8743
2014-12-02 17:38:53 +01:00
javanna 0c70b3ca7d [TEST] remove duplicated tests.slow from randomized testing maven plugin conf 2014-11-27 11:36:15 +01:00
Chris Earle 08521a4066 Revert "Update to Jackson 2.4.3"
This reverts commit 7523d0b150.
2014-11-25 16:41:33 -05:00
Chris Earle 7523d0b150 Update to Jackson 2.4.3
- Update pom to 2.4.3 from 2.4.2
- Enable the CBOR data header (aka tag) from the CBOR Generator to provide binary identification like the Smile format
- Check for the CBOR header and ensure that the data sent in represents a "major type" that is an object
- Cleans up `JsonVsCborTests` unused imports
2014-11-25 14:03:16 -05:00
Michael McCandless dfb6d6081c Core: upgrade to current Lucene 5.0.0 snapshot
Elasticsearch no longer unlocks the Lucene index on startup (this was
dangerous, and could possibly lead to corruption).

Added the new serbian_normalization TokenFilter from Lucene.

NoLockFactory is no longer supported (index.store.fs.fs_lock = none),
and if you have a typo in your fs_lock you'll now hit a StoreException
instead of silently using NoLockFactory.

Closes #8588
2014-11-24 05:08:42 -05:00
Uwe Schindler 866571f4d7 Switch to forbidden-apis 1.7 2014-11-24 09:58:44 +01:00
Ryan Ernst cca5934e9d Tests: Pass through locale and timezone to test runner, and print in
repro command line.

The carrot runner currently randomizes both locale and timezone, but
these are not set in the maven reproduce line.  Since they aren't
even printed, we have no idea what locale/timezone the tests
actually ran with.
2014-11-19 22:01:26 -08:00
Simon Willnauer 5c6fe2593e [CORE] Ban all useage of Future#cancel(true)
Interrupting a thread while blocking on an NIO Read / Write Operation
can cause a file to be closed due to the interrupts. This can have unpredictable
effects when files are open by index readers etc. we should prevent interruptions
across the board if possible.

Closes #8494
2014-11-18 14:14:09 +01:00
Philip McMahon 4194a699c0 Logging: Add log4j-extras dependency
Close #7927
2014-11-13 12:39:30 +00:00