Commit Graph

16287 Commits

Author SHA1 Message Date
Simon Willnauer bdc21078ca [TEST] don't copy write locks or extraFS files 2015-09-29 17:01:46 +02:00
Jason Tedor 464d48cda6 Merge pull request #13847 from jasontedor/workaround-jdk-8056984
Workaround JDK-8056984
2015-09-29 16:49:44 +02:00
Jason Tedor ba0643da2b Workaround JDK-8056984
This commit works around JDK bug JDK-8056984 in the javac compiler.
This bug is impacting CI compilations on JDK 8u25.
2015-09-29 16:45:57 +02:00
Robert Muir d54dd63825 Merge pull request #13844 from rmuir/lock_down_classloader_and_reflection
Clean up scripting permissions.
2015-09-29 10:35:52 -04:00
Simon Willnauer 6bb7b9b567 Merge pull request #13766 from s1monw/store_recovery_helper
Refactor StoreRecoveryService to be a simple package private util class
2015-09-29 15:59:45 +02:00
Simon Willnauer 3b5ed08d49 Refactor StoreRecoveryService to be a simple package private util class
StoreRecoveryService used to be a pretty heavy class with lots of dependencies.
This class was basically not testable in isolation and had an async API with a listener.
This commit refactors this class to be a simple utility classs with a sync API hidden behind
the IndexShard interface. It includes single node tests and moves all the async properities to
the caller side.
Note, this change also removes the mapping update on master from the store recovery code since
it's not needed anymore in 3.0 because all stores have been subject to sync mapping updates such
that the master already has all the mappings for documents that made it into the transaction log.

Closes #13766
2015-09-29 15:59:21 +02:00
Robert Muir eeeb42abef Clean up scripting permissions.
Now that groovy is factored out, we contain this dangerous stuff there.

TODO: look into those test hacks inspecting class protection domains, maybe we can
clean that one up too.

TODO: generalize the GroovyCodeSourcePermission to something all script engines check,
before entering accesscontrollerblocks. this way e.g. groovy script cannot coerce
python engine into creating something with more privs if it gets ahold of it... we
should probably protect the aws/gce hacks in the same way.
2015-09-29 09:54:12 -04:00
Jason Tedor 882fe0784c Merge pull request #13842 from jasontedor/workaround-jdk-8056014
Workaround JDK-8056014
2015-09-29 15:43:25 +02:00
Nik Everett 8a7f9edf02 Merge pull request #13820 from nik9000/immutable_set_be_gone_2
Finish banning ImmutableSet
2015-09-29 15:34:34 +02:00
Jason Tedor ad090296fc Workaround JDK-8056014
This commit works around JDK bug JDK-8056014 in the javac compiler.
This bug is impacting CI compilations on JDK 8u11 and 8u25.
2015-09-29 15:03:32 +02:00
Robert Muir 94972bb39d Merge pull request #13829 from rmuir/add_seatbelt
improve seccomp syscall filtering
2015-09-29 08:54:06 -04:00
Colin Goodheart-Smithe 8c7c7652f6 Make sure equivalent geohashCellQueries are equal after toQuery called
Previous to this change if to equal geohash cell query builders were created and then toQuery was called on one, they would no longer be equal.

This change also adds a test to AbstractQueryTestCase to make sure calling toQuery on any query builder does not affect the query builder's equality
2015-09-29 14:34:58 +02:00
Simon Willnauer 2f1b3ccde2 Merge pull request #13816 from s1monw/fix_jackson_line_feed
Ensure XContent is consistent across platforms
2015-09-29 14:28:03 +02:00
Simon Willnauer e814102bfb Ensure XContent is consistent across platforms
Today we generate XContent with platform dependent linefeeds. This
commit makes the pretty-printed json etc. consistent with \n across all
platforms.
2015-09-29 14:22:33 +02:00
Simon Willnauer 5ec7f04021 Merge pull request #13797 from s1monw/remove_index_service_dep
Remove IndexService dep. from IndexShard
2015-09-29 14:20:58 +02:00
Simon Willnauer e3b38e5af1 Remove IndexService dep. from IndexShard
there is no reason for the index shard to hold on to it's corresponding
index service. This dependency is unnecessary.
2015-09-29 14:07:58 +02:00
Robert Muir cbf894a2e7 Add abstract TestScript to the test framework jar 2015-09-29 06:37:33 -04:00
Lee Hinman 1811286a60 Fix compilation error due to empty <p> tag 2015-09-29 04:17:36 -06:00
Robert Muir 20a203b1a2 Merge pull request #13834 from rmuir/groovy_factor_out
Factor groovy out of core into lang-groovy
2015-09-29 05:58:21 -04:00
Tanguy Leroux 9cb27ad068 Update to Jackson 2.6.2 2015-09-29 10:20:54 +02:00
Boaz Leskes c1ab2dc244 Test: make sure TranslogTests.randomNonTranslogPatternString returns a reference to a file
It can return a directory now (like /  or ..)
2015-09-29 07:53:31 +02:00
Robert Muir e0d42739dd Factor groovy out of core into lang-groovy 2015-09-28 20:17:45 -04:00
Robert Muir d7a087a41b improve seccomp syscall filtering
* Add OS X support via "seatbelt" mechanism. This gives consistency across dev and prod, since many devs use OS X.
* block execveat system call: it may be new, but we should not allow it.
2015-09-28 11:35:07 -04:00
Nik Everett 85b99d2011 Finish banning ImmutableSet
Ban ImmutableSet$Builder because that let you sneak some `ImmutableSet`s in.

Remove all remaining imports of ImmutableSet.
2015-09-28 01:49:46 +02:00
Boaz Leskes 148265bd16 Internal: an inactive shard is temporarily activated by triggered synced flush
When a shard becomes in active we trigger a sync flush in order to speed up future recoveries. The sync flush causes a new translog generation to be made, which in turn confuses the IndexingMemoryController making it think that the shard is active. If no documents comes along in the next 5m, the shard is made inactive again , triggering a sync flush and so forth.

To avoid this, the IndexingMemoryController is changed to ignore empty translogs when checking if a shard became active. This comes with the price of potentially missing indexing operations which are followed by a flush. This is acceptable as if no more index operation come in, it's OK to leave the shard in active.

A new unit test is introduced and comparable integration tests are removed.

Closes #13802
2015-09-27 07:16:54 +02:00
Simon Willnauer ebe02ec54a [TEST] Make test pass on windows - jackson uses platform line.separator which messes up comparisons 2015-09-26 21:25:49 +02:00
Robert Muir e01a74d608 Merge pull request #13813 from rmuir/crazy_ubuntu
Don't let ubuntu try to install its crazy jayatana agent.
2015-09-25 23:46:41 -04:00
Robert Muir c0d098b233 Don't let ubuntu try to install its crazy jayatana agent.
By default, our security stuff will reject this (as do other apps).
See https://bugs.launchpad.net/ubuntu/+source/jayatana/+bug/1441487

However its not really the user's fault, ubuntu screws up here by
installing this agent by default. We don't want any agents.

So instead, we drop it like this:

```
$ bin/elasticsearch
Warning: Ignoring JAVA_TOOL_OPTIONS=-Bogus1 -Bogus2
Please pass JVM parameters via JAVA_OPTS instead
[2015-09-25 23:34:39,777][INFO ][node                     ] [Doctor Bong] version[3.0.0-SNAPSHOT], pid[19044], build[2f5b6ea/2015-09-26T03:18:16Z]
...
```

Closes #13785
2015-09-25 23:32:52 -04:00
Nik Everett 2f5b6eabab Merge pull request #13754 from nik9000/immutable_set_be_gone
Removes and bans ImmutableSet
2015-09-25 21:46:18 -04:00
Nik Everett 892d6b3b89 Merge branch 'master' into immutable_set_be_gone 2015-09-25 21:05:51 -04:00
Jason Tedor f40ae25352 Another sync of ElasticsearchException ids with 2.0 2015-09-25 16:43:01 -04:00
Jason Tedor 5571c327c5 Sync ElasticsearchException ids with 2.0 2015-09-25 16:24:49 -04:00
Jason Tedor 65577816de Merge pull request #13796 from s1monw/remove_reflection_hacks_from_es_exception
Remove reflection hacks from ElasticsearchException
2015-09-25 15:02:48 -04:00
Robert Muir aa8d1740e4 Don't log multi-megabyte guice exceptions.
Instead just log the same thing we print to the startup console for that case (magic logic),
it sucks to do this, but guice exceptions are too much.

All other non-guice exceptions will still be fully logged.

Closes #13782
2015-09-25 15:02:16 -04:00
Jason Tedor 75ecc54f33 Add clarifying comment on removing an ElasticsearchException 2015-09-25 14:59:10 -04:00
Jason Tedor ab19bfbe9b Removing and reordering ElasticsearchExceptions should be okay 2015-09-25 14:43:40 -04:00
Jason Tedor 752b4798d1 Use explicit ids for ElasticsearchExceptions
This commit adds explicit ids for managing ElasticsearchException
serialization. By adding explicit ids and unit tests for them, the ids
are less brittle and breakage can be more clearly detected.
2015-09-25 14:32:44 -04:00
Boaz Leskes 00d8a84475 Test: reduce size in ByteSizeValueTests.testEquality to avoid double rounding issues 2015-09-25 20:24:11 +02:00
Boaz Leskes 5a0f2fd1ac ByteSizeValue.equals should normalize units
currently ByteSizeValue.parse("1GB") is not equal to ByteSizeValue.parse("1024MB")

Closes #13784
2015-09-25 20:05:13 +02:00
Britta Weber de73910a13 add -DtestScript option 2015-09-25 19:04:28 +02:00
Colin Goodheart-Smithe c379505525 Changed pom to output first 30 test failures instead of just 3 2015-09-25 17:09:09 +01:00
Simon Willnauer f4ff6647ad Remove reflection hacks from ElasticsearchException
Today we use reflection where it's not needed anymore since java8 can
pass ctors around. This commit replaces runtime checks with compile time
checks which is always preferrable.
2015-09-25 16:42:42 +02:00
Nik Everett bfa3dc5a0c Fix merge error 2015-09-25 10:25:47 -04:00
Simon Willnauer 2a94085605 Merge pull request #13794 from s1monw/kill_es_deletion_policy
Remove ES internal deletion policies in favour of Lucenes implementations
2015-09-25 16:12:56 +02:00
Simon Willnauer d5d4c9b140 Remove ES internal deletion policies in favour of Lucenes implementations
These classes are really duplicates and are just here for historical reasons.
We don't need these anymore since the same classes exist in lucene today.
This also removes the guice injection for DeletionPolicy and make them shard private.
2015-09-25 16:11:46 +02:00
Clinton Gormley 0c0f530c31 Merge pull request #13781 from KimTaehee/master
fix typo "Seting" to "Setting"
2015-09-25 15:28:04 +02:00
Nik Everett 5949b83115 Merge branch 'master' into immutable_set_be_gone 2015-09-25 09:09:24 -04:00
javanna eaafc62f53 [DOCS] added query-refactoring changes to 3.0 migrate guide 2015-09-25 14:55:37 +02:00
javanna 3d3197342d Merge branch 'master' into merge/query-refactoring 2015-09-25 14:38:53 +02:00
Clinton Gormley c2296b1d68 Merge pull request #13757 from andrestc/docs-now-time_zone
"now" is not affected by "time_zone" in range queries
2015-09-25 13:55:23 +02:00