1815 Commits

Author SHA1 Message Date
Jason Tedor
ad090296fc Workaround JDK-8056014
This commit works around JDK bug JDK-8056014 in the javac compiler.
This bug is impacting CI compilations on JDK 8u11 and 8u25.
2015-09-29 15:03:32 +02:00
Robert Muir
94972bb39d Merge pull request #13829 from rmuir/add_seatbelt
improve seccomp syscall filtering
2015-09-29 08:54:06 -04:00
Colin Goodheart-Smithe
8c7c7652f6 Make sure equivalent geohashCellQueries are equal after toQuery called
Previous to this change if to equal geohash cell query builders were created and then toQuery was called on one, they would no longer be equal.

This change also adds a test to AbstractQueryTestCase to make sure calling toQuery on any query builder does not affect the query builder's equality
2015-09-29 14:34:58 +02:00
Simon Willnauer
e814102bfb Ensure XContent is consistent across platforms
Today we generate XContent with platform dependent linefeeds. This
commit makes the pretty-printed json etc. consistent with \n across all
platforms.
2015-09-29 14:22:33 +02:00
Simon Willnauer
e3b38e5af1 Remove IndexService dep. from IndexShard
there is no reason for the index shard to hold on to it's corresponding
index service. This dependency is unnecessary.
2015-09-29 14:07:58 +02:00
Robert Muir
cbf894a2e7 Add abstract TestScript to the test framework jar 2015-09-29 06:37:33 -04:00
Robert Muir
20a203b1a2 Merge pull request #13834 from rmuir/groovy_factor_out
Factor groovy out of core into lang-groovy
2015-09-29 05:58:21 -04:00
Tanguy Leroux
9cb27ad068 Update to Jackson 2.6.2 2015-09-29 10:20:54 +02:00
Boaz Leskes
c1ab2dc244 Test: make sure TranslogTests.randomNonTranslogPatternString returns a reference to a file
It can return a directory now (like /  or ..)
2015-09-29 07:53:31 +02:00
Robert Muir
e0d42739dd Factor groovy out of core into lang-groovy 2015-09-28 20:17:45 -04:00
Robert Muir
d7a087a41b improve seccomp syscall filtering
* Add OS X support via "seatbelt" mechanism. This gives consistency across dev and prod, since many devs use OS X.
* block execveat system call: it may be new, but we should not allow it.
2015-09-28 11:35:07 -04:00
Nik Everett
85b99d2011 Finish banning ImmutableSet
Ban ImmutableSet$Builder because that let you sneak some `ImmutableSet`s in.

Remove all remaining imports of ImmutableSet.
2015-09-28 01:49:46 +02:00
Boaz Leskes
148265bd16 Internal: an inactive shard is temporarily activated by triggered synced flush
When a shard becomes in active we trigger a sync flush in order to speed up future recoveries. The sync flush causes a new translog generation to be made, which in turn confuses the IndexingMemoryController making it think that the shard is active. If no documents comes along in the next 5m, the shard is made inactive again , triggering a sync flush and so forth.

To avoid this, the IndexingMemoryController is changed to ignore empty translogs when checking if a shard became active. This comes with the price of potentially missing indexing operations which are followed by a flush. This is acceptable as if no more index operation come in, it's OK to leave the shard in active.

A new unit test is introduced and comparable integration tests are removed.

Closes #13802
2015-09-27 07:16:54 +02:00
Simon Willnauer
ebe02ec54a [TEST] Make test pass on windows - jackson uses platform line.separator which messes up comparisons 2015-09-26 21:25:49 +02:00
Nik Everett
892d6b3b89 Merge branch 'master' into immutable_set_be_gone 2015-09-25 21:05:51 -04:00
Jason Tedor
f40ae25352 Another sync of ElasticsearchException ids with 2.0 2015-09-25 16:43:01 -04:00
Jason Tedor
5571c327c5 Sync ElasticsearchException ids with 2.0 2015-09-25 16:24:49 -04:00
Jason Tedor
65577816de Merge pull request #13796 from s1monw/remove_reflection_hacks_from_es_exception
Remove reflection hacks from ElasticsearchException
2015-09-25 15:02:48 -04:00
Robert Muir
aa8d1740e4 Don't log multi-megabyte guice exceptions.
Instead just log the same thing we print to the startup console for that case (magic logic),
it sucks to do this, but guice exceptions are too much.

All other non-guice exceptions will still be fully logged.

Closes #13782
2015-09-25 15:02:16 -04:00
Jason Tedor
75ecc54f33 Add clarifying comment on removing an ElasticsearchException 2015-09-25 14:59:10 -04:00
Jason Tedor
ab19bfbe9b Removing and reordering ElasticsearchExceptions should be okay 2015-09-25 14:43:40 -04:00
Jason Tedor
752b4798d1 Use explicit ids for ElasticsearchExceptions
This commit adds explicit ids for managing ElasticsearchException
serialization. By adding explicit ids and unit tests for them, the ids
are less brittle and breakage can be more clearly detected.
2015-09-25 14:32:44 -04:00
Boaz Leskes
00d8a84475 Test: reduce size in ByteSizeValueTests.testEquality to avoid double rounding issues 2015-09-25 20:24:11 +02:00
Boaz Leskes
5a0f2fd1ac ByteSizeValue.equals should normalize units
currently ByteSizeValue.parse("1GB") is not equal to ByteSizeValue.parse("1024MB")

Closes #13784
2015-09-25 20:05:13 +02:00
Simon Willnauer
f4ff6647ad Remove reflection hacks from ElasticsearchException
Today we use reflection where it's not needed anymore since java8 can
pass ctors around. This commit replaces runtime checks with compile time
checks which is always preferrable.
2015-09-25 16:42:42 +02:00
Nik Everett
bfa3dc5a0c Fix merge error 2015-09-25 10:25:47 -04:00
Simon Willnauer
d5d4c9b140 Remove ES internal deletion policies in favour of Lucenes implementations
These classes are really duplicates and are just here for historical reasons.
We don't need these anymore since the same classes exist in lucene today.
This also removes the guice injection for DeletionPolicy and make them shard private.
2015-09-25 16:11:46 +02:00
Nik Everett
5949b83115 Merge branch 'master' into immutable_set_be_gone 2015-09-25 09:09:24 -04:00
javanna
3ac4da5f84 Merge branch 'master' into feature/query-refactoring 2015-09-25 13:53:06 +02:00
Luca Cavanna
ee71fc61aa Merge pull request #13787 from javanna/enhancement/terms_lookup_query_builder
Java api: remove TermsLookupQueryBuilder
2015-09-25 12:06:33 +02:00
Martijn van Groningen
04cfbe361b date math expressions should also work when indexing documents into a none existing index.
Closes #13570
2015-09-25 12:05:47 +02:00
javanna
629ac8e014 Java api: remove TermsLookupQueryBuilder
TermsLookupQueryBuilder was left around only for bw comp reasons, but TermsQueryBuilder is its replacement. We can remove it now that it is clear query refactoring goes in master (3.0).
2015-09-25 11:44:05 +02:00
Martijn van Groningen
93ad696966 Index name expressions should not be broken up
Closes #13665
2015-09-25 11:28:52 +02:00
Simon Willnauer
64d5ea13d5 Move ShardTermVectorService to be on indices level as TermVectorService
There is no need to have term vectors service on the shard level where it's
created for every shard. This commit moves it to a higher level which makes
shard creation slightly simpler and reduces the number of long living objects.
2015-09-25 10:56:27 +02:00
Christoph Büscher
74d641181e Query Refactoring: validate GeoShapeQueryBuilder strategy and relation parameter
Before the refactoring we didn't check any invalid settings for strategy and relation
in the GeoShapeQueryBuilder. However, using SpatialStrategy.TERM and ShapeRelation.INTERSECTS
together is invalid and we tried to protect against that in the validate() method.

This PR moves these checks to setter for strategy and relation and adds tests for the new
behaviour.

Relates to #10217
2015-09-25 10:52:12 +02:00
javanna
a50a0da183 [TEST] adapt to changes upstream, expression has been moved out
Queries that can include scripts make now use of the mock script engine
2015-09-25 10:20:22 +02:00
javanna
34de79370f Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/ElasticsearchException.java
2015-09-25 09:38:24 +02:00
javanna
4425cd6570 [TEST] fix test bug 2015-09-25 09:16:02 +02:00
Nik Everett
fe8f0b97e4 Another formatting fix 2015-09-24 16:50:04 -04:00
Isabel Drost-Fromm
18d3ab381f Switch geo validation to enum 2015-09-24 22:33:08 +02:00
Simon Willnauer
152775c515 Move ShardPercolateService creation into IndexShard
This commit removes an unnecessarily guice injected service and
simply creates an instance where it's needed using plain old java ctors
2015-09-24 21:29:59 +02:00
Robert Muir
8b88a69d9c Block process execution with seccomp on linux/amd64
Block execve(), fork(), and vfork() system calls, returning EACCES instead,
on kernels that support seccomp-bpf: either via seccomp() or falling back
to prctl().

Only linux/amd64 is supported. This feature can be disabled (in case
of problems) with bootstrap.seccomp=false.

Closes #13753

Squashed commit of the following:

commit 92cee05c72b49e532d41be7b16709e1c9f919fa9
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Sep 24 10:12:51 2015 -0400

    Add a note about why we don't parse uname() or anything

commit b427971f45cbda4d0b964ddc4a55fae638880335
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Sep 24 09:44:31 2015 -0400

    style only: we already pull errno into a local, use it for catch-all case

commit ddf93305525ed1546baf91f7902148a8f5b1ad06
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Sep 24 08:36:01 2015 -0400

    add TODO

commit f29d1b7b809a9d4c1fcf15f6064e43f7d1b24696
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Sep 24 08:33:28 2015 -0400

    Add full stacktrace at debug level always

commit a3c991ff8b0b16dc5e128af8fb3dfa6346c6d6f1
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Sep 24 00:08:19 2015 -0400

    Add missing check just in case.

commit 628ed9c77603699aa9c67890fe7632b0e662a911
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 23 22:47:16 2015 -0400

    Add public getter, for stats or whatever if they need to know this

commit 3e2265b5f89d42043d9a07d4525ce42e2cb1c727
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 23 22:43:06 2015 -0400

    Enable use of seccomp(2) on Linux 3.17+ which provides more protection.
    Add nice errors.
    Add all kinds of checks and paranoia.
    Add documentation.
    Add boolean switch.

commit 0e421f7fa2d5236c8fa2cd073bcb616f5bcd2d23
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 23 21:36:32 2015 -0400

    Add defensive checks and nice error messages

commit 6231c3b7c96a81af8460cde30135e077f60a3f39
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 23 20:52:40 2015 -0400

    clean up JNA and BPF. block fork and vfork too.

commit bb31e8a6ef03ceeb1d5137c84d50378c270af85a
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 23 19:00:32 2015 -0400

    order is LE already for the JNA buffer, but be explicit about it

commit 10456d2f08f12ddc3d60989acb86b37be6a4b12b
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 23 17:47:07 2015 -0400

    block process execution with seccomp on linux/amd64
2015-09-24 13:55:26 -04:00
Nik Everett
f8fa00185f Better format some stream usage 2015-09-24 13:53:05 -04:00
Nik Everett
a4f709c6b8 Fix mistake and revert Stream 2015-09-24 13:52:49 -04:00
Nik Everett
63da3b1a7a Remove methods from code review 2015-09-24 13:43:05 -04:00
javanna
db9c2796b3 Query refactoring: last //norelease addressed
Some methods had default implementation while queries were going to be refactored, now that they are all refactored all those methods can be made abstract.
2015-09-24 18:00:36 +02:00
javanna
aae7faa88e Query refactoring: minor cleanups before we merge into master
Remove ParseFieldMatcher duplication query in both contexts. QueryParseContext is still contained in QueryShardContext, as parsing still happens in the shards here and there. Most of the norelease comments have been removed simply because the scope of the refactoring has become smaller. Some could only be removed once everything, the whole search request, gets parsed on the coordinating node. We will get there eventually.
2015-09-24 17:53:03 +02:00
Christoph Büscher
1968d3e0f0 Tests: fixing minor issues in RangeQueryBuilderTests and MatchQueryBuilderTests 2015-09-24 15:43:56 +02:00
javanna
8976934d3b Remove IndexQueryParseService#parse methods used only in tests
SimpleIndexQueryParserTests was the main responsible: deleted lots of duplicated tests, moved the ones that made sense to keep to their corresponding unit tests (note they were ESSingleNode tests before while are now converted to unit tests).

Closes #13750
2015-09-24 13:35:36 +02:00
Christoph Büscher
eb8ea63626 Query Refactoring: remove deprecated methods and temporary classes
After all queries now have a `toQuery` method and the parsers all
support `fromXContent` it is possible to remove the following
workarounds and deprecated methods we kept around while doing the
refactoring:

* remove the BaseQueryParser and BaseQueryParserTemp. All parsers
  implement QueryParser directly now
* remove deprecated methods in QueryParseContext that either returned
  a Query or a Filter.
* remove the temporary QueryWrapperQueryBuilder

Relates to #10217
2015-09-24 12:51:19 +02:00