Commit Graph

2062 Commits

Author SHA1 Message Date
Jason Tedor 7f12806a7d Fix typo in o.e.c.r.a.d.ConcurrentRebalanceAllocationDecider 2015-10-16 10:06:33 -04:00
javanna a1c1a8b230 Query DSL: don't change state of the bool query while converting to lucene query
We used to change the minimumShouldMatch field of the query depending on the context, the final minimim should match should still be applied based on that, but the original minimumShouldMatch of the query shouldn't change. This was revelead by some recent test failure.

Closes #14153
2015-10-16 14:33:09 +02:00
Adrien Grand 5ae810991c Upgrade to lucene-5.4-snapshot-1708254. 2015-10-16 09:41:36 +02:00
Simon Willnauer cd0084915d Allow parser to move on the START_OBJECT token when parsing search source
Currently we require parser to be right before the sources START_OBJECT
but if we are parsing embedded search sources this won't work since we potentially
moved already on to the START_OBJECT. This commit make this optional such that
both ways work.
2015-10-15 22:28:05 +02:00
Nik Everett 68d708a75f Merge pull request #11398 from Collaborne/bugs/explain-function-script-quotes
Fix the quotes in the explain message for a script score function without parameters
2015-10-15 13:56:21 -04:00
Colin Goodheart-Smithe cf3e92ae92 Merge branch 'master' into feature/search-request-refactoring 2015-10-15 13:51:00 +01:00
Colin Goodheart-Smithe ef5fc721e5 reverted non-null searchSourceBuilder in searchRequest for now 2015-10-15 13:49:43 +01:00
Robert Muir c3751801c6 Test that the lucene "unmap hack" is supported.
Closes #14120

Squashed commit of the following:

commit 556b7f5783211bd82a5d9796996e21be87b0404b
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 14 15:16:54 2015 -0400

    Add bugid link

commit b44aac7e9f1a97974938c17e013812ebf4c2fa76
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 14 15:01:41 2015 -0400

    Test that the lucene "unmap hack" is supported.

    We should know if this is not working for any configuration, otherwise resources such as address space, file handles, and even disk space become tied to Java's garbage collector.
2015-10-15 08:47:05 -04:00
Robert Muir 0dcac1448d Merge pull request #14126 from rmuir/seccomp_testbug
Fix SeccompTests bug on older kernels / add defense
2015-10-15 08:37:53 -04:00
javanna 0cab4a25fb Revert changes made to IndexQueryParserService, fixed upstream meanwhile 2015-10-15 14:35:16 +02:00
javanna 8a87448758 Revert changes made to IdsQueryBuilder, fixed upstream meanwhile 2015-10-15 14:28:39 +02:00
Colin Goodheart-Smithe 63c51b78b2 review comment fixes 2015-10-15 11:34:31 +01:00
Alexander Reelsen bbaf7ea0f0 Release 1.7.3: Added bwc indices and updated Version.java 2015-10-15 12:26:18 +02:00
Colin Goodheart-Smithe c618f75b76 Merge branch 'master' into feature/search-request-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/search/SearchService.java
2015-10-15 11:02:34 +01:00
Simon Willnauer 5b1ee8bd74 Fix GeoCenteriod Aggregation serialization 2015-10-15 09:57:47 +02:00
Robert Muir 9c109cae0d Fix SeccompTests bug on older kernels
This test failed only on a feature branch, because that feature branch has
a different build randomization script and is the only place
randomizing tests.security.manager (this test cannot run with it enabled).

On old kernels without TSYNC support, the test fails because (surprise to me) the thread that
runs the test is not the same thread that runs static initializers:
7571489190/randomized-runner/src/main/java/com/carrotsearch/randomizedtesting/RandomizedRunner.java (L573-L574)

To fix this test (its not an issue in practice, since we do this before creating threadpools),
we just record for testing purposes that we couldn't TSYNC, and re-run the whole thing for the test thread
in setUp(), failing if something goes wrong.

Also add a bunch of additional paranoia and narrow our defensive checks better here after reading
through more chrome bug reports: they don't impact us but those linux distros are too cowboy
with the backports and the spirit of the checks makes me feel better.
2015-10-15 01:40:32 -04:00
Jason Tedor 68561ce55c Merge pull request #14119 from jasontedor/index-meta-data-clean-up
Cleanup IndexMetaData
2015-10-14 19:29:08 -04:00
Nicholas Knize ceefe2e91a remove centroid calculation from GeoHashGridAggregation 2015-10-14 16:19:28 -05:00
Nicholas Knize b31d3ddd3e Adds geo_centroid metric aggregator
This commit adds a new metric aggregator for computing the geo_centroid over a set of geo_point fields. This can be combined with other aggregators (e.g., geohash_grid, significant_terms) for computing the geospatial centroid based on the document sets from other aggregation results.
2015-10-14 16:19:09 -05:00
Jason Tedor 88dd3256fb Cleanup IndexMetaData
This commit cleans up IndexMetaData. In particular, all duplicate
getters (X and getX) have been collapsed into one (getX). Further, the
number of shards and number of replicas settings are now parsed once
and saved off as fields.
2015-10-14 14:59:35 -04:00
Robert Muir 5d001d1578 Decentralize plugin security
* Add ability for plugins to declare additional permissions with a custom plugin-security.policy file and corresponding AccessController logic. See the plugin author's guide for more information.
* Add warning messages to users for extra plugin permissions in bin/plugin.
* When bin/plugin is run interactively (stdin is a controlling terminal and -b/--batch not supplied), require user confirmation.
* Improve unit test and IDE support for plugins with additional permissions by exposing plugin's metadata as a maven test resource.

Closes #14108

Squashed commit of the following:

commit cf8ace65a7397aaccd356bf55f95d6fbb8bb571c
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 14 13:36:05 2015 -0400

    fix new unit test from master merge

commit 9be3c5aa38f2d9ae50f3d54924a30ad9cddeeb65
Merge: 2f168b8 7368231
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 14 12:58:31 2015 -0400

    Merge branch 'master' into off_my_back

commit 2f168b8038e32672f01ad0279fb5db77ba902ae8
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 14 12:56:04 2015 -0400

    improve plugin author documentation

commit 6e6c2bfda68a418d92733ac22a58eec35508b2d0
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 14 12:52:14 2015 -0400

    move security confirmation after 'plugin already installed' check, to prevent user from answering unnecessary questions.

commit 08233a2972554afef2a6a7521990283102e20d92
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 14 05:36:42 2015 -0400

    Add documentation and pluginmanager support

commit 05dad86c51488ba43ccbd749f0164f3fbd3aee62
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Oct 14 02:22:24 2015 -0400

    Decentralize plugin permissions (modulo docs and pluginmanager work)
2015-10-14 14:46:45 -04:00
javanna 736823163f Plugins: plugin script to set proper plugin bin dir attributes
This commit makes sure that the plugin script looks at user, group and permissions of the elasticsearch bin dir and copies them over to the plugin bin subdirectory, whatever they are, so that they get properly setup depending on how elasticsearch was installed. We also make sure that execute permissions are added for files (we already did this before).

Relates to #11016
Closes #14088
2015-10-14 14:59:38 +02:00
Simon Willnauer c133bec4bd Ensure searcher is release if wrapping fails
Today we leak an index searcher if we fail to wrap the seacher in
IndexShard. This commit ensures that the seacher is released if the wrapper
throws an exception.

This commit also restructures some test to be more atomic and only test a single
feature / attribute of the wrapper.
2015-10-14 10:44:57 +02:00
Simon Willnauer 5828796848 Merge pull request #14084 from s1monw/close_the_wrapper_only
Streamline top level reader close listeners and forbid general usage
2015-10-14 09:39:30 +02:00
Simon Willnauer db710c576f add javadocs 2015-10-14 09:28:05 +02:00
Simon Willnauer 7999027bf5 apply review comments 2015-10-14 09:19:02 +02:00
Jason Tedor 2e445d3ede Do not pollute Cache with failed futures 2015-10-13 22:14:13 -04:00
Jason Tedor b154c84306 Avoid deadlocks in Cache#computeIfAbsent
This commit changes the behavior of Cache#computeIfAbsent to not invoke
load for a key under the segment lock. Instead, the synchronization
mechanism to ensure that load is invoked at most once per key is
through the use of a future. Under the segment lock, we put a future in
the cache and through this ensure that load is invoked at most once per
key. This will not lead to the same deadlock situation as before
because a dependent key load on the same thread can not be triggered
while the segment lock is held.

Closes #14090
2015-10-13 21:15:28 -04:00
Simon Willnauer cac073dafa enforce that wrappers delegate core cache key and ban getCombinedCoreAndDeletesKey() entirely 2015-10-13 23:31:25 +02:00
Simon Willnauer ec60018e34 add comment why and when we have a leaf reader in the warmer 2015-10-13 21:06:20 +02:00
Simon Willnauer 0ead0faa1b fix typo 2015-10-13 20:51:43 +02:00
Simon Willnauer 1dca0e8f9b Add simple tests to ensure we can load and unload global ordinal with a wrapped reader 2015-10-13 20:51:10 +02:00
Simon Willnauer e3f00e302c Add more pickyness to index warming and searcher wrappping
this commit also fixes a bug where we wramed a leaf reader in a top level context
which caused atomic segment readers to be used in our top level caches.
2015-10-13 17:21:46 +02:00
Simon Willnauer dac1799305 Cast DirectoryReader only when really requested 2015-10-13 17:21:46 +02:00
Simon Willnauer ba8de12429 remove unnecessary cast 2015-10-13 17:21:46 +02:00
Simon Willnauer bd5ac9ce1a enforce DirectoryReader when globals are loaded 2015-10-13 17:21:46 +02:00
Simon Willnauer d3436ff592 Streamline top level reader close listeners and forbid general usage
IndexReader#addReaderCloseListener is very error prone when it comes to
caching and reader wrapping. The listeners are not delegated to the sub readers
nor can it's implementation change since it's final in the base class. This commit
only allows installing close listeners on the top level ElasticsearchDirecotryReader
which is known to work an has a defined lifetime which corresponds to its subreader.
This ensure that cachesa re cleared once the reader goes out of scope.
2015-10-13 17:21:18 +02:00
jaymode 077a401c28 [TEST] do not wipe excluded templates
In #13606, the ability to exclude a template from being wiped was added but a call to
wipeTemplates() was not removed, so all index templates were still being removed.

This change removes the wipeTemplates() call in the wipe method so that only the
non-excluded templates are removed.
2015-10-13 09:59:32 -04:00
javanna 4d7d29f65b Plugins: plugin script to set proper plugin config dir attributes
Depending on how elasticsearch is installed, we have two scenarios to take into account that relate to user, group and permissions assigned to the config directory:

1) deb/rpm package: /etc/elasticsearch is root:elasticsearch 750 and the plugin script is run from root user
2) tar/zip archive: es config dir is most likely elasticsearch:elasticsearch and the plugin script is most likely run from elasticsearch user

When the plugin script copies over the plugin config dir within the es config dir, it should take care of setting the proper user, group and permissions, which vary depending on how elasticsearch was installed in the first place. Should be root:elasticsearch 750 if installed from a package, or elasticsearch:elasticsearch if installed from an archive.

This commit makes sure that the plugin script looks at user, group and permissions of the config dir and copies them over to the plugin config subdirectory, whatever they are, so that they get properly setup depending on how elasticsearch was installed in the first place. We also make sure that execute permissions are left untouched for files.

Relates to #11016
Closes #14048
2015-10-13 15:56:26 +02:00
Britta Weber 1634c8364e Merge pull request #14085 from brwe/fix-score-accuracy
fix numerical issue in function score query
2015-10-13 15:21:04 +02:00
Jason Tedor b7e09582ce Merge pull request #14087 from jasontedor/vle-integral-arrays
Add methods for variable-length encoding integral arrays
2015-10-13 09:12:10 -04:00
Robert Muir ceb969a13e Make PerThreadIDAndVersionLookup per-segment
Closes #14070

Squashed commit of the following:

commit ad5829e590cf6150763c31886e97d69976b7f02d
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Oct 13 06:50:25 2015 -0400

    Add more tests for version map caching

commit 617333eefd6d4ddc99db491d0843827b1b5936b2
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Oct 12 13:25:51 2015 -0400

    fix javadocs ref

commit 9626dd02e01a2676538e1071e302d882fc870837
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Oct 12 13:23:16 2015 -0400

    make the version map cache per-segment
2015-10-13 09:00:04 -04:00
Yannick Welsch a550ebf381 Merge pull request #13998 from ywelsch/tests/transportmasternodeaction
Tests for TransportMasterNodeAction
2015-10-13 05:48:40 -07:00
Jason Tedor 2162172f30 Add methods for variable-length encoding integral arrays
This commit adds methods to serialize the elements of int and long
arrays using variable-length encodings. This can be useful for
serializing int and long arrays containing mostly non-negative “not
large” values in a compressed form.
2015-10-13 08:47:42 -04:00
Yannick Welsch c2df88a21d [TEST] Add tests for TransportMasterNodeAction
Closes #13998
2015-10-13 14:46:59 +02:00
Britta Weber d021b1bfeb assert that doc was actually found 2015-10-13 14:35:32 +02:00
Yannick Welsch 57770d4d6c Fix import in SharedClusterSnapshotRestoreIT 2015-10-13 13:56:46 +02:00
Yannick Welsch 889edd00f0 Merge pull request #13931 from ywelsch/fix/index-settings-restore
Snapshot restore and index creates should keep index settings and cluster blocks in sync
2015-10-13 04:36:02 -07:00
Britta Weber 318dfba464 use same score computation for actual scoring and explanation
FiltersFunctionScoreQuery sums up scores and weights and scores as double but when
we explain we cannot get the double scores from the explanation of score
functions. as a result we cannot compute the exact score from the explanations
of the functions alone.
this commit makes the explanation more accurate but also causes the score to be
computed one additional time.
2015-10-13 13:30:53 +02:00
Robert Muir a68c163959 Merge pull request #14064 from rmuir/blinded_forever
Allow tests to run from eclipse on windows
2015-10-13 06:58:19 -04:00