Commit Graph

2245 Commits

Author SHA1 Message Date
Simon Willnauer 255338111b test and fix TranslogStats - you wouldn't believe how hard it is to sum up two values 2015-10-08 09:24:22 +02:00
David Pilato c73ab50df1 Rename cloud-gce plugin to discovery-gce plugin
Follow up azure and aws splits, we need to be consistent and rename `cloud-gce` to `discovery-gce`.
2015-10-08 06:53:37 +02:00
Jason Tedor 23cd64b767 Merge pull request #13903 from jasontedor/evicting-queue-be-gone
Remove and forbid use of com.google.common.collect.EvictingQueue
2015-10-07 21:04:59 -04:00
Robert Muir 780ccb98e8 Test: add address formatting round trip tests 2015-10-07 20:57:55 -04:00
Jason Tedor e61e7463e3 Add Javadocs for EvictingQueue 2015-10-07 20:51:53 -04:00
Jason Tedor b7c7c5f4f2 Remove and forbid use of com.google.common.collect.EvictingQueue
This commit removes and now forbids all uses of
com.google.common.collect.EvictingQueue across the codebase. This is
one of the few remaining steps in the eventual removal of Guava as a
dependency.

Relates #13224
2015-10-07 20:51:53 -04:00
Igor Motov ea99d97d8f TransportNodesAction shouldn't hold to cluster state
Long running TransportNodesAction requests can retain old cluster states in memory for much longer than needed. This can cause nodes with frequent cluster state updates and long running requests to run out of memory.
2015-10-07 20:42:20 -04:00
Jason Tedor 9eddc3c1c9 Remove and forbid use of com.google.common.net.InetAddresses
This commit removes and now forbids all uses of
com.google.common.net.InetAddresses across the codebase. This is one of
the few remaining steps in the eventual removal of Guava as a
dependency.

Relates #13224
2015-10-07 20:11:48 -04:00
Itamar Syn-Hershko 1f7a7bacbb Update TransportClientNodesService.java 2015-10-08 01:08:35 +03:00
Igor Motov a358f34276 Expose nodes operation timeout in REST API
Currently it's not possible to specify a timeout for nodes operations (such as node info, node stats, cluster stats and hot threads) via REST-based APIs.
2015-10-07 18:07:59 -04:00
Itamar Syn-Hershko 4a7810944d Fix ensureNodesAreAvailable's error message
listedNodes are the "configured nodes" and not the empty list of nodes that is passed to the method and causes this exception to be thrown
2015-10-08 01:07:31 +03:00
Ryan Ernst 42718936d9 Merge pull request #14003 from rjernst/fix/13740
Mappings: Enforce metadata fields are not passed in documents
2015-10-07 14:40:28 -07:00
Simon Willnauer 890f607b95 add test and fix another bug on the way 2015-10-07 22:30:35 +02:00
David Pilato 289cd5dcf4 [discovery-gce] add _gce_ network host setting
When running in GCE platform, an instance has access to:

http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/ip

Which gives back the private IP address, for example `10.240.0.2`.

http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/0/externalIp

Gives back the public Ip address, for example `130.211.108.21`.

As we have for `ec2`, we can support new network host settings:

* `_gce:privateIp:X_`: The private IP address of the machine for a given network interface.
* `_gce:hostname_`: The hostname of the machine.
* `_gce_`: Same as `_gce:privateIp:0_` (recommended).

Closes #13605.
Closes #13590.

BTW resolveIfPossible now throws IOException so code is also updated for ec2 discovery and
some basic tests have been added.
2015-10-07 22:04:34 +02:00
Ryan Ernst 566fef0cde Mappings: Enforce metadata fields are not passed in documents
We previously removed the ability to specify metadata fields inside
documents in #11074, but the backcompat left leniency that allowed this
to still occur. This change locks down parsing so any metadata field
found while parsing a document results in an exception. This only
affects 2.0+ indexes; backcompat is maintained.

closes #13740
2015-10-07 12:52:13 -07:00
Simon Willnauer bfa0202816 Don't pull translog from shadow engine
ShadowEngine doesn't have a translog but instead throws an
UOE when it's requested. ShadowIndexShard should not try to pull
stats for the translog either and should return null instead.

Closes #12730
2015-10-07 20:47:58 +02:00
Simon Willnauer 2064b17e3f Simplify similarity module and friends
SimilarityModule was binding two different interfaces SimilaritySerivce and SimilarityLookupSerice.
Both used a class Similarities which was holding some default impls etc. Thit commit folds
all the logic into a rather simplified SimilarityService which has not construction time dependency to
any other service in the system anymore. It's soely constructued from custom similarities, the index name
and index settings and SimilarityModule is just trivial glue code with out much logic.
This also adds a simple unittest for basic test coverage of the service.
2015-10-07 20:36:46 +02:00
Jason Tedor 2881c4fa94 Remove and forbid use of com.google.common.collect.Iterators
This commit removes and now forbids all uses of
com.google.common.collect.Iterators across the codebase. This is one of
the final steps in the eventual removal of Guava as a dependency.

Relates #13224
2015-10-07 12:41:45 -04:00
Michael McCandless 23f97c30a0 Merge branch 'master' into immediate_shard_active
Conflicts:
	core/src/main/java/org/elasticsearch/index/engine/Engine.java
	core/src/main/java/org/elasticsearch/index/shard/IndexShard.java
2015-10-07 12:20:27 -04:00
Colin Goodheart-Smithe a16b025f0a re-enables InnerHits Tests 2015-10-07 16:59:56 +01:00
Jason Tedor 5328f145ef Clarify use of System.nanoTime for measuring eviction times 2015-10-07 11:03:29 -04:00
Jason Tedor ff8f9c9332 Add comment clarifying CacheTests.testCacheStats 2015-10-07 10:56:42 -04:00
Jason Tedor 37becf3dc2 Rename Cache.exceedsSize for clarity 2015-10-07 10:46:09 -04:00
Jason Tedor 44b471b008 Rename LRU and segment locks for clarity 2015-10-07 10:43:56 -04:00
Jason Tedor 1f61384988 Protect against null RemovalListener in setter 2015-10-07 10:40:22 -04:00
Jason Tedor 0fb908178d Clarify Javadoc comment regarding CacheSegment locks 2015-10-07 10:36:05 -04:00
Colin Goodheart-Smithe ee5d2847b5 more clean up 2015-10-07 15:34:21 +01:00
Colin Goodheart-Smithe 5b23d9abdc more test clean ups 2015-10-07 15:29:13 +01:00
Michael McCandless 7f435e2f4f feedback 2015-10-07 10:18:52 -04:00
Colin Goodheart-Smithe 3313742826 removed defaultRescoreWindow 2015-10-07 15:09:35 +01:00
Clinton Gormley 884215a782 Added version ID for 2.0.0-rc1 2015-10-07 15:13:09 +02:00
Clinton Gormley 90df3c6195 Added bwc indices for 2.0.0-rc1 2015-10-07 15:13:08 +02:00
Britta Weber 871df97e18 Merge pull request #13961 from brwe/function-score-unit-tests
Convert some messy function score tests to unit tests
2015-10-07 14:51:22 +02:00
Britta Weber bb939f819c use searcher.createNormalizedWeight() instead of query.getWeight() and cleanup 2015-10-07 14:46:47 +02:00
javanna 53f316b540 GeoDistanceRangeQueryBuilder to not change its state when calling toQuery
The geoPoint gets normalized while calling toQuery. That should happen on a copy of the point though, the state of the request should never change as part of toQuery execution. Also updated corresponding test to support point normalization.

Closes #13984
2015-10-07 12:54:24 +02:00
javanna a024125f8d Java api: GeoDistanceRangeQueryBuilder#getValidationMethod to not take any argument 2015-10-07 12:54:24 +02:00
Britta Weber ab74e68c5c Merge pull request #13101 from brwe/update-ang
Send response for update request when it timed out
2015-10-07 12:53:22 +02:00
Britta Weber 13d3a3e257 send response for update request when it timed out 2015-10-07 12:51:57 +02:00
Boaz Leskes bcb3fab6ac Engine: Remove Engine.Create
The `_create` API is handy way to specify an index operation should only be done if the document doesn't exist. This is currently implemented in explicit code paths all the way down to the engine. However, conceptually this is no different than any other versioned operation - instead of requiring a document is on a specific version, we require it to be deleted (or non-existent). This PR removes Engine.Create in favor of a slight extension in the VersionType logic.

There are however a couple of side effects:
- DocumentAlreadyExistsException is removed and VersionConflictException is used instead (with an improved error message)
- Update will reject version parameters if the upsert option is used (it doesn't compute anyway).
- Translog.Create is also removed infavor of Translog.Index (that's OK because their binary format was the same, so we can just read Translog.Index of the translog file)

Closes #13955
2015-10-07 12:37:34 +02:00
Michael McCandless 934827d25f fold feedback 2015-10-06 17:19:39 -04:00
Britta Weber 71aefd5a06 remove option to configure custom config file via CONF_FILE or -Des.default.conf
It is rarely used and was not consistently handled by different distributions anyway.
This commit also adds a test for specifying CONF_DIR when installing plugins and
starting elasticsearch.

relates to #12712 and #12954
closes #5329
closes #13715
2015-10-06 19:02:43 +02:00
javanna 9d1c45f3e0 [TEST] Minor SearchSourceBuilderTests infra changes 2015-10-06 18:13:05 +02:00
javanna cbcc23b8be [TEST] Simplify SearchSourceBuilderTests setup
Mappings and many modules are not needed here compared to AbstractQueryTestCase, as we will never call toQuery in this other test. Parsing is independent from indices and types as well.
2015-10-06 17:45:41 +02:00
javanna 1915c74e93 Merge branch 'master' into feature/search-request-refactoring 2015-10-06 16:21:58 +02:00
javanna 108f5c5701 Count Request: minScore can be null, its getter should be Float rather than float 2015-10-06 16:16:57 +02:00
javanna c2fc65352a [TEST] move variable assignment one line above 2015-10-06 16:00:06 +02:00
javanna fcc60609ae Count api: simplify default handling
For minScore and terminateAfter we can just rely on defaults set to SearchSourceBuilder.
2015-10-06 15:55:34 +02:00
Colin Goodheart-Smithe 10a559348b more fixes for tests 2015-10-06 14:50:48 +01:00
javanna 86be9db7b9 Fix some RequestBuilder#toString that produced broken json
Also restored a couple of old tests around search and count request builder that used to get wiped when calling toString.
2015-10-06 15:46:11 +02:00
Colin Goodheart-Smithe 669a5893bb fixed some NORELEASE comments in tests 2015-10-06 14:36:50 +01:00
Nik Everett bb2611d2f5 Merge branch 'master' into immutable_map_be_gone 2015-10-06 09:13:58 -04:00
javanna 80d236e1a0 Revert "Count api: simplify default handling and toString representation"
This reverts commit ef286ce972.
2015-10-06 14:57:34 +02:00
javanna ef286ce972 Count api: simplify default handling and toString representation
For minScore and terminateAfter we can just rely on defaults set to SearchSourceBuilder. Also simplified toString representation of CountRequestBuilder
2015-10-06 14:41:03 +02:00
javanna f39e23b40e pass in QuerySourceBuilder directly to ValidateQueryRequest 2015-10-06 14:39:35 +02:00
javanna f89de33548 Merge branch 'master' into feature/search-request-refactoring 2015-10-06 14:28:31 +02:00
Nik Everett 82f9e977ad Fully remove and ban ImmutableMap 2015-10-06 08:16:13 -04:00
Britta Weber d2ca694200 Merge pull request #13933 from brwe/plugin-cli-logging
plugin cli tool should not create empty log files
2015-10-06 14:13:59 +02:00
Britta Weber 9492be65d4 plugin cli tool should not create empty log files
Plugin cli tools configures logging with whatever is in the logging.yml.
If a file appender is configured for any of the logs this will cause creation
of an empty log file. If a plugin was for example installed as root it will
create empty logs at es.home/logs.
This is problematic when for example plugins are installed as root and es is run
as service. Logs will then be created in /usr/share/elasticsearch/logs
and can later not be removed by for example dpkg -r or -purge.

To avoid this, configure the logger to use an appender that writes to the same
output that plugin cli tool does. This allows other components that are called
from Plugin cli tool to write to the same terminal that plugin cli tool writes to
by using the logging mechanism already in place.
The logging conf is not read at all pb plugin cli tool.

As a side effect, the loging level for components that are called
from the plugin command such as the jar hell check can now be configured
with -Des.logger.level which makes it easier to debug the jar hell check.
2015-10-06 14:13:24 +02:00
Adrien Grand 56c2c24f5a Remove ScriptEngineService.execute.
This methods was only used in tests and can be replaced by calling
`ScriptEngineService.executable(compiledScript, vars).run()` instead.
2015-10-06 13:27:27 +02:00
javanna 7e840532c1 Make strategy optional in GeoShapeQueryBuilder readFrom and writeTo
The field is optional everywhere else but in the serialization methods, which causes problems. Also expanded tests so that they can catch this type of problem.

Closes #13963
2015-10-06 13:18:47 +02:00
Britta Weber 473d25beed convert weight functions tests to unit tests 2015-10-06 12:08:59 +02:00
Britta Weber 0915adaa71 convert explain function score tests to unit tests 2015-10-06 11:20:35 +02:00
Adrien Grand bc98895d18 Remove ScriptEngineService.unwrap.
The ability to unwrap script values is already exposed via ExecutableScript.unwrap.
2015-10-06 10:30:15 +02:00
Michael McCandless 0aabfe383c improve javadocs 2015-10-06 04:27:52 -04:00
Michael McCandless a082135538 Merge branch 'master' into immediate_shard_active
Conflicts:
	core/src/main/java/org/elasticsearch/index/shard/IndexShard.java
	core/src/main/java/org/elasticsearch/index/shard/ShadowIndexShard.java
	core/src/main/java/org/elasticsearch/indices/memory/IndexingMemoryController.java
2015-10-06 04:19:56 -04:00
Michael McCandless c5971272ee fold feedback 2015-10-06 03:59:37 -04:00
javanna 2fd1cde35e [TEST] move back some test from groovy plugin to core
Relates to #13837

Closes #13945
2015-10-06 09:19:54 +02:00
Nik Everett 34e28ea6ff Remove ImmutableMap$Builder from tests! 2015-10-05 16:37:24 -04:00
Nik Everett 380dbbfb23 Ban ImmutableMap$Builder in core's main
Almost there!
2015-10-05 15:42:17 -04:00
Nik Everett ba68a8df63 Merge branch 'master' into immutable_map_be_gone 2015-10-05 14:00:53 -04:00
Simon Willnauer 3ab3938501 Merge pull request #13944 from s1monw/simplify_index_modules
Remove unneeded Module abstractions
2015-10-05 17:43:20 +02:00
Simon Willnauer fce55a15b8 Remove unneeded Module abstractions
These abstractions don't really do anything nor can they be extended.
We can just fold them into IndexModule for now. There are more but they
are tricky due to some test dependencies which I need to resolve first.
2015-10-05 17:33:03 +02:00
Robert Muir 4ad1bf0716 Merge pull request #13924 from rmuir/ireallyhatescripts
lock down javascript and python script engines better
2015-10-05 11:29:26 -04:00
Nik Everett 672a54b39a Make return type of MapBuilder#immutableMap Map
We won't be removing that method - just deprecating it. So we need to remove
ImmutableMap from it. And the only way to do that is not to return one.
2015-10-05 10:49:27 -04:00
Colin Goodheart-Smithe a3a2432ebd Merge branch 'master' into feature/search-request-refactoring 2015-10-05 14:43:04 +01:00
Colin Goodheart-Smithe 9b1d1b239c fixes 2015-10-05 14:31:45 +01:00
Colin Goodheart-Smithe ec93531303 fixes 2015-10-05 14:28:25 +01:00
Colin Goodheart-Smithe 7055a05e6e test fixes 2015-10-05 14:19:26 +01:00
Colin Goodheart-Smithe d59e959c80 rename RestActions.parseQuerySource() to RestActions.urlParamsToQueryBuilder() 2015-10-05 14:11:59 +01:00
javanna e8653f5156 Java api: IdsQueryBuilder to accept only non null ids and types
Types are still optional, but if you do provide them, they can't be null. Split the existing constructor that accepted nnull into two, one that accepts no arguments, and another one that accepts the types argument, which must be not null.

Also trimmed down different ways of setting ids, some were misleading as they would always add the ids to the existing ones and not set them, the add prefix makes that clear. Left `addIds` method that accepts a varargs argument. Added check for ids not be null.
2015-10-05 15:10:30 +02:00
Colin Goodheart-Smithe 81491843f1 test fixes 2015-10-05 14:10:16 +01:00
javanna 3a0d1841d9 Query refactoring: simplify IndexQueryParserService parse methods and prepare the field for #13859
Relates to #13859
2015-10-05 14:50:15 +02:00
Colin Goodheart-Smithe e8ef63cc00 compile fix 2015-10-05 13:43:34 +01:00
Nik Everett f484290e5e Deprecate MapBuilder#immutableMap
I'm not 100% sure we should remove it as part of the pull request to drop
ImmutableMap. It might be more prudent to change its return type to map
and its implementation to an unmodifiable copy of the map being built
and then remove all consumers after banning ImmutableMap.
2015-10-05 08:41:30 -04:00
Nik Everett 46d10f1b6f More progress 2015-10-05 08:38:03 -04:00
Colin Goodheart-Smithe ec51e8e31d Merge branch 'master' into feature/search-request-refactoring 2015-10-05 13:31:52 +01:00
Colin Goodheart-Smithe 7fbd565c5c review comment fixes 2015-10-05 13:27:14 +01:00
Simon Willnauer efdecfa161 [TEST] Add tests to ensure that Get uses wrapped searcher / reader 2015-10-05 14:23:55 +02:00
Simon Willnauer 4676eb19a4 add tests for IndexSearcherWrapper 2015-10-05 14:07:18 +02:00
Simon Willnauer 623a519988 also wrap searcher when it's used for Get calls 2015-10-05 14:07:18 +02:00
Simon Willnauer 674a9851cf rename fooSafe into getFoo and getFooOrNull 2015-10-05 14:07:17 +02:00
Simon Willnauer e94f242456 more index level cleanups 2015-10-05 14:07:17 +02:00
Simon Willnauer d6b1f4ce6c Make Percolator a first class citizen in IndexShard and prevent premature index searcher access 2015-10-05 14:07:17 +02:00
Simon Willnauer a892a35f40 Hide engine entirely in IndexShard and do searcher wrapping only on top of the engine 2015-10-05 14:07:17 +02:00
Simon Willnauer d2e3e8cc7b more cleanups 2015-10-05 14:07:17 +02:00
Simon Willnauer c0eca94a04 Remove shard-level injector
Today we use a hirachical injector on the shard level for each shard
created. This commit removes the shard level injetor and replaces
it with good old constructor calls. This also removes all shard level plugin
facilities such that plugins can only have node or index level modules.
For plugins that need to track shard lifecycles they should use the relevant
callback from the lifecycle we already provide.
2015-10-05 14:07:17 +02:00
Simon Willnauer 188aa684ac Merge pull request #13923 from s1monw/off_by_one
Record all bytes of the checksum in VerifyingIndexOutput
2015-10-05 14:03:47 +02:00
Britta Weber 6e29facd0a Merge pull request #13934 from brwe/log-config-order
settings in log config file should not overwrite custom parameters
2015-10-05 13:49:55 +02:00
Britta Weber ceefb06752 settings in log config file should not overwrite custom parameters 2015-10-05 13:29:35 +02:00
xuzha 668371c945 Forbid index name with '.' and '..'.
Fixes #13858
2015-10-04 20:05:43 -07:00
Jason Tedor d3cef85352 Remove unnecessary call to MessageDigest.reset 2015-10-04 21:02:50 -04:00
Robert Muir 8ff42834e9 lock down javascript and python permissions 2015-10-04 17:13:47 -04:00
Jason Tedor bde4889daf Add field for expiration conditions 2015-10-04 16:56:15 -04:00
Jason Tedor e5a10e9520 Merge pull request #13907 from jasontedor/hash-be-gone
Remove and forbid use of com.google.common.hash.*
2015-10-04 16:26:12 -04:00
Jason Tedor 67d1c70c2d Remove and forbid use of com.google.common.hash.*
This commit removes and now forbids all uses of
com.google.common.hash.HashCode, com.google.common.hash.HashFunction,
and com.google.common.hash.Hashing across the codebase. This is one of
the few remaining steps in the eventual removal of Guava as a
dependency.

Relates #13224
2015-10-04 16:01:24 -04:00
Simon Willnauer 96206dfd2a [TEST] Work around how OS / GIT handles line separator 2015-10-04 21:46:40 +02:00
Simon Willnauer 95406c4701 Record all bytes of the checksum in VerifyingIndexOutput
The fix in #13848 has an off by one issue where the first byte of the checksum
was never written. Unfortunately most tests shadowed the problem and the first
byte of the checksum seems to be very likely a 0 which causes only very rare
failures.

Relates to #13896
Relates to #13848
2015-10-04 21:11:02 +02:00
xuzha b19be2c34a DiskThresholdDecider check data nodes number
Right now, we allow allocation if there is only a single node in the
cluster. it would be nice to fail open when there is only one data node
(instead of only one node total).

closes #9391
2015-10-03 16:16:51 -07:00
Michael McCandless 19ab16b9a5 set last indexing time before invoking IMC 2015-10-03 17:29:01 -04:00
Michael McCandless 934cc091e6 fix tests; pull out translog buffer size constant 2015-10-03 17:15:59 -04:00
Yannick Welsch cdb00371da Merge pull request #13828 from ywelsch/fix-snapshot-restore-throttling
Snapshot restore operations throttle more than specified
2015-10-03 16:40:27 +02:00
Yannick Welsch 03a4e226f1 Snapshot restore operations throttle more than specified
Lucene's RateLimiter can do too much sleeping on small values (see also #6018).
The issue here is that calls to "pause" are not properly guarded in "restoreFile".

Instead of simply adding the guard, this commit uses the RateLimitingInputStream similar as for "snapshotFile".

Closes #13828
2015-10-03 16:39:00 +02:00
Michael McCandless f27c0adb0b a start 2015-10-03 05:09:07 -04:00
Simon Willnauer 04e892634e Verify Checksum once it has been fully written to fail as soon as possible
Today we are relying on calling Store.verify on the closed stream to validate the
checksum. This is still necessary to catch file truncation but for an actually corrupted
file or checksum we can fail early and check the checksum against the actual metadata
once it's been fully written to the VerifyingIndexOutput.
2015-10-02 21:49:07 +02:00
Jason Tedor 5fbf3494fe Merge pull request #13909 from jasontedor/immutable-collections-be-gone
Remove and forbid use of com.google.common.collect.ImmutableCollection
2015-10-02 15:18:34 -04:00
Jason Tedor 7447eb9842 Remove and forbid use of com.google.common.collect.ImmutableCollection
This commit removes and now forbids all uses of
com.google.common.collect.ImmutableCollection across the codebase. This
is one of the final steps in the eventual removal of Guava as a
dependency.

Relates #13224
2015-10-02 18:27:30 +02:00
Jason Tedor a5f9cd98d8 Remove and forbid use of com.google.common.o.Resources
This commit removes and now forbids all uses of
com.google.common.io.Resources across the codebase. This is one of the
few remaining steps in the eventual removal of Guava as a dependency.

Relates #13224
2015-10-02 17:42:40 +02:00
Michael McCandless 5278cf0d5e Merge pull request #13870 from mikemccand/close_tokenstream
Close TokenStream in finally clause
2015-10-02 14:43:26 +01:00
Nik Everett a378cc6866 Remove ImmutableMap#copyOf from core/src/main
Mostly favoring unmodifiableMap and making sure to only wrap maps that aren't
otherwise returned and so cannot be copied.

MapMaker#immutableMap has to go next.
2015-10-02 13:22:05 +02:00
Jason Tedor 105d830114 Correct condition for when time is needed 2015-10-02 11:55:36 +02:00
Nik Everett ab7fa7fe9e Remove multi-arg ImmutableMap#of variants 2015-10-02 04:01:40 +02:00
Nik Everett e75f1137f2 Finish removing ImmutableMap#of from tests 2015-10-02 02:44:28 +02:00
Nik Everett f68dabe615 Finish removing ImmutableMap#of
We'll need to be more careful with CopyOnWriteHashMap than I was at first.
2015-10-02 01:35:32 +02:00
Nik Everett e6303302e6 Remove more ImmutableMap#of
TODO - look for places where CopyOnWriteHashMap will do.
2015-10-02 00:40:38 +02:00
Nik Everett 9430e17f70 Yet more ImmutableMap#of removal
Did some refactoring of PathTrie along the way to remove warnings. Added a
simple toString to it to make it easier to debug.
2015-10-02 00:18:05 +02:00
Nik Everett a9d59024b9 More removing ImmutableMap#of() 2015-10-01 23:29:29 +02:00
Nik Everett bd2202bf21 Start removing ImmutableMap#of 2015-10-01 22:52:07 +02:00
Nik Everett a0288742e7 Remove and ban unmodifiableMap in cluster package
We're concerned that unmodifiableMap uses significantly more memory than
ImmutableMap did - especially in cluster state - so we ban it there outright
and move to ImmutableOpenMap.

Removes ClusterState$Builder#routingTable(RoutingTable$Builder) because that
method had the side effect of building the routing table which can only be
done once per RoutingTable$Builder now that it uses ImmutableOpenMap.
2015-10-01 21:48:40 +02:00
Xu Zhang 7b74f0ddc9 Merge pull request #13783 from andrestc/feature/disk-used-by-es
Adds disk used by indices to _cat/allocation

closes #13529
2015-10-01 11:26:58 -07:00
Simon Willnauer d9b68a7674 Call verify on index input after copying 2015-10-01 16:48:37 +02:00
Simon Willnauer bd66c4bb0d [TEST] Add some debug output to HasChildQueryBuilderTests 2015-10-01 16:13:34 +02:00
André Carvalho 03c6e8e1cb Adds disk used by indices to _cat/allocation
Sets Store flag on request
2015-10-01 08:16:58 -03:00
Simon Willnauer edac9c17fa [TEST] ensure files are synced otherwise MDW will corrupt them afterwards 2015-10-01 10:46:55 +02:00
David Pilato 264fb5f3a2 Merge remote-tracking branch 'origin/master' 2015-10-01 07:14:08 +02:00
Jason Tedor 01e7378804 Cache#computeIfAbsent loader can throw checked exceptions 2015-10-01 02:08:14 +02:00
Jason Tedor c100d18f86 Start test threads at the same time 2015-10-01 01:41:29 +02:00
Jason Tedor 50cfe71b2a Forbidden means verboten! 2015-10-01 01:18:26 +02:00
Jason Tedor 4efe7b9c18 Replace CyclicBarrier with CountDownLatch 2015-10-01 01:16:33 +02:00
Jason Tedor 716be91345 Remove hidden synchronization from test 2015-10-01 01:04:20 +02:00
Robert Muir 8c4bc7d10b Nuke ES_CLASSPATH appending, JarHell fail on empty classpath elements
Closes #13880

Squashed commit of the following:

commit 316a328e5032e580ba840db993d907631334aac0
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 30 16:57:47 2015 -0400

    windows is terrible

commit 0406b560c58bf833f8d77af9c7cf3386771dd9c5
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 30 16:43:09 2015 -0400

    Nuke ES_CLASSPATH appending

    Out of box, ES expects its stuff to be in particular places. We should not be appending to ES_CLASSPATH, allowing users to specify stuff there, like we do in elasticsearch.bin.sh

    If the user sets it, its not going to work out of box.

    Closes #13812

commit 415d8972df28eddec322bb6d70100a1993fa95f6
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 30 16:26:35 2015 -0400

    Fail hard on empty classpath elements.

    This can happen easily, if somehow old 1.x shellscripts survive and try to launch 2.x code.
    I have the feeling this happens maybe because of packaging upgrades or something.
    Either way: we can just fail hard and clear in this situation, rather than the current situation
    where CWD might be /, and we might traverse the entire filesystem until we hit an error...

    Relates to #13864
2015-09-30 18:59:27 -04:00
Jason Tedor 7a5e90fc45 Lookups from computeIfAbsent should propogate now 2015-10-01 00:47:08 +02:00
Jason Tedor 30bfea741e Cleanup formatting 2015-10-01 00:37:49 +02:00
David Pilato 28f82fb568 Update version incompatibility message for plugin manager
When the plugin manager does not find in `plugin-descriptor.properties` the exact same elasticsearch version it was built on
as the current elasticsearch version, it fails with a message like:

```
ERROR: Elasticsearch version [2.0.0-beta1] is too old for plugin [elasticsearch-mapper-attachments]
```

Actually, the message should be:

```
Plugin [elasticsearch-mapper-attachments] is incompatible with Elasticsearch [2.0.0.beta2]. Was designed for version [2.0.0.beta1].
```

The opposite is true. If you try to install a version of a plugin which was built with a newer version of elasticsearch, it will fail the same way:

```
Plugin [elasticsearch-mapper-attachments] is incompatible with Elasticsearch [2.0.0.beta1]. Was designed for version [2.0.0.beta2].
```
2015-10-01 00:35:17 +02:00
Jason Tedor 11d75226a9 Ensure that computeIfAbsent loader is invoked at-most once 2015-10-01 00:32:44 +02:00
Jason Tedor eb2ea01106 Use try-with-resources for lock acquisition 2015-10-01 00:15:34 +02:00
Jason Tedor 8c05d4f43d Remove unnecessary overrides of equals/hashCode in Cache.Entry 2015-09-30 22:46:14 +02:00
Jason Tedor 64727b78de Add support for expiration after write to Cache
This commit adds supports for expiration after writes to Cache. This
enables entries to expire after they were initially placed in the cache
without prolonging their life on retrieval. Replacements are considered
new writes.
2015-09-30 22:44:41 +02:00
Martijn van Groningen eace065931 Merge pull request #13878 from martijnvg/fix_some_messy_tests
fixed some messy tests
2015-09-30 22:13:50 +02:00
Martijn van Groningen dc858d2008 test: fixed some messy tests 2015-09-30 21:48:17 +02:00
Jason Tedor 5d340f5e6e Remove and forbid use of com.google.common.cache.*
This commit removes and now forbids all uses of
com.google.common.cache.Cache, com.google.common.cache.CacheBuilder,
com.google.common.cache.RemovalListener,
com.google.common.cache.RemovalNotification,
com.google.common.cache.Weigher across the codebase. This is a major
step in the eventual removal of Guava as a dependency.

Relates #13224
2015-09-30 21:45:57 +02:00
Jason Tedor aa8bfeb88c Create concurrent cache with flexible eviction policies
This commit adds a concurrent cache with flexible eviction policies. In
particular, this cache supports:

1. concurrency
2. weight-based evictions
3. time-based evictions
4. manual invalidation
5. removal notification
6. cache statistics

Closes #13717
2015-09-30 21:43:00 +02:00
Simon Willnauer 5915309939 Fix part size calculation if all parts are of the same size
Relates to #13574
2015-09-30 21:29:00 +02:00
Christoph Büscher 97db3d5ef6 Tests: fix test by making copy of geopoints in assertLuceneQuery 2015-09-30 20:47:39 +02:00
Simon Willnauer 4886562d78 Don't get IndexShard instance in the executor but use already availale instance
We try to get the index shard instance again from the index service on a different
threads while that shard might have already been closed or removed which can cause a NPE
instead of another expected expecption.
2015-09-30 20:43:22 +02:00
Simon Willnauer 9d12c38f44 [TEST] Don't corrupt extraFS files 2015-09-30 20:02:55 +02:00
Simon Willnauer f2ae29a1cb [TEST] don't run check index on an intentionally corrupted index 2015-09-30 20:00:53 +02:00
mikemccand a321300e9c another try-with-resources 2015-09-30 17:46:22 +02:00
Simon Willnauer e38f4cf01e Start making RecoverySourceHandler unittestable
This commit shuffels and rewrites some code in RecoverySourceHandler to make it
simpler and more unittestable. This commit doesn't change all parts of this class
neither is it fully tested yet. It's an important part of the infrastrucutre so I started
to make it better tested but I don't want to change everything in one go since it makes
review simpler and more detailed. Future commits will continue cleaning up the class and
add more tests.
2015-09-30 17:45:29 +02:00
mikemccand dc01450d95 cutover more Analyzer.tokenStream to try-with-resources 2015-09-30 17:37:26 +02:00
Simon Willnauer f0a8c10c87 [TEST] only trim for the comparison 2015-09-30 17:00:39 +02:00
Boaz Leskes 169d06cf9e Internal: Remove the disabled autogenerated id optimization from InternalEngine
If a document is indexed into ES with no id, ES will generate one for it. We used to have an optimization for this case where the engine will not try to resolve the ids of these request in the existing index but immediately try to index them. This optimization has proven to be the source of brittle bugs (solved!) and we disabled it in 1.5, preparing for it to be removed if no performance degradation was found. Since we haven't seen any such degradation we can remove it.

Along with the removal of the optmization, we can remove the autogenerate id flag on indexing requests and the can have duplicate flag. The only downside of the removal of the canHaveDuplicate flag is that we can't make sure any more that when we retry an autogenerated id create operation we will ignore any document already exists exception (See #9125 for background and discussion). To work around this, we don't set the operation to CREATE any more when we generate an id, so the resulting request will never fail when it finds an existing doc but do return a version of 2. I think that's acceptable.

Closes #13857
2015-09-30 16:16:51 +02:00
mikemccand bb613bcacd move close responsibility back down to SuggestUtils.analyze 2015-09-30 15:44:35 +02:00
Simon Willnauer c185a12ef8 Verify actually written checksum in VerifyingIndexOutput
today we don't verify that the actual checksum written to VerifyingIndexOutput
is the actual checksum we are expecting.
2015-09-30 15:33:05 +02:00
Simon Willnauer 6fff824402 Remove ClusterSerivce and IndexSettingsService dependency from IndexShard
We have two unneded heavy dependencies on IndexShard that are unneeded and only cause
trouble if you try to mock index shard. This commit removes IndexSettingsService as well as
ClusterSerivce from IndexShard to simplify future mocking and construction.
2015-09-30 14:19:26 +02:00
mikemccand 433774fa6a close TokenStream in finally 2015-09-30 12:59:00 +02:00
Tanguy Leroux d715dfd16c Fix blob size in writeBlob() method 2015-09-30 12:33:14 +02:00
javanna a3abfab865 Query refactoring: set has_parent & has_child types context properly
While refactoring has_child and has_parent query we lost an important detail around types. The types that the inner query gets executed against shouldn't be the main types of the search request but the parent or child type set to the parent query. We used to use QueryParseContext#setTypesWithPrevious as part of XContentStructure class which has been deleted, without taking care though of setting the types and restoring them as part of the innerQuery#toQuery call.

Meanwhile also we make sure that the original context types are restored in PercolatorQueriesRegistry

Closes #13863
2015-09-30 12:07:26 +02:00
Robert Muir b8b8bdac40 Add 2.2.0 version to master 2015-09-29 19:50:13 -04:00
Robert Muir 6d8c035f70 Add SpecialPermission to guard exceptions to security policy.
Closes #13854

Squashed commit of the following:

commit 42c1166efc55adda0d13fed77de583c0973e44b3
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Sep 29 11:59:43 2015 -0400

    Add paranoia

    Groovy holds on to a classloader, so check it before compilation too.
    I have not reviewed yet what Rhino is doing, but just be safe.

commit b58668a81428e964dd5ffa712872c0a34897fc91
Author: Robert Muir <rmuir@apache.org>
Date:   Tue Sep 29 11:46:06 2015 -0400

    Add SpecialPermission to guard exceptions to security policy.

    In some cases (e.g. buggy cloud libraries, scripting engines), we must
    grant dangerous permissions to contained cases. Those AccessController blocks
    are dangerous, since they truncate the stack, and can allow privilege escalation.

    This PR adds a simple permission to check before each one, so that unprivileged code
    like groovy scripts, can't do anything they shouldn't be allowed to do otherwise.
2015-09-29 17:32:56 -04:00
Robert Muir ad6bc5b94c Throw exception on unsupported OS, so we don't falsely report 'enabled' 2015-09-29 17:16:00 -04:00
Alex Chow 9bea7cbda8 Omit current* stats for OldShardStats (closes #13386) 2015-09-29 11:18:45 -07:00
Colin Goodheart-Smithe 117d8d2606 Merge branch 'master' into feature/search-request-refactoring 2015-09-29 18:50:52 +02:00
Colin Goodheart-Smithe 458f87c2fc removed OldSearchSourceBuilder 2015-09-29 18:50:13 +02:00
Simon Willnauer bdc21078ca [TEST] don't copy write locks or extraFS files 2015-09-29 17:01:46 +02:00
Jason Tedor 464d48cda6 Merge pull request #13847 from jasontedor/workaround-jdk-8056984
Workaround JDK-8056984
2015-09-29 16:49:44 +02:00
Jason Tedor ba0643da2b Workaround JDK-8056984
This commit works around JDK bug JDK-8056984 in the javac compiler.
This bug is impacting CI compilations on JDK 8u25.
2015-09-29 16:45:57 +02:00
Robert Muir d54dd63825 Merge pull request #13844 from rmuir/lock_down_classloader_and_reflection
Clean up scripting permissions.
2015-09-29 10:35:52 -04:00
Simon Willnauer 3b5ed08d49 Refactor StoreRecoveryService to be a simple package private util class
StoreRecoveryService used to be a pretty heavy class with lots of dependencies.
This class was basically not testable in isolation and had an async API with a listener.
This commit refactors this class to be a simple utility classs with a sync API hidden behind
the IndexShard interface. It includes single node tests and moves all the async properities to
the caller side.
Note, this change also removes the mapping update on master from the store recovery code since
it's not needed anymore in 3.0 because all stores have been subject to sync mapping updates such
that the master already has all the mappings for documents that made it into the transaction log.

Closes #13766
2015-09-29 15:59:21 +02:00
Robert Muir eeeb42abef Clean up scripting permissions.
Now that groovy is factored out, we contain this dangerous stuff there.

TODO: look into those test hacks inspecting class protection domains, maybe we can
clean that one up too.

TODO: generalize the GroovyCodeSourcePermission to something all script engines check,
before entering accesscontrollerblocks. this way e.g. groovy script cannot coerce
python engine into creating something with more privs if it gets ahold of it... we
should probably protect the aws/gce hacks in the same way.
2015-09-29 09:54:12 -04:00
Colin Goodheart-Smithe e15736aa97 fixes for some NORELEASE comments 2015-09-29 15:53:04 +02:00
Jason Tedor 882fe0784c Merge pull request #13842 from jasontedor/workaround-jdk-8056014
Workaround JDK-8056014
2015-09-29 15:43:25 +02:00
Nik Everett 8a7f9edf02 Merge pull request #13820 from nik9000/immutable_set_be_gone_2
Finish banning ImmutableSet
2015-09-29 15:34:34 +02:00
Jason Tedor ad090296fc Workaround JDK-8056014
This commit works around JDK bug JDK-8056014 in the javac compiler.
This bug is impacting CI compilations on JDK 8u11 and 8u25.
2015-09-29 15:03:32 +02:00
Robert Muir 94972bb39d Merge pull request #13829 from rmuir/add_seatbelt
improve seccomp syscall filtering
2015-09-29 08:54:06 -04:00
Colin Goodheart-Smithe a21238beda Merge branch 'master' into feature/search-request-refactoring
# Conflicts:
#	plugins/lang-groovy/src/test/java/org/elasticsearch/messy/tests/ChildQuerySearchTests.java
#	plugins/lang-groovy/src/test/java/org/elasticsearch/messy/tests/IndexedScriptTests.java
#	plugins/lang-groovy/src/test/java/org/elasticsearch/messy/tests/SearchFieldsTests.java
#	plugins/lang-groovy/src/test/java/org/elasticsearch/messy/tests/SearchStatsTests.java
#	plugins/lang-groovy/src/test/java/org/elasticsearch/messy/tests/SearchTimeoutTests.java
#	plugins/lang-groovy/src/test/java/org/elasticsearch/messy/tests/SimpleSortTests.java
#	plugins/lang-groovy/src/test/java/org/elasticsearch/script/groovy/GroovyScriptTests.java
#	plugins/lang-groovy/src/test/java/org/elasticsearch/script/groovy/GroovySecurityTests.java
2015-09-29 14:40:27 +02:00
Colin Goodheart-Smithe 8c7c7652f6 Make sure equivalent geohashCellQueries are equal after toQuery called
Previous to this change if to equal geohash cell query builders were created and then toQuery was called on one, they would no longer be equal.

This change also adds a test to AbstractQueryTestCase to make sure calling toQuery on any query builder does not affect the query builder's equality
2015-09-29 14:34:58 +02:00
Simon Willnauer e814102bfb Ensure XContent is consistent across platforms
Today we generate XContent with platform dependent linefeeds. This
commit makes the pretty-printed json etc. consistent with \n across all
platforms.
2015-09-29 14:22:33 +02:00
Simon Willnauer e3b38e5af1 Remove IndexService dep. from IndexShard
there is no reason for the index shard to hold on to it's corresponding
index service. This dependency is unnecessary.
2015-09-29 14:07:58 +02:00
Robert Muir cbf894a2e7 Add abstract TestScript to the test framework jar 2015-09-29 06:37:33 -04:00
Robert Muir 20a203b1a2 Merge pull request #13834 from rmuir/groovy_factor_out
Factor groovy out of core into lang-groovy
2015-09-29 05:58:21 -04:00
Tanguy Leroux 9cb27ad068 Update to Jackson 2.6.2 2015-09-29 10:20:54 +02:00
Boaz Leskes c1ab2dc244 Test: make sure TranslogTests.randomNonTranslogPatternString returns a reference to a file
It can return a directory now (like /  or ..)
2015-09-29 07:53:31 +02:00
Robert Muir e0d42739dd Factor groovy out of core into lang-groovy 2015-09-28 20:17:45 -04:00
Robert Muir d7a087a41b improve seccomp syscall filtering
* Add OS X support via "seatbelt" mechanism. This gives consistency across dev and prod, since many devs use OS X.
* block execveat system call: it may be new, but we should not allow it.
2015-09-28 11:35:07 -04:00
Nik Everett 85b99d2011 Finish banning ImmutableSet
Ban ImmutableSet$Builder because that let you sneak some `ImmutableSet`s in.

Remove all remaining imports of ImmutableSet.
2015-09-28 01:49:46 +02:00
Boaz Leskes 148265bd16 Internal: an inactive shard is temporarily activated by triggered synced flush
When a shard becomes in active we trigger a sync flush in order to speed up future recoveries. The sync flush causes a new translog generation to be made, which in turn confuses the IndexingMemoryController making it think that the shard is active. If no documents comes along in the next 5m, the shard is made inactive again , triggering a sync flush and so forth.

To avoid this, the IndexingMemoryController is changed to ignore empty translogs when checking if a shard became active. This comes with the price of potentially missing indexing operations which are followed by a flush. This is acceptable as if no more index operation come in, it's OK to leave the shard in active.

A new unit test is introduced and comparable integration tests are removed.

Closes #13802
2015-09-27 07:16:54 +02:00
Simon Willnauer ebe02ec54a [TEST] Make test pass on windows - jackson uses platform line.separator which messes up comparisons 2015-09-26 21:25:49 +02:00
Nik Everett 892d6b3b89 Merge branch 'master' into immutable_set_be_gone 2015-09-25 21:05:51 -04:00
Jason Tedor f40ae25352 Another sync of ElasticsearchException ids with 2.0 2015-09-25 16:43:01 -04:00
Jason Tedor 5571c327c5 Sync ElasticsearchException ids with 2.0 2015-09-25 16:24:49 -04:00
Jason Tedor 65577816de Merge pull request #13796 from s1monw/remove_reflection_hacks_from_es_exception
Remove reflection hacks from ElasticsearchException
2015-09-25 15:02:48 -04:00
Robert Muir aa8d1740e4 Don't log multi-megabyte guice exceptions.
Instead just log the same thing we print to the startup console for that case (magic logic),
it sucks to do this, but guice exceptions are too much.

All other non-guice exceptions will still be fully logged.

Closes #13782
2015-09-25 15:02:16 -04:00
Jason Tedor 75ecc54f33 Add clarifying comment on removing an ElasticsearchException 2015-09-25 14:59:10 -04:00
Jason Tedor ab19bfbe9b Removing and reordering ElasticsearchExceptions should be okay 2015-09-25 14:43:40 -04:00
Jason Tedor 752b4798d1 Use explicit ids for ElasticsearchExceptions
This commit adds explicit ids for managing ElasticsearchException
serialization. By adding explicit ids and unit tests for them, the ids
are less brittle and breakage can be more clearly detected.
2015-09-25 14:32:44 -04:00
Boaz Leskes 00d8a84475 Test: reduce size in ByteSizeValueTests.testEquality to avoid double rounding issues 2015-09-25 20:24:11 +02:00
Boaz Leskes 5a0f2fd1ac ByteSizeValue.equals should normalize units
currently ByteSizeValue.parse("1GB") is not equal to ByteSizeValue.parse("1024MB")

Closes #13784
2015-09-25 20:05:13 +02:00
Simon Willnauer f4ff6647ad Remove reflection hacks from ElasticsearchException
Today we use reflection where it's not needed anymore since java8 can
pass ctors around. This commit replaces runtime checks with compile time
checks which is always preferrable.
2015-09-25 16:42:42 +02:00
Nik Everett bfa3dc5a0c Fix merge error 2015-09-25 10:25:47 -04:00
Simon Willnauer d5d4c9b140 Remove ES internal deletion policies in favour of Lucenes implementations
These classes are really duplicates and are just here for historical reasons.
We don't need these anymore since the same classes exist in lucene today.
This also removes the guice injection for DeletionPolicy and make them shard private.
2015-09-25 16:11:46 +02:00
Nik Everett 5949b83115 Merge branch 'master' into immutable_set_be_gone 2015-09-25 09:09:24 -04:00
Colin Goodheart-Smithe 44722700b0 Merge branch 'feature/query-refactoring' into feature/search-request-refactoring
# Conflicts:
#	core/src/test/java/org/elasticsearch/search/fetch/FetchSubPhasePluginIT.java
2015-09-25 13:04:50 +01:00
Colin Goodheart-Smithe f1da2bb54c test fixes 2015-09-25 13:01:32 +01:00
javanna 3ac4da5f84 Merge branch 'master' into feature/query-refactoring 2015-09-25 13:53:06 +02:00
Colin Goodheart-Smithe 8d488c5088 Removed delete tests which accidentally got re-added on merge 2015-09-25 11:20:00 +01:00
Luca Cavanna ee71fc61aa Merge pull request #13787 from javanna/enhancement/terms_lookup_query_builder
Java api: remove TermsLookupQueryBuilder
2015-09-25 12:06:33 +02:00
Martijn van Groningen 04cfbe361b date math expressions should also work when indexing documents into a none existing index.
Closes #13570
2015-09-25 12:05:47 +02:00
javanna 629ac8e014 Java api: remove TermsLookupQueryBuilder
TermsLookupQueryBuilder was left around only for bw comp reasons, but TermsQueryBuilder is its replacement. We can remove it now that it is clear query refactoring goes in master (3.0).
2015-09-25 11:44:05 +02:00
Martijn van Groningen 93ad696966 Index name expressions should not be broken up
Closes #13665
2015-09-25 11:28:52 +02:00
Colin Goodheart-Smithe f1cf592020 Test Fixes 2015-09-25 10:28:15 +01:00
Colin Goodheart-Smithe 9f08d48d34 Merge branch 'feature/query-refactoring' into feature/search-request-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/index/query/IndexQueryParserService.java
#	core/src/test/java/org/elasticsearch/script/IndexedScriptIT.java
#	core/src/test/java/org/elasticsearch/script/OnDiskScriptIT.java
2015-09-25 10:14:53 +01:00
Colin Goodheart-Smithe 2b94126567 removed obselete NOCOMMIT 2015-09-25 09:57:28 +01:00
Simon Willnauer 64d5ea13d5 Move ShardTermVectorService to be on indices level as TermVectorService
There is no need to have term vectors service on the shard level where it's
created for every shard. This commit moves it to a higher level which makes
shard creation slightly simpler and reduces the number of long living objects.
2015-09-25 10:56:27 +02:00
Christoph Büscher 74d641181e Query Refactoring: validate GeoShapeQueryBuilder strategy and relation parameter
Before the refactoring we didn't check any invalid settings for strategy and relation
in the GeoShapeQueryBuilder. However, using SpatialStrategy.TERM and ShapeRelation.INTERSECTS
together is invalid and we tried to protect against that in the validate() method.

This PR moves these checks to setter for strategy and relation and adds tests for the new
behaviour.

Relates to #10217
2015-09-25 10:52:12 +02:00
Colin Goodheart-Smithe f3b581a6e0 Added ext section to search request for plugins that extend the fetch phase
tested with term vectors fetch
2015-09-25 09:40:16 +01:00
javanna a50a0da183 [TEST] adapt to changes upstream, expression has been moved out
Queries that can include scripts make now use of the mock script engine
2015-09-25 10:20:22 +02:00
javanna 34de79370f Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/ElasticsearchException.java
2015-09-25 09:38:24 +02:00
javanna 4425cd6570 [TEST] fix test bug 2015-09-25 09:16:02 +02:00
Nik Everett fe8f0b97e4 Another formatting fix 2015-09-24 16:50:04 -04:00
Isabel Drost-Fromm 18d3ab381f Switch geo validation to enum 2015-09-24 22:33:08 +02:00
Simon Willnauer 152775c515 Move ShardPercolateService creation into IndexShard
This commit removes an unnecessarily guice injected service and
simply creates an instance where it's needed using plain old java ctors
2015-09-24 21:29:59 +02:00
Robert Muir 8b88a69d9c Block process execution with seccomp on linux/amd64
Block execve(), fork(), and vfork() system calls, returning EACCES instead,
on kernels that support seccomp-bpf: either via seccomp() or falling back
to prctl().

Only linux/amd64 is supported. This feature can be disabled (in case
of problems) with bootstrap.seccomp=false.

Closes #13753

Squashed commit of the following:

commit 92cee05c72b49e532d41be7b16709e1c9f919fa9
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Sep 24 10:12:51 2015 -0400

    Add a note about why we don't parse uname() or anything

commit b427971f45cbda4d0b964ddc4a55fae638880335
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Sep 24 09:44:31 2015 -0400

    style only: we already pull errno into a local, use it for catch-all case

commit ddf93305525ed1546baf91f7902148a8f5b1ad06
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Sep 24 08:36:01 2015 -0400

    add TODO

commit f29d1b7b809a9d4c1fcf15f6064e43f7d1b24696
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Sep 24 08:33:28 2015 -0400

    Add full stacktrace at debug level always

commit a3c991ff8b0b16dc5e128af8fb3dfa6346c6d6f1
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Sep 24 00:08:19 2015 -0400

    Add missing check just in case.

commit 628ed9c77603699aa9c67890fe7632b0e662a911
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 23 22:47:16 2015 -0400

    Add public getter, for stats or whatever if they need to know this

commit 3e2265b5f89d42043d9a07d4525ce42e2cb1c727
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 23 22:43:06 2015 -0400

    Enable use of seccomp(2) on Linux 3.17+ which provides more protection.
    Add nice errors.
    Add all kinds of checks and paranoia.
    Add documentation.
    Add boolean switch.

commit 0e421f7fa2d5236c8fa2cd073bcb616f5bcd2d23
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 23 21:36:32 2015 -0400

    Add defensive checks and nice error messages

commit 6231c3b7c96a81af8460cde30135e077f60a3f39
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 23 20:52:40 2015 -0400

    clean up JNA and BPF. block fork and vfork too.

commit bb31e8a6ef03ceeb1d5137c84d50378c270af85a
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 23 19:00:32 2015 -0400

    order is LE already for the JNA buffer, but be explicit about it

commit 10456d2f08f12ddc3d60989acb86b37be6a4b12b
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Sep 23 17:47:07 2015 -0400

    block process execution with seccomp on linux/amd64
2015-09-24 13:55:26 -04:00
Nik Everett f8fa00185f Better format some stream usage 2015-09-24 13:53:05 -04:00
Nik Everett a4f709c6b8 Fix mistake and revert Stream 2015-09-24 13:52:49 -04:00
Nik Everett 63da3b1a7a Remove methods from code review 2015-09-24 13:43:05 -04:00
javanna db9c2796b3 Query refactoring: last //norelease addressed
Some methods had default implementation while queries were going to be refactored, now that they are all refactored all those methods can be made abstract.
2015-09-24 18:00:36 +02:00
javanna aae7faa88e Query refactoring: minor cleanups before we merge into master
Remove ParseFieldMatcher duplication query in both contexts. QueryParseContext is still contained in QueryShardContext, as parsing still happens in the shards here and there. Most of the norelease comments have been removed simply because the scope of the refactoring has become smaller. Some could only be removed once everything, the whole search request, gets parsed on the coordinating node. We will get there eventually.
2015-09-24 17:53:03 +02:00
Simon Willnauer 78d6d2987f fix groovy tests - we don't pass json anymore so no need to double escape quotes 2015-09-24 16:55:28 +02:00
Colin Goodheart-Smithe 35fc73f094 fix to make parseSource in SearchService work with source overriding template 2015-09-24 15:48:55 +01:00
Colin Goodheart-Smithe bd42d26a76 text fix 2015-09-24 15:19:34 +01:00
Colin Goodheart-Smithe 67237ba269 test fixes 2015-09-24 15:16:16 +01:00
Christoph Büscher 1968d3e0f0 Tests: fixing minor issues in RangeQueryBuilderTests and MatchQueryBuilderTests 2015-09-24 15:43:56 +02:00
Colin Goodheart-Smithe a9c6e4c051 Merge branch 'feature/query-refactoring' into feature/search-request-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/index/query/IndexQueryParserService.java
#	core/src/main/java/org/elasticsearch/index/query/TemplateQueryParser.java
2015-09-24 14:33:35 +01:00
Colin Goodheart-Smithe 851af9e100 fixing up NOCOMMITs and making the IndicesQueryParserService not serialise the QueryBuilder to and from son 2015-09-24 14:20:50 +01:00
javanna 8976934d3b Remove IndexQueryParseService#parse methods used only in tests
SimpleIndexQueryParserTests was the main responsible: deleted lots of duplicated tests, moved the ones that made sense to keep to their corresponding unit tests (note they were ESSingleNode tests before while are now converted to unit tests).

Closes #13750
2015-09-24 13:35:36 +02:00
Christoph Büscher eb8ea63626 Query Refactoring: remove deprecated methods and temporary classes
After all queries now have a `toQuery` method and the parsers all
support `fromXContent` it is possible to remove the following
workarounds and deprecated methods we kept around while doing the
refactoring:

* remove the BaseQueryParser and BaseQueryParserTemp. All parsers
  implement QueryParser directly now
* remove deprecated methods in QueryParseContext that either returned
  a Query or a Filter.
* remove the temporary QueryWrapperQueryBuilder

Relates to #10217
2015-09-24 12:51:19 +02:00
Boaz Leskes d121550a4f Internal: pending operations in the translog prevent shard from being marked as inactive
The IndexingMemoryController checks periodically if there is any indexing activity on the shard. If no activity is sean for 5m (default) the shard is marked as inactive allowing it's indexing buffer quota to given to other active shards.

Sadly the current check is bad as it checks for 0 translog operation. This makes the inactive wait for a flush to happen - which used to take 30m and since #13707 doesn't happen at all (as we rely on the synced flush triggered by inactivity). This commit fixes the check so it will work with any translog size.

Closes #13759
2015-09-24 12:01:07 +02:00
Simon Willnauer 6cbf2de592 Fire off another maybeFlush on finish to ensure we don't miss a pending flush 2015-09-24 09:28:36 +02:00
Colin Goodheart-Smithe 510d1896f7 Revert "correct IndexQueryParserService.parse(QueryBuilder<?>) so that it doesn't convert to JSON and back again"
This reverts commit cc9519eee3.
2015-09-24 00:00:09 +01:00
Colin Goodheart-Smithe cc9519eee3 correct IndexQueryParserService.parse(QueryBuilder<?>) so that it doesn't convert to JSON and back again 2015-09-23 23:58:13 +01:00
Colin Goodheart-Smithe ce7dde08d0 fix SearchSourceBuilder.fromXContent to allow sort to be an object 2015-09-23 23:12:36 +01:00
Nik Everett ffa5b27c75 Use utility 2015-09-23 17:39:43 -04:00
Nik Everett aa06433868 Fix silly mistake 2015-09-23 17:25:37 -04:00
Nik Everett 65041a8121 Entirely remove and ban ImmutableSet
The last usage was ImmutableMap#keySet
2015-09-23 16:26:16 -04:00
Simon Willnauer 1fd8c64f1f [TEST] Add more logging to IndexShardTests#testMaybeFlush() 2015-09-23 22:22:25 +02:00
Nik Everett 04c570461e Remove and ban ImmutableSet#of 2015-09-23 16:14:38 -04:00
Simon Willnauer c93eeb88ca fix inner_hits parsing in SearchService 2015-09-23 22:05:49 +02:00
Colin Goodheart-Smithe 7a209cc600 more test fixes 2015-09-23 20:52:57 +01:00
Simon Willnauer 04864dc7d7 Impl SearchSource#hashCode|equals|toString 2015-09-23 21:50:30 +02:00
Nik Everett b0ab02e35c Remove and ban ImmutableSet#copyOf
It was used heavily in the Guice we've embedded.
2015-09-23 15:37:49 -04:00
Simon Willnauer 95a7b6aa83 Add a late-parsing SearchSoruce wrapper for index warmers 2015-09-23 21:31:00 +02:00
Colin Goodheart-Smithe 2de450ca0b fixes and debugging 2015-09-23 19:57:59 +01:00
Simon Willnauer 0f6e40ede1 Merge pull request #13742 from s1monw/remove_dead_code
Remove `ExpressionScriptCompilationException` and `ExpressionScriptExecutionException`
2015-09-23 20:41:03 +02:00
Colin Goodheart-Smithe 85f4c39b69 more test fixes 2015-09-23 18:58:16 +01:00
Nik Everett 6ecda41485 Remove and ban ImmutableSet#builder 2015-09-23 13:55:34 -04:00
Colin Goodheart-Smithe 63da68e480 test fixes 2015-09-23 18:50:45 +01:00
Jason Tedor 4fe243a5ca Merge pull request #13701 from jasontedor/fix-ping-timeout-settings-inconsistency
Fix ping timeout settings inconsistencies
2015-09-23 13:17:46 -04:00
Colin Goodheart-Smithe 08ae68c195 Merge branch 'feature/query-refactoring' into feature/search-request-refactoring
# Conflicts:
#	core/src/test/java/org/elasticsearch/search/functionscore/DecayFunctionScoreIT.java
#	core/src/test/java/org/elasticsearch/search/functionscore/FunctionScoreFieldValueIT.java
#	core/src/test/java/org/elasticsearch/search/functionscore/FunctionScoreIT.java
#	core/src/test/java/org/elasticsearch/search/rescore/QueryRescorerIT.java
2015-09-23 18:07:30 +01:00
Jason Tedor d8b29f7beb Fix ping timeout settings inconsistencies
This commit fixes ping timeout settings inconsistencies in
ZenDiscovery. In particular, the documentation refers to the ping
timeout setting as discovery.zen.ping_timeout but the code was
ultimately using discovery.zen.ping.timeout if this was set.

This commit also changes all instances of the raw string
“discovery.zen.ping_timeout” to the constant
o.e.d.z.ZenDiscovery.SETTING_PING_TIMEOUT.

Finally, this commit removes the legacy setting
"discovery.zen.initial_ping_timeout".

Closes #6579, #9581, #9908
2015-09-23 12:58:23 -04:00
Colin Goodheart-Smithe 394e551365 removed sysouts 2015-09-23 17:34:28 +01:00
Colin Goodheart-Smithe 52d9664993 fix to sort parsing in SearchService 2015-09-23 17:33:05 +01:00
Colin Goodheart-Smithe 585ed07c5b unit test fixes 2015-09-23 17:26:18 +01:00
Christoph Büscher dbb01f5b43 Query Refactoring: Move MoreLikeThisQueryBuilder validation to constructor
The current MoreLikeThisQueryBuilder validation checks for existence of at
least one `like` text or item. This is hard to check in setters, so this PR
tries to change the construction of the query so that we can do these checks
already at construction time.

Changing to using arrays for fieldnames, likeTexts, likeItems, unlikeTexts
and unlikeItems. `likeTexts` and/or `likeItems` need to be specified at
construction time to validate we have at least one item there.

Relates to #10217
2015-09-23 18:20:46 +02:00
Colin Goodheart-Smithe 2e3cc50a0f removed sysout 2015-09-23 16:57:25 +01:00
Colin Goodheart-Smithe d7429a6c63 Search service can now fully populate SearchContext from SearchSourceBuilder 2015-09-23 16:40:58 +01:00
Nik Everett 13720eb583 Merge pull request #13724 from nik9000/immutable_map_entry_set_be_gone
Remove and ban ImmutableMap#entrySet
2015-09-23 11:10:03 -04:00
Nik Everett 52f3c89c3b Remove and ban ImmutableMap#entrySet
Banning `ImmutableSet` outright is too much to do all at once - this starts
the process by banning `ImmutableMap#entrySet` - one of the more common ways
that `ImmutableSet`s come up. It then starts to remove calls to
`ImmutableMap#entrySet` by changing declarations from `ImmutableMap` to `Map`.

Unfortunately this process is like pulling on a long, windy string and one
declaration change requires another which requires 5 more which in turn
require another few. So this change is rather large.

As such, to keep the changes manageable they only remove `ImmutableMap` from
the signatures that are needed for `entrySet` and make little effort to stop
using `ImmutableMap` internally. Removing the usages of `ImmutableMap`
complicates immutability guarantees and will be done separately.
2015-09-23 11:07:28 -04:00
jaymode 5b8b15e729 expose all addresses that the transports are bound to
In #12942, the NettyTransport and NettyHttpServerTransport were updated to allow for binding
to multiple addresses. However, the BoundTransportAddress holder only exposed the first address
that the transport was bound to and this object is used to populate the values returned to the user
via our APIs.

This change exposes all of the bound addresses in the BoundTransportAddress holder, which allows
for an accurate representation of all interfaces that elasticsearch is bound to and listening on.
2015-09-23 11:06:35 -04:00
Colin Goodheart-Smithe a53cf896dc Parsing of rescuer and sort ByteReferences.
Not the neatest implementation in the world. Maybe we should consider changing the builders so it is a single builder for sort and a single builder for rescore instead of a list of builders for each?
2015-09-23 15:43:35 +01:00
Jason Tedor b1965267ec Fix broken test in BootstrapCliParserTests 2015-09-23 10:37:55 -04:00
Jason Tedor 8d1d8f9c46 Merge pull request #13737 from jasontedor/cli-parameter-order-error-message
More helpful error message on parameter order
2015-09-23 10:14:13 -04:00
Jason Tedor 9e7c56b430 More helpful error message on parameter order
This commit addresses a confusing error message that arises when a
property parameter (e.g. -D) is after a double-dash parameter. The
current error message reports to the user that the parameter does not
start with “--". Adding the second dash as the error message suggests
causes the parameter to be silently ignored. This is confusing for the
user. With this commit, the user is now informed that the parameter
order is violated.

Relates e27ede48ce
2015-09-23 10:13:40 -04:00
Nicholas Knize 8fe9a11943 Fix GeoPointFieldMapper to index geohash at correct precision(s). Add test coverage. 2015-09-23 09:07:52 -05:00
Colin Goodheart-Smithe 083f28a704 remove index from QueryParseContext Constructor 2015-09-23 15:07:33 +01:00
Simon Willnauer a63a30bcc4 Remove `ExpressionScriptCompilationException` and `ExpressionScriptExecutionException`
These exceptions are useless and unused, since we are on a major verison we should remove
them. This commit also makes it easier to remove excepitons in the future.
2015-09-23 15:09:30 +02:00
Simon Willnauer 67a911ff24 [TEST] force REQUEST durabilty otherwise test will fail 2015-09-23 14:10:40 +02:00
Simon Willnauer c32b9c3846 Add back presumably redundant shouldFlush() check.
The check prevents a race condition since we can't use real locks here.
Relates to #13707
2015-09-23 13:57:12 +02:00
Boaz Leskes d62f864317 Reduced the number of ClusterStateUpdateTask variants
In the past ClusterStateUpdateTask was an interface and we had various derived marker interfaces to control behavior. Since then we moved ClusterStateUpdateTask to be an abstract class but we kept the old hierarchy of implementations. All of those (but the AckedClusterStateUpdateTask) can be folded into ClusterStateUpdateTask, adding correct default behavior.

Closes #13735
2015-09-23 13:43:20 +02:00
Robert Muir d6c1880f08 Merge pull request #13726 from rmuir/lang-expression
Factor expressions scripts out to lang-expression plugin
2015-09-23 07:32:15 -04:00
Clinton Gormley e27ede48ce Added a clarifying note about argument order in the bin/elasticsearch start script 2015-09-23 13:17:57 +02:00
Simon Willnauer 75e816400c Remove TranslogService and fold it into synchronous IndexShard API
This commit moves the size and ops based flush into a synchronous API into
IndexShard and removes the time-based flush alltogether since it' basically
covered by the inactive async flush API we have today. The functionality doesn't
need to be covered by scheduled task and async APIs while we can actually make all
the decisions in a sync manner which is way easier to control and to test.

Closes #13707
2015-09-23 12:39:06 +02:00
Simon Willnauer 6c335f4a47 Parse source, extra-source and template-source on the coordinating node
This commit removes all the opaque bytes for extra_source and template_source.
Instead source and extra_source etc. are represented as SearchSourceBuilder which can
in-place be modified and is updated with the content of the request parameters.
Template Source is parsed and evaluated which in-turn replaces the actual source.
2015-09-23 12:24:07 +02:00
Colin Goodheart-Smithe ac8c1722ac fix compile errors 2015-09-23 11:07:20 +01:00
Colin Goodheart-Smithe b98f7cf023 updates to fix some of the NOCOMMITs 2015-09-23 10:44:24 +01:00
Colin Goodheart-Smithe ab01ec5b28 SearchRequest.source() now returns a SearchSourceBuilder.
Lots of NOCOMMITS are in the code as lots of compile errors have been temporarily commented out
2015-09-23 10:44:23 +01:00
Colin Goodheart-Smithe 41e63505b9 muted failing test 2015-09-23 10:44:23 +01:00
Colin Goodheart-Smithe aa3d29dfbc small parsing fix for fields 2015-09-23 10:44:23 +01:00
Colin Goodheart-Smithe b4dd6a5429 fixes to tests 2015-09-23 10:44:23 +01:00
Colin Goodheart-Smithe 3cb3514474 clean up for methods in SearchRequestBuilder so it works with SearchSourceBuilder 2015-09-23 10:44:23 +01:00
Colin Goodheart-Smithe e1759b1a60 cleanup codebase to work with SearchSourceBuilder 2015-09-23 10:44:22 +01:00
Colin Goodheart-Smithe 9116c1c19f Renamed SearchSourceBuilder to OldSearchSourceBuilder and NewSearchSourceBuilder to SearchSourceBuilder 2015-09-23 10:44:22 +01:00
Colin Goodheart-Smithe 3c8c6c0250 Completed serialisation to/from son and transport protocol and added a unit test 2015-09-23 10:44:22 +01:00
Colin Goodheart-Smithe eed6ab7b82 various fixes after a review 2015-09-23 10:44:22 +01:00
Colin Goodheart-Smithe e73052bce2 cleanup of commented out code 2015-09-23 10:44:22 +01:00
Colin Goodheart-Smithe 5623dd2950 implemented hashCode, equals and fromXContent in NewSearchSourceBuidler 2015-09-23 10:44:22 +01:00
Colin Goodheart-Smithe a96f14ecc6 new SearchSourceBuilder to use as the search request 2015-09-23 10:44:22 +01:00
javanna 7eedd84dc3 Query refactoring: refactor query_string query and score functions
Refactor the function_score query so it can be parsed on the coordinating node, split parse into fromXContent and toQuery, make FunctionScoreQueryBuilder Writeable.

Closes #13653
2015-09-23 11:32:54 +02:00
javanna 6350470ee3 Remove index member from QueryParseContext
Given that we are moving to parsing queries on the coordinating node, the index name is not relevant anymore in QueryParseContext, as the parsing phase cannot be related to any specific index. On the contrary, the QueryShardContext is the one that holds mappings etc. and the index name too, as the lucene query creation happens on the data node and can still be related with the index that it happens against.

Changes are mainly around tests that were expecting the index name, moved to using QueryShardException in some of them, removed the index name elsewhere.

Closes #13631
2015-09-23 10:30:53 +02:00
Michael McCandless 4fb6386df3 Merge pull request #13718 from mikemccand/catch_throwable
LoggingRunnable.run should catch and log all errors, not just Exception
2015-09-23 04:24:27 -04:00
Ryan Ernst cfb1e6c6c2 Refactor file script tests to be real unit tests 2015-09-22 23:39:22 -07:00
Ryan Ernst 05957cc4f3 rewrite custom scriptcontext tests to be unit tests 2015-09-22 21:16:10 -07:00
Robert Muir b65a631d9b i am fucking done taking out your trash: have it back, clean it up yourself. just let me secure the system. 2015-09-22 23:50:28 -04:00
Robert Muir 689af1a6d6 Factor expressions scripts out to lang-expression plugin 2015-09-22 20:33:47 -04:00
Nik Everett e284210652 [core] Forbid ForwardingSet
Removes CopyOnWriteHashSet, our only usage of ForwardingSet. We weren't
using it.

Related to #13224
2015-09-22 12:31:08 -04:00
Michael McCandless de8eadab93 catch Throwable instead of Exception in LoggingRunnable.run 2015-09-22 11:36:16 -04:00
Chris Earle 6d3202b6b5 Remove unnecessary suppression 2015-09-22 10:41:50 -04:00
Chris Earle 764051b4e3 Moving system property setting to before it can be used 2015-09-22 10:41:50 -04:00
Christoph Büscher 5ebb074e57 Fixing warnings and errors in javadoc 2015-09-22 13:59:34 +02:00
Christoph Büscher 1aae68d2e8 Merge branch 'master' into feature/query-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/index/query/CommonTermsQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/GeoPolygonQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/MatchQueryBuilder.java
	core/src/main/java/org/elasticsearch/index/query/QueryParseContext.java
	core/src/main/java/org/elasticsearch/index/query/QueryParser.java
2015-09-22 12:43:55 +02:00
Christoph Büscher 7d887a1e05 Tests: fix failing GeoDistanceRangeQueryTest 2015-09-22 12:13:53 +02:00
Britta Weber 997ea08e7b remove tests. they are fragile, hard to debug and add no value
closes #13542
closes #13343
closes #13316
2015-09-22 11:42:24 +02:00
Christoph Büscher 03035a28a3 Query Refactoring: moving validation to setters and constructors
Moving validation from validate() to constructors and setters for the
following query builders:

* GeoDistanceQueryBuilder
* GeoDistanceRangeQueryBuilder
* GeoPolygonQueryBuilder
* GeoShapeQueryBuilder
* GeohashCellQuery
* TermsQueryBuilder

Relates to #10217
2015-09-22 11:05:56 +02:00
Simon Willnauer ac2be641b3 add missing license header 2015-09-22 10:03:19 +02:00
Simon Willnauer 2c9a280878 only set matcher if parser is not null 2015-09-22 09:34:00 +02:00
Simon Willnauer 5f258b1585 Add a ObjectParser helper for stream parsing
This parser prototype allows to decleratively define parsers for XContent
instead of writing messy and error prone while loops. It encapsulates all the error handling logic
and only even tries to parse if the token types match the declaration.
2015-09-22 09:26:17 +02:00
Robert Muir b582de79ae Merge pull request #13702 from rmuir/broke_javadocs
Fix all javadocs issues, re-enable compiler warnings (but disable on java 9 where maven is broken)
2015-09-22 00:46:31 -04:00
Robert Muir 2f67cacaa3 Fix all javadocs issues, re-enable compiler warnings (but disable on java9 where maven is broken) 2015-09-21 23:35:32 -04:00
Ryan Ernst 8aa6aec344 Merge pull request #13663 from rjernst/license_cleanup
Remove unnecessary copies of license and notice files
2015-09-21 18:04:57 -07:00
Nik Everett 6876321581 Merge 'RestUtils.decodeQueryString ignores the URI fragment when parsing a query string'
Fixes #13365
2015-09-21 15:59:11 -04:00
Camilo Díaz Repka c2b3b8d983 Teach decodeQueryString about fragments
Fixes #13320
2015-09-21 15:50:29 -04:00
Robert Muir 7bcdae28ca Merge pull request #13695 from rmuir/factor_out_scripts
Get lang-javascript, lang-python, securemock ready for script refactoring
2015-09-21 15:44:08 -04:00
Robert Muir f0733bd829 Get lang-javascript, lang-python, securemock ready for script refactoring.
I want to refactor scripting engines so we can contain dangerous "God-like" permissions
like createClassloader/sun.reflect. These are used for dynamic class generation (scripts, mocks).
This will mean some refactoring to ES core.

But first lets get the plugins in order first. I removed those permissions globally, and
fixed grants for lang-javascript, lang-python, securemock so that everything works.

lang-javascript needs no code changes, because rhino is properly written :)
lang-python needs accesscontroller blocks. securemock was already working as of 1.1

This is just a baby step, to try to do some of this incrementally! It doesn't yet provide
us anything.
2015-09-21 15:13:17 -04:00
Igor Motov e09e43a325 Increment tribe node version on updates
Currently the tribe node version always stays 0, which can cause issues for the services that rely on cluster state version. For example, ClusterStateObserver doesn't revalidate the cluster state after change, which leads to cluster health check with wait flags to take much longer then actually needed.
2015-09-21 14:45:59 -04:00
David Pilato f230eabc15 [cloud-azure] Split azure plugin in 3 plugins
Until now we had a cloud-azure plugin which is providing 3 distinct features:

* discovery on Azure
* snapshot/restore on Aure
* SMB store

This commit splits the plugin by feature so people can use either one or the other or both features.

Doc is updated accordingly.
2015-09-21 17:55:23 +02:00
Christoph Büscher cc69de5c5f Merge branch 'master' into feature/query-refactoring 2015-09-21 15:38:43 +02:00
Simon Willnauer 3f94b5a395 Merge pull request #13465 from s1monw/issues/13404
Don't be lenient if JarHell is found
2015-09-21 09:53:30 +02:00
Isabel Drost-Fromm 91b97a6f7b Refactors GeoBoundingBoxQueryBuilder/-Parser
This add equals, hashcode, read/write methods, validation, separates toQuery
and JSON parsing and adds serialization and query generation tests.

Deprecates two types of initializing the bounding box: In our documentation we
speak about specifying top/left and bottom/right corner of a bounding box. Here
we also allow for top/right and bottom/left. This adds not only to the amount
of code but also testing needed w/o too much benefit for the user other than
more chances to confuse top/right/bottom/left/latitude/longitude IMHO.

Missing: The toQuery method with type set to "indexed" is not tested at the
moment.

Cleanup changes unrelated to base refactoring:
* Switched from type String to enum for types in GeoBoundingBoxQueryBuilder.
* Switched to using type GeoPoint for storing the bounding box coordinates
  instead of array of double values.

Relates to #10217 for the query refactoring part.
Relates to #12016 for how missing mappings are handled.

Adds a utility class for generating random geo data.

Adds some missing documentation.

Extend test to MEMORY type config

Fix final review comments and rebase
2015-09-20 19:49:52 +02:00
Ryan Ernst 18c519145d Remove unnecessary copies of license and notice files
We moved a lot of repositories into elasticsearch, but in their new
location they retained their LICENSE.txt and NOTICE.txt files. These are
all the same, and having the license and notice and the root of the
repository should be sufficient.
2015-09-18 17:48:30 -07:00
Simon Willnauer 37cff7b01c Add tests and fixes for primitive numeric ctors on TermsQueryBuilder 2015-09-18 22:56:50 +02:00
Simon Willnauer 2305ccef43 Remove forbidden API usage 2015-09-18 22:02:38 +02:00
Colin Goodheart-Smithe a2334981fa Fix messed up merge 2015-09-18 20:47:05 +02:00
Simon Willnauer 2da1baf546 Merge branch 'master' into feature/query-refactoring 2015-09-18 20:38:15 +02:00
Simon Willnauer 1228a9fe55 Merge branch 'master' into feature/query-refactoring 2015-09-18 20:35:56 +02:00
Jason Tedor dea7989a0f Move method to CollectionUtils to reduce number of source files 2015-09-18 08:43:17 -04:00
Jason Tedor 702cf7be72 Replace Integers.checkedCast with built-in method 2015-09-18 08:43:17 -04:00
Jason Tedor f0e20cf594 Remove need for lenient tryParse method 2015-09-18 08:43:17 -04:00
Jason Tedor c281826702 Remove and forbid use of com.google.common.primitives.Ints
This commit removes and now forbids all uses of
com.google.common.primitives.Ints across the codebase. This is one of
many steps in the eventual removal of Guava as a dependency.

Relates #13224
2015-09-18 08:43:17 -04:00
Colin Goodheart-Smithe c8d1f7aa67 [TESTS] fix failing tests due to ParsingException rename 2015-09-18 13:28:48 +01:00
Colin Goodheart-Smithe a8368b5dcd Merge pull request #13631 from s1monw/refactor_query_parsing_exception
Rename QueryParsingException to a more generic ParsingException
2015-09-18 12:30:49 +01:00
Christoph Büscher c6d254ad7e Add QueryShardException to list in ElasticsearchException 2015-09-18 12:38:56 +02:00