Commit Graph

1272 Commits

Author SHA1 Message Date
Ryan Ernst f3142e0c93 Merge pull request #13497 from rjernst/test_jar_cleanup
Reorganize sharing of constants for mock cluster info service
2015-09-10 21:10:29 -07:00
Jason Tedor b5b22d4b01 Remove and forbid use of com.google.common.collect.Queues
This commit removes and now forbids all uses of
com.google.common.collect.Queues across the codebase. This is one of
many steps in the eventual removal of Guava as a dependency.

Relates #13224
2015-09-10 23:59:59 -04:00
Jason Tedor 65353b8a32 Remove and forbid use of com.google.common.collect.ImmutableSortedSet
This commit removes and now forbids all uses of
com.google.common.collect.ImmutableSortedSet across the codebase. This
is one of many steps in the eventual removal of Guava as a dependency.

Relates #13224
2015-09-10 23:33:55 -04:00
Robert Muir af2df9aef6 Update monitor probe tests for java 9: this stuff is no longer accessible 2015-09-10 23:18:58 -04:00
Ryan Ernst 4a9f5bcf1d Tests: Reorganize sharing of constants for mock cluster info service
The MockInternalClusterInfoService depends on a constant and helper
method from actual test cases. This moves the constant and helper method
into the mock itself. Without this change, the classes put into the test
jar are not completely useable on their own (since this mock is in the
test jar, but the test cases are not).
2015-09-10 20:08:04 -07:00
Robert Muir 6ba4d132df Disable some exception serialization asserts on java 9, likely StackTraceElement has new stuff in it and we have to think about what to do 2015-09-10 22:45:53 -04:00
Robert Muir 552d5b0f65 Add java 9 support for bootclasspath to JvmInfo (it throws UOE otherwise which is not properly handled) 2015-09-10 22:15:27 -04:00
Robert Muir c663d9b6a5 Merge branch 'jarhell2.0' into 1702090 2015-09-10 22:07:37 -04:00
Robert Muir 7aeff9f526 fix merge 2015-09-10 22:07:20 -04:00
Robert Muir dd208002c9 Merge branch 'master' into 1702090 2015-09-10 21:41:07 -04:00
Robert Muir 6f59e3fca1 cutover jarhell to parsing classpath all the time 2015-09-10 21:31:25 -04:00
Jason Tedor 5d4d34ab16 Remove and forbid use of c.g.c.b.Preconditions#checkNotNull
This commit removes and now forbids all uses of
com.google.common.base.Preconditions#checkNotNull across the codebase.
This is one of many steps in the eventual removal of Guava as a
dependency.

Relates #13224
2015-09-10 17:57:00 -04:00
Robert Muir c1f2fc76c2 Upgrade lucene to r1702090
The semantics of the `boost` parameter for `function_score` changed. This is
due to the fact that Lucene now requires that query boosts and top-level boosts
are applied the same way.
2015-09-10 23:36:43 +02:00
Nik Everett e4981968ad [search] Limit the size of the result window
Requesting a million hits, or page 100,000 is always a bad idea, but users
may not be aware of this. This adds a per-index limit on the maximum size +
from that can be requested which defaults to 10,000.

This should not interfere with deep-scrolling.

Closes #9311
2015-09-10 15:38:29 -04:00
Robert Muir 6cdcc805cc hack jython tests to work on windows
windows needs access to the "root" holding the jar file (see https://github.com/int3/jython/blob/master/src/org/python/core/PySystemState.java#L571-L616)

Its different on windows, because when canonicalizing the file (case sensitivity), it needs access to the file.

Closes #13476
2015-09-10 14:59:23 -04:00
Ryan Ernst ca8867a334 Test: Fix tests looking for old indexes to resolve against the correct dir 2015-09-10 11:07:02 -07:00
Lee Hinman b21d3d2fb5 [TEST] Additional logging for testDelayedUnassignedScheduleReroute
Relates to #13485
2015-09-10 11:44:09 -06:00
Ryan Ernst 47de1bd923 Merge pull request #13443 from rjernst/simplify_bwc_path
Move static bwc indexes to a shared location
2015-09-10 10:31:50 -07:00
Jason Tedor 3bd1d38176 Remove and forbid use of com.google.common.collect.Sets
This commit removes and now forbids all uses of
com.google.common.collect.Sets across the codebase. This is one of many
steps in the eventual removal of Guava as a dependency.

Relates #13224
2015-09-10 11:22:09 -04:00
Martijn van Groningen ab0847e0df parent/child: several cleanups
* Dropped ScoreType in favour of Lucene's ScoreMode
* Removed `score_type` option from `has_child` and `has_parent` queries in favour for the already existing `score_mode` option.
* Removed the score mode `sum` in favour for the already existing `total` score mode. (`sum` doesn't exist in Lucene's ScoreMode class)
* If `max_children` is set to `0` it now really means that zero children are allowed to match.
2015-09-10 17:15:41 +02:00
Nik Everett 60a2dd7020 Merge pull request #13189 from nik9000/fix_nodeattrs_test
Fix test for _cat/nodeattrs
2015-09-10 11:12:01 -04:00
Nik Everett 56c3471851 Fix test for _cat/nodeattrs
Adds a node attribute to all test runs and uses the attribute to test
`_cat/nodeattrs`.

Note that its quite possible create an impressively slow regex while doing
this and you have to be careful. See comment in commit for more if curious.

Closes #12558
2015-09-10 10:50:51 -04:00
Colin Goodheart-Smithe 53d29e51e0 Aggregations: Pipeline Aggregations at the root of the agg tree are now validated
Previously PipelineAggregatorFactory's at the root to the agg tree (top-level aggs) were not validated. This commit adds a call to PipelineAggregatoFactory.validate() for that case.

Closes #13179
2015-09-10 15:44:52 +01:00
Nik Everett 3839d15ea0 Merge branch 'pr/13130' 2015-09-10 10:13:01 -04:00
Martijn van Groningen a0fea6d365 set query cache to null 2015-09-10 15:19:12 +02:00
André Carvalho 9802c38fa7 [Stats] Adds counter for failed indexing requests
Adds the stats to the _cat/indices, _cat/shards, and the _stats apis.

Closes #8938
2015-09-10 09:03:24 -04:00
Britta Weber c10f116a84 [test] don't catch AssertionError from indexRandom(). just index and catch MapperParsingException 2015-09-10 14:51:23 +02:00
Martijn van Groningen 314e1c8a3e parent/child: has_child has_parent queries shouldn't require search context 2015-09-10 12:47:06 +02:00
Martijn van Groningen 2eadc6d595 nested sorting: If sorting by nested field then the `nested_path` should always be specified.
Closes #13420
2015-09-10 12:21:12 +02:00
Martijn van Groningen 89159f073c Merge pull request #13430 from martijnvg/pc/parent_field_always_store_doc_values
Parent field mapper should always store doc values join field.
2015-09-10 12:15:39 +02:00
Simon Willnauer 7fff399834 Reenable XTestSecurityManager 2015-09-10 10:49:13 +02:00
Adrien Grand e89ec468c3 Remove support for deprecated queries.
This removes support for the `and`, `or`, `fquery`, `limit` and `filtered`
queries. `query` is still supported until #13326 is fixed.
2015-09-10 10:38:11 +02:00
Simon Willnauer 7ab0e2c532 Swap out XTestSecurityManager until we figured out why it crashes the JVM 2015-09-10 09:57:02 +02:00
Ryan Ernst c1dd9b8a98 add temp test security manager to test jar 2015-09-09 16:08:50 -07:00
Ryan Ernst 5c810eff59 Add helper method so indices path is not copied all over 2015-09-09 15:51:34 -07:00
Ryan Ernst 8f75c2b3a8 Tests: Move static bwc indexes to a shared location
There are a few tests that currently use the statically generated
backcompat indexes. This change moves them to a shared location, so they
no longer have to build a path based on the package name of the old
index tests.
2015-09-09 15:42:26 -07:00
Robert Muir e67140b954 Merge pull request #13442 from rmuir/ide_noise
Remove noise when IDE test runners try to System.exit
2015-09-09 18:22:05 -04:00
Ryan Ernst 7854ccf797 Tweak intellij junit package name 2015-09-09 15:17:48 -07:00
Robert Muir e7b98ac513 lie to the stupid license checker 2015-09-09 18:03:32 -04:00
Robert Muir c273895d82 Remove noise when IDE test runners try to System.exit 2015-09-09 18:00:46 -04:00
Jason Tedor 6b19aebab0 Merge pull request #13438 from jasontedor/maps-be-gone
Remove and forbid use of com.google.common.collect.Maps
2015-09-09 17:58:44 -04:00
Jason Tedor 2a5412ebf0 Remove and forbid use of com.google.common.collect.Maps
This commit removes and now forbids all uses of
com.google.common.collect.Maps across the codebase. This is one of many
steps in the eventual removal of Guava as a dependency.

Relates #13224
2015-09-09 17:41:41 -04:00
Jason Tedor b18ee60f80 Merge pull request #13440 from jasontedor/remove-thread-dumps-on-test-failures
Do not dump stack traces of threads on test failure
2015-09-09 17:23:41 -04:00
Jason Tedor 3dcb1a81c8 Do not dump stack traces of threads on test failure
Reverts 878301c795, relates #12425
2015-09-09 17:07:13 -04:00
Ryan Ernst 9d71f0726e Merge pull request #13410 from rjernst/you_were_warned
Fix compiler warnings
2015-09-09 14:04:54 -07:00
Ryan Ernst 9e8a90a657 Add xlint ignores for warning classes, where appropriate. 2015-09-09 12:47:07 -07:00
gmarz 9e6115b066 Packaging: Fix Windows service start/stop issues
This commit addresses several bugs that prevented the Windows
service from being started or stopped:

- Extra white space in the concatenation of java options in
  elasticsearch.in.bat which tripped up Apache Commons Daemon
  and caused ES to startup without any params, eventually leading
  to the "path.home is not configured" exception.

- service.bat was not passing the start argument to ES

- The service could not be stopped gracefully via the stop command
  because there wasn't a method for procrun to call.

Closes #13247
Closes #13401
2015-09-09 13:31:46 -04:00
Martijn van Groningen c54135c1e1 parent/child: Always store doc values join field.
Now that the pre 2.0 parent child implementation has been removed there is no need to have logic that decides not to store the join doc values field.
2015-09-09 18:46:40 +02:00
Martijn van Groningen 73d84e4797 Merge pull request #13376 from martijnvg/remove_pre2x_slow_parent_child
Removed pre 2.x parent child implementation
2015-09-09 15:20:10 +02:00
David Pilato 5b9d183bf4 Merge remote-tracking branch 'origin/master' 2015-09-09 15:17:48 +02:00
David Pilato 35049a05c3 Allocation: add support for filtering by transport IP address
Allocation filtering by IP only works today using the node host address. But in some cases, you might want to filter using the publish address which could be different.
2015-09-09 15:15:53 +02:00
Robert Muir f5270849fa Merge pull request #13407 from rmuir/ide_security_manager
Enable security manager by default in tests (e.g. IDEs)
2015-09-09 09:12:35 -04:00
Boaz Leskes 30ca6d3970 Internal: extract gateway required allocation calculations
and add a basic test

Closes #13391
2015-09-09 13:23:14 +02:00
Martijn van Groningen 2fb2a12c52 Removed pre 2.x parent child implementation 2015-09-09 13:21:40 +02:00
Boaz Leskes 37d4727c0a Gateway: allow overriding the Gateway implementation
Allows mocking the gateway and overriding it from plugins.

Closes #13412
2015-09-09 13:19:25 +02:00
Martijn van Groningen 65e7aba780 inner hits: Protected against specifying a size larger than the total amount of documents in an index.
Closes #13394
2015-09-09 13:13:41 +02:00
Britta Weber 281eac757d [test] make sure result is never null and that flush and get start at same time 2015-09-09 12:41:32 +02:00
Britta Weber fa9696fb8c Merge pull request #13414 from brwe/commit-refresh-order
Engine: refresh before translog commit
2015-09-09 12:06:26 +02:00
Alex Ksikes a45ee273e3 MLT: builder takes a new Item object like its parser
Previously the parser could take any Term Vectors request, but this would be
not the case of the builder which would still use MultiGetRequest.Item. This
introduces a new Item class which is used by both the builder and parser.

Beyond that the rest is mostly cleanups such as:

1) Deprecating the ignoreLike methods, in favor to using unlike.

2) Deprecating and renaming MoreLikeThisBuilder#addItem to addLikeItem.

3) Ordering the methods of MoreLikeThisBuilder more logically.

This change is needed for the upcoming query refactoring of MLT.

Closes #13372
2015-09-09 11:40:27 +02:00
Martijn van Groningen e23d116bc5 parent/child: Split the _parent field mapping's field type into three field types:
1) A shared immutable fieldtype for the _parent field (used for direct access to that field in the dsl). This field type is stored and indexed.
2) A per type field type for the child join field. The field type has doc values enabled if index is created on or post 2.0 and field data type is allowed to be changed.
3) A per type field type for the parent join field. The field type has doc values enabled if index is created on or post 2.0.

This resolves the issue that a mapping is not compatible if parent and child types have different field data loading settings.

Closes #13169
2015-09-09 11:10:43 +02:00
Britta Weber 0ce66b4d70 Engine: refresh before translog commit
When we commit the translog, documents that were in it before cannot be retrieved from
it anymore via get and have to be retrieved from the index instead. But they will only
be visible if between index and get a refresh is called. Therfore we have to call
first refresh and then translog.commit() because otherwise there is a small gap
in which we cannot read from the translog anymore but also not from the index.

closes #13379
2015-09-09 10:37:13 +02:00
Ryan Ernst b04160d9f5 Add Xlint options and fix some warnings 2015-09-08 23:49:20 -07:00
Ryan Ernst 2d15cab68c Build: Fix compiler warnings
We have a handful of compiler warnings, mostly because of passing an
array to varargs methods. This change fixes these warnings and adds
-Werror so we don't get anymore of these warnings.

Note this does *not* enable deprecation or unchecked type warnings, so
these remain "hidden". We should work towards removing those as well,
but this is a first step.
2015-09-08 22:15:34 -07:00
Jason Tedor 1806c1e0c7 Remove and forbid use of com.google.common.base.Throwables
This commit removes and now forbids all uses of
com.google.common.base.Throwables across the codebase.

For uses of com.google.common.base.Throwables#getStackTraceAsString,
use org.elasticsearch.ExceptionsHelper#stackTrace.

Relates #13224
2015-09-08 21:56:23 -04:00
Ryan Ernst ea4c27a089 Add back path.logs setup in internal settings preparer 2015-09-08 15:29:06 -07:00
Nicholas Knize e4e71d8a9a add points_only option to GeoShapeFieldMapper for optimizing indexing performance on geo_shape indexes designed to store only points. Includes updated documentation and exception handling for ensuring index integrity on points only data. 2015-09-08 16:17:50 -05:00
Ryan Ernst 55795f8ec2 Merge pull request #13383 from rjernst/go_away_transport_paths
Remove environment from transport client
2015-09-08 13:28:21 -07:00
Ryan Ernst 4914b0c465 Merge pull request #13385 from rjernst/wrong_resources
Move resources to their proper location
2015-09-08 13:25:31 -07:00
Ryan Ernst f3fb702e88 Move logging specific settings setup to log configurator. 2015-09-08 13:13:10 -07:00
Ryan Ernst e218b31322 Address PR comments 2015-09-08 12:57:28 -07:00
Robert Muir 29698f6945 Enable security manager by default in tests (e.g. IDEs)
Otherwise people will be confused when they use maven.
2015-09-08 15:55:33 -04:00
Robert Muir 164cf2775f Give intellij its jar hell back. This is not an invitation for more leniency. 2015-09-08 15:32:07 -04:00
Ryan Ernst 2e63290548 Finish removing loadConfigSettings, and fix node startup to use correct
settings object
2015-09-08 10:16:25 -07:00
Ryan Ernst 92b62c0c6b Merge branch 'master' into go_away_transport_paths 2015-09-08 08:55:17 -07:00
Ryan Ernst d19e04fbb6 Change prepareSettingsAndEnvironment to prepareEnvironment 2015-09-08 08:54:18 -07:00
Robert Muir c354165bc9 remove intellij leniency. fix your IDE instead. it gives people the wrong idea. 2015-09-08 11:32:12 -04:00
Britta Weber 3de8cbb54c Merge pull request #13380 from brwe/exception-on-force-merge
Engine: Let AlreadyClosedException and EngineClosedExceptionBubble up
2015-09-08 17:01:10 +02:00
Britta Weber 2288d44eeb Engine: Let AlreadyClosedException and EngineClosedException bubble up
Whe we call optimize we ignore Exceptions that indicate a closed shard.
However, when a shard is closed while an optimize request is in flight it
might also trigger an AlreadyClosedException from the IndexWriter when we
get the config or ForceMergeFailedEngineException with the EngineClosedException
wrapped inside. Because these are not identified as exceptions that indicate
a closed shard (TransportActions.isShardNotAvailableException(..)) optimize
would sometimes report failures when shards were relocating while optimize was called
and sometimes not. This caused weird test failures, see #13266 .
Instead, we should let EngineClosedException bubble up and also recognize
AlreadyClosedException as an indicator for a closed shard.
2015-09-08 16:05:40 +02:00
Simon Willnauer f208aaa47c Also use PriorityComparator in shard balancer
Today we try to allocate primaries first and then replicas
but don't take the index creation date and priority into account
as we do in the GatewayAlloactor.

Closes #13249
2015-09-08 15:22:22 +02:00
Jason Tedor cf539a2158 Add trace logging before sending single shard requests 2015-09-08 08:41:39 -04:00
Simon Willnauer ed5f6e5a0c Remove cyclic dependencies between IndexService and FieldData / BitSet caches
Adds a listeners to each of the caches that allows us to remove the dependency on IndexService which is cyclic since
the IndexService depends on both of these caches. This cyclic dependency makes
testing the indiviual parts very hard and is only added for the sake of
incrementing some stats.
2015-09-08 13:06:32 +02:00
Boaz Leskes a2f94f2e2f move ZenDiscovery.rejoin to protected
So we can override it from tests and plugins.
2015-09-08 11:02:29 +02:00
Ryan Ernst 6b88093f78 Add back filtering for build properties, and remove unused resource
file.
2015-09-08 01:15:46 -07:00
Ryan Ernst 1b99b1d8b4 Remove unnecessary maven resources config 2015-09-07 22:50:01 -07:00
Ryan Ernst bcc4175b53 Build: Move resources to their proper location
We have a a number of resources in the java source tree, mostly for
tests. This change moves them to their proper location.
2015-09-07 19:45:08 -07:00
Ryan Ernst 1ff49eb8de Settings: Remove environment from transport client
Transport clients run embedded within external applications, so
elasticsearch should not be doing anything with the filesystem, as there
is not elasticsearch home.

This change makes a number of cleanups to the internal API for loading
settings and creating an environment. The loadFromConfig option was
removed, since it was always true except for tests. We now always
attempt to load settings from config a file when an environment is
created. The prepare methods were also simplified so there is now
prepareSettingsAndEnvironment which nodes use, and prepareSettings which
the transport client uses. I also attempted to improve the tests, but
there is a still a lot of follow up work to do there.

closes #13155
2015-09-07 17:26:41 -07:00
Adrien Grand 4173b00241 Merge pull request #13310 from jpountz/remove/count_scan
Remove the scan and count search types.
2015-09-07 16:21:14 +02:00
Jason Tedor a6ffe8f6d5 Remove and forbid use of com.google.common.base.Strings
This commit removes and now forbids all uses of
com.google.common.base.Strings across the codebase.

For uses of com.google.common.base.Strings.isNullOrEmpty, use
org.elasticsearch.common.Strings.isNullOrEmpty.

For uses of com.google.common.base.Strings.padStart use
org.elasticsearch.common.Strings.padStart.

For uses of com.google.common.base.Strings.nullToEmpty use
org.elasticsearch.common.Strings.coalesceToEmpty.

Relates #13224
2015-09-07 09:40:14 -04:00
Adrien Grand 0c26e7cd83 Remove the scan and count search types.
These search types have been deprecated in 2.1 and 2.0 respectively, and will
be removed in 3.0.
2015-09-07 15:18:45 +02:00
Britta Weber c5cb559014 Merge pull request #13341 from brwe/shard-not-available-broadcast-replication
fix exception handling for unavailable shards in broadcast replicatio…
2015-09-07 14:47:55 +02:00
Britta Weber 41a2375c9e fix exception handling for unavailable shards in broadcast replication action
Before #13068 refresh and flush ignored all exceptions that matched
TransportActions.isShardNotAvailableException(e) and this should not change.
In addition, refresh and flush which are based on broadcast replication
might now get UnavailableShardsException from TransportReplicationAction if a shard
is unavailable and this is not caught by TransportActions.isShardNotAvailableException(e).
This must be ignored as well.
2015-09-07 14:47:28 +02:00
Britta Weber 972c1cc398 [test] remove timeout test 0s for flush and refresh - we decided we use 1m timeout
see #13238
2015-09-07 14:39:59 +02:00
Jason Tedor 8a3411e5e4 Remove and forbid the use of com.google.common.base.Predicate(s)?
This commit removes and now forbids all uses of
com.google.common.base.Predicate and com.google.common.base.Predicates
across the codebase. This is one of the many steps in the eventual
removal of Guava as a dependency. This was enabled by #13314.

Relates #13224
2015-09-06 07:20:24 -04:00
Jason Tedor 14e4882425 Remove and forbid use of com.google.common.base.Objects
This commit removes and now forbids all uses of
com.google.common.base.Objects across the codebase. This is a small
step in the eventual removal of Guava as a dependency.

Relates #13224
2015-09-05 08:52:30 -04:00
Lee Hinman 701dc340a8 Allow deleting closed indices with shadow replicas
Previously we skip deleting the index store for indices on a shared
filesystem, because we don't want to delete the data when the shard is
relocating around the cluster. This adds a flag to the
`deleteIndexStore` method signifying that the index is closed and that
we should allow deleting the contents even if it is on a shared
filesystem.

Includes a unit test for the IndicesService.canDeleteIndexContents and
integration tests ensure a closed shadow replica index deletes files
correctly.

Resolves #13297
2015-09-04 15:28:18 -06:00
Martijn van Groningen ac951e86d1 fix npe 2015-09-04 22:03:25 +02:00
Zachary Tong 5aa9639034 Add missing License header 2015-09-04 15:50:02 -04:00
Zachary Tong 397d5beae1 Aggregations: Add stats_bucket / extended_stats_bucket pipeline aggregations
These are the complements to the stats/extended_stats metric aggregations, and can be used
to calculate a variety of statistics over buckets
2015-09-04 15:23:48 -04:00
Jason Tedor f4213614c3 Remove use of underscore as an identifier
As a refinement to Project Coin (JEP-213, JDK-8042880), Java 9 is going
to disallow the use of ‘_’ as a one-character identifier. This will be
done by adding ‘_’ as a keyword to the Java language (JDK-8065599).
Currently, uses of ‘_’ as a one-character identifier are warnings in
the Java 8 compiler. This commit removes all uses of ‘_’ as a
one-character identifier from the codebase.
2015-09-04 14:18:30 -04:00
Jason Tedor b98cd5f611 Should be asserting JVM plugins do not have a URL 2015-09-04 12:13:54 -04:00