Commit Graph

1705 Commits

Author SHA1 Message Date
Armin Braun bdf632b6f9
SNAPSHOTING+MINOR: Simplify SnapshortShardService (#35769) 2018-11-21 13:50:17 +01:00
Christoph Büscher 6638708b56
Remove deprecated QueryStringQueryBuilder#splitOnWhiteSpace (#35763)
This parameter has been deprecated and was ignored since 6.0, so its Java API
methods can be removed.
2018-11-21 10:29:08 +01:00
Christoph Büscher 5847f8379c
Move ScoreAccessor to test-framework (#35766)
This class is only used by RandomScoreFunctionIT and the MockScriptEngine, so it
shouldn't be part of the server codebase.
2018-11-21 10:28:31 +01:00
Christoph Büscher ff03443ab9
Fix problem with MatchNoDocsQuery in disjunction queries (#35726)
Queries across multiple fields generate MatchNoDocsQuerys for fields that are
unmapped. In certain situation this can lead to erroneous behaviour,
for example when an umapped field is used in a query_string query across
several fields. If some of the tokens in the query string get eliminated by an
analyzer on the mapped fields, the same token will currently generate
MatchNoDocsQuerys combined into a disjunction, which in turn
leads to no matches in the overall query. Instead we should simply not add
MatchNoDocsQuerys to those disjunctions.

Closes #34708
2018-11-21 03:49:49 +01:00
Mayya Sharipova 643bb20137
Add a new query type - ScriptScoreQuery (#34533)
* Add a new query type - ScriptScoreQuery

script_score query uses script to calculate document scores.
Added as a substitute for function_score with an intentation
to deprecate function_scoreq query.

```http
GET /_search
{
    "query": {
        "script_score" : {
            "query": {
                "match": { "message": "elasticsearch" }
            },
            "script" : {
              "source": "Math.log(2 + doc['likes'].value)"
            },
            "min_score" : 2
        }
    }
}
```

Add several functions to painless to be used inside script_score:

double rational(double, double)
double sigmoid(double, double, double)
double randomNotReproducible() 
double randomReproducible(String, int) 

double decayGeoLinear(String, String, String, double, GeoPoint)
double decayGeoExp(String, String, String, double, GeoPoint)
double decayGeoGauss(String, String, String, double, GeoPoint)

double decayNumericLinear(String, String, String, double, double)
double decayNumericExp(String, String, String, double, double)
double decayNumericGauss(String, String, String, double, double)

double decayDateLinear(String, String, String, double, JodaCompatibleZonedDateTime)
double decayDateExp(String, String, String, double, JodaCompatibleZonedDateTime)
double decayDateGauss(String, String, String, double, JodaCompatibleZonedDateTime)

Date functions only works on dates in  the default format and default time zone
2018-11-20 16:10:06 -05:00
Ryan Ernst c07ad67718 Add 6.5.2 version 2018-11-20 12:33:26 -08:00
Gordon Brown 4bda469861
Fix line lengths in misc other files (#35650) 2018-11-20 12:29:48 -07:00
Vladimir Dolzhenko 953c8586df
Fix PrimaryAllocationIT#testForceStaleReplicaToBePromotedToPrimary (#35728)
Closes 35497
2018-11-20 20:05:23 +01:00
Christoph Büscher e91f404d16
Remove remains of 'auto_generate_phrase_queries' (#35735)
This parameter in the `query_string` query was deprecated in 6.0 and ignored
since then. Its API methods and remaining uses can be removed in the upcoming
major version.

Relates to #35734
2018-11-20 16:07:17 +01:00
Jim Ferenczi a5f5ceb869
Remove remaining line length violations for o.e.index (#35652)
This commit removes line length violations in the classes under
org.elasticsearch.index.
2018-11-20 08:09:14 +01:00
Simon Willnauer 29ef442841
Add a `_freeze` / `_unfreeze` API (#35592)
This commit adds a rest endpoint for freezing and unfreezing an index.
Among other cleanups mainly fixing an issue accessing package private APIs
from a plugin that got caught by integration tests this change also adds
documentation for frozen indices.
Note: frozen indices are marked as `beta` and available as a basic feature.

Relates to #34352
2018-11-20 08:03:24 +01:00
Benjamin Trent 7a779a9988
[TEST] fixing ClusterApplierService#testClusterStateUpdateLogging() (#35714) 2018-11-19 14:23:49 -06:00
Gordon Brown b2057138a7
Remove AbstractComponent from AbstractLifecycleComponent (#35560)
AbstractLifecycleComponent now no longer extends AbstractComponent. In
order to accomplish this, many, many classes now instantiate their own
logger.
2018-11-19 09:51:32 -07:00
Nik Everett 036c15d8a3
Build: Fix official plugins list (#35661)
The list of official plugins accidentally included `qa` projects like,
well, `qa` and `amazon-ec2`. This changes the mechanism that we use to
build the list and adds a test to catch this.

Closes #35623
2018-11-19 10:17:05 -05:00
Ekal Golas 450db7fcf7 [Tests] Fix slowness of AutoDateHistogramAggregatorTests (#35072)
Randomize test assertion and test set size instead of asserting on an
exhaustive list of dates with fixed test set size. Also refactor common 
objects used to avoid recreating them, avoid date to string conversion
and reduce duplicate test code

Closes #33181
2018-11-19 15:21:01 +01:00
Martijn van Groningen c468d928b8
Remove line length violations in o.e.indices package (#35647)
Relates to #34884
2018-11-19 07:36:24 +01:00
Lee Hinman 22933ebcd5
Fix line length for org.elasticsearch.action.* files (#35607)
This removes the checkstyle suppressions for things in the `action` package.

Relates to #34884
2018-11-16 14:11:39 -07:00
Arthur Gavlyukovskiy 022726011c Remove use of AbstractComponent in server (#35444)
Removed extending of AbstractComponent and changed logger usage to
explicit declaration. Abstract classes still have logger
declaration using this.getClass() in order to show implementation class
name in its logs.

See #34488
2018-11-16 16:10:32 -05:00
Julie Tibshirani c6a0904e0e
Deprecate types in count and msearch. (#35421)
* Deprecate types in count requests.
* Move RestCountAction to the 'search' package.
* Deprecate types in multi search requests.
* Add tests for types deprecation in the _search endpoint.
2018-11-16 13:04:43 -08:00
Jack Conradson d62bbca56d
[Scripting] Use Number as a return value for BucketAggregationScript (#35653)
This change fixes #35351. Users were no longer able to return types of numbers other than doubles for bucket aggregation scripts. This change reverts to the previous behavior of being able to return any type of number and having it converted to a double outside of the script.
2018-11-16 12:33:01 -08:00
Jernej Klancic baf33b3162 Removes AbstractComponent from several classes (#35566)
Removes inhertiting from AbstractComponent for some classes (mostly
in the plugins module).

Relates to #34488
2018-11-16 20:50:18 +01:00
Julie Tibshirani 40ba4de5e6
Deprecate types in validate query requests. (#35575) 2018-11-16 08:59:04 -08:00
Tanguy Leroux d3d7c0158e Revert "[RCI] Check blocks while having index shard permit in TransportReplicationAction (#35332)"
This reverts commit 31567cefb4.
2018-11-16 15:38:51 +01:00
Tom Callahan 76b77dbaa4
Remove remaining line length violations for o.e.action.admin.cluster (#35156)
This inserts newlines in order to reduce line lengths in the
o.e.action.admin.cluster package to 140 characters or less.  This
also remves the checkstyle suppressions for affected files.

Relates #34884, #34923
2018-11-16 09:37:42 -05:00
Christoph Büscher 2da4bc85cf
Remove usages of CharSequence in Sets (#35501)
The javadocs of the CharSequence interface state that not all of its
implementations define the general contracts of the Object#equals and
Object#hashCode methods, therefore it is dangerous to use different CharSequence
instances as elements in a set or as keys in a map. While we probably mostly use
Strings in sets, in some places this is not enforced. To prevent this from
accidentally happening, this change replaces all occurances of Set<CharSequence>
which are currently mostly used in the completion suggester code with the more
concrete usage of Set<String>.
2018-11-16 09:35:22 +01:00
Jim Ferenczi 903d27a8fd
Fix phrase_slop in query_string query (#35533)
This change ensures that custom `phrase_slop`s are correctly handled
in the `query_string` query.

Fixes #35125
2018-11-16 09:08:07 +01:00
Lee Hinman ce35d049e9 [TEST] Fix ClusterApplierServiceTests.testClusterStateUpdateLogging
This changes the test to not use a `CountDownlatch`, instead adding an assertion
for the final logging message and waiting until the `MockAppender` has seen it
before proceeding.

Resolves #23739
2018-11-15 14:15:23 -07:00
Jim Ferenczi c7a2c6d549
Handle IndexOrDocValuesQuery in composite aggregation (#35392)
The `composite` aggregation can optimize its execution when the query
is a `match_all` or a `range` over the field that is used in the first source
of the aggregation. However we only check for instances of `PointRangeQuery` whereas
the range query builder creates an  `IndexOrDocValuesQuery`. This means that
today the optimization does not apply to `range` query even if the code could handle it.
This change fixes this issue by extracting the index query inside `IndexOrDocValuesQuery`.
2018-11-15 17:52:06 +01:00
David Turner 928ab6a5b1
Suppress CachedTimeThread in hot threads output (#35558)
In #23175 we renamed `ThreadPool$EstimatedTimeThread` to
`ThreadPool$CachedTimeThread` but did not update the corresponding entry in
`HotThreads#isIdleThread`. This commit addresses this.
2018-11-15 08:25:09 +00:00
Tanguy Leroux c9b4ef0dfd
Use RunOnce when appropriate (#35553)
This pull request replaces some blocks of code that must be run once 
and that are currently based on AtomicBoolean by the convient RunOnce 
class added in #35489.
2018-11-15 09:24:40 +01:00
Tanguy Leroux 5cd9b518ac
Fix usages of XContentParserUtils.ensureExpectedToken() (#35530)
Some usages of XContentParserUtils.ensureExpectedToken() are inverting
the expected and actual tokens, resulting in wrong error messages.
2018-11-15 09:18:32 +01:00
Hendrik Muhs fc774a3776
add ES 6.5.1 (on master) (#35549)
add ES 6.5.1
2018-11-14 21:08:21 +01:00
Tanguy Leroux c8c8ce2374
Extract RunOnce into a dedicated class (#35489)
This commit extracts the static inner class RunOnce from 
WorkerBulkByScrollTaskState so that it can be reused at 
other places.
2018-11-14 17:33:04 +01:00
Tanguy Leroux bbe50e7a86
Remove LoggingRunnable class (#35486)
This commit removes the unused LoggingRunnable class.
2018-11-14 10:12:25 +01:00
Tanguy Leroux 31567cefb4
[RCI] Check blocks while having index shard permit in TransportReplicationAction (#35332)
Today, the TransportReplicationAction checks the global level blocks and 
the index level blocks before routing the operation to the primary, in the 
ReroutePhase, and it happens at the very beginning of the transport 
replication action execution. For the upcoming rework of the Close Index 
API and in order to deal with primary relocation, we'll need to also check 
for blocks before executing the operation on the primary (while holding a 
permit) but before routing to the new primary.

This pull request change the AsyncPrimaryAction so that it checks for 
replication action's blocks before executing the operation locally or before 
routing the primary action to the newly primary shard. The check is done 
while holding a PrimaryShardReference.

Related to #33888
2018-11-14 09:43:55 +01:00
Hendrik Muhs 5c84708ee5 test: expose error message on failure 2018-11-14 08:25:41 +01:00
Christoph Büscher d8b1c23e1d
Remove Comparable interface from ScoreAccessor (#35519)
The way ScoreAccessor implements `compareTo()` is problematic because it doesn't
completely follow the Comparable contract, specificaly symmetry (if x is a
ScoreAccessor and y any Number then x.comparTo(y) works, but y.compareTo(x)
generally does not even compile). Fortunately we don't seem to use the fact that
ScoreAccessor is a Comparable anywhere, so we can simply remove it.
2018-11-14 05:58:05 +01:00
Vladimir Dolzhenko 9728119b82 [CI] AllocationIdIT testFailedRecoveryOnAllocateStalePrimaryRequiresAnotherAllocateStalePrimary failure
Closes #35504
2018-11-13 20:57:50 +01:00
Simon Willnauer 3229dfc4de
Allow efficient can_match phases on frozen indices (#35431)
This change adds a special caching reader that caches all relevant
values for a range query to rewrite correctly in a can_match phase
without actually opening the underlying directory reader. This
allows frozen indices to be filtered with can_match and in-turn
searched with wildcards in a efficient way since it allows us to
exclude shards that won't match based on their date-ranges without
opening their directory readers.

Relates to #34352
Depends on #34357
2018-11-13 14:53:55 +01:00
Christoph Büscher 0a6614a03a
Correct implemented interface of ParsedReverseNested (#35455)
The ParsedReverseNested implementation should implement the ReverseNested
interface and not the Nested interface. Although this is an empty marker
interface it is confusing and can lead to casting errors. Also adding a test to
check that both ParsedNested and ParsedReverseNested implement the correct
interface.

Closes #35449
2018-11-13 10:34:29 +01:00
Jason Tedor a18b599d64
Handle OS pretty name on old OS without OS release (#35453)
Some very old ancient versions of Linux do not have /etc/os-release. For
example, old Red Hat-like OS. This commit adds a fallback for handling
pretty name for these OS.
2018-11-12 19:31:12 -05:00
Tim Brooks 71cfb730f6
Register remote cluster compress setting (#35464)
This is a follow up to #35357. That commit failed to register the new
cluster.remote.cluster_name.transport.compress setting with
`ClusterSettings`. This commit fixes that.
2018-11-12 16:07:42 -07:00
Igor Motov e7896bcefc
Geo: enables coerce support in WKT polygon parser (#35414)
WKT parser now automatically closes open polygons similar to GeoJSON
parser if coerce flag in mapping is set to true.

Closes to #35059
2018-11-12 09:40:04 -10:00
Jason Tedor 40ca62c298
Address handling of OS pretty name on some OS (#35451)
Some OS (e.g., Oracle Linux Server 6.9) have a trailing space at the end
of the PRETTY_NAME line in /etc/os-release. This commit addresses this
by accounting for this trailing space when extracting the pretty name.
2018-11-12 14:27:57 -05:00
Nick Knize 2591f66a33
upgrade to lucene-8.0.0-snapshot-6d9c714052 (#35428) 2018-11-12 10:48:27 -06:00
Christoph Büscher 09cac321e7
Upgrade to Joda 2.10.1 (#35410)
This version contains a bugfix that allows us to reenable one of our muted tests
in DateTimeUnitTests.

Closes #33749
2018-11-12 10:02:41 +01:00
Tim Brooks ba478827ad
Improve MockTcpTransport memory usage (#35402)
The MockTcpTransport is not friendly in regards to memory usage. It must
allocate multiple byte arrays for every message. This improves the
memory situation by failing fast if the message is improperly formatted.
Additionally, it uses reusable big arrays for at least half of the
allocated byte arrays.
2018-11-09 10:12:49 -07:00
Jim Ferenczi 7054e289fa
Add trace log of the request for the query and fetch phases (#34479)
This change adds a logger for the query and fetch phases that prints all requests
before their execution at the trace level. This will help debugging cases where an issue
occurs during the execution since only completed queries are logged by the slow logs.
2018-11-09 09:41:51 +01:00
Tim Brooks bccc99c2be
Fix TcpTransport compression test (#35396)
This commit fixes an assertion in the TcpTransportTests compresssion
test.
2018-11-08 18:04:48 -07:00
Tim Brooks 93c2c604e5
Move compression config to ConnectionProfile (#35357)
This is related to #34483. It introduces a namespaced setting for
compression that allows users to configure compression on a per remote
cluster basis. The transport.tcp.compress remains as a fallback
setting. If transport.tcp.compress is set to true, then all requests
and responses are compressed. If it is set to false, only requests to
clusters based on the cluster.remote.cluster_name.transport.compress
setting are compressed. However, after this change regardless of any
local settings, responses will be compressed if the request that is
received was compressed.
2018-11-08 10:37:59 -07:00