Commit Graph

42990 Commits

Author SHA1 Message Date
Costin Leau f0a3d32602
SQL: Introduce INTERVAL support (#35521)
Introduce INTERVAL as a DataType
Add INTERVAL to the grammar which supports the standard SQL declaration
 (without precision):
 > INTERVAL '1 23:45:01.123456789' DAY TO SECOND
 but also number for single unit intervals:
 > INTERVAL 1 YEAR
 as well as the plurals of the units:
 > INTERVAL 2 YEARS
Interval are internally supported as just another Literal being backed
 by java.time.Period and java.time.Duration
Move JDBC away from JDBCType enum to SQLType interface
Refactor DataType by moving it into server core and adding dedicated (and
 much simpler) JDBC driver type
Improve internal JDBC conversion by normalizing on the DataType
Rename JDBC columnInfo to JdbcColumnInfo to differentiate between it and
 the SQL ColumnInfo

Fix #29990
2018-11-21 15:45:59 +02:00
Yannick Welsch 8939a7894f
Zen2: Move disruption tests to Zen2 (#35724)
- Moves disruption tests to Zen2
- Registers a few missing settings
- Removes .put(TestZenDiscovery.USE_ZEN2.getKey(), true) from tests where Zen2 is now enabled
by default through the parent test class
- Moves QuorumGatewayIT back to Zen1, as it is not stable with Zen2 as it currently relies on
dangling indices due to the lack of proper CS persistence, which triggers secondary failures
2018-11-21 14:43:33 +01:00
Armin Braun bdf632b6f9
SNAPSHOTING+MINOR: Simplify SnapshortShardService (#35769) 2018-11-21 13:50:17 +01:00
Martijn van Groningen 9b2ab064cf
[HLRC] Added support for CCR Resume Follow API (#35638)
This change also adds documentation for the Resume Follow API

Relates to #33824
2018-11-21 10:52:03 +01:00
Armin Braun 7a210342ab
TESTS: Remove Dead Code in Disruption Tests (#35768)
* Neither this class nor the constructor are used anywhere
2018-11-21 10:33:50 +01:00
Dimitrios Liappis 1cb578b435
[DOCS] Update sysctl instructions for Docker on Mac (#35755)
Recent Docker for Mac releases[1] have a different path to the tty for
accessing the console of the xhyve vm, required for altering the
`vm.max_map_count` sysctl.

Update instructions on how to enter the xhyve vm for altering the
`vm.max_map_count` sysctl setting on Docker for Mac.

Closes #34817 

[1]
https://forums.docker.com/t/is-it-possible-to-ssh-to-the-xhyve-machine/17426/13
2018-11-21 11:32:29 +02:00
Christoph Büscher 6638708b56
Remove deprecated QueryStringQueryBuilder#splitOnWhiteSpace (#35763)
This parameter has been deprecated and was ignored since 6.0, so its Java API
methods can be removed.
2018-11-21 10:29:08 +01:00
Christoph Büscher 5847f8379c
Move ScoreAccessor to test-framework (#35766)
This class is only used by RandomScoreFunctionIT and the MockScriptEngine, so it
shouldn't be part of the server codebase.
2018-11-21 10:28:31 +01:00
Luca Cavanna 65d8fdf3da
Clean up PutLicenseResponse (#35689)
This commit removes the parsing code from the PutLicenseResponse server variant, and the toXContent portion from the corresponding client variant.

Relates to #35547
2018-11-21 10:21:45 +01:00
Luca Cavanna 778550a97d
Clean up StartBasicResponse (#35688)
This commit removes the parsing code from the PostStartBasicResponse server variant. It also makes the server response implement StatusToXContent which allows us to save a couple of lines of code in the corredponding REST action.

Relates to #35547
2018-11-21 10:21:10 +01:00
Ed Savage 4f857c4f8d
[HLRC][ML] Add ML revert model snapshot API (#35750)
Relates to #29827
2018-11-21 09:10:37 +00:00
Alan Woodward f6a43b5939
Add a prebuilt ICU Analyzer (#34958)
The ICU plugin provides the building blocks of an analysis chain, but doesn't actually have a prebuilt analyzer. It would be a better for users if there was a simple analyzer that they could use out of the box, and also something we can point to from the CJK Analyzer docs as a superior alternative.

Relates to #34285
2018-11-21 09:00:48 +00:00
Ioannis Kakavas e8ec4fad7b
[DOCS] Adjust Invalidate Token REST API docs (#35622)
- Renames API to Invalidate Token
- Explicitly calls out the possibility to invalidate refresh tokens
via this API
2018-11-21 09:32:56 +02:00
Martijn van Groningen a6647a20a9
[HLRC] Added support for CCR Unfollow API (#35693)
This change also adds documentation for the Unfollow API

Relates to #33824
2018-11-21 07:48:29 +01:00
Tim Vernum 30c5422561
Move XContent generation to HasPrivilegesResponse (#35616)
The RestHasPrivilegesAction previously handled its own XContent
generation. This change moves that into HasPrivilegesResponse and
makes the response implement ToXContent.

This allows HasPrivilegesResponseTests to be used to test
compatibility between HLRC and X-Pack internals.

A serialization bug (cluster privs) was also fixed here.
2018-11-21 14:33:10 +11:00
Christoph Büscher ff03443ab9
Fix problem with MatchNoDocsQuery in disjunction queries (#35726)
Queries across multiple fields generate MatchNoDocsQuerys for fields that are
unmapped. In certain situation this can lead to erroneous behaviour,
for example when an umapped field is used in a query_string query across
several fields. If some of the tokens in the query string get eliminated by an
analyzer on the mapped fields, the same token will currently generate
MatchNoDocsQuerys combined into a disjunction, which in turn
leads to no matches in the overall query. Instead we should simply not add
MatchNoDocsQuerys to those disjunctions.

Closes #34708
2018-11-21 03:49:49 +01:00
lcawl fa6d8a5004 [DOCS] Fixes navtitle for CCR API 2018-11-20 17:31:01 -08:00
Marios Trivyzas b1818dbdce
SQL: [docs] Add documentation for COALESCE (#35740)
Follows: #35253
2018-11-21 01:43:05 +01:00
Lisa Cawley 04a087fe2f
[DOCS] Adds authorization info for CCR APIs (#35557) 2018-11-20 16:23:01 -08:00
Benjamin Trent 84db1e42c0
HLRC: ML Get Calendar Events (#35747)
* HLRC: ML Get Calendar Events

* Addressing PR comments
2018-11-20 16:40:31 -06:00
Mayya Sharipova 643bb20137
Add a new query type - ScriptScoreQuery (#34533)
* Add a new query type - ScriptScoreQuery

script_score query uses script to calculate document scores.
Added as a substitute for function_score with an intentation
to deprecate function_scoreq query.

```http
GET /_search
{
    "query": {
        "script_score" : {
            "query": {
                "match": { "message": "elasticsearch" }
            },
            "script" : {
              "source": "Math.log(2 + doc['likes'].value)"
            },
            "min_score" : 2
        }
    }
}
```

Add several functions to painless to be used inside script_score:

double rational(double, double)
double sigmoid(double, double, double)
double randomNotReproducible() 
double randomReproducible(String, int) 

double decayGeoLinear(String, String, String, double, GeoPoint)
double decayGeoExp(String, String, String, double, GeoPoint)
double decayGeoGauss(String, String, String, double, GeoPoint)

double decayNumericLinear(String, String, String, double, double)
double decayNumericExp(String, String, String, double, double)
double decayNumericGauss(String, String, String, double, double)

double decayDateLinear(String, String, String, double, JodaCompatibleZonedDateTime)
double decayDateExp(String, String, String, double, JodaCompatibleZonedDateTime)
double decayDateGauss(String, String, String, double, JodaCompatibleZonedDateTime)

Date functions only works on dates in  the default format and default time zone
2018-11-20 16:10:06 -05:00
Ryan Ernst c07ad67718 Add 6.5.2 version 2018-11-20 12:33:26 -08:00
Gordon Brown 4bda469861
Fix line lengths in misc other files (#35650) 2018-11-20 12:29:48 -07:00
Vladimir Dolzhenko 953c8586df
Fix PrimaryAllocationIT#testForceStaleReplicaToBePromotedToPrimary (#35728)
Closes 35497
2018-11-20 20:05:23 +01:00
Alpar Torok 328448e8cd
Implement a property to allow for splitting tests. (#35473) 2018-11-20 20:35:53 +02:00
Gordon Brown 17780ce07e
Add HLRC docs for Delete Lifecycle Policy (#35664)
Adds documenatation for the Delete Lifecycle Policy API to the HLRC
documentation.
2018-11-20 11:32:41 -07:00
Armin Braun 89cf4a7397
NETWORKING: Fix IpFiltering Test (#35730)
* The port assigned to all loopback interfaces doesn't necessarily have to be the same  for ipv4 and ipv6
=> use actual address from profile instead of just port + loopback in test
* Closes #35584
2018-11-20 17:50:08 +01:00
Christoph Büscher e91f404d16
Remove remains of 'auto_generate_phrase_queries' (#35735)
This parameter in the `query_string` query was deprecated in 6.0 and ignored
since then. Its API methods and remaining uses can be removed in the upcoming
major version.

Relates to #35734
2018-11-20 16:07:17 +01:00
Tal Levy d061b3999a
[ILM] HLRC-ILM Retry Lifecycle Policy docs (#35715)
this adds documentation for the retry method in the
high-level-ilm-rest-client.

this PR also renames retryLifecycleStep to retryLifecyclePolicy in the index-lifecycle-client
2018-11-20 07:05:27 -08:00
Armin Braun 33c713ba60
TESTS: More Logging in LongGcDisruptionTests (#35702)
* The existing logging is not helpful enough to track down which threads hang, we need the hanging thread's stacktraces too
* Relates #35686
2018-11-20 15:36:01 +01:00
Benjamin Trent 7657e6d274
HLRC ML Add Event To Calendar API (#35704)
* HLRC: ML Adding Post event to calendar api

* Fixing tests and serialization

* removing unused import
2018-11-20 08:15:21 -06:00
Benjamin Trent d707838c02
HLRC: ML Delete job from calendar (#35713) 2018-11-20 07:43:34 -06:00
Armin Braun 8892408edd
SNAPSHOT: Fix Leaking Snapshot Task in IT (#35657)
* SNAPSHOT: Fix Leaking Snapshot Task in IT

* If the test randomization causes the request to not wait for snapshot completion
then the deleting of the still in progress snapshot could fail in the after hook
   * Fixed by deleting the snapshot in the test
* Closes #35642
2018-11-20 14:09:15 +01:00
Ed Savage 844483a99a
[HLRC][ML] Add ML update model snapshot API (#35537) (#35694)
Relates to #29827
2018-11-20 12:18:29 +00:00
Alpar Torok 8659af68e0
Auto skip license headers on no source (#35640)
* Unmute BuildExamplePluginsIT

* Skip licenseHeaders when there are no sources
2018-11-20 13:02:33 +02:00
Takuro Wada 7b2d547e8e [Docs] Delete inappropriate backtick (#35722) 2018-11-20 10:08:32 +01:00
Jim Ferenczi a5f5ceb869
Remove remaining line length violations for o.e.index (#35652)
This commit removes line length violations in the classes under
org.elasticsearch.index.
2018-11-20 08:09:14 +01:00
Simon Willnauer 29ef442841
Add a `_freeze` / `_unfreeze` API (#35592)
This commit adds a rest endpoint for freezing and unfreezing an index.
Among other cleanups mainly fixing an issue accessing package private APIs
from a plugin that got caught by integration tests this change also adds
documentation for frozen indices.
Note: frozen indices are marked as `beta` and available as a basic feature.

Relates to #34352
2018-11-20 08:03:24 +01:00
Tim Brooks a989b675b5
Remove NPE from IndexFollowingIT (#35717)
Currently there is a common NPE in the IndexFollowingIT that does not
indicate the test failing. This is when a cluster state listener is
called and certain index metadata is not yet available.

This commit checks that the metadata is not null before performing the
logic that depends on the metadata.
2018-11-19 20:38:49 -07:00
Tim Vernum d367894c31
DOCS: Add password_hash & refresh to Put User API (#35574)
PR #35242 formalised support for the password_hash field in the body
of the Put User security API.
Since this field is now validated and tested, it can also be
documented.

The Put User API also supports a "refresh" query parameter that was
not documented. This commit adds it to the docs.
2018-11-20 13:03:22 +11:00
Tim Vernum 3bba9a56de
TEST: Enable skipped test (#35682)
This test was skipped due to infr complications during the work
to open x-pack, but wasn't enabled again when the issue was resolved.

Relates: #29890
2018-11-20 12:51:01 +11:00
Jason Tedor 220c175484
Provide links to auto-follow docs (#35674)
This commit adds a link to the auto-follow docs from the CCR getting
started docs and the relevant CCR APIs.
2018-11-19 20:40:58 -05:00
Gordon Brown e012ac4649
Add HLRC docs for ILM Start/Stop/Status (#35672)
Adds HLRC documentation for Start ILM, Stop ILM, and ILM Status APIs, as
these APIs are tightly linked to each other.
2018-11-19 17:17:01 -07:00
Gordon Brown bf02897173
Fix doc link for HLRC async Put Lifecycle API (#35656)
The link to the HLRC documentation for the async Put Lifecycle
method was never updated to the correct link. This commit fixes that
link.
2018-11-19 16:30:32 -07:00
Tal Levy 90c0d444cf
[ILM] DOCS: clarify interaction between rollover and min_age (#35663)
This adds an example to drive home the semantics of `min_age` and explain
how actions from one phase must complete before `min_age` is even tested

closes #34020.
2018-11-19 13:26:20 -08:00
Benjamin Trent 7a779a9988
[TEST] fixing ClusterApplierService#testClusterStateUpdateLogging() (#35714) 2018-11-19 14:23:49 -06:00
Yannick Welsch 47ada69c46
Zen2: Move most integration tests to Zen2 (#35678)
Zen2 is now feature-complete enough to run most ESIntegTestCase tests. The changes in this PR
are as follows:
- ClusterSettingsIT is adapted to not be Zen1 specific anymore (it was using Zen1 settings).
- Some of the integration tests require persistent storage of the cluster state, which is not fully
implemented yet (see #33958). These tests keep running with Zen1 for now but will be switched
over as soon as that is fully implemented.
- Some very few integration tests are not running yet with Zen2 for other reasons, depending on
some of the other open points in #32006.
2018-11-19 21:15:29 +01:00
lcawl 3a232d7897 [DOCS] Adds anchor for SAML Kibana basic info 2018-11-19 11:42:45 -08:00
Andrey Ershov f9ecd0c49e
[Zen2] Write manifest file (#35049)
Elasticsearch node is responsible for storing cluster metadata. 
There are 2 types of metadata: global metadata and index metadata. 
`GatewayMetaState` implements `ClusterStateApplier` and receives all 
`ClusterStateChanged` events and is responsible for storing modified 
metadata to disk. 

When new `ClusterStateChanged` event is received, `GatewayMetaState` 
checks if global metadata has changed and if it's the case writes new 
global metadata to disk. After that `GatewayMetaState` checks if index 
metadata has changed or there are new indices assigned to this node and 
if it's the case writes new index metadata to disk. Atomicity of global 
metadata and index metadata writes is ensured by `MetaDataStateFormat` 
class.

Unfortunately, there is no atomicity when more than one metadata changes
(global and index, or metadata for two indices). And atomicity is 
important for Zen2 correctness.
This commit adds atomicity by adding a notion of manifest file, 
represented by `MetaState` class. `MetaState` contains pointers to
current metadata.
More precisely, it stores global state generation as long and map from 
`Index` to index metadata generation as long. Atomicity of writes for 
manifest file is ensured by `MetaStateFormat` class.

The algorithm of writing changes to the disk would be the following:

1. Write global metadata state file to disk and remember
it's generation.
2. For each new/changed index write state file to disk and remember
it's generation. For each not-changed index use generation from 
previous manifest file. If index is removed or this node is no longer
responsible for this index - forget about the index.
3. Create `MetaState` object using previously remembered generations and
write it to disk.
4. Remove old state files for global metadata, indices metadata and 
manifest.

Additonally new implementation relies on enhanced `MetaDataStateFormat` 
failure semantics, `applyClusterState` throws IOException, whose 
descendant `WriteStateException` could be (and should be in Zen2) 
explicitly handled.
2018-11-19 19:49:44 +01:00
Benjamin Trent 80bcf7c704
[ML]: adjusting post backport of (#35645) (#35696) 2018-11-19 12:49:07 -06:00