Commit Graph

39348 Commits

Author SHA1 Message Date
Christoph Büscher fa1052017c
[Test] Minor changes to rank_eval tests (#29577)
Removing an enum in favour of local constants to simplify tests and
removing a few deprecated method calls and warnings.
2018-04-19 13:50:18 +02:00
Tanguy Leroux e2d770d9b9
Fix missing node id prefix in startup logs (#29534)
When `node.name` is not set, some log traces at startup time does not
show the node id.
2018-04-19 09:40:25 +02:00
Martijn van Groningen 8afa7c174f
Added painless execute api. (#29164)
Added an api that allows to execute an arbitrary script and a result to be returned.

```
POST /_scripts/painless/_execute
{
  "script": {
    "source": "params.var1 / params.var2",
    "params": {
      "var1": 1,
      "var2": 1
    }
  }
}
```

Relates to #27875
2018-04-19 09:33:34 +02:00
Martijn van Groningen 621a1935b8
test: also assert deprecation warning after clusters have been closed. 2018-04-19 09:20:04 +02:00
Ryan Ernst 1cb3a9d9dc Test: Guard deprecation check when 0 nodes created
The internal test cluster can sometimes have 0 nodes. In this situation,
the http.enabled flag will never be read, and thus no deprecation
warning will be emitted. This commit guards the deprecation warning
check in this case.
2018-04-18 21:18:56 -07:00
Jason Tedor 4e376070a3 Remove index thread pool from monitoring filters (elastic/x-pack-elasticsearch#4409)
The index thread pool is gone, so the index thread pool stats do not any
longer need to be whitelisted in the monitoring stats.

Original commit: elastic/x-pack-elasticsearch@83f7742805
2018-04-18 21:02:06 -04:00
Ryan Ernst d0ee36b192 Update tests using http.enabled (elastic/x-pack-elasticsearch#4413)
This commit updates tests using http.enabled to expect a deprecation
warning.

relates https://github.com/elastic/elasticsearch/pull/29591

Original commit: elastic/x-pack-elasticsearch@92e5ea9c74
2018-04-18 17:36:14 -07:00
Ryan Ernst 98d776edaf
Networking: Deprecate http.enabled setting (#29591)
This commit deprecates the http.enabled, in preparation for removing the
feature in 7.0.

relates #12792
2018-04-18 17:36:09 -07:00
debadair 0c9baebe15
[DOCS] Added include for internal highlighters section. (#29597) 2018-04-18 16:56:09 -07:00
Mayya Sharipova bf6cfff080
[DOCS] Update highlighting docs (#28802)
- add more explanation to some highlighting parameters
- add a document describing how highlighters work internally
2018-04-18 17:41:19 -04:00
Jack Conradson da9a6899ff
Painless: modify grammar to allow more statement delimiters (#29566)
This allows the grammar to determine when and what delimiters statements will use by
splitting up the statements into regular statements and delimited statements, those that do
not require a delimiter versus those that do. This allows consumers of the statements to
determine what delimiters the statements will use so that in certain cases semicolons are
not necessary like when there's a closing right bracket.

This change removes the need for semicolon insertion in the lexer, simplifying the existing
lexer quite a bit. It also ensures that there isn't a need to track semicolons being inserted
into places that aren't necessary such as array initializers.
2018-04-18 10:32:42 -07:00
Lisa Cawley dcedffb0f0 [DOCS] Replaces InetSocketTransportAddress in code samples (elastic/x-pack-elasticsearch#4404)
Original commit: elastic/x-pack-elasticsearch@85b11cccc7
2018-04-18 09:15:43 -07:00
Nhat Nguyen f97aec7b8b Sibling of enforce access to translog via engine
Since elastic/elasticsearch#29542, we no longer expose translog instance
but only provide creating translog snapshot method. This commit adapts
that change in CCR branch.

Relates elastic/elasticsearch#29542
2018-04-18 11:54:00 -04:00
Nhat Nguyen 4be1488324 Merge branch 'master' into ccr
* master:
  Remove the index thread pool (#29556)
  Remove extra copy in ScriptDocValues.Strings
  Fix full cluster restart test recovery (#29545)
  Fix binary doc values fetching in _search (#29567)
  Mutes failing MovAvgIT tests
  Fix the assertion message for an incorrect current version. (#29572)
  Fix the version ID for v5.6.10. (#29570)
  Painless Spec Documentation Clean Up (#29441)
  Add versions 5.6.10 and 6.2.5
  [TEST] test against scaled value instead of fixed epsilon in MovAvgIT
  Remove `flatSettings` support from request classes (#29560)
  MapperService to wrap a single DocumentMapper. (#29511)
  Fix dependency checks on libs when generating Eclipse configuration. (#29550)
  Add null_value support to geo_point type (#29451)
  Add documentation about the include_type_name option. (#29555)
  Enforce translog access via engine (#29542)
2018-04-18 11:41:08 -04:00
Paul Sanwald 3e7fccddaf
Add a CHANGELOG file for release notes. (#29450)
* Add a CHANGELOG file for 7.x release notes.

* update file to include 6.x

* remove confusing comment and small edit to section title

* moving CHANGELOG file under docs directory, as it pertains to release notes.
2018-04-18 07:42:05 -07:00
Jason Tedor 2b47d67d95
Remove the index thread pool (#29556)
Now that single-document indexing requests are executed on the bulk
thread pool the index thread pool is no longer needed. This commit
removes this thread pool from Elasticsearch.
2018-04-18 09:18:08 -04:00
Jim Ferenczi 9d11c7a6c1 Remove extra copy in ScriptDocValues.Strings
This commit removes a BytesRef copy introduced in #29567 and not
required.

Relates #29567
2018-04-18 15:13:24 +02:00
Jason Tedor a548a7f2cb
Fix full cluster restart test recovery (#29545)
The test was using a parameter on GET /_cluster/health that older nodes
do not understand. Yet, we do no even need to make this call here, we
can use ensure green for the index.
2018-04-18 08:43:43 -04:00
Jim Ferenczi a7c9857976
Fix binary doc values fetching in _search (#29567)
Binary doc values are retrieved during the DocValueFetchSubPhase through an instance of ScriptDocValues.
Since 6.0 ScriptDocValues instances are not allowed to reuse the object that they return (https://github.com/elastic/elasticsearch/issues/26775) but BinaryScriptDocValues doesn't follow this restriction and reuses instances of BytesRefBuilder among different documents.
This results in `field` values assigned to the wrong document in the response.
This commit fixes this issue by recreating the BytesRef for each value that needs to be returned.

 Fixes #29565
2018-04-18 13:01:06 +02:00
David Roberts fd3f079276 [ML] Exclude failed jobs from node allocation decision (elastic/x-pack-elasticsearch#4395)
When calculating the current load on each ML node during the
node allocation process we should be ignoring failed jobs.
This is because failed jobs do not have a corresponding native
process, so do not consume memory or CPU resources.

relates elastic/x-pack-elasticsearch#4381

Original commit: elastic/x-pack-elasticsearch@1cb0ca973e
2018-04-18 10:10:55 +01:00
Jim Ferenczi 8b34066d8b Mutes failing MovAvgIT tests
Relates #29456
2018-04-18 10:54:45 +02:00
Julie Tibshirani c8209fa7b1
Fix the assertion message for an incorrect current version. (#29572) 2018-04-17 19:27:02 -07:00
Lisa Cawley 70f9bcc0d1 [DOCS] Adds link to clear role cache API (elastic/x-pack-elasticsearch#4405)
Original commit: elastic/x-pack-elasticsearch@d2d74069bf
2018-04-17 18:03:46 -07:00
Lisa Cawley 4c38e610fd [DOCS] Adds enabled parameter to user management API (elastic/x-pack-elasticsearch#4364)
Original commit: elastic/x-pack-elasticsearch@166bef4702
2018-04-17 16:15:54 -07:00
Julie Tibshirani 52858ba760
Fix the version ID for v5.6.10. (#29570) 2018-04-17 16:04:16 -07:00
Nhat Nguyen 566d1826c8
Upgrade to Lucene-7.4-snapshot-330fd18f20 (#29562)
This snapshot includes the fix for LUCENE-8253.
See: https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=330fd18
2018-04-17 18:11:26 -04:00
Alexander Reelsen 4c00361b01 Watcher: Remove broken tests (elastic/x-pack-elasticsearch#4399)
These tests test actually watcher 6.x features and are not needed in the
master branch anymore, as there is no way that watcher is running on the
master node only in Elaticsearch 7.x.

Original commit: elastic/x-pack-elasticsearch@ac5b933994
2018-04-17 23:12:49 +02:00
Zachary Tong 97fa56443d [Docs] Document limitations around rolling up heterogeneous schemas
Rolling up indices that contain multiple schemas requires extra
considerations for the user, and in some cases, some limitations.  This
commit tries to enumerate the issues so the user is aware.

Original commit: elastic/x-pack-elasticsearch@bf96eeab4e
2018-04-17 19:45:22 +00:00
Jack Conradson c833167d84
Painless Spec Documentation Clean Up (#29441)
Created a flatter structure for the different sections.  Cleaned up comments, keywords, and literals.  Used callouts for examples where it made sense.
2018-04-17 12:16:08 -07:00
Dimitris Athanasiou 7969eb7db7 Add versions 5.6.10 and 6.2.5 2018-04-17 18:47:20 +01:00
Zachary Tong cfc9d12acc [TEST] test against scaled value instead of fixed epsilon in MovAvgIT
When comparing doubles, fixed epsilons can fail because the absolute
difference in values may be quite large, even though the relative
difference is tiny (e.g. with two very large numbers).

Instead, we can scale epsilon by the absolute value of the expected
value.  This means we are looking for a diff that is epsilon-percent
away from the value, rather than just epsilon.

This is basically checking the relative error using junit's assertEqual.

Closes #29456, unmutes the test
2018-04-17 17:33:18 +00:00
Lisa Cawley ed2c6ead7f [DOCS] Move monitoring security and configuration information to Elasticsearch Reference (elastic/x-pack-elasticsearch#4370)
Original commit: elastic/x-pack-elasticsearch@5ff35f7ec6
2018-04-17 10:27:31 -07:00
Nhat Nguyen 67990121fc Merge branch 'master' into ccr 2018-04-17 12:34:31 -04:00
Luca Cavanna 9c8ebb608f
Remove `flatSettings` support from request classes (#29560)
As part of adding support for new API to the high-level REST client,
we added support for the `flat_settings` parameter to some of our
request classes. We added documentation that such flag is only ever
read by the high-level REST client, but the truth is that it doesn't
do anything given that settings are always parsed back into a `Settings`
object, no matter whether they are returned in a flat format or not.

It was a mistake to add support for this flag in the context of the
high-level REST client, hence this commit removes it.
2018-04-17 18:18:21 +02:00
Adrien Grand c228c48e70 Fix compilation after elastic/elasticsearch#29511.
Original commit: elastic/x-pack-elasticsearch@4240815b5b
2018-04-17 17:38:13 +02:00
Chris Earle 51d87994ca [Monitoring] Thread _xpack/monitoring/_bulk (elastic/x-pack-elasticsearch#4393)
Instead of allowing the `_xpack/monitoring/_bulk` to remain on the same
thread, it should execute on a separate thread to avoid blocking the
http worker thread whenever the exporters get stuck waiting on the
monitoring cluster.

Original commit: elastic/x-pack-elasticsearch@25ce9a4df0
2018-04-17 11:16:49 -04:00
Adrien Grand d7be9185c8
MapperService to wrap a single DocumentMapper. (#29511)
This refactors MapperService so that it wraps a single `DocumentMapper` rather
than a `Map<String, DocumentMapper>`. We will need follow-ups since I haven't
fixed most APIs that still expose collections of types of mappers, but this is
a start...
2018-04-17 17:11:27 +02:00
Adrien Grand a8c2cc6ce7
Fix dependency checks on libs when generating Eclipse configuration. (#29550)
Currently this fails because the Eclipse configuration splits the main and test
folders into separate projects to avoid circular dependencies.

Relates #29336
2018-04-17 17:11:12 +02:00
Igor Motov 983d6c15a2
Add null_value support to geo_point type (#29451)
Adds support for null_value attribute to the geo_point types.

Closes #12998
2018-04-17 10:19:54 -04:00
Adrien Grand 3367948be6
Add documentation about the include_type_name option. (#29555)
This option will be useful in 7.x to prepare for upgrade to 8.0 which won't
know about types anymore.
2018-04-17 15:04:46 +02:00
Nhat Nguyen 45c6c20467
Enforce translog access via engine (#29542)
Today the translog of an engine is exposed and can be accessed directly.
While this exposure offers much flexibility, it also causes these troubles:

- Inconsistent behavior between translog method and engine method.
For example, rolling a translog generation via an engine also trims
unreferenced files, but translog's method does not.

- An engine does not get notified when critical errors happen in translog
as the access is direct.

This change isolates translog of an engine and enforces all accesses to
translog via the engine.
2018-04-17 08:03:41 -04:00
Ioannis Kakavas f1902aba39 [DOCS] Update documentation for SAML metadata signing (elastic/x-pack-elasticsearch#4356)
Updates documentation describing the parameters for optionally signing the
SAML medatata document that the saml-metadata command creates.

Original commit: elastic/x-pack-elasticsearch@0e8146541a
2018-04-17 14:23:26 +03:00
Jason Tedor 1dd0fd4874
Deprecate the index thread pool (#29540)
The index thread pool is no longer needed as its primary use-case for
single-document indexing requests has been relieved now that
single-document indexing requests are converted to bulk indexing
requests (with a single document payload).
2018-04-17 06:47:30 -04:00
Jason Tedor faa7fe86c5
Introduce analyze thread pool (#29541)
We want to remove the index thread pool as it is no longer needed since
single-document indexing requests are executed as bulk requests
now. Analyze requests are also executed on the index thread pool though
and they need a thread pool to execute on. The bulk thread does not seem
like the right thread pool, let us keep that thread pool conceptually
for bulk requests and free for bulk requests. None of the existing
thread pools make sense for analyze requests either. The generic thread
pool would be a terrible choice since it has an unbounded queue and that
is a bad idea for user-facing APIs. This commit introduces a small by
default (size=1, queue_size=16) thread pool for analyze requests.
2018-04-17 06:46:15 -04:00
Adrien Grand d223bcf7ab
Add the `include_type_name` option to the search and document APIs. (#29506)
This commit add the `include_type_name` option to the `index`, `update`,
`delete`, `get`, `bulk` and `search` APIs. When set to `false`, the response
will omit the `_type` in the response. This option doesn't work if the endpoint
contains a type. For instance, the following call would succeed:

```
GET index/_doc/1?include_type_name=false
```

But the following one would fail:

```
GET index/some_type/1?include_type_name=false
```

Relates #15613
2018-04-17 11:29:08 +02:00
Martijn van Groningen 56ca59a513 Add the ability to the follow index to follow an index in a remote cluster.
The follow index api completely reuses CCS infrastructure that was exposed via:
https://github.com/elastic/elasticsearch/pull/29495

This means that the leader index parameter support the same ccs index
to indicate that an index resides in a different cluster.

I also added a qa module that smoke tests the cross cluster nature of ccr.
The idea is that this test just verifies that ccr can read data from a
remote leader index and that is it, no crazy randomization or indirectly
testing other features.
2018-04-17 07:36:40 +02:00
Tim Vernum eccf3899a2 Tie zip output to command line options in certutil (elastic/x-pack-elasticsearch#4354)
Previously "certutil" would generate a zip file if there were multiple
certificates.
However, this means that if the user specified "-multiple" or "-in"
then the output format will vary based on whether they entered
multiple instance names (-multiple) or whether the input file
contained multiple instance records (-in).
It is better if the output format is predictable based on the command
line options, so this change forces zip output whenever any of the
following command line options are supplied:
    -pem
    -keep-ca-key
    -multiple
    -in

Original commit: elastic/x-pack-elasticsearch@344baa5f17
2018-04-17 11:05:30 +10:00
Tim Vernum 5eac9fd1a4 [DOCS] Improve docs for "--pass" option in certutil (elastic/x-pack-elasticsearch#4352)
The handling of the "--pass" option has complex rules due to the
different requirements of different output formats, and the way
joptsimple handles specs with optional arguments.

A standard unix shell will parse
    certutil --pass=""
as being identical to
    certutil --pass=
and joptsimple treats that as being "no argument", i.e. the same as
    certutil --pass

The only way to pass a blank password is
    certutil --pass ""
(though this is shell dependent)

Original commit: elastic/x-pack-elasticsearch@bc09d0f298
2018-04-17 11:05:09 +10:00
Tim Vernum 7cd99d2f96 Trigger log4j loading prior to context classloader (elastic/x-pack-elasticsearch#4355)
OpenSAML requires that the context classloader be set to the
classloader that contains the opensaml jar files in order to
initialise itself. However, sometimesrthis seems to cause the
classloading for slf4j/log4j to take place while that context
classloader is set, which fails.

This change forces slf4j/log4j to be initialised prior to setting
the context classloader.

Original commit: elastic/x-pack-elasticsearch@85ed8eb488
2018-04-17 11:04:31 +10:00
Nhat Nguyen fd161d2659 TEST: Mute testEnsureWeReconnect
Relates #29547
2018-04-16 18:31:34 -04:00