Commit Graph

41664 Commits

Author SHA1 Message Date
Jake Landis ad94e79350
ingest: processor stats (#34724)
This change introduces stats per processors. Total, time, failed,
current are currently supported. All pipelines will now show all
top level processors that belong to it. Failure processors are not
displayed, however, the time taken to execute the failure chain is part
of the stats for the top level processor.

The processor name is the type of the processor, ordered as defined in
the pipeline. If a tag for the processor is found, then the tag is
appended to the type.

Pipeline processors will have the pipeline name appended to the name of
the name of the processors (before the tag if one exists). If more
then one pipeline is used to process the document, then each pipeline
will carry its own stats. The outer most pipeline will also include the
inner most pipeline stats.

Conditional processors will only included in the stats if the condition evaluates
to true.
2018-10-23 07:30:52 -05:00
Marios Trivyzas 4a8386f271
SQL: Implement IN(value1, value2, ...) expression. (#34581)
Implement the functionality to translate the
`field IN (value1, value2,...)` expressions to proper Lucene queries
or painless script or local processors depending on the use case.

The `IN` expression can be used in SELECT, WHERE and HAVING clauses.

Closes: #32955
2018-10-23 14:28:23 +02:00
Igor Motov 123f784e32
Tests: Add checks to GeoDistanceQueryBuilderTests (#34273)
Adds checks for parsed geo distance query. It is a bit hack-ish since it
compares with query's toString() output, but it is better than no
checks. The parsed query itself has default visibility, so we cannot
access it here unless we move the test to org.apache.lucene.document
package.

Fixes #34043
2018-10-23 07:55:41 -04:00
Armin Braun 8e155b8430
INGEST: Rename Pipeline Processor Param. (#34733)
* `name` is more readable/ergnomic than having `pipeline` twice
2018-10-23 13:43:26 +02:00
Alexander Reelsen 83fd93b2fd
Core: Move IndexNameExpressionResolver to java time (#34507)
This switches from joda time to java time when resolving index names
using date math. This commit also removes two non registered settings
from the code, which could not be used anyway. An unused method was
removed as well.

Relates #27330
2018-10-23 13:26:02 +02:00
Frederik Bosch 183c32d4c3 [DOCS] Force Merge: clarify execution and storage requirements (#33882) 2018-10-23 12:25:04 +02:00
lipsill 4bda9bdab0 TESTING.asciidoc fix examples using forbidden annotation (#34515)
Clean up examples not to use forbidden test annotation `@Nightly`.
remove references to unused annotations
2018-10-23 12:36:42 +03:00
Marios Trivyzas e9e140790a
SQL: Implement `CONVERT`, an alternative to `CAST` (#34660)
`CONVERT` works exactly like cast with slightly different syntax:
`CONVERT(<value>, <data_type)` as opposed to `CAST(<value> AS <data_type>)`

Moreover it support format of the MS-SQL data types `SQL_<type>`,
e.g.: `SQL_INTEGER`

Closes: #34513
2018-10-23 11:21:15 +02:00
Alpar Torok 5fb4f32c91
[Build] Simplify testclusters configuration (#34334)
Remove interface, prefer straught implementaton instead.
2018-10-23 12:13:49 +03:00
Marios Trivyzas 1eb76f16b1
SQL: Allow min/max aggregates on date fields (#34699)
Allow `MIN()` and `MAX()` aggregate functions to operate
also on arguments of type DATE apart from the numeric ones.

Fixes: #34477
2018-10-23 11:11:22 +02:00
Alpar Torok 0536635c44
Upgrade forbiddenapis to 2.6 (#33809)
* Upgrade forbiddenapis to 2.6

Closes #33759

* Switch forbiddenApis back to official plugin

* Remove CLI based task

* Fix forbiddenApisJava9
2018-10-23 12:06:46 +03:00
Andrei Stefan 91434f7721
SQL: the SSL default configuration shouldn't override the https protocol if used (#34635)
* The default SSL option shouldn't override the https protocol if specified. Fixes https://github.com/elastic/elasticsearch/issues/33817
2018-10-23 11:04:16 +03:00
Jason Tedor f55843c9f0
Enable docs tests to have tear down snippet (#34716)
This commit adds the ability for docs tests to add a tear down
snippet. This snippet will be converted to a tear down section of the
generated REST tests.
2018-10-22 19:41:12 -04:00
Tim Brooks d4bb3d1ce5
Update SSLDriver for JDK 11 changes (#34398)
JDK11 introduced some changes with the SSLEngine. A number of error
messages were changed. Additionally, there were some behavior changes
in regard to how the SSLEngine handles closes during the handshake
process. This commit updates our tests and SSLDriver to support these
changes.
2018-10-22 19:01:28 -04:00
jaymode ae1e46b852
Test: add empty test to PainlessDomainSplitIT
All of the tests in PainlessDomainSplitIT have an awaitsfix, which
causes the build to fail since no tests are run. This adds an empty
test to get the build going again.

Relates #34683
Relates #32966
2018-10-22 14:01:44 -06:00
Julie Tibshirani f854330e06
Make sure to use the type _doc in the REST documentation. (#34662)
* Replace custom type names with _doc in REST examples.
* Avoid using two mapping types in the percolator docs.
* Rename doc -> _doc in the main repository README.
* Also replace some custom type names in the HLRC docs.
2018-10-22 11:54:04 -07:00
Zachary Tong d981746142
[Docs] clarification about cardinality accuracy (#34616)
Adds a bit more clarification about how accuracy is dependent
on the dataset in question.

Closes #18231
2018-10-22 13:15:45 -04:00
Jay Modi c344293aed
Security: don't call prepare index for reads (#34568)
The security native stores follow a pattern where
`SecurityIndexManager#prepareIndexIfNeededThenExecute` wraps most calls
made for the security index. The reasoning behind this was to check if
the security index had been upgraded to the latest version in a
consistent manner. However, this has the potential side effect that a
read will trigger the creation of the security index or an updating of
its mappings, which can lead to issues such as failures due to put
mapping requests timing out even though we might have been able to read
from the index and get the data necessary.

This change introduces a new method, `checkIndexVersionThenExecute`,
that provides the consistent checking of the security index to make
sure it has been upgraded. That is the only check that this method
performs prior to running the passed in operation, which removes the
possible triggering of index creation and mapping updates for reads.

Additionally, areas where we do reads now check the availability of the
security index and can short circuit requests. Availability in this
context means that the index exists and all primaries are active.

This is the fixed version of #34246, which was reverted.

Relates #33205
2018-10-22 10:12:37 -06:00
Lee Hinman 5dd79bf58c
Make accounting circuit breaker settings dynamic (#34372)
* Make accounting circuit breaker settings dynamic

These missed the original property making them dynamic. This fixes the issue so
these can now be set at any time.

Resolves #34368
2018-10-22 09:55:00 -06:00
Julie Tibshirani fbb9ac34f9
Deprecate type exists requests. (#34663) 2018-10-22 08:46:11 -07:00
Jason Tedor 52fc502b7e
Fix the casing in the names of some CCR classes
We should be consistent here. We were already using the casing "Ccr" and
this is the preferred casing for Java class names. This commit adjusts
the names of some classes that were using the casing "CCR" to be "Ccr".
2018-10-22 11:25:00 -04:00
Christoph Büscher e869f9a78c [Docs] Update synonym-tokenfilter.asciidoc (#34706)
Remove ugly double-dot.
2018-10-22 17:18:29 +02:00
Jason Tedor 66ebcdc112
Enable using defined setups from test setup in docs (#34681)
In the docs tests, we have pre-defined setups in the build.gradle file,
and we can also define test setup sections within the doc page
itself. Alas, these two are incompatible in that if you try to use a
pre-defined setup alongside a test setup section, the pre-defined setup
will be silently ignored. This commit enables pre-defined setup sections
to be used together with test setup sections. The ordering here is that
pre-defined setup sections will be executed first, followed by the test
setup section.
2018-10-22 11:14:39 -04:00
Jason Tedor 243335e2ba
Allow set section in setup section of REST tests (#34678)
This commit enables using a set section in the setup section of REST
tests.
2018-10-22 11:14:27 -04:00
Jason Tedor 7af19b8f81
Migrate wait for pending tasks helper to server (#34675)
In some of our X-Pack REST tests we have to wait for pending tasks to
complete. We are now needing this functionality in ESRestTestCase for
the docs tests where we run against X-Pack features. This commit moves
the helper method that we have in X-Pack to ESRestTestCase, and removes
duplicate logic from waiting for rollup tasks to complete.
2018-10-22 11:14:02 -04:00
Martijn van Groningen 92e34732f5
[CCR] Remove ccr related metadata between tests for single node tests too 2018-10-22 09:15:22 +02:00
Martijn van Groningen b6750cf6c2
[CCR] Muted tests
Relates to #34696
2018-10-22 08:47:31 +02:00
Martijn van Groningen f51301a1a6
[CCR] Moved integration test 2018-10-22 08:44:41 +02:00
Martijn van Groningen b816837d39
[CCR] Always remove persistent tasks metadata between tests and
better handle assertion errors between tests.
2018-10-22 08:15:43 +02:00
Jason Tedor e562afad69
Awaits fix PainlessDomainSplitIT#testIsolated
This test fails reliably with a compilation error. This commit awaits
fix the test.
2018-10-21 19:13:53 -04:00
Jason Tedor 0577703183
Revert "ingest: processor stats (#34202)"
This reverts commit 6567729600.
2018-10-21 13:16:15 -04:00
Jason Tedor bf5f0af491
Use trial license in docs tests (#34673)
This commit switches to using a trial license in the docs tests that run
on the default distribution. This is needed so that docs tests can be
executed against non-basic features.
2018-10-21 06:23:42 -04:00
Ryan Ernst 222652dfce
Scripting: Convert script fields to use script context (#34164)
This commit removes the use of SearchScript for script fields and adds
a new FieldScript.
2018-10-20 16:33:49 -07:00
Nhat Nguyen 7ab464807d TEST: Mute testDedupByPrimaryTerm
Should be fixed by #34667
2018-10-20 18:23:02 -04:00
Jake Landis 6567729600
ingest: processor stats (#34202)
This change introduces stats per processors. Total, time, failed,
current are currently supported. All pipelines will now show all
top level processors that belong to it. Failure processors are not
displayed, however, the time taken to execute the failure chain is part
of the stats for the top level processor.

The processor name is the type of the processor, ordered as defined in
the pipeline. If a tag for the processor is found, then the tag is
appended to the type.

Pipeline processors will have the pipeline name appended to the name of
the name of the processors (before the tag if one exists). If more
then one pipeline is used to process the document, then each pipeline
will carry its own stats. The outer most pipeline will also include the
inner most pipeline stats.

Conditional processors will only included in the stats if the condition evaluates
to true.
2018-10-20 16:01:01 -05:00
Jason Tedor dca452f043
Fix JDK references in contributing docs (#34669)
After we bumped the JDK compiler minimum requirements to JDK 11, we
missed updating the contributing docs to point to JDK 11 instead of JDK
10. This commit fixes that.

Additionally, when we stopped requiring JDK 7 to run the reindex from
old tests, we did not update the contributing docs to reflect this. This
commit also fixes that.
2018-10-20 16:01:32 -04:00
Jason Tedor 19cc4439d7
Fix cluster formation BWC versions
We support JDK 10 through the 6.3 and 6.4 series. This commit adjusts
the BWC logic when starting nodes during tests to account for this.
2018-10-20 15:58:38 -04:00
Jason Tedor aebb855f38
Add missing word to remote clusters docs
This commit adds a missing word in the initial section of the remote
clusters docs.
2018-10-20 12:11:41 -04:00
Jason Tedor aca8c705ae
Fix name of remote clusters reference
This commit fixes the name of the reference to the remote clusters
module on the modules page by adding a missing character.
2018-10-20 11:18:27 -04:00
Jason Tedor 8bed0c68e2
Fix remote clusters section link
This commit fixes the remote clusters section link by changing some
underscores to dashes.
2018-10-20 11:16:39 -04:00
Jason Tedor 77d23a2007
Fix remote clusters section link
This fixes a link to the configuring remote clusters section of the
docs.
2018-10-20 10:28:25 -04:00
Jason Tedor 19ee5da037
Fix reference to remote clusters configuration
This commit fixes a reference to configuring a remote cluster in the
cross-cluster search and security docs.
2018-10-20 09:35:24 -04:00
Jason Tedor ab5a0b15f6
Fix formatting of remotes clusters module section
This commit fixes the formatting of the remote clusters module section
on the modules page. We add a missing newline and wrap the columns at
80.
2018-10-20 09:04:48 -04:00
Jason Tedor 91dbb22950
Fix reference to remote clusters docs
This commit fixes a reference to the remote clusters docs from the
modules page.
2018-10-20 09:02:00 -04:00
Jason Tedor b2dc58450e
Separate remote clusters docs from CCS (#34612)
With remote clusters taking on a larger role, we have make the
infrastructure more generic than being tied to cross-cluster search
(CCS). We want to refer to the remote clusters configuration in the
cross-cluster replication (CCR) docs. Yet, these docs are still tied to
CCS. This commit extracts the remote clusters docs from CCS (with some
wording changes to make them more general) so that we can refer to them
in the CCR docs.
2018-10-20 08:33:59 -04:00
Yannick Welsch 9fb2f7cc20
HLRC: Delete role API (#34620)
Adds the "Delete role" API to the high-level REST client.
2018-10-20 12:11:36 +02:00
Alpar Torok adc3816e32
Enable the Gradle daemon (#34545) 2018-10-20 11:14:41 +03:00
Nhat Nguyen d90b6730c7
CCR: Following primary should process NoOps once (#34408)
This is a follow-up for #34288.

Relates #34412
2018-10-19 21:10:13 -04:00
Jim Ferenczi ba87c543c0 [TEST] Fix sporadic failures in CompletionSuggestSearchIT#testTiebreak
Relates #34508
2018-10-20 01:05:48 +02:00
Jason Tedor 48ab6d7c92
Switch old reindex tests to use use JDK 8 (#34646)
This commit switches the old reindex tests to use JDK 8. This enables us
to not require that JDK 7 be installed, easing the burden on developers.
2018-10-19 15:11:48 -04:00