Commit Graph

6920 Commits

Author SHA1 Message Date
Nik Everett b91e9d85e9 Fix SHAs
Original commit: elastic/x-pack-elasticsearch@15c2669416
2017-10-27 17:12:15 -04:00
Costin Leau f26a9acdc4 Update ANTLR in qa project as well
Original commit: elastic/x-pack-elasticsearch@58c10398b4
2017-10-27 23:43:37 +03:00
Costin Leau 660cb3b814 Upgrade ANTLR to 4.5.3
Relates to elastic/x-pack-elasticsearch#26834

Original commit: elastic/x-pack-elasticsearch@57ba8f1d5f
2017-10-27 23:29:00 +03:00
Costin Leau 8d5572a77b SQL: improve validation of properties (both names and values) (elastic/x-pack-elasticsearch#2759)
* refactor Configuration class

move away from Properties
perform validation of settings names and values at startup
better exception handling

Original commit: elastic/x-pack-elasticsearch@d8a9edeccf
2017-10-27 22:54:03 +03:00
Costin Leau af591b9edd SQL: Remove usage of Settings inside SqlSettings (elastic/x-pack-elasticsearch#2757)
* Remove usage of Settings inside SqlSettings

Also hook client timeouts to the backend
Set UTC as default timezone when using CSV
As the JVM timezone changes, make sure to pin it to UTC since this is what the results are computed against

Original commit: elastic/x-pack-elasticsearch@3e7aad8c1f
2017-10-27 18:55:59 +03:00
Costin Leau 6b1d0d1c8e SQL: Remove time threshold for rule executor (elastic/x-pack-elasticsearch#2750)
Refactor RuleExecutor to eliminate time-based thresholds

Original commit: elastic/x-pack-elasticsearch@ba131f8058
2017-10-27 18:43:13 +03:00
Alexander Reelsen 940eabd5f3 Watcher: Add thread pool rejection to execution state (elastic/x-pack-elasticsearch#2805)
The execution state of a watch did not differentiate between failures of
the execution like a broken painless script and a thread pool rejection.

This adds an own state, which allows to aggregate on such data in the
watch history, which should ease debugging issues a bit.

Original commit: elastic/x-pack-elasticsearch@351e64e14d
2017-10-27 16:37:14 +02:00
Martijn van Groningen 96b0b4e96d test: refresh only once to workaround phrase suggester edge case
Relates to elastic/x-pack-elasticsearch#2804

Original commit: elastic/x-pack-elasticsearch@3f2b6b7eea
2017-10-27 15:05:18 +02:00
Hendrik Muhs f74e680142 [ML] add detectorIndex to modelplot and forecast (elastic/x-pack-elasticsearch#2796)
add detector_index to model plots and forecast

relates elastic/x-pack-elasticsearch#2547

corresponding ml-cpp change: elastic/machine-learning-cpp#361

Original commit: elastic/x-pack-elasticsearch@5927d8578e
2017-10-27 12:54:42 +02:00
Dimitris Athanasiou c7e94b3b4c [ML] Enable overall buckets aggregation at a custom bucket span (elastic/x-pack-elasticsearch#2782)
For the purpose of getting this API consumed by our UI, returning
overall buckets that match the job's largest `bucket_span` can
result in too much data. The UI only ever displays a few buckets
in the swimlane. Their span depends on the time range selected and
the screen resolution, but it will only ever be a relatively
low number.

This PR adds the ability to aggregate overall buckets in a user
specified `bucket_span`. That `bucket_span` may be equal or
greater to the largest job's `bucket_span`. The `overall_score`
of the result overall buckets is the max score of the
corresponding overall buckets with a span equal to the job's
largest `bucket_span`.

The implementation is now chunking the bucket requests
as otherwise the aggregation would fail when too many buckets
are matching.

Original commit: elastic/x-pack-elasticsearch@981f7a40e5
2017-10-27 11:14:13 +01:00
Martijn van Groningen e028716bec test: use a single primary shard to workaround an edge case with the phrase suggester
Relates to elastic/x-pack-elasticsearch#2804

Original commit: elastic/x-pack-elasticsearch@afd028faf7
2017-10-27 10:41:19 +02:00
Lisa Cawley 215f289a8c [DOCS] Reformatted security troubleshooting pages (elastic/x-pack-elasticsearch#2799)
Original commit: elastic/x-pack-elasticsearch@ec9969ec7a
2017-10-26 13:56:57 -07:00
Nik Everett 3d0f57d976 Add remaining security tests (elastic/x-pack-elasticsearch#2797)
This adds all of the security tests I think SQL is going to need for the initial release. SQL is still missing an entire scenario though: SSL enabled. Either way, this removes some `NOCOMMIT`s in `qa/sql/security`. Adding the SSL testing can come later.

Original commit: elastic/x-pack-elasticsearch@851620b606
2017-10-26 17:23:35 +00:00
Lisa Cawley 61bfa39331 [DOCS] Added setup-passwords command parameters (elastic/x-pack-elasticsearch#2735)
* [DOCS] Added setup-passwords command parameters

* [DOCS] Addressed feedback in setup-passwords command

Original commit: elastic/x-pack-elasticsearch@5401994c56
2017-10-26 08:28:32 -07:00
Nik Everett 52d9de1de7 Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@dd027d9ca5
2017-10-26 09:51:04 -04:00
Martijn van Groningen 62215f1fae security: Fail request if suggesters are used and DLS is active.
Original commit: elastic/x-pack-elasticsearch@056c735e77
2017-10-26 08:02:31 +02:00
Tim Vernum 8985625ea5 [Security] BulkShardRequest may have multiple indices (elastic/x-pack-elasticsearch#2742)
If a bulk update references aliases rather than concrete indices,
it is possible that a single shard level request could have multiple distinct "index names", potentially including "date math".
Those names will resolve to the same concrete index, but they might have different privileges.

Original commit: elastic/x-pack-elasticsearch@34cfd11df8
2017-10-26 15:34:58 +11:00
Jason Tedor 70a38ec545 Enable BWC testing against other remotes
This commit enables BWC testing against remotes on GitHub other than
elastic/elasticsearch.git and elastic/x-pack-elasticsearch.git.

Relates elastic/x-pack-elasticsearch#2707

Original commit: elastic/x-pack-elasticsearch@9028f2e089
2017-10-25 22:39:58 -04:00
lcawley 615de68711 [DOCS] Added xpack.watcher.history.cleaner_service_enabled
Original commit: elastic/x-pack-elasticsearch@ceead2a1c2
2017-10-25 16:35:20 -07:00
Lisa Cawley c67ec73cb5 [DOCS] Add xpack.ml.max_model_memory_limit (elastic/x-pack-elasticsearch#2787)
* [DOCS] Add xpack.ml.max.model_memory_limit

* [DOCS] Addressed feedback on model limit setting

Original commit: elastic/x-pack-elasticsearch@77a10bfe0e
2017-10-25 09:15:19 -07:00
Lee Hinman cdaa047d56 [TEST] Fix compilation for ARS stats exposing
Original commit: elastic/x-pack-elasticsearch@eb0ad99434
2017-10-24 11:09:19 -06:00
Lisa Cawley 73e819b0bd [DOCS] Add secure versions of SSL passphrases (elastic/x-pack-elasticsearch#2478)
* [DOCS] Add secure versions of SSL passphrases

* [DOCS] Add secure xpack settings

* [DOCS] Clarify type of keystore

* [DOCS] Added secure settings to security page

* [DOCS] Clarify X-Pack secure settings

* [DOCS] Reformat secure X-Pack settings

Original commit: elastic/x-pack-elasticsearch@efe043fb67
2017-10-24 16:38:37 +01:00
Lisa Cawley 3a6870f0dc [DOCS] Added SSL settings to X-Pack installation (elastic/x-pack-elasticsearch#2733)
Original commit: elastic/x-pack-elasticsearch@c88cce763a
2017-10-24 08:34:07 -07:00
Albert Zaharovits 403912b8a2 SecureSettings ignored by customAuditIndexSettings (elastic/x-pack-elasticsearch#2748)
customAuditIndexSettings does not submit SecureSettings with putIndexMapping.

relates elastic/x-pack-elasticsearch#2705

* Randomize SecureSetting in testcase

* Apply feedback

Original commit: elastic/x-pack-elasticsearch@1a5414b057
2017-10-24 13:50:35 +03:00
Alexander Reelsen 6f437c973a Watcher: Ensure all templates exist before starting watcher (elastic/x-pack-elasticsearch#2765)
This is to ensure that the required templates exist (which are added by
the WatcherIndexTemplateRegistry) before actually starting watcher.

Relates elastic/x-pack-elasticsearch#2761 

Original commit: elastic/x-pack-elasticsearch@568088061f
2017-10-23 11:57:40 +02:00
Martijn van Groningen c9682d02d4 fix test
Original commit: elastic/x-pack-elasticsearch@7ca5e0fb54
2017-10-23 09:58:33 +02:00
Martijn van Groningen 652f6560b8 security: Always allow access to a rootdoc's nested documents if access to rootdoc is allowed
relates elastic/x-pack-elasticsearch#2665

Original commit: elastic/x-pack-elasticsearch@2bbddd1dd2
2017-10-23 09:28:53 +02:00
Igor Motov 387944d176 SQL: Move special joda time handling into DocValueExtractor (elastic/x-pack-elasticsearch#2758)
Moves joda time handling into DocValueExtractor, that's the only place where it occurs at the moment.

Original commit: elastic/x-pack-elasticsearch@205e82990a
2017-10-22 15:54:00 +02:00
Nik Everett d7ab14ee54 Fix SQL security build's run config
Now it is the same as the integTest config agian. I should have done
this in elastic/x-pack-elasticsearch#2753 but I forgot.

Original commit: elastic/x-pack-elasticsearch@bbbb8b1dc7
2017-10-19 13:14:30 -04:00
Nik Everett 56ce29c6bf Security tests for SQL's CLI and JDBC (elastic/x-pack-elasticsearch#2770)
Add security tests for SQL's CLI and JDBC features. I do this by factoring out all the "actions" from the existing REST tests into an interface and implement it for REST, CLI, and JDBC. This way we can share the same audit log assertions across tests and we can be sure that the REST, CLI, and JDBC tests cover all the same use cases.

Original commit: elastic/x-pack-elasticsearch@82ff66a520
2017-10-19 17:13:31 +00:00
Alexander Reelsen b76c85e7fd Docs: Change port to be an integer in htttp input docs
Original commit: elastic/x-pack-elasticsearch@768ec54e03
2017-10-19 13:53:24 +02:00
Nik Everett 65f2b9fe01 Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@9fc67cbfee
2017-10-18 10:27:19 -04:00
Simon Willnauer 2d1ce76194 Adopt core that `_flush` and `_force_merge` doesn't refresh anymore (elastic/x-pack-elasticsearch#2752)
Relates to elastic/elasticsearch#27000

Original commit: elastic/x-pack-elasticsearch@52e9951094
2017-10-16 10:16:50 +02:00
Nik Everett 770bc9516c Switch sql audit tests from index to the log file (elastic/x-pack-elasticsearch#2753)
This is *way* faster because we don't have to wait for the audit
events from previous test runs to drain into the index. And we
don't have to wait for the index's refresh cycle. We have to parse
the log lines which is a bit more brittle but it feels worth it
at this point.

Original commit: elastic/x-pack-elasticsearch@4b1758fc32
2017-10-14 12:27:51 +00:00
Nik Everett 269b656cf2 Prune the forbiddenPatterns tasks that are skipped (elastic/x-pack-elasticsearch#2754)
We don't need to blindly skip all of these tasks. We don't have
NOCOMMITs everywhere, just lots of places....

Original commit: elastic/x-pack-elasticsearch@d772cb1b96
2017-10-13 16:21:33 +00:00
Igor Motov e1d4fdc6d3 SQL: Unknown filter type [pattern_capture] warning in SQL IT Tests (elastic/x-pack-elasticsearch#2746)
Fixes java.lang.IllegalArgumentException: Unknown filter type [pattern_capture] for [email] warnings that pollute log files during execution of the AbstractSqlIntegTestCase-based tests.

Related to elastic/x-pack-elasticsearch#2501

Original commit: elastic/x-pack-elasticsearch@b9ec3c20d0
2017-10-13 11:55:48 -04:00
Lisa Cawley 84b00995e3 [DOCS] Removed ML GA limitation (elastic/x-pack-elasticsearch#2751)
Original commit: elastic/x-pack-elasticsearch@30aeb0d269
2017-10-13 16:52:40 +01:00
Nik Everett 99fea5f448 Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@d521ecda35
2017-10-13 10:39:28 -04:00
Jason Tedor c35efb7adf Replace global checkpoint sync test
This commit replaces the REST test that the global checkpoint sync
action runs successfully as a privileged user. The test needs to be
replaced because it has a small race condition. Namely, the check that
the post-operation global checkpoint sync was successful could run
before the sync finishes running. To address this, we replace the REST
test with a test where we have a little more control and can assert busy
to avoid this race from failing the test.

Relates elastic/x-pack-elasticsearch#2749

Original commit: elastic/x-pack-elasticsearch@ea585b843c
2017-10-13 10:05:59 -04:00
Costin Leau 353c0c500b Fix and enable datetime tests (elastic/x-pack-elasticsearch#2680)
Firstly, data in H2 is now stored in TIMESTAMP WITH TIME ZONE since H2
does not allow a global TZ to be set and picks the JVM TZ when a record
is read.
JdbcAssert is now aware of this allows TIMESTAMP with TZ == TIMESTAMP

Discovered a serious bug in DateTimeFunction - unfortunately date
histogram is not useful except for year since most extract functions
avoid ordering which a histogram preserves.
Thus most DTF are now terms aggs with scripting.

Improved a bug that caused duplicate functions to not be detected because
of aliasing.

Moved some datetime tests to CSV but the aggs tests now are in sync with
H2

Fixed bug that caused arithmetic on aggs to not be properly resolved by
splitting the processor definition tree to aggName (unresolved) and
aggPath (resolved)

Original commit: elastic/x-pack-elasticsearch@e75ada68f1
2017-10-13 13:52:48 +03:00
Tanguy Leroux a6776cef97 [Monitoring] Add interval_ms to Monitoring documents (elastic/x-pack-elasticsearch#2650)
This commit adds a new interval_ms field to the monitoring documents. 
This field indicates the current collection interval for Elasticsearch or 
external monitored systems. The value is indexed as a long.

Related to elastic/x-pack-elasticsearch#212

Original commit: elastic/x-pack-elasticsearch@2ceb20455c
2017-10-13 11:18:47 +02:00
David Roberts 10cc0088e4 [DOCS] Make clearer that xpack.xyz.enabled settings are node settings (elastic/x-pack-elasticsearch#2731)
The discussion in elastic/x-pack-elasticsearch#2697 shows that this was not clear before.

relates elastic/x-pack-elasticsearch#2697

Original commit: elastic/x-pack-elasticsearch@87553faa2c
2017-10-13 09:22:21 +01:00
Costin Leau 2bbb86eff7 Eliminate NOCOMMITS per discussion
Original commit: elastic/x-pack-elasticsearch@bd091a6608
2017-10-13 00:33:16 +03:00
Costin Leau 21c8a9168b Add antlr to qa tests for IDE execution
Original commit: elastic/x-pack-elasticsearch@d146b98be3
2017-10-12 23:59:17 +03:00
Nik Everett 69d1a5c5dd Give audit logs more time to show up in SQL tests
CI has been failing since we had to remove our tweaks to the audit
logging configuration due to
https://github.com/elastic/x-pack-elasticsearch/issues/2705

So we increase the timeout. The timeout is very very long, but it
seems like we need it for CI which is often slow.

Original commit: elastic/x-pack-elasticsearch@91a926a031
2017-10-12 13:53:09 -04:00
Nik Everett 69a326dd94 Fix CliErrorsIT in sql security tests
It was disabled because the CLI didn't work with security but
we've since fixed that so we can enable it.

Original commit: elastic/x-pack-elasticsearch@8d9b5ad89b
2017-10-12 12:42:13 -04:00
Nik Everett 852af7de57 Fix error handling in SQL's CLI (elastic/x-pack-elasticsearch#2730)
We weren't returning errors correctly from the server
or catching them correctly in the CLI. This fixes that
and adds simple integration tests.

Original commit: elastic/x-pack-elasticsearch@259da0da6f
2017-10-12 16:32:15 +00:00
Jay Modi 9028c0a642 Allow PkiRealm to use truststore.password setting (elastic/x-pack-elasticsearch#2727)
This change fixes an incorrect check for a missing password setting for the PKI realm. The check
only allowed the secure setting to be used for the PkiRealm password even though the legacy setting
is still valid. This change fixes the check.

Relates elastic/x-pack-elasticsearch#2487

Original commit: elastic/x-pack-elasticsearch@a4524c2c05
2017-10-12 10:07:08 -06:00
Nik Everett 6478713304 Add support for username and password in SQL CLI (elastic/x-pack-elasticsearch#2718)
Add support for username and password in SQL CLI and adds tests that CLI works with security.

Original commit: elastic/x-pack-elasticsearch@39c8dbfc97
2017-10-12 15:55:29 +00:00
lcawley 7f37c2c431 [DOCS] Fixed broken link to java transport client
Original commit: elastic/x-pack-elasticsearch@f3036bb2a0
2017-10-12 14:29:20 +01:00