Commit Graph

6999 Commits

Author SHA1 Message Date
Lisa Cawley 73e819b0bd [DOCS] Add secure versions of SSL passphrases (elastic/x-pack-elasticsearch#2478)
* [DOCS] Add secure versions of SSL passphrases

* [DOCS] Add secure xpack settings

* [DOCS] Clarify type of keystore

* [DOCS] Added secure settings to security page

* [DOCS] Clarify X-Pack secure settings

* [DOCS] Reformat secure X-Pack settings

Original commit: elastic/x-pack-elasticsearch@efe043fb67
2017-10-24 16:38:37 +01:00
Lisa Cawley 3a6870f0dc [DOCS] Added SSL settings to X-Pack installation (elastic/x-pack-elasticsearch#2733)
Original commit: elastic/x-pack-elasticsearch@c88cce763a
2017-10-24 08:34:07 -07:00
Albert Zaharovits 403912b8a2 SecureSettings ignored by customAuditIndexSettings (elastic/x-pack-elasticsearch#2748)
customAuditIndexSettings does not submit SecureSettings with putIndexMapping.

relates elastic/x-pack-elasticsearch#2705

* Randomize SecureSetting in testcase

* Apply feedback

Original commit: elastic/x-pack-elasticsearch@1a5414b057
2017-10-24 13:50:35 +03:00
Alexander Reelsen 6f437c973a Watcher: Ensure all templates exist before starting watcher (elastic/x-pack-elasticsearch#2765)
This is to ensure that the required templates exist (which are added by
the WatcherIndexTemplateRegistry) before actually starting watcher.

Relates elastic/x-pack-elasticsearch#2761 

Original commit: elastic/x-pack-elasticsearch@568088061f
2017-10-23 11:57:40 +02:00
Martijn van Groningen c9682d02d4 fix test
Original commit: elastic/x-pack-elasticsearch@7ca5e0fb54
2017-10-23 09:58:33 +02:00
Martijn van Groningen 652f6560b8 security: Always allow access to a rootdoc's nested documents if access to rootdoc is allowed
relates elastic/x-pack-elasticsearch#2665

Original commit: elastic/x-pack-elasticsearch@2bbddd1dd2
2017-10-23 09:28:53 +02:00
Igor Motov 387944d176 SQL: Move special joda time handling into DocValueExtractor (elastic/x-pack-elasticsearch#2758)
Moves joda time handling into DocValueExtractor, that's the only place where it occurs at the moment.

Original commit: elastic/x-pack-elasticsearch@205e82990a
2017-10-22 15:54:00 +02:00
Nik Everett d7ab14ee54 Fix SQL security build's run config
Now it is the same as the integTest config agian. I should have done
this in elastic/x-pack-elasticsearch#2753 but I forgot.

Original commit: elastic/x-pack-elasticsearch@bbbb8b1dc7
2017-10-19 13:14:30 -04:00
Nik Everett 56ce29c6bf Security tests for SQL's CLI and JDBC (elastic/x-pack-elasticsearch#2770)
Add security tests for SQL's CLI and JDBC features. I do this by factoring out all the "actions" from the existing REST tests into an interface and implement it for REST, CLI, and JDBC. This way we can share the same audit log assertions across tests and we can be sure that the REST, CLI, and JDBC tests cover all the same use cases.

Original commit: elastic/x-pack-elasticsearch@82ff66a520
2017-10-19 17:13:31 +00:00
Alexander Reelsen b76c85e7fd Docs: Change port to be an integer in htttp input docs
Original commit: elastic/x-pack-elasticsearch@768ec54e03
2017-10-19 13:53:24 +02:00
Nik Everett 65f2b9fe01 Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@9fc67cbfee
2017-10-18 10:27:19 -04:00
Simon Willnauer 2d1ce76194 Adopt core that `_flush` and `_force_merge` doesn't refresh anymore (elastic/x-pack-elasticsearch#2752)
Relates to elastic/elasticsearch#27000

Original commit: elastic/x-pack-elasticsearch@52e9951094
2017-10-16 10:16:50 +02:00
Nik Everett 770bc9516c Switch sql audit tests from index to the log file (elastic/x-pack-elasticsearch#2753)
This is *way* faster because we don't have to wait for the audit
events from previous test runs to drain into the index. And we
don't have to wait for the index's refresh cycle. We have to parse
the log lines which is a bit more brittle but it feels worth it
at this point.

Original commit: elastic/x-pack-elasticsearch@4b1758fc32
2017-10-14 12:27:51 +00:00
Nik Everett 269b656cf2 Prune the forbiddenPatterns tasks that are skipped (elastic/x-pack-elasticsearch#2754)
We don't need to blindly skip all of these tasks. We don't have
NOCOMMITs everywhere, just lots of places....

Original commit: elastic/x-pack-elasticsearch@d772cb1b96
2017-10-13 16:21:33 +00:00
Igor Motov e1d4fdc6d3 SQL: Unknown filter type [pattern_capture] warning in SQL IT Tests (elastic/x-pack-elasticsearch#2746)
Fixes java.lang.IllegalArgumentException: Unknown filter type [pattern_capture] for [email] warnings that pollute log files during execution of the AbstractSqlIntegTestCase-based tests.

Related to elastic/x-pack-elasticsearch#2501

Original commit: elastic/x-pack-elasticsearch@b9ec3c20d0
2017-10-13 11:55:48 -04:00
Lisa Cawley 84b00995e3 [DOCS] Removed ML GA limitation (elastic/x-pack-elasticsearch#2751)
Original commit: elastic/x-pack-elasticsearch@30aeb0d269
2017-10-13 16:52:40 +01:00
Nik Everett 99fea5f448 Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@d521ecda35
2017-10-13 10:39:28 -04:00
Jason Tedor c35efb7adf Replace global checkpoint sync test
This commit replaces the REST test that the global checkpoint sync
action runs successfully as a privileged user. The test needs to be
replaced because it has a small race condition. Namely, the check that
the post-operation global checkpoint sync was successful could run
before the sync finishes running. To address this, we replace the REST
test with a test where we have a little more control and can assert busy
to avoid this race from failing the test.

Relates elastic/x-pack-elasticsearch#2749

Original commit: elastic/x-pack-elasticsearch@ea585b843c
2017-10-13 10:05:59 -04:00
Costin Leau 353c0c500b Fix and enable datetime tests (elastic/x-pack-elasticsearch#2680)
Firstly, data in H2 is now stored in TIMESTAMP WITH TIME ZONE since H2
does not allow a global TZ to be set and picks the JVM TZ when a record
is read.
JdbcAssert is now aware of this allows TIMESTAMP with TZ == TIMESTAMP

Discovered a serious bug in DateTimeFunction - unfortunately date
histogram is not useful except for year since most extract functions
avoid ordering which a histogram preserves.
Thus most DTF are now terms aggs with scripting.

Improved a bug that caused duplicate functions to not be detected because
of aliasing.

Moved some datetime tests to CSV but the aggs tests now are in sync with
H2

Fixed bug that caused arithmetic on aggs to not be properly resolved by
splitting the processor definition tree to aggName (unresolved) and
aggPath (resolved)

Original commit: elastic/x-pack-elasticsearch@e75ada68f1
2017-10-13 13:52:48 +03:00
Tanguy Leroux a6776cef97 [Monitoring] Add interval_ms to Monitoring documents (elastic/x-pack-elasticsearch#2650)
This commit adds a new interval_ms field to the monitoring documents. 
This field indicates the current collection interval for Elasticsearch or 
external monitored systems. The value is indexed as a long.

Related to elastic/x-pack-elasticsearch#212

Original commit: elastic/x-pack-elasticsearch@2ceb20455c
2017-10-13 11:18:47 +02:00
David Roberts 10cc0088e4 [DOCS] Make clearer that xpack.xyz.enabled settings are node settings (elastic/x-pack-elasticsearch#2731)
The discussion in elastic/x-pack-elasticsearch#2697 shows that this was not clear before.

relates elastic/x-pack-elasticsearch#2697

Original commit: elastic/x-pack-elasticsearch@87553faa2c
2017-10-13 09:22:21 +01:00
Costin Leau 2bbb86eff7 Eliminate NOCOMMITS per discussion
Original commit: elastic/x-pack-elasticsearch@bd091a6608
2017-10-13 00:33:16 +03:00
Costin Leau 21c8a9168b Add antlr to qa tests for IDE execution
Original commit: elastic/x-pack-elasticsearch@d146b98be3
2017-10-12 23:59:17 +03:00
Nik Everett 69d1a5c5dd Give audit logs more time to show up in SQL tests
CI has been failing since we had to remove our tweaks to the audit
logging configuration due to
https://github.com/elastic/x-pack-elasticsearch/issues/2705

So we increase the timeout. The timeout is very very long, but it
seems like we need it for CI which is often slow.

Original commit: elastic/x-pack-elasticsearch@91a926a031
2017-10-12 13:53:09 -04:00
Nik Everett 69a326dd94 Fix CliErrorsIT in sql security tests
It was disabled because the CLI didn't work with security but
we've since fixed that so we can enable it.

Original commit: elastic/x-pack-elasticsearch@8d9b5ad89b
2017-10-12 12:42:13 -04:00
Nik Everett 852af7de57 Fix error handling in SQL's CLI (elastic/x-pack-elasticsearch#2730)
We weren't returning errors correctly from the server
or catching them correctly in the CLI. This fixes that
and adds simple integration tests.

Original commit: elastic/x-pack-elasticsearch@259da0da6f
2017-10-12 16:32:15 +00:00
Jay Modi 9028c0a642 Allow PkiRealm to use truststore.password setting (elastic/x-pack-elasticsearch#2727)
This change fixes an incorrect check for a missing password setting for the PKI realm. The check
only allowed the secure setting to be used for the PkiRealm password even though the legacy setting
is still valid. This change fixes the check.

Relates elastic/x-pack-elasticsearch#2487

Original commit: elastic/x-pack-elasticsearch@a4524c2c05
2017-10-12 10:07:08 -06:00
Nik Everett 6478713304 Add support for username and password in SQL CLI (elastic/x-pack-elasticsearch#2718)
Add support for username and password in SQL CLI and adds tests that CLI works with security.

Original commit: elastic/x-pack-elasticsearch@39c8dbfc97
2017-10-12 15:55:29 +00:00
lcawley 7f37c2c431 [DOCS] Fixed broken link to java transport client
Original commit: elastic/x-pack-elasticsearch@f3036bb2a0
2017-10-12 14:29:20 +01:00
Jason Tedor aece28c286 Add test for global checkpoint sync with security
After a write operation on an index, a post-operation global checkpoint
sync fires. Previously, this action fired on the same user as executed
the write action. If the user did not have priviledges for this action,
the global checkpoint sync would fail. With an upstream change in core,
this action now fires as the system user. This commit adds a test that
create a user that has minimal write permissions on an index, but none
that would imply it could execute the global checkpoint sync. This then
serves as a test that the upstream change to fire the global checkpoint
sync as the system user is correct. This test must run as a mulit-node
test so that a replica is a assigned so that the global checkpoint sync
fires in the first place. This test does indeed fail without the
upstream change, and passes with it.

Relates elastic/x-pack-elasticsearch#2744

Original commit: elastic/x-pack-elasticsearch@bf7e771756
2017-10-12 09:19:17 -04:00
lcawley b628815dbe [DOCS] Fixed link to X-Pack transport client
Original commit: elastic/x-pack-elasticsearch@0870334e4b
2017-10-12 13:41:14 +01:00
Lisa Cawley 604229cd4d [DOCS] Added transport client info for X-Pack (elastic/x-pack-elasticsearch#2737)
* [DOCS] Added transport client info for X-Pack

* [DOCS] Relocated X-Pack java client info

* [DOCS] Added transport client deprecation info

Original commit: elastic/x-pack-elasticsearch@416aab1d76
2017-10-12 13:18:44 +01:00
Costin Leau fa4504ed28 Fix some NOCOMMITs
Original commit: elastic/x-pack-elasticsearch@1a6ac1e6c6
2017-10-12 14:24:56 +03:00
Costin Leau e3d072aeea Make JDBC driver throw only SQLException (elastic/x-pack-elasticsearch#2708)
A JDBC driver should throw only checked SQLExceptions.
Introduce JdbcSQLException and fix some no-commits along the way.

Original commit: elastic/x-pack-elasticsearch@299fcf9ace
2017-10-12 12:27:19 +03:00
Tanguy Leroux 0299886388 [Tests] Use XPack Usage API to verify Monitoring exporters are disabled (elastic/x-pack-elasticsearch#2648)
This commit changes the MonitoringIt and XPackRestIT tests so that the
disableMonitoring() method now use the XPack Usage API in order to check
that the monitoring exporters are correctly disabled. It checks at the
beginning of the tests (all exporters must be disabled before running
the test) and also at the end of the test.

This commit also fixes a bug in MonitoringIT where the Bulk thread pool
active queue was wrongly extracted from the response's map, forcing the
test to always wait for 30sec.

relates elastic/x-pack-elasticsearch#2459

Original commit: elastic/x-pack-elasticsearch@2d349e840f
2017-10-12 09:36:44 +02:00
Igor Motov 7f0c44f138 SQL: Fix string to boolean conversion (elastic/x-pack-elasticsearch#2728)
This commit also adds tests for conversion to boolean

Original commit: elastic/x-pack-elasticsearch@d7740929a2
2017-10-11 19:56:36 -04:00
Igor Motov 06b4c043b9 SQL: Add toXcontent test for SqlResponse (elastic/x-pack-elasticsearch#2726)
Original commit: elastic/x-pack-elasticsearch@aa7b8f1a91
2017-10-11 19:55:59 -04:00
Costin Leau 2026198dd4 Add size for column tests (elastic/x-pack-elasticsearch#2685)
Add displaySize to columnInfo

Original commit: elastic/x-pack-elasticsearch@ed1d265e98
2017-10-12 00:03:41 +03:00
Costin Leau c9a41e111e Improve Sql exception reporting (elastic/x-pack-elasticsearch#2679)
Better handling of SQL exceptions (result of incorrect queries) vs
unexpected ones (engine failure, ES...)

Original commit: elastic/x-pack-elasticsearch@2698402cdb
2017-10-11 23:23:52 +03:00
Costin Leau 57fcbb81cb SQL: Improve JDBC communication (elastic/x-pack-elasticsearch#2660)
* Improve JDBC communication
Jdbc HTTP client uses only one url for messages and relies on / for ping
Fixed ES prefix being discarded (missing /)
Add HEAD handler for JDBC endpoint

Original commit: elastic/x-pack-elasticsearch@389f82262e
2017-10-11 23:03:03 +03:00
Costin Leau 6a1806a3eb Fix line too long
Original commit: elastic/x-pack-elasticsearch@084095a944
2017-10-11 22:32:14 +03:00
Costin Leau d6881e25c9 Use embedded client inside the planner as well
Original commit: elastic/x-pack-elasticsearch@c9d20672be
2017-10-11 22:08:20 +03:00
Nik Everett 125f140620 Fix SQL CLI tests in windows (elastic/x-pack-elasticsearch#2738)
We put the CLI in unix mode so if we send it
`\r\n` (the default in windows) then it'll
spit out extra "you are on a line continuation"
characters (`|`). Instead, we can use `\n`
directly and everything works.

I've also added a timeout to the reads from the
CLI because it makes the tests easier to debug.

Original commit: elastic/x-pack-elasticsearch@69f69f4092
2017-10-11 18:31:45 +00:00
Tanguy Leroux ea4bff1d43 [Monitoring] Align MonitoringBulkDoc serialization with 6.0 (elastic/x-pack-elasticsearch#2736)
The version used in serialization must be aligned with 6.0/6.x.

Original commit: elastic/x-pack-elasticsearch@db63b91bc6
2017-10-11 17:56:24 +02:00
Lisa Cawley 2455415a04 [DOCS] Small fixes in the overall buckets API (elastic/x-pack-elasticsearch#2732)
* [DOCS] Small fixes in the overall buckets API

* [DOCS] Addressed feedback in overall buckets API

Original commit: elastic/x-pack-elasticsearch@4f79bc9a50
2017-10-11 16:25:05 +01:00
lcawley 723dd49905 [DOCS] Fixed typo in count function
Original commit: elastic/x-pack-elasticsearch@34c821796b
2017-10-11 16:16:28 +01:00
Tanguy Leroux 8484680007 Few fixes in packaging tests
This commit fixes indentation in certgen.bash, adds a check on cluster
health in bootstrap_password.bash and fixes a bug in xpack.bash

Original commit: elastic/x-pack-elasticsearch@d6847f6640
2017-10-11 11:53:10 +02:00
David Roberts c84d69fde3 [DOCS] Fix ML post_data docs (elastic/x-pack-elasticsearch#2689)
It was pointed out in
https://github.com/elastic/elasticsearch-net/pull/2856#discussion_r142830656
that our post_data docs incorrectly say that reset_start and reset_end are
body parameters.  In fact they are query parameters.

There were also a number of other errors and ommissions on this page that I
have attempted to correct.

Original commit: elastic/x-pack-elasticsearch@c83decacc7
2017-10-11 10:47:07 +01:00
Tim Vernum a4f7db4f66 [Security] Improve error messages in setup-passwords (elastic/x-pack-elasticsearch#2724)
Provides more verbose messaging around errors and possible causes when the tool aborts.

This change is primarily focused on errors connecting to the Elasticsearch node when TLS is enabled on the HTTP connection.

Original commit: elastic/x-pack-elasticsearch@aa8f7c6143
2017-10-11 12:32:35 +10:00
Tim Vernum bc038b323d [Security] Apply validation when parsing certgen input (elastic/x-pack-elasticsearch#2711)
When certgen configuration was read from an input file (`-in` option) validation errors were collected but never reported. Depending on the type of error this may have caused the tool to exit with an internal error (e.g. NPE).

Validation is now applied after parsing the file and if errors are found the tool exits.

Original commit: elastic/x-pack-elasticsearch@b2262ed1d7
2017-10-11 12:30:19 +10:00