611 Commits

Author SHA1 Message Date
Costin Leau
d6881e25c9 Use embedded client inside the planner as well
Original commit: elastic/x-pack-elasticsearch@c9d20672be
2017-10-11 22:08:20 +03:00
Nik Everett
125f140620 Fix SQL CLI tests in windows (elastic/x-pack-elasticsearch#2738)
We put the CLI in unix mode so if we send it
`\r\n` (the default in windows) then it'll
spit out extra "you are on a line continuation"
characters (`|`). Instead, we can use `\n`
directly and everything works.

I've also added a timeout to the reads from the
CLI because it makes the tests easier to debug.

Original commit: elastic/x-pack-elasticsearch@69f69f4092
2017-10-11 18:31:45 +00:00
Tanguy Leroux
8484680007 Few fixes in packaging tests
This commit fixes indentation in certgen.bash, adds a check on cluster
health in bootstrap_password.bash and fixes a bug in xpack.bash

Original commit: elastic/x-pack-elasticsearch@d6847f6640
2017-10-11 11:53:10 +02:00
Nik Everett
b02e569b38 Speed up audit log testing for SQL (elastic/x-pack-elasticsearch#2721)
SQL was cleaning up the audit logs *after* each test
but this switches it to cleaning up the audit logs
*before* each test. This is faster because we can
generate a lot of audit logs before the first test.

Original commit: elastic/x-pack-elasticsearch@71d8f76667
2017-10-10 16:48:37 -04:00
Nik Everett
9633b1d7bc Remove an @AwaitsFix on test in SQL (elastic/x-pack-elasticsearch#2719)
It has been fixed already, actually.

Original commit: elastic/x-pack-elasticsearch@46c8dacc50
2017-10-10 16:44:21 -04:00
Costin Leau
1cd6fb23ec Make SearchCursor limit aware
Original commit: elastic/x-pack-elasticsearch@c4839bc293
2017-10-10 19:22:42 +03:00
Dimitris Athanasiou
5eea355b33 [ML] Add overall buckets api (elastic/x-pack-elasticsearch#2713)
Adds the GET overall_buckets API.

The REST end point is: GET
/_xpack/ml/anomaly_detectors/job_id/results/overall_buckets

The API returns overall bucket results. An overall bucket
is a summarized bucket result over multiple jobs.
It has the `bucket_span` of the longest job's `bucket_span`.
It also has an `overall_score` that is the `top_n` average of the
max anomaly scores per job.

relates elastic/x-pack-elasticsearch#2693

Original commit: elastic/x-pack-elasticsearch@ba6061482d
2017-10-10 14:41:24 +01:00
Tanguy Leroux
1ed4be1471 Show exit code in Bootstrap Password packaging tests
Also cleans up some files before the test is executed,
and explicitly binds to 127.0.0.1/9200.

Original commit: elastic/x-pack-elasticsearch@778584ea78
2017-10-10 09:51:22 +02:00
Igor Motov
79d6b88763 Merge remote-tracking branch 'elastic/master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@7503023447
2017-10-09 15:47:35 -04:00
David Roberts
9ad961088d [TEST] Wait a little longer for named pipes to open in unit tests (elastic/x-pack-elasticsearch#2712)
Same fix as elastic/x-pack-elasticsearch#987, but for the unit tests.  The slowness affecting EBS
volumes created from snapshots can affect CI as it runs on AWS instances.

Original commit: elastic/x-pack-elasticsearch@306b8110b7
2017-10-09 13:09:17 +01:00
Albert Zaharovits
98347088f9 Fix LDAP Authc connections deadlock (elastic/x-pack-elasticsearch#2587)
Do not execute bind on on the LDAP reader thread

Each LDAP connection has a single associated thread, executing the handlers for async requests; this is managed by the LDAP library. The bind operation is blocking for the connection. It is a deadlock to call bind, if on the LDAP reader thread for the same connection, because waiting for the bind response blocks the thread processing responses (for this connection).
This will execute the bind operation (and the subsequent runnable) on a thread pool after checking for the conflict above.

Closes: elastic/x-pack-elasticsearch#2570, elastic/x-pack-elasticsearch#2620

Original commit: elastic/x-pack-elasticsearch@404a3d8737
2017-10-09 13:06:12 +03:00
Yannick Welsch
20c0e01523 Set minimum_master_nodes on rolling-upgrade test
Companion commit to elastic/elasticsearch#26911

Original commit: elastic/x-pack-elasticsearch@dcdbd14f78
2017-10-09 10:59:58 +02:00
Simon Willnauer
cd14f33ae2 Return List instead of an array from settings (elastic/x-pack-elasticsearch#2694)
XPack side of elastic/elasticsearch#26903

Original commit: elastic/x-pack-elasticsearch@f0390974ab
2017-10-09 09:52:34 +02:00
Boaz Leskes
9041211690 Setup debug logging for qa.full-cluster-restart
Original commit: elastic/x-pack-elasticsearch@1f7f8f2a92
2017-10-06 23:02:48 +02:00
Nik Everett
5239e49b75 Fix security tests
The security tests were broken because of
https://github.com/elastic/x-pack-elasticsearch/issues/2705
so this works around it.

Original commit: elastic/x-pack-elasticsearch@34f499d55e
2017-10-06 16:03:01 -04:00
Nik Everett
5806b620c5 Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@141332d3fc
2017-10-06 13:57:55 -04:00
Tanguy Leroux
a9d7c232be Fix packaging tests
Since elastic/elasticsearch#26878, array and list of settings are
internally represented as actual lists. This makes filtering works
as expected when it comes to filter out arrays/lists.

The packaging tests used to check the presence of the XPack SSL
certificated_authorities setting which should have always been filtered.

By fixing the filtering of settings, elastic/elasticsearch#26878 broke
this packaging test.

This commit changes this test so that it does not expect certificated_authorities
setting to exist in the Nodes Info response.

relates elastic/x-pack-elasticsearch#2688

Original commit: elastic/x-pack-elasticsearch@cb299186b8
2017-10-06 14:36:44 +02:00
Costin Leau
31a952993a Merge remote-tracking branch 'remotes/upstream/master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@2ac0dab27b
2017-10-05 19:16:36 +03:00
Boaz Leskes
45c62cca63 full-cluster-restart tests: prevent shards from going inactive
FullClusterRestartIT.testRecovery relies on the translogs not being flushed

Original commit: elastic/x-pack-elasticsearch@4ee41372b6
2017-10-05 10:10:10 +02:00
Boaz Leskes
0d08e6cb73 Promote common rest test utility methods to ESRestTestCase
We have duplicates in some classes and I was about to create one more.

Original commit: elastic/x-pack-elasticsearch@78ff553992
2017-10-05 10:10:10 +02:00
Simon Willnauer
f5864c7291 Move away from Settings#getAsMap() (elastic/x-pack-elasticsearch#2661)
Relates to elastic/elasticsearch#26845

Original commit: elastic/x-pack-elasticsearch@0323ea07a5
2017-10-04 01:21:59 -06:00
Costin Leau
e0d02033de More JDBC improvements
properly return the precision for VARCHAR
ignore type when specified in index pattern

Original commit: elastic/x-pack-elasticsearch@71a5ac1812
2017-09-30 00:40:57 +03:00
Costin Leau
cc66bbaa00 All jdbc client escaping is done on the server
Original commit: elastic/x-pack-elasticsearch@2b8b7c8c2e
2017-09-29 20:09:58 +03:00
Nik Everett
c8e69b160e SQL: Fix build
Fix a few forgetten things from the validation change.

Original commit: elastic/x-pack-elasticsearch@807098dc6a
2017-09-28 16:57:58 -04:00
Costin Leau
d634314dd1 Add comments to SqlSession
Original commit: elastic/x-pack-elasticsearch@82291d41c8
2017-09-28 22:29:23 +03:00
Costin Leau
aca8a5b6c0 Analysis validation (elastic/x-pack-elasticsearch#2651)
Rework unresolved items messages
Update URLs in embedded HttpServers
Add antlr-runtime for embedded classpath

Original commit: elastic/x-pack-elasticsearch@36f0331d90
2017-09-28 22:27:20 +03:00
Igor Motov
5f385d9155 SQL: Disable column type autodetection in CSV tests (elastic/x-pack-elasticsearch#2634)
Related to elastic/x-pack-elasticsearch#2608

Original commit: elastic/x-pack-elasticsearch@d1a45eab15
2017-09-27 18:19:07 -04:00
Costin Leau
c33dfe7dbe Hook _translate rest endpoint
Add rest testcase plus fix some NOCOMMITs

Original commit: elastic/x-pack-elasticsearch@150576869c
2017-09-27 18:50:35 +03:00
Igor Motov
fdd98f01ed Merge remote-tracking branch 'elastic/master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@273e0a110e
2017-09-26 11:59:49 -04:00
Nik Everett
42dd1947cd SQL: Randomize fetch size in tests (elastic/x-pack-elasticsearch#2626)
Now that we have fetch size working consistently we should randomize
the fetch size that we use in the tests to detect any errors caused
by strange fetch sizes.

Original commit: elastic/x-pack-elasticsearch@2c41fb5309
2017-09-26 10:16:54 -04:00
Tal Levy
048418aca7 add dummy test to LicensingTribeIT for build to pass (elastic/x-pack-elasticsearch#2627)
The true purpose of this test is to introduce another test alongside
the original, so that the test suite passes even if the other test
is skipped due to the assumption it makes about `build.snapshot`.

Original commit: elastic/x-pack-elasticsearch@709d7a5dc5
2017-09-25 22:03:46 -07:00
Nik Everett
c7c79bc1c0 Add scrolling support to jdbc (elastic/x-pack-elasticsearch#2524)
* Switch `ResultSet#getFetchSize` from returning the *requested*
fetch size to returning the size of the current page of results.
For scrolling searches without parent/child this'll match the
requested fetch size but for other things it won't. The nice thing
about this is that it lets us tell users a thing to look at if
they are wondering why they are using a bunch of memory.
* Remove all the entire JDBC action and implement it on the REST
layer instead.
* Create some code that can be shared between the cli and jdbc
actions to properly handle errors and request deserialization so
there isn't as much copy and paste between the two. This helps
because it calls out explicitly the places where they are different.
  * I have not moved the CLI REST handle to shared code because
I think it'd be clearer to make that change in a followup PR.
* There is now no more need for constructs that allow iterating
all of the results in the same request. I have not removed these
because I feel that that cleanup is big enough to deserve its own
PR.

Original commit: elastic/x-pack-elasticsearch@3b12afd11c
2017-09-25 14:41:46 -04:00
Igor Motov
c31c2af872 SQL: Fix fulltext CSV spec tests (elastic/x-pack-elasticsearch#2608)
Column type autodetect of integer types is broken in JDBC CSV library when it is used in tr-TR locale. The library is using toLowerCase() calls with default locale, which causes it to convert autodetected type name "Int" to lowercase "ınt" in tr-TR locale and not recognize it as an int type afterwards.

This commit adds a temporary workaround that makes the prevents that test from failing by specifying explicit column types in all tests where integer columns are present.

Original commit: elastic/x-pack-elasticsearch@86ca2acd8c
2017-09-25 14:23:07 -04:00
Nik Everett
216058035b Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@b3f6d3fd70
2017-09-25 09:55:17 -04:00
Simon Willnauer
e7b5702f50 Adopt settings cleanups from core (elastic/x-pack-elasticsearch#2605)
Relates to elastic/elasticsearch#26739

Original commit: elastic/x-pack-elasticsearch@dd13d099de
2017-09-25 12:27:07 +02:00
Tal Levy
d9554955f2 make tribe-node-license validation assume snapshot context (elastic/x-pack-elasticsearch#2589)
Release tests were introduced that sets the `build.snapshot`
system property to `false` to mimic release builds. This invalidates
the hardcoded license signatures that were signed against the
integration test pub/priv keys. This commit modifies the
license-validation assertions to assume the test is running against
those test keys, and will be skipped/ignored when these assertions
fail (which should only occur with `build.snapshot=true`)

Original commit: elastic/x-pack-elasticsearch@871704a3af
2017-09-21 14:39:01 -07:00
Nik Everett
2c183d566e Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@53dc6d4ce7
2017-09-21 15:20:20 -04:00
Nik Everett
71a33323ff SQL: Soften some more NOCOMMITs
Original commit: elastic/x-pack-elasticsearch@910e2485df
2017-09-21 11:33:30 -04:00
Nik Everett
8a05c1b81f Move all sql integration tests into qa (elastic/x-pack-elasticsearch#2432)
Builds on elastic/x-pack-elasticsearch#2403 to move all of sql's integration testing into
qa modules with different running server configurations. The
big advantage of this is that it allows us to test the cli and
jdbc with security present.

Creating a project that depends on both cli and jdbc and the
server has some prickly jar hell issues because cli and jdbc
package their dependencies in the jar. This works around it
in a few days:
1. Include only a single copy of the JDBC dependencies with
careful gradle work.
2. Do not include the CLI on the classpath at all and instead
run it externally.

I say "run it externally" rather than "fork it" because Elasticsearch
tests aren't allowed to fork other processes. This is forbidden
by seccomp on linux and seatbelt on osx and cannot be explicitly
requested like additional security manager settings. So instead
of forking the CLI process directly the tests interact with a test
fixture that isn't bound by Elasticsearch's rules and *can* fork
it.

This forking of the CLI has a nice side effect: it forces us to
make sure that things like security and connection strings other
than `localhost:9200` work. The old test could and did work around
missing features like that. The new tests cannot so I added the
ability to set the connection string. Configuring usernames and
passwords was also not supported but I did not add support for
that, only created the failing test and marked it as `@AwaitsFix`.

Original commit: elastic/x-pack-elasticsearch@560c6815e3
2017-09-21 09:58:52 -04:00
Tanguy Leroux
70687fbef3 [Tests] Add packaging tests for SSL/TLS communication (elastic/x-pack-elasticsearch#2556)
This commit adds a packaging test that uses the certgen tool
to set up a two nodes cluster that uses encrypted communication.

relates elastic/x-pack-elasticsearch#2485

Original commit: elastic/x-pack-elasticsearch@6d2e3c5cd0
2017-09-21 10:12:07 +02:00
Nik Everett
1405773acb Fix serialization for HitExtractorProcessor
Also get more information when SQL fails in IT.

Original commit: elastic/x-pack-elasticsearch@09f6625274
2017-09-20 14:15:35 -04:00
Nik Everett
2df8b0c144 Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@dff8c22d99
2017-09-20 12:06:27 -04:00
Tanguy Leroux
0aef18333f Add packaging test for bootstrap password setup (elastic/x-pack-elasticsearch#2509)
relates elastic/x-pack-elasticsearch#2388

Original commit: elastic/x-pack-elasticsearch@cc750155d0
2017-09-19 10:07:39 +02:00
Nik Everett
52ee02da27 Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@c25c179ce6
2017-09-18 12:32:46 -04:00
Tim Brooks
b3914afd30 Reenable TribeWithSecurityIT tests (elastic/x-pack-elasticsearch#2511)
This is related to elastic/x-pack-elasticsearch#1996. These tests were disabled during the bootstrap
password work. They can now be reenabled. Additionally, I made the test
password used in tests consistent.

Original commit: elastic/x-pack-elasticsearch@5b490c8231
2017-09-15 12:50:54 -06:00
Jay Modi
57de66476c Disable TLS by default (elastic/x-pack-elasticsearch#2481)
This commit adds back the ability to disable TLS on the transport layer and also disables TLS by
default to restore the 5.x behavior. The auto generation of key/cert and bundled CA certificate
have also been removed.

Relates elastic/x-pack-elasticsearch#2463

Original commit: elastic/x-pack-elasticsearch@abc66ec67d
2017-09-14 12:18:54 -06:00
Nik Everett
858f0b2dac Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@9945382d90
2017-09-13 16:45:27 -04:00
Nik Everett
94de0d8041 Replace exception catching with fancy returns (elastic/x-pack-elasticsearch#2454)
Instead of throwing and catching an exception for invalid
indices this returns *why* they are invalid in a convenient
object form that can be thrown as an exception when the index
is required or the entire index can be ignored when listing
indices.

Original commit: elastic/x-pack-elasticsearch@f45cbce647
2017-09-11 14:27:50 -04:00
Jay Modi
aaa0510821 Run core's full cluster restart tests with x-pack (elastic/x-pack-elasticsearch#2433)
This change pulls in the o.e.u.FullClusterRestartIT class from core and runs it as part of the
x-pack full cluster restart tests.

Relates elastic/x-pack-elasticsearch#1629

Original commit: elastic/x-pack-elasticsearch@87da59485f
2017-09-08 13:33:33 -06:00
Nik Everett
3f8bf7ccc8 Integrate sql's metadata with security (elastic/x-pack-elasticsearch#2446)
This integrates SQL's metadata calls with security by creating
`SqlIndicesAction` and routing all of SQL's metadata calls through
it. Since it *does* know up from which indices it is working against
it can be an `IndicesRequest.Replaceable` and integrate with the
existing security infrastructure for filtering indices.

This request is implemented fairly similarly to the `GetIndexAction`
with the option to read from the master or from a local copy of
cluster state. Currently SQL forces it to run on the local copy
because the request doesn't properly support serialization. I'd
like to implement that in a followup.

Original commit: elastic/x-pack-elasticsearch@15f9512820
2017-09-08 10:59:47 -04:00