Commit Graph

599 Commits

Author SHA1 Message Date
Yogesh Gaikwad 8114646e12
[Kerberos] Add debug log statement for exceptions (#32663)
This commit adds missing debug log statements for exceptions
that occur during ticket validation. I thought these
get logged somewhere else in authentication chain
but even after enabling trace logs I could not see them
logged. As the Kerberos exception messages are cryptic
adding full stack trace would help debugging faster.
2018-08-12 00:49:08 +10:00
Tim Brooks 38ec0ff6ca
Make x-pack core pull transport-nio (#32757)
The security nio transports depend on transport-nio. This commit
modifies x-pack core to include the transport-nio jar into the x-pack
core module.
2018-08-10 17:36:29 -06:00
Benjamin Trent b08416b899
Clear Job#finished_time when it is opened (#32605) (#32755)
* Clear Job#finished_time when it is opened (#32605)

* not returning failure when Job#finished_time is not reset

* Changing error log string and source string
2018-08-10 13:52:00 -05:00
Nik Everett f5ba801c6b Test: Only sniff host metadata for node_selectors (#32750)
Our rest testing framework has support for sniffing the host metadata on
startup and, before this change, it'd sniff that metadata before running
the first test. This prevents running these tests against
elasticsearch installations that won't support sniffing like Elastic
Cloud. This change allows tests to only sniff for metadata when they
encounter a test with a `node_selector`. These selectors are the things
that need the metadata anyway and they are super rare. Tests that use
these won't be able to run against installations that don't support
sniffing but we can just skip them. In the case of Elastic Cloud, these
tests were never going to work against Elastic Cloud anyway.
2018-08-10 13:35:47 -04:00
Albert Zaharovits 1dcf80795a
[TEST] Certificate NONE not allowed in FIPS JVM (#32753)
Certificate NONE not allowed when running in a FIPS JVM
2018-08-10 19:37:05 +03:00
Dimitris Athanasiou c7b1ba33aa
[ML] Refactor ProcessCtrl into Autodetect and Normalizer builders (#32720)
This moves the helper functionality for creating the autodetect
and mormalizer processes into corresponding builders.
2018-08-10 17:28:20 +01:00
Alexander Reelsen e49717393a
Tests: Disable rolling upgrade tests with system key on fips JVM (#32775)
This disables the x-pack rolling upgrade tests using a fips JVM, as
there are problems creating the keystore.

Relates #32737
2018-08-10 16:50:11 +02:00
Benjamin Trent 94a9b253db
Adding ML HLRC wrapper and put_job API call (#32726)
* Adding ML HLRC wrapper and put_job API call

* Changing integration test job to have consistent stucture
2018-08-10 07:16:55 -05:00
David Roberts ae0c303dad
Move icu4j and super-csv version numbers to versions file (#32769)
The upcoming ML log structure finder functionality will use these
libraries, and it makes sense to use the same versions that are
being used elsewhere in Elasticsearch.  This is especially true
with icu4j, which is pretty big.
2018-08-10 12:19:06 +01:00
Alexander Reelsen 3fa3af1d11 Tests: Mute test in KerberosRealmTests
Relates #32701
2018-08-10 10:40:37 +02:00
Tim Vernum 93d4f84b21
[TEST] Add "ne" as an unsupported SimpleKdc locale (#32700)
SimpleKdcServer cannot handle some locales, so we explicitly avoid
using them in tests.
Adding "ne" (Nepali) to that list.
2018-08-10 13:36:36 +10:00
Yogesh Gaikwad 0161bbc77f
[Test] Handle file permissions for Windows (#32681)
This commit modifies the test to handle file permission
tests in windows/dos environments. The test requires access
to UserPrincipal and so have modified the plugin-security policy
to access user information.

Closes#32637
2018-08-10 09:55:43 +10:00
Alpar Torok 5bbed5ed9a Merge branch 'master' of github.com:elastic/elasticsearch 2018-08-09 21:58:03 +03:00
Alpar Torok c5e66cb345 mute test #32737 2018-08-09 21:54:58 +03:00
Andrei Stefan 7b618f3db4
SQL: Bug fix for the optional "start" parameter usage inside LOCATE function (#32576)
The incorrect NodeInfo is created when the optional parameter is not used, leading to the incorrect constructor being used. Simplified LocateFunctionProcessorDefinition by using one constructor instead of two.
Fixes https://github.com/elastic/elasticsearch/issues/32554
2018-08-09 19:30:02 +03:00
Andrei Stefan 6750e15773
SQL: Ignore H2 comparative tests for uppercasing/lowercasing string functions (#32604)
Skip the comparative tests using lowercasing/uppercasing against H2 (which considers the Locale).
ES-SQL is, so far, ignoring the Locale.
Still, the same queries are executed against ES-SQL alone and results asserted to be correct.
2018-08-09 19:28:48 +03:00
Nicholas Knize e162127ff3 Upgrade to Lucene-7.5.0-snapshot-13b9e28f9d
The main feature is the inclusion of bkd backed geo_shape with
INTERSECT, DISJOINT, WITHIN bounding box and polygon query support.
2018-08-09 11:15:02 -05:00
Lee Hinman 7af28c48c3
Switch WritePipelineResponse to AcknowledgedResponse (#32722)
We previously discussed moving the classes extending `AcknowledgedResponse` to
simply use `AcknowledgedResponse`, making the class non-abstract.

This moves the first class to do this, removing `WritePipelineResponse` in the
process.

If we like the way this looks, I will switch the remaining classes over to using
`AcknowledgedResponse`.
2018-08-08 16:21:58 -06:00
Benjamin Trent d586e4cfd3
Adding `Job` and `AnalysisConfig` for HLRC (#32687)
* Adding `Job` and `AnalysisConfig` for HLRC

* Removing println used for local debugging

* Adding null checks and removing unnecessary field
2018-08-08 11:54:11 -05:00
Jim Ferenczi f1869cca35
Fix role query that can match nested documents (#32705)
This change makes sure that the role query excludes nested documents
when it is ran to select the parent documents allowed by the role.
2018-08-08 16:36:51 +02:00
Armin Braun 580d59e2d7
CORE: Upgrade to Jackson 2.8.11 (#32670)
* closes #30352
2018-08-08 12:04:25 +02:00
Tim Vernum 4cbcc1d659 [TEST] Add empty test case for FIPS
All other tests in security:cli are skipped on FIPS JVMs
2018-08-08 13:47:18 +10:00
Tanguy Leroux 2e65bac5dd
[Rollup] Remove builders from RollupJobConfig (#32669) 2018-08-07 18:54:42 +02:00
Benjamin Trent 6d50d8b5a9
Adding job process pojos to protocol pkg (#32657)
* Adding job process pojos to protocol pkg

* Removing unused `RESULTS_FIELD`

* Addressing PR comments, removing unnecessary methods
2018-08-07 10:51:52 -05:00
Albert Zaharovits 1f50950099 Add @AwaitsFix for #32673 2018-08-07 13:22:12 +03:00
Tanguy Leroux 1122314b3b
[Rollup] Remove builders from GroupConfig (#32614) 2018-08-07 09:39:24 +02:00
Benjamin Trent b2a0f38a0c
Adding xpack.core.ml.datafeed to protocol.xpack.ml.datafeed (#32625)
* Adding org.elasticsearch.xpack.core.ml.datafeed to org.elasticsearch.protocol.xpack.ml.datafeed

* removing unused ParseField and import

* Addressing PR feed back and fixing tests

* Simplifying Datafeed(Config|Update) ctor parser
2018-08-06 15:33:18 -05:00
Igor Motov e641fccfe3
Rest HL client: Add get license action (#32438)
Rest HL client: Add get license action

Continues to use String instead of a more complex License class to
hold the license text similarly to put license.

Relates #29827
2018-08-06 07:15:40 -07:00
Yogesh Gaikwad 615aa85f4e
[Kerberos] Use canonical host name (#32588)
The Apache Http components support for Spnego scheme
uses canonical name by default.
Also when resolving host name, on centos by default
there are other aliases so adding them to the
DelegationPermission.

Closes#32498
2018-08-06 23:51:43 +10:00
Armin Braun 0a67cb4133
LOGGING: Upgrade to Log4J 2.11.1 (#32616)
* LOGGING: Upgrade to Log4J 2.11.1
* Upgrade to `2.11.1` to fix memory leaks in slow logger when logging large requests
   * This was caused by a bug in Log4J https://issues.apache.org/jira/browse/LOG4J2-2269 and is fixed in `2.11.1` via https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=9496c0c
* Fixes #32537
* Fixes #27300
2018-08-06 14:56:21 +02:00
Ioannis Kakavas 66edba2012 [TEST] Allow to run in FIPS JVM (#32607)
* Change SecurityNioHttpServerTransportTests to use PEM key and
certificate files instead of a JKS keystore so that this tests
can also run in a FIPS 140 JVM
* Do not attempt to run cases with ssl.verification_mode NONE in
SessionFactoryTests so that the tests can run in a FIPS 140 JVM
2018-08-06 07:42:26 +03:00
Ioannis Kakavas ceb1ae4d7b [Test] Add ckb to the list of unsupported languages (#32611) 2018-08-06 10:00:45 +10:00
David Roberts b99aa81fe4
[ML] Add ML result classes to protocol library (#32587)
This commit adds the ML results classes to the X-Pack protocol
library used by the high level REST client.

(Other commits will add the config classes and stats classes.)

These classes:

- Are publically immutable
- Are privately mutable - this is perhaps not as nice as the
  config classes, but to do otherwise would require adding
  builders and the corresponding server-side classes that the
  old transport client used don't have builders
- Have little/no validation of field values beyond null checks
- Are convertible to and from X-Content, but NOT wire transportable
- Have lenient parsers to maximize compatibility across versions
- Have the same class names and getter names as the corresponding
  classes in X-Pack core to ease migration for transport client
  users
- Don't reproduce all the methods that do calculations or
  transformations that the the corresponding classes in X-Pack core
  have
2018-08-03 20:48:38 +01:00
Zachary Tong 992ec4be5d [Rollup] Update wire version check after backport
Bumping down the version to 6.4 since the backport is complete.  Also
adds some missing version checks to the bwc tests to make sure it
only runs on the correct versions
2018-08-03 14:09:01 -04:00
Zachary Tong fc9fb64ad5
[Rollup] Improve ID scheme for rollup documents (#32558)
Previously, we were using a simple CRC32 for the IDs of rollup documents.
This is a very poor choice however, since 32bit IDs leads to collisions
between documents very quickly.

This commit moves Rollups over to a 128bit ID.  The ID is a concatenation
of all the keys in the document (similar to the rolling CRC before),
hashed with 128bit Murmur3, then base64 encoded.  Finally, the job
ID and a delimiter (`$`) are prepended to the ID.

This gurantees that there are 128bits per-job.  128bits should
essentially remove all chances of collisions, and the prepended
job ID means that _if_ there is a collision, it stays "within"
the job.

BWC notes:

We can only upgrade the ID scheme after we know there has been a good
checkpoint during indexing.  We don't rely on a STARTED/STOPPED
status since we can't guarantee that resulted from a real checkpoint,
or other state.  So we only upgrade the ID after we have reached
a checkpoint state during an active index run, and only after the
checkpoint has been confirmed.

Once a job has been upgraded and checkpointed, the version increments
and the new ID is used in the future.  All new jobs use the
new ID from the start
2018-08-03 11:13:25 -04:00
David Roberts bc274b2ff2
[ML] Add some ML config classes to protocol library (#32502)
This commit adds four ML config classes to the X-Pack protocol
library used by the high level REST client.

(Other commits will add the remaining config classes, plus results
and stats classes.)

These classes:

- Are immutable
- Have little/no validation of field values beyond null checks
- Are convertible to and from X-Content, but NOT wire transportable
- Have lenient parsers to maximize compatibility across versions
- Have the same class names, member names and getter/setter names
  as the corresponding classes in X-Pack core to ease migration
  for transport client users
- Don't reproduce all the methods that do calculations or
  transformations that the the corresponding classes in X-Pack core
  have
2018-08-03 13:21:08 +01:00
Ioannis Kakavas 1ee6393117
[TEST]Split transport verification mode none tests (#32488)
This commit splits SecurityNetty4TransportTests in two methods
one handling verification mode certificate and full and one
handling verification mode none. This is done so that the second
method can be muted in a FIPS 140 JVM where verification mode none
cannot be used.
2018-08-03 14:44:40 +03:00
Tanguy Leroux 21f660d801
[Rollup] Remove builders from DateHistogramGroupConfig (#32555)
Same motivation as #32507 but for the DateHistogramGroupConfig
configuration object. This pull request also changes the format of the
time zone from a Joda's DateTimeZone to a simple String.

It should help to port the API to the high level rest client and allows
clients to not be forced to use the Joda Time library. Serialization is
impacted but does not need a backward compatibility layer as
DateTimeZone are serialized as String anyway. XContent also expects
a String for timezone, so I found it easier to move everything to String.

Related to #29827
2018-08-03 13:11:00 +02:00
David Roberts eb17128b9c
[ML] Add Detector config classes to protocol library (#32495)
This commit adds the Detector class and its dependencies to the
X-Pack protocol library used by the high level REST client.

(Future commits will add the remaining config classes, plus results
and stats classes.)

These classes:

- Are immutable, with builders, but the builders do no validation
  beyond null checks
- Are convertible to and from X-Content, but NOT wire transportable
- Have lenient parsers to maximize compatibility across versions
- Have the same class names, member names and getter/setter names
  as the corresponding classes in X-Pack core to ease migration
  for transport client users
- Don't reproduce all the methods that do calculations or
  transformations that the the corresponding classes in X-Pack core
  have
2018-08-03 10:39:29 +01:00
Tanguy Leroux 937dcfd716
[Rollup] Remove builders from MetricConfig (#32536)
Related to #29827
2018-08-03 10:01:20 +02:00
Alexander Reelsen f809d6fff4
Tests: Add rolling upgrade tests for watcher (#32428)
These tests ensure, that the basic watch APIs are tested in the rolling
upgrade tests. After initially adding a watch, the tests try to get,
execute, deactivate and activate a watch. Watcher stats are tested as
well, and an own java based test has been added for restarting, as that
requires waiting for a state change. Watcher history is also checked.

Closes #31216
2018-08-03 09:41:29 +02:00
Shaunak Kashyap 0a83968650
Add cluster UUID to Cluster Stats API response (#32206)
* Make cluster stats response contain cluster UUID

* Updating constructor usage in Monitoring tests

* Adding cluster_uuid field to Cluster Stats API reference doc

* Adding rest api spec test for expecting cluster_uuid in cluster stats response

* Adding missing newline

* Indenting do section properly

* Missed a spot!

* Fixing the test cluster ID
2018-08-02 17:14:19 -07:00
Jay Modi eb3accb721 Security: move User to protocol project (#32367)
The User class has been moved to the protocol project for upcoming work
to add more security APIs to the high level rest client. As part of
this change, the toString method no longer uses a custom output method
from MetadataUtils and instead just relies on Java's toString
implementation.
2018-08-03 00:21:34 +03:00
Andrei Stefan 9e1e38ff51
Minor fix for javadoc (applicable for java 11). (#32573) 2018-08-02 23:46:08 +03:00
Tanguy Leroux 08e4f4be42
[Rollup] Remove builders from HistoGroupConfig (#32533)
Related to #29827
2018-08-02 17:55:00 +02:00
Colin Goodheart-Smithe 4cdbb42542
Mutes failing SQL string function tests due to #32589 2018-08-02 16:15:59 +01:00
Yannick Welsch db6e8c736d
Remove cluster state initial customs (#32501)
This infrastructure was introduced in #26144 and made obsolete in #30743
2018-08-02 15:49:59 +02:00
Dimitris Athanasiou f30bb0ebf8
[ML] Remove multiple_bucket_spans (#32496)
This commit removes the never released multiple_bucket_spans
configuration parameter. This is now replaced with the new
multibucket feature that requires no configuration.
2018-08-02 11:25:56 +01:00
David Kyle 15679315e3
[ML] Rename JobProvider to JobResultsProvider (#32551) 2018-08-02 09:53:47 +01:00
Dimitris Athanasiou 8bf83647f5
[ML] Improve error when no available field exists for rule scope (#32550)
Closes #32542
2018-08-01 18:58:20 +01:00