Commit Graph

41545 Commits

Author SHA1 Message Date
markharwood 2a413abb0b
Docs - remove experimental marker from significant_text aggregation (#34598) 2018-10-19 09:32:02 +01:00
Daniel Mitterdorfer dbb6fe58fa
Remove hand-coded XContent duplicate checks
With this commit we cleanup hand-coded duplicate checks in XContent
parsing. They were necessary previously but since we reconfigured the
underlying parser in #22073 and #22225, these checks are obsolete and
were also ineffective unless an undocumented system property has been
set. As we also remove this escape hatch, we can remove the additional
checks as well.

Closes #22253
Relates #34588
2018-10-19 10:13:13 +02:00
Alexander Reelsen e498b7d437
Core: Parse floats in epoch millis parser (#34504)
In order to stay BWC compatible with joda time, the epoch millis date
formatter needs to parse dates with a dot like `123.45`. This
adds this functionality for the epoch millis parser in the same way as
for the epoch seconds parser. It also adds support for scientific
notations like `1.0e3` and fixes parsing of negative values for epoch
seconds and epoch millis.
2018-10-19 10:02:45 +02:00
Christoph Büscher 4f7895800e
Remove unused methods in ValueType (#34624)
The removed methods seem unused in the rest of the project.
2018-10-19 09:50:45 +02:00
Tim Vernum 670ccfb853
Handle missing user in user privilege APIs (#34575)
For user/_has_privileges and user/_privileges, handle the case where
there is no user in the security context. This is likely to indicate
that the server is running with a basic license, in which case the
action will be rejected with a non-compliance exception (provided
we don't throw a NPE).

The implementation here is based on the _authenticate API.

Resolves: #34567
2018-10-19 17:54:01 +11:00
Martijn van Groningen 56d4f69718
Renamed remaining leader_cluster_alias / cluster_alias to leader_cluster 2018-10-19 07:59:56 +02:00
Martijn van Groningen 44b461aff2
[CCR] Make leader cluster a required argument. (#34580)
This change makes it no longer possible to follow / auto follow without
specifying a leader cluster. If a local index needs to be followed
then `cluster.remote.*.seeds` should point to nodes in the local cluster.

Closes #34258
2018-10-19 07:41:46 +02:00
Yogesh Gaikwad 39a6163316
[HLRC] Add support for Delete role mapping API (#34531)
Building on expression dsl and create role mapping API, this
commit adds support for delete role mapping API to high level client.
2018-10-19 13:46:26 +11:00
Tim Vernum 2283a337b9 [TEST] Fix bit indexing in test mutator
There are 2 hard problems in computer science: cache invalidation,
naming things, and off-by-1 errors.
 --  Leon Bambrick (@secretGeek)

Fixes: #34610
2018-10-19 11:40:53 +11:00
Christoph Büscher 7cc3ac0fc4
[Docs] Add _cache/clear parameter examples (#34614)
Add example for selectively clearing just the request, query or fielddata cache
and for selectively clearing the cache for specific fields.

Closes #34287
2018-10-19 00:44:37 +02:00
Zachary Tong 45546e71c2
Add GetRollupCaps API to high level rest client (#32880)
Adds GetRollupCaps API to the HLRC, and tweaks some of the
Caps objects to be immutable.  Also various style tweaks
2018-10-18 17:12:38 -04:00
Benjamin Trent fa973007b8
[DOCS] Adds new lazy ml node setting (#34600)
* Adding new xpack.ml.max_lazy_ml_nodes setting to docs

* Fixing docs, making it clearer what the setting does

* Adding note about external process need
2018-10-18 16:11:36 -05:00
Nik Everett 2c0f67318a
Docs: Reenable rollup docs tests (#34564)
We'd disabled them because we didn't have a way to clean up after each
test. I implemented #34342 which adds the clean ups so now we can
re-enable the tests.

In the `setup` sections we have to use `raw` requests instead of
`x-pack` requests because we don't have the json config for x-pack.

Closes #33319
2018-10-18 15:24:02 -04:00
Zachary Tong ca51fb6873
[Rollup] Add support for date histo `format` (#34537)
Adds support for query-time formatting of the date histo keys
when executing a rollup search.

Closes #34391
2018-10-18 12:12:17 -04:00
Ioannis Kakavas fe41b2cc0b
Change wording for FIPS 140-2 (#34471)
Changes wording in the FIPS 140-2 related documentation. 

Co-authored-by: derickson <dave@elastic.co>
2018-10-18 18:22:46 +03:00
Benjamin Trent 6ccaa83d2a
Fixing line lengths in murmur3 and hdfs plugins (#34603) 2018-10-18 09:57:20 -05:00
Igor Motov f11a23be2f Fix line longer than 140 characters in Murmur3FieldMapper 2018-10-18 10:49:23 -04:00
Nik Everett 6c07d105f3
Amazon: Wrap at 140 columns (#34495)
Applies our standard column wrapping to the `discovery-ec2` and
`repository-s3` plugins.
2018-10-18 09:09:09 -04:00
David Roberts 0f8d05f2d0
[TEST] Reduce forecast disk space requirement for tests (#34552)
The setting that reduces the disk space requirement
for the forecasting integration tests was accidentally
removed in #31757 when files were moved around.  This
change simply adds back the setting that existed before
that.
2018-10-18 12:43:41 +01:00
Andrei Stefan 60b8118d3a
A constant can be used outside aggregation only queries (#34576)
A constant can now be used outside aggregation only queries.
Don't skip an ES query in case of constants-only selects.
Loosen the binary pipe restriction of being used only in aggregation queries.
Fixes https://github.com/elastic/elasticsearch/issues/31863
2018-10-18 13:33:01 +03:00
Martijn van Groningen 0d62f6102c
[CCR] Split cluster alias from leader index field into its own field in follow APIs (#34366) 2018-10-18 12:11:48 +02:00
Alexander Reelsen 4e36251835
Watcher: Validate email adresses when storing a watch (#34042)
Right now, watches fail on runtime, when invalid email addresses are
used.

All those fields can be checked on parsing, if no mustache is used in
any email address template. In that case we can return immediate
feedback, that invalid email addresses should not be specified when
trying to store a watch.
2018-10-18 11:54:50 +02:00
Christoph Büscher 7bcf496315
[Tests] Correct map lookup in ReplicationTrackerTests (#34565) 2018-10-18 11:23:53 +02:00
Albert Zaharovits df3726dc9b
Rename and deprecated `prefix` logfile audit setting names (#34475)
The logfile audit log format is no longer formed by prefix fields followed
by key value fields, it is all formed by key value fields only (JSON format).

Consequently, the following settings, which toggled some of the prefix
fields, have been renamed:
audit.logfile .prefix.emit_node_host_address
audit.logfile .prefix.emit_node_host_name
audit.logfile .prefix.emit_node_name
2018-10-18 12:06:47 +03:00
Ioannis Kakavas 6874ee8938 Remove deprecated setting from test
Replace xpack.security.transport.ssl.truststore.password deprecated
setting with xpack.security.transport.ssl.truststore.secure_password
in the HLRC integTestCluster.
2018-10-18 10:17:25 +03:00
Vladimir Dolzhenko 3621277969 eliminate NPE on closing restHighLevelClient (if test suite skips all test cases)
(cherry picked from 277704c5972148ee62900e781ac64b061aa4b767)
2018-10-18 09:07:10 +02:00
Tim Vernum e41c5cb47a Fix compilation issue due to 47e9082 and 9200e15
47e9082 removed a method that 9200e15 made use of.
2018-10-18 18:03:54 +11:00
Tim Vernum 47e9082bb4
Replace Streamable w/ Writeable in RoleDescriptor (#34544)
This commit replaces usage of Streamable with Writeable within the
RoleDescriptor class (and inner classes).

Relates: #34389
2018-10-18 17:28:43 +11:00
Tim Vernum 9200e15b74
Add get-user-privileges API (#33928)
This API is intended as a companion to the _has_privileges API.
It returns the list of privileges that are held by the current user.

This information is difficult to reason about, and consumers should
avoid making direct security decisions based solely on this data.
For example, each of the following index privileges (as well as many
more) would grant a user access to index a new document into the
"metrics-2018-08-30" index, but clients should not try and deduce
that information from this API.
- "all" on "*"
- "all" on "metrics-*"
- "write" on "metrics-2018-*"
- "write" on "metrics-2018-08-30"

Rather, if a client wished to know if a user had "index" access to
_any_ index, it would be possible to use this API to determine whether
the user has any index privileges, and on which index patterns, and
then feed those index patterns into _has_privileges in order to
determine whether the "index" privilege had been granted.

The result JSON is modelled on the Role API, with a few small changes
to reflect how privileges are modelled when multiple roles are merged
together (multiple DLS queries, multiple FLS grants, multiple global
conditions, etc).
2018-10-18 14:09:04 +11:00
Ryan Ernst d445785f1a
Scripting: Convert domainSplit function for ML to whitelist (#34426)
This commit moves the definition of domainSplit into java and exposes it
as a painless whitelist extension. The method also no longer needs
params, and version which ignores params is added and deprecated.
2018-10-17 15:54:21 -07:00
Ryan Ernst 8734540345
Ensure map keys cannot be self referencing (#34569)
This commit improves self reference checking to map keys, as well as
adds it to ingest script processing.
2018-10-17 15:16:13 -07:00
Jason Tedor 9be87adb95
Increment settings version when upgrading index (#34566)
When we upgrade an index, we set the settings version upgraded
setting. This should be considered a settings change, and therefore we
need to increment the settings version. This commit addresses that.
2018-10-17 18:00:17 -04:00
Nhat Nguyen eb36f10394
TEST: Capture replication targets when replication group ready (#34407)
Today, WriteReplicationAction uses a set of replication targets directly
from the primary shard of ReplicationGroup. It should be fine except
when we add/remove or promote a shard while a write action is executing.
We have encountered these two issues:

1. Replicas are not found in the replication targets. This happens
because we remove replicas but the WriteReplicationAction still uses the
old replication targets which include the removed replicas.

2. Access ReplicationGroup from a primary shard which hasn't activated
the primary-mode yet. This is because we won't activate the primary-mode
for a promoting shard after bumping the primary term which is executed
asynchronously.

This commit captures the replication targets when the replication group
is ready and continue using those targets until we re-compute the new
targets after the group is changed.

Closes #33457
2018-10-17 17:37:52 -04:00
Nik Everett a45626deb5
Analysis: Wrap at 140 columns (#34494)
Applies our standard column width to all analysis plugins.
2018-10-17 16:17:25 -04:00
Nik Everett 1452d55155
Expressions: Wrap at 140 columns (#34493)
Apply our standard line length to all of the java files in the
`lang-expression` plugin.
2018-10-17 16:12:21 -04:00
Nik Everett 5d74f8085b
SMB: Wrap at 140 columns (#34492)
Applies our max column width to the store-smb project.
2018-10-17 16:04:41 -04:00
Nik Everett b6aa42777a
Search: Wrap lucene classes at 140 columns (#34491)
Applies our line length guidance for all classes in the server in `lucene`
directories *except* `XMoreLikeThis`. The only long line in
`XMoreLikeThis` says "remove this when we upgrade to Lucene 5. Given
that we're on Lucene 8, this is a little terrifying and deserves another
look.
2018-10-17 15:54:35 -04:00
Armin Braun 08d4bf6e84
TESTS: Remove Dead Code in Test Infra. (#34548)
* None of this infrastructure is used
* Some redundant throws and resulting catch code removed
2018-10-17 20:08:39 +01:00
Simon Willnauer b0e98cbce2
Pass the host name on as `server_name` if proxy mode is on (#34559)
In remote cluster setup if we see a configured proxy we should set
the seed nodes host name as the `server_name` to trigger SNI based
routing even for seed nodes. Since remote cluster connections are
plain TCP connections we have to set the host manually since the other
side can't take it from the request URL like in the HTTP case.
This also adds some more informative logging to remote cluster connection.
2018-10-17 19:11:50 +02:00
jaymode 46c7b5ee6e
Revert "Security: don't call prepare index for reads (#34246)"
This reverts commit 0b4e8db1d3 as some
issues have been identified with the changed handling of a primary
shard of the security index not being available.
2018-10-17 10:37:40 -06:00
Andy Bristol 18aa1c1381
add start trial API to HLRC (#33406)
Introduces client-specific request and response classes that do not
depend on the server

The `type` parameter is named `licenseType` in the response class to be
more descriptive. The parts that make up the acknowledged-required
response are given slightly different names than their server-response
types to be consistent with the naming in the put license API

Tests do not cover all cases because the integ test cluster starts up
with a trial license - this will be addressed in a future commit
2018-10-17 08:02:04 -07:00
Nik Everett 139bbc3f03
Rollup: Consolidate rollup cleanup for http tests (#34342)
This moves the rollup cleanup code for http tests from the high level rest
client into the test framework and then entirely removes the rollup cleanup
code for http tests that lived in x-pack. This is nice because it
consolidates the cleanup into one spot, automatically invokes the cleanup
without the test having to know that it is "about rollup", and should allow
us to run the rollup docs tests.

Part of #34530
2018-10-17 09:32:16 -04:00
Benjamin Trent fb579d2d9a
ML: Adding support for lazy nodes (#29991) (#34538) 2018-10-17 08:30:15 -05:00
Nik Everett 2cccd4c669
LLREST: Wrap at 140 columns (#34489)
Wraps a single line in a test to fit on 140 characters, removing the
only checkstyle suppression for the clients directory.
2018-10-17 09:17:02 -04:00
Nik Everett 030e8c8fe3 Docs: Tweak upgrade docs
Tweak the upgrade instructions for moving from pre-6.3-with-x-pack to
post-6.3-default distribution. Specifically, you have to remove the
x-pack plugin before upgrading because 6.4 doesn't understand how to
remove it.

Relates to #34307
2018-10-17 08:17:12 -04:00
Tim Vernum e0a1803638
Add Debug/Trace logging to token service (#34022)
The token service has fairly strict validation and there are a range
of reasons why request may be rejected.
The detail is typically returned in the client exception / json body
but the ES admin can only debug that if they have access to detailed
logs from the client.

This commit adds debug & trace logging to the token service so that it
is possible to perform this debugging from the server side if
necessary.
2018-10-17 20:55:18 +11:00
Shaunak Kashyap e0cab14c6e
Cleanup: removing unused class (#34541)
* Cleanup: removing unused field in other similar classes

* Removing unused class
2018-10-17 10:15:12 +01:00
Armin Braun 3954d041a0
SCRIPTING: Move sort Context to its Own Class (#33717)
* SCRIPTING: Move sort Context to its own Class
2018-10-17 10:02:44 +01:00
Alpar Torok 2e5e4e1a69
Switch to parametric CI jobs (#34179)
Switch to parametric CI jobs. Given the changes in CI this also switches to running Gradle with the build java version only ( we used to also run it with run-time java version )
2018-10-17 07:57:30 +03:00
Armin Braun b3be96aeb9
MINOR: Remove Deadcode in X-Pack Tests (#34511) 2018-10-16 22:48:28 +01:00