Commit Graph

8008 Commits

Author SHA1 Message Date
Costin Leau 03ecbc0ccb SQL: implement COT, RANDOM, SIGN math functions (elastic/x-pack-elasticsearch#4394)
Add a number of missing math functions

Original commit: elastic/x-pack-elasticsearch@a26d9d2454
2018-04-20 15:07:14 +03:00
David Kyle 043a877a31 [ML] Prevent unnecessary job updates. (elastic/x-pack-elasticsearch#4424)
Original commit: elastic/x-pack-elasticsearch@8e0629789e
2018-04-20 12:42:31 +01:00
Ioannis Kakavas 14ad96ea8b Fix link to certificate settings in PKI realm (elastic/x-pack-elasticsearch#4402)
Remove indentation so that the two lines are not interpreted as
code block and the link is rendered correctly

Original commit: elastic/x-pack-elasticsearch@a73d52dfba
2018-04-20 12:43:55 +03:00
Costin Leau bc2ef139a8 SQL: Add Atan2 and Power functions (elastic/x-pack-elasticsearch#4412)
Add missing Atan2 & Power(and introduce BinaryMath operations), similar
to MathOperation.
Also align arithmetic package with binary math for code reuse.

Original commit: elastic/x-pack-elasticsearch@311961815e
2018-04-20 12:23:42 +03:00
Costin Leau e6f69ee269 SQL: Fix version loading in JDBC (elastic/x-pack-elasticsearch#4425)
When deployed, JDBC version needs to look at jars inside the classpath
not on the file system. Failing to do that, causes the version
information to be missing

Original commit: elastic/x-pack-elasticsearch@d35e8abb29
2018-04-20 12:19:45 +03:00
Ryan Ernst 935df40f70 Docs: Fix reference to legacy tribe section
Original commit: elastic/x-pack-elasticsearch@5261193fc1
2018-04-19 18:40:32 -07:00
Ryan Ernst ba9c13302a Remove remaining references to tribe node (elastic/x-pack-elasticsearch#4407)
This commit review remaining docs and build references from the docs.

Original commit: elastic/x-pack-elasticsearch@33830b63ed
2018-04-19 18:02:09 -07:00
Zachary Tong 2d6d3c3d74 [Rollup] Add scaled_float to allowed metric mapper types (elastic/x-pack-elasticsearch#4423)
We grab all the NumberFieldMappers to determine the whitelist of allowed
number types, but `scaled_float` is in a module so it isn't picked up
automatically.  This commit adds `scaled_float` to the whitelist
manually.

Original commit: elastic/x-pack-elasticsearch@fb35440315
2018-04-19 13:54:04 -07:00
Jay Modi 3bc6fd76a3 Allow license installation in dev mode without TLS (elastic/x-pack-elasticsearch#4155)
This commit allows license installation without TLS being enabled when
the cluster is in dev mode. The main difference this change enables is
the ability to install a production license on a single node cluster
that is bound to localhost and does not have the single-node discovery
enabled.

relates elastic/x-pack-elasticsearch#4123

Original commit: elastic/x-pack-elasticsearch@04ebcc0fab
2018-04-19 13:49:27 -06:00
Nik Everett f0be979541 SQL: Mimick Elasticsearch defaults for sql action (elastic/x-pack-elasticsearch#4398)
The SQL action supports several text outputs and used to default to an
output that looked like the SQL CLI. It is a lovely output format but
this changes output selection behavior to mimick Elasticsearch's
standard behavior: it'll now default to the same format as the request.

That means that if you want the pretty text format then you need to ask
for it. The way to do that is:

```
POST /_xpack/sql?format=text/plain
{
    "query": "SELECT * FROM library ORDER BY page_count DESC LIMIT 5"
}
```

Original commit: elastic/x-pack-elasticsearch@4a15a23b18
2018-04-19 15:40:19 -04:00
Lisa Cawley d835ad42f1 [DOCS] Remove X-Pakc installation page (elastic/x-pack-elasticsearch#4416)
Original commit: elastic/x-pack-elasticsearch@a5ca5c7230
2018-04-19 11:40:26 -07:00
Lee Hinman 0bd9163c28 Move sql-proto and jdbc to java.util.TimeZone (elastic/x-pack-elasticsearch#4350)
* Move sql-proto and jdbc to java.util.TimeZone

This moves sql-proto and jdbc from Joda's `DateTimeZone` to
`java.util.TimeZone`, this will allow us in the future to be able to decouple
JDBC from Joda.

This does not decouple all of SQL from joda, it focuses on as small a piece as I
could for sql-proto and jdbc.

Requires https://github.com/elastic/elasticsearch/pull/29487 to be merged first.

Original commit: elastic/x-pack-elasticsearch@7c9d52e675
2018-04-19 11:27:33 -06:00
Ryan Ernst 2574d4e704 Build: Change bwc java home check to new signature (elastic/x-pack-elasticsearch#4392)
The signature for getJavaHome is changing with
elastic/elasticsearch#29548.


Original commit: elastic/x-pack-elasticsearch@85434d0a13
2018-04-19 09:52:04 -07:00
Lisa Cawley 2578346877 [DOCS] Fixes monitor privilege description (elastic/x-pack-elasticsearch#4403)
Original commit: elastic/x-pack-elasticsearch@579996d671
2018-04-19 09:27:40 -07:00
Jay Modi 61542a5543 Wait for Netty threads after Security tests (elastic/x-pack-elasticsearch#4390)
This commit adds waits for two Netty threads that run in the background
and require explicit calls to await inactivity in them. This shows up
as an issue in security tests since we always use netty and do not have
these waits so these threads can trip the thread leak detector.

relates elastic/x-pack-elasticsearch#4367

Original commit: elastic/x-pack-elasticsearch@76d84553ba
2018-04-19 09:54:25 -06:00
Jay Modi 502bc6c572 Clarify index operations for create privilege (elastic/x-pack-elasticsearch#4337)
The `create` privilege can be misleading based on a user's
interpretation of the meaning of the privilege. The create privilege
grants the user access to the index api and the put mapping api. The
index api allows for a document to be created but it also allows for
an existing document to be overwritten (assuming the CREATE operation
type is not used). However, the name `create` implies the ability to
only create a document and not be able to modify an existing document,
which may not be clear to users unfamiliar with the index API.

This commit adds a small note under the privilege in the documentation
that should add some clarity to the operations allowed by this
privilege.

Original commit: elastic/x-pack-elasticsearch@24596c41b0
2018-04-19 08:52:30 -06:00
Russ Cam d62b6eb3b8 [DOCS] Update forecast_id to string in response (elastic/x-pack-elasticsearch#4245)
This commit updates forecast_id to be a string within the response,
to match source code.

Original commit: elastic/x-pack-elasticsearch@b6fbc87a84
2018-04-19 15:01:17 +01:00
tomcallahan 8d6f36c507 Remove copy-pasted code (elastic/x-pack-elasticsearch#4298)
* Remove copy-pasted code

An examination of the x-pack source code revealed copy-pasted code
in ActiveDirectoryGroupsResolver in the binarySidToStringSid
method.  I have replaced this with an apache2 implementation from
the apache directory studio project.  Furthermore, I have added
a test that leverages a real binary/string SID pair retrieved
from an active directory domain controller.  The apache2-based
implementation is exempt for the license checker.

Original commit: elastic/x-pack-elasticsearch@81a7471261
2018-04-19 09:12:15 -04:00
Jason Tedor 650b80a982 Reflect renaming of bulk thread pool (elastic/x-pack-elasticsearch#4414)
The bulk thread pool was renamed to the write thread pool. This commit
adds support for this in X-Pack. Specifically, a change is needed in
monitoring to reflect the name change (and support the possibility that
the user has the display name for the write thread pool as "bulk").

Original commit: elastic/x-pack-elasticsearch@c3c4b99be5
2018-04-19 08:22:10 -04:00
Jason Tedor 4e376070a3 Remove index thread pool from monitoring filters (elastic/x-pack-elasticsearch#4409)
The index thread pool is gone, so the index thread pool stats do not any
longer need to be whitelisted in the monitoring stats.

Original commit: elastic/x-pack-elasticsearch@83f7742805
2018-04-18 21:02:06 -04:00
Ryan Ernst d0ee36b192 Update tests using http.enabled (elastic/x-pack-elasticsearch#4413)
This commit updates tests using http.enabled to expect a deprecation
warning.

relates https://github.com/elastic/elasticsearch/pull/29591

Original commit: elastic/x-pack-elasticsearch@92e5ea9c74
2018-04-18 17:36:14 -07:00
Lisa Cawley dcedffb0f0 [DOCS] Replaces InetSocketTransportAddress in code samples (elastic/x-pack-elasticsearch#4404)
Original commit: elastic/x-pack-elasticsearch@85b11cccc7
2018-04-18 09:15:43 -07:00
David Roberts fd3f079276 [ML] Exclude failed jobs from node allocation decision (elastic/x-pack-elasticsearch#4395)
When calculating the current load on each ML node during the
node allocation process we should be ignoring failed jobs.
This is because failed jobs do not have a corresponding native
process, so do not consume memory or CPU resources.

relates elastic/x-pack-elasticsearch#4381

Original commit: elastic/x-pack-elasticsearch@1cb0ca973e
2018-04-18 10:10:55 +01:00
Lisa Cawley 70f9bcc0d1 [DOCS] Adds link to clear role cache API (elastic/x-pack-elasticsearch#4405)
Original commit: elastic/x-pack-elasticsearch@d2d74069bf
2018-04-17 18:03:46 -07:00
Lisa Cawley 4c38e610fd [DOCS] Adds enabled parameter to user management API (elastic/x-pack-elasticsearch#4364)
Original commit: elastic/x-pack-elasticsearch@166bef4702
2018-04-17 16:15:54 -07:00
Alexander Reelsen 4c00361b01 Watcher: Remove broken tests (elastic/x-pack-elasticsearch#4399)
These tests test actually watcher 6.x features and are not needed in the
master branch anymore, as there is no way that watcher is running on the
master node only in Elaticsearch 7.x.

Original commit: elastic/x-pack-elasticsearch@ac5b933994
2018-04-17 23:12:49 +02:00
Zachary Tong 97fa56443d [Docs] Document limitations around rolling up heterogeneous schemas
Rolling up indices that contain multiple schemas requires extra
considerations for the user, and in some cases, some limitations.  This
commit tries to enumerate the issues so the user is aware.

Original commit: elastic/x-pack-elasticsearch@bf96eeab4e
2018-04-17 19:45:22 +00:00
Lisa Cawley ed2c6ead7f [DOCS] Move monitoring security and configuration information to Elasticsearch Reference (elastic/x-pack-elasticsearch#4370)
Original commit: elastic/x-pack-elasticsearch@5ff35f7ec6
2018-04-17 10:27:31 -07:00
Adrien Grand c228c48e70 Fix compilation after elastic/elasticsearch#29511.
Original commit: elastic/x-pack-elasticsearch@4240815b5b
2018-04-17 17:38:13 +02:00
Chris Earle 51d87994ca [Monitoring] Thread _xpack/monitoring/_bulk (elastic/x-pack-elasticsearch#4393)
Instead of allowing the `_xpack/monitoring/_bulk` to remain on the same
thread, it should execute on a separate thread to avoid blocking the
http worker thread whenever the exporters get stuck waiting on the
monitoring cluster.

Original commit: elastic/x-pack-elasticsearch@25ce9a4df0
2018-04-17 11:16:49 -04:00
Ioannis Kakavas f1902aba39 [DOCS] Update documentation for SAML metadata signing (elastic/x-pack-elasticsearch#4356)
Updates documentation describing the parameters for optionally signing the
SAML medatata document that the saml-metadata command creates.

Original commit: elastic/x-pack-elasticsearch@0e8146541a
2018-04-17 14:23:26 +03:00
Tim Vernum eccf3899a2 Tie zip output to command line options in certutil (elastic/x-pack-elasticsearch#4354)
Previously "certutil" would generate a zip file if there were multiple
certificates.
However, this means that if the user specified "-multiple" or "-in"
then the output format will vary based on whether they entered
multiple instance names (-multiple) or whether the input file
contained multiple instance records (-in).
It is better if the output format is predictable based on the command
line options, so this change forces zip output whenever any of the
following command line options are supplied:
    -pem
    -keep-ca-key
    -multiple
    -in

Original commit: elastic/x-pack-elasticsearch@344baa5f17
2018-04-17 11:05:30 +10:00
Tim Vernum 5eac9fd1a4 [DOCS] Improve docs for "--pass" option in certutil (elastic/x-pack-elasticsearch#4352)
The handling of the "--pass" option has complex rules due to the
different requirements of different output formats, and the way
joptsimple handles specs with optional arguments.

A standard unix shell will parse
    certutil --pass=""
as being identical to
    certutil --pass=
and joptsimple treats that as being "no argument", i.e. the same as
    certutil --pass

The only way to pass a blank password is
    certutil --pass ""
(though this is shell dependent)

Original commit: elastic/x-pack-elasticsearch@bc09d0f298
2018-04-17 11:05:09 +10:00
Tim Vernum 7cd99d2f96 Trigger log4j loading prior to context classloader (elastic/x-pack-elasticsearch#4355)
OpenSAML requires that the context classloader be set to the
classloader that contains the opensaml jar files in order to
initialise itself. However, sometimesrthis seems to cause the
classloading for slf4j/log4j to take place while that context
classloader is set, which fails.

This change forces slf4j/log4j to be initialised prior to setting
the context classloader.

Original commit: elastic/x-pack-elasticsearch@85ed8eb488
2018-04-17 11:04:31 +10:00
Lisa Cawley fa44406cea [DOCS] Add X-Pack monitoring details for Elasticsearch (elastic/x-pack-elasticsearch#4328)
Original commit: elastic/x-pack-elasticsearch@0d160df9b6
2018-04-16 14:57:42 -07:00
Ryan Ernst 0ac8b78986 Remove legacy help files (elastic/x-pack-elasticsearch#4343)
This commit removes help files that existed from the cli infrastructure
long ago.

Original commit: elastic/x-pack-elasticsearch@df11f59763
2018-04-16 12:26:42 -07:00
Chris Earle 64a3339178 [Monitoring] Ignore data when no Cluster UUID exists (elastic/x-pack-elasticsearch#4344)
This ignores data collection when the cluster is not ready, in addition to
the existing check that ignores when the cluster state's version is
unknown.

Original commit: elastic/x-pack-elasticsearch@54257d7e6f
2018-04-16 13:58:20 -04:00
Zachary Tong b73c16287b [Rollup] Document type should be _doc (elastic/x-pack-elasticsearch#4363)
Rollup was using a historical `rollup` doc type, when we should be using
`_doc` so that it is forward-compatible with the removal in 8.0

Original commit: elastic/x-pack-elasticsearch@4b3188e6c8
2018-04-16 10:52:13 -07:00
Shaunak Kashyap 31b118552f Give the logstash_admin role cluster:monitor/main privilege (elastic/x-pack-elasticsearch#4318)
This is required so the Logstash Centralized Configuration Management UI in Kibana may make the GET / request to Elasticsearch and retrieve the cluster UUID. It then uses this cluster UUID to make a call to a Kibana Monitoring API to retrieve a list of pipelines from Monitoring. In order for the Kibana Monitoring API request to succeed, the logged-in user needs to have the built-in monitoring_user role anyway, so we give this role the cluster:monitor/main privilege.

Original commit: elastic/x-pack-elasticsearch@bf6ad5c1df
2018-04-16 10:47:01 -07:00
Lisa Cawley 5742ec92b2 [DOCS] Adds links to Getting Started with Security (elastic/x-pack-elasticsearch#4349)
Original commit: elastic/x-pack-elasticsearch@28148bd72e
2018-04-16 10:37:45 -07:00
Costin Leau f7bed219f3 SQL: improve conversion of Date types (elastic/x-pack-elasticsearch#4382)
When dealing with dates, the conversion now returns a proper DateTime
instance instead of a long

Relates elastic/x-pack-elasticsearch#4331

Original commit: elastic/x-pack-elasticsearch@bba9f2c79f
2018-04-16 19:58:32 +03:00
Alexander Reelsen 25895e0a3c Tests: Check watcher state before starting/stopping (elastic/x-pack-elasticsearch#4362)
This changes the behaviour of AbstractWatcherIntegrationTestCase and its
startWatcher/stopWatcher methods. Instead of checking for the target
state and just starting or stopping if it does not match, the methods
now wait for certain states to be reached before starting or stopping.

This will fix test failures where a failure is started instead of
stopped or vice versa.

Original commit: elastic/x-pack-elasticsearch@f0b0954803
2018-04-16 18:53:04 +02:00
Costin Leau fd7b3e4d0b SQL: add conversion to same and NULL data type (elastic/x-pack-elasticsearch#4378)
Add trivial data type conversions to same (identity) and NULL type

Original commit: elastic/x-pack-elasticsearch@dc825443d5
2018-04-16 19:48:22 +03:00
Costin Leau 7266042546 SQL: Add folding to Math functions (elastic/x-pack-elasticsearch#4377)
Math functions now support folding

Original commit: elastic/x-pack-elasticsearch@4af7a22b32
2018-04-16 19:48:10 +03:00
Costin Leau e69f820423 SQL: Expand SYS TABLES behavior to support ODBC spec (elastic/x-pack-elasticsearch#4375)
Add behavior for enumerating catalogs, table types, etc when the main
param is SQL_ALL_* while all the rest are empty strings

Fix elastic/x-pack-elasticsearch#4334

Original commit: elastic/x-pack-elasticsearch@28e7b15904
2018-04-16 19:47:38 +03:00
Costin Leau bf5f7e1847 SQL: Enable some date agg tests (elastic/x-pack-elasticsearch#4374)
Now that composite aggs return dates as long (instead of Strings) enable
relevant tests

Relates https://github.com/elastic/elasticsearch/pull/29370

Original commit: elastic/x-pack-elasticsearch@6653db6d0f
2018-04-16 19:47:08 +03:00
Costin Leau e113d5dda1 SQL: Fix bug in transforming Expression properties (elastic/x-pack-elasticsearch#4373)
Fix bug in checking whether the expression properties has been  
modified or not.

Change test to move from SIN to SQRT (as the former seem to return
different values (extra digit) across JDK versions - 8 vs 10)

Fix elastic/x-pack-elasticsearch#4335

Original commit: elastic/x-pack-elasticsearch@8f672c455d
2018-04-16 19:46:33 +03:00
jaymode 13d08f9c42 Test: don't reset node in ClusterPrivilegeTests
In the ClusterPrivilegeTests class, the code was resetting the node
after each test and failures were seen in CI that were HTTP 401 when
a 403 was expected. This commit removes the resetting of the node
between tests as this was not necessary.

Additionally, there is an issue in the SecuritySingleNodeTestCase where
the rest client was not torn down afterstopping a node and starting a
new node. This means the client used in other tests would not be
connected to the right cluster. This change resolves this by tearing
down the rest client after the old node is torn down.

relates elastic/x-pack-elasticsearch#4383

Original commit: elastic/x-pack-elasticsearch@2f81a4b2e2
2018-04-16 10:31:13 -06:00
Lisa Cawley da6ff8fddd [DOCS] Clarified xpack.monitoring.enabled setting (elastic/x-pack-elasticsearch#4384)
Original commit: elastic/x-pack-elasticsearch@e713aee367
2018-04-16 09:16:29 -07:00
javanna 719bad0cc3 Mute ClusterPrivilegeTests#testThatSnapshotAndRestore
See elastic/x-pack-elasticsearch#4383

Original commit: elastic/x-pack-elasticsearch@95d57374bc
2018-04-16 17:38:44 +02:00