Commit Graph

369 Commits

Author SHA1 Message Date
Luca Cavanna 00a6ad0e9e
Remove aliases resolution limitations when security is enabled (#31952)
Resolving wildcards in aliases expression is challenging as we may end
up with no aliases to replace the original expression with, but if we
replace with an empty array that means _all which is quite the opposite.
Now that we support and serialize the original requested aliases,
whenever aliases are replaced we will be able to know what was
initially requested. `MetaData#findAliases` can then be updated to not
return anything in case it gets empty aliases, but the original aliases
were not empty. That means that empty aliases are interpreted as _all
only if they were originally requested that way.

Relates to #31516
2018-07-20 09:23:32 +02:00
Toby McLaughlin 487cfc3b3f
[DOCS] Update TLS on Docker for 6.3 (#32114)
Remove references to the `platinum` image and add a self-generated trial
licence to the example for TLS on Docker.

Fixes elastic/elasticsearch-docker#176
2018-07-18 17:07:31 +10:00
Zachary Tong 791b9b147c
[Rollup] Add new capabilities endpoint for concrete rollup indices (#30401)
This introduces a new GetRollupIndexCaps API which allows the user to retrieve rollup capabilities of a specific rollup index (or index pattern). This is distinct from the existing RollupCaps endpoint.

- Multiple jobs can be stored in multiple indices and point to a single target data index pattern (logstash-*). The existing API finds capabilities/config of all jobs matching that data index pattern.
- One rollup index can hold data from multiple jobs, targeting multiple data index patterns. This new API finds the capabilities based on the concrete rollup indices.
2018-07-16 17:20:50 -04:00
Michael Basnight 637cac9061
Watcher: Store username on watch execution (#31873)
There is currently no way to see what user executed a watch. This commit
adds the decrypted username to each execution in the watch history, in a
new field "user".

Closes #31772
2018-07-16 14:20:16 -05:00
Zachary Tong b7f07f03ed
[Rollup] Use composite's missing_bucket (#31402)
We can leverage the composite agg's new `missing_bucket` feature on
terms groupings.  This means the aggregation criteria used in the indexer
will now return null buckets for missing keys.  

Because all buckets are now returned (even if a key is null),
we can guarantee correct doc counts with
"combined" jobs (where a job rolls up multiple schemas).  This was
previously impossible since composite would ignore documents that
didn't have _all_ the keys, meaning non-overlapping schemas would
cause composite to return no buckets.

Note: date_histo does not use `missing_bucket`, since a timestamp is
always required.

The docs have been adjusted to recommend a single, combined job.  It
also makes reference to the previous issue to help users that are upgrading
(rather than just deleting the sections).
2018-07-13 10:07:42 -04:00
Michael Basnight e85bb734cf
Docs: add security delete role to api call table (#31907) 2018-07-10 11:17:21 -05:00
Damien Alexandre 5dcaac32b0 [Docs] Fix typo in the Rollup API Quick Reference (#31855)
The Stop endpoint was not correct in the Quick Reference.
2018-07-09 12:51:57 +02:00
Hendrik Muhs e9f8442bee
[ML] Return statistics about forecasts as part of the jobsstats and usage API (#31647)
This change adds stats about forecasts, to the jobstats api as well as xpack/_usage. The following 
information is collected:

_xpack/ml/anomaly_detectors/{jobid|_all}/_stats:

 -  total number of forecasts
 -  memory statistics (mean/min/max)
 -  runtime statistics
 -  record statistics
 -  counts by status

_xpack/usage

 -  collected by job status as well as overall (_all):
     -  total number of forecasts
     -  number of jobs that have at least 1 forecast
     -  memory, runtime, record statistics
     -  counts by status

Fixes #31395
2018-07-04 08:15:45 +02:00
David Kyle 5d94003dc0
[DOCS] Check for Windows and *nix file paths (#31648)
Proper cleanup of the docs snippet tests depends on detecting what is being tested (ML, Watcher, etc) this is deduced from the file path and so we must account for Windows and Unix path separators
2018-07-02 13:10:52 +01:00
Lisa Cawley 101d675f90
[DOCS] Replace CONFIG_DIR with ES_PATH_CONF (#31635) 2018-06-28 08:27:04 -07:00
Alexander Reelsen 8a6d062180
Docs: Clarify sensitive fields watcher encryption (#31551)
Clarify the scope of encrypting sensitive settings in watcher, which fields
are encrypted and if users can have their own encrypted fields.
2018-06-26 16:24:28 +02:00
Lisa Cawley b6cc6fc2bc
[DOCS] Updates Watcher examples for code testing (#31152) 2018-06-22 18:09:37 -07:00
Lisa Cawley 638b9fd88c
[DOCS] Move sql to docs (#31474) 2018-06-22 15:40:25 -07:00
Lisa Cawley eb81a305ae
[DOCS] Move monitoring to docs folder (#31477) 2018-06-22 15:39:34 -07:00
Luca Cavanna 16e4e7a7cf
Node selector per client rather than per request (#31471)
We have made node selectors configurable per request, but all 
of other language clients don't allow for that.
A good reason not to do so, is that having a different node selector 
per request breaks round-robin. This commit makes NodeSelector 
configurable only at client initialization. It also improves the docs 
on this matter, important given that a single node selector can still 
affect round-robin.
2018-06-22 17:15:29 +02:00
Costin Leau 99f503e3be [DOCS] Fix REST tests in SQL docs
Fixed a search & replace gone awry
Tweaked the docs a bit
2018-06-22 00:16:03 +03:00
Lisa Cawley 438591566f
[DOCS] Add code snippet testing in more ML APIs (#31339) 2018-06-21 11:32:11 -07:00
lcawl 60204af0cb [DOCS] Remove fixed file from build.gradle 2018-06-21 11:13:19 -07:00
Lisa Cawley bd06563e78
[DOCS] Creates field and document level security overview (#30937) 2018-06-21 10:08:50 -07:00
Costin Leau 872418ff94 [DOCS] Significantly improve SQL docs
Introduce SQL commands
Move reserved keywords into an appendix
Add section on security
Introduce concepts section
2018-06-21 20:03:08 +03:00
Lisa Cawley 68ec958873
[DOCS] Move migration APIs to docs (#31473) 2018-06-21 08:19:23 -07:00
Costin Leau e202923225 [DOCS] Fix JDBC Maven client group/artifact ID 2018-06-20 21:42:15 +03:00
Lisa Cawley f012de0f00
[DOCS] Move licensing APIs to docs (#31445) 2018-06-20 08:17:11 -07:00
Lisa Cawley 5971eb83c4
[DOCS] Fixes code snippet testing for machine learning (#31189) 2018-06-19 13:57:10 -07:00
Lisa Cawley 8fd1f5fbed
[DOCS] Moves the info API to docs (#31121) 2018-06-19 10:33:57 -07:00
Lisa Cawley cfb470429e
[DOCS] Add code snippet testing for more ML APIs (#31404) 2018-06-19 08:49:32 -07:00
Lisa Cawley ea92864eb1
[DOCS] Adds testing for security APIs (#31345) 2018-06-18 08:48:23 -07:00
Costin Leau bbcfcd1ca5 [DOCS] Fix version in SQL JDBC Maven template 2018-06-17 00:40:01 +03:00
Costin Leau 215c5f292a [DOCS] Improve install and setup section for SQL JDBC 2018-06-17 00:33:55 +03:00
Tim Brooks 605dbbeabd
Remove trial status info from start trial doc (#31365)
This is related to #31325. There is currently information about the
get-trial-status api on the start-trial api documentation page. It also
has the incorrect route for that api. This commit removes that
information as the start-trial page properly links to a page providing
documenation about get-trial-status.
2018-06-15 12:39:25 -06:00
Tim Vernum 74c6f18fea Fix non-REST doc snippet 2018-06-14 16:26:17 +10:00
Costin Leau 870a913217 [DOC] Extend SQL docs
Add overview section
Add data type section
Improve function section
2018-06-14 00:38:53 +03:00
lcawl 509729f9c1 [DOCS] Shortens ML API intros 2018-06-13 13:43:15 -07:00
Zachary Tong d4262de83a
[Docs] All Rollup docs experimental, agg limitations, clarify DeleteJob (#31299)
- All rollup pages should be marked as experimental instead of just
the top page
- While the job config docs state which aggregations are allowed, adding
a section which specifically details this in one place is more convenient
for the user
- Add a clarification that the DeleteJob API does not delete the rollup
data, just the rollup job.
2018-06-13 15:42:20 -04:00
Costin Leau bd81b95e95 [DOCS] Mark SQL feature as experimental 2018-06-12 19:46:08 +03:00
Lisa Cawley ff0f4ece2f
[DOCS] Updates machine learning custom URL screenshots (#31222) 2018-06-12 08:57:11 -07:00
Albert Zaharovits e988ace5f7
[DOCS] Clarify audit index settings when remote indexing (#30923) 2018-06-12 16:19:43 +03:00
Yuri Tceretian ef453b31b4
Delete typos in SAML docs (#31199)
* Delete typos in SAML docs saml-guide.asciidoc
2018-06-12 14:45:27 +03:00
lcawl 1de38a2488 [DOCS] Moves ML content to stack-docs 2018-06-07 09:26:00 -07:00
Lisa Cawley d0f35d204e
[DOCS] Clarifies recommendation for audit index output type (#31146) 2018-06-07 08:55:14 -07:00
Lisa Cawley 7f0c2e89c2
[DOCS] Moves X-Pack setup to docs (#31145) 2018-06-06 14:46:20 -07:00
Lisa Cawley 68827fc046
[DOCS] Enables testing for monitoring examples (#31119) 2018-06-06 13:25:36 -07:00
Lisa Cawley b4514d3cc1
[DOCS] Moves ML node info to docs (#31142) 2018-06-06 12:39:24 -07:00
Lisa Cawley 45537c59e5
[DOCS] Moves X-Pack settings to docs folder (#31120) 2018-06-06 10:05:32 -07:00
Lisa Cawley 6fd4eb52b8
[DOCS] Moves commands to docs folder (#31114) 2018-06-06 07:49:15 -07:00
David Kyle 3767bdc98d
[ML][DOCS] Add example of top N derivative aggregation (#31109)
Add example of top N derivative aggregation to the ML datafeed docs
2018-06-06 13:21:16 +01:00
lcawl 7c05f69c39 [DOCS] Creates rest-api folder in docs 2018-06-05 16:43:55 -07:00
Lisa Cawley 840a3bd5a6
[DOCS] Fixes security example (#31082) 2018-06-05 08:50:06 -07:00
lcawl 024400bcb8 [DOCS] Removes redundant authorization pages 2018-06-04 12:02:50 -07:00
lcawl 409da09a81 [DOCS] Re-adds custom realm 2018-06-04 11:30:11 -07:00
lcawl 53357e7d14 [DOCS] Removes duplicated authentication pages 2018-06-04 11:04:54 -07:00
lcawl 7e565797e7 [DOCS] Moves machine learning overview to stack-docs 2018-06-04 08:39:22 -07:00
Michael Russell b665d591f1 [Docs] Fix typo in watcher conditions documentation (#30989) 2018-06-04 13:00:57 +02:00
Lisa Cawley 7c74318580
[DOCS] Fixes links (#31011) 2018-05-31 17:02:18 -07:00
Albert Zaharovits d9ab1469c9
[DOCS] Clarify not all PKCS12 usable as truststores (#30750)
Although elasticsearch-certutil generates PKCS#12
files which are usable as both keystore and truststore
this is uncommon in practice. Settle these expectations
for the users following our security guides.
2018-05-31 21:49:27 +03:00
lcawl 94e1bdba33 [DOCS] Fixes kibana security file location 2018-05-30 07:47:20 -07:00
lcawl b5c997b10b [DOCS] Fix watcher file location 2018-05-29 21:56:57 -07:00
Toby McLaughlin 83a7ade7c5
[DOCS] Remove reference to platinum Docker image (#30916) 2018-05-30 11:17:02 +10:00
Lisa Cawley eb2e43b695
[DOCS] Adds missing TLS settings for auditing (#30822) 2018-05-29 15:53:19 -07:00
lcawl e76c09f642 [DOCS] Fixes typos in security settings 2018-05-23 16:41:04 -07:00
lcawl 94ba78e09a [DOCS] Splits auditing.asciidoc into smaller files 2018-05-23 15:43:43 -07:00
Lisa Cawley ab047ca28d
[DOCS] Fix more edit URLs in Stack Overview (#30704) 2018-05-23 09:37:55 -07:00
lcawl fcd54cdb7c [DOCS] Fixes broken link for native realm 2018-05-23 09:32:07 -07:00
Lisa Cawley b3619bdd3e
[DOCS] Clarified audit.index.client.hosts (#30797) 2018-05-23 08:41:06 -07:00
Adrien Grand a19df4ab3b
Add a `format` option to `docvalue_fields`. (#29639)
This commit adds the ability to configure how a docvalue field should be
formatted, so that it would be possible eg. to return a date field
formatted as the number of milliseconds since Epoch.

Closes #27740
2018-05-23 14:39:04 +02:00
Lisa Cawley 9ffeb171e0
[DOCS] Add SAML configuration information (#30548) 2018-05-22 08:50:35 -07:00
Lisa Cawley 1094ec0917
[DOCS] Remove X-Pack references from SQL CLI (#30694) 2018-05-22 08:33:14 -07:00
lcawl 8180e4431f [DOCS] Removes out-dated x-pack/docs/en/index.asciidoc 2018-05-18 11:31:25 -07:00
Lisa Cawley e750462e0c
[DOCS] Moves X-Pack configurationg pages in table of contents (#30702) 2018-05-18 10:26:03 -07:00
Hendrik Muhs 6c313a9871 This implementation lazily (on 1st forecast request) checks for available
diskspace and creates a subfolder for storing data outside of Lucene
indexes, but as part of the ES data paths.

Details:
 - tmp storage is managed and does not allow allocation if disk space is
   below a threshold (5GB at the moment)
 - tmp storage is supposed to be managed by the native component but in
   case this fails cleanup is provided:
    - on job close
    - on process crash
    - after node crash, on restart
 - available space is re-checked for every forecast call (the native
   component has to check again before writing)

Note: The 1st path that has enough space is chosen on job open (job
close/reopen triggers a new search)
2018-05-18 14:04:09 +02:00
Lisa Cawley a8b4a98ee5
[DOCS] Fixes edit URLs for stack overview (#30583) 2018-05-17 14:20:03 -07:00
lcawl 663295d635 [DOCS] Replace X-Pack terms with attributes 2018-05-17 09:57:11 -07:00
Christoph Büscher 712473b558
[Docs] Replace InetSocketTransportAddress with TransportAdress (#30673)
The former class has been removed in 6.0, the documentation code
snippets should be updated accordingly.
2018-05-17 14:23:08 +02:00
lcawl 8ff9baeb82 [DOCS] Fixes list of unconverted snippets in build.gradle 2018-05-16 13:11:06 -07:00
lcawl 4f41018753 [DOCS] Reorganizes RBAC documentation 2018-05-16 12:22:52 -07:00
Jason Tedor abc06d5b79
Expose master version in REST test context (#30623)
This commit exposes the master version to the REST test context. This
will be needed in a follow-up where the master version will be used to
determine whether or not a certain warning header is expected.
2018-05-15 17:26:43 -04:00
David Roberts 50c34b2a9b
[ML] Reverse engineer Grok patterns from categorization results (#30125)
This change adds a grok_pattern field to the GET categories API
output in ML. It's calculated using the regex and examples in the
categorization result, and applying a list of candidate Grok
patterns to the bits in between the tokens that are considered to
define the category.

This can currently be considered a prototype, as the Grok patterns
it produces are not optimal. However, enough people have said it
would be useful for it to be worthwhile exposing it as experimental
functionality for interested parties to try out.
2018-05-15 09:02:38 +01:00
Tim Vernum 7dd816e77c Update build file due to doc file rename
A file with uncoverted snippets was change as part of 7f47ff9, but
build.gradle was not updated to reflect the rename.
2018-05-15 14:28:29 +10:00
lcawl 7f47ff9fcd [DOCS] Fixes title capitalization in security content 2018-05-14 15:36:03 -07:00
Lisa Cawley 7928270610
[DOCS] Fix realm setting names (#30499) 2018-05-14 13:13:26 -07:00
Lisa Cawley fa45c6c9a6
[DOCS] Fix path info for various security files (#30502) 2018-05-14 13:07:27 -07:00
Jay Modi 13e2223680
Docs: remove transport_client from CCS role example (#30263)
This commit removes the unnecessary transport_client cluster permission
from the role that is used as an example in our documentation. This
permission is not needed to use cross cluster search.
2018-05-04 11:52:11 -06:00
lcawley b3516786ec [DOCS] Removed X-Pack Breaking Changes 2018-05-02 15:52:14 -07:00
lcawley 427d6912ea [DOCS] Fixes link to TLS LDAP info 2018-05-02 15:30:49 -07:00
Lisa Cawley e697299103
[DOCS] Fixes broken links to bootstrap user (#30349) 2018-05-02 13:09:25 -07:00
Lisa Cawley 3e9fe3c9cd
[DOCS] Reorganizes authentication details in Stack Overview (#30280) 2018-05-02 12:08:02 -07:00
Ryan Ernst fb0aa562a5
Network: Remove http.enabled setting (#29601)
This commit removes the http.enabled setting. While all real nodes (started with bin/elasticsearch) will always have an http binding, there are many tests that rely on the quickness of not actually needing to bind to 2 ports. For this case, the MockHttpTransport.TestPlugin provides a dummy http transport implementation which is used by default in ESIntegTestCase.

closes #12792
2018-05-02 11:42:05 -07:00
Lisa Cawley 383856a175
[DOCS] Adds LDAP realm configuration details (#30214) 2018-05-02 11:22:32 -07:00
Lisa Cawley 5064ff6ad4
[DOCS] Adds native realm configuration details (#30215) 2018-05-02 10:56:31 -07:00
Lisa Cawley 0d7ac9a74c
[DOCS] Enables edit links for X-Pack pages (#30278) 2018-05-02 10:13:42 -07:00
lcawley 42cc122a9b [DOCS] Fixes links to breaking changes 2018-05-01 17:21:28 -07:00
Lisa Cawley 092dd6cb89
[DOCS] Removes X-Pack Elasticsearch release notes (#30272) 2018-05-01 16:05:23 -07:00
lcawley 6a55eead94 [DOCS] Removes broken link 2018-05-01 14:42:29 -07:00
Lisa Cawley db44a4cddb
[DOCS] Adds file realm configuration details (#30221) 2018-05-01 14:11:38 -07:00
Lisa Cawley 86addc0c8b
[DOCS] Adds PKI realm configuration details (#30225) 2018-05-01 13:47:24 -07:00
Lisa Cawley 1a0838bd0b
[DOCS] Removes redundant SAML realm settings (#30196) 2018-05-01 11:53:11 -07:00
Lisa Cawley 5b5c98c96b
[DOCS] Adds changelog to Elasticsearch Reference (#30271) 2018-05-01 10:34:26 -07:00
lcawley 0f688e0867 [DOCS] Fixes section error 2018-05-01 10:21:39 -07:00
Lisa Cawley 7933f5e28e
[DOCS] Adds Active Directory realm configuration details (#30223) 2018-05-01 09:15:13 -07:00
Lisa Cawley 846783c2b6
[DOCS] Removes redundant file realm settings (#30192) 2018-05-01 08:55:35 -07:00
Lisa Cawley b34e5cf216
[DOCS] Fixes users command name (#30275) 2018-05-01 08:52:22 -07:00
lcawley 51902238f3 [DOCS] Fixes syskeygen command name 2018-04-30 13:20:22 -07:00
Lisa Cawley e95a7aa6f0
[DOCS] Removes redundant Active Directory realm settings (#30190) 2018-04-30 08:52:21 -07:00
Lisa Cawley 05160e6cd8
[DOCS] Removes redundant LDAP realm settings (#30193) 2018-04-30 08:04:15 -07:00
Lisa Cawley 7eaec6031d
[DOCS] Fixes broken links (#30219) 2018-04-27 13:24:15 -07:00
Lisa Cawley 962c965812
[DOCS] Adds native realm security settings (#30186) 2018-04-27 12:40:05 -07:00
Tim Brooks 592481e4ed
Require acknowledgement to start_trial license (#30135)
This is related to #30134. It modifies the start_trial action to require
an acknowledgement parameter in the rest request to actually start the
trial license. There are backwards compatibility issues as prior ES
versions did not support this parameter. To handle this, it is assumed
that a request coming from a node prior to 6.3 is acknowledged. And
attempts to write a non-acknowledged request to a prior to 6.3 node will
throw an exception.

Additionally this PR adds messages about the trial license the user is
generating.
2018-04-26 21:42:44 -04:00
Lisa Cawley eebcdce7f9
[DOCS] Updates docker installation package details (#30110) 2018-04-26 08:54:29 -07:00
Ryan Ernst d8636d0fe3 Rename syskeygen
This commit renames syskeygen to elasticsearch-syskeygen
2018-04-20 15:34:01 -07:00
Jason Tedor c7c0e330b8 Rename users
This commit renames users to elasticsearch-users.
2018-04-20 15:34:01 -07:00
Jason Tedor b05c8bf781 Rename migrate
This commit renames migrate to elasticsearch-migrate.
2018-04-20 15:34:00 -07:00
Jason Tedor 2d3fac307a Rename saml-metadata
This commit renames saml-metadata to elasticsearch-saml-metadata.
2018-04-20 15:34:00 -07:00
Jason Tedor 8a8a1dfc2e Rename sql-cli
This commit renames sql-cli to elasticsearch-sql-cli.
2018-04-20 15:34:00 -07:00
Ryan Ernst 6dbb7be067 Rename croneval
This commit renames croneval to elasticsearch-croneval
2018-04-20 15:34:00 -07:00
Jason Tedor 7ba2defdbf Rename setup-passwords
This commit renames setup-passwords to elasticsearch-setup-passwords.
2018-04-20 15:34:00 -07:00
Jason Tedor 03ffd16921 Rename certutil
This commit renames certutil to elasticsearch-certutil.
2018-04-20 15:33:59 -07:00
Jason Tedor a6b4682711 Rename certgen
This commit renames certgen to elasticsearch-certgen.
2018-04-20 15:33:59 -07:00
Ryan Ernst fab5e21e7d Build: Split distributions into oss and default
This commit makes x-pack a module and adds it to the default
distrubtion. It also creates distributions for zip, tar, deb and rpm
which contain only oss code.
2018-04-20 15:33:57 -07:00
Ryan Ernst 2efd22454a Migrate x-pack-elasticsearch source to elasticsearch 2018-04-20 15:29:54 -07:00