X-Pack security supports built-in authentication service
`token-service` that allows access tokens to be used to
access Elasticsearch without using Basic authentication.
The tokens are generated by `token-service` based on
OAuth2 spec. The access token is a short-lived token
(defaults to 20m) and refresh token with a lifetime of 24 hours,
making them unsuitable for long-lived or recurring tasks where
the system might go offline thereby failing refresh of tokens.
This commit introduces a built-in authentication service
`api-key-service` that adds support for long-lived tokens aka API
keys to access Elasticsearch. The `api-key-service` is consulted
after `token-service` in the authentication chain. By default,
if TLS is enabled then `api-key-service` is also enabled.
The service can be disabled using the configuration setting.
The API keys:-
- by default do not have an expiration but expiration can be
configured where the API keys need to be expired after a
certain amount of time.
- when generated will keep authentication information of the user that
generated them.
- can be defined with a role describing the privileges for accessing
Elasticsearch and will be limited by the role of the user that
generated them
- can be invalidated via invalidation API
- information can be retrieved via a get API
- that have been expired or invalidated will be retained for 1 week
before being deleted. The expired API keys remover task handles this.
Following are the API key management APIs:-
1. Create API Key - `PUT/POST /_security/api_key`
2. Get API key(s) - `GET /_security/api_key`
3. Invalidate API Key(s) `DELETE /_security/api_key`
The API keys can be used to access Elasticsearch using `Authorization`
header, where the auth scheme is `ApiKey` and the credentials, is the
base64 encoding of API key Id and API key separated by a colon.
Example:-
```
curl -H "Authorization: ApiKey YXBpLWtleS1pZDphcGkta2V5" http://localhost:9200/_cluster/health
```
Closes#34383
This adds the _security/user/_privileges API to the High
Level Rest Client.
This also makes some changes to the Java model for the Role APIs
in order to better accommodate the GetPrivileges API
This commit adds support for the index templates exist API, creating
new client-side request types for that API and the get index
templates API. Also adds links in hlrc docs to pages for supported
index template APIs
this adds documentation for the retry method in the
high-level-ilm-rest-client.
this PR also renames retryLifecycleStep to retryLifecyclePolicy in the index-lifecycle-client
* Adds HLRC docs for put lifecycle policy
* Adds link to docs in client javadocs
* Fixes checkstyle
* Make the documentation use the right ack response
Implement high level client for migration upgrade API. It should wrap
RestHighLevelClient and expose high level IndexUpgradeRequest (new),
IndexTaskResponse for submissions with wait_for_completion=false and
BulkByScrollResponse (already used) objects.
refers: #29827
* Moved `AcknowledgedResponse` to core package
* Made AcknowledgedResponse not abstract and provided a default parser,
so that in cases when the field name is not overwritten then there
is no need for a subclass.
Relates to #33824
This change adds support for clearing the cache of a realm. The realms
cache may contain a stale set of credentials or incorrect role
assignment, which can be corrected by clearing the cache of the entire
realm or just that of a specific user.
Relates #29827
This adds the security `_authenticate` API to the HLREST client.
It is unlike some of the other APIs because the request does not
have a body.
The commit also creates the `User` entity. It is important
to note that the `User` entity does not have the `enabled`
flag. The `enabled` flag is part of the response, alongside
the `User` entity.
Moreover this adds the `SecurityIT` test class
(extending `ESRestHighLevelClientTestCase`).
Relates #29827
Adds support for the Clear Roles Cache API to the High Level Rest
Client. As part of this a helper class, NodesResponseHeader, has been
added that enables parsing the nodes header from responses that are
node requests.
Relates to #29827
Introduces client-specific request and response classes that do not
depend on the server
The `type` parameter is named `licenseType` in the response class to be
more descriptive. The parts that make up the acknowledged-required
response are given slightly different names than their server-response
types to be consistent with the naming in the put license API
Tests do not cover all cases because the integ test cluster starts up
with a trial license - this will be addressed in a future commit
We added support for role mapper expression DSL in #33745,
that allows us to build the role mapper expression used in the
role mapping (as rules for determining user roles based on what
the boolean expression resolves to).
This change now adds support for create/update role mapping
API to the high-level rest client.
This further applies the pattern set in #34125 to reduce copy-and-paste
in the single document CRUD portion of the High Level REST Client docs.
It also adds line wraps to snippets that are too wide to fit into the box
when rendered in the docs, following up on the work started in #34163.
* HLRC: ML Add preview datafeed api
* Changing deprecation handling for parser
* Removing some duplication in docs, will address other APIs in another PR
This change adds throttling to the update-by-query and delete-by-query cases
similar to throttling for reindex. This mostly means additional methods on the
client class itself, since the request hits the same RestHandler, just with
slightly different endpoints, and also the return values are similar.
Adds support for the get rollup job to the High Level REST Client. I had
to do three interesting and unexpected things:
1. I ported the rollup state wiping code into the high level client
tests. I'll move this into the test framework in a followup and remove
the x-pack version.
2. The `timeout` in the rollup config was serialized using the
`toString` representation of `TimeValue` which produces fractional time
values which are more human readable but aren't supported by parsing. So
I switched it to `getStringRep`.
3. Refactor the xcontent round trip testing utilities so we can test
parsing of classes that don't implements `ToXContent`.
This commit adds the Create Rollup Job API to the high level REST
client. It supersedes #32703 and adds dedicated request/response
objects so that it does not depend on server side components.
Related #29827
Yogesh Gaikwad
Benjamin Trent
Nick Knize
Tim Vernum
Andy Bristol
Albert Zaharovits
Martijn van Groningen
Nik Everett
Nhat Nguyen
Michael Basnight
Alan Woodward
Jim Ferenczi
Jim Ferenczi
Ignacio Vera
Simon Willnauer
Vladimir Dolzhenko
Tal Levy
Gordon Brown
Ed Savage
Ioannis Kakavas
Ioannis Kakavas
Mayya Sharipova
Tanguy Leroux
Christoph Büscher
Colin Goodheart-Smithe
Luca Cavanna
Igor Motov
Przemyslaw Gomulka
Zachary Tong
Jay Modi
Milan Mrdjen
Boaz Leskes
Mark Tozzi
Yannick Welsch
Jack Conradson
Jack Conradson
Paul Sanwald
Julie Tibshirani
David Kyle
Dimitris Athanasiou