From d7e42fcb768809696a6faf1d3497b3df70f9c669 Mon Sep 17 00:00:00 2001 From: Peter-Josef Meisch Date: Thu, 10 Nov 2022 21:41:10 +0100 Subject: [PATCH] Update documentation. Original Pull Request #2361 Closes #2360 --- src/main/asciidoc/preface.adoc | 2 +- .../reference/elasticsearch-clients.adoc | 92 +++++++++------ .../asciidoc/reference/elasticsearch-new.adoc | 1 + .../elasticsearch-object-mapping.adoc | 27 ++--- .../reference/elasticsearch-operations.adoc | 27 ++--- .../reactive-elasticsearch-operations.adoc | 106 +++++------------- 6 files changed, 105 insertions(+), 150 deletions(-) diff --git a/src/main/asciidoc/preface.adoc b/src/main/asciidoc/preface.adoc index 1c4efd02e..5806fbcef 100644 --- a/src/main/asciidoc/preface.adoc +++ b/src/main/asciidoc/preface.adoc @@ -37,7 +37,7 @@ built and tested. [cols="^,^,^,^,^",options="header"] |=== | Spring Data Release Train | Spring Data Elasticsearch | Elasticsearch | Spring Framework | Spring Boot -| 2022.0 (Turing) | 5.0.x | 8.5.0 | 6.0.x | 3.0.x? +| 2022.0 (Turing) | 5.0.x | 8.5.0 | 6.0.x | 3.0.x | 2021.2 (Raj) | 4.4.x | 7.17.3 | 5.3.x | 2.7.x | 2021.1 (Q) | 4.3.x | 7.15.2 | 5.3.x | 2.6.x | 2021.0 (Pascal) | 4.2.xfootnote:oom[Out of maintenance] | 7.12.0 | 5.3.x | 2.5.x diff --git a/src/main/asciidoc/reference/elasticsearch-clients.adoc b/src/main/asciidoc/reference/elasticsearch-clients.adoc index 8abb192e5..04fe74b16 100644 --- a/src/main/asciidoc/reference/elasticsearch-clients.adoc +++ b/src/main/asciidoc/reference/elasticsearch-clients.adoc @@ -3,8 +3,10 @@ This chapter illustrates configuration and usage of supported Elasticsearch client implementations. -Spring Data Elasticsearch operates upon an Elasticsearch client that is connected to a single Elasticsearch node or a cluster. -Although the Elasticsearch Client can be used to work with the cluster, applications using Spring Data Elasticsearch normally use the higher level abstractions of <> and <>. +Spring Data Elasticsearch operates upon an Elasticsearch client (provided by Elasticsearch client libraries) that is +connected to a single Elasticsearch node or a cluster. +Although the Elasticsearch Client can be used directly to work with the cluster, applications using Spring Data +Elasticsearch normally use the higher level abstractions of <> and <>. [[elasticsearch.clients.restclient]] == Imperative Rest Client @@ -21,14 +23,20 @@ public class MyClientConfig extends ElasticsearchConfiguration { @Override public ClientConfiguration clientConfiguration() { - return ClientConfiguration.builder() // - .connectedTo("localhost:9200") // + return ClientConfiguration.builder() <.> + .connectedTo("localhost:9200") .build(); } } +---- +<.> for a detailed description of the builder methods see <> +==== -// ... +The following beans can then be injected in other Spring components: +==== +[source,java] +---- @Autowired ElasticsearchOperations operations; <.> @@ -39,11 +47,8 @@ ElasticsearchClient elasticsearchClient; <.> RestClient restClient; <.> ---- -the following can be injected: - <.> an implementation of `ElasticsearchOperations` <.> the `co.elastic.clients.elasticsearch.ElasticsearchClient` that is used. -This is new Elasticsearch client implementation. <.> the low level `RestClient` from the Elasticsearch libraries ==== @@ -61,18 +66,24 @@ When working with the reactive stack, the configuration must be derived from a d import org.springframework.data.elasticsearch.client.elc.ReactiveElasticsearchConfiguration; @Configuration -public class MyClientConfig extends ElasticsearchConfiguration { +public class MyClientConfig extends ReactiveElasticsearchConfiguration { @Override public ClientConfiguration clientConfiguration() { - return ClientConfiguration.builder() // - .connectedTo("localhost:9200") // + return ClientConfiguration.builder() <.> + .connectedTo("localhost:9200") .build(); } } +---- +<.> for a detailed description of the builder methods see <> +==== -// ... +The following beans can then be injected in other Spring components: +==== +[source,java] +---- @Autowired ReactiveElasticsearchOperations operations; <.> @@ -85,9 +96,9 @@ RestClient restClient; <.> the following can be injected: -<.> an implementation of `ElasticsearchOperations` +<.> an implementation of `ReactiveElasticsearchOperations` <.> the `org.springframework.data.elasticsearch.client.elc.ReactiveElasticsearchClient` that is used. -This is based on the new Elasticsearch client implementation. +This is a reactive implementation based on the Elasticsearch client implementation. <.> the low level `RestClient` from the Elasticsearch libraries ==== @@ -97,8 +108,14 @@ When using repositories, this instance is used under the hood as well. [[elasticsearch.clients.resthighlevelclient]] == High Level REST Client (deprecated) -The Java RestHighLevelClient is deprecated, but still can be configured like shown (read -<> as well): +[CAUTION] +==== +The Elasticsearch Java RestHighLevelClient is deprecated, but still can be configured like shown (make sure to read +<> as well). + +It should only be used to access an Elasticsearch cluster running version 7, even with the compatibility headers set +there are cases where the `RestHighLevelClient` cannot read the responses sent from a version 8 cluster. +==== .RestHighLevelClient ==== @@ -127,15 +144,6 @@ public class RestClientConfig extends AbstractElasticsearchConfiguration { RestHighLevelClient highLevelClient; RestClient lowLevelClient = highLevelClient.lowLevelClient(); <3> - -// ... - -IndexRequest request = new IndexRequest("spring-data") - .id(randomID()) - .source(singletonMap("feature", "high-level-rest-client")) - .setRefreshPolicy(IMMEDIATE); - -IndexResponse response = highLevelClient.index(request,RequestOptions.DEFAULT); ---- <1> Use the builder to provide cluster addresses, set default `HttpHeaders` or enable SSL. @@ -150,8 +158,12 @@ The `org.springframework.data.elasticsearch.client.erhlc.ReactiveElasticsearchCl It uses the request/response objects provided by the Elasticsearch core project. Calls are directly operated on the reactive stack, **not** wrapping async (thread pool bound) responses into reactive types. -This was the first reactive implementation Spring Data Elasticsearch provided, but now is deprecated in favour of the `org.springframework.data.elasticsearch.client.elc.ReactiveElasticsearchClient` +[CAUTION] +==== +This was the first reactive implementation Spring Data Elasticsearch provided, but now is deprecated in favour +of the `org.springframework.data.elasticsearch.client.elc.ReactiveElasticsearchClient` which uses the functionality offered by the new Elasticsearch client libraries. +==== .Reactive REST Client (deprecated) ==== @@ -194,6 +206,11 @@ Client behaviour can be changed via the `ClientConfiguration` that allows to set ==== [source,java] ---- +import org.springframework.data.elasticsearch.client.ClientConfiguration; +import org.springframework.data.elasticsearch.support.HttpHeaders; + +import static org.springframework.data.elasticsearch.client.elc.ElasticsearchClients.*; + HttpHeaders httpHeaders = new HttpHeaders(); httpHeaders.add("some-header", "on every request") <.> @@ -212,7 +229,7 @@ ClientConfiguration clientConfiguration = ClientConfiguration.builder() return headers; }) .withClientConfigurer( <.> - ElasticsearchClients.ElasticsearchClientConfigurationCallback.from(clientBuilder -> { + ElasticsearchClientConfigurationCallback.from(clientBuilder -> { // ... return clientBuilder; })) @@ -227,12 +244,10 @@ ClientConfiguration clientConfiguration = ClientConfiguration.builder() <.> Optionally set a proxy. <.> Optionally set a path prefix, mostly used when different clusters a behind some reverse proxy. <.> Set the connection timeout. -Default is 10 sec. <.> Set the socket timeout. -Default is 5 sec. <.> Optionally set headers. <.> Add basic authentication. -<.> A `Supplier
` function can be specified which is called every time before a request is sent to Elasticsearch - here, as an example, the current time is written in a header. +<.> A `Supplier` function can be specified which is called every time before a request is sent to Elasticsearch - here, as an example, the current time is written in a header. <.> a function to configure the created client (see <>), can be added multiple times. ==== @@ -251,6 +266,8 @@ The following callbacks are provided: [[elasticsearch.clients.configuration.callbacks.rest]] ==== Configuration of the low level Elasticsearch `RestClient`: +This callback provides a `org.elasticsearch.client.RestClientBuilder` that can be used to configure the Elasticsearch +`RestClient`: ==== [source,java] ---- @@ -266,6 +283,9 @@ ClientConfiguration.builder() [[elasticsearch.clients.configurationcallbacks.httpasync]] ==== Configuration of the HttpAsyncClient used by the low level Elasticsearch `RestClient`: +This callback provides a `org.apache.http.impl.nio.client.HttpAsyncClientBuilder` to configure the HttpCLient that is +used by the `RestClient`. + ==== [source,java] ---- @@ -287,6 +307,9 @@ documentation]. For the imperative client this must be done by setting the default headers, for the reactive code this must be done using a header supplier: +CAUTION: Even when these headers are set, there are cases where the response returned from the cluster cannot be +parsed with the client. This is not an error in Spring Data Elasticsearch. + ==== [source,java] ---- @@ -310,12 +333,13 @@ ClientConfiguration clientConfiguration = ClientConfiguration.builder() [[elasticsearch.clients.logging]] == Client Logging -To see what is actually sent to and received from the server `Request` / `Response` logging on the transport level needs to be turned on as outlined in the snippet below. +To see what is actually sent to and received from the server `Request` / `Response` logging on the transport level +needs to be turned on as outlined in the snippet below. This can be enabled in the Elasticsearch client by setting +the level of the `tracer` package to "trace" (see +https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current/java-rest-low-usage-logging.html) .Enable transport layer logging [source,xml] ---- - + ---- - -NOTE: The above applies to both the `RestHighLevelClient` and `ReactiveElasticsearchClient` when obtained via `RestClients` respectively `ReactiveRestClients`. diff --git a/src/main/asciidoc/reference/elasticsearch-new.adoc b/src/main/asciidoc/reference/elasticsearch-new.adoc index 859e98468..65aae6123 100644 --- a/src/main/asciidoc/reference/elasticsearch-new.adoc +++ b/src/main/asciidoc/reference/elasticsearch-new.adoc @@ -6,6 +6,7 @@ * Upgrade to Java 17 baseline * Upgrade to Spring Framework 6 +* Upograde to Elasticsearch 8.5.0 * Use the new Elasticsearch client library [[new-features.4-4-0]] diff --git a/src/main/asciidoc/reference/elasticsearch-object-mapping.adoc b/src/main/asciidoc/reference/elasticsearch-object-mapping.adoc index c3a2ab668..bc119f108 100644 --- a/src/main/asciidoc/reference/elasticsearch-object-mapping.adoc +++ b/src/main/asciidoc/reference/elasticsearch-object-mapping.adoc @@ -1,20 +1,9 @@ [[elasticsearch.mapping]] = Elasticsearch Object Mapping -Spring Data Elasticsearch Object Mapping is the process that maps a Java object - the domain entity - into the JSON representation that is stored in Elasticsearch and back. - -Earlier versions of Spring Data Elasticsearch used a Jackson based conversion, Spring Data Elasticsearch 3.2.x introduced the <>. -As of version 4.0 only the Meta Object Mapping is used, the Jackson based mapper is not available anymore and the `MappingElasticsearchConverter` is used. - -The main reasons for the removal of the Jackson based mapper are: - -* Custom mappings of fields needed to be done with annotations like `@JsonFormat` or `@JsonInclude`. -This often caused problems when the same object was used in different JSON based datastores or sent over a JSON based API. -* Custom field types and formats also need to be stored into the Elasticsearch index mappings. -The Jackson based annotations did not fully provide all the information that is necessary to represent the types of Elasticsearch. -* Fields must be mapped not only when converting from and to entities, but also in query argument, returned data and on other places. - -Using the `MappingElasticsearchConverter` now covers all these cases. +Spring Data Elasticsearch Object Mapping is the process that maps a Java object - the domain entity - into the JSON +representation that is stored in Elasticsearch and back. The class that is internally used for this mapping is the +`MappingElasticsearcvhConverter`. [[elasticsearch.mapping.meta-model]] == Meta Model Object Mapping @@ -31,14 +20,13 @@ The metadata is taken from the entity's properties which can be annotated. The following annotations are available: * `@Document`: Applied at the class level to indicate this class is a candidate for mapping to the database. -The most important attributes are: +The most important attributes are (check the API documentation for the complete list of attributes): ** `indexName`: the name of the index to store this entity in. This can contain a SpEL template expression like `"log-#{T(java.time.LocalDate).now().toString()}"` ** `createIndex`: flag whether to create an index on repository bootstrapping. Default value is _true_. See <> -** `versionType`: Configuration of version management. -Default value is _EXTERNAL_. + * `@Id`: Applied at the field level to mark the field used for identity purpose. * `@Transient`: By default all fields are mapped to the document when it is stored or retrieved, this annotation excludes the field. @@ -103,6 +91,9 @@ The following table shows the different attributes and the mapping created from NOTE: If you are using a custom date format, you need to use _uuuu_ for the year instead of _yyyy_. This is due to a https://www.elastic.co/guide/en/elasticsearch/reference/current/migrate-to-java-time.html#java-time-migration-incompatible-date-formats[change in Elasticsearch 7]. +Check the code of the `org.springframework.data.elasticsearch.annotations.DateFormat` enum for a complete list of +predefined values and their patterns. + [[elasticsearch.mapping.meta-model.annotations.range]] ==== Range types @@ -194,7 +185,6 @@ Those type hints are represented as `_class` attributes within the document and [source,java] ---- public class Person { <1> - @Id String id; String firstname; String lastname; @@ -274,7 +264,6 @@ Geospatial types like `Point` & `GeoPoint` are converted into _lat/lon_ pairs. [source,java] ---- public class Address { - String city, street; Point location; } diff --git a/src/main/asciidoc/reference/elasticsearch-operations.adoc b/src/main/asciidoc/reference/elasticsearch-operations.adoc index c2b8fb2de..90fa0182f 100644 --- a/src/main/asciidoc/reference/elasticsearch-operations.adoc +++ b/src/main/asciidoc/reference/elasticsearch-operations.adoc @@ -34,7 +34,7 @@ There is support for automatic creation of indices and writing the mappings when == Usage examples The example shows how to use an injected `ElasticsearchOperations` instance in a Spring REST controller. - +The example assumes that `Person` is a class that is annotated with `@Document`, `@Id` etc (see <>). .ElasticsearchOperations usage ==== [source,java] @@ -45,34 +45,29 @@ public class TestController { private ElasticsearchOperations elasticsearchOperations; - public TestController(ElasticsearchOperations elasticsearchOperations) { <1> + public TestController(ElasticsearchOperations elasticsearchOperations) { <.> this.elasticsearchOperations = elasticsearchOperations; } @PostMapping("/person") - public String save(@RequestBody Person person) { <2> - - IndexQuery indexQuery = new IndexQueryBuilder() - .withId(person.getId().toString()) - .withObject(person) - .build(); - String documentId = elasticsearchOperations.index(indexQuery); - return documentId; + public String save(@RequestBody Person person) { <.> + Person savedEntity = elasticsearchOperations.save(person); + return savedEntity.getId(); } @GetMapping("/person/{id}") - public Person findById(@PathVariable("id") Long id) { <3> - Person person = elasticsearchOperations - .queryForObject(GetQuery.getById(id.toString()), Person.class); + public Person findById(@PathVariable("id") Long id) { <.> + Person person = elasticsearchOperations.get(id.toString(), Person.class); return person; } } ---- -<1> Let Spring inject the provided `ElasticsearchOperations` bean in the constructor. -<2> Store some entity in the Elasticsearch cluster. -<3> Retrieve the entity with a query by id. +<.> Let Spring inject the provided `ElasticsearchOperations` bean in the constructor. +<.> Store some entity in the Elasticsearch cluster. The id is read from the returned entity, as it might have been +null in the `person` object and been created by Elasticsearch. +<.> Retrieve the entity with a get by id. ==== To see the full possibilities of `ElasticsearchOperations` please refer to the API documentation. diff --git a/src/main/asciidoc/reference/reactive-elasticsearch-operations.adoc b/src/main/asciidoc/reference/reactive-elasticsearch-operations.adoc index 98e1f4b5b..3e7d93111 100644 --- a/src/main/asciidoc/reference/reactive-elasticsearch-operations.adoc +++ b/src/main/asciidoc/reference/reactive-elasticsearch-operations.adoc @@ -5,80 +5,22 @@ The `ReactiveElasticsearchTemplate` is the default implementation of `ReactiveElasticsearchOperations`. -[[elasticsearch.reactive.template]] -== Reactive Elasticsearch Template +[[elasticsearch.reactive.operations]] +== Reactive Elasticsearch Operations -To get started the `ReactiveElasticsearchTemplate` needs to know about the actual client to work with. -Please see <> for details on the client. +To get started the `ReactiveElasticsearchOperations` needs to know about the actual client to work with. +Please see <> for details on the client and how to configure it. -[[elasticsearch.reactive.template.configuration]] -=== Reactive Template Configuration +[[elasticsearch.reactive.operations.usage]] +=== Reactive Operations Usage -The easiest way of setting up the `ReactiveElasticsearchTemplate` is via `AbstractReactiveElasticsearchConfiguration` providing -dedicated configuration method hooks for `base package`, the `initial entity set` etc. - -.The AbstractReactiveElasticsearchConfiguration -==== -[source,java] ----- -@Configuration -public class Config extends AbstractReactiveElasticsearchConfiguration { - - @Bean <1> - @Override - public ReactiveElasticsearchClient reactiveElasticsearchClient() { - // ... - } -} ----- -<1> Configure the client to use. This can be done by `ReactiveRestClients` or directly via `DefaultReactiveElasticsearchClient`. -==== - -NOTE: If applicable set default `HttpHeaders` via the `ClientConfiguration` of the `ReactiveElasticsearchClient`. See <>. - -TIP: If needed the `ReactiveElasticsearchTemplate` can be configured with default `RefreshPolicy` and `IndicesOptions` that get applied to the related requests by overriding the defaults of `refreshPolicy()` and `indicesOptions()`. - -However one might want to be more in control over the actual components and use a more verbose approach. - -.Configure the ReactiveElasticsearchTemplate -==== -[source,java] ----- -@Configuration -public class Config { - - @Bean <1> - public ReactiveElasticsearchClient reactiveElasticsearchClient() { - // ... - } - @Bean <2> - public ElasticsearchConverter elasticsearchConverter() { - return new MappingElasticsearchConverter(elasticsearchMappingContext()); - } - @Bean <3> - public SimpleElasticsearchMappingContext elasticsearchMappingContext() { - return new SimpleElasticsearchMappingContext(); - } - @Bean <4> - public ReactiveElasticsearchOperations reactiveElasticsearchOperations() { - return new ReactiveElasticsearchTemplate(reactiveElasticsearchClient(), elasticsearchConverter()); - } -} ----- -<1> Configure the client to use. This can be done by `ReactiveRestClients` or directly via `DefaultReactiveElasticsearchClient`. -<2> Set up the `ElasticsearchConverter` used for domain type mapping utilizing metadata provided by the mapping context. -<3> The Elasticsearch specific mapping context for domain type metadata. -<4> The actual template based on the client and conversion infrastructure. -==== - -[[elasticsearch.reactive.template.usage]] -=== Reactive Template Usage - -`ReactiveElasticsearchTemplate` lets you save, find and delete your domain objects and map those objects to documents stored in Elasticsearch. +`ReactiveElasticsearchOperations` lets you save, find and delete your domain objects and map those objects to documents +stored +in Elasticsearch. Consider the following: -.Use the ReactiveElasticsearchTemplate +.Use the ReactiveElasticsearchOperations ==== [source,java] ---- @@ -94,15 +36,20 @@ public class Person { [source,java] ---- -template.save(new Person("Bruce Banner", 42)) <1> + +ReactiveELasticsearchOperations operations; <.> + +// ... + +operations.save(new Person("Bruce Banner", 42)) <.> .doOnNext(System.out::println) - .flatMap(person -> template.findById(person.id, Person.class)) <2> + .flatMap(person -> operations.get(person.id, Person.class)) <.> .doOnNext(System.out::println) - .flatMap(person -> template.delete(person)) <3> + .flatMap(person -> operations.delete(person)) <.> .doOnNext(System.out::println) - .flatMap(id -> template.count(Person.class)) <4> + .flatMap(id -> operations.count(Person.class)) <.> .doOnNext(System.out::println) - .subscribe(); <5> + .subscribe(); <.> ---- The above outputs the following sequence on the console. @@ -114,11 +61,10 @@ The above outputs the following sequence on the console. > QjWCWWcBXiLAnp77ksfR > 0 ---- -<1> Insert a new `Person` document into the _marvel_ index under type _characters_. The `id` is generated on server side and set into the instance returned. -<2> Lookup the `Person` with matching `id` in the _marvel_ index under type _characters_. -<3> Delete the `Person` with matching `id`, extracted from the given instance, in the _marvel_ index under type _characters_. -<4> Count the total number of documents in the _marvel_ index under type _characters_. -<5> Don't forget to _subscribe()_. +<.> Insert a new `Person` document into the _marvel_ index . The `id` is generated on server +side and set into the instance returned. +<.> Lookup the `Person` with matching `id` in the _marvel_ index. +<.> Delete the `Person` with matching `id`, extracted from the given instance, in the _marvel_ index. +<.> Count the total number of documents in the _marvel_ index. +<.> Don't forget to _subscribe()_. ==== - -