Compare commits

...

110 Commits

Author SHA1 Message Date
Mark Paluch
679be76847
DATAES-888 - Release version 3.2.10 (Moore SR10). 2020-09-16 10:53:23 +02:00
Mark Paluch
1fea502355
DATAES-888 - Prepare 3.2.10 (Moore SR10). 2020-09-16 10:52:54 +02:00
Mark Paluch
52e725f9a2
DATAES-888 - Updated changelog. 2020-09-16 10:52:51 +02:00
Mark Paluch
160fd851e6
DATAES-887 - Updated changelog. 2020-09-16 10:39:04 +02:00
Peter-Josef Meisch
a409440e4e
DATAES-903 - Update to Elasticsearch 6.8.12.
Original PR: #511
2020-08-24 21:06:58 +02:00
Mark Paluch
83d5a58737
DATAES-890 - Updated changelog. 2020-08-12 13:25:55 +02:00
Mark Paluch
1b62e311fd
DATAES-872 - Updated changelog. 2020-08-12 12:01:26 +02:00
Peter-Josef Meisch
bf5eaae357 DATAES-892 - Fix ElasticsearchEntityMapper recursive descent when reading Map objects. 2020-07-29 21:17:21 +02:00
Mark Paluch
ee660bb208
DATAES-862 - Updated changelog. 2020-07-22 10:38:05 +02:00
Mark Paluch
e01d618c83
DATAES-861 - After release cleanups. 2020-07-22 10:07:20 +02:00
Mark Paluch
d48bfcc820
DATAES-861 - Prepare next development iteration. 2020-07-22 10:07:18 +02:00
Mark Paluch
c87e4a2fa5
DATAES-861 - Release version 3.2.9 (Moore SR9). 2020-07-22 09:55:56 +02:00
Mark Paluch
3770f37855
DATAES-861 - Prepare 3.2.9 (Moore SR9). 2020-07-22 09:55:30 +02:00
Mark Paluch
c5287dc326
DATAES-861 - Updated changelog. 2020-07-22 09:55:26 +02:00
Mark Paluch
53a9181d5f
DATAES-860 - Updated changelog. 2020-07-22 09:44:35 +02:00
Mark Paluch
0c36132c5c
DATAES-824 - Updated changelog. 2020-06-25 12:00:24 +02:00
Mark Paluch
33174ff28e
DATAES-823 - Updated changelog. 2020-06-10 14:31:03 +02:00
Mark Paluch
67972f5b29
DATAES-807 - After release cleanups. 2020-06-10 12:29:17 +02:00
Mark Paluch
e943a815b9
DATAES-807 - Prepare next development iteration. 2020-06-10 12:29:14 +02:00
Mark Paluch
43e8e26b5b
DATAES-807 - Release version 3.2.8 (Moore SR8). 2020-06-10 12:11:27 +02:00
Mark Paluch
43133d6a9f
DATAES-807 - Prepare 3.2.8 (Moore SR8). 2020-06-10 12:11:00 +02:00
Mark Paluch
5184fc44a7
DATAES-807 - Updated changelog. 2020-06-10 12:10:58 +02:00
Mark Paluch
93b353e3bf
DATAES-806 - Updated changelog. 2020-06-10 11:22:52 +02:00
Peter-Josef Meisch
401893eb34
DATAES-851 - Upgrade to Elasticsearch 6.8.10.
Original PR: #474
2020-06-05 14:22:58 +02:00
Peter-Josef Meisch
b19af5cf17
DATAES-837 - Update to Elasticsearch 6.8.9.
Original PR: #464
2020-05-21 06:22:35 +02:00
Peter-Josef Meisch
fd713bfc8e
DATAES-822 - Convert Reactive Client exceptions to ElasticsearchStatusException. (#453)
Original PR: #453
2020-05-12 06:07:21 +02:00
Peter-Josef Meisch
c3bde816fe
DATES-821 - Fix code for adding an alias.
Original PR: #452
2020-05-10 20:08:05 +02:00
Greg Turnquist
187a8b9e15
DATAES-811 - Remove Travis CI. 2020-05-04 15:30:29 -05:00
Peter-Josef Meisch
ee02073142 DATAES-767 - Fix ReactiveElasticsearch handling of 4xx HTTP responses. 2020-05-03 04:46:01 +02:00
Peter-Josef Meisch
0df58615e9
DATAES-767 - Fix ReactiveElasticsearch handling of 4xx HTTP responses. (#446)
Original PR: #445

(cherry picked from commit e605cad688a0e49875cf2624e7674319eb2e8c57)
2020-05-03 04:25:36 +02:00
Peter-Josef Meisch
a69658dc8b DATAES-776 - Adapt RestClients class to change in InetSocketAddress class in JDK14. 2020-04-30 17:47:11 +02:00
Mark Paluch
f3e83bf4ae DATAES-774 - Updated changelog. 2020-04-28 15:12:26 +02:00
Mark Paluch
88552cfe1a DATAES-770 - After release cleanups. 2020-04-28 14:45:30 +02:00
Mark Paluch
40ecf9211d DATAES-770 - Prepare next development iteration. 2020-04-28 14:45:29 +02:00
Mark Paluch
bd898f0363 DATAES-770 - Release version 3.2.7 (Moore SR7). 2020-04-28 14:35:55 +02:00
Mark Paluch
b1a6dc524c DATAES-770 - Prepare 3.2.7 (Moore SR7). 2020-04-28 14:35:25 +02:00
Mark Paluch
197956c4d4 DATAES-770 - Updated changelog. 2020-04-28 14:35:23 +02:00
Mark Paluch
bc0bead9b8 DATAES-755 - Updated changelog. 2020-04-28 11:59:02 +02:00
Peter-Josef Meisch
1459dd491d DATAES-778 - Fix SSL setup in the reactive client.
Original PR: #429

(cherry picked from commit 539c1ee6e75968b5874cd05a99be78079ecd9cc9)
2020-04-16 21:47:10 +02:00
Peter-Josef Meisch
952e0c8985
DATAES-780 - Upgrade 3.2.x to Elasticsearch 6.8.8.
Original PR: #420
2020-04-04 16:14:45 +02:00
Mark Paluch
f6caa731dd DATAES-762 - Updated changelog. 2020-03-31 15:08:50 +02:00
Mark Paluch
b7dbdbf0e0 DATAES-756 - After release cleanups. 2020-03-25 10:58:27 +01:00
Mark Paluch
80a61e27c4 DATAES-756 - Prepare next development iteration. 2020-03-25 10:58:26 +01:00
Mark Paluch
0ce8e7ce0e DATAES-756 - Release version 3.2.6 (Moore SR6). 2020-03-25 10:46:02 +01:00
Mark Paluch
fde9f42735 DATAES-756 - Prepare 3.2.6 (Moore SR6). 2020-03-25 10:45:35 +01:00
Mark Paluch
6b2ad6cdc6 DATAES-756 - Updated changelog. 2020-03-25 10:45:34 +01:00
Mark Paluch
7e7ceb885d
DATAES-769 - Upgrade to Elasticsearch 6.8.7. 2020-03-23 16:45:41 +01:00
Peter-Josef Meisch
54c80f3375 DATAES-763 - Allow map properties in entity with null values. 2020-03-22 08:58:11 +01:00
Peter-Josef Meisch
39b318caa7
DATAES-765 - Pageable.unpaged() is not used to build a query returning all documents.
Original PR: #409
2020-03-20 19:07:58 +01:00
Peter-Josef Meisch
17af36702f DATAES-764 - Adapt to JUnit4. 2020-03-18 20:49:28 +01:00
Sascha Woo
e22e2bbfc3 DATAES-764 - StreamQueries#streamResults does not clear scroll context when finished.
Original PR: #406

(cherry picked from commit f103bdb9d81bef1ad58cee005fe4958bbda8f26e)
2020-03-18 20:39:30 +01:00
Jens Schauder
b11f8d2c38 DATAES-744 - Updated changelog. 2020-03-11 09:59:33 +01:00
Peter-Josef Meisch
6062896568
DATAES-758 - Fix documentation for @Query annotation.
Original PR: #402
2020-03-06 18:44:49 +01:00
Mark Paluch
33057cafb1 DATAES-730 - After release cleanups. 2020-02-26 11:54:09 +01:00
Mark Paluch
643b8b1e64 DATAES-730 - Prepare next development iteration. 2020-02-26 11:54:08 +01:00
Mark Paluch
8f5947bfc7 DATAES-730 - Release version 3.2.5 (Moore SR5). 2020-02-26 11:38:18 +01:00
Mark Paluch
af78e0bf3d DATAES-730 - Prepare 3.2.5 (Moore SR5). 2020-02-26 11:37:58 +01:00
Mark Paluch
e4f1c8ba58 DATAES-730 - Updated changelog. 2020-02-26 11:37:57 +01:00
Mark Paluch
be4924a214 DATAES-729 - Updated changelog. 2020-02-26 11:31:49 +01:00
Mark Paluch
55119989f7
DATAES-752 - Upgrade to Elasticsearch 6.8.6. 2020-02-24 09:06:56 +01:00
Peter-Josef Meisch
068de487b2 DATAES-741 - Tests fail due to Elasticsearch cluster 'blocks' on nearly-full file-systems. 2020-02-15 16:32:58 +01:00
Peter-Josef Meisch
bacad5ca0c DATAES-214 - Polishing. 2020-02-12 20:58:41 +01:00
Alexander Shabunevich
864d41cb01
DATAES-214 - ElasticsearchTemplate's prepareSearch(Query query) method should use getOffset().
Original PR: #392
2020-02-12 20:55:36 +01:00
Mark Paluch
8dd7cfcc6e DATAES-732 - Updated changelog. 2020-02-12 15:05:02 +01:00
Mark Paluch
868664aa78 DATAES-731 - Updated changelog. 2020-01-17 09:58:35 +01:00
Mark Paluch
5f9eab9a97 DATAES-663 - Updated changelog. 2020-01-16 16:12:38 +01:00
Mark Paluch
090ed0be5c DATAES-704 - After release cleanups. 2020-01-15 12:48:23 +01:00
Mark Paluch
fc9e47ccd7 DATAES-704 - Prepare next development iteration. 2020-01-15 12:48:22 +01:00
Mark Paluch
4c12979bb0 DATAES-704 - Release version 3.2.4 (Moore SR4). 2020-01-15 12:37:44 +01:00
Mark Paluch
2c1613cac0 DATAES-704 - Prepare 3.2.4 (Moore SR4). 2020-01-15 12:37:18 +01:00
Mark Paluch
5a92340829 DATAES-704 - Updated changelog. 2020-01-15 12:37:16 +01:00
Mark Paluch
e317bef992 DATAES-703 - Updated changelog. 2020-01-15 10:36:37 +01:00
Mark Paluch
be7ae214a4
DATAES-725 - Update copyright years to 2020. 2020-01-07 09:08:12 +01:00
Peter-Josef Meisch
6eda05ddd7 DATAES-720 - SimpleReactiveElasticsearchRepository findAll() returns only 10 elements.
Original PR: #364

(cherry picked from commit b634f318abe771b98d01bc38a3885ffb20e5368f)
2019-12-25 11:34:13 +01:00
Peter-Josef Meisch
283b27d170 DATAES-719 - Add customization hook for reactive WebClient.
Original PR: #363

(cherry picked from commit f7a14c1135189a62675a3345d70ccb1a6ec16af4)
2019-12-25 09:09:43 +01:00
Peter-Josef Meisch
b731b47b1b DATAES-719 - Add customization hook for reactive WebClient.
Original PR: #363

(cherry picked from commit f7a14c1135189a62675a3345d70ccb1a6ec16af4)
2019-12-25 09:07:56 +01:00
Sascha Woo
31a391522a DATAES-702 - Travis CI builds currently broken.
Original PR: #351
2019-12-13 13:08:33 +01:00
Peter-Josef Meisch
0a0ac102cc
DATAES-705_-_Add-support-for-PathPrefix-to-clients-in-3_2_x.
Original PR: #354
2019-12-07 15:42:09 +01:00
Jens Schauder
325fdb47c6 DATAES-692 - After release cleanups. 2019-12-04 14:29:17 +01:00
Jens Schauder
fdee09dc04 DATAES-692 - Prepare next development iteration. 2019-12-04 14:29:15 +01:00
Jens Schauder
060cab76d5 DATAES-692 - Release version 3.2.3 (Moore SR3). 2019-12-04 14:12:24 +01:00
Jens Schauder
afb8a35eac DATAES-692 - Prepare 3.2.3 (Moore SR3). 2019-12-04 14:11:44 +01:00
Jens Schauder
1c0dd71020 DATAES-692 - Updated changelog. 2019-12-04 14:11:42 +01:00
Jens Schauder
1633668d7f DATAES-691 - Updated changelog. 2019-12-04 12:09:51 +01:00
Peter-Josef Meisch
6756f792c8
DATAES-699 - Fix count implementation.
Original PR: #349
2019-12-01 10:03:56 +01:00
Peter-Josef Meisch
bae4db8a7f DATAES-700 - Enable proxy support for RestClient. 2019-11-30 23:30:59 +01:00
Mark Paluch
c47fd2cfce DATAES-685 - After release cleanups. 2019-11-18 12:42:05 +01:00
Mark Paluch
cb08bb7196 DATAES-685 - Prepare next development iteration. 2019-11-18 12:42:04 +01:00
Mark Paluch
f3500623ff DATAES-685 - Release version 3.2.2 (Moore SR2). 2019-11-18 12:32:12 +01:00
Mark Paluch
4c4cbed43b DATAES-685 - Prepare 3.2.2 (Moore SR2). 2019-11-18 12:31:53 +01:00
Mark Paluch
23fb5689b7 DATAES-685 - Updated changelog. 2019-11-18 12:31:52 +01:00
Mark Paluch
b551466f94 DATAES-683 - Updated changelog. 2019-11-18 12:16:28 +01:00
Peter-Josef Meisch
0a7af69d69 DATAES-680 - Fix missing import. 2019-11-08 17:03:37 +01:00
Peter-Josef Meisch
2cf42a4763 DATAES-680 - ReactiveElasticsearchTemplate-should-use-the-count-API.
Original PR: #341

(cherry picked from commit 62385edaa568e9e3ebbfc56ad59e76ffd72b47f8)
2019-11-07 17:07:39 +01:00
Peter-Josef Meisch
6f0d1ee9e7 DATAES-684 - Polishing.
(cherry picked from commit 24751972a8f266281e948eaf5f73f66003d4b896)
2019-11-05 10:41:21 +01:00
Henrique Amaral
f82dd229d9 DATAES-684 Implement bulk request from reactive client
Original PR: #342

* DATAES-684 Implement bulk request from reactive client

* Update src/main/java/org/springframework/data/elasticsearch/client/reactive/ReactiveElasticsearchClient.java

Co-Authored-By: Peter-Josef Meisch <pj.meisch@sothawo.com>

* DATAES-684 Implement bulk request from reactive client

Added author

(cherry picked from commit 6ae424428ccc45d72b80dcd3ddb7e3fbc6f32073)
2019-11-05 10:39:40 +01:00
Christoph Strobl
3b833f6f63 DATAES-662 - After release cleanups. 2019-11-04 15:34:27 +01:00
Christoph Strobl
2517fd5c90 DATAES-662 - Prepare next development iteration. 2019-11-04 15:34:26 +01:00
Christoph Strobl
4e572679dd DATAES-662 - Release version 3.2.1 (Moore SR1). 2019-11-04 14:55:02 +01:00
Christoph Strobl
de9c664d1e DATAES-662 - Prepare 3.2.1 (Moore SR1). 2019-11-04 14:54:16 +01:00
Christoph Strobl
8187c5362a DATAES-662 - Updated changelog. 2019-11-04 14:54:15 +01:00
Christoph Strobl
a8eb260bbd DATAES-660 - Updated changelog. 2019-11-04 10:34:54 +01:00
Peter-Josef Meisch
bb944f595f
DATAES-679 - Upgrade_to_Elasticsearch_6.8.4.
Original PR: #339
2019-10-31 17:40:24 +01:00
Henrique Amaral
598626238b DATAES-673 Create a Ssl Rest Client using SslContext and HostnameVerifier 2019-10-23 19:22:34 +02:00
Gyula Csörögi
a451f8dca4 DATAES-671 - Missing indicesOptions support for scrolling queries.
Original PR: #332
2019-10-16 09:00:44 +02:00
Peter-Josef Meisch
4fd070c332
DATAES-670 - Fix version compatibility matrix in documentation.
Original PR: #330
2019-10-12 10:42:27 +02:00
Mark Paluch
091413dd69 DATAES-665 - Fix plugin ordering to deploy Javadoc. 2019-10-01 08:39:10 +02:00
Greg Turnquist
ff74425132
DATAES-625 - Create CI job. 2019-09-30 14:23:41 -05:00
Mark Paluch
6f4d1dcc57 DATAES-625 - After release cleanups. 2019-09-30 16:17:54 +02:00
Mark Paluch
9713e33fed DATAES-625 - Prepare next development iteration. 2019-09-30 16:17:53 +02:00
303 changed files with 2108 additions and 622 deletions

View File

@ -1,22 +0,0 @@
language: java
jdk:
- oraclejdk8
before_install:
- curl -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-${ES_VERSION}.deb && sudo dpkg -i --force-confnew elasticsearch-${ES_VERSION}.deb && sudo service elasticsearch restart
- sleep 10
env:
global:
- ES_VERSION=6.5.1
addons:
apt:
packages:
- oracle-java8-set-default
- oracle-java8-installer
sudo: true
script: "mvn clean dependency:list test -Dsort"

8
Jenkinsfile vendored
View File

@ -3,7 +3,7 @@ pipeline {
triggers {
pollSCM 'H/10 * * * *'
upstream(upstreamProjects: "spring-data-commons/master", threshold: hudson.model.Result.SUCCESS)
upstream(upstreamProjects: "spring-data-commons/2.2.x", threshold: hudson.model.Result.SUCCESS)
}
options {
@ -15,7 +15,7 @@ pipeline {
stage("Test") {
when {
anyOf {
branch 'master'
branch '3.2.x'
not { triggeredBy 'UpstreamCause' }
}
}
@ -39,7 +39,7 @@ pipeline {
stage('Release to artifactory') {
when {
anyOf {
branch 'master'
branch '3.2.x'
not { triggeredBy 'UpstreamCause' }
}
}
@ -70,7 +70,7 @@ pipeline {
}
stage('Publish documentation') {
when {
branch 'master'
branch '3.2.x'
}
agent {
docker {

47
pom.xml
View File

@ -5,12 +5,12 @@
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-elasticsearch</artifactId>
<version>3.2.0.RELEASE</version>
<version>3.2.10.RELEASE</version>
<parent>
<groupId>org.springframework.data.build</groupId>
<artifactId>spring-data-parent</artifactId>
<version>2.2.0.RELEASE</version>
<version>2.2.10.RELEASE</version>
</parent>
<name>Spring Data Elasticsearch</name>
@ -19,9 +19,9 @@
<properties>
<commonslang>2.6</commonslang>
<elasticsearch>6.8.1</elasticsearch>
<elasticsearch>6.8.12</elasticsearch>
<log4j>2.9.1</log4j>
<springdata.commons>2.2.0.RELEASE</springdata.commons>
<springdata.commons>2.2.10.RELEASE</springdata.commons>
<netty>4.1.39.Final</netty>
<java-module-name>spring.data.elasticsearch</java-module-name>
</properties>
@ -248,6 +248,24 @@
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.github.tomakehurst</groupId>
<artifactId>wiremock-jre8</artifactId>
<version>2.25.1</version>
<scope>test</scope>
<exclusions>
<!-- these exclusions are needed because of Elasticsearch JarHell-->
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
<exclusion>
<groupId>org.ow2.asm</groupId>
<artifactId>asm</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- Upgrade xbean to 4.5 to prevent incompatibilities due to ASM versions -->
<dependency>
<groupId>org.apache.xbean</groupId>
@ -258,8 +276,8 @@
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>3.0-alpha-1</version>
<artifactId>javax.servlet-api</artifactId>
<version>3.1.0</version>
<scope>test</scope>
</dependency>
@ -267,14 +285,6 @@
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.asciidoctor</groupId>
<artifactId>asciidoctor-maven-plugin</artifactId>
</plugin>
<!--
please do not remove this configuration for surefire - we need that to avoid issue with jar hell
-->
@ -286,12 +296,21 @@
<useFile>false</useFile>
<includes>
<include>**/*Tests.java</include>
<include>**/*Test.java</include>
</includes>
<systemPropertyVariables>
<es.set.netty.runtime.available.processors>false</es.set.netty.runtime.available.processors>
</systemPropertyVariables>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.asciidoctor</groupId>
<artifactId>asciidoctor-maven-plugin</artifactId>
</plugin>
</plugins>
</build>

View File

@ -35,8 +35,8 @@ The following table shows the Elasticsearch versions that are used by Spring Dat
[cols="^,^,^,^",options="header"]
|===
|Spring Data Release Train |Spring Data Elasticsearch |Elasticsearch | Spring Boot
|Moorefootnote:cdv[Currently in development] |3.2.xfootnote:cdv[]|6.8.1 / 7.xfootnote:hrc[via the <<elasticsearch.clients.rest,high-level REST client>>]|2.2.0footnote:cdv[]
|Lovelace |3.1.x |6.2.2 / 7.xfootnote:hrc[]|2.1.x
|Moore |3.2.x |6.8.12 |2.2.x
|Lovelace |3.1.x |6.2.2|2.1.x
|Kayfootnote:oom[Out of maintenance]|3.0.xfootnote:oom[] |5.5.0 |2.0.xfootnote:oom[]
|Ingallsfootnote:oom[]|2.1.xfootnote:oom[] |2.4.0 |1.5.xfootnote:oom[]
|===

View File

@ -104,8 +104,15 @@ static class Config {
@Bean
ReactiveElasticsearchClient client() {
ClientConfiguration clientConfiguration = ClientConfiguration.builder() <1>
ClientConfiguration clientConfiguration = ClientConfiguration.builder() <1>
.connectedTo("localhost:9200", "localhost:9291")
.withWebClientConfigurer(webClient -> { <2>
ExchangeStrategies exchangeStrategies = ExchangeStrategies.builder()
.codecs(configurer -> configurer.defaultCodecs()
.maxInMemorySize(-1))
.build();
return webClient.mutate().exchangeStrategies(exchangeStrategies).build();
})
.build();
return ReactiveRestClients.create(clientConfiguration);
@ -124,6 +131,7 @@ Mono<IndexResponse> response = client.index(request ->
);
----
<1> Use the builder to provide cluster addresses, set default `HttpHeaders` or enable SSL.
<2> when configuring a reactive client, the `withWebClientConfigurer` hook can be used to customize the WebClient.
====
NOTE: The ReactiveClient response, especially for search operations, is bound to the `from` (offset) & `size` (limit) options of the request.

View File

@ -8,7 +8,7 @@ The Elasticsearch module supports all basic query building feature as string que
=== Declared queries
Deriving the query from the method name is not always sufficient and/or may result in unreadable method names. In this case one might make either use of `@Query` annotation (see <<elasticsearch.query-methods.at-query>> ).
Deriving the query from the method name is not always sufficient and/or may result in unreadable method names. In this case one might make use of the `@Query` annotation (see <<elasticsearch.query-methods.at-query>> ).
[[elasticsearch.query-methods.criterions]]
== Query creation
@ -29,12 +29,14 @@ The method name above will be translated into the following Elasticsearch json q
[source]
----
{ "bool" :
{ "must" :
[
{ "field" : {"name" : "?"} },
{ "field" : {"price" : "?"} }
]
{
"query": {
"bool" : {
"must" : [
{ "query_string" : { "query" : "?", "fields" : [ "name" ] } },
{ "query_string" : { "query" : "?", "fields" : [ "price" ] } }
]
}
}
}
----
@ -48,80 +50,184 @@ A list of supported keywords for Elasticsearch is shown below.
| Sample
| Elasticsearch Query String| `And`
| `findByNameAndPrice`
| `{"bool" : {"must" : [ {"field" : {"name" : "?"}},
{"field" : {"price" : "?"}} ]}}`
| `{ "query" : {
"bool" : {
"must" : [
{ "query_string" : { "query" : "?", "fields" : [ "name" ] } },
{ "query_string" : { "query" : "?", "fields" : [ "price" ] } }
]
}
}}`
| `Or`
| `findByNameOrPrice`
| `{"bool" : {"should" : [ {"field" : {"name" : "?"}},
{"field" : {"price" : "?"}} ]}}`
| `{ "query" : {
"bool" : {
"should" : [
{ "query_string" : { "query" : "?", "fields" : [ "name" ] } },
{ "query_string" : { "query" : "?", "fields" : [ "price" ] } }
]
}
}}`
| `Is`
| `findByName`
| `{"bool" : {"must" : {"field" : {"name" : "?"}}}}`
| `{ "query" : {
"bool" : {
"must" : [
{ "query_string" : { "query" : "?", "fields" : [ "name" ] } }
]
}
}}`
| `Not`
| `findByNameNot`
| `{"bool" : {"must_not" : {"field" : {"name" : "?"}}}}`
| `{ "query" : {
"bool" : {
"must_not" : [
{ "query_string" : { "query" : "?", "fields" : [ "name" ] } }
]
}
}}`
| `Between`
| `findByPriceBetween`
| `{"bool" : {"must" : {"range" : {"price" : {"from" :
?,"to" : ?,"include_lower" : true,"include_upper" : true}}}}}`
| `{ "query" : {
"bool" : {
"must" : [
{"range" : {"price" : {"from" : ?, "to" : ?, "include_lower" : true, "include_upper" : true } } }
]
}
}}`
| `LessThan`
| `findByPriceLessThan`
| `{ "query" : {
"bool" : {
"must" : [
{"range" : {"price" : {"from" : null, "to" : ?, "include_lower" : true, "include_upper" : false } } }
]
}
}}`
| `LessThanEqual`
| `findByPriceLessThan`
| `{"bool" : {"must" : {"range" : {"price" : {"from" :
null,"to" : ?,"include_lower" : true,"include_upper" :
true}}}}}`
| `findByPriceLessThanEqual`
| `{ "query" : {
"bool" : {
"must" : [
{"range" : {"price" : {"from" : null, "to" : ?, "include_lower" : true, "include_upper" : true } } }
]
}
}}`
| `GreaterThan`
| `findByPriceGreaterThan`
| `{ "query" : {
"bool" : {
"must" : [
{"range" : {"price" : {"from" : ?, "to" : null, "include_lower" : false, "include_upper" : true } } }
]
}
}}`
| `GreaterThanEqual`
| `findByPriceGreaterThan`
| `{"bool" : {"must" : {"range" : {"price" : {"from" :
?,"to" : null,"include_lower" : true,"include_upper" :
true}}}}}`
| `{ "query" : {
"bool" : {
"must" : [
{"range" : {"price" : {"from" : ?, "to" : null, "include_lower" : true, "include_upper" : true } } }
]
}
}}`
| `Before`
| `findByPriceBefore`
| `{"bool" : {"must" : {"range" : {"price" : {"from" :
null,"to" : ?,"include_lower" : true,"include_upper" :
true}}}}}`
| `{ "query" : {
"bool" : {
"must" : [
{"range" : {"price" : {"from" : null, "to" : ?, "include_lower" : true, "include_upper" : true } } }
]
}
}}`
| `After`
| `findByPriceAfter`
| `{"bool" : {"must" : {"range" : {"price" : {"from" :
?,"to" : null,"include_lower" : true,"include_upper" :
true}}}}}`
| `{ "query" : {
"bool" : {
"must" : [
{"range" : {"price" : {"from" : ?, "to" : null, "include_lower" : true, "include_upper" : true } } }
]
}
}}`
| `Like`
| `findByNameLike`
| `{"bool" : {"must" : {"field" : {"name" : {"query" :
"?*","analyze_wildcard" : true}}}}}`
| `{ "query" : {
"bool" : {
"must" : [
{ "query_string" : { "query" : "?*", "fields" : [ "name" ] }, "analyze_wildcard": true }
]
}
}}`
| `StartingWith`
| `findByNameStartingWith`
| `{"bool" : {"must" : {"field" : {"name" : {"query" :
"?*","analyze_wildcard" : true}}}}}`
| `{ "query" : {
"bool" : {
"must" : [
{ "query_string" : { "query" : "?*", "fields" : [ "name" ] }, "analyze_wildcard": true }
]
}
}}`
| `EndingWith`
| `findByNameEndingWith`
| `{"bool" : {"must" : {"field" : {"name" : {"query" :
"*?","analyze_wildcard" : true}}}}}`
| `{ "query" : {
"bool" : {
"must" : [
{ "query_string" : { "query" : "*?", "fields" : [ "name" ] }, "analyze_wildcard": true }
]
}
}}`
| `Contains/Containing`
| `findByNameContaining`
| `{"bool" : {"must" : {"field" : {"name" : {"query" :
"*?*","analyze_wildcard" : true}}}}}`
| `{ "query" : {
"bool" : {
"must" : [
{ "query_string" : { "query" : "\*?*", "fields" : [ "name" ] }, "analyze_wildcard": true }
]
}
}}`
| `In`
| `findByNameIn(Collection<String>names)`
| `{"bool" : {"must" : {"bool" : {"should" : [ {"field" :
{"name" : "?"}}, {"field" : {"name" : "?"}} ]}}}}`
| `{ "query" : {
"bool" : {
"must" : [
{"bool" : {"must" : [
{"terms" : {"name" : ["?","?"]}}
]
}
}
]
}
}}`
| `NotIn`
| `findByNameNotIn(Collection<String>names)`
| `{"bool" : {"must_not" : {"bool" : {"should" : {"field" :
{"name" : "?"}}}}}}`
| `{ "query" : {
"bool" : {
"must" : [
{"bool" : {"must_not" : [
{"terms" : {"name" : ["?","?"]}}
]
}
}
]
}
}}`
| `Near`
| `findByStoreNear`
@ -129,18 +235,45 @@ A list of supported keywords for Elasticsearch is shown below.
| `True`
| `findByAvailableTrue`
| `{"bool" : {"must" : {"field" : {"available" : true}}}}`
| `{ "query" : {
"bool" : {
"must" : [
{ "query_string" : { "query" : "true", "fields" : [ "available" ] } }
]
}
}}`
| `False`
| `findByAvailableFalse`
| `{"bool" : {"must" : {"field" : {"available" : false}}}}`
| `{ "query" : {
"bool" : {
"must" : [
{ "query_string" : { "query" : "false", "fields" : [ "available" ] } }
]
}
}}`
| `OrderBy`
| `findByAvailableTrueOrderByNameDesc`
| `{"sort" : [{ "name" : {"order" : "desc"} }],"bool" :
{"must" : {"field" : {"available" : true}}}}`
| `{ "query" : {
"bool" : {
"must" : [
{ "query_string" : { "query" : "true", "fields" : [ "available" ] } }
]
}
}, "sort":[{"name":{"order":"desc"}}]
}`
|===
== Method return types
Repository methods can be defined to have the following return types for returning multiple Elements:
* `List<T>`
* `Stream<T>`
* `AggregatedPage<T>`
[[elasticsearch.query-methods.at-query]]
== Using @Query Annotation
@ -149,8 +282,21 @@ A list of supported keywords for Elasticsearch is shown below.
[source,java]
----
interface BookRepository extends ElasticsearchRepository<Book, String> {
@Query("{\"bool\" : {\"must\" : {\"field\" : {\"name\" : \"?0\"}}}}")
@Query("{\"match\": {\"name\": {\"query\": \"?0\"}}}")
Page<Book> findByName(String name,Pageable pageable);
}
----
The String that is set as the annotation argument must be a valid Elasticsearch JSON query. It will be sent to Easticsearch as value of the query element; if for example the function is called with the parameter _John_, it would produce the following query body:
[source,json]
----
{
"query": {
"match": {
"name": {
"query": "John"
}
}
}
}
----
====

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2019 the original author or authors.
* Copyright 2019-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -20,16 +20,21 @@ import java.net.SocketAddress;
import java.time.Duration;
import java.util.List;
import java.util.Optional;
import java.util.function.Function;
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.SSLContext;
import org.springframework.http.HttpHeaders;
import org.springframework.web.reactive.function.client.WebClient;
/**
* Configuration interface exposing common client configuration properties for Elasticsearch clients.
*
* @author Mark Paluch
* @author Peter-Josef Meisch
* @author Huw Ayling-Miller
* @author Henrique Amaral
* @since 3.2
*/
public interface ClientConfiguration {
@ -118,6 +123,13 @@ public interface ClientConfiguration {
*/
Optional<SSLContext> getSslContext();
/**
* Returns the {@link HostnameVerifier} to use. Can be {@link Optional#empty()} if unconfigured.
*
* @return the {@link HostnameVerifier} to use. Can be {@link Optional#empty()} if unconfigured.
*/
Optional<HostnameVerifier> getHostNameVerifier();
/**
* Returns the {@link java.time.Duration connect timeout}.
*
@ -135,6 +147,27 @@ public interface ClientConfiguration {
*/
Duration getSocketTimeout();
/**
* Returns the path prefix that should be prepended to HTTP(s) requests for Elasticsearch behind a proxy.
*
* @return the path prefix.
* @since 3.2.4
*/
String getPathPrefix();
/**
* returns an optionally set proxy in the form host:port
*
* @return the optional proxy
* @since 3.2.4
*/
Optional<String> getProxy();
/**
* @return the function for configuring a WebClient.
*/
Function<WebClient, WebClient> getWebClientConfigurer();
/**
* @author Christoph Strobl
*/
@ -201,6 +234,16 @@ public interface ClientConfiguration {
* @return the {@link TerminalClientConfigurationBuilder}.
*/
TerminalClientConfigurationBuilder usingSsl(SSLContext sslContext);
/**
* Connect via {@literal https} using the givens {@link SSLContext} and HostnameVerifier {@link HostnameVerifier}
* .<br />
* <strong>NOTE</strong> You need to leave out the protocol in
* {@link ClientConfigurationBuilderWithRequiredEndpoint#connectedTo(String)}.
*
* @return the {@link TerminalClientConfigurationBuilder}.
*/
TerminalClientConfigurationBuilder usingSsl(SSLContext sslContext, HostnameVerifier hostnameVerifier);
}
/**
@ -267,6 +310,29 @@ public interface ClientConfiguration {
*/
TerminalClientConfigurationBuilder withBasicAuth(String username, String password);
/**
* Configure the path prefix that will be prepended to any HTTP(s) requests
*
* @param pathPrefix the pathPrefix.
* @return the {@link TerminalClientConfigurationBuilder}
* @since 3.2.4
*/
TerminalClientConfigurationBuilder withPathPrefix(String pathPrefix);
/**
* @param proxy a proxy formatted as String {@literal host:port}.
* @return the {@link TerminalClientConfigurationBuilder}.
*/
TerminalClientConfigurationBuilder withProxy(String proxy);
/**
* set customization hook in case of a reactive configuration
*
* @param webClientConfigurer function to configure the WebClient
* @return the {@link TerminalClientConfigurationBuilder}.
*/
TerminalClientConfigurationBuilder withWebClientConfigurer(Function<WebClient, WebClient> webClientConfigurer);
/**
* Build the {@link ClientConfiguration} object.
*

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -20,8 +20,10 @@ import java.time.Duration;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.function.Function;
import java.util.stream.Collectors;
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.SSLContext;
import org.springframework.data.elasticsearch.client.ClientConfiguration.ClientConfigurationBuilderWithRequiredEndpoint;
@ -30,6 +32,7 @@ import org.springframework.data.elasticsearch.client.ClientConfiguration.Termina
import org.springframework.http.HttpHeaders;
import org.springframework.lang.Nullable;
import org.springframework.util.Assert;
import org.springframework.web.reactive.function.client.WebClient;
/**
* Default builder implementation for {@link ClientConfiguration}.
@ -37,6 +40,8 @@ import org.springframework.util.Assert;
* @author Christoph Strobl
* @author Mark Paluch
* @author Peter-Josef Meisch
* @author Huw Ayling-Miller
* @author Henrique Amaral
* @since 3.2
*/
class ClientConfigurationBuilder
@ -46,10 +51,14 @@ class ClientConfigurationBuilder
private HttpHeaders headers = HttpHeaders.EMPTY;
private boolean useSsl;
private @Nullable SSLContext sslContext;
private @Nullable HostnameVerifier hostnameVerifier;
private Duration connectTimeout = Duration.ofSeconds(10);
private Duration soTimeout = Duration.ofSeconds(5);
private String username;
private String password;
private String pathPrefix;
private String proxy;
private Function<WebClient, WebClient> webClientConfigurer;
/*
* (non-Javadoc)
@ -78,6 +87,13 @@ class ClientConfigurationBuilder
return this;
}
@Override
public MaybeSecureClientConfigurationBuilder withProxy(String proxy) {
Assert.hasLength(proxy, "proxy must not be null or empty");
this.proxy = proxy;
return this;
}
/*
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.client.ClientConfiguration.MaybeSecureClientConfigurationBuilder#usingSsl()
@ -103,6 +119,22 @@ class ClientConfigurationBuilder
return this;
}
/*
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.client.ClientConfiguration.MaybeSecureClientConfigurationBuilder#usingSsl(javax.net.ssl.SSLContext, javax.net.ssl.HostnameVerifier)
*/
@Override
public TerminalClientConfigurationBuilder usingSsl(SSLContext sslContext, HostnameVerifier hostnameVerifier) {
Assert.notNull(sslContext, "SSL Context must not be null");
Assert.notNull(hostnameVerifier, "Host Name Verifier must not be null");
this.useSsl = true;
this.sslContext = sslContext;
this.hostnameVerifier = hostnameVerifier;
return this;
}
/*
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.client.ClientConfiguration.TerminalClientConfigurationBuilder#withDefaultHeaders(org.springframework.http.HttpHeaders)
@ -156,6 +188,22 @@ class ClientConfigurationBuilder
return this;
}
@Override
public TerminalClientConfigurationBuilder withPathPrefix(String pathPrefix) {
this.pathPrefix = pathPrefix;
return this;
}
@Override
public TerminalClientConfigurationBuilder withWebClientConfigurer(Function<WebClient, WebClient> webClientConfigurer) {
Assert.notNull(webClientConfigurer, "webClientConfigurer must not be null");
this.webClientConfigurer = webClientConfigurer;
return this;
}
/*
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.client.ClientConfiguration.ClientConfigurationBuilderWithOptionalDefaultHeaders#build()
@ -170,8 +218,8 @@ class ClientConfigurationBuilder
headers.setBasicAuth(username, password);
}
return new DefaultClientConfiguration(this.hosts, this.headers, this.useSsl, this.sslContext, this.soTimeout,
this.connectTimeout);
return new DefaultClientConfiguration(hosts, headers, useSsl, sslContext, soTimeout, connectTimeout, pathPrefix,
hostnameVerifier, proxy, webClientConfigurer);
}
private static InetSocketAddress parse(String hostAndPort) {

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -21,17 +21,22 @@ import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Optional;
import java.util.function.Function;
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.SSLContext;
import org.springframework.http.HttpHeaders;
import org.springframework.lang.Nullable;
import org.springframework.web.reactive.function.client.WebClient;
/**
* Default {@link ClientConfiguration} implementation.
*
* @author Mark Paluch
* @author Christoph Strobl
* @author Huw Ayling-Miller
* @author Peter-Josef Meisch
* @since 3.2
*/
class DefaultClientConfiguration implements ClientConfiguration {
@ -42,9 +47,14 @@ class DefaultClientConfiguration implements ClientConfiguration {
private final @Nullable SSLContext sslContext;
private final Duration soTimeout;
private final Duration connectTimeout;
private final String pathPrefix;
private final @Nullable HostnameVerifier hostnameVerifier;
private final String proxy;
private final Function<WebClient, WebClient> webClientConfigurer;
DefaultClientConfiguration(List<InetSocketAddress> hosts, HttpHeaders headers, boolean useSsl,
@Nullable SSLContext sslContext, Duration soTimeout, Duration connectTimeout) {
@Nullable SSLContext sslContext, Duration soTimeout, Duration connectTimeout, @Nullable String pathPrefix,
@Nullable HostnameVerifier hostnameVerifier, String proxy, Function<WebClient, WebClient> webClientConfigurer) {
this.hosts = Collections.unmodifiableList(new ArrayList<>(hosts));
this.headers = new HttpHeaders(headers);
@ -52,60 +62,59 @@ class DefaultClientConfiguration implements ClientConfiguration {
this.sslContext = sslContext;
this.soTimeout = soTimeout;
this.connectTimeout = connectTimeout;
this.pathPrefix = pathPrefix;
this.hostnameVerifier = hostnameVerifier;
this.proxy = proxy;
this.webClientConfigurer = webClientConfigurer;
}
/*
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.client.ClientConfiguration#getEndpoints()
*/
@Override
public List<InetSocketAddress> getEndpoints() {
return this.hosts;
}
/*
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.client.ClientConfiguration#getDefaultHeaders()
*/
@Override
public HttpHeaders getDefaultHeaders() {
return this.headers;
}
/*
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.client.ClientConfiguration#useSsl()
*/
@Override
public boolean useSsl() {
return this.useSsl;
}
/*
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.client.ClientConfiguration#getSslContext()
*/
@Override
public Optional<SSLContext> getSslContext() {
return Optional.ofNullable(this.sslContext);
}
/*
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.client.ClientConfiguration#getConnectTimeout()
*/
@Override
public Optional<HostnameVerifier> getHostNameVerifier() {
return Optional.ofNullable(this.hostnameVerifier);
}
@Override
public Duration getConnectTimeout() {
return this.connectTimeout;
}
/*
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.client.ClientConfiguration#getSocketTimeout()
*/
@Override
public Duration getSocketTimeout() {
return this.soTimeout;
}
@Override
public String getPathPrefix() {
return this.pathPrefix;
}
@Override
public Optional<String> getProxy() {
return Optional.ofNullable(proxy);
}
@Override
public Function<WebClient, WebClient> getWebClientConfigurer() {
return webClientConfigurer != null ? webClientConfigurer : Function.identity();
}
}

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2015-2019 the original author or authors.
* Copyright 2015-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -24,6 +24,7 @@ import java.util.List;
import java.util.Optional;
import java.util.stream.Collectors;
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.SSLContext;
import org.apache.http.Header;
@ -52,6 +53,9 @@ import org.springframework.util.Assert;
*
* @author Christoph Strobl
* @author Mark Paluch
* @author Huw Ayling-Miller
* @author Henrique Amaral
* @author Peter-Josef Meisch
* @since 3.2
*/
public final class RestClients {
@ -75,6 +79,11 @@ public final class RestClients {
HttpHost[] httpHosts = formattedHosts(clientConfiguration.getEndpoints(), clientConfiguration.useSsl()).stream()
.map(HttpHost::create).toArray(HttpHost[]::new);
RestClientBuilder builder = RestClient.builder(httpHosts);
if (clientConfiguration.getPathPrefix() != null) {
builder.setPathPrefix(clientConfiguration.getPathPrefix());
}
HttpHeaders headers = clientConfiguration.getDefaultHeaders();
if (!headers.isEmpty()) {
@ -87,7 +96,9 @@ public final class RestClients {
builder.setHttpClientConfigCallback(clientBuilder -> {
Optional<SSLContext> sslContext = clientConfiguration.getSslContext();
Optional<HostnameVerifier> hostNameVerifier = clientConfiguration.getHostNameVerifier();
sslContext.ifPresent(clientBuilder::setSSLContext);
hostNameVerifier.ifPresent(clientBuilder::setSSLHostnameVerifier);
if (ClientLogger.isEnabled()) {
@ -114,6 +125,8 @@ public final class RestClients {
clientBuilder.setDefaultRequestConfig(requestConfigBuilder.build());
clientConfiguration.getProxy().map(HttpHost::create).ifPresent(clientBuilder::setProxy);
return clientBuilder;
});
@ -122,7 +135,8 @@ public final class RestClients {
}
private static List<String> formattedHosts(List<InetSocketAddress> hosts, boolean useSsl) {
return hosts.stream().map(it -> (useSsl ? "https" : "http") + "://" + it).collect(Collectors.toList());
return hosts.stream().map(it -> (useSsl ? "https" : "http") + "://" + it.getHostString() + ':' + it.getPort())
.collect(Collectors.toList());
}
/**

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -16,7 +16,9 @@
package org.springframework.data.elasticsearch.client.reactive;
import io.netty.channel.ChannelOption;
import io.netty.handler.ssl.ApplicationProtocolConfig;
import io.netty.handler.ssl.ClientAuth;
import io.netty.handler.ssl.IdentityCipherSuiteFilter;
import io.netty.handler.ssl.JdkSslContext;
import io.netty.handler.timeout.ReadTimeoutHandler;
import io.netty.handler.timeout.WriteTimeoutHandler;
@ -45,9 +47,9 @@ import java.util.function.Function;
import javax.net.ssl.SSLContext;
import org.apache.http.util.EntityUtils;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.ElasticsearchStatusException;
import org.elasticsearch.action.ActionRequest;
import org.elasticsearch.action.ActionResponse;
import org.elasticsearch.action.admin.indices.close.CloseIndexRequest;
import org.elasticsearch.action.admin.indices.create.CreateIndexRequest;
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
@ -58,6 +60,8 @@ import org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest;
import org.elasticsearch.action.admin.indices.open.OpenIndexRequest;
import org.elasticsearch.action.admin.indices.refresh.RefreshRequest;
import org.elasticsearch.action.admin.indices.refresh.RefreshResponse;
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.bulk.BulkResponse;
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.action.get.GetRequest;
@ -77,6 +81,8 @@ import org.elasticsearch.action.support.master.AcknowledgedResponse;
import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.action.update.UpdateResponse;
import org.elasticsearch.client.Request;
import org.elasticsearch.client.core.CountRequest;
import org.elasticsearch.client.core.CountResponse;
import org.elasticsearch.common.unit.TimeValue;
import org.elasticsearch.common.xcontent.DeprecationHandler;
import org.elasticsearch.common.xcontent.NamedXContentRegistry;
@ -91,7 +97,6 @@ import org.elasticsearch.search.Scroll;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.reactivestreams.Publisher;
import org.springframework.data.elasticsearch.ElasticsearchException;
import org.springframework.data.elasticsearch.client.ClientConfiguration;
import org.springframework.data.elasticsearch.client.ClientLogger;
import org.springframework.data.elasticsearch.client.ElasticsearchHost;
@ -110,7 +115,6 @@ import org.springframework.util.Assert;
import org.springframework.util.ObjectUtils;
import org.springframework.util.ReflectionUtils;
import org.springframework.util.StringUtils;
import org.springframework.web.client.HttpServerErrorException;
import org.springframework.web.reactive.function.BodyExtractors;
import org.springframework.web.reactive.function.client.ClientRequest;
import org.springframework.web.reactive.function.client.ClientResponse;
@ -122,6 +126,9 @@ import org.springframework.web.reactive.function.client.WebClient.RequestBodySpe
*
* @author Christoph Strobl
* @author Mark Paluch
* @author Peter-Josef Meisch
* @author Huw Ayling-Miller
* @author Henrique Amaral
* @since 3.2
* @see ClientConfiguration
* @see ReactiveRestClients
@ -193,7 +200,6 @@ public class DefaultReactiveElasticsearchClient implements ReactiveElasticsearch
}
if (!soTimeout.isNegative()) {
tcpClient = tcpClient.doOnConnected(connection -> connection //
.addHandlerLast(new ReadTimeoutHandler(soTimeout.toMillis(), TimeUnit.MILLISECONDS))
.addHandlerLast(new WriteTimeoutHandler(soTimeout.toMillis(), TimeUnit.MILLISECONDS)));
@ -204,11 +210,16 @@ public class DefaultReactiveElasticsearchClient implements ReactiveElasticsearch
if (clientConfiguration.useSsl()) {
httpClient = httpClient.secure(sslConfig -> {
Optional<SSLContext> sslContext = clientConfiguration.getSslContext();
Optional<SSLContext> sslContext = clientConfiguration.getSslContext();
sslContext.ifPresent(it -> sslConfig.sslContext(new JdkSslContext(it, true, ClientAuth.NONE)));
});
if (sslContext.isPresent()) {
httpClient = httpClient.secure(sslContextSpec -> {
sslContextSpec.sslContext(new JdkSslContext(sslContext.get(), true, null, IdentityCipherSuiteFilter.INSTANCE,
ApplicationProtocolConfig.DISABLED, ClientAuth.NONE, null, false));
});
} else {
httpClient = httpClient.secure();
}
scheme = "https";
}
@ -216,7 +227,13 @@ public class DefaultReactiveElasticsearchClient implements ReactiveElasticsearch
ReactorClientHttpConnector connector = new ReactorClientHttpConnector(httpClient);
WebClientProvider provider = WebClientProvider.create(scheme, connector);
return provider.withDefaultHeaders(clientConfiguration.getDefaultHeaders());
if (clientConfiguration.getPathPrefix() != null) {
provider = provider.withPathPrefix(clientConfiguration.getPathPrefix());
}
provider = provider.withDefaultHeaders(clientConfiguration.getDefaultHeaders()) //
.withWebClientConfigurer(clientConfiguration.getWebClientConfigurer());
return provider;
}
/*
@ -319,6 +336,17 @@ public class DefaultReactiveElasticsearchClient implements ReactiveElasticsearch
.publishNext();
}
/*
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.client.reactive.ReactiveElasticsearchClient#count(org.springframework.http.HttpHeaders, org.elasticsearch.action.search.SearchRequest)
*/
@Override
public Mono<Long> count(HttpHeaders headers, CountRequest countRequest) {
return sendRequest(countRequest, RequestCreator.count(), CountResponse.class, headers) //
.map(CountResponse::getCount) //
.next();
}
/*
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.client.reactive.ReactiveElasticsearchClient#ping(org.springframework.http.HttpHeaders, org.elasticsearch.action.search.SearchRequest)
@ -417,12 +445,23 @@ public class DefaultReactiveElasticsearchClient implements ReactiveElasticsearch
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.client.reactive.ReactiveElasticsearchClient#ping(org.springframework.http.HttpHeaders, org.elasticsearch.index.reindex.DeleteByQueryRequest)
*/
@Override
public Mono<BulkByScrollResponse> deleteBy(HttpHeaders headers, DeleteByQueryRequest deleteRequest) {
return sendRequest(deleteRequest, RequestCreator.deleteByQuery(), BulkByScrollResponse.class, headers) //
.publishNext();
}
/*
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.client.reactive.ReactiveElasticsearchClient#bulk(org.springframework.http.HttpHeaders, org.elasticsearch.action.bulk.BulkRequest)
*/
@Override
public Mono<BulkResponse> bulk(HttpHeaders headers, BulkRequest bulkRequest) {
return sendRequest(bulkRequest, RequestCreator.bulk(), BulkResponse.class, headers) //
.publishNext();
}
// --> INDICES
/*
@ -553,13 +592,12 @@ public class DefaultReactiveElasticsearchClient implements ReactiveElasticsearch
// -->
private <Req extends ActionRequest, Resp extends ActionResponse> Flux<Resp> sendRequest(Req request,
Function<Req, Request> converter, Class<Resp> responseType, HttpHeaders headers) {
private <Req extends ActionRequest, Resp> Flux<Resp> sendRequest(Req request, Function<Req, Request> converter,
Class<Resp> responseType, HttpHeaders headers) {
return sendRequest(converter.apply(request), responseType, headers);
}
private <AR extends ActionResponse> Flux<AR> sendRequest(Request request, Class<AR> responseType,
HttpHeaders headers) {
private <Resp> Flux<Resp> sendRequest(Request request, Class<Resp> responseType, HttpHeaders headers) {
String logId = ClientLogger.newLogId();
@ -641,6 +679,12 @@ public class DefaultReactiveElasticsearchClient implements ReactiveElasticsearch
return handleServerError(request, response);
}
if (response.statusCode().is4xxClientError()) {
ClientLogger.logRawResponse(logId, response.statusCode());
return handleClientError(logId, request, response, responseType);
}
return response.body(BodyExtractors.toMono(byte[].class)) //
.map(it -> new String(it, StandardCharsets.UTF_8)) //
.doOnNext(it -> ClientLogger.logResponse(logId, response.statusCode(), it)) //
@ -677,13 +721,68 @@ public class DefaultReactiveElasticsearchClient implements ReactiveElasticsearch
.createParser(NamedXContentRegistry.EMPTY, DeprecationHandler.THROW_UNSUPPORTED_OPERATION, content);
}
private static <T> Publisher<? extends T> handleServerError(Request request, ClientResponse response) {
private <T> Publisher<? extends T> handleServerError(Request request, ClientResponse response) {
return Mono.error(
new HttpServerErrorException(response.statusCode(), String.format("%s request to %s returned error code %s.",
request.getMethod(), request.getEndpoint(), response.statusCode().value())));
RestStatus status = RestStatus.fromCode(response.statusCode().value());
return Mono.error(new ElasticsearchStatusException(String.format("%s request to %s returned error code %s.",
request.getMethod(), request.getEndpoint(), response.statusCode().value()), status));
}
private <T> Publisher<? extends T> handleClientError(String logId, Request request, ClientResponse response,
Class<T> responseType) {
return response.body(BodyExtractors.toMono(byte[].class)) //
.map(bytes -> new String(bytes, StandardCharsets.UTF_8)) //
.flatMap(content -> {
String mediaType = response.headers().contentType().map(MediaType::toString)
.orElse(XContentType.JSON.mediaType());
RestStatus status = RestStatus.fromCode(response.statusCode().value());
try {
ElasticsearchException exception = getElasticsearchException(response, content, mediaType);
if (exception != null) {
StringBuilder sb = new StringBuilder();
buildExceptionMessages(sb, exception);
return Mono.error(new ElasticsearchStatusException(sb.toString(), status, exception));
}
} catch (Exception e) {
return Mono.error(new ElasticsearchStatusException(content, status));
}
return Mono.just(content);
}).doOnNext(it -> ClientLogger.logResponse(logId, response.statusCode(), it)) //
.flatMap(content -> doDecode(response, responseType, content));
}
// region ElasticsearchException helper
@Nullable
private ElasticsearchException getElasticsearchException(ClientResponse response, String content, String mediaType)
throws IOException {
XContentParser parser = createParser(mediaType, content);
// we have a JSON object with an error and a status field
XContentParser.Token token = parser.nextToken(); // Skip START_OBJECT
do {
token = parser.nextToken();
if (parser.currentName().equals("error")) {
return ElasticsearchException.failureFromXContent(parser);
}
} while (token == XContentParser.Token.FIELD_NAME);
return null;
}
private static void buildExceptionMessages(StringBuilder sb, Throwable t) {
sb.append(t.getMessage());
for (Throwable throwable : t.getSuppressed()) {
sb.append(", ");
buildExceptionMessages(sb, throwable);
}
}
// endregion
// region internal classes
static class RequestCreator {
static Function<SearchRequest, Request> search() {
@ -737,7 +836,19 @@ public class DefaultReactiveElasticsearchClient implements ReactiveElasticsearch
try {
return RequestConverters.deleteByQuery(request);
} catch (IOException e) {
throw new ElasticsearchException("Could not parse request", e);
throw new org.springframework.data.elasticsearch.ElasticsearchException("Could not parse request", e);
}
};
}
static Function<BulkRequest, Request> bulk() {
return request -> {
try {
return RequestConverters.bulk(request);
} catch (IOException e) {
throw new org.springframework.data.elasticsearch.ElasticsearchException("Could not parse request", e);
}
};
}
@ -776,6 +887,10 @@ public class DefaultReactiveElasticsearchClient implements ReactiveElasticsearch
return RequestConverters::flushIndex;
}
static Function<CountRequest, Request> count() {
return RequestConverters::count;
}
}
/**
@ -837,4 +952,5 @@ public class DefaultReactiveElasticsearchClient implements ReactiveElasticsearch
}
}
}
// endregion
}

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -19,6 +19,7 @@ import java.net.InetSocketAddress;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Consumer;
import java.util.function.Function;
import org.springframework.http.HttpHeaders;
import org.springframework.http.client.reactive.ClientHttpConnector;
@ -32,6 +33,8 @@ import org.springframework.web.reactive.function.client.WebClient.Builder;
*
* @author Mark Paluch
* @author Christoph Strobl
* @author Huw Ayling-Miller
* @author Peter-Josef Meisch
* @since 3.2
*/
class DefaultWebClientProvider implements WebClientProvider {
@ -42,6 +45,8 @@ class DefaultWebClientProvider implements WebClientProvider {
private final @Nullable ClientHttpConnector connector;
private final Consumer<Throwable> errorListener;
private final HttpHeaders headers;
private final String pathPrefix;
private final Function<WebClient, WebClient> webClientConfigurer;
/**
* Create new {@link DefaultWebClientProvider} with empty {@link HttpHeaders} and no-op {@literal error listener}.
@ -50,35 +55,38 @@ class DefaultWebClientProvider implements WebClientProvider {
* @param connector can be {@literal null}.
*/
DefaultWebClientProvider(String scheme, @Nullable ClientHttpConnector connector) {
this(scheme, connector, e -> {}, HttpHeaders.EMPTY);
this(scheme, connector, e -> {}, HttpHeaders.EMPTY, null, Function.identity());
}
/**
* Create new {@link DefaultWebClientProvider} with empty {@link HttpHeaders} and no-op {@literal error listener}.
*
*
* @param scheme must not be {@literal null}.
* @param connector can be {@literal null}.
* @param errorListener must not be {@literal null}.
* @param headers must not be {@literal null}.
* @param connector can be {@literal null}.
* @param errorListener must not be {@literal null}.
* @param headers must not be {@literal null}.
* @param pathPrefix can be {@literal null}
* @param webClientConfigurer must not be {@literal null}.
*/
private DefaultWebClientProvider(String scheme, @Nullable ClientHttpConnector connector,
Consumer<Throwable> errorListener, HttpHeaders headers) {
Consumer<Throwable> errorListener, HttpHeaders headers, @Nullable String pathPrefix,
Function<WebClient, WebClient> webClientConfigurer) {
Assert.notNull(scheme, "Scheme must not be null! A common scheme would be 'http'.");
Assert.notNull(errorListener, "ErrorListener must not be null! You may want use a no-op one 'e -> {}' instead.");
Assert.notNull(errorListener, "errorListener must not be null! You may want use a no-op one 'e -> {}' instead.");
Assert.notNull(headers, "headers must not be null! Think about using 'HttpHeaders.EMPTY' as an alternative.");
Assert.notNull(webClientConfigurer,
"webClientConfigurer must not be null! You may want use a no-op one 'Function.identity()' instead.");
this.cachedClients = new ConcurrentHashMap<>();
this.scheme = scheme;
this.connector = connector;
this.errorListener = errorListener;
this.headers = headers;
this.pathPrefix = pathPrefix;
this.webClientConfigurer = webClientConfigurer;
}
/*
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.client.reactive.WebClientProvider#get(java.net.InetSocketAddress)
*/
@Override
public WebClient get(InetSocketAddress endpoint) {
@ -87,19 +95,21 @@ class DefaultWebClientProvider implements WebClientProvider {
return this.cachedClients.computeIfAbsent(endpoint, this::createWebClientForSocketAddress);
}
/*
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.client.reactive.WebClientProvider#getDefaultHeaders()
*/
@Override
public HttpHeaders getDefaultHeaders() {
return headers;
}
/*
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.client.reactive.WebClientProvider#withDefaultHeaders(org.springframework.http.HttpHeaders)
*/
@Override
public Consumer<Throwable> getErrorListener() {
return this.errorListener;
}
@Override
public String getPathPrefix() {
return pathPrefix;
}
@Override
public WebClientProvider withDefaultHeaders(HttpHeaders headers) {
@ -109,31 +119,33 @@ class DefaultWebClientProvider implements WebClientProvider {
merged.addAll(this.headers);
merged.addAll(headers);
return new DefaultWebClientProvider(this.scheme, this.connector, errorListener, merged);
return new DefaultWebClientProvider(scheme, connector, errorListener, merged, pathPrefix, webClientConfigurer);
}
/*
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.client.reactive.WebClientProvider#getErrorListener()
*/
@Override
public Consumer<Throwable> getErrorListener() {
return this.errorListener;
}
/*
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.client.reactive.WebClientProvider#withErrorListener(java.util.function.Consumer)
*/
@Override
public WebClientProvider withErrorListener(Consumer<Throwable> errorListener) {
Assert.notNull(errorListener, "Error listener must not be null.");
Consumer<Throwable> listener = this.errorListener.andThen(errorListener);
return new DefaultWebClientProvider(this.scheme, this.connector, listener, this.headers);
return new DefaultWebClientProvider(scheme, this.connector, listener, headers, pathPrefix, webClientConfigurer);
}
@Override
public WebClientProvider withPathPrefix(String pathPrefix) {
Assert.notNull(pathPrefix, "pathPrefix must not be null.");
return new DefaultWebClientProvider(this.scheme, this.connector, this.errorListener, this.headers, pathPrefix,
webClientConfigurer);
}
@Override
public WebClientProvider withWebClientConfigurer(Function<WebClient, WebClient> webClientConfigurer) {
return new DefaultWebClientProvider(scheme, connector, errorListener, headers, pathPrefix, webClientConfigurer);
}
protected WebClient createWebClientForSocketAddress(InetSocketAddress socketAddress) {
Builder builder = WebClient.builder().defaultHeaders(it -> it.addAll(getDefaultHeaders()));
@ -142,7 +154,9 @@ class DefaultWebClientProvider implements WebClientProvider {
builder = builder.clientConnector(connector);
}
String baseUrl = String.format("%s://%s:%d", this.scheme, socketAddress.getHostString(), socketAddress.getPort());
return builder.baseUrl(baseUrl).filter((request, next) -> next.exchange(request).doOnError(errorListener)).build();
String baseUrl = String.format("%s://%s:%d%s", this.scheme, socketAddress.getHostString(), socketAddress.getPort(),
pathPrefix == null ? "" : '/' + pathPrefix);
WebClient webClient = builder.baseUrl(baseUrl).filter((request, next) -> next.exchange(request).doOnError(errorListener)).build();
return webClientConfigurer.apply(webClient);
}
}

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -30,6 +30,8 @@ import org.elasticsearch.action.admin.indices.get.GetIndexRequest;
import org.elasticsearch.action.admin.indices.mapping.put.PutMappingRequest;
import org.elasticsearch.action.admin.indices.open.OpenIndexRequest;
import org.elasticsearch.action.admin.indices.refresh.RefreshRequest;
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.bulk.BulkResponse;
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.action.get.GetRequest;
@ -41,11 +43,11 @@ import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.action.update.UpdateResponse;
import org.elasticsearch.client.core.CountRequest;
import org.elasticsearch.index.get.GetResult;
import org.elasticsearch.index.reindex.BulkByScrollResponse;
import org.elasticsearch.index.reindex.DeleteByQueryRequest;
import org.elasticsearch.search.SearchHit;
import org.springframework.data.elasticsearch.client.ClientConfiguration;
import org.springframework.data.elasticsearch.client.ElasticsearchHost;
import org.springframework.http.HttpHeaders;
@ -58,6 +60,8 @@ import org.springframework.web.reactive.function.client.WebClient;
*
* @author Christoph Strobl
* @author Mark Paluch
* @author Peter-Josef Meisch
* @author Henrique Amaral
* @since 3.2
* @see ClientConfiguration
* @see ReactiveRestClients
@ -329,6 +333,47 @@ public interface ReactiveElasticsearchClient {
*/
Mono<DeleteResponse> delete(HttpHeaders headers, DeleteRequest deleteRequest);
/**
* Execute a {@link SearchRequest} against the {@literal count} API.
*
* @param consumer new {@literal null}.
* @see <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/search-count.html">Count API on
* elastic.co</a>
* @return the {@link Mono} emitting the count result.
* @since 3.2.4
*/
default Mono<Long> count(Consumer<CountRequest> consumer) {
CountRequest countRequest = new CountRequest();
consumer.accept(countRequest);
return count(countRequest);
}
/**
* Execute a {@link SearchRequest} against the {@literal count} API.
*
* @param countRequest must not be {@literal null}.
* @see <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/search-count.html">Count API on
* elastic.co</a>
* @return the {@link Mono} emitting the count result.
* @since 3.2.4
*/
default Mono<Long> count(CountRequest countRequest) {
return count(HttpHeaders.EMPTY, countRequest);
}
/**
* Execute a {@link SearchRequest} against the {@literal count} API.
*
* @param headers Use {@link HttpHeaders} to provide eg. authentication data. Must not be {@literal null}.
* @param countRequest must not be {@literal null}.
* @see <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/search-count.html">Count API on
* elastic.co</a>
* @return the {@link Mono} emitting the count result.
* @since 3.2.4
*/
Mono<Long> count(HttpHeaders headers, CountRequest countRequest);
/**
* Execute a {@link SearchRequest} against the {@literal search} API.
*
@ -430,6 +475,44 @@ public interface ReactiveElasticsearchClient {
*/
Mono<BulkByScrollResponse> deleteBy(HttpHeaders headers, DeleteByQueryRequest deleteRequest);
/**
* Execute a {@link BulkRequest} against the {@literal bulk} API.
*
* @param consumer never {@literal null}.
* @see <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html">Bulk API on
* elastic.co</a>
* @return a {@link Mono} emitting the emitting operation response.
*/
default Mono<BulkResponse> bulk(Consumer<BulkRequest> consumer) {
BulkRequest request = new BulkRequest();
consumer.accept(request);
return bulk(request);
}
/**
* Execute a {@link BulkRequest} against the {@literal bulk} API.
*
* @param bulkRequest must not be {@literal null}.
* @see <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html">Bulk API on
* elastic.co</a>
* @return a {@link Mono} emitting the emitting operation response.
*/
default Mono<BulkResponse> bulk(BulkRequest bulkRequest) {
return bulk(HttpHeaders.EMPTY, bulkRequest);
}
/**
* Execute a {@link BulkRequest} against the {@literal bulk} API.
*
* @param headers Use {@link HttpHeaders} to provide eg. authentication data. Must not be {@literal null}.
* @param bulkRequest must not be {@literal null}.
* @see <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html">Bulk API on
* elastic.co</a>
* @return a {@link Mono} emitting operation response.
*/
Mono<BulkResponse> bulk(HttpHeaders headers, BulkRequest bulkRequest);
/**
* Compose the actual command/s to run against Elasticsearch using the underlying {@link WebClient connection}.
* {@link #execute(ReactiveElasticsearchClientCallback) Execute} selects an active server from the available ones and

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -17,6 +17,7 @@ package org.springframework.data.elasticsearch.client.reactive;
import java.net.InetSocketAddress;
import java.util.function.Consumer;
import java.util.function.Function;
import org.springframework.http.HttpHeaders;
import org.springframework.http.client.reactive.ClientHttpConnector;
@ -34,6 +35,8 @@ import org.springframework.web.reactive.function.client.WebClient;
*
* @author Christoph Strobl
* @author Mark Paluch
* @author Huw Ayling-Miller
* @author Peter-Josef Meisch
* @since 3.2
*/
public interface WebClientProvider {
@ -96,6 +99,14 @@ public interface WebClientProvider {
*/
Consumer<Throwable> getErrorListener();
/**
* Obtain the {@link String pathPrefix} to be used.
*
* @return the pathPrefix if set.
* @since 3.2.4
*/
String getPathPrefix();
/**
* Create a new instance of {@link WebClientProvider} applying the given headers by default.
*
@ -111,4 +122,21 @@ public interface WebClientProvider {
* @return new instance of {@link WebClientProvider}.
*/
WebClientProvider withErrorListener(Consumer<Throwable> errorListener);
/**
* Create a new instance of {@link WebClientProvider} where HTTP requests are called with the given path prefix.
*
* @param pathPrefix Path prefix to add to requests
* @return new instance of {@link WebClientProvider}
* @since 3.2.4
*/
WebClientProvider withPathPrefix(String pathPrefix);
/**
* Create a new instance of {@link WebClientProvider} calling the given {@link Function} to configure the {@link WebClient}.
* @param webClientConfigurer configuration function
* @return new instance of {@link WebClientProvider}
* @since 3.2.4
*/
WebClientProvider withWebClientConfigurer(Function<WebClient, WebClient> webClientConfigurer);
}

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -60,6 +60,7 @@ import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.client.Request;
import org.elasticsearch.client.Requests;
import org.elasticsearch.client.RethrottleRequest;
import org.elasticsearch.client.core.CountRequest;
import org.elasticsearch.cluster.health.ClusterHealthStatus;
import org.elasticsearch.common.Priority;
import org.elasticsearch.common.Strings;
@ -98,6 +99,8 @@ import org.springframework.lang.Nullable;
* <p>
* Only intended for internal use.
*
* @author Christoph Strobl
* @author Peter-Josef Meisch
* @since 3.2
*/
public class RequestConverters {
@ -386,6 +389,32 @@ public class RequestConverters {
return request;
}
/**
* Creates a count request.
*
* @param countRequest the search defining the data to be counted
* @return Elasticsearch count request
* @since 3.2
*/
public static Request count(CountRequest countRequest) {
Request request = new Request(HttpMethod.POST.name(),
endpoint(countRequest.indices(), countRequest.types(), "_count"));
Params params = new Params(request);
addCountRequestParams(params, countRequest);
if (countRequest.source() != null) {
request.setEntity(createEntity(countRequest.source(), REQUEST_BODY_CONTENT_TYPE));
}
return request;
}
private static void addCountRequestParams(Params params, CountRequest countRequest) {
params.withRouting(countRequest.routing());
params.withPreference(countRequest.preference());
params.withIndicesOptions(countRequest.indicesOptions());
}
private static void addSearchRequestParams(Params params, SearchRequest searchRequest) {
params.putParam("typed_keys", "true");
params.withRouting(searchRequest.routing());

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2015-2019 the original author or authors.
* Copyright 2015-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -15,6 +15,8 @@ import org.springframework.util.StringUtils;
*/
public abstract class AbstractElasticsearchTemplate {
static final Integer INDEX_MAX_RESULT_WINDOW = 10_000;
private static final Logger LOGGER = LoggerFactory.getLogger(AbstractElasticsearchTemplate.class);
protected ElasticsearchConverter elasticsearchConverter;

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2019 the original author or authors.
* Copyright 2019-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -15,8 +15,6 @@
*/
package org.springframework.data.elasticsearch.core;
import com.fasterxml.jackson.databind.ObjectReader;
import com.fasterxml.jackson.databind.ObjectWriter;
import lombok.RequiredArgsConstructor;
import java.io.IOException;
@ -52,6 +50,8 @@ import org.springframework.util.ClassUtils;
import org.springframework.util.ObjectUtils;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.ObjectReader;
import com.fasterxml.jackson.databind.ObjectWriter;
/**
* Elasticsearch specific {@link EntityReader} & {@link EntityWriter} implementation based on domain type
@ -241,27 +241,31 @@ public class ElasticsearchEntityMapper implements
Map<String, Object> target = new LinkedHashMap<>();
for (Entry<String, Object> entry : source.entrySet()) {
if (isSimpleType(entry.getValue())) {
target.put(entry.getKey(),
readSimpleValue(entry.getValue(), targetType.isMap() ? targetType.getComponentType() : targetType));
String entryKey = entry.getKey();
Object entryValue = entry.getValue();
if (entryValue == null) {
target.put(entryKey, null);
} else if (isSimpleType(entryValue)) {
target.put(entryKey,
readSimpleValue(entryValue, targetType.isMap() ? targetType.getComponentType() : targetType));
} else {
ElasticsearchPersistentEntity<?> targetEntity = computeGenericValueTypeForRead(property, entry.getValue());
ElasticsearchPersistentEntity<?> targetEntity = computeGenericValueTypeForRead(property, entryValue);
if (targetEntity.getTypeInformation().isMap()) {
Map<String, Object> valueMap = (Map) entry.getValue();
Map<String, Object> valueMap = (Map) entryValue;
if (typeMapper.containsTypeInformation(valueMap)) {
target.put(entry.getKey(), readEntity(targetEntity, (Map) entry.getValue()));
target.put(entryKey, readEntity(targetEntity, (Map) entryValue));
} else {
target.put(entry.getKey(), readValue(valueMap, property, targetEntity.getTypeInformation()));
target.put(entryKey, readValue(valueMap, property, targetEntity.getTypeInformation()));
}
} else if (targetEntity.getTypeInformation().isCollectionLike()) {
target.put(entry.getKey(),
readValue(entry.getValue(), property, targetEntity.getTypeInformation().getActualType()));
target.put(entryKey, readValue(entryValue, property, targetEntity.getTypeInformation().getActualType()));
} else {
target.put(entry.getKey(), readEntity(targetEntity, (Map) entry.getValue()));
target.put(entryKey, readEntity(targetEntity, (Map) entryValue));
}
}
}
@ -281,13 +285,18 @@ public class ElasticsearchEntityMapper implements
for (Object value : source) {
if (isSimpleType(value)) {
if (value == null) {
target.add(null);
} else if (isSimpleType(value)) {
target.add(
readSimpleValue(value, targetType.getComponentType() != null ? targetType.getComponentType() : targetType));
} else {
if (value instanceof List) {
target.add(readValue(value, property, property.getTypeInformation().getActualType()));
} else if (value instanceof Map) {
target
.add(readMapValue((Map<String, Object>) value, property, property.getTypeInformation().getActualType()));
} else {
target.add(readEntity(computeGenericValueTypeForRead(property, value), (Map) value));
}
@ -493,7 +502,14 @@ public class ElasticsearchEntityMapper implements
if (!typeHint.getActualType().getType().equals(Object.class)
&& isSimpleType(typeHint.getMapValueType().getType())) {
mapSource.forEach(it -> target.put(it.getKey(), getWriteSimpleValue(it.getValue())));
mapSource.forEach(it -> {
if (it.getValue() == null) {
target.put(it.getKey(), null);
} else {
target.put(it.getKey(), getWriteSimpleValue(it.getValue()));
}
});
} else {
mapSource.forEach(it -> {

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -143,6 +143,7 @@ import com.fasterxml.jackson.databind.ObjectMapper;
* @author Peter-Josef Meisch
* @author Mathias Teier
* @author Gyula Attila Csorogi
* @author Alexander Shabunevich
*/
public class ElasticsearchRestTemplate extends AbstractElasticsearchTemplate
implements ElasticsearchOperations, EsClient<RestHighLevelClient>, ApplicationContextAware {
@ -208,11 +209,17 @@ public class ElasticsearchRestTemplate extends AbstractElasticsearchTemplate
aliasAction.filter(query.getFilterBuilder());
} else if (query.getFilter() != null) {
aliasAction.filter(query.getFilter());
} else if (hasText(query.getRouting())) {
}
if (hasText(query.getRouting())) {
aliasAction.routing(query.getRouting());
} else if (hasText(query.getSearchRouting())) {
}
if (hasText(query.getSearchRouting())) {
aliasAction.searchRouting(query.getSearchRouting());
} else if (hasText(query.getIndexRouting())) {
}
if (hasText(query.getIndexRouting())) {
aliasAction.indexRouting(query.getIndexRouting());
}
@ -598,6 +605,7 @@ public class ElasticsearchRestTemplate extends AbstractElasticsearchTemplate
if (elasticsearchQuery != null) {
sourceBuilder.query(elasticsearchQuery);
}
sourceBuilder.size(0);
countRequest.source(sourceBuilder);
try {
@ -616,6 +624,7 @@ public class ElasticsearchRestTemplate extends AbstractElasticsearchTemplate
if (elasticsearchFilter != null) {
searchRequest.source().postFilter(elasticsearchFilter);
}
searchRequest.source().size(0);
SearchResponse response;
try {
response = client.search(searchRequest, RequestOptions.DEFAULT);
@ -960,6 +969,10 @@ public class ElasticsearchRestTemplate extends AbstractElasticsearchTemplate
prepareSort(query, searchSourceBuilder, entity);
}
if (query.getIndicesOptions() != null) {
request.indicesOptions(query.getIndicesOptions());
}
if (query instanceof SearchQuery) {
SearchQuery searchQuery = (SearchQuery) query;
@ -1332,8 +1345,17 @@ public class ElasticsearchRestTemplate extends AbstractElasticsearchTemplate
}
if (query.getPageable().isPaged()) {
startRecord = query.getPageable().getPageNumber() * query.getPageable().getPageSize();
long offset = query.getPageable().getOffset();
if (offset > Integer.MAX_VALUE) {
throw new IllegalArgumentException(String.format("Offset must not be more than %d", Integer.MAX_VALUE));
}
startRecord = (int) offset;
sourceBuilder.size(query.getPageable().getPageSize());
} else {
startRecord = 0;
sourceBuilder.size(INDEX_MAX_RESULT_WINDOW);
}
sourceBuilder.from(startRecord);

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -123,6 +123,7 @@ import org.springframework.util.StringUtils;
* @author Martin Choraine
* @author Farid Azaza
* @author Gyula Attila Csorogi
* @author Alexander Shabunevich
*/
public class ElasticsearchTemplate extends AbstractElasticsearchTemplate
implements ElasticsearchOperations, EsClient<Client>, ApplicationContextAware {
@ -189,13 +190,20 @@ public class ElasticsearchTemplate extends AbstractElasticsearchTemplate
aliasAction.filter(query.getFilterBuilder());
} else if (query.getFilter() != null) {
aliasAction.filter(query.getFilter());
} else if (!StringUtils.isEmpty(query.getRouting())) {
}
if (!StringUtils.isEmpty(query.getRouting())) {
aliasAction.routing(query.getRouting());
} else if (!StringUtils.isEmpty(query.getSearchRouting())) {
}
if (!StringUtils.isEmpty(query.getSearchRouting())) {
aliasAction.searchRouting(query.getSearchRouting());
} else if (!StringUtils.isEmpty(query.getIndexRouting())) {
}
if (!StringUtils.isEmpty(query.getIndexRouting())) {
aliasAction.indexRouting(query.getIndexRouting());
}
return client.admin().indices().prepareAliases().addAliasAction(aliasAction).execute().actionGet().isAcknowledged();
}
@ -508,6 +516,7 @@ public class ElasticsearchTemplate extends AbstractElasticsearchTemplate
if (elasticsearchQuery != null) {
countRequestBuilder.setQuery(elasticsearchQuery);
}
countRequestBuilder.setSize(0);
return countRequestBuilder.execute().actionGet().getHits().getTotalHits();
}
@ -521,6 +530,7 @@ public class ElasticsearchTemplate extends AbstractElasticsearchTemplate
if (elasticsearchFilter != null) {
searchRequestBuilder.setPostFilter(elasticsearchFilter);
}
searchRequestBuilder.setSize(0);
return searchRequestBuilder.execute().actionGet().getHits().getTotalHits();
}
@ -810,6 +820,10 @@ public class ElasticsearchTemplate extends AbstractElasticsearchTemplate
prepareSort(query, requestBuilder, entity);
}
if (query.getIndicesOptions() != null) {
requestBuilder.setIndicesOptions(query.getIndicesOptions());
}
if (query instanceof SearchQuery) {
SearchQuery searchQuery = (SearchQuery) query;
@ -1109,8 +1123,17 @@ public class ElasticsearchTemplate extends AbstractElasticsearchTemplate
}
if (query.getPageable().isPaged()) {
startRecord = query.getPageable().getPageNumber() * query.getPageable().getPageSize();
long offset = query.getPageable().getOffset();
if (offset > Integer.MAX_VALUE) {
throw new IllegalArgumentException(String.format("Offset must not be more than %d", Integer.MAX_VALUE));
}
startRecord = (int) offset;
searchRequestBuilder.setSize(query.getPageable().getPageSize());
} else {
startRecord = 0;
searchRequestBuilder.setSize(INDEX_MAX_RESULT_WINDOW);
}
searchRequestBuilder.setFrom(startRecord);

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -36,6 +36,7 @@ import org.elasticsearch.action.support.IndicesOptions;
import org.elasticsearch.action.support.WriteRequest;
import org.elasticsearch.action.support.WriteRequest.RefreshPolicy;
import org.elasticsearch.client.Requests;
import org.elasticsearch.client.core.CountRequest;
import org.elasticsearch.index.get.GetResult;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
@ -247,80 +248,135 @@ public class ReactiveElasticsearchTemplate implements ReactiveElasticsearchOpera
@Nullable String type) {
return Flux.defer(() -> {
SearchRequest request = prepareSearchRequest(buildSearchRequest(query, entity, index, type));
IndexCoordinates indexCoordinates = operations.determineIndex(entity, index, type);
SearchRequest request = new SearchRequest(indices(query, indexCoordinates::getIndexName));
request.types(indexTypes(query, indexCoordinates::getTypeName));
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(mappedQuery(query, entity));
searchSourceBuilder.version(entity.hasVersionProperty());
searchSourceBuilder.trackScores(query.getTrackScores());
QueryBuilder postFilterQuery = mappedFilterQuery(query, entity);
if (postFilterQuery != null) {
searchSourceBuilder.postFilter(postFilterQuery);
}
if (query.getSourceFilter() != null) {
searchSourceBuilder.fetchSource(query.getSourceFilter().getIncludes(), query.getSourceFilter().getExcludes());
}
if (query instanceof SearchQuery && ((SearchQuery) query).getCollapseBuilder() != null) {
searchSourceBuilder.collapse(((SearchQuery) query).getCollapseBuilder());
}
sort(query, entity).forEach(searchSourceBuilder::sort);
if (query.getMinScore() > 0) {
searchSourceBuilder.minScore(query.getMinScore());
}
if (query.getIndicesOptions() != null) {
request.indicesOptions(query.getIndicesOptions());
}
if (query.getPreference() != null) {
request.preference(query.getPreference());
}
if (query.getSearchType() != null) {
request.searchType(query.getSearchType());
}
Pageable pageable = query.getPageable();
if (pageable.isPaged()) {
long offset = pageable.getOffset();
if (offset > Integer.MAX_VALUE) {
throw new IllegalArgumentException(String.format("Offset must not be more than %s", Integer.MAX_VALUE));
}
searchSourceBuilder.from((int) offset);
searchSourceBuilder.size(pageable.getPageSize());
request.source(searchSourceBuilder);
return doFind(prepareSearchRequest(request));
if (query.getPageable().isPaged()) {
return doFind(request);
} else {
request.source(searchSourceBuilder);
return doScan(prepareSearchRequest(request));
return doScroll(request);
}
});
}
@Override
public Mono<Long> count(Query query, Class<?> entityType, String index, String type) {
return doCount(query, getPersistentEntity(entityType), index, type);
}
private Mono<Long> doCount(Query query, ElasticsearchPersistentEntity<?> entity, @Nullable String index,
@Nullable String type) {
return Mono.defer(() -> {
CountRequest countRequest = buildCountRequest(query, entity, index, type);
CountRequest request = prepareCountRequest(countRequest);
return doCount(request);
});
}
private CountRequest buildCountRequest(Query query, ElasticsearchPersistentEntity<?> entity, @Nullable String index,
@Nullable String type) {
IndexCoordinates indexCoordinates = operations.determineIndex(entity, index, type);
CountRequest request = new CountRequest(indices(query, indexCoordinates::getIndexName));
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(mappedQuery(query, entity));
searchSourceBuilder.trackScores(query.getTrackScores());
QueryBuilder postFilterQuery = mappedFilterQuery(query, entity);
if (postFilterQuery != null) {
searchSourceBuilder.postFilter(postFilterQuery);
}
if (query.getSourceFilter() != null) {
searchSourceBuilder.fetchSource(query.getSourceFilter().getIncludes(), query.getSourceFilter().getExcludes());
}
if (query instanceof SearchQuery && ((SearchQuery) query).getCollapseBuilder() != null) {
searchSourceBuilder.collapse(((SearchQuery) query).getCollapseBuilder());
}
sort(query, entity).forEach(searchSourceBuilder::sort);
if (query.getMinScore() > 0) {
searchSourceBuilder.minScore(query.getMinScore());
}
if (query.getIndicesOptions() != null) {
request.indicesOptions(query.getIndicesOptions());
}
if (query.getPreference() != null) {
request.preference(query.getPreference());
}
request.source(searchSourceBuilder);
return request;
}
private SearchRequest buildSearchRequest(Query query, ElasticsearchPersistentEntity<?> entity, @Nullable String index,
@Nullable String type) {
IndexCoordinates indexCoordinates = operations.determineIndex(entity, index, type);
SearchRequest request = new SearchRequest(indices(query, indexCoordinates::getIndexName));
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(mappedQuery(query, entity));
searchSourceBuilder.version(entity.hasVersionProperty());
searchSourceBuilder.trackScores(query.getTrackScores());
QueryBuilder postFilterQuery = mappedFilterQuery(query, entity);
if (postFilterQuery != null) {
searchSourceBuilder.postFilter(postFilterQuery);
}
if (query.getSourceFilter() != null) {
searchSourceBuilder.fetchSource(query.getSourceFilter().getIncludes(), query.getSourceFilter().getExcludes());
}
if (query instanceof SearchQuery && ((SearchQuery) query).getCollapseBuilder() != null) {
searchSourceBuilder.collapse(((SearchQuery) query).getCollapseBuilder());
}
sort(query, entity).forEach(searchSourceBuilder::sort);
if (query.getMinScore() > 0) {
searchSourceBuilder.minScore(query.getMinScore());
}
if (query.getIndicesOptions() != null) {
request.indicesOptions(query.getIndicesOptions());
}
if (query.getPreference() != null) {
request.preference(query.getPreference());
}
if (query.getSearchType() != null) {
request.searchType(query.getSearchType());
}
Pageable pageable = query.getPageable();
if (pageable.isPaged()) {
long offset = pageable.getOffset();
if (offset > Integer.MAX_VALUE) {
throw new IllegalArgumentException(String.format("Offset must not be more than %s", Integer.MAX_VALUE));
}
searchSourceBuilder.from((int) offset);
searchSourceBuilder.size(pageable.getPageSize());
request.source(searchSourceBuilder);
} else {
searchSourceBuilder.from(0);
searchSourceBuilder.size(AbstractElasticsearchTemplate.INDEX_MAX_RESULT_WINDOW);
request.source(searchSourceBuilder);
}
return request;
}
/*
* (non-Javadoc)
* @see org.springframework.data.elasticsearch.core.ReactiveElasticsearchOperations#count(Query, Class, String, String)
*/
@Override
public Mono<Long> count(Query query, Class<?> entityType, String index, String type) {
// TODO: ES 7.0 has a dedicated CountRequest - use that one once available.
return find(query, entityType, index, type).count();
}
/*
* (non-Javadoc)
@ -457,6 +513,22 @@ public class ReactiveElasticsearchTemplate implements ReactiveElasticsearchOpera
return prepareWriteRequest(request);
}
/**
* Customization hook to modify a generated {@link SearchRequest} prior to its execution. Eg. by setting the
* {@link SearchRequest#indicesOptions(IndicesOptions) indices options} if applicable.
*
* @param request the generated {@link CountRequest}.
* @return never {@literal null}.
*/
protected CountRequest prepareCountRequest(CountRequest request) {
if (indicesOptions == null) {
return request;
}
return request.indicesOptions(indicesOptions);
}
/**
* Customization hook to modify a generated {@link SearchRequest} prior to its execution. Eg. by setting the
* {@link SearchRequest#indicesOptions(IndicesOptions) indices options} if applicable.
@ -557,16 +629,32 @@ public class ReactiveElasticsearchTemplate implements ReactiveElasticsearchOpera
.onErrorResume(NoSuchIndexException.class, it -> Mono.empty());
}
/**
* Customization hook on the actual execution result {@link Publisher}. <br />
*
* @param request the already prepared {@link CountRequest} ready to be executed.
* @return a {@link Mono} emitting the result of the operation.
*/
protected Mono<Long> doCount(CountRequest request) {
if (QUERY_LOGGER.isDebugEnabled()) {
QUERY_LOGGER.debug("Executing doCount: {}", request);
}
return Mono.from(execute(client -> client.count(request))) //
.onErrorResume(NoSuchIndexException.class, it -> Mono.empty());
}
/**
* Customization hook on the actual execution result {@link Publisher}. <br />
*
* @param request the already prepared {@link SearchRequest} ready to be executed.
* @return a {@link Flux} emitting the result of the operation.
*/
protected Flux<SearchHit> doScan(SearchRequest request) {
protected Flux<SearchHit> doScroll(SearchRequest request) {
if (QUERY_LOGGER.isDebugEnabled()) {
QUERY_LOGGER.debug("Executing doScan: {}", request);
QUERY_LOGGER.debug("Executing doScroll: {}", request);
}
return Flux.from(execute(client -> client.scroll(request))) //
@ -665,9 +753,7 @@ public class ReactiveElasticsearchTemplate implements ReactiveElasticsearchOpera
elasticsearchQuery = new WrapperQueryBuilder(((StringQuery) query).getSource());
} else if (query instanceof NativeSearchQuery) {
elasticsearchQuery = ((NativeSearchQuery) query).getQuery();
}
else {
} else {
throw new IllegalArgumentException(String.format("Unknown query type '%s'.", query.getClass()));
}

View File

@ -1,5 +1,5 @@
/*
* Copyright 2019 the original author or authors.
* Copyright 2019-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -2,12 +2,14 @@
package org.springframework.data.elasticsearch.core;
import org.springframework.data.domain.Page;
import org.springframework.lang.Nullable;
/**
* @author Artur Konczak
* @author Peter-Josef Meisch
* @author Sascha Woo
*/
public interface ScrolledPage<T> extends Page<T> {
String getScrollId();
String getScrollId();
}

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2019 the original author or authors.
* Copyright 2019-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2019 the original author or authors.
* Copyright 2019-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -21,11 +21,13 @@ import java.util.function.Consumer;
import java.util.function.Function;
import org.springframework.data.util.CloseableIterator;
import org.springframework.util.Assert;
/**
* Utility to support streaming queries.
*
* @author Mark Paluch
* @author Sascha Woo
* @since 3.2
*/
abstract class StreamQueries {
@ -33,72 +35,71 @@ abstract class StreamQueries {
/**
* Stream query results using {@link ScrolledPage}.
*
* @param page the initial page.
* @param continueFunction continuation function accepting the current scrollId.
* @param clearScroll cleanup function accepting the current scrollId.
* @param page the initial scrolled page.
* @param continueScrollFunction function to continue scrolling applies to the current scrollId.
* @param clearScrollConsumer consumer to clear the scroll context by accepting the current scrollId.
* @param <T>
* @return the {@link CloseableIterator}.
*/
static <T> CloseableIterator<T> streamResults(ScrolledPage<T> page,
Function<String, ScrolledPage<T>> continueFunction, Consumer<String> clearScroll) {
Function<String, ScrolledPage<T>> continueScrollFunction, Consumer<String> clearScrollConsumer) {
Assert.notNull(page, "page must not be null.");
Assert.notNull(page.getScrollId(), "scrollId must not be null.");
Assert.notNull(continueScrollFunction, "continueScrollFunction must not be null.");
Assert.notNull(clearScrollConsumer, "clearScrollConsumer must not be null.");
return new CloseableIterator<T>() {
/** As we couldn't retrieve single result with scroll, store current hits. */
private volatile Iterator<T> currentHits = page.iterator();
/** The scroll id. */
// As we couldn't retrieve single result with scroll, store current hits.
private volatile Iterator<T> scrollHits = page.iterator();
private volatile String scrollId = page.getScrollId();
/** If stream is finished (ie: cluster returns no results. */
private volatile boolean finished = !currentHits.hasNext();
private volatile boolean continueScroll = scrollHits.hasNext();
@Override
public void close() {
try {
// Clear scroll on cluster only in case of error (cause elasticsearch auto clear scroll when it's done)
if (!finished && scrollId != null && currentHits != null && currentHits.hasNext()) {
clearScroll.accept(scrollId);
}
clearScrollConsumer.accept(scrollId);
} finally {
currentHits = null;
scrollHits = null;
scrollId = null;
}
}
@Override
public boolean hasNext() {
// Test if stream is finished
if (finished) {
if (!continueScroll) {
return false;
}
// Test if it remains hits
if (currentHits == null || !currentHits.hasNext()) {
// Do a new request
ScrolledPage<T> scroll = continueFunction.apply(scrollId);
// Save hits and scroll id
currentHits = scroll.iterator();
finished = !currentHits.hasNext();
scrollId = scroll.getScrollId();
if (!scrollHits.hasNext()) {
ScrolledPage<T> nextPage = continueScrollFunction.apply(scrollId);
scrollHits = nextPage.iterator();
scrollId = nextPage.getScrollId();
continueScroll = scrollHits.hasNext();
}
return currentHits.hasNext();
return scrollHits.hasNext();
}
@Override
public T next() {
if (hasNext()) {
return currentHits.next();
return scrollHits.next();
}
throw new NoSuchElementException();
}
@Override
public void remove() {
throw new UnsupportedOperationException("remove");
throw new UnsupportedOperationException();
}
};
}
// utility constructor
private StreamQueries() {}
private StreamQueries() {
}
}

View File

@ -1,5 +1,5 @@
/*
* Copyright 2017-2019 the original author or authors.
* Copyright 2017-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2019 the original author or authors.
* Copyright 2019-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2018-2019 the original author or authors.
* Copyright 2018-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2019 the original author or authors.
* Copyright 2019-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2019 the original author or authors.
* Copyright 2019-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2013-2019 the original author or authors.
* Copyright 2013-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

View File

@ -1,5 +1,5 @@
/*
* Copyright 2014-2019 the original author or authors.
* Copyright 2014-2020 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

Some files were not shown because too many files have changed in this diff Show More