Compare commits

...

16 Commits
main ... 5.5.3

Author SHA1 Message Date
Mark Paluch
5bbccbf9d0
Release version 5.5.3 (2025.0.3).
See #3134
2025-08-15 10:01:35 +02:00
Mark Paluch
19af94f87e
Prepare 5.5.3 (2025.0.3).
See #3134
2025-08-15 10:01:14 +02:00
Mark Paluch
71a3e24096
Polishing.
Use documentation variables for references, reorder antora keys.

See #3135
2025-08-14 17:28:01 +02:00
Peter-Josef Meisch
23eacd4d4b
Upgrade to Elasticsearch 8.18.5.
Original Pull Request #3151
Closes #3149
Signed-off-by: Peter-Josef Meisch <pj.meisch@sothawo.com>
2025-08-12 18:39:04 +02:00
Mark Paluch
56bda34666
After release cleanups.
See #3120
2025-07-18 10:30:34 +02:00
Mark Paluch
76ba2324a2
Prepare next development iteration.
See #3120
2025-07-18 10:30:33 +02:00
Mark Paluch
4c217fa9c4
Release version 5.5.2 (2025.0.2).
See #3120
2025-07-18 10:27:59 +02:00
Mark Paluch
003213d4b0
Prepare 5.5.2 (2025.0.2).
See #3120
2025-07-18 10:27:38 +02:00
Mark Paluch
85d52014dc
Upgrade to Maven Wrapper 3.9.11.
See #3131
2025-07-17 14:00:55 +02:00
Peter-Josef Meisch
786e0928ed
Fix the calculation of the requested number of documents.
Original Pull Request #3128
Closes #3127

Signed-off-by: Peter-Josef Meisch <pj.meisch@sothawo.com>
(cherry picked from commit 12ddb74faebb122132b0b4079ede4daa735d9670)
2025-07-15 18:41:17 +02:00
Mark Paluch
ef6f091d6b
After release cleanups.
See #3114
2025-06-13 13:42:19 +02:00
Mark Paluch
9ff829a829
Prepare next development iteration.
See #3114
2025-06-13 13:42:18 +02:00
Mark Paluch
30f32b6bbe
Release version 5.5.1 (2025.0.1).
See #3114
2025-06-13 13:39:35 +02:00
Mark Paluch
8ce113a083
Prepare 5.5.1 (2025.0.1).
See #3114
2025-06-13 13:39:14 +02:00
Mark Paluch
262781c0a0
After release cleanups.
See #3096
2025-05-16 11:31:36 +02:00
Mark Paluch
ffe8293365
Prepare next development iteration.
See #3096
2025-05-16 11:31:35 +02:00
13 changed files with 198 additions and 45 deletions

View File

@ -1,3 +1,3 @@
#Thu Nov 07 09:47:28 CET 2024 #Thu Jul 17 14:00:55 CEST 2025
wrapperUrl=https\://repo.maven.apache.org/maven2/io/takari/maven-wrapper/0.5.6/maven-wrapper-0.5.6.jar wrapperUrl=https\://repo.maven.apache.org/maven2/io/takari/maven-wrapper/0.5.6/maven-wrapper-0.5.6.jar
distributionUrl=https\://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.9/apache-maven-3.9.9-bin.zip distributionUrl=https\://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.11/apache-maven-3.9.11-bin.zip

2
Jenkinsfile vendored
View File

@ -9,7 +9,7 @@ pipeline {
triggers { triggers {
pollSCM 'H/10 * * * *' pollSCM 'H/10 * * * *'
upstream(upstreamProjects: "spring-data-commons/main", threshold: hudson.model.Result.SUCCESS) upstream(upstreamProjects: "spring-data-commons/3.5.x", threshold: hudson.model.Result.SUCCESS)
} }
options { options {

10
pom.xml
View File

@ -1,16 +1,16 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion> <modelVersion>4.0.0</modelVersion>
<groupId>org.springframework.data</groupId> <groupId>org.springframework.data</groupId>
<artifactId>spring-data-elasticsearch</artifactId> <artifactId>spring-data-elasticsearch</artifactId>
<version>5.5.0</version> <version>5.5.3</version>
<parent> <parent>
<groupId>org.springframework.data.build</groupId> <groupId>org.springframework.data.build</groupId>
<artifactId>spring-data-parent</artifactId> <artifactId>spring-data-parent</artifactId>
<version>3.5.0</version> <version>3.5.3</version>
</parent> </parent>
<name>Spring Data Elasticsearch</name> <name>Spring Data Elasticsearch</name>
@ -18,10 +18,10 @@
<url>https://github.com/spring-projects/spring-data-elasticsearch</url> <url>https://github.com/spring-projects/spring-data-elasticsearch</url>
<properties> <properties>
<springdata.commons>3.5.0</springdata.commons> <springdata.commons>3.5.3</springdata.commons>
<!-- version of the ElasticsearchClient --> <!-- version of the ElasticsearchClient -->
<elasticsearch-java>8.18.1</elasticsearch-java> <elasticsearch-java>8.18.5</elasticsearch-java>
<hoverfly>0.19.0</hoverfly> <hoverfly>0.19.0</hoverfly>
<log4j>2.23.1</log4j> <log4j>2.23.1</log4j>

View File

@ -376,7 +376,7 @@ So calling the method with a `List` of `["id1", "id2", "id3"]` would produce the
.Declare query on the method using the `@Query` annotation with SpEL expression. .Declare query on the method using the `@Query` annotation with SpEL expression.
==== ====
https://docs.spring.io/spring-framework/reference/core/expressions.html[SpEL expression] is also supported when defining query in `@Query`. {spring-framework-docs}/core/expressions.html[SpEL expression] is also supported when defining query in `@Query`.
[source,java] [source,java]
---- ----
@ -453,7 +453,7 @@ We can pass `new QueryParameter("John")` as the parameter now, and it will produ
.accessing bean property. .accessing bean property.
==== ====
https://docs.spring.io/spring-framework/reference/core/expressions/language-ref/bean-references.html[Bean property] is also supported to access. {spring-framework-docs}/core/expressions/language-ref/bean-references.html[Bean property] is also supported to access.
Given that there is a bean named `queryParameter` of type `QueryParameter`, we can access the bean with symbol `@` rather than `#`, and there is no need to declare a parameter of type `QueryParameter` in the query method: Given that there is a bean named `queryParameter` of type `QueryParameter`, we can access the bean with symbol `@` rather than `#`, and there is no need to declare a parameter of type `QueryParameter` in the query method:
[source,java] [source,java]
@ -523,7 +523,7 @@ A collection of `names` like `List.of("name1", "name2")` will produce the follow
.access property in the `Collection` param. .access property in the `Collection` param.
==== ====
https://docs.spring.io/spring-framework/reference/core/expressions/language-ref/collection-projection.html[SpEL Collection Projection] is convenient to use when values in the `Collection` parameter is not plain `String`: {spring-framework-docs}/core/expressions/language-ref/collection-projection.html[SpEL Collection Projection] is convenient to use when values in the `Collection` parameter is not plain `String`:
[source,java] [source,java]
---- ----

View File

@ -6,7 +6,7 @@ The following table shows the Elasticsearch and Spring versions that are used by
[cols="^,^,^,^",options="header"] [cols="^,^,^,^",options="header"]
|=== |===
| Spring Data Release Train | Spring Data Elasticsearch | Elasticsearch | Spring Framework | Spring Data Release Train | Spring Data Elasticsearch | Elasticsearch | Spring Framework
| 2025.0 | 5.5.x | 8.18.1 | 6.2.x | 2025.0 | 5.5.x | 8.18.5 | 6.2.x
| 2024.1 | 5.4.x | 8.15.5 | 6.1.x | 2024.1 | 5.4.x | 8.15.5 | 6.1.x
| 2024.0 | 5.3.xfootnote:oom[Out of maintenance] | 8.13.4 | 6.1.x | 2024.0 | 5.3.xfootnote:oom[Out of maintenance] | 8.13.4 | 6.1.x
| 2023.1 (Vaughan) | 5.2.xfootnote:oom[] | 8.11.1 | 6.1.x | 2023.1 (Vaughan) | 5.2.xfootnote:oom[] | 8.11.1 | 6.1.x

View File

@ -3,18 +3,19 @@ prerelease: ${antora-component.prerelease}
asciidoc: asciidoc:
attributes: attributes:
copyright-year: ${current.year} attribute-missing: 'warn'
chomp: 'all'
version: ${project.version} version: ${project.version}
copyright-year: ${current.year}
springversionshort: ${spring.short} springversionshort: ${spring.short}
springversion: ${spring} springversion: ${spring}
attribute-missing: 'warn'
commons: ${springdata.commons.docs} commons: ${springdata.commons.docs}
include-xml-namespaces: false include-xml-namespaces: false
spring-data-commons-docs-url: https://docs.spring.io/spring-data/commons/reference spring-data-commons-docs-url: https://docs.spring.io/spring-data/commons/reference/{commons}
spring-data-commons-javadoc-base: https://docs.spring.io/spring-data/commons/docs/${springdata.commons}/api/ spring-data-commons-javadoc-base: '{spring-data-commons-docs-url}/api/java'
springdocsurl: https://docs.spring.io/spring-framework/reference/{springversionshort} springdocsurl: https://docs.spring.io/spring-framework/reference/{springversionshort}
springjavadocurl: https://docs.spring.io/spring-framework/docs/${spring}/javadoc-api
spring-framework-docs: '{springdocsurl}' spring-framework-docs: '{springdocsurl}'
springjavadocurl: https://docs.spring.io/spring-framework/docs/${spring}/javadoc-api
spring-framework-javadoc: '{springjavadocurl}' spring-framework-javadoc: '{springjavadocurl}'
springhateoasversion: ${spring-hateoas} springhateoasversion: ${spring-hateoas}
releasetrainversion: ${releasetrain} releasetrainversion: ${releasetrain}

View File

@ -120,9 +120,6 @@ class RequestConverter extends AbstractQueryProcessor {
private static final Log LOGGER = LogFactory.getLog(RequestConverter.class); private static final Log LOGGER = LogFactory.getLog(RequestConverter.class);
// the default max result window size of Elasticsearch
public static final Integer INDEX_MAX_RESULT_WINDOW = 10_000;
protected final JsonpMapper jsonpMapper; protected final JsonpMapper jsonpMapper;
protected final ElasticsearchConverter elasticsearchConverter; protected final ElasticsearchConverter elasticsearchConverter;
@ -1295,15 +1292,8 @@ class RequestConverter extends AbstractQueryProcessor {
.timeout(timeStringMs(query.getTimeout())) // .timeout(timeStringMs(query.getTimeout())) //
; ;
var offset = query.getPageable().isPaged() ? query.getPageable().getOffset() : 0; bb.from((int) (query.getPageable().isPaged() ? query.getPageable().getOffset() : 0))
var pageSize = query.getPageable().isPaged() ? query.getPageable().getPageSize() .size(query.getRequestSize());
: INDEX_MAX_RESULT_WINDOW;
// if we have both a page size and a max results, we take the min, this is necessary for
// searchForStream to work correctly (#3098) as there the page size defines what is
// returned in a single request, and the max result determines the total number of
// documents returned
var size = query.isLimiting() ? Math.min(pageSize, query.getMaxResults()) : pageSize;
bb.from((int) offset).size(size);
if (!isEmpty(query.getFields())) { if (!isEmpty(query.getFields())) {
bb.fields(fb -> { bb.fields(fb -> {
@ -1473,14 +1463,8 @@ class RequestConverter extends AbstractQueryProcessor {
builder.seqNoPrimaryTerm(true); builder.seqNoPrimaryTerm(true);
} }
var offset = query.getPageable().isPaged() ? query.getPageable().getOffset() : 0; builder.from((int) (query.getPageable().isPaged() ? query.getPageable().getOffset() : 0))
var pageSize = query.getPageable().isPaged() ? query.getPageable().getPageSize() : INDEX_MAX_RESULT_WINDOW; .size(query.getRequestSize());
// if we have both a page size and a max results, we take the min, this is necessary for
// searchForStream to work correctly (#3098) as there the page size defines what is
// returned in a single request, and the max result determines the total number of
// documents returned
var size = query.isLimiting() ? Math.min(pageSize, query.getMaxResults()) : pageSize;
builder.from((int) offset).size(size);
if (!isEmpty(query.getFields())) { if (!isEmpty(query.getFields())) {
var fieldAndFormats = query.getFields().stream().map(field -> FieldAndFormat.of(b -> b.field(field))).toList(); var fieldAndFormats = query.getFields().stream().map(field -> FieldAndFormat.of(b -> b.field(field))).toList();

View File

@ -27,6 +27,7 @@ import java.util.List;
import java.util.Optional; import java.util.Optional;
import java.util.stream.Collectors; import java.util.stream.Collectors;
import org.springframework.data.domain.PageRequest;
import org.springframework.data.domain.Pageable; import org.springframework.data.domain.Pageable;
import org.springframework.data.domain.Sort; import org.springframework.data.domain.Sort;
import org.springframework.lang.Nullable; import org.springframework.lang.Nullable;
@ -47,10 +48,15 @@ import org.springframework.util.Assert;
*/ */
public class BaseQuery implements Query { public class BaseQuery implements Query {
public static final int INDEX_MAX_RESULT_WINDOW = 10_000;
private static final int DEFAULT_REACTIVE_BATCH_SIZE = 500; private static final int DEFAULT_REACTIVE_BATCH_SIZE = 500;
// the instance to mark the query pageable initial status, needed to distinguish between the initial
// value and a user-set unpaged value; values don't matter, the RequestConverter compares to the isntance.
private static final Pageable UNSET_PAGE = PageRequest.of(0, 1);
@Nullable protected Sort sort; @Nullable protected Sort sort;
protected Pageable pageable = DEFAULT_PAGE; protected Pageable pageable = UNSET_PAGE;
protected List<String> fields = new ArrayList<>(); protected List<String> fields = new ArrayList<>();
@Nullable protected List<String> storedFields; @Nullable protected List<String> storedFields;
@Nullable protected SourceFilter sourceFilter; @Nullable protected SourceFilter sourceFilter;
@ -78,7 +84,7 @@ public class BaseQuery implements Query {
private boolean queryIsUpdatedByConverter = false; private boolean queryIsUpdatedByConverter = false;
@Nullable private Integer reactiveBatchSize = null; @Nullable private Integer reactiveBatchSize = null;
@Nullable private Boolean allowNoIndices = null; @Nullable private Boolean allowNoIndices = null;
private EnumSet<IndicesOptions.WildcardStates> expandWildcards; private EnumSet<IndicesOptions.WildcardStates> expandWildcards = EnumSet.noneOf(IndicesOptions.WildcardStates.class);
private List<DocValueField> docValueFields = new ArrayList<>(); private List<DocValueField> docValueFields = new ArrayList<>();
private List<ScriptedField> scriptedFields = new ArrayList<>(); private List<ScriptedField> scriptedFields = new ArrayList<>();
@ -87,7 +93,7 @@ public class BaseQuery implements Query {
public <Q extends BaseQuery, B extends BaseQueryBuilder<Q, B>> BaseQuery(BaseQueryBuilder<Q, B> builder) { public <Q extends BaseQuery, B extends BaseQueryBuilder<Q, B>> BaseQuery(BaseQueryBuilder<Q, B> builder) {
this.sort = builder.getSort(); this.sort = builder.getSort();
// do a setPageable after setting the sort, because the pageable may contain an additional sort // do a setPageable after setting the sort, because the pageable may contain an additional sort
this.setPageable(builder.getPageable() != null ? builder.getPageable() : DEFAULT_PAGE); this.setPageable(builder.getPageable() != null ? builder.getPageable() : UNSET_PAGE);
this.fields = builder.getFields(); this.fields = builder.getFields();
this.storedFields = builder.getStoredFields(); this.storedFields = builder.getStoredFields();
this.sourceFilter = builder.getSourceFilter(); this.sourceFilter = builder.getSourceFilter();
@ -203,7 +209,7 @@ public class BaseQuery implements Query {
@Override @Override
@SuppressWarnings("unchecked") @SuppressWarnings("unchecked")
public final <T extends Query> T addSort(@Nullable Sort sort) { public final <T extends Query> T addSort(@Nullable Sort sort) {
if (sort == null) { if (sort == null || sort.isUnsorted()) {
return (T) this; return (T) this;
} }
@ -561,4 +567,52 @@ public class BaseQuery implements Query {
public List<ScriptedField> getScriptedFields() { public List<ScriptedField> getScriptedFields() {
return scriptedFields; return scriptedFields;
} }
@Override
public Integer getRequestSize() {
var pageable = getPageable();
Integer requestSize = null;
if (pageable.isPaged() && pageable != UNSET_PAGE) {
// pagesize defined by the user
if (!isLimiting()) {
// no maxResults
requestSize = pageable.getPageSize();
} else {
// if we have both a page size and a max results, we take the min, this is necessary for
// searchForStream to work correctly (#3098) as there the page size defines what is
// returned in a single request, and the max result determines the total number of
// documents returned.
requestSize = Math.min(pageable.getPageSize(), getMaxResults());
}
} else if (pageable == UNSET_PAGE) {
// no user defined pageable
if (isLimiting()) {
// maxResults
requestSize = getMaxResults();
} else {
requestSize = DEFAULT_PAGE_SIZE;
}
} else {
// explicitly set unpaged
if (!isLimiting()) {
// no maxResults
requestSize = INDEX_MAX_RESULT_WINDOW;
} else {
// if we have both a implicit page size and a max results, we take the min, this is necessary for
// searchForStream to work correctly (#3098) as there the page size defines what is
// returned in a single request, and the max result determines the total number of
// documents returned.
requestSize = Math.min(INDEX_MAX_RESULT_WINDOW, getMaxResults());
}
}
if (requestSize == null) {
// this should not happen
requestSize = DEFAULT_PAGE_SIZE;
}
return requestSize;
}
} }

View File

@ -484,6 +484,13 @@ public interface Query {
*/ */
List<ScriptedField> getScriptedFields(); List<ScriptedField> getScriptedFields();
/**
* @return the number of documents that should be requested from Elasticsearch in this query. Depends wether a
* Pageable and/or maxResult size is set on the query.
* @since 5.4.8 5.5.2
*/
public Integer getRequestSize();
/** /**
* @since 4.3 * @since 4.3
*/ */

View File

@ -1,4 +1,4 @@
Spring Data Elasticsearch 5.5 GA (2025.0.0) Spring Data Elasticsearch 5.5.3 (2025.0.3)
Copyright (c) [2013-2022] Pivotal Software, Inc. Copyright (c) [2013-2022] Pivotal Software, Inc.
This product is licensed to you under the Apache License, Version 2.0 (the "License"). This product is licensed to you under the Apache License, Version 2.0 (the "License").
@ -25,6 +25,9 @@ conditions of the subcomponent's license, as noted in the LICENSE file.

View File

@ -105,8 +105,6 @@ import org.springframework.lang.Nullable;
@SpringIntegrationTest @SpringIntegrationTest
public abstract class ElasticsearchIntegrationTests { public abstract class ElasticsearchIntegrationTests {
static final Integer INDEX_MAX_RESULT_WINDOW = 10_000;
private static final String MULTI_INDEX_PREFIX = "test-index"; private static final String MULTI_INDEX_PREFIX = "test-index";
private static final String MULTI_INDEX_ALL = MULTI_INDEX_PREFIX + "*"; private static final String MULTI_INDEX_ALL = MULTI_INDEX_PREFIX + "*";
private static final String MULTI_INDEX_1_NAME = MULTI_INDEX_PREFIX + "-1"; private static final String MULTI_INDEX_1_NAME = MULTI_INDEX_PREFIX + "-1";

View File

@ -0,0 +1,106 @@
/*
* Copyright 2025 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.data.elasticsearch.core.query;
import static org.assertj.core.api.Assertions.*;
import static org.springframework.data.elasticsearch.core.query.BaseQuery.*;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.springframework.data.domain.Pageable;
class BaseQueryTests {
private static final String MATCH_ALL_QUERY = "{\"match_all\":{}}";
@Test // #3127
@DisplayName("query with no Pageable and no maxResults requests 10 docs from 0")
void queryWithNoPageableAndNoMaxResultsRequests10DocsFrom0() {
var query = StringQuery.builder(MATCH_ALL_QUERY)
.build();
var requestSize = query.getRequestSize();
assertThat(requestSize).isEqualTo(10);
}
@Test // #3127
@DisplayName("query with a Pageable and no MaxResults request with values from Pageable")
void queryWithAPageableAndNoMaxResultsRequestWithValuesFromPageable() {
var query = StringQuery.builder(MATCH_ALL_QUERY)
.withPageable(Pageable.ofSize(42))
.build();
var requestSize = query.getRequestSize();
assertThat(requestSize).isEqualTo(42);
}
@Test // #3127
@DisplayName("query with no Pageable and maxResults requests maxResults")
void queryWithNoPageableAndMaxResultsRequestsMaxResults() {
var query = StringQuery.builder(MATCH_ALL_QUERY)
.withMaxResults(12_345)
.build();
var requestSize = query.getRequestSize();
assertThat(requestSize).isEqualTo(12_345);
}
@Test // #3127
@DisplayName("query with Pageable and maxResults requests with values from Pageable if Pageable is less than maxResults")
void queryWithPageableAndMaxResultsRequestsWithValuesFromPageableIfPageableIsLessThanMaxResults() {
var query = StringQuery.builder(MATCH_ALL_QUERY)
.withPageable(Pageable.ofSize(42))
.withMaxResults(123)
.build();
var requestSize = query.getRequestSize();
assertThat(requestSize).isEqualTo(42);
}
@Test // #3127
@DisplayName("query with Pageable and maxResults requests with values from maxResults if Pageable is more than maxResults")
void queryWithPageableAndMaxResultsRequestsWithValuesFromMaxResultsIfPageableIsMoreThanMaxResults() {
var query = StringQuery.builder(MATCH_ALL_QUERY)
.withPageable(Pageable.ofSize(420))
.withMaxResults(123)
.build();
var requestSize = query.getRequestSize();
assertThat(requestSize).isEqualTo(123);
}
@Test // #3127
@DisplayName("query with explicit unpaged request and no maxResults requests max request window size")
void queryWithExplicitUnpagedRequestAndNoMaxResultsRequestsMaxRequestWindowSize() {
var query = StringQuery.builder(MATCH_ALL_QUERY)
.withPageable(Pageable.unpaged())
.build();
var requestSize = query.getRequestSize();
assertThat(requestSize).isEqualTo(INDEX_MAX_RESULT_WINDOW);
}
}

View File

@ -15,7 +15,7 @@
# #
# #
sde.testcontainers.image-name=docker.elastic.co/elasticsearch/elasticsearch sde.testcontainers.image-name=docker.elastic.co/elasticsearch/elasticsearch
sde.testcontainers.image-version=8.18.1 sde.testcontainers.image-version=8.18.5
# #
# #
# needed as we do a DELETE /* at the end of the tests, will be required from 8.0 on, produces a warning since 7.13 # needed as we do a DELETE /* at the end of the tests, will be required from 8.0 on, produces a warning since 7.13