520 Commits

Author SHA1 Message Date
Dmitriy Yakovlev
f0ae72141f DATAES-565 - prepareScroll doesn't respect SourceFilter from the Query.
Original pull request: #272
2019-04-18 12:46:31 +02:00
Jens Schauder
0ccc54da34 DATAES-558 - Upgrade Elastic Search version to 6.6.2. 2019-04-11 11:07:07 +02:00
Christoph Strobl
3ea7dea5cb DATAES-552 - Polishing.
Update lookup to replace all matches & add some tests.

Original Pull Request: #267
2019-04-10 10:20:12 +02:00
Taylor
b93bd07a3a DATAES-552 - @Query annotation fail when passing over 10 parameters
Updated the replacePlaceholders methods to use replaceFirst to prevent duplication substitutions.

Original Pull Request: #267
2019-04-10 10:19:25 +02:00
Christoph Strobl
16bf8450f0 DATAES-547 - Polishing.
Add test and directly use SearchHit to pass on the index name.
Fix minor flaw in Exception translation for non existing indices along the way.

Original Pull Request: #257
2019-04-09 15:40:49 +02:00
Spring Operator
9818ed0af9 DATAES-549 - URL Cleanup.
This commit updates URLs to prefer the https protocol. Redirects are not followed to avoid accidentally expanding intentionally shortened URLs (i.e. if using a URL shortener).

# Fixed URLs

## Fixed Success
These URLs were switched to an https URL with a 2xx status. While the status was successful, your review is still recommended.

* [ ] http://www.apache.org/licenses/ with 1 occurrences migrated to:
  https://www.apache.org/licenses/ ([https](https://www.apache.org/licenses/) result 200).
* [ ] http://www.apache.org/licenses/LICENSE-2.0 with 317 occurrences migrated to:
  https://www.apache.org/licenses/LICENSE-2.0 ([https](https://www.apache.org/licenses/LICENSE-2.0) result 200).

Original Pull Request: #266
2019-03-22 08:20:19 +01:00
Spring Operator
75b039a477 DATAES-549 - URL Cleanup.
This commit updates URLs to prefer the https protocol. Redirects are not followed to avoid accidentally expanding intentionally shortened URLs (i.e. if using a URL shortener).

# Fixed URLs

## Fixed Success
These URLs were switched to an https URL with a 2xx status. While the status was successful, your review is still recommended.

* http://maven.apache.org/xsd/maven-4.0.0.xsd with 1 occurrences migrated to:
  https://maven.apache.org/xsd/maven-4.0.0.xsd ([https](https://maven.apache.org/xsd/maven-4.0.0.xsd) result 200).
* http://www.springframework.org/schema/beans/spring-beans-3.1.xsd with 19 occurrences migrated to:
  https://www.springframework.org/schema/beans/spring-beans-3.1.xsd ([https](https://www.springframework.org/schema/beans/spring-beans-3.1.xsd) result 200).
* http://www.springframework.org/schema/beans/spring-beans.xsd with 3 occurrences migrated to:
  https://www.springframework.org/schema/beans/spring-beans.xsd ([https](https://www.springframework.org/schema/beans/spring-beans.xsd) result 200).
* http://www.springframework.org/schema/data/elasticsearch/spring-elasticsearch.xsd with 20 occurrences migrated to:
  https://www.springframework.org/schema/data/elasticsearch/spring-elasticsearch.xsd ([https](https://www.springframework.org/schema/data/elasticsearch/spring-elasticsearch.xsd) result 200).
* http://java.sun.com/xml/ns/javaee/beans_1_0.xsd with 1 occurrences migrated to:
  https://java.sun.com/xml/ns/javaee/beans_1_0.xsd ([https](https://java.sun.com/xml/ns/javaee/beans_1_0.xsd) result 302).

# Ignored
These URLs were intentionally ignored.

* http://java.sun.com/xml/ns/javaee with 2 occurrences
* http://maven.apache.org/POM/4.0.0 with 2 occurrences
* http://www.springframework.org/schema/beans with 44 occurrences
* http://www.springframework.org/schema/data/elasticsearch with 40 occurrences
* http://www.w3.org/2001/XMLSchema-instance with 24 occurrences

Original Pull Request: #258
2019-03-20 12:38:23 +01:00
Oliver Drotbohm
13d369de20
DATAES-545 - Move off deprecations in Spring Data Commons.
Related ticket: DATACMNS-1496.
2019-03-13 19:20:40 +01:00
Petr Kukrál
11ba4c55ad DATAES-535 - Add mapping annotation @DynamicTemplates.
Original pull request: #238
2019-03-13 14:16:12 +01:00
Robert Gründler
092690d1b9 DATAES-536 - Add support for context suggester
Original pull request: #241
2019-03-13 10:27:11 +01:00
Mark Paluch
c3e7056d7b DATAES-530 - Polishing.
Fix generics in ElasticsearchEntityMapper. Extract methods. Move enum conversion handling to simple type handling for symmetric implementation. Swap comparison of nullable types to avoid potential null dereference.

Rename ElasticsearchDefaultTypeMapper to DefaultElasticsearchTypeMapper. Move MapTypeAliasAccessor to DefaultElasticsearchTypeMapper.

Introduce SearchResultMapperAdapter to avoid empty method implementations in anonymous classes. Javadoc, reference docs, formatting.

Original Pull Request: #237
2019-03-06 10:03:02 +01:00
Christoph Strobl
a64af54e26 DATAES-530 - Add converter based EntityMapper.
ElasticsearchEntityMapper provides an alternative to the Jackson based EntityMapper implementation.

Original Pull Request: #237
2019-03-06 10:02:13 +01:00
Mark Paluch
1dc113d57b DATAES-537 - Upgrade to Elasticsearch 6.6.1.
Adapt tests to use new IndicesOptions defaults.

Original Pull Request: #243
2019-03-06 09:29:37 +01:00
Peter Nowak
e0831fa234 DATAES-525 - Fixed issue in scrolling not applying the search query.
The root cause of the deletion problem was the doScroll which did not apply the given query and therefor returned all entries from the index.
The doScroll implementation has been fixed to apply the query and multiple unit tests have been added to ensure that the delete methods delete the desired documents only and leave the rest untouched.
Also added unit tests for the scrolling to test it against real queries (not only matchAll).

Original pull request: #240
2019-02-21 08:27:30 +01:00
Ivan Greene
c77b543fb8 DATAES-523 - Allow specifying version type.
Allow specifying the version type for documents.

Original pull request: #236
2019-02-21 08:24:29 +01:00
Peter Nowak
a89a44b08f DATAES-531 - Fixed getMapping from ElasticsearchRestTemplate.
The getMapping methods in the ElasticsearchRestTemplate now behave like the methods in the ElasticsearchTemplate and return the mapping for the specified index and type.

Original pull request: #239
2019-01-24 17:24:38 +01:00
Christoph Strobl
69dc36c6c3 DATAES-519 - Add reactive repository support.
Reactive Elasticsearch repository support builds on the core repository support utilizing
operations provided via ReactiveElasticsearchOperations executed by a ReactiveElasticsearchClient.

Spring Data Elasticsearchs reactive repository support uses Project Reactor as its reactive
composition library of choice.

There are 3 main interfaces to be used:

* ReactiveRepository
* ReactiveCrudRepository
* ReactiveSortingRepository

For Java configuration, use the @EnableReactiveElasticsearchRepositories annotation.
The following listing shows how to use Java configuration for a repository:

@Configuration
@EnableReactiveElasticsearchRepositories
public class Config extends AbstractReactiveElasticsearchConfiguration {

  @Override
  public ReactiveElasticsearchClient reactiveElasticsearchClient() {
    return ReactiveRestClients.create(ClientConfiguration.localhost());
  }
}

Using a repository that extends ReactiveSortingRepository makes all CRUD operations available
as well as methods for sorted access to the entities. Working with the repository instance is a matter of dependency
injecting it into a client.

The repository itself allows defining additional methods backed by the inferred proxy.

public interface ReactivePersonRepository extends ReactiveSortingRepository<Person, String> {

  Flux<Person> findByFirstname(String firstname);

  Flux<Person> findByFirstname(Publisher<String> firstname);

  Flux<Person> findByFirstnameOrderByLastname(String firstname);

  Flux<Person> findByFirstname(String firstname, Sort sort);

  Flux<Person> findByFirstname(String firstname, Pageable page);

  Mono<Person> findByFirstnameAndLastname(String firstname, String lastname);

  Mono<Person> findFirstByLastname(String lastname);

  @Query("{ \"bool\" : { \"must\" : { \"term\" : { \"lastname\" : \"?0\" } } } }")
  Flux<Person> findByLastname(String lastname);

  Mono<Long> countByFirstname(String firstname)

  Mono<Boolean> existsByFirstname(String firstname)

  Mono<Long> deleteByFirstname(String firstname)
}

Original Pull Request: #235
2019-01-23 13:54:44 +01:00
Mark Paluch
f3fb779923 DATAES-524 - Update copyright years to 2019. 2019-01-02 12:36:07 +01:00
zzt
e264bd7f84 DATAES-487 - Support for multi search API. 2018-12-21 15:26:14 +00:00
Christoph Strobl
a1c139445f DATAES-518 - Use scroll for unpaged find operations in ReactiveElasticsearchTemplate.
We now use scroll instead of a max page size for unpaged queries.

Original pull request: #233.
2018-12-14 09:43:25 +01:00
Christoph Strobl
a2fad72e86 DATAES-515 - Revert override of Elasticsearch's JarHell for tests.
This reverts commit 25b02f29a741aa659cb8e49ce1d4c85e43c72e84 as the upgrade to Elasticsearch 6.5 made the changes obsolete.
2018-12-11 08:28:03 +01:00
Christoph Strobl
a3a46b2e11 DATAES-510 - Add tests for resource clean up.
Original Pull Request: #231
2018-12-11 07:17:10 +01:00
Christoph Strobl
da9de6bc49 DATAES-510 - Add reactive scroll support.
The ReactiveElasticsearchClient now support scrolling through large result sets issuing subsequent _search/scroll requests while emitting data on the outbound channel. Resources bound via their scrollId get freed on completion of the flux.

Original Pull Request: #231
2018-12-11 07:17:10 +01:00
Christoph Strobl
ce124a2d9e DATAES-512 - Fix request parameters not getting added to the URI as query string parameters.
We now make sure to include request parameters in the constructed URI and add potential request option headers.
2018-12-11 07:17:10 +01:00
Christoph Strobl
1ea73d2fb9 DATAES-504 - Update documentation.
Update documentation to cover newly added configuration options for the ReactiveElasticsearchClient.
Make sure to apply postFilter correctly and set a default limit for unpaged search requests.

Also fix some code format issues.
2018-12-11 07:17:10 +01:00
Mark Paluch
2dcd1cfbad DATAES-504 - Add configuration options for logging and timeouts to ReactiveElasticsearchClient.
We now log HTTP requests and responses with the org.springframework.data.elasticsearch.client.WIRE logger for both, the HighLevelRestClient and our reactive client and associate a logging Id for improved traceability.

Configuration of connection/socket timeouts is now available via the ClientConfiguration.

Along the lines we also aligned entity handling to EntityOperations already common in other modules. EntityOperations centralizes how aspects of entities (versioning, retrieval of index name/index type) are handled.

Original Pull Request: #229
2018-12-11 07:17:10 +01:00
Christoph Strobl
ba890cb7eb DATAES-504 - Add ReactiveElasticsearchOperations & ReactiveElasticsearchTemplate
ReactiveElasticsearchOperations is the gateway to executing high level commands against an Elasticsearch cluster using the ReactiveElasticsearchClient.

The ReactiveElasticsearchTemplate is the default implementation of ReactiveElasticsearchOperations and offers the following set of features.

* Read/Write mapping support for domain types.
* A rich query and criteria api.
* Resource management and Exception translation.

To get started the ReactiveElasticsearchTemplate needs to know about the actual client to work with.
The easiest way of setting up the ReactiveElasticsearchTemplate is via AbstractReactiveElasticsearchConfiguration providing
dedicated configuration method hooks for base package, the initial entity set etc.

@Configuration
public class Config extends AbstractReactiveElasticsearchConfiguration {

    @Bean
    @Override
    public ReactiveElasticsearchClient reactiveElasticsearchClient() {
        // ...
    }
}

NOTE: If applicable set default HttpHeaders via the ClientConfiguration of the ReactiveElasticsearchClient.

TIP: If needed the ReactiveElasticsearchTemplate can be configured with default RefreshPolicy and IndicesOptions that get applied to the related requests by overriding the defaults of refreshPolicy() and indicesOptions().

The ReactiveElasticsearchTemplate lets you save, find and delete your domain objects and map those objects to documents stored in Elasticsearch.

@Document(indexName = "marvel", type = "characters")
public class Person {

    private @Id String id;
    private String name;
    private int age;

    // Getter/Setter omitted...
}

template.save(new Person("Bruce Banner", 42)) // save a new document
    .doOnNext(System.out::println)
    .flatMap(person -> template.findById(person.id, Person.class)) // then go find it
    .doOnNext(System.out::println)
    .flatMap(person -> template.delete(person)) // just to remove remove it again
    .doOnNext(System.out::println)
    .flatMap(id -> template.count(Person.class)) // so we've got nothing at the end
    .doOnNext(System.out::println)
    .subscribe(); // yeah :)

The above outputs the following sequence on the console.

> Person(id=QjWCWWcBXiLAnp77ksfR, name=Bruce Banner, age=42)
> Person(id=QjWCWWcBXiLAnp77ksfR, name=Bruce Banner, age=42)
> QjWCWWcBXiLAnp77ksfR
> 0

Original Pull Request: #229
2018-12-11 07:17:10 +01:00
Christoph Strobl
a39c34058b DATAES-488 - Polishing & Documentation.
Rename VerificationMode -> Verification. Reorder methods in ReactiveElasticsearchClient, add test for DefaultWebClientProvider. Enforce assertions and fix some overall code style issues.
Add client reference documentation section.
2018-12-11 07:17:10 +01:00
Mark Paluch
390d7e8273 DATAES-488 - Polishing.
Convert spaces to tabs for pom.xml. Switch reactive dependencies to optional. Remove unused commonscollections property. Use managed versions for reactor and Spring dependencies.

Introduce WebClientProvider to avoid reinstantiation of WebClient instances. Introduce ClientConfiguration to encapsulate common Elasticsearch client configuration properties. Split ElasticsearchClients into RestClients and ReactiveRestClients to avoid mandatory dependency on WebFlux/Project Reactor. Adapt tests and code referring to WebClient creation.

Extract response body as byte array instead of Flux of DataBuffer to avoid chunking and to parse an entire response.

Encapsulate hostAndPort string used across configuration/HostProvider with InetSocketAddress. Add parser for InetSocketAddress.

Original Pull Request: #226
2018-12-11 07:17:10 +01:00
Christoph Strobl
691a8c57bc DATAES-488 - Add reactive Elasticsearch client support.
Initial implementation of a ReactiveElasticsearchClient using WebClient to connect to cluster nodes.

ReactiveElasticsearchClient client = ElasticsearchClients.createClient()
  .connectedTo("http://localhost:9200", "http://localhost:9201")
  .reactive();
A HostProvider selects active nodes and routes requests.

client.index(request ->

  request.index("spring-data")
    .type("elasticsearch")
    .id(randomUUID().toString())
    .source(singletonMap("feature", "reactive-client"))
    .setRefreshPolicy(IMMEDIATE);
);
This implementation provides the first building block for reactive Template and Repository support to be added subsequently.

Along the lines we upgraded to Elasticsearch 6.5.

Original Pull Request: #226
2018-12-11 07:17:10 +01:00
Mark Paluch
25b02f29a7 DATAES-515 - Override Elasticsearch's JarHell for tests. 2018-12-10 18:39:26 +01:00
xhaggi
89d0633fd5 DATAES-33 - Polishing
* Move @Parent property recognition to ElasticsearchPersistentProperty
* Move type contraints for @Version and @Parent fields to ElasticsearchPersistentProperty to fail faster
* remove unused constant SUPPORTED_ID_TYPES from SimpleElasticsearchPersistentProperty

Original pull request: #208
2018-11-20 11:45:26 +01:00
xhaggi
527f669f76 DATAES-503 - Added missing copy_to property to @Field annotation.
Original pull request: #227
2018-11-20 11:23:05 +01:00
xhaggi
dcf4cbe32a DATAES-492 - Add missing normalizer property to @Field and @InnerField.
Original pull request: #222
2018-11-20 09:21:51 +01:00
Artur Konczak
32a4f4ea7a DATAES-407 - improved tests for rest/transport template 2018-09-29 10:12:57 +01:00
Artur Konczak
4d4a6390e1 DATAES-407 - removed dependency with apache commons 2018-09-29 10:12:57 +01:00
Don Wellington
2f0b9b718b DATAES-407 - Support for HighLevelRestClient via ElasticsearchRestTemplate
Original pull request: #216
2018-09-29 10:10:54 +01:00
jnizet
b8324f9205 DATAES-479 - Allow specifying a HighlightBuilder when creating a query.
Original pull request: #217.
2018-08-24 15:59:12 +02:00
Oliver Gierke
469455383f
DATAES-470 - Fixed parsing of cluster nodes in TransportClientFactoryBean.
Extracted ClusterNodes value object to capture the parsing logic and actually properly test it. Added unit tests to verify the proper rejection and the two cases outlined in the ticket.

Related tickets: DATAES-283.
2018-07-12 21:48:31 +02:00
xhaggi
ba3eba5734
DATAES-467 - Fix sorting by _score if Spring Data sort is used.
Original pull request: #209.
2018-06-18 11:18:31 +02:00
Oliver Gierke
5ddb46c435 DATAES-462 - Polishing.
SimpleElasticsearchPersistentProperty now already checks for the correct type of score properties. Added unit tests for that. Also added unit tests for SimpleElasticsearchPersistentEntity rejecting more than one score property being present.

Additional non-null assertions on components that are required so that we can remove superfluous null checks.

A bit o formatting, Javadoc, missing @since tags and license headers.

Original pull request: #207.
2018-06-13 19:00:36 +02:00
xhaggi
d996406113 DATAES-462 - Add support for mapping max score and document scores.
Original pull request: #207.
2018-06-13 18:58:13 +02:00
Oliver Gierke
62a03a8fb7 DATAES-464 - DefaultEntityWriter now considers read-only and transient properties.
We now register a custom Jackson module that filters read-only and transient properties for serialization.
2018-06-13 18:21:37 +02:00
Ted Liang
e8e407f93b DATAES-312 - Fix NullHandling.NULLS_LAST in query.sort
Original pull request: #163
2018-05-17 09:31:59 +02:00
Nordine Bittich
1b0006f9f7 DATAES-420 - Analyzer of main field ignored when using @MultiField annotation 2018-05-15 17:16:29 +02:00
xhaggi
ace5a21b05 DATAES-285 - Polishing
* drop superfluous class FieldIndex
* change FieldType text and keyword starting with capital letters

Original commit: 089d7746be2f2fc4a395bd5c814f664729121f21
2018-05-07 13:19:37 +02:00
xhaggi
9c235cd53c DATAES-412 - only the last highlight field is added to SearchRequest 2018-05-07 11:39:58 +02:00
Chris White
ef1dca31e4 DATAES-198 - Fixed @Version annotation on fields. 2018-04-23 17:33:54 +02:00
Alen Turkovic
de1afe8bb0 DATAES-422 - Add support for IndicesOptions in search queries.
See also: DATAES-191..
2018-04-20 14:32:01 +02:00
Oliver Gierke
92c3bbebed DATAES-363 - Polishing.
Original pull request: #183.
2018-04-18 12:36:27 +02:00