Commit Graph

20 Commits

Author SHA1 Message Date
Chris Sampson 53809dd83f
NIFI-9439:
- Add PutElasticsearchJson processor to Elasticsearch REST bundle
- Deprecate PutElasticsearchHttp/PutElasticsearchHttpReccord in favour of Elasticsearch REST processors

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5566.
2021-12-05 15:39:35 -05:00
Chris Sampson d45b23d89f
NIFI-8153 custom date/time format properties for PutElasticsearchRecord
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5552.
2021-12-01 07:28:41 -05:00
Chris Sampson 1745d2a88e
- NIFI-9388: add GetElasticsearch to fetch Elasticsearch document using the ElasticsearchClientService
- NIFI-9387: add Proxy capability to ElasticsearchClientService
- NIFI-1576: allow GetElasticsearch to run without requiring FlowFile input

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5535.
2021-11-20 06:26:55 -05:00
Chris Sampson f50c002692
- NIFI-8001: Dynamic Properties as Request Parameters for Elasticsearch Client Service processors
- NIFI-8003: UpdateByQueryElasticsearchProcessor
- Addressed various warnings and inefficiencies found in existing processor code

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #4693.
2021-10-08 07:33:49 -04:00
Chris Sampson 3892e50991
NIFI-7990: Add properties to map Record field as @timestamp in output to Elasticsearch for PutElasticsearchRecord and PutElasticsearchHttpRecord processors; NIFI-7474 allow mapped id field to be retained within the Record for PutElasticsearchRecord
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #4691.
2021-10-07 05:58:10 -04:00
exceptionfactory 4bcd03024a
NIFI-9283 Excluded Log4j 1.2 and upgraded Log4j2 to 2.14.1
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5440.
2021-10-06 11:17:36 -04:00
Chris Sampson 3a63bd579e
NIFI-8002:
- Creation of a PaginatedJsonQueryElasticsearch and ConsumeElasticsearch processors
- Integration Tests for ES Client Service with version and flavour-specific tests for applicable functionality

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5193.
2021-10-04 19:27:12 -04:00
Chris Sampson 2e1f276f06
NIFI-8790 allow Expression Language for Index Operation in PutElasticsearchRecord
Improved validation for PutElasticsearchRecord Index Operation property

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5220
2021-08-09 13:51:15 -04:00
Chris Sampson e0a1cb8a01
NIFI-8133 Add ability to supress null/empty values in ElasticSearchClientService for PutElasticsearchRecord output
NIFI-8133 pass null/empty field handling from PutElasticsearchRecord to ElasticSearchClientImpl

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4755
2021-05-29 13:29:14 -04:00
Chris Sampson 124cdbd3fe
NIFI-6403 and NIFI-6404: Elasticsearch 7 support
Addressed PR#4153 comments; removed ES Version property and made Type optional in all ES HTTP/Record processors, applying sensible default values where required; use _source queyr parameter instead of _source_include/s as it's compatible between ES versions

Fix unit test compilation to use JDK8-compatible library/method

Better optional type and id handling for PutElasticsearchRecord; update nifi-elasticsearch-client-service build dependencies to use latest versions of Elasticsearch in each supported major version (5/6/7); addressed several warnings in ElasticSearchClientServiceImpl

This closes #4667.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2020-11-26 15:08:13 +09:00
Chris Sampson 5c925371c2
NIFI-7474 Allow index operation type to be provided for PUT Elasticsearch processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4665.
2020-11-16 18:19:35 +01:00
Matthew Burgess 71226ce077
NIFI-7044: Fixed 'InputStream not closed' issue in PutElasticsearchRecord and DeleteHBaseCells
This closes #3997

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-01-18 10:39:47 -05:00
Mike Thomsen 4c79ff0576
NIFI-5248 Added new Elasticsearch processors.
NIFI-5248 Fixed a few stray 1.7.0 references.
NIFI-5248 Removed build helper plugin.
NIFI-5248 Made changes requested in a review.
NIFI-5248 Updated dummy service.
NIFI-5248 Made a few changes from a code review.
NIFI-5248 Added logic for removing nulls so record paths can be removed when no longer needed.
NIFI-5248 Switched from variable registry to flowfile level EL.
NIFI-5248 Added JsonPath code to remove index, id and type path statements.
NIFI-5248 Updated validation.
NIFI-5248 Set the field to null instead of empty string when nulling records.
NIFI-5248 Fixed TestElasticSearchClientService.
NIFI-5248 Removed high level client and switched over to low level client for everything.
NIFI-5248 Added profiles for ES 6 and ES 7 integration testing.
NIFI-5248 Updated integration tests to support 5 and 6.
NIFI-5248 Fixed some style check breaks.
NIFI-5248 Added create operation type.
NIFI-5248 Updated documentation.
NIFI-5248 Added error handling to PutElasticsearchRecord.
NIFI-5248 Added error logging to PutElasticsearchJson.
NIFI-5248 Added split failed records option to PutElasticsearchJson.
NIFI-5248 Added documentation for PutElasticsearchRecord.
NIFI-5248 Updated import to not use * import.
NIFI-5248 Removed processor that is no longer relevant due to schema inference.
NIFI-5248 Renamed ElasticSearch instances to Elasticsearch where we can within API guidelines.

NIFI-5248 Added groovy-json test dependency.

NIFI-5248 Updated PutElasticsearchRecord to only do index operations.

NIFI-5248 Added batch size property and refactored the way relationships and properties are added.

NIFI-5248 Added batch processing support.

NIFI-5248 Updated error handling.

NIFI-5248 Updated to 1.11.0-SNAPSHOT.

NIFI-5248 Made changes requested in a code review.

NIFI-5248 Made a few more changes from a code review.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2861
2019-11-19 13:53:34 -05:00
Mike Thomsen 12e384b4be NIFI-5389 Added additional documentation for JsonQueryElasticsearch.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2864
2018-07-09 12:24:02 -04:00
Mike Thomsen 0ff8518068 NIFI-5390 Added more documentation for DeleteByQueryElasticsearch.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2865
2018-07-09 11:51:15 -04:00
Mike Thomsen d2fd7e5e7d NIFI-5271 Moved JSON_VALIDATOR to its own maven module.
This closes #2765

Signed-off-by: zenfenan <zenfenan@apache.org>
2018-06-07 20:38:25 +05:30
zenfenan 504152eaa1 NIFI-5261: Added JSON_VALIDATOR to StandardValidators
This closes #2758

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-06-05 18:24:47 -04:00
Mike Thomsen 3a29c1e4ca NIFI-5052 Added DeleteByQuery ElasticSearch processor.
NIFI-5052 fixed checkstyle issue.

NIFI-5052 Added changes requested in a code review.

NIFI-5052 Fixed a typo.

NIFI-5052 Added changes requested in a code review.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2616
2018-05-23 09:14:04 -04:00
Pierre Villard 7cc77937d8 NIFI-4149 - fixed contrib-check errors 2018-04-06 20:38:04 +02:00
Mike Thomsen aa947e4d3e NIFI-4325 Added new processor that uses the JSON DSL.
NIFI-4325 Cleaned up how ElasticSearch client service builds SSLContext, added query attribute to flowfiles and other changes requested in a code review.

This closes #2113.

Signed-off-by: Joe Percivall <JPercivall@apache.org>
2018-03-26 15:01:37 -04:00