Commit Graph

90 Commits

Author SHA1 Message Date
dan-s1 8c045268ba
NIFI-11778 refactored Groovy tests in nifi-elasticsearch-restapi-processors to Java (and JUnit 5)
This closes #7537

Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>
2023-08-29 05:56:15 +01:00
Chris Sampson fb502cdf9a NIFI-11480 add option to group PutElasticsearchRecord errors by Elasticsearch _bulk error type
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7441
2023-08-23 16:00:28 -04:00
Pierre Villard 150bfbe990
NIFI-11813 Removed Event Driven Scheduling Strategy
This closes #7483

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-07-18 12:22:52 -05:00
Ryan Van Den Bos 658f2547d8
NIFI-11430 - PaginatedJsonQueryElasticsearch processors should not output empty FlowFile if hits have been found; PaginatedJsonQueryElasticsearch processors should be able to use _source and _meta only result formats when grouping by query
This closes #7163

Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>
2023-05-13 19:35:20 +01:00
Chris Sampson 3a3b9edc71
NIFI-10067: enable use of script when updating documents in Elasticsearch via PutElasticsearchJson or PutElasticsearchRecord
NIFI-3262: enable use of dynamic_templates in Elasticsearch _bulk operations

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #6628.
2023-05-02 07:17:17 -04:00
Pierre Villard 7fb2468f4b NIFI-11502 - Upgrade json-path to 2.8.0
NIFI-11502 fixing unit tests
NIFI-11502 review

This closes #7208

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2023-04-29 16:03:16 -04:00
Chris Sampson 99b2412ad4
NIFI-11483 Correctly use DescribedValue for JsonQueryElasticsearch
This closes #7195

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-26 11:57:14 -05:00
Chris Sampson 05418d94f0 NIFI-11111 add option to output Elasticsearch error responses as FlowFile to PutElasticsearchJson and PutElasticsearchRecord
NIFI-11111 clarify error_responses relationships in PutElasticsearchJson/Record processors
NIFI-11111 Refactor exception handling for error response flowfile transfer
NIFI-11111 Add elasticsearch.bulk.error attributes containing the Elasticsearch _bulk response for error documents in PutElasticsearchJson

This closes #6903

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2023-04-25 06:41:44 -04:00
Chris Sampson f71ba8aadb
NIFI-11274 only add @timestamp to PutElasticsearchRecord document if not null
This closes #7036.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2023-03-13 14:11:19 -04:00
Joe Witt 43eab4c81d
NIFI-11103 prepping for 2.0.0 line 2023-02-09 15:32:53 -07:00
dan-s1 e86786433a
NIFI-11108 Further changes to upgrade from JUnit 4 to JUnit 5.
Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>

This closes #6919
2023-02-02 12:27:42 +00:00
Chris Sampson d9420afb60
NIFI-10844 Allow _source only output for GetElasticsearch and JsonQueryElasticsearch
This closes #6687

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-01-04 15:30:41 -06:00
Joe Witt 1046b2c308
Merge branch 'NIFI-10854-RC1' 2022-11-28 08:43:56 -07:00
Ryan Van Den Bos 2ad33eea80 NIFI-10845 - JsonQueryElasticsearch processors are not outputting an empty flow file for a combined response with output_no_hits set to true
Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>

This closes #6701
2022-11-23 18:27:28 +00:00
Joe Witt 75e7c9e47c
NIFI-10854-RC1 prepare for next development iteration 2022-11-22 19:22:47 -07:00
Joe Witt ec87bf93ad
NIFI-10854-RC1 prepare release nifi-1.19.0-RC1 2022-11-22 19:22:45 -07:00
Chris Sampson 2acfd1e496 NIFI-9398 add verification to ElasticSearchClientService (with integration tests) and Elasticsearch REST API processors
NIFI-9398 processor verification integration-tests for nifi-elasticsearch-restapi-processors using common nifi-elasticsearch-test-utils refactored from nifi-elasticsearch-client-service
NIFI-9398 fail index exists verification for Elasticsearch processors whose operations would not create the index if it doesn't already exist

This closes #6544

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-11-05 18:17:13 -04:00
Chris Sampson b6026f5709 NIFI-10303 route GetElasticsearch to failure if _id is blank after attribute evaluation
This closes #6573

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-10-25 14:30:09 -04:00
Chris Sampson db49a861b3 NIFI-10687 set Elasticsearch document _id to null if ID attribute evaluated to blank String for PutElasticsearchRecord or PutElasticsearchJson; use @timestamp default value if @timestamp record path evaluates to blank String in PutElasticsearchRecord
This closes #6575

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-10-25 14:23:19 -04:00
Joe Witt c9ebdd2025
NIFI-10521-RC4 prepare for next development iteration 2022-10-03 10:59:36 -07:00
Joe Witt 109e54cd58
NIFI-10521-RC4 prepare release nifi-1.18.0-RC4 2022-10-03 10:59:34 -07:00
Joe Witt 8c66bf948c
NIFI-10272-RC2 prepare for next development iteration 2022-07-27 13:24:09 -07:00
Joe Witt 8d256784d8
NIFI-10272-RC2 prepare release nifi-1.17.0-RC2 2022-07-27 13:24:06 -07:00
Chris Sampson 9c9c708c14
NIFI-10043 fix ElasticsearchClient integration tests following JUnit5 upgrade; update Elasticsearch and json-path dependency versions
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #6062.
2022-05-20 12:27:53 -04:00
exceptionfactory 0c8305950c
NIFI-9987 Standardize Commons IO on version 2.11.0
- Removed version declarations from multiple modules
- Adjusted PutDynamoDB to remove catch for IOException not thrown in Commons IO 2.11.0

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6015.
2022-05-09 12:06:37 +02:00
exceptionfactory 4f423a59ba
NIFI-9952 Upgraded Jackson BOM to 2.13.2.20220328
- Removed unnecessary references to jackson.version property
- Removed unnecessary dependency management references to Jackson libraries

This closes #5992

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-04-23 08:23:39 -04:00
Joe Witt 0419dc2939
NIFI-9780-RC3 prepare for next development iteration 2022-03-21 11:58:27 -07:00
Joe Witt b019a9191f
NIFI-9780-RC3 prepare release nifi-1.16.0-RC3 2022-03-21 11:58:09 -07:00
Mike Thomsen f35f010deb
NIFI-9085 Refactored the Elasticsearch bundle to use JUnit 5
- NIFI-9086 Refactored nifi-accumulo-bundle to use JUnit 5
- NIFI-9087 Updated nifi-ambari-bundle to use JUnit 5
- NIFI-9088 Refactored nifi-amqp-bundle to use JUnit 5
- NIFI-9089 Updated nifi-asn1-bundle to use JUnit 5
- NIFI-9090 Refactored nifi-atlas-bundle to use JUnit 5
- NIFI-9091 Refactored nifi-avro-bundle to use JUnit 5
- NIFI-9092 Refactored nifi-aws-bundle to use JUnit 5
- NIFI-9095 Refactored nifi-azure-bundle to use JUnit 5
- NIFI-9145 Refactored nifi-rethinkdb-bundle to use JUnit 5
- NIFI-9143 Refactored nifi-redis-bundle to use JUnit 5

This closes #5350

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-07 21:49:23 -06:00
Chris Sampson 12685a4af4
NIFI-9766 avoid intermittent SearchElasticsearchTest failures in CI pipeline
This closes #5842

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-05 19:05:02 -05:00
Chris Sampson 4cc20e6c06
NIFI-9678 update Elasticsearch REST API processor integration-tests for Elasticsearch 8.x
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5822.
2022-03-02 10:04:34 -05:00
Chris Sampson 04d061a7bd
NIFI-9581 add PutElasticsearchRecord relationship for output of successful Records sent to Elasticsearch
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5665.
2022-02-22 11:42:01 -05:00
Chris Sampson 07131a66ea
NIFI-9715 add option to output empty FlowFile from Elasticsearch REST API Json Query processors when there are no hits from query
This closes #5786

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-02-22 06:32:56 -05:00
exceptionfactory 9075fc2cd8
NIFI-9611 Removed unnecessary references to nifi-processor-utils
- Replaced nifi-processor-utils with nifi-utils where necessary
- Added direct dependencies on commons-lang3 where necessary

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5693.
2022-01-26 11:52:08 -05:00
Chris Sampson 53809dd83f
NIFI-9439:
- Add PutElasticsearchJson processor to Elasticsearch REST bundle
- Deprecate PutElasticsearchHttp/PutElasticsearchHttpReccord in favour of Elasticsearch REST processors

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5566.
2021-12-05 15:39:35 -05:00
Chris Sampson d45b23d89f
NIFI-8153 custom date/time format properties for PutElasticsearchRecord
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5552.
2021-12-01 07:28:41 -05:00
Chris Sampson 1745d2a88e
- NIFI-9388: add GetElasticsearch to fetch Elasticsearch document using the ElasticsearchClientService
- NIFI-9387: add Proxy capability to ElasticsearchClientService
- NIFI-1576: allow GetElasticsearch to run without requiring FlowFile input

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5535.
2021-11-20 06:26:55 -05:00
Joe Witt c055895952
NIFI-9344-RC3 prepare for next development iteration 2021-11-03 08:53:37 -07:00
Joe Witt 7fdc07cccd
NIFI-9344-RC3 prepare release nifi-1.15.0-RC3 2021-11-03 08:53:32 -07:00
Chris Sampson f50c002692
- NIFI-8001: Dynamic Properties as Request Parameters for Elasticsearch Client Service processors
- NIFI-8003: UpdateByQueryElasticsearchProcessor
- Addressed various warnings and inefficiencies found in existing processor code

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #4693.
2021-10-08 07:33:49 -04:00
Chris Sampson 3892e50991
NIFI-7990: Add properties to map Record field as @timestamp in output to Elasticsearch for PutElasticsearchRecord and PutElasticsearchHttpRecord processors; NIFI-7474 allow mapped id field to be retained within the Record for PutElasticsearchRecord
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #4691.
2021-10-07 05:58:10 -04:00
exceptionfactory 4bcd03024a
NIFI-9283 Excluded Log4j 1.2 and upgraded Log4j2 to 2.14.1
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5440.
2021-10-06 11:17:36 -04:00
Chris Sampson 3a63bd579e
NIFI-8002:
- Creation of a PaginatedJsonQueryElasticsearch and ConsumeElasticsearch processors
- Integration Tests for ES Client Service with version and flavour-specific tests for applicable functionality

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5193.
2021-10-04 19:27:12 -04:00
Chris Sampson 2e1f276f06
NIFI-8790 allow Expression Language for Index Operation in PutElasticsearchRecord
Improved validation for PutElasticsearchRecord Index Operation property

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5220
2021-08-09 13:51:15 -04:00
Joe Witt 97feacc181
NIFI-8767-RC2 prepare for next development iteration 2021-07-10 12:17:09 -07:00
Joe Witt fcbf1d5f97
NIFI-8767-RC2 prepare release nifi-1.14.0-RC2 2021-07-10 12:17:05 -07:00
exceptionfactory 2efc2b6b8b
NIFI-8718 Upgraded Apache Commons IO to 2.10.0
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5169
2021-06-21 11:29:27 -04:00
rhavermans 9fd8042cfb
NIFI-8664 Update json-path lib from 2.4.0 -> 2.6.0
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5132
2021-06-21 11:01:10 -04:00
Chris Sampson e0a1cb8a01
NIFI-8133 Add ability to supress null/empty values in ElasticSearchClientService for PutElasticsearchRecord output
NIFI-8133 pass null/empty field handling from PutElasticsearchRecord to ElasticSearchClientImpl

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4755
2021-05-29 13:29:14 -04:00
exceptionfactory 6776765a92
NIFI-8538 Upgraded Apache Commons IO to 2.8.0
- Upgraded direct dependencies from 2.6 to 2.8.0
- Added dependency management configuration to use 2.8.0 for some modules
- Updated scripted Groovy tests to avoid copying unnecessary files

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5073
2021-05-14 09:31:48 -04:00