Commit Graph

275 Commits

Author SHA1 Message Date
Peter Turcsanyi d2f80b1645
NIFI-11260 Added STS SSL Context Service for AWS Credentials Provider
This closes #7021

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-13 09:39:30 -05:00
exceptionfactory 512155ba2f
NIFI-10975 Add Kubernetes Leader Election and State Provider (#6779)
* NIFI-10975 Added Kubernetes Leader Election and State Provider
- Added Kubernetes Leader Election Manager based on Kubernetes Leases
- Added Kubernetes State Provider based on Kubernetes ConfigMaps
- Added nifi-kubernetes-client for generalized access to Fabric8 Kubernetes Client
- Added nifi.cluster.leader.election.implementation Property defaulting to CuratorLeaderElectionManager
- Refactored LeaderElectionManager to nifi-framework-api for Extension Discovering Manager
- Refactored shared ZooKeeper configuration to nifi-framework-cluster-zookeeper

* NIFI-10975 Updated Kubernetes Client and StateMap
- Upgraded Kubernetes Client from 6.2.0 to 6.3.0
- Added getStateVersion to StateMap and deprecated getVersion
- Updated Docker start.sh with additional properties

* NIFI-10975 Corrected MockStateManager.assertStateSet()
* NIFI-10975 Upgraded Kubernetes Client from 6.3.0 to 6.3.1
* NIFI-10975 Corrected unregister leader and disabled release on cancel

* NIFI-10975 Corrected findLeader handling of Lease expiration
- Changed LeaderElectionManager.getLeader() return to Optional String

* NIFI-10975 Corrected StandardNiFiServiceFacade handling of Optional Leader
* NIFI-10975 Changed getLeader() to call findLeader() to avoid stale cached values
* NIFI-10975 Updated LeaderElectionCommand to run LeaderElector in loop
* NIFI-10975 Rebased on project version 2.0.0-SNAPSHOT

* NIFI-10975 Corrected Gson and AspectJ versions
- Updated versions to match current main branch and avoid reverting
2023-03-07 14:14:24 -05:00
Joe Witt 43eab4c81d
NIFI-11103 prepping for 2.0.0 line 2023-02-09 15:32:53 -07:00
dan-s1 e86786433a
NIFI-11108 Further changes to upgrade from JUnit 4 to JUnit 5.
Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>

This closes #6919
2023-02-02 12:27:42 +00:00
Kalman Jantner cc8d108975
NIFI-10710 Added Processors for AWS Polly, Textract, Translate, Transcribe
This closes #6589

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-01-27 11:46:28 -06:00
Pierre Villard bb886af69d NIFI-11010 - Update Kinesis client to 1.14.9
Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>

This closes #6809
2022-12-29 11:29:48 +00:00
Peter Turcsanyi ae5d23db47
NIFI-10987: Added default implementation of AWS SDK V2 credentials in AWSCredentialsProviderService
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #6787.
2022-12-15 16:43:04 -05:00
Peter Turcsanyi 1e23e5146f NIFI-10969: Created extension point for Signer Override in AWS S3 processors
Also added Signer Override property in AWSCredentialsProviderControllerService with Custom Signer extension point.
Made Assume Role related properties dependent on Assume Role ARN property in AWSCredentialsProviderControllerService.
Fixed S3 IT tests.

This closes #6777.

Signed-off-by: Tamas Palfy <tpalfy@apache.org>
2022-12-15 18:43:22 +01:00
Joe Gresock 31af82275e
NIFI-10791 Added AWS SDK v2 support to AWSCredentialsProviderControllerService
This closes #6661

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-11-28 14:36:12 -06:00
Joe Witt 1046b2c308
Merge branch 'NIFI-10854-RC1' 2022-11-28 08:43:56 -07:00
Peter Turcsanyi 40b46b4f5d
NIFI-10850: Fixed Query Parameters property in InvokeAWSGatewayApi should support FlowFile attributes in EL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6692.
2022-11-23 13:04:58 +01:00
Joe Witt 75e7c9e47c
NIFI-10854-RC1 prepare for next development iteration 2022-11-22 19:22:47 -07:00
Joe Witt ec87bf93ad
NIFI-10854-RC1 prepare release nifi-1.19.0-RC1 2022-11-22 19:22:45 -07:00
Marco Carlino 6ce2e3799c
NIFI-10349 add maximum object age property to list s3
This closes #6293.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-11-07 14:44:53 +01:00
Lehel Boér feb3148b68
NIFI-10628: Added Default Schedule Time and Run Duration to processors
This closes #6512

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-10-21 14:57:46 -05:00
Joe Gresock 704ca63cc7
NIFI-10643: Paging secrets in AwsSecretsManagerParameterProvider
This closes #6532

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-10-15 09:09:14 -05:00
Andrew Lim da0fb234d6
NIFI-10288 Better describe forward slash requirements for Prefix property in ListS3
This closes #6477

Co-authored-by: Andrew Lim <andrewlim.apache@gmail.com>
Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-10-10 11:40:57 -05:00
Matt Gilman e638bcee2f
NIFI-10617:
- Adding dynamic property annotation on a number of extensions that were missing it.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6501
2022-10-10 10:32:50 -04:00
Joe Witt c9ebdd2025
NIFI-10521-RC4 prepare for next development iteration 2022-10-03 10:59:36 -07:00
Joe Witt 109e54cd58
NIFI-10521-RC4 prepare release nifi-1.18.0-RC4 2022-10-03 10:59:34 -07:00
Joe Gresock 315e54a812
NIFI-9636: Adding AwsSecretsManagerParameterProvider
This closes #6392

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-09-29 11:59:04 -05:00
exceptionfactory ae14ef2d1f
NIFI-10373 Set managed version for AWS 1 and 2 SDK
- Replaced individual AWS SDK versions with root managed dependency version
- Set AWS SDK 1 version to 1.12.299
- Set AWS SDK 2 version to 2.17.270
- Suppressed false positive dependency vulnerability for aws-sdk-swf-libraries

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6405.
2022-09-13 15:06:42 +02:00
mr1716 5303dd13aa
NIFI-10389 Upgrade AWS Kinesis to 1.14.8
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6328.
2022-08-24 11:46:32 +02:00
Joe Witt 8c66bf948c
NIFI-10272-RC2 prepare for next development iteration 2022-07-27 13:24:09 -07:00
Joe Witt 8d256784d8
NIFI-10272-RC2 prepare release nifi-1.17.0-RC2 2022-07-27 13:24:06 -07:00
exceptionfactory 7d920b9bc6
NIFI-10270 This closes #6240. Upgraded AWS SDK from 1.12.182 to 1.12.267
- Upgraded MiNiFi AWS SDK references from 1.11.172 to 1.12.267

Signed-off-by: Joe Witt <joewitt@apache.org>
2022-07-23 17:59:04 -07:00
patalwell 0cd438d9e9
NIFI-9817 Added Unit Validator to PutCloudWatchMetric
This closes #5905

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-07-23 10:18:27 -05:00
Lehel Boér 2072e4ed2a NIFI-10105: FetchS3Object decodes Content-Disposition
This closes #6109.

Signed-off-by: Tamas Palfy <tpalfy@apache.org>
2022-06-09 14:26:32 +02:00
exceptionfactory 0c8305950c
NIFI-9987 Standardize Commons IO on version 2.11.0
- Removed version declarations from multiple modules
- Adjusted PutDynamoDB to remove catch for IOException not thrown in Commons IO 2.11.0

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6015.
2022-05-09 12:06:37 +02:00
exceptionfactory 4f423a59ba
NIFI-9952 Upgraded Jackson BOM to 2.13.2.20220328
- Removed unnecessary references to jackson.version property
- Removed unnecessary dependency management references to Jackson libraries

This closes #5992

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-04-23 08:23:39 -04:00
Joe Witt 873d25585c
NIFI-9780 Merge branch 'NIFI-9780-RC3' 2022-04-04 08:35:23 -07:00
Joe Gresock a75381abbe NIFI-9827: Upgrading AWS Java SDK to 1.12.182 to pick up new AWS Regions
This closes #5892

Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>
2022-03-24 13:24:28 +00:00
Joe Witt 0419dc2939
NIFI-9780-RC3 prepare for next development iteration 2022-03-21 11:58:27 -07:00
Joe Witt b019a9191f
NIFI-9780-RC3 prepare release nifi-1.16.0-RC3 2022-03-21 11:58:09 -07:00
Bence Simon b86466b4a5 NIFI-9669 Adding PutDynamoDBRecord processor
This closes #5761.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-03-09 16:20:33 +01:00
Mike Thomsen f35f010deb
NIFI-9085 Refactored the Elasticsearch bundle to use JUnit 5
- NIFI-9086 Refactored nifi-accumulo-bundle to use JUnit 5
- NIFI-9087 Updated nifi-ambari-bundle to use JUnit 5
- NIFI-9088 Refactored nifi-amqp-bundle to use JUnit 5
- NIFI-9089 Updated nifi-asn1-bundle to use JUnit 5
- NIFI-9090 Refactored nifi-atlas-bundle to use JUnit 5
- NIFI-9091 Refactored nifi-avro-bundle to use JUnit 5
- NIFI-9092 Refactored nifi-aws-bundle to use JUnit 5
- NIFI-9095 Refactored nifi-azure-bundle to use JUnit 5
- NIFI-9145 Refactored nifi-rethinkdb-bundle to use JUnit 5
- NIFI-9143 Refactored nifi-redis-bundle to use JUnit 5

This closes #5350

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-07 21:49:23 -06:00
exceptionfactory fee7c16732
NIFI-9679 Added access-environment-credentials permission
- Applied new permission restrictions to AWSCredentialsProviderControllerService and GCPCredentialsControllerService

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5796.
2022-03-02 12:31:09 -05:00
lawye1973 0ade040b76
NIFI-9696 : DeleteS3Object don't have provenance
Changed from AmazonS3 to AmazonS3client.
Add REMOTE_INVOCATION provenance event with URL and "Object deleted"

This closes #5778

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-02-22 08:01:59 -05:00
lawye1973 54b4209798
NIFI-9713 : TagS3Object do not have provenance data
Changed from AmazonS3 to AmazonS3client.
Add REMOTE_INVOCATION provenance event with URL and "Object tagged"

This closes #5785

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-02-22 07:58:45 -05:00
lawye1973 8d80fb2930
NIFI-9694 : Correct FetchS3Object.java provenance data URL
Current provenance data URL is incorrect as it's hardcoded.
Follow PutS3Object.java method, using AmazonS3Client (instead of AmazonS3) to use getResourceUrl(bucket,key) to build the correct URL for provenance data

This closes #5779

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-02-19 18:46:20 -05:00
exceptionfactory fe1139b8bb
NIFI-9692 Upgraded Apache Commons Lang3 to 3.12.0
- Moved commons-lang3 version management to root Maven configuration
- Refactored limited usage of StringUtils is nifi-reporting-utils to remove commons-lang3
- Refactored limited usage of StringUtils in nifi-websocket-processors to remove commons-lang3

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5773
2022-02-16 09:28:21 -05:00
exceptionfactory a2f6420f43
NIFI-9638 Refactored Google Guava usage in extensions
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5742
2022-02-09 16:39:53 -05:00
exceptionfactory ba0bf2ede0
NIFI-9650 Upgraded OkHttp from 4.9.2 to 4.9.3
- Added okhttp-bom to root Maven configuration
- Removed repeated okhttp.version property from specific dependencies

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5737.
2022-02-02 11:45:18 +01:00
exceptionfactory dc7d9510cd NIFI-9610 Refactored nifi-processor-utils to separate modules
- Added nifi-bin-manager
- Added nifi-event-listen
- Added nifi-event-put
- Added nifi-listed-entity
- Added nifi-put-pattern
- Removed nifi-processor-utils

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5729.
2022-02-01 12:59:16 -05:00
Tamas Palfy 990285ba1c NIFI-8676 Added 'Tracking Entities' listing strategy to 'ListS3' and 'ListGCSBucket'
This closes #5413.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-01-10 20:35:04 +01:00
exceptionfactory 6039095625
NIFI-9267 Replaced nifi-standard-web-test-utils with mockwebserver
- Replaced instances of custom Jetty TestServer with OkHttp MockWebServer
- Removed integration tests referencing nifi-standard-web-test-utils SslContextUtils
- Removed nifi-standard-web-test-utils

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5620.
2022-01-09 05:25:32 -05:00
Chris Sampson bd1b2b4d69 NIFI-9520 update AWS SDK and Kinesis Client Library versions; Allow PutS3Object to use all available StorageClasses
This closes #5632.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-01-06 13:43:23 +01:00
Paul Grey 0d49371c53
NIFI-7371 Added FlowFile attributes for exceptions in S3 processors
This closes #5606

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-12-21 22:35:31 -06:00
exceptionfactory 7c6bdcb035
NIFI-9491 Excluded commons-logging and added jcl-over-slf4j references
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5608
2021-12-17 07:21:03 -05:00
Joe Gresock f07c37285e NIFI-9353: Adding Config Verification to AWS Processors
This closes #5504
2021-12-11 14:33:23 +00:00