Commit Graph

315 Commits

Author SHA1 Message Date
Peter Turcsanyi d370b470b8
NIFI-12782 Migrated GCS processors' Proxy properties to ProxyConfigurationService
Extracted proxy service migration code into a common util module because the same logic was already used in AWS module,
and it is also reusable in other components for proxy property migration.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #8400.
2024-03-19 08:56:42 +01:00
Freedom9339 12fc2f8508
NIFI-6379 Added SSL Context to PutSNS, DeleteSQS, GetSQS, and PutSQS
This closes #8505

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-03-18 16:24:28 -05:00
Balázs Gerner 40d9750bb3
NIFI-12671 Added S3FileResourceService
This closes #8368.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2024-02-26 17:23:35 +01:00
Mark Payne ae423fc6ba
NIFI-12832 Removed unnecessary dependencies from nifi-mock
- Moved StandardValidators to nifi-api
- Moved URL creation method from UriUtils to URLValidator
- Separated FormatUtils into FormatUtils and DurationFormat classes
- Added DurationFormat to nifi-api

This closes #8442

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-02-23 15:13:21 -06:00
Peter Turcsanyi 0f4defa49a
NIFI-12771 Restored Client-side KMS encryption strategy for S3
This closes #8388

Signed-off-by: David Handermann <exceptionfactory@apache.org>v
2024-02-20 10:59:49 -06:00
Peter Turcsanyi 9ba68edb5f
NIFI-12766 Fixed Region handling for AWS Assume Role Credentials
This closes #8382

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-02-12 18:05:09 -06:00
Peter Turcsanyi d5206b1ddd
NIFI-12736 Fixed migration of credential properties in AWS processors
This closes #8353

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-02-09 08:39:29 -06:00
Peter Turcsanyi 8f63e928a9 NIFI-12732 ListS3 resets its tracking state after configuration change
This closes #8348.

Signed-off-by: Tamas Palfy <tpalfy@apache.org>
2024-02-06 14:52:16 +01:00
Balázs Gerner c1a21ad078
NIFI-12642 Added support for FileResourceService in PutS3Object
This closes #8295.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2024-01-29 18:40:19 +01:00
David Handermann 250fe90b34
NIFI-9458 Replaced SimpleDateFormat with DateTimeFormatter (#8248)
- Added FieldConverterRegistry and Standard implementation
- Moved DateTypeUtils methods to FieldConverter implementations
2024-01-16 16:39:40 -05:00
Juldrixx 9a91933909
NIFI-12441 Added No Tracking Strategy to ListS3
This closes #8088

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-01-16 09:42:48 -06:00
Juldrixx 281a28c5d4
NIFI-11288 Add AWS STS dependency for AssumeRoleWithWebIdentity method
This closes #7974

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-01-15 14:16:52 -06:00
p-kimberley 3ebad40fae
NIFI-12594: ListS3 - observe min/max object age when entity state tracking is used
This closes #8231.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2024-01-15 09:41:46 +01:00
Mark Payne 283abadf97
NIFI-12457: Added additional use case based documentation and fixed some typos/made some clarifications in related docs
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #8108.
2023-12-04 14:59:53 +01:00
Joe Gresock ec884ac091
NIFI-12212 Upgraded DynamoDB Processors to AWS SDK 2
This closes #7911

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-11-11 23:30:30 -06:00
mr1716 582c3ac7cc
NIFI-12347 Upgraded AWS Kinesis from 2.5.2 to 2.5.3
This closes #8007

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-11-11 23:28:11 -06:00
Joe Gresock 77834c92df
NIFI-12263 Upgraded AWS Machine Learning processors to SDK 2
This closes #7953

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-11-03 13:05:22 -05:00
exceptionfactory bd4ba34339
NIFI-12266 Added Standard Shared NAR and BOM (#7925)
- Added nifi-standard-shared-nar with common dependencies
- Added nifi-standard-shared-bom with provided scope for parent module references
2023-10-25 13:22:26 -04:00
Mark Payne a44b633252
NIFI-12220 Added ability to create Controller Services from migrateProperties
- Added ability to get raw property values from PropertyConfiguration instead of just effective values
- Updated TestRunner to allow for testing these migration methods
- Auto-enable newly created controller services if they are valid
- Eliminated Proxy properties in all AWS processors and instead just make use of the Proxy Configuration controller service
- Eliminated authentication properties from AWS processors and migrated all processors to using Controller Service or authentication

This closes #7874

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-10-24 09:14:19 -05:00
Joe Gresock 9627c4c0e8
NIFI-12174 Upgraded AWS PutLambda Processor to SDK 2
This closes #7851

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-10-11 16:41:39 -05:00
Mark Payne 0fd4ec50ad
NIFI-12144: Eliminated deprecation warnings by moving to newer Amazon SDK APIs. Eliminated several abstract classes that provided no value. Code cleanup. Refactored many of the integration tests to make use of Test Containers. Removed some tests that were incorrect
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes #7854.
2023-10-10 20:32:49 -04:00
exceptionfactory 98cebc6687
NIFI-12185 Added nifi-bom Bill of Materials (#7852)
- Imported nifi-bom to nifi-commons and nifi-nar-bundles parent modules
- Removed explicit version and scope properties for dependencies declared in nifi-bom
2023-10-10 13:55:22 -04:00
Joe Gresock a584251f0a
NIFI-12189: Upgrading PutCloudwatchMetric to use AWS SDK 2.x
This closes #7857

Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>
2023-10-08 18:24:37 +01:00
Joe Gresock ff8f1ff686
NIFI-8531: This closes #7529. Upgrading Kinesis processors to AWS SDK 2.x
Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-10-01 16:26:51 -07:00
Mark Payne db727aa419
NIFI-12142: This closes #7806. Deleted many methods, classes, and references that were deprecated.
Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-09-28 14:07:48 -07:00
Pierre Villard 43cc5ad122
NIFI-12079 This closes #7750. Remove Variables / Variable Registry
Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-09-25 09:39:53 -07:00
Pierre Villard 2aca08910c
NIFI-12103 Replaced deprecated usage of new URL(String)
This closes #7771

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-23 10:30:42 -05:00
Mark Payne 140ac52e9d
NIFI-12068: This closes #7737. Added the ability to annotate components with @UseCase and @MultiProcessorUseCase annotations; updated several processors to make use of them.
Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-09-22 10:47:47 -07:00
Peter Turcsanyi ebd0d60e49
NIFI-12075 Deprecated ComponentLog methods with Object[] and Throwable
- Updated component references to remove use of Object[] wrapping for log methods

This closes #7748

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-19 08:53:00 -05:00
Joseph Witt 71bdbc080b
NIFI-12066 Upgraded framework and cloud provider dependencies
- Upgraded SLF4J from 2.0.7 to 2.0.9
- Upgraded Logback from 1.3.8 to 1.3.11
- Upgraded Testcontainers from 1.18.3 to 1.19.0
- Upgraded Fabric8 Kubernetes from 6.5.1 to 6.8.1
- Upgraded AspectJ from 1.9.19 to 1.9.20.1
- Upgraded Caffeine from 3.1.6 to 3.1.8
- Upgraded AWS SDK from 2.20.103 to 2.20.148
- Upgraded Guava from 32.0.1 to 32.1.2
- Upgraded Nimbus JOSE JWT from 9.31 to 9.33
- Upgraded Apache Tika from 2.8.0 to 2.9.0
- Upgraded gRPC from 1.57.2 to 1.58.0
- Upgraded Google Libraries from 26.17.0 to 26.22.0
- Upgraded Azure SDK from 1.2.13 to 1.2.16

This closes #7733

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-15 13:05:31 -05:00
Pierre Villard 47f4c8ce2e NIFI-12028 - Add s3.region attribute to ListS3 generated flow files
This closes #7672

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2023-09-10 12:22:35 -04:00
mr1716 c878310e36 NIFI-11930 Update aws-kinesis-client-library-version to 1.15.0
This closes #7594

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2023-08-11 17:42:14 -04:00
Mark Payne 964f56043c
NIFI-11910: Clarified some points in docs
This closes #7574

Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>
2023-08-08 08:10:49 +01:00
Lehel 8a5f7f0092
NIFI-11827 Added AWS Glue Schema Registry Service
This closes #7492

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-07-20 14:07:25 -05:00
exceptionfactory 305277b833 NIFI-11709 Upgraded Guava from 32.0.0 to 32.0.1
- Added version override for Guava 27 to 32.0.1 in Hadoop and Hive modules

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7415
2023-06-20 16:22:00 -04:00
Emilio Setiadarma 31e1c17d44
NIFI-11642 Updated PutSNS using AWS SDK 2
This closes #7362

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-13 13:50:22 -05:00
exceptionfactory c5c711c91d
NIFI-11645 This closes #7342. Upgraded Guava from 31.1 to 32.0.0
- Added selective overrides to upgrade transitive dependencies

Signed-off-by: Joe Witt <joewitt@apache.org>
2023-06-05 14:33:10 -07:00
Joe Gresock 3ef4439879
NIFI-8287 Upgraded SQS Processors to use AWS SDK 2
This closes #7211

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-01 21:53:33 -05:00
mr1716 bfa4b178ee NIFI-11606 Update aws-kinesis-client-library-version to 1.14.10
This closes #7307

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2023-05-29 14:07:44 -04:00
krisztina-zsihovszki 9a002d9a43
NIFI-11255 Allowable value for 'Use s3.region Attribute'
This closes #7051.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-04-18 17:58:49 +02:00
Peter Turcsanyi d2f80b1645
NIFI-11260 Added STS SSL Context Service for AWS Credentials Provider
This closes #7021

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-13 09:39:30 -05:00
exceptionfactory 512155ba2f
NIFI-10975 Add Kubernetes Leader Election and State Provider (#6779)
* NIFI-10975 Added Kubernetes Leader Election and State Provider
- Added Kubernetes Leader Election Manager based on Kubernetes Leases
- Added Kubernetes State Provider based on Kubernetes ConfigMaps
- Added nifi-kubernetes-client for generalized access to Fabric8 Kubernetes Client
- Added nifi.cluster.leader.election.implementation Property defaulting to CuratorLeaderElectionManager
- Refactored LeaderElectionManager to nifi-framework-api for Extension Discovering Manager
- Refactored shared ZooKeeper configuration to nifi-framework-cluster-zookeeper

* NIFI-10975 Updated Kubernetes Client and StateMap
- Upgraded Kubernetes Client from 6.2.0 to 6.3.0
- Added getStateVersion to StateMap and deprecated getVersion
- Updated Docker start.sh with additional properties

* NIFI-10975 Corrected MockStateManager.assertStateSet()
* NIFI-10975 Upgraded Kubernetes Client from 6.3.0 to 6.3.1
* NIFI-10975 Corrected unregister leader and disabled release on cancel

* NIFI-10975 Corrected findLeader handling of Lease expiration
- Changed LeaderElectionManager.getLeader() return to Optional String

* NIFI-10975 Corrected StandardNiFiServiceFacade handling of Optional Leader
* NIFI-10975 Changed getLeader() to call findLeader() to avoid stale cached values
* NIFI-10975 Updated LeaderElectionCommand to run LeaderElector in loop
* NIFI-10975 Rebased on project version 2.0.0-SNAPSHOT

* NIFI-10975 Corrected Gson and AspectJ versions
- Updated versions to match current main branch and avoid reverting
2023-03-07 14:14:24 -05:00
Joe Witt 43eab4c81d
NIFI-11103 prepping for 2.0.0 line 2023-02-09 15:32:53 -07:00
dan-s1 e86786433a
NIFI-11108 Further changes to upgrade from JUnit 4 to JUnit 5.
Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>

This closes #6919
2023-02-02 12:27:42 +00:00
Kalman Jantner cc8d108975
NIFI-10710 Added Processors for AWS Polly, Textract, Translate, Transcribe
This closes #6589

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-01-27 11:46:28 -06:00
Pierre Villard bb886af69d NIFI-11010 - Update Kinesis client to 1.14.9
Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>

This closes #6809
2022-12-29 11:29:48 +00:00
Peter Turcsanyi ae5d23db47
NIFI-10987: Added default implementation of AWS SDK V2 credentials in AWSCredentialsProviderService
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #6787.
2022-12-15 16:43:04 -05:00
Peter Turcsanyi 1e23e5146f NIFI-10969: Created extension point for Signer Override in AWS S3 processors
Also added Signer Override property in AWSCredentialsProviderControllerService with Custom Signer extension point.
Made Assume Role related properties dependent on Assume Role ARN property in AWSCredentialsProviderControllerService.
Fixed S3 IT tests.

This closes #6777.

Signed-off-by: Tamas Palfy <tpalfy@apache.org>
2022-12-15 18:43:22 +01:00
Joe Gresock 31af82275e
NIFI-10791 Added AWS SDK v2 support to AWSCredentialsProviderControllerService
This closes #6661

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-11-28 14:36:12 -06:00
Joe Witt 1046b2c308
Merge branch 'NIFI-10854-RC1' 2022-11-28 08:43:56 -07:00