Commit Graph

173 Commits

Author SHA1 Message Date
exceptionfactory d382b378a8
NIFI-11387 Added Transport Type property to Azure Event Hub Components
- Transport Type defaults to AMQP and also supports AMQP with WebSockets

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7126.
2023-04-19 15:04:56 +03:00
Pierre Villard b0997f843d
NIFI-11416 Upgraded Caffeine from 2.8.1 to 3.1.6
This closes #7147

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-10 08:45:12 -05:00
exceptionfactory dbc627e0b0
NIFI-11261 Added Primary Node State handling to GetAzureEventHub
- Updated Qpid Proton J from 0.34.0 to 0.34.1

This closes #7023.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-03-09 22:31:51 +01:00
Peter Turcsanyi 5088934b79
NIFI-11256 Improved documentation for Azure Event Hubs Processors
- used the same ordering of 'Event Hub Namespace', 'Event Hub Name' and 'Service Bus Endpoint' properties
- removed "Primary" tag from 'Shared Access Policy Key' property name
- added some notes on clustering and consumer group in the documentation

This closes #7018

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-07 15:31:30 -06:00
Robert Kalmar c7c1a245a9
NIFI-11246 Document differences in GetAzureEventHub and ConsumeAzureEventHub
This closes #7006

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-06 07:54:23 -06:00
Nandor Soma Abonyi dd2fc4bb9d
NIFI-11173 Add default values to DeleteAzureBlobStorage_v12
This closes #6945.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-02-14 11:42:37 +01:00
Joe Witt 43eab4c81d
NIFI-11103 prepping for 2.0.0 line 2023-02-09 15:32:53 -07:00
dan-s1 e86786433a
NIFI-11108 Further changes to upgrade from JUnit 4 to JUnit 5.
Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>

This closes #6919
2023-02-02 12:27:42 +00:00
Malthe Borch 41649660be
NIFI-10721 Avoid querying properties after Azure Blob upload
This closes #6730

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-01-26 15:45:52 -06:00
Nandor Soma Abonyi caee606706
NIFI-10884 log target filename
This closes #6792.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-12-18 22:49:52 +01:00
Joe Witt 75e7c9e47c
NIFI-10854-RC1 prepare for next development iteration 2022-11-22 19:22:47 -07:00
Joe Witt ec87bf93ad
NIFI-10854-RC1 prepare release nifi-1.19.0-RC1 2022-11-22 19:22:45 -07:00
exceptionfactory 6062f16667
NIFI-10861 This closes #6705. Deprecated Azure Storage Processors using legacy Microsoft SDK
Signed-off-by: Joe Witt <joewitt@apache.org>
2022-11-22 15:34:06 -07:00
Nandor Soma Abonyi 4ea425e235
NIFI-10656 Log ignored event with info instead of warning
This closes #6540.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-11-22 22:13:56 +01:00
Malthe Borch a71556f115
NIFI-10491 Added Conflict Resolution Strategy to PutAzureBlobStorage_v12
This closes #6443

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-11-15 14:40:23 -06:00
Emilio Setiadarma d06c19180a
NIFI-9697: Implemented AzureKeyVaultSecretsParameterProvider and AzureKeyVaultCredentialsService; Implemented unit tests as well as the documentation for the Parameter Provider
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #6545.
2022-11-14 13:25:47 -05:00
Emilio Setiadarma 08bc44715e
NIFI-10789 Set FlowFile attributes for errors fetching from Azure Data Lake Storage
This closes #6644

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-11-14 07:27:59 -06:00
Lehel Boér feb3148b68
NIFI-10628: Added Default Schedule Time and Run Duration to processors
This closes #6512

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-10-21 14:57:46 -05:00
Joe Witt c9ebdd2025
NIFI-10521-RC4 prepare for next development iteration 2022-10-03 10:59:36 -07:00
Joe Witt 109e54cd58
NIFI-10521-RC4 prepare release nifi-1.18.0-RC4 2022-10-03 10:59:34 -07:00
Peter Turcsanyi 271c773ee4
NIFI-10574: Fixed PutAzureDataLakeStorage fails with HTTP 412 error
This closes #6470

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-10-02 16:44:53 -05:00
exceptionfactory 6461061e8f
NIFI-10381 Refactored Azure Event Hubs components with current SDK
- Upgraded Azure Event Hubs components from legacy version 3 to current version 5
- Refactored Azure dependencies using azure-sdk-bom
- Merged nifi-azure-record-sink-nar with nifi-azure-nar
- Refactored PutAzureCosmosDBRecordTest to use Jackson instead of Gson

This closes #6319.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-09-28 18:22:01 +02:00
Nandor Soma Abonyi 320aed024f
NIFI-10152 Storage client caching in Azure ADLS processors
This closes #6158.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-09-26 08:26:44 +02:00
Timea Barna 22e8901fce NIFI-8248 Modified PutAzureDataLakeStorage processor to use temp file instead of inline replacement
This closes #6159

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2022-08-19 17:10:32 +00:00
Joe Witt 8c66bf948c
NIFI-10272-RC2 prepare for next development iteration 2022-07-27 13:24:09 -07:00
Joe Witt 8d256784d8
NIFI-10272-RC2 prepare release nifi-1.17.0-RC2 2022-07-27 13:24:06 -07:00
Nandor Soma Abonyi f66540eb6d NIFI-9951 Add proxy support to Azure ADLS and Blob v12 processors
This closes #5990.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-05-25 20:33:14 +02:00
Timea Barna 205c58230b NIFI-9879 Add min/max age and min/max size properties to ListAzure processors
This closes #5933.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-05-24 17:22:52 +02:00
exceptionfactory 0c8305950c
NIFI-9987 Standardize Commons IO on version 2.11.0
- Removed version declarations from multiple modules
- Adjusted PutDynamoDB to remove catch for IOException not thrown in Commons IO 2.11.0

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6015.
2022-05-09 12:06:37 +02:00
exceptionfactory 4f423a59ba
NIFI-9952 Upgraded Jackson BOM to 2.13.2.20220328
- Removed unnecessary references to jackson.version property
- Removed unnecessary dependency management references to Jackson libraries

This closes #5992

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-04-23 08:23:39 -04:00
Timea Barna 39a08025c3 NIFI-8677 Added endpoint suffix for Azure EventHub Processors
This closes #5303

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2022-04-06 13:22:29 -07:00
Joe Witt 873d25585c
NIFI-9780 Merge branch 'NIFI-9780-RC3' 2022-04-04 08:35:23 -07:00
Timea Barna 4a90334728 NIFI-9846 Implement pagination listing for Azure List processors
NIFI-9846 removing paging from ListAzureBlobStorage_v12 and ListAzureDataLakeStorage, adding = to filtering

This closes #5916.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-04-04 15:18:49 +02:00
Joe Witt 0419dc2939
NIFI-9780-RC3 prepare for next development iteration 2022-03-21 11:58:27 -07:00
Joe Witt b019a9191f
NIFI-9780-RC3 prepare release nifi-1.16.0-RC3 2022-03-21 11:58:09 -07:00
Peter Turcsanyi 3219c105eb
NIFI-9728: Added support for User Assigned Managed Identity authentication for Azure ADLS and Blob_v12 processors
Also bumped Azure dependencies.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5846.
2022-03-09 16:50:38 +01:00
Mike Thomsen f35f010deb
NIFI-9085 Refactored the Elasticsearch bundle to use JUnit 5
- NIFI-9086 Refactored nifi-accumulo-bundle to use JUnit 5
- NIFI-9087 Updated nifi-ambari-bundle to use JUnit 5
- NIFI-9088 Refactored nifi-amqp-bundle to use JUnit 5
- NIFI-9089 Updated nifi-asn1-bundle to use JUnit 5
- NIFI-9090 Refactored nifi-atlas-bundle to use JUnit 5
- NIFI-9091 Refactored nifi-avro-bundle to use JUnit 5
- NIFI-9092 Refactored nifi-aws-bundle to use JUnit 5
- NIFI-9095 Refactored nifi-azure-bundle to use JUnit 5
- NIFI-9145 Refactored nifi-rethinkdb-bundle to use JUnit 5
- NIFI-9143 Refactored nifi-redis-bundle to use JUnit 5

This closes #5350

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-07 21:49:23 -06:00
Timea Barna a3e1f32cae NIFI-9657 Create MoveADLS processor
This closes #5752.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-02-23 16:09:31 +01:00
exceptionfactory a2f6420f43
NIFI-9638 Refactored Google Guava usage in extensions
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5742
2022-02-09 16:39:53 -05:00
exceptionfactory dc7d9510cd NIFI-9610 Refactored nifi-processor-utils to separate modules
- Added nifi-bin-manager
- Added nifi-event-listen
- Added nifi-event-put
- Added nifi-listed-entity
- Added nifi-put-pattern
- Removed nifi-processor-utils

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5729.
2022-02-01 12:59:16 -05:00
Peter Turcsanyi 01f9d59ee3
NIFI-9338 - Add Azure Blob processors using Azure Blob Storage client library v12 for Java
- changed default credential type to SAS Token
- added endpointSuffix property in integration tests
- added AzureServiceEndpoints to provide endpoint suffix constants and methods to get endpoint urls
- fixed typos, validators
- downgraded azure-identity to 1.3.7 due to a bug in the latest version
- added Access Token credential type (similar to the existing Blob/ADLS processors)
- used JUnit 5 in tests

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5486.
2021-11-09 11:38:36 +01:00
Joe Witt c055895952
NIFI-9344-RC3 prepare for next development iteration 2021-11-03 08:53:37 -07:00
Joe Witt 7fdc07cccd
NIFI-9344-RC3 prepare release nifi-1.15.0-RC3 2021-11-03 08:53:32 -07:00
Joe Gresock d2f8f97b10
NIFI-9276: Adding config verification to AbstractListProcessor subclasses (#5453) 2021-10-21 12:22:50 -04:00
Peter Turcsanyi 1e2783ba63
NIFI-9305: Upgraded Azure dependencies
Also fixed ITPutAzureDataLakeStorage (was broken due to a former dependency upgrade).

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5472.
2021-10-21 10:55:54 +02:00
Timea Barna 462306369f NIFI-8668 ConsumeAzureEventHub NiFi processors need to support storage SAS token authentication
This closes #5136.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-08-09 12:57:56 +02:00
Lehel Boér 9bf3b55034 NIFI-7947: Add directory deletion functionality in DeleteAzureDataLakeStorage
This closes #5190.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-08-05 13:21:44 +02:00
Denes Arvay 47eeabd8a5
NIFI-8762 ADLSCredentialControllerService does not support EL for Storage Account name
- Added FLOWFILE_ATTRIBUTES expression language support to the Storage Account Name and
  and also to the Storage Account Key property to be consistent with
  AzureStorageCredentialsControllerService
- ADLSCredentialControllerService.ACCOUNT_KEY and ADLSCredentialControllerService.SAS_TOKEN
  PropertyDescriptor public constants are the same as AzureStorageUtils.ACCOUNT_KEY and
  AzureStorageUtils.PROP_SAS_TOKEN respectively, but they haven't been removed to keep
  backward compatibility.

NIFI-8762 Removed ADLSCredentialsControllerService.ACCOUNT_KEY and SAS_TOKEN static fields

NIFI-8762 Add test for EL in Account Name and Account Key

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5229.
2021-07-20 15:05:48 +02:00
Joe Witt 97feacc181
NIFI-8767-RC2 prepare for next development iteration 2021-07-10 12:17:09 -07:00
Joe Witt fcbf1d5f97
NIFI-8767-RC2 prepare release nifi-1.14.0-RC2 2021-07-10 12:17:05 -07:00