- Updated Mock Framework to now fail tests that use ProcessSession.commit() unless they first call TestRunner.setAllowSynchronousSessionCommits(true)
- Updated stateless nifi in order to make use of async session commits
- Fixed bug that caused stateless to not properly handle Additional Classpath URLs and bug that caused warnings about validation to get generated when a flow that used controller services was initialized. While this is not really in scope of NIFI-8469, it was found when testing and blocked further progress so addresssed here.
- If Processor fails to progress when run from stateless, trigger from start of flow until that is no longer the case
- Introduced notion of TransactionThresholds that can limit the amount of data that a flow will bring in for a given invocation of stateless dataflow
- Several new system-level tests
- Updated components to make use of new feature
NIFI-8206: Added a ResourceType of TEXT. This requires that the ResourceReferenceFactory know which types are allowed in order to create the ResourceReference. PropertyValue needs to then have the PropertyDescriptor available to it. This resulted in highlighting many bugs in unit tests where components were not exposing property descriptors via getSupportedPropertyDescriptors() or were evaluating Expression Language using the wrong scope, so fixed many unit tests/components to properly declare Expression Language scope when using it
NIFI-8206: Removed problematic unit test that required directory names with special characters that are not allowed on some operating systems
This closes#4890.
Signed-off-by: Bryan Bende <bbende@apache.org>
- Removed Expression Language support indicators from sensitive properties
This closes#4843
Signed-off-by: David Handermann <exceptionfactory@apache.org>
NIFI-7964 Remove unused import
NIFI-7964 Reverted to using BufferedInputStream along with unmarkable
NIFI-7964 Made UnmarkableInputStream static and added factory method
NIFI-7964 Remove unnecessary factory method from UnmarkableInputStream
This closes#4632.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
Also added validator for Directory Name property in AbstractAzureDataLakeStorageProcessor
Fix Tracking Entities strategy: use milliseconds for lastModified
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#4438.
NIFI-7581 Separated Controller Service for providing Azure credentials for ADLS (ADLSCredentialsControllerService) form the one that does the same for Blob storages (AzureStorageCredentialsDetails). (This was done due to the considerable difference in the APIs of the libraries used to connect to both.)
NIFI-7581 Fix: Register controller service in META-INF. Minor fixes.
NIFI-7581 Minor changes (documentation, type etc.)
NIFI-7581 Updated integration tests.
NIFI-7581 Minor changes (renaming).
This closes#4369.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
NIFI-7445: Add Conflict Resolution property to PutAzureDataLakeStorage processor
Made warning and error messages more informative.
Refactored flowFile assertion in the tests.
This closes#4287.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
DeleteAzureDataLakeStorage now throws exception if fileSystem or fileName is empty string
NIFI-7336: Add tests for DeleteAzureDataLakeStorage - typos fixed
NIFI-7336: Add tests for DeleteAzureDataLakeStorage - fixed a test case
This closes#4272.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
NIFI-7367: Negative test cases for expression language in FetchAzureDataLakeStorage
FetchAzureDataLakeStorage throws exception when filesystem or filename is blank.
Fixed logged error messages in all 3 of the Delete, Fetch and Put ADLS processors.
testFetchDirectory test case marked as ignored.
This closes#4257.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
Updated to remove unused variables
NIFI-7259 import and property description changes
This closes#4189.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
The controller service api jars have their own bundles, they do not need to be
packaged into the processor nars.
Signed-off-by: Joe Witt <joewitt@apache.org>
Added AzureStorageCredentialsService controller service interface
with 2 implementations:
- AzureStorageCredentialsControllerService: holds the credential properties
(account name, account key, sas token)
- AzureStorageCredentialsControllerServiceLookup: can be used to dynamically
look up another AzureStorageCredentialsService (similar to
DBCPConnectionPoolLookup)
The controller service can be used by the Azure Storage processors:
{List|Fetch|Put|Delete}AzureBlobStorage, {Get|Put}AzureQueueStorage
NIFI-6550: Review changes.
NIFI-6550: Review changes #2.
This closes#3742.
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
Mark Payne
Guillaume Schaer
Paul Kelly
sjyang18
Denes Arvay
Peter Turcsanyi
Timea Barna
Joe Witt
Joey Frazee
Sushil Kumar
Fabio Torchetti
Tamas Palfy
Peter Gyori
muazmaz
Pierre Villard
Martin Šimek
Shayne Burgess
Tamas Palfy
Peter Turcsanyi
mysunnytime