NIFI-1833 Moved AbstractListProcessor.java, EntityListing.java, and ListableEntity.java from nifi-standard-processors into nifi-processor-utils
Moved TestAbstractListProcessor.java into nifi-processor-utils
Set nifi-azure-nar's nar dependency back to nifi-standard-services-api-nar
Fixed failing integration tests (ITFetchAzureBlobStorage.java, ITListAzureBlobStorage.java, and ITPutAzureStorageBlob.java) and refactored them to be able to run in parallel
NIFI-1833 Moved security notice info in the additional details documentation into the descriptions of the specific attributes for which those notices are intended
Added displayName usage to properties
Updated exception handling in FetchAzureBlobStorage.java and PutAzureBlobStorage.java to cause flowfiles with Output/InputStreamCallback failures to be routed to the processor's failure relationship
Cleaned up dependencies in pom
NIFI-1833 Removed unnecessary calls to map on Optional in the onTrigger exception handling of FetchAzureBlobStorage.java and PutAzureBlobStorage.java
NIFI-1833 Updates due to nifi-processor-utils being moved under nifi-nar-bundles
This closes#1719.
Signed-off-by: Bryan Rosander <brosander@apache.org>
Addressed dependency issues from the review.
Addressed a checkstyle issue.
Review: reworded the descriptions.
Review: implemented the reset condition logic.
Review: dropped static qualifier from method signatures, not required really
Review: removed sys.out, inlined a single method to get access to the ProcessContext.getName()
Switched to HTTPS as per MSFT recommendation. Some DRY. Dropped cruft.
Addressing review suggestions from 4/5
Review: documentation improvements
Review: documentation improvements
This closes#1636.
Signed-off-by: Bryan Rosander <brosander@apache.org>
Added src/test/resources/logback-test.xml files resetting log level from DEBUG (in nifi-data-provenance-utils) to WARN because later tests depend on MockComponentLog recording a certain number of messages and this number is different than expected if the log level is DEBUG.
This closes#1686.
Signed-off-by: Bryan Bende, Yolanda M. Davis, and Mark Payne
- Added description on what session maintenance does.
- Added calling deregister when initial connection attempt fails so that a processor can retry connecting at next onTrigger.
This closes#1597
Signed-off-by: Jeremy Dyer <jeremydyer@apache.org>
- Removed unused disconnect method from WebSocketService interface.
- Added session maintenance background thread at JettyWebSocketClient
which reconnects sessions those are still referred by ConnectWebSocket
processor but no longer active.
- Added Session Maintenance Interval property to JettyWebSocketClient.
- Allowed specifying existing session id so that it can be recovered
transparently.
- Moved test classes to appropriate package.
- Added test cases that verify the same session id can be used after
WebSocket server restarts.
- Addressing issue when enabling/disabling controller services when referencing services are not in scope.
- Fixing enabled/disabled icons for referencing services.
- Rendering when a reference cycle is detected.
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
This closes#1725
Added screenshot of encrypted provenance repository contents on disk.
Added note about clearing existing provenance repository when switching to encrypted implementation (see PR 1686 @ https://github.com/apache/nifi/pull/1686#issuecomment-298432578).
This closes#1713.
Signed-off-by: Andy LoPresto <alopresto@apache.org>
- Creating nifi-records-utils to share utility code from record services
- Refactoring Parquet tests to use MockRecorderParser and MockRecordWriter
- Refactoring AbstractPutHDFSRecord to use schema access strategy
- Adding custom validate to AbstractPutHDFSRecord and adding handling of UNION types when writing Records as Avro
- Refactoring project structure to get CS API references out of nifi-commons, introducing nifi-extension-utils under nifi-nar-bundles
- Updating abstract put/fetch processors to obtain the WriteResult and update flow file attributes
This closes#1712.
Signed-off-by: Andy LoPresto <alopresto@apache.org>
- Added org.apache.nifi.processor.util.pattern package in nifi-processor-utils containing reusable functions to mix-in 'Rollback on Failure' capability.
- Created a process pattern classes, Put and PutGroup. It will be helpful to standardize Processor implementations.
- Applied Rollback on Failure to PutSQL, PutHiveQL, PutHiveStreaming and PutDatabaseRecord.
- Stop using AbstractProcessor for these processors, as it penalizes FlowFiles being processed when it rollback a process session. If FlowFiles are penalized, it will not be fetched again until penalization expires.
- Yield processor when a failure occurs and RollbackOnFailure is enabled. If we do not penalize nor yield, a failed FlowFile retries too frequently.
- When Rollback on Failure is enabled but processor is not transactional, discontinue when an error occurred after successful processes.
- Fixed existing issues on PutHiveStreaming:
- Output FlowFile Avro format was corrupted by concatenating multiple Avro files.
- Output FlowFile records had incorrect values because of reusing GenericRecord instance.
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes#1658
- added proxy dn flag to support providing authorized username for secured environments
- addressed pr comments including fix to ensure proxy info added when getting cluster info, showing cleaner error messaging and improving help text. Also fixed potential issue with versioning comparison (mismatched lengths)
- Printing response body when requests fails.
- This closes#1697
- Added batchCount, batchSize, batchDuration to limit flow files to be
included in a single Site-to-Site transaction.
- Added batch throttling logic when StandardRemoteGroupPort transfers
flow files to a remote input port using the batch limit configurations,
so that users can limit batch not only for pulling data, but also pushing data.
- Added destination list shuffle to provide better load distribution.
Previously, the load distribution algorithm produced the same host consecutively.
- Added new batch settings to FlowConfiguration.xsd.
- Added new batch settings to Flow Fingerprint.
- Added new batch settings to Audit.
- Sort ports by name at 'Remote Process Group Ports' dialog.
- Show 'No value set' when a batch configuration is not set
- Updated batch settings tooltip to clearly explain how it works the configuration works differently for input and output ports.
- Updated DTO by separating batch settings to BatchSettingsDTO to indicate count, size and duration are a set of configurations.
- This closes#1306
NIFI-3704: Incorporated review comments
NIFI-3704: Updated PutDatabaseRecord documentation, including limitations on Update records
NIFI-3704: Added retry paths for transient SQL errors
NIFI-3704: Updated to use renamed RecordReaderFactory and catch SchemaNotFoundException
NIFI-3704: Added better error handling and rollback
This closes#1677.
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
Mark Payne
Sarah Olson
Andrew Lim
Jeff Storck
Andrew Grande
Simon Elliston Ball
Andy LoPresto
Yolanda M. Davis
Andre F de Miranda
Koji Kawamura
Matt Gilman
Bryan Bende
Jon Anderson III
Joe Gresock
Pierre Villard
Bryan Rosander
Matt Burgess
joewitt
Joe Percivall