Also added validator for Directory Name property in AbstractAzureDataLakeStorageProcessor
Fix Tracking Entities strategy: use milliseconds for lastModified
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#4438.
* NIFI-7540: Fix TestListenSMTP and TestListFile on macOS build environment
This also fixes NIFI-4760.
* NIFI-7540: Remove duplicate mail.smtp.starttls.enable from TestListenSMTP
Signed-off-by: Andy LoPresto <alopresto@apache.org>
Still recognizes 'atlas.cluster.name' as well, but takes lower precedence than the new property.
Also Atlas URL can be provided via the 'atlas.rest.address' property in the atlas-application.properties.
NIFI-7280 In ReportLineageToAtlas improved documentation and adjusted property ordering for better user experience. Minor refactor.
NIFI-7280 In ReportLineageToAtlas amended documentation. Minor refactor.
NIFI-7280 In ReportLineageToAtlas amended more documentation. More minor refactor.
NIFI-7280 - In Atlas reporting: complete clusterName -> namespace overhaul where appropriate.
This closes#4213.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
- Create a mock Validator to track count of calls to validate().
We cannot use Mockito for this, because it can't mock all the
StandardValidators
refactor based on review comments
fix naming in comments
moved to main based on review
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#4043
NIFI-6707: Added System Test to verify behavior when services depend on one another
NIFI-6707: Ensure that when tearing down flow after test, we wait for all processors and controller services to complete stop/disable before attempting to delete them
NIFI-6707: Fixed bug in RemoteProcessGroupIT that caused the test to fail intermittently based on timing. Improved logging for Process Group, Standard Process Group
NIFI-6707: Updated to include java11 subdirectory under lib/
NIFI-6707: Rebased against master and changed dependencies from 1.10.0-SNAPSHOT to 1.11.0-SNAPSHOT
NIFI-6707: Updates to see if it will help travis build correctly
NIFI-6707: Commenting out java commands in bootstrap.conf files
This closes#3831.
Signed-off-by: Bryan Bende <bbende@apache.org>
ISO8061_INSTANT_VALIDATOR was misnamed - the correct standard is ISO 8601.
Also updated the error messages to have spaces in them, which is how the ISO site
displays them.
This closes#3465
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
These tests were written using Diffblue Cover.
Fixed bug in FormatUtils.formatNanos
Fix import asterisk warning
Add Apache license header.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#3354
- Updates to make UpdateRecord and RecordPath automatically update Record schema when performing update and perform the updates on the first record in UpdateRecord before obtaining Writer Schema. This allows the Writer to to inherit the Schema of the updated Record instead of the Schema of the Record as it was when it was read.
- Updated JoltTransformRecord so that schema is inferred on the first transformed object before passing the schema to the Record Writer, so that if writer inherits schema from record, the schema that is inherited is the trans transformed schema
- Updated LookupRecord to allow for Record fields to be arbitrarily added
- Implemented ContentClaimInputStream
- Added controller service for caching schemas
- UpdatedQueryRecord to cache schemas automatically up to some number of schemas, which will significantly inprove throughput in many cases, especially with inferred schemas.
NIFI-5938: Updated AvroTypeUtil so that if creating an Avro Schema using a field name that is not valid for Avro, it creates a Schema that uses a different, valid field name and adds an alias for the given field name so that the fields still are looked up appropriately. Fixed a bug in finding the appropriate Avro field when aliases are used. Updated ContentClaimInputStream so that if mark() is called followed by multiple calls to reset(), that each reset() call is successful instead of failing after the first one (the JavaDoc for InputStream appears to indicate that the InputStream is free to do either and in fact the InputStream is even free to allow reset() to reset to the beginning of file if mark() is not even called, if it chooses to do so instead of requiring a call to mark()).
NIFI-5938: Added another unit test for AvroTypeUtil
NIFI-5938: If using inferred schema in CSV Reader, do not consider first record as a header line. Also addressed a bug in StandardConfigurationContext that was exposed by CSVReader, in which calling getProperty(PropertyDescriptor) did not properly lookup the canonical representation of the Property Descriptor from the component before attempting to get a default value
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#3253
Added helper methods.
Added unit tests.
NIFI-5854 [WIP] Cleaned up logic.
Resolved failing unit tests due to error message change.
NIFI-5854 [WIP] All helper method unit tests pass.
NIFI-5854 [WIP] FormatUtils#getPreciseTimeDuration() now handles all tested inputs correctly.
Added unit tests.
NIFI-5854 [WIP] FormatUtils#getTimeDuration() still using long.
Added unit tests.
Renamed existing unit tests to reflect method under test.
NIFI-5854 FormatUtils#getTimeDuration() returns long but now accepts decimal inputs.
Added @Deprecation warnings (will update callers where possible).
All unit tests pass.
NIFI-5854 Fixed unit tests (ran in IDE but not Maven) due to int overflows.
Fixed checkstyle issues.
NIFI-5854 Fixed typo in Javadoc.
NIFI-5854 Fixed typo in Javadoc.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#3193
Refactoring StandardFlowFileQueue to have an AbstractFlowFileQueue
Refactored more into AbstractFlowFileQueue
Added documentation, cleaned up code some
Refactored FlowFileQueue so that there is SwappablePriorityQueue
Several unit tests written
Added REST API Endpoint to allow PUT to update connection to use load balancing or not. When enabling load balancing, though, I saw the queue size go from 9 to 18. Then was only able to process 9 FlowFiles.
Bug fixes
Code refactoring
Added integration tests, bug fixes
Refactored clients to use NIO
Bug fixes. Appears to finally be working with NIO Client!!!!!
NIFI-5516: Refactored some code from NioAsyncLoadBalanceClient to LoadBalanceSession
Bug fixes and allowed load balancing socket connections to be reused
Implemented ability to compress Nothing, Attributes, or Content + Attributes when performing load-balancing
Added flag to ConnectionDTO to indicate Load Balance Status
Updated Diagnostics DTO for connections
Store state about cluster topology in NodeClusterCoordinator so that the state is known upon restart
Code cleanup
Fixed checkstyle and unit tests
NIFI-5516: Updating logic for Cluster Node Firewall so that the node's identity comes from its certificate, not from whatever it says it is.
NIFI-5516: FIxed missing License headers
NIFI-5516: Some minor code cleanup
NIFI-5516: Adddressed review feedback; Bug fixes; some code cleanup. Changed dependency on nifi-registry from SNAPSHOT to official 0.3.0 release
NIFI-5516: Take backpressure configuration into account
NIFI-5516: Fixed ConnectionDiagnosticsSnapshot to include node identifier
NIFI-5516: Addressed review feedback
This closes#2947
Updated Javadoc for SiteToSiteClient#createTransaction() and HttpClient implementation.
Reverted exception listing in method contract for SiteToSiteClient#createTransaction and HttpClient tion of same.
Reverted import ordering in TestSiteToSiteClient.
Reverted exception listing in TestGetHDFSFileInfo, TestListHDFS, and StandardHttpFlowFileServerProtocol.
Restored @SuppressWarnings annotation and removed unnecessary "public static" keywords from inner classes in SiteToSiteClient.
This closes#2841.
Signed-off-by: Joe Witt <joewitt@apache.org>
- Added configurable character set encoding for JMS TextMessages
- Improved PublishJMS/ConsumeJMS documentation
- Validate character set in property validator instead of OnScheduled
Added failing unit test for #available() at various states (initial, during read, after read).
Implemented #available() delegation.
All tests pass.
This closes#2461.
Signed-off-by: Kevin Doran <kdoran@apache.org>
- Removing the usage of SimpleDateFormat when formatting hours/minutes/seconds as the current timezone could cause unintended results.
This closes#1871.
use the FileNameFilter when not passing down explit jar paths
Filter out ^. files when reading lists of files from directories
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
- Support counters at Wait/Notify processors so that NiFi flow can be
configured to wait for N signals
- Extract Wait/Notify logics into WaitNotifyProtocol
- Added FragmentAttributes to manage commonly used fragment attributes
- Changed existing split processors to set 'fragment.identifier' and
'fragment.count', so that Wait can use those to wait for all splits
get processed
This closes#1420.
Signed-off-by: Bryan Bende <bbende@apache.org>
NIFI-1582 removing the option to use cluster state
NIFI-1582 addressing Oleg's comments
NIFI-1582 No longer forcing numbers as the init value and adding getStateValue() to EL instead of using 'ATTRIBUTE_state'
NIFI-1582 Removing init state value
NIFI-1582 Adding documentation for the changes to Init State value
This closes#319
NIFI-2854: Incorporated PR review feedback
NIFI-2854: Implemented feedback from PR Review
NIFI-2854: Ensure that all resources are closed on CompressableRecordReader.close() even if an IOException is thrown when closing one of them
This closes#1202
- introduced org.apache.nifi.stream.io.util.TextLineDemarcator
- refactored SplitText to use org.apache.nifi.stream.io.util.TextLineDemarcator
- updated SplitText's capability discription to provide more clarity around splits with headers.
Otto Fowler
Peter Turcsanyi
Joey
Tamas Palfy
Mark Payne
Jeff Storck
Alan Jackoway
Freddy Tuxworth
Andy LoPresto
Matthew Burgess
Koji Kawamura
Mike Thomsen
zenfenan
Bryan Bende
Mike Moser
Pierre Villard
Matt Gilman
Oleg Zhurakousky
Otto Fowler
jpercivall
joewitt