- Removed hamcrest-all from default dependencies
- Added groovy-test to selected modules with Groovy test classes
- Added junit-vintage-engine to selected modules with JUnit 4 test classes
- Corrected references to JUnit 4 assertions in JUnit 5 test classes
- Removed several unnecessary test classes from nifi-socket-utils
- Removed duplicative Registry toolkit test classes
- Removed Kudu integration tests
NIFI-11532 Corrected scope for junit-vintage-engine for Elasticsearch
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes#7233
Rather than creating many FlowFiles with the same Content Claim, refactored content repos' OutputStreams and ClaimWriteCache so that a new ContentClaim is created for each FlowFile. This ensures that we have a content claim offset of 0. The poor performance was due to having to use StreamUtils.skip() in conjunction with the CipherInputStream, which would only skip a max of 511 bytes at a time. By using a separate Content Claim per FlowFile, we no longer need to seek after creating the CipherInputStream
This closes#7363
Signed-off-by: David Handermann <exceptionfactory@apache.org>
After terminating Processor in DynamicallyModifiedClasspathIT system test, keep trying to empty queue until successful instead of assuming success after the first iteration.
When writing to a replacement file in AsyncCommitCallbackIT, write to a temp file and then rename. This prevents the processor from picking up the file while it's still being written.
This closes#7302
Signed-off-by: David Handermann <exceptionfactory@apache.org>
When waiting for a Processor to stop completely in system tests, also consider any terminated threads. Consider the processor stopped only when all active threads and all terminated threads have gone to 0. Additionally, when stopping all components in a Process Group, wait for the PG to reflect that there are no running components. Previously, we checked for processor active thread counts. However, this was problematic because the flow that was fetched was not fetched recursively and as a result, processors running in embedded groups were not always stopped when the waitForProcessorsStopped(String groupId) method returned. Finally, removed the step for gathering surefire-reports in failed system tests because it was not working as expected and was unneeded because the tests' logs were already gathered into the diagnostic dump that is uploaded.
Added fix for BatchFlowBetweenGroupsIT. The test would intermittently fail, based on timing. The test assumed that when Input Port in Group B was scheduled to run, it would trigger before FlowFiles in Group A finished processing. The Sleep processor caused a 10 ms delay for each of 5 FlowFiles which gives the Input Port about 50 ms to trigger. If it took longer to trigger the first time, the test failed. Adjusted the Sleep processor to use 2 seconds. So now instead of 50 milliseconds to start, it has 10 seconds. Overall this adds about 30-45 seconds to the run time of the test but ensures more-than-adequate time for the test to run, even on extremely slow hardware
This closes#7289
Signed-off-by: David Handermann <exceptionfactory@apache.org>
NIFI-11557: Added an additional system test and updated github actions to include surefire-report in order to help diagnose problem that occurred in one of the last system-test runs in Github. Could not replicate problem locally
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#7265
- Updated GitHub workflow so that system tests include Python 3.9
- Updated GitHub actions to build necessary modules for system tests
This closes#7003
Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Fixed system tests so that they work properly in Clustered version of RegistryClientIT
- Fixed system test - ensure that we wait for processors to become valid before attempting to start them; also added an additional system test around Controller Services in versioned flows
This closes#7095
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Ensure that components are notified that primary node has changed in a background thread instead of the Leader Election thread and activate/deactivate the thread in the case of Processors so that they can be viewed in the UI and terminated
- Fixed system tests that would fail intermittently because they did not wait for node disconnection to complete and did not properly switch the client to look at the connected node before checking cluster status
This closes#7052
Signed-off-by: David Handermann <exceptionfactory@apache.org>
NIFI-11310: Fixed META-INF/services file that was mistakenly listing an extra extension point, due to rebase
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#7061
- Updated algorithm from PBEWITHMD5AND256BITAES-CBC-OPENSSL to NIFI_PBKDF2_AES_GCM_256
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#7041.
- Replaced OSUtils.getProcessId() with java.lang.Process.pid() available on Java 9 and following
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#6989.
NIFI-11192: If a failure is encountered when changing the version of a flow from 1 version to another, attempt to rollback the changes instead of just failing with the flow in a bad state
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#6981
- Upgraded Logback from 1.2.11 to 1.3.5
- Updated Logback DelayingShutdownHook to DefaultShutdownHook
- Disabled Spring Boot Logging System in favor of standard Logback initialization
- Excluded logback-classic from ZooKeeper and other dependencies to avoid conflicts when running tests
- Excluded spring-boot-starter-logging to avoid failures related to Logback 1.2 and Spring Boot 2.7
- Removed ZooKeeperMigratorTest.groovy based on Apache Curator test server usage of Logback 1.2
NIFI-10580 Added logback-core as explicit dependency
- Set logback-core as provided in root configuration
- Added logback-core as compile dependency in assembly configurations
Signed-off-by: Joe Witt <joewitt@apache.org>
- NIFI-10981 Ensure NarAutoLoader starts after provider retrieves NARs, and ensure the auto-loader attempts to process any NARs available before starting to watch for new files
- Add system test with NAR Provider that copies NARs from a local directory and verifies the NARs were loaded and linked correctly
- Exclude new assembly from CI test-compile
* NIFI-9958 Implemented Sensitive Dynamic Properties
- Added SupportsSensitiveDynamicProperties annotation for components
- Added optional sensitive query parameter to Property Descriptor REST Resource methods
- Added system tests for components supporting sensitive dynamic properties
- Updated REST Resources to support Sensitive Dynamic Property Names
- Updated Documentation Writer to indicate component support for Sensitive Dynamic Properties
- Updated InvokeHTTP to support Sensitive Dynamic Properties
- Updated Auditor components to handle masking Sensitive Dynamic Properties
* Refactored Property Descriptor REST method handling
- Corrected AbstractDocumentationWriter evaluation of support for sensitive dynamic properties
- Refactored Controller Service Dynamic Properties system tests to new class
* Updated AbstractComponentNode.getProperties() to get canonical descriptor
* NIFI-10001: Fixed issue in which some components may fail to update the scheduled state when comparing flows
* NIFI-10001: Fixed bugs that caused some components to not have their scheduled state updated. When comparing two flows, now allow specifying how to determine a VersionedComponent's ID for comparison. When comparing local flow against flow from registry, use Versioned Component ID. But when comparing two instantiated flows, such as local flow vs. cluster flow, use the VersionedComponent's Instance ID instead. This ensures that we can properly compare two components even if there are several instances of a given flow
- Downgraded Jakarta Activation from 2.0.1 to 1.2.2 in nifi-bootstrap
- Updated assembly configurations to include Jakarta Activation in java11 directory
exceptionfactory
Mark Payne
Lehel Boér
Bryan Bende
Joe Witt
dan-s1
Joe Gresock
Nissim Shiman