Ensured that if a Process Group doesn't have a Versioned Component ID we use the ComponentIdLookup to create one based on its Instance ID in the same way that is done when serializing the flow; this ensures matching ID's when we synchronize flows across the cluster. Also included some code cleanup around failure handling on startup
This closes#8432
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Adjusted test flow to find available version of commons-lang3 instead of depending on a specific version from the default property value
This closes#8307
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Added nifi-standard-shared-nar with common dependencies
- Added nifi-standard-shared-bom with provided scope for parent module references
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#8008.
- Added nar-packaging and war-packaging profiles to root Maven configuration
- Removed individual properties from NAR module configurations
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#7947.
(cherry picked from commit 2366132e92951a8fc27097c777d3104a1c396f7b)
- Added EntityStoreAuditService implementation using JetBrains Xodus libraries
- Added FlowConfigurationHistoryMigrator for checking and migrating H2 databases containing Flow Configuration History to Xodus persistent store implementation
- Added nifi-security-cert for reusable components without dependencies
- Added nifi-security-cert-builder for certificate generation
NIFI-12001: Changes related to backporting
Signed-off-by: Matt Burgess <mattyb149@apache.org>
- Timing issues in system integration tests could sometimes fail with a NullPointerException when calling the poison method multiple times
This closes#7618
Signed-off-by: David Handermann <exceptionfactory@apache.org>
(cherry picked from commit 25d775b9651dc5a69120e0bed7545daa18281f3d)
- Ensure that testComponentsRecreatedOnRestart only counts components in its own group
This closes#7466
Signed-off-by: David Handermann <exceptionfactory@apache.org>
(cherry picked from commit 96be7377b3f678271f13a9085d63e223670ca522)
Also changed some properties around so that we can have both a clustered and a standalone instance running at the same time.
This closes#7427
Signed-off-by: David Handermann <exceptionfactory@apache.org>
(cherry picked from commit 5d3443c1915ae7231ade4162437dc1debf9c42e7)
- FlowSynchronizationIT no longer requires isDestroyEnvironmentAfterEachTest to return true
This closes#7420
Signed-off-by: David Handermann <exceptionfactory@apache.org>
(cherry picked from commit 9709bd6fb749e1a1c7bc52076e69d96f5235802a)
- Removed hamcrest-all from default dependencies
- Added groovy-test to selected modules with Groovy test classes
- Added junit-vintage-engine to selected modules with JUnit 4 test classes
- Corrected references to JUnit 4 assertions in JUnit 5 test classes
- Removed several unnecessary test classes from nifi-socket-utils
- Removed duplicative Registry toolkit test classes
- Removed Kudu integration tests
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes#7233
(cherry picked from commit ca6c7f704164e922e11d72e7c972a50ac46e6b0e)
Rather than creating many FlowFiles with the same Content Claim, refactored content repos' OutputStreams and ClaimWriteCache so that a new ContentClaim is created for each FlowFile. This ensures that we have a content claim offset of 0. The poor performance was due to having to use StreamUtils.skip() in conjunction with the CipherInputStream, which would only skip a max of 511 bytes at a time. By using a separate Content Claim per FlowFile, we no longer need to seek after creating the CipherInputStream
This closes#7363
Signed-off-by: David Handermann <exceptionfactory@apache.org>
(cherry picked from commit 702c6350344e4ae4cd31349747e96c1384017ed4)
After terminating Processor in DynamicallyModifiedClasspathIT system test, keep trying to empty queue until successful instead of assuming success after the first iteration.
When writing to a replacement file in AsyncCommitCallbackIT, write to a temp file and then rename. This prevents the processor from picking up the file while it's still being written.
This closes#7302
Signed-off-by: David Handermann <exceptionfactory@apache.org>
(cherry picked from commit 9f61b5f957e0389d4929fd7f4e9fd83521bcd4c5)
When waiting for a Processor to stop completely in system tests, also consider any terminated threads. Consider the processor stopped only when all active threads and all terminated threads have gone to 0. Additionally, when stopping all components in a Process Group, wait for the PG to reflect that there are no running components. Previously, we checked for processor active thread counts. However, this was problematic because the flow that was fetched was not fetched recursively and as a result, processors running in embedded groups were not always stopped when the waitForProcessorsStopped(String groupId) method returned. Finally, removed the step for gathering surefire-reports in failed system tests because it was not working as expected and was unneeded because the tests' logs were already gathered into the diagnostic dump that is uploaded.
Added fix for BatchFlowBetweenGroupsIT. The test would intermittently fail, based on timing. The test assumed that when Input Port in Group B was scheduled to run, it would trigger before FlowFiles in Group A finished processing. The Sleep processor caused a 10 ms delay for each of 5 FlowFiles which gives the Input Port about 50 ms to trigger. If it took longer to trigger the first time, the test failed. Adjusted the Sleep processor to use 2 seconds. So now instead of 50 milliseconds to start, it has 10 seconds. Overall this adds about 30-45 seconds to the run time of the test but ensures more-than-adequate time for the test to run, even on extremely slow hardware
This closes#7289
Signed-off-by: David Handermann <exceptionfactory@apache.org>
(cherry picked from commit 108f841525f98aab8144c0421a1ea5e5cb2a3b3b)
NIFI-11557: Added an additional system test and updated github actions to include surefire-report in order to help diagnose problem that occurred in one of the last system-test runs in Github. Could not replicate problem locally
- Fixed system tests so that they work properly in Clustered version of RegistryClientIT
- Fixed system test - ensure that we wait for processors to become valid before attempting to start them; also added an additional system test around Controller Services in versioned flows
This closes#7095
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Ensure that components are notified that primary node has changed in a background thread instead of the Leader Election thread and activate/deactivate the thread in the case of Processors so that they can be viewed in the UI and terminated
- Fixed system tests that would fail intermittently because they did not wait for node disconnection to complete and did not properly switch the client to look at the connected node before checking cluster status
This closes#7052
Signed-off-by: David Handermann <exceptionfactory@apache.org>
NIFI-11310: Fixed META-INF/services file that was mistakenly listing an extra extension point, due to rebase
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
- Updated algorithm from PBEWITHMD5AND256BITAES-CBC-OPENSSL to NIFI_PBKDF2_AES_GCM_256
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#7041.
NIFI-11192: If a failure is encountered when changing the version of a flow from 1 version to another, attempt to rollback the changes instead of just failing with the flow in a bad state
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#6981
- Upgraded Logback from 1.2.11 to 1.3.5
- Updated Logback DelayingShutdownHook to DefaultShutdownHook
- Disabled Spring Boot Logging System in favor of standard Logback initialization
- Excluded logback-classic from ZooKeeper and other dependencies to avoid conflicts when running tests
- Excluded spring-boot-starter-logging to avoid failures related to Logback 1.2 and Spring Boot 2.7
- Removed ZooKeeperMigratorTest.groovy based on Apache Curator test server usage of Logback 1.2
NIFI-10580 Added logback-core as explicit dependency
- Set logback-core as provided in root configuration
- Added logback-core as compile dependency in assembly configurations
Signed-off-by: Joe Witt <joewitt@apache.org>
- NIFI-10981 Ensure NarAutoLoader starts after provider retrieves NARs, and ensure the auto-loader attempts to process any NARs available before starting to watch for new files
- Add system test with NAR Provider that copies NARs from a local directory and verifies the NARs were loaded and linked correctly
- Exclude new assembly from CI test-compile
