Commit Graph

365 Commits

Author SHA1 Message Date
Mark Payne b3c620411a
NIFI-11556: Added ability to use a Process Group as a Stateless Flow
When shutting down FlowController, wait up to gracefulShutdownSeconds seconds for the components to stop before shutting down thread pools. This allows for asynchronous operations such as disableControllerServicesAsync to complete during shutdown. Updated StandardStatelessFlow so that on shutdown it catches more general Exception to ensure that shutdown succeeds

Ensure that Max Concurrent Tasks cannot be set less than 1 for stateless group; fixed typo in ProcessGroupDTO's docs; on shutdown, we may need to disable controller services asynchronously. At that point, the thread pool used to do so may already be shutdown. If so, catch this and create a new single-thread pool, disable the service, and immediately shutdown the pool. Also, if we fail to disable services on shutdown of a stateless flow, instead of throwing an Exception, just log it and move on - it doesn't make much sense for shutdown() to throw an Exception in that case.

Updated system tests so that when emptying a queue, we check the result and if the queue still has data (because a Processor hasn't acknowledged the data, for example) then continue issuing request until the queue fully becomes empty.

When shutting down input/output ports for stateless group, ensure that we wait for the ports' active threads to complete before returning

When stopping StatelessGroupNode, ensure that all the Processors, Controller Services (even those not executed by the Stateless Engine) are stopped/disabled before considering the Stateless Group to be fully STOPPED.

This closes #7253

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-08-09 12:08:22 -05:00
mr1716 8979c20f59
NIFI-11869 Upgraded opencsv from 5.7.1 to 5.8
This closes #7536

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-07-31 19:48:06 -05:00
Pierre Villard dbb2d6905f
NIFI-11842 - remove getTargetUri() and code cleanup
This closes #7511

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-07-24 13:53:17 -05:00
Pierre Villard e0d6b49cd5
NIFI-11850 - CLI - add keep existing parameter context flag
This closes #7520

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-07-24 13:15:11 -05:00
Pierre Villard d9b04770ef
NIFI-11844 - CLI - command to stop version control of a process group
This closes #7513

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-07-24 11:47:57 -05:00
Nandor Soma Abonyi 84b4d100a8
NIFI-11848 Allocate unused port in TlsCertificateAuthorityTest
This closes #7514

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-07-24 10:54:45 -05:00
exceptionfactory 1f1c5df5a3
NIFI-11673 Removed Legacy TLS Configuration Versions
This closes #7367

Signed-off-by: Nandor Soma Abonyi <nsabonyi@apache.org>
2023-06-26 11:10:39 +02:00
exceptionfactory 305277b833 NIFI-11709 Upgraded Guava from 32.0.0 to 32.0.1
- Added version override for Guava 27 to 32.0.1 in Hadoop and Hive modules

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7415
2023-06-20 16:22:00 -04:00
mr1716 a5ae155839
NIFI-11725 Upgraded Woodstox from 6.5.0 to 6.5.1
This closes #7407

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-19 16:05:01 -05:00
exceptionfactory 0ed1efc50a
NIFI-11712 Removed java.net.preferIPv4Stack from bootstrap.conf
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7392.
2023-06-19 10:22:31 +02:00
exceptionfactory ca6c7f7041 NIFI-11532 Removed JUnit 4 and Groovy Test from default dependencies
- Removed hamcrest-all from default dependencies
- Added groovy-test to selected modules with Groovy test classes
- Added junit-vintage-engine to selected modules with JUnit 4 test classes
- Corrected references to JUnit 4 assertions in JUnit 5 test classes
- Removed several unnecessary test classes from nifi-socket-utils
- Removed duplicative Registry toolkit test classes
- Removed Kudu integration tests

NIFI-11532 Corrected scope for junit-vintage-engine for Elasticsearch

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7233
2023-06-13 15:36:23 -04:00
exceptionfactory c5c711c91d
NIFI-11645 This closes #7342. Upgraded Guava from 31.1 to 32.0.0
- Added selective overrides to upgrade transitive dependencies

Signed-off-by: Joe Witt <joewitt@apache.org>
2023-06-05 14:33:10 -07:00
Mark Payne 50811660d0
NIFI-11603 Refactored Socket-based tests to use Dynamic Ports
- Removed NetworkUtils methods for getting available ports
- Updated Socket-based components to support using 0 to listen on a random available port for improved test reliability

This closes #7299

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-01 13:33:06 -05:00
Mark Payne a12c9ca9c7
NIFI-11557: Avoid using the expensive and unnecessary Files.walkFileTree on startup and initialization of Content Repository. Also performed some code cleanup: IntelliJ flagged many warnings in the class, mostly around methods that are no longer used and potential NullPointerExceptions, so those were cleaned up. Additionally, removed the nifi property for max flowfiles per claim - this property was never implemented. It was referenced, but the way in which is was used curiously had nothing to do with what the property was intended to be used for or for how it was documented. Instead, it was used to limit the max number of claims that could remain writable. As a result, it was removed.
NIFI-11557: Added an additional system test and updated github actions to include surefire-report in order to help diagnose problem that occurred in one of the last system-test runs in Github. Could not replicate problem locally
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #7265
2023-05-23 14:29:29 -04:00
Joe Gresock 6ee4632267
NIFI-11566: Adding updateTimeout argument to parameter commands in CLI
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7267.
2023-05-19 12:05:44 +02:00
exceptionfactory b9f4d02094
NIFI-11539 Removed ZooKeeper Migrator Toolkit (#7237) 2023-05-15 14:44:23 -04:00
Pierre Villard 65daa33dfa
NIFI-11506 Upgraded Curator from 5.4.0 to 5.5.0
This closes #7215

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-01 14:09:42 -05:00
exceptionfactory d9f49bca14
NIFI-11491 Removed deprecated PersistentProvenanceRepository
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7200.
2023-04-28 13:42:36 +03:00
exceptionfactory ee03db0e8f
NIFI-11488 Removed Deprecated Sensitive Properties Algorithms
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7198.
2023-04-27 20:16:26 +03:00
timeabarna 3df8c9d2ac
NIFI-11327 Add Export/Import All - NiFi CLI - NiFi Registry (#7092)
* NIFI-11327 Add Export/Import All - NiFi CLI - NiFi Registry
* NIFI-11327 Addressing review comment, updating storage location, improving serialization
* NIFI-11327 Addressing review comments, updating Rest endpoints and storage location handling
* NIFI-11327 Restoring EventType to its original version
* NIFI-11327 Addressing review comments changing skipExisting, updating documentation, adding some more error handling
2023-04-19 13:27:41 -04:00
timeabarna 51f7d6d747
NiFi CLI - add possibility to set 'Maximum Timer Driven Thread Count' (#7165) 2023-04-18 10:18:44 -04:00
exceptionfactory 946064b1c4
NIFI-11460 Switched to JLine 3.23.0 FileNameCompleter
Removed project StandardFileNameCompleter no longer required as of JLine 3.23.0 which incorporated JLine Pull Request 817

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7175.
2023-04-18 09:55:17 +03:00
mr1716 f77659913f
NIFI-11457 Upgraded commons-configuration2 from 2.8.0 to 2.9.0
This closes #7176

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-17 18:05:09 -05:00
Pierre Villard 594e45cf9a
NIFI-11428 Upgraded Groovy to 3.0.17 and spock-core to 2.3
This closes #7159

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-11 08:01:36 -05:00
Pierre Villard 3b216ced92
NIFI-11426 Upgraded JLine from 3.22.0 to 3.23.0
This closes #7157

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-11 08:01:34 -05:00
Pierre Villard a148b6f348
NIFI-11407 Upgraded Azure BOM from 1.2.9 to 1.2.11
- Added Woodstox Core to nifi-toolkit-encrypt-config

This closes #7136

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-10 14:36:48 -05:00
Pierre Villard 0160d6d6b9
NIFI-11410 Upgraded multiple integration dependencies
- Upgraded Spring Boot from 2.7.9 to 2.7.10

This closes #7141

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-10 08:40:09 -05:00
Pierre Villard 4a0b38f4dd
NIFI-11405 Upgraded multiple drivers and test dependencies
This closes #7134

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-07 15:54:51 -05:00
Kevin Doran 08e18220e7
NIFI-11351 Support Docker image builds on ARM64
- Switch from spotify docker-maven-plugin to fabric8 docker-maven-plugin
- Add consistency to docker maven image names and tags
- Other minor Dockerfile changes for consistency

This closes #7105

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-30 14:41:23 -05:00
Kevin Doran 335552f9ff
NIFI-10017 Increase JVM heap size for tls-toolkit
This closes #7100

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-29 17:22:19 -05:00
exceptionfactory 0afd1557e1
NIFI-11316 Removed Admin Toolkit Commands (#7065) 2023-03-29 16:04:30 -04:00
exceptionfactory ce484a0418
NIFI-11307 Removed S2S Toolkit (#7060) 2023-03-29 10:32:01 -04:00
exceptionfactory 132bce4245
NIFI-11276 Upgraded nifi-toolkit-api to Swagger Codegen 3
- Added dependency on swagger-annotations version 3
- Disabled API and Model test generation

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #7038.
2023-03-23 14:07:57 -04:00
Joe Witt 58517e4dd0
NIFI-11329 Updated NAR Plugin to 1.5.0 and Commons Compress to 1.23.0
This closes #7077

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 12:28:40 -05:00
Mark Payne 016d58612c
NIFI-11290 Run Component Primary Node State changes in background thread
- Ensure that components are notified that primary node has changed in a background thread instead of the Leader Election thread and activate/deactivate the thread in the case of Processors so that they can be viewed in the UI and terminated

- Fixed system tests that would fail intermittently because they did not wait for node disconnection to complete and did not properly switch the client to look at the connected node before checking cluster status

This closes #7052

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 12:20:37 -05:00
Mark Payne 6adfb6131d
NIFI-11310: Provide appropriate classpath resources to the ReloadComponent when a processor is terminated
NIFI-11310: Fixed META-INF/services file that was mistakenly listing an extra extension point, due to rebase
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #7061
2023-03-21 14:29:44 -04:00
Chris Sampson d604c87247
NIFI-11311 Updated maven-antrun-plugin to 3.1.0 across all modules
This closes #7062

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-21 08:06:02 -05:00
exceptionfactory 3aecd3bda9
NIFI-11265 Removed nifi-toolkit-flowanalyzer
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7028.
2023-03-13 09:17:54 +01:00
Mark Payne 87e61c50ee
NIFI-11192: Ensure that if ports moved between parent/child group in between flow versions that we can properly handle that. Added system tests to verify.
NIFI-11192: If a failure is encountered when changing the version of a flow from 1 version to another, attempt to rollback the changes instead of just failing with the flow in a bad state
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6981
2023-02-22 15:38:00 -05:00
Peter Turcsanyi c19ec90030 Revert "NIFI-11044 Script/commands to migrate Kafka processors"
This reverts commit 00985edd80.

Reason for reverting: 0.x Kafka processors are not present in NiFi 2.x.
The migration tool needs to be added on the 1.x branch only.
2023-02-14 20:08:55 +01:00
Timea Barna 00985edd80
NIFI-11044 Script/commands to migrate Kafka processors
This closes #6838.

Reviewed-by: Robert Kalmar <rfrostkalmar@gmail.com>
Reviewed-by: Zoltan Kornel Torok <taz19880922@gmail.com>

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-02-14 17:05:20 +01:00
exceptionfactory 1156f4cbc5
NIFI-11165 This closes #6944. Removed optional jigsaw build profiles
- Removed jigsaw build profiles
- Removed Java 11 directories from assemblies

Signed-off-by: Joe Witt <joewitt@apache.org>
2023-02-13 13:01:58 -05:00
Joe Witt 43eab4c81d
NIFI-11103 prepping for 2.0.0 line 2023-02-09 15:32:53 -07:00
Joe Witt efe7b7d30f
NIFI-11103 bumping version refs for 1.20 2023-02-06 11:20:34 -07:00
exceptionfactory e9f5e49d5b
NIFI-11133 This closes #6920. Updated TLS Toolkit Standalone to set Client Key Password
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-02-03 09:49:22 -07:00
exceptionfactory 4b97936d38 NIFI-10932 Changed PKCS12 KeyStore Type Provider to SunJSSE
- Changed from Bouncy Castle to Sun JSSE Provider for Key Stores to improve reading and writing Trust Stores formatted in PKCS12
- Updated TLS Toolkit Key Password handling to remove setting null for PKCS12

Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>

This closes #6881
2023-01-24 15:06:20 +00:00
exceptionfactory 972667461b
NIFI-10580 This closes #6867. Upgraded SLF4J from 1.7.36 to 2.0.6
- Upgraded Logback from 1.2.11 to 1.3.5
- Updated Logback DelayingShutdownHook to DefaultShutdownHook
- Disabled Spring Boot Logging System in favor of standard Logback initialization
- Excluded logback-classic from ZooKeeper and other dependencies to avoid conflicts when running tests
- Excluded spring-boot-starter-logging to avoid failures related to Logback 1.2 and Spring Boot 2.7
- Removed ZooKeeperMigratorTest.groovy based on Apache Curator test server usage of Logback 1.2

NIFI-10580 Added logback-core as explicit dependency

- Set logback-core as provided in root configuration
- Added logback-core as compile dependency in assembly configurations

Signed-off-by: Joe Witt <joewitt@apache.org>
2023-01-23 11:38:10 -07:00
Peter Turcsanyi cb77a96059
NIFI-11088: This closes #6878. updated copyright year to 2023 in NOTICE files
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-01-23 09:42:44 -07:00
exceptionfactory a37887305c
NIFI-11048 This closes #6858. Added FileNameCompleter to avoid style parsing failures
- Added StandardFileNameCompleter with static LS_COLORS to avoid parsing environment variables in Styles.lsStyle()

Signed-off-by: Joe Witt <joewitt@apache.org>
2023-01-18 14:31:55 -07:00
exceptionfactory 87c583042e
NIFI-10941 Removed TestNG references
- Replaced TestNG assertions with JUnit 5
- Excluded testng transitive dependencies from groovy-all
- Added testng exclusion to Maven Enforcer configuration

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6753.
2022-12-05 13:11:59 +01:00