213 Commits

Author SHA1 Message Date
Zackeus Bengtsson
387c22b22c
NIFI-12122 Fixed persistence of Parameter Context descriptions
Fixed bug where parameter context descriptions were not loaded on NiFi startup and overwritten as empty

This closes: #7787

Signed-off-by: David Handermann <exceptionfactory@apache.org>
(cherry picked from commit 0a47157640c1502a0379a7f66d764d2d7d344872)
2023-10-13 18:39:55 -05:00
Mark Payne
4d00f8e156
NIFI-12228: This closes #7882. Fixed issue with FlowFile Concucrrency that can occasionally bring in more data than it should.
Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-10-13 13:36:54 -07:00
Bryan Bende
ad915f773d
NIFI-12186 Add ability to export a versioned reporting task snapshot (#7853)
* NIFI-12186 Add ability to export a versioned reporting task snapshot
- Add CLI commands and optional query param to specify specific reporting task
2023-10-09 11:39:43 -04:00
Peter Turcsanyi
e07bb19233
NIFI-12075 Deprecated ComponentLog methods with Object[] and Throwable
- Updated component references to remove use of Object[] wrapping for log methods

This closes #7758

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-19 13:23:07 -05:00
exceptionfactory
27a690a30a NIFI-11832-RC3 prepare for next development iteration 2023-07-25 14:03:12 -05:00
exceptionfactory
b7792f03b7 NIFI-11832-RC3 prepare release nifi-1.23.0-RC3 2023-07-25 14:03:11 -05:00
Mark Payne
eb09df672b NIFI-11783: Fixed a bug in which a FlowFile split into multiple and then removed did not properly handle retries when one of the outputs was retried.
Signed-off-by: Matt Burgess <mattyb149@apache.org>
2023-07-19 15:50:35 -04:00
Bence Simon
5acbb8a702
NIFI-11706 Add option to create dedicated Parameter Contexts for Imported Flows
This closes #7401

Signed-off-by: David Handermann <exceptionfactory@apache.org>
(cherry picked from commit cadf2fb6f25caf334bf787292380af5b25fdcbb1)
2023-06-30 13:15:41 -05:00
exceptionfactory
cb76a62a14
NIFI-3065 Corrected TestStandardLoggingContext for Java 8
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-26 11:09:50 -05:00
timeabarna
d5fb6b4518 NIFI-3065 Per Process Group logging (#7315)
* NIFI-3065 Per Process Group logging
2023-06-23 21:20:34 -04:00
exceptionfactory
0ecd15537e
NIFI-11532 Removed JUnit 4 and Groovy Test from default dependencies
- Removed hamcrest-all from default dependencies
- Added groovy-test to selected modules with Groovy test classes
- Added junit-vintage-engine to selected modules with JUnit 4 test classes
- Corrected references to JUnit 4 assertions in JUnit 5 test classes
- Removed several unnecessary test classes from nifi-socket-utils
- Removed duplicative Registry toolkit test classes
- Removed Kudu integration tests

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7233

(cherry picked from commit ca6c7f704164e922e11d72e7c972a50ac46e6b0e)
2023-06-13 15:22:40 -05:00
Mark Payne
181b287b1d
NIFI-11670 Refactored Content Repo OutputStream to create new Claim per FlowFile
Rather than creating many FlowFiles with the same Content Claim, refactored content repos' OutputStreams and ClaimWriteCache so that a new ContentClaim is created for each FlowFile. This ensures that we have a content claim offset of 0. The poor performance was due to having to use StreamUtils.skip() in conjunction with the CipherInputStream, which would only skip a max of 511 bytes at a time. By using a separate Content Claim per FlowFile, we no longer need to seek after creating the CipherInputStream

This closes #7363

Signed-off-by: David Handermann <exceptionfactory@apache.org>
(cherry picked from commit 702c6350344e4ae4cd31349747e96c1384017ed4)
2023-06-12 10:23:29 -05:00
Joe Witt
f1d5a8c4de
NIFI-11654-RC1 prepare for next development iteration 2023-06-06 12:32:00 -07:00
Joe Witt
71e3ea9f2c
NIFI-11654-RC1 prepare release nifi-1.22.0-RC1 2023-06-06 12:31:58 -07:00
Bryan Bende
64b110c292
NIFI-11464 Improvements for importing nested versioned flows (#7218)
* NIFI-11464 Improvements for importing nested versioned flows
- Introduce FlowSnapshotContainer to return root snapshot + children
- Introduce ControllerServiceResolver to extract logic from service facade
- Update resolution logic to correctly consider all services in the hierarchy
- Merge additional parameter contexts and parameter providers from child to parent
- Add unit test for controller service resolver
- Replace use of emptSet/emptyMap with new set/map instance
2023-06-05 10:34:10 -04:00
Peter Turcsanyi
8af6f9416d
NIFI-11628 Fixed Object[] and Throwable argument substitution in SimpleProcessLogger
This closes #7362

Signed-off-by: David Handermann <exceptionfactory@apache.org>
(cherry picked from commit 803868935afc424bf0b69b6b82802e557e689c04)
2023-06-02 15:01:38 -05:00
Joe Gresock
0e6a099eb6
NIFI-11563: Allowing source connectables to be restarted on new connections in the StandardVersionedComponentSynchronizer (#7261) 2023-05-19 08:25:17 -04:00
Nissim Shiman
434154b514 NIFI-10756 Generate error message when processor and/or controller
service is unable to transition to start and/or enabled state

Signed-off-by: Mike Moser <mosermw@apache.org>
2023-05-12 14:46:32 +00:00
Timea Barna
b146ab7407
NIFI-11473 Flow version change in NiFi should not stop a component when only position is changed
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
2023-04-27 12:58:05 -04:00
Joe Witt
9c27ddcb1c
NIFI-11357-RC2 prepare for next development iteration 2023-04-03 14:40:08 -07:00
Joe Witt
892f822107
NIFI-11357-RC2 prepare release nifi-1.21.0-RC2 2023-04-03 14:40:07 -07:00
Mark Payne
451941aad9
NIFI-11251: Fixed case in which changing Flow Version was improperly attempting to use a Temporary Funnel destination for a connection due to differences in Group ID by instead checking if the destination is reachable; fixed issue in which processors in an inner versioned group were not stopped even though they changed, when a higher-level parent (that was itself under version control) had its version changed. Wrote system tests to verify
- Fixed system tests so that they work properly in Clustered version of RegistryClientIT
- Fixed system test - ensure that we wait for processors to become valid before attempting to start them; also added an additional system test around Controller Services in versioned flows

This closes #7095

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-29 08:35:32 -05:00
Mark Payne
474e124667
NIFI-11341 Fixed OutputStream.close() handling for Content Claims
Fixed issue in StandardContentClaimWriteCache in which inner OutputStream class did not have an idempotent close() method; as a result, the stream could be written to while already in use for another active FlowFile; fixed bug in ContentClaimInputStream in which skip() method ignored its own BufferedInputStream - this was discovered because it was causing failures in StandardProcessSessionIT; fixed bug in StandardProcessSessionIT in which the length of StandardContentClaim was being doubled because the OutputStream was setting the claim length but that is already handled at a lower level.

This closes #7087

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-27 16:13:11 -05:00
Christian Wahl
06b0093130
NIFI-11232 Fixed buffer handling in ContentClaimInputStream
This closes #6996

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-27 16:13:10 -05:00
Mark Payne
8eeebdb614
NIFI-11290 Run Component Primary Node State changes in background thread
- Ensure that components are notified that primary node has changed in a background thread instead of the Leader Election thread and activate/deactivate the thread in the case of Processors so that they can be viewed in the UI and terminated

- Fixed system tests that would fail intermittently because they did not wait for node disconnection to complete and did not properly switch the client to look at the connected node before checking cluster status

This closes #7052

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 12:21:45 -05:00
Joe Gresock
9208d056ef
NIFI-11330: Restarting service referenced by parameter only when service is active (#7078) 2023-03-23 13:14:48 -04:00
simonbence
4a6e6751d5 NIFI-11213 Showing version change in older (pre 1.18.0) contained version flows properly (#7017) 2023-03-20 16:59:05 +01:00
Mark Payne
f571d5704e
NIFI-11289: Avoid obtaining read locks on queues when fetching Group Status, except in those few specific situations where it's needed.
This closes #7046

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-15 15:26:44 -05:00
Matt Burgess
9e804fec4e NIFI-11279: Allow event stream processing to continue in CaptureChangeMySQL after sync issue (#7039) 2023-03-14 16:30:41 -04:00
Joe Gresock
bf75070071
NIFI-11238: Creating missing parameter providers when importing inherited provided contexts
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7002.
2023-03-03 22:13:38 +01:00
Mark Payne
51577ff20a
NIFI-11176: When comparing a VersionedConnection with an existing connection, we examined the source component's VersionedComponentId. But the existing component may not have a VersionedComponentId. We should only be comparing these if they are populated
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-03-02 11:16:50 -07:00
Mark Payne
6b15c2544a
NIFI-11192: Ensure that if ports moved between parent/child group in between flow versions that we can properly handle that. Added system tests to verify.
NIFI-11192: If a failure is encountered when changing the version of a flow from 1 version to another, attempt to rollback the changes instead of just failing with the flow in a bad state
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6981
2023-02-22 15:46:11 -05:00
Mark Payne
6f2b9e5f65 NIFI-11189: When synchronizing a ProcessGroup to match a VersionedProcessGroup, do not remove the temporary funnel until the very end. This is important if the temporary funnel already exists in the flow on startup
Signed-off-by: Bence Simon <bsimon@apache.org>
This closes #6963
2023-02-16 15:13:07 +01:00
Bence Simon
f432d98166 NIFI-11159 Fixing connections with source having reassigned id 2023-02-10 11:30:52 -05:00
Joe Witt
8300477ed1
NIFI-11103 preparing support/nifi-1.x line for any subsequent feature/maint releases starting with 1.21-snap 2023-02-09 15:40:01 -07:00
Joe Witt
896fdacbbd
NIFI-11103-RC1 prepare for next development iteration 2023-02-06 11:41:41 -07:00
Joe Witt
81296b5b69
NIFI-11103-RC1 prepare release nifi-1.20.0-RC1 2023-02-06 11:41:39 -07:00
Mark Payne
28b9f6fc02
NIFI-11118: This closes #6909. When changing version of a versioned flow, ensure that we properly set the Version Control Information's StorageLocation for the NiFi Registry client in order to maintain backward compatibility
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-02-03 14:02:38 -07:00
Joe Gresock
2609fd5f55
NIFI-11122 Corrected provided parameter context inheritance after creation
This closes #6913

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-02-02 13:20:35 -06:00
Joe Gresock
afdf49b3cd
NIFI-11100: Limiting scope of external controller services during version sync to services outside the versioning context (#6898) 2023-02-01 12:14:38 -05:00
simonbence
ae66ebdc22
NIFI-10973 Introduce differention of comparing embedded versioned flows for UI and synchronisation purposes (#6870)
* NIFI-10973 Introduce differention of comparing embedded versioned flows for UI and synchronisation purposes
2023-01-31 10:58:42 -05:00
Mark Payne
d9f35b8974
NIFI-11105: This closes #6895. When mapping Parameter Context to VersionedParameterContext, don't bother mapping those that have already been mapped, as it is redundant
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-01-26 15:35:18 -07:00
exceptionfactory
d60f541d7e
NIFI-11098 Deprecated ProcessContext encrypt and decrypt methods
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6891.
2023-01-26 18:18:09 +01:00
Joe Gresock
7d99521d4c
NIFI-11089: Resolving previously resolved parameter providers when possible on version sync (#6886) 2023-01-25 13:42:52 -05:00
Paul Grey
567e95aa28
NIFI-11047 - Issue when upgrading version of dataflow with external service
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6849
2023-01-25 12:52:52 -05:00
Mark Payne
228a3441b5
NIFI-11001: This closes #6836. When new components are added, track a mapping of its Versioned Compoenent ID to the component itself. Then, use this when creating connections. This is important because if a Process Group has multiple components with the same Versioned Component ID across multiple child groups, the Versioned Component ID may be changed upon adding the component. Maintaining this mapping allows us to still create the connection correctly.
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-01-25 09:55:16 -07:00
exceptionfactory
84f48b5e8c
NIFI-11090 This closes #6883. Removed unused Jasypt test dependency
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-01-24 08:42:33 -07:00
Joe Gresock
8640989ad7
NIFI-11039: Fixing bug when importing parameter providers from nifi-registry (#6868)
* NIFI-11039: Fixing NPE in registry retrieval
* NIFI-11039: Fixing bug when importing parameter providers from nifi registry
2023-01-23 11:59:34 -05:00
Joe Gresock
29618ae5c6
NIFI-11076: Resolving deadlock issue in StandardParameterContext (#6865)
* NIFI-11076: Resolving deadlock issue in StandardParameterContext

* NIFI-11076: Resolving deadlock issue in StandardParameterContext
2023-01-23 10:54:38 -05:00
Timea Barna
a9274c12c5
NIFI-11079 Extend QueryNifiReportingTask and ProcessGroup Summary Table
Correcting EmbeddedQuestDbStatusHistoryRepositoryForComponentsTest

Correcting TestQueryNiFiReportingTask

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6869
2023-01-20 15:22:20 -05:00