203 Commits

Author SHA1 Message Date
exceptionfactory
0ecd15537e
NIFI-11532 Removed JUnit 4 and Groovy Test from default dependencies
- Removed hamcrest-all from default dependencies
- Added groovy-test to selected modules with Groovy test classes
- Added junit-vintage-engine to selected modules with JUnit 4 test classes
- Corrected references to JUnit 4 assertions in JUnit 5 test classes
- Removed several unnecessary test classes from nifi-socket-utils
- Removed duplicative Registry toolkit test classes
- Removed Kudu integration tests

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7233

(cherry picked from commit ca6c7f704164e922e11d72e7c972a50ac46e6b0e)
2023-06-13 15:22:40 -05:00
Mark Payne
181b287b1d
NIFI-11670 Refactored Content Repo OutputStream to create new Claim per FlowFile
Rather than creating many FlowFiles with the same Content Claim, refactored content repos' OutputStreams and ClaimWriteCache so that a new ContentClaim is created for each FlowFile. This ensures that we have a content claim offset of 0. The poor performance was due to having to use StreamUtils.skip() in conjunction with the CipherInputStream, which would only skip a max of 511 bytes at a time. By using a separate Content Claim per FlowFile, we no longer need to seek after creating the CipherInputStream

This closes #7363

Signed-off-by: David Handermann <exceptionfactory@apache.org>
(cherry picked from commit 702c6350344e4ae4cd31349747e96c1384017ed4)
2023-06-12 10:23:29 -05:00
Joe Witt
f1d5a8c4de
NIFI-11654-RC1 prepare for next development iteration 2023-06-06 12:32:00 -07:00
Joe Witt
71e3ea9f2c
NIFI-11654-RC1 prepare release nifi-1.22.0-RC1 2023-06-06 12:31:58 -07:00
Bryan Bende
64b110c292
NIFI-11464 Improvements for importing nested versioned flows (#7218)
* NIFI-11464 Improvements for importing nested versioned flows
- Introduce FlowSnapshotContainer to return root snapshot + children
- Introduce ControllerServiceResolver to extract logic from service facade
- Update resolution logic to correctly consider all services in the hierarchy
- Merge additional parameter contexts and parameter providers from child to parent
- Add unit test for controller service resolver
- Replace use of emptSet/emptyMap with new set/map instance
2023-06-05 10:34:10 -04:00
Peter Turcsanyi
8af6f9416d
NIFI-11628 Fixed Object[] and Throwable argument substitution in SimpleProcessLogger
This closes #7362

Signed-off-by: David Handermann <exceptionfactory@apache.org>
(cherry picked from commit 803868935afc424bf0b69b6b82802e557e689c04)
2023-06-02 15:01:38 -05:00
Joe Gresock
0e6a099eb6
NIFI-11563: Allowing source connectables to be restarted on new connections in the StandardVersionedComponentSynchronizer (#7261) 2023-05-19 08:25:17 -04:00
Nissim Shiman
434154b514 NIFI-10756 Generate error message when processor and/or controller
service is unable to transition to start and/or enabled state

Signed-off-by: Mike Moser <mosermw@apache.org>
2023-05-12 14:46:32 +00:00
Timea Barna
b146ab7407
NIFI-11473 Flow version change in NiFi should not stop a component when only position is changed
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
2023-04-27 12:58:05 -04:00
Joe Witt
9c27ddcb1c
NIFI-11357-RC2 prepare for next development iteration 2023-04-03 14:40:08 -07:00
Joe Witt
892f822107
NIFI-11357-RC2 prepare release nifi-1.21.0-RC2 2023-04-03 14:40:07 -07:00
Mark Payne
451941aad9
NIFI-11251: Fixed case in which changing Flow Version was improperly attempting to use a Temporary Funnel destination for a connection due to differences in Group ID by instead checking if the destination is reachable; fixed issue in which processors in an inner versioned group were not stopped even though they changed, when a higher-level parent (that was itself under version control) had its version changed. Wrote system tests to verify
- Fixed system tests so that they work properly in Clustered version of RegistryClientIT
- Fixed system test - ensure that we wait for processors to become valid before attempting to start them; also added an additional system test around Controller Services in versioned flows

This closes #7095

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-29 08:35:32 -05:00
Mark Payne
474e124667
NIFI-11341 Fixed OutputStream.close() handling for Content Claims
Fixed issue in StandardContentClaimWriteCache in which inner OutputStream class did not have an idempotent close() method; as a result, the stream could be written to while already in use for another active FlowFile; fixed bug in ContentClaimInputStream in which skip() method ignored its own BufferedInputStream - this was discovered because it was causing failures in StandardProcessSessionIT; fixed bug in StandardProcessSessionIT in which the length of StandardContentClaim was being doubled because the OutputStream was setting the claim length but that is already handled at a lower level.

This closes #7087

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-27 16:13:11 -05:00
Christian Wahl
06b0093130
NIFI-11232 Fixed buffer handling in ContentClaimInputStream
This closes #6996

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-27 16:13:10 -05:00
Mark Payne
8eeebdb614
NIFI-11290 Run Component Primary Node State changes in background thread
- Ensure that components are notified that primary node has changed in a background thread instead of the Leader Election thread and activate/deactivate the thread in the case of Processors so that they can be viewed in the UI and terminated

- Fixed system tests that would fail intermittently because they did not wait for node disconnection to complete and did not properly switch the client to look at the connected node before checking cluster status

This closes #7052

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 12:21:45 -05:00
Joe Gresock
9208d056ef
NIFI-11330: Restarting service referenced by parameter only when service is active (#7078) 2023-03-23 13:14:48 -04:00
simonbence
4a6e6751d5 NIFI-11213 Showing version change in older (pre 1.18.0) contained version flows properly (#7017) 2023-03-20 16:59:05 +01:00
Mark Payne
f571d5704e
NIFI-11289: Avoid obtaining read locks on queues when fetching Group Status, except in those few specific situations where it's needed.
This closes #7046

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-15 15:26:44 -05:00
Matt Burgess
9e804fec4e NIFI-11279: Allow event stream processing to continue in CaptureChangeMySQL after sync issue (#7039) 2023-03-14 16:30:41 -04:00
Joe Gresock
bf75070071
NIFI-11238: Creating missing parameter providers when importing inherited provided contexts
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7002.
2023-03-03 22:13:38 +01:00
Mark Payne
51577ff20a
NIFI-11176: When comparing a VersionedConnection with an existing connection, we examined the source component's VersionedComponentId. But the existing component may not have a VersionedComponentId. We should only be comparing these if they are populated
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-03-02 11:16:50 -07:00
Mark Payne
6b15c2544a
NIFI-11192: Ensure that if ports moved between parent/child group in between flow versions that we can properly handle that. Added system tests to verify.
NIFI-11192: If a failure is encountered when changing the version of a flow from 1 version to another, attempt to rollback the changes instead of just failing with the flow in a bad state
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6981
2023-02-22 15:46:11 -05:00
Mark Payne
6f2b9e5f65 NIFI-11189: When synchronizing a ProcessGroup to match a VersionedProcessGroup, do not remove the temporary funnel until the very end. This is important if the temporary funnel already exists in the flow on startup
Signed-off-by: Bence Simon <bsimon@apache.org>
This closes #6963
2023-02-16 15:13:07 +01:00
Bence Simon
f432d98166 NIFI-11159 Fixing connections with source having reassigned id 2023-02-10 11:30:52 -05:00
Joe Witt
8300477ed1
NIFI-11103 preparing support/nifi-1.x line for any subsequent feature/maint releases starting with 1.21-snap 2023-02-09 15:40:01 -07:00
Joe Witt
896fdacbbd
NIFI-11103-RC1 prepare for next development iteration 2023-02-06 11:41:41 -07:00
Joe Witt
81296b5b69
NIFI-11103-RC1 prepare release nifi-1.20.0-RC1 2023-02-06 11:41:39 -07:00
Mark Payne
28b9f6fc02
NIFI-11118: This closes #6909. When changing version of a versioned flow, ensure that we properly set the Version Control Information's StorageLocation for the NiFi Registry client in order to maintain backward compatibility
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-02-03 14:02:38 -07:00
Joe Gresock
2609fd5f55
NIFI-11122 Corrected provided parameter context inheritance after creation
This closes #6913

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-02-02 13:20:35 -06:00
Joe Gresock
afdf49b3cd
NIFI-11100: Limiting scope of external controller services during version sync to services outside the versioning context (#6898) 2023-02-01 12:14:38 -05:00
simonbence
ae66ebdc22
NIFI-10973 Introduce differention of comparing embedded versioned flows for UI and synchronisation purposes (#6870)
* NIFI-10973 Introduce differention of comparing embedded versioned flows for UI and synchronisation purposes
2023-01-31 10:58:42 -05:00
Mark Payne
d9f35b8974
NIFI-11105: This closes #6895. When mapping Parameter Context to VersionedParameterContext, don't bother mapping those that have already been mapped, as it is redundant
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-01-26 15:35:18 -07:00
exceptionfactory
d60f541d7e
NIFI-11098 Deprecated ProcessContext encrypt and decrypt methods
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6891.
2023-01-26 18:18:09 +01:00
Joe Gresock
7d99521d4c
NIFI-11089: Resolving previously resolved parameter providers when possible on version sync (#6886) 2023-01-25 13:42:52 -05:00
Paul Grey
567e95aa28
NIFI-11047 - Issue when upgrading version of dataflow with external service
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6849
2023-01-25 12:52:52 -05:00
Mark Payne
228a3441b5
NIFI-11001: This closes #6836. When new components are added, track a mapping of its Versioned Compoenent ID to the component itself. Then, use this when creating connections. This is important because if a Process Group has multiple components with the same Versioned Component ID across multiple child groups, the Versioned Component ID may be changed upon adding the component. Maintaining this mapping allows us to still create the connection correctly.
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-01-25 09:55:16 -07:00
exceptionfactory
84f48b5e8c
NIFI-11090 This closes #6883. Removed unused Jasypt test dependency
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-01-24 08:42:33 -07:00
Joe Gresock
8640989ad7
NIFI-11039: Fixing bug when importing parameter providers from nifi-registry (#6868)
* NIFI-11039: Fixing NPE in registry retrieval
* NIFI-11039: Fixing bug when importing parameter providers from nifi registry
2023-01-23 11:59:34 -05:00
Joe Gresock
29618ae5c6
NIFI-11076: Resolving deadlock issue in StandardParameterContext (#6865)
* NIFI-11076: Resolving deadlock issue in StandardParameterContext

* NIFI-11076: Resolving deadlock issue in StandardParameterContext
2023-01-23 10:54:38 -05:00
Timea Barna
a9274c12c5
NIFI-11079 Extend QueryNifiReportingTask and ProcessGroup Summary Table
Correcting EmbeddedQuestDbStatusHistoryRepositoryForComponentsTest

Correcting TestQueryNiFiReportingTask

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6869
2023-01-20 15:22:20 -05:00
krisztina-zsihovszki
9ad1d8a7ae
NIFI-11028 Handle controller service 'Scheduled State' change (#6831) 2023-01-17 18:40:47 -05:00
Mark Payne
d5c79fdcd1
NIFI-10887: Addressed performance concerned. Use String.indexOf() instead of Pattern.matcher() when using Literal Replace. Use a NonFlushableOutputStream when ProcessSession.write() is called. Implemented hashCode() on AbstractConnection. Updated default Run Schedule on ReplaceText from 0 ms to 25 ms. Added a Surround Replacement strategy that allows both prepending and appending text. Updated unit tests to account for this.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6724
2023-01-11 12:28:59 -05:00
Nissim Shiman
fe25424233
NIFI-10772 Clarify logs on shutdown where controller service and/or
processor were unable to properly start

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6829
2023-01-10 13:56:00 -05:00
exceptionfactory
0d9dc6c540
NIFI-9167 Converted remaining nifi-framework tests to JUnit 5
NIFI-9167 Addressed feedback and improved tests using TempDir

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6823
2023-01-10 13:53:51 -05:00
dan-s1
b74c5b423f
NIFI-10979 Additional nifi-framework upgrades from JUnit 4 to 5
This closes #6806

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-12-30 15:18:38 -06:00
Mark Payne
78be613a0f
NIFI-10888: When inferring a schema using a Record Reader, buffer up to 1 MB of FlowFile content for the schema inference so that when we read the contents to obtain records we can use the buffered data. This helps in cases of small FlowFiles by not having to seek back to the beginning of the FlowFile every time.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6725
2022-12-14 12:09:32 -05:00
Mark Payne
2473683ce5 NIFI-10918: When fetching a flow from a Flow Registry, if it references any 'internal versioned flows' instead of requiring that we have a client configured for the appropriate URL, attempt to fetch the flow from each client. We will start with the clients that do report that they can handle the URL but will try others as well. As soon as we successfully fetch the flow, we stop.
NIFI-10918: Fixed checkstyle violations

This closes #6736
Signed-off-by: Bence Simon <bsimon@apache.org>
2022-12-02 14:24:40 +01:00
dan-s1
e8d554d9b4
NIFI-10876 Converted some nifi-framework tests from JUnit 4 to JUnit 5
This closes #6720

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-11-28 14:47:17 -06:00
Joe Witt
75e7c9e47c
NIFI-10854-RC1 prepare for next development iteration 2022-11-22 19:22:47 -07:00
Joe Witt
ec87bf93ad
NIFI-10854-RC1 prepare release nifi-1.19.0-RC1 2022-11-22 19:22:45 -07:00