2447 Commits

Author SHA1 Message Date
Matthew Burgess
a84a7cb60a
NIFI-11557: Fixed error with Java 11 code 2023-05-23 16:05:26 -04:00
Mark Payne
82a55ebdd4
NIFI-11557: Avoid using the expensive and unnecessary Files.walkFileTree on startup and initialization of Content Repository. Also performed some code cleanup: IntelliJ flagged many warnings in the class, mostly around methods that are no longer used and potential NullPointerExceptions, so those were cleaned up. Additionally, removed the nifi property for max flowfiles per claim - this property was never implemented. It was referenced, but the way in which is was used curiously had nothing to do with what the property was intended to be used for or for how it was documented. Instead, it was used to limit the max number of claims that could remain writable. As a result, it was removed.
NIFI-11557: Added an additional system test and updated github actions to include surefire-report in order to help diagnose problem that occurred in one of the last system-test runs in Github. Could not replicate problem locally
2023-05-23 14:30:59 -04:00
Bence Simon
9cebdc2824
NIFI-11559 Increased poll time in CollisionAwareResourceProviderWorkerTest
- Incremental adjustment avoids breaking test on slower environments

This closes #7260

Signed-off-by: David Handermann <exceptionfactory@apache.org>
(cherry picked from commit ec01bce20786dfef0bd457fa7de8e04edbef9589)
2023-05-23 10:15:19 -05:00
Joe Gresock
0e6a099eb6
NIFI-11563: Allowing source connectables to be restarted on new connections in the StandardVersionedComponentSynchronizer (#7261) 2023-05-19 08:25:17 -04:00
Nissim Shiman
434154b514 NIFI-10756 Generate error message when processor and/or controller
service is unable to transition to start and/or enabled state

Signed-off-by: Mike Moser <mosermw@apache.org>
2023-05-12 14:46:32 +00:00
M Tien
7f660d7d87
NIFI-11287: detect dependent properties when the property it depends on references a parameter (UI) (#7117)
* NIFI-11287: detect dependent properties when the property it depends on references a parameter

* address review feedback

* address more review comments

* - loadProperties checks type before getting the parameter context
- retrieve the parameter context with ajax call from inside the class

* add type parameter to all places calling loadProperties

* get reference parameters by invoking provided callbacks from the client

* check for user permissions before requesting for referenced parameters

* address review feedback

This closes #7117
2023-05-11 17:05:06 -04:00
Timea Barna
e1ddc44b07
NIFI-11385 Added JMX Metrics REST Resource for Diagnostics
This closes #7124

Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
(cherry picked from commit 5811a9c579a11aa8fc8358851df33d0456a5e10c)
2023-05-08 08:12:16 -05:00
Lehel Boér
ab20a93a90 NIFI-11493: Defaulted dynamically modified classpath fix
This closes #7201.

Co-authored-by: Peter Turcsanyi <turcsanyi@apache.org>
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-05-04 15:43:03 +02:00
Pierre Villard
4bc38d0d93
NIFI-11506 Upgraded Curator from 5.4.0 to 5.5.0
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-01 14:17:25 -05:00
Timea Barna
b146ab7407
NIFI-11473 Flow version change in NiFi should not stop a component when only position is changed
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
2023-04-27 12:58:05 -04:00
Lehel Boér
4e1b1e7926
NIFI-11067 Delete Property History when changing Sensitive status
This closes #7082

Signed-off-by: David Handermann <exceptionfactory@apache.org>
(cherry picked from commit ffaf81ec462c4093f0fcf6d61d36be9849f0eda2)
2023-04-26 11:11:54 -05:00
Scott Aslan
8d04559e07 NIFI-11433 - use .add() for set instead of .push() (#7193)
* replace d3.nest, use add instead of push for Set, use forEach to loop over Set

* es5 function syntax

Merged #7193 into main.
2023-04-25 10:46:39 -07:00
Scott Aslan
10eab2a6d3
[NIFI-11433] update angular, d3, moment, slickgrid, and jquery depen… (#7174)
* [NIFIF-11433] update angular, d3, moment, slickgrid, and jquery dependencies

* address review feedback

* .entries instead of .keys

Merged #7174 into main.
2023-04-24 13:47:01 -07:00
exceptionfactory
290a3ac217
NIFI-11438 Set standard OpenID Connect Scopes
- Restored previous behavior of sending openid and email scopes for OpenID Connect token requests
- Added offline_access scope as the default value in nifi.properties to support Refresh Tokens

This closes #7168
Signed-off-by: Paul Grey <greyp@apache.org>
2023-04-21 11:48:27 -04:00
exceptionfactory
033134e843
NIFI-11461 Improve User and Group Tenants Search (#7181)
* NIFI-11461 Improved User and Group Tenants Search

- Added searchTenants method to NiFiServiceFacade and removed unnecessary object creation
- Updated TenantsResource to use delegated NiFiServiceFacade.searchTenants method
- Changed autocomplete delay from default 300 ms to 500 ms

* NIFI-11461 Adjusted implementation to use EntityFactory.createTenantEntity

This closes #7181

(cherry picked from commit bdff3abcd6d072a61e6399e694e3213732696561)
2023-04-19 09:11:59 -05:00
Bryan Bende
7cb9de69c4 NIFI-11436 Fix NPE during updateFlow when called from a replace request for a PG that is not under version control (#7167) 2023-04-12 19:24:46 -04:00
Pierre Villard
5255d1bad3
NIFI-11417 Upgraded Caffeine from 2.8.1 to 2.9.3
This closes #7148

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-10 08:55:45 -05:00
Pierre Villard
e0c70ca45b
NIFI-11410 Upgraded multiple integration dependencies
- Upgraded Spring Boot from 2.7.9 to 2.7.10

This closes #7141

Signed-off-by: David Handermann <exceptionfactory@apache.org>

(cherry picked from commit 0160d6d6b90ef2d2862d01520161b37d489e2669)
2023-04-10 08:53:20 -05:00
Pierre Villard
2b2d571b82
NIFI-11405 Upgraded multiple drivers and test dependencies
This closes #7134

Signed-off-by: David Handermann <exceptionfactory@apache.org>

(cherry picked from commit 4a0b38f4dd4b497fad3d071d99bfeae0fb15e3d8)
2023-04-07 16:49:44 -05:00
Joe Witt
9c27ddcb1c
NIFI-11357-RC2 prepare for next development iteration 2023-04-03 14:40:08 -07:00
Joe Witt
892f822107
NIFI-11357-RC2 prepare release nifi-1.21.0-RC2 2023-04-03 14:40:07 -07:00
exceptionfactory
fbbb8046df
NIFI-11370 Corrected JWK Set retrieval for NIFI Trust Strategy (#7108)
- Added StandardOidcIdTokenDecoderFactory based on Spring Security OidcIdTokenDecoderFactory with custom REST Operations

Merged #7108 into main.

(cherry picked from commit e4f0508c90f7f354b4210dd80f3dbed5b3254bcd)
2023-03-31 18:28:58 -05:00
exceptionfactory
006d1507d4
NIFI-11365 Corrected OIDC Redirect URI resolution for reverse proxies (#7104)
- Added Authorization Request Resolver with support for building the base redirect URI using allowed proxy headers

This closes #7104

(cherry picked from commit 75eb449a312815b84545c1ee157144255b8fc97d)
2023-03-30 15:29:04 -05:00
Mark Payne
451941aad9
NIFI-11251: Fixed case in which changing Flow Version was improperly attempting to use a Temporary Funnel destination for a connection due to differences in Group ID by instead checking if the destination is reachable; fixed issue in which processors in an inner versioned group were not stopped even though they changed, when a higher-level parent (that was itself under version control) had its version changed. Wrote system tests to verify
- Fixed system tests so that they work properly in Clustered version of RegistryClientIT
- Fixed system test - ensure that we wait for processors to become valid before attempting to start them; also added an additional system test around Controller Services in versioned flows

This closes #7095

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-29 08:35:32 -05:00
exceptionfactory
6a5b8c39ef
NIFI-11356 This closes #7096. Upgraded Nimbus JOSE JWT to 9.31
- Upgraded NiFi Framework from 9.24.3
- Upgraded NiFi Registry from 8.20
- Upgraded transitive msal4j dependency from 9.22
- Upgraded transitive hadoop-auth dependency from 9.8.1

Signed-off-by: Joe Witt <joewitt@apache.org>
2023-03-28 14:23:07 -07:00
exceptionfactory
ff5a783943
NIFI-4890 Refactor OIDC with support for Refresh Tokens (#7013)
* NIFI-4890 Refactored OIDC with support for Refresh Tokens

- Implemented OIDC Authorization Code Grant Flow using Spring Security Filters
- Implemented OIDC RP-Initiated Logout 1.0
- Implemented OAuth2 Token Revocation RFC 7009 for Refresh Tokens
- Added OIDC Bearer Token Refresh Filter for updating application Bearer Tokens from Refresh Token exchanges
- Added configurable Token Refresh Window to application properties
- Removed original implementation and supporting classes

* NIFI-4890 Set Bearer Token expiration based on Access Token

* NIFI-4890 Corrected spelling and naming issues based on feedback

This closes #7013
2023-03-28 09:38:58 -05:00
Mark Payne
d2fdff7b93
NIFI-11333: Do not allow components to be removed while a node is disconnected
This closes #7085

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-27 17:44:31 -05:00
Mark Payne
474e124667
NIFI-11341 Fixed OutputStream.close() handling for Content Claims
Fixed issue in StandardContentClaimWriteCache in which inner OutputStream class did not have an idempotent close() method; as a result, the stream could be written to while already in use for another active FlowFile; fixed bug in ContentClaimInputStream in which skip() method ignored its own BufferedInputStream - this was discovered because it was causing failures in StandardProcessSessionIT; fixed bug in StandardProcessSessionIT in which the length of StandardContentClaim was being doubled because the OutputStream was setting the claim length but that is already handled at a lower level.

This closes #7087

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-27 16:13:11 -05:00
Christian Wahl
06b0093130
NIFI-11232 Fixed buffer handling in ContentClaimInputStream
This closes #6996

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-27 16:13:10 -05:00
Bryan Bende
093935e1cd
NIFI-11332 Close Stream from Files.lines in StandardRuntimeManifestService
This closes #7079

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 14:30:26 -05:00
Joe Witt
fb31fcca54
NIFI-11329 Updated NAR Plugin to 1.5.0 and Commons Compress to 1.23.0
This closes #7077

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 12:32:43 -05:00
Mark Payne
8eeebdb614
NIFI-11290 Run Component Primary Node State changes in background thread
- Ensure that components are notified that primary node has changed in a background thread instead of the Leader Election thread and activate/deactivate the thread in the case of Processors so that they can be viewed in the UI and terminated

- Fixed system tests that would fail intermittently because they did not wait for node disconnection to complete and did not properly switch the client to look at the connected node before checking cluster status

This closes #7052

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 12:21:45 -05:00
Joe Gresock
9208d056ef
NIFI-11330: Restarting service referenced by parameter only when service is active (#7078) 2023-03-23 13:14:48 -04:00
dependabot[bot]
573187eaf1
NIFI-11328 Upgraded Jettison from 1.5.2 to 1.5.4
- Upgraded direct dependencies in Framework and Registry modules as well as Spark bundle

This closes #7073
This closes #7074
This closes #7075

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 10:49:58 -05:00
exceptionfactory
a7daebc1af
NIFI-11325 Upgraded Mockito from 3.12.4 to 4.11.0
- Replaced deprecated Matchers references with ArgumentMatchers
- Removed unnecessary Mockito versions for Registry
- Refactored test configuration to Java for mocking

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7071.
2023-03-23 09:56:43 +01:00
Mark Payne
cece57268e
NIFI-11310: Provide appropriate classpath resources to the ReloadComponent when a processor is terminated
NIFI-11310: Fixed META-INF/services file that was mistakenly listing an extra extension point, due to rebase
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
2023-03-21 14:17:17 -04:00
simonbence
4a6e6751d5 NIFI-11213 Showing version change in older (pre 1.18.0) contained version flows properly (#7017) 2023-03-20 16:59:05 +01:00
Mark Payne
f571d5704e
NIFI-11289: Avoid obtaining read locks on queues when fetching Group Status, except in those few specific situations where it's needed.
This closes #7046

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-15 15:26:44 -05:00
Matt Burgess
9e804fec4e NIFI-11279: Allow event stream processing to continue in CaptureChangeMySQL after sync issue (#7039) 2023-03-14 16:30:41 -04:00
exceptionfactory
cc1e5b314b
NIFI-11194 Remove gzip CSS and JS from nifi-web-ui (#6968)
* NIFI-11194 Removed gzip CSS and JS from nifi-web-ui

* NIFI-11194 Updated comment for static content aggregation

This closes #6968
2023-03-08 11:10:03 -06:00
Joe Gresock
a6687ae71a
NIFI-11247: Using parameterContextName instead of group name when fetching parameters (#7011)
Merged #7011.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
2023-03-07 19:07:13 +01:00
Joe Gresock
bf75070071
NIFI-11238: Creating missing parameter providers when importing inherited provided contexts
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7002.
2023-03-03 22:13:38 +01:00
exceptionfactory
88d2c03b37 NIFI-11245 Corrected reducing Maximum Thread Count while running (#7005)
- Corrected implementation to allow reducing Maximum Thread Count below the default of 10
2023-03-03 11:38:00 -05:00
Mark Payne
51577ff20a
NIFI-11176: When comparing a VersionedConnection with an existing connection, we examined the source component's VersionedComponentId. But the existing component may not have a VersionedComponentId. We should only be comparing these if they are populated
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-03-02 11:16:50 -07:00
Mark Bean
443ef1d9e2
NIFI-11134 Added Label auditing to Flow Configuration History
This closes #6949

Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-02-27 13:02:28 -06:00
Mark Payne
6b15c2544a
NIFI-11192: Ensure that if ports moved between parent/child group in between flow versions that we can properly handle that. Added system tests to verify.
NIFI-11192: If a failure is encountered when changing the version of a flow from 1 version to another, attempt to rollback the changes instead of just failing with the flow in a bad state
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6981
2023-02-22 15:46:11 -05:00
Mark Payne
6f2b9e5f65 NIFI-11189: When synchronizing a ProcessGroup to match a VersionedProcessGroup, do not remove the temporary funnel until the very end. This is important if the temporary funnel already exists in the flow on startup
Signed-off-by: Bence Simon <bsimon@apache.org>
This closes #6963
2023-02-16 15:13:07 +01:00
Bence Simon
f432d98166 NIFI-11159 Fixing connections with source having reassigned id 2023-02-10 11:30:52 -05:00
Joe Witt
8300477ed1
NIFI-11103 preparing support/nifi-1.x line for any subsequent feature/maint releases starting with 1.21-snap 2023-02-09 15:40:01 -07:00
Joe Witt
896fdacbbd
NIFI-11103-RC1 prepare for next development iteration 2023-02-06 11:41:41 -07:00