Commit Graph

2403 Commits

Author SHA1 Message Date
exceptionfactory 0ed1efc50a
NIFI-11712 Removed java.net.preferIPv4Stack from bootstrap.conf
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7392.
2023-06-19 10:22:31 +02:00
exceptionfactory b6b621bf9e
NIFI-11717 Updated Required Java Version to 17.0.6
- Added Groovy Plugin Repository for groovy-eclipse-compiler

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7397.
2023-06-19 10:19:55 +02:00
exceptionfactory ca6c7f7041 NIFI-11532 Removed JUnit 4 and Groovy Test from default dependencies
- Removed hamcrest-all from default dependencies
- Added groovy-test to selected modules with Groovy test classes
- Added junit-vintage-engine to selected modules with JUnit 4 test classes
- Corrected references to JUnit 4 assertions in JUnit 5 test classes
- Removed several unnecessary test classes from nifi-socket-utils
- Removed duplicative Registry toolkit test classes
- Removed Kudu integration tests

NIFI-11532 Corrected scope for junit-vintage-engine for Elasticsearch

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7233
2023-06-13 15:36:23 -04:00
exceptionfactory 9c2f15cc18
NIFI-11680 Corrected Buffer Size Calculation for Connection Balancing (#7370)
- Resolved BufferOverflowException in PeerChannel with Bouncy Castle Provider
- Changed prepareForWrite() to use Destination Buffer remaining instead of Application Buffer Size
- Changed encrypt() to Packet Buffer Size instead of Application Buffer Size
2023-06-13 11:01:27 -04:00
Mark Payne 787e0d8261
NIFI-11681: Terminate Process Sessions before interrupting processor threads
This closes #7371

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-13 08:24:03 -05:00
Mark Payne 702c635034
NIFI-11670 Refactored Content Repo OutputStream to create new Claim per FlowFile
Rather than creating many FlowFiles with the same Content Claim, refactored content repos' OutputStreams and ClaimWriteCache so that a new ContentClaim is created for each FlowFile. This ensures that we have a content claim offset of 0. The poor performance was due to having to use StreamUtils.skip() in conjunction with the CipherInputStream, which would only skip a max of 511 bytes at a time. By using a separate Content Claim per FlowFile, we no longer need to seek after creating the CipherInputStream

This closes #7363

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-12 09:47:44 -05:00
siddr dd2486d3e0 NIFI-11587: Update questdb to 6.7
This closes #7319
Signed-off-by: Bence Simon <bsimon@apache.org>
2023-06-07 09:39:44 +02:00
Shane Ardell 7a7f6f6229
NIFI-11593: Fix hiding logic for properties (#7339)
* NIFI-11593: fix hiding logic for properties with references or without dependent values

* NIFI-11593: remove arrow functions

Merged #7339 into main.
2023-06-05 13:36:49 -07:00
Bryan Bende ce50ffbe0e
NIFI-11464 Improvements for importing nested versioned flows (#7218)
* NIFI-11464 Improvements for importing nested versioned flows
- Introduce FlowSnapshotContainer to return root snapshot + children
- Introduce ControllerServiceResolver to extract logic from service facade
- Update resolution logic to correctly consider all services in the hierarchy
- Merge additional parameter contexts and parameter providers from child to parent
- Add unit test for controller service resolver
- Replace use of emptSet/emptyMap with new set/map instance
2023-06-05 10:11:33 -04:00
Peter Turcsanyi 803868935a
NIFI-11628 Fixed Object[] and Throwable argument substitution in SimpleProcessLogger
This closes #7362

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-02 14:59:07 -05:00
Mark Payne df9f8ddcfb NIFI-11636: Do not buffer Parquet content into memory unnecessarily
NIFI-11636: Change default log level from parquet internal reader to WARN as it logs excessively at INFO level

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7334
2023-06-02 15:35:10 -04:00
Scott Aslan 7a10ba6165
[NIFI-11560] call forEach on Map (#7297)
* [NIFI-11560] call forEach on Map

* getComponentRestrictions returns Map, use forEach

This closes #7297
2023-06-01 16:13:08 -04:00
Mark Payne 50811660d0
NIFI-11603 Refactored Socket-based tests to use Dynamic Ports
- Removed NetworkUtils methods for getting available ports
- Updated Socket-based components to support using 0 to listen on a random available port for improved test reliability

This closes #7299

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-01 13:33:06 -05:00
Mark Payne 4b7e20740e
NIFI-11591: Added additional logging for DynamicallyModifyClasspath system test
This closes #7312

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-31 16:23:07 -05:00
Bence Simon 007bf3bcec
NIFI-11596 Adjusting Swagger contract to cover RegistryClients as possible reference type for Controller Services
This closes #7295

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-31 11:02:48 -05:00
Michael Moser eca4f5d68f
NIFI-11392 for CRON scheduled components, improved cancellation of … (#7232)
NIFI-11392 for CRON scheduled components, improved cancellation of futures for thread cleanup
2023-05-30 11:09:06 -04:00
NissimShiman a59ea4a135
NIFI-11109 Allow registry client's class in flow.xml/json to not change (#7273)
when underlying nar for nifi-flow-registry-client-nar is missing
2023-05-24 13:32:31 -04:00
Mark Payne a12c9ca9c7
NIFI-11557: Avoid using the expensive and unnecessary Files.walkFileTree on startup and initialization of Content Repository. Also performed some code cleanup: IntelliJ flagged many warnings in the class, mostly around methods that are no longer used and potential NullPointerExceptions, so those were cleaned up. Additionally, removed the nifi property for max flowfiles per claim - this property was never implemented. It was referenced, but the way in which is was used curiously had nothing to do with what the property was intended to be used for or for how it was documented. Instead, it was used to limit the max number of claims that could remain writable. As a result, it was removed.
NIFI-11557: Added an additional system test and updated github actions to include surefire-report in order to help diagnose problem that occurred in one of the last system-test runs in Github. Could not replicate problem locally
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #7265
2023-05-23 14:29:29 -04:00
Joe Gresock 28f003d886
NIFI-11563: Allowing source connectables to be restarted on new connections in the StandardVersionedComponentSynchronizer (#7261) 2023-05-19 08:24:13 -04:00
exceptionfactory bfe4110b56
NIFI-11513 Remove deprecated NarProvider and implementations (#7222)
* NIFI-11513 Removed deprecated NarProvider and implementations

- Removed HDFSNarProvider
- Removed NiFiRegistryNarProvider
- Removed nifi-external-resource-utils module from nifi-commons
- Removed unnecessary deprecated annotations from EncryptedSchemaRepositoryRecordSerde
- Removed deprecated method from ContentRequestContext interface
- Restored NiFi Registry Provider example properties
2023-05-16 16:10:25 -04:00
exceptionfactory 8ebecdc3ab
NIFI-11554 Upgraded OpenSAML from 3.4.6 to 4.3.0
- Added Shibboleth repository for OpenSAML
- Replaced deprecated OpenSAML 3 Spring Security components with OpenSAML 4

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7251.
2023-05-16 10:34:23 +02:00
Nissim Shiman fd2138b8cf NIFI-10756 Generate error message when processor and/or controller
service is unable to transition to start and/or enabled state

Signed-off-by: Mike Moser <mosermw@apache.org>
2023-05-12 14:40:47 +00:00
M Tien 09cef4ff99
NIFI-11287: detect dependent properties when the property it depends on references a parameter (UI) (#7117)
* NIFI-11287: detect dependent properties when the property it depends on references a parameter

* address review feedback

* address more review comments

* - loadProperties checks type before getting the parameter context
- retrieve the parameter context with ajax call from inside the class

* add type parameter to all places calling loadProperties

* get reference parameters by invoking provided callbacks from the client

* check for user permissions before requesting for referenced parameters

* address review feedback

This closes #7117
2023-05-11 17:00:57 -04:00
exceptionfactory e302f2aff7
NIFI-10976 Added Previous Cluster State Provider configuration (#7235)
- Added methods to enumerate Stored Component Identifiers on State Provider interface and implementations
- Added nifi.state.management.provider.cluster.previous to nifi.properties
- Updated State Manager Provider to restore Cluster State from Previous Cluster Provider
- Updated Configuring State Providers documentation for new property
2023-05-10 16:22:05 -04:00
Timea Barna 5811a9c579
NIFI-11385 Added JMX Metrics REST Resource for Diagnostics
This closes #7124

Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-08 08:11:20 -05:00
Lehel Boér 265b132e73
NIFI-11493: Defaulted dynamically modified classpath fix
This closes #7201.

Co-authored-by: Peter Turcsanyi <turcsanyi@apache.org>
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-05-04 15:20:56 +02:00
exceptionfactory d9f49bca14
NIFI-11491 Removed deprecated PersistentProvenanceRepository
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7200.
2023-04-28 13:42:36 +03:00
exceptionfactory ee03db0e8f
NIFI-11488 Removed Deprecated Sensitive Properties Algorithms
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7198.
2023-04-27 20:16:26 +03:00
Timea Barna d54940c397
NIFI-11473 Flow version change in NiFi should not stop a component when only position is changed
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #7185
2023-04-27 12:59:24 -04:00
Lehel Boér ffaf81ec46
NIFI-11067 Delete Property History when changing Sensitive status
This closes #7082

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-26 09:55:21 -05:00
Scott Aslan 387504b66f
NIFI-11433 - use .add() for set instead of .push() (#7193)
* replace d3.nest, use add instead of push for Set, use forEach to loop over Set

* es5 function syntax

Merged #7193 into main.
2023-04-25 10:43:45 -07:00
Scott Aslan 38ece1d7d9
[NIFI-11433] update angular, d3, moment, slickgrid, and jquery depen… (#7174)
* [NIFIF-11433] update angular, d3, moment, slickgrid, and jquery dependencies

* address review feedback

* .entries instead of .keys

Merged #7174 into main.
2023-04-24 10:16:20 -07:00
exceptionfactory 1639ecee11
NIFI-11438 Set standard OpenID Connect Scopes
- Restored previous behavior of sending openid and email scopes for OpenID Connect token requests
- Added offline_access scope as the default value in nifi.properties to support Refresh Tokens

This closes #7168
Signed-off-by: Paul Grey <greyp@apache.org>
2023-04-21 11:29:16 -04:00
exceptionfactory bdff3abcd6
NIFI-11461 Improve User and Group Tenants Search (#7181)
* NIFI-11461 Improved User and Group Tenants Search

- Added searchTenants method to NiFiServiceFacade and removed unnecessary object creation
- Updated TenantsResource to use delegated NiFiServiceFacade.searchTenants method
- Changed autocomplete delay from default 300 ms to 500 ms

* NIFI-11461 Adjusted implementation to use EntityFactory.createTenantEntity

This closes #7181
2023-04-19 09:58:25 -04:00
Mark Payne 7f7cccfdc2
NIFI-11241 Initial implementation of Python-based Processor API with Py4J
- Updated GitHub workflow so that system tests include Python 3.9
- Updated GitHub actions to build necessary modules for system tests

This closes #7003

Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-14 15:03:34 -05:00
Bryan Bende ba93f60731
NIFI-11436 Fix NPE during updateFlow when called from a replace request for a PG that is not under version control (#7167) 2023-04-12 19:24:22 -04:00
Pierre Villard b0997f843d
NIFI-11416 Upgraded Caffeine from 2.8.1 to 3.1.6
This closes #7147

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-10 08:45:12 -05:00
Pierre Villard 4a0b38f4dd
NIFI-11405 Upgraded multiple drivers and test dependencies
This closes #7134

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-07 15:54:51 -05:00
exceptionfactory e4f0508c90
NIFI-11370 Corrected JWK Set retrieval for NIFI Trust Strategy (#7108)
- Added StandardOidcIdTokenDecoderFactory based on Spring Security OidcIdTokenDecoderFactory with custom REST Operations

Merged #7108 into main.
2023-03-31 16:26:02 -07:00
Matthew Burgess 59eebb9906
NIFI-8710: Pass details instead of source ID for Provenance per the interface doc
This closes #5224

Signed-off-by: Nandor Soma Abonyi <nsabonyi@apache.org>
2023-03-31 18:43:22 +02:00
exceptionfactory 75eb449a31
NIFI-11365 Corrected OIDC Redirect URI resolution for reverse proxies (#7104)
- Added Authorization Request Resolver with support for building the base redirect URI using allowed proxy headers

This closes #7104
2023-03-30 16:26:35 -04:00
Mark Payne 23d6d6ede4
NIFI-11251: Fixed case in which changing Flow Version was improperly attempting to use a Temporary Funnel destination for a connection due to differences in Group ID by instead checking if the destination is reachable; fixed issue in which processors in an inner versioned group were not stopped even though they changed, when a higher-level parent (that was itself under version control) had its version changed. Wrote system tests to verify
- Fixed system tests so that they work properly in Clustered version of RegistryClientIT
- Fixed system test - ensure that we wait for processors to become valid before attempting to start them; also added an additional system test around Controller Services in versioned flows

This closes #7095

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-29 08:30:18 -05:00
exceptionfactory 26400fcbe9
NIFI-4890 Refactor OIDC with support for Refresh Tokens (#7013)
* NIFI-4890 Refactored OIDC with support for Refresh Tokens

- Implemented OIDC Authorization Code Grant Flow using Spring Security Filters
- Implemented OIDC RP-Initiated Logout 1.0
- Implemented OAuth2 Token Revocation RFC 7009 for Refresh Tokens
- Added OIDC Bearer Token Refresh Filter for updating application Bearer Tokens from Refresh Token exchanges
- Added configurable Token Refresh Window to application properties
- Removed original implementation and supporting classes

* NIFI-4890 Set Bearer Token expiration based on Access Token

* NIFI-4890 Corrected spelling and naming issues based on feedback

This closes #7013
2023-03-28 08:35:10 -04:00
Mark Payne 94ae926c42
NIFI-11333: Do not allow components to be removed while a node is disconnected
This closes #7085

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-27 17:43:38 -05:00
Mark Payne 969fc50778
NIFI-11341 Fixed OutputStream.close() handling for Content Claims
Fixed issue in StandardContentClaimWriteCache in which inner OutputStream class did not have an idempotent close() method; as a result, the stream could be written to while already in use for another active FlowFile; fixed bug in ContentClaimInputStream in which skip() method ignored its own BufferedInputStream - this was discovered because it was causing failures in StandardProcessSessionIT; fixed bug in StandardProcessSessionIT in which the length of StandardContentClaim was being doubled because the OutputStream was setting the claim length but that is already handled at a lower level.

This closes #7087

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-27 16:05:50 -05:00
Christian Wahl 6bd893da16
NIFI-11232 Fixed buffer handling in ContentClaimInputStream
This closes #6996

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-27 16:02:27 -05:00
Bryan Bende 5386095083
NIFI-11332 Close Stream from Files.lines in StandardRuntimeManifestService
This closes #7079

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 14:29:17 -05:00
Mark Payne 016d58612c
NIFI-11290 Run Component Primary Node State changes in background thread
- Ensure that components are notified that primary node has changed in a background thread instead of the Leader Election thread and activate/deactivate the thread in the case of Processors so that they can be viewed in the UI and terminated

- Fixed system tests that would fail intermittently because they did not wait for node disconnection to complete and did not properly switch the client to look at the connected node before checking cluster status

This closes #7052

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 12:20:37 -05:00
Joe Gresock 9de93f19a4
NIFI-11330: Restarting service referenced by parameter only when service is active (#7078) 2023-03-23 13:13:22 -04:00
exceptionfactory 8154fcce49
NIFI-11325 Upgraded Mockito from 3.12.4 to 4.11.0
- Replaced deprecated Matchers references with ArgumentMatchers
- Removed unnecessary Mockito versions for Registry
- Refactored test configuration to Java for mocking

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7071.
2023-03-23 09:56:15 +01:00