227 Commits

Author SHA1 Message Date
Joe Gresock
6258f5fe04
NIFI-13831: Adding inheritance to versioned component synchronizer parameter context synchronization when considering referencing components to restart
Signed-off-by: Csaba Bejan <bejan.csaba@gmail.com>

This closes #9338.
2024-10-04 17:56:23 -04:00
Pierre Villard
3af800a017
NIFI-13826 Upgraded Jetty to 12.0.14 and Jackson to 2.18.0 along with others (#9331)
- Jackson from 2.17.2 to 2.18.0
- zstd-jni from 1.5.6-5 to 1.5.6-6
- gson from 2.10.1 to 2.11.0
- okio from 3.9.0 to 3.9.1
- Fabric8 Kubernetes from 6.13.1 to 6.13.4
- Netty from 4.1.113.Final to 4.1.114.Final
- swagger-annotations from 2.2.23 to 2.2.24
- avro from 1.11.3 to 1.11.4
- commons-lang3 from 3.16.0 to 3.17.0
- log4j from 2.24.0 to 2.24.1
- Jetty from 12.0.13 to 12.0.14
- junit-bom from 5.10.3 to 5.10.4
- junit-platform-commons from 1.10.3 to 1.10.4
- mockito from 5.13.0 to 5.14.1
- testcontainers from 1.20.1 to 1.20.2
- snakeyaml from 2.2 to 2.3
- AWS SDK v2 from 2.28.4 to 2.28.13
- flyway from 10.18.0 to 10.18.2
- jline from 3.26.3 to 3.27.0
- neo4j driver from 5.24.0 to 5.25.0
- maxmind from 3.1.0 to 3.1.1
- geoip2 from 4.2.0 to 4.2.1
- amqp-client from 5.21.0 to 5.22.0
- commons-csv from 1.11.0 to 1.12.0
- splunk from 1.9.4 to 1.9.5
- lucene from 9.11.1 to 9.12.0
- Google BOM from 26.46.0 to 26.47.0
- Azure BOM from 1.2.26 to 1.2.28
- msal4j from 1.16.1 to 1.17.1
- json-schema-validator from 1.5.1 to 1.5.2
- checker-qual from 3.45.0 to 3.47.0
- checkstyle from 10.16.0 to 10.18.2

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-10-02 18:13:08 -05:00
Bryan Bende
d1fbf19ec7
NIFI-13817 Protect against null asset ids when synchronizing parameter context
This closes #9324

Signed-off-by: Joseph Witt <joewitt@apache.org>
2024-09-29 08:01:33 -07:00
Bryan Bende
dffa88aaae
NIFI-13803 Improved Clustered Flow Loading and Synchronization Process
- Only synchronize flow with the coordinator after getting cluster response

This closes #9317

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-09-28 22:25:23 -05:00
Matt Gilman
fb2a82c035
NIFI-13721: Migrating documentation into the NiFi UI (#9288)
* NIFI-13721: Migrating additional details to markdown.

* NIFI-13721: Rebuilding documentation to generate mark up on the client.

* NIFI-13721: Removing old documentation approach.

* NIFI-13721: Rebase main and final fixing button colors.

* NIFI-13721: Fixing rat issues.

* NIFI-13721: Removing generated comonent documentation.
- Removing links and images from additional details markdown.

* NIFI-13721: Recreating package lock following rebase.

* NIFI-13721: Updating documentation for M3.

* NIFI-13721: Fixing broken test.

* NIFI-13721: Addressing review feedback.

* NIFI-13721: Addressing review feedback.

* NIFI-13721: Ensuring the selected item is visible after routing.

* NIFI-13721: Fixing reportingTaskTypes in finding extensions for a given type.

* NIFI-13721: Updating the see also component to handle when the referenced extension is not installed.

* NIFI-13721: Fixing expansion panel header styles.
2024-09-27 11:11:55 -04:00
Bryan Bende
ee29562517
NIFI-13795 Improve NAR Manager handling of dependent NARs (#9310)
- Ensure NARs are loaded in parent-first order during restart
- Allow replacing a NAR when other NARs are dependent on it
- Improvements to catch Throwable in several places to handle NoClassDefFoundError

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-09-26 12:28:17 -05:00
Mark Payne
f39cce5dec
NIFI-13807 Fixed Stream Handling in Runtime Management Server HTTP Exchanges (#9316)
- Drained request InputStream
- Added try-with-resources for response OutputStream

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-09-26 10:57:37 -05:00
Mark Payne
e9ce40d209
NIFI-12992 Reset Validation State when Updating Process Group Execution Engine (#9187)
- Do not reset validation status for a component if it is not in a state where validation will be performed

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-09-25 16:19:46 -05:00
David Handermann
2e7a39d200
NIFI-13799 Improved Replicated Cluster Response Handling (#9312)
- Return the remote Response Stream in the Replicated Response for unknown or large content length values
- Buffered smaller responses
- Addressed code warnings in ThreadPoolRequestReplicator
2024-09-25 15:03:49 -04:00
Mark Payne
69b99390e6
NIFI-13800 Fixed Missing Registry Client Handling when Inheriting Cluster Flow (#9309)
When inheriting Registry Clients from a cluster's flow, any missing registry clients are removed at the end instead of the beginning of the synchronization logic. This allows those registry clients to still be referenced while performing synchronization. Also found that if a Processor is missing from cluster's flow but is running in local flow, on startup we get an error indicating that the processor does not belong to the associated process group so fixed that in tandem. Finally, noticed while verifying the fix that we check if the proposed flow is empty with a null check instead of using the isFlowEmpty() method - this could result in inheriting an empty flow from cluster even when a flow is loaded locally.

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-09-24 16:35:49 -05:00
Mark Payne
457d83ef84
NIFI-13796: When writing to a content claim and its size + offset exceeds the max appendable size, do not write to that content claim again
This closes #9306.

Signed-off-by: Joseph Witt <joewitt@apache.org>
2024-09-23 21:06:08 -07:00
Mark Bathori
b12d26b6ba
NIFI-13720: Component is not reloaded when the isolation key depends on service property
This closes #9236.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2024-09-23 12:00:00 +02:00
Gabor Gyimesi
426b8feaa3
NIFI-13714 Fixed RecordTransform Python Processor Partition Handling (#9253)
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-09-21 08:18:42 -05:00
Mark Payne
e9577ded29
NIFI-13777 Fixed Stateless Group Startup around Controller Services (#9291)
Fixed stateless group startup around Controller Services by changing the startup logic in two ways. First, we enable Controller Services before we create the Stateless Flow. Secondly, we do not bother enabling the ephemeral Controller Services that are created during the Stateless Flow creation/synchronization. This ensures that if Processors make use of a Controller Service during its @OnScheduled method (which is part of the Stateless Flow initialization process) that the Controller Service is Enabled. It also avoids calling @OnEnabled methods of Controller Services that will never actually be used, as Processors make use of the existing Controller Services when running within standard NiFi .

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-09-20 13:47:47 -05:00
David Handermann
563d7ea614
NIFI-13770 Removed Placeholder Prefix from Python Component Types (#9283)
This closes #9283
2024-09-20 11:26:41 -04:00
Joseph Witt
e155e6bb26
NIFI-13769 bump various deps for logback, amazon, spring, and more
ch.qos.logback 1.5.7 1.5.8
com.amazonaws 1.12.770 1.12.772
com.github.luben 1.5.6-4 1.5.6-5
commons-io 2.16.1 2.17.0
io.netty 4.1.112.Final 4.1.113.Final
org.apache.ant 1.10.14 1.10.15
org.apache.logging.log4j 2.23.1 2.24.0  (why is this here!?)
org.eclipse.jetty 12.0.12 12.0.13
org.springframework 6.1.12 6.1.13
org.xerial.snappy 1.1.10.6 1.1.10.7
software.amazon.awssdk 2.27.14 2.28.4
com.google.apis v3-rev20240730-2.0.0	 v3-rev20240903-2.0.0
com.slack.api bolt-socket-mode 1.42.0 1.42.1
com.squareup.wire wire-schema-jvm 5.0.0 5.1.0
io.projectreactor (core/test) 3.6.9 3.6.10
net.java.dev.jna (jna/jna-platform) 5.14.0 5.15.0
org.apache.groovy 4.0.22 4.0.23
org.apache.maven maven-artifact 3.9.8 3.9.9
org.eclipse.jgit 6.10.0.202406032230-r	 7.0.0.202409031743-r
org.flywaydb 10.17.2 10.18.0
org.kosuke 1.324 1.326
org.mongodb 4.11.3 4.11.4
org.neo4j.driver 5.23.0 5.24.0
org.postgresql 42.7.3 42.7.4
org.springframework.boot 3.3.3 3.3.4
org.springframework.integration 6.3.3 6.3.4
org.springframework.retry 2.0.8 2.0.9

Signed-off-by: Joseph Witt <joewitt@apache.org>
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #9280.
2024-09-20 13:46:02 +01:00
Bryan Bende
44abe39983
NIFI-13757 Improve handling when a NAR contains an extension that produces an Error (#9274)
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-09-17 21:41:45 -05:00
David Handermann
6e5a276cb2
NIFI-13718 Switched Request Replication to Web Client Service (#9234)
- Added Request Replication Header enumeration with lowercased header names for HTTP/2
2024-09-12 13:50:03 -04:00
Bryan Bende
68b74547e8
NIFI-13713 Use Asynchronous Loading for Extension UIs in Jetty Server (#9231)
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-09-09 07:47:35 -05:00
Matt Gilman
39ab3129f0
NIFI-13632: Updating content access. (#9221)
* NIFI-13632: Updating content access.
- Removing existing content viewer application.
- Introduced new front end based content viewer applicaiton.
- Introduced new bundled content viewers for hex and images.
- Deleted previous image content viewer.
- Migrated existing standard content viewer (which handles json, xml, yaml, csv, and text) to utilize new content access interface.
- Moved standard content viewer into its own NAR.
- Moved and renamed custom ui utils and content viewer utils into nifi-common.
- Added mime type to FlowFileSummary response payload to help drive the initially opened content viewer.

* NIFI-13632: Fixing rat issue.

* NIFI-13632: Fixing CI issue.

* NIFI-13632: Removing standard content viewer war from code coverage.

* NIFI-13632: Addressing review feedback.

* NIFI-13632: Fixing import.

* NIFI-13632: Further simplification of content viewer state.

* NIFI-13632: Removing unneeded code.

* NIFI-13632: Adding a min height around codemirror.

* NIFI-13632: Addressing review feedback.
- Rendering mime type in the content viewer.
- Preventing additions to the browser history as the user changes how the content is viewed.
- Appending forward slash to content viewer UIs to prevent unnecessary redirect.
- Removing Standard Content Viewer title based on review feedback.
- Fixing double scroll bar in the Standard Content Viewer on small browser heights.
- Removed unnecessary application.yml.
- Bumping spring boot version.

This closes #9221
2024-09-06 09:12:38 -04:00
David Handermann
d56f92d738
NIFI-13707 Switched Kubernetes HTTP Client from OkHttp to JDK (#9228) 2024-09-05 11:31:15 -04:00
Peter Gyori
5f0cbed5a6
NIFI-13660 ProcessContext.yieldResources() can be called from Python code
Closes #9182

Signed-off-by: Marton Szasz <szaszm@apache.org>
2024-09-04 20:26:32 +02:00
David Handermann
0510e716b4
NIFI-13706 Handled Exceptions Fetching Parameters on Startup (#9227) 2024-09-03 16:29:32 -05:00
Mike Moser
ba13c5d48a
NIFI-13664 Support Component Search on NAR Bundle ID
This closes #9197

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-08-31 10:50:48 -05:00
Bryan Bende
5845d631cf
NIFI-13694 Remove MissingComponentsCheck from VersionFlowSynchronizer (#9217)
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-08-30 07:59:08 -05:00
exceptionfactory
e540f72989
NIFI-13688 Run Validation on Parameter Providers before fetching
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #9220.
2024-08-30 13:42:47 +02:00
Bryan Bende
71face8ea7
NIFI-13659 Call migrateProperties on components changed in synchronization (#9201)
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-08-29 09:25:49 -05:00
exceptionfactory
bdaf514018
NIFI-13688 Checked Validation Status before Fetching Parameters
- Added perform validation for Controller-level Services
- Added wait on Parameter Provider validation status to allow for Controller Service enabling

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #9210.
2024-08-29 15:28:54 +02:00
Joseph Witt
a8e204aceb
NIFI-13690 Upgraded AWS SDK to 2.27.14 and other dependencies
- Upgraded Logback from 1.5.6 to 1.5.7
- Upgraded brotli4j from 1.16.0 to 1.17.0
- Upgraded AWS SDK from 1.12.767 to 1.12.770
- Upgraded AWS SDK from 2.27.1 to 2.27.14
- Upgraded H2 from 2.3.230 to 2.3.232
- Upgraded Commons CLI from 1.8.0 to 1.9.0
- Upgraded Dropwizard Metrics from 4.2.26 to 4.2.27
- Upgraded Kubernetes Client from 6.13.2 to 6.13.3
- Upgraded Reactor Netty from 1.1.21 to 1.1.22
- Upgraded Commons Compress from 1.27.0 to 1.27.1
- Upgraded Mockito from 5.12.0 to 5.13.0
- Upgraded Slack API from 1.40.3 to 1.42.0
- Upgraded Project Reactor Core from 3.6.8 to 3.6.9
- Upgraded Iceberg from 1.6.0 to 1.6.1
- Upgraded FlywayDB from 10.17.0 to 10.17.2
- Upgraded GitHub API 1.323 to 1.324
- Upgraded PostgreSQL from 42.7.3 to 42.7.4
- Upgraded Spring Integration from 6.3.2 to 6.3.3
- Upgraded Spring Retry 2.0.7 to 2.0.8
- Upgraded Spring Vault from 3.1.1 to 3.1.2

This closes #9206

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-08-28 14:17:37 -05:00
David Handermann
e9f84083eb
NIFI-13684 Set Cluster Status HTTP Code based on Connection (#9202) 2024-08-27 13:38:58 -04:00
David Handermann
6b6e8d6f78
NIFI-13665 Refactored Bootstrap and Runtime Process Handling (#9192)
- Added BootstrapProcess main class to nifi-bootstrap
- Added HTTP Management Server to nifi-runtime for status
- Added HTTP client to nifi-bootstrap for status
- Removed TCP socket listeners from nifi-bootstrap and nifi-runtime
- Removed dump and env commands
- Removed java property from bootstrap.conf
- Removed nifi.bootstrap.listen.port from bootstrap.conf
2024-08-26 14:22:04 -04:00
exceptionfactory
fa6e55f1de
NIFI-13677 Removed install command from nifi.sh
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #9194.
2024-08-23 18:01:30 +02:00
Mark Payne
da9c296820
NIFI-12991 Fixed Recursive Validation of Group Execution Engine (#9186)
Fixed issue in which we recursively validate that we can change Execution Engine of a group, but without changing the top level, inherited groups can't actually be changed; instead, replaced logic to ensure that we can change the top level, that we don't validate the constraint that a STATELESS group cannot have a STATEFUL child, and then validate that all descendant groups can be updated, but without checking the descendants' parent groups.

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-08-20 15:26:44 -05:00
exceptionfactory
7348740ecc
NIFI-13639 Replaced OkHttp with web-client-api in web-security
This closes #9157

Signed-off-by: Joseph Witt <joewitt@apache.org>
2024-08-16 14:53:55 -07:00
exceptionfactory
98f55b21b8
NIFI-13532 Fixed Flow Configuration History Sorting when Filtered
This closes #9179

Signed-off-by: Joseph Witt <joewitt@apache.org>
2024-08-16 14:41:40 -07:00
Joseph Witt
e4ec0cb20d
NIFI-13651 Moved framework-related modules under framework-bundle
- Removed references to nifi-property-utils under nifi-extension-bundles

This closes #9169

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-08-16 13:44:13 -05:00
Bryan Bende
e46c7db1ec
NIFI-13657 Fixed Reducer for Processor Load Average in Node Status
This closes #9177

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-08-14 20:28:21 -05:00
David Handermann
89c757999b
NIFI-13652 Added Range Header Handling for Content Download (#9172)
This closes #9172
2024-08-14 14:44:21 -04:00
Mark Payne
60e99184f8
NIFI-12898 Added Asset Management REST API and CLI Capabilities
- Added REST resource methods for managing assets associated with Parameter Contexts
- Added CLI commands for managing assets
- Added and updated system test for assets and Parameter Contexts
- Updated flow synchronization to consider referenced asset changes
- Adjusted authorization for cluster nodes to read Parameter Contexts

This closes #9138

Co-authored-by: Bryan Bende <bbende@apache.org>
Co-authored-by: Mark Payne <markap14@hotmail.com>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-08-13 11:42:02 -05:00
David Handermann
265ef593a6
NIFI-13653 Fixed Sensitive Dynamic Property Update Handling (#9175)
- Avoided removing Sensitive Dynamic Property Names unless the property itself is removed

This closes #9175
2024-08-13 09:34:44 -04:00
Bryan Bende
e349b7e7b4
NIFI-13649 Check Cluster Node Address Status before marking CONNECTED (#9168)
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-08-12 16:27:52 -05:00
Joseph Witt
3096c983f5
NIFI-13648 Upgraded Jetty to 12.0.12 along with other common dependencies
- Upgraded AWS SDK from 1.12.762 to 1.12.767
- Upgraded AWSK SDK from 2.26.21 to 2.27.1
- Upgraded zstd-jni from 1.5.6-3 to 1.5.6-4
- Upgraded excel-streaming-reader from 4.4.0 to 5.0.2
- Upgraded azure-sdk-bom from 1.2.25 to 1.2.26
- Upgraded json-schema-validator from 1.5.0 to 1.5.1
- Upgraded oauth2-oidc-sdk from 11.13 to 11.14
- Upgraded Fabric8 Kubernetes Client from 6.13.1 to 6.13.2
- Upgraded commons-compress from 1.26.2 to 1.27.0
- Upgraded commons-lang3 from 3.15.0 to 3.16.0
- Upgraded Apache SSHD from 2.13.1 to 2.13.2
- Upgraded Jetty from 12.0.11 to 12.0.12
- Upgraded Elasticsearch Client from 8.14.3 to 8.15.0
- Upgraded Jersey from 3.1.7 to 3.1.8
- Upgraded SLF4J from 2.0.13 to 2.0.15
- Upgraded Testcontainers from 1.20.0 to 1.20.1
- Upgraded org.tukaani xz from 1.9 to 1.10
- Upgraded snappy-java from 1.1.10.5 to 1.1.10.6
- Upgraded google-api-services-drive from v3-rev20240628-2.0.0 to v3-rev20240730-2.0.0
- Upgraded Hazelcast from 5.4.0 to 5.5.0
- Upgraded Azure Kusto from 5.1.0 to 5.1.1
- Upgraded ActiveMQ from 6.1.2 to 6.1.3
- Upgraded Clojure from 1.11.3 to 1.11.4
- Upgraded FlywayDB from 10.16.0 to 10.17.0
- Upgraded MongoDB from 4.11.2 to 4.11.3
- Upgraded Neo4j from 5.22.0 to 5.23.0

This closes #9167

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-08-12 13:03:46 -05:00
David Handermann
9cd319e3e5
NIFI-13646 Corrected Current User Replicated Response Handling (#9164)
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-08-07 16:22:39 -05:00
Matt Gilman
38292762bd
NIFI-13638: Limiting the maximum number of provenance events to return when fetching the latest events. (#9156) 2024-08-06 17:26:34 -04:00
Matt Gilman
da6c9c4791
NIFI-13017: Replace JoltTransformJSON custom UI (#9117)
* NIFI-13017:
- Remove old JoltTransformJSON custom UI.
- Update build to use new JoltTransformJSON custom UI.
- Fix revision query parameter.

* NIFI-13017:
- Excluding jolt transform json ui in workflow ci.

* NIFI-13017:
- Excluding jolt nar in workflow ci.

* NIFI-13017:
- Excluding jolt ui war from code coverage.

* NIFI-13017:
- Incorporating nifi-web-servlet-shared.
- Moving QueryStringToFragmentFilter to nifi-web-servlet-shared.

This closes #9117
2024-08-06 13:51:41 -04:00
dan-s1
f64e68b23c
NIFI-13625 Removed or Replaced System.out.println in Tests (#9150)
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-08-05 20:35:40 -05:00
David Handermann
141ca71cbc
NIFI-13622 Corrected Logout Complete Redirect for Proxies (#9143)
This closes #9143
2024-08-02 16:40:47 -04:00
David Handermann
8ad6aa6ede
NIFI-13609 Correct Component Search Configuration (#9131)
* NIFI-13609 Corrected Component Search Configuration
- Removed duplicate Controller Search Service bean definition
- Wired Controller Search Service to Controller Facade bean definition
- Added Flow Controller reference for SearchableMatcher
2024-07-31 15:04:49 -04:00
Matt Gilman
6c930626dc
NIFI-13606 Corrected Custom UI loading in reverse proxy deployments
This closes #9130

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-07-31 11:10:07 -05:00
David Szabo
2542927b53 NIFI-12741 - Remove write permission requirement for the referenced controller service when changing component property referencing a controller service through parameter
This closes #9093.

Signed-off-by: Tamas Palfy <tpalfy@apache.org>
2024-07-31 17:26:10 +02:00