Commit Graph

2190 Commits

Author SHA1 Message Date
exceptionfactory a661b035e8 NIFI-10259 Improved HTTP error handling for authentication failures
- Added Standard AuthenticationEntryPoint
- Configured AuthenticationEntryPoint for SecurityFilterChain and BearerTokenAuthenticationFilter

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6233.
2022-07-22 17:33:51 -04:00
Mark Payne d7ed66032e
NIFI-10050: When copying a process group and 1 CS references a higher-level CS, ensure that we don't drop that reference
This closes #6077

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-07-22 16:25:58 -05:00
Mark Payne bc4e3e850b
NIFI-10246: When syncing Controller Services, wait until any DISABLING Controller Service has fully disabled before attempting to synchronize them. Also performed some minor refactoring/cleanup for System Tests in order to make writing a system test for this simpler
This closes #6219

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-07-22 13:31:06 -05:00
Kevin Doran ccf3866261
NIFI-9812 Disabled Native Lib loading tests on ARM64
- Migrate tests in nifi-framework-mark-loading-utils to JUnit5
- Annotate tests that use x86_64 native binaries to be conditional on x86_64 os.arch

This closes #6215

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-07-19 11:38:26 -05:00
Paul Grey 148834e71b
NIFI-10247 Corrected REST API documentation for empty-all-connection-requests
This closes #6220

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-07-19 07:51:32 -05:00
exceptionfactory 047b3611bf NIFI-10233 Corrected Anonymous Authentication for HTTP Access
- Enabled Standard Anonymous Authentication Filter when unencrypted HTTP is enabled

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6209.
2022-07-14 23:27:54 -04:00
exceptionfactory a9b5bebb15 NIFI-10216 Refactored NiFi Web API Security Configuration
- Removed extension of deprecated WebSecurityConfigurerAdapter
- Moved Filter bean configuration associated configuration classes
- Set default Spring Security log level to INFO
- Adjusted CSRF Token Repository to leverage simplified RequestUriBuilder for retrieving allowed context paths

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6195
2022-07-13 19:11:03 -04:00
Mark Payne 4c0555a476
NIFI-10203: Ensure that when a FlowFile is transferred and is not retried that we remove the retryCount.<uuid> attribute
This closes #6201
Signed-off-by: Paul Grey <greyp@apache.org>
2022-07-12 19:01:41 -04:00
Mark Payne 996d8faaf4
NIFI-10203: Fixed bug in which same FlowFile Builder was used repeatedly for multiple FlowFiles; this caused mingling of their attributes when a FlowFile is routed to a relationship that has more than 1 destination (i.e., many connections with the same Relationship)
This closes #6187
Signed-off-by: Paul Grey <greyp@apache.org>
2022-07-12 09:52:49 -04:00
Otto Fowler cf4eed0224 NIFI-10207: ensure allowableValues matches SUPPORTED_STATE in ProcessorRunStatusEntity
This closes #6188.

Signed-off-by: Tamas Palfy <tpalfy@apache.org>
2022-07-11 21:34:27 +02:00
mr1716 291bd40013
NIFI-10195 Upgraded Testcontainers to 1.17.3
This closes #6179

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-07-11 10:37:39 -05:00
Ferenc Erdei d78c667c19
NIFI-10166 Improved MiNiFi bootstrap test coverage
This closes #6160

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-07-01 14:45:31 -05:00
exceptionfactory 1f18345929
NIFI-10163 Corrected StandardProcessSessionTest after NIFI-10167 2022-06-30 19:10:57 -05:00
exceptionfactory 943197c18b NIFI-10163
- Corrected StandardProcessSession.exportTo() byte counting
- Removed bytesWritten incrementing from exportTo methods

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6153.
2022-06-30 18:42:44 -04:00
Kevin Doran 9cf5c170b1
NIFI-9810 Deprecated RocksDB repository and moved to NAR
- Bump version to 6.29.5 for arm64 compatibility
- Extract RocksDBFlowFileRepository and supporting code to its own module
- Mark RocksDBFlowFileRepository as deprecated, both in code and documentation
- Log deprecation warning at startup if RocksDBFlowFileRepository is used
- Move native RocksDB info logs to NiFi debug level logs

This closes #6155

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-30 14:23:10 -05:00
Mark Payne 5e83bda9e9
NIFI-10167: Added advanced timing metrics for processors, such as CPU Usage, time reading/writing content repo, process session commit time, etc. Exposed via nifi.sh diagnostics and made configurable via nifi.properties
This closes #6156

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-30 10:50:24 -05:00
timeabarna d5b626f0e4
NIFI-10108 Processor scheduling via parameter (#6115)
- NIFI-10108 Processor scheduling via parameter
- Refactoring component referencing check in AbstractComponentNode and StandardParameterReferenceManager classes.
2022-06-30 10:11:13 -04:00
exceptionfactory 7a47c8cfbd
NIFI-10162 Reorganized InvokeHTTP Properties
- Corrected handling of documentation for allowable values

NIFI-10162 Updated Multipart Form-Data Name description

This closes #6163.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-06-29 16:52:19 -04:00
Emilio Setiadarma a098653d07
NIFI-10165 Increased NiFi bootstrap heap size to 48 MB
- Increased bootstrap heap size from 24 MB to 48 to avoid heap constraints when using HTTP notifications

This closes #6161

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-28 14:56:55 -05:00
exceptionfactory 0de83292de NIFI-9849 Refactored SAML Support with Spring Security 5
- Updated SAML Authentication Configuration with Spring Security SAML 2 components
- Updated Administration Guide with REST Resources
- Replaced SAMLAccessResource methods with applicable Spring Security Filters
- Removed IDP Credential Service and supporting components
- Removed message.logging.enabled, metadata.signing.enabled, and signature.digest.algorithm properties
- Added Access Token Expiration resource method
- Removed Saml2AccessResource and replaced with Access Token Expiration to avoid unnecessary conflicts with SAML login consumer
- Corrected Resource URI handling to support proxy server access

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6149.
2022-06-28 13:57:35 -04:00
Csaba Bejan 1465c2c629
NIFI-9908 C2 refactor and test coverage improvements
This closes #6149

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-28 12:02:31 -05:00
Nissim Shiman db11961026
NIFI-9440 Allow Controller Services to have configurable Bulletins
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6035
2022-06-28 10:44:45 -04:00
exceptionfactory c1c1d0da82
NIFI-10153 Refactored DocGeneratorTest and removed binary test files
This closes #6146

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-06-27 22:05:53 -04:00
Mark Payne 1442dcef23
NIFI-10111: Added option to unpack NARs into an Uber Jar and updated stateless nifi to always use this.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6117
2022-06-24 13:12:14 -04:00
Joe Gresock 3dda694fd6
NIFI-10139: Adding fields to RemoteProcessGroupStatus (#6137) 2022-06-22 12:24:23 -04:00
Mark Payne ac8c1b0326
NIFI-10148: Fixed bug in SwappablePriorityQueue, in which the Active Queue Size could be decremented before the Unacknowledged Queue Size is incremented when FlowFiles are polled. This can result in the SwappablePrioriotyQueue incorrectly returning a value of true from the isEmpty() method. Additionally, updated the NiFiSystemIT so that if waiting for a particular queue size we periodically log size of all queues, which aids in debugging, and added necessary methods to FlowClient to make that happen.
This closes #6143

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-21 15:55:47 -05:00
Bryan Bende 1de9c72779
NIFI-10140 Handle deleted parameters in StandardVersionedComponentSynchronizer
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes #6138.
2022-06-21 10:43:38 -04:00
Joe Gresock 5e01d3de06
NIFI-10117: Allowing remote group ports to be synchronized from Stand… (#6128)
NIFI-10117: Allowing remote group ports to be synchronized from StandardVersionedComponentSynchronizer
2022-06-17 14:28:14 -04:00
Matthew Burgess d89f98f5b4
NIFI-9666 Implemented MiNiFi C2 client and refactored bootstrap
- Implemented MiNiFi C2 client using OkHttp
- Refactored MiNiFi bootstrap command handling and socket communication
- Added C2 Client Service to nifi-framework-core

This closes #6075

Co-authored-by: Matthew Burgess <mattyb149@apache.org>
Co-authored-by: Csaba Bejan <bejan.csaba@gmail.com>
Co-authored-by: Ferenc Erdei <ferdei@cloudera.com>

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-17 09:23:52 -05:00
exceptionfactory bd45eb4995 NIFI-10100 Upgraded Jersey to 2.35
- Replaced individual version references with Jersey BOM dependency

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6106.
2022-06-16 23:09:22 -04:00
Emilio Setiadarma 2161d0fe9c
NIFI-10104 Refactored usage of PropertyEncryptor
This closes #6119

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-16 16:38:34 -05:00
Mark Bean 0f5d8517b5
NIFI-5378 Prevent duplicate keys with different values in nifi.properties
This closes #6061

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-14 11:06:35 -05:00
exceptionfactory f91d877043
NIFI-10114 This closes #6122. Removed Single Refresh from ShellUserGroupProvider
Signed-off-by: Joe Witt <joewitt@apache.org>
2022-06-13 14:38:19 -07:00
Nandor Soma Abonyi f99298f98e
NIFI-7535 add detailed error message when .nar file is not readable
This closes #6056
Signed-off-by: Bence Simon <bsimon@apache.org>
2022-06-10 09:40:27 +02:00
exceptionfactory c51f70b6e6
NIFI-10096 Corrected nested inherited Parameter Context loading
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes #6114.
2022-06-09 21:10:44 -04:00
exceptionfactory e19efa6658
NIFI-10095 Use Allowable Value Display Name for Default Value when found
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6103.
2022-06-07 14:36:47 +01:00
exceptionfactory 2455ba2c45
NIFI-10094 Removed unused test HttpServer classes
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6102.
2022-06-07 14:33:57 +01:00
exceptionfactory e618e85da7
NIFI-10093 Set timeout to 5 seconds on SocketProtocolListenerTest (#6101) 2022-06-06 16:23:17 -04:00
Matthew Burgess 5591bec44b
NIFI-9742: Use dependent property Display Names in documentation
This closes #5820

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-06 13:43:02 -05:00
exceptionfactory 1342ea7025 NIFI-10079 Upgraded ZooKeeper from 3.5.9 to 3.8.0
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6096.
2022-06-03 19:05:21 -04:00
exceptionfactory 4b655ecb68
NIFI-9804 Added HTTP/2 support to Application Server
- Added nifi.web.https.application.protocols property
- Set default protocol to HTTP/1.1 and provided documentation for enabling HTTP/2
- Changed StandardALPNProcessor handshakeFailed log to debug

Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes #6093.
2022-06-03 11:54:21 -04:00
exceptionfactory 5a2992c243
NIFI-9960 Add documentation for Sensitive Dynamic Properties (#6089)
* NIFI-9960 Added documentation for Sensitive Dynamic Properties

- Updated User Guide with Add Property details
- Updated Developer Guide with annotation configuration details
- Updated Document Writer to indicate Supports Sensitive Dynamic Properties status

* NIFI-9960 Adjusted User Guide wording based on feedback
2022-06-02 08:12:22 -04:00
Joe Gresock 48de70a568
NIFI-10071: Adding support for HashiCorp Vault K/V version 2 Secrets Engine (#6087) 2022-06-01 15:09:01 -04:00
Joe Gresock 314232ca6c
NIFI-10072: Adding LoadBalanceStatus to ConnectionStatus (#6086) 2022-05-31 16:27:23 -04:00
Bryan Bende 98ec2563e9
NIFI-10059 Remove incorrect removal of controller service from cache (#6082) 2022-05-26 17:13:28 -04:00
Mark Payne 35d8bb5ca9
NIFI-10051: Clear Controller Services' state in a background thread instead of the web thread.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6078
2022-05-26 10:31:55 -04:00
exceptionfactory 83316736f8
NIFI-9959 Add UI Support for Sensitive Dynamic Properties (#6073)
* NIFI-9959 Added UI Support for Sensitive Dynamic Properties

- Added SupportsSensitiveDynamicProperties to DBCPConnectionPool and ScriptedReportingTask

* NIFI-9959 Added sensitive parameter argument for Controller Service descriptors

* NIFI-9959 Adjusted sensitive property descriptor handling to support changing status

* NIFI-9959 Added info icon for Sensitive Value field

* NIFI-9959 Corrected handling of descriptor for existing dynamic properties

* NIFI-9959 Cleaning up dialog markup.

Co-authored-by: Matt Gilman <matt.c.gilman@gmail.com>

This closes #6073
2022-05-25 21:36:31 -04:00
Mark Payne d664708dd3
NIFI-10049: When unscheduling reporting task, increment its concurrent task count until we've finished all shutdown logic and then decrement it, in much the same way that we do for processors
This closes #6076

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-05-24 17:54:17 -05:00
exceptionfactory 99be62be67
NIFI-9958 Add Framework Support for Sensitive Dynamic Properties (#6057)
* NIFI-9958 Implemented Sensitive Dynamic Properties

- Added SupportsSensitiveDynamicProperties annotation for components
- Added optional sensitive query parameter to Property Descriptor REST Resource methods
- Added system tests for components supporting sensitive dynamic properties
- Updated REST Resources to support Sensitive Dynamic Property Names
- Updated Documentation Writer to indicate component support for Sensitive Dynamic Properties
- Updated InvokeHTTP to support Sensitive Dynamic Properties
- Updated Auditor components to handle masking Sensitive Dynamic Properties

* Refactored Property Descriptor REST method handling

- Corrected AbstractDocumentationWriter evaluation of support for sensitive dynamic properties
- Refactored Controller Service Dynamic Properties system tests to new class

* Updated AbstractComponentNode.getProperties() to get canonical descriptor
2022-05-23 17:20:54 -04:00
Mark Payne 38b51b0dde
NIFI-10037: When system test fails to clean up flow, destroy the entire environment so that the next test starts in a healthy state. Name troubleshooting directories with the name of the test class to avoid ambiguity. Also added a log statement so that we know which test is running when looking at the log output from the tests themselves. Finally, found an issue in AbstractComponentNode in which we iterate over the elements in a Map and call setProperty, which can update the underlying Map - updated to first create a copy of the HashMap. Updated that in this Jira because I suspect it is causing one of the tests failures that I've been investigating.
This closes #6059

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-05-19 13:14:30 -05:00