Commit Graph

1995 Commits

Author SHA1 Message Date
Joe Witt 7fdc07cccd
NIFI-9344-RC3 prepare release nifi-1.15.0-RC3 2021-11-03 08:53:32 -07:00
Mark Payne 09feed9df6
NIFI-9352: Consider all property values in component references
- Ensure that we consider all property values to determine if a component references a variable, not just 'supported' (i.e., well-known/non-user-defined) properties

This closes #5499

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-10-30 11:16:57 -05:00
exceptionfactory 9865ea2bfb NIFI-9322 Refactored OIDC and SAML Access Resources
- Removed parent AccessResource from OIDCAccessResource and SAMLAccessResource to avoid unexpected inherited methods
- Moved Token Expiration validation from AccessResource to StandardBearerTokenProvider

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5489.
2021-10-28 12:12:57 -04:00
Joe Gresock 28cd5d1300
NIFI-9345: Resolving NPE in ProcessorEntityMerger (#5491)
NIFI-9345: Resolving NPE in ProcessorEntityMerger, Streamlining forEach calls
2021-10-28 12:06:56 -04:00
Mark Payne 3114bdb701
NIFI-9311: When determining property values, be sure to fetch the property descriptor from the component itself, rather than using the PropertyDescriptor in the Map. This allows us to ensure that if the definition of the PropertyDescriptor changes, the most up-to-date definition is picked up
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5488
2021-10-27 17:08:06 -04:00
Lehel Boér 059f14fd62
NIFI-8385: Add FlowFiles from logging to bulletins (#4952) 2021-10-27 16:06:30 -04:00
Matthew Burgess 104078868e
NIFI-7012: Refactored OnConfigurationRestored to support sensitive property validation (#5415) 2021-10-27 14:40:17 -04:00
omkadmi 8506a6012f
NIFI-9337: added missing enum value SINGLE_BATCH_PER_NODE of FlowFileConcurrencyType (#5484) 2021-10-26 11:25:52 -04:00
exceptionfactory 7b2f364cf5
NIFI-9331 Corrected ThreadUtils using Objects.equals() for LockedStackFrame
- Removed unused arguments from ThreadUtils.createStackTrace()

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5480.
2021-10-26 09:56:28 +02:00
markap14 5766d33fce
NIFI-9309: Include a uiOnly flag when requesting flow for a given pro… (#5462)
NIFI-9309: Include a uiOnly flag when requesting flow for a given process group. In that case, do not include the property descriptors, property values, etc. for Processors. When fetching Variable Registry, improved logic to cache the VariableImpact for each property value instead of parsing/recreating it every time. When fetching bulletins for a component, avoid filtering through all components' bulletins and instead only look at bulletins that might pertain to the appropriate component
2021-10-23 19:42:31 -04:00
Matt Gilman 1bec905890 NIFI-9288:
- Allowing the user to submit a verification request for Processors, Controller Services, and Reporting Tasks.
- Tracking progress of verification requests.
- Showing the verification results.

NIFI-9288:
- Fixing class name which prevented styles from being applied.

NIFI-9288:
- Ensuring that previously entered referenced attribute values take precedence.

NIFI-9288:
- Positioning the property listing and verification results based on percentages instead of fixed values.
- Removing the additional dialog height.

NIFI-9288:
- Allowing attribute value entry to be skipped when appropriate.

NIFI-9288:
- Working around an issue caused by css minification.

NIFI-9288:
- Adding some padding to the verifying progress dialog.

This closes #5461

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2021-10-21 10:06:58 -04:00
Matt Gilman 51c6910510 NIFI-9237:
- Similar to NIFI-9215, converting integer identifiers to strings to ensure the items are successfully retrieved when attempting to apply a tooltip.

This closes #5454

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2021-10-20 16:47:21 -04:00
markap14 90ae271692
NIFI-9310: Addressed issues found while testing the component verification features. Added a supportsVerification flag to the ConfigAnalysis DTO (#5469)
This closes #5469
2021-10-20 11:07:17 -04:00
Mark Payne be2b80d909
NIFI-9292: Added getInputPortNames() and getOutputPortNames() to StatelessDataflowDefinition and getReadOnlyExtensionsDirectories() to StatelessEngineConfiguration
- Addressed issue identified in system test where a flow can fail due to FailurePortEncounteredException but then purge the flow in the background after the call to trigger completes

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5450.
2021-10-18 12:22:12 -04:00
Mark Payne 151a9361fa
NIFI-9288: Updated data model to use for cnofig verification requests so that only properties are necessary not the full component/config dtos. Also added endpoint necessary for determining which attributes are referenced by a component's properties and removed the referencedAttributes field from the compnoents themselves, since there's now a new endpoint for it. Also fixed a bug that was encountered where the VerifyConfigRequestDTO's complete flag was incorrect in case of failures
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5452.
2021-10-18 11:42:47 -04:00
Joe Gresock 1b69c1f9f1
NIFI-7588: Handling dynamic properties on termination (#5261)
- Update to StandardProcessorNodeIT
2021-10-13 12:14:01 -04:00
Matt Gilman 6d3a7539eb NIFI-9215:
- During mouse over events the items in the Controller Service Types table could not be looked up because the identifier of the item was an integer value and the identifier was a string value. Addressing the issue by always using a string.

This closes #5439

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2021-10-12 15:35:26 -04:00
Lehel Boér f7df326521 NIFI-9183: Add a command-line option to save status history
Fixed typo error.

Fixed error.

Fixed review items.

Fix contrib-check. Added missing test file to excludes.

Fix review items.

Fix review items: error handling, input validation, added more unit tests.

Improved status history repository creation in HeadlessNiFiServer.
2021-10-12 16:35:20 +02:00
Mark Payne 231dd57e11
NIFI-9289: On startup, when enabling a Controller Service & its dependencies, do not wait for the dependencies to fully enable. Doing so can take 30 seconds per each Controller Service (and per each reference). Due to some previous refactoring, this waiting period is no longer necessary, as the referencing service can now be enabled and will asynchronously complete the enabling once it becomes valid (due to the referenced service becoming enabled).
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5449.
2021-10-12 07:32:42 -04:00
Matt Gilman 650da75f2d NIFI-9258:
- Populating the empty state when a parameter is not referenced by any component.
- Tweaking margins so the references are slightly more compact.

This closes #5442

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2021-10-11 16:15:21 -04:00
Matt Gilman ebcb1cc0d5 NIFI-9214:
- Migrating away from forceFitColumns which is deprecated. Updating to use the same strategy in the new configuration which will prevent the warning logs to the console.

This closes #5428

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2021-10-11 09:50:58 -04:00
Joe Gresock f3136f07eb
NIFI-9221: Add AWS SecretsManager Sensitive Props Provider
This closes #5410

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-10-08 11:44:43 -05:00
exceptionfactory 70432507df
NIFI-6617 Refactored Encrypted Repository configuration
- Updated documentation with new properties
- Refactored cipher operations to common RepositoryEncryptor classes
- Abstracted record metadata serialization for better compatibility

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5407.
2021-10-08 11:57:10 -04:00
exceptionfactory 8d0eec2d62
NIFI-9266 Added Azure Key Vault Secret SPP
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5435.
2021-10-08 10:01:09 +02:00
exceptionfactory 4bcd03024a
NIFI-9283 Excluded Log4j 1.2 and upgraded Log4j2 to 2.14.1
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5440.
2021-10-06 11:17:36 -04:00
exceptionfactory fe42326335
NIFI-9284 Added QuestDB qlog.conf to test resources
- Set default logging level to ERROR to avoid excessive INFO messages

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5441.
2021-10-06 09:40:27 +02:00
Mark Payne 607480c1a0
NIFI-9239: Updated Consume/Publish Kafka processors to support Exactly Once Semantics. Added ExecuteStateless processor/bundle
- A few minor updates to Stateless in order to surface some concepts from the stateless engine up to the caller, such as bulletins, counters, etc.

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5412.
2021-10-04 15:32:21 -04:00
Mark Payne a675023b71
NIFI-9272: When determining if Property dependency is satisfied, consider property default values also
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5432.
2021-10-04 14:22:39 -04:00
timeabarna 72660af479
NIFI-9229 Flow upgrade not possible if a Output Port changes to a funnel (#5402)
* NIFI-9229 Flow upgrade not possible if a Output Port changes to a funnel
* NIFI-9229 Addressing review comments modified log message and added comments
2021-10-04 09:14:36 +02:00
exceptionfactory 00b2358c27
NIFI-9252 Removed unused nifi.sensitive.props.provider Property
- Removed nifi.minifi.sensitive.props.provider Property from MiNiFi
- Removed property from example NiFi properties files
- Removed provider from MiNiFi SensitivePropsSchema
- Removed BC provider value from MiNiFi test cases

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5422.
2021-10-01 22:20:13 -04:00
exceptionfactory e16a6c2b89 NIFI-9241 Refactored CSRF mitigation using random Request-Token
- Replaced use of Authorization header with custom Request-Token header for CSRF mitigation
- Added Request-Token cookie for CSRF mitigation
- Replaced session storage of JWT with expiration in seconds
- Removed and disabled CORS configuration
- Disabled HTTP OPTIONS method
- Refactored HTTP Proxy URI construction using RequestUriBuilder

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5417.
2021-09-30 20:36:15 -04:00
Matt Gilman be57a210ff
NIFI-8491: Adding support for configuring parameter context inheritance (#5371)
* NIFI-8491:
- Adding support for configuring parameter context inheritance.

* NIFI-8491:
- Allowing changes to the parameter context inheritance to drive Apply disabled state.

* NIFI-8491: Updating StandardParameterContext#isAuthorized check

* NIFI-8491:
- Showing selected inherited parameter contexts in ready only form when appropriate.
- Allowing available parameter contexts to be inherited by double clicking.
- Removing support for rendering unauthorized inherited parameter contexts as they can no longer be opened.

* NIFI-8491: Adding inherited param context verification earlier

* NIFI-8491:
- Addressing CI failures by rolling back to some order JS language spec to allow yui-compress to minify and compress.

* NIFI-8491:
- Ensuring selected context sort order is honored.
- Ensuring the Apply button is correctly enabled.
- Showing Pending Apply message when selected Parameter Context changes.
- Ensuring the Parameter's tab is selected now that there is a third tab.

* Updates to inherited param context verification

* Improving validation between parameters/inherited parameters

* NIFI-8491:
- Ensuring the available parameter contexts are loaded whether the edit dialog is opened from the listing or outside of the listing.

* NIFI-8491:
- Fixing conditions we check if the parameter context listing is currently open.

* NIFI-8491:
- Waiting for the parameter contexts to load prior to rendering the parameter context inheritance tab and showing the dialog.

* NIFI-8491:
- Fixing pending apply message clipping.
- Hiding pending apply message after clicking Apply.

Co-authored-by: Joe Gresock <jgresock@gmail.com>

This closes #5371
2021-09-29 17:04:35 -04:00
Mark Payne efc1cb012f
NIFI-8727: This closes #5418. Addressed bug in which ProcessSession doesn't properly decrement claimant count when a FlowFile is cloned and then the clone written to. Added automated tests to ensure that we are properly handling cases where a FlowFile is clone and then the contents modified
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-09-29 11:02:41 -07:00
Matthew Burgess fa2be2ee63
NIFI-9009 NPE defensive code for referenced attributes
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
2021-09-27 22:11:29 -04:00
exceptionfactory 84dbf915a9 NIFI-9060 Refactored HTTP Cookie Path Handling
- Implemented ApplicationCookieService for adding and retrieving HTTP Cookies
- Added getCookieResourceUri() leveraging allowed proxy headers to support optional Cookie Paths
- Refactored Access Resources to use ApplicationCookieService for processing
- Changed __Host- prefix to __Secure- prefix for Bearer Token cookie to support Cookie Path processing
- Removed unnecessary jetty-http dependency from nifi-web-api
- Corrected NiFi path references in JavaScript to support prefixed paths

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5329.
2021-09-23 18:03:48 -04:00
Mark Payne e29c8af708
NIFI-9182: When calling ProcessSession.append(), do not allow calls from processor to flush the underlying BufferedOutputStream. Instead, wrap in a NonFlushableOutputStream and only flush when session commit is called
This closes #5359

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-09-22 18:54:32 -05:00
Mark Payne baf29e59f1
NIFI-9009: Created VerifiableProcessor, VerifiableControllerService, VerifiableReportingTask components; implemented backend work to call the methods. Added REST APIs and created/updated data models for component configuration verification
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5288
2021-09-22 19:00:23 -04:00
exceptionfactory b4e0416b5e
NIFI-9228 Refactored tests using TemporaryKeyStoreBuilder
- Added TemporaryKeyStoreBuilder with File.deleteOnExit() for KeyStore and TrustStore files
- Removed JKS files from nifi-security-utils tests
- Refactored usage of KeyStoreUtils.createKeyStoreAndGetX509Certificate() to TemporaryKeyStoreBuilder
- Removed unnecesary hadoop-minikdc test dependency in security-utils
- Replaced Mini KDC Hex utility with Bouncy Castle Hex utility in unit tests

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5406
2021-09-21 16:07:13 -04:00
exceptionfactory e78674ec59 NIFI-9184 Refactored shared methods in Sensitive Property Providers
- Added EncodedSensitivePropertiesProvider with Base64 encoding methods
- Added ClientBasedEncodedSensitivePropertiesProvider with validate method
- Abstracted client configuration to ClientProvider interface and implementations
- Added unit tests for AWS and Azure Property Providers

NIFI-9184 Adjusted abstract provider class names and updated documentation

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5363.
2021-09-21 15:47:19 -04:00
exceptionfactory ba775d28de
NIFI-9228 Refactored tests to use generated KeyStores
- Refactored multiple tests using KeyStoreUtils
- Removed static KeyStore and TrustStore files

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5401
2021-09-21 11:06:19 -04:00
greyp9 4af3fac07a
NIFI-9217 - avoid deadlock on cluster operation (#5390) 2021-09-15 15:40:33 -04:00
markap14 73f88b3239
NIFI-9061: Eliminated the nifi.cluster.node.protocol.threads property in favor of nifi.cluster.node.protocol.max.threads property so that we can properly scale out the number of threads used for HTTP request replication. Implementing a caching mechanism for creating the DateTimeFormatter used by TimeAdapter in order to improve performance when parsing timestamps in web requests. Implementing caching logic for caching the number of characters that can rendered without needing an ellipsis for some components in the UI (#5316)
This closes #5316
2021-09-13 14:36:35 -04:00
Mohammed Nadeem 5e2ec9b454
NIFI-9038: Fix fingerprinting group access control policies for Remote
Port

This closes #5300

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-09-11 16:45:21 -05:00
Mohammed Nadeem acb25c1130
NIFI-9170 Upgrade commons-beanutils to 1.9.4 to mitigate CVE-2019-10086
NIFI-9170 Add two more 1.9.4 references to close out the few things identified by the Maven dependency plugin.

This closes #5351

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2021-09-07 16:24:57 -04:00
exceptionfactory f5a421ccb1
NIFI-9067 Upgraded JAXB Runtime from 2.3.0 to 2.3.5
- Replaced old com.sun.xml.bind:jaxb-impl and jaxb-core with current org.glassfish.jaxb:jaxb-runtime
- Replaced old javax.xml.bind:jaxb-api with current jakarta.xml.bind-api
- Removed unnecessary dependency references to javax.activation-api

This closes #5320

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2021-09-07 13:27:53 -04:00
Chris Sampson 05adcbcc61
NIFI-9181 Prevent unwanted test artifacts in Windows builds
This closes #5355

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-09-06 17:03:46 -05:00
Emilio Setiadarma 2ab23efb74
NIFI-6616 Added GCP Sensitive Property Provider
This closes #5242

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-30 19:15:34 -05:00
Hsin-Ying Lee 59f0b6bac3
NIFI-8739 Penalized flowfiles should be able to be polled from the queue in some cases (#5189) 2021-08-30 11:26:02 -04:00
Joe Gresock 4a3e81531b
NIFI-8490: Adding inherited parameter contexts (#5072)
- Allowing inherited param contexts on creation, updating PC authorization
2021-08-30 10:44:31 -04:00
markap14 d90ef06752
NIFI-9017: Update Load Balanced Connection logic so that if a node connects to the cluster with a different load balancing hostname/port, it starts sending to the new endpoint instead of failing to send to the old endpoint (#5287)
Self-merging based on +1 feedback from multiple active community members who have reviewed & tested code
2021-08-27 21:37:06 -04:00