2124 Commits

Author SHA1 Message Date
exceptionfactory
784ee680d5
NIFI-9495 Corrected BulletinMergerTest using static timestamps
- Replaced JUnit 4 and TestNG references with JUnit 5
- Added test method for bulletin sorting

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5607
2021-12-16 18:44:07 -05:00
Shane Ardell
293919f881
NIFI-9460: Update Lodash to 4.17.21 (#5605)
* NIFI-9460: update lodash transitive dep in registry

* NIFI-9460: use lodash instead of lodash-core

Merged #5605 into main.
2021-12-16 14:28:02 -08:00
M Tien
3d5f357de8
[NIFI-9423-NIFI-9429]: Show icon and tooltip for Parameters with leading and/or trailing whitespace (#5569)
* NIFI-9423 - Show icon and tooltip for Parameter values that have leading and/or trailing whitespaces

NIFI-9429 - Parameters should allow blank values that are non-null (only whitespace)

* - Update areas to clean up tooltips in parameter values
- Show whitespaces and ellipsis in parameter and property values and tooltips
- Update serializeValue to accommodate for blank values

* - Address review findings

* - Remove commented out code

* - Add multiline check for ellipsis

* NIFI-9459 - Empty string checked will disable Edit Parameter value field on dialog open

* - Add multi-line style to parameter and property table

* - Safely insert title attribute content

* - Fix Edit Parameter bug that clears textarea for sensitive and empty string values on dialog open

This closes #5569
2021-12-13 16:01:42 -05:00
Joe Gresock
429087c11d
NIFI-9386: Adding status task schedule to Stateless engine config
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5532.
2021-12-13 12:08:54 +01:00
Mark Payne
97198e35a0
NIFI-9382: This closes #5584. Added system test that replicates issue in which a closed shared classloader causes issues when used again
NIFI-9382: Fixed issue with SharedInstanceClassLoader where the classloader may get closed but then get used again. When the SharedInstanceClassLoader is closed, we will now ensure that we don't use anymore and instead create a new one.

Signed-off-by: Joe Witt <joewitt@apache.org>
2021-12-09 12:41:57 -07:00
Yiming Li
c1bb0c0c34
NIFI-9417: Update BulletinMergerTest.java
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5572
2021-12-09 13:27:35 -05:00
Lehel Boér
bf288f3ba1
NIFI-9454: Moved file renaming after closing the OutputStream in StandardFlowConfigurationDAO (#5581) 2021-12-08 13:38:45 -05:00
Mark Payne
2a3f7dafd6
NIFI-9442: When deleting a connection, ensure that when the source of the connection is a funnel that its upstream components are checked (recursively)
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5568.
2021-12-05 06:39:52 -05:00
Mark Payne
f95044ff87
NIFI-9433: When a Connection is unregistered from the NioAsyncLoadBalanceClient, make sure that we only cancel its active transaction if the transaction belongs to the appropriate connection. Also ensure that when we do cancel a transaction / call its failure callback, we purge the collection of any FlowFiles that have been sent in that transaction. This ensures that we cannot later attempt to failure the transaction again, decrementing the count of FlowFiles for the connection more than we should.
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5564.
2021-12-03 10:31:31 -05:00
Mark Payne
90b39b593a
NIFI-9069 Changed framework dataflow serialization to support JSON
- Changed framework so that it serializes the dataflow into a VersionedDataflow using JSON as well as XML, and prefers the JSON representation on load, if it's available. This also results in the need for the cluster protocol to exchange its representation of the dataflow to using JSON. Rather than re-implementing all of the complex logic of Flow Fingerprinting, updated to just inherit the cluster's flow.
- Moved logic to synchronize Process Group with Versioned Process Group into a new ProcessGroupSynchronizer class instead of having all of the logic within StandardProcessGroup
- Reworked versioned components to use an instance id.
- Renamed StandardFlowSynchronizer to XmlFlowSynchronizer; introduced new StandardFlowSynchronizer that delegates to the appropriate (Xml or Versioned)FlowSynchronzer
- Updated to allow import of VersionedProcessGroup even if not all bundles are available - will now use ghost components
- Introduced a VersionedDataflow object to hold controller-level services, reporting tasks, parameter contexts, templates, etc.
- Allow mutable requests to be made while nodes are disconnected. Also fixed issue in AbstractPolicyBasedAuthorizer that caused ClassNotFoundException / NoClassDefFoundError if the authorizations were changed and then a node attempts to rejoin the cluster. The Authorizer was attempting to use XmlUtils, which is in nifi-security-utils and so so by madking nifi-security-utils a provided dependency of nifi-framework-api, but this doesn't work, because nifi-framework-api is loaded by a higher-level classloader, so the classloader that loads AbstractPolicyBasedAuthorizer will never have the appropriate classloader to provide nifi-security-utils. Addressed this by copying the code for creating a safe document builder from XmlUtils to AbstractPolicyBasedAuthorizer.
- Fixed bug that occurred when importing a Process Group that has 2 parameter contexts, one inheriting from another, where neither is pre-defined in the existing flow
- Fixed bug that was encountered when Updating a Versioned Process Group where one version had a disabled processor and the other had the processor running.
- Increased system-tests workflow timeout to 120 minutes
- Added additional exception handling to system tests

This closes #5514

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-12-02 15:10:38 -06:00
Pierre Villard
fede6b9354
NIFI-9432 - fix typo in diagnostic output (#5562) 2021-12-02 15:25:23 -05:00
M Tien
2b415de912
NIFI-9336 - Show icon for property values with leading or trailing whitespace (#5559)
* NIFI-9336 - Show icon in processor and controller services configurations when property values contain leading or trailing whitespace

* - Address PR feedback

* - Fix a bug to clean up tooltips to prevent a DOM leak

This closes #5559
2021-12-02 13:31:19 -05:00
exceptionfactory
6a6b755aaa
NIFI-9394 Removed RequestLogger and TimerFilter
- Removed logger references from logback.xml

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5543.
2021-11-29 12:55:39 -05:00
Duan
8171ecb2c1
NIFI-9381 Updated CompositeUserGroupProviderTestBase with LinkedHashMap
- Replaced HashMap with LinkedHashMap to avoid potential non-deterministic results for user group properties

This closes #5524

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-11-29 10:45:59 -06:00
Joe Gresock
a5581562bf
NIFI-9416: Fixing NPE when updating param context without inheritedParameterContexts (#5553) 2021-11-29 11:29:33 -05:00
Joe Gresock
dfd8adfd8e
NIFI-9411: Adding missing 'setSubtitle' call for Controller Services modal
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5551.
2021-11-24 18:01:48 +01:00
exceptionfactory
c1e54276b8 NIFI-9355 Upgraded Apache Curator from 4.2.0 to 5.2.0
- Removed unused Curator dependency declaration from MiNiFi

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5503.
2021-11-22 18:24:39 -05:00
exceptionfactory
0e09750a4d NIFI-9393 Set Port Scheduled State for Flow Definitions
- Set Scheduled State for Versioned Port and Versioned Remote Port when mapping Flow Definition
- Updated StandardProcessGroup to set disable Port based on Scheduled State of DISABLED
- Updated StandardProcessGroup to set Remote Port transmitting based on Scheduled State of ENABLED

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5534.
2021-11-22 18:16:42 -05:00
markap14
839fbf7d19
NIFI-9382: Created a new ClassloaderIsolationKey mechanism by which H… (#5533)
* NIFI-9382: Created a new ClassloaderIsolationKey mechanism by which Hadoop related processors (and potentially others) can indicate that they need full classloaders to be cloned but can share with other instances in certain circumstances
- Added system tests

* NIFI-9382: Renamed interface based on review feedback

* NIFI-9382: Removed ReentrantKerberosUser.
2021-11-22 14:55:19 -05:00
Timea Barna
848a22eb29
Revert "NIFI-9385 Add Flow Metrics producer for Cloudera Manager"
This reverts commit 785021dade215fdbf9b6ffcce2817d4509cadeac.

This closes #5540

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-11-19 08:52:20 -06:00
Timea Barna
785021dade
NIFI-9385 Add Flow Metrics producer for Cloudera Manager
This closes #5528

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-11-18 11:51:53 -06:00
exceptionfactory
605346ae0f
NIFI-9291 Added NiFi HTTP request logging
- Added nifi.web.request.log.format property
- Added Filters to set and retrieve authenticated username for logging

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5527.
2021-11-18 10:30:50 -05:00
M Tien
5016eac010
NIFI-9205 - Update prioritizer configuration (#5516)
* NIFI-9205 - Update prioritizer configuration

* - Address review feedback

This closes #5516
2021-11-17 13:55:02 -05:00
exceptionfactory
ab662434ce
NIFI-9372 Corrected NiFi application log messages
- Removed unnecessary spaces from initialization log
- Changed bootstrap temporary password file processing messages to debug
- Updated several log statements using parameterized strings
- Refactored NiFi unit test class

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5515.
2021-11-14 16:00:40 +01:00
exceptionfactory
5aced2b4bc
NIFI-9371 Removed synchronized keyword from Active Threads methods (#5513)
- ConcurrentHashMap collection for Active Threads eliminates the need for method synchronization
- Replaced System.out.println() with Logger.info() in nifi-system-test-suite classes
2021-11-10 09:18:41 -05:00
Mark Payne
119ba17bf1
NIFI-9365: Changed HashMap to ConcurrentHashMap in StandardProcessorNode for activeThreads
- Methods without the synchronized keyword, such as terminate(), iterate over activeThreads

This closes #5509

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-11-09 09:50:48 -06:00
Peter Turcsanyi
9d54fc6a3c
NIFI-9368: Fixed the ordering of the log entries on the processor level bulletin pop-up
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5512.
2021-11-09 15:57:41 +01:00
noblenumbat360
f410c8df0a
NIFI-8272 Delete stale metrics from REST API Prometheus endpoint.
Added <scope>test</scope> tag to the nifi-web-api pom.xml and corrected imports.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5447
2021-11-08 20:26:30 -05:00
Bryan Bende
c96809012b
NIFI-9350 Add NiFi Registry NarProvider implementation
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5497.
2021-11-08 09:18:10 -05:00
Joe Witt
9e845d87ff
Merge branch 'NIFI-9344-RC3' 2021-11-07 16:23:10 -07:00
shadon
a9685b36fc
optimize import. (#5323)
Co-authored-by: pengxd <pengxd@hua-cloud.com.cn>
2021-11-04 09:29:11 -04:00
Tamas Palfy
2bd752d868
NIFI-9202 Improve Allowable Values merging to handle cases when different nodes have different set of Allowable Values.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5380
2021-11-03 15:09:37 -04:00
Joe Witt
c055895952
NIFI-9344-RC3 prepare for next development iteration 2021-11-03 08:53:37 -07:00
Joe Witt
7fdc07cccd
NIFI-9344-RC3 prepare release nifi-1.15.0-RC3 2021-11-03 08:53:32 -07:00
Mark Payne
09feed9df6
NIFI-9352: Consider all property values in component references
- Ensure that we consider all property values to determine if a component references a variable, not just 'supported' (i.e., well-known/non-user-defined) properties

This closes #5499

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-10-30 11:16:57 -05:00
exceptionfactory
9865ea2bfb NIFI-9322 Refactored OIDC and SAML Access Resources
- Removed parent AccessResource from OIDCAccessResource and SAMLAccessResource to avoid unexpected inherited methods
- Moved Token Expiration validation from AccessResource to StandardBearerTokenProvider

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5489.
2021-10-28 12:12:57 -04:00
Joe Gresock
28cd5d1300
NIFI-9345: Resolving NPE in ProcessorEntityMerger (#5491)
NIFI-9345: Resolving NPE in ProcessorEntityMerger, Streamlining forEach calls
2021-10-28 12:06:56 -04:00
exceptionfactory
60d6d469bf
NIFI-9342 Upgraded to Netty 3.10.6 and 4.1.69
- Replaced Netty 3.6.9 and 3.7.1 with 3.10.6
- Replaced Netty 4.1 with 4.1.69

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5490.
2021-10-28 09:34:31 +02:00
Mark Payne
3114bdb701
NIFI-9311: When determining property values, be sure to fetch the property descriptor from the component itself, rather than using the PropertyDescriptor in the Map. This allows us to ensure that if the definition of the PropertyDescriptor changes, the most up-to-date definition is picked up
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5488
2021-10-27 17:08:06 -04:00
Lehel Boér
059f14fd62
NIFI-8385: Add FlowFiles from logging to bulletins (#4952) 2021-10-27 16:06:30 -04:00
Matthew Burgess
104078868e
NIFI-7012: Refactored OnConfigurationRestored to support sensitive property validation (#5415) 2021-10-27 14:40:17 -04:00
omkadmi
8506a6012f
NIFI-9337: added missing enum value SINGLE_BATCH_PER_NODE of FlowFileConcurrencyType (#5484) 2021-10-26 11:25:52 -04:00
exceptionfactory
7b2f364cf5
NIFI-9331 Corrected ThreadUtils using Objects.equals() for LockedStackFrame
- Removed unused arguments from ThreadUtils.createStackTrace()

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5480.
2021-10-26 09:56:28 +02:00
markap14
5766d33fce
NIFI-9309: Include a uiOnly flag when requesting flow for a given pro… (#5462)
NIFI-9309: Include a uiOnly flag when requesting flow for a given process group. In that case, do not include the property descriptors, property values, etc. for Processors. When fetching Variable Registry, improved logic to cache the VariableImpact for each property value instead of parsing/recreating it every time. When fetching bulletins for a component, avoid filtering through all components' bulletins and instead only look at bulletins that might pertain to the appropriate component
2021-10-23 19:42:31 -04:00
Matt Gilman
1bec905890 NIFI-9288:
- Allowing the user to submit a verification request for Processors, Controller Services, and Reporting Tasks.
- Tracking progress of verification requests.
- Showing the verification results.

NIFI-9288:
- Fixing class name which prevented styles from being applied.

NIFI-9288:
- Ensuring that previously entered referenced attribute values take precedence.

NIFI-9288:
- Positioning the property listing and verification results based on percentages instead of fixed values.
- Removing the additional dialog height.

NIFI-9288:
- Allowing attribute value entry to be skipped when appropriate.

NIFI-9288:
- Working around an issue caused by css minification.

NIFI-9288:
- Adding some padding to the verifying progress dialog.

This closes #5461

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2021-10-21 10:06:58 -04:00
Matt Gilman
51c6910510 NIFI-9237:
- Similar to NIFI-9215, converting integer identifiers to strings to ensure the items are successfully retrieved when attempting to apply a tooltip.

This closes #5454

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2021-10-20 16:47:21 -04:00
markap14
90ae271692
NIFI-9310: Addressed issues found while testing the component verification features. Added a supportsVerification flag to the ConfigAnalysis DTO (#5469)
This closes #5469
2021-10-20 11:07:17 -04:00
exceptionfactory
70274ebcfc
NIFI-9306 Upgraded Spring Framework to 5.3.11
- Upgraded Spring Security from 5.5.1 to 5.5.2
- Upgraded Spring Boot from 2.5.2 to 2.5.5 in Registry
- Upgraded Spring Data Redis from 2.5.3 to 2.5.5 in Redis modules

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5463.
2021-10-19 11:26:37 +02:00
Mark Payne
be2b80d909
NIFI-9292: Added getInputPortNames() and getOutputPortNames() to StatelessDataflowDefinition and getReadOnlyExtensionsDirectories() to StatelessEngineConfiguration
- Addressed issue identified in system test where a flow can fail due to FailurePortEncounteredException but then purge the flow in the background after the call to trigger completes

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5450.
2021-10-18 12:22:12 -04:00
Mark Payne
151a9361fa
NIFI-9288: Updated data model to use for cnofig verification requests so that only properties are necessary not the full component/config dtos. Also added endpoint necessary for determining which attributes are referenced by a component's properties and removed the referencedAttributes field from the compnoents themselves, since there's now a new endpoint for it. Also fixed a bug that was encountered where the VerifyConfigRequestDTO's complete flag was incorrect in case of failures
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5452.
2021-10-18 11:42:47 -04:00