Commit Graph

127 Commits

Author SHA1 Message Date
Mark Payne 016d58612c
NIFI-11290 Run Component Primary Node State changes in background thread
- Ensure that components are notified that primary node has changed in a background thread instead of the Leader Election thread and activate/deactivate the thread in the case of Processors so that they can be viewed in the UI and terminated

- Fixed system tests that would fail intermittently because they did not wait for node disconnection to complete and did not properly switch the client to look at the connected node before checking cluster status

This closes #7052

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 12:20:37 -05:00
Mark Payne 6adfb6131d
NIFI-11310: Provide appropriate classpath resources to the ReloadComponent when a processor is terminated
NIFI-11310: Fixed META-INF/services file that was mistakenly listing an extra extension point, due to rebase
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #7061
2023-03-21 14:29:44 -04:00
Mark Payne 87e61c50ee
NIFI-11192: Ensure that if ports moved between parent/child group in between flow versions that we can properly handle that. Added system tests to verify.
NIFI-11192: If a failure is encountered when changing the version of a flow from 1 version to another, attempt to rollback the changes instead of just failing with the flow in a bad state
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6981
2023-02-22 15:38:00 -05:00
Joe Witt 43eab4c81d
NIFI-11103 prepping for 2.0.0 line 2023-02-09 15:32:53 -07:00
exceptionfactory a37887305c
NIFI-11048 This closes #6858. Added FileNameCompleter to avoid style parsing failures
- Added StandardFileNameCompleter with static LS_COLORS to avoid parsing environment variables in Styles.lsStyle()

Signed-off-by: Joe Witt <joewitt@apache.org>
2023-01-18 14:31:55 -07:00
Joe Witt 75e7c9e47c
NIFI-10854-RC1 prepare for next development iteration 2022-11-22 19:22:47 -07:00
Joe Witt ec87bf93ad
NIFI-10854-RC1 prepare release nifi-1.19.0-RC1 2022-11-22 19:22:45 -07:00
exceptionfactory 5bc8e49c7a NIFI-10755 Refactored SSLContext creation using nifi-security-ssl
- Added TrustManagerBuilder to nifi-security-ssl
- Removed SslContextFactory and CertificateUtils from nifi-registry
- Refactored c2-client-http
- Refactored minifi-bootstrap
- Refactored nifi-site-to-site-client
- Refactored nifi-registry-client
- Refactored nifi-registry-framework
- Refactored nifi-toolkit-admin
- Refactored nifi-toolkit-cli

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6618.
2022-11-22 18:33:11 -05:00
mr1716 0f8a04f8d9
NIFI-10726 Upgraded OpenCSV from 5.7.0 to 5.7.1
This closes #6605

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-10-31 15:16:41 -05:00
Mark Payne d390a0becf
NIFI-10673: When a component is added to a ProcessGroup, we had a method called ensureUniqueVersionControlId but the method only took into accounts in that Process Group. Updated methods to now consider components in any Process Group that is part of the same Versioned Flow. Also added system test to verify the problem and the fix. Added Toolkit clients and additional methods as necessary in order to implement system tests
NIFI-10673: Addresed checkstyle issue

NIFI-10673: Removed TODO comment
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
2022-10-26 14:38:01 -04:00
exceptionfactory 969e2dc7cc
NIFI-10697 Refactored nifi-toolkit tests to use JUnit 5
- Removed RegistryManualIT for nifi-toolkit-cli
- Simplified EncryptConfigMainTest to avoid duplication with ConfigEncryptionToolTest

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6579
2022-10-26 10:49:04 -04:00
UcanInfosec c50e677ee3
NIFI-10649 Upgraded commons-cli to 1.5.0
This closes #6552

Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-10-20 19:38:25 -05:00
Joe Witt c9ebdd2025
NIFI-10521-RC4 prepare for next development iteration 2022-10-03 10:59:36 -07:00
Joe Witt 109e54cd58
NIFI-10521-RC4 prepare release nifi-1.18.0-RC4 2022-10-03 10:59:34 -07:00
simonbence 94fdc794b0
NIFI-10497 Making RegistryClient an extension point (#6433)
- NIFI-10497 Mae FlowRegistryClient an extension point
2022-09-26 12:16:21 -04:00
UcanInfosec 860d550435
NIFI-10504 Upgraded OpenCSV from 5.4 to 5.7.0 for nifi-toolkit-cli
This closes #6419

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-09-16 16:52:11 -05:00
Nandor Soma Abonyi 398af783c2
NIFI-10488 Fix NullPointerException on updating empty ParameterContext property with NiFi Toolkit (#6408)
NIFI-10488 Fix NullPointerException on updating empty ParameterContext property
2022-09-13 12:54:43 -04:00
Joe Gresock de7793e69b
NIFI-9003 Added framework support for Parameter Providers
- Added Environment Variable and File Parameter Provider implementations
- Added Parameter Provider commands to CLI
- Added component search handling for Parameter Providers
- Added documentation for integrating Parameter Providers

This closes #5369

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-09-09 13:04:56 -05:00
Mark Payne a0c705715b
NIFI-10429: Added the ability to Replay latest provenance event for a given Processor.
This closes #6359

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-09-08 15:55:26 -05:00
Joe Witt 8c66bf948c
NIFI-10272-RC2 prepare for next development iteration 2022-07-27 13:24:09 -07:00
Joe Witt 8d256784d8
NIFI-10272-RC2 prepare release nifi-1.17.0-RC2 2022-07-27 13:24:06 -07:00
Mark Payne ac8c1b0326
NIFI-10148: Fixed bug in SwappablePriorityQueue, in which the Active Queue Size could be decremented before the Unacknowledged Queue Size is incremented when FlowFiles are polled. This can result in the SwappablePrioriotyQueue incorrectly returning a value of true from the isEmpty() method. Additionally, updated the NiFiSystemIT so that if waiting for a particular queue size we periodically log size of all queues, which aids in debugging, and added necessary methods to FlowClient to make that happen.
This closes #6143

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-21 15:55:47 -05:00
exceptionfactory bd45eb4995 NIFI-10100 Upgraded Jersey to 2.35
- Replaced individual version references with Jersey BOM dependency

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6106.
2022-06-16 23:09:22 -04:00
Zoltan Kornel Torok a30ac23e90
NIFI-10116 Added CLI toolkit command to delete report tasks
This closes #6126

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-16 14:53:04 -05:00
exceptionfactory 99be62be67
NIFI-9958 Add Framework Support for Sensitive Dynamic Properties (#6057)
* NIFI-9958 Implemented Sensitive Dynamic Properties

- Added SupportsSensitiveDynamicProperties annotation for components
- Added optional sensitive query parameter to Property Descriptor REST Resource methods
- Added system tests for components supporting sensitive dynamic properties
- Updated REST Resources to support Sensitive Dynamic Property Names
- Updated Documentation Writer to indicate component support for Sensitive Dynamic Properties
- Updated InvokeHTTP to support Sensitive Dynamic Properties
- Updated Auditor components to handle masking Sensitive Dynamic Properties

* Refactored Property Descriptor REST method handling

- Corrected AbstractDocumentationWriter evaluation of support for sensitive dynamic properties
- Refactored Controller Service Dynamic Properties system tests to new class

* Updated AbstractComponentNode.getProperties() to get canonical descriptor
2022-05-23 17:20:54 -04:00
exceptionfactory 0c8305950c
NIFI-9987 Standardize Commons IO on version 2.11.0
- Removed version declarations from multiple modules
- Adjusted PutDynamoDB to remove catch for IOException not thrown in Commons IO 2.11.0

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6015.
2022-05-09 12:06:37 +02:00
exceptionfactory 4f423a59ba
NIFI-9952 Upgraded Jackson BOM to 2.13.2.20220328
- Removed unnecessary references to jackson.version property
- Removed unnecessary dependency management references to Jackson libraries

This closes #5992

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-04-23 08:23:39 -04:00
Joe Witt 0419dc2939
NIFI-9780-RC3 prepare for next development iteration 2022-03-21 11:58:27 -07:00
Joe Witt b019a9191f
NIFI-9780-RC3 prepare release nifi-1.16.0-RC3 2022-03-21 11:58:09 -07:00
Mike Thomsen 2bb1a06a8c
NIFI-9788 Updated commons-codec to 1.15 across all modules.
This closes #5881

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-21 09:13:05 -05:00
exceptionfactory b09a778d90
NIFI-9685 Upgraded JNA to 5.10.0
- Upgraded nifi-bootstrap-utils JNA from 4.4.0 to 5.10.0
- Upgraded nifi-windows-event-log-processors JNA from 4.5.2 to 5.10.0
- Upgraded nifi-toolkit-cli JLine from 3.5.2 to 3.21.0

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5766.
2022-02-15 10:52:54 +01:00
Joe Gresock a3fb61d9e6
NIFI-9668: Adding informative error when setting same value in 'set-param' CLI command
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5757.
2022-02-10 15:03:38 +01:00
Bryan Bende d6efc69585
NIFI-9502 Move extension manifest model and parser out of nifi-registry and into common modules
NIFI-9502 Move ExtensionManifest parser and impl from nifi registry into common module

NIFI-9502 Add jigsaw profile to nifi-extension-manifest-model in order to build on Java 11

Add apache license header to test resources

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5619
2021-12-22 10:46:59 -05:00
exceptionfactory 7c6bdcb035
NIFI-9491 Excluded commons-logging and added jcl-over-slf4j references
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5608
2021-12-17 07:21:03 -05:00
Mark Payne 90b39b593a
NIFI-9069 Changed framework dataflow serialization to support JSON
- Changed framework so that it serializes the dataflow into a VersionedDataflow using JSON as well as XML, and prefers the JSON representation on load, if it's available. This also results in the need for the cluster protocol to exchange its representation of the dataflow to using JSON. Rather than re-implementing all of the complex logic of Flow Fingerprinting, updated to just inherit the cluster's flow.
- Moved logic to synchronize Process Group with Versioned Process Group into a new ProcessGroupSynchronizer class instead of having all of the logic within StandardProcessGroup
- Reworked versioned components to use an instance id.
- Renamed StandardFlowSynchronizer to XmlFlowSynchronizer; introduced new StandardFlowSynchronizer that delegates to the appropriate (Xml or Versioned)FlowSynchronzer
- Updated to allow import of VersionedProcessGroup even if not all bundles are available - will now use ghost components
- Introduced a VersionedDataflow object to hold controller-level services, reporting tasks, parameter contexts, templates, etc.
- Allow mutable requests to be made while nodes are disconnected. Also fixed issue in AbstractPolicyBasedAuthorizer that caused ClassNotFoundException / NoClassDefFoundError if the authorizations were changed and then a node attempts to rejoin the cluster. The Authorizer was attempting to use XmlUtils, which is in nifi-security-utils and so so by madking nifi-security-utils a provided dependency of nifi-framework-api, but this doesn't work, because nifi-framework-api is loaded by a higher-level classloader, so the classloader that loads AbstractPolicyBasedAuthorizer will never have the appropriate classloader to provide nifi-security-utils. Addressed this by copying the code for creating a safe document builder from XmlUtils to AbstractPolicyBasedAuthorizer.
- Fixed bug that occurred when importing a Process Group that has 2 parameter contexts, one inheriting from another, where neither is pre-defined in the existing flow
- Fixed bug that was encountered when Updating a Versioned Process Group where one version had a disabled processor and the other had the processor running.
- Increased system-tests workflow timeout to 120 minutes
- Added additional exception handling to system tests

This closes #5514

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-12-02 15:10:38 -06:00
Joe Witt c055895952
NIFI-9344-RC3 prepare for next development iteration 2021-11-03 08:53:37 -07:00
Joe Witt 7fdc07cccd
NIFI-9344-RC3 prepare release nifi-1.15.0-RC3 2021-11-03 08:53:32 -07:00
Mark Payne 151a9361fa
NIFI-9288: Updated data model to use for cnofig verification requests so that only properties are necessary not the full component/config dtos. Also added endpoint necessary for determining which attributes are referenced by a component's properties and removed the referencedAttributes field from the compnoents themselves, since there's now a new endpoint for it. Also fixed a bug that was encountered where the VerifyConfigRequestDTO's complete flag was incorrect in case of failures
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5452.
2021-10-18 11:42:47 -04:00
Mark Payne baf29e59f1
NIFI-9009: Created VerifiableProcessor, VerifiableControllerService, VerifiableReportingTask components; implemented backend work to call the methods. Added REST APIs and created/updated data models for component configuration verification
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5288
2021-09-22 19:00:23 -04:00
Joe Gresock 4a3e81531b
NIFI-8490: Adding inherited parameter contexts (#5072)
- Allowing inherited param contexts on creation, updating PC authorization
2021-08-30 10:44:31 -04:00
Joe Witt 97feacc181
NIFI-8767-RC2 prepare for next development iteration 2021-07-10 12:17:09 -07:00
Joe Witt fcbf1d5f97
NIFI-8767-RC2 prepare release nifi-1.14.0-RC2 2021-07-10 12:17:05 -07:00
exceptionfactory 2efc2b6b8b
NIFI-8718 Upgraded Apache Commons IO to 2.10.0
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5169
2021-06-21 11:29:27 -04:00
Mohammed Nadeem 14265a4015
NIFI-8682: Upgrade opencsv from 4.5 to 5.4 to mitigate CVE_2014_0114
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5144.
2021-06-15 19:53:43 +02:00
tpalfy dfa683af0e
NIFI-8528 Migrate NiFi Registry into NiFi codebase (#5065)
NIFI-8528 Migrate NiFi Registry fully codebase into NiFi as a module. No changes except certain dependency scopes to preserve the NiFi Registry original by overriding the new parent (nifi).
- Version adjustments. Removed distinct checkstye rules form nifi-registry. (Using nifi's instead.)
- Made some tests Windows-compatible.
- Consolidated LICENSE, NOTICE and README.md.
- Fixed CryptoKeyLoaderGroovyTest.groovy.
- Disable frontend-maven-plugin on Windows.
- Skipping all goals of the frontend-maven-plugin on Windows.
- Registry integration tests not to run in github jobs (same as the original settings). Skip all registry tests (build and run) on Windows.
- Removed Husky from registry.
2021-05-26 09:34:55 -04:00
exceptionfactory 6776765a92
NIFI-8538 Upgraded Apache Commons IO to 2.8.0
- Upgraded direct dependencies from 2.6 to 2.8.0
- Added dependency management configuration to use 2.8.0 for some modules
- Updated scripted Groovy tests to avoid copying unnecessary files

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5073
2021-05-14 09:31:48 -04:00
Nathan Gough 07a4966d10
NIFI-8329 - Updated dependencies with no build failures
NIFI-8329 - Removed unnecessary jackson.version from azure bundle to use the global property instead.

NIFI-8329 - Updated jackson/jackson-databind version and removed the 'jackson-databind.version' pom property in favor of 'jackson.version'

Updated dependencies include the following:

- jackson-core
- jackson-databind
- icu4j
- snakeyaml
- spring-integration-mail
- spring-core and framework modules
- activemq-client
- activemq-broker
- xercesImpl

This closes #4911

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-03-19 14:46:33 -05:00
exceptionfactory 0e659981d4 NIFI-8298 Refactored Kerberos and Socket classes from security-utils to new modules
- Created nifi-security-socket-ssl
- Created nifi-security-kerberos
- Removed nifi-security-utils dependency from nifi-processor-utils
- Updated modules to reference new dependencies
- Eliminated unnecessary transitive dependencies on bcprov-jdk15on from over 30 modules

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4881.
2021-03-19 13:12:19 -04:00
Tamas Palfy aa726040c5 NIFI-8188 - Add 'Run Once' for processors in context menu.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-02-18 14:51:28 -05:00
Joe Witt 88fab00e29
NIFI-7873 merging release branch to latest and updating to 1.14.0-SNAPSHOT 2021-02-15 12:09:32 -07:00