2573 Commits

Author SHA1 Message Date
Zackeus Bengtsson
0a47157640
NIFI-12122 Fixed persistence of Parameter Context descriptions
Fixed bug where parameter context descriptions were not loaded on NiFi startup and overwritten as empty

This closes: #7787

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-10-13 18:30:50 -05:00
Mark Payne
f4ae292a45
NIFI-12221: This closes #7876. Be more lenient about which Disconnection Codes we allow a node to be reconnected to a cluster vs. when we notify the node to disconnect again. Also updated the timeout for OffloadIT because it occasionally times ou out while running properly.
Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-10-13 13:59:05 -07:00
Mark Payne
0eabbcdf19
NIFI-12228: This closes #7881. Fixed issue with FlowFile Concucrrency that can occasionally bring in more data than it should.
Code cleanup, fixing logback to avoid INFO-level stack trace from xodus

Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-10-13 13:36:09 -07:00
Bryan Bende
96eb1d825a
NIFI-12222 Protect against missing parameter context when syncing a PG in component synchronizer (#7877) 2023-10-13 13:51:47 -04:00
exceptionfactory
22ad7d542d
NIFI-12206 Refactor Flow History using JetBrains Xodus (#7870)
* NIFI-12206 Refactored Flow History using JetBrains Xodus

- Replaced H2 Database Engine with JetBrains Xodus for persistent storage of FlowConfigurationHistory
- Added EntityStoreAuditService implementation using Xodus PersistentEntityStore
- Removed nifi.h2.url.append from properties
2023-10-12 14:50:25 -04:00
Mark Payne
abfc49e212
NIFI-12139: Implemented migrateProperties, migrateRelationships for processors, as well as migrateProperties for ReportingTasks and Controller Services. Added system tests to verify behavior.
- Ensure that after restoring nars in the lib/ directory we restart NiFi so that they take effect. This is important if this test is not the last one to run

This closes #7837

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-10-12 11:40:18 -05:00
exceptionfactory
a849ca044c
NIFI-12218 Removed SensitiveValueEncoder and SecureHasher
- SensitiveValueEncoder and SecureHasher are no longer required following the removal of support for flow.xml.gz

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7873.
2023-10-12 17:39:43 +02:00
Bence Simon
3f13604c36
NIFI-11772 Removed flow.xml.gz support
- Created NIFI-12203 to evaluate issues with flow comparison surfaced in JoinClusterWithDifferentFlow

This closes #7661

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-10-11 13:53:01 -05:00
timeabarna
d2aec89738
NIFI-12166 Add repo usage to the monitoring endpoint (#7836)
* NIFI-12166 Add repo usage to the monitoring endpoint
2023-10-10 14:55:33 -04:00
exceptionfactory
98cebc6687
NIFI-12185 Added nifi-bom Bill of Materials (#7852)
- Imported nifi-bom to nifi-commons and nifi-nar-bundles parent modules
- Removed explicit version and scope properties for dependencies declared in nifi-bom
2023-10-10 13:55:22 -04:00
Bryan Bende
ae61ebb5ed
NIFI-12186 Add ability to export a versioned reporting task snapshot (#7853)
* NIFI-12186 Add ability to export a versioned reporting task snapshot
- Add CLI commands and optional query param to specify specific reporting task
2023-10-09 10:50:36 -04:00
Nissim Shiman
5afda79f28
NIFI-11782 Resolved NPE when moving snippet with label to process group
This closes #7838

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-10-05 16:05:40 -05:00
Mark Payne
711f2f1188
NIFI-12137: This closes #7808. Fixed name of Expression Language Scope from LIMITED to ENVIRONMENT, ; fixed issue in which flowfiles routed to failure by FlowFileTransform were also being routed to original; defined 'identifier' and 'logger' as member variables for FlowFileTransform and RecordTransform so that IDEs know about them
Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-10-04 09:16:04 -07:00
Mark Payne
1d9ccb3857
NIFI-12161: This closes #7829. Ensuring framework threads use lightweight threads instead of a capped thread pool. This prevents framework threads from livelocking in the event enough framework threads are holding threads while those needing to run cannot get them.
Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-10-04 08:48:12 -07:00
Tamas Palfy
9c425d273c
NIFI-12131 Fixed potential NPE related to FlowAnalyzer
Revamped RuleViolationsManager and FlowAnalyzer handling to make sure no issue occurs when these are not set. Fix ResourceNotFoundException: When determining the subject permissions for a rule violation the type of the subject is now known, so we try to lookup all possible types. Non-matching tpyes throw a ResourceNotFoundException exception though. Going to ignore those.

This closes #7809

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-10-02 15:35:05 -05:00
exceptionfactory
19b4be40aa
NIFI-12157 This closes #7826. Removed nifi-logging-utils and NiFiLog
- Replaced NiFiLog references with direct references to SLF4J LoggerFactory

Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-10-02 08:17:51 -07:00
Joe Gresock
ff8f1ff686
NIFI-8531: This closes #7529. Upgrading Kinesis processors to AWS SDK 2.x
Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-10-01 16:26:51 -07:00
Emilio Setiadarma
a8c1bb1d9c
NIFI-10425 Corrected auto-reloading KeyStore files with symlinks
- Changed StoreScanner scanner instantiation to not report real paths

This closes #7821

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-30 12:28:54 -05:00
exceptionfactory
c76191fa4b
NIFI-12152 This closes #7818. Refactored addProvider() Bouncy Castle references
- Removed Security.addProvider() references from several tests
- Refactored KeyStoreUtils to use instance of BouncyCastleProvider instead of BC provider name string
- Refactored MiNiFi references to pass BouncyCastleProvider for BCFKS

Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-09-30 07:56:53 -07:00
exceptionfactory
75c3b19979
NIFI-12142 This closes #7822. Corrected Cluster Pool Size Method Reference
- Set getClusterNodeProtocolMaxPoolSize method reference in place of deprecated getClusterNodeProtocolThreads

Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-09-30 07:54:13 -07:00
Joe Witt
476f30813d
NIFI-12132 change default content repo archive percentage and age to 90 percent and 3 hours to better align to default user needs (#7797) 2023-09-28 17:32:21 -04:00
Mark Payne
db727aa419
NIFI-12142: This closes #7806. Deleted many methods, classes, and references that were deprecated.
Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-09-28 14:07:48 -07:00
Pierre Villard
43cc5ad122
NIFI-12079 This closes #7750. Remove Variables / Variable Registry
Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-09-25 09:39:53 -07:00
exceptionfactory
bcc54953c4
NIFI-12120 This closes #7784. Removed Obsolete Framework Integration Tests
- Removed integration tests from org.apache.nifi.integration in nifi-framework-core

Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-09-24 15:48:47 -07:00
Pierre Villard
2aca08910c
NIFI-12103 Replaced deprecated usage of new URL(String)
This closes #7771

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-23 10:30:42 -05:00
Mark Payne
140ac52e9d
NIFI-12068: This closes #7737. Added the ability to annotate components with @UseCase and @MultiProcessorUseCase annotations; updated several processors to make use of them.
Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-09-22 10:47:47 -07:00
Mark Payne
713be8b539
NIFI-12101: This closes #7768. Fixed bugs that caused StatelessBasicsIT.testChangeFlowVersion to fail. There were a couple of related issues. Firstly, for Stateless Groups we were not waiting for the group to fully stop before attempting to update the flow but were waiting only for child/descendant groups to stop. This was due to misleading naming in the ProcessGroupUpdateStrategy enum. Renamed enum values to more clearly convey what they mean. Additionally, renamed the enum to more clearly convey what how it is used, since it is used not only for updating groups but also for retrieving groups. Additionally, when waiting for stateless groups to stop, we inadvertently checked the Set of affectedComponents' IDs against Process Group IDs so addressed the logic there and also ensured that we were consistent in setting the AffectedComponentEntity's ProcessGroup (i.e., its ProcessGroupNameDTO) as we were inconsistently setting it across the codebase.
Fixed issue in logic of LocalComponentLifecycle, which was waiting for all PGs to reach desired stateless run schedule, even when the group itself was not stateless

Fixed JavaScript to specify correct value for recursivity when updating process group parameter context
2023-09-22 09:00:52 -07:00
Nandor Soma Abonyi
9ae6921272
NIFI-12095 This closes #7762. Increase default Xmx to 1g
Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-09-21 19:48:34 -07:00
Pierre Villard
5405a3b08b
NIFI-12102 Fixed deprecation warnings when instantiating primitive types
This closes #7769

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-21 13:41:30 -05:00
exceptionfactory
11d8519914
NIFI-12086 This closes #7757. Removed unused jgit dependency from framework-core
Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-09-20 14:28:38 -07:00
Joseph Witt
e5894cf943
NIFI-12053 Updated Minimum Build Version to Java 21
- Configured GitHub Workflows with Azul Zulu and Amazon Corretto
- Added bootstrap.conf option for reflective access supporting Hive 3

This closes #7744

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-19 22:59:59 -05:00
Joe Witt
017b16254b
NIFI-11903 disable if windows (#7566) 2023-09-19 16:22:49 -04:00
Peter Turcsanyi
520840d93b
NIFI-12076 Removed ComponentLog methods with Object[] and Throwable
This closes #7759

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-19 13:10:19 -05:00
Peter Turcsanyi
ac80b9d900
NIFI-11595 StateProvider.replace() supports creating the initial state
- Extracted common logic from setState() and replace() into modifyState()
- Removed redundant code from createNode() because exceptions are handled on the caller side
- NodeExistsException and InterruptedException are handled in setState() and replace()
- Also used KeeperException's subclasses instead of KeeperException.code()

This closes #7324

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-19 10:06:38 -05:00
Peter Turcsanyi
ebd0d60e49
NIFI-12075 Deprecated ComponentLog methods with Object[] and Throwable
- Updated component references to remove use of Object[] wrapping for log methods

This closes #7748

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-19 08:53:00 -05:00
Joe Witt
cbf5bb98f9
NIFI-12072 moved to StackWalker API elimnating usage of SecurityManager (#7742) 2023-09-16 17:05:22 -04:00
Joseph Witt
bd2276260e
NIFI-12051 Upgraded Groovy from 3.0.19 to 4.0.15
- Adjusted Groovy Maven coordinates to org.apache.groovy
- Adjusted build configuration and tests for Groovy 4

This closes #7692

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-15 13:38:24 -05:00
Joseph Witt
71bdbc080b
NIFI-12066 Upgraded framework and cloud provider dependencies
- Upgraded SLF4J from 2.0.7 to 2.0.9
- Upgraded Logback from 1.3.8 to 1.3.11
- Upgraded Testcontainers from 1.18.3 to 1.19.0
- Upgraded Fabric8 Kubernetes from 6.5.1 to 6.8.1
- Upgraded AspectJ from 1.9.19 to 1.9.20.1
- Upgraded Caffeine from 3.1.6 to 3.1.8
- Upgraded AWS SDK from 2.20.103 to 2.20.148
- Upgraded Guava from 32.0.1 to 32.1.2
- Upgraded Nimbus JOSE JWT from 9.31 to 9.33
- Upgraded Apache Tika from 2.8.0 to 2.9.0
- Upgraded gRPC from 1.57.2 to 1.58.0
- Upgraded Google Libraries from 26.17.0 to 26.22.0
- Upgraded Azure SDK from 1.2.13 to 1.2.16

This closes #7733

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-15 13:05:31 -05:00
Joseph Witt
a1fca9b44e
NIFI-12064 Upgraded Maven Plugin versions
This closes #7731

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-14 19:35:51 -05:00
exceptionfactory
aee9f2c65d
NIFI-12034 This closes #7729. Upgraded Apache Commons Compress from 1.23.0 to 1.24.0
Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-09-14 13:21:17 -07:00
exceptionfactory
98fa7d0290
NIFI-12060 This closes #7694. Removed Groovy Tests using Spock Framework
- Added org.spockframework to the list of banned dependencies
- Removed several other Groovy tests and associated build profiles

Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-09-14 11:22:54 -07:00
Mark Payne
f27ace1ccf
NIFI-12016: This closes #7662. Allow use of compatible NAR bundles when loading flow from cluster connection; when determining what bundles are compatible, consider not just any bundle if it's the only one but also any bundle whose version matches the framework version so that when NiFi is upgraded, it is handled more gracefully.
Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-09-14 10:29:23 -07:00
Joseph Witt
4d4c97d091
NIFI-12050 Upgraded Mockito from 4.11.0 to 5.5.0
This closes #7689

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-14 09:49:42 -05:00
exceptionfactory
6841aa6e99
NIFI-12047 Added HTTP/2 to Default Application Protocols
- Changed default value of nifi.web.https.application.protocols to include both h2 and http/1.1
- Changed default value of nifi.registry.web.https.application.protocols to include both h2 and http/1.1
- Updated HostHeaderHandler logging

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7684.
2023-09-14 13:50:01 +02:00
Timea Barna
c30fd9b66d
NIFI-12006 Removed Templates from Framework and UI
This closes #7658

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-11 14:52:41 -05:00
Matt Burgess
932cfe22a3
NIFI-12010: Handle auto-commit and commit based on driver capabilities in SQL components
Signed-off-by: Arpad Boda <aboda@apache.org>

This closes #7663
2023-09-09 11:13:31 +02:00
Peter Turcsanyi
6cffc78ad2
NIFI-12025: Fixed duplicated bulletin messages (#7670) 2023-09-08 13:16:44 -04:00
Freedom9339
2b330d9fee
NIFI-10904 Changed the font color for drop down menus to avoid the app… (#7502)
* NIFI-10904 Changed the font color for dropdown menus to avoid the appearance of menu items being disabled

* NIFI-10904 Changed the CSS tag used to set the color of the dropdown menu text

* NIFI-10904 Used a new css class to set the color of dropdown options

* nifi-10904 Changed the css class for create/reference drop down menu items.

Merged #7502 into main.
2023-08-31 09:41:08 -07:00
exceptionfactory
5dc1ea27d3 NIFI-12001 Refactored CertificateUtils to separate modules
- Added nifi-security-cert for reusable components without dependencies
- Added nifi-security-cert-builder for certificate generation

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7651
2023-08-30 15:08:46 -04:00
Tamas Palfy
d4147ca442
NIFI-8650 Added Flow Analysis Rules
- Added new extensible Component Type: FlowAnalysisRule
- Added DisallowComponentType Rule implementation
- Flow Analysis Rules can be managed from the UI under Controller Settings -> Flow Analysis Rules
- Flow Analysis Rules can be set up with an enforcement policy of WARN or ENFORCE
- Flow Analysis Rules can evaluate an individual Component or a Process Group

This closes #7191

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-08-29 15:50:22 -05:00