Commit Graph

1008 Commits

Author SHA1 Message Date
exceptionfactory 6841aa6e99
NIFI-12047 Added HTTP/2 to Default Application Protocols
- Changed default value of nifi.web.https.application.protocols to include both h2 and http/1.1
- Changed default value of nifi.registry.web.https.application.protocols to include both h2 and http/1.1
- Updated HostHeaderHandler logging

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7684.
2023-09-14 13:50:01 +02:00
Timea Barna c30fd9b66d
NIFI-12006 Removed Templates from Framework and UI
This closes #7658

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-11 14:52:41 -05:00
Mike Moser 1449325027 NIFI-12019 bugfix for SynchronousFileWatcher time check interval
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7664
2023-09-06 12:41:58 -04:00
exceptionfactory 5dc1ea27d3 NIFI-12001 Refactored CertificateUtils to separate modules
- Added nifi-security-cert for reusable components without dependencies
- Added nifi-security-cert-builder for certificate generation

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7651
2023-08-30 15:08:46 -04:00
Tamas Palfy d4147ca442
NIFI-8650 Added Flow Analysis Rules
- Added new extensible Component Type: FlowAnalysisRule
- Added DisallowComponentType Rule implementation
- Flow Analysis Rules can be managed from the UI under Controller Settings -> Flow Analysis Rules
- Flow Analysis Rules can be set up with an enforcement policy of WARN or ENFORCE
- Flow Analysis Rules can evaluate an individual Component or a Process Group

This closes #7191

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-08-29 15:50:22 -05:00
exceptionfactory 807df22af9
NIFI-11990 Disabled Kubernetes Auto Config in Test Class
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7643.
2023-08-24 18:31:06 +02:00
Mike Thomsen cba6b00271
NIFI-11383 Updated Java documentation links from 8 to 17
This closes #7581

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-08-09 09:17:48 -05:00
exceptionfactory bbd8fb6f63
NIFI-11895 Removed deprecated Repository Encryption configuration
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #7557.
2023-08-09 07:07:31 -04:00
Tamas Neumer 80e71068be NIFI-11860 fixed bug in ResultSetRecordSet affecting decimalScale when resultSetScale was zero.
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7545
2023-08-02 12:22:55 -04:00
mr1716 da953c1fac
NIFI-11882 Upgraded Spring Vault from 2.3.3 to 2.3.4
This closes #7547

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-07-31 19:54:20 -05:00
Mark Payne c68f5ec9a2
NIFI-11793 Added Python debugging properties and documentation
Added documentation to indicate how to debug Python side of nifi framework, as well as debugging Python processors themselves using VSCode's Remote debugger.
This also provides the ability to launch the Controller process in such a way that it will listen to incoming remote debug connections.

This closes #7469

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-07-24 13:23:19 -05:00
Pierre Villard c06bb97e85
NIFI-11833 - remove deprecated classes in nifi-commons (#7503)
Signed-off-by: Otto Fowler <ottobackwards@gmail.com>
2023-07-20 11:58:51 -04:00
exceptionfactory 09e36f4d3e NIFI-11757 Upgraded Google Cloud Libraries from 26.15.0 to 26.17.0
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7439
2023-06-27 12:48:51 -04:00
Chris Sampson 3f5ed23504
NIFI-9206 Added RemoveRecordField Processor
This closes #6816

Co-authored-by: Peter Gyori <peter.gyori.dev@gmail.com>
Co-authored-by: Chris Sampson <chris.sampson82@gmail.com>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-26 09:50:41 -05:00
exceptionfactory 1f1c5df5a3
NIFI-11673 Removed Legacy TLS Configuration Versions
This closes #7367

Signed-off-by: Nandor Soma Abonyi <nsabonyi@apache.org>
2023-06-26 11:10:39 +02:00
timeabarna ba6797bb94
NIFI-3065 Per Process Group logging (#7315)
* NIFI-3065 Per Process Group logging
2023-06-23 21:14:34 -04:00
exceptionfactory 305277b833 NIFI-11709 Upgraded Guava from 32.0.0 to 32.0.1
- Added version override for Guava 27 to 32.0.1 in Hadoop and Hive modules

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7415
2023-06-20 16:22:00 -04:00
exceptionfactory b6b621bf9e
NIFI-11717 Updated Required Java Version to 17.0.6
- Added Groovy Plugin Repository for groovy-eclipse-compiler

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7397.
2023-06-19 10:19:55 +02:00
exceptionfactory 6b19ab8eaa NIFI-11696 Upgraded Bouncy Castle from 1.71 to 1.74
- Adjusted nifi-repository-encryption to remove dependency on Bouncy Castle Provider
- Updated Google Cloud Provider dependencies to remove exclusions and dependencies on Bouncy Castle that no longer apply to current versions

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7384
2023-06-15 16:33:20 -04:00
exceptionfactory ca6c7f7041 NIFI-11532 Removed JUnit 4 and Groovy Test from default dependencies
- Removed hamcrest-all from default dependencies
- Added groovy-test to selected modules with Groovy test classes
- Added junit-vintage-engine to selected modules with JUnit 4 test classes
- Corrected references to JUnit 4 assertions in JUnit 5 test classes
- Removed several unnecessary test classes from nifi-socket-utils
- Removed duplicative Registry toolkit test classes
- Removed Kudu integration tests

NIFI-11532 Corrected scope for junit-vintage-engine for Elasticsearch

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7233
2023-06-13 15:36:23 -04:00
mr1716 054df20aed
NIFI-11665 Upgraded oauth2-oidc-sdk from 9.43.1 to 9.43.3
This closes #7358

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-08 16:18:44 -05:00
Matt Burgess f3e8651522 NIFI-11647: Support UUID type in DataTypeUtils.getSQLTypeValue
This closes #7355

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2023-06-08 11:13:07 -04:00
exceptionfactory c5c711c91d
NIFI-11645 This closes #7342. Upgraded Guava from 31.1 to 32.0.0
- Added selective overrides to upgrade transitive dependencies

Signed-off-by: Joe Witt <joewitt@apache.org>
2023-06-05 14:33:10 -07:00
dan-s1 27b5a2f931
NIFI-11613 Refactored SanitizeContextPathFilterTest from Groovy to Java
- Moved Java tests from groovy directory to java directory and removed groovy directory from nifi-web-utils

This closes #7333

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-02 16:28:53 -05:00
Mark Payne 50811660d0
NIFI-11603 Refactored Socket-based tests to use Dynamic Ports
- Removed NetworkUtils methods for getting available ports
- Updated Socket-based components to support using 0 to listen on a random available port for improved test reliability

This closes #7299

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-01 13:33:06 -05:00
Pierre Villard d4f301f473
NIFI-11620 Upgraded Google libraries-bom from 26.12.0 to 26.15.0
This closes #7321

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-31 16:31:44 -05:00
Pierre Villard 06b49cbc32
NIFI-11619 Upgraded Azure BOM from 1.2.11 to 1.2.13
This closes #7320

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-31 16:31:04 -05:00
Bryan Bende 921e3d0f2a
NIFI-11604 Improve handling of non-renewable tickets in AbstractKerberosUser
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7301.
2023-05-31 15:13:41 +02:00
dan-s1 818747d84b
NIFI-11612 Refactored SocketUtilsTest from Groovy to Java
This closes #7310

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-30 14:39:17 -05:00
dan-s1 2f8e5b27b0
NIFI-11611 Refactored TlsConfigurationTest from Groovy to Java
This closes #7309

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-30 14:32:29 -05:00
dan-s1 1dfd85b40b
NIFI-11600 Combined the two TestFormatUtils into one class
This closes #7298

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-25 16:15:31 -05:00
Matthew Burgess df04c60e01
NIFI-8326: Send records as individual messages in Kafka RecordSinks
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4901.
2023-05-24 10:43:30 +02:00
Mark Payne a12c9ca9c7
NIFI-11557: Avoid using the expensive and unnecessary Files.walkFileTree on startup and initialization of Content Repository. Also performed some code cleanup: IntelliJ flagged many warnings in the class, mostly around methods that are no longer used and potential NullPointerExceptions, so those were cleaned up. Additionally, removed the nifi property for max flowfiles per claim - this property was never implemented. It was referenced, but the way in which is was used curiously had nothing to do with what the property was intended to be used for or for how it was documented. Instead, it was used to limit the max number of claims that could remain writable. As a result, it was removed.
NIFI-11557: Added an additional system test and updated github actions to include surefire-report in order to help diagnose problem that occurred in one of the last system-test runs in Github. Could not replicate problem locally
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #7265
2023-05-23 14:29:29 -04:00
dan-s1 e77835758c
NIFI-11581 Refactored JSON Validator Test from Groovy to Java
This closes #7283

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-23 10:02:44 -05:00
dan-s1 57f5b518f8
NIFI-11528 Refactored TestFormatUtilsGroovy to Java Parameterized Test
This closes #7248

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-23 09:23:18 -05:00
Arek Burdach 8a4c3cab84
NIFI-8161 Migrated EL from SimpleDateFormat to DateTimeFormatter
This closes #7169

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-19 16:10:12 -05:00
Emilio Setiadarma c4f7251b23
NIFI-11531 Migrated tests in nifi-security-utils from Groovy to Java
This closes #7238

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-17 13:39:15 -05:00
dan-s1 c382ab6f06
NIFI-11150 Refactored QueryGroovyTest to TestQuery
This closes #7250

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-17 09:08:01 -05:00
exceptionfactory bfe4110b56
NIFI-11513 Remove deprecated NarProvider and implementations (#7222)
* NIFI-11513 Removed deprecated NarProvider and implementations

- Removed HDFSNarProvider
- Removed NiFiRegistryNarProvider
- Removed nifi-external-resource-utils module from nifi-commons
- Removed unnecessary deprecated annotations from EncryptedSchemaRepositoryRecordSerde
- Removed deprecated method from ContentRequestContext interface
- Restored NiFi Registry Provider example properties
2023-05-16 16:10:25 -04:00
exceptionfactory b9f4d02094
NIFI-11539 Removed ZooKeeper Migrator Toolkit (#7237) 2023-05-15 14:44:23 -04:00
exceptionfactory e302f2aff7
NIFI-10976 Added Previous Cluster State Provider configuration (#7235)
- Added methods to enumerate Stored Component Identifiers on State Provider interface and implementations
- Added nifi.state.management.provider.cluster.previous to nifi.properties
- Updated State Manager Provider to restore Cluster State from Previous Cluster Provider
- Updated Configuring State Providers documentation for new property
2023-05-10 16:22:05 -04:00
Timea Barna 5811a9c579
NIFI-11385 Added JMX Metrics REST Resource for Diagnostics
This closes #7124

Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-08 08:11:20 -05:00
Pierre Villard 7fb2468f4b NIFI-11502 - Upgrade json-path to 2.8.0
NIFI-11502 fixing unit tests
NIFI-11502 review

This closes #7208

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2023-04-29 16:03:16 -04:00
Chris Sampson 9274d0c9b0 NIFI-10865 allow RecordPath's unescapeJson to convert de-serialised JSON Objects into Records
NIFI-10865 allow UpdateRecord to replace the Record root for relative paths, e.g. when a RecordPath function is used to modify selected field(s)

This closes #6708

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2023-04-29 12:26:07 -04:00
exceptionfactory ee03db0e8f
NIFI-11488 Removed Deprecated Sensitive Properties Algorithms
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7198.
2023-04-27 20:16:26 +03:00
Mark Payne 7f7cccfdc2
NIFI-11241 Initial implementation of Python-based Processor API with Py4J
- Updated GitHub workflow so that system tests include Python 3.9
- Updated GitHub actions to build necessary modules for system tests

This closes #7003

Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-14 15:03:34 -05:00
Pierre Villard a148b6f348
NIFI-11407 Upgraded Azure BOM from 1.2.9 to 1.2.11
- Added Woodstox Core to nifi-toolkit-encrypt-config

This closes #7136

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-10 14:36:48 -05:00
Pierre Villard 2b06306b07
NIFI-11418 Upgraded bcrypt from 0.9.0 to 0.10.2
This closes #7149

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-10 08:45:55 -05:00
Pierre Villard b0997f843d
NIFI-11416 Upgraded Caffeine from 2.8.1 to 3.1.6
This closes #7147

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-10 08:45:12 -05:00
Pierre Villard 0160d6d6b9
NIFI-11410 Upgraded multiple integration dependencies
- Upgraded Spring Boot from 2.7.9 to 2.7.10

This closes #7141

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-10 08:40:09 -05:00
Pierre Villard 4a0b38f4dd
NIFI-11405 Upgraded multiple drivers and test dependencies
This closes #7134

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-07 15:54:51 -05:00
Pierre Villard 9149bd47c4
NIFI-11406 Upgraded Google GCP from 26.4.0 to 26.12.0
- Upgraded Google Drive from Revision 20221219 to 20230306

This closes #7135

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-07 15:47:15 -05:00
krisztina-zsihovszki a36040fded
NIFI-11367 Database DATE value displayed incorrectly in record output if timezone is GMT-X
This closes #7103.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-04-02 22:47:42 +02:00
exceptionfactory f61d547e3e NIFI-11363 Fixed precision number handling in Flow Encryption Commands
Signed-off-by: Bence Simon <bsimon@apache.org>
This closes #11363
2023-03-30 16:29:52 +02:00
exceptionfactory 7cb20d6f7f
NIFI-11356 This closes #7096. Upgraded Nimbus JOSE JWT to 9.31
- Upgraded NiFi Framework from 9.24.3
- Upgraded NiFi Registry from 8.20
- Upgraded transitive msal4j dependency from 9.22
- Upgraded transitive hadoop-auth dependency from 9.8.1

Signed-off-by: Joe Witt <joewitt@apache.org>
2023-03-28 14:22:12 -07:00
exceptionfactory 26400fcbe9
NIFI-4890 Refactor OIDC with support for Refresh Tokens (#7013)
* NIFI-4890 Refactored OIDC with support for Refresh Tokens

- Implemented OIDC Authorization Code Grant Flow using Spring Security Filters
- Implemented OIDC RP-Initiated Logout 1.0
- Implemented OAuth2 Token Revocation RFC 7009 for Refresh Tokens
- Added OIDC Bearer Token Refresh Filter for updating application Bearer Tokens from Refresh Token exchanges
- Added configurable Token Refresh Window to application properties
- Removed original implementation and supporting classes

* NIFI-4890 Set Bearer Token expiration based on Access Token

* NIFI-4890 Corrected spelling and naming issues based on feedback

This closes #7013
2023-03-28 08:35:10 -04:00
Mark Payne 969fc50778
NIFI-11341 Fixed OutputStream.close() handling for Content Claims
Fixed issue in StandardContentClaimWriteCache in which inner OutputStream class did not have an idempotent close() method; as a result, the stream could be written to while already in use for another active FlowFile; fixed bug in ContentClaimInputStream in which skip() method ignored its own BufferedInputStream - this was discovered because it was causing failures in StandardProcessSessionIT; fixed bug in StandardProcessSessionIT in which the length of StandardContentClaim was being doubled because the OutputStream was setting the claim length but that is already handled at a lower level.

This closes #7087

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-27 16:05:50 -05:00
dependabot[bot] feebb2c399
NIFI-11338 Upgraded Spring Vault Core from 2.3.2 to 2.3.3
This closes #7083

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 19:49:32 -05:00
Joe Witt 58517e4dd0
NIFI-11329 Updated NAR Plugin to 1.5.0 and Commons Compress to 1.23.0
This closes #7077

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 12:28:40 -05:00
Peter Turcsanyi 00707f684f
NIFI-11323: Fixed last modified change detection of dynamicallyModifiesClasspath resources (#7069) 2023-03-22 09:15:48 -04:00
exceptionfactory 665b1696ef
NIFI-11268 Removed deprecated OpenPGP support from EncryptContent
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7030.
2023-03-13 09:20:33 +01:00
exceptionfactory 25d3fc27c7
NIFI-11262 Corrected scope for bcprov-jdk18on in nifi-security-kerberos
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #7022.
2023-03-09 06:11:54 -05:00
exceptionfactory 512155ba2f
NIFI-10975 Add Kubernetes Leader Election and State Provider (#6779)
* NIFI-10975 Added Kubernetes Leader Election and State Provider
- Added Kubernetes Leader Election Manager based on Kubernetes Leases
- Added Kubernetes State Provider based on Kubernetes ConfigMaps
- Added nifi-kubernetes-client for generalized access to Fabric8 Kubernetes Client
- Added nifi.cluster.leader.election.implementation Property defaulting to CuratorLeaderElectionManager
- Refactored LeaderElectionManager to nifi-framework-api for Extension Discovering Manager
- Refactored shared ZooKeeper configuration to nifi-framework-cluster-zookeeper

* NIFI-10975 Updated Kubernetes Client and StateMap
- Upgraded Kubernetes Client from 6.2.0 to 6.3.0
- Added getStateVersion to StateMap and deprecated getVersion
- Updated Docker start.sh with additional properties

* NIFI-10975 Corrected MockStateManager.assertStateSet()
* NIFI-10975 Upgraded Kubernetes Client from 6.3.0 to 6.3.1
* NIFI-10975 Corrected unregister leader and disabled release on cancel

* NIFI-10975 Corrected findLeader handling of Lease expiration
- Changed LeaderElectionManager.getLeader() return to Optional String

* NIFI-10975 Corrected StandardNiFiServiceFacade handling of Optional Leader
* NIFI-10975 Changed getLeader() to call findLeader() to avoid stale cached values
* NIFI-10975 Updated LeaderElectionCommand to run LeaderElector in loop
* NIFI-10975 Rebased on project version 2.0.0-SNAPSHOT

* NIFI-10975 Corrected Gson and AspectJ versions
- Updated versions to match current main branch and avoid reverting
2023-03-07 14:14:24 -05:00
exceptionfactory 9773105841
NIFI-11161 Simplified KeyedCipherPropertyEncryptor
- Replaced KeyedCipherProvider references with direct Cipher instantiation

This closes #6939
Signed-off-by: Paul Grey <greyp@apache.org>
2023-02-28 12:00:11 -05:00
exceptionfactory c8b5c0ce7b
NIFI-11220 Removed nifi-bootstrap-utils
- Replaced OSUtils.getProcessId() with java.lang.Process.pid() available on Java 9 and following

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6989.
2023-02-27 13:51:14 +01:00
exceptionfactory 48689a2567
NIFI-11195 Refactored Identity Mapping to nifi-security-identity
- Moved StringUtils from nifi-properties to nifi-property-utils
- Moved Peer Identity methods from CertificateUtils to specific Site-to-Site classes

Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes #6977.
2023-02-23 10:33:45 -05:00
dan-s1 22a4bce1ed
NIFI-11156 Updated SchemaValidator to include error message for ValidateXml
- Restores behavior so that XML parsing failure details will be included in FlowFile error attribute from ValidateXml

This closes #6970

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-02-18 10:28:50 -06:00
exceptionfactory 1156f4cbc5
NIFI-11165 This closes #6944. Removed optional jigsaw build profiles
- Removed jigsaw build profiles
- Removed Java 11 directories from assemblies

Signed-off-by: Joe Witt <joewitt@apache.org>
2023-02-13 13:01:58 -05:00
Joe Witt 43eab4c81d
NIFI-11103 prepping for 2.0.0 line 2023-02-09 15:32:53 -07:00
Joe Witt 4f91420cc1
NIFI-11139 Updated multiple Maven Plugins
This closes #6925

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-02-03 16:07:02 -06:00
exceptionfactory b3319c7119
NIFI-11141 This closes #6923. Upgraded Azure SDK BOM from 1.2.6 to 1.2.9
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-02-03 14:01:32 -07:00
exceptionfactory 0c676b9633
NIFI-11022 Added DecryptContent Compatibility Processors
- Added nifi-cipher-bundle with nifi-cipher-nar for new Processors
- Added DecryptContentCompatibilityMode Processor supporting PKCS5 and PKCS12 Password-Based Encryption Schemes
- Added DecryptContentEncoded Processor supporting NiFi Key Derivation Functions and associated formatting
- Added nifi-security-crypt-key module with Key Derivation Functions and Parameter Readers
- Added Additional Details documentation for Processors

This closes #6821
Signed-off-by: Paul Grey <greyp@apache.org>
2023-01-26 17:56:02 -05:00
exceptionfactory 4fe2c318e6
NIFI-11091 This closes #6884. Upgraded Google libraries-bom from 25.4.0 to 26.4.0
- Upgraded Google Drive library to v3-rev20221219-2.0.0
- Corrected test encryption keys to match expected encoding

Signed-off-by: Joe Witt <joewitt@apache.org>
2023-01-25 10:00:00 -07:00
dan-s1 53371844a4
NIFI-11035 Replaced remaining JUnit 4 assertions in nifi-commons with JUnit 5
- Replaced Groovy asserts with JUnit 5 assertions and Groovy shouldFail method Junit 5 with assertThrow method

This closes #6880

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-01-24 15:30:53 -06:00
exceptionfactory 4b97936d38 NIFI-10932 Changed PKCS12 KeyStore Type Provider to SunJSSE
- Changed from Bouncy Castle to Sun JSSE Provider for Key Stores to improve reading and writing Trust Stores formatted in PKCS12
- Updated TLS Toolkit Key Password handling to remove setting null for PKCS12

Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>

This closes #6881
2023-01-24 15:06:20 +00:00
Mark Payne 2ead9e5494 NIFI-10970: Added a count RecordPath function
Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>

This closes #6778
2023-01-23 20:32:09 +00:00
nathluu ea93dec079
NIFI-10481 Support disabling ZooKeeper Ensemble Tracking
This closes #6400

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-01-09 10:42:01 -06:00
dan-s1 3a60013876
NIFI-10754 Added getUri NIFI Expression Language function
This closes #6689

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-01-05 15:44:41 -06:00
exceptionfactory b556322749
NIFI-11005 Added Illegal and Redundant Import Modules to Checkstyle
- Updated impacted classes to remove redundant import lines
- Removed WebUtilsGroovyTest.groovy class due to use of internal sun.security classes

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6804.
2022-12-23 18:34:38 +01:00
Lehel Boér ed6ba53724 NIFI-10832: Create PutSalesforceRecord processor
This closes #6670.

Signed-off-by: Tamas Palfy <tpalfy@apache.org>
2022-12-13 18:59:41 +01:00
Matt Burgess b744fac479
NIFI-10956: Fix inference issues with mixed arrays (#6763) 2022-12-08 09:28:19 -05:00
exceptionfactory ab7ce58fb1
NIFI-10947 This closes #6758. Upgraded Apache Commons Net to 3.9.0
- Added Commons Net to managed dependencies in root Maven configuration
- Removed version references to 3.3 in MiNiFi and 3.6 in NiFi standard modules

Signed-off-by: Joe Witt <joewitt@apache.org>
2022-12-05 09:01:29 -07:00
mr1716 74b8d9296b
NIFI-10859 Upgraded metrics-jvm from 4.1.2 to 4.1.33
This closes #6721

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-11-29 08:38:18 -06:00
Joe Witt 75e7c9e47c
NIFI-10854-RC1 prepare for next development iteration 2022-11-22 19:22:47 -07:00
Joe Witt ec87bf93ad
NIFI-10854-RC1 prepare release nifi-1.19.0-RC1 2022-11-22 19:22:45 -07:00
exceptionfactory 5bc8e49c7a NIFI-10755 Refactored SSLContext creation using nifi-security-ssl
- Added TrustManagerBuilder to nifi-security-ssl
- Removed SslContextFactory and CertificateUtils from nifi-registry
- Refactored c2-client-http
- Refactored minifi-bootstrap
- Refactored nifi-site-to-site-client
- Refactored nifi-registry-client
- Refactored nifi-registry-framework
- Refactored nifi-toolkit-admin
- Refactored nifi-toolkit-cli

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6618.
2022-11-22 18:33:11 -05:00
exceptionfactory 7019e182b5
NIFI-10835 Improved performance of TestControlRate
- Added LongSupplier for TimedBuffer and ControlRate classes to support overriding System.currentTimeMillis()

This closes #6671
Signed-off-by: Paul Grey <greyp@apache.org>
2022-11-22 13:14:04 -05:00
Pierre Villard 7374f9d2b2 NIFI-10810 - Update HTTPAsyncClient to 4.1.5
This closes #6678

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-11-18 08:32:52 -05:00
exceptionfactory 3a536e261f NIFI-10798 Added Deprecation Logging for Java 8 on Startup
- Added deprecation warnings for NiFi, Registry, and MiNiFi
- Added RuntimeVersionProvider for shared reference to deprecated and minimum versions

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6648.
2022-11-10 21:48:50 -05:00
dan-s1 2afe2b36b9 [NIFI-10612] Initial check in of isJson code.
[NIFI-10612] Made suggested change to only test subject value where it is formatted like a Json array or object.

This closes #6574

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-11-02 15:46:40 -04:00
Hervé Boutemy fa1ea87a83
NIFI-10616 Resolve Reproducible ANTLR3 and JAXB Build Issues
This closes #6500

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-10-31 17:32:43 -05:00
Malthe Borch 831a11d0b5
NIFI-7823 Added groups mapping from OIDC token claim
This closes #6454

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-10-24 09:12:24 -05:00
exceptionfactory 3e9b7e27a5
NIFI-10674 Block evaluateELString from reading Parameters (#6562)
* NIFI-10674 Blocked access to Parameters from evaluateELString()
2022-10-21 14:19:41 -04:00
Joe Witt c155d29658
NIFI-10614 Upgraded Bouncy Castle from 1.70 to 1.71
- Blocked Bouncy Castle jdk15on dependencies
- Forced jdk18on dependencies and validated across entire build

This closes #6498

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-10-20 19:35:02 -05:00
exceptionfactory b753c1c72d NIFI-10625 Added support for HTTP/2 in Registry
- Added nifi-security-ssl for generalized SSLContext creation
- Removed static keystore and truststore test files from nifi-registry-jetty

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6514.
2022-10-17 23:52:09 -04:00
Matthew Burgess a76abef270 NIFI-10635: Fix handling of enums in PutDatabaseRecord
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6518.
2022-10-17 11:29:53 -04:00
exceptionfactory 0dc02e26d2 NIFI-10648 Upgraded Apache Commons Text to 1.10.0
- Added org.apache.commons.text.version property in root Maven configuration
- Removed direct version references in favor of managed dependency version

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6531.
2022-10-14 14:07:23 -04:00
Joe Gresock 283669ced3
NIFI-10633 Adding references to Key/Value Version 1 secrets engine in HashiCorp Vault documentation
This closes #6520

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-10-14 09:55:18 -05:00
Mark Bathori e87bced147
NIFI-10442: Create PutIceberg processor
This closes #6368.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-10-11 21:51:18 +02:00
Joe Witt c9ebdd2025
NIFI-10521-RC4 prepare for next development iteration 2022-10-03 10:59:36 -07:00
Joe Witt 109e54cd58
NIFI-10521-RC4 prepare release nifi-1.18.0-RC4 2022-10-03 10:59:34 -07:00
simonbence 94fdc794b0
NIFI-10497 Making RegistryClient an extension point (#6433)
- NIFI-10497 Mae FlowRegistryClient an extension point
2022-09-26 12:16:21 -04:00
Nathan Gough a52974780d
NIFI-10525 - Upgraded hadoop.version to 3.3.4.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6431.
2022-09-24 10:33:18 +04:00
Csaba Bejan 35fb66f50f
NIFI-10403 Add processor supporting the new BigQuery Write API
Signed-off-by: Arpad Boda <aboda@apache.org>

This closes #6344
2022-09-19 14:32:11 +02:00
Mark Payne ec09c56e93
NIFI-10508: When inferring data types for values, allow float and double to encapsulate byte/short/int/long values
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6421
2022-09-16 17:58:19 -04:00
Joe Gresock 3987d39cdc
NIFI-9401 Added HashiCorpVaultParameterProvider
- Refactored nifi-vault-utils to nifi-hashicorp-vault-api and nifi-hashcorp-vault modules
- Added HashiCorpVaultClientService and Standard implementation

This closes #6304

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-09-14 19:46:09 -05:00
exceptionfactory ae14ef2d1f
NIFI-10373 Set managed version for AWS 1 and 2 SDK
- Replaced individual AWS SDK versions with root managed dependency version
- Set AWS SDK 1 version to 1.12.299
- Set AWS SDK 2 version to 2.17.270
- Suppressed false positive dependency vulnerability for aws-sdk-swf-libraries

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6405.
2022-09-13 15:06:42 +02:00
exceptionfactory fa85a05a2b
NIFI-9374 Added Deprecation Logger
- Added nifi-deprecation-log module with interface and implementation using SLF4J
- Updated standard logback.xml with nifi-deprecation.log appender
- Updated NiFiLegacyCipherProvider with deprecation logging
- Set Size, Time Policy, and Total Size Limit for Deprecation Log

This closes #6300
Signed-off-by: Paul Grey <greyp@apache.org>
2022-08-23 11:25:03 -04:00
exceptionfactory 864036674e
NIFI-10244 Added nifi-web-client-api and implementation
- Added nifi-web-client implementation based on OkHttp
- Added WebClientServiceProvider Controller Service interface and implementation
- Corrected comments and added unmodifiableMap wrapper
- Added getHeaderNames() and corrected ProxyContext comments

This closes #6268
Signed-off-by: Paul Grey <greyp@apache.org>
2022-08-11 17:25:18 -04:00
Joe Witt 8c66bf948c
NIFI-10272-RC2 prepare for next development iteration 2022-07-27 13:24:09 -07:00
Joe Witt 8d256784d8
NIFI-10272-RC2 prepare release nifi-1.17.0-RC2 2022-07-27 13:24:06 -07:00
NissimShiman 74b25c7306
NIFI-10154 ReplaceText AdminYielding on long line (#6151)
* NIFI-10154 ReplaceText AdminYielding on long line

* NIFI-10154 code review changes

Signed-off-by: Otto Fowler <otto@apache.org>
2022-07-22 11:14:23 -04:00
exceptionfactory 047b3611bf NIFI-10233 Corrected Anonymous Authentication for HTTP Access
- Enabled Standard Anonymous Authentication Filter when unencrypted HTTP is enabled

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6209.
2022-07-14 23:27:54 -04:00
exceptionfactory a9b5bebb15 NIFI-10216 Refactored NiFi Web API Security Configuration
- Removed extension of deprecated WebSecurityConfigurerAdapter
- Moved Filter bean configuration associated configuration classes
- Set default Spring Security log level to INFO
- Adjusted CSRF Token Repository to leverage simplified RequestUriBuilder for retrieving allowed context paths

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6195
2022-07-13 19:11:03 -04:00
exceptionfactory 2a9139c57a NIFI-10219 Removed jna-platform from nifi-bootstrap-utils
- Removed OSUtils.getWindowsProcessId() which removes JNA-based retrieval of Process ID retrieval for Windows on Java 8
- Removing jna-platform from nifi-bootstrap-utils eliminates the library from lib/bootstrap and allows it as a dependency in lib/properties

This closes #6199

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2022-07-12 18:36:10 +00:00
Kevin Doran 9cf5c170b1
NIFI-9810 Deprecated RocksDB repository and moved to NAR
- Bump version to 6.29.5 for arm64 compatibility
- Extract RocksDBFlowFileRepository and supporting code to its own module
- Mark RocksDBFlowFileRepository as deprecated, both in code and documentation
- Log deprecation warning at startup if RocksDBFlowFileRepository is used
- Move native RocksDB info logs to NiFi debug level logs

This closes #6155

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-30 14:23:10 -05:00
Mark Payne 5e83bda9e9
NIFI-10167: Added advanced timing metrics for processors, such as CPU Usage, time reading/writing content repo, process session commit time, etc. Exposed via nifi.sh diagnostics and made configurable via nifi.properties
This closes #6156

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-30 10:50:24 -05:00
exceptionfactory 0de83292de NIFI-9849 Refactored SAML Support with Spring Security 5
- Updated SAML Authentication Configuration with Spring Security SAML 2 components
- Updated Administration Guide with REST Resources
- Replaced SAMLAccessResource methods with applicable Spring Security Filters
- Removed IDP Credential Service and supporting components
- Removed message.logging.enabled, metadata.signing.enabled, and signature.digest.algorithm properties
- Added Access Token Expiration resource method
- Removed Saml2AccessResource and replaced with Access Token Expiration to avoid unnecessary conflicts with SAML login consumer
- Corrected Resource URI handling to support proxy server access

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6149.
2022-06-28 13:57:35 -04:00
exceptionfactory b3b13a4ee0
NIFI-10145 Upgraded Hadoop from 3.3.2 to 3.3.3
- Excluded slf4j-reload4j implementation of Log4j 1
- Updated nifi-hive3-processors to leverage shared Hadoop version from 3.1.1
- Updated nifi-accumulo-bundle to leverage shared Hadoop version from 3.1.1
- Updated nifi-atlas-bundle to leverage shared Hadoop version from 3.3.2
- Updated nifi-spark-receiver to leverage shared Hadoop version from 3.3.2

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6139.
2022-06-28 16:19:38 +02:00
Mark Payne 1442dcef23
NIFI-10111: Added option to unpack NARs into an Uber Jar and updated stateless nifi to always use this.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6117
2022-06-24 13:12:14 -04:00
Mike Thomsen a3e8048b2d
NIFI-9981 Added support for Avro UUID types
This closes #6013

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-23 08:35:31 -05:00
exceptionfactory 5febd47c72
NIFI-10109 Changed HashMap to LinkedHashMap in JsonTreeRowRecordReader (#6112)
- Added unit test to confirm preservation of field ordering
2022-06-21 17:00:34 -04:00
exceptionfactory bd45eb4995 NIFI-10100 Upgraded Jersey to 2.35
- Replaced individual version references with Jersey BOM dependency

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6106.
2022-06-16 23:09:22 -04:00
UcanInfosec 1f2820a39a
NIFI-10119 Upgraded test hadoop-minikdc from 3.1.0 to 3.3.3
- Corrected JUnit 5 Assetions usage in KerberosUserIT

This closes #6129

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-15 13:14:59 -05:00
UcanInfosec d298a3ab83
NIFI-10082 Upgraded Google libraries BOM to 25.4.0
This closes #6125

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-14 14:26:14 -05:00
Emilio Setiadarma 24adc73862
NIFI-10012 Added XML and JSON FlowEncryptor implementations
This closes #6054

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-06 20:14:39 -05:00
Emilio Setiadarma 58f93d1f8b
NIFI-10085 Removed CryptoUtility.isUnlimitedStrengthCryptoSupported() and references
This closes #6098

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-06 08:49:39 -05:00
exceptionfactory 4b655ecb68
NIFI-9804 Added HTTP/2 support to Application Server
- Added nifi.web.https.application.protocols property
- Set default protocol to HTTP/1.1 and provided documentation for enabling HTTP/2
- Changed StandardALPNProcessor handshakeFailed log to debug

Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes #6093.
2022-06-03 11:54:21 -04:00
Matthew Burgess 639ab328c9
NIFI-10057: Add support for SQL Server's DATETIMEOFFSET type to ExecuteSQL processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6080.
2022-06-03 10:33:41 +02:00
Joe Gresock 48de70a568
NIFI-10071: Adding support for HashiCorp Vault K/V version 2 Secrets Engine (#6087) 2022-06-01 15:09:01 -04:00
Lehel Boér a4797327fc NIFI-7230: Created toInstant(), formatInstant(), toNanos() and toMicros() expression language functions.
This closes #5888.

Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
2022-05-23 19:19:06 +02:00
exceptionfactory e0976f42d3 NIFI-3869 Added HTTP/2 support to ListenHTTP and HandleHttpRequest
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6048.
2022-05-19 16:21:53 -04:00
exceptionfactory fe424a2d42
NIFI-9805 Refactored Distributed Cache Servers using Netty
- Added Map and Set Cache Servers based on nifi-event-transport components
- Removed custom servers and unused socket stream components
- Reduced duplication on protocol classes
- Added checks for readable bytes
- Added mark and reset handling for buffer reads

This closes #6040
Signed-off-by: Paul Grey <greyp@apache.org>
2022-05-18 12:40:16 -04:00
exceptionfactory 0c8305950c
NIFI-9987 Standardize Commons IO on version 2.11.0
- Removed version declarations from multiple modules
- Adjusted PutDynamoDB to remove catch for IOException not thrown in Commons IO 2.11.0

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6015.
2022-05-09 12:06:37 +02:00
exceptionfactory 272325cb4e NIFI-9988 Corrected Property Decryption for Authorizers and Providers
- Updated Protection Scheme Resolver to support both Name matching and Path matching

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6017.
2022-05-04 15:19:17 -04:00
exceptionfactory ac2d7d1e54
NIFI-9968 Added null check before System.setProperty() in test methods
- Resolves build failures on Java 17 where the original user.timezone property returns null from System.getProperty()

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6000
2022-04-27 14:38:33 -04:00
exceptionfactory 4f423a59ba
NIFI-9952 Upgraded Jackson BOM to 2.13.2.20220328
- Removed unnecessary references to jackson.version property
- Removed unnecessary dependency management references to Jackson libraries

This closes #5992

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-04-23 08:23:39 -04:00
exceptionfactory b288810316
NIFI-9943 Added Transform Provider to nifi-xml-processing
- Refactored TransformerFactory references using StandardTransformerProvider

This closes #5986
Signed-off-by: Paul Grey <greyp@apache.org>
2022-04-21 15:14:54 -04:00
Paul Grey c88c404dd8
NIFI-9826 Moved build locale settings from Maven to GitHub Workflow
- Adjusted Java 17 build locale settings
- Added test to verify match between System properties and build environment variables

This closes #5980

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-04-20 14:19:00 -05:00
exceptionfactory 18a4182cb0
NIFI-9883 Refactor Property Protection using Isolated ClassLoader (#5972)
* NIFI-9883 Refactored property protection to isolated ClassLoader

- Added nifi-property-protection-loader for abstracting access to implementation classes using ServiceLoader
- Updated Authorizer and Login Identity Provider configuration using isolated ClassLoader
- Updated NiFi Properties Loader using isolated ClassLoader
- Updated nifi-assembly to place property protection dependencies in lib/properties directory
- Updated and refactored unit tests
- Corrected LoginIdentityProviderFactoryBean getObject() Type
2022-04-19 11:08:04 -04:00
Joe Witt 84340962d4
NIFI-9927 Initial Commit to clean up all extraneous version/group declarations.
This closes #5969

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-04-16 12:05:07 -04:00
exceptionfactory 15f7590f7a
NIFI-9901 Added nifi-xml-processing to nifi-commons
- Refactored XML parsing to use providers from nifi-xml-processing
- Configured spotbugs-maven-plugin with findsecbugs-plugin in nifi-xml-processing
- Disabled Validate DTD in default configuration for EvaluateXPath and EvaluateXQuery
- Replaced configuration of DocumentBuilder and streaming XML Readers with shared components
- Removed XML utilities from nifi-security-utils
- Moved Commons Configuration classes to nifi-lookup-services

This closes #5962
Signed-off-by: Paul Grey <greyp@apache.org>
2022-04-13 19:56:57 -04:00
Bence Simon 9c71d83c5c NIFI-9615 Extending capabilities of NAR provider with restraining, conflict resolution strategy and refactors to make it more flexible 2022-04-06 11:54:31 -04:00
exceptionfactory e052cf81ba
NIFI-9872 Upgraded azure-sdk-bom to 1.2.0 for Sensitive Properties
- Replaced individual Azure dependencies with azure-sdk-bom in nifi-property-protection-azure
- Removed woodstox-core and stax2-api exclusions
- Refactored TemplateDeserializer test class to avoid dependency on specific XML implementation

This closes #5929

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-04-04 21:26:32 -04:00
Joe Witt 873d25585c
NIFI-9780 Merge branch 'NIFI-9780-RC3' 2022-04-04 08:35:23 -07:00
exceptionfactory 0fa4490a98
NIFI-9842 Refactored nifi-commons using JUnit 5
- Refactored nifi-bootstrap using JUnit 5
- Refactored nifi-maven-archetypes using JUnit 5
- Refactored nifi-stateless using JUnit 5

This closes #5912

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-28 16:00:23 -04:00
Joe Witt 0419dc2939
NIFI-9780-RC3 prepare for next development iteration 2022-03-21 11:58:27 -07:00
Joe Witt b019a9191f
NIFI-9780-RC3 prepare release nifi-1.16.0-RC3 2022-03-21 11:58:09 -07:00
Tamas Horvath a8cd7e5200
NIFI-9786 Added debug to KeyStoreUtils.isStoreValid
This closes #5884

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-21 10:48:00 -05:00
Mike Thomsen 2bb1a06a8c
NIFI-9788 Updated commons-codec to 1.15 across all modules.
This closes #5881

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-21 09:13:05 -05:00
在红尘中成仙 8d5de6b292
NIFI-9818 fix flaky tests
This closes #5883

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-21 08:23:40 -04:00
exceptionfactory 412e21be97
NIFI-9281 Enabled building on Java 17
- Added Ubuntu Zulu JDK 17 GitHub build
- Adjusted MiNiFi C2 FileSystemConfigurationCache test to avoid using environment variables
- Adjusted MiNiFi StatusLogger and StatusLoggerTest to avoid overriding private logger
- Adjusted failure reason attribute check in TestGetIgniteCache
- Adjusted TestRangerAuthorizer and TestRangerNiFiAuthorizer to avoid checking nested exceptions
- Adjusted encrypt-config TestUtil to avoid unnecessary comparison of different types
- Disabled Javascript tests on Java 15 and higher
- Disabled several Hive 3 tests on Java 17 for StringInternUtils illegal access
- Refactored nifi-enrich-processors tests to use Mockito without Powermock
- Refactored nifi-toolkit-tls tests to avoid illegal reflective access
- Removed deprecated X509Certificate test in CertificateUtilsTest
- Removed kryo serialization from nifi-site-to-site-client test
- Updated TestHashContent to use SHA-1 instead of SHA for hash algorithm
- Upgraded maven-war-plugin from 2.5 to 3.3.2
- Upgraded nifi-graph-bundle dependencies from Groovy 2.5.14 to 3.0.8
- Upgraded QuestDB from 4.2.1 to 6.2.1 in nifi-framework-core

This closes #5870

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-17 14:08:38 -04:00
Joe Witt 72fadf9e51
NIFI-9780 fixing many inconsistent pom project declarations and updated org apache parent version 2022-03-10 11:15:16 -07:00
exceptionfactory 859d5fe8cf NIFI-9785 Improved Login Credentials Writer File Handling
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5856.
2022-03-10 11:47:32 -05:00
Mark Payne 6a1c7c72d5
NIFI-9781: Fix handling when selecting array element via QueryRecord
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5853.
2022-03-09 13:07:15 -05:00
zhangcheng a85cafe771
NIFI-9064:Support Oracle timestamp when `Use Avro Logical Types` is true for ExecuteSQLRecord and QueryDatabaseTableRecord
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5807.
2022-03-02 11:18:58 -05:00
Mike Thomsen 4b41570d17
NIFI-9714 Added overloaded toMap to MapRecord that can convert sub-records into maps.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5784
2022-02-28 09:12:51 -05:00
exceptionfactory 49d1c747ca
NIFI-9724 Added set-sensitive-properties-algorithm command
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5801.
2022-02-25 11:03:55 -05:00
Mark Payne ce0122bd25
NIFI-9691: Added ForkEnrichment, JoinEnrichment processors
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5772.
2022-02-22 18:54:32 -05:00
exceptionfactory 45f8795177
NIFI-9711 Added support for flow.json.gz in SetSensitivePropertiesKey
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5783.
2022-02-22 12:58:30 -05:00
Nathan Gough 7ef2fd2986
NIFI-7333 Added OIDC trust store strategy property
This closes #5753

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-18 15:19:22 -05:00
exceptionfactory fe1139b8bb
NIFI-9692 Upgraded Apache Commons Lang3 to 3.12.0
- Moved commons-lang3 version management to root Maven configuration
- Refactored limited usage of StringUtils is nifi-reporting-utils to remove commons-lang3
- Refactored limited usage of StringUtils in nifi-websocket-processors to remove commons-lang3

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5773
2022-02-16 09:28:21 -05:00
Steve Lawrence 4815df29db
NIFI-6740: Add configuration options to specify NiFi/Bootstrap communication ports
The NiFi and NiFi Bootstrap processes both bind to random ephemeral
ports to allow for inter-process communication (e.g. shutdown, port,
ping, etc.). However, the randomness of these ephemeral ports can pose
challenges for some security policies and firewall rules.

This adds two configuration options, nifi.bootstrap.listen.port and
nifi.listener.bootstrap.port, that allow an administrator to define
which ports the two processes should bind to for this communication,
making it easier to define security policies. The options default to
zero to maintain the current ephemeral port behavior.

NIFI-6740: Add configuration options to specify NiFi/Bootstrap communication ports

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5746
2022-02-15 11:46:21 -05:00
exceptionfactory b09a778d90
NIFI-9685 Upgraded JNA to 5.10.0
- Upgraded nifi-bootstrap-utils JNA from 4.4.0 to 5.10.0
- Upgraded nifi-windows-event-log-processors JNA from 4.5.2 to 5.10.0
- Upgraded nifi-toolkit-cli JLine from 3.5.2 to 3.21.0

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5766.
2022-02-15 10:52:54 +01:00
exceptionfactory ba0bf2ede0
NIFI-9650 Upgraded OkHttp from 4.9.2 to 4.9.3
- Added okhttp-bom to root Maven configuration
- Removed repeated okhttp.version property from specific dependencies

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5737.
2022-02-02 11:45:18 +01:00
exceptionfactory e2243500d3
NIFI-9618 Upgraded Checkstyle to 9.2.1
- Corrected several missing return statements
- Increased Maven heap size to 3 GB for ci-workflow

This closes #5700

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-21 20:08:16 -06:00
sedadgn e603b0179b NIFI-7865 amqp$header is splitted in the wrong way for "," and "}"
This PR introduces 2 new properties for the ConsumeAMQP processor
And one new property for PublishAMQP

This allows to configure the processors to use escaping for commas and to consistently not use curly braces in the amqp$header attribute.

The default values ensure backwards compatibility.

This closes #5458.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-01-18 16:45:45 -05:00
exceptionfactory 2ffd4a5a9a
NIFI-9438 Refactored sensitive-property-provider to multiple modules
- Added nifi-property-protection-api with provider interfaces
- Added nifi-property-protection-factory with implementation references
- Added ProtectionSchemeResolver for abstracting conversion from command arguments
- Refactored PropertyProtectionScheme to package private visibility
- Refactored multiple unit test and removed provider integration tests
- Renamed AESSensitivePropertyProvider to AesGcmSensitivePropertyProvider
- Added getSupportedProtectionSchemes() to StandardProtectionSchemeResolver
- Updated command argument descriptions for protection schemes to include supported values

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5650.
2022-01-13 13:18:04 -05:00
exceptionfactory 5054fc3ac6
NIFI-9564 This closes #5652. Removed unnecessary logback-classic test dependencies
Signed-off-by: Joe Witt <joewitt@apache.org>
2022-01-12 10:39:32 -07:00
exceptionfactory 5832dff25e
NIFI-9556 Upgraded Apache HttpClient to 4.5.13
- Upgraded Apache HttpCore to 4.4.15
- Added dependency management declarations in root Maven configuration for HttpClient and HttpCore
- Removed version numbers from multiple modules

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5647.
2022-01-08 11:06:07 +01:00
Mark Payne d9b863a84b
NIFI-9549: Delegate NonFlushableOutputStream write methods to wrapped OutputStream
Ensure that we delegate calls to write(byte[]) and write(byte[], int, int) to the underlying OutputStream for NonFlushableOutputStream, instead of allowing FilterOutputStream to iterate over every byte

This closes #5642

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-06 22:46:55 -06:00
exceptionfactory 12ba579b8f NIFI-9539 Upgraded Netty from 4.1.69 to 4.1.72
- Added Netty 4 BOM to root Maven configuration
- Removed individual Netty 4 dependency overrides
- Removed commented dependencies and unnecessary declarations from MiNiFi

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5635.
2022-01-05 18:11:03 -05:00
exceptionfactory 81e57c9651
NIFI-9535 Upgraded Spring Framework to 5.3.14
- Centralized Spring Framework and Spring Security versions using BOM dependencies
- Upgraded Spring Security from 5.5.2 to 5.6.1
- Upgraded Spring Boot from 2.5.5 to 2.5.8 in Registry
- Upgraded Spring Integration from 5.5.2 to 5.5.7

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5631.
2022-01-05 15:30:03 +01:00
Pierre Villard 90930ca197
NIFI-9476 - Fix QueryRecord when no result and with array type column
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5594
2021-12-15 17:37:24 -05:00
exceptionfactory b7ad1f924d
NIFI-9457 Support microseconds for String Timestamps in PutKudu
- Implemented override for Timestamp Record Field Type format handling to add support for optional microseconds
- Added FieldConverter and ObjectTimestampFieldConverter implementation for generalized Timestamp parsing using DateTimeFormatter
- Updated PutKudu unit tests for standard Timestamp and Timestamp with microseconds

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5589.
2021-12-10 10:31:52 +01:00
exceptionfactory 563df24067
NIFI-9448 Improved S2S HTTP Extend Transaction Exception Handling
- Refactor background transaction extension to ExtendTransactionCommand
- Avoid closing S2S HTTP client for IllegalStateExceptions
- Avoid creating additional S2S HTTP client instance for transaction extension commands
- Add check for extend transaction requests received in client test class
- Add null check for Peer Persistence implementation in PeerSelector

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5577.
2021-12-08 09:33:55 -05:00
Mark Payne 90b39b593a
NIFI-9069 Changed framework dataflow serialization to support JSON
- Changed framework so that it serializes the dataflow into a VersionedDataflow using JSON as well as XML, and prefers the JSON representation on load, if it's available. This also results in the need for the cluster protocol to exchange its representation of the dataflow to using JSON. Rather than re-implementing all of the complex logic of Flow Fingerprinting, updated to just inherit the cluster's flow.
- Moved logic to synchronize Process Group with Versioned Process Group into a new ProcessGroupSynchronizer class instead of having all of the logic within StandardProcessGroup
- Reworked versioned components to use an instance id.
- Renamed StandardFlowSynchronizer to XmlFlowSynchronizer; introduced new StandardFlowSynchronizer that delegates to the appropriate (Xml or Versioned)FlowSynchronzer
- Updated to allow import of VersionedProcessGroup even if not all bundles are available - will now use ghost components
- Introduced a VersionedDataflow object to hold controller-level services, reporting tasks, parameter contexts, templates, etc.
- Allow mutable requests to be made while nodes are disconnected. Also fixed issue in AbstractPolicyBasedAuthorizer that caused ClassNotFoundException / NoClassDefFoundError if the authorizations were changed and then a node attempts to rejoin the cluster. The Authorizer was attempting to use XmlUtils, which is in nifi-security-utils and so so by madking nifi-security-utils a provided dependency of nifi-framework-api, but this doesn't work, because nifi-framework-api is loaded by a higher-level classloader, so the classloader that loads AbstractPolicyBasedAuthorizer will never have the appropriate classloader to provide nifi-security-utils. Addressed this by copying the code for creating a safe document builder from XmlUtils to AbstractPolicyBasedAuthorizer.
- Fixed bug that occurred when importing a Process Group that has 2 parameter contexts, one inheriting from another, where neither is pre-defined in the existing flow
- Fixed bug that was encountered when Updating a Versioned Process Group where one version had a disabled processor and the other had the processor running.
- Increased system-tests workflow timeout to 120 minutes
- Added additional exception handling to system tests

This closes #5514

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-12-02 15:10:38 -06:00
markap14 839fbf7d19
NIFI-9382: Created a new ClassloaderIsolationKey mechanism by which H… (#5533)
* NIFI-9382: Created a new ClassloaderIsolationKey mechanism by which Hadoop related processors (and potentially others) can indicate that they need full classloaders to be cloned but can share with other instances in certain circumstances
- Added system tests

* NIFI-9382: Renamed interface based on review feedback

* NIFI-9382: Removed ReentrantKerberosUser.
2021-11-22 14:55:19 -05:00
exceptionfactory 605346ae0f
NIFI-9291 Added NiFi HTTP request logging
- Added nifi.web.request.log.format property
- Added Filters to set and retrieve authenticated username for logging

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5527.
2021-11-18 10:30:50 -05:00
Matthew Burgess fe4161b1c2
NIFI-8392: Translate JDBC CHAR type to RecordFieldType STRING
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5506.
2021-11-14 16:03:17 +01:00
Joe Witt c055895952
NIFI-9344-RC3 prepare for next development iteration 2021-11-03 08:53:37 -07:00
Joe Witt 7fdc07cccd
NIFI-9344-RC3 prepare release nifi-1.15.0-RC3 2021-11-03 08:53:32 -07:00
exceptionfactory 60d6d469bf
NIFI-9342 Upgraded to Netty 3.10.6 and 4.1.69
- Replaced Netty 3.6.9 and 3.7.1 with 3.10.6
- Replaced Netty 4.1 with 4.1.69

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5490.
2021-10-28 09:34:31 +02:00
exceptionfactory a9af5d3465
NIFI-9332 Corrected SecureHasher conditional test methods
- Removed GroovyTestCase super class
- Replaced shouldFail() with assertThrows()

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5481.
2021-10-26 09:59:08 +02:00
Mike Thomsen 8d513c5ed3
NIFI-9080 Converted nifi-commons to use JUnit 5
This closes #5332

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-10-25 21:07:54 -05:00
Nathan Gough 309ab8f4ab
NIFI-8792 - Modified ListenRELP to use Netty
- Refactored RELP encoders and decoders

This closes #5398

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-10-25 15:53:58 -05:00
Joe Gresock 207da405ee NIFI-9263: Using secret-per-context model for Stateless Vault PVP
Rebasing after PVP refactor

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5427.
2021-10-21 16:43:53 -04:00
exceptionfactory 70274ebcfc
NIFI-9306 Upgraded Spring Framework to 5.3.11
- Upgraded Spring Security from 5.5.1 to 5.5.2
- Upgraded Spring Boot from 2.5.2 to 2.5.5 in Registry
- Upgraded Spring Data Redis from 2.5.3 to 2.5.5 in Redis modules

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5463.
2021-10-19 11:26:37 +02:00
Joe Gresock f3136f07eb
NIFI-9221: Add AWS SecretsManager Sensitive Props Provider
This closes #5410

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-10-08 11:44:43 -05:00
exceptionfactory 70432507df
NIFI-6617 Refactored Encrypted Repository configuration
- Updated documentation with new properties
- Refactored cipher operations to common RepositoryEncryptor classes
- Abstracted record metadata serialization for better compatibility

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5407.
2021-10-08 11:57:10 -04:00
exceptionfactory 8d0eec2d62
NIFI-9266 Added Azure Key Vault Secret SPP
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5435.
2021-10-08 10:01:09 +02:00
exceptionfactory 4bcd03024a
NIFI-9283 Excluded Log4j 1.2 and upgraded Log4j2 to 2.14.1
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5440.
2021-10-06 11:17:36 -04:00
exceptionfactory defea61075
NIFI-9253 Corrected SSLSocketChannel.available() for TLSv1.3
- Added unit tests to reproduce issues with available() method
- Changed available() to return size of application buffer
- Removed unused isDataAvailable()
- Refactored unwrap handling to read from channel for buffer underflow

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5421.
2021-10-02 18:31:05 +02:00
exceptionfactory ae0154de5a
NIFI-9268 Upgraded OkHttp from 4.9.1 to 4.9.2
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5429.
2021-10-02 18:26:40 +02:00
exceptionfactory 00b2358c27
NIFI-9252 Removed unused nifi.sensitive.props.provider Property
- Removed nifi.minifi.sensitive.props.provider Property from MiNiFi
- Removed property from example NiFi properties files
- Removed provider from MiNiFi SensitivePropsSchema
- Removed BC provider value from MiNiFi test cases

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5422.
2021-10-01 22:20:13 -04:00
exceptionfactory a94b47ecf8
NIFI-9254 Updated default Stateless Sensitive Property configuration
- Set NIFI_PBKDF2_AES_GCM_256 as property encryption method
- Replaced static default sensitive properties key with random UUID
- Added unit test for PropertiesFileEngineConfigurationParser
- Added random encryption key generation method
- Changed Stateless to use PropertyEncryptionMethod enum

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5424
2021-10-01 15:55:06 -04:00
exceptionfactory e16a6c2b89 NIFI-9241 Refactored CSRF mitigation using random Request-Token
- Replaced use of Authorization header with custom Request-Token header for CSRF mitigation
- Added Request-Token cookie for CSRF mitigation
- Replaced session storage of JWT with expiration in seconds
- Removed and disabled CORS configuration
- Disabled HTTP OPTIONS method
- Refactored HTTP Proxy URI construction using RequestUriBuilder

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5417.
2021-09-30 20:36:15 -04:00
Paul Grey 720e7ef353
NIFI-8919 - Refactor Distributed Cache Clients using Netty
This closes #5311

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-09-27 21:00:58 -05:00
Peter Gyori 38d26dafa3
NIFI-9192: ResultSetRecordSet considers value of useLogicalTypes flag when determining the object's schema.
NIFI-9192: ResultSetRecordSet consider value of useLogicalType flag when sqlType is not handled separately (default) and readerSchema is not null

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: Unit tests added to test the use of useLogicalType flag

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: ResultSetRecordSet consider not using logical type when sqlType is OTHER

NIFI-9192: Fixed checkstyle violations

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: Renamed JdbcCommon.getUseLogicalTypes to isUseLogicalTypes

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: Added unit tests for Array types with and without using logical types

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: Review comments considered and unit tests refactored

NIFI-9192: Added further unit tests

NIFI-9192: Minor modifications based on review comments

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5388
2021-09-27 16:27:39 -04:00
exceptionfactory 84dbf915a9 NIFI-9060 Refactored HTTP Cookie Path Handling
- Implemented ApplicationCookieService for adding and retrieving HTTP Cookies
- Added getCookieResourceUri() leveraging allowed proxy headers to support optional Cookie Paths
- Refactored Access Resources to use ApplicationCookieService for processing
- Changed __Host- prefix to __Secure- prefix for Bearer Token cookie to support Cookie Path processing
- Removed unnecessary jetty-http dependency from nifi-web-api
- Corrected NiFi path references in JavaScript to support prefixed paths

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5329.
2021-09-23 18:03:48 -04:00