Commit Graph

653 Commits

Author SHA1 Message Date
markap14 73f88b3239
NIFI-9061: Eliminated the nifi.cluster.node.protocol.threads property in favor of nifi.cluster.node.protocol.max.threads property so that we can properly scale out the number of threads used for HTTP request replication. Implementing a caching mechanism for creating the DateTimeFormatter used by TimeAdapter in order to improve performance when parsing timestamps in web requests. Implementing caching logic for caching the number of characters that can rendered without needing an ellipsis for some components in the UI (#5316)
This closes #5316
2021-09-13 14:36:35 -04:00
denes 3fbd9c9e14
NIFI-9201 NullPointerException in AbstractKerberosUser if the tgt is not renewable (#5383) 2021-09-13 11:04:39 -04:00
Matthew Burgess 98e3b465c3
NIFI-8376: Gracefully handle SQL exceptions in ResultSetRecordSet
This closes #4951

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2021-09-08 09:11:26 -04:00
exceptionfactory f5a421ccb1
NIFI-9067 Upgraded JAXB Runtime from 2.3.0 to 2.3.5
- Replaced old com.sun.xml.bind:jaxb-impl and jaxb-core with current org.glassfish.jaxb:jaxb-runtime
- Replaced old javax.xml.bind:jaxb-api with current jakarta.xml.bind-api
- Removed unnecessary dependency references to javax.activation-api

This closes #5320

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2021-09-07 13:27:53 -04:00
exceptionfactory 20804ff2b1
NIFI-8749 Removed implicit time zone conversion to GMT
- Updated DataTypeUtils.getDateFormat() to use system default time zone
- Updated Record Path Guide to match Expression Language Guide regarding default time zone
- Updated impacted unit tests to expect localized dates instead of dates converted to GMT

NIFI-8749 Set user.timezone in TestRecordPath to avoid unexpected date conversions

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5361
2021-08-31 16:07:53 -04:00
Matthew Burgess dfa6592a47
NIFI-8749 Revert "Removed implicit time zone conversion to GMT"
This reverts commit 758e1850ed.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>
2021-08-31 13:33:53 -04:00
exceptionfactory 758e1850ed
NIFI-8749 Removed implicit time zone conversion to GMT
- Updated DataTypeUtils.getDateFormat() to use system default time zone
- Updated Record Path Guide to match Expression Language Guide regarding default time zone
- Updated impacted unit tests to expect localized dates instead of dates converted to GMT

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5305
2021-08-31 10:40:57 -04:00
Emilio Setiadarma 2ab23efb74
NIFI-6616 Added GCP Sensitive Property Provider
This closes #5242

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-30 19:15:34 -05:00
Joe Gresock 4a3e81531b
NIFI-8490: Adding inherited parameter contexts (#5072)
- Allowing inherited param contexts on creation, updating PC authorization
2021-08-30 10:44:31 -04:00
Bryan Bende e436381c3a
NIFI-8975 Integrate KerberosUserService into HBase processors/services
This closes #5322

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-27 17:41:58 -05:00
exceptionfactory 0cf09840ea
NIFI-9082 Add nifi.zookeeper.jute.maxbuffer property (#5335)
* NIFI-9082 Added nifi.zookeeper.jute.maxbuffer property

- Updated ZooKeeperStateProvider to check jute.maxbuffer instead of hard-coded number
- Updated internal ZooKeeperClientConfig class to include new juteMaxBuffer property

* NIFI-9082 Updated jute.maxbuffer documentation and added hexadecimal representation

- Renamed DefaultWatcher to NoOpWatcher
- Removed unnecessary validateDataSize() from createNode()
2021-08-26 21:12:34 +02:00
Paul Grey aa5babbb97
NIFI-9073 - Limit Surefire Output during Build
This closes #5325

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-24 10:37:07 -05:00
exceptionfactory a652280fbb NIFI-8766 Implemented RS512 Algorithm for JWT Signing
- Replaced per-user symmetric-key HS256 with shared and rotated RSA asymmetric-key RS512 implementation
- Added nifi.security.user.jws.key.rotation.period property for RSA Key Pair rotation
- Added JSON Web Tokens section to Administration Guide
- Implemented persistent storage of RSA Public Keys for verification using Local State Manager
- Implemented JWT revocation on logout with persistence using Local State Manager
- Refactored JWT implementation using Spring Security OAuth2 and Nimbus JWT
- Refactored Spring Security Provider configuration using Java instead of XML
- Removed H2 storage of per-user keys
- Upgraded nimbus-jose-jwt from 7.9 to 9.11.2

NIFI-8766 Corrected AuthenticationException handling in AccessResource.getAccessStatus

- Added nifi.user.security.jws.key.rotation.period to default nifi.properties
- Updated logging statements and clarified configuration and method documentation

NIFI-8766 Changed Algorithm to PS512 and updated documentation

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5262.
2021-08-19 12:26:12 -04:00
Lehel Boér 9bcbf83e5a NIFI-8752: Automatic diagnostic at NiFi restart/stop
This closes #5195.

Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
2021-08-19 18:09:58 +02:00
Bryan Bende 21c2fb95d3
NIFI-8973 Implement KerberosUserService API and keytab, password, and ticket cache implementations
NIFI-8974 Integrate KerberosUserService with HDFS processors

NIFI-8980 Integrate KerberosUserService with Kafka 2.6 processors
- Introduced SelfContainerKerberosUserService to restrict which impls can be used with Kafka
- Add variations of KerberosUser doAs that allow setting the context ClassLoader
- Add additional unit tests for configurations

This closes #5277

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-18 19:54:56 -05:00
Emilio Setiadarma 714670b8e6
NIFI-6615 Added Azure Key Vault Key Sensitive Property Provider
This closes #5274

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-13 10:54:48 -05:00
Mark Payne fb4edfa0d6
NIFI-8950: Added ability to optionally use a file-system backed Content Repository for Stateless NiFi
This closes #5254

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-11 12:01:07 -05:00
exceptionfactory f27cd012f3
NIFI-9035 Refactored isKeystoreValid() to avoid NullPointerException
- Removed unnecessary debug logging from StandardTlsConfiguration
- Replaced internal string labels with StoreType enum

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5297
2021-08-09 15:55:20 -04:00
Timea Barna 462306369f NIFI-8668 ConsumeAzureEventHub NiFi processors need to support storage SAS token authentication
This closes #5136.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-08-09 12:57:56 +02:00
Joe Gresock cc1e9665cd
NIFI-8696: Added HashiCorp Vault KeyValue SPP
This closes #5255

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-06 07:39:34 -05:00
Mark Payne 787f16da3f
NIFI-9013: Moved nifi-properties-loader and nifi-properties dependencies from data-provenance-utils to the repository where they are needed
This closes #5283

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2021-08-05 20:36:01 -04:00
exceptionfactory d93e9f152c
NIFI-8986 Upgraded Commons Compress to 1.21
- Upgraded direct and transitive dependencies from 1.20 and earlier to 1.21

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5273.
2021-08-03 15:52:45 +02:00
Joe Gresock 2daac5714a
NIFI-8695: Adding context to sensitive property providers
This closes #5206

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-07-27 15:37:29 -05:00
exceptionfactory e16bf644e3
NIFI-8948 Upgraded Spring to 5.3.9 and Security to 5.5.1
- Upgrades Spring Framework from 5.3.8 to 5.3.9
- Upgrades Spring Security from 5.4.6 to 5.5.1
- Upgrades Spring Boot from 2.5.1 to 2.5.2 in Registry
- Upgrades Spring Data Redis from 2.5.1 to 2.5.3

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5252.
2021-07-27 10:21:27 +02:00
Emilio Setiadarma d4a560c59a
NIFI-6325 Added AWS KMS Sensitive Properties Provider
This closes #5202

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-07-22 08:24:52 -05:00
Bryan Bende 74c0a91b6c
NIFI-8933 Configure Jersey's ObjectMapper to ingnore unknown fields
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5236.
2021-07-21 11:46:45 +02:00
exceptionfactory c668d3df1b NIFI-8782 Added Rate-Limiting for Access Token Requests
- Added Jetty DoSFilter configured for /access/token
- Added nifi.web.max.access.token.requests.per.second property with default value of 25

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5215.
2021-07-14 14:24:45 -04:00
Joe Witt 7356332852
Merge branch 'NIFI-8767-RC2' 2021-07-14 09:07:22 -05:00
exceptionfactory 05d7867a16
NIFI-8768 Added toLocalDate() for convertType() handling of DATE fields
- Updated PutKudu to use DataTypeUtils.toLocalDate() for DATE fields
- Updated PutDatabaseRecord to remove convertDateToLocalTZ() since convertType() uses toLocalDate()
- Updated PutElasticsearchHttpRecord to use default time zone format for DATE fields
- Updated WriteXMLResult to use default time zone format for DATE fields
- Updated WriteJsonResult to use default time zone format for DATE fields
- Updated AvroTypeUtil to use toLocalDate() for logical DATE fields
- Updated JdbcCommon to avoid conversion to UTC for logical DATE fields
- Updated Processor and RecordReader unit tests for consistency in DATE comparison

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5210
2021-07-13 16:17:36 -04:00
Joe Witt 97feacc181
NIFI-8767-RC2 prepare for next development iteration 2021-07-10 12:17:09 -07:00
Joe Witt fcbf1d5f97
NIFI-8767-RC2 prepare release nifi-1.14.0-RC2 2021-07-10 12:17:05 -07:00
exceptionfactory aedbd0d753 NIFI-8511 Added KeyStore implementation of KeyProvider
- KeyStoreKeyProvider supports PKCS12 and BCFKS
- Refactored KeyProvider and implementations to nifi-security-kms
- Updated Admin Guide and User Guide with KeyStoreKeyProvider details

NIFI-8511 Improved documentation and streamlined several methods

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5110.
2021-07-09 12:12:25 -04:00
Peter Turcsanyi f11c99f9ec
NIFI-8475: Fixed EL query parser when input contains multiple escaped expressions
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5031
2021-06-25 13:59:11 -04:00
Joe Gresock 726082ffa6
NIFI-8447 Added HashiCorp Vault Transit Sensitive Properties Provider
- Added default bootstrap-hashicorp-vault.conf
- Updated Toolkit Guide documentation with HashiCorp Vault properties

This closes #5154

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-24 22:06:42 -05:00
exceptionfactory 6a83115d6a NIFI-7468 Updated SSLSocketChannel to support TLS 1.3
- Handling additional FINISHED Handshake Status for TLS 1.3 Post-Handshake Messages per RFC 8446 Section 4.6
- Removed clearing buffers after handshake to avoid losing packets
- Updated read() method to check Handshake Status after SSLEngine.unwrap()
- Changed SSLSocketChannelSender to close SSLSocketChannel before other resources
- Added ChannelStatus enum and convenience logging methods for tracing status
- Added unit tests for TLS 1.2 and 1.3 using Netty server and client handlers

NIFI-8704 Updated netty-handler to 4.1.65.Final

NIFI-7468 Corrected SSLSocketChannel.read() to return byte read

NIFI-7468 Adjusted comment formatting

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5152.
2021-06-22 22:29:16 -04:00
exceptionfactory 2efc2b6b8b
NIFI-8718 Upgraded Apache Commons IO to 2.10.0
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5169
2021-06-21 11:29:27 -04:00
rhavermans 9fd8042cfb
NIFI-8664 Update json-path lib from 2.4.0 -> 2.6.0
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5132
2021-06-21 11:01:10 -04:00
exceptionfactory 9be9b9dbe2
NIFI-8708 Upgraded Spring Framework to 5.3.8 for several extensions
- nifi-vault-utils
- nifi-email-bundle
- nifi-redis-bundle

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5163.
2021-06-16 18:18:54 +02:00
markap14 96a8b2d090
NIFI-8666: Allow users to escape parameter names in Expression Language using quotes. (#5133) 2021-06-15 09:37:34 -04:00
Peter Turcsanyi 48befe22f6
NIFI-8645: Disable LongRunningTaskMonitor by default
This closes #5111

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-14 08:50:34 -05:00
Joe Gresock 6f04b45773
NIFI-8643: Correcting nifi.cluster.load.balance.address in properties (#5146)
NIFI-8643: Correcting nifi.cluster.load.balance.address in properties
2021-06-11 16:58:20 -04:00
Joe Gresock 1ccc4fbb0f
NIFI-8651: Refactor Sensitive Properties Providers for extension
This closes #5131

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-11 15:44:37 -05:00
Mark Payne 64f600d0ce
NIFI-8689: This closes #5150. Avoid flushing the socket buffer unnecessarily when sending a series of FlowFiles via site-to-site
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-06-11 13:15:03 -07:00
exceptionfactory db40989b48 NIFI-8516 Enabled HTTPS and Single User Authentication in default configuration
- Set default HTTPS Port to 9443
- Set default authorizer to single-user-authorizer
- Set default login-identity-provider to single-user-provider
- Updated README.md with authentication instructions using generated credentials
- Updated default URL and port information in Administration and User Guides
- Updated Getting Started Guide with authentication and URL changes
- Updated Docker images to set HTTPS as default configuration
- Updated default HTTPS port to 8443
- Set Cluster Protocol secure property in Docker start scripts
- Added set-single-user-credentials command
- Refactored shared classes to nifi-single-user-utils
- Updated Getting Started documentation and logging
- Updated documentation and TLS Toolkit default ports
- Updated Toolkit Guide and Administration Guide
- Updated README.md with HTTPS links
2021-06-11 14:18:53 -04:00
Mark Payne c9dee30294
NIFI-8658: Allow Filter Functions and expressions to be specified as a RecordPaths
NIFI-8658: Addressed issue where the RecordField that was provided from Function Filters were not accurate

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5125
2021-06-04 17:17:33 -04:00
Chris Sampson e883aa6b0b
Rename RecordPath un/escape JSON functions (#5124)
Signed-off-by: Otto Fowler <ottobackwards@gmail.com>

This closes #5124.
2021-06-04 11:56:28 -04:00
Chris Sampson bc5204d4df
NIFI-8137 Record Path EscapeJson/UnescapeJson functions (#4756)
* NIFI-8137 Record Path EscapeJson/UnescapeJson functions

* Correct jackson-databind dependency version

* Add negative tests for RecordPath JSON handling; rename RecordPath JSON classes to better match existing functions

Signed-off-by: Otto Fowler <ottobackwards@gmail.com>

This closes #4756.
2021-06-04 09:19:24 -04:00
exceptionfactory a3365c8833 NIFI-8462 Refactored PutSyslog and ListenSyslog using Netty
- Added nifi-event-transport module encapsulating Netty classes
- Refactored unit tests for PutSyslog and ListenSyslog
- Removed integration tests for PutSyslog and ListenSyslog

NIFI-8462 Added context.yield() in PutSyslog when no FlowFiles and addressed other issues

NIFI-8462 Removed unused import of ExpressionLanguageScope

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5044.
2021-05-25 10:54:15 -04:00
exceptionfactory 01783a295c NIFI-8037 Changed SSLContextServices to use runtime supported protocols for TLSv1.3
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4827.
2021-05-23 23:52:01 -04:00
Lehel Boér a3eaf0a37a NIFI-8325: Complete SNMP refactor: SNMP GET and SET processors reworked, unit tests added
This closes #5028.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-05-19 23:26:31 +02:00