759 Commits

Author SHA1 Message Date
exceptionfactory
84dbf915a9 NIFI-9060 Refactored HTTP Cookie Path Handling
- Implemented ApplicationCookieService for adding and retrieving HTTP Cookies
- Added getCookieResourceUri() leveraging allowed proxy headers to support optional Cookie Paths
- Refactored Access Resources to use ApplicationCookieService for processing
- Changed __Host- prefix to __Secure- prefix for Bearer Token cookie to support Cookie Path processing
- Removed unnecessary jetty-http dependency from nifi-web-api
- Corrected NiFi path references in JavaScript to support prefixed paths

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5329.
2021-09-23 18:03:48 -04:00
Mark Payne
e29c8af708
NIFI-9182: When calling ProcessSession.append(), do not allow calls from processor to flush the underlying BufferedOutputStream. Instead, wrap in a NonFlushableOutputStream and only flush when session commit is called
This closes #5359

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-09-22 18:54:32 -05:00
Mark Payne
baf29e59f1
NIFI-9009: Created VerifiableProcessor, VerifiableControllerService, VerifiableReportingTask components; implemented backend work to call the methods. Added REST APIs and created/updated data models for component configuration verification
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5288
2021-09-22 19:00:23 -04:00
exceptionfactory
b4e0416b5e
NIFI-9228 Refactored tests using TemporaryKeyStoreBuilder
- Added TemporaryKeyStoreBuilder with File.deleteOnExit() for KeyStore and TrustStore files
- Removed JKS files from nifi-security-utils tests
- Refactored usage of KeyStoreUtils.createKeyStoreAndGetX509Certificate() to TemporaryKeyStoreBuilder
- Removed unnecesary hadoop-minikdc test dependency in security-utils
- Replaced Mini KDC Hex utility with Bouncy Castle Hex utility in unit tests

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5406
2021-09-21 16:07:13 -04:00
exceptionfactory
e78674ec59 NIFI-9184 Refactored shared methods in Sensitive Property Providers
- Added EncodedSensitivePropertiesProvider with Base64 encoding methods
- Added ClientBasedEncodedSensitivePropertiesProvider with validate method
- Abstracted client configuration to ClientProvider interface and implementations
- Added unit tests for AWS and Azure Property Providers

NIFI-9184 Adjusted abstract provider class names and updated documentation

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5363.
2021-09-21 15:47:19 -04:00
exceptionfactory
ba775d28de
NIFI-9228 Refactored tests to use generated KeyStores
- Refactored multiple tests using KeyStoreUtils
- Removed static KeyStore and TrustStore files

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5401
2021-09-21 11:06:19 -04:00
markap14
73f88b3239
NIFI-9061: Eliminated the nifi.cluster.node.protocol.threads property in favor of nifi.cluster.node.protocol.max.threads property so that we can properly scale out the number of threads used for HTTP request replication. Implementing a caching mechanism for creating the DateTimeFormatter used by TimeAdapter in order to improve performance when parsing timestamps in web requests. Implementing caching logic for caching the number of characters that can rendered without needing an ellipsis for some components in the UI (#5316)
This closes #5316
2021-09-13 14:36:35 -04:00
denes
3fbd9c9e14
NIFI-9201 NullPointerException in AbstractKerberosUser if the tgt is not renewable (#5383) 2021-09-13 11:04:39 -04:00
Matthew Burgess
98e3b465c3
NIFI-8376: Gracefully handle SQL exceptions in ResultSetRecordSet
This closes #4951

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2021-09-08 09:11:26 -04:00
exceptionfactory
f5a421ccb1
NIFI-9067 Upgraded JAXB Runtime from 2.3.0 to 2.3.5
- Replaced old com.sun.xml.bind:jaxb-impl and jaxb-core with current org.glassfish.jaxb:jaxb-runtime
- Replaced old javax.xml.bind:jaxb-api with current jakarta.xml.bind-api
- Removed unnecessary dependency references to javax.activation-api

This closes #5320

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2021-09-07 13:27:53 -04:00
exceptionfactory
20804ff2b1
NIFI-8749 Removed implicit time zone conversion to GMT
- Updated DataTypeUtils.getDateFormat() to use system default time zone
- Updated Record Path Guide to match Expression Language Guide regarding default time zone
- Updated impacted unit tests to expect localized dates instead of dates converted to GMT

NIFI-8749 Set user.timezone in TestRecordPath to avoid unexpected date conversions

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5361
2021-08-31 16:07:53 -04:00
Matthew Burgess
dfa6592a47
NIFI-8749 Revert "Removed implicit time zone conversion to GMT"
This reverts commit 758e1850ed0e4af5063877c7e3a56628e5e7ed78.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>
2021-08-31 13:33:53 -04:00
exceptionfactory
758e1850ed
NIFI-8749 Removed implicit time zone conversion to GMT
- Updated DataTypeUtils.getDateFormat() to use system default time zone
- Updated Record Path Guide to match Expression Language Guide regarding default time zone
- Updated impacted unit tests to expect localized dates instead of dates converted to GMT

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5305
2021-08-31 10:40:57 -04:00
Emilio Setiadarma
2ab23efb74
NIFI-6616 Added GCP Sensitive Property Provider
This closes #5242

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-30 19:15:34 -05:00
Joe Gresock
4a3e81531b
NIFI-8490: Adding inherited parameter contexts (#5072)
- Allowing inherited param contexts on creation, updating PC authorization
2021-08-30 10:44:31 -04:00
Bryan Bende
e436381c3a
NIFI-8975 Integrate KerberosUserService into HBase processors/services
This closes #5322

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-27 17:41:58 -05:00
exceptionfactory
0cf09840ea
NIFI-9082 Add nifi.zookeeper.jute.maxbuffer property (#5335)
* NIFI-9082 Added nifi.zookeeper.jute.maxbuffer property

- Updated ZooKeeperStateProvider to check jute.maxbuffer instead of hard-coded number
- Updated internal ZooKeeperClientConfig class to include new juteMaxBuffer property

* NIFI-9082 Updated jute.maxbuffer documentation and added hexadecimal representation

- Renamed DefaultWatcher to NoOpWatcher
- Removed unnecessary validateDataSize() from createNode()
2021-08-26 21:12:34 +02:00
Paul Grey
aa5babbb97
NIFI-9073 - Limit Surefire Output during Build
This closes #5325

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-24 10:37:07 -05:00
exceptionfactory
a652280fbb NIFI-8766 Implemented RS512 Algorithm for JWT Signing
- Replaced per-user symmetric-key HS256 with shared and rotated RSA asymmetric-key RS512 implementation
- Added nifi.security.user.jws.key.rotation.period property for RSA Key Pair rotation
- Added JSON Web Tokens section to Administration Guide
- Implemented persistent storage of RSA Public Keys for verification using Local State Manager
- Implemented JWT revocation on logout with persistence using Local State Manager
- Refactored JWT implementation using Spring Security OAuth2 and Nimbus JWT
- Refactored Spring Security Provider configuration using Java instead of XML
- Removed H2 storage of per-user keys
- Upgraded nimbus-jose-jwt from 7.9 to 9.11.2

NIFI-8766 Corrected AuthenticationException handling in AccessResource.getAccessStatus

- Added nifi.user.security.jws.key.rotation.period to default nifi.properties
- Updated logging statements and clarified configuration and method documentation

NIFI-8766 Changed Algorithm to PS512 and updated documentation

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5262.
2021-08-19 12:26:12 -04:00
Lehel Boér
9bcbf83e5a NIFI-8752: Automatic diagnostic at NiFi restart/stop
This closes #5195.

Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
2021-08-19 18:09:58 +02:00
Bryan Bende
21c2fb95d3
NIFI-8973 Implement KerberosUserService API and keytab, password, and ticket cache implementations
NIFI-8974 Integrate KerberosUserService with HDFS processors

NIFI-8980 Integrate KerberosUserService with Kafka 2.6 processors
- Introduced SelfContainerKerberosUserService to restrict which impls can be used with Kafka
- Add variations of KerberosUser doAs that allow setting the context ClassLoader
- Add additional unit tests for configurations

This closes #5277

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-18 19:54:56 -05:00
Emilio Setiadarma
714670b8e6
NIFI-6615 Added Azure Key Vault Key Sensitive Property Provider
This closes #5274

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-13 10:54:48 -05:00
Mark Payne
fb4edfa0d6
NIFI-8950: Added ability to optionally use a file-system backed Content Repository for Stateless NiFi
This closes #5254

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-11 12:01:07 -05:00
exceptionfactory
f27cd012f3
NIFI-9035 Refactored isKeystoreValid() to avoid NullPointerException
- Removed unnecessary debug logging from StandardTlsConfiguration
- Replaced internal string labels with StoreType enum

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5297
2021-08-09 15:55:20 -04:00
Timea Barna
462306369f NIFI-8668 ConsumeAzureEventHub NiFi processors need to support storage SAS token authentication
This closes #5136.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-08-09 12:57:56 +02:00
Joe Gresock
cc1e9665cd
NIFI-8696: Added HashiCorp Vault KeyValue SPP
This closes #5255

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-06 07:39:34 -05:00
Mark Payne
787f16da3f
NIFI-9013: Moved nifi-properties-loader and nifi-properties dependencies from data-provenance-utils to the repository where they are needed
This closes #5283

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2021-08-05 20:36:01 -04:00
exceptionfactory
d93e9f152c
NIFI-8986 Upgraded Commons Compress to 1.21
- Upgraded direct and transitive dependencies from 1.20 and earlier to 1.21

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5273.
2021-08-03 15:52:45 +02:00
Joe Gresock
2daac5714a
NIFI-8695: Adding context to sensitive property providers
This closes #5206

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-07-27 15:37:29 -05:00
exceptionfactory
e16bf644e3
NIFI-8948 Upgraded Spring to 5.3.9 and Security to 5.5.1
- Upgrades Spring Framework from 5.3.8 to 5.3.9
- Upgrades Spring Security from 5.4.6 to 5.5.1
- Upgrades Spring Boot from 2.5.1 to 2.5.2 in Registry
- Upgrades Spring Data Redis from 2.5.1 to 2.5.3

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5252.
2021-07-27 10:21:27 +02:00
Emilio Setiadarma
d4a560c59a
NIFI-6325 Added AWS KMS Sensitive Properties Provider
This closes #5202

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-07-22 08:24:52 -05:00
Bryan Bende
74c0a91b6c
NIFI-8933 Configure Jersey's ObjectMapper to ingnore unknown fields
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5236.
2021-07-21 11:46:45 +02:00
exceptionfactory
c668d3df1b NIFI-8782 Added Rate-Limiting for Access Token Requests
- Added Jetty DoSFilter configured for /access/token
- Added nifi.web.max.access.token.requests.per.second property with default value of 25

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5215.
2021-07-14 14:24:45 -04:00
Joe Witt
7356332852
Merge branch 'NIFI-8767-RC2' 2021-07-14 09:07:22 -05:00
exceptionfactory
05d7867a16
NIFI-8768 Added toLocalDate() for convertType() handling of DATE fields
- Updated PutKudu to use DataTypeUtils.toLocalDate() for DATE fields
- Updated PutDatabaseRecord to remove convertDateToLocalTZ() since convertType() uses toLocalDate()
- Updated PutElasticsearchHttpRecord to use default time zone format for DATE fields
- Updated WriteXMLResult to use default time zone format for DATE fields
- Updated WriteJsonResult to use default time zone format for DATE fields
- Updated AvroTypeUtil to use toLocalDate() for logical DATE fields
- Updated JdbcCommon to avoid conversion to UTC for logical DATE fields
- Updated Processor and RecordReader unit tests for consistency in DATE comparison

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5210
2021-07-13 16:17:36 -04:00
Joe Witt
97feacc181
NIFI-8767-RC2 prepare for next development iteration 2021-07-10 12:17:09 -07:00
Joe Witt
fcbf1d5f97
NIFI-8767-RC2 prepare release nifi-1.14.0-RC2 2021-07-10 12:17:05 -07:00
exceptionfactory
aedbd0d753 NIFI-8511 Added KeyStore implementation of KeyProvider
- KeyStoreKeyProvider supports PKCS12 and BCFKS
- Refactored KeyProvider and implementations to nifi-security-kms
- Updated Admin Guide and User Guide with KeyStoreKeyProvider details

NIFI-8511 Improved documentation and streamlined several methods

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5110.
2021-07-09 12:12:25 -04:00
Peter Turcsanyi
f11c99f9ec
NIFI-8475: Fixed EL query parser when input contains multiple escaped expressions
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5031
2021-06-25 13:59:11 -04:00
Joe Gresock
726082ffa6
NIFI-8447 Added HashiCorp Vault Transit Sensitive Properties Provider
- Added default bootstrap-hashicorp-vault.conf
- Updated Toolkit Guide documentation with HashiCorp Vault properties

This closes #5154

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-24 22:06:42 -05:00
exceptionfactory
6a83115d6a NIFI-7468 Updated SSLSocketChannel to support TLS 1.3
- Handling additional FINISHED Handshake Status for TLS 1.3 Post-Handshake Messages per RFC 8446 Section 4.6
- Removed clearing buffers after handshake to avoid losing packets
- Updated read() method to check Handshake Status after SSLEngine.unwrap()
- Changed SSLSocketChannelSender to close SSLSocketChannel before other resources
- Added ChannelStatus enum and convenience logging methods for tracing status
- Added unit tests for TLS 1.2 and 1.3 using Netty server and client handlers

NIFI-8704 Updated netty-handler to 4.1.65.Final

NIFI-7468 Corrected SSLSocketChannel.read() to return byte read

NIFI-7468 Adjusted comment formatting

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5152.
2021-06-22 22:29:16 -04:00
exceptionfactory
2efc2b6b8b
NIFI-8718 Upgraded Apache Commons IO to 2.10.0
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5169
2021-06-21 11:29:27 -04:00
rhavermans
9fd8042cfb
NIFI-8664 Update json-path lib from 2.4.0 -> 2.6.0
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5132
2021-06-21 11:01:10 -04:00
exceptionfactory
9be9b9dbe2
NIFI-8708 Upgraded Spring Framework to 5.3.8 for several extensions
- nifi-vault-utils
- nifi-email-bundle
- nifi-redis-bundle

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5163.
2021-06-16 18:18:54 +02:00
markap14
96a8b2d090
NIFI-8666: Allow users to escape parameter names in Expression Language using quotes. (#5133) 2021-06-15 09:37:34 -04:00
Peter Turcsanyi
48befe22f6
NIFI-8645: Disable LongRunningTaskMonitor by default
This closes #5111

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-14 08:50:34 -05:00
Joe Gresock
6f04b45773
NIFI-8643: Correcting nifi.cluster.load.balance.address in properties (#5146)
NIFI-8643: Correcting nifi.cluster.load.balance.address in properties
2021-06-11 16:58:20 -04:00
Joe Gresock
1ccc4fbb0f
NIFI-8651: Refactor Sensitive Properties Providers for extension
This closes #5131

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-11 15:44:37 -05:00
Mark Payne
64f600d0ce
NIFI-8689: This closes #5150. Avoid flushing the socket buffer unnecessarily when sending a series of FlowFiles via site-to-site
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-06-11 13:15:03 -07:00
exceptionfactory
db40989b48 NIFI-8516 Enabled HTTPS and Single User Authentication in default configuration
- Set default HTTPS Port to 9443
- Set default authorizer to single-user-authorizer
- Set default login-identity-provider to single-user-provider
- Updated README.md with authentication instructions using generated credentials
- Updated default URL and port information in Administration and User Guides
- Updated Getting Started Guide with authentication and URL changes
- Updated Docker images to set HTTPS as default configuration
- Updated default HTTPS port to 8443
- Set Cluster Protocol secure property in Docker start scripts
- Added set-single-user-credentials command
- Refactored shared classes to nifi-single-user-utils
- Updated Getting Started documentation and logging
- Updated documentation and TLS Toolkit default ports
- Updated Toolkit Guide and Administration Guide
- Updated README.md with HTTPS links
2021-06-11 14:18:53 -04:00