570 Commits

Author SHA1 Message Date
exceptionfactory
a652280fbb NIFI-8766 Implemented RS512 Algorithm for JWT Signing
- Replaced per-user symmetric-key HS256 with shared and rotated RSA asymmetric-key RS512 implementation
- Added nifi.security.user.jws.key.rotation.period property for RSA Key Pair rotation
- Added JSON Web Tokens section to Administration Guide
- Implemented persistent storage of RSA Public Keys for verification using Local State Manager
- Implemented JWT revocation on logout with persistence using Local State Manager
- Refactored JWT implementation using Spring Security OAuth2 and Nimbus JWT
- Refactored Spring Security Provider configuration using Java instead of XML
- Removed H2 storage of per-user keys
- Upgraded nimbus-jose-jwt from 7.9 to 9.11.2

NIFI-8766 Corrected AuthenticationException handling in AccessResource.getAccessStatus

- Added nifi.user.security.jws.key.rotation.period to default nifi.properties
- Updated logging statements and clarified configuration and method documentation

NIFI-8766 Changed Algorithm to PS512 and updated documentation

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5262.
2021-08-19 12:26:12 -04:00
Lehel Boér
9bcbf83e5a NIFI-8752: Automatic diagnostic at NiFi restart/stop
This closes #5195.

Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
2021-08-19 18:09:58 +02:00
Emilio Setiadarma
714670b8e6
NIFI-6615 Added Azure Key Vault Key Sensitive Property Provider
This closes #5274

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-13 10:54:48 -05:00
Joe Gresock
cc1e9665cd
NIFI-8696: Added HashiCorp Vault KeyValue SPP
This closes #5255

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-06 07:39:34 -05:00
exceptionfactory
633cdab121
NIFI-8964 Add Cluster Firewall Configuration to Admin Guide (#5264)
NIFI-8964 Added Cluster Firewall Configuration to Admin Guide
2021-08-03 10:38:59 -04:00
Emilio Setiadarma
d4a560c59a
NIFI-6325 Added AWS KMS Sensitive Properties Provider
This closes #5202

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-07-22 08:24:52 -05:00
exceptionfactory
c668d3df1b NIFI-8782 Added Rate-Limiting for Access Token Requests
- Added Jetty DoSFilter configured for /access/token
- Added nifi.web.max.access.token.requests.per.second property with default value of 25

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5215.
2021-07-14 14:24:45 -04:00
Joe Witt
97feacc181
NIFI-8767-RC2 prepare for next development iteration 2021-07-10 12:17:09 -07:00
Joe Witt
fcbf1d5f97
NIFI-8767-RC2 prepare release nifi-1.14.0-RC2 2021-07-10 12:17:05 -07:00
exceptionfactory
aedbd0d753 NIFI-8511 Added KeyStore implementation of KeyProvider
- KeyStoreKeyProvider supports PKCS12 and BCFKS
- Refactored KeyProvider and implementations to nifi-security-kms
- Updated Admin Guide and User Guide with KeyStoreKeyProvider details

NIFI-8511 Improved documentation and streamlined several methods

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5110.
2021-07-09 12:12:25 -04:00
Mark Bean
ea31634ea7
NIFI-8195: add default connection settings to process group configuration
- include new process group property support in NiFi Registry
  - updated documentation to describe and show new feature
  - added elements to XSD schema definition

NIFI-8195: update to DAO to fix PG move and copy/paste

update condition to not null vice null

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5192
2021-07-07 17:45:54 -04:00
theBestAndrew
1033e77e7f
NIFI-8686 Update administration guide with kernel 3 time_wait setting
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5149
2021-06-25 16:15:24 -04:00
Joe Gresock
726082ffa6
NIFI-8447 Added HashiCorp Vault Transit Sensitive Properties Provider
- Added default bootstrap-hashicorp-vault.conf
- Updated Toolkit Guide documentation with HashiCorp Vault properties

This closes #5154

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-24 22:06:42 -05:00
Matthew Burgess
5e4f32663e Revert "NIFI-8195: add default connection settings to process group configuration"
This reverts commit 5ebbe0028b9ff4d23f86d87e4942665db62e9bc0.
2021-06-24 18:52:17 -04:00
Mark Bean
5ebbe0028b
NIFI-8195: add default connection settings to process group configuration
NIFIDEVS-8195: fixed properties not properly inheriting from template/snippet values

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5094
2021-06-24 14:03:51 -04:00
Bence Simon
a3f54fa578
NIFI-8736 Adding capability to override HDFS location for NAR autoloading
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5185.
2021-06-24 19:37:19 +02:00
Mathieu Rossignol
5196ba9e14
Update developer-guide.adoc
Typo: "created created" (twice)

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5168
2021-06-21 13:14:10 -04:00
Andrew Lim
e88d610da6
NIFI-8700 Corrections/improvements to doc added for default HTTPS
This closes #5158

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-14 19:49:53 -05:00
Peter Turcsanyi
48befe22f6
NIFI-8645: Disable LongRunningTaskMonitor by default
This closes #5111

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-14 08:50:34 -05:00
exceptionfactory
db40989b48 NIFI-8516 Enabled HTTPS and Single User Authentication in default configuration
- Set default HTTPS Port to 9443
- Set default authorizer to single-user-authorizer
- Set default login-identity-provider to single-user-provider
- Updated README.md with authentication instructions using generated credentials
- Updated default URL and port information in Administration and User Guides
- Updated Getting Started Guide with authentication and URL changes
- Updated Docker images to set HTTPS as default configuration
- Updated default HTTPS port to 8443
- Set Cluster Protocol secure property in Docker start scripts
- Added set-single-user-credentials command
- Refactored shared classes to nifi-single-user-utils
- Updated Getting Started documentation and logging
- Updated documentation and TLS Toolkit default ports
- Updated Toolkit Guide and Administration Guide
- Updated README.md with HTTPS links
2021-06-11 14:18:53 -04:00
Paul Grey
a3c1cd074b
NIFI-8499 - Added encrypted FlowFile repository swap file implementation
This closes #5122

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-08 13:23:22 -05:00
Chris Sampson
e883aa6b0b
Rename RecordPath un/escape JSON functions (#5124)
Signed-off-by: Otto Fowler <ottobackwards@gmail.com>

This closes #5124.
2021-06-04 11:56:28 -04:00
Chris Sampson
bc5204d4df
NIFI-8137 Record Path EscapeJson/UnescapeJson functions (#4756)
* NIFI-8137 Record Path EscapeJson/UnescapeJson functions

* Correct jackson-databind dependency version

* Add negative tests for RecordPath JSON handling; rename RecordPath JSON classes to better match existing functions

Signed-off-by: Otto Fowler <ottobackwards@gmail.com>

This closes #4756.
2021-06-04 09:19:24 -04:00
Bence Simon
51aae5bcf6 NIFI-8519 Adding HDFS support for NAR autoload
- Refining classloader management with the help of @markap14

This closes #5059

Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-05-14 13:14:06 -04:00
exceptionfactory
09e54c1dad
NIFI-8246 Set NIFI_PBKDF2_AES_GCM_256 as default properties encryption algorithm
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5055.
2021-05-12 12:58:50 +02:00
Sushil Kumar
48a9054731 NIFI-4913 - Added "-E" with sudo to read environment variables set in current shell environment while running nifi with bootstrap property "run.as" set as to some user
NIFI-4913 - Updated administration-guide.adoc

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #3750.
2021-05-11 11:48:41 -04:00
exceptionfactory
13d5be622b NIFI-8230 Removed default Sensitive Properties Key and added random generation
- Retained legacy default Sensitive Properties Key in ConfigEncryptionTool to support migration
- Streamlined default file path and moved key generation conditional
- Refactored with getDefaultProperties()
- Cleared System Property in ConfigEncryptionToolTest
- Added checking and error handling for clustered status
- Added set-sensitive-properties-key command
- Refactored PropertyEncryptor classes to nifi-property-encryptor
- Added nifi-flow-encryptor
- Refactored ConfigEncryptionTool to use FlowEncryptor for supporting AEAD algorithms
- Added Admin Guide section Updating the Sensitive Properties Key

This closes #4857.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-05-04 08:49:34 -04:00
Joe Gresock
54a0e27c93
NIFI-7134: Adding auto-reloading of Keystore and Truststore
- NIFI-7261 Included TrustStoreScanner for auto-reloading of truststore

This closes #4991

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-04-29 08:29:42 -05:00
Paul Grey
17fa0cf3c1
NIFI-3580 Add configurable TLS Cipher Suite properties
This closes #5018

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-04-22 11:31:19 -05:00
Andrew Lim
55236e6fd7
NIFI-8396 Update incorrect case in references to ZooKeeper in Admin and Toolkit guides
This closes #4985

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-04-12 11:50:05 -05:00
sjyang18
365dde2d11 NIFI-7573 Added Azure Active Directory User Group Provider
This closes #4367

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-04-09 15:06:14 -07:00
Nathan Gough
9da3b1ec01
NIFI-7912 - Added properties to configure DoSFilter timeout and whitelisted addresses
- Added nifi.web.request.ip.whitelist property to set DoSFilter.ipWhitelist
- Added nifi.web.request.timeout property to set DoSFilter.maxRequestMs with default of 60 seconds

This closes #4972

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-04-07 07:40:03 -05:00
Nathan Gough
61c4261bb7 NIFI-8335 Remove the nifi-storm-spout module that is no longer maintained
This closes #4912

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-03-18 12:52:33 -07:00
Joey Frazee
f9e469ffa2
NIFI-8324 Fix NGINX X-ProxiedEntitiesChain examples in Admin Guide
This closes #4897

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-03-17 08:10:40 -05:00
exceptionfactory
5608f4389a NIFI-7668 Implemented support for additional AEAD property encryption methods
- Added support for PBKDF2 and Scrypt property encryption methods in addition to Argon2
- Refactored StringEncryptor class to PropertyEncryptor interface with implementations
- Added PasswordBasedCipherPropertyEncryptor and KeyedCipherPropertyEncryptor
- Replaced direct instantiation of encryptor with PropertyEncryptorFactory
- Refactored applicable unit tests to use mocked PropertyEncryptor

NIFI-7668 Consolidated similar methods to CipherPropertyEncryptor

NIFI-7668 Updated AbstractTimeBasedSchedulingAgent with PropertyEncryptor

NIFI-7668 Added support for bcrypt secure hashing algorithm

NIFI-7668 Updated comments to clarify implementation of bcrypt key derivation

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4809.
2021-02-25 17:00:32 -05:00
Mark Payne
99fe548e6b NIFI-8113: Minor updates to admin guide to clarify documentation of some new properties 2021-02-25 16:08:07 -05:00
Bence Simon
7730777d66 NIFI-8113 Adding persistent status history repository backed by embedded QuestDB
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-02-25 15:59:01 -05:00
Tamas Palfy
aa726040c5 NIFI-8188 - Add 'Run Once' for processors in context menu.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-02-18 14:51:28 -05:00
Joe Witt
88fab00e29
NIFI-7873 merging release branch to latest and updating to 1.14.0-SNAPSHOT 2021-02-15 12:09:32 -07:00
Joe Witt
4afb2ba743
NIFI-7873-RC4 prepare for next development iteration 2021-02-15 12:09:31 -07:00
Joe Witt
487280bee9
NIFI-7873-RC4 prepare release nifi-1.13.0-RC4 2021-02-15 12:09:30 -07:00
Nathan Gough
8057f8f6c5
NIFI-8221 - Set the default HTTP listening interface to 127.0.0.1.
This closes #4817

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-02-10 14:46:37 -06:00
Mark Payne
03fd59eb2f
NIFI-8196: When node is reconnected to cluster, ensure that it re-registers for election of cluster coordinator / primary node. On startup, if cluster coordinator is already registered and is 'this node' then register silently as coordinator and do not join the cluster until there is no Cluster Coordinator or another node is elected. This allows the zookeeper session timeout to elapse.
Signed-off-by: Bryan Bende <bbende@apache.org>
2021-02-05 15:18:07 -05:00
Peter Gyori
5a8b18b12d NIFI-8123: Added support for --wait-for-init when NiFi started with the 'start' command
This closes #4748.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-02-03 09:40:09 -05:00
Joe Witt
d826416217
NIFI-8192 updating Copyright years for nifi things to include 2021
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4802.
2021-02-03 12:56:28 +04:00
Mark Payne
4ff9cddf15
NIFI-8173, NIFI-8174: This closes #4784. Updated Parameter Contexts to allow for unsetting a parameter value / setting it to null. Allow Parameters to make use of Expression Language. Updated docs to illustrated how EL is evaluated
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-01-26 21:55:18 -07:00
Troy Melhase
76648bdc0b NIFI-7356 - Config TLS for embedded ZooKeeper when NiFi TLS enabled.
NIFI-7356 - Addresses PR feedback.

NIFI-7356 - Additional changes from PR feedback.

NIFI-7356 - Adding integration tests for ZooKeeperStateServer for TLS.

NIFI-7356 - TLS + Zookeeper now working with single and quorum. Needs code cleanup, need to fix IT tests and docs.

NIFI-7356 - Fixed up tests and removed some irrelevant ones. Refactored some of ZooKeeperStateServer. Tested successfully with a secure and insecure 3 node NiFi + Quorum.

NIFI-7356 - Checkstyle fixes.

NIFI-7356 - Updated administration guide with embedded ZooKeeper TLS configuration.

NIFI-7356 - Updated the way ZooKeeper TLS properties are set/mapped from NiFi properties.

NIFI-7356 - Updated per review, using NiFiProperties keystore strings, classname for ocnnection factory, adjusted TLS configuration checks in NiFiProperties.

NIFI-7356 - Updated configuration validation logic and added tests.

NIFI-7356 - Codestyle check fixes.

NIFI-7356 - Updated some of the log messages.

NIFI-7356 - Updated as per code review.

NIFI-7356 - Fixed max port number.

NIFI-7356 - Updated admin guide and updated small code issues as per code review.

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4753.
2021-01-21 13:47:53 -05:00
Andrew Lim
27f57e6463
NIFI-8150 Change Download flow to Download flow definition for process groups (#4766) 2021-01-20 16:00:41 -05:00
Nissim Shiman
3cc8d767b3 NIFI-7738 Reverse Provenance Query
This closes #4563.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-01-13 09:35:05 -05:00
exceptionfactory
7d76bcd520 NIFI-8094 Added support for BCFKS Keystore Type
NIFI-8094 Updated Administration Guide to include BCFKS

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4729.
2021-01-12 10:52:49 -05:00