661 Commits

Author SHA1 Message Date
Joey Frazee
55cb8d73cb NIFI-7401 Add ZooKeeper client TLS to CuratorLeaderElectionManager
NIFI-7401 Rebased to 1.13.0-SNAPSHOT and simplified tests

NIFI-7401 Added keystore types and changed properties to match nifi.security.*

NIFI-7401 Removed dead code from SecureClientZooKeeperFactory test

NIFI-7401 Renamed bean methods, moved helper code into NiFiProperties

NIFI-7401 Changed connection socket constants to use .class.getName()

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4592.
2020-10-20 11:20:43 -04:00
Andrew Lim
dee2fcedfe
NIFI-7871 Correct errors for UUID3, UUID5 and hash functions in EL Guide
Added links to UUID function in docs.

This closes #4583.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-10-07 16:35:21 -07:00
abrown
b4faf210d8
NIFI-4897 Add documentation to user guide for terminating a processor
NIFI-4897 fixed typos in new section

Fixed some typoes

This closes #4568

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-10-05 09:17:27 -04:00
Pierre Villard
f32405ed16
NIFI-7844 - substring should return empty instead of throwing IndexOutOfBoundsException
This closes #4553

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-10-01 19:45:26 -04:00
VKadam
7e0bcb98e1
NIFI-7841: Made corrections in the nifi-walkthroughs docs (#4548)
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-09-23 16:37:26 -07:00
Mohammed Nadeem
4040664886
NIFI-7816: Correct documentation example for urlEncode function in Expression Language Guide (#4536)
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-09-21 12:12:23 -07:00
abrown
953abba6d6
NIFI-5061-NiFi documentation incomplete/wrong for EL hierarchy.
Co-authored-by: Pierre Villard <pierre.villard.fr@gmail.com>
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4530.
2020-09-20 19:48:14 +02:00
Andy LoPresto
9370571131
NIFI-7804 Split nifi-security-utils into sub-module for nifi-security… (#4533)
* NIFI-7804 Split nifi-security-utils into sub-module for nifi-security-utils-api (no external dependencies).
Separated interface and implementation of TlsConfiguration.
Reabsorbed nifi-security-xml-config into nifi-security-utils.

* NIFI-7804 Resolved failing unit test on Java 8.
Removed accidental module dependency.

* NIFI-7804 Resolved failing unit test.

* NIFI-7804 Removed legacy dependency.

* NIFI-7804 Marked nifi-security-utils-api as provided and overrode with compile scope in specific modules which are not children of nifi-standard-services-api-nar.
2020-09-17 12:52:22 -04:00
abrown
624678466b
NIFI-7580-Add documentation around autoloading NARs
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4529.
2020-09-17 12:50:13 +02:00
Bence Simon
0dff3bc065
NIFI-7429 Adding status history for system level metrics
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4420.
2020-09-10 16:19:11 +02:00
Andrew Lim
29e23e57b9
NIFI-7743 Document Empty all queues option for Process Groups
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4506
2020-09-02 17:14:17 -04:00
VKadam
e884b3cdb2
NIFI-7778: Made corrections in descriptions of padLeft, padRight, plus (#4504)
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-09-01 13:07:47 -07:00
Andrew Lim
861b83f314
NIFI-7309 Update Admin Guide by removing unused properties and updating default values
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4488.
2020-08-28 14:38:21 +02:00
Joe Witt
8baa5c9940
NIFI-7692 updating for next dev release 1.13.0 2020-08-18 14:48:02 -07:00
Joe Witt
fb57bcbc11
NIFI-7692-RC1 prepare for next development iteration 2020-08-13 09:20:39 -07:00
Joe Witt
303d6c59ba
NIFI-7692-RC1 prepare release nifi-1.12.0-RC1 2020-08-13 09:20:36 -07:00
Jaya Aditya
339e09a6e0
NIFI-7681 - Add update-bucket-policy command, add option to specify timeout and fix documentation to include previously implemented commands (#4450)
* NIFI-7681 - Add update-bucket-policy command, add option to specify
timeout and fix documentation to include previously implemented commands

* Fix return type of UpdateBucketPolicy and add missing registry commands to the documentation

Co-authored-by: Jaya Aditya <jchandra@yahoo-corp.jp>
2020-08-07 09:06:56 -04:00
Ruben Laguna
30b328ceab
NIFI-7694 - Move misplaced text related to oidc
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4445.
2020-07-31 11:00:14 +02:00
Tamás Bunth
455f48fce4
NIFI-7640 Add documentation: temporary directory (#4414)
NiFi uses the Java IO temporary directory for storing HTTP multipart
files when using HandleHttpRequest processor. The directory can be
overwritten with Java command line parameter.
2020-07-29 13:57:34 -07:00
Andy LoPresto
716ba992f5
NIFI-7669 Changed custom PBE AEAD algorithm to derive key once rather than on every encrypt/decrypt operation, leading to substantial performance gains.
Updated documentation.
Added unit tests.

NIFI-7669 Moved time-based encryption tests to integration tests to avoid running during CI builds.

NIFI-7669 Fixed failing test due to nifi.properties initialization.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4435.
2020-07-29 09:29:00 +02:00
Andy LoPresto
7d20c03f89
NIFI-7638 Implemented custom nifi.sensitive.props.algorithm for AES-G/CM with Argon2 KDF.
Added documentation for encryption of flow sensitive values.
Added unit tests.

This closes #4427.
2020-07-24 18:11:37 -07:00
Andy LoPresto
eeeda84474
NIFI-7122 Introduced improvements to EncryptContent processor.
Added unit tests.
Refactored shared logic from various algorithm-specific secure hasher implementations to AbstractSecureHasher.
Introduced secure hasher implementations for various KDFs.
Added custom validation to EncryptContent processor.
Implemented logic for EncryptContent to write operational metadata to flowfile attributes.
Added encryption metadata attribute annotations to EncryptContent.
Added Argon2 KDF documentation and Bcrypt key derivation change notes to Admin Guide.
Updated unit tests to calculate default/recommended cost parameters for Argon2.

This closes #4421.

Co-authored-by: mtien <mtien.apache@gmail.com>
2020-07-24 12:31:39 -07:00
Bryan Bende
e9a443fe2e
NIFI-7521 Remove additional unused property from nifi proeprties
This closes #4327.
2020-07-23 08:14:39 -04:00
Mark Payne
a473fc0373
NIFI-7633: Added FlowFileConcurrency of SINGLE_BATCH_PER_NODE to allow data to be pulled into a ProcessGroup as a batch
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4405.
2020-07-20 12:02:54 +02:00
Mark Payne
44fc4d9f27
NIFI-7552: When Process Group is configured to transfer data in batch, add an attribute to each outbound FlowFile that indicates how many FlowFiles went to each port. Updated user guide to explain the new attributes.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4345.
2020-07-17 10:33:59 +02:00
Andy LoPresto
dbee774c5b
NIFI-7304 Removed default value for nifi.web.max.content.size.
Added Bundle#toString() method.
Refactored implementation of filter addition logic.
Added logging.
Added unit tests to check for filter enablement.
Introduced content-length exception handling in StandardPublicPort.
Added filter bypass functionality for framework requests in ContentLengthFilter.
Updated property documentation in Admin Guide.
Renamed methods & added Javadoc to clarify purpose of filters in JettyServer.
Cleaned up conditional logic in StandardPublicPort.
Moved ContentLengthFilterTest to correct module.
Refactored unit tests for accuracy and clarity.
Fixed remaining merge conflict due to method renaming.

Signed-off-by: Joe Witt <joe.witt@gmail.com>
2020-07-14 10:42:00 -07:00
Andy LoPresto
0fa8776f4d
NIFI-7621 & NIFI-7614 Updated terminology throughout codebase and documentation.
Fixed unit test failures introduced from static imports during refactoring.

Signed-off-by: Joe Witt <joe.witt@gmail.com>
2020-07-14 10:39:28 -07:00
mtien
43fb57e7bb
NIFI-7332 Added method to log available claim names from the ID provider response when the OIDC Identifying User claim is not found. Revised log message to print available claims.
Added new StandardOidcIdentityProviderGroovyTest file.
Updated deprecated methods in StandardOidcIdentityProvider. Changed log output to print all available claim names from JWTClaimsSet. Added unit test.
Added comments in getAvailableClaims() method.
Fixed typos in NiFi Docs Admin Guide.
Added license to Groovy test.
Fixed a checkstyle error.
Refactor exchangeAuthorizationCode method.
Added unit tests.
Verified all unit tests added so far are passing.
Refactored code. Added unit tests.
Refactored OIDC provider to decouple constructor & network-dependent initialization.
Added unit tests.
Added unit tests.
Refactored OIDC provider to separately authorize the client. Added unit tests.
Added unit tests.

NIFI-7332 Refactored exchangeAuthorizationCode method to separately retrieve the NiFi JWT.

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4344.
2020-07-07 15:54:32 -04:00
VKadam
239a2e884c
NIFI-7513 Added custom DNS resolution steps to walkthrough (#4359) 2020-07-01 10:52:22 -07:00
Andy LoPresto
94c98c019f
NIFI-7558 Fixed CatchAllFilter init logic by calling super.init().
Renamed legacy terms.
Updated documentation.

This closes #4351.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-06-22 12:20:28 -07:00
Tamas Palfy
a126d0a6b6
NIFI-7442 Added missing use cases (list users and user groups), made update-access-policy use case more in line with the NiFi side. Added some tests. Additional refactor, documentation revision.
This closes #4329.
2020-06-12 12:43:41 -04:00
Bence Simon
d99983bdc7
NIFI-7442 Add CLI commands to the registry in order to support automatic registry setup 2020-06-12 12:43:27 -04:00
Mark Payne
463d72117b
NIFI-7507: Added section to User Guide on configuring a Process Group
NIFI-7507: Fixed Flowfile Expiration header in doc

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4318
2020-06-08 14:03:00 -04:00
Alessandro D'Armiento
788f8b0389
NIFI-6674 MinusEvaluator throws an Arithmetic Exception in case of Long overflow.
TestQuery checks that Long overflow is detected and Double overflow is correctly promoted to NEGATIVE_INFINITY

MinusEvaluator throws an Arithmetic Exception in case of Long overflow.
TestQuery checks that Long overflow is detected and Double overflow is correctly promoted to NEGATIVE_INFINITY

The behaviour change is reverted until further investigations.
The overflow behaviour is still enforced by unit tests and documented in the expression language doc

fixed mispositioned # in doc

This closes #3740

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-05-25 07:06:22 -04:00
Alessandro D'Armiento
1ba8f76a44
NIFI-6673 MultiplyEvaluator throws an Arithmetic Exception in case of Long overflow.
TestQuery checks that Long overflow is detected and Double overflow is correctly promoted to POSITIVE_INFINITY or NEGATIVE_INFINITY

The behaviour change is reverted until further investigations.
The overflow behaviour is still enforced by unit tests and documented in the expression language doc

This closes #3739

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-05-25 07:01:02 -04:00
Alessandro D'Armiento
b025117824
NIFI-6672 PlusEvaluator throws an Arithmetic Exception in case of Long overflow.
TestQuery checks that Long overflow is detected and Double overflow is correctly promoted to POSITIVE_INFINITY

The behaviour change is reverted until further investigations.
The overflow behaviour is still enforced by unit tests and documented in the expression language doc
NIFI-6672 Removed test code.

This closes #3738

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-05-25 06:58:15 -04:00
Phillip Grenier
0f4b79b55e
NIFI-6255 NIFI-6287: Hash function for expression language and record path.
NIFI-6255 NIFI-6287: Rebased to match the new expression language interface
NIFI-6255 NIFI-6287: Fix wildcard imports and unused imports
NIFI-6255 NIFI-6287: Move to the common codec DigetUtils
Update commons-codec

This closes #3624

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-05-25 06:23:00 -04:00
Mark Payne
d195702ee2 Fixed a couple of typos in the RecordPath guide 2020-05-20 17:09:40 -04:00
Peter Gyori
6b73ea48bc
NIFI-7413: Documented REMOTE_INVOCATION provenance event type in user/dev guides
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4267
2020-05-12 16:00:41 -04:00
Andy LoPresto
7d494011ea
Added note about unique initial user identity names to walkthrough doc. 2020-04-30 12:36:30 -07:00
Matt Gilman
e81960f8e8 NIFI-7170:
- Adding a flag to nifi.properties to disable anonymous authentication.

NIFI-7170:
- Fixing checkstyle issues.

NIFI-7170:
- Adding missing license header.

NIFI-7170:
- Initial PR feedback.

NIFI-7170:
- Fixing broken integration tests.
- Creating new integration tests for verifying allowing and preventing anonymous access.

NIFI-7170:
- Ensuring the new anonymous authentication property is considered for proxied requests.

NIFI-7170 - Fixed comment.

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4099.
2020-04-30 15:20:19 -04:00
Sushil Kumar
996688b419
NIFI-7389 Makes Missable heartbeat counts configurable
This closes #4236.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-04-27 13:21:21 -07:00
Andy LoPresto
07a8311b4c
NIFI-7319 Add walkthrough document (#4193)
* NIFI-7319 Added first draft of walkthroughs doc.

* NIFI-7319 Added instructions and screenshots for securing standalone NiFi instance.

* NIFI-7319 Added instructions and screenshots for instructing OS & browser to trust self-signed certificate.

* NIFI-7319 Added instructions and screenshots for securing NiFi with externally-provided certificates.

* NIFI-7319 Added instructions and screenshots for building NiFi from source.

* NIFI-7319 [WIP] Converting secure cluster instructions to match format.
Fixed instructions regarding embedded ZooKeeper configuration.

* NIFI-7319 Completed secure cluster walkthrough.

* NIFI-7319 Added walkthroughs to documentation navigation list.

* NIFI-7319 Incorporated PR feedback on broken links.

* NIFI-7319 Removed line number helpers from update sections.

* NIFI-7319 Incorporated final PR review items.

Co-authored-by: Sandra Pius <spiusapache@gmail.com>
2020-04-10 22:25:40 -07:00
Andy LoPresto
1ec7e31f11
NIFI-7341 Updated certificate commands and source code formatting in Toolkit Guide. (#4196) 2020-04-09 10:13:53 -07:00
Mark Payne
683b1d9952
NIFI-6849: Reworked how nodes inherit cluster information when joining a cluster. Now, if there are conflicts, a local copy is made of the flow/authorizations/etc. and the cluster's flow is inherited.
- Refactored Flow Synchronization to make code cleaner
 - Updated Authorizers to forcibly inherit Users, Groups, and Access Policies if the local flow is empty.
 - Updated FlowFileRepositories to use SerializedRepositoryRecord instead of RepositoryRecord, so that we have the ability to read records without already knowing the Queue objects. Updated StandardFlowSynchronizer so that if the flow is not inheritable but the controller has not yet been initialized, the flow is backed up and replaced instead of NiFi failing to start
- Added system tests. Updated FlowController so that if it fails to inherit flow due to flow uninheritability that it notifies the cluster of this instead of remaining in the 'CONNECTING' state.
- Added additional log statements to aid in debugging

NIFI-6849: Rebased against master. Updated Admin Guide to describe new cluster flow inheritance behavior

NIFI-6849: Addressed review feedback

NIFI-6849: Addressed review feedback: Relocated logic for bundle compatibility into the BundleCompatibilityCheck class. Fixed logic that prevented users/groups/policies from being forcibly inherited during startup

This closes #3891
2020-04-09 10:42:05 -04:00
Bence Simon
c0f5fcb484 NIFI-7188 Extending UI search with filters and refactoring existing solution
This closes #4123.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-04-02 21:49:40 +02:00
Troy Melhase
483f23a8aa
NIFI-7153 Adds ContentLengthFilter to enforce configurable maximum length on incoming HTTP requests.
Adds DoSFilter to enforce configurable maximum on incoming HTTP requests per second.
Redirected log messages for ContentLengthFilter to nifi-app.log in logback.xml.

This closes #4125.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-03-25 17:23:22 -07:00
Nathan Gough
7374361b5c
NIFI-7223 - Fixed a minor issue where the OkHttpReplicationClient class loaded blank properties as empty string instead of an expected null value. Added a isNotBlank check. Added unit tests for replication client and HTTPNotificationService.
NIFI-7223 - Renamed some variables and methods.
NIFI-7223 - Removed unused dependency. Corrected security properties in administration-guide.
2020-03-16 12:26:50 -04:00
Mike
60b6327757
Merge branch 'master' into NIFI-6791 2020-03-15 09:26:44 -04:00
Andrew Lim
7b0ae56a2c
NIFI-7179 Documented Download flow option in Process group context menu (#4124)
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-03-09 16:54:56 -07:00