2008 Commits

Author SHA1 Message Date
exceptionfactory
817f621d6f NIFI-8096 Deprecated ClientAuth references in SSLContextService
- Added SSLContextService.createContext() and refactored referencing components
- Removed references to ClientAuth from SslContextFactory methods

This closes #4737.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-01-06 16:24:26 -05:00
Shane Ardell
5c79553af6 NIFI-8062: make tabs functional with keyboard 2021-01-04 11:58:47 -05:00
exceptionfactory
28ca7478d6 NIFI-8019 Added TlsPlatform to provide runtime TLS protocol configuration
NIFI-8019 Renamed getDefaultProtocols() to getSupportedProtocols()

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4673.
2020-12-14 19:34:58 -05:00
exceptionfactory
0af736dc0b
NIFI-7885 Added Environment Variable to deny LFS access using Hadoop (#4712)
* NIFI-7885 Added Environment Variable to deny LFS access using Hadoop

* NIFI-7885 Changed Hadoop Processor validation to check LFS access
2020-12-09 09:29:42 -05:00
Matthew Burgess
04aaf25131 NIFI-1121: Use display name for dependent property documentation 2020-12-03 16:39:46 -05:00
mtien
8055c47a84
NIFI-8058 Fixed a UI error to correctly delete dynamic properties while configuring processors.
Changed to check the length of all unfiltered properties instead of only filtered properties.
Added additional check if descriptor is a dynamic property.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4707
2020-12-03 15:44:56 -05:00
Bryan Bende
312fa8e85e NIFI-8064 Convert TestSecureClientZooKeeperFactory to integration test 2020-12-03 14:46:13 -05:00
Bryan Bende
2b1359a808 NIFI-8060 Addressed review feedback 2020-12-03 14:42:50 -05:00
Bryan Bende
8ac8a2bd1f NIFI-8060 Added minimal VolatileProvenanceRepository to nifi-stateless-engine and remove dependency on nifi-volatile-provenance-repo module 2020-12-03 14:42:50 -05:00
markap14
aaa1452d04
NIFI-8054: Updated ReflectionUtils to use a WeakHashMap for the mapping of annotations to methods with that annotation. This way, the ReflectionUtils class will not hold a reference to Classes that are no longer referenced elsewhere. (#4694) 2020-11-30 16:14:12 -05:00
Mark Payne
7ad9520079
NIFI-8041: Refactored API for stateless nifi so that calling StatelessDataflow.trigger() returns an object that allows the caller to wait for the result, cancel the result, etc. and then optionally acknowledge the completion. This allows the caller to block the completion of ProcessSession.commit() until it has handled the output of the dataflow execution.
NIFI-8038: Fixed deadlock that can occur when updating Parameter Context

This closes #4684.

Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-24 15:12:51 -05:00
Mark Payne
3c9d8a7007
NIFI-7897: Refactoring to create a new nifi-framework-components module.
- Refactored nifi-stateless to make use of nifi-framework-components
- Removed requirement for nifi-framework-nar to be provided.
- Refactored stateless nifi into api, engine, nar, and bootstrap modules, with a parent 'bundle' module
- Creation of nifi-stateless-system-tests
- Added unit tests and logging
- Changed flow configuration to use properties file instead of json
- Allow for -p parameter to specify parameters on command line
- Moved implementations of Authorizer, NiFiUser, and UserGroupProviders to new module named nifi-framework-authorization-providers so that those that depend on nifi-framework-authorization don't have to bring in the providers. This way, we can have stateless not bring in those providers, as we otherwise get warnings on startup about the provider already being registered. Additionally, it avoids needing dependencies on spring-security-core
- Updated bin/nifi.sh script to run new stateless bootstrap
- Added Reporting Tasks to stateless.
- Download bundles as necessary on stateless nifi startup

NIFI-7897: Addressing review feedback

NIFI-7897: Fixed typos in README and also addressed issue that caused parameters with spaces not to be parsed properly

This closes #4669.

Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-24 11:37:45 -05:00
exceptionfactory
dc6ba2541f NIFI-8024 Added null claim check to EncryptedFileSystemRepository.read() to avoid EOFException 2020-11-19 09:11:49 -05:00
humpfhumpf
40fdcd8144 NIFI-7771 - Infinite loop on WebUI when node stopped in cluster (PB1)
This commit resolves "Problem 1" described in issue #7771.

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4496.
2020-11-18 14:56:59 -05:00
Bryan Bende
dcc4fb00a5
NIFI-7888 Added support for authenticating via SAML
- Add dependency on spring-security-saml2-core
- Updated AccessResource with new SAML end-points
- Updated Login/Logout filters to handle SAML scenario
- Updated logout process to track a logout request using a cookie
- Added database storage for cached SAML credential and user groups
- Updated proxied requests when clustered to send IDP groups in a header
- Updated X509 filter to process the IDP groups from the header if present
- Updated admin guide
- Fixed logout action on error page

- Updated UserGroupProvider with a default method for getGroupByName
- Updated StandardManagedAuthorizer to combine groups from request with groups from lookup
- Updated UserGroupProvider implementations with more efficient impl of getGroupByName
- Added/updated unit tests

- Ensure signing algorithm is applied to all signatures and not just metadata signatures
- Added property to specify signature digest algorithm

- Added option to specify whether JDK truststore or NiFi's truststore should be used when connecting to IDP over https
- Added properties to configure connect and read timeouts for http client

- Added URL encoding of issuer when generating JWT to prevent potential issue with the frontend performing base64 decoding

- Made atomic replace methods for storing groups and saml credential in database

- Added properties to control AuthnRequestsSigned and WantAssertionsSigned in the generated service provider metadata

- Dynamically determine the private key alias from the keystore and remove the property for specifying the signing key alias

- Fixed unit test

- Added property to specify an optional identity attribute which would be used instead of NameID

- Cleaned up logging

- Fallback to keystore password when key password is blank

- Make signature and digest default to SHA-256 when no value provided in nifi.properties

This closes #4614
2020-11-18 12:44:02 -05:00
Kevin Doran
a0328ff8d1
NIFI-7744 Add Support for Unicode in X-ProxiedEntitiesChain (#4664)
- Adds detection and encoding of non-ascii characters to creation of chain
- Adds unit tests that use proxied entities with Unicode
2020-11-17 10:23:30 -05:00
s9514171
c57d0abcd3 NIFI-6820 If Processor adjusts a counter but does not interact with any FlowFiles, the counter isn't updated 2020-11-13 11:25:34 -05:00
Nathan Gough
8a4079cd78
NIFI-7993 - Upgraded Jetty dependency and fixed a minor issue with Jetty security filter tests.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4658.
2020-11-13 13:27:12 +01:00
Mark Payne
c79ad1502e
NIFI-7999: Do not call NioAsyncLoadBalanceClient.nodeDisconnected() if node was already in a disconnected state. Doing so was resulting in that method being called constantly on startup, and with the synchronization in place that can result in a huge performance hit on startup. Also updated RemoteQueuePartition to move a small predicate into its own method. This was done because the predicate was previously defined within a synchronized method, which meant that invoking that predicate required synchronization.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4657.
2020-11-12 17:44:46 +01:00
Nathan Gough
479ee6e3db
NIFI-7819 - Added ZooKeeperStateProvider TLS properties.
- Added tests for TLS with ZooKeeperStateProvider.
- Added docs to administration guide.
- Small fixes for PR comments.
- Changed the ZooKeeperStateProvider to receive configuration from the nifi.properties file. Uses the Zookeeper TLS properties or if they are not declared, uses the standard NiFi TLS properties.
- Updated administration-guide.
- Fixed some boolean literalsl. Set the ZooKeeper watcher to null. Removed stacktrace prints to standard out. Added getPreferredProperty for key/truststore types.
- Removing some unused code. Fixing up NiFi properties methods. Removed whitespace.
- Added some tests for getPreferredProperty().
- Checkstyle fixes.
- Passing through nifi properties to the state provider using an annotation to avoid ZooKeeper references in the StateManagerProvider.
- Fixed comment.
- Added CLIENT_SECURE property to isZooKeeperTlsConfigurationPresent() check.
- Small change to getPreferredProperty, added more tests.
- Added checkstyle fix.
- Moved StateProviderContext to nifi-framework-api.
- Changed combine properties to handle null NiFiProperties. Inject NiFiProperties object for tests.
- Checkstyle fix.
- Changed the connect string in state-management.xml to be required. Rearranged order of property validation to validate before initialization.
- Rearranged the way ZooKeeperClientConfig is initialized and added a non blank validator to connect string.
- Minor change to ZooKeeperClientConfig member variable set and get.

This closes #4613.

Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-10 17:05:05 -05:00
Mark Payne
badcfe1ab7
NIFI-7992: Periodically check disk usage for content repo to see if backpressure should be applied. Log progress in background task. Improve performance of background cleanup task by not using an ArrayList Iterator and constantly calling remove but instead wait until the end of our cleanup loop and then removed from the list all elements that should be removed in a single update
This closes #4652.

Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-10 15:27:51 -05:00
Pierre Villard
14ec02f21d
NIFI-7981 - add support for enum type in avro schema
This closes #4648

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-11-05 18:19:55 -05:00
Bence Simon
59e00c4b6f NIFI-7959 Handling node disconnection in MonitorActivity processor
- Make reporting in clustered scope to dependent of expected cluster state in order to prevent unexpected flow file emission

This closes #4642.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-11-04 10:09:46 -05:00
mtien
42c2cda9a2
NIFI-1121 Fixed a dependent value check error.
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:51 -05:00
Mark Payne
4b9014b959
NIFI-1121: Updated backend to perform appropriate validation. Added tests. Updated documentation writer. Updated dev guide to explain how PropertyDescriptor.Builder#dependsOn affects validation. Updated JavaDocs for PropertyDescriptor.Builder#dependsOn
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:42 -05:00
mtien
535cab3167
NIFI-1121: Added an additional check for hidden properties to account for transitive dependent properties.
- Added a 'dependent' attribute to determine whether or not to save dependent property values

Co-authored-by: Scott Aslan <scottyaslan@gmail.com>
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:33 -05:00
mtien
4bd9d7b413
NIFI-1121 Show and hide properties that depend on another property.
Co-authored-by: Scott Aslan <scottyaslan@gmail.com>
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:16 -05:00
Mark Payne
f7f336a4b0
NIFI-1121: Added API changes for having one Property depend on another
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:08 -05:00
mtien
8b78277a45
NIFI-7914 Bumped H2 dependency to 1.4.199.
Bumped icu4j dependency to 60.2.
Replaced jackson-mapper-asl dependency with jackson-databind.
Fixed an error comparing key identities in TestKeyService.
Replaced jackson-mapper-asl ObjectMapper with jackson-databind ObjectMapper in LivySessionController.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4640.
2020-11-03 11:46:29 +01:00
Mark Bean
a115894984 NIFI-7963: fix package name for TestZooKeeperStateServer
This closes #4631

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2020-10-28 21:09:10 -05:00
mtien
bf962f6227 NIFI-7584 Added OIDC logout mechanism.
Added method to validate the OIDC Access Token for the revoke endpoint.
Created a new callback URI of oidc/logoutCallback to handle certain OIDC logout cases.
Changed method to exchange the Authorization Code for a Login Authentication Token.
Added a new method to exchange the AuthN Code for an Access Token.
Changed method to convert OIDC Token to a Login AuthN Token instead of a NiFi JWT.
Created new OidcServiceGroovyTest class.

NIFI-7584-rebase Added test.

NIFI-7584 Fixed a checkstyle issue.

NIFI-7584 Removed a dependency not in use.

NIFI-7584 Made revisions based on PR review.
Refactored revoke endpoint POST request to a private method.
Removed unnecessary dependencies.
Fixed Regex Pattern to search for literal dot character.
Fixed logging the Exception message.
Fixed caught Exception.
Changed timeout value to a static variable.
Changed repeating error messages to a static string.
Reduced sleep duration in unit test.
Refactored cookie generation to private method.

NIFI-7584 Fixed the snapshot version.

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4593.
2020-10-27 17:18:05 -04:00
Mark Payne
bcf625f4c8 NIFI-7920: When node that is connected to cluster is asked to reconnect, ensure that it relinquishes role of Cluster Coordinator and Primary Node 2020-10-27 14:30:13 -04:00
Bryan Bende
e7f244f267
NIFI-7923 Catch Throwable instead of Exception in NarAutoLoaderTask so that thread won't die (#4623) 2020-10-27 12:33:08 -04:00
markap14
67d1b73a85
NIFI-7944: When checking if a Component can reference a Controller Service, NiFi would sometimes fail to find that Controller Service because it was looking based on the wrong Process Group. Fixed that. (#4619) 2020-10-27 08:57:05 -04:00
Peter Turcsanyi
3dd024fb66
NIFI-7825: Support native library loading via absolute path
Use an AspectJ aspect and agent to intercept the load native library calls
at runtime, copy the native library file to temp folder and proceed with
the newly created file in order to provide classloader isolation.

Remove AspectJ jars from lib directory, move the necessary jar to lib/aspectj subdirectory.

This closes #4540.

Signed-off-by: Bryan Bende <bbende@apache.org>
2020-10-26 16:21:16 -04:00
Bence Simon
c8ea7523ef
NIFI-7942 Fixing ordering issue when counter based metrics are added by component
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4621.
2020-10-26 19:31:05 +01:00
Bence Simon
b980a8ea8c NIFI-7549 Adding Hazelcast based DistributedMapCacheClient support
NIFI-7549 Refining documentation; Changing explicit HA mode; Smaller review comments
NIFI-7549 Code review responses about license, documentation and dependencies
NIFI-7549 Fixing issue when explicit HA; Some further review based adjustments
NIFI-7549 Response to code review comments
NIFI-7549 Adding extra serialization test
NIFI-7549 Minor changes based on review comments
NIFI-7549 Adding hook point to the shutdown

This closes #4510.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-10-22 19:22:49 +02:00
Joey Frazee
55cb8d73cb NIFI-7401 Add ZooKeeper client TLS to CuratorLeaderElectionManager
NIFI-7401 Rebased to 1.13.0-SNAPSHOT and simplified tests

NIFI-7401 Added keystore types and changed properties to match nifi.security.*

NIFI-7401 Removed dead code from SecureClientZooKeeperFactory test

NIFI-7401 Renamed bean methods, moved helper code into NiFiProperties

NIFI-7401 Changed connection socket constants to use .class.getName()

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4592.
2020-10-20 11:20:43 -04:00
dependabot[bot]
8aa001a918
Bump junit from 4.13 to 4.13.1
Bumps [junit](https://github.com/junit-team/junit4) from 4.13 to 4.13.1.
- [Release notes](https://github.com/junit-team/junit4/releases)
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.13.1.md)
- [Commits](https://github.com/junit-team/junit4/compare/r4.13...r4.13.1)

Signed-off-by: dependabot[bot] <support@github.com>
2020-10-12 21:54:52 +00:00
Andy LoPresto
82d3fd6cd6 NIFI-7898 Fixed unit test which failed with simultaneous services running on localhost:80. 2020-10-08 08:58:14 -04:00
M Tien
de39a3d9f4
NIFI-7892 Created a Logout page to inform users of a complete logout when OIDC is configured. (#4579) 2020-10-07 12:17:00 -04:00
Matthew Burgess
325a49556c
NIFI-7592: Allow NiFi to be started without a GUI/REST interface (#4509)
* NIFI-7592: Allow NiFi to be started without a GUI/REST interface

* NIFI-7592: Enable all controller services when starting headless

* NIFI-7592: Marked duplicate dependencies as provided

* NIFI-7592: Incorporated additional review comments
2020-10-07 09:12:16 -04:00
Matthew Burgess
7cc3713389
NIFI-7796: Add Prometheus counters for total bytes sent/received (#4522)
* NIFI-7796: Add Prometheus metrics for total bytes sent/received, fixed read/written metrics

* NIFI-7796: Incorporated review comments
2020-10-06 09:26:27 -04:00
markap14
b2489f7644
NIFI-7868: Track number of flowfiles/bytes sent/received/fetched in provenance reporter so that events that are emitted with 'immediate' flag true are still counted (#4564) 2020-09-30 19:58:20 -04:00
Andy LoPresto
9370571131
NIFI-7804 Split nifi-security-utils into sub-module for nifi-security… (#4533)
* NIFI-7804 Split nifi-security-utils into sub-module for nifi-security-utils-api (no external dependencies).
Separated interface and implementation of TlsConfiguration.
Reabsorbed nifi-security-xml-config into nifi-security-utils.

* NIFI-7804 Resolved failing unit test on Java 8.
Removed accidental module dependency.

* NIFI-7804 Resolved failing unit test.

* NIFI-7804 Removed legacy dependency.

* NIFI-7804 Marked nifi-security-utils-api as provided and overrode with compile scope in specific modules which are not children of nifi-standard-services-api-nar.
2020-09-17 12:52:22 -04:00
Shane Ardell
dc4daa2923 NIFI-7742: remove defined and null check
This closes #4524

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2020-09-14 10:50:06 -04:00
Shane Ardell
80bc40a9af NIFI-7742: update case clause logic 2020-09-14 10:03:41 -04:00
Shane Ardell
e3d551e87d NIFI-7742: add case for controller service selections 2020-09-14 10:03:41 -04:00
Mohammed Nadeem
6b34a8b36e
NIFI-7759: Minor misspelling in rollover help for 'Stream When Available' process group configuration
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4521.
2020-09-11 16:41:25 +02:00
Bence Simon
0dff3bc065
NIFI-7429 Adding status history for system level metrics
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4420.
2020-09-10 16:19:11 +02:00