Instead, it will look at the ServiceLoader file and read the names of the classes but avoid instantiating all of the objects or loading the classes into memory.
- Updated Doc Generation so that if the documentation for a given NAR already exists, it doesn't delete it and re-generate it. This was necessary because we are no longer instantiating an instance of each component and instead lazily creating the components as necessary.
- Removed stateless version of extension registry because it's no longer necessary
This closes#4852
Signed-off-by: David Handermann <exceptionfactory@apache.org>
NIFI-8260 [WIP] Fixed server side logic to upload a flow file. Cleaned up the front end logic.
NIFI-8260 [WIP] Finished the server side upload logic.
Added a client ID parameter to the endpoint.
Added JSON parsing error response.
Fixed the client side file form to reset after submit.
Fixed the canvas to instantly update and show the process group after submitting the file.
Changed the Add Processor Group dialog UI based on design notes.
Changed the Upload File link to an icon and moved to the process group name input.
Changed the Registry Import link to say 'Import from Registry' and moved to the bottom of the dialog.
Display the filename when a file is selected.
NIFI-8260 [WIP] Added a cancel file button to the Process Group dialog.
Fixed some CSS styles.
NIFI-8260 - Removed accessing the snapshot metadata to avoid an NPE.
Added a title attribute to the html of the dialog file cancel button.
NIFI-8260 - Disabled the dialog 'Add' button.
Revised based on PR feedback.
Refactored the upload file endpoint and client side filename extraction methods.
Fixed some CSS.
Reverted some unnecessary changes.
NIFI-8260 - Revised based on PR feedback.
Refactored uploadProcessGroup.
Fixed some exception handling.
Hid the Upload File button when grouping components.
Refactored nf-ng-group-component.js replacing jquery selectors with variables.
Extracted the resetValues function to clear dialog values.
NIFI-8260 - Fixed the cluster replicate request.
Created a new endpoint to handle the cluster replicate request.
Created ProcessGroupUploadEntity.
Renamed positionX and positionY parameters.
NIFI-8260 - Fixed a checkstyle error.
Removed unnecessary httpServletRequest parameter.
Reverted some re-ordering of imports.
NIFI-8260 - Changed the ProcessGroupUploadEntity to be consistent with other Entity and DTO models.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#4846.
- Configuration based on Sensitive Properties Algorithm defaults to Argon2
- Added SensitiveValueEncoder interface
- Standard implementation uses existing approach with HmacSHA256
This closes#4867
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Added support for PBKDF2 and Scrypt property encryption methods in addition to Argon2
- Refactored StringEncryptor class to PropertyEncryptor interface with implementations
- Added PasswordBasedCipherPropertyEncryptor and KeyedCipherPropertyEncryptor
- Replaced direct instantiation of encryptor with PropertyEncryptorFactory
- Refactored applicable unit tests to use mocked PropertyEncryptor
NIFI-7668 Consolidated similar methods to CipherPropertyEncryptor
NIFI-7668 Updated AbstractTimeBasedSchedulingAgent with PropertyEncryptor
NIFI-7668 Added support for bcrypt secure hashing algorithm
NIFI-7668 Updated comments to clarify implementation of bcrypt key derivation
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#4809.
- Instead of entering a 'synchronized' block for every provenance event, serialize up to 1 MB worth of data, and then enter synchronized block to write that data out. This avoids large amounts of lock contention and context switches
NIFI-7646: Removed TODO and unused Jackson dependency
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#4818
NIFI-8132 Added FileDigestUtils in nifi-nar-utils to avoid dependency on nifi-utils
NIFI-8132 Removed unused imports from NarUnpacker
NIFI-8132 Removed MD5 references from FileUtils documentation
NIFI-8132 Replaced StringBuffer with StringBuilder and made new DigestUtils classes final
NIFI-8132 Replaced Collections.sort() with Stream.sorted()
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#4788.
NIFI-8171 Increased response and idle timeouts for HTTP unit tests
NIFI-8171 Increased TestServer idle timeout to 45 seconds for HTTP unit tests
NIFI-8171 Adjusted timeout and sleep on TestPutTCPCommon.testPruneSenders
NIFI-8171 Increased TestServer idle timeout to 60 seconds and removed 500ms Thread.sleep() in TestInvokeHttpSSL
NIFI-8171 Optimized OkHttpClientUtils to avoid reading trust store twice during initialization
NIFI-8171 Added static variable for server startup sleep
NIFI-8171 Increased TestInvokeHTTP Connect Timeout and TestListenHTTP Response Timeout to 30 seconds
NIFI-8171 Refactored unit tests for InvokeHTTP and ListenHTTP to optimize SSLContext creation
NIFI-8171 Updated TestListenHTTP for static creation of SSLContext
NIFI-8171 Added started check for ListenHTTP Server in TestListenHTTP
NIFI-8171 Refactored TestPutTCP classes to optimize SSLContext creation
NIFI-8171 Increased TestListenHTTP timeout for server start to 120 seconds and added exception when not connected
NIFI-8171 Increased Connect and Read Timeouts for InvokeHTTP SSL unit tests
Signed-off-by: Joe Witt <joewitt@apache.org>
NIFI-7356 - Addresses PR feedback.
NIFI-7356 - Additional changes from PR feedback.
NIFI-7356 - Adding integration tests for ZooKeeperStateServer for TLS.
NIFI-7356 - TLS + Zookeeper now working with single and quorum. Needs code cleanup, need to fix IT tests and docs.
NIFI-7356 - Fixed up tests and removed some irrelevant ones. Refactored some of ZooKeeperStateServer. Tested successfully with a secure and insecure 3 node NiFi + Quorum.
NIFI-7356 - Checkstyle fixes.
NIFI-7356 - Updated administration guide with embedded ZooKeeper TLS configuration.
NIFI-7356 - Updated the way ZooKeeper TLS properties are set/mapped from NiFi properties.
NIFI-7356 - Updated per review, using NiFiProperties keystore strings, classname for ocnnection factory, adjusted TLS configuration checks in NiFiProperties.
NIFI-7356 - Updated configuration validation logic and added tests.
NIFI-7356 - Codestyle check fixes.
NIFI-7356 - Updated some of the log messages.
NIFI-7356 - Updated as per code review.
NIFI-7356 - Fixed max port number.
NIFI-7356 - Updated admin guide and updated small code issues as per code review.
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#4753.
- Added SSLContextService.createContext() and refactored referencing components
- Removed references to ClientAuth from SslContextFactory methods
This closes#4737.
Signed-off-by: Mark Payne <markap14@hotmail.com>
Changed to check the length of all unfiltered properties instead of only filtered properties.
Added additional check if descriptor is a dynamic property.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#4707
- Refactored nifi-stateless to make use of nifi-framework-components
- Removed requirement for nifi-framework-nar to be provided.
- Refactored stateless nifi into api, engine, nar, and bootstrap modules, with a parent 'bundle' module
- Creation of nifi-stateless-system-tests
- Added unit tests and logging
- Changed flow configuration to use properties file instead of json
- Allow for -p parameter to specify parameters on command line
- Moved implementations of Authorizer, NiFiUser, and UserGroupProviders to new module named nifi-framework-authorization-providers so that those that depend on nifi-framework-authorization don't have to bring in the providers. This way, we can have stateless not bring in those providers, as we otherwise get warnings on startup about the provider already being registered. Additionally, it avoids needing dependencies on spring-security-core
- Updated bin/nifi.sh script to run new stateless bootstrap
- Added Reporting Tasks to stateless.
- Download bundles as necessary on stateless nifi startup
NIFI-7897: Addressing review feedback
NIFI-7897: Fixed typos in README and also addressed issue that caused parameters with spaces not to be parsed properly
This closes#4669.
Signed-off-by: Bryan Bende <bbende@apache.org>
- Add dependency on spring-security-saml2-core
- Updated AccessResource with new SAML end-points
- Updated Login/Logout filters to handle SAML scenario
- Updated logout process to track a logout request using a cookie
- Added database storage for cached SAML credential and user groups
- Updated proxied requests when clustered to send IDP groups in a header
- Updated X509 filter to process the IDP groups from the header if present
- Updated admin guide
- Fixed logout action on error page
- Updated UserGroupProvider with a default method for getGroupByName
- Updated StandardManagedAuthorizer to combine groups from request with groups from lookup
- Updated UserGroupProvider implementations with more efficient impl of getGroupByName
- Added/updated unit tests
- Ensure signing algorithm is applied to all signatures and not just metadata signatures
- Added property to specify signature digest algorithm
- Added option to specify whether JDK truststore or NiFi's truststore should be used when connecting to IDP over https
- Added properties to configure connect and read timeouts for http client
- Added URL encoding of issuer when generating JWT to prevent potential issue with the frontend performing base64 decoding
- Made atomic replace methods for storing groups and saml credential in database
- Added properties to control AuthnRequestsSigned and WantAssertionsSigned in the generated service provider metadata
- Dynamically determine the private key alias from the keystore and remove the property for specifying the signing key alias
- Fixed unit test
- Added property to specify an optional identity attribute which would be used instead of NameID
- Cleaned up logging
- Fallback to keystore password when key password is blank
- Make signature and digest default to SHA-256 when no value provided in nifi.properties
This closes#4614
- Added tests for TLS with ZooKeeperStateProvider.
- Added docs to administration guide.
- Small fixes for PR comments.
- Changed the ZooKeeperStateProvider to receive configuration from the nifi.properties file. Uses the Zookeeper TLS properties or if they are not declared, uses the standard NiFi TLS properties.
- Updated administration-guide.
- Fixed some boolean literalsl. Set the ZooKeeper watcher to null. Removed stacktrace prints to standard out. Added getPreferredProperty for key/truststore types.
- Removing some unused code. Fixing up NiFi properties methods. Removed whitespace.
- Added some tests for getPreferredProperty().
- Checkstyle fixes.
- Passing through nifi properties to the state provider using an annotation to avoid ZooKeeper references in the StateManagerProvider.
- Fixed comment.
- Added CLIENT_SECURE property to isZooKeeperTlsConfigurationPresent() check.
- Small change to getPreferredProperty, added more tests.
- Added checkstyle fix.
- Moved StateProviderContext to nifi-framework-api.
- Changed combine properties to handle null NiFiProperties. Inject NiFiProperties object for tests.
- Checkstyle fix.
- Changed the connect string in state-management.xml to be required. Rearranged order of property validation to validate before initialization.
- Rearranged the way ZooKeeperClientConfig is initialized and added a non blank validator to connect string.
- Minor change to ZooKeeperClientConfig member variable set and get.
This closes#4613.
Signed-off-by: Bryan Bende <bbende@apache.org>
- Make reporting in clustered scope to dependent of expected cluster state in order to prevent unexpected flow file emission
This closes#4642.
Signed-off-by: Mark Payne <markap14@hotmail.com>
- Added a 'dependent' attribute to determine whether or not to save dependent property values
Co-authored-by: Scott Aslan <scottyaslan@gmail.com>
Signed-off-by: Bryan Bende <bbende@apache.org>
Bumped icu4j dependency to 60.2.
Replaced jackson-mapper-asl dependency with jackson-databind.
Fixed an error comparing key identities in TestKeyService.
Replaced jackson-mapper-asl ObjectMapper with jackson-databind ObjectMapper in LivySessionController.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#4640.
Added method to validate the OIDC Access Token for the revoke endpoint.
Created a new callback URI of oidc/logoutCallback to handle certain OIDC logout cases.
Changed method to exchange the Authorization Code for a Login Authentication Token.
Added a new method to exchange the AuthN Code for an Access Token.
Changed method to convert OIDC Token to a Login AuthN Token instead of a NiFi JWT.
Created new OidcServiceGroovyTest class.
NIFI-7584-rebase Added test.
NIFI-7584 Fixed a checkstyle issue.
NIFI-7584 Removed a dependency not in use.
NIFI-7584 Made revisions based on PR review.
Refactored revoke endpoint POST request to a private method.
Removed unnecessary dependencies.
Fixed Regex Pattern to search for literal dot character.
Fixed logging the Exception message.
Fixed caught Exception.
Changed timeout value to a static variable.
Changed repeating error messages to a static string.
Reduced sleep duration in unit test.
Refactored cookie generation to private method.
NIFI-7584 Fixed the snapshot version.
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#4593.
Use an AspectJ aspect and agent to intercept the load native library calls
at runtime, copy the native library file to temp folder and proceed with
the newly created file in order to provide classloader isolation.
Remove AspectJ jars from lib directory, move the necessary jar to lib/aspectj subdirectory.
This closes#4540.
Signed-off-by: Bryan Bende <bbende@apache.org>
NIFI-7549 Refining documentation; Changing explicit HA mode; Smaller review comments
NIFI-7549 Code review responses about license, documentation and dependencies
NIFI-7549 Fixing issue when explicit HA; Some further review based adjustments
NIFI-7549 Response to code review comments
NIFI-7549 Adding extra serialization test
NIFI-7549 Minor changes based on review comments
NIFI-7549 Adding hook point to the shutdown
This closes#4510.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
NIFI-7401 Rebased to 1.13.0-SNAPSHOT and simplified tests
NIFI-7401 Added keystore types and changed properties to match nifi.security.*
NIFI-7401 Removed dead code from SecureClientZooKeeperFactory test
NIFI-7401 Renamed bean methods, moved helper code into NiFiProperties
NIFI-7401 Changed connection socket constants to use .class.getName()
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#4592.
* NIFI-7592: Allow NiFi to be started without a GUI/REST interface
* NIFI-7592: Enable all controller services when starting headless
* NIFI-7592: Marked duplicate dependencies as provided
* NIFI-7592: Incorporated additional review comments
* NIFI-7804 Split nifi-security-utils into sub-module for nifi-security-utils-api (no external dependencies).
Separated interface and implementation of TlsConfiguration.
Reabsorbed nifi-security-xml-config into nifi-security-utils.
* NIFI-7804 Resolved failing unit test on Java 8.
Removed accidental module dependency.
* NIFI-7804 Resolved failing unit test.
* NIFI-7804 Removed legacy dependency.
* NIFI-7804 Marked nifi-security-utils-api as provided and overrode with compile scope in specific modules which are not children of nifi-standard-services-api-nar.
Cleaned up JettyServer code.
Changed test logging severity to include debug statements.
Added test resources.
This closes#4498.
Co-authored-by: Kotaro Terada <kotarot@apache.org>
NIFI-7663 Minor changes (variable name refactor, javadoc, GUI message). Merging Drop All Flowfiles responses across all nodes in a cluster.
NIFI-7663 Reloading the canvas after completing a Drop All Flowfiles request.
NIFI-7663 Fixed typos.
This closes#4425.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
Updated documentation.
Added unit tests.
NIFI-7669 Moved time-based encryption tests to integration tests to avoid running during CI builds.
NIFI-7669 Fixed failing test due to nifi.properties initialization.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#4435.
Added unit tests.
NIFI-7680 Duplicated DocumentBuilder creation method in NotificationServiceManager to avoid nifi-bootstrap dependency on nifi-security-utils.
Explicitly added commons-lang3 to lib/bootstrap/ directory in nifi-assembly.
NIFI-7680 Reverted unnecessary dependency changes.
Added explicit dependencies where necessary.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#4436
NIFI-7678: Check if debug is enabled before logging message about processor.onTrigger because obtaining class name is expensive
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#4431.
Logout now deletes signing key by key ID rather than identity.
Validate token expiration now uses mapped identity instead, which allows logging of the mapped identity.
Updated delete key to expect only 0 or 1 keys deleted.
This closes#4416.
Signed-off-by: Andy LoPresto <alopresto@apache.org>
Added unit tests.
Refactored shared logic from various algorithm-specific secure hasher implementations to AbstractSecureHasher.
Introduced secure hasher implementations for various KDFs.
Added custom validation to EncryptContent processor.
Implemented logic for EncryptContent to write operational metadata to flowfile attributes.
Added encryption metadata attribute annotations to EncryptContent.
Added Argon2 KDF documentation and Bcrypt key derivation change notes to Admin Guide.
Updated unit tests to calculate default/recommended cost parameters for Argon2.
This closes#4421.
Co-authored-by: mtien <mtien.apache@gmail.com>
Added exception mapper.
NIFI-7657 Renamed exception & exception mapper to reflect scope of authentication not supported.
Registered exception mapper.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#4418.
Bence Simon
Mark Payne
Bryan Bende
exceptionfactory
mtien
Nathan Gough
Matthew Burgess
exceptionfactory
Lehel Boér
Tamas Palfy
Joe Witt
Pierre Villard
Peter Gyori
Mark Bean
Jon Kessler
Troy Melhase
Andrew Lim
s9514171
Nissim Shiman
tlsmith
sjyang18
Shane Ardell
humpfhumpf
Kevin Doran
Peter Turcsanyi
Joey Frazee
dependabot[bot]
Andy LoPresto
M Tien
Andy LoPresto
Mohammed Nadeem
Kotaro Terada
jmconte
Kristjan Antunovic
si
Scott Aslan
shreeju
Karthikeyan Singaravelan
Jan Hentschel
Vasily Makarov