Adds support for NiFI Registry config files to the encrypt-config tool
in NiFi Toolkit.
Also adds decryption capability to encrypt-config tool.
This closes#2376.
Signed-off-by: Andy LoPresto <alopresto@apache.org>
Adds authorizers.xml to the files understood by the encrypt-config
tool in the NiFi Toolkit. If enabled, then the sensitive properties
for LdapUserGroupProvider in authorizers.xml will be encrypted.
Also fixes a bug wherein encrypt-config replaces multiple XML nodes
in login-indentity-providers.xml when LdapProvider is not the first
provider listed in the file.
Enable properties in authorizers.xml to be encrypted by the master key.
This closes#2350.
Signed-off-by: Andy LoPresto <alopresto.apache@gmail.com>
- Refactoring NarDetails to include all info from MANIFEST
- Adding the concept of a Bundle and refactoring NarClassLoaders to pass Bundles to ExtensionManager
- Adding logic to fail start-up when multiple NARs with same coordinates exist, moving Bundle classes to framework API
- Refactoring bundle API to classes and creating BundleCoordinate
- Updating FlowController to use BundleCoordinate
- Updating the UI and DTO model to support showing bundle details that loaded an extension type.
- Adding bundle details for processor canvas node, processor dialogs, controller service dialogs, and reporting task dialogs.
- Updating the formating of the bundle coordinates.
- Addressing text overflow in the configuration/details dialog.
- Fixing self referencing functions.
- Updating extension UI mapping to incorporate bundle coordinates.
- Discovering custom UIs through the supplied bundles.
- Adding verification methods for creating extensions through the rest api.
- Only returning extensions that are common amongst all nodes.
- Rendering the ghost processors using a dotted border.
- Adding bundle details to the flow.xml.
- Loading NiFi build and version details from the framework NAR.
- Removing properties for build and version details.
- Wiring together front end and back end changes.
- Including bundle coordinates in the component data model.
- Wiring together component data model and flow.xml.
- Addressing issue when resolve unvesioned dependent NARs.
Updating unit tests to pass based on framework changes
- Fixing logging of extension types during start up
- Allowing the application to start if there is a compatible bundle found. - Reporting missing bundle when the a compatible bundle is not found. - Fixing table height in new component dialogs.
Fixing chechstyle error and increasing test timeout for TestStandardControllerServiceProvider
- Adding ability to change processor type at runtime
- Adding backend code to change type for controller services
- Cleaning up instance classloaders for temp components.
- Creating a dialog for changing the version of a component.
- Updating the formatting of the component type and bundle throughout.
- Updating the new component dialogs to support selecting source group.
- Cleaning up new component dialogs.
- Cleaning up documentation in the cluster node endpoint.
Adding missing include in nifi-web-ui pom compressor plugin
- Refactoring so ConfigurableComponent provides getLogger() and so the nodes provide the ConfigurableComponent
- Creating LoggableComponent to pass around the component, logger, and coordinate with in the framework
- Finishing clean up following rebase.
Calling lifecycle methods for add and remove when changing versions of a component
- Introducing verifyCanUpdateBundle(coordinate) to ConfiguredComponent, and adding unit tests
- Ensuring documentation is available for all components. Including those of the same type that are loaded from different bundles.
Adding lookup from ClassLoader to Bundle, adding fix for instance class loading to include all parent NARs, and adding additional unit tests for FlowController
- Adding validation to ensure referenced controller services implement the required API
- Fixing template instantiation to look up compatible bundle
- Requiring services/reporting tasks to be disabled/stopped.
- Only supporting a change version option when the item has multiple versions available.
- Limiting the possible new controller services to the applicable API version.
- Showing the implemented API versions for Controller Services.
- Updating the property descriptor tooltip to indicate the required service requirements.
- Introducing version based sorting in the new component dialog, change version dialog, and new controller service dialog.
- Addressing remainder of the issues from recent rebase.
Ensuring bundles have been added to the flow before proposing a flow, and incorporating bundle information into flow fingerprinting
- Refactoring the way missing bundles work to retain the desired bundle if available
- Fixing logger.isDebugEnabled to be logger.isTraceEnabled
- Auditing when user changes the bundle. - Ensuring bundle details are present in templates.
Moving standard prioritizers to framework NAR and refactoring ExtensionManager logic to handle cases where an extension is in a JAR directly in the lib directory
- Ensuring all nodes attempt to instantiate the same template instance when the available bundles may differ. - Fixing the auditing of copy/paste and template instantiation. - Running addtional verification methods when running standalone.
Refactoring controller service invocation handler to allow updating the node used by the invocation handler
- Ensuring the bundles in a proposed flow are compatible with the current instance when the current instance has no flow is going to accept the proposed flow
- Merging whether multiple versions of the component are available
- Setting NAR plugin back to current released version
- Cleaning up DocGenerator to not process multiple times
Addressing incorrect usage of nf.Common. - Using formatType in the new component type dialogs.
Improving error messages when looking for bundles
Addressing comments from PR. - Fixing references to global nf namespace. - Fixing injection of nfProcessGroupConfiguration in nfComponentVersion. - Fixing web api integration tests.
Not rendering unversioned in help documentation. - Ensuring the isExtentionMissing flag is correct after changing the component type.
Adding synchronization in node classes to ensure changing component can't occur when component is running, introducing MissingBundleException for better reporting when a node can't join cluster due to a missing bundle, and bumping NAR plugin to released version 1.2.0
Adding concept of missing components to fingerprinting to ensure nodes agree on missing components when joining a cluster
NIFI-3380: NIFI-3520: - Fixing hive nar dependency. - Marking DBCPService as provided. - Skipping services that require instance classloading and are cobundled with their service API. - Skipping components that require instance classloading and reference service APIs that are cobundled. - Addressing UI issues in the new component dialogs when re-opening with a filter applied.
Fixing checkstyles issue and adding back assume checks to distributed cache server test
Ensuring new component types are sorted correctly when shown initially.
This closes#1585.
Added and updated unit tests.
Cleaned up unnecessary and commented code.
Removed ignored unit tests.
Cleaned up commented code.
This closes#1262.
Signed-off-by: Andy LoPresto <alopresto@apache.org>
Cleaned up commented pom.xml contents. (+4 squashed commits)
Squashed commits:
[725860b] NIFI-3051 Switched CET test logging dependency from logback to log4j as Zookeeper migration has an explicit dependency on log4j as the logging provider and the two libraries were causing classpath collisions. Now the tests run in both Maven and Intellij, the tools all build successfully, and the tools all run (TLS Toolkit, CET, and ZK) without logging provider warnings and print the expected output to the console.
[0e604c7] NIFI-3051 Changed provider element selection in serialize method to be by class (org.apache.nifi.ldap.LdapProvider) rather than identifier in case it has been modified.
Added unit tests.
[300a23d] NIFI-3051 Changed provider element selection to be by class (org.apache.nifi.ldap.LdapProvider) rather than identifier in case it has been modified.
Added unit tests.
[a0cdd40] NIFI-3051 Fixed issue serializing commented or empty login-identity-providers.xml.
Updated and added unit tests. (+1 squashed commit)
Squashed commits:
[b187202] NIFI-3051 - checked in test demonstrating failure to serialize commented ldap-provider section.
This closes#1238.
Signed-off-by: Andy LoPresto <alopresto@apache.org>
Removed logback usage from classpath
Added slf4j-log4j12 dependency in nifi-toolkit pom
Added logback-classic exclusion for nifi-properties-loader used by nifi-toolkit-encrypt-config
Updated log4j.properties logging pattern and logger config in nifi-toolkit-assembly and nifi-toolkit-zookeeper-migrator, filtering zookeeper messages below WARN
Removed logback.groovy since log4j is the single logging implementation
Updated ZooKeeperMigratorMain command line output to match standards established by other tools in nifi-toolkit
This closes#1237.
Signed-off-by: Andy LoPresto <alopresto@apache.org>
Squashed commits:
[5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags.
Exposed master key retrieval code in NiFiPropertiesLoader.
Added logic to decrypt login identity providers XML configuration.
Updated login-identity-providers.xsd to include encryption scheme attribute.
Added unit tests. (+18 squashed commits)
Squashed commits:
[57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted.
Added unit test and resource.
[27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP.
Added comprehensive unit test for LIP & NFP in same test.
[b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing.
[5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP.
Added unit tests (w/o encryption works; w/ does not).
[b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key.
[2d9686c] NIFI-2654 Updated tool description and various logging statements.
Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key.
[8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content.
Added unit tests.
[8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files.
Added unit tests.
[077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements.
Added unit tests and resources.
[d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues.
Removed test resources.
[7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace.
Added unit test.
[b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors.
[6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources.
[d289ffa] NIFI-2654 Added LIP XML decryption.
Added unit tests.
[a482245] NIFI-2654 Added LIP test resources.
[7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided.
[729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired.
Added/updated unit tests.
[7dba5ef] NIFI-2654 Started LIP work (arguments & parsing).
Added unit tests.
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>
This closes#1216
Added test resources with 128-bit encryption for environments without unlimited strength cryptographic jurisdiction policies installed. All tests pass in both 128- and 256-bit environments. (+8 squashed commits)
Squashed commits:
[55f127c] NIFI-2652 Updated Admin Guide with instructions for encrypted config key migration.
[05abf0e] NIFI-2652 Added unit tests for negative cases for migration argument parsing.
Cleaned up TODOs and comments.
[9b73b22] NIFI-2652 Removed SCrypt mock from one unit test that didn't need it. Test pollution is removed and all tests pass.
[d17ea77] NIFI-2652 Removed SCrypt mock from one redundant unit test. One offender remains ignored.
[0924ce0] NIFI-2652 Removed SCrypt mock from one unit test that did not need it. Two offenders remain ignored.
[cb5f850] NIFI-2652 Expanded unit test for combinations into individual tests due to System.exit() only be capturable once per test.
Three tests which mock Scrypt for speed are temporarily ignored to perform test pollution identification.
[c9cc5dc] NIFI-2652 Added logic and unit test for all combinations of original key/password and new key/password.
[19713ec] NIFI-2652 Implemented first pass of key migration logic and provided single comprehensive unit test.
This closes#1186.
joewitt
Kevin Doran
Bryan Bende
Jeff Storck
Matt Gilman
Andy LoPresto
Joseph Percivall