Commit Graph

37 Commits

Author SHA1 Message Date
Jeff Storck af2861f105
NIFI-3300 Implemented usage of ZooKeeper chroot capability in the connect string
Updated ZooKeeper connect string parsing tests
Updated admin doc for ZooKeeper Migrator migration of nifi root nodes, updated source and destination ZK check by servers in the connection string instead of the entire connection string
Added check between source and destination ZooKeeper paths to allow data to be written to the same ZooKeeper with a different path
Added test for writing to the same ZooKeeper with a different path
Added type parameter to server list in for ZooKeeperEndpointConfig

This closes #1456.

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-02-01 17:02:17 -05:00
Jeff Storck 516075de02
NIFI-3158 Added cleanup of resources and refactored exception handling
This closes #1427.

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-01-20 10:17:21 -05:00
Bryan Rosander 97d2d30423
NIFI-3131 - Qualifying nifi-env in nifi bat scripts, removing popd from toolkit bat scripts
This closes #1293

Signed-off-by: Bryan Rosander <brosander@apache.org>
2016-12-05 23:03:41 -05:00
Bryan Rosander d1f2492dec
NIFI-3132 - Using cygpath and quoting paths correctly in sh scripts when run in cygwin
This closes #1292

Signed-off-by: Bryan Rosander <brosander@apache.org>
2016-12-05 22:16:24 -05:00
Jeff Storck 78de10dec0
NIFI-3150 Added logic to wait for the zk client to connect to the configured server
Signed-off-by: Bryan Rosander <brosander@apache.org>
2016-12-05 22:06:00 -05:00
joewitt 92f17a995b NIFI-3100-rc2 prepare for next development iteration 2016-11-25 23:49:27 -05:00
joewitt 5536f690a8 NIFI-3100-rc2 prepare release nifi-1.1.0-RC2 2016-11-25 23:49:13 -05:00
Andy LoPresto 1b2b9f136d
NIFI-2655 Implemented logic check for missing -n or -l arguments.
Added and updated unit tests.
Cleaned up unnecessary and commented code.
Removed ignored unit tests.
Cleaned up commented code.

This closes #1262.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-23 15:12:05 -08:00
Andy LoPresto 2c3714536f
NIFI-3024 Added key migration for sensitive processor properties contained in flow.xml.gz. (nifi.sensitive.props.key)
This closes #1261.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-23 13:26:18 -08:00
Bryan Rosander cdb9b81f42 NIFI-3092 This closes #1268. Double quoting java invocation in bat files with %* 2016-11-23 15:00:40 -05:00
Andy LoPresto 8568d40cd8
NIFI-3051 Fixed issue serializing commented or empty login-identity-providers.xml.
Cleaned up commented pom.xml contents. (+4 squashed commits)
Squashed commits:
[725860b] NIFI-3051 Switched CET test logging dependency from logback to log4j as Zookeeper migration has an explicit dependency on log4j as the logging provider and the two libraries were causing classpath collisions. Now the tests run in both Maven and Intellij, the tools all build successfully, and the tools all run (TLS Toolkit, CET, and ZK) without logging provider warnings and print the expected output to the console.
[0e604c7] NIFI-3051 Changed provider element selection in serialize method to be by class (org.apache.nifi.ldap.LdapProvider) rather than identifier in case it has been modified.
Added unit tests.
[300a23d] NIFI-3051 Changed provider element selection to be by class (org.apache.nifi.ldap.LdapProvider) rather than identifier in case it has been modified.
Added unit tests.
[a0cdd40] NIFI-3051 Fixed issue serializing commented or empty login-identity-providers.xml.
Updated and added unit tests. (+1 squashed commit)
Squashed commits:
[b187202] NIFI-3051 - checked in test demonstrating failure to serialize commented ldap-provider section.

This closes #1238.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-18 01:12:00 -08:00
Jeff Storck 06d7ecd324 NIFI-3059 Adds --ignore-source option to the ZooKeeper Migrator to allow data read from a source zookeeper to be written back to the same zookeeper
Added unit test to test the --ignore-source option

This closes #1242
2016-11-17 18:24:58 -05:00
Bryan Rosander e5eda63705
NIFI-2943 - Toolkit uses JKS type over PKCS12 when creating truststore because non-Bouncy Castle providers cannot read certificates from PKCS12 truststore.
Peer review feedback (+2 squashed commits)
Squashed commits:
[0102c8e] NIFI-2943 - Peer review feedback
[9bcd495] NIFI-2943 - pkcs12 keystore improvements

1. loading pkcs12 keystores with bouncy castle everywhere
2. tls-toolkit client using jks truststore when keystore type is specified differently
3. tests

This closes #1165.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-16 17:13:10 -08:00
Jeff Storck fa13832a9c
NIFI-3049 Fixes logging issues due to logback and log4j being on the classpath
Removed logback usage from classpath
Added slf4j-log4j12 dependency in nifi-toolkit pom
Added logback-classic exclusion for nifi-properties-loader used by nifi-toolkit-encrypt-config
Updated log4j.properties logging pattern and logger config in nifi-toolkit-assembly and nifi-toolkit-zookeeper-migrator, filtering zookeeper messages below WARN
Removed logback.groovy since log4j is the single logging implementation
Updated ZooKeeperMigratorMain command line output to match standards established by other tools in nifi-toolkit

This closes #1237.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-16 15:33:11 -08:00
Andy LoPresto 59fea1cb4e NIFI-2654 Enabled encryption coverage for login-identity-providers.xml.
Squashed commits:
[5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags.

Exposed master key retrieval code in NiFiPropertiesLoader.
Added logic to decrypt login identity providers XML configuration.
Updated login-identity-providers.xsd to include encryption scheme attribute.
Added unit tests. (+18 squashed commits)
Squashed commits:
[57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted.
Added unit test and resource.
[27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP.
Added comprehensive unit test for LIP & NFP in same test.
[b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing.
[5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP.
Added unit tests (w/o encryption works; w/ does not).
[b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key.
[2d9686c] NIFI-2654 Updated tool description and various logging statements.
Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key.
[8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content.
Added unit tests.
[8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files.
Added unit tests.
[077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements.
Added unit tests and resources.
[d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues.
Removed test resources.
[7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace.
Added unit test.
[b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors.
[6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources.
[d289ffa] NIFI-2654 Added LIP XML decryption.
Added unit tests.
[a482245] NIFI-2654 Added LIP test resources.
[7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided.
[729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired.
Added/updated unit tests.
[7dba5ef] NIFI-2654 Started LIP work (arguments & parsing).
Added unit tests.

Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1216
2016-11-14 22:39:06 -05:00
Jeff Storck 7e23734181
NIFI-2957 Corrections of typos, documentation, and strings
Updates made regarding Intellij code inspections

This closes #1218

Signed-off-by: Bryan Rosander <brosander@apache.org>
2016-11-14 16:59:24 -05:00
Jeff Storck ed6e03399f
NIFI-2957 ZooKeeper Migration Toolkit
Reads from and writes to a Zookeeper that is open or secured via user/password digest or SASL
Supports persisting of data obtained from Zookeeper to a file or standard out as JSON
Supports sending of Zookeeper data to Zookeeper from a file or standard out as JSON
Does not allow data obtained from a source Zookeeper to be written back to the same Zookeeper connect string and path

This closes #1193

Signed-off-by: Bryan Rosander <brosander@apache.org>
2016-11-11 14:24:56 -05:00
Bryan Rosander 0cc1765531 NIFI-2823 - Adding nifiDnPrefix, nifiDnSuffix options to standalone tls-toolkit
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1066
2016-11-09 16:49:04 -05:00
Andy LoPresto 89eb2ce28c
NIFI-2652 Provided key migration capability for encrypted config tool.
Added test resources with 128-bit encryption for environments without unlimited strength cryptographic jurisdiction policies installed. All tests pass in both 128- and 256-bit environments. (+8 squashed commits)
Squashed commits:
[55f127c] NIFI-2652 Updated Admin Guide with instructions for encrypted config key migration.
[05abf0e] NIFI-2652 Added unit tests for negative cases for migration argument parsing.
Cleaned up TODOs and comments.
[9b73b22] NIFI-2652 Removed SCrypt mock from one unit test that didn't need it. Test pollution is removed and all tests pass.
[d17ea77] NIFI-2652 Removed SCrypt mock from one redundant unit test. One offender remains ignored.
[0924ce0] NIFI-2652 Removed SCrypt mock from one unit test that did not need it. Two offenders remain ignored.
[cb5f850] NIFI-2652 Expanded unit test for combinations into individual tests due to System.exit() only be capturable once per test.
Three tests which mock Scrypt for speed are temporarily ignored to perform test pollution identification.
[c9cc5dc] NIFI-2652 Added logic and unit test for all combinations of original key/password and new key/password.
[19713ec] NIFI-2652 Implemented first pass of key migration logic and provided single comprehensive unit test.

This closes #1186.
2016-11-07 15:33:10 -08:00
Bryan Rosander 8c09bef4f8 NIFI-2937 - Adding configJsonIn option to tls-toolkit client and server
This closes #1158

Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>
2016-10-25 14:30:04 -04:00
Bryan Rosander 4e13fef724
NIFI-2783 - Site-to-site command line client - s2s.sh shell script warnings -> stderr, usage improvement with examples - Increasing heap settings for s2s cli
This closes #1056.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-09-30 09:12:16 -04:00
Andy LoPresto bc50329d5e
NIFI-2688 Fixed regex pattern for European and Australian time zones in serialized nifi.properties.
This closes #974.
2016-08-31 15:21:04 -07:00
jpercivall 1752a57197 Merge branch 'NIFI-2676-rc1' of https://git-wip-us.apache.org/repos/asf/nifi 2016-08-31 14:01:57 +10:00
Koji Kawamura 6916c5363a
NIFI-2686: Confusing log No hostnames specified
instanceDefinitions.isEmpty() was negate wrongly.

This closes #965.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-08-29 10:43:11 -07:00
Joseph Percivall 1fe18a1567 NIFI-2676-rc1 prepare for next development iteration 2016-08-26 11:40:58 -04:00
Joseph Percivall 74d5224783 NIFI-2676-rc1 prepare release nifi-1.0.0-RC1 2016-08-26 11:40:44 -04:00
Bryan Rosander e9da90812b NIFI-2650 - Adding dependency on nifi-resources to match maven-dependency-plugin
This closes #936

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-24 21:31:29 -04:00
Andy LoPresto 1a9d505b4e NIFI-2640 Fixed Windows encrypt-config.bat script to correctly invoke ConfigEncryptionTool. - Resolved failing Windows tests by adding OS-agnostic file permission read/write methods and new regex for different date formats. - Ignored tests that fail on Windows due to file permission and line ending issues. These are captured in NIFI-2644.
This closes #925

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-24 17:07:35 -04:00
Andy LoPresto c638191a47
NIFI-1831 Added internal logic and command-line tool to allow AES-encrypted sensitive configuration values in nifi.properties.
This closes #834.
2016-08-23 20:34:31 -07:00
Bryan Rosander d4bbd647c5
NIFI-2586 - Javadocs, ExitCode cleanup, password generation improvement, setting cluster secure to true, making properties writer more flexible
This closes #879.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-18 09:26:32 -04:00
Matt Gilman 3f7216ab84 NIFI-2561: - Decoupling kerberos service and spnego principles and keytabs.
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #873
2016-08-17 08:14:05 -04:00
Bryan Rosander 2fd39676a8
NIFI-2569 - Multiple instances on same host, range enhancements, nifi.properties incrementing ports NIFI-2569 - Updating main class in windows bat file
This closes #861.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-16 15:20:46 -04:00
Bryan Rosander fa5da543e6
NIFI-2526 - DN order, multiple standalone runs, client certificates
- Logic for sorting DN, reversing X500Names before using them to generate certificate
- Logging reordered dn
- Accounting for limited crypto pkcs12, allowing password specification for client certificates
- Updating tests to work with or without jce unlimited
- Loading keystore for test in try-with

This closes #824.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-15 10:21:34 -04:00
Bryan Rosander 2dc094765d NIFI-2516 - Extract version info into parent pom, upgrade to commons-io 2.5
NIFI-2516 - Removing slf4j-api, version from slf4j-log4j12

This closes #809.
2016-08-09 22:43:29 +02:00
Joseph Percivall 8d380dcdac NIFI-2497 Fixing testOutputDirectory to be system agnostic
This closes #795
2016-08-05 12:27:09 -04:00
Bryan Rosander d223423de9
NIFI-2485 Added log messages and default values to standalone tool.
NIFI-2485 - Updating standalone success message (+2 squashed commits)
Squashed commits:
[68638bc] NIFI-2485 - Logging successful host processing in standalone
[9b41e0f] NIFI-2485 - Updating standalone defaults, adding logging

This closes #790.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-08-04 13:53:53 -07:00
Bryan Rosander fa4c6ab03c
NIFI-2193 - Added functionality to automate certificate generation, keystore and truststore generation, and nifi.properties keystore and truststore password population. Follow-on changes will be made under NIFI-2476.
This closes #695.

Signed-off-by: Andy LoPresto <alopresto@apache.org>

Defaulting to same keyStore, key password (+18 squashed commits)
Squashed commits:
[9d01ba0] NIFI-2193 - Fixing typo
[55440bc] NIFI-2193 - Standalone can run as long as there are no conflicting files/folders
[0ca34ed] NIFI-2193 - Fixing some filename, absolute path issues
[9d4f65b] NIFI-2193 - Incorporating feedback
[f7550b4] NIFI-2193 - Cleaning up imports
[59a7637] NIFI-2193 - Updating umask to allow owner to execute
[cf824e7] NIFI-2193 - Moving DN arg to CA service specific parent class
[921ee13] NIFI-2193 - Making keystore getInstance more consistent
[a283c4b] NIFI-2193 - Updating sample config files in assembly to reflect new structure
[8d3a21d] NIFI-2193 - Making TlsHelper static, adding option to use same password for Key, KeyStore
[b13d247] NIFI-2193 - Addressing PR feedback
[46ef8ed] NIFI-2193 - Removing commons-logging, log4j from notice
[d4cf41a] NIFI-2193 - Adding option to specify output file for CA certificate when using cli client
[b74bf25] NIFI-2193 - Removing Bouncy Castle from notice
[6e34f9a] NIFI-2193 - Adding CLI client for easier generation of client certificates
[2924fca] NIFI-2193 - nifi-toolkit-ssl -> nifi-toolkit-tls, removing unused constants
[886167e] NIFI-2193 - Adding slf4j to avoid runtime issue
[082de46] NIFI-2193 - Command line SSL config utility as well as certificate authority client/server
2016-08-03 21:34:11 -07:00