Commit Graph

300 Commits

Author SHA1 Message Date
Tony Kurc 303f8eabf1 NIFI-1379: Move to 0.5.0-SNAPSHOT, add tkurc code signing key to keys 2016-02-06 08:49:48 -05:00
Aldrin Piri 4df6512126 Merge branch 'NIFI-259' 2016-02-05 14:09:29 -05:00
Andy LoPresto 498b5023ce NIFI-1257 NIFI-1259
Added utility method to return the maximum acceptable password length for PBE ciphers on JVM with limited strength crypto because BC implementation is undocumented (based on empirical evidence).
Updated EncryptionMethod definitions to accurately reflect need for unlimited strength crypto according to algorithm key length.
Added processor logic to invoke keyed cipher.
Added EncryptContent processor property for raw hex key (always visible until NIFI-1121).
Added validations for KDF (keyed and PBE) and hex key.
Added utility method to return list of valid key lengths for algorithm.
Added description to allowable values for KDF and encryption method in EncryptContent processor.
Added IV read/write to KeyedCipherProvider and changed from interface to abstract class.
Added salt read/write logic to NifiLegacy and OpenSSL cipher providers.
Changed RandomIVPBECipherProvider from interface to abstract class.
Updated strong KDF implementations.
Renamed CipherFactory to CipherProviderFactory.
Added unit test for registered KDF resolution from factory.
Updated default iteration count for PBKDF2 cipher provider.
Implemented Scrypt cipher provider.
Added salt translator from mcrypt format to Java format.
Added unit tests for salt formatting and validation.
Added surefire block to groovy unit test profile to enforce 3072 MB heap for Scrypt test.
Added local Java implementation of Scrypt KDF (and underlying PBKDF2 KDF) from Will Glozer.
Defined interface for KeyedCipherProvider.
Implemented AES implementation for KeyedCipherProvider.
Added Ruby script to test/resources for external compatibility check.
Added key length check to PBKDF2 cipher provider.
Changed default PRF to SHA-512.
Added salt and key length check to PBKDF2 cipher provider.
Added utility method to check key length validity for cipher families.
Added Bcrypt implementation.
Implemented PBKDF2 cipher provider.
Added default constructor with strong choices for PBKDF2 cipher provider.
Implemented NiFiLegacyCipherProvider and added unit tests.
Added key length parameter to PBKDF2 cipher provider.
Added PRF resolution to PBKDF2 cipher provider.
Added RandomIVPBECipherProvider to allow for non-deterministic IVs.
Added new keyed encryption methods and added boolean field for compatibility with new KDFs.
Added CipherFactory.
Improved Javadoc in NiFi legacy cipher provider and OpenSSL cipher provider.
Added KeyedCipherProvider interface.
Added OpenSSL PKCS#5 v1.5 EVP_BytesToKey cipher provider and unit test.

This closes #201.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-04 19:40:55 -05:00
Aldrin Piri 6902812678 NIFI-259: Adjusting property description of the connect string to be more specific concerning socket tuple format and the associated default port if one is not specified. Providing minor cleanup and removing unused import. 2016-02-01 10:38:45 -05:00
Bryan Bende b7f7e6ed80 NIFI-1436 This closes #189. Combining stop() and close() into a single method to simplify, and adding checks on stopped flag in the run method of SocketChannelDispatcher and DatagramChannelDispatcher to ensure the run() method exits as soon as possible upon close() being called
NIFI-1436 Adding synchronization on keys set in close() method  based on Selector JavaDoc

Signed-off-by: joewitt <joewitt@apache.org>
2016-01-25 22:50:25 -05:00
Bryan Bende 1089f0a956 NIFI-1273 Adding ListenRELP processor which includes refactoring code that was previously part of ListenSyslog into a reusable framework for implementing listener processors, as well as back-end code for handling the RELP protocol. Addressing some feedback from review and fixing logging statements Addressing review feedback, added AsyncChannelDispatcher and made ChannelHandlerFactory use generics for the dispatcher This closes #179
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-22 16:28:58 -05:00
Mark Payne 4d88aaedc5 NIFI-1258: Added a new function named getDelimitedField to the Expression Language and put together a guide that walks through how to add a new function
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-21 22:09:25 -08:00
Mark Payne 09357297e2 NIFI-259: Ensured that thread pools were being shutdown properly 2016-01-21 12:00:40 -05:00
Mark Payne d2a969e3d6 NIFI-259: Initial implementation of State Management feature 2016-01-11 16:38:52 -05:00
gresockj 34bd2061f7 NIFI-1291: Adding BinFiles in nifi-processor-utilities, deprecating version in nifi-standard-processors. This closes #155.
Signed-off-by: joewitt <joewitt@apache.org>
2016-01-03 23:05:28 -05:00
Oleg Zhurakousky ebcefaac23 NIFI-1289 reverted new method of NiFiProperties in favor of the localized reflection call in test to refresh properties.
Reviewed and Amended (added comments) by Tony Kurc (tkurc@apache.org). This closes #150
2015-12-26 09:58:08 -05:00
Oleg Zhurakousky 2845e93812 NIFI-1289 added support for refreshing properties - Added _getNewInstance()_ operation to NiFiProperties to ensure there is a way to refresh/reload NiFi properties - Fixed javadocs
Signed-off-by: Bryan Bende <bbende@apache.org>

This closes #142
2015-12-23 14:30:08 -05:00
joewitt f4ac8d75c5 NIFI-1312-RC1 prepare for next development iteration 2015-12-19 00:41:04 -05:00
joewitt d624ea4866 NIFI-1312-RC1 prepare release nifi-0.4.1-RC1 2015-12-19 00:40:53 -05:00
Mark Payne bef3fc8b40 NIFI-1301: Ensure that when creating site-to-site connection, if remote instance is applying backpressure that we do not block indefinitely waiting for the connection to be made
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-18 16:08:41 -05:00
Bryan Bende 4249fc943a NIFI-1284 Creating inner class for SiteToSiteClientConfig to fix serialization issue 2015-12-18 13:08:51 -05:00
joewitt d755e43ec8 NIFI-1122_nifi-0.4.0-RC2prepare for next development iteration 2015-12-08 13:00:10 -05:00
joewitt b66c029090 NIFI-1122_nifi-0.4.0-RC2prepare release nifi-0.4.0-RC2 2015-12-08 12:59:59 -05:00
Andy LoPresto f83e6d33c5 NIFI-1242:
Added logic and test resources to debug JCE unlimited strength cryptography policy issues and incorporated into processor property validation.
Excluded test resources from RAT check
Added KeyDerivationFunction enum.
Added kdf property in EncryptContent processor and provided to PasswordBasedEncryptor.
Added logic in PasswordBasedEncryptor to handle variable KDF.
Added unit tests for EncryptContent processor.
Added test resources and excluded from RAT check.

plain.txt: This is a plaintext message.

0s @ 12:20:32 $ openssl enc -aes-256-cbc -e -in plain.txt -out salted_raw.enc -k thisIsABadPassword -p
salt=31DC301A6C7B8A0B
key=CB878A6E167A5B530B8F2BD175E6359E3092AFF7C83274A22A5B421D79E599AC
iv =0C614A72FC06B454B84E035B3FA8F877
0s @ 12:20:44 $ xxd salted_raw.enc
0000000: 5361 6c74 6564 5f5f 31dc 301a 6c7b 8a0b  Salted__1.0.l{..
0000010: 616b c65d f767 504d c085 ba7a c517 d0cb  ak.].gPM...z....
0000020: 7832 211e f573 b6f1 ded2 8f59 88e8 088f  x2!..s.....Y....

0s @ 20:14:00 $ openssl enc -aes-256-cbc -e -in plain.txt -out unsalted_raw.enc -k thisIsABadPassword -p -nosalt
key=711E85689CE7AFF6F410AEA43ABC5446842F685B84879B2E00F977C22B9E9A7D
iv =0C90ABF8ECE84B92BAA2CD448EC760F0
0s @ 20:14:17 $ xxd unsalted_raw.enc
0000000: 70cd 2984 fdbb 0e7c c01b 7206 88b1 6b50  p.)....|..r...kP
0000010: 5eeb e4f3 4036 773b 00ce dd8e 85d8 f90a  ^...@6w;........

This closes #140

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-12-04 19:45:10 -05:00
Mark Payne f378ee9021 NIFI-1249: Allow Processors to add their own variables to those referencable by Expression Language. Make ReplaceText allow users to reference back-references of regex matches 2015-12-04 13:17:37 -05:00
Bryan Bende 4653908b22 NIFI-1227 Adding SSL/TLS support to Syslog processors
-Removing connection creation from PutSyslog onScheduled, and reusing constructor in SSLSocketChannel
-Reducing visiblity of inner classes
2015-12-03 11:42:33 -05:00
Tony Kurc 3a7ddc6a35 NIFI-1054: Fixed DOS line endings in xml, java and js source files
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-01 22:49:51 -05:00
Matt Gilman c073253366 NIFI-655:
- Update admin guide with documentation for username/password authentication.
- Setting default anonymous roles to none.
- Making account status messages to users more clear.
- Deleting user keys when an admin revokes/deletes an account.
- Updating authentication filter to error back whenever authentication fails.
2015-11-25 14:17:23 -05:00
Matt Gilman aaf14c45c9 NIFI-655:
- Refactoring web security to use Spring Security Java Configuration.
- Introducing security in Web UI in order to get JWT.

NIFI-655:
- Setting up the resources (js/css) for the login page.

NIFI-655:
- Adding support for configuring anonymous roles.
- Addressing checkstyle violations.

NIFI-655:
- Moving to token api to web-api.
- Creating an LoginProvider API for user/pass based authentication.
- Creating a module for funneling access to the authorized useres.

NIFI-655:
- Moving away from usage of DN to identity throughout the application (from the user db to the authorization provider).
- Updating the authorized users schema to support login users.
- Creating an extension point for authentication of users based on username/password.

NIFI-655:
- Creating an endpoint for returning the identity of the current user.
- Updating the LoginAuthenticationFilter.

NIFI-655:
- Moving NiFi registration to the login page.
- Running the authentication filters in a different order to ensure we can disambiguate each case.
- Starting to layout each case... Forbidden, Login, Create User, Create NiFi Account.

NIFI-655:
- Addressing checkstyle issues.

NIFI-655:
- Making nf-storage available in the login page.
- Requiring use of local storage.
- Ignoring security for GET requests when obtaining the login configuration.

NIFI-655:
- Adding a new endpoint to obtain the status of a user registration.
- Updated the login page loading to ensure all possible states work.

NIFI-655:
- Ensuring we know the necessary state before we attempt to render the login page.
- Building the proxy chain in the JWT authentication filter.
- Only rendering the login when appropriate.

NIFI-655:
- Starting to style the login page.
- Added simple 'login' support by identifying username/password. Issuing JWT token coming...
- Added logout support
- Rendering the username when appropriate.

NIFI-655:
- Extracting certificate validation into a utility class.
- Fixing checkstyle issues.
- Cleaning up the web security context.
- Removing proxy chain checking where possible.

NIFI-655:
- Starting to add support for registration.
- Creating registration form.

NIFI-655:
- Starting to implement the JWT service.
- Parsing JWT on client side in order to render who the user currently is when logged in.

NIFI-655:
- Allowing the user to link back to the log in page from the new account page.
- Renaming DN to identity where possible.

NIFI-655:
- Fixing checkstyle issues.

NIFI-655:
- Adding more/better support for logging out.

NIFI-655:
- Fixing checkstyle issues.

NIFI-655:
- Adding a few new exceptions for the login identity provider.

NIFI-655:
- Disabling log in by default initially.
- Restoring authorization service unit test.

NIFI-655:
- Fixing checkstyle issues.

NIFI-655:
- Updating packages for log in filters.
- Handling new registration exceptions.
- Code clean up.

NIFI-655:
- Removing registration support.
- Removing file based implementation.

NIFI-655:
- Removing file based implementation.

NIFI-655:
- Removing unused spring configuration files.

NIFI-655:
- Making the auto wiring more explicit.

NIFI-655:
- Removing unused dependencies.

NIFI-655:
- Removing unused filter.

NIFI-655:
- Updating the login API authenticate method to use a richer set of exceptions.
- UI code clean.

NIFI-655:
- Ensuring the login identity provider is able to switch context classloaders via the standard NAR mechanisms.

NIFI-655:
- Initial commit of the LDAP based identity providers.
- Fixed issue when attempting to log into a NiFi that does not support new account requests.

NIFI-655:
- Allowing the ldap provider to specify if client authentication is required/desired.

NIFI-655:
- Persisting keys to sign user tokens.
- Allowing the identity provider to specify the token expiration.
- Code clean up.

NIFI-655:
- Ensuring identities are unique in the key table.

NIFI-655:
- Adding support for specifying the user search base and user search filter in the active directory provider.

NIFI-655:
- Fixing checkstyle issues.

NIFI-655:
- Adding automatic client side token renewal.

NIFI-655:
- Ensuring the logout link is rendered when appropriate.

NIFI-655:
- Adding configuration options for referrals and connect/read timeouts

NIFI-655:
- Added an endpoint for access details including configuration, creating tokens, and checking status.
- Updated DTOs and client side to utilize new endpoints.

NIFI-655:
- Refactoring certificate extraction and validation.
- Refactoring how expiration is specified in the login identity providers.
- Adding unit tests for the access endpoints.
- Code clean up.

NIFI-655:
- Keeping token expiration between 1 minute and 12 hours.

NIFI-655:
- Using the user identity provided by the login identity provider.

NIFI-655: - Fixed typo in error message for unrecognized authentication strategy.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655. - Added logback-test.xml configuration resource for nifi-web-security.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655. - Added issuer field to LoginAuthenticationToken. - Updated AccessResource to pass identity provider class name when creating LoginAuthenticationTokens. - Began refactoring JWT logic from request parsing logic in JwtService. - Added unit tests for JWT logic.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655. - Changed issuer field to use FQ class name because some classes return an empty string for getSimpleName(). - Finished refactoring JWT logic from request parsing logic in JwtService. - Updated AccessResource and JwtAuthenticationFilter to call new JwtService methods decoupled from request header parsing. - Added extensive unit tests for JWT logic.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655:
- Refactoring key service to expose the key id.
- Handling client side expiration better.
- Removing specialized active directory provider and abstract ldap provider.

NIFI-655. - Updated JwtService and JwtServiceTest to use Key POJO instead of raw String key from KeyService.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655:
- Fixing typo when loading the ldap connect timeout.
- Providing a better experience for session expiration.
- Using ellipsis for lengthly user name.
- Adding an issuer to the authentication response so the LIP can specify the appropriate value.

NIFI-655:
- Showing a logging in notification during the log in process.

NIFI-655:
- Removing unnecessary class.

NIFI-655:
- Fixing checkstyle issues.
- Showing the progress spinner while submitting account justification.

NIFI-655:
- Removing deprecated authentication strategy.
- Renaming TLS to START_TLS.
- Allowing the protocol to be configured.

NIFI-655:
- Fixing issue detecting the presence of DN column

NIFI-655:
- Pre-populating the login-identity-providers.xml file with necessary properties and documentation.
- Renaming the Authentication Duration property name.

NIFI-655:
- Updating documentation for the failure response codes.

NIFI-655:
- Ensuring the user identity is not too long.

NIFI-655:
- Updating default authentication expiration to 12 hours.

NIFI-655:
- Remaining on the login form when there is any unsuccessful login attempt.
- Fixing checkstyle issues.
2015-11-23 14:50:13 -05:00
Aldrin Piri 1580edb558 NIFI-1196 Correcting treatment of FETCH events as a continuation of a lineage trail and not the start of a new one.
Reviewed by Tony Kurc (tkurc@apache.org)
2015-11-21 20:21:36 -05:00
Aldrin Piri 08d59e4374 NIFI-1196 Providing handling of FETCH provenance events for their "unique" property, transit URI, within the framework and UI.
Reviewed by Tony Kurc (tkurc@apache.org)
2015-11-19 17:42:15 -05:00
Tony Kurc 528dab78d6 NIFI-1073: Fixing coverity discovered errors. Resource leaks, and statics
Reviewed by Bryan Bende (bbende@apache.org)
2015-11-14 18:56:43 -05:00
Mark Payne 358cabafa3 NIFI-1142: Added typo that referenced the wrong function name when indicating that isEmpty has the wrong number of arguments 2015-11-10 10:55:08 -05:00
Mark Payne 73c1671975 NIFI-1132: Limited number of Lineage Identifiers held to 100 and marked the getLineageIdentifiers() method as deprecated 2015-11-09 12:09:56 -05:00
Mark Payne 5f8fdae909 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2015-11-09 12:07:45 -05:00
Mark Payne 59a49aea12 Revert "NIFI-1082: Ensure that events returned from the provenance repository are ordered such that newest events are provided first"
This reverts commit cf8ca3dc2c.
2015-11-08 21:46:06 -05:00
joewitt 99629646fe NIFI-1122 moved to 0.4.0-SNAPSHOT 2015-11-06 23:41:15 -05:00
Joseph Percivall a549621267 NIFI-1077 enabling expression language on ConvertCharacterSet input and output
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-02 11:46:17 -05:00
Mark Payne 814e8b212c Merge branch 'NIFI-1082' 2015-11-02 11:29:52 -05:00
Joseph Percivall 37e2f178f8 NIFI-1068 Fix EndpointConnectionPool to properly remove connections from activeConnections when terminating connections
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-01 14:47:23 -05:00
Mark Payne cf8ca3dc2c NIFI-1082: Ensure that events returned from the provenance repository are ordered such that newest events are provided first 2015-10-28 17:32:51 -04:00
Mark Payne 193dbe6533 Merge branch 'NIFI-604' of https://github.com/rickysaltzer/nifi into NIFI-604 2015-09-23 16:03:46 -04:00
Matt Gilman ded74ec94c NIFI-876 prepare for next development iteration 2015-09-14 21:48:11 -04:00
Matt Gilman 2ec735e350 NIFI-876 prepare release nifi-0.3.0-RC1 2015-09-14 21:48:00 -04:00
Mark Payne e4e263c292 NIFI-914: If no krb5 file set in nifi properties, treat it the same as an empty string set 2015-08-31 15:18:33 -04:00
Mark Payne a8b063d61b NIFI-902: Ensure that we close the underlying file stream when we roll over a partition instead of the bufferedoutputstream, which could cause corruption of there was a failure to flush the entire buffer previously. 2015-08-30 19:48:19 -04:00
Mark Payne 5de37f63d9 NIFI-902: Ensure that if we get an IOException during rollover of WAL, we are able to recover 2015-08-28 10:04:58 -04:00
ricky 9cefc4a5a5 NIFI-604: Custom Argument Delimiters ExecuteStreamCommand / ExecuteProcess
- Unified the way ExecuteStreamCommand and ExecuteProcess handle arguments
- Argument delimiters can now be specified. Their default being what they were using before (; and space)
2015-08-27 15:26:56 -04:00
Mark Payne 4baffacc42 NIFI-892: If nifi.flowfile.repository.partitions property is changed, but repository already exists, just previous value 2015-08-25 09:58:37 -04:00
ricky 7fb6e884a7 NIFI-866: Add Kerberos Support for Hadoop
- Add krb5.conf to nifi.properties

	nifi.kerberos.krb5.file | path to krb5.conf

- Connections to secure Hadoop clusters will be determined by their config,
  that is, hadoop.security.authentication should be set to kerberos.

- Added two optional arguments to AbstractHadoopProcessor (principal and keytab),
  these are only required if the cluster you're connecting to is secured. Both of
  these options require the krb5.conf to be present in nifi.properties.

Signed-off-by: Bryan Bende <bbende@apache.org>
2015-08-24 15:36:22 -04:00
Mark Payne 68d94cc01b NIFI-744: Refactored ContentClaim into ContentClaim and ResourceClaim and allowed resource claim to be reused across sessions 2015-08-21 11:08:34 -04:00
Mark Payne 8ebf1f03c2 Merge branch 'nifi-868' of https://github.com/pukkaone/nifi into NIFI-868 2015-08-19 12:56:26 -04:00
Mark Payne 19f7db6986 NIFI-793: Added multi-threading to the indexing in the Persistent Provenance Repository 2015-08-19 12:24:25 -04:00
Chin Huang 8f98f80938 NIFI-868 Configure advertised host name of Remote Process Group Input Port
Before this change, the host given out to clients to connect to a Remote
Process Group Input Port is the host where the NiFi instance runs.
However, sometimes the binding host is different from the host that
clients connect to. For example, when a NiFi instance runs inside a
Docker container, a client on a separate machine must connect to the
Docker host which forwards the connection to the container.

Add a configuration property to specify the host name to give out to
clients to connect to a Remote Process Group Input Port. If the property
is not configured, then give out the name of host running the NiFi
instance.
2015-08-18 21:35:28 -07:00
joewitt aa99884782 NIFI-850 removed nifi parent, updated nifi pom, moved all nifi subdirs up one level, fixed readme. 2015-08-15 13:12:22 -04:00