Commit Graph

97 Commits

Author SHA1 Message Date
Oleg Zhurakousky 812da19cad NIFI-614 Added initial support for new style JMS
NIFI-614 finalized JMSConnectionFactoryProvider ControllerService

NIFI-614 finalized implementation of both Processors and ControllerService

NIFI-614 added initial documentation

NIFI-614 addressed PR comment with unused import and squashed

NIFI-614 added @OnDisabled method

NIFI-614 changed POMs to 0.6

NIFI-614 removed local .gitignore

NIFI-614 added support for parsing Tibco URL

NIFI-614 removed setting of jms message id

NIFI-614 addressed PR comments, fixed tests

NIFI-614 addressed latest PR comments

NIFI-614 second round of PR comments addressed

NIFI-614 3rd round of PR comments addressed

NIFI-614 finalizing on PR comments

NIFI-614 more PR comments

This closes #222

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-03-15 15:48:03 -04:00
Bryan Bende 6f5fb59479 NIFI-1420 Adding Splunk bundle containing PutSplunk, and GetSplunk, and adding a ListenTCP processor to standard processors. Refactored internal code from PutSyslog to create a generic AbstractPutEventProcessor which PutSplunk extends from.
This closes #233
2016-03-07 18:21:17 -05:00
Bryan Bende 9576f41603 NIFI-1529 Forcing hadoop-libraries-nar to version 4.2.5 of httpclient and httpcore since that is what hadoop-client 2.6.2 normally brings in - Removing dependencies from the nar pom that should be transitively brought in by hadoop-client - Updating NOTICE files in hadoop-libraries-nar and nifi-assembly according to dependency changes
This closes #236
2016-02-19 16:35:49 -05:00
Tony Kurc ad73b5c9d9 NIFI-1379-RC3 prepare for next development iteration 2016-02-12 17:28:10 -05:00
Tony Kurc 8309dba80b NIFI-1379-RC3 prepare release nifi-0.5.0-RC3 2016-02-12 17:27:59 -05:00
Tony Kurc cae5b109c0 NIFI-1507: Update bouncycastle license reference 2016-02-12 16:58:26 -05:00
joewitt f44eb643dd NIFI-1461 added run.as filtered property for RPM and removed extraneous properties 2016-02-10 21:21:19 -05:00
puspendu.banerjee@gmail.com e00652e625 NIFI-1461 This closes #204. Implemented RPM before/after logic to create user and more as listed in JIRA
Signed-off-by: joewitt <joewitt@apache.org>
2016-02-10 21:21:10 -05:00
Tony Kurc 5be83166ee NIFI-1485: Updated 'header' section of NiFi NOTICE files 2016-02-07 13:52:00 -05:00
Tony Kurc 303f8eabf1 NIFI-1379: Move to 0.5.0-SNAPSHOT, add tkurc code signing key to keys 2016-02-06 08:49:48 -05:00
Aldrin Piri 4df6512126 Merge branch 'NIFI-259' 2016-02-05 14:09:29 -05:00
Andy LoPresto 498b5023ce NIFI-1257 NIFI-1259
Added utility method to return the maximum acceptable password length for PBE ciphers on JVM with limited strength crypto because BC implementation is undocumented (based on empirical evidence).
Updated EncryptionMethod definitions to accurately reflect need for unlimited strength crypto according to algorithm key length.
Added processor logic to invoke keyed cipher.
Added EncryptContent processor property for raw hex key (always visible until NIFI-1121).
Added validations for KDF (keyed and PBE) and hex key.
Added utility method to return list of valid key lengths for algorithm.
Added description to allowable values for KDF and encryption method in EncryptContent processor.
Added IV read/write to KeyedCipherProvider and changed from interface to abstract class.
Added salt read/write logic to NifiLegacy and OpenSSL cipher providers.
Changed RandomIVPBECipherProvider from interface to abstract class.
Updated strong KDF implementations.
Renamed CipherFactory to CipherProviderFactory.
Added unit test for registered KDF resolution from factory.
Updated default iteration count for PBKDF2 cipher provider.
Implemented Scrypt cipher provider.
Added salt translator from mcrypt format to Java format.
Added unit tests for salt formatting and validation.
Added surefire block to groovy unit test profile to enforce 3072 MB heap for Scrypt test.
Added local Java implementation of Scrypt KDF (and underlying PBKDF2 KDF) from Will Glozer.
Defined interface for KeyedCipherProvider.
Implemented AES implementation for KeyedCipherProvider.
Added Ruby script to test/resources for external compatibility check.
Added key length check to PBKDF2 cipher provider.
Changed default PRF to SHA-512.
Added salt and key length check to PBKDF2 cipher provider.
Added utility method to check key length validity for cipher families.
Added Bcrypt implementation.
Implemented PBKDF2 cipher provider.
Added default constructor with strong choices for PBKDF2 cipher provider.
Implemented NiFiLegacyCipherProvider and added unit tests.
Added key length parameter to PBKDF2 cipher provider.
Added PRF resolution to PBKDF2 cipher provider.
Added RandomIVPBECipherProvider to allow for non-deterministic IVs.
Added new keyed encryption methods and added boolean field for compatibility with new KDFs.
Added CipherFactory.
Improved Javadoc in NiFi legacy cipher provider and OpenSSL cipher provider.
Added KeyedCipherProvider interface.
Added OpenSSL PKCS#5 v1.5 EVP_BytesToKey cipher provider and unit test.

This closes #201.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-04 19:40:55 -05:00
Oleg Zhurakousky dc8b62c3a6 NIFI-865 Added processors to Get and Put to/from AMQP-based messaging systems
This closes #200

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-02-02 20:57:10 -05:00
Matt Burgess 75af3a2ebe NIFI-1275: Added FetchElasticsearch processor and support for secure clusters
This closes #180.

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-02-02 18:21:31 -05:00
Matt Burgess 943d0a6e53 Adding to PutElasticsearch - validators, unit tests, etc. 2016-02-02 17:26:40 -05:00
Matt Burgess 77ab5d3684 NIFI-210: This closes #185. Add ExecuteScript and InvokeScriptedProcessor
Signed-off-by: joewitt <joewitt@apache.org>
2016-02-01 21:09:06 -05:00
joewitt 06d621f358 NIFI-1454 fixed rpm name, bootstrap deps, updated rpm plugin version, removed extraneous license and notice 2016-01-31 20:19:59 -05:00
Mark Payne f6ec437bc7 Merge branch 'master' into NIFI-259 2016-01-21 09:32:02 -05:00
Mark Payne 2c9fb676cd NIFI-1156: Updated version from 0.4.0-SNAPSHOT to 0.4.2-SNAPSHOT; addressed some error handling issues; updated licensing information 2016-01-20 15:27:58 -05:00
Jeremy Dyer ee7400ef53 NIFI-1156 2016-01-13 15:59:47 -05:00
Mark Payne d2a969e3d6 NIFI-259: Initial implementation of State Management feature 2016-01-11 16:38:52 -05:00
ricky c45060f703 NIFI-987 Added Riemann (PutRiemann) Reporting
- Introduced nifi-riemann-bundle for future Riemann backed monitoring

- Added initial PutRiemann processor for writing events to Riemann
  using the Riemann batch client.

	- Values for events are provided using the NiFi expression language
		e.g. Metric -> ${latency.milliseconds:divide(1000)}
2015-12-28 15:45:11 -05:00
joewitt f4ac8d75c5 NIFI-1312-RC1 prepare for next development iteration 2015-12-19 00:41:04 -05:00
joewitt d624ea4866 NIFI-1312-RC1 prepare release nifi-0.4.1-RC1 2015-12-19 00:40:53 -05:00
joewitt d755e43ec8 NIFI-1122_nifi-0.4.0-RC2prepare for next development iteration 2015-12-08 13:00:10 -05:00
joewitt b66c029090 NIFI-1122_nifi-0.4.0-RC2prepare release nifi-0.4.0-RC2 2015-12-08 12:59:59 -05:00
Matt Gilman 1f4ef9f5fa NIFI-1266:
- Adding license file for LDAP provider NAR.
- Adding spring security to NOTICE where appropriate.
2015-12-07 15:04:09 -05:00
Jeremy Dyer 3e26ed8105 Merge remote-tracking branch 'apache/master' into NIFI-1156 2015-11-30 10:12:39 -05:00
Matt Gilman c073253366 NIFI-655:
- Update admin guide with documentation for username/password authentication.
- Setting default anonymous roles to none.
- Making account status messages to users more clear.
- Deleting user keys when an admin revokes/deletes an account.
- Updating authentication filter to error back whenever authentication fails.
2015-11-25 14:17:23 -05:00
Matt Gilman aaf14c45c9 NIFI-655:
- Refactoring web security to use Spring Security Java Configuration.
- Introducing security in Web UI in order to get JWT.

NIFI-655:
- Setting up the resources (js/css) for the login page.

NIFI-655:
- Adding support for configuring anonymous roles.
- Addressing checkstyle violations.

NIFI-655:
- Moving to token api to web-api.
- Creating an LoginProvider API for user/pass based authentication.
- Creating a module for funneling access to the authorized useres.

NIFI-655:
- Moving away from usage of DN to identity throughout the application (from the user db to the authorization provider).
- Updating the authorized users schema to support login users.
- Creating an extension point for authentication of users based on username/password.

NIFI-655:
- Creating an endpoint for returning the identity of the current user.
- Updating the LoginAuthenticationFilter.

NIFI-655:
- Moving NiFi registration to the login page.
- Running the authentication filters in a different order to ensure we can disambiguate each case.
- Starting to layout each case... Forbidden, Login, Create User, Create NiFi Account.

NIFI-655:
- Addressing checkstyle issues.

NIFI-655:
- Making nf-storage available in the login page.
- Requiring use of local storage.
- Ignoring security for GET requests when obtaining the login configuration.

NIFI-655:
- Adding a new endpoint to obtain the status of a user registration.
- Updated the login page loading to ensure all possible states work.

NIFI-655:
- Ensuring we know the necessary state before we attempt to render the login page.
- Building the proxy chain in the JWT authentication filter.
- Only rendering the login when appropriate.

NIFI-655:
- Starting to style the login page.
- Added simple 'login' support by identifying username/password. Issuing JWT token coming...
- Added logout support
- Rendering the username when appropriate.

NIFI-655:
- Extracting certificate validation into a utility class.
- Fixing checkstyle issues.
- Cleaning up the web security context.
- Removing proxy chain checking where possible.

NIFI-655:
- Starting to add support for registration.
- Creating registration form.

NIFI-655:
- Starting to implement the JWT service.
- Parsing JWT on client side in order to render who the user currently is when logged in.

NIFI-655:
- Allowing the user to link back to the log in page from the new account page.
- Renaming DN to identity where possible.

NIFI-655:
- Fixing checkstyle issues.

NIFI-655:
- Adding more/better support for logging out.

NIFI-655:
- Fixing checkstyle issues.

NIFI-655:
- Adding a few new exceptions for the login identity provider.

NIFI-655:
- Disabling log in by default initially.
- Restoring authorization service unit test.

NIFI-655:
- Fixing checkstyle issues.

NIFI-655:
- Updating packages for log in filters.
- Handling new registration exceptions.
- Code clean up.

NIFI-655:
- Removing registration support.
- Removing file based implementation.

NIFI-655:
- Removing file based implementation.

NIFI-655:
- Removing unused spring configuration files.

NIFI-655:
- Making the auto wiring more explicit.

NIFI-655:
- Removing unused dependencies.

NIFI-655:
- Removing unused filter.

NIFI-655:
- Updating the login API authenticate method to use a richer set of exceptions.
- UI code clean.

NIFI-655:
- Ensuring the login identity provider is able to switch context classloaders via the standard NAR mechanisms.

NIFI-655:
- Initial commit of the LDAP based identity providers.
- Fixed issue when attempting to log into a NiFi that does not support new account requests.

NIFI-655:
- Allowing the ldap provider to specify if client authentication is required/desired.

NIFI-655:
- Persisting keys to sign user tokens.
- Allowing the identity provider to specify the token expiration.
- Code clean up.

NIFI-655:
- Ensuring identities are unique in the key table.

NIFI-655:
- Adding support for specifying the user search base and user search filter in the active directory provider.

NIFI-655:
- Fixing checkstyle issues.

NIFI-655:
- Adding automatic client side token renewal.

NIFI-655:
- Ensuring the logout link is rendered when appropriate.

NIFI-655:
- Adding configuration options for referrals and connect/read timeouts

NIFI-655:
- Added an endpoint for access details including configuration, creating tokens, and checking status.
- Updated DTOs and client side to utilize new endpoints.

NIFI-655:
- Refactoring certificate extraction and validation.
- Refactoring how expiration is specified in the login identity providers.
- Adding unit tests for the access endpoints.
- Code clean up.

NIFI-655:
- Keeping token expiration between 1 minute and 12 hours.

NIFI-655:
- Using the user identity provided by the login identity provider.

NIFI-655: - Fixed typo in error message for unrecognized authentication strategy.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655. - Added logback-test.xml configuration resource for nifi-web-security.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655. - Added issuer field to LoginAuthenticationToken. - Updated AccessResource to pass identity provider class name when creating LoginAuthenticationTokens. - Began refactoring JWT logic from request parsing logic in JwtService. - Added unit tests for JWT logic.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655. - Changed issuer field to use FQ class name because some classes return an empty string for getSimpleName(). - Finished refactoring JWT logic from request parsing logic in JwtService. - Updated AccessResource and JwtAuthenticationFilter to call new JwtService methods decoupled from request header parsing. - Added extensive unit tests for JWT logic.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655:
- Refactoring key service to expose the key id.
- Handling client side expiration better.
- Removing specialized active directory provider and abstract ldap provider.

NIFI-655. - Updated JwtService and JwtServiceTest to use Key POJO instead of raw String key from KeyService.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655:
- Fixing typo when loading the ldap connect timeout.
- Providing a better experience for session expiration.
- Using ellipsis for lengthly user name.
- Adding an issuer to the authentication response so the LIP can specify the appropriate value.

NIFI-655:
- Showing a logging in notification during the log in process.

NIFI-655:
- Removing unnecessary class.

NIFI-655:
- Fixing checkstyle issues.
- Showing the progress spinner while submitting account justification.

NIFI-655:
- Removing deprecated authentication strategy.
- Renaming TLS to START_TLS.
- Allowing the protocol to be configured.

NIFI-655:
- Fixing issue detecting the presence of DN column

NIFI-655:
- Pre-populating the login-identity-providers.xml file with necessary properties and documentation.
- Renaming the Authentication Duration property name.

NIFI-655:
- Updating documentation for the failure response codes.

NIFI-655:
- Ensuring the user identity is not too long.

NIFI-655:
- Updating default authentication expiration to 12 hours.

NIFI-655:
- Remaining on the login form when there is any unsuccessful login attempt.
- Fixing checkstyle issues.
2015-11-23 14:50:13 -05:00
Joseph Percivall 1e5cc070a3 NIFI-1081 Adding option to ExecuteStreamCommand to put output value to an attribute
Reviewed and amended (comments,whitespace,and some code readability (discussed in ticket)) by Tony Kurc (tkurc@apache.org)
2015-11-18 23:23:10 -05:00
Jeremy Dyer c82fc18f8e HTML Parsing Processors Bundle
NIFI-1156 HTML Parsing Processors Bundle
2015-11-13 15:01:10 -05:00
Mark Payne 56ad22aea6 NIFI-900: Created Processors for interacting with Microsoft Azure EventHubs
Reviewed (with amendments needed for clean merge, whitespace and NOTICEs) by Tony Kurc (tkurc@apache.org)
2015-11-11 23:06:04 -05:00
Bryan Bende e748fd5848 NIFI-817 Processors for interacting with HBase
- Refactoring PutHBaseCell to batch Puts by table
- Adding optional Columns property to GetHBase to return only selected column families or columns
- Making GetHBase cluster friendly by storing state in the distributed cache and a local file
- Adding Initial Time Range property to GetHBase
- Adding Filter Expression property and custom validate to prevent using columns and a filter at the same time
- Creating an HBaseClientService controller service to isolate the HBase client and support multiple versions
- Creating appropriate LICENSE/NOTICE files
- Adding @InputRequirement to processors
- Addressing comments from review, moving hbase client services under standard services
- Making sure result of session.penalize() is assinged to FlowFile variable before transferring
2015-11-10 11:13:51 -05:00
joewitt 64b1fc983c NIFI-925 reviewed patch/ builds well/ fixed licensing gaps. 2015-11-08 22:15:11 -05:00
joewitt 99629646fe NIFI-1122 moved to 0.4.0-SNAPSHOT 2015-11-06 23:41:15 -05:00
ijokarumawak 2466a24530 nifi-992 Adding nifi-couchbase-bundle.
- new CouchbaseClusterControllerService
- new Processors
  - GetCouchbaseKey
  - PutCouchbaseKey

Signed-off-by: Bryan Bende <bbende@apache.org>
2015-09-28 11:21:42 -04:00
Matt Gilman ded74ec94c NIFI-876 prepare for next development iteration 2015-09-14 21:48:11 -04:00
Matt Gilman 2ec735e350 NIFI-876 prepare release nifi-0.3.0-RC1 2015-09-14 21:48:00 -04:00
Mark Payne 992e841027 NIFI-948: Provide capability to register notifications for nifi lifecycle events 2015-09-13 19:55:12 -04:00
ricky 7fb6e884a7 NIFI-866: Add Kerberos Support for Hadoop
- Add krb5.conf to nifi.properties

	nifi.kerberos.krb5.file | path to krb5.conf

- Connections to secure Hadoop clusters will be determined by their config,
  that is, hadoop.security.authentication should be set to kerberos.

- Added two optional arguments to AbstractHadoopProcessor (principal and keytab),
  these are only required if the cluster you're connecting to is secured. Both of
  these options require the krb5.conf to be present in nifi.properties.

Signed-off-by: Bryan Bende <bbende@apache.org>
2015-08-24 15:36:22 -04:00
Mark Payne a9c2bf58b8 NIFI-844: Initial import of the ConvertAvroToJSON processor 2015-08-21 17:11:56 -04:00
Joe Percivall d992730fb7 NIFI-864 first contrib to nifi supports working with images and addressing comments
Committing modified files

Addressing code review comments

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-08-21 08:56:41 -04:00
Mark Payne 19f7db6986 NIFI-793: Added multi-threading to the indexing in the Persistent Provenance Repository 2015-08-19 12:24:25 -04:00
Bryan Bende 1c34478eb8 NIFI-790 Create reporting task to deliver metrics to Apache Ambari
- Adding NOTICE for nifi-ambari-nar, and fixing formatting issues in nifi-ambari-reporting-task pom
- Addressing review feedback: updating assembly NOTICE, fixing unit test, and minor clean up
- Adding additionalDetails.html
2015-08-17 15:47:49 -04:00
joewitt a837fa5f38 NIFI-801 fixed reference to BSD and name of NOTICE 2015-08-16 22:42:28 -04:00
joewitt aa99884782 NIFI-850 removed nifi parent, updated nifi pom, moved all nifi subdirs up one level, fixed readme. 2015-08-15 13:12:22 -04:00