Commit Graph

125 Commits

Author SHA1 Message Date
Oleg Zhurakousky 3f295b13c9 NIFI-1519 upgraded to the latest version of Spark 1.6.0
Signed-off-by: Matt Burgess <mattyb149@gmail.com>
2016-03-15 10:03:32 -04:00
joewitt f6705f234c NIFI-1511 we now ensure groovy eclipse compiler is only used during testCompile goal
This closes #270

Signed-off-by: Matt Burgess <mattyb149@gmail.com>
2016-03-11 11:15:15 -05:00
Bryan Bende 6f5fb59479 NIFI-1420 Adding Splunk bundle containing PutSplunk, and GetSplunk, and adding a ListenTCP processor to standard processors. Refactored internal code from PutSyslog to create a generic AbstractPutEventProcessor which PutSplunk extends from.
This closes #233
2016-03-07 18:21:17 -05:00
Aldrin Piri 58e50ae39d NIFI-1511 Incorporating Groovy unit tests as part of the main build and providing explicit compile scope on the Groovy dependency for the scripting bundle. This closes #220
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-02-26 10:25:45 -05:00
Bryan Bende 9576f41603 NIFI-1529 Forcing hadoop-libraries-nar to version 4.2.5 of httpclient and httpcore since that is what hadoop-client 2.6.2 normally brings in - Removing dependencies from the nar pom that should be transitively brought in by hadoop-client - Updating NOTICE files in hadoop-libraries-nar and nifi-assembly according to dependency changes
This closes #236
2016-02-19 16:35:49 -05:00
Tony Kurc ad73b5c9d9 NIFI-1379-RC3 prepare for next development iteration 2016-02-12 17:28:10 -05:00
Tony Kurc 8309dba80b NIFI-1379-RC3 prepare release nifi-0.5.0-RC3 2016-02-12 17:27:59 -05:00
Tony Kurc 0849489a87 NIFI-1379: increasing timeout of staging plugin for release 2016-02-06 16:51:34 -05:00
Tony Kurc 303f8eabf1 NIFI-1379: Move to 0.5.0-SNAPSHOT, add tkurc code signing key to keys 2016-02-06 08:49:48 -05:00
Aldrin Piri 4df6512126 Merge branch 'NIFI-259' 2016-02-05 14:09:29 -05:00
Oleg Zhurakousky dc8b62c3a6 NIFI-865 Added processors to Get and Put to/from AMQP-based messaging systems
This closes #200

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-02-02 20:57:10 -05:00
Matt Burgess 943d0a6e53 Adding to PutElasticsearch - validators, unit tests, etc. 2016-02-02 17:26:40 -05:00
Matt Burgess 77ab5d3684 NIFI-210: This closes #185. Add ExecuteScript and InvokeScriptedProcessor
Signed-off-by: joewitt <joewitt@apache.org>
2016-02-01 21:09:06 -05:00
joewitt 06d621f358 NIFI-1454 fixed rpm name, bootstrap deps, updated rpm plugin version, removed extraneous license and notice 2016-01-31 20:19:59 -05:00
Andy LoPresto 93aac8cff3 NIFI-1365
Added Groovy support for unit tests to pom with skeleton test.
Added Groovy unit tests for OCSPCertificateValidator.
Implemented positive & negative unit tests with cache injection for valid/revoked OCSP certificate.
Modified pom.xml to support Groovy unit tests with custom variable.

mvn clean test -Dgroovy=test

Added local cache injection into Groovy tests for OCSP certificate validation (see NIFI-1324 and NIFI-1364).
Set Java version to 1.7 for Groovy test src/target.
Moved Groovy unit test profile from nifi-web-security to root pom.
Added null check for algorithm argument in PGPUtil.
Changed buffer length check from ">= 0" to "> -1" because it was confusing other developers.
Resolved contrib-check line length issues.
Fixed contrib-check issues in OpenPGPKeyBasedEncryptorTest.
This closes #163

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-26 17:37:59 -05:00
Matt Gilman f8e3bef606 NIFI-923: - Upgrading to the current release of spring. - This closes #82
Signed-off-by: joewitt <joewitt@apache.org>
2016-01-25 10:14:45 -05:00
Mark Payne f6ec437bc7 Merge branch 'master' into NIFI-259 2016-01-21 09:32:02 -05:00
Mark Payne 2c9fb676cd NIFI-1156: Updated version from 0.4.0-SNAPSHOT to 0.4.2-SNAPSHOT; addressed some error handling issues; updated licensing information 2016-01-20 15:27:58 -05:00
Aldrin Piri f53aaed122 NIFI-569 Introducing the maven-failsafe-plugin to the root pom.xml. Refactoring AWS integration level tests to make use of the IT* prefix and removing @Ignore from these classes. This closes #173
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-19 09:09:24 -05:00
Mark Payne 16dcf4595c Merge branch 'master' into NIFI-259 2016-01-18 08:48:30 -05:00
Andy LoPresto c3ac772b92 NIFI-1324: bump version on bouncycastle artifacts from 1.53 to 1.54
Reviewed by Tony Kurc (tkurc@apache.org) This closes #170
2016-01-12 19:37:45 -05:00
Andy LoPresto ffbfffce6d NIFI-1324:
Changed Maven dependencies for BouncyCastle bcprov and bcpg from jdk16:1.46 to jdk15on:1.53 (kept nifi-web-security on jdk16:1.46 because jdk15on:1.53 splits OCSP logic into new module bcpkix).
Added individual unit tests for PGP public keyring validation.
Passes all legacy unit tests.
Added TODOs for customizable brick encryption and refactoring shared code.
Cleaned up magic numbers to constants.
Added unit tests for OpenPGPPasswordBasedEncryptor (internal consistency and legacy file decrypt).
Began refactoring shared encrypt code from OpenPGP* implementations.
Extracted encrypt utility method from OpenPGPPasswordBasedEncryptor to PGPUtil class.
Added test resources (signed and unsigned key-encrypted files).
Added unit tests for OpenPGPKeyBasedEncryptor (internal consistency and external file decrypt).
Changed BC dependency for nifi-web-security to bcprov-jdk15on:1.53 and bcpkix-jdk15on:1.53.
Updated OCSPValidator to use new BC logic for OCSP validation. This code compiles but should be fully audited, as the legacy OCSP validation was not completely implemented.
Added skeleton of OCSP validator unit tests with successful keypair and certificate generation and signing code.
Added further unit tests for issued certificates.
Annotated unimplemented unit tests with note about Groovy integration.
Refactored Jersey call in OCSPCertificateValidator to internal method.
Added toString() to NiFi local OcspRequest.
Implemented positive & negative unit tests with cache injection for valid/revoked OCSP certificate.
Resolved contrib-check issues.
Removed ignored code in unit test.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-12 09:22:51 -05:00
Mark Payne d2a969e3d6 NIFI-259: Initial implementation of State Management feature 2016-01-11 16:38:52 -05:00
ricky c45060f703 NIFI-987 Added Riemann (PutRiemann) Reporting
- Introduced nifi-riemann-bundle for future Riemann backed monitoring

- Added initial PutRiemann processor for writing events to Riemann
  using the Riemann batch client.

	- Values for events are provided using the NiFi expression language
		e.g. Metric -> ${latency.milliseconds:divide(1000)}
2015-12-28 15:45:11 -05:00
joewitt f4ac8d75c5 NIFI-1312-RC1 prepare for next development iteration 2015-12-19 00:41:04 -05:00
joewitt d624ea4866 NIFI-1312-RC1 prepare release nifi-0.4.1-RC1 2015-12-19 00:40:53 -05:00
joewitt a7b09a57c5 NIFI-1122 release vote passess. Merge branch 'NIFI-1122_nifi-0.4.0-RC2' 2015-12-11 17:14:26 -05:00
joewitt d755e43ec8 NIFI-1122_nifi-0.4.0-RC2prepare for next development iteration 2015-12-08 13:00:10 -05:00
joewitt b66c029090 NIFI-1122_nifi-0.4.0-RC2prepare release nifi-0.4.0-RC2 2015-12-08 12:59:59 -05:00
Aldrin Piri 854c667bd3 NIFI-1211 Adding a .travis.yml to provide CI and adding an exclusion for this file in the RAT check configuration. Adding Travis CI build status graphic to README 2015-12-08 11:13:05 -05:00
joewitt 0f15f261f8 NIFI-1236 adding prefer IPv4 java arg property for tests to have same env we do in prod
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-01 20:43:31 -05:00
Jeremy Dyer 3e26ed8105 Merge remote-tracking branch 'apache/master' into NIFI-1156 2015-11-30 10:12:39 -05:00
Matt Gilman 36eaddb7de NIFI-655:
- Updating the version of ldap provider nar.
2015-11-23 16:51:24 -05:00
Matt Gilman aaf14c45c9 NIFI-655:
- Refactoring web security to use Spring Security Java Configuration.
- Introducing security in Web UI in order to get JWT.

NIFI-655:
- Setting up the resources (js/css) for the login page.

NIFI-655:
- Adding support for configuring anonymous roles.
- Addressing checkstyle violations.

NIFI-655:
- Moving to token api to web-api.
- Creating an LoginProvider API for user/pass based authentication.
- Creating a module for funneling access to the authorized useres.

NIFI-655:
- Moving away from usage of DN to identity throughout the application (from the user db to the authorization provider).
- Updating the authorized users schema to support login users.
- Creating an extension point for authentication of users based on username/password.

NIFI-655:
- Creating an endpoint for returning the identity of the current user.
- Updating the LoginAuthenticationFilter.

NIFI-655:
- Moving NiFi registration to the login page.
- Running the authentication filters in a different order to ensure we can disambiguate each case.
- Starting to layout each case... Forbidden, Login, Create User, Create NiFi Account.

NIFI-655:
- Addressing checkstyle issues.

NIFI-655:
- Making nf-storage available in the login page.
- Requiring use of local storage.
- Ignoring security for GET requests when obtaining the login configuration.

NIFI-655:
- Adding a new endpoint to obtain the status of a user registration.
- Updated the login page loading to ensure all possible states work.

NIFI-655:
- Ensuring we know the necessary state before we attempt to render the login page.
- Building the proxy chain in the JWT authentication filter.
- Only rendering the login when appropriate.

NIFI-655:
- Starting to style the login page.
- Added simple 'login' support by identifying username/password. Issuing JWT token coming...
- Added logout support
- Rendering the username when appropriate.

NIFI-655:
- Extracting certificate validation into a utility class.
- Fixing checkstyle issues.
- Cleaning up the web security context.
- Removing proxy chain checking where possible.

NIFI-655:
- Starting to add support for registration.
- Creating registration form.

NIFI-655:
- Starting to implement the JWT service.
- Parsing JWT on client side in order to render who the user currently is when logged in.

NIFI-655:
- Allowing the user to link back to the log in page from the new account page.
- Renaming DN to identity where possible.

NIFI-655:
- Fixing checkstyle issues.

NIFI-655:
- Adding more/better support for logging out.

NIFI-655:
- Fixing checkstyle issues.

NIFI-655:
- Adding a few new exceptions for the login identity provider.

NIFI-655:
- Disabling log in by default initially.
- Restoring authorization service unit test.

NIFI-655:
- Fixing checkstyle issues.

NIFI-655:
- Updating packages for log in filters.
- Handling new registration exceptions.
- Code clean up.

NIFI-655:
- Removing registration support.
- Removing file based implementation.

NIFI-655:
- Removing file based implementation.

NIFI-655:
- Removing unused spring configuration files.

NIFI-655:
- Making the auto wiring more explicit.

NIFI-655:
- Removing unused dependencies.

NIFI-655:
- Removing unused filter.

NIFI-655:
- Updating the login API authenticate method to use a richer set of exceptions.
- UI code clean.

NIFI-655:
- Ensuring the login identity provider is able to switch context classloaders via the standard NAR mechanisms.

NIFI-655:
- Initial commit of the LDAP based identity providers.
- Fixed issue when attempting to log into a NiFi that does not support new account requests.

NIFI-655:
- Allowing the ldap provider to specify if client authentication is required/desired.

NIFI-655:
- Persisting keys to sign user tokens.
- Allowing the identity provider to specify the token expiration.
- Code clean up.

NIFI-655:
- Ensuring identities are unique in the key table.

NIFI-655:
- Adding support for specifying the user search base and user search filter in the active directory provider.

NIFI-655:
- Fixing checkstyle issues.

NIFI-655:
- Adding automatic client side token renewal.

NIFI-655:
- Ensuring the logout link is rendered when appropriate.

NIFI-655:
- Adding configuration options for referrals and connect/read timeouts

NIFI-655:
- Added an endpoint for access details including configuration, creating tokens, and checking status.
- Updated DTOs and client side to utilize new endpoints.

NIFI-655:
- Refactoring certificate extraction and validation.
- Refactoring how expiration is specified in the login identity providers.
- Adding unit tests for the access endpoints.
- Code clean up.

NIFI-655:
- Keeping token expiration between 1 minute and 12 hours.

NIFI-655:
- Using the user identity provided by the login identity provider.

NIFI-655: - Fixed typo in error message for unrecognized authentication strategy.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655. - Added logback-test.xml configuration resource for nifi-web-security.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655. - Added issuer field to LoginAuthenticationToken. - Updated AccessResource to pass identity provider class name when creating LoginAuthenticationTokens. - Began refactoring JWT logic from request parsing logic in JwtService. - Added unit tests for JWT logic.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655. - Changed issuer field to use FQ class name because some classes return an empty string for getSimpleName(). - Finished refactoring JWT logic from request parsing logic in JwtService. - Updated AccessResource and JwtAuthenticationFilter to call new JwtService methods decoupled from request header parsing. - Added extensive unit tests for JWT logic.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655:
- Refactoring key service to expose the key id.
- Handling client side expiration better.
- Removing specialized active directory provider and abstract ldap provider.

NIFI-655. - Updated JwtService and JwtServiceTest to use Key POJO instead of raw String key from KeyService.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655:
- Fixing typo when loading the ldap connect timeout.
- Providing a better experience for session expiration.
- Using ellipsis for lengthly user name.
- Adding an issuer to the authentication response so the LIP can specify the appropriate value.

NIFI-655:
- Showing a logging in notification during the log in process.

NIFI-655:
- Removing unnecessary class.

NIFI-655:
- Fixing checkstyle issues.
- Showing the progress spinner while submitting account justification.

NIFI-655:
- Removing deprecated authentication strategy.
- Renaming TLS to START_TLS.
- Allowing the protocol to be configured.

NIFI-655:
- Fixing issue detecting the presence of DN column

NIFI-655:
- Pre-populating the login-identity-providers.xml file with necessary properties and documentation.
- Renaming the Authentication Duration property name.

NIFI-655:
- Updating documentation for the failure response codes.

NIFI-655:
- Ensuring the user identity is not too long.

NIFI-655:
- Updating default authentication expiration to 12 hours.

NIFI-655:
- Remaining on the login form when there is any unsuccessful login attempt.
- Fixing checkstyle issues.
2015-11-23 14:50:13 -05:00
Joseph Percivall 8c2323dc8d NIFI-1086 Provide refactoring of InvokeHTTP
NIFI-980 Add support for HTTP Digest authentication to InvokeHttp
NIFI-1080 Provide additional InvokeHttp unit tests
NIFI-1133 InvokeHTTP Processor does not save Location header for 3xx responses
NIFI-1009 InvokeHTTP should be able to be scheduled without any incoming connection for GET operations
NIFI-61 Multiple improvements for InvokeHTTP inclusive of providing unique tx.id across clusters, dynamic HTTP header properties

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-11-19 01:40:21 -05:00
Jeremy Dyer c82fc18f8e HTML Parsing Processors Bundle
NIFI-1156 HTML Parsing Processors Bundle
2015-11-13 15:01:10 -05:00
Mark Payne 56ad22aea6 NIFI-900: Created Processors for interacting with Microsoft Azure EventHubs
Reviewed (with amendments needed for clean merge, whitespace and NOTICEs) by Tony Kurc (tkurc@apache.org)
2015-11-11 23:06:04 -05:00
Mark Payne 72008035b1 Merge branch 'NIFI-1000' 2015-11-10 15:02:02 -05:00
Mark Payne 8f942fcdcc NIFI-1000: Fixed checkstyle violation in TestGetJSMQueue use activemq-broker with test scope and activemq-client with default scope rather than pulling in activemq-all; this reduces size of standard nar from 30 MB to 18 MB 2015-11-10 14:31:20 -05:00
Bryan Bende e748fd5848 NIFI-817 Processors for interacting with HBase
- Refactoring PutHBaseCell to batch Puts by table
- Adding optional Columns property to GetHBase to return only selected column families or columns
- Making GetHBase cluster friendly by storing state in the distributed cache and a local file
- Adding Initial Time Range property to GetHBase
- Adding Filter Expression property and custom validate to prevent using columns and a filter at the same time
- Creating an HBaseClientService controller service to isolate the HBase client and support multiple versions
- Creating appropriate LICENSE/NOTICE files
- Adding @InputRequirement to processors
- Addressing comments from review, moving hbase client services under standard services
- Making sure result of session.penalize() is assinged to FlowFile variable before transferring
2015-11-10 11:13:51 -05:00
Oleg Zhurakousky ef0be5a5d6 NIFI-1000 Fixed JmsFactory to properly obtain destiniation name
Re-enabled JMS Tests that were annotated with @Ignore
2015-11-09 18:35:31 -05:00
Jean-Baptiste Onofré de498df4c9 NIFI-925 - Upgrade to hadoop 2.6.2. This closes #83.
Signed-off-by: joewitt <joewitt@apache.org>
2015-11-08 22:10:30 -05:00
Venkatesh Sellappa 77de51df19 NIFI-1025 Updated aws-sdk and joda-time to resolve AWS issues with Java 1.8 update 60.
- Refactored tests - created AbstractS3Test for common utility methods
 - Corrected incorrect unit test in TestDeleteS3Object, and adjusted processor documentation to reflect behavior
 - moved aws dependency management to root pom

This closes #107

Tested, Reviewed and Amended by Tony Kurc (<tkurc@apache.org>)
2015-11-07 21:09:18 -05:00
joewitt 99629646fe NIFI-1122 moved to 0.4.0-SNAPSHOT 2015-11-06 23:41:15 -05:00
Jean-Baptiste Onofré bd47f36c06 NIFI-922: Upgrade to ActiveMQ 5.12.0. closes #81.
Signed-off-by: joewitt <joewitt@apache.org>
2015-10-18 19:13:26 -04:00
ijokarumawak 2466a24530 nifi-992 Adding nifi-couchbase-bundle.
- new CouchbaseClusterControllerService
- new Processors
  - GetCouchbaseKey
  - PutCouchbaseKey

Signed-off-by: Bryan Bende <bbende@apache.org>
2015-09-28 11:21:42 -04:00
Matt Gilman ded74ec94c NIFI-876 prepare for next development iteration 2015-09-14 21:48:11 -04:00
Matt Gilman 2ec735e350 NIFI-876 prepare release nifi-0.3.0-RC1 2015-09-14 21:48:00 -04:00
Matt Gilman 291e65c6ef NIFI-947:
- Added a profile to disable doclint when running jdk 1.8.
2015-09-11 10:17:33 -04:00
Matt Gilman 6f8eb9f5e3 NIFI-876:
- Updating the version of the nifi-nar-maven-plugin.
- Updating the minimum version of maven.
2015-08-23 12:32:52 -04:00
Mark Payne a9c2bf58b8 NIFI-844: Initial import of the ConvertAvroToJSON processor 2015-08-21 17:11:56 -04:00
Mark Payne 28549c2b1e NIFI-853: Made updates to processors 2015-08-19 17:36:29 -04:00
Bryan Bende 1c34478eb8 NIFI-790 Create reporting task to deliver metrics to Apache Ambari
- Adding NOTICE for nifi-ambari-nar, and fixing formatting issues in nifi-ambari-reporting-task pom
- Addressing review feedback: updating assembly NOTICE, fixing unit test, and minor clean up
- Adding additionalDetails.html
2015-08-17 15:47:49 -04:00
joewitt aa99884782 NIFI-850 removed nifi parent, updated nifi pom, moved all nifi subdirs up one level, fixed readme. 2015-08-15 13:12:22 -04:00
joewitt 300952a984 Reworked overall directory structure to make releasing nifi vs maven plugis easier 2015-01-15 21:11:07 -05:00
Mark Payne 94a06fc5d5 NIFI-265: Fixed typos 2015-01-14 20:41:18 -05:00
joewitt 7ded9970a2 NIFI-242 additional rat exclusion for ide generated file already ignored by Git - wish Rat honored Git ignores 2015-01-10 22:49:57 -05:00
joewitt 7d85962e4d NIFI-242 RTC process deviation as this changeset is for must-have license headers. Apache RAT now integrated and working in build 2015-01-10 22:29:13 -05:00
joewitt ed8b2f6027 NIFI-242 added more stuff i learned from accumulo's pom. 2015-01-09 15:46:14 -05:00
joewitt 002926882b NIFI-242 added section for generating release sources and signing and running apache rat - borrowed from accumulo 2015-01-09 14:56:59 -05:00
Mark Payne 2d0b163961 NIFI-221: included controller service needed in nar-bundles pom 2015-01-06 10:16:00 -05:00
joewitt 9e35c7da9f NIFI-222 moved away from project.version and instead use explicit version declaration. Release plugin should take care of it anyway 2015-01-03 00:58:58 -05:00
Mark Payne 3e2f790672 NIFI-220: Initial Import of GetKafka and PutKafka processors 2015-01-01 20:52:36 -05:00
joewitt 87b07384a5 NIFI-162 filled in a basic overview. Definitely still needs work though. Too wordy. Too specific. 2014-12-28 15:23:45 -05:00
joewitt 9b0a9a6415 NIFI-198 formatting adjustment, removed comments provided as guides to understand previous patch 2014-12-23 22:28:42 -05:00
joewitt 32bc17db85 Merge branch 'NIFI-199' of https://github.com/khmarbaise/incubator-nifi into develop 2014-12-23 22:12:14 -05:00
Karl-Heinz Marbaise 03f2ce5000 [NIFI-199] Removed plugin dependencies from main project cause they are
not needed at the moment.
2014-12-23 22:58:39 +01:00
Karl-Heinz Marbaise 07547a74f1 [NIFI-198] Prerequisites is not correctly checked
Using maven-enforcer-plugin to check the correct maven version.
2014-12-23 22:53:33 +01:00
Karl-Heinz Marbaise 31921212da [NIFI-195] Take advantage of inheritance from ASF parent version 16 2014-12-22 22:41:07 +01:00
joewitt 0d8da145bd NIFI-183 Updated LICENSE and NOTICE based on thorough analysis of all dependencies. Removed HORNETQ and c3p0 as they are incompatible with ASF licensing policies. 2014-12-21 15:14:53 -05:00
joewitt 1357a17b67 NIFI-189 Collapsed several commons libs into one 2014-12-20 00:40:14 -05:00
joewitt 21182ac260 NIFI-150 updated to ensure docs get built and end up in the assembly under docs. 2014-12-19 16:34:25 -05:00
Mark Payne a23c7f2e2b NIFI-150: Added documentation in asciidoc format 2014-12-18 17:10:42 -05:00
joewitt 614349bdfd NIFI-169 updating a few tests and build 2014-12-15 10:43:07 -05:00
joewitt 19d4a150a8 NIFI-169 well it finally all builds. There is a classpath issue still to sort out which impacts startup 2014-12-15 05:14:32 -05:00