Commit Graph

237 Commits

Author SHA1 Message Date
Andy LoPresto 378ccf53c2
NIFI-1753 Replaced usage of javax.security.cert.X509Certificate with java.security.cert.X509Certificate and resolved user-reported ClassCastException when handling client certificates during TLS mutual authentication.
Fixed nifi-utils pom.xml comment about additional dependencies. (+5 squashed commits)
Squashed commits:
[965b766] NIFI-1753 Removed temporary work-around of duplicate certificate conversion util method and added nifi-security-utils as dependency of nifi-utils.
[cd35f9b] NIFI-1753 Replaced legacy X.509 certificate declarations with new declarations in SSLSocketChannel and EndpointConnectionPool.
Temporary work-around of duplicate certificate conversion util method because nifi-utils cannot depend on nifi-security-utils.
[6420897] NIFI-1753 Replaced legacy X.509 certificate declarations with new declarations in PostHTTP.
[b9868ef] NIFI-1753 Added convenience method for extracting DN from peer certificate chain in SSL socket (canonical implementation to reduce code duplication and references to legacy certificate implementations).
Refactored logic retrieving legacy X.509 certificates with reference to convenience method in NodeProtocolSenderImpl.
Replaced logic retrieving legacy X.509 certificates with reference to convenience method in SocketProtocolListener.
Cleaned up exception handling in SocketProtocolListener.
Replaced legacy X.509 certificate declarations with new declarations in HandleHttpRequest (needs manual test).
[e2d1c35] NIFI-1753 Added convenience methods for converting legacy X.509 certificates and abstract certificates to correct X.509 format.
Added unit tests for certificate manipulation.
Replaced logic retrieving legacy X.509 certificates with new logic in NodeProtocolSenderImpl.
Added bcpkix (Bouncy Castle PKI implementation) dependency to nifi-standard-processors pom.

This closes #346.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-04-13 18:30:21 -07:00
Zachary Jaffee 41583e6dc0 NIFI-400 remove deprecated processor EvaluateRegularExpression
This closes #77

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-04-08 09:11:20 -04:00
Bryan Bende 3adb45eafc NIFI-1630 This closes #332. Making TestPutUDP select an open port for testing to avoid conflicts
Signed-off-by: joewitt <joewitt@apache.org>
2016-04-06 23:39:30 -04:00
Matt Brown b0cc6ae7e8 NIFI-1630 PutUDP processor created.
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-04-06 11:04:01 -04:00
Aldrin Piri 3effc6cca5 NIFI-1689 Correcting TimeUnit conversion to be milliseconds instead of the incorrect microseconds and using historical modified times to avoid issues with second granularity of HFS+
Signed-off-by: joewitt <joewitt@apache.org>
2016-04-05 09:38:48 -04:00
Mike Moser 1c025dd4be NIFI-1714 use standard HttpClient cookie policy instead of default
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-04-04 13:13:17 -04:00
Matt Gilman 2de7f3f884 Updating versions to 1.0.0-SNAPSHOT. 2016-04-04 11:36:20 -04:00
Matt Burgess 65b26e6f41 NIFI-1691: Add Fetch Size property to QueryDatabaseTable
This closes #307

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-03-29 10:51:30 -04:00
Aldrin Piri e977729b56 NIFI-1634-rc2 prepare for next development iteration 2016-03-23 18:56:34 -04:00
Aldrin Piri 0b9bd20d31 NIFI-1634-rc2 prepare release nifi-0.6.0-RC2 2016-03-23 18:56:22 -04:00
Aldrin Piri 4babd067c1 NIFI-1664 Preferring System.nanoTime to System.currentTimeMillis and providing explicit handling of timestamps for files in those tests that are testing other attributes of the ListFile process besides timestamp which could lead to erroneous transmissions depending on exactly when files were created.
Adding unsalted_128_raw.enc and salted_128_raw.enc to the list of exclusions for the RAT plugin which caused issues in the Windows environment for contrib-checks.

This closes #297.
2016-03-22 14:13:37 -04:00
Mark Payne 736896246c NIFI-1649: Addressed bug in SplitText that resulted in improper handling of input data if Remove Trailing Newlines is true and there is a FlowFile that is only made up of newlines 2016-03-21 13:26:32 -04:00
Joey Frazee 3a4546c08a NIFI-1617 Add source filename metadata to IdentifyMimeType
Signed-off-by: Matt Burgess <mattyb149@apache.org>
2016-03-18 00:23:16 -04:00
Michal Klempa e12e7a55b7 NIFI-1562 ExecuteStreamCommand and ExecuteProcess now support empty command line arguments
This closes #247

Signed-off-by: Matt Burgess <mattyb149@apache.org>
2016-03-17 23:15:12 -04:00
Pierre Villard e7a254f78e NIFI-1332 Fixed HTTP-204 handling + unit test
This closes #259.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-03-17 23:03:51 -04:00
Matt Burgess 98395de74f NIFI-1575: Add QueryDatabaseTable processor 2016-03-16 15:55:57 -04:00
Mark Payne 68cfc8c612 NIFI-1632: Fixed NPE that occurs if a capturing group is optional and not matched; updated docs to illustrate how this is handled
Signed-off-by: joewitt <joewitt@apache.org>
2016-03-16 15:19:17 -04:00
Mark Payne 56f79e1e85 NIFI-1588: Reworked how ListHDFS store state so that only 2 timestamps are stored and no paths
Signed-off-by: joewitt <joewitt@apache.org>
2016-03-14 15:59:24 -04:00
joewitt 730d999d4b NIFI-627 minor formating adjustments and removed exception based handling logic. This closes #268. This closes #274.
Signed-off-by: joewitt <joewitt@apache.org>
2016-03-13 13:28:48 -04:00
Mike Moser 0cb4047add NIFI-627 removed flowfile penalization which could skew behavior when processor's Time Duration was less than Penalty Duration, improved over throttle penalization NIFI-990 corrected failure path NIFI-1329 refactored using FlowFileFilter to avoid repeatedly returning flowfiles to the input queue, producing misleading stats and excessive Tasks/Time used
Signed-off-by: joewitt <joewitt@apache.org>
2016-03-13 13:28:43 -04:00
Bryan Bende 1373bf6725 NIFI-899 Rewrite of ListenUDP to use new listener framework, includes the following changes:
- Adding Network Interface property to AbstractListenEventProcessor and ListenSyslog
- Adding sending host and sending port to DatagramChannelDispatcher
- Creation of common base class AbstractListenEventBatchingProcessor
- Refactor of ListenUDP, ListenTCP, and ListenRELP to all extend from AbstractListenEventBatchingProcessor
- Changing DatagramChannelDispatcher, socket handlers, and RELP handler to use offer() when queueing instead of put(), and log an error if the offer failed

This closes #266
2016-03-11 09:07:05 -05:00
Bryan Bende 6f5fb59479 NIFI-1420 Adding Splunk bundle containing PutSplunk, and GetSplunk, and adding a ListenTCP processor to standard processors. Refactored internal code from PutSyslog to create a generic AbstractPutEventProcessor which PutSplunk extends from.
This closes #233
2016-03-07 18:21:17 -05:00
Matt Burgess a7e6820fd9 NIFI-1596: ExecuteSQL Error Creating Schema 2016-03-07 10:42:21 -05:00
Bryan Bende 19e53962ca NIFI-1579 Performance improvements for ListenSyslog which include removing an unnecessary yield and exposing a configurable size for the internal queue used by the processor, changing ListenSyslog to use a 20ms poll and use a long poll when batching, also including same improvements for ListenRELP 2016-03-04 09:17:45 -05:00
Mark Payne 28c2a3e5a6 NIFI-1256: Added a 'Satisfies Expression' option for Matching Strategy of RouteText
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-02-25 15:59:57 -05:00
Tony Kurc c7e24c7569 NIFI-1513: fixed some easy to fix errors
Addressing checkstyle issue.

This closes #221

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-02-25 15:21:40 -05:00
Jeremy Dyer 32f476aaa7 NIFI-1357 Add Snappy compression to "CompressContent" Processor
This closes #164.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-25 12:47:01 -05:00
puspendu.banerjee@gmail.com 1c73d9090a NIFI-1501 : Test Monitor Activity has spurious failures.
Added Timeout for the test. So, in case something goes South, threshold will never be more that that Timeout period.

Reviewed with amendments for whitespace by Tony Kurc (tkurc@apache.org). This closes #225
2016-02-17 21:55:10 -05:00
Michal Klempa c2998a71ec NIFI-1514 Fixed ExecuteStreamCommand when expressions in command arguments are used, contents are not split by command separator
This closes #223.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-17 17:48:23 -05:00
Tony Kurc ad73b5c9d9 NIFI-1379-RC3 prepare for next development iteration 2016-02-12 17:28:10 -05:00
Tony Kurc 8309dba80b NIFI-1379-RC3 prepare release nifi-0.5.0-RC3 2016-02-12 17:27:59 -05:00
Aldrin Piri 1a512cd1e6 NIFI-1484 Making use of timestamps at various points of execution to provide listing of all but the latest files which are held until a subsequent execution.
Correcting nifi-amqp-nar bundle's pom description.

This closes #212.
2016-02-10 10:25:47 -05:00
ledor473 eb6f9f0fec NIFI-1423 Allow to penalize FlowFiles that are routed to No Retry relationship
This closes #183

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-02-08 11:20:08 -05:00
Aldrin Piri 2673370cba NIFI-1483 Correcting logic in terms of when local persistence files are removed during the migration process.
Reviewed by Tony Kurc (tkurc@apache.org). This closes #206
2016-02-07 17:40:12 -05:00
Tony Kurc 303f8eabf1 NIFI-1379: Move to 0.5.0-SNAPSHOT, add tkurc code signing key to keys 2016-02-06 08:49:48 -05:00
Andy LoPresto b407379670 NIFI-1257 Resolved legacy compatibility issue with NiFi legacy KDF salt length dependent on cipher block size.
Replaced screenshot for NiFiLegacy salt encoding.
Added description of legacy salt length determination in admin guide.
Added logic for NiFiLegacyCipherProvider to generate and validate salts of the length determined by the cipher block size.
Changed EncryptContent to default to Bcrypt KDF.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-05 23:38:58 -05:00
Aldrin Piri 4df6512126 Merge branch 'NIFI-259' 2016-02-05 14:09:29 -05:00
Aldrin Piri 16f1852453 NIFI-259 Updating logger message for AbstractListProcessor to reference State Manager in onTrigger instead of the distributed cache. 2016-02-05 12:55:06 -05:00
Andy LoPresto 498b5023ce NIFI-1257 NIFI-1259
Added utility method to return the maximum acceptable password length for PBE ciphers on JVM with limited strength crypto because BC implementation is undocumented (based on empirical evidence).
Updated EncryptionMethod definitions to accurately reflect need for unlimited strength crypto according to algorithm key length.
Added processor logic to invoke keyed cipher.
Added EncryptContent processor property for raw hex key (always visible until NIFI-1121).
Added validations for KDF (keyed and PBE) and hex key.
Added utility method to return list of valid key lengths for algorithm.
Added description to allowable values for KDF and encryption method in EncryptContent processor.
Added IV read/write to KeyedCipherProvider and changed from interface to abstract class.
Added salt read/write logic to NifiLegacy and OpenSSL cipher providers.
Changed RandomIVPBECipherProvider from interface to abstract class.
Updated strong KDF implementations.
Renamed CipherFactory to CipherProviderFactory.
Added unit test for registered KDF resolution from factory.
Updated default iteration count for PBKDF2 cipher provider.
Implemented Scrypt cipher provider.
Added salt translator from mcrypt format to Java format.
Added unit tests for salt formatting and validation.
Added surefire block to groovy unit test profile to enforce 3072 MB heap for Scrypt test.
Added local Java implementation of Scrypt KDF (and underlying PBKDF2 KDF) from Will Glozer.
Defined interface for KeyedCipherProvider.
Implemented AES implementation for KeyedCipherProvider.
Added Ruby script to test/resources for external compatibility check.
Added key length check to PBKDF2 cipher provider.
Changed default PRF to SHA-512.
Added salt and key length check to PBKDF2 cipher provider.
Added utility method to check key length validity for cipher families.
Added Bcrypt implementation.
Implemented PBKDF2 cipher provider.
Added default constructor with strong choices for PBKDF2 cipher provider.
Implemented NiFiLegacyCipherProvider and added unit tests.
Added key length parameter to PBKDF2 cipher provider.
Added PRF resolution to PBKDF2 cipher provider.
Added RandomIVPBECipherProvider to allow for non-deterministic IVs.
Added new keyed encryption methods and added boolean field for compatibility with new KDFs.
Added CipherFactory.
Improved Javadoc in NiFi legacy cipher provider and OpenSSL cipher provider.
Added KeyedCipherProvider interface.
Added OpenSSL PKCS#5 v1.5 EVP_BytesToKey cipher provider and unit test.

This closes #201.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-04 19:40:55 -05:00
jpercivall 29fb9c9393 NIFI-259 Corrected some logic in AbstractListProcessor regarding being elected primary node and improved readability in GetHBase a smidge 2016-02-04 16:07:18 -05:00
jpercivall 88d4d2ce5f NIFI-259 Corrected GetHttp state managment and added a new unit test 2016-02-03 20:04:39 -05:00
Matt Gilman 55b77fe03b NIFI-1223: - Removing conf directory from unit test.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-03 16:05:38 -05:00
Mark Payne ef80549d63 Merge branch 'NIFI-1093' of https://github.com/olegz/nifi into NIFI-1093 2016-02-01 11:51:27 -05:00
Daniel Cave 171b9c4e94 NIFI-1093 - Added settings for processing JSON documents when validating against database columns to fail, log a warning, or ignore unmatched columns. Added tests for new settings.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2016-02-01 11:45:15 -05:00
joewitt 1bf7a585b2 NIFI-1456 This closes #198. reduced tests to 10MB instead of 100MB datasets and resolved IO issues which impact test stability 2016-01-31 21:56:53 -05:00
Richard Miskin 8ef337b65e NIFI-1434 Prevent array index exception in PutEmail
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-01-30 10:56:31 -05:00
Oleg Zhurakousky 0126800201 NIFI-1093 added support for handling non-required columns
Mostly based on contribution from Daniel Cave
2016-01-29 13:41:27 -05:00
jpercivall be27c07c7f NIFI-1382 Created FetchDistributedMapCache processor 2016-01-29 12:12:23 -05:00
Mark Payne a0224f1b4c NIFI-1431: Added documentation about how ETag and LastModified dates are used by the Processor
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-28 16:02:05 -05:00
jpercivall ee30b27cc8 NIFI-1396 Correcting PostHttp chunked encoding functionality 2016-01-28 14:50:22 -05:00