Commit Graph

2063 Commits

Author SHA1 Message Date
Matt Gilman 7314af6177 NIFI-259: - Addressing issues that arose from a merge conflict.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-05 17:47:56 -05:00
Aldrin Piri 4df6512126 Merge branch 'NIFI-259' 2016-02-05 14:09:29 -05:00
Aldrin Piri 16f1852453 NIFI-259 Updating logger message for AbstractListProcessor to reference State Manager in onTrigger instead of the distributed cache. 2016-02-05 12:55:06 -05:00
Andy LoPresto 498b5023ce NIFI-1257 NIFI-1259
Added utility method to return the maximum acceptable password length for PBE ciphers on JVM with limited strength crypto because BC implementation is undocumented (based on empirical evidence).
Updated EncryptionMethod definitions to accurately reflect need for unlimited strength crypto according to algorithm key length.
Added processor logic to invoke keyed cipher.
Added EncryptContent processor property for raw hex key (always visible until NIFI-1121).
Added validations for KDF (keyed and PBE) and hex key.
Added utility method to return list of valid key lengths for algorithm.
Added description to allowable values for KDF and encryption method in EncryptContent processor.
Added IV read/write to KeyedCipherProvider and changed from interface to abstract class.
Added salt read/write logic to NifiLegacy and OpenSSL cipher providers.
Changed RandomIVPBECipherProvider from interface to abstract class.
Updated strong KDF implementations.
Renamed CipherFactory to CipherProviderFactory.
Added unit test for registered KDF resolution from factory.
Updated default iteration count for PBKDF2 cipher provider.
Implemented Scrypt cipher provider.
Added salt translator from mcrypt format to Java format.
Added unit tests for salt formatting and validation.
Added surefire block to groovy unit test profile to enforce 3072 MB heap for Scrypt test.
Added local Java implementation of Scrypt KDF (and underlying PBKDF2 KDF) from Will Glozer.
Defined interface for KeyedCipherProvider.
Implemented AES implementation for KeyedCipherProvider.
Added Ruby script to test/resources for external compatibility check.
Added key length check to PBKDF2 cipher provider.
Changed default PRF to SHA-512.
Added salt and key length check to PBKDF2 cipher provider.
Added utility method to check key length validity for cipher families.
Added Bcrypt implementation.
Implemented PBKDF2 cipher provider.
Added default constructor with strong choices for PBKDF2 cipher provider.
Implemented NiFiLegacyCipherProvider and added unit tests.
Added key length parameter to PBKDF2 cipher provider.
Added PRF resolution to PBKDF2 cipher provider.
Added RandomIVPBECipherProvider to allow for non-deterministic IVs.
Added new keyed encryption methods and added boolean field for compatibility with new KDFs.
Added CipherFactory.
Improved Javadoc in NiFi legacy cipher provider and OpenSSL cipher provider.
Added KeyedCipherProvider interface.
Added OpenSSL PKCS#5 v1.5 EVP_BytesToKey cipher provider and unit test.

This closes #201.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-04 19:40:55 -05:00
jpercivall 29fb9c9393 NIFI-259 Corrected some logic in AbstractListProcessor regarding being elected primary node and improved readability in GetHBase a smidge 2016-02-04 16:07:18 -05:00
Matt Gilman 6f4c3db186 NIFI-259:
- Fixing the line height to ensure the multiline ellipsis is consistent across browsers.
2016-02-04 08:59:26 -05:00
puspendu.banerjee@gmail.com 0690aee452 NIFI-1460 - Performance improvement. test Timeout Mitigation. less IO, less dependency on /dev/(u)random.
- Incorporated comments for UUID format conformance & optimized.
          - polished to satisfy this test-case. we don't need incrementAndGet
Reviewed by Tony Kurc (tkurc@apache.org). This closes #202
2016-02-03 21:50:24 -05:00
Joe Skora 769b044e30 NIFI-1107: Re-integrate Multipart Upload changes into PutS3Object. 1. add Multipart upload logic to allow resuming an upload after process/instance restart, 2. add local state management to track the part uploaded for a flowfile, 3. add configurable AWS S3 state management to abort orphaned uploads, and 4. adapt to IT test naming.
Pull request updates: * Cleanup property description to reflect processor details, not code specifics. * Only resume an upload from local state if uploadID still exists in S3, otherwise delete local state and start a new upload. * Check that local state from was actually created before trying to delete it.

Re-integrate Multipart Upload changes into PutS3Object.
1. add Multipart upload logic to allow resuming an upload after process/instance restart,
2. add local state management to track the part uploaded for a flowfile,
3. add configurable AWS S3 state management to abort orphaned uploads, and
4. adapt to IT test naming.

Fixed test issues
 * added mock client for testing local state without matching S3 state,
 * updated regex for multipart etag pattern.

Reviewed by Tony Kurc (tkurc@apache.org). This closes #192
2016-02-03 20:54:32 -05:00
jpercivall 88d4d2ce5f NIFI-259 Corrected GetHttp state managment and added a new unit test 2016-02-03 20:04:39 -05:00
Matt Gilman 55b77fe03b NIFI-1223: - Removing conf directory from unit test.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-03 16:05:38 -05:00
Matt Burgess 478226451c NIFI-210: Add support for multiple module paths for scripting 2016-02-03 10:41:47 -05:00
Matt Gilman adfa5dc0eb NIFI-259: - Moving the state management documentation to after the Clustering configuration.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-03 10:13:23 -05:00
Matt Gilman 7711106d62 NIFI-259: - Adding additional documentation for embedded ZK.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-03 10:13:19 -05:00
Aldrin Piri 8a668fd344 NIFI-259: Fixed bug that caused StateProvider.replace to return true if the value had never been set, instead of false. Fixed typos in administration-guide
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-02 23:03:31 -05:00
joewitt 2d6fec5fa9 NIFI-865 mild cleanup for LICENSE NOTICE 2016-02-02 21:13:28 -05:00
Oleg Zhurakousky dc8b62c3a6 NIFI-865 Added processors to Get and Put to/from AMQP-based messaging systems
This closes #200

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-02-02 20:57:10 -05:00
Matt Burgess 75af3a2ebe NIFI-1275: Added FetchElasticsearch processor and support for secure clusters
This closes #180.

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-02-02 18:21:31 -05:00
scarpacci 0c137bc22d Added path home property and unit test to elasticsearch processor in support of the node client 2016-02-02 17:26:40 -05:00
Matt Burgess 943d0a6e53 Adding to PutElasticsearch - validators, unit tests, etc. 2016-02-02 17:26:40 -05:00
scarpacci e6cfcf40d0 Initial commit for the elasticsearch bundle to Nifi 2016-02-02 17:26:39 -05:00
Matt Gilman 8cf34c3ea5 NIFI-1457:
- Update heartbeat bean when primary state changes.
2016-02-02 13:16:10 -05:00
Matt Burgess 77ab5d3684 NIFI-210: This closes #185. Add ExecuteScript and InvokeScriptedProcessor
Signed-off-by: joewitt <joewitt@apache.org>
2016-02-01 21:09:06 -05:00
Bryan Bende 1e56de9521 NIFI-1417 Exposing several connection settings on the Solr processors to help deal with timeouts and adding an @OnStopped method to close the SolrClient which didn't appear to be happening before
This closes #194
2016-02-01 18:42:00 -05:00
Mark Payne 35d2b921ea NIFI-259: Distinguish between unable to communicate with ZK and 'bad version' when performing 'replace' method of ZooKeeperStateProvider
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-01 16:29:12 -05:00
Aldrin Piri a931e72787 NIFI-259: Expose the ZK exception code in the event of a KeeperException within the ZooKeeperStateProvider. 2016-02-01 16:29:12 -05:00
Aldrin Piri 447e401912 NIFI-259: Extending the StateProvider interface to provide a getSupportedScopes method and implemented this based on the capabilities of each of its implementations. Used supported scope to evaluated configurations at startup and prevent issues when trying to make use of state 2016-02-01 16:29:12 -05:00
Aldrin Piri 257eca9c46 NIFI-259 Removing StateProviderScope and refactoring to use Scope in its place. 2016-02-01 16:29:11 -05:00
Aldrin Piri f2c366cf9c NIFI-259: Correcting path to the ZooKeeperStateProvider. 2016-02-01 15:26:27 -05:00
Matt Gilman 72c8467b9f NIFI-259:
- Addressing typo in documentation.
- Minor tweaks to admin guide.
- Adding support to stand up a ZooKeeperServer when a quorum peer is not distributed (ie supporting both embedded standalone and cluster).

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-01 14:07:38 -05:00
Aldrin Piri f47168213b NIFI-259: Adjusting validation logic for Connect String to catch errors at startup. 2016-02-01 14:07:38 -05:00
Mark Payne ef80549d63 Merge branch 'NIFI-1093' of https://github.com/olegz/nifi into NIFI-1093 2016-02-01 11:51:27 -05:00
Daniel Cave 171b9c4e94 NIFI-1093 - Added settings for processing JSON documents when validating against database columns to fail, log a warning, or ignore unmatched columns. Added tests for new settings.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2016-02-01 11:45:15 -05:00
Aldrin Piri 6902812678 NIFI-259: Adjusting property description of the connect string to be more specific concerning socket tuple format and the associated default port if one is not specified. Providing minor cleanup and removing unused import. 2016-02-01 10:38:45 -05:00
Matt Gilman dbe8ff3f44 NIFI-1426:
- Ensuring we aren't preventing default when focused in a textarea.
2016-02-01 08:28:18 -05:00
joewitt 1bf7a585b2 NIFI-1456 This closes #198. reduced tests to 10MB instead of 100MB datasets and resolved IO issues which impact test stability 2016-01-31 21:56:53 -05:00
joewitt 06d621f358 NIFI-1454 fixed rpm name, bootstrap deps, updated rpm plugin version, removed extraneous license and notice 2016-01-31 20:19:59 -05:00
Richard Miskin 8ef337b65e NIFI-1434 Prevent array index exception in PutEmail
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-01-30 10:56:31 -05:00
jpercivall 2f80cd504b This closes #193 Feedback on the PR has been provided in NIFI-1447 2016-01-29 14:52:56 -05:00
Oleg Zhurakousky 0126800201 NIFI-1093 added support for handling non-required columns
Mostly based on contribution from Daniel Cave
2016-01-29 13:41:27 -05:00
Oleg Zhurakousky 807eb8de6d NIFI-1447 modified SimpleProcessLogger Modified SimpleProcessLogger to ensure that whenever is called it prints full stack trace.
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-01-29 12:39:27 -05:00
jpercivall be27c07c7f NIFI-1382 Created FetchDistributedMapCache processor 2016-01-29 12:12:23 -05:00
Ryan Blue 73c0637c25 NIFI-1193: Add support for storing data in Hive tables.
Thanks to Ricky for helping with the test dependencies.
2016-01-29 10:19:29 -05:00
Mark Payne a0224f1b4c NIFI-1431: Added documentation about how ETag and LastModified dates are used by the Processor
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-28 16:02:05 -05:00
Matt Gilman 4e50263581 NIFI-1445:
- Updating supported browsers sections to accurately reflect the capabilities the UI employs and the dependencies the UI uses.
2016-01-28 15:04:48 -05:00
jpercivall ee30b27cc8 NIFI-1396 Correcting PostHttp chunked encoding functionality 2016-01-28 14:50:22 -05:00
Matt Gilman b3990ecdcf NIFI-1426: - Introducing a universal capture for key events to ensure a consistent behavior throughout the application. - Allowing backspace to remove components from the canvas. - Introducing a more consistent behavior around the escape button.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-28 12:23:02 -06:00
Mark Payne 8a9a44c102 NIFI-1078: Fixed typo in docs and added additional examples 2016-01-28 09:35:05 -05:00
jpercivall 23364f554c NIFI-1078 Fixing the 'now' EL documentation
Signed-off-by: Mark Payne <markap14@hotmail.com>
2016-01-28 08:56:31 -05:00
Oleg Zhurakousky f70f7e3447 NIFI-1333 fixed FlowController shutdown deadlock. put read lock back. This closes #148
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-28 08:31:30 -05:00
Matt Burgess aa6bc5d505 NIFI-1409: Fix missing transfer on error in ExecuteSql
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-01-27 18:23:32 -05:00