Commit Graph

1905 Commits

Author SHA1 Message Date
Joe Skora 769b044e30 NIFI-1107: Re-integrate Multipart Upload changes into PutS3Object. 1. add Multipart upload logic to allow resuming an upload after process/instance restart, 2. add local state management to track the part uploaded for a flowfile, 3. add configurable AWS S3 state management to abort orphaned uploads, and 4. adapt to IT test naming.
Pull request updates: * Cleanup property description to reflect processor details, not code specifics. * Only resume an upload from local state if uploadID still exists in S3, otherwise delete local state and start a new upload. * Check that local state from was actually created before trying to delete it.

Re-integrate Multipart Upload changes into PutS3Object.
1. add Multipart upload logic to allow resuming an upload after process/instance restart,
2. add local state management to track the part uploaded for a flowfile,
3. add configurable AWS S3 state management to abort orphaned uploads, and
4. adapt to IT test naming.

Fixed test issues
 * added mock client for testing local state without matching S3 state,
 * updated regex for multipart etag pattern.

Reviewed by Tony Kurc (tkurc@apache.org). This closes #192
2016-02-03 20:54:32 -05:00
Matt Burgess 478226451c NIFI-210: Add support for multiple module paths for scripting 2016-02-03 10:41:47 -05:00
joewitt 2d6fec5fa9 NIFI-865 mild cleanup for LICENSE NOTICE 2016-02-02 21:13:28 -05:00
Oleg Zhurakousky dc8b62c3a6 NIFI-865 Added processors to Get and Put to/from AMQP-based messaging systems
This closes #200

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-02-02 20:57:10 -05:00
Matt Burgess 75af3a2ebe NIFI-1275: Added FetchElasticsearch processor and support for secure clusters
This closes #180.

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-02-02 18:21:31 -05:00
scarpacci 0c137bc22d Added path home property and unit test to elasticsearch processor in support of the node client 2016-02-02 17:26:40 -05:00
Matt Burgess 943d0a6e53 Adding to PutElasticsearch - validators, unit tests, etc. 2016-02-02 17:26:40 -05:00
scarpacci e6cfcf40d0 Initial commit for the elasticsearch bundle to Nifi 2016-02-02 17:26:39 -05:00
Matt Gilman 8cf34c3ea5 NIFI-1457:
- Update heartbeat bean when primary state changes.
2016-02-02 13:16:10 -05:00
Matt Burgess 77ab5d3684 NIFI-210: This closes #185. Add ExecuteScript and InvokeScriptedProcessor
Signed-off-by: joewitt <joewitt@apache.org>
2016-02-01 21:09:06 -05:00
Bryan Bende 1e56de9521 NIFI-1417 Exposing several connection settings on the Solr processors to help deal with timeouts and adding an @OnStopped method to close the SolrClient which didn't appear to be happening before
This closes #194
2016-02-01 18:42:00 -05:00
Mark Payne ef80549d63 Merge branch 'NIFI-1093' of https://github.com/olegz/nifi into NIFI-1093 2016-02-01 11:51:27 -05:00
Daniel Cave 171b9c4e94 NIFI-1093 - Added settings for processing JSON documents when validating against database columns to fail, log a warning, or ignore unmatched columns. Added tests for new settings.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2016-02-01 11:45:15 -05:00
Matt Gilman dbe8ff3f44 NIFI-1426:
- Ensuring we aren't preventing default when focused in a textarea.
2016-02-01 08:28:18 -05:00
joewitt 1bf7a585b2 NIFI-1456 This closes #198. reduced tests to 10MB instead of 100MB datasets and resolved IO issues which impact test stability 2016-01-31 21:56:53 -05:00
joewitt 06d621f358 NIFI-1454 fixed rpm name, bootstrap deps, updated rpm plugin version, removed extraneous license and notice 2016-01-31 20:19:59 -05:00
Richard Miskin 8ef337b65e NIFI-1434 Prevent array index exception in PutEmail
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-01-30 10:56:31 -05:00
jpercivall 2f80cd504b This closes #193 Feedback on the PR has been provided in NIFI-1447 2016-01-29 14:52:56 -05:00
Oleg Zhurakousky 0126800201 NIFI-1093 added support for handling non-required columns
Mostly based on contribution from Daniel Cave
2016-01-29 13:41:27 -05:00
Oleg Zhurakousky 807eb8de6d NIFI-1447 modified SimpleProcessLogger Modified SimpleProcessLogger to ensure that whenever is called it prints full stack trace.
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-01-29 12:39:27 -05:00
jpercivall be27c07c7f NIFI-1382 Created FetchDistributedMapCache processor 2016-01-29 12:12:23 -05:00
Ryan Blue 73c0637c25 NIFI-1193: Add support for storing data in Hive tables.
Thanks to Ricky for helping with the test dependencies.
2016-01-29 10:19:29 -05:00
Mark Payne a0224f1b4c NIFI-1431: Added documentation about how ETag and LastModified dates are used by the Processor
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-28 16:02:05 -05:00
Matt Gilman 4e50263581 NIFI-1445:
- Updating supported browsers sections to accurately reflect the capabilities the UI employs and the dependencies the UI uses.
2016-01-28 15:04:48 -05:00
jpercivall ee30b27cc8 NIFI-1396 Correcting PostHttp chunked encoding functionality 2016-01-28 14:50:22 -05:00
Matt Gilman b3990ecdcf NIFI-1426: - Introducing a universal capture for key events to ensure a consistent behavior throughout the application. - Allowing backspace to remove components from the canvas. - Introducing a more consistent behavior around the escape button.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-28 12:23:02 -06:00
Mark Payne 8a9a44c102 NIFI-1078: Fixed typo in docs and added additional examples 2016-01-28 09:35:05 -05:00
jpercivall 23364f554c NIFI-1078 Fixing the 'now' EL documentation
Signed-off-by: Mark Payne <markap14@hotmail.com>
2016-01-28 08:56:31 -05:00
Oleg Zhurakousky f70f7e3447 NIFI-1333 fixed FlowController shutdown deadlock. put read lock back. This closes #148
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-28 08:31:30 -05:00
Matt Burgess aa6bc5d505 NIFI-1409: Fix missing transfer on error in ExecuteSql
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-01-27 18:23:32 -05:00
Aldrin Piri c5a174ae6d Revert "NIFI-1193: Add support for storing data in Hive tables."
This reverts commit 9dfef8f432.
2016-01-27 10:03:41 -05:00
Andy LoPresto 93aac8cff3 NIFI-1365
Added Groovy support for unit tests to pom with skeleton test.
Added Groovy unit tests for OCSPCertificateValidator.
Implemented positive & negative unit tests with cache injection for valid/revoked OCSP certificate.
Modified pom.xml to support Groovy unit tests with custom variable.

mvn clean test -Dgroovy=test

Added local cache injection into Groovy tests for OCSP certificate validation (see NIFI-1324 and NIFI-1364).
Set Java version to 1.7 for Groovy test src/target.
Moved Groovy unit test profile from nifi-web-security to root pom.
Added null check for algorithm argument in PGPUtil.
Changed buffer length check from ">= 0" to "> -1" because it was confusing other developers.
Resolved contrib-check line length issues.
Fixed contrib-check issues in OpenPGPKeyBasedEncryptorTest.
This closes #163

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-26 17:37:59 -05:00
Mark Payne 0d45f21f4b Merge branch 'NIFI-1200' of https://github.com/olegz/nifi into NIFI-1200 2016-01-26 16:38:47 -05:00
Oleg Zhurakousky 0395b90315 NIFI-1200 fixed CPU saturation in FileSystemRepository
added minimal interval
added WARN message if set interval is lower

NIFI-1200 fixed unused imports
2016-01-26 16:37:19 -05:00
Ryan Blue 9dfef8f432 NIFI-1193: Add support for storing data in Hive tables. 2016-01-26 13:52:03 -05:00
Mark Payne 0e92607466 Merge branch 'NIFI-1412' of https://github.com/mattyb149/nifi into NIFI-1412 2016-01-26 12:30:19 -05:00
Mark Payne 1c6abcbd55 Merge branch 'NIFI-1421' of https://github.com/rpmiskin/nifi into NIFI-1421 2016-01-26 09:20:33 -05:00
Mark Payne 68975dc2d7 NIFI-1356: Fixed typo in unit test that caused it to look for Shapes_Header.csv.avro instead of Shapes_header.csv.avro (i.e., case of h in Header) 2016-01-26 09:04:13 -05:00
Mark Payne 5ded6b223b Merge branch 'NIFI-1356' of https://github.com/jdye64/nifi into NIFI-1356 2016-01-26 08:51:34 -05:00
Richard Miskin c16c8db459 NIFI-1421 Make namespace ordering deterministic in SplitXML 2016-01-26 05:35:07 +00:00
Bryan Bende b7f7e6ed80 NIFI-1436 This closes #189. Combining stop() and close() into a single method to simplify, and adding checks on stopped flag in the run method of SocketChannelDispatcher and DatagramChannelDispatcher to ensure the run() method exits as soon as possible upon close() being called
NIFI-1436 Adding synchronization on keys set in close() method  based on Selector JavaDoc

Signed-off-by: joewitt <joewitt@apache.org>
2016-01-25 22:50:25 -05:00
Mark Payne 925138b6c4 NIFI-1404: Fixed NPE that resulted on startup if there was a Provenance file that had no events
Signed-off-by: joewitt <joewitt@apache.org>
2016-01-25 21:55:48 -05:00
Jeremy Dyer 7008a3054e NIFI-1356 2016-01-25 18:28:38 -05:00
Mark Payne fca0b4c433 NIFI-1376: This closes #165 2016-01-25 16:40:09 -05:00
Richard Miskin 43b4cb728a NIFI-1376 Fixed checkstyle errors 2016-01-25 16:32:28 -05:00
Richard Miskin 13c6aeec95 NIFI-1376 Provide access to logged messages from TestRunner 2016-01-25 16:32:28 -05:00
joewitt beb92492be NIFI-1421 This closes #184. Added content assertion to test and removed extra space
Signed-off-by: Mark Payne <markap14@hotmail.com>
2016-01-25 16:04:47 -05:00
Mark Payne 4a5fb37b33 Merge branch 'NIFI-1421' of https://github.com/rpmiskin/nifi into NIFI-1421 2016-01-25 15:38:08 -05:00
Matt Gilman d71266502c NIFI-1435: - Addressing typo when initializing and showing the reporting task property table which prevented dynamic properties to be created. 2016-01-25 10:42:37 -05:00
Matt Gilman f8e3bef606 NIFI-923: - Upgrading to the current release of spring. - This closes #82
Signed-off-by: joewitt <joewitt@apache.org>
2016-01-25 10:14:45 -05:00