Apache
/
nifi
mirror of https://github.com/apache/nifi.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,210 Commits 143 Branches 197 Tags 390 MiB
Commit Graph

2210 Commits

Author SHA1 Message Date
Oleg Zhurakousky 3f295b13c9 NIFI-1519 upgraded to the latest version of Spark 1.6.0 
Signed-off-by: Matt Burgess <mattyb149@gmail.com>
 2016-03-15 10:03:32 -04:00
trkurc f22db218e3 NIFI-1481 fixed a couple bugs in bash script for cygwin and when JAVA_HOME not set 
Reviewed by Aldrin Piri (aldrin@apache.org). This closes #277
 2016-03-14 22:40:25 -04:00
puspendu.banerjee@gmail.com 76ce3f7520 NIFI-1481 support for dumping environment using nifi.sh env 
Reviewed and amended (Amendments reviewed by Aldrin Piri (aldrin@apache.org)) by Tony Kurc (tkurc@apache.org). This closes #218
 2016-03-14 19:24:21 -04:00
Oleg Zhurakousky c7df94e00f NIFI-1464 life-cycle refactoring part-2 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-14 17:13:55 -04:00
Mark Payne 56f79e1e85 NIFI-1588: Reworked how ListHDFS store state so that only 2 timestamps are stored and no paths 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-14 15:59:24 -04:00
Mark Payne 8c488d7e8e NIFI-1587: Always poll state from State Manager when running ListHDFS instead of relying on local state over the cluster state 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-13 14:36:36 -04:00
Mark Payne 5a8b2cf7f1 NIFI-1606: Run the onComponentRemoved logic of state providers in a background thread 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-13 14:08:24 -04:00
Mark Payne 07d4d7005b NIFI-1612: Found a spot within the site-to-site client where the timeout was not utilized properly 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-13 13:40:56 -04:00
Mark Payne ce068ffc6c NIFI-1612: Do not ignore Remote Process Group's timeout configuration 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-13 13:40:45 -04:00
joewitt 730d999d4b NIFI-627 minor formating adjustments and removed exception based handling logic. This closes #268. This closes #274. 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-13 13:28:48 -04:00
Mike Moser 0cb4047add NIFI-627 removed flowfile penalization which could skew behavior when processor's Time Duration was less than Penalty Duration, improved over throttle penalization NIFI-990 corrected failure path NIFI-1329 refactored using FlowFileFilter to avoid repeatedly returning flowfiles to the input queue, producing misleading stats and excessive Tasks/Time used 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-13 13:28:43 -04:00
Mark Payne 7400b6f7c5 NIFI-1622: Ensure that the Nar Context Class Loader is used when calling Processor lifecycle methods 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-12 20:06:23 -05:00
Mark Payne a85c119533 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2016-03-11 12:55:23 -05:00
Oleg Zhurakousky 8e7e2916fa NIFI-1464 fixed admin guide 2016-03-11 12:54:50 -05:00
Oleg Zhurakousky 59fac58c96 NIFI-1464 ensured that OnUnscheduled is treated the same as OnScheduled 
NIFI-1464 polished javadocs, error messages and docs
 2016-03-11 12:54:50 -05:00
Oleg Zhurakousky 1c22f3f012 NIFI-1464 refactored the latest commit 2016-03-11 12:54:50 -05:00
Oleg Zhurakousky 48af0bfbc5 NIFI-1464 addressed latest PR comments 
NIFI-1464 polishing
 2016-03-11 12:54:50 -05:00
Oleg Zhurakousky f53f45def3 NIFI-1464 addressed PR comments from @apiri and @markap14 2016-03-11 12:54:50 -05:00
Oleg Zhurakousky 0c5b1c27f2 NIFI-1464, Refactored Processor's life-cycle operation sequence 
* Simplified and cleaned StandardProcessScheduler.start/stopProcessor methods
* Added stop/start operations to ProcessorNode.
* Removed unnecessary synchronization blocks related to ScheduledState in favor of enforcing order and idempotency via CAS operations. Those synchronization blocks were causing intermittent deadlocks whenever @OnScheduled blocks indefinitely.
* Added support for stopping the service when @OnScheduled operation hangs.
* Fixed the order of life-cycle operation invocation ensuring that each operation can *only* be invoked at the appropriate time
* Removed unnecessary locks from StandardProcessNode since Atomic variables are used.
* Removed calls to @OnStopped from ContinuallyRunningProcessTask while ensuring that procesor's full shut down in implementation of StandardProcessorNode.stop() method.
* Removed dead code
* Added comprehensive tests suite that covers 95% of Processor's life-cycle operations within the scope of FlowController, StandardProcesssScheduler and StandardProcessNode
* Improved and added javadocs on covered operations with detailed explanations.
 2016-03-11 12:54:50 -05:00
Matt Burgess e1cf37fb89 NIFI-1619: Fix Elasticsearch processor bug when flow file missing ID attribute 
This closes #269

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-03-11 12:01:28 -05:00
joewitt f6705f234c NIFI-1511 we now ensure groovy eclipse compiler is only used during testCompile goal 
This closes #270

Signed-off-by: Matt Burgess <mattyb149@gmail.com>
 2016-03-11 11:15:15 -05:00
Bryan Bende 1373bf6725 NIFI-899 Rewrite of ListenUDP to use new listener framework, includes the following changes: 
- Adding Network Interface property to AbstractListenEventProcessor and ListenSyslog
- Adding sending host and sending port to DatagramChannelDispatcher
- Creation of common base class AbstractListenEventBatchingProcessor
- Refactor of ListenUDP, ListenTCP, and ListenRELP to all extend from AbstractListenEventBatchingProcessor
- Changing DatagramChannelDispatcher, socket handlers, and RELP handler to use offer() when queueing instead of put(), and log an error if the offer failed

This closes #266
 2016-03-11 09:07:05 -05:00
Pierre Villard 8f0116544a NIFI-1491 Deprecated existing method and added new one throwing eception 
Reviewed and amended (to add javadoc @deprecation link and to carry forward other deprecated methods) by Tony Kurc (tkurc@apache.org). This closes #227
 2016-03-10 22:15:13 -05:00
Michal Klempa 784f2a2c20 NIFI-1518 InferAvroSchema note has an option to set CSV delimiter 
Reviewed (and amended based on acknowledgement in PR review) by Tony Kurc (tkurc@apache.org). This closes #235.
 2016-03-10 21:36:31 -05:00
Mark Payne d3367a7dc3 nNFI-1618: Updated unit test to use 30,000 FlowFiles to test dropping of swapped out FlowFiles intead of using 210,000. Thiss unit test creates a lot of objects and when reducing the number of FlowFiles generated to 30K instead of 210K, all appears to work okay 2016-03-10 16:01:29 -05:00
Pasqualino Ferrentino 73e168e954 NIFI-1003 A relationship can be auto-terminable. In this case the processor will auto-terminate the relationship and allow the user to run it even if he does not connect those relationships and does not terminate them. 
This closes #217

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-03-10 14:56:46 -05:00
Aldrin Piri da234abd76 NIFI-1488 Adjusting unused imports and adding license exclusions to RAT plugin for build. 2016-03-10 08:58:12 -05:00
Mark Payne 2839a2f215 NIFI-15: Address issue where incomplete swap files can result in continually attempting to swap in data without ever being successful 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-03-09 23:46:53 -05:00
Jeff Lord 7f15626af5 NIFI-1488: Add HBase Kerberos Support with UGI 
Added `kerberos-principal` and `kerberos-keytab` properties
to the HBase service.
 2016-03-09 15:51:30 -05:00
Mark Payne a2164136db Merge branch 'NIFI-1573' of https://github.com/rpmiskin/nifi into NIFI-1573 2016-03-09 09:40:37 -05:00
Mark Payne 0f9b55afb6 Merge branch 'nifi-1509' of https://github.com/mans2singh/nifi into NIFI-1509 2016-03-09 08:19:45 -05:00
mans2singh 1211996d35 updated attributes using hashmap rather than individually 2016-03-08 21:12:49 -08:00
Mark Payne 9874d35b6b NIFI-1600: Ensure that if we move a RemoteProcessGroup, it also updates the Parent Process Group of the ports 2016-03-08 12:09:45 -05:00
Oleg Zhurakousky 998ac1c830 NIFI-1534 removed SupportsBatching annotation from PutKafka 
Signed-off-by: Matt Burgess <mattyb149@gmail.com>

This closes #249
 2016-03-08 10:03:27 -05:00
Bryan Bende 6f5fb59479 NIFI-1420 Adding Splunk bundle containing PutSplunk, and GetSplunk, and adding a ListenTCP processor to standard processors. Refactored internal code from PutSyslog to create a generic AbstractPutEventProcessor which PutSplunk extends from. 
This closes #233
 2016-03-07 18:21:17 -05:00
Oleg Zhurakousky 4ce7b679e1 NIFI-1595 fixed ReflectionUtils to honor bridge methods 
Refactored and simplified ReflectionUtils while at it
Added ReflectionUtilsTest

This closes #260.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
With adjustments to formatting and whitespace.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-03-07 15:27:35 -05:00
Matt Burgess a7e6820fd9 NIFI-1596: ExecuteSQL Error Creating Schema 2016-03-07 10:42:21 -05:00
Bryan Bende 19e53962ca NIFI-1579 Performance improvements for ListenSyslog which include removing an unnecessary yield and exposing a configurable size for the internal queue used by the processor, changing ListenSyslog to use a 20ms poll and use a long poll when batching, also including same improvements for ListenRELP 2016-03-04 09:17:45 -05:00
Mark Payne 6776060ac8 NIFI-1577: Close any streams that are left open for 'append' when the session is checkpointed, rather than waiting for it to be committed 2016-02-29 16:02:26 -05:00
Mark Payne 62333c9e0a NIFI-1574: Ensure that we never flush a BufferedOutputStream's buffer on close of the write-ahead log 2016-02-29 15:42:54 -05:00
Mark Payne 1149bc61cb NIFI-1572: Ensure that if an Exception is thrown when processing a bin, all sessions involved are rolled back or otherwise accounted for 2016-02-29 15:42:33 -05:00
Richard Miskin 738236a41d NIFI-1573 Allow programmatic access to a Processor's name 2016-02-27 15:18:22 +00:00
Richard Miskin 99c7fe3b44 NIFI-963 Update admin guide to cover configuration of multiple lib directories 
This closes #250.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-27 10:01:38 -05:00
Aldrin Piri 58e50ae39d NIFI-1511 Incorporating Groovy unit tests as part of the main build and providing explicit compile scope on the Groovy dependency for the scripting bundle. This closes #220 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-02-26 10:25:45 -05:00
Mark Payne 28c2a3e5a6 NIFI-1256: Added a 'Satisfies Expression' option for Matching Strategy of RouteText 
Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-02-25 15:59:57 -05:00
Tony Kurc c7e24c7569 NIFI-1513: fixed some easy to fix errors 
Addressing checkstyle issue.

This closes #221

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-02-25 15:21:40 -05:00
Jeremy Dyer 32f476aaa7 NIFI-1357 Add Snappy compression to "CompressContent" Processor 
This closes #164.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-25 12:47:01 -05:00
Matt Gilman 0d13de0cf3 NIFI-1539: - Comparing octet stream content type by using starts with and ignores case. 2016-02-25 10:13:07 -05:00
Sönke Liebau fc92441981 NIFI-1539 - Add normalization of content type for content viewing 
Add code to ContentViewerController to strip content type of any trailing parameters and lowercase the type and subtype.

Added function to ViewableContent to enable retrieving the original value of the content type if needed.

This closes #242

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-02-25 10:12:45 -05:00
Mark Payne 6af108c0ca NIFI-1557: Ensure that Reporting Tasks & controller services are ordered by ID before fingerprinting 2016-02-23 18:02:58 -05:00