25 Commits

Author SHA1 Message Date
Matt Gilman
7bcccb10f4 NIFI-3719:
- Removing the usage of SimpleDateFormat when formatting hours/minutes/seconds as the current timezone could cause unintended results.

This closes #1871.
2017-05-30 15:01:39 -04:00
Oleg Zhurakousky
8e17929d6a NIFI-2876 refactored demarcators into a common abstract class
This closes #1214.
2017-02-24 10:06:53 -05:00
Oleg Zhurakousky
ec868362f3 NIFI-3495 fixed the index issue with TextLineDemarcator
This closes #1518.
2017-02-17 10:27:40 -05:00
Otto Fowler
5cfba6ace3 NIFI-3454: Use FileNameFilter to make test consistent
use the FileNameFilter when not passing down explit jar paths
Filter out ^. files when reading lists of files from directories

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-02-11 08:54:06 +09:00
Pierre Villard
4e4d14f86f NIFI-3139 Added host:port list and non empty EL validators in StandardValidators 2017-02-10 11:22:43 -05:00
Oleg Zhurakousky
bba675a11d NIFI-3278 Fixed ArrayIndexOutOfBoundsException in TextLineDemarcator
NIFI-3278 addressed PR comments

This closes #1389
2017-01-04 15:52:23 -05:00
Mark Payne
e59cf86656 NIFI-3188: Added unit test to test corner cases
This closes #1321.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-12-14 10:57:11 +09:00
Mark Payne
8f8b8cdf46 NIFI-3071: Deprecated InputStreams & OutputStream sin org.apache.nifi.stream.io package in favor of using their Java counterparts 2016-12-05 09:29:19 -05:00
joewitt
fb9cbccc38 NIFI-2954 This closes #1244. Moved StandardPropertyValidator to nifi-utils, documented scope/purpose of a few util libs, removed deps from nifi-utils. 2016-11-21 16:30:42 -05:00
Mark Payne
ad92474593 NIFI-2851: Added additional unit test to ensure correctness of demarcation when demarcator falls between buffered data
This closes #1116.
2016-11-11 15:45:40 -05:00
Oleg Zhurakousky
41f519e84c NIFI-2851 initial commit of perf improvements on SplitText
- introduced org.apache.nifi.stream.io.util.TextLineDemarcator
- refactored SplitText to use org.apache.nifi.stream.io.util.TextLineDemarcator
- updated SplitText's capability discription to provide more clarity around splits with headers.
2016-11-11 15:45:40 -05:00
Bryan Bende
d1d053725b NIFI-2909 Adding per-instance class loading capability through @RequiresInstanceClassLoading annotation
NIFI-1712 Applying per-instance class loading to HBaseClientService to allow specifying Phoenix Client JAR

-Refactoring the ClassLoading so that every processor, controller service, and reporting task gets an InstanceClassLoader with a parent of the NAR ClassLoader, and only components with @RequiresInstanceClassLoading will make a copy of the NAR ClassLoader resources, and addressing some review feedback

This closes #1156
2016-11-08 13:21:27 -05:00
Mark Payne
b9cb6b1b47
NIFI-2865: Refactored PublishKafka and PublishKafka_0_10 to allow batching of FlowFiles within a single publish and to let messages timeout if not acknowledged
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-10-06 15:51:16 -04:00
Bryan Bende
a4ed622152
Revert "NIFI-2865: Refactored PublishKafka and PublishKafka_0_10 to allow batching of FlowFiles within a single publish and to let messages timeout if not acknowledged"
This reverts commit 92cca96d49042f9898f93b3a2d2210b924708e52.
2016-10-06 15:19:00 -04:00
Mark Payne
92cca96d49
NIFI-2865: Refactored PublishKafka and PublishKafka_0_10 to allow batching of FlowFiles within a single publish and to let messages timeout if not acknowledged
This closes #1097.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-10-06 10:05:51 -04:00
Matt Burgess
7123a1a276 NIFI-2619: Added unit test showing bugs, Added logic to ClassLoaderUtils to trim module paths and accept URLs
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #907
2016-08-22 15:29:03 -04:00
joewitt
d3b96dcac1 NIFI-2519 This closes #856. aligned threading model with subethastmp 2016-08-14 13:32:00 -04:00
Oleg Zhurakousky
48fa76ecff NIFI-2519 Fixed and refactored ListenSMTP processor
- Removed message queueing which could result in data loss
- Fixed life-cycle issues that coudl put processor in an unstable state
- Fixed PropertyDescriptor translation for Time units and Byte sizes
- Fixed broken tests
- Added additional tests

NIFI-2519 added default for SMTP_MAXIMUM_CONNECTIONS

NIFI-2519 addressed PR comments, polishing
- fixed intermittent deadlock on processor stop and added test for it
- the attributes that can not be extracted from the message but available via MessageContext are written into the outgoing FlowFile
- other minor fixes

NIFI-2519 addressed lates PR comments

NIFI-2519 added better messaging when server closes the connection

NIFI-2519 some polishing and additional tests to validate deadlocks

NIFI-2519 address latest PR comments
fixed deadlock condition for when the consumer is stopped while server is distributing messages
fixed MAX message size issue ensuring it is validated
set max connections to SMTPServer
polished pom
added L&N

NIFI-2519 PR comments
- fixed LICENSE
- Added usage of LimitingInputStream
- simplified SmtpConsumer by removing hasMessage operation
2016-08-14 13:31:46 -04:00
Yolanda M. Davis
048ba5366c NIFI-2020 - initial commit for custom transformation support
NIFI-2020 - updates to use lambdas/stream wherever possible and fix potential nullpointer issue.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #564
2016-07-14 18:04:14 -04:00
Will Song
9546bef86e
NIFI-1958 Added "wks" as an acceptable parameter and added unit test (+6 squashed commits)
Squashed commits:
[16dd4ba] NIFI-1958 fixed logic on incoming time units. Removed feature tests to convert to weeks because it will not be implemented.
[1b22e58] NIFI-1958 added logic to getTimeDuration to handle weeks as a string value to parse but did not add TimeUnit yet. 3/5 feature test pass.
[7136544] NIFI-1958 Moved tests to correct module.
[7d95653] NIFI-1958 Added feature tests for negative values.
[ffc3941] NIFI-1958 Added second feature test.
[7d16bbe] NIFI-1958 Added new feature test and regression test for week conversion.

This closes #544.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-06-17 17:24:19 -07:00
Andrew Grande
d42ea467ca Reviewed, typos.
This closes #460
2016-05-22 10:58:48 -04:00
Oleg Zhurakousky
2d03489ec5 NIFI-1296, NIFI-1680, NIFI-1764, NIFI-1837, NIFI-1827, NIFI-1699 implemented new Kafka processors that leverage Kafka 0.9 API
- Improved StreamScanner for better performance
- Renamed StreamScanner to StreamDemarcator as suggested by Joe
- Added failure handling logic to ensure both processors can be reset to their initial state (as if they were just started)
- Provided comprehensive test suite to validate various aspects of both Publish and Consume from Kafka
- Added relevant javadocs
- Added initial additionalDetails docs
- Addressed NPE reported by NIFI-1764
- Life-cycle refactoring for the existing PutKafka to ensure producer restart after errors
- Incorporated code changes contributed by Ralph Perko (see NIFI-1837)
- Addressed partition issue in RoundRobinPartitioner discussed in NIFI-1827
- Updated PropertyDescriptor descriptions to reflect their purpose

NIFI-1296 added @Ignore on some Kafka tests to improve test time

NIFI-1296 reworked tests to avoid dependency on embedded Kafka

NIFI-1296 fixed spelling error

NIFI-1296 fixed trailing whitespaces in non-java files

This closes #366
2016-05-19 07:33:03 -04:00
ijokarumawak
9235a28f82 NIFI-1736 Move kafka.StreamScanner to nifi-utils. This closes #333 2016-04-07 07:37:28 -04:00
Tony Kurc
528dab78d6 NIFI-1073: Fixing coverity discovered errors. Resource leaks, and statics
Reviewed by Bryan Bende (bbende@apache.org)
2015-11-14 18:56:43 -05:00
joewitt
aa99884782 NIFI-850 removed nifi parent, updated nifi pom, moved all nifi subdirs up one level, fixed readme. 2015-08-15 13:12:22 -04:00