Commit Graph

61 Commits

Author SHA1 Message Date
Oleg Zhurakousky 29f53c07f5 NIFI-2509 This closes #805. fixed NPE condition in KafkaConsumer 2016-08-07 23:48:57 -04:00
Mark Payne 7ffa30d21b NIFI-2423: Make use of the SSLContextService to provide SSL information
Signed-off-by: Oleg Zhurakousky <oleg@suitcase.io>
2016-08-05 14:14:38 -04:00
Oleg Zhurakousky c39a127ec8 NIFI-2322, NIFI-2423, NIFI-2412 Kafka improvements
- Fixed KafkaConsumer's connection block when broker is not available
- Fixed Serializer/Deserializer configs in both Consume/Publish Kafka
- Added sensitive properties for SSL ket/trust stores

NIFI-2322 fixed tests
2016-08-05 14:14:38 -04:00
Oleg Zhurakousky 54549891e3 NIFI-2444 NIFI-2445 fixed PublishKafka
- fixed the logging issue NIFI-2444 by ensuring the ProcessLog is added to KafkaPublisher
- fixed KafkaPublisher's isAllAcked operation to ensure that it properly reports that the flow file has failed.
- added additional test
This closes #758.
2016-08-02 20:30:34 -04:00
jpercivall 2a6580992b NIFI-2455 Removing 'Queue Buffering Max Time' as a required property of PutKafka. This closes #765 2016-08-02 17:01:58 -04:00
Oleg Zhurakousky 0dbba811f3 NIFI-2298 This closes #687 added Kafka consume attributes to a FlowFile 2016-07-21 22:56:24 -04:00
Oleg Zhurakousky 7a901952b5 NIFI-2192: Fixed OOM issue in KafkaPublisher
This closes #618.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2016-07-08 15:33:49 -04:00
Aldrin Piri 1bd2cf0d09 NIFI-1811 Renaming MockProcessorLogger to MockComponentLogger for consistency. Removing unused imports from ExecuteScript causing checkstyle failures. 2016-05-19 14:38:41 -04:00
Pierre Villard 372ffb8aa0 NIFI-1811 Removed ProcessorLog and updated dependent interfaces
This closes #403.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-05-19 13:08:09 -04:00
Oleg Zhurakousky 2d03489ec5 NIFI-1296, NIFI-1680, NIFI-1764, NIFI-1837, NIFI-1827, NIFI-1699 implemented new Kafka processors that leverage Kafka 0.9 API
- Improved StreamScanner for better performance
- Renamed StreamScanner to StreamDemarcator as suggested by Joe
- Added failure handling logic to ensure both processors can be reset to their initial state (as if they were just started)
- Provided comprehensive test suite to validate various aspects of both Publish and Consume from Kafka
- Added relevant javadocs
- Added initial additionalDetails docs
- Addressed NPE reported by NIFI-1764
- Life-cycle refactoring for the existing PutKafka to ensure producer restart after errors
- Incorporated code changes contributed by Ralph Perko (see NIFI-1837)
- Addressed partition issue in RoundRobinPartitioner discussed in NIFI-1827
- Updated PropertyDescriptor descriptions to reflect their purpose

NIFI-1296 added @Ignore on some Kafka tests to improve test time

NIFI-1296 reworked tests to avoid dependency on embedded Kafka

NIFI-1296 fixed spelling error

NIFI-1296 fixed trailing whitespaces in non-java files

This closes #366
2016-05-19 07:33:03 -04:00
Pierre Villard 3d6e664097 NIFI-1672 Improved the Provenance Events emitted by PutKafka
This closes #355
2016-04-20 10:33:03 -04:00
Oleg Zhurakousky bd1f0a82ae NIFI-1748 removed un-used test data
This closes #344.

Signed-off-by: Joe Skora <jskora@apache.org>
2016-04-11 19:50:22 -04:00
Oleg Zhurakousky 95dda1d920 NIFI-1748 removed un-used test data
This closes #341.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-04-11 02:24:45 -04:00
Joe Skora 696a12e4e7 Revert "NIFI-1748 removed un-used test data"
This reverts commit 47cd9ff22c.
2016-04-11 02:05:24 -04:00
Oleg Zhurakousky 47cd9ff22c NIFI-1748 removed un-used test data
This closes #341.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-04-10 13:21:37 -04:00
ijokarumawak 9235a28f82 NIFI-1736 Move kafka.StreamScanner to nifi-utils. This closes #333 2016-04-07 07:37:28 -04:00
Aldrin Piri 0e9e9b36d9 NIFI-1728 Specifying charsets for messages sent to/received from Kafka in Kafka processor and related tests to remedy failures in Windows environments. Specifying EOF as an int instead of a byte.
Signed-off-by: joewitt <joewitt@apache.org>
2016-04-05 09:38:39 -04:00
Matt Gilman 2de7f3f884 Updating versions to 1.0.0-SNAPSHOT. 2016-04-04 11:36:20 -04:00
Oleg Zhurakousky 25290cedc4 NIFI-1701 fixed StreamScanner, added more tests
NIFI-1701 additional refactoring, clean up and more tests
2016-04-04 09:55:16 -04:00
Oleg Zhurakousky c3d54ab724 NIFI-1684 This closes #308. fixed ZKClient connection leak
Signed-off-by: joewitt <joewitt@apache.org>
2016-03-29 09:56:24 -04:00
Oleg Zhurakousky 9912f18de5 NIFI-1684 This closes #302. fixed random partitioner initialization
Signed-off-by: joewitt <joewitt@apache.org>
2016-03-29 09:56:08 -04:00
Oleg Zhurakousky 1292581ec8 NIFI-1684 added penalization on failure
Signed-off-by: joewitt <joewitt@apache.org>
2016-03-29 09:56:04 -04:00
Oleg Zhurakousky 8d960f5243 NIFI-1684 fixed NPE in PutKafka when retrieving key attribute bytes
Signed-off-by: joewitt <joewitt@apache.org>
2016-03-29 09:56:04 -04:00
Oleg Zhurakousky e35c40b0fd NIFI-1684 fixed NPE, added tests
Signed-off-by: joewitt <joewitt@apache.org>
2016-03-29 09:56:04 -04:00
Aldrin Piri e977729b56 NIFI-1634-rc2 prepare for next development iteration 2016-03-23 18:56:34 -04:00
Aldrin Piri 0b9bd20d31 NIFI-1634-rc2 prepare release nifi-0.6.0-RC2 2016-03-23 18:56:22 -04:00
Mark Payne f7ecb47e29 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2016-03-23 11:24:28 -04:00
Oleg Zhurakousky e0e00ff282 NIFI-1645 refactored PutKafka
- used newest API available in 0.8.* version
- added PutKafka integration tests
- Kafka module code coverage is at 85%

NIFI-1645 polishing

NIFI-1645 PR comments round 1

NIFI-1645 PR comments round 2

NIFI-1645 change to use async Kafka producer

NIFI-1645 polishing

NIFI-1645 polishing

NIFI-1645 polishing

NIFI-1645 changed from java.util.Scanner to custom StreamScanner

NIFI-1645 polishing

NIFI-1645 final polish
2016-03-23 10:31:34 -04:00
Oleg Zhurakousky a68f87f96e NIFI-1665 This closes #296. fixed GetKafka to reset consumer in case of timeout
NIFI-1665 polishing

Signed-off-by: joewitt <joewitt@apache.org>
2016-03-22 21:48:47 -06:00
Oleg Zhurakousky 148b4497b4 NIFI-1629 This closes #282. downgraded Kafka back to 0.8 - added context.yield to PutKafka - added lifecycle hooks to defend from Kafka deadlocks
NIFI-1629 changd thread pool implementation in Get/PutKafka

Signed-off-by: joewitt <joewitt@apache.org>
2016-03-16 15:38:46 -04:00
Oleg Zhurakousky 998ac1c830 NIFI-1534 removed SupportsBatching annotation from PutKafka
Signed-off-by: Matt Burgess <mattyb149@gmail.com>

This closes #249
2016-03-08 10:03:27 -05:00
joewitt 86ab4428f0 NIFI-1520 by default skip javadoc and source jar generation in nars and wars
Reviewed by Tony Kurc (tkurc@apache.org). This closes #234
2016-02-18 18:45:26 -05:00
Tony Kurc ad73b5c9d9 NIFI-1379-RC3 prepare for next development iteration 2016-02-12 17:28:10 -05:00
Tony Kurc 8309dba80b NIFI-1379-RC3 prepare release nifi-0.5.0-RC3 2016-02-12 17:27:59 -05:00
Tony Kurc 5be83166ee NIFI-1485: Updated 'header' section of NiFi NOTICE files 2016-02-07 13:52:00 -05:00
Tony Kurc 303f8eabf1 NIFI-1379: Move to 0.5.0-SNAPSHOT, add tkurc code signing key to keys 2016-02-06 08:49:48 -05:00
Mark Payne 16dcf4595c Merge branch 'master' into NIFI-259 2016-01-18 08:48:30 -05:00
Oleg Zhurakousky 133838a93f NIFI-1233 upgraded to Kafka 0.9.0.0
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-01-12 18:21:38 -05:00
Mark Payne d2a969e3d6 NIFI-259: Initial implementation of State Management feature 2016-01-11 16:38:52 -05:00
Mark Payne 4ab6e3cb3f NIFI-1219: Moved the kafka-tmp directory that gets created to the target directory; added missing license info to beginning of EmbeddedKafkaProducerHelper.java 2016-01-01 13:48:22 -05:00
Oleg Zhurakousky b043d04ecf added embedded Kafka server and tests 2016-01-01 13:48:22 -05:00
joewitt f4ac8d75c5 NIFI-1312-RC1 prepare for next development iteration 2015-12-19 00:41:04 -05:00
joewitt d624ea4866 NIFI-1312-RC1 prepare release nifi-0.4.1-RC1 2015-12-19 00:40:53 -05:00
joewitt fb51483777 Revert "NIFI-1218 upgraded Kafka to 0.9.0.0 client API Tested and validated that it is still compatible with 0.8.* Kafka brokers"
This reverts commit 37635232c7.
2015-12-19 00:07:34 -05:00
joewitt 51821e01b6 Revert "NIFI-1218 addressed PR comments"
This reverts commit 7c87968260.
2015-12-19 00:06:35 -05:00
Oleg Zhurakousky 7c87968260 NIFI-1218 addressed PR comments
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2015-12-16 14:25:28 -05:00
Oleg Zhurakousky 37635232c7 NIFI-1218 upgraded Kafka to 0.9.0.0 client API Tested and validated that it is still compatible with 0.8.* Kafka brokers
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2015-12-16 14:25:28 -05:00
joewitt d755e43ec8 NIFI-1122_nifi-0.4.0-RC2prepare for next development iteration 2015-12-08 13:00:10 -05:00
joewitt b66c029090 NIFI-1122_nifi-0.4.0-RC2prepare release nifi-0.4.0-RC2 2015-12-08 12:59:59 -05:00
Mark Payne 4fa2a713a2 NIFI-1192: Removed some additional white space 2015-11-25 12:57:06 -05:00