10 Commits

Author SHA1 Message Date
Koji Kawamura
7f0171ffa2
NIFI-3216: Add N signals to Wait/Notify
- Support counters at Wait/Notify processors so that NiFi flow can be
  configured to wait for N signals
- Extract Wait/Notify logics into WaitNotifyProtocol
- Added FragmentAttributes to manage commonly used fragment attributes
- Changed existing split processors to set 'fragment.identifier' and
  'fragment.count', so that Wait can use those to wait for all splits
get processed

This closes #1420.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-01-19 15:17:59 -05:00
Matt Burgess
49297b725d NIFI-2805: Add fragment attributes to SplitAvro
This closes #1046.
2016-09-22 21:46:23 +02:00
Ryan Persaud
274dc0902e NIFI-1909 Adding ability to process schemaless Avro records to ConvertAvrotoJson.
- Made suggested changes and removed unused imports found by checkstyle
 - This closes #459.
2016-06-20 13:48:03 -04:00
Aldrin Piri
ecc240b918 NIFI-1234 Augmenting container handling functionality for single Avro records and adjusting formatting in ConvertAvroToJSON.
Reviewed by Tony Kurc (tkurc@apache.org). This closes #136.
2015-12-02 20:16:08 -05:00
Tony Kurc
ba5b1d837c NIFI-1117 Changed line endings to unix line endings in java source 2015-11-10 18:50:57 -05:00
Tony Kurc
9756e9f126 NIFI-1117 Added unit tests and default behavior for avro files with no records. Also fixed a findbugs warning in SplitAvro (made inner classes static) 2015-11-10 18:46:17 -05:00
Joe
a5a5badb88 NIFI-945 Create a new property (JSON Container) in ConvertAvroToJson, which determines how stream of records is exposed: either as a sequence of single Objects ("none"),
writing every Object to a new line, or as an array of Objects.

Let's assume you have an Avro content as stream of records (record1, record2, ...). If JSON container is "none", the converter will expose the records as sequence of
single JSON objects:

record1
record2
...
recordN

Please bear in mind, that the final output is not a valid JSON content. You can then forward this content e.g. to Kafka, where every record will be a single Kafka message.

If JSON container is "array", the output looks like this:

[record1,record2,...,recordN]

It is useful when you want to convert your Avro content to a valid JSON array.

This closes #88

Reviewed and Amended (amendments reviewed by original patch author on github) by Tony Kurc (tkurc@apache.org)
2015-10-21 20:15:29 -04:00
Bryan Bende
b36841e213 NIFI-919 Adding SplitAvro processor that splits binary datafiles into smaller datafiles, or bare records.
- Adding documentation about bare record use, renaming Split Size to Output Size, and adding a test case with 0 records
- Removing validators on properties that have allowable values, using positive integer validator for Output Size, and fixing type on processor description
2015-09-16 10:11:30 -04:00
Bryan Bende
1007999415 NIFI-912 Initial version of ExtractAvroMetadata processor.
- Adding optional ability to extract record count
- Renaming record.count to item.count for clarity, and updating documentation
- Adding a test case with 0 records
- Removing validators from properties that use allowable values
2015-09-16 09:59:50 -04:00
Mark Payne
a9c2bf58b8 NIFI-844: Initial import of the ConvertAvroToJSON processor 2015-08-21 17:11:56 -04:00