Apache/nifi
1
0
Fork 0
mirror of https://github.com/apache/nifi.git synced 2025-02-09 19:45:09 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,310 Commits 147 Branches 203 Tags
Commit Graph

68 Commits

Author SHA1 Message Date
joewitt
f8466cb16d NIFI-4995-RC3 prepare release nifi-1.6.0-RC3 2018-04-03 08:28:15 -07:00
joewitt
7b5bf265a6 NIFI-4995 updating copyright year on all notices 2018-03-26 21:54:10 -04:00
Derek Straka
95dd1ebffc NIFI-4882: Resolve issue with parsing custom date, time, and timestamp formats in CSV files 
Refactor the two existing CSV classes to inherit from an abstract base class that parses the data
Add unit tests for calls to nextRecord that do not coerce types

This closes #2473.

Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2018-03-22 09:18:00 +09:00
Derek Straka
5bdb7cf6e7 NIFI-4912: This closes #2494. Update jackson version to latest stable version (2.9.4) 
Signed-off-by: joewitt <joewitt@apache.org>
 2018-03-19 10:22:50 -04:00
Bryan Bende
de71a41bd0 NIFI-4935 Refactoring to support specifying schema branch or schema version when using schema by name strategy 
This closes #2523.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-03-15 16:16:12 -04:00
Pierre Villard
11a7d5c09f NIFI-4955 - Preserve columns ordering with CSV and ValidateRecord 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2528
 2018-03-13 17:09:07 -04:00
Pierre Villard
95e543d4cc NIFI-4967 - CSVRecordReader does not read header with specific formats 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2536
 2018-03-13 16:54:46 -04:00
Pierre Villard
c75604a564 NIFI-4966 - JacksonCSVRecordReader - NPE with some CSV formats 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2535
 2018-03-13 16:49:47 -04:00
joewitt
c71409fb5d
NIFI-4936 trying to quiet down the mvn output a bit so we dont exceed the travis-ci 4MB max 
NIFI-4936 updated dependency handling pushing down delcarations where they belong
This closes #2512
 2018-03-09 16:34:53 -05:00
Mark Payne
7c1ce17223
NIFI-4756: Updated PublishKafkaRecord processors to include attributes generated from schema write strategy into the message headers when appropriate 
This closes #2396.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2018-01-16 16:31:34 -05:00
joewitt
41ce788812 NIFI-4751 changed to next minor release version snapshot 2018-01-12 15:15:32 -05:00
joewitt
36405e888c NIFI-4751-RC1 prepare for next development iteration 2018-01-08 23:39:49 -07:00
joewitt
46d30c7e92 NIFI-4751-RC1 prepare release nifi-1.5.0-RC1 2018-01-08 23:39:32 -07:00
Mark Payne
c91d99884a NIFI-4717: Several minor bug fixes and performance improvements around record-oriented processors 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2359
 2017-12-29 10:43:21 -05:00
Matthew Burgess
14d2291db8 NIFI-4496: Added JacksonCSVRecordReader to allow choice of CSV parser. This closes #2245. 2017-12-22 08:56:29 -05:00
Mark Payne
406db4867a NIFI-4702: When we check the next line for matches in Grok Reader, store the Map that is returned so that we don't have to re-evaluate the regexes the next time that nextRecord() is called 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2349
 2017-12-18 09:41:05 -05:00
Mark Payne
f772f2f093 NIFI-4671: This closes #2328. Ensure that Avro Schemas that are created properly denote fields as being nullable iff the schemas says they are, for non-top-level fields 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-12-11 11:46:15 -05:00
Matt Gilman
6b75921725 NIFI-4617: 
- Ensuring a charset is specified when convert a string into bytes.
- Ensuring the aggregate snapshot is non null before attempting to promote the runStatus.

This closes #2278.
 2017-11-20 09:39:23 -05:00
patricker
e3482cc772 NIFI-4534 Choose Character Set for CSV Record Read/Write streams 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2229.
 2017-10-27 10:34:15 +02:00
Mark Payne
95e86c1f51 NIFI-4495: Expose the option for supressing null values on the JSON Writer 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2218.
 2017-10-19 10:20:40 +02:00
patricker
fd00df3d2f NIFI-4465 ConvertExcelToCSV Data Formatting and Delimiters 
This closes #2194.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2017-10-17 14:56:49 +09:00
Matt Gilman
6baea8ccff
NIFI-4444: 
- Upgrading to Jersey 2.x.
- Updating NOTICE files where necessary.
- Fixing checkstyle issues.

This closes #2206.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
 2017-10-12 10:27:02 -07:00
Andy LoPresto
d4168f5ff1
NIFI-4297 
- Upgraded immediately actionable dependency versions from Meterian report.
- Upgraded jackson-core test dependencies for HBase and Elasticsearch modules.
- Only 3 instances of jackson-core < 2.8.6 (Google Cloud Platform and Spark Receiver modules).
- Upgraded version of poi dependency in nifi-email-processors to 3.16.
- Resolving dependency issues after rebasing against 1.5.0-SNAPSHOT.
- Removed jackson-databind from <dependencyManagement> block in nifi/pom.xml and added explicit reference to ${jackson.version} in all referenced artifacts.
- Removed jackson-mapper-asl from <dependencyManagement> block in nifi/pom.xml and added explicit reference to ${jackson.old.version} in all referenced artifacts.
- Removed Jasypt from <dependencyManagement> and added explicit version in test dependency for legacy compatibility.
- This closes #2084
 2017-10-05 15:23:52 -04:00
Jeff Storck
a57911d3db NIFI-4412-RC2 prepare for next development iteration 2017-09-28 13:45:36 -04:00
Jeff Storck
e6508ba7d3 NIFI-4412-RC2 prepare release nifi-1.4.0-RC2 2017-09-28 13:45:21 -04:00
Arun Manivannan
3b1b326b13 NIFI-4416 CSVRecordReader does not accept escaped character as delimiter 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2172
 2017-09-25 22:32:15 -04:00
Mark Payne
b452f8c251 NIFI-4378: Allow suppression of null values in JSON 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2148.
 2017-09-12 21:41:59 +02:00
Wesley-Lawrence
e203358bf3 NIFI-4258 Corrected a bug with the 'Informix Unload Escape Disabled' CSV format. This closes #2090. 2017-09-08 16:17:38 -04:00
Pierre Villard
6559604456 NIFI-4081 - Added raw message option in GrokReader 
This closes #1921.
 2017-09-08 12:44:37 -04:00
Koji Kawamura
1f67cbf628 NIFI-4004: Use RecordReaderFactory without FlowFile. 
- Removed FlowFile from RecordReaderFactory, RecordSetWriterFactory and SchemaAccessStrategy.
- Renamed variable 'allowableValue' to 'strategy' to represent its meaning better.
- Removed creation of temporal FlowFile to resolve Record Schema from ConsumerLease.

- Removed unnecessary 'InputStream content' argument from
  RecordSetWriterFactory.getSchema method.

This closes #1877.
 2017-09-08 12:37:40 -04:00
Mark Payne
451f9cf124 NIFI-4142: This closes #2015. Refactored Record Reader/Writer to allow for reading/writing "raw records". Implemented ValidateRecord. Updated Record Reader to take two parameters for nextRecord: (boolean coerceTypes) and (boolean dropUnknownFields) 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-08-11 22:01:46 -07:00
Mark Payne
3906d4e1d2 NIFI-1763: Initial implementation of ConfluentSchemaRegistry. 
NIFI-1763: Fixed bug where the Confluent Schema Registry Schema Access Writer was not being created

Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1938
 2017-07-06 15:52:07 -04:00
Steve Champagne
68b42c9e54 NIFI-4055: Add a compression option to AvroRecordSetWriter 
This closes #1909.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
 2017-06-26 19:16:31 +02:00
Mark Payne
73e601bc55 NIFI-4102: If first line read does not match Grok expression, skip line 
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1932
 2017-06-21 15:10:22 -04:00
Matt Gilman
490e1da5db NIFI-4019: 
- Adding support for X-Forwarded-* headers.
- Unrelated code clean up.
- Addressing additional proxying issues.
- Addressing mistyped landing page.
- Handled trailing slashes in proxy headers for context path
 2017-06-15 14:17:35 -04:00
Mark Payne
e7dcb6f6c5 NIFI-3921: Allow Record Writers to inherit schema from Record 
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1902
 2017-06-09 16:13:25 -04:00
Matt Gilman
cc741d2be6
NIFI-3997: 
- Bumping to next minor version.
 2017-06-08 15:22:51 -04:00
Matt Gilman
6ee12e9b47
NIFI-3997-RC1prepare for next development iteration 2017-06-05 11:07:43 -04:00
Matt Gilman
ddb73612bd
NIFI-3997-RC1prepare release nifi-1.3.0-RC1 2017-06-05 11:07:28 -04:00
Mark Payne
37be0b9820 NIFI-3990: This closes #1870. Avoided creating garbage when we can avoid it in the JSON and csv readers and json writer 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-06-02 10:01:40 -04:00
Mark Payne
a1b07b1e9c
NIFI-3949: Updated Grok Reader to allow for sub-patterns to be used when determining the schema 
This closes #1839.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2017-05-22 16:30:47 -04:00
Mark Payne
6937a6cf64 NIFI-3953: This closes #1837. Allow multiple schemas on same kafka topic/partition for ConsumeKafkaRecord_0_10 
Also, updated record writers to ensure that they write the schema as appropriate if not using a RecordSet. Updated ConsumeKafkaRecord to allow for multiple schemas to be on same topic and partition

Signed-off-by: joewitt <joewitt@apache.org>
 2017-05-22 14:37:53 -04:00
Mark Payne
c49933f03d NIFI-3948: This closes #1834. Added flush() method to RecordWriter and call it when writing a single record to OutputStream for PublishKafkaRecord. Also removed no-longer-used class WriteAvroResult 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-05-19 23:05:04 -04:00
Mark Payne
ae9953db64 NIFI-3857: This closes #1825. Added PartitionRecord processor 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-05-19 02:08:52 -04:00
Mark Payne
9bd0246a96 NIFI-3863: Initial implementation of Lookup Services. Implemented LookupRecord processors. This required some refactoring of RecordSetWriter interface, so refactored that interface and all implementations and references of it 2017-05-19 01:02:41 -04:00
Mark Payne
5cf4bf61b3
NIFI-3917: Use a default value of 'false' for CSV Reader's Skip Header Line property. This closes #1819 2017-05-17 13:31:53 -04:00
Koji Kawamura
40a9cd4f2e NIFI-3919: Let AvroTypeUtil try every possible type 
Before this fix, AvroTypeUtil can throw an Exception before trying every possible data types defined within a union field.

This closes #1816.
 2017-05-17 10:53:27 -04:00
Koji Kawamura
f019d509f3 NIFI-3918: Added Choice mapping to JsonTreeRowRecordReader. 2017-05-17 09:49:11 -04:00
Koji Kawamura
33dc3e36fb NIFI-3920: Remove unnecessary code from AvroTypeUtil 
- Removed remaining duplicate lines of code left by NIFI-3861 refactoring.
- Added test case that writes Avro record having union field.

This closes #1813.
 2017-05-17 09:45:21 -04:00
Mark Payne
ce1bc42ac5 NIFI-3912: This closes #1809. Fixed NPE that would result in validation failure for FreeFormTextRecordSetWriter 
Signed-off-by: joewitt <joewitt@apache.org>
 2017-05-16 16:17:04 -04:00