Commit Graph

81 Commits

Author SHA1 Message Date
Andy LoPresto d42a1e8bf4
NIFI-5323-RC1 prepare for next development iteration 2018-06-19 20:02:21 -07:00
Andy LoPresto 99bcd1f88d
NIFI-5323-RC1 prepare release nifi-1.7.0-RC1 2018-06-19 20:02:01 -07:00
zenfenan cf3c666683 NIFI-5286: Updated FasterXML Jackson libraries to 2.9.5
This closes #2775

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-06-09 14:19:45 -04:00
Mark Payne 49228aa5dc NIFI-5281: If value is not valid according to the schema's CHOICE field, JSON Writer should write null value instead of throwing NullPointerException
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2772.
2018-06-08 10:07:43 +02:00
Mike Thomsen 716587d09f NIFI-5169 This closes #2705. Upgrade to JSONPath 2.4
NIFI-5169 Added explicit checking for empty strings in JsonPath expression property.

Signed-off-by: joewitt <joewitt@apache.org>
2018-05-23 16:16:25 -05:00
JohannesDaniel 2c8c9374af NIFI-5113: Add XMLRecordSetWriter
This closes #2675.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-05-15 11:07:19 -04:00
Otto Fowler 61fe493786 NIFI-5170 Upgrad Grok to version 0.1.9
This closes #2691

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-13 08:40:20 -04:00
Mark Payne 4700b8653d NIFI-5138: Bug fix to ensure that when we have a CHOICE between two or more REOCRD types that we choose the appropriate RECORD type when creating the Record in the JSON Reader.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2670.
2018-05-09 18:38:04 +02:00
Matthew Burgess 0289ca7114 NIFI-4456: Support multiple JSON objects in JSON record reader/writers
This closes #2640.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-05-02 15:22:01 -04:00
Matthew Burgess 159b64b4c8
NIFI-5123: Move SchemaRegistryService to nifi-avro-record-utils
This closes #2661.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-04-26 11:50:01 -04:00
Mark Payne 0e736f59fd NIFI-4185: Minor tweaks to XML Record Reader, around documentation and error handling
This closes #2587.
2018-04-23 14:41:17 -04:00
JohannesDaniel d21bd3870b NIFI-4185: Add XML Record Reader 2018-04-23 14:39:18 -04:00
joewitt 59f625d3c2 Merge branch 'NIFI-4995-RC3' as part of NiFi 1.6.0 release process 2018-04-06 16:48:08 -07:00
Pierre Villard 4c787799ff NIFI-4149 - Indicate if EL is evaluated against FFs or not
- take into account input requirement for documentation rendering
- Renamed variable registry scope and added comments
- Doc + change in mock framework to check scope + update of components + UI
2018-04-06 12:40:54 -04:00
joewitt d511fe3e4b NIFI-4995-RC3 prepare for next development iteration 2018-04-03 08:28:34 -07:00
joewitt f8466cb16d NIFI-4995-RC3 prepare release nifi-1.6.0-RC3 2018-04-03 08:28:15 -07:00
Derek Straka 95dd1ebffc NIFI-4882: Resolve issue with parsing custom date, time, and timestamp formats in CSV files
Refactor the two existing CSV classes to inherit from an abstract base class that parses the data
Add unit tests for calls to nextRecord that do not coerce types

This closes #2473.

Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-03-22 09:18:00 +09:00
Derek Straka 5bdb7cf6e7 NIFI-4912: This closes #2494. Update jackson version to latest stable version (2.9.4)
Signed-off-by: joewitt <joewitt@apache.org>
2018-03-19 10:22:50 -04:00
Bryan Bende de71a41bd0 NIFI-4935 Refactoring to support specifying schema branch or schema version when using schema by name strategy
This closes #2523.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-15 16:16:12 -04:00
Pierre Villard 11a7d5c09f NIFI-4955 - Preserve columns ordering with CSV and ValidateRecord
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2528
2018-03-13 17:09:07 -04:00
Pierre Villard 95e543d4cc NIFI-4967 - CSVRecordReader does not read header with specific formats
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2536
2018-03-13 16:54:46 -04:00
Pierre Villard c75604a564 NIFI-4966 - JacksonCSVRecordReader - NPE with some CSV formats
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2535
2018-03-13 16:49:47 -04:00
joewitt c71409fb5d
NIFI-4936 trying to quiet down the mvn output a bit so we dont exceed the travis-ci 4MB max
NIFI-4936 updated dependency handling pushing down delcarations where they belong
This closes #2512
2018-03-09 16:34:53 -05:00
Mark Payne 7c1ce17223
NIFI-4756: Updated PublishKafkaRecord processors to include attributes generated from schema write strategy into the message headers when appropriate
This closes #2396.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-01-16 16:31:34 -05:00
joewitt 41ce788812 NIFI-4751 changed to next minor release version snapshot 2018-01-12 15:15:32 -05:00
joewitt 36405e888c NIFI-4751-RC1 prepare for next development iteration 2018-01-08 23:39:49 -07:00
joewitt 46d30c7e92 NIFI-4751-RC1 prepare release nifi-1.5.0-RC1 2018-01-08 23:39:32 -07:00
Mark Payne c91d99884a NIFI-4717: Several minor bug fixes and performance improvements around record-oriented processors
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2359
2017-12-29 10:43:21 -05:00
Matthew Burgess 14d2291db8 NIFI-4496: Added JacksonCSVRecordReader to allow choice of CSV parser. This closes #2245. 2017-12-22 08:56:29 -05:00
Mark Payne 406db4867a NIFI-4702: When we check the next line for matches in Grok Reader, store the Map that is returned so that we don't have to re-evaluate the regexes the next time that nextRecord() is called
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2349
2017-12-18 09:41:05 -05:00
Mark Payne f772f2f093 NIFI-4671: This closes #2328. Ensure that Avro Schemas that are created properly denote fields as being nullable iff the schemas says they are, for non-top-level fields
Signed-off-by: joewitt <joewitt@apache.org>
2017-12-11 11:46:15 -05:00
Matt Gilman 6b75921725 NIFI-4617:
- Ensuring a charset is specified when convert a string into bytes.
- Ensuring the aggregate snapshot is non null before attempting to promote the runStatus.

This closes #2278.
2017-11-20 09:39:23 -05:00
patricker e3482cc772 NIFI-4534 Choose Character Set for CSV Record Read/Write streams
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2229.
2017-10-27 10:34:15 +02:00
Mark Payne 95e86c1f51 NIFI-4495: Expose the option for supressing null values on the JSON Writer
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2218.
2017-10-19 10:20:40 +02:00
patricker fd00df3d2f NIFI-4465 ConvertExcelToCSV Data Formatting and Delimiters
This closes #2194.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-10-17 14:56:49 +09:00
Matt Gilman 6baea8ccff
NIFI-4444:
- Upgrading to Jersey 2.x.
- Updating NOTICE files where necessary.
- Fixing checkstyle issues.

This closes #2206.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-10-12 10:27:02 -07:00
Andy LoPresto d4168f5ff1
NIFI-4297
- Upgraded immediately actionable dependency versions from Meterian report.
- Upgraded jackson-core test dependencies for HBase and Elasticsearch modules.
- Only 3 instances of jackson-core < 2.8.6 (Google Cloud Platform and Spark Receiver modules).
- Upgraded version of poi dependency in nifi-email-processors to 3.16.
- Resolving dependency issues after rebasing against 1.5.0-SNAPSHOT.
- Removed jackson-databind from <dependencyManagement> block in nifi/pom.xml and added explicit reference to ${jackson.version} in all referenced artifacts.
- Removed jackson-mapper-asl from <dependencyManagement> block in nifi/pom.xml and added explicit reference to ${jackson.old.version} in all referenced artifacts.
- Removed Jasypt from <dependencyManagement> and added explicit version in test dependency for legacy compatibility.
- This closes #2084
2017-10-05 15:23:52 -04:00
Jeff Storck a57911d3db NIFI-4412-RC2 prepare for next development iteration 2017-09-28 13:45:36 -04:00
Jeff Storck e6508ba7d3 NIFI-4412-RC2 prepare release nifi-1.4.0-RC2 2017-09-28 13:45:21 -04:00
Arun Manivannan 3b1b326b13 NIFI-4416 CSVRecordReader does not accept escaped character as delimiter
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2172
2017-09-25 22:32:15 -04:00
Mark Payne b452f8c251 NIFI-4378: Allow suppression of null values in JSON
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2148.
2017-09-12 21:41:59 +02:00
Wesley-Lawrence e203358bf3 NIFI-4258 Corrected a bug with the 'Informix Unload Escape Disabled' CSV format. This closes #2090. 2017-09-08 16:17:38 -04:00
Pierre Villard 6559604456 NIFI-4081 - Added raw message option in GrokReader
This closes #1921.
2017-09-08 12:44:37 -04:00
Koji Kawamura 1f67cbf628 NIFI-4004: Use RecordReaderFactory without FlowFile.
- Removed FlowFile from RecordReaderFactory, RecordSetWriterFactory and SchemaAccessStrategy.
- Renamed variable 'allowableValue' to 'strategy' to represent its meaning better.
- Removed creation of temporal FlowFile to resolve Record Schema from ConsumerLease.

- Removed unnecessary 'InputStream content' argument from
  RecordSetWriterFactory.getSchema method.

This closes #1877.
2017-09-08 12:37:40 -04:00
Mark Payne 451f9cf124 NIFI-4142: This closes #2015. Refactored Record Reader/Writer to allow for reading/writing "raw records". Implemented ValidateRecord. Updated Record Reader to take two parameters for nextRecord: (boolean coerceTypes) and (boolean dropUnknownFields)
Signed-off-by: joewitt <joewitt@apache.org>
2017-08-11 22:01:46 -07:00
Mark Payne 3906d4e1d2 NIFI-1763: Initial implementation of ConfluentSchemaRegistry.
NIFI-1763: Fixed bug where the Confluent Schema Registry Schema Access Writer was not being created

Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1938
2017-07-06 15:52:07 -04:00
Steve Champagne 68b42c9e54 NIFI-4055: Add a compression option to AvroRecordSetWriter
This closes #1909.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
2017-06-26 19:16:31 +02:00
Mark Payne 73e601bc55 NIFI-4102: If first line read does not match Grok expression, skip line
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1932
2017-06-21 15:10:22 -04:00
Matt Gilman 490e1da5db NIFI-4019:
- Adding support for X-Forwarded-* headers.
- Unrelated code clean up.
- Addressing additional proxying issues.
- Addressing mistyped landing page.
- Handled trailing slashes in proxy headers for context path
2017-06-15 14:17:35 -04:00
Mark Payne e7dcb6f6c5 NIFI-3921: Allow Record Writers to inherit schema from Record
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1902
2017-06-09 16:13:25 -04:00