36bbc77723
NIFI-6097 - Upgraded the fasterxml jackson version to 2.9.8. Ensure that the version is consistent across modules using a maven property defined in the root pom.
...
This closes #3347
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-03-04 19:30:35 -05:00
e5fa18d63c
NIFI-6088: Widen type inference for BIGINT and DOUBLE
...
This closes #3342
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-02-28 05:51:33 -05:00
25cc7b4a1e
NIFI-6029 merging nifi 1.9.0 release into master
2019-02-19 22:55:49 -05:00
76e92c8682
NIFI-6052 Update NOTICE files to reflect 2019
...
This closes #3319
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-02-19 18:32:15 -05:00
0e204f3576
NIFI-6029-RC2 prepare for next development iteration
2019-02-16 21:50:35 -05:00
45bb53d2aa
NIFI-6029-RC2 prepare release nifi-1.9.0-RC2
2019-02-16 21:50:15 -05:00
82f44155f6
NIFI-6044: This closes #3314 . Retain the input data's order in the CSV Reader's inferred schema
...
Signed-off-by: joewitt <joewitt@apache.org>
2019-02-16 20:57:42 -05:00
b508d6bfbc
NIFI-6033, NIFI-6034, NIFI-6035, NIFI-6036, NIFI-6037: Fixed bugs that were found during 1.9.0-RC1 validation. If multiple FlowFiles were written to same Content Claim, and a Processor attempted to read two of them wi within a single session, it would seek to the wrong part of the content or else throw a ContentNotFoundException. Updated logic for considering a processor to be 'running' / having 'active threads' if the processor is invalid upon NiFi restart but scheduled to run. Fixed NPE in FreeFormTextWriter. If MergeRecord reaches minimum number of records, flush writer after writing content out so that its minimum size can accurately be checked.
...
This closes #3309 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-02-15 11:59:47 -05:00
36c0a99e91
NIFI-5938: Added ability to infer record schema on read from JsonTreeReader, JsonPathReader, XML Reader, and CSV Reader.
...
- Updates to make UpdateRecord and RecordPath automatically update Record schema when performing update and perform the updates on the first record in UpdateRecord before obtaining Writer Schema. This allows the Writer to to inherit the Schema of the updated Record instead of the Schema of the Record as it was when it was read.
- Updated JoltTransformRecord so that schema is inferred on the first transformed object before passing the schema to the Record Writer, so that if writer inherits schema from record, the schema that is inherited is the trans transformed schema
- Updated LookupRecord to allow for Record fields to be arbitrarily added
- Implemented ContentClaimInputStream
- Added controller service for caching schemas
- UpdatedQueryRecord to cache schemas automatically up to some number of schemas, which will significantly inprove throughput in many cases, especially with inferred schemas.
NIFI-5938: Updated AvroTypeUtil so that if creating an Avro Schema using a field name that is not valid for Avro, it creates a Schema that uses a different, valid field name and adds an alias for the given field name so that the fields still are looked up appropriately. Fixed a bug in finding the appropriate Avro field when aliases are used. Updated ContentClaimInputStream so that if mark() is called followed by multiple calls to reset(), that each reset() call is successful instead of failing after the first one (the JavaDoc for InputStream appears to indicate that the InputStream is free to do either and in fact the InputStream is even free to allow reset() to reset to the beginning of file if mark() is not even called, if it chooses to do so instead of requiring a call to mark()).
NIFI-5938: Added another unit test for AvroTypeUtil
NIFI-5938: If using inferred schema in CSV Reader, do not consider first record as a header line. Also addressed a bug in StandardConfigurationContext that was exposed by CSVReader, in which calling getProperty(PropertyDescriptor) did not properly lookup the canonical representation of the Property Descriptor from the component before attempting to get a default value
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #3253
2019-02-11 12:56:50 -05:00
24a7d480c8
NIFI-5983: handling parse problems in recordReader implementations
...
Fixed Checkstyle violation
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #3282
2019-02-06 15:54:15 -05:00
a6f91a1975
NIFI-5872 - Added compression option to JsonRecordSetWriter
...
This closes #3208 .
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-12-10 16:25:59 +09:00
d3b1674813
NIFI-5805: Pool the BinaryEncoders used by the WriteAvroResultWithExternalSchema writer. Unfortunately, the writer that embeds schemas does not allow for this optimization due to the Avro API
...
This closes #3160 .
2018-11-12 09:32:12 -05:00
765df67817
NIFI-5757 Using Caffeine instead of slow synchronization on LinkedHashMap for caches - mainly avro schema caches
...
This closes #3111 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-11-09 14:50:24 -05:00
2812fe60a2
NIFI-5728 - XML Writer to populate record tag name properly
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #3098 .
2018-11-02 09:46:45 +01:00
db966cf348
NIFI-5765 Fixing WriteJsonResult to use chosenDataType when writing an Array value
...
Fixing the same bug in PutElasticsearchHttpRecord.java
This closes #3114 .
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-10-30 11:51:17 +09:00
c0182294ed
NIFI-5720-RC3 prepare for next development iteration
2018-10-22 22:16:43 -04:00
98aabf2c50
NIFI-5720-RC3 prepare release nifi-1.8.0-RC3
2018-10-22 22:16:23 -04:00
c6106d1d88
NIFI-5525 - CSVRecordReader fails with StringIndexOutOfBoundsException when field is a double quote
...
review
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #3092
2018-10-22 16:58:36 -04:00
8e233ca2ef
NIFI-4806 updated tika and a ton of other deps as found by dependency versions plugin
...
This closes #3028
2018-10-04 09:32:04 -04:00
2e1005e884
NIFI-5640: Improved efficiency of Avro Reader and some methods of AvroTypeUtil. Also switched ServiceStateTransition to using read/write locks instead of synchronized blocks because profiling showed that significant time was spent in determining state of a Controller Service when attempting to use it. Switching to a ReadLock should provide better performance there.
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #3036
2018-09-27 15:38:47 -04:00
647152ab16
NIFI-5597: Restored FreeFormTextRecordSetWriter to the META-INF services file
...
This closes #3004 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2018-09-14 09:52:39 -04:00
410176ed22
NIFI-5525: CSVRecordReader fails with StringIndexOutOfBoundsException when field is a double quote
...
This closes #2953 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-08-16 14:22:27 -04:00
0a493bf7fd
NIFI-5432 Add Syslog Record Reader legacy Syslog
...
- Add additional details for schema
This closes #2900 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2018-07-17 13:40:47 -04:00
b10220439c
NIFI-5337 Syslog 5424 Record Reader and nifi-syslog-utils
...
- Create nifi-syslog-utils to move syslog parsing functionalty to a central location shared by the processors and serialization/record system.
- Refactor Processors to use these utils
- Update 5424 syslog classes using simple-syslog-5424 to pick up new changes to support this work, as well as keep dependencies/types from bleeding out to the
processors or readers
- Refactor Syslog5424Event and Parser
- Create Syslog5424RecordReader
- per review, handle blank message differently from eof
- name schema per review
This closes #2816 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2018-07-16 11:50:55 -04:00
b279624398
NIFI-5368 controller services validated prior to enabling; referenced controller services must be enabled for referencing component to be valid (mock framework)
...
This closes #2873 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-07-10 11:23:40 -04:00
d42a1e8bf4
NIFI-5323-RC1 prepare for next development iteration
2018-06-19 20:02:21 -07:00
99bcd1f88d
NIFI-5323-RC1 prepare release nifi-1.7.0-RC1
2018-06-19 20:02:01 -07:00
cf3c666683
NIFI-5286: Updated FasterXML Jackson libraries to 2.9.5
...
This closes #2775
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-06-09 14:19:45 -04:00
49228aa5dc
NIFI-5281: If value is not valid according to the schema's CHOICE field, JSON Writer should write null value instead of throwing NullPointerException
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #2772 .
2018-06-08 10:07:43 +02:00
716587d09f
NIFI-5169 This closes #2705 . Upgrade to JSONPath 2.4
...
NIFI-5169 Added explicit checking for empty strings in JsonPath expression property.
Signed-off-by: joewitt <joewitt@apache.org>
2018-05-23 16:16:25 -05:00
2c8c9374af
NIFI-5113: Add XMLRecordSetWriter
...
This closes #2675 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-05-15 11:07:19 -04:00
61fe493786
NIFI-5170 Upgrad Grok to version 0.1.9
...
This closes #2691
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-13 08:40:20 -04:00
4700b8653d
NIFI-5138: Bug fix to ensure that when we have a CHOICE between two or more REOCRD types that we choose the appropriate RECORD type when creating the Record in the JSON Reader.
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #2670 .
2018-05-09 18:38:04 +02:00
0289ca7114
NIFI-4456: Support multiple JSON objects in JSON record reader/writers
...
This closes #2640 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-05-02 15:22:01 -04:00
159b64b4c8
NIFI-5123: Move SchemaRegistryService to nifi-avro-record-utils
...
This closes #2661 .
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-04-26 11:50:01 -04:00
0e736f59fd
NIFI-4185: Minor tweaks to XML Record Reader, around documentation and error handling
...
This closes #2587 .
2018-04-23 14:41:17 -04:00
d21bd3870b
NIFI-4185: Add XML Record Reader
2018-04-23 14:39:18 -04:00
59f625d3c2
Merge branch 'NIFI-4995-RC3' as part of NiFi 1.6.0 release process
2018-04-06 16:48:08 -07:00
4c787799ff
NIFI-4149 - Indicate if EL is evaluated against FFs or not
...
- take into account input requirement for documentation rendering
- Renamed variable registry scope and added comments
- Doc + change in mock framework to check scope + update of components + UI
2018-04-06 12:40:54 -04:00
d511fe3e4b
NIFI-4995-RC3 prepare for next development iteration
2018-04-03 08:28:34 -07:00
f8466cb16d
NIFI-4995-RC3 prepare release nifi-1.6.0-RC3
2018-04-03 08:28:15 -07:00
7b5bf265a6
NIFI-4995 updating copyright year on all notices
2018-03-26 21:54:10 -04:00
95dd1ebffc
NIFI-4882: Resolve issue with parsing custom date, time, and timestamp formats in CSV files
...
Refactor the two existing CSV classes to inherit from an abstract base class that parses the data
Add unit tests for calls to nextRecord that do not coerce types
This closes #2473 .
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-03-22 09:18:00 +09:00
5bdb7cf6e7
NIFI-4912: This closes #2494 . Update jackson version to latest stable version (2.9.4)
...
Signed-off-by: joewitt <joewitt@apache.org>
2018-03-19 10:22:50 -04:00
de71a41bd0
NIFI-4935 Refactoring to support specifying schema branch or schema version when using schema by name strategy
...
This closes #2523 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-15 16:16:12 -04:00
11a7d5c09f
NIFI-4955 - Preserve columns ordering with CSV and ValidateRecord
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #2528
2018-03-13 17:09:07 -04:00
95e543d4cc
NIFI-4967 - CSVRecordReader does not read header with specific formats
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #2536
2018-03-13 16:54:46 -04:00
c75604a564
NIFI-4966 - JacksonCSVRecordReader - NPE with some CSV formats
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #2535
2018-03-13 16:49:47 -04:00
c71409fb5d
NIFI-4936 trying to quiet down the mvn output a bit so we dont exceed the travis-ci 4MB max
...
NIFI-4936 updated dependency handling pushing down delcarations where they belong
This closes #2512
2018-03-09 16:34:53 -05:00
7c1ce17223
NIFI-4756: Updated PublishKafkaRecord processors to include attributes generated from schema write strategy into the message headers when appropriate
...
This closes #2396 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2018-01-16 16:31:34 -05:00
thenatog
Matthew Burgess
joewitt
Denes Arvay
Mark Payne
Endre Zoltan Kovacs
Pierre Villard
Arek Burdach
Ed B
Joe Percivall
Jeff Storck
Vadim Arshavsky
Otto Fowler
Mark Bean
Andy LoPresto
zenfenan
Mike Thomsen
JohannesDaniel
Derek Straka
Bryan Bende