Apache/nifi
1
0
Fork 0
mirror of https://github.com/apache/nifi.git synced 2025-02-07 10:38:33 +00:00
Code Issues Packages Projects Releases Wiki Activity
6,284 Commits 147 Branches 203 Tags
Commit Graph

164 Commits

Author SHA1 Message Date
Chris Sampson
3cb26aec72
NIFI-8232 CSV Parsers optionally allow/reject duplicate header names 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4828.
 2021-02-23 09:50:18 +04:00
Joe Witt
88fab00e29
NIFI-7873 merging release branch to latest and updating to 1.14.0-SNAPSHOT 2021-02-15 12:09:32 -07:00
Joe Witt
4afb2ba743
NIFI-7873-RC4 prepare for next development iteration 2021-02-15 12:09:31 -07:00
Joe Witt
487280bee9
NIFI-7873-RC4 prepare release nifi-1.13.0-RC4 2021-02-15 12:09:30 -07:00
Joe Witt
d826416217
NIFI-8192 updating Copyright years for nifi things to include 2021 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4802.
 2021-02-03 12:56:28 +04:00
Matthew Burgess
5e84cf6ff9 NIFI-8175: Add WindowsEventLogReader controller service 
This closes #4785

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2021-01-29 15:16:19 -06:00
Mark Payne
6741317cc4 NIFI-8162: Fixed outdated docs. Self-merging as this is a trivial docs fix. 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2021-01-21 12:52:30 -05:00
Mark Payne
37c2284d04
NIFI-8068: Ensure that when we determine the best of multiple possible types in a UNION that we handle Arrays of Records properly. Also refactored code to be a bit cleaner by extracting blocks of it into appropriately named methods 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4706.
 2020-12-04 19:31:23 +01:00
Pierre Villard
14ec02f21d
NIFI-7981 - add support for enum type in avro schema 
This closes #4648

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
 2020-11-05 18:19:55 -05:00
Matthew Burgess
71a5735f63
NIFI-7976: Infer ints in JSON when all values fit in integer type 
NIFI-7976: Fixed unit test

This closes #4645

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
 2020-11-03 15:43:15 -05:00
Mark Payne
4b9014b959
NIFI-1121: Updated backend to perform appropriate validation. Added tests. Updated documentation writer. Updated dev guide to explain how PropertyDescriptor.Builder#dependsOn affects validation. Updated JavaDocs for PropertyDescriptor.Builder#dependsOn 
Signed-off-by: Bryan Bende <bbende@apache.org>
 2020-11-03 15:37:42 -05:00
Denes Arvay
f73a019f36 NIFI-7843 Recursive avro schemas fail to write with RecordWriter 
NIFI-7843 Recursive avro schemas fail to write with RecordWriter
Add new test case to TestSimpleRecordSchema to test the scenario
when schema name and schema namespace match.

This closes #4550.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
 2020-09-28 19:52:41 +02:00
Peter Wicks
dbf92b9a72
NIFI-5829 Create Lookup Controller Services for RecordSetWriter and RecordReader 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3188
 2020-09-21 14:57:06 -04:00
Matthew Burgess
df64416e91
NIFI-7800: Fix line endings for changed files 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4527.
 2020-09-16 09:26:15 +02:00
Mohammed Nadeem
fd8b0b286f
NIFI-7800: Provide an option to omit XML declaration for XMLRecordSetWriter 
NIFI-7800: Mark new property as required

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4520
 2020-09-11 13:05:40 -04:00
Joe Witt
8baa5c9940
NIFI-7692 updating for next dev release 1.13.0 2020-08-18 14:48:02 -07:00
Joe Witt
fb57bcbc11
NIFI-7692-RC1 prepare for next development iteration 2020-08-13 09:20:39 -07:00
Joe Witt
303d6c59ba
NIFI-7692-RC1 prepare release nifi-1.12.0-RC1 2020-08-13 09:20:36 -07:00
Matthew Burgess
b7b2533ffe NIFI-7734: Added Record Separator property to CSVReader 2020-08-12 22:05:08 -04:00
Joe Witt
4417b9d64a
NIFI-6666 removing all modifications to the nifi-api and the newly created builder class 
This closes #4475.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
 2020-08-12 17:49:51 -07:00
Pierre Villard
958d9c4cb6
NIFI-7493 - fix checkstyle issue 2020-07-17 12:22:47 +02:00
Mark Payne
7e09e0db33
NIFI-7493: When inferring schema for XML data, if we find a text element that also has attributes, infer it as a Record type, in order to match how the data will be read when using the XML Reader 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4375.
 2020-07-17 11:35:10 +02:00
Matthew Burgess
f040c6aadb
NIFI-5213: Allow AvroReader to process files w embedded schema even when the access strategy is explicit schema 
NIFI-5213: Incorporated review comments

This closes #2718

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
 2020-07-07 14:43:57 -04:00
Pierre Villard
a2f521297f
NIFI-7490 - Add optional raw field to Syslog readers 
review

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4299
 2020-06-09 15:37:08 -04:00
Bence Simon
5c2bfcf7d3 NIFI-7369 Adding decimal support for record handling in order to avoid missing precision when reading in records 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2020-06-02 15:13:14 -04:00
Niels Basjes
2a82efc417
NIFI-6666 Add Useragent Header to InvokeHTTP requests 
This closes #3734

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
 2020-05-29 21:39:14 -04:00
Matthew Burgess
a3cc2c58ff NIFI-6497: Allow FreeFormTextRecordSetWriter to access FlowFile Attributes 
This closes #4275.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2020-05-18 15:11:33 -04:00
Bryan Bende
2feeb57159
NIFI-7221 Support v2 and v3 protocol version for Hortonworks Schema Registry 
- Update nifi-nar-bundles/nifi-extension-utils/nifi-record-utils/nifi-avro-record-utils/src/main/java/org/apache/nifi/serialization/SchemaRegistryRecordSetWriter.java
- Addressing review feedback

This closes #4120.
 2020-03-20 10:56:47 -04:00
Joe Witt
f694e6464f NIFI-7187 adding missing version strings from accumulo bundle pom 
- Removed Cat X JSON.org dep inclusion which seems to not be necessary
- Updated a ton of easier/safer looking deps
- Updated tika due to CVE

This closes #4086

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2020-03-20 10:07:56 -04:00
Joe Witt
561be89a21
NIFI-7250 activate user.timezone appropriate to each region 
NIFI-7250 fix a test which appears brittle at least on windows builds on slow environments
NIFI-7250 activated a timezone run for AU Australia/Melbourne which exposed a poor magic number and needless assertion but interesting results worth keeping

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4140.
 2020-03-13 17:30:19 +01:00
Joe Witt
97e250cdaa
NIFI-7244 Updated all tests which dont run well on windows to either work or be ignored on windows 
Also dealt with unreliable tests which depend on timing by ignoring them or converting to IT.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4132.
 2020-03-12 19:13:59 +01:00
Mark Payne
67676ba5b4 Fixed bug in JsonRowRecordReader when reading a 'raw' record with a schema that indicates that a field should be a Map. Also updated unit test to explicitly define schema, since schema inference will never return a Map but rather a Record 2020-03-11 14:29:44 -04:00
Pierre Villard
cd83e70b91 unit test reproducing the issue 2020-03-11 14:29:44 -04:00
Zoltan Kornel Torok
e631851e0f
NIFI-7178 - Handle the case when schema is not available 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4067.
 2020-02-22 11:07:31 -08:00
Joe Witt
3de77ebacc
NIFI-7021-RC3 prepare for next development iteration 2020-01-19 14:14:40 -05:00
Joe Witt
633408bce7
NIFI-7021-RC3 prepare release nifi-1.11.0-RC3 2020-01-19 14:14:38 -05:00
Joe Witt
23c8234586
NIFI-7031 updating copyright year on NOTICES 2020-01-15 16:10:31 -05:00
Pierre Villard
ac5bacccb8
NIFI-6839 - Upgrade jackson-databind direct dependencies 
This closes #3870

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
 2019-11-25 10:58:22 -05:00
Joe Witt
f8c3d877cf
NIFI-6733 updating to next release version for master branch 2019-11-04 13:31:39 -05:00
Joe Witt
418179f5b2
NIFI-6733-RC3 prepare for next development iteration 2019-10-28 15:13:13 -07:00
Joe Witt
b217ae20ad
NIFI-6733-RC3 prepare release nifi-1.10.0-RC3 2019-10-28 15:12:57 -07:00
Mark Payne
e394f6683a
NIFI-6753: Fixed bug where all values being provided to the CSV Writer were String objects, which resulted in the CSV Writer improperly quoting numeric values when the schema indicates that the value is a number. Now, we will only convert the value to a String if the value is not a Number and/or the schema does not indicate a numeric ty type 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3797
 2019-10-09 09:54:45 -04:00
Mark Payne
e18d9ce1e8
NIFI-6369: This closes #3560. Updated JSON Readers to convert String values to Date/Time/Timestamp objects when appropriate according to the schema and the configured pattern 
NIFI-6939: Fix to WriteJsonRecord to deal with date/time/timestamp when no format is explicitly set
 2019-09-19 23:16:34 -04:00
Tamas Palfy
34112519c2 NIFI-6640 - UNION/CHOICE types not handled correctly 
3 important changes:
1. FieldTypeInference had a bug when dealing with multiple datatypes for
 the same field where some (but not all) were in a wider-than-the-other
 relationship.
 Before: Some datatypes could be lost. String was wider than any other.
 After: Consistent behaviour. String is NOT wider than any other.
2. Choosing a datatype for a value from a ChoiceDataType:
 Before it chose the first compatible datatype as the basis of conversion.
 After change it tries to find the most suitable datatype.
3. Conversion of a value of avro union type:
 Before it chose the first compatible datatype as the basis of conversion.
 After change it tries to find the most suitable datatype.

Change: In the RecordFieldType enum moved TIMESTAMP ahead of DATE.

This closes #3724.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2019-09-19 11:02:30 -04:00
FaizFizy Rosle
f6842a5200
Fix syntax error in schema example 
This closes #3745

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
 2019-09-19 07:56:42 -04:00
Bryan Bende
76c2c3fee2
NIFI-6089 Add Parquet record reader and writer 
NIFI-5755 Allow PutParquet processor to specify avro write configuration
Review feedback
Additional review feedback

This closes #3679

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
 2019-09-04 08:52:17 -04:00
Ferenc Szabó
f851a9e96f
NIFI-6519 GrokReader does not evaluate EL while validating "Grok Pattern File" property 
added evaluateAttributeExpressions call where it was missing.

tested manually

This closes #3627

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2019-08-02 09:03:49 -04:00
Peter Turcsanyi
c6e6a418aa
NIFI-6318: Support EL in CSV formatting properties 
CSVReader and CSVRecordSetWriter controller services and also ConvertExcelToCSVProcessor
support EL for Value Separator, Quote Character and Escape Character properties.

NIFI-6318: Fixed null checks and compound OR expression.

NIFI-6318: RecordSetWriterFactory.createWriter() changes.

NIFI-6318: Initialize CSVFormat in onEnabled() if there are no dynamic formatting properties.

NIFI-6318: Comment Marker supports EL.

NIFI-6318: Various review changes.

This closes #3504.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2019-07-25 17:04:22 +09:00
mike
3fb4454375
NIFI-6443 - ParseSyslog5424 n-th SD-ELEMENT isn't parsed 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3588.
 2019-07-18 16:01:32 +02:00
Phillip Grenier
e277545cea
NIFI-6419: Flush the buffers on close to ensure all data is written. 
This closes #3568.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2019-07-12 17:07:11 +09:00