937 Commits

Author SHA1 Message Date
Pierre Villard
578430c9d9 NIFI-7197 - In-place replacement in LookupRecord processor
This closes #4088

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-03-06 15:35:59 -05:00
Joe Witt
20dda05f26
NIFI-7232 if users do not supply a remote path PutSFTP with conflict resolution will fail with an NPE 2020-03-06 12:46:02 -05:00
Joe Witt
040c8a0af9
NIFI-7222 Cleaned up API for FTP/SFTP remote file retrieval and ensure we close remote file resources for SFTP pulls in particular
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4115.
2020-03-06 12:48:15 +01:00
Matthew Burgess
74b1b2fc59 NIFI-7208: Fixed PutSQL/JdbcCommon handling of timestamps (nanoseconds, e.g.) 2020-03-04 11:46:47 -05:00
Eduardo Fontes
bad254ec5b
NIFI-4970 - EOF Exception in InvokeHttp when body's response is empty with gzip
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4109.
2020-03-04 10:36:06 +01:00
karthik.kadajji
2cf4fde686
NIFI-5644 Fixed typo in getWrappedQuery method of AbstractDatabaseFetchProcessor class
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4106.
2020-03-03 16:43:56 +01:00
Joe Witt
4cd63c99e8
NIFI-7205 NIFI-7206
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4093.
2020-02-26 14:02:49 -08:00
Shawn Weeks
0bb8ce7438
NIFI-7139 Add record.error.message on failure of a record reader or writer
Handle scenario where message might be null.

Update to test case that was failing because adding attributes modified a flow file even if you don't change the contents.

Fixed Style Issues and Updated WritesAttributes.

Added Test Case for Error Message

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4052
2020-02-26 08:50:32 -05:00
Pierre Villard
4f315edc6e
NIFI-7133 - This closes #4049. Clarification of EnforceOrder description
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-02-23 11:59:49 -08:00
Pierre Villard
fd7c01453b
NIFI-7184 - This closes #4074. Added mime type property to GenerateFlowFile
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-02-23 11:54:28 -08:00
Pierre Villard
9a8a551e03
NIFI-7183 - This closes #4073. Improve ReplaceText when removing FF's content
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-02-23 11:50:34 -08:00
Joe Witt
f8a5d1295c
NIFI-7175 removed appveyor and travis builds which have become unreliable. Added Github Actions based CI build for pull requests and pushes. Removed unit tests which make unreadable or unaccesible dirs. Includes windows build as well as ubuntu. we can do localization based builds later as this is all proven stable.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4058.
2020-02-20 16:36:51 -08:00
Troy Melhase
0de89452f1
NIFI-6927 Consolidate SSL context and trust managers for OkHttp on JDK9.
Fixes name conflicts.

This closes #4047.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-02-19 16:19:39 -08:00
David Savage
5e964fbc47
NIFI-4792: Add support for querying array fields in QueryRecord
Work in progress adding support for array based queries
updated calcite dependency

tidy up unused imports highlighted by checkstyle in travis build

tidy up }s highlighted by checkstyle in travis build

Add test for use case referenced in NIFI-4792

Bumped Calcite version to 1.21.0

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4015
2020-02-11 10:59:20 -05:00
John Highcock
f738e19a75
NIFI-2537: Add custom MIME type configuration support to IdentifyMimeType
Add two new properties to IdentifyMimeType (Config File and Config Body).
Specifying one of these properties will override the default NiFi MIME
type configuration and use the configured property's MIME config instead.
Add additional runtime documentation for IdentifyMimeType's usage. The
default behavior of IdentifyMimeType is unchanged.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4016.
2020-02-06 19:34:18 -05:00
mdayakar
850869c6d2
NIFI-7049 : SFTP processors shouldn't silently try to access known hosts file of the user
Signed-off-by: Arpad Boda <aboda@apache.org>

This closes #4014
2020-01-30 15:15:43 +01:00
Joe Witt
3de77ebacc
NIFI-7021-RC3 prepare for next development iteration 2020-01-19 14:14:40 -05:00
Joe Witt
633408bce7
NIFI-7021-RC3 prepare release nifi-1.11.0-RC3 2020-01-19 14:14:38 -05:00
Joe Witt
b205b99668
NIFI-7041 This closes #4000. Ensure that if the permissions arent set by the flowfile or processor property that we dont attempt to set perms on the remote host
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-19 08:47:58 -05:00
Joe Witt
0789ec3a6b
NIFI-6882 from plain patch file in JIRA contributed by 'Josh'
Removed tests which created dirs outside target location and which do not appear unit testable.
2020-01-18 21:02:50 -05:00
Nathan Gough
4ec9155cbc
NIFI-6770 - Set validator to Validator.VALID to allow empty password for truststores.
Added no-password keystore for tests
System NiFi truststore now allows a passwordless truststore. Added a unit test to prove this.
Forgot no-password-truststore.jks file for the unit test.
Refactored utility method from CertificateUtils to KeyStoreUtils.
Added utility methods to verify keystore and key passwords.
Added unit tests.
Implemented different keystore and truststore validation logic.
Refactored internal custom validation in StandardSSLContextService.
Added unit test resource for keystore with different key and keystore passwords.
Added unit test to generate passwordless truststore for https://nifi.apache.org for live testing.
Resolved NPE in SSLContext generation in StandardSSLContextService
Added unit test to generate passwordless truststore for localhost for InvokeHTTP testing.
Resolved TrustManagerFactoryImpl initialization error.
Fixed unit test without proper cleanup which caused RAT failures.

Co-authored-by: Andy LoPresto <alopresto@apache.org>

This closes #3823.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-01-17 20:01:06 -08:00
Joe Witt
23c8234586
NIFI-7031 updating copyright year on NOTICES 2020-01-15 16:10:31 -05:00
Michael Hogue
421bfdd5ff
NIFI-6919: Added relationship attribute to DistributeLoad
NIFI-6919: Cleaned up docs

NIFI-6919: Cleanup

NIFI-6919: Cleanup

NIFI-6919: added negative unit test

NIFI-6919: Removed unnecesary feature flag

Updated attribute description

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3939
2020-01-14 16:14:48 -05:00
John Pierce
0dc6439ddb
NIFI-6964 This closes #3975. Use compression level for xz-lzma2 format of the CompressContent processor
NIFI-6965 adding resource warning on memory and highlighting this with the compression level property

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-13 15:08:13 -05:00
Yolanda Davis
ebd33452c4 NIFI-6962 - moved HashAlgorithm and HashService to nifi-security-utils (#3947) 2020-01-08 11:10:58 -08:00
Matthew Burgess
c604923c0b NIFI-6947: Add PutRecord processor using RecordSinkService (#3943)
* NIFI-6947: Add PutRecord processor using RecordSinkService

* NIFI-6947: Incorporated review comments
2020-01-08 04:25:14 -04:00
Matthew Burgess
f4ef45678e
NIFI-6963: Fix ValidateRecord handling of missing required array fields
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3948.
2020-01-08 02:02:04 +07:00
Shawn Weeks
b470db620b
NIFI-6979 Add record.index field to UpdateRecord
This closes #3955

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-01-03 19:25:01 -05:00
Matthew Burgess
8b3f6fa61c NIFI-6953: Add missing documentation for input.flowfile.uuid attribute of ExecuteSQL processors 2019-12-16 14:48:12 -05:00
Pierre Villard
ac5bacccb8
NIFI-6839 - Upgrade jackson-databind direct dependencies
This closes #3870

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-11-25 10:58:22 -05:00
Matthew Burgess
d617c0b96a NIFI-6865: Added Fetch Size property to ExecuteSQL processors
This closes #3888.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2019-11-19 23:33:46 +01:00
Koji Kawamura
b3880a4a06
NIFI-5970 Handle multiple input FlowFiles at Put.initConnection
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3583
2019-11-19 16:48:14 -05:00
Koji Kawamura
86cae184ff
NIFI-6599 Fix MergeRecord failure in defragment mode
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3678.
2019-11-14 14:44:01 +01:00
Ivan Ezequiel Rodriguez
7f96fa1d0d
NIFI-6395: Thread-safety bug fixed and added new flag property to handle the way to adjust the counters
Update CountText.java

Local variable changes by AtomicInteger

NIFI-6395 - Fix line is longer than 200 characters

This closes #3552.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-11-05 20:54:25 +09:00
Nicholas Zhan
af0777b2c9
NIFI-6834 Encoding attachment filename
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3865.
2019-11-05 11:10:15 +01:00
Joe Witt
f8c3d877cf
NIFI-6733 updating to next release version for master branch 2019-11-04 13:31:39 -05:00
Joe Witt
418179f5b2
NIFI-6733-RC3 prepare for next development iteration 2019-10-28 15:13:13 -07:00
Joe Witt
b217ae20ad
NIFI-6733-RC3 prepare release nifi-1.10.0-RC3 2019-10-28 15:12:57 -07:00
Jan Hentschel
ccf85777c4
NIFI-6816 Removed duplicated dependency declarations
This closes #3851

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-10-28 12:49:08 -04:00
Otto Fowler
cfcf2e698d
NIFI-6682 SplitJson does not work if the input is null
This closes #3772

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-10-25 17:43:13 -04:00
Andrew Rodriguez
cd90b1d3e1
NIFI-6773 This closes #3820. Adding lz4-framed option for compression/decompression
NIFI-6773 Adding two unit tests for lz4-framed compression/decompression
NIFI-6773 Adding compressed lz4 file for decompression test

Signed-off-by: Joe Witt <joewitt@apache.org>
2019-10-16 17:45:32 -04:00
Mark Bean
c4ccd5f505
NIFI-6750: This closes #3810. add FETCH provenance event to FetchFile
Signed-off-by: Joe Witt <joewitt@apache.org>
2019-10-15 14:52:39 -04:00
Joe Witt
0f02de6002
NIFI-6733 updating key apache commons dependencies and apache base dependency for build
This closes #3791.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2019-10-07 22:20:48 -04:00
Joe Witt
6d8968cc87
NIFI-6520 update to tika 1.22
This closes #3756

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-09-23 08:18:55 -04:00
Mark Payne
e18d9ce1e8
NIFI-6369: This closes #3560. Updated JSON Readers to convert String values to Date/Time/Timestamp objects when appropriate according to the schema and the configured pattern
NIFI-6939: Fix to WriteJsonRecord to deal with date/time/timestamp when no format is explicitly set
2019-09-19 23:16:34 -04:00
Mark Payne
d0371a5ef1
NIFI-6691: This closes #3754. Fixed bug in MapRecord's algorithm for incorporating Inactive Fields, by recursing into any 'child fields' that contain records; also fixed bug in UpdateRecord that caused it to incorrectly map the first result to all elements if the key of the property points to an array 2019-09-19 22:53:39 -04:00
Joe Witt
13381c2254
NIFI-5816 converting test to integration test as it is test environment specific 2019-09-19 21:46:06 -04:00
Tamas Palfy
34112519c2 NIFI-6640 - UNION/CHOICE types not handled correctly
3 important changes:
1. FieldTypeInference had a bug when dealing with multiple datatypes for
 the same field where some (but not all) were in a wider-than-the-other
 relationship.
 Before: Some datatypes could be lost. String was wider than any other.
 After: Consistent behaviour. String is NOT wider than any other.
2. Choosing a datatype for a value from a ChoiceDataType:
 Before it chose the first compatible datatype as the basis of conversion.
 After change it tries to find the most suitable datatype.
3. Conversion of a value of avro union type:
 Before it chose the first compatible datatype as the basis of conversion.
 After change it tries to find the most suitable datatype.

Change: In the RecordFieldType enum moved TIMESTAMP ahead of DATE.

This closes #3724.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-09-19 11:02:30 -04:00
Bryan Bende
ff6a7d9561
NIFI-5816 This closes #3726. Remove unused SFTP classes that reference Jsch
NIFI-5816 Switching SFTP processors from JSCH to SSHJ
NIFI-5816 LICENSE/NOTICIE updates to reflect changing from JSch to SSHJ

Signed-off-by: Joe Witt <joewitt@apache.org>
2019-09-19 10:58:53 -04:00
Hsin-Ying Lee
758035b964 NIFI-6567 HandleHttpRequest does not shutdown HTTP server in some cir… (#3673)
* NIFI-6567 HandleHttpRequest does not shutdown HTTP server in some circumstances

signed-off by: Peter Wicks <patricker@gmail.com>
2019-09-11 09:27:52 -06:00