1143 Commits

Author SHA1 Message Date
exceptionfactory
3621a2182c
NIFI-8424 Removed SeeAlso references to service interfaces
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5269.
2021-08-02 20:20:11 +02:00
exceptionfactory
f539d9886e
NIFI-8956 Changed TestListenHTTP to expect IOException
- Replaced expected SSLException with IOException in assertThrows
- Removed conditional override of jdk.tls.disabledAlgorithms in test SslContextUtils

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5260.
2021-07-29 09:44:43 +02:00
exceptionfactory
cb020072f7 NIFI-8770 Use queue drainTo() on shutdown in HandleHttpRequest
- Refactored response handling to use shared sendError() method
- Standardized request logging to include HTTP Method and URI

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5218.
2021-07-20 13:21:48 -04:00
exceptionfactory
828b6c1bcc
NIFI-5174 Update README.md Minimum Requirements
- Remove reference to ongoing work for Java 11
- Remove references to Bower which is no longer used as of NIFI-2781

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5232
2021-07-19 17:58:05 -04:00
exceptionfactory
ebdc5eb92b
NIFI-7443 Corrected SFTP Keep Alive behavior
- Configured SSHJ Keep Alive Interval of 5 seconds
- Updated Send Keep Alive On Timeout property description

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5223.
2021-07-19 20:26:45 +02:00
Nathan Gough
af0f3403a5
NIFI-8788 - Upgraded dependencies and removed unnecessary log4j test dependencies.
This closes #5222

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-07-16 14:12:09 -05:00
exceptionfactory
00f385b51b
NIFI-8789 Corrected TestListenUDP to use getAvailableUdpPort()
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5219.
2021-07-16 15:13:05 +02:00
Joe Witt
7356332852
Merge branch 'NIFI-8767-RC2' 2021-07-14 09:07:22 -05:00
exceptionfactory
05d7867a16
NIFI-8768 Added toLocalDate() for convertType() handling of DATE fields
- Updated PutKudu to use DataTypeUtils.toLocalDate() for DATE fields
- Updated PutDatabaseRecord to remove convertDateToLocalTZ() since convertType() uses toLocalDate()
- Updated PutElasticsearchHttpRecord to use default time zone format for DATE fields
- Updated WriteXMLResult to use default time zone format for DATE fields
- Updated WriteJsonResult to use default time zone format for DATE fields
- Updated AvroTypeUtil to use toLocalDate() for logical DATE fields
- Updated JdbcCommon to avoid conversion to UTC for logical DATE fields
- Updated Processor and RecordReader unit tests for consistency in DATE comparison

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5210
2021-07-13 16:17:36 -04:00
Joe Witt
97feacc181
NIFI-8767-RC2 prepare for next development iteration 2021-07-10 12:17:09 -07:00
Joe Witt
fcbf1d5f97
NIFI-8767-RC2 prepare release nifi-1.14.0-RC2 2021-07-10 12:17:05 -07:00
exceptionfactory
75de68e013
NIFI-8764 Refactored UnpackContent to use both Commons Compress and Zip4j
- UnpackContent uses Zip4j when configured with a password property
- UnpackContent uses Commons Compress when a password is not specified

NIFI-8764 Updated Password property description mentioning disabled algorithms

NIFI-8764 Adjusted Password property description

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5201
2021-07-09 13:15:44 -04:00
Nathan Gough
9a125dcaaf
NIFI-8616 Refactored PutTCP, PutUDP, and PutSplunk to use Netty nifi-event-transport classes
This closes #5182

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-07-07 16:24:49 -05:00
Matthew Burgess
e16016b4ab
NIFI-8530: Improved DELETE handling in PutDatabaseRecord for non-nullable columns
This closes #5173

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-07-07 14:51:40 -05:00
exceptionfactory
6c7c3b3c1b
NIFI-8711 Added SFTP support for keyboard-interactive authentication
- Included debug logging of configured Authentication Methods

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5166.
2021-06-30 14:03:28 +02:00
exceptionfactory
786f17182b
NIFI-8630 Upgraded javax.mail 1.4.7 to jakarta.mail 2.0.1 for PutEmail
- Added jakarata.activation 2.0.1 to support jakarta.mail 2.0.1
- Clarified description of SMTP TLS property enabling STARTTLS

NIFI-8630 Added final to several variables and adjusted variable declaration

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5179.
2021-06-25 19:18:16 +02:00
Jon Kessler
115bba9ab0
NIFI-8523 Added SFTP algorithm and cipher properties
- Updated secure FTP processors to configure which algorithms, ciphers and message authentication codes are allowed to be used by the SSH Client
- Included Expression Language Variable Registry support for properties

This closes #5061

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-24 11:51:04 -05:00
Mark Payne
6fc9f33ba2
NIFI-8720: Ensure that when we use Expression Language for Records Per Split property of SplitRecord, we treat the result as a value of 1 if it's less than 1.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5171
2021-06-21 13:09:20 -04:00
exceptionfactory
2efc2b6b8b
NIFI-8718 Upgraded Apache Commons IO to 2.10.0
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5169
2021-06-21 11:29:27 -04:00
rhavermans
9fd8042cfb
NIFI-8664 Update json-path lib from 2.4.0 -> 2.6.0
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5132
2021-06-21 11:01:10 -04:00
exceptionfactory
3cdab0d752
NIFI-8707 Upgraded SSHJ to 0.31.0
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5162.
2021-06-16 12:06:06 +02:00
Kent Nguyen
87c02c17d2
NIFI-8674: Update Apache Calcite version to 1.27.0
This closes #5138

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-14 15:47:47 -05:00
exceptionfactory
ab851b9cde NIFI-8672 Added syncUninterruptibly() to Netty shutdown and close methods
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5139.
2021-06-12 17:14:34 -04:00
Timea Barna
d7a8d275c9 NIFI-8642 Select the default Old Gen Memory Pool for Memory Reporting Task
This closes #5115.

Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
2021-06-11 14:06:15 +02:00
Mark Payne
c9dee30294
NIFI-8658: Allow Filter Functions and expressions to be specified as a RecordPaths
NIFI-8658: Addressed issue where the RecordField that was provided from Function Filters were not accurate

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5125
2021-06-04 17:17:33 -04:00
exceptionfactory
7fb0293c3e NIFI-8626 Upgraded okhttp-digest to 2.5
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5096.
2021-05-27 12:41:11 -04:00
Mark Payne
08edc33eb7
NIFI-8629: Implemented the LogComponentStatuses task that runs periodically in stateless.
Also noticed a typo in the ControllerStatusReportingTask and found in comparing outputs
that it had a bug that caused it to log counters generated only by processors at the root level so fixed that.

This closes #5101

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-05-26 20:07:26 -05:00
exceptionfactory
1e6161c0aa
NIFI-8361 Upgraded Zip4j to 2.8.0
- Upgrade resolves issue unpacking Zip files with temporary spanning markers

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5103.
2021-05-26 20:21:11 +02:00
exceptionfactory
a3365c8833 NIFI-8462 Refactored PutSyslog and ListenSyslog using Netty
- Added nifi-event-transport module encapsulating Netty classes
- Refactored unit tests for PutSyslog and ListenSyslog
- Removed integration tests for PutSyslog and ListenSyslog

NIFI-8462 Added context.yield() in PutSyslog when no FlowFiles and addressed other issues

NIFI-8462 Removed unused import of ExpressionLanguageScope

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5044.
2021-05-25 10:54:15 -04:00
Tamas Palfy
5108d7cdd0 NIFI-8439 Update parquet-avro to allow reading parquet INT96 timestamps as byte arrays (instead of throwing an exception).
Also allow to write them as such (byte-arrays) - again, instead of throwing an exception.

NIFI-8439 Fixed unit tests.
NIFI-8439 Allow writing parquet INT96 timestamps if they were read by the same parquet-avro library.

This closes #5006.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-05-25 13:18:46 +02:00
Mohammed Nadeem
622507f994
NIFI-8627: Upgrade apache derby to 10.14.2.0 to mitigate CVE_2015_1832
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5095
2021-05-24 14:35:27 -04:00
Mohammed Nadeem
c113960b81
NIFI-8613: Improve FlattenJson Processor
- Unflattening a flattened json
- Preserving primitive arrays such as strings, numbers, booleans and null in a nested json
- Logging errors when failure
- Pretty printing resulted json

This closes #5083

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-05-20 15:04:23 -05:00
Mark Payne
ecacfdaa4c NIFI-8469: Introduced ProcessSession.commitAsync and updated processors to use it. Deprecated ProcessSession.commit()
- Updated Mock Framework to now fail tests that use ProcessSession.commit() unless they first call TestRunner.setAllowSynchronousSessionCommits(true)
- Updated stateless nifi in order to make use of async session commits
- Fixed bug that caused stateless to not properly handle Additional Classpath URLs and bug that caused warnings about validation to get generated when a flow that used controller services was initialized. While this is not really in scope of NIFI-8469, it was found when testing and blocked further progress so addresssed here.
- If Processor fails to progress when run from stateless, trigger from start of flow until that is no longer the case
- Introduced notion of TransactionThresholds that can limit the amount of data that a flow will bring in for a given invocation of stateless dataflow
- Several new system-level tests
2021-05-19 08:51:16 -04:00
Matthew Burgess
f812dfdfc0
NIFI-8535: Better error message in PutDatabaseRecord when table does not exist (#5070)
* NIFI-8535: Better error message in PutDatabaseRecord when table does not exist
2021-05-14 15:13:00 -04:00
exceptionfactory
6776765a92
NIFI-8538 Upgraded Apache Commons IO to 2.8.0
- Upgraded direct dependencies from 2.6 to 2.8.0
- Added dependency management configuration to use 2.8.0 for some modules
- Updated scripted Groovy tests to avoid copying unnecessary files

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5073
2021-05-14 09:31:48 -04:00
exceptionfactory
340b13033f NIFI-8367 Removed joda-time from nifi-standard-processors 2021-05-11 14:50:49 -04:00
matthewlannan
86d5b08d23 Adding addititonal documentation to EvalluateJsonPath per https://issues.apache.org/jira/browse/NIFI-8518 2021-05-11 14:40:12 -04:00
exceptionfactory
5062f2e54e
NIFI-8515 Upgraded Apache Tika to 1.26
- Upgraded tika-core from 1.24 to 1.26 in nifi-framework-bundle and nifi-standard-bundle
- Upgraded tika-parsers from 1.24.1 to 1.26 in nifi-media-processors

NIFI-8515 Removed jsr-275 exclusion and confirmed it is no longer included

- TIKA-2535 Replaced the jsr-275 dependency to resolve licensing issues

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5056
2021-05-10 11:13:31 -04:00
Matthew Burgess
30fc26647e
NIFI-6061: Fix CLOB/BLOB handling in PutDatabaseRecord
NIFI-6061: Force getBytes() in BLOB handling to use UTF-8 charset

NIFI-6061: Use setClob(), added unit tests, incorporated review comments

This closes #5049

Co-authored-by: zhangcheng <zhangchengk@foxmail.com>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-05-04 14:58:14 -05:00
Paul Grey
cc554a6b11
NIFI-8478 Addressed i18n test issues; CI supplies locale to surefire
This closes #5040

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-05-03 13:40:01 -05:00
Matthew Burgess
3963f66dff
NIFI-8320: Fix column mismatch in PutDatabaseRecord
This closes #5024

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-04-30 10:41:36 -05:00
Tamas Palfy
16dc61e151 NIFI-8458 TailFile - Fix some bugs. Add more tests.
- During cleanup keep "tailingPostRollover" in the updated state.
- Skipping tests that can't run on Windows.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-04-27 09:44:39 -04:00
Peter Turcsanyi
525bfe00b3
NIFI-8474: Added new Replacement Strategy for variable substitution in ReplaceText.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5032.
2021-04-27 13:03:21 +02:00
Paul Grey
835f50c83a
NIFI-8465 Handle bcrypt legacy decrypt failures in testing
This closes #5029

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-04-26 11:34:20 -05:00
Paul Kelly
aedacdf86f
NIFI-8454: Allow InvokeHTTP to output final URL from response request property
This closes #5016

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-04-21 18:18:25 -05:00
Mark Payne
1ef1905461 NIFI-8344: Addressed corner case in which we didn't handle a case where we tailed a file after being rolled over and then processor encounters a newline followed by NUL characters
This closes #5009.

Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
2021-04-20 14:51:24 +02:00
Joe Gresock
be716c1621 NIFI-3862: Adding Issuer DN support to ListenHTTP
This closes #4976.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-04-20 13:09:53 +02:00
exceptionfactory
ef794c56b6
NIFI-8398 Updated dependencies to build using Apache Maven 3.8.1
- Updated ParCEFone to 1.2.8
- Added conjars.org HTTPS repository for Hive transitive dependencies

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4989
2021-04-12 14:45:02 -04:00
Mark Payne
7d1d536da6
NIFI-8206: Added identifiesExternalResource() method to PropertyDescriptor.Builder and implemented functionality.
- Updated components to make use of new feature

NIFI-8206: Added a ResourceType of TEXT. This requires that the ResourceReferenceFactory know which types are allowed in order to create the ResourceReference. PropertyValue needs to then have the PropertyDescriptor available to it. This resulted in highlighting many bugs in unit tests where components were not exposing property descriptors via getSupportedPropertyDescriptors() or were evaluating Expression Language using the wrong scope, so fixed many unit tests/components to properly declare Expression Language scope when using it

NIFI-8206: Removed problematic unit test that required directory names with special characters that are not allowed on some operating systems

This closes #4890.

Signed-off-by: Bryan Bende <bbende@apache.org>
2021-04-12 11:48:04 -04:00
Mark Payne
2edf5514b7 NIFI-8344: If configured to tail a file for some period of time post-rollover, ensure that we only consume lines that are fully written (i.e., end with a newline). Once we stop tailing that file post-rollover, consume any data from that file that has not yet been consumed, up to the end of the file, even if there is no newline.
This closes #4937.

Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
2021-03-31 13:40:08 +02:00