1023 Commits

Author SHA1 Message Date
Tamas Palfy
4aaec5aa38
NIFI-8020 Make sure TailFile doesn't leave FileChannel open when handling NUL characters
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4671.
2020-11-18 00:26:27 +01:00
r65535
d5dc63ded9
NIFI-7982 Added tags to FlattenJSON processor
Adding missing imports

This closes #4662

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-11-17 07:05:52 -05:00
Tamas Palfy
47cc2867ba NIFI-7972 Add a boolean property by which the user can tell the processor to yield (and try again later) whenever it encounters a NUL character. 2020-11-16 15:16:12 -05:00
Denes Arvay
83948db989
NIFI-7996 Conversion with ConvertRecord to avro results in invalid date
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4666.
2020-11-16 17:40:40 +01:00
Peter Gyori
e66362194d NIFI-7922: Added support for GET in ListenHTTP for health check
NIFI-7922: Fixes based on review comments

ListenHTTP: changed if(!sslRequired) to if(sslRequired) so that the positive case comes first.
HealthCheckServlet: response body for GET contains "OK".
ContentAcknowledgmentServlet: super.doDelete() is called when a DELETE should be rejected because of port mismatch.

NIFI-7922: Refactored, based on review comments

NIFI-7922: Fixed a checkstyle violation (organized the imports)

This closes #4603.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-11-16 12:20:30 +01:00
Peter Turcsanyi
018778a25d NIFI-7994: Fixed ReplaceText concurrency issue
Moving RegexReplace.additionalAttrs field to local variable in replace() method.
Concurrent usage of the additionalAttrs map caused data corruption.
2020-11-12 10:45:10 -05:00
Pierre Villard
14ec02f21d
NIFI-7981 - add support for enum type in avro schema
This closes #4648

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-11-05 18:19:55 -05:00
Bence Simon
59e00c4b6f NIFI-7959 Handling node disconnection in MonitorActivity processor
- Make reporting in clustered scope to dependent of expected cluster state in order to prevent unexpected flow file emission

This closes #4642.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-11-04 10:09:46 -05:00
Mark Payne
4b9014b959
NIFI-1121: Updated backend to perform appropriate validation. Added tests. Updated documentation writer. Updated dev guide to explain how PropertyDescriptor.Builder#dependsOn affects validation. Updated JavaDocs for PropertyDescriptor.Builder#dependsOn
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:42 -05:00
mtien
535cab3167
NIFI-1121: Added an additional check for hidden properties to account for transitive dependent properties.
- Added a 'dependent' attribute to determine whether or not to save dependent property values

Co-authored-by: Scott Aslan <scottyaslan@gmail.com>
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:33 -05:00
Mark Payne
e2e901b6b9
NIFI-1121: Added property dependencies to MergeContent
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:26 -05:00
Mark Payne
f7f336a4b0
NIFI-1121: Added API changes for having one Property depend on another
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:08 -05:00
r65535
d8d9aa9cec Added regex support for attribute header selection on HandleHTTPResponse 2020-11-03 12:22:41 -05:00
Denes Arvay
d05d0c6240 NIFI-7925: ValidateRecord reports false positive for avro arrays with null elements 2020-10-21 09:17:29 -04:00
Dan Kim
5b5b9a7800
NIFI-7916 refactor CountText to ignore whitespace words when counting
This closes #4595

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-10-14 09:09:42 -04:00
Matthew Burgess
4c235f0405
NIFI-7909: Change DataTypeUtils.toInteger() to use Math.toIntExact()
This closes #4596

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-10-12 17:53:27 -04:00
Peter Gyori
668c278e30
NIFI-7815: Enhance the logging in MergeContent processor
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4534.
2020-10-08 11:44:52 +02:00
exceptionfactory
efb629e37d
NIFI-7777 Removed support for Expression Language from Password property
Added unit test for no password configured on Zip files

This closes #4572.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-10-07 10:59:50 -07:00
exceptionfactory
ea6b01159d
NIFI-7777 Added optional Password property to UnpackContent for decrypting Zip archives
This closes #4572.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-10-07 10:56:23 -07:00
Denes Arvay
6990f0d3a9
NIFI-7685: Add UTF8 support for FetchFTP
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4446
2020-10-01 17:35:46 -04:00
Mike Thomsen
a66c3d8168
NIFI-2580 Added a little explanation note about a common issue new users face with jsonpath.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4561.
2020-09-30 14:15:07 +02:00
Matthew Burgess
95fb8e3144
NIFI-7503: PutSQL - only call commit() and rollback() if autocommit is disabled
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4558.
2020-09-29 11:08:44 +02:00
Mohammed Nadeem
9c83908c9c
NIFI-7852: Small correction in additional details of LookupRecord
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4555.
2020-09-28 15:10:46 +02:00
Andy LoPresto
9370571131
NIFI-7804 Split nifi-security-utils into sub-module for nifi-security… (#4533)
* NIFI-7804 Split nifi-security-utils into sub-module for nifi-security-utils-api (no external dependencies).
Separated interface and implementation of TlsConfiguration.
Reabsorbed nifi-security-xml-config into nifi-security-utils.

* NIFI-7804 Resolved failing unit test on Java 8.
Removed accidental module dependency.

* NIFI-7804 Resolved failing unit test.

* NIFI-7804 Removed legacy dependency.

* NIFI-7804 Marked nifi-security-utils-api as provided and overrode with compile scope in specific modules which are not children of nifi-standard-services-api-nar.
2020-09-17 12:52:22 -04:00
Peter Gyori
266433e13d
NIFI-7624: ListenFTP processor
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4481.
2020-09-17 12:37:58 +02:00
Matthew Burgess
60a5d71809
NIFI-7803: Allow ExecuteSQL(Record) properties to evaluate flowfile attributes where possible
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4523.
2020-09-15 21:02:52 +02:00
noedetore
85501e08e6
NIFI-7506 add snappy-hadoop to CompressContent
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4321.
2020-09-07 13:54:33 +02:00
Cameron E. Tidd
9a1ae469d8
NIFI-7167 This closes #4486. Fixing resource leaks in IdentifyMimeType
The TikaInputStream and FileInputStream instances utilized in IdentifyMimeType are now explicitly closed. The FileInputStream is additionally wrapped by a BufferedInputStream.

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-09-02 13:22:26 -05:00
Kotaro Terada
c3cab48325
NIFI-7730 Added regression tests for multiple certificate keystores.
Cleaned up JettyServer code.
Changed test logging severity to include debug statements.
Added test resources.

This closes #4498.

Co-authored-by: Kotaro Terada <kotarot@apache.org>
2020-09-01 18:32:03 -07:00
Matthew Burgess
3952c70448 NIFI-7745: Add SampleRecord processor
This closes #4482.

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2020-08-28 14:28:19 -05:00
Tamás Bunth
aaab9ee0cf NIFI-7760 Remove invalid permission check
Because even though permissions are only the 7 least significant bits of the file
mode but the file mode can be wider and can contain further info (like the
sticky bit).

Extend unit test for converting file mode with sticky bit into 'rwx' style
permission string.

Remove old test cases

This closes #4490.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-08-25 19:37:18 +02:00
Pierre Villard
d0c0f9704c
NIFI-7751 - This closes #4483. fix for TestExecuteStreamCommand unit tests
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-08-24 12:52:07 -07:00
Joe Witt
8baa5c9940
NIFI-7692 updating for next dev release 1.13.0 2020-08-18 14:48:02 -07:00
Joe Witt
fb57bcbc11
NIFI-7692-RC1 prepare for next development iteration 2020-08-13 09:20:39 -07:00
Joe Witt
303d6c59ba
NIFI-7692-RC1 prepare release nifi-1.12.0-RC1 2020-08-13 09:20:36 -07:00
Joe Witt
4417b9d64a
NIFI-6666 removing all modifications to the nifi-api and the newly created builder class
This closes #4475.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-08-12 17:49:51 -07:00
Kyle Miracle
a79493082c
NIFI-XXXX Fixed typo in documentation.
This closes #4455.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-08-11 14:43:06 -07:00
Andy LoPresto
96810f677f NIFI-7723 Upgraded BouncyCastle version to 1.66. 2020-08-11 14:08:57 -04:00
Joe Witt
536dbb72bb
NIFI-7703 updated all commons codec references to 1.14
This closes #4448.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-08-04 12:11:57 -07:00
Mike Thomsen
0861b2f632
NIFI-7605 Removed user-agent default value so no header will be sent by default.
Added and updated unit tests.

This closes #4428.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
Signed-off-by: Joe Witt <joe.witt@gmail.com>
2020-08-03 17:23:56 -07:00
Tamás Bunth
455f48fce4
NIFI-7640 Add documentation: temporary directory (#4414)
NiFi uses the Java IO temporary directory for storing HTTP multipart
files when using HandleHttpRequest processor. The directory can be
overwritten with Java command line parameter.
2020-07-29 13:57:34 -07:00
Andy LoPresto
7f0416ee8b
NIFI-7680 Added convenience methods for creating XML DocumentBuilder instances.
Added unit tests.

NIFI-7680 Duplicated DocumentBuilder creation method in NotificationServiceManager to avoid nifi-bootstrap dependency on nifi-security-utils.
Explicitly added commons-lang3 to lib/bootstrap/ directory in nifi-assembly.

NIFI-7680 Reverted unnecessary dependency changes.
Added explicit dependencies where necessary.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4436
2020-07-28 17:02:42 -04:00
Mark Payne
e926a17968
NIFI-7683: Fixed bug that resulted in a byte[] being allocated and held onto by a member variable that was unprotected. This caused the byte array to be modified by two different threads concurrently, which can potentially cause corruption of FlowFile's data
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4439
2020-07-28 14:52:57 -04:00
Andy LoPresto
eeeda84474
NIFI-7122 Introduced improvements to EncryptContent processor.
Added unit tests.
Refactored shared logic from various algorithm-specific secure hasher implementations to AbstractSecureHasher.
Introduced secure hasher implementations for various KDFs.
Added custom validation to EncryptContent processor.
Implemented logic for EncryptContent to write operational metadata to flowfile attributes.
Added encryption metadata attribute annotations to EncryptContent.
Added Argon2 KDF documentation and Bcrypt key derivation change notes to Admin Guide.
Updated unit tests to calculate default/recommended cost parameters for Argon2.

This closes #4421.

Co-authored-by: mtien <mtien.apache@gmail.com>
2020-07-24 12:31:39 -07:00
Tamás Bunth
b1e77019ac NIFI-6128 UnpackContent: Store unpacked file data
Tar format allows us to archive files with their original permission,
owner, group name and last modification time.

When unpacking with Tar unpacker, these information are stored in new
attributes with names: "file.inner.*". This way, it preserves backward
compatibility when using parallel with GetFile processor (which stores
information in "file.*").

NIFI-6128 Tar unpackContent: assert date of last modification of content is a valid date format.

NIFI-6128 UnpackContent: use original attributes

In case of tar format:
- Use "file.*" attributes instead of "file.inner.*" (which eventually
  lead to overwrite if tar had been fetched with GetFile)
- Store file permission in "rwx" format, instead of integer
  representation.
- Also replace SimpleDateFormat with DateTimeFormatter.

Replace generic error with IllegalArgumentException
Also refactor: move permission string to top of file as static constant.

Update test

Remove 'file systems' text

Unpackcontent: Fill file.creationTime attribute
which always holds the very same value as file.lastModifiedTime.

This closes #4370.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-07-14 18:04:56 +02:00
Tamás Bunth
72666a9c7e NIFI-7589 Fix path value when unpacking tar
Refactor: remove variable reassignment

This closes #4371.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-07-14 17:32:51 +02:00
Tamas Palfy
ad95287e78
NIFI-6934 In PutDatabaseRecord added DatabaseAdapter-based UPSERT support for Postgres (9.5+)
NIFI-6934 Added more documentation and unit tests.

NIFI-6934 Added missing license for new test class.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4350
2020-07-09 11:26:41 -04:00
Tamas Palfy
c396927299 NIFI-7594 In HandleHttpRequest deleting multipart file resources after processing.
This closes #4379.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-07-06 14:15:43 +02:00
Tamás Bunth
91cd74d725 NIFI-7576 ListenHTTP: Honor primary node only
ListenHTTP processor now binds port and creates a HTTP connection only
if one of the following conditions apply:
- Primary node execution is 'false'
- Primary node execution is 'true' and node is elected as primary node.

Changes:
- Connection is established in 'onTrigger' annotated method instead of
  'onSchedule'. (This is similar to how handleHTTPRequest processor
  handles connections.)
- 'onPrimaryNodeStateChange' annotated method is introduced to tear down
  server on reelection of primary node

This closes #4356.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-06-23 14:26:45 -04:00
mtien
005d05f20b
NIFI-7542 Override jackson-databind version.
NIFI-7542 Override additional jackson-databind versions.
NIFI-7542 Upgrade jackson-databind dependency to 2.9.10.5 in the root pom.xml.

This closes #4343

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-06-22 17:37:57 -04:00