Commit Graph

3990 Commits

Author SHA1 Message Date
Cameron E. Tidd 9a1ae469d8
NIFI-7167 This closes #4486. Fixing resource leaks in IdentifyMimeType
The TikaInputStream and FileInputStream instances utilized in IdentifyMimeType are now explicitly closed. The FileInputStream is additionally wrapped by a BufferedInputStream.

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-09-02 13:22:26 -05:00
Kotaro Terada c3cab48325
NIFI-7730 Added regression tests for multiple certificate keystores.
Cleaned up JettyServer code.
Changed test logging severity to include debug statements.
Added test resources.

This closes #4498.

Co-authored-by: Kotaro Terada <kotarot@apache.org>
2020-09-01 18:32:03 -07:00
Matthew Burgess 45470b0984 NIFI-7740: Add Records Per Transaction and Transactions Per Batch properties to PutHive3Streaming
NIFI-7740: Incorporated review comments

NIFI-7740: Restore RecordsEOFException superclass to SerializationError

This closes #4489.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-09-01 17:17:02 +02:00
Mohammed Nadeem 943904c12c NIFI-7779: Avoid NPE when a ExecuteScript Processor has null value properties during a component search from canvas 2020-09-01 09:40:32 -04:00
jmconte bdead4d040 NIFI-7762 - support copy-paste on Disabled ports 2020-08-31 15:56:12 -04:00
Bryan Bende 87685dd4e1 NIFI-7661 Change validation of controller service against it's API to use comparison of inteface methods instead of bundle coordinates 2020-08-31 15:33:45 -04:00
Matthew Burgess 3952c70448 NIFI-7745: Add SampleRecord processor
This closes #4482.

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2020-08-28 14:28:19 -05:00
Kristjan Antunovic 483af3a460
NIFI-6767 Persist registry URL after service restart.
This closes #4006.

Signed-off-by: Bryan Bende <bbende@apache.org>
2020-08-28 09:58:39 -04:00
Denes Arvay ca43615702 NIFI-7714: QueryCassandra loses precision when converting timestamps to JSON
Updated the patch based on @tpalfy's review
Updated the patch based on @mattyb149's review
Rename DATE_FORMAT_PATTERN to JSON_TIMESTAMP_FORMAT_PATTERN
Changed convertToJsonStream method's visibility to package private.
Removed json prefix from timestamp-format-pattern property to make it more generic

This closes #4463.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-08-27 17:06:57 +02:00
Tamás Bunth aaab9ee0cf NIFI-7760 Remove invalid permission check
Because even though permissions are only the 7 least significant bits of the file
mode but the file mode can be wider and can contain further info (like the
sticky bit).

Extend unit test for converting file mode with sticky bit into 'rwx' style
permission string.

Remove old test cases

This closes #4490.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-08-25 19:37:18 +02:00
Pierre Villard d0c0f9704c
NIFI-7751 - This closes #4483. fix for TestExecuteStreamCommand unit tests
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-08-24 12:52:07 -07:00
Joe Witt 8baa5c9940
NIFI-7692 updating for next dev release 1.13.0 2020-08-18 14:48:02 -07:00
Joe Witt fb57bcbc11
NIFI-7692-RC1 prepare for next development iteration 2020-08-13 09:20:39 -07:00
Joe Witt 303d6c59ba
NIFI-7692-RC1 prepare release nifi-1.12.0-RC1 2020-08-13 09:20:36 -07:00
Joe Witt 1b121ba18f
NIFI-7692 reduced excessive test file size
Self merging as part of RC1 generation.

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-08-13 08:49:07 -07:00
Tamas Palfy 6c3c2c173b NIFI-7736 Fix: Empty All Queues performance - When authorizing request, checking read/write credentials on encapsulated process groups and connections was doubly recursive (for each element it added all child elements multiple times again, making this exponential instead of linear). 2020-08-13 10:42:44 -04:00
Matthew Burgess b7b2533ffe NIFI-7734: Added Record Separator property to CSVReader 2020-08-12 22:05:08 -04:00
Joe Witt 4417b9d64a
NIFI-6666 removing all modifications to the nifi-api and the newly created builder class
This closes #4475.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-08-12 17:49:51 -07:00
Tamas Palfy 3e18d5b236 NIFI-7722 Catching 'Throwable' instead of 'Exception' in 'NodeHttpRequest.run()' to make sure subsequent 'Object.notify()' (when request replication has completed) is not missed out even if an 'Error' is thrown. 2020-08-12 12:38:56 -04:00
Mark Payne 869c4236c4
NIFI-7729: Ensure that if a script used in the ScriptedTransformRecord processor introduces a new field to the schema that the field gets incorporated into the schema.
NIFI-7729: Updated docs to explain how to add new fields to Records via ScriptedTransformRecord and added example. Fixed checkstyle violation.

NIFI-7729: Always call Record.incorporateInactiveFields with ScriptedTransformRecord

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4470
2020-08-11 18:06:55 -04:00
Kyle Miracle a79493082c
NIFI-XXXX Fixed typo in documentation.
This closes #4455.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-08-11 14:43:06 -07:00
Andy LoPresto 96810f677f NIFI-7723 Upgraded BouncyCastle version to 1.66. 2020-08-11 14:08:57 -04:00
Bryan Bende cea569169f NIFI-7672 Remove filtering of invalid components when getting service ids to enable, lower layers will already skip enabling invalid services, so we want to include services that may become valid when enabling their dependent services 2020-08-10 14:33:49 -04:00
bibistroc 51091a5fca
NIFI-7338 This closes #4169. Added comment about com.hierynomus.smbj.SMBClient syncronicity
- added file.size attribute on GetSmbFile

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-08-10 09:44:52 -07:00
si 71d155e874
NIFI-7338 fixed provenance, top level dir,
chatty smbj log, pw validation
update accessmask to be more restrictive

Author:    si <si.sun@roche.com>

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-08-10 09:43:43 -07:00
Joe Witt 58e324e9f1
NIFI-7522 bumping from Apache Kafka 2.5 to Apache Kafka 2.6
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4462.
2020-08-10 09:55:53 +02:00
Mark Payne 0f18c5966e NIFI-7592: Updated nifi system test to no longer include nifi-server-nar and also changed a unit test to an integration test because it starts an embedded server on a specific port
This closes #4459.
2020-08-07 12:51:17 -04:00
Matthew Burgess 55adb11f06 Revert "NIFI-7592: Allow NiFi to be started without a GUI/REST interface"
This reverts commit 1b91a6caf3.
2020-08-07 12:50:48 -04:00
Pierre Villard 27e7d0b02a
NIFI-7707 - This closes #4451. fixed Publish GCP PubSub processor for binary data
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-08-07 08:14:19 -07:00
Tamas Palfy 3ef566f7da NIFI-7663 Added option for emptying all queues in a process group. Available from context menu.
NIFI-7663 Minor changes (variable name refactor, javadoc, GUI message). Merging Drop All Flowfiles responses across all nodes in a cluster.

NIFI-7663 Reloading the canvas after completing a Drop All Flowfiles request.

NIFI-7663 Fixed typos.

This closes #4425.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-08-07 16:23:00 +02:00
Matthew Burgess 6488db1376
NIFI-7572: Force ScriptedTransformRecord to recompile the script when started
This closes #4458.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-08-06 17:40:12 -07:00
Peter Gyori 5cc7c0485a NIFI-7708: Add Azure SDK for Java (and transitive dependencies) to LICENCE and NOTICE files
This closes #4452

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2020-08-06 10:55:31 -07:00
Mark Payne eca7f153d0
NIFI-7706, NIFI-5702: Allow NiFi to keep FlowFiles if their queue is unknown. This way, if a Flow is inadvertently removed, updated, etc., and NiFi is restarted, the data will not be dropped by default. The old mechanism of dropping data is exposed via a property
This closes #4454.

Signed-off-by: Bryan Bende <bbende@apache.org>
2020-08-06 10:55:56 -04:00
Scott Aslan 6596fb1f87
[NIFI-7705] update fronted deps
make variable and parameters tables position relative

Signed-off-by: Rob Fellows <rob.fellows@gmail.com>

This closes #4449
2020-08-06 10:48:24 -04:00
Bryan Bende 93e5abf274
NIFI-7522 Updating LICENSE and NOTICE for Kafka 2.5 NAR
This closes #4383.
2020-08-04 16:59:54 -04:00
Mathieu 1df7025ae8
NIFI-7522 : Add Kafka 2.5 Processors set 2020-08-04 16:59:39 -04:00
Joe Witt 536dbb72bb
NIFI-7703 updated all commons codec references to 1.14
This closes #4448.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-08-04 12:11:57 -07:00
shreeju 24d4ad348a
NIFI-7702 This closes #4447. validate requestentity not null
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-08-04 09:48:54 -07:00
Mike Thomsen 0861b2f632
NIFI-7605 Removed user-agent default value so no header will be sent by default.
Added and updated unit tests.

This closes #4428.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
Signed-off-by: Joe Witt <joe.witt@gmail.com>
2020-08-03 17:23:56 -07:00
Peter Turcsanyi c980b64bf5
NIFI-7340: Adding ListAzureDataLakeStorage
Also added validator for Directory Name property in AbstractAzureDataLakeStorageProcessor
Fix Tracking Entities strategy: use milliseconds for lastModified

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4438.
2020-08-03 19:27:21 +02:00
Peter Turcsanyi 7b4cce9e21
NIFI-6312: Improved connection handling in AMQP processors
Disable connection automatic recovery which can lead to uncontrolled/stale threads. Handle the recovery in the processors instead.
Use poisoning in case of errors, then discarding and recreating the poisoned consumer/publisher.
NIFI-6312: Use conventional exception handling instead of poisoning
Use component logger in workers.
Remove basicNack()/basicReject() calls as they are not needed because all unacknowledged messages will be redelivered.
NIFI-6312: Further improve exception handling and error logging.
NIFI-6312: Fix consumer closing in previous commit
NIFI-6312: Use custom executor with a single thread (no more is used by the processor)

Reviewed by tamas palfy and simon bence

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-08-03 09:43:25 -07:00
Kent Nguyen ddb304a927 NIFI-7664 Add Content Disposition property to PutS3Object processor
Add new property 'Content Disposition' to allow user
to set the content-disposition http header on the S3 object.

Allowed values are 'inline' (default) and 'attachment'.
If 'attachment' is selected, the filename will be set to the S3 Object key.

Remove default value and keep backward compatibility
Update fetchS3Object filename attribute settin
Update constant names
Update order of if-else condition
NIFI-7664 Update condition in FetchS3Processor
NIFI-7664 Undo the unexpected indent
NIFI-7664 Update international chars unit test
NIFI-7664 Set fetchS3 file path name
NIFI-7664 Update code style

This closes #4423.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-08-03 18:27:35 +02:00
Karthikeyan Singaravelan 6a497aa8c3 NIFI-7691: Validate request object to be not null. 2020-07-31 15:31:32 -04:00
Matthew Burgess 1b91a6caf3 NIFI-7592: Allow NiFi to be started without a GUI/REST interface
- Rename MiNiFi references to Headless NiFi, added profile to build headless assembly

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-07-31 14:09:12 -04:00
Jan Hentschel 84586a6772 NIFI-7315 Fixed typo in bootstrap.conf for 17th argument 2020-07-31 11:17:17 -04:00
Mike Thomsen 83fd4ca291
NIFI-7526 Updated code to use the new helper library for OkHttp.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4331.
2020-07-31 10:51:15 +02:00
Joey Frazee 633fd8883d
NIFI-7693: Bump aspectjweaver version to resolve BeanCreationException on Java 11 2018-09-25
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4444.
2020-07-31 10:44:47 +02:00
Kent Nguyen 0cff54097e NIFI-6332: Add Cache Control property to PutS3Object processor
Add new property 'Cache Control' to allow user to
set the cache-control http header on the S3 object.

This property is not required, and has no default value.

The implementation is similar to the Content-Type property,
except that this property does not allow Expression Language.

Update property description

Add support EL for cache-control property

This closes #4422.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-07-30 08:37:10 +02:00
Tamás Bunth 455f48fce4
NIFI-7640 Add documentation: temporary directory (#4414)
NiFi uses the Java IO temporary directory for storing HTTP multipart
files when using HandleHttpRequest processor. The directory can be
overwritten with Java command line parameter.
2020-07-29 13:57:34 -07:00
Andy LoPresto 716ba992f5
NIFI-7669 Changed custom PBE AEAD algorithm to derive key once rather than on every encrypt/decrypt operation, leading to substantial performance gains.
Updated documentation.
Added unit tests.

NIFI-7669 Moved time-based encryption tests to integration tests to avoid running during CI builds.

NIFI-7669 Fixed failing test due to nifi.properties initialization.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4435.
2020-07-29 09:29:00 +02:00