5963 Commits

Author SHA1 Message Date
ben
5470de949e
NIFI-7715 This closes #4456. Added IntelliJ generated .ipr to .gitignore
- IntelliJ generates a `.ipr` Project File when exec the `mvn` goals
`mvn idea:idea` and `mvn idea:project`
  - [Maven `idea` Doc](https://maven.apache.org/plugins/maven-idea-plugin/usage.html)

Signed-off-by: ben <git@benjamin.ie>
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-08-10 10:22:34 -07:00
bibistroc
51091a5fca
NIFI-7338 This closes #4169. Added comment about com.hierynomus.smbj.SMBClient syncronicity
- added file.size attribute on GetSmbFile

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-08-10 09:44:52 -07:00
si
71d155e874
NIFI-7338 fixed provenance, top level dir,
chatty smbj log, pw validation
update accessmask to be more restrictive

Author:    si <si.sun@roche.com>

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-08-10 09:43:43 -07:00
Joe Witt
58e324e9f1
NIFI-7522 bumping from Apache Kafka 2.5 to Apache Kafka 2.6
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4462.
2020-08-10 09:55:53 +02:00
Mark Payne
0f18c5966e NIFI-7592: Updated nifi system test to no longer include nifi-server-nar and also changed a unit test to an integration test because it starts an embedded server on a specific port
This closes #4459.
2020-08-07 12:51:17 -04:00
Matthew Burgess
55adb11f06 Revert "NIFI-7592: Allow NiFi to be started without a GUI/REST interface"
This reverts commit 1b91a6caf32f60e08c2b4f0a7b2d95d9fb8ed22d.
2020-08-07 12:50:48 -04:00
Matthew Burgess
b8a1a4c578 Revert "NIFI-7592: Fixed typo in comment in pom.xml and changed headless profile to not active by default."
This reverts commit 2de4962ff1a46a0f1b9c9029ccce5c8206a04eb5.
2020-08-07 12:50:48 -04:00
Pierre Villard
27e7d0b02a
NIFI-7707 - This closes #4451. fixed Publish GCP PubSub processor for binary data
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-08-07 08:14:19 -07:00
Tamas Palfy
3ef566f7da NIFI-7663 Added option for emptying all queues in a process group. Available from context menu.
NIFI-7663 Minor changes (variable name refactor, javadoc, GUI message). Merging Drop All Flowfiles responses across all nodes in a cluster.

NIFI-7663 Reloading the canvas after completing a Drop All Flowfiles request.

NIFI-7663 Fixed typos.

This closes #4425.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-08-07 16:23:00 +02:00
Jaya Aditya
339e09a6e0
NIFI-7681 - Add update-bucket-policy command, add option to specify timeout and fix documentation to include previously implemented commands (#4450)
* NIFI-7681 - Add update-bucket-policy command, add option to specify
timeout and fix documentation to include previously implemented commands

* Fix return type of UpdateBucketPolicy and add missing registry commands to the documentation

Co-authored-by: Jaya Aditya <jchandra@yahoo-corp.jp>
2020-08-07 09:06:56 -04:00
Matthew Burgess
6488db1376
NIFI-7572: Force ScriptedTransformRecord to recompile the script when started
This closes #4458.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-08-06 17:40:12 -07:00
Peter Gyori
5cc7c0485a NIFI-7708: Add Azure SDK for Java (and transitive dependencies) to LICENCE and NOTICE files
This closes #4452

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2020-08-06 10:55:31 -07:00
Mark Payne
eca7f153d0
NIFI-7706, NIFI-5702: Allow NiFi to keep FlowFiles if their queue is unknown. This way, if a Flow is inadvertently removed, updated, etc., and NiFi is restarted, the data will not be dropped by default. The old mechanism of dropping data is exposed via a property
This closes #4454.

Signed-off-by: Bryan Bende <bbende@apache.org>
2020-08-06 10:55:56 -04:00
Scott Aslan
6596fb1f87
[NIFI-7705] update fronted deps
make variable and parameters tables position relative

Signed-off-by: Rob Fellows <rob.fellows@gmail.com>

This closes #4449
2020-08-06 10:48:24 -04:00
Bryan Bende
93e5abf274
NIFI-7522 Updating LICENSE and NOTICE for Kafka 2.5 NAR
This closes #4383.
2020-08-04 16:59:54 -04:00
Mathieu
1df7025ae8
NIFI-7522 : Add Kafka 2.5 Processors set 2020-08-04 16:59:39 -04:00
Joe Witt
536dbb72bb
NIFI-7703 updated all commons codec references to 1.14
This closes #4448.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-08-04 12:11:57 -07:00
shreeju
24d4ad348a
NIFI-7702 This closes #4447. validate requestentity not null
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-08-04 09:48:54 -07:00
Mike Thomsen
0861b2f632
NIFI-7605 Removed user-agent default value so no header will be sent by default.
Added and updated unit tests.

This closes #4428.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
Signed-off-by: Joe Witt <joe.witt@gmail.com>
2020-08-03 17:23:56 -07:00
Peter Turcsanyi
c980b64bf5
NIFI-7340: Adding ListAzureDataLakeStorage
Also added validator for Directory Name property in AbstractAzureDataLakeStorageProcessor
Fix Tracking Entities strategy: use milliseconds for lastModified

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4438.
2020-08-03 19:27:21 +02:00
Peter Turcsanyi
7b4cce9e21
NIFI-6312: Improved connection handling in AMQP processors
Disable connection automatic recovery which can lead to uncontrolled/stale threads. Handle the recovery in the processors instead.
Use poisoning in case of errors, then discarding and recreating the poisoned consumer/publisher.
NIFI-6312: Use conventional exception handling instead of poisoning
Use component logger in workers.
Remove basicNack()/basicReject() calls as they are not needed because all unacknowledged messages will be redelivered.
NIFI-6312: Further improve exception handling and error logging.
NIFI-6312: Fix consumer closing in previous commit
NIFI-6312: Use custom executor with a single thread (no more is used by the processor)

Reviewed by tamas palfy and simon bence

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-08-03 09:43:25 -07:00
Kent Nguyen
ddb304a927 NIFI-7664 Add Content Disposition property to PutS3Object processor
Add new property 'Content Disposition' to allow user
to set the content-disposition http header on the S3 object.

Allowed values are 'inline' (default) and 'attachment'.
If 'attachment' is selected, the filename will be set to the S3 Object key.

Remove default value and keep backward compatibility
Update fetchS3Object filename attribute settin
Update constant names
Update order of if-else condition
NIFI-7664 Update condition in FetchS3Processor
NIFI-7664 Undo the unexpected indent
NIFI-7664 Update international chars unit test
NIFI-7664 Set fetchS3 file path name
NIFI-7664 Update code style

This closes #4423.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-08-03 18:27:35 +02:00
Mike Thomsen
255d0c0f19
NIFI-7604 Moved nifi-elasticsearch-5 NAR to a profile to save space.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4389
2020-07-31 16:12:20 -04:00
Karthikeyan Singaravelan
6a497aa8c3 NIFI-7691: Validate request object to be not null. 2020-07-31 15:31:32 -04:00
Mark Payne
2de4962ff1 NIFI-7592: Fixed typo in comment in pom.xml and changed headless profile to not active by default.
This closes #4376.
2020-07-31 14:11:48 -04:00
Matthew Burgess
1b91a6caf3 NIFI-7592: Allow NiFi to be started without a GUI/REST interface
- Rename MiNiFi references to Headless NiFi, added profile to build headless assembly

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-07-31 14:09:12 -04:00
Jan Hentschel
84586a6772 NIFI-7315 Fixed typo in bootstrap.conf for 17th argument 2020-07-31 11:17:17 -04:00
Ruben Laguna
30b328ceab
NIFI-7694 - Move misplaced text related to oidc
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4445.
2020-07-31 11:00:14 +02:00
Mike Thomsen
83fd4ca291
NIFI-7526 Updated code to use the new helper library for OkHttp.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4331.
2020-07-31 10:51:15 +02:00
Joey Frazee
633fd8883d
NIFI-7693: Bump aspectjweaver version to resolve BeanCreationException on Java 11 2018-09-25
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4444.
2020-07-31 10:44:47 +02:00
Andy LoPresto
ac60bf6507
NIFI-7687 Improved JVM version handling in RunNiFi and OSUtils.
Added unit tests.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4442.
2020-07-30 09:47:14 +02:00
Kent Nguyen
0cff54097e NIFI-6332: Add Cache Control property to PutS3Object processor
Add new property 'Cache Control' to allow user to
set the cache-control http header on the S3 object.

This property is not required, and has no default value.

The implementation is similar to the Content-Type property,
except that this property does not allow Expression Language.

Update property description

Add support EL for cache-control property

This closes #4422.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-07-30 08:37:10 +02:00
Tamás Bunth
455f48fce4
NIFI-7640 Add documentation: temporary directory (#4414)
NiFi uses the Java IO temporary directory for storing HTTP multipart
files when using HandleHttpRequest processor. The directory can be
overwritten with Java command line parameter.
2020-07-29 13:57:34 -07:00
Andy LoPresto
716ba992f5
NIFI-7669 Changed custom PBE AEAD algorithm to derive key once rather than on every encrypt/decrypt operation, leading to substantial performance gains.
Updated documentation.
Added unit tests.

NIFI-7669 Moved time-based encryption tests to integration tests to avoid running during CI builds.

NIFI-7669 Fixed failing test due to nifi.properties initialization.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4435.
2020-07-29 09:29:00 +02:00
Andy LoPresto
7f0416ee8b
NIFI-7680 Added convenience methods for creating XML DocumentBuilder instances.
Added unit tests.

NIFI-7680 Duplicated DocumentBuilder creation method in NotificationServiceManager to avoid nifi-bootstrap dependency on nifi-security-utils.
Explicitly added commons-lang3 to lib/bootstrap/ directory in nifi-assembly.

NIFI-7680 Reverted unnecessary dependency changes.
Added explicit dependencies where necessary.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4436
2020-07-28 17:02:42 -04:00
Karthikeyan Singaravelan
cee6cfddf2
NIFI-7682: Check for request object to be null.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4437.
2020-07-28 21:17:46 +02:00
Mark Payne
e926a17968
NIFI-7683: Fixed bug that resulted in a byte[] being allocated and held onto by a member variable that was unprotected. This caused the byte array to be modified by two different threads concurrently, which can potentially cause corruption of FlowFile's data
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4439
2020-07-28 14:52:57 -04:00
Andy LoPresto
90c9db8208
NIFI-7679 Changed flow fingerprint masking logic to derive a static key once (slowly) from nifi.sensitive.props.key and use a (fast) HMAC/SHA-256 operation during fingerprinting to resolve unacceptable delays.
Added unit tests.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4434.
2020-07-28 10:22:24 +02:00
Mark Payne
0b9f2fbe3b
NIFI-7678: Fixed memory leak, improved some logic that constantly created collections unnecessarily. Changed types of Collections being used to much more efficient implementations. Removed using Streams for performance-critical parts of application
NIFI-7678: Check if debug is enabled before logging message about processor.onTrigger because obtaining class name is expensive

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4431.
2020-07-27 13:28:15 +02:00
Peter Turcsanyi
cdd766d649
NIFI-7671: Support Message Selector in ConsumeJMS processor
Also fixed some display names and variable names.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4432.
2020-07-27 12:50:44 +02:00
Andy LoPresto
7d20c03f89
NIFI-7638 Implemented custom nifi.sensitive.props.algorithm for AES-G/CM with Argon2 KDF.
Added documentation for encryption of flow sensitive values.
Added unit tests.

This closes #4427.
2020-07-24 18:11:37 -07:00
Bryan Bende
5cb8d24689
NIFI-7622 Use param context name from inside nested versioned PG when importing
This closes #4401.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-07-24 14:15:57 -07:00
Nathan Gough
d846a74730
NIFI-7568 - Applied Kerberos mappings to authentication requests. Kerberos mappings should now be applied correctly in H2 database for username/password based login. Added tests.
Logout now deletes signing key by key ID rather than identity.
Validate token expiration now uses mapped identity instead, which allows logging of the mapped identity.
Updated delete key to expect only 0 or 1 keys deleted.

This closes #4416.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-07-24 13:28:01 -07:00
Andy LoPresto
eeeda84474
NIFI-7122 Introduced improvements to EncryptContent processor.
Added unit tests.
Refactored shared logic from various algorithm-specific secure hasher implementations to AbstractSecureHasher.
Introduced secure hasher implementations for various KDFs.
Added custom validation to EncryptContent processor.
Implemented logic for EncryptContent to write operational metadata to flowfile attributes.
Added encryption metadata attribute annotations to EncryptContent.
Added Argon2 KDF documentation and Bcrypt key derivation change notes to Admin Guide.
Updated unit tests to calculate default/recommended cost parameters for Argon2.

This closes #4421.

Co-authored-by: mtien <mtien.apache@gmail.com>
2020-07-24 12:31:39 -07:00
Pierre Villard
4b0d349a49
NIFI-7667 - add variable support to GetHBase properties (#4424) 2020-07-23 09:29:44 -04:00
Bryan Bende
e9a443fe2e
NIFI-7521 Remove additional unused property from nifi proeprties
This closes #4327.
2020-07-23 08:14:39 -04:00
Mark Payne
e371f4ac7c
NIFI-7521: Removed unused properties from nifi properties. Updated the defaults to use for File System's archive cleanup, flowfile checkpoint interval 2020-07-23 08:14:21 -04:00
Bryan Bende
99b3780ad9
NIFI-7373 Setting TTL on keys in Redis putAll
This closes #4217.
2020-07-23 07:39:15 -04:00
Mike Thomsen
0e61dbc9a0
NIFI-7373
Added new methods to DistributedMapCacheClient for bulk get and put. Updated HBase 1.1.2 clients.
Added HBase 2 support.
Added Redis support.
2020-07-23 07:38:59 -04:00
Pierre Villard
7f0b188be4 NIFI-7531 - changed RW to RO access requirement on autoload NARs directory 2020-07-22 14:31:48 -04:00