Commit Graph

6471 Commits

Author SHA1 Message Date
Joe Gresock ed591e0f22
NIFI-8445: Implemented HashiCorpVaultCommunicationService in nifi-vault-utils
This closes #5034

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-05-14 12:27:01 -05:00
Bence Simon 51aae5bcf6 NIFI-8519 Adding HDFS support for NAR autoload
- Refining classloader management with the help of @markap14

This closes #5059

Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-05-14 13:14:06 -04:00
Mark Payne 7c08fbc4d4
NIFI-8542: When returning content via TriggerResult.readContent(FlowFile), ensure that we take into account the content claim offset and length
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5076
2021-05-14 12:57:01 -04:00
Nathan Gough 7e54ef9421
NIFI-8343 - Updated solr from 8.4.1 to 8.8.2. Small code changes were required.
This closes #5074

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-05-14 11:38:07 -05:00
exceptionfactory 6776765a92
NIFI-8538 Upgraded Apache Commons IO to 2.8.0
- Upgraded direct dependencies from 2.6 to 2.8.0
- Added dependency management configuration to use 2.8.0 for some modules
- Updated scripted Groovy tests to avoid copying unnecessary files

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5073
2021-05-14 09:31:48 -04:00
Mohammed Nadeem 73a2283c27
NIFI-8536: PrometheusReportingTask and PrometheusRecordSink do not support variable registry for port property
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5075
2021-05-13 14:05:30 -04:00
Mark Payne 97ed0efbc2
NIFI-8537: Check that a property's dependencies are satisfied before checking if the referenced controller service valid/enabled
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5071
2021-05-13 11:37:57 -04:00
Chris Sampson a274c12bbb NIFI-2892 Implement AWS Kinesis Stream Consume Processor
This closes #4822.

Co-authored-by: uday <udaygkale@gmail.com>

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-05-12 20:03:28 +02:00
Mark Payne ab8b7444b5
NIFI-8457: Fixed bug in load balanced connections that can result in the node never completing OFFLOAD action. Also fixed issue in which data destined for a disconnected/offloaded node was never rebalanced even for partitioning strategies that call for rebalancing on failure
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5019.
2021-05-12 13:05:48 +02:00
exceptionfactory 09e54c1dad
NIFI-8246 Set NIFI_PBKDF2_AES_GCM_256 as default properties encryption algorithm
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5055.
2021-05-12 12:58:50 +02:00
exceptionfactory 4db50f2f40
NIFI-8502 Upgraded Spring Framework to 5.3.6
- Upgraded Spring Framework references from version 4.3.30 to 5.3.6
- Upgraded Spring Security from version 4.2.20 to 5.4.6
- Upgraded Spring Data Redis from 2.1.16 to 2.5.0
- Upgraded Jedis from 2.9.0 to 3.6.0 to match Spring Data Redis 2.5.0
- Upgraded Easy Rules from 3.4.0 to 4.1.0 to support Spring 5
- Upgraded Hortonworks Schema Registry Client from 0.8.1 to 0.9.1 to support Spring 5
- Refactored ThreadPoolRequestReplicatorFactoryBean to implement DisposableBean to handle executor shutdown

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5066.
2021-05-12 12:49:01 +02:00
Sandro Tolaini fdda3582e3
NIFI-8470: Fix DataTypeUtils for CHAR() sequences
NIFI-8470: Added unit test

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5027
2021-05-11 17:46:38 -04:00
Matthew Burgess a4d435c995 NIFI-8330: Fix compilation of Jython scripts for scripting components
- Add onPropertyModified to ExecuteScript to recompile on change

This closes #4903.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-05-11 16:51:23 -04:00
exceptionfactory 340b13033f NIFI-8367 Removed joda-time from nifi-standard-processors 2021-05-11 14:50:49 -04:00
matthewlannan 86d5b08d23 Adding addititonal documentation to EvalluateJsonPath per https://issues.apache.org/jira/browse/NIFI-8518 2021-05-11 14:40:12 -04:00
Sushil Kumar 48a9054731 NIFI-4913 - Added "-E" with sudo to read environment variables set in current shell environment while running nifi with bootstrap property "run.as" set as to some user
NIFI-4913 - Updated administration-guide.adoc

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #3750.
2021-05-11 11:48:41 -04:00
s9514171 1791f4cc35
NIFI-8417 emoji in processor group name cause nifi to not start up
- Excluded Xalan dependency from spring-security-saml2-core

This closes #5012

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-05-11 08:27:49 -05:00
Mark Payne 1645886e5a NIFI-8477: If interrupted while waiting for Node Status Update to be replicated to other nodes, do not throw ProtocolException; instead just log a warning and return.
This closes #5039
2021-05-10 16:15:00 -04:00
Mark Bean 5c2df42be3
NIFI-8431: remove redundant validation of dynamic properties
removed unused variable

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5045
2021-05-10 13:54:18 -04:00
exceptionfactory 5062f2e54e
NIFI-8515 Upgraded Apache Tika to 1.26
- Upgraded tika-core from 1.24 to 1.26 in nifi-framework-bundle and nifi-standard-bundle
- Upgraded tika-parsers from 1.24.1 to 1.26 in nifi-media-processors

NIFI-8515 Removed jsr-275 exclusion and confirmed it is no longer included

- TIKA-2535 Replaced the jsr-275 dependency to resolve licensing issues

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5056
2021-05-10 11:13:31 -04:00
Matthew Burgess 284322feed
MINIFI-554: Move OSUtils to nifi-bootstrap-utils for MiNiFi and NiFi
This closes #5062

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-05-10 08:39:10 -05:00
Mark Bean 1e75b7ef06
NIFI-8496: eliminate 500 error when uploading templates
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5053
2021-05-07 11:26:21 -04:00
Mark Payne 2895bac2c0
NIFI-8512: When converting to/from Avro UNION type, we can be more efficient when the UNION consists of a Null type and one other type by determinine the non-null type and just using that. Also eliminated a call to List.stream() and related .collect() call by using an existing method that performs the logic without the very expensive call to stream()
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5051
2021-05-06 16:06:23 -04:00
Matthew Burgess 30fc26647e
NIFI-6061: Fix CLOB/BLOB handling in PutDatabaseRecord
NIFI-6061: Force getBytes() in BLOB handling to use UTF-8 charset

NIFI-6061: Use setClob(), added unit tests, incorporated review comments

This closes #5049

Co-authored-by: zhangcheng <zhangchengk@foxmail.com>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-05-04 14:58:14 -05:00
exceptionfactory 328b9e540b
NIFI-8513 Upgraded Spring Framework to 4.3.30
- Upgraded direct Spring Framework references to 4.3.30.RELEASE
- Upgraded direct Spring Security references to 4.2.20.RELEASE

NIFI-8513 Updated Spring Framework and Security Notices with major version

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5054.
2021-05-04 18:45:26 +02:00
Tamas Palfy 5bcfcf42bb NIFI-7788 Created a new endpoint in RemoteProcessGroupResource to allow updating run statuses/transmission state of all remote process groups within a process group.
When selecting run/stop on a process group/canvas/selection, it will try to enable/disable transmission of all involved remote process groups.

NIFI-7788 Supplied same functionality missed when selecting a process group.
NIFI-7788 Updated endpoint URL paths.
NIFI-7788 No need to return list of remote process groups when updating en masse.
NIFI-7788 Added some null checks in RemoteProcessGroupsEndpointMerger.merge.
NIFI-7788 Fix checkstyle violation.

This closes #4516.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-05-04 16:49:21 +02:00
exceptionfactory 13d5be622b NIFI-8230 Removed default Sensitive Properties Key and added random generation
- Retained legacy default Sensitive Properties Key in ConfigEncryptionTool to support migration
- Streamlined default file path and moved key generation conditional
- Refactored with getDefaultProperties()
- Cleared System Property in ConfigEncryptionToolTest
- Added checking and error handling for clustered status
- Added set-sensitive-properties-key command
- Refactored PropertyEncryptor classes to nifi-property-encryptor
- Added nifi-flow-encryptor
- Refactored ConfigEncryptionTool to use FlowEncryptor for supporting AEAD algorithms
- Added Admin Guide section Updating the Sensitive Properties Key

This closes #4857.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-05-04 08:49:34 -04:00
Paul Grey a74c17616d
NIFI-8478 - This closes #5052. handle undefined system property 'user.region'
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-05-03 16:55:37 -07:00
Guillaume Schaer 24d3d4325c NIFI-8506 Fix Azure Storage integration tests
This closes #5047

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-05-03 13:19:31 -07:00
Paul Grey cc554a6b11
NIFI-8478 Addressed i18n test issues; CI supplies locale to surefire
This closes #5040

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-05-03 13:40:01 -05:00
Nathan Gough e1697a8d58
NIFI-8510 - Added a more specific CSRF cookie request matcher to avoid issues with CSRF 403 response.
This closes #5050

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-05-03 13:30:08 -05:00
Mark Payne 935566ba23 NIFI-8433 Added ability to decommission a node in a cluster
This closes #5004

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-05-01 12:45:48 -07:00
Nathan Gough 1090a9748a
NIFI-7870 Resolved access to extension resources when using JWT
- Added SameSite Session Cookie __Host-Authorization-Bearer for sending JWT
- Configured Spring Security CSRF Filter comparing Authorization header and Cookie JWT
- Implemented BearerTokenResolver for retrieving JWT

This closes #4988

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-04-30 22:29:55 -05:00
Matthew Burgess 3963f66dff
NIFI-8320: Fix column mismatch in PutDatabaseRecord
This closes #5024

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-04-30 10:41:36 -05:00
Matthew Burgess e1c99e3a5c
MINIFI-553: Fix timeout-based tests in StatusLoggerTest
This closes #5041

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-04-29 10:45:43 -05:00
Joe Gresock 54a0e27c93
NIFI-7134: Adding auto-reloading of Keystore and Truststore
- NIFI-7261 Included TrustStoreScanner for auto-reloading of truststore

This closes #4991

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-04-29 08:29:42 -05:00
herbert 93dcf25d33
MINIFI-552: Fix new SSL SAN Behaviour from new Jetty Version
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5036
2021-04-28 11:47:04 -04:00
Matthew Burgess a0655bff1a
MINIFI-422: Incorporate MiNiFi Java into NiFi
- Initial copy of MiNiFi Java into NiFi
- Checkpoint for basic integration, load empty flow without startup errors
- Refactor provenance repositories, replace some MiNiFi NARs with NiFi NARs
- Remove MiNiFi LICENSE, NOTICE, and KEYS, change file permissions on build.properties
- Updated MiNiFi LICENSE and NOTICE files
- Fix headless NiFi config, set krb5 file in MiNiFi for Kerberos support
- Removed commented-out lines from POMs, added nifi-kerberos-credentials-service NAR to MiNiFi assembly
- Update Git repo URLs, replace NiFi version property

This closes #4933.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2021-04-27 21:06:56 -04:00
exceptionfactory 3fb76400db
NIFI-8485 Upgraded Jetty to 9.4.40
This closes #5035

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-04-27 17:49:23 -05:00
Paul Kelly 0ed3534524 NIFI-7886 Added read by range to Azure, GCS, S3 fetch processors
This closes #4576

Co-authored-by: Joey Frazee <jfrazee@apache.org>
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-04-27 12:08:06 -07:00
Tamas Palfy 16dc61e151 NIFI-8458 TailFile - Fix some bugs. Add more tests.
- During cleanup keep "tailingPostRollover" in the updated state.
- Skipping tests that can't run on Windows.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-04-27 09:44:39 -04:00
Joe Gresock 90c7d03ed3
NIFI-8403: Generating Self-signed cert on startup when applicable (#4986)
* NIFI-8403: Implementing auto-generated certificates for secure startup

* Adding check for passwords in SecureNiFiConfigUtil
2021-04-27 09:34:44 -04:00
Peter Turcsanyi 525bfe00b3
NIFI-8474: Added new Replacement Strategy for variable substitution in ReplaceText.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5032.
2021-04-27 13:03:21 +02:00
exceptionfactory f9d3bb7f69
NIFI-8473 Added Parameter Contexts section to REST API documentation
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5030.
2021-04-27 12:23:33 +02:00
Paul Grey 835f50c83a
NIFI-8465 Handle bcrypt legacy decrypt failures in testing
This closes #5029

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-04-26 11:34:20 -05:00
Denes Arvay 9f45b48866 NIFI-8437 RecordReader 'Infer Schema' for large records may throw BufferedInputStream error
This closes #5011.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-04-23 13:46:41 -04:00
exceptionfactory 92bdc23adb NIFI-8251 Added EncryptContentPGP and DecryptContentPGP Processors
- Added PGPPrivateKeyService and PGPPublicKeyService interfaces with standard implementations
- NIFI-7396 EncryptContentPGP writes encryption metadata attributes
- NIFI-6708 Controller Services support ElGamal Public and Private Keys
- NIFI-5346 Controller Services support Keyring Files and ASCII Key properties
- NIFI-5335 Controller Services support multiple public or private keys from keyrings
- NIFI-2983 DecryptContentPGP finds and decrypts Encrypted Data Packets regardless of signing
- NIFI-1694 Controller Services support individual key files or keyrings

NIFI-8251 Refactored Public Key ID Property to Public Key Search

NIFI-8251 Corrected handling of multiple Encrypted Data packets in DecryptContentPGP

- Added unit tests for encryption and decryption with both password-based and public key
- Added PGP NAR dependencies to nifi-assembly

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4842.
2021-04-22 15:48:49 -04:00
Paul Grey a1597ced89
NIFI-8460 Remove unused BouncyCastle reference in test class
This closes #5022

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-04-22 12:01:44 -05:00
Paul Grey 17fa0cf3c1
NIFI-3580 Add configurable TLS Cipher Suite properties
This closes #5018

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-04-22 11:31:19 -05:00
Paul Kelly aedacdf86f
NIFI-8454: Allow InvokeHTTP to output final URL from response request property
This closes #5016

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-04-21 18:18:25 -05:00