Apache/nifi
1
0
Fork 0
mirror of https://github.com/apache/nifi.git synced 2025-02-08 02:58:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
6,575 Commits 147 Branches 203 Tags
Commit Graph

4476 Commits

Author SHA1 Message Date
Pierre Villard
1313ee3d90
NIFI-8735 - Increase polling frequency for NAR autoloading providers (#5183) 2021-06-25 17:37:51 +02:00
Lars Francke
49d10dbcbd
NIFI-5573 Allow overriding of nifi-env.sh 
This closes #2985.

Signed-off-by: Kevin Doran <kdoran@apache.org>
 2021-06-25 10:46:23 -04:00
Peter Turcsanyi
39ffa4a8ac
NIFI-8737: Fixed incorrect provenance data in HDFS processors when Directory property is inconsistent with core-site.xml 2021-06-25 14:19:57 +02:00
Joe Gresock
726082ffa6
NIFI-8447 Added HashiCorp Vault Transit Sensitive Properties Provider 
- Added default bootstrap-hashicorp-vault.conf
- Updated Toolkit Guide documentation with HashiCorp Vault properties

This closes #5154

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2021-06-24 22:06:42 -05:00
Matthew Burgess
5e4f32663e Revert "NIFI-8195: add default connection settings to process group configuration" 
This reverts commit 5ebbe0028b9ff4d23f86d87e4942665db62e9bc0.
 2021-06-24 18:52:17 -04:00
Attila Bukor
27c35c8c42
NIFI-8463: Support custom SASL protocol name in PutKudu (#5023) 
As of KUDU-1884, Kudu supports custom Kerberos principals on server-side
and custom SASL protocol (service) names on client-side which must match
the SPN base, i.e. if the SPN is kudu/_HOST, SASL protocol name *must*
be "kudu" in the client to be able to connect to the cluster.

This patch adds the ability to configure this in the PutKudu processor.

Signed-off-by: Attila Bukor <abukor@apache.org>
 2021-06-24 17:42:04 -04:00
Mark Bean
5ebbe0028b
NIFI-8195: add default connection settings to process group configuration 
NIFIDEVS-8195: fixed properties not properly inheriting from template/snippet values

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5094
 2021-06-24 14:03:51 -04:00
Bence Simon
a3f54fa578
NIFI-8736 Adding capability to override HDFS location for NAR autoloading 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5185.
 2021-06-24 19:37:19 +02:00
Jon Kessler
115bba9ab0
NIFI-8523 Added SFTP algorithm and cipher properties 
- Updated secure FTP processors to configure which algorithms, ciphers and message authentication codes are allowed to be used by the SSH Client
- Included Expression Language Variable Registry support for properties

This closes #5061

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2021-06-24 11:51:04 -05:00
markap14
ea710e44a3
NIFI-8722: Updated Quest DB to account for newly added connection status metrics (#5181) 
Co-authored-by: Bence Simon <simonbence>
 2021-06-23 16:50:10 -04:00
Lehel Boér
0c748a5a2b NIFI-4344: Improve bulletin messages with exception details. 
This closes #5093.

Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
 2021-06-23 17:34:01 +02:00
exceptionfactory
6a83115d6a NIFI-7468 Updated SSLSocketChannel to support TLS 1.3 
- Handling additional FINISHED Handshake Status for TLS 1.3 Post-Handshake Messages per RFC 8446 Section 4.6
- Removed clearing buffers after handshake to avoid losing packets
- Updated read() method to check Handshake Status after SSLEngine.unwrap()
- Changed SSLSocketChannelSender to close SSLSocketChannel before other resources
- Added ChannelStatus enum and convenience logging methods for tracing status
- Added unit tests for TLS 1.2 and 1.3 using Netty server and client handlers

NIFI-8704 Updated netty-handler to 4.1.65.Final

NIFI-7468 Corrected SSLSocketChannel.read() to return byte read

NIFI-7468 Adjusted comment formatting

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5152.
 2021-06-22 22:29:16 -04:00
simonbence
73363d9aea
NIFI-8717 Refactoring PutHDFS (#5175) 
NIFI-8717 Refactoring PutHDFS
 2021-06-22 13:13:14 -04:00
exceptionfactory
f2698a4e1e
NIFI-8723 Upgraded Jackson to 2.12.3 
- Updated MiNiFi to reference root jackson.version
- Updated Registry assembly to reference root jackson.version
- Added dependency management in NAR modules to override transitive versions

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5174
 2021-06-22 11:21:45 -04:00
Nathan Gough
9744644b9d
NIFI-8025 - Refactored SAML and OIDC Resources to separate classes 
This closes #5079

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2021-06-21 16:37:06 -05:00
Chris Sampson
02b4e33aa6
NIFI-8683 support Expression Language for the Truststore/Keystore properties of SSLContextService 
This closes #5147

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2021-06-21 14:39:35 -05:00
Mark Payne
6fc9f33ba2
NIFI-8720: Ensure that when we use Expression Language for Records Per Split property of SplitRecord, we treat the result as a value of 1 if it's less than 1. 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5171
 2021-06-21 13:09:20 -04:00
Mark Payne
b1ad13c835
NIFI-8719: Ensure that when OutputStream is returned from StandardProcessSession.write, upon close of the OutputStream we close the underlying resources. 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5170
 2021-06-21 12:55:17 -04:00
exceptionfactory
2efc2b6b8b
NIFI-8718 Upgraded Apache Commons IO to 2.10.0 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5169
 2021-06-21 11:29:27 -04:00
rhavermans
9fd8042cfb
NIFI-8664 Update json-path lib from 2.4.0 -> 2.6.0 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5132
 2021-06-21 11:01:10 -04:00
Peter Turcsanyi
820b2cff29 NIFI-8662: Using the configured AWS region when parsing from VPCE endpoint URL fails in AbstractAWSProcessor 
This closes #5140.

Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
 2021-06-16 21:50:25 +02:00
Bernhard Geisberger
90f44d9d62
NiFi-8152: Fix NPE in RedisDistributedMapCacheClientService caused by newer versions of Jedis 
This closes #4783

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
 2021-06-16 14:07:16 -04:00
exceptionfactory
9be9b9dbe2
NIFI-8708 Upgraded Spring Framework to 5.3.8 for several extensions 
- nifi-vault-utils
- nifi-email-bundle
- nifi-redis-bundle

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5163.
 2021-06-16 18:18:54 +02:00
Abhishek Kumar
520ee82745
NIFI-8709 - added XSAnyImpl in attribute check for group 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5164.
 2021-06-16 13:42:04 +02:00
exceptionfactory
3cdab0d752
NIFI-8707 Upgraded SSHJ to 0.31.0 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5162.
 2021-06-16 12:06:06 +02:00
exceptionfactory
8ea27d515a
NIFI-8704 Upgraded Spring Framework from 5.3.6 to 5.3.8 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5160.
 2021-06-15 20:06:04 +02:00
Mark Payne
e06afbdd22
NIFI-8609: Optimized AvroTypeUtil Record creation and conversion 
Added unit test that is ignored so that it can be manually run for testing performance before/after changes to AvroTypeUtil. Updated AvroTypeUtil to be more efficient by not using Record.getValue() and instead iterating over the Map of values directly. getValue() is less efficient here because we know the RecordField's we are iterating over exist in the schema since they are retrieved from there directly; as a result, any null values still have be looked up by aliaases, but that step can be skipped in this situation. Also avoided looking for fields that exist in Avro Schema and not in RecordSchema just to set default values on GenericRecord - there's no need to set them if they are default values.

This closes #5080

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2021-06-15 09:05:38 -05:00
Mark Payne
8004aa5e6e
NIFI-8698: If using a Data RecordPath and handling schema drift, ensure that the RecordPath is evaluated against the data before attempting to determine the fields to update. Also refactored to move the handling of schema drift into its own method for the purpose of clarity 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5156
 2021-06-15 09:45:18 -04:00
markap14
96a8b2d090
NIFI-8666: Allow users to escape parameter names in Expression Language using quotes. (#5133) 2021-06-15 09:37:34 -04:00
Nathan Gough
b49a62e1b5
NIFI-8699 - Upgrade Lucene to 8.8.2 in Provenance Bundle 
This closes #5157

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2021-06-15 08:30:50 -05:00
Kent Nguyen
87c02c17d2
NIFI-8674: Update Apache Calcite version to 1.27.0 
This closes #5138

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2021-06-14 15:47:47 -05:00
Mark Payne
576338cd55 NIFI-8697: When login endpoint is encountered and JWT is no longer valid, request cookie to be deleted. Also fixed NPE that was encountered when going to /logout without the expected cookie being present 
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5155.
 2021-06-14 16:06:18 -04:00
Mark Payne
20c889cf82
NIFI-8620: Ensure that we provider appropriate error messages if attempting to migrate FlowFiles from one session to another without including full hierarchy; added tests to verify behavior 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5099
 2021-06-14 10:00:49 -04:00
Peter Turcsanyi
48befe22f6
NIFI-8645: Disable LongRunningTaskMonitor by default 
This closes #5111

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2021-06-14 08:50:34 -05:00
exceptionfactory
ab851b9cde NIFI-8672 Added syncUninterruptibly() to Netty shutdown and close methods 
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5139.
 2021-06-12 17:14:34 -04:00
Peter Turcsanyi
1a515ee74a NIFI-8656 Support EL for SAS Token in the ADLS Gen2 processors 
This closes #5119

Signed-off-by: Joey Frazee <jfrazee@apache.org>
 2021-06-11 14:11:07 -07:00
markap14
38cf25c7cf
NIFI-8647: Updated VersionedPropertyDescriptor to include information about which properties are referencing resources, how many resource, and the types of resources; updated NiFiRegistryFlowMapper to perform the mapping. (#5114) 2021-06-11 16:48:58 -04:00
Joe Gresock
1ccc4fbb0f
NIFI-8651: Refactor Sensitive Properties Providers for extension 
This closes #5131

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2021-06-11 15:44:37 -05:00
Mark Payne
172afac6ab
NIFI-8633: This closes #5104. When reading a Content/Resource Claim from FileSystemRepository, avoid the unnecessary Files.exists call and instead just create a FileInputStream, catching FileNotFoundException 
Signed-off-by: Joe Witt <joewitt@apache.org>
 2021-06-11 13:10:32 -07:00
Matthew Burgess
3e774bc5be
NIFI-8625: Refactor scripted components to use ScriptRunner to fix concurrency issues (#5116) 
NIFI-8625: Refactor scripted components to use ScriptRunner to fix concurrency issues
 2021-06-11 14:44:55 -04:00
exceptionfactory
db40989b48 NIFI-8516 Enabled HTTPS and Single User Authentication in default configuration 
- Set default HTTPS Port to 9443
- Set default authorizer to single-user-authorizer
- Set default login-identity-provider to single-user-provider
- Updated README.md with authentication instructions using generated credentials
- Updated default URL and port information in Administration and User Guides
- Updated Getting Started Guide with authentication and URL changes
- Updated Docker images to set HTTPS as default configuration
- Updated default HTTPS port to 8443
- Set Cluster Protocol secure property in Docker start scripts
- Added set-single-user-credentials command
- Refactored shared classes to nifi-single-user-utils
- Updated Getting Started documentation and logging
- Updated documentation and TLS Toolkit default ports
- Updated Toolkit Guide and Administration Guide
- Updated README.md with HTTPS links
 2021-06-11 14:18:53 -04:00
Timea Barna
d7a8d275c9 NIFI-8642 Select the default Old Gen Memory Pool for Memory Reporting Task 
This closes #5115.

Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
 2021-06-11 14:06:15 +02:00
Mark Payne
5c91143dc2
NIFI-8680: Ensure that if commitAsync() is called and an Exception caught that it is rethrown 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5143
 2021-06-10 10:10:51 -04:00
exceptionfactory
4df2a79c8a
NIFI-8678 Forced Calcite 1.27.0 and Calcite Avatica 1.6.0 for Hive Bundle 
- Overriding Calcite incubating versions avoids dependencies from third party repositories

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5142
 2021-06-10 10:02:50 -04:00
Peter Gyori
7cb3bce6a7
NIFI-8622: Reuse of GenericRecord instance eliminated from AvroReaderWithExplicitSchema to avoid issues when parsing fields of type "bytes" (#5090) 2021-06-09 15:22:22 -04:00
markap14
d44dec7345
NIFI-8670: Fixed bug in which a Parameter Context Update would fail if an updated parameter was referenced by a Controller Service whose state was ENABLING. Created system test to verify and addressed bug that was encountered in SingleFlowFileConcurrencyIT. (#5137) 2021-06-09 13:07:33 -04:00
Paul Grey
a3c1cd074b
NIFI-8499 - Added encrypted FlowFile repository swap file implementation 
This closes #5122

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2021-06-08 13:23:22 -05:00
Mark Payne
07ff4f2592
NIFI-8667: When marking a Controller Service as enabled, ensure that we release the write lock before calling validation methods of referencing components. Otherwise, we can encounter a deadlock. 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5134
 2021-06-08 11:59:24 -04:00
Matthew Burgess
659fda10f3
NIFI-8659: Support multiple output records in JoltTransformRecord (#5126) 
NIFI-8659: Support multiple output records in JoltTransformRecord
 2021-06-07 15:30:41 -04:00
Dominik Przybysz
f07e17ba74
[NIFI-8610] Do not reuse generic records in convertAvroToORC 
[NIFI-8610] Simplify decimal test for convertAvroToORC

Signed-off-by: Arpad Boda <aboda@apache.org>

This closes #1081
 2021-06-07 11:26:16 +02:00