Commit Graph

4502 Commits

Author SHA1 Message Date
exceptionfactory c668d3df1b NIFI-8782 Added Rate-Limiting for Access Token Requests
- Added Jetty DoSFilter configured for /access/token
- Added nifi.web.max.access.token.requests.per.second property with default value of 25

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5215.
2021-07-14 14:24:45 -04:00
Joe Witt 7356332852
Merge branch 'NIFI-8767-RC2' 2021-07-14 09:07:22 -05:00
exceptionfactory 05d7867a16
NIFI-8768 Added toLocalDate() for convertType() handling of DATE fields
- Updated PutKudu to use DataTypeUtils.toLocalDate() for DATE fields
- Updated PutDatabaseRecord to remove convertDateToLocalTZ() since convertType() uses toLocalDate()
- Updated PutElasticsearchHttpRecord to use default time zone format for DATE fields
- Updated WriteXMLResult to use default time zone format for DATE fields
- Updated WriteJsonResult to use default time zone format for DATE fields
- Updated AvroTypeUtil to use toLocalDate() for logical DATE fields
- Updated JdbcCommon to avoid conversion to UTC for logical DATE fields
- Updated Processor and RecordReader unit tests for consistency in DATE comparison

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5210
2021-07-13 16:17:36 -04:00
Denes Arvay 4fde9648dd
NIFI-8759 ExecuteSQL and ExecuteSQLRecord unnecessarily fall back to default decimal scale
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5212
2021-07-13 13:27:58 -04:00
Paul Grey ce90249f7c
NIFI-8772 - address Maven build warnings
This closes #5207

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-07-12 16:55:21 -05:00
Joe Gresock 1793af4987
NIFI-8776: Using mocks in Test*IgniteCache
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5211
2021-07-12 12:45:07 -04:00
Joe Witt 97feacc181
NIFI-8767-RC2 prepare for next development iteration 2021-07-10 12:17:09 -07:00
Joe Witt fcbf1d5f97
NIFI-8767-RC2 prepare release nifi-1.14.0-RC2 2021-07-10 12:17:05 -07:00
Mark Payne a4cfdbb695
NIFI-8774: Fixed NullPointerExceptions
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5208
2021-07-09 21:56:00 -04:00
exceptionfactory 75de68e013
NIFI-8764 Refactored UnpackContent to use both Commons Compress and Zip4j
- UnpackContent uses Zip4j when configured with a password property
- UnpackContent uses Commons Compress when a password is not specified

NIFI-8764 Updated Password property description mentioning disabled algorithms

NIFI-8764 Adjusted Password property description

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5201
2021-07-09 13:15:44 -04:00
exceptionfactory aedbd0d753 NIFI-8511 Added KeyStore implementation of KeyProvider
- KeyStoreKeyProvider supports PKCS12 and BCFKS
- Refactored KeyProvider and implementations to nifi-security-kms
- Updated Admin Guide and User Guide with KeyStoreKeyProvider details

NIFI-8511 Improved documentation and streamlined several methods

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5110.
2021-07-09 12:12:25 -04:00
Mark Payne 2025999310
NIFI-8771: This closes #5205. Ensure that we consider root process group level Controller Services when determining whether or not the dataflow is empty
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-07-08 14:26:06 -07:00
thenatog 775130580a
NIFI-8616 - Updated DelimitedInputStream to extend the FilterInputStream class. (#5203) 2021-07-07 18:36:58 -04:00
Mark Bean ea31634ea7
NIFI-8195: add default connection settings to process group configuration
- include new process group property support in NiFi Registry
  - updated documentation to describe and show new feature
  - added elements to XSD schema definition

NIFI-8195: update to DAO to fix PG move and copy/paste

update condition to not null vice null

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5192
2021-07-07 17:45:54 -04:00
Nathan Gough 9a125dcaaf
NIFI-8616 Refactored PutTCP, PutUDP, and PutSplunk to use Netty nifi-event-transport classes
This closes #5182

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-07-07 16:24:49 -05:00
Matthew Burgess e16016b4ab
NIFI-8530: Improved DELETE handling in PutDatabaseRecord for non-nullable columns
This closes #5173

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-07-07 14:51:40 -05:00
exceptionfactory 1de01e34da
NIFI-8741 Changed JettyServerGroovyTest to use NetworkUtils.getAvailableTcpPort()
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5200.
2021-07-07 10:55:32 +02:00
exceptionfactory edee7a71ec
NIFI-8756 Upgraded AngularJS to 1.8.2 and JQuery to 3.6.0
- Upgraded Angular Material from 1.1.10 to 1.1.26
- Upgraded Moment from 2.24.0 to 2.29.1
- Upgraded JSON Lint from 1.6.2 to 1.6.3
- Upgraded Slickgrid from 2.4.27 to 2.4.38
- Upgraded frontend-maven-plugin from 1.4 to 1.12.0
- Upgraded frontend-maven-plugin NodeJS from 12.7.0 to 12.22.2

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5197.
2021-07-07 10:53:03 +02:00
Mark Payne 6df07df3b2
NIFI-8644: Introduced a notion of ParameterProviderDefinition
- Refactored stateless to use this when creating a dataflow so that Parameter Provider implementations can be externalized into NARs. Also updated ExtensionDiscoveringManager such that callers are able to provide a new type of class to be discovered (e.g., ParameterProvider) so that the extensions will be automatically discovered
- Put specific command-line overrides as highest precedence for parameter overrides
- Make ParameterOverrideProvider valid by allowing for dynamically added parameters
- Fixed bug in validation logic, added new system tests to verify proper handling of Required and Optional properties
- Addressed review feedback and fixed some bugs. Also added system test to verify Parameter Providers are working as expected

This closes #5113

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-07-06 16:45:02 -05:00
Guillaume Schaer ace27e5f69 NIFI-8501 Added Azure blob client side encryption
This closes #5078

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-07-06 13:00:15 -07:00
exceptionfactory 6c7c3b3c1b
NIFI-8711 Added SFTP support for keyboard-interactive authentication
- Included debug logging of configured Authentication Methods

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5166.
2021-06-30 14:03:28 +02:00
Matthew Burgess 632fbcca7d
NIFI-8730: Invalidate (don't evaluate) empty script in scripting components
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5184.
2021-06-30 13:44:52 +02:00
exceptionfactory b27c2b500e
NIFI-8748 Corrected PutKudu String to java.sql.Date parsing
- Added getDateFormat() using default time zone instead of GMT time zone from DataTypeUtils.getDateFormat()

NIFI-8748 Adjusted Date Format to use DataType.getFormat()

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5194.
2021-06-30 13:40:34 +02:00
exceptionfactory 786f17182b
NIFI-8630 Upgraded javax.mail 1.4.7 to jakarta.mail 2.0.1 for PutEmail
- Added jakarata.activation 2.0.1 to support jakarta.mail 2.0.1
- Clarified description of SMTP TLS property enabling STARTTLS

NIFI-8630 Added final to several variables and adjusted variable declaration

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5179.
2021-06-25 19:18:16 +02:00
Nathan Gough 1dc528ae80
NIFI-8699 - Adding the lucene-backward-codecs dependency to maintain backward compatibility with old provenance repos.
This closes #5187

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-25 11:53:29 -05:00
markap14 88cc232f15
NIFI-8731: If a processor is running but made invalid due to a parameter update, it ends up in a state of STARTING. If the parameter is then updated again, it transitions the state to STOPPING but the processor is not fully stopped yet. At that point, the parameter is updated and the processor is attempted to be started again. Fixed this by keeping the number of active threads to >= 1 if processor is STOPPING in order to convey that it is not fully stopped. Also addressed a few minor bugs discovered in the process: when stopping a processor, if status == invalid, it should be skipped instead of waiting for the status to become stopped since it never will be. In the DTO's run status use Stopped instead of Invalid if there is at least 1 active thread / if stopping but not stopped. When considering if a processor has transitioned to the desired state for parameter updates, do not consider validation status if still transitioning to stopped or if the desired state has already been reached. Added new system tests to verify behavior. (#5180)
This closes #5180
2021-06-25 12:40:49 -04:00
Pierre Villard 1313ee3d90
NIFI-8735 - Increase polling frequency for NAR autoloading providers (#5183) 2021-06-25 17:37:51 +02:00
Lars Francke 49d10dbcbd
NIFI-5573 Allow overriding of nifi-env.sh
This closes #2985.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2021-06-25 10:46:23 -04:00
Peter Turcsanyi 39ffa4a8ac
NIFI-8737: Fixed incorrect provenance data in HDFS processors when Directory property is inconsistent with core-site.xml 2021-06-25 14:19:57 +02:00
Joe Gresock 726082ffa6
NIFI-8447 Added HashiCorp Vault Transit Sensitive Properties Provider
- Added default bootstrap-hashicorp-vault.conf
- Updated Toolkit Guide documentation with HashiCorp Vault properties

This closes #5154

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-24 22:06:42 -05:00
Matthew Burgess 5e4f32663e Revert "NIFI-8195: add default connection settings to process group configuration"
This reverts commit 5ebbe0028b.
2021-06-24 18:52:17 -04:00
Attila Bukor 27c35c8c42
NIFI-8463: Support custom SASL protocol name in PutKudu (#5023)
As of KUDU-1884, Kudu supports custom Kerberos principals on server-side
and custom SASL protocol (service) names on client-side which must match
the SPN base, i.e. if the SPN is kudu/_HOST, SASL protocol name *must*
be "kudu" in the client to be able to connect to the cluster.

This patch adds the ability to configure this in the PutKudu processor.

Signed-off-by: Attila Bukor <abukor@apache.org>
2021-06-24 17:42:04 -04:00
Mark Bean 5ebbe0028b
NIFI-8195: add default connection settings to process group configuration
NIFIDEVS-8195: fixed properties not properly inheriting from template/snippet values

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5094
2021-06-24 14:03:51 -04:00
Bence Simon a3f54fa578
NIFI-8736 Adding capability to override HDFS location for NAR autoloading
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5185.
2021-06-24 19:37:19 +02:00
Jon Kessler 115bba9ab0
NIFI-8523 Added SFTP algorithm and cipher properties
- Updated secure FTP processors to configure which algorithms, ciphers and message authentication codes are allowed to be used by the SSH Client
- Included Expression Language Variable Registry support for properties

This closes #5061

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-24 11:51:04 -05:00
markap14 ea710e44a3
NIFI-8722: Updated Quest DB to account for newly added connection status metrics (#5181)
Co-authored-by: Bence Simon <simonbence>
2021-06-23 16:50:10 -04:00
Lehel Boér 0c748a5a2b NIFI-4344: Improve bulletin messages with exception details.
This closes #5093.

Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
2021-06-23 17:34:01 +02:00
exceptionfactory 6a83115d6a NIFI-7468 Updated SSLSocketChannel to support TLS 1.3
- Handling additional FINISHED Handshake Status for TLS 1.3 Post-Handshake Messages per RFC 8446 Section 4.6
- Removed clearing buffers after handshake to avoid losing packets
- Updated read() method to check Handshake Status after SSLEngine.unwrap()
- Changed SSLSocketChannelSender to close SSLSocketChannel before other resources
- Added ChannelStatus enum and convenience logging methods for tracing status
- Added unit tests for TLS 1.2 and 1.3 using Netty server and client handlers

NIFI-8704 Updated netty-handler to 4.1.65.Final

NIFI-7468 Corrected SSLSocketChannel.read() to return byte read

NIFI-7468 Adjusted comment formatting

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5152.
2021-06-22 22:29:16 -04:00
simonbence 73363d9aea
NIFI-8717 Refactoring PutHDFS (#5175)
NIFI-8717 Refactoring PutHDFS
2021-06-22 13:13:14 -04:00
exceptionfactory f2698a4e1e
NIFI-8723 Upgraded Jackson to 2.12.3
- Updated MiNiFi to reference root jackson.version
- Updated Registry assembly to reference root jackson.version
- Added dependency management in NAR modules to override transitive versions

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5174
2021-06-22 11:21:45 -04:00
Nathan Gough 9744644b9d
NIFI-8025 - Refactored SAML and OIDC Resources to separate classes
This closes #5079

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-21 16:37:06 -05:00
Chris Sampson 02b4e33aa6
NIFI-8683 support Expression Language for the Truststore/Keystore properties of SSLContextService
This closes #5147

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-21 14:39:35 -05:00
Mark Payne 6fc9f33ba2
NIFI-8720: Ensure that when we use Expression Language for Records Per Split property of SplitRecord, we treat the result as a value of 1 if it's less than 1.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5171
2021-06-21 13:09:20 -04:00
Mark Payne b1ad13c835
NIFI-8719: Ensure that when OutputStream is returned from StandardProcessSession.write, upon close of the OutputStream we close the underlying resources.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5170
2021-06-21 12:55:17 -04:00
exceptionfactory 2efc2b6b8b
NIFI-8718 Upgraded Apache Commons IO to 2.10.0
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5169
2021-06-21 11:29:27 -04:00
rhavermans 9fd8042cfb
NIFI-8664 Update json-path lib from 2.4.0 -> 2.6.0
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5132
2021-06-21 11:01:10 -04:00
Peter Turcsanyi 820b2cff29 NIFI-8662: Using the configured AWS region when parsing from VPCE endpoint URL fails in AbstractAWSProcessor
This closes #5140.

Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
2021-06-16 21:50:25 +02:00
Bernhard Geisberger 90f44d9d62
NiFi-8152: Fix NPE in RedisDistributedMapCacheClientService caused by newer versions of Jedis
This closes #4783

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2021-06-16 14:07:16 -04:00
exceptionfactory 9be9b9dbe2
NIFI-8708 Upgraded Spring Framework to 5.3.8 for several extensions
- nifi-vault-utils
- nifi-email-bundle
- nifi-redis-bundle

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5163.
2021-06-16 18:18:54 +02:00
Abhishek Kumar 520ee82745
NIFI-8709 - added XSAnyImpl in attribute check for group
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5164.
2021-06-16 13:42:04 +02:00