Commit Graph

6603 Commits

Author SHA1 Message Date
Joe Witt fcbf1d5f97
NIFI-8767-RC2 prepare release nifi-1.14.0-RC2 2021-07-10 12:17:05 -07:00
Mark Payne 3bc64754c3
NIFI-8775: Fixing system test assembly
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5209
2021-07-09 22:26:37 -04:00
Mark Payne a4cfdbb695
NIFI-8774: Fixed NullPointerExceptions
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5208
2021-07-09 21:56:00 -04:00
exceptionfactory 75de68e013
NIFI-8764 Refactored UnpackContent to use both Commons Compress and Zip4j
- UnpackContent uses Zip4j when configured with a password property
- UnpackContent uses Commons Compress when a password is not specified

NIFI-8764 Updated Password property description mentioning disabled algorithms

NIFI-8764 Adjusted Password property description

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5201
2021-07-09 13:15:44 -04:00
Joe Witt b75965d08f
NIFI-8767 fixing conflicting/not needed declarations from former subproject poms for registry and minifi 2021-07-09 09:25:46 -07:00
exceptionfactory aedbd0d753 NIFI-8511 Added KeyStore implementation of KeyProvider
- KeyStoreKeyProvider supports PKCS12 and BCFKS
- Refactored KeyProvider and implementations to nifi-security-kms
- Updated Admin Guide and User Guide with KeyStoreKeyProvider details

NIFI-8511 Improved documentation and streamlined several methods

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5110.
2021-07-09 12:12:25 -04:00
Mark Payne 2025999310
NIFI-8771: This closes #5205. Ensure that we consider root process group level Controller Services when determining whether or not the dataflow is empty
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-07-08 14:26:06 -07:00
Joe Witt 777f9029b1
NIFI-8767 fixing project reference 2021-07-08 07:40:07 -07:00
exceptionfactory 9d29e8bae4
NIFI-8769 This closes #5204. Removed Bintray repository from MiNiFi test modules
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-07-08 07:14:22 -07:00
thenatog 775130580a
NIFI-8616 - Updated DelimitedInputStream to extend the FilterInputStream class. (#5203) 2021-07-07 18:36:58 -04:00
Mark Bean ea31634ea7
NIFI-8195: add default connection settings to process group configuration
- include new process group property support in NiFi Registry
  - updated documentation to describe and show new feature
  - added elements to XSD schema definition

NIFI-8195: update to DAO to fix PG move and copy/paste

update condition to not null vice null

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5192
2021-07-07 17:45:54 -04:00
Nathan Gough 9a125dcaaf
NIFI-8616 Refactored PutTCP, PutUDP, and PutSplunk to use Netty nifi-event-transport classes
This closes #5182

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-07-07 16:24:49 -05:00
Matthew Burgess e16016b4ab
NIFI-8530: Improved DELETE handling in PutDatabaseRecord for non-nullable columns
This closes #5173

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-07-07 14:51:40 -05:00
exceptionfactory 1de01e34da
NIFI-8741 Changed JettyServerGroovyTest to use NetworkUtils.getAvailableTcpPort()
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5200.
2021-07-07 10:55:32 +02:00
exceptionfactory edee7a71ec
NIFI-8756 Upgraded AngularJS to 1.8.2 and JQuery to 3.6.0
- Upgraded Angular Material from 1.1.10 to 1.1.26
- Upgraded Moment from 2.24.0 to 2.29.1
- Upgraded JSON Lint from 1.6.2 to 1.6.3
- Upgraded Slickgrid from 2.4.27 to 2.4.38
- Upgraded frontend-maven-plugin from 1.4 to 1.12.0
- Upgraded frontend-maven-plugin NodeJS from 12.7.0 to 12.22.2

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5197.
2021-07-07 10:53:03 +02:00
Mark Payne 6df07df3b2
NIFI-8644: Introduced a notion of ParameterProviderDefinition
- Refactored stateless to use this when creating a dataflow so that Parameter Provider implementations can be externalized into NARs. Also updated ExtensionDiscoveringManager such that callers are able to provide a new type of class to be discovered (e.g., ParameterProvider) so that the extensions will be automatically discovered
- Put specific command-line overrides as highest precedence for parameter overrides
- Make ParameterOverrideProvider valid by allowing for dynamically added parameters
- Fixed bug in validation logic, added new system tests to verify proper handling of Required and Optional properties
- Addressed review feedback and fixed some bugs. Also added system test to verify Parameter Providers are working as expected

This closes #5113

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-07-06 16:45:02 -05:00
Guillaume Schaer ace27e5f69 NIFI-8501 Added Azure blob client side encryption
This closes #5078

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-07-06 13:00:15 -07:00
exceptionfactory 54624bc26d
NIFI-8758 Increased GitHub build timeout to 120 minutes
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5199.
2021-07-06 11:29:53 +02:00
exceptionfactory 0b15606fae
NIFI-8757 Upgraded MiNiFi docker-compose-rule-junit4 to 1.5.0
This closes #5198.

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-07-05 13:09:54 -05:00
exceptionfactory 6c7c3b3c1b
NIFI-8711 Added SFTP support for keyboard-interactive authentication
- Included debug logging of configured Authentication Methods

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5166.
2021-06-30 14:03:28 +02:00
Matthew Burgess 632fbcca7d
NIFI-8730: Invalidate (don't evaluate) empty script in scripting components
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5184.
2021-06-30 13:44:52 +02:00
exceptionfactory b27c2b500e
NIFI-8748 Corrected PutKudu String to java.sql.Date parsing
- Added getDateFormat() using default time zone instead of GMT time zone from DataTypeUtils.getDateFormat()

NIFI-8748 Adjusted Date Format to use DataType.getFormat()

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5194.
2021-06-30 13:40:34 +02:00
Lockie Richter 70d8fe8e00
NIFI-8624 Updated Maven archetypes to conform to SonarQube standards
This closes #5092

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-28 20:47:35 -05:00
theBestAndrew 1033e77e7f
NIFI-8686 Update administration guide with kernel 3 time_wait setting
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5149
2021-06-25 16:15:24 -04:00
Peter Turcsanyi f11c99f9ec
NIFI-8475: Fixed EL query parser when input contains multiple escaped expressions
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5031
2021-06-25 13:59:11 -04:00
exceptionfactory 786f17182b
NIFI-8630 Upgraded javax.mail 1.4.7 to jakarta.mail 2.0.1 for PutEmail
- Added jakarata.activation 2.0.1 to support jakarta.mail 2.0.1
- Clarified description of SMTP TLS property enabling STARTTLS

NIFI-8630 Added final to several variables and adjusted variable declaration

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5179.
2021-06-25 19:18:16 +02:00
Nathan Gough 1dc528ae80
NIFI-8699 - Adding the lucene-backward-codecs dependency to maintain backward compatibility with old provenance repos.
This closes #5187

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-25 11:53:29 -05:00
markap14 88cc232f15
NIFI-8731: If a processor is running but made invalid due to a parameter update, it ends up in a state of STARTING. If the parameter is then updated again, it transitions the state to STOPPING but the processor is not fully stopped yet. At that point, the parameter is updated and the processor is attempted to be started again. Fixed this by keeping the number of active threads to >= 1 if processor is STOPPING in order to convey that it is not fully stopped. Also addressed a few minor bugs discovered in the process: when stopping a processor, if status == invalid, it should be skipped instead of waiting for the status to become stopped since it never will be. In the DTO's run status use Stopped instead of Invalid if there is at least 1 active thread / if stopping but not stopped. When considering if a processor has transitioned to the desired state for parameter updates, do not consider validation status if still transitioning to stopped or if the desired state has already been reached. Added new system tests to verify behavior. (#5180)
This closes #5180
2021-06-25 12:40:49 -04:00
Pierre Villard 1313ee3d90
NIFI-8735 - Increase polling frequency for NAR autoloading providers (#5183) 2021-06-25 17:37:51 +02:00
Lars Francke 49d10dbcbd
NIFI-5573 Allow overriding of nifi-env.sh
This closes #2985.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2021-06-25 10:46:23 -04:00
Peter Turcsanyi 39ffa4a8ac
NIFI-8737: Fixed incorrect provenance data in HDFS processors when Directory property is inconsistent with core-site.xml 2021-06-25 14:19:57 +02:00
Joe Gresock 726082ffa6
NIFI-8447 Added HashiCorp Vault Transit Sensitive Properties Provider
- Added default bootstrap-hashicorp-vault.conf
- Updated Toolkit Guide documentation with HashiCorp Vault properties

This closes #5154

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-24 22:06:42 -05:00
Matthew Burgess 5e4f32663e Revert "NIFI-8195: add default connection settings to process group configuration"
This reverts commit 5ebbe0028b.
2021-06-24 18:52:17 -04:00
Attila Bukor 27c35c8c42
NIFI-8463: Support custom SASL protocol name in PutKudu (#5023)
As of KUDU-1884, Kudu supports custom Kerberos principals on server-side
and custom SASL protocol (service) names on client-side which must match
the SPN base, i.e. if the SPN is kudu/_HOST, SASL protocol name *must*
be "kudu" in the client to be able to connect to the cluster.

This patch adds the ability to configure this in the PutKudu processor.

Signed-off-by: Attila Bukor <abukor@apache.org>
2021-06-24 17:42:04 -04:00
Mark Bean 5ebbe0028b
NIFI-8195: add default connection settings to process group configuration
NIFIDEVS-8195: fixed properties not properly inheriting from template/snippet values

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5094
2021-06-24 14:03:51 -04:00
Bence Simon a3f54fa578
NIFI-8736 Adding capability to override HDFS location for NAR autoloading
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5185.
2021-06-24 19:37:19 +02:00
Jon Kessler 115bba9ab0
NIFI-8523 Added SFTP algorithm and cipher properties
- Updated secure FTP processors to configure which algorithms, ciphers and message authentication codes are allowed to be used by the SSH Client
- Included Expression Language Variable Registry support for properties

This closes #5061

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-24 11:51:04 -05:00
markap14 ea710e44a3
NIFI-8722: Updated Quest DB to account for newly added connection status metrics (#5181)
Co-authored-by: Bence Simon <simonbence>
2021-06-23 16:50:10 -04:00
exceptionfactory 1d9ea6d4cf
NIFI-8725 Added Single User Environment Variables for Docker (#5178)
* NIFI-8725 Added Single User Environment Variables for Docker

- Updated Docker README.md with example command and link to Administration Guide section
- Updated Docker integration test to check HTTP response status
- Disabled integration test plugin for dockerhub because it depends on changes on the main branch. Can be re-enabled after 1.14.0 is released.
2021-06-23 12:42:44 -04:00
Lehel Boér 0c748a5a2b NIFI-4344: Improve bulletin messages with exception details.
This closes #5093.

Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
2021-06-23 17:34:01 +02:00
exceptionfactory 8e5ce46f6c
NIFI-8721 Upgraded Spring Boot to 2.5.1 for Registry
- Upgraded spring-jdbc to 5.3.8 for Registry

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5172.
2021-06-23 10:38:19 +02:00
exceptionfactory 6a83115d6a NIFI-7468 Updated SSLSocketChannel to support TLS 1.3
- Handling additional FINISHED Handshake Status for TLS 1.3 Post-Handshake Messages per RFC 8446 Section 4.6
- Removed clearing buffers after handshake to avoid losing packets
- Updated read() method to check Handshake Status after SSLEngine.unwrap()
- Changed SSLSocketChannelSender to close SSLSocketChannel before other resources
- Added ChannelStatus enum and convenience logging methods for tracing status
- Added unit tests for TLS 1.2 and 1.3 using Netty server and client handlers

NIFI-8704 Updated netty-handler to 4.1.65.Final

NIFI-7468 Corrected SSLSocketChannel.read() to return byte read

NIFI-7468 Adjusted comment formatting

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5152.
2021-06-22 22:29:16 -04:00
simonbence 73363d9aea
NIFI-8717 Refactoring PutHDFS (#5175)
NIFI-8717 Refactoring PutHDFS
2021-06-22 13:13:14 -04:00
Chris Sampson bd265bd508
NIFIREG-455 upgrade jgit version for nifi-registry GitFlowPersistenceProvider
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5123
2021-06-22 12:28:28 -04:00
exceptionfactory f2698a4e1e
NIFI-8723 Upgraded Jackson to 2.12.3
- Updated MiNiFi to reference root jackson.version
- Updated Registry assembly to reference root jackson.version
- Added dependency management in NAR modules to override transitive versions

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5174
2021-06-22 11:21:45 -04:00
exceptionfactory 006e3498b7
NIFI-8724 Upgraded Bouncy Castle to 1.69
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5177
2021-06-22 11:13:37 -04:00
Nathan Gough 9744644b9d
NIFI-8025 - Refactored SAML and OIDC Resources to separate classes
This closes #5079

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-21 16:37:06 -05:00
Chris Sampson 02b4e33aa6
NIFI-8683 support Expression Language for the Truststore/Keystore properties of SSLContextService
This closes #5147

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-06-21 14:39:35 -05:00
Mathieu Rossignol 5196ba9e14
Update developer-guide.adoc
Typo: "created created" (twice)

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5168
2021-06-21 13:14:10 -04:00
Mark Payne 6fc9f33ba2
NIFI-8720: Ensure that when we use Expression Language for Records Per Split property of SplitRecord, we treat the result as a value of 1 if it's less than 1.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5171
2021-06-21 13:09:20 -04:00