Commit Graph

7016 Commits

Author SHA1 Message Date
Mark Payne a675023b71
NIFI-9272: When determining if Property dependency is satisfied, consider property default values also
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5432.
2021-10-04 14:22:39 -04:00
timeabarna 72660af479
NIFI-9229 Flow upgrade not possible if a Output Port changes to a funnel (#5402)
* NIFI-9229 Flow upgrade not possible if a Output Port changes to a funnel
* NIFI-9229 Addressing review comments modified log message and added comments
2021-10-04 09:14:36 +02:00
exceptionfactory cacd6bb88a
NIFI-9270 Upgraded JGit from 5.11.1 to 5.13.0
- Adjusted OWASP dependency-check suppressions to match Registry and MiNiFi packages

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5431.
2021-10-03 10:54:30 +02:00
exceptionfactory defea61075
NIFI-9253 Corrected SSLSocketChannel.available() for TLSv1.3
- Added unit tests to reproduce issues with available() method
- Changed available() to return size of application buffer
- Removed unused isDataAvailable()
- Refactored unwrap handling to read from channel for buffer underflow

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5421.
2021-10-02 18:31:05 +02:00
exceptionfactory ae0154de5a
NIFI-9268 Upgraded OkHttp from 4.9.1 to 4.9.2
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5429.
2021-10-02 18:26:40 +02:00
exceptionfactory 00b2358c27
NIFI-9252 Removed unused nifi.sensitive.props.provider Property
- Removed nifi.minifi.sensitive.props.provider Property from MiNiFi
- Removed property from example NiFi properties files
- Removed provider from MiNiFi SensitivePropsSchema
- Removed BC provider value from MiNiFi test cases

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5422.
2021-10-01 22:20:13 -04:00
exceptionfactory a94b47ecf8
NIFI-9254 Updated default Stateless Sensitive Property configuration
- Set NIFI_PBKDF2_AES_GCM_256 as property encryption method
- Replaced static default sensitive properties key with random UUID
- Added unit test for PropertiesFileEngineConfigurationParser
- Added random encryption key generation method
- Changed Stateless to use PropertyEncryptionMethod enum

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5424
2021-10-01 15:55:06 -04:00
Peter Turcsanyi 24c0c39ebb
NIFI-9261: Make ActiveMQ client configurable via SSL Context Service in JMSConnectionFactoryProvider
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5425.
2021-10-01 09:29:02 +02:00
exceptionfactory 4943560521
NIFI-9223 Corrected ListenSyslog with default address of 0.0.0.0
- Refactored NettyEventServerFactory to accept nullable InetAddress
- Updated unit tests referencing NettyEventServerFactory

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5426.
2021-10-01 09:24:46 +02:00
exceptionfactory e16a6c2b89 NIFI-9241 Refactored CSRF mitigation using random Request-Token
- Replaced use of Authorization header with custom Request-Token header for CSRF mitigation
- Added Request-Token cookie for CSRF mitigation
- Replaced session storage of JWT with expiration in seconds
- Removed and disabled CORS configuration
- Disabled HTTP OPTIONS method
- Refactored HTTP Proxy URI construction using RequestUriBuilder

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5417.
2021-09-30 20:36:15 -04:00
Peter Turcsanyi 1379674240
NIFI-9255: Support asynchronous session commits in ListenHTTP (#5419)
NIFI-9255: Support asynchronous session commits in ListenHTTP
2021-09-29 18:16:47 -04:00
Matt Gilman be57a210ff
NIFI-8491: Adding support for configuring parameter context inheritance (#5371)
* NIFI-8491:
- Adding support for configuring parameter context inheritance.

* NIFI-8491:
- Allowing changes to the parameter context inheritance to drive Apply disabled state.

* NIFI-8491: Updating StandardParameterContext#isAuthorized check

* NIFI-8491:
- Showing selected inherited parameter contexts in ready only form when appropriate.
- Allowing available parameter contexts to be inherited by double clicking.
- Removing support for rendering unauthorized inherited parameter contexts as they can no longer be opened.

* NIFI-8491: Adding inherited param context verification earlier

* NIFI-8491:
- Addressing CI failures by rolling back to some order JS language spec to allow yui-compress to minify and compress.

* NIFI-8491:
- Ensuring selected context sort order is honored.
- Ensuring the Apply button is correctly enabled.
- Showing Pending Apply message when selected Parameter Context changes.
- Ensuring the Parameter's tab is selected now that there is a third tab.

* Updates to inherited param context verification

* Improving validation between parameters/inherited parameters

* NIFI-8491:
- Ensuring the available parameter contexts are loaded whether the edit dialog is opened from the listing or outside of the listing.

* NIFI-8491:
- Fixing conditions we check if the parameter context listing is currently open.

* NIFI-8491:
- Waiting for the parameter contexts to load prior to rendering the parameter context inheritance tab and showing the dialog.

* NIFI-8491:
- Fixing pending apply message clipping.
- Hiding pending apply message after clicking Apply.

Co-authored-by: Joe Gresock <jgresock@gmail.com>

This closes #5371
2021-09-29 17:04:35 -04:00
Paul Grey 46a5e3f096
NIFI-9207 - Added Max Read Size to Distributed Cache Servers
This closes #5420

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-09-29 13:48:08 -05:00
Mark Payne efc1cb012f
NIFI-8727: This closes #5418. Addressed bug in which ProcessSession doesn't properly decrement claimant count when a FlowFile is cloned and then the clone written to. Added automated tests to ensure that we are properly handling cases where a FlowFile is clone and then the contents modified
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-09-29 11:02:41 -07:00
exceptionfactory 6ef638d13d
NIFI-9251 Added AuthenticationManager Bean to avoid unnecessary password generation
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5416.
2021-09-28 19:23:19 +02:00
Matthew Burgess fa2be2ee63
NIFI-9009 NPE defensive code for referenced attributes
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
2021-09-27 22:11:29 -04:00
Paul Grey 720e7ef353
NIFI-8919 - Refactor Distributed Cache Clients using Netty
This closes #5311

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-09-27 21:00:58 -05:00
Peter Gyori 38d26dafa3
NIFI-9192: ResultSetRecordSet considers value of useLogicalTypes flag when determining the object's schema.
NIFI-9192: ResultSetRecordSet consider value of useLogicalType flag when sqlType is not handled separately (default) and readerSchema is not null

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: Unit tests added to test the use of useLogicalType flag

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: ResultSetRecordSet consider not using logical type when sqlType is OTHER

NIFI-9192: Fixed checkstyle violations

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: Renamed JdbcCommon.getUseLogicalTypes to isUseLogicalTypes

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: Added unit tests for Array types with and without using logical types

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: Review comments considered and unit tests refactored

NIFI-9192: Added further unit tests

NIFI-9192: Minor modifications based on review comments

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5388
2021-09-27 16:27:39 -04:00
Bryan Bende 2ae4f908f5
NIFI-8978 Add KerberosUserService to DBCPConnectionPool/HadoopDBCPConnectionPool
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5399.
2021-09-27 10:12:31 +02:00
Bryan Bende 3a8da7b33e
NIFI-8979 Add KerberosUserService to PutKudu
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5400.
2021-09-27 10:09:57 +02:00
Bryan Bende bf7d1ffa1b
NIFI-8982 Add KerberosUserService to Accumulo processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5405.
2021-09-27 10:08:00 +02:00
hondawei 30cf49db7e
NIFI-4542 - add target.dir.created to indicate if the target directory created
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5397.
2021-09-26 11:48:18 +02:00
exceptionfactory 2d7e9c1c9a
NIFI-9240 Upgraded Logback to 1.2.6 and SLF4J to 1.7.32
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5411.
2021-09-24 11:08:19 +02:00
exceptionfactory 84dbf915a9 NIFI-9060 Refactored HTTP Cookie Path Handling
- Implemented ApplicationCookieService for adding and retrieving HTTP Cookies
- Added getCookieResourceUri() leveraging allowed proxy headers to support optional Cookie Paths
- Refactored Access Resources to use ApplicationCookieService for processing
- Changed __Host- prefix to __Secure- prefix for Bearer Token cookie to support Cookie Path processing
- Removed unnecessary jetty-http dependency from nifi-web-api
- Corrected NiFi path references in JavaScript to support prefixed paths

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5329.
2021-09-23 18:03:48 -04:00
Mark Payne 5b7af511fc NIFI-8273: Fix typos in docs/clarify some statements. Addressed handling of arrays as partitions
This closes #4948.
2021-09-23 09:14:00 -04:00
Bence Simon db5b618550 NIFI-8273 Adding Scripted Record processors 2021-09-23 09:13:36 -04:00
Pierre Villard 4a5fe698f7 NIFI-9234 - Upgrade Atlas dependency to 2.2.0
This closes #5404.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-09-23 14:25:54 +02:00
Mark Payne e29c8af708
NIFI-9182: When calling ProcessSession.append(), do not allow calls from processor to flush the underlying BufferedOutputStream. Instead, wrap in a NonFlushableOutputStream and only flush when session commit is called
This closes #5359

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-09-22 18:54:32 -05:00
Mark Payne baf29e59f1
NIFI-9009: Created VerifiableProcessor, VerifiableControllerService, VerifiableReportingTask components; implemented backend work to call the methods. Added REST APIs and created/updated data models for component configuration verification
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5288
2021-09-22 19:00:23 -04:00
Peter Turcsanyi 4240a9f285
NIFI-9238: Mapped JMSConnectionFactoryProvider properties to Qpid JMS client setters
NIFI-9238: Added documentation

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5408.
2021-09-22 19:49:32 +02:00
exceptionfactory b4e0416b5e
NIFI-9228 Refactored tests using TemporaryKeyStoreBuilder
- Added TemporaryKeyStoreBuilder with File.deleteOnExit() for KeyStore and TrustStore files
- Removed JKS files from nifi-security-utils tests
- Refactored usage of KeyStoreUtils.createKeyStoreAndGetX509Certificate() to TemporaryKeyStoreBuilder
- Removed unnecesary hadoop-minikdc test dependency in security-utils
- Replaced Mini KDC Hex utility with Bouncy Castle Hex utility in unit tests

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5406
2021-09-21 16:07:13 -04:00
exceptionfactory e78674ec59 NIFI-9184 Refactored shared methods in Sensitive Property Providers
- Added EncodedSensitivePropertiesProvider with Base64 encoding methods
- Added ClientBasedEncodedSensitivePropertiesProvider with validate method
- Abstracted client configuration to ClientProvider interface and implementations
- Added unit tests for AWS and Azure Property Providers

NIFI-9184 Adjusted abstract provider class names and updated documentation

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5363.
2021-09-21 15:47:19 -04:00
exceptionfactory ba775d28de
NIFI-9228 Refactored tests to use generated KeyStores
- Refactored multiple tests using KeyStoreUtils
- Removed static KeyStore and TrustStore files

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5401
2021-09-21 11:06:19 -04:00
Pierre Villard 2e998839eb
NIFI-8966 - Confluent Schema Registry - get schema without subject/version
This closes #5266

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-09-20 07:43:56 -05:00
Pierre Villard d70359d3e8
NIFI-6717 - Add Job ID in attributes for PutBigQueryBatch
This closes #3768

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-09-18 09:08:18 -05:00
ScottGiles d97bec63ed
NIFI-8941: Adding ENDPOINT_OVERRIDE to properties List of PutLambda.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5244.
2021-09-18 15:17:06 +02:00
Gabor Gyimesi 1fdae088cf
NIFI-4692 Fix AttributesToJSON property docs to match functionality
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5313.
2021-09-18 15:13:24 +02:00
Alasdair Brown 50e60c812d
NIFI-8087 Adds MessageGroupID and MessageDuplicationId to PutSNS
Updated topic arn assert to match dummy topic name with .fifo suffix

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5367.
2021-09-18 15:08:02 +02:00
exceptionfactory 9dfcba24cd
NIFI-9179 Enabled system-tests workflow for selected modules
- Increased system test NiFi heap memory to 512 MB

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5389.
2021-09-18 15:02:54 +02:00
Bryan Bende 1374f66f09
NIFI-8977 Add KerberosUserService to Solr processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5395.
2021-09-18 14:55:42 +02:00
exceptionfactory 6f87865e27
NIFI-9213 Upgraded Dependency Check Plugin to 6.3.1
- Suppressed false positive for ftpserver
- Suppressed false positive for com.metamx:http-client
- Suppressed false positive for Jetty servlet-api
- Suppressed false positive for Testcontainers MySQL
- Suppressed false positive for vorbis-java-tika

This closes #5384

Signed-off-by: Joe Gresock <jgresock@gmail.com>
2021-09-18 07:42:42 -04:00
Joe Gresock 3df2fff6e0
NIFI-8732: Adding a HashiCorp Vault K/V Stateless Param Provider
This closes #5289

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-09-17 12:26:07 -05:00
Tamas Palfy 229f45997d NIFI-9203 Improve GrokReader to be able to handle complex grok expression properly.
This closes #5376.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-09-17 16:24:04 +02:00
Paul Grey 1ab47ce931
NIFI-9212 Add timeout and quiet period for Netty shutdown
This closes #5386

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-09-16 09:38:49 -05:00
greyp9 4af3fac07a
NIFI-9217 - avoid deadlock on cluster operation (#5390) 2021-09-15 15:40:33 -04:00
exceptionfactory bb638b13aa
NIFI-9216 Upgraded Derby to 10.14.2.0 in nifi-hive-bundle
This closes #5385

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2021-09-14 15:19:18 -04:00
markap14 73f88b3239
NIFI-9061: Eliminated the nifi.cluster.node.protocol.threads property in favor of nifi.cluster.node.protocol.max.threads property so that we can properly scale out the number of threads used for HTTP request replication. Implementing a caching mechanism for creating the DateTimeFormatter used by TimeAdapter in order to improve performance when parsing timestamps in web requests. Implementing caching logic for caching the number of characters that can rendered without needing an ellipsis for some components in the UI (#5316)
This closes #5316
2021-09-13 14:36:35 -04:00
Joey 38a1f476e3
NIFI-9055 Added handling for 0- read range to FetchS3Object (#5317) 2021-09-13 19:19:19 +02:00
denes 3fbd9c9e14
NIFI-9201 NullPointerException in AbstractKerberosUser if the tgt is not renewable (#5383) 2021-09-13 11:04:39 -04:00
Mohammed Nadeem 5e2ec9b454
NIFI-9038: Fix fingerprinting group access control policies for Remote
Port

This closes #5300

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-09-11 16:45:21 -05:00