Commit Graph

5100 Commits

Author SHA1 Message Date
exceptionfactory e618e85da7
NIFI-10093 Set timeout to 5 seconds on SocketProtocolListenerTest (#6101) 2022-06-06 16:23:17 -04:00
Matthew Burgess 5591bec44b
NIFI-9742: Use dependent property Display Names in documentation
This closes #5820

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-06 13:43:02 -05:00
Emilio Setiadarma 58f93d1f8b
NIFI-10085 Removed CryptoUtility.isUnlimitedStrengthCryptoSupported() and references
This closes #6098

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-06 08:49:39 -05:00
exceptionfactory 4aa0d31d9f NIFI-10088 Set SSH Transport Timeout using SFTP Data Timeout property
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6097.
2022-06-03 19:10:28 -04:00
exceptionfactory 1342ea7025 NIFI-10079 Upgraded ZooKeeper from 3.5.9 to 3.8.0
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6096.
2022-06-03 19:05:21 -04:00
exceptionfactory 4b655ecb68
NIFI-9804 Added HTTP/2 support to Application Server
- Added nifi.web.https.application.protocols property
- Set default protocol to HTTP/1.1 and provided documentation for enabling HTTP/2
- Changed StandardALPNProcessor handshakeFailed log to debug

Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes #6093.
2022-06-03 11:54:21 -04:00
Matthew Burgess 639ab328c9
NIFI-10057: Add support for SQL Server's DATETIMEOFFSET type to ExecuteSQL processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6080.
2022-06-03 10:33:41 +02:00
exceptionfactory 4c2fc216ab
NIFI-9671 Deprecated GetTwitter in favor of ConsumeTwitter
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6094.
2022-06-03 10:31:41 +02:00
exceptionfactory 5a2992c243
NIFI-9960 Add documentation for Sensitive Dynamic Properties (#6089)
* NIFI-9960 Added documentation for Sensitive Dynamic Properties

- Updated User Guide with Add Property details
- Updated Developer Guide with annotation configuration details
- Updated Document Writer to indicate Supports Sensitive Dynamic Properties status

* NIFI-9960 Adjusted User Guide wording based on feedback
2022-06-02 08:12:22 -04:00
Lehel 04aa32cd57
NIFI-10054: Improve SendTrapSNMP error handling
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6081.
2022-06-02 11:43:24 +02:00
exceptionfactory 7f93cb4c91
NIFI-6943 Set DataDogReportingTask API Key Property as Sensitive
- Corrected Logger handling and improved unit tests

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6090.
2022-06-02 11:40:53 +02:00
exceptionfactory 1f9672c18b
NIFI-10075 Improved TestListSFTP configuration verification
- Changed assertVerificationSuccess() to separate testVerificationSuccessful() method
- Upgraded TestListSFTP to JUnit 5

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6092.
2022-06-02 11:36:25 +02:00
Joe Gresock 48de70a568
NIFI-10071: Adding support for HashiCorp Vault K/V version 2 Secrets Engine (#6087) 2022-06-01 15:09:01 -04:00
tpalfy 0d45fef702
NIFI-10074 Fix potential UnsupportedOperationException in PutSmbFile.customValidate. (#6088) 2022-06-01 14:27:07 -04:00
exceptionfactory 45cc3cefe5 NIFI-10069 Updated multiple components to support Sensitive Dynamic Properties
- ExecuteScript
- ExecuteStreamCommand
- InvokeScriptedProcessor
- HikariCPConnectionPool

This closes #6085.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-06-01 11:26:41 -04:00
Joe Gresock 314232ca6c
NIFI-10072: Adding LoadBalanceStatus to ConnectionStatus (#6086) 2022-05-31 16:27:23 -04:00
Paula Yamashita 9878df9f5a
NIFI-9867 Adding new tests to increase Test Coverage of GetTwitter
This closes #5978

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-05-31 09:53:29 -05:00
Tamas Palfy d79b390836
NIFI-10056 Wrap table scan in a kerberos action in KuduLookupService so that expired TGTs are renewed automatically.
This closes #10056.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-05-27 18:43:46 +02:00
Bryan Bende 98ec2563e9
NIFI-10059 Remove incorrect removal of controller service from cache (#6082) 2022-05-26 17:13:28 -04:00
Mark Payne 35d8bb5ca9
NIFI-10051: Clear Controller Services' state in a background thread instead of the web thread.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6078
2022-05-26 10:31:55 -04:00
exceptionfactory 83316736f8
NIFI-9959 Add UI Support for Sensitive Dynamic Properties (#6073)
* NIFI-9959 Added UI Support for Sensitive Dynamic Properties

- Added SupportsSensitiveDynamicProperties to DBCPConnectionPool and ScriptedReportingTask

* NIFI-9959 Added sensitive parameter argument for Controller Service descriptors

* NIFI-9959 Adjusted sensitive property descriptor handling to support changing status

* NIFI-9959 Added info icon for Sensitive Value field

* NIFI-9959 Corrected handling of descriptor for existing dynamic properties

* NIFI-9959 Cleaning up dialog markup.

Co-authored-by: Matt Gilman <matt.c.gilman@gmail.com>

This closes #6073
2022-05-25 21:36:31 -04:00
Nandor Soma Abonyi f66540eb6d NIFI-9951 Add proxy support to Azure ADLS and Blob v12 processors
This closes #5990.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-05-25 20:33:14 +02:00
exceptionfactory e5203db960
NIFI-10046 Upgraded Gson to 2.9.0
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6072
2022-05-25 14:14:44 -04:00
Mark Payne d664708dd3
NIFI-10049: When unscheduling reporting task, increment its concurrent task count until we've finished all shutdown logic and then decrement it, in much the same way that we do for processors
This closes #6076

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-05-24 17:54:17 -05:00
Timea Barna 205c58230b NIFI-9879 Add min/max age and min/max size properties to ListAzure processors
This closes #5933.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-05-24 17:22:52 +02:00
exceptionfactory 99be62be67
NIFI-9958 Add Framework Support for Sensitive Dynamic Properties (#6057)
* NIFI-9958 Implemented Sensitive Dynamic Properties

- Added SupportsSensitiveDynamicProperties annotation for components
- Added optional sensitive query parameter to Property Descriptor REST Resource methods
- Added system tests for components supporting sensitive dynamic properties
- Updated REST Resources to support Sensitive Dynamic Property Names
- Updated Documentation Writer to indicate component support for Sensitive Dynamic Properties
- Updated InvokeHTTP to support Sensitive Dynamic Properties
- Updated Auditor components to handle masking Sensitive Dynamic Properties

* Refactored Property Descriptor REST method handling

- Corrected AbstractDocumentationWriter evaluation of support for sensitive dynamic properties
- Refactored Controller Service Dynamic Properties system tests to new class

* Updated AbstractComponentNode.getProperties() to get canonical descriptor
2022-05-23 17:20:54 -04:00
Matthew Burgess 41b3778d67
NIFI-10044: Add content viewer support for YAML
This closes #6063

Signed-off-by: David Handermann <exceptionfactory@apache.org?
2022-05-23 11:46:15 -05:00
Matthew Burgess e74557f990
NIFI-10027: Add support for WEBP images to the image viewer
This closes #6064

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-05-23 09:06:27 -05:00
Chris Sampson 9c9c708c14
NIFI-10043 fix ElasticsearchClient integration tests following JUnit5 upgrade; update Elasticsearch and json-path dependency versions
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #6062.
2022-05-20 12:27:53 -04:00
exceptionfactory e0976f42d3 NIFI-3869 Added HTTP/2 support to ListenHTTP and HandleHttpRequest
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6048.
2022-05-19 16:21:53 -04:00
Mark Payne 38b51b0dde
NIFI-10037: When system test fails to clean up flow, destroy the entire environment so that the next test starts in a healthy state. Name troubleshooting directories with the name of the test class to avoid ambiguity. Also added a log statement so that we know which test is running when looking at the log output from the tests themselves. Finally, found an issue in AbstractComponentNode in which we iterate over the elements in a Map and call setProperty, which can update the underlying Map - updated to first create a copy of the HashMap. Updated that in this Jira because I suspect it is causing one of the tests failures that I've been investigating.
This closes #6059

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-05-19 13:14:30 -05:00
Joe Gresock 1cfca0d978
NIFI-10036: Corrected Elasticsearch Client Service to prefix paths for all requests
- Prefixing endpoint paths with a forward slash ensures correct HTTP request formatting required for some deployments with a forwarding proxy

This closes #6058

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-05-19 13:02:40 -05:00
markap14 88e53eae8c
NIFI-10001: Fixed bugs that caused some components to not have their scheduled state updated. (#6049)
* NIFI-10001: Fixed issue in which some components may fail to update the scheduled state when comparing flows

* NIFI-10001: Fixed bugs that caused some components to not have their scheduled state updated. When comparing two flows, now allow specifying how to determine a VersionedComponent's ID for comparison. When comparing local flow against flow from registry, use Versioned Component ID. But when comparing two instantiated flows, such as local flow vs. cluster flow, use the VersionedComponent's Instance ID instead. This ensures that we can properly compare two components even if there are several instances of a given flow
2022-05-18 13:53:36 -04:00
exceptionfactory fe424a2d42
NIFI-9805 Refactored Distributed Cache Servers using Netty
- Added Map and Set Cache Servers based on nifi-event-transport components
- Removed custom servers and unused socket stream components
- Reduced duplication on protocol classes
- Added checks for readable bytes
- Added mark and reset handling for buffer reads

This closes #6040
Signed-off-by: Paul Grey <greyp@apache.org>
2022-05-18 12:40:16 -04:00
tpalfy 2a8d07641e
NIFI-9895 Allow parameters to reference controller services (#5958)
* NIFI-9895 Allow parameter to reference controller service. Check read and write authorization for both previous and newly set controller service. Authorization done for both property or parameter change. Import/export handled by switching between instance id and versioned id.
2022-05-16 17:55:03 -04:00
Lehel eba1318529
NIFI-10028: Added QuerySalesforceObject additionalDetails and improved age field property descriptions
This closes #6047

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-05-16 16:24:20 -05:00
Matthew Burgess bd1e033d55 NIFI-9967: Added FlowSerializationStrategy to determine which flow formats (XML,JSON) to save
This closes #6001.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-05-16 16:22:19 -04:00
Peter Gyori 3f16a41ca1
NIFI-10010: ListenTCP adds Certificate Subject and Issuer FlowFile attributes
This closes #6032

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-05-13 15:48:12 -05:00
markap14 8031b62351
NIFI-10001: When enabling a collection of Controller Services, change… (#6042)
* NIFI-10001: When enabling a collection of Controller Services, changed logic. Instead of enabling dependent services and waiting for them to complete enablement before starting a given service, just start the services given. The previous logic was necessary long ago because we couldn't enable a service unless all dependent services were fully enabled. But that changed a while ago. Now, we can enable a service when it's invalid. It'll just keep trying to enable until it becomes valid. At that point, it will complete its transition from ENABLING to ENABLED.

* NIFI-10001: Restored previous implementation for StandardControllerServiceProvider, as the changes were not ultimately what we needed. Changed StandardProcessGroup to use a ConcurrentHashMap for controller services instead of a HashMap with readLock. This was causing a deadlock when we enable a Controller Service that references another service during flow synchronization. Flow Synchronization was happening within a write lock and enabling the service required a read lock on the group. Eventually the thread holding the write lock would timeout and release the write lock. But this caused significant delays on startup. By changing to a ConcurrentHashMap, we alleviate the need for the Read Lock. Also noticed in testing that the StandardNiFiServiceFacade did not save flow changes when enabling dependent services so added call to controllerFacade.save().
2022-05-13 15:09:23 -04:00
exceptionfactory 30f7c1ba1e
NIFI-9990 This closes #6030. Improved FTP 550 file unavailable handling
- Improved File Not Found reply detection
- Added Permission Denied reply handling

Signed-off-by: Joe Witt <joewitt@apache.org>
2022-05-13 07:44:22 -07:00
Chris Sampson 313d70520d
NIFI-9470 This closes #6027. Allow creation of Parameter Context without any Inherited Parameter Contexts
Signed-off-by: Joe Witt <joewitt@apache.org>
2022-05-13 07:39:11 -07:00
Mark Payne 4a60d1673e
NIFI-10023: Ensure that any files that are archived upon startup properly increment the archival count in the content repo
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6037
2022-05-12 15:38:49 -04:00
Paul Grey a528b89169
NIFI-10003 - NiFi build fails in UTC+ time zones
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6038
2022-05-12 13:55:51 -04:00
Pierre Villard dfc8b9c4c8
NIFI-10009 - bump Kudu client from 1.15 to 1.16
This closes #6029

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-05-10 13:21:07 -05:00
Lehel Boér 8247910a81 NIFI-9865: Add Atlas lineage for GCS processors (PutHDFS)
This closes #5926.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-05-10 11:04:30 +00:00
Tamas Palfy 8aaa3d2f10 NIFI-9991 Fix: The asn1 StandardRecordModelIteratorProvider's interator returns only the first object read from the input stream.
This closes #6019.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-05-09 20:37:41 +00:00
Matthew Burgess 728744a49b
NIFI-9998: This closes #6023. Upgrade Hive3 to 3.1.3
Signed-off-by: Joe Witt <joewitt@apache.org>
2022-05-09 10:53:32 -07:00
Mark Payne b25201dd22
NIFI-9993: Fixed bug in initialization in which the Content Repo did not properly increment the counter for how many files exist in the archive directories. This was causing the counter to become negative in some cases, which caused processors to incorrectly pause, waiting for content archive cleanup to occur when, in fact, there were no files archived
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes #6021.
2022-05-09 12:22:00 -04:00
exceptionfactory 9c2e255fae
NIFI-9986 Upgraded commons-configuration2 to 2.7 for Accumulo
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6014.
2022-05-09 16:53:34 +02:00
exceptionfactory 0c8305950c
NIFI-9987 Standardize Commons IO on version 2.11.0
- Removed version declarations from multiple modules
- Adjusted PutDynamoDB to remove catch for IOException not thrown in Commons IO 2.11.0

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6015.
2022-05-09 12:06:37 +02:00