Commit Graph

5071 Commits

Author SHA1 Message Date
exceptionfactory e0976f42d3 NIFI-3869 Added HTTP/2 support to ListenHTTP and HandleHttpRequest
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6048.
2022-05-19 16:21:53 -04:00
Mark Payne 38b51b0dde
NIFI-10037: When system test fails to clean up flow, destroy the entire environment so that the next test starts in a healthy state. Name troubleshooting directories with the name of the test class to avoid ambiguity. Also added a log statement so that we know which test is running when looking at the log output from the tests themselves. Finally, found an issue in AbstractComponentNode in which we iterate over the elements in a Map and call setProperty, which can update the underlying Map - updated to first create a copy of the HashMap. Updated that in this Jira because I suspect it is causing one of the tests failures that I've been investigating.
This closes #6059

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-05-19 13:14:30 -05:00
Joe Gresock 1cfca0d978
NIFI-10036: Corrected Elasticsearch Client Service to prefix paths for all requests
- Prefixing endpoint paths with a forward slash ensures correct HTTP request formatting required for some deployments with a forwarding proxy

This closes #6058

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-05-19 13:02:40 -05:00
markap14 88e53eae8c
NIFI-10001: Fixed bugs that caused some components to not have their scheduled state updated. (#6049)
* NIFI-10001: Fixed issue in which some components may fail to update the scheduled state when comparing flows

* NIFI-10001: Fixed bugs that caused some components to not have their scheduled state updated. When comparing two flows, now allow specifying how to determine a VersionedComponent's ID for comparison. When comparing local flow against flow from registry, use Versioned Component ID. But when comparing two instantiated flows, such as local flow vs. cluster flow, use the VersionedComponent's Instance ID instead. This ensures that we can properly compare two components even if there are several instances of a given flow
2022-05-18 13:53:36 -04:00
exceptionfactory fe424a2d42
NIFI-9805 Refactored Distributed Cache Servers using Netty
- Added Map and Set Cache Servers based on nifi-event-transport components
- Removed custom servers and unused socket stream components
- Reduced duplication on protocol classes
- Added checks for readable bytes
- Added mark and reset handling for buffer reads

This closes #6040
Signed-off-by: Paul Grey <greyp@apache.org>
2022-05-18 12:40:16 -04:00
tpalfy 2a8d07641e
NIFI-9895 Allow parameters to reference controller services (#5958)
* NIFI-9895 Allow parameter to reference controller service. Check read and write authorization for both previous and newly set controller service. Authorization done for both property or parameter change. Import/export handled by switching between instance id and versioned id.
2022-05-16 17:55:03 -04:00
Lehel eba1318529
NIFI-10028: Added QuerySalesforceObject additionalDetails and improved age field property descriptions
This closes #6047

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-05-16 16:24:20 -05:00
Matthew Burgess bd1e033d55 NIFI-9967: Added FlowSerializationStrategy to determine which flow formats (XML,JSON) to save
This closes #6001.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-05-16 16:22:19 -04:00
Peter Gyori 3f16a41ca1
NIFI-10010: ListenTCP adds Certificate Subject and Issuer FlowFile attributes
This closes #6032

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-05-13 15:48:12 -05:00
markap14 8031b62351
NIFI-10001: When enabling a collection of Controller Services, change… (#6042)
* NIFI-10001: When enabling a collection of Controller Services, changed logic. Instead of enabling dependent services and waiting for them to complete enablement before starting a given service, just start the services given. The previous logic was necessary long ago because we couldn't enable a service unless all dependent services were fully enabled. But that changed a while ago. Now, we can enable a service when it's invalid. It'll just keep trying to enable until it becomes valid. At that point, it will complete its transition from ENABLING to ENABLED.

* NIFI-10001: Restored previous implementation for StandardControllerServiceProvider, as the changes were not ultimately what we needed. Changed StandardProcessGroup to use a ConcurrentHashMap for controller services instead of a HashMap with readLock. This was causing a deadlock when we enable a Controller Service that references another service during flow synchronization. Flow Synchronization was happening within a write lock and enabling the service required a read lock on the group. Eventually the thread holding the write lock would timeout and release the write lock. But this caused significant delays on startup. By changing to a ConcurrentHashMap, we alleviate the need for the Read Lock. Also noticed in testing that the StandardNiFiServiceFacade did not save flow changes when enabling dependent services so added call to controllerFacade.save().
2022-05-13 15:09:23 -04:00
exceptionfactory 30f7c1ba1e
NIFI-9990 This closes #6030. Improved FTP 550 file unavailable handling
- Improved File Not Found reply detection
- Added Permission Denied reply handling

Signed-off-by: Joe Witt <joewitt@apache.org>
2022-05-13 07:44:22 -07:00
Chris Sampson 313d70520d
NIFI-9470 This closes #6027. Allow creation of Parameter Context without any Inherited Parameter Contexts
Signed-off-by: Joe Witt <joewitt@apache.org>
2022-05-13 07:39:11 -07:00
Mark Payne 4a60d1673e
NIFI-10023: Ensure that any files that are archived upon startup properly increment the archival count in the content repo
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6037
2022-05-12 15:38:49 -04:00
Paul Grey a528b89169
NIFI-10003 - NiFi build fails in UTC+ time zones
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6038
2022-05-12 13:55:51 -04:00
Pierre Villard dfc8b9c4c8
NIFI-10009 - bump Kudu client from 1.15 to 1.16
This closes #6029

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-05-10 13:21:07 -05:00
Lehel Boér 8247910a81 NIFI-9865: Add Atlas lineage for GCS processors (PutHDFS)
This closes #5926.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-05-10 11:04:30 +00:00
Tamas Palfy 8aaa3d2f10 NIFI-9991 Fix: The asn1 StandardRecordModelIteratorProvider's interator returns only the first object read from the input stream.
This closes #6019.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-05-09 20:37:41 +00:00
Matthew Burgess 728744a49b
NIFI-9998: This closes #6023. Upgrade Hive3 to 3.1.3
Signed-off-by: Joe Witt <joewitt@apache.org>
2022-05-09 10:53:32 -07:00
Mark Payne b25201dd22
NIFI-9993: Fixed bug in initialization in which the Content Repo did not properly increment the counter for how many files exist in the archive directories. This was causing the counter to become negative in some cases, which caused processors to incorrectly pause, waiting for content archive cleanup to occur when, in fact, there were no files archived
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes #6021.
2022-05-09 12:22:00 -04:00
exceptionfactory 9c2e255fae
NIFI-9986 Upgraded commons-configuration2 to 2.7 for Accumulo
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6014.
2022-05-09 16:53:34 +02:00
exceptionfactory 0c8305950c
NIFI-9987 Standardize Commons IO on version 2.11.0
- Removed version declarations from multiple modules
- Adjusted PutDynamoDB to remove catch for IOException not thrown in Commons IO 2.11.0

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6015.
2022-05-09 12:06:37 +02:00
exceptionfactory 5e8b4d3144
NIFI-9997 Upgraded Ranger from 2.1.0 to 2.2.0 (#6022)
- Upgraded Hadoop from 3.1.1 to 3.3.2 for Ranger Plugins
2022-05-06 16:07:53 -04:00
exceptionfactory ea75a0a996 NIFI-9995 Replaced Custom Filters with Spring Security HeaderWriter
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6020.
2022-05-06 12:22:30 -04:00
exceptionfactory 64f9b66141
NIFI-9984 Allow 200-series responses in OAuth2 Access Token Provider
This closes #6016

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-05-04 17:31:07 -04:00
Emilio Setiadarma e20aa0ea2a
NIFI-9755 Added ConsumeTwitter Processor
This closes #5947

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-05-03 21:36:48 -05:00
exceptionfactory 0f6679f2df
NIFI-9980 Corrected conflicting GCP dependencies
- Replaced google-cloud-bom 0.172.0 with libraries-bom 25.2.0 in nifi-gcp-bundle
- Removed specific versions from Google dependencies in nifi-gcp-processors and nifi-gcp-services-api

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6010.
2022-05-03 13:51:34 +02:00
Matthew Burgess 04121c9d6a
NIFI-8533: Replace deprecated jython-shaded with jython-slim
This closes #5531

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-05-02 15:47:23 -05:00
Matt Gilman e175b146cd NIFI-9978:
- Updating the condition under which we show the Download Flow menu items.

This closes #6008.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-05-02 19:40:48 +02:00
Juldrixx b04632129f
NIFI-9798 Added Proxy Support for ConsumeGCPubSub and PublishGCPubSub
This closes #5868

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-05-02 11:37:47 -05:00
exceptionfactory 21b263bd71
NIFI-9979 Removed nifi-druid-bundle and associated components
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6009.
2022-04-30 16:43:36 +02:00
Tamas Palfy 5aa71570ff
NIFI-9977 In StandardOauth2AccessTokenProvider add new property to be able to set "scope".
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6006.
2022-04-30 16:41:27 +02:00
Mike Thomsen 7271e8cea7
NIFI-7234 Standardized on Avro 1.11.0
NIFI-7234 Replaced Jackson 1.X references with Jackson 2.X references in various classes.
NIFI-7234 Added jackson-annotations to nifi-hdfs-processors.
NIFI-7234 Various updates to bring our test cases into better alignment with the Avro specification as of 1.11.
Fixed a checkstyle issue.

NIFI-7234 Made changes requested in a review.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5900
2022-04-29 17:48:59 -04:00
exceptionfactory fc2f539c7d
NIFI-9976 Upgraded json-smart to 2.4.8
- Replaced nifi-framework-bundle managed dependency to root managed dependency

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6005.
2022-04-29 10:12:41 +02:00
Lehel 569929269a NIFI-9956: Fix QuerySalesforceObject schema parsing with nested field strategy
This closes #5998.

Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
2022-04-28 17:17:24 +02:00
exceptionfactory ac2d7d1e54
NIFI-9968 Added null check before System.setProperty() in test methods
- Resolves build failures on Java 17 where the original user.timezone property returns null from System.getProperty()

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6000
2022-04-27 14:38:33 -04:00
exceptionfactory 8b08795e18
NIFI-9944 Configured maven-dependency-plugin for JavaScript test JAR
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5997
2022-04-27 09:52:47 -04:00
Matthew Burgess 4b66299cf6
NIFI-9944: Fixed issue with using modules in InvokeScriptedProcessor 2022-04-27 09:52:44 -04:00
Peter Gyori 6ee3d32ca3 NIFI-9918: 'Parse XML Attributes' property added to XMLReader
NIFI-9918: Fixed review findings
NIFI-9918: Example added to XMLReader's additionalDetails.html on the new property. Minor documentation fixes.

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5964.
2022-04-26 22:07:59 -04:00
exceptionfactory 56fecbb7d8 NIFI-9955 Upgraded ZooKeeper 3.4.14 to 3.5.9
This closes #5995.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-04-24 15:46:55 +02:00
Malthe Borch 83a9e965d2 NIFI-9888 Fix Azure header datetime format
This fixes the occasional "403 Forbidden" bug that we have seen, simply because
the signature ends up being invalid due to an invalid datetime format used.

Issue: NIFI-9888
Reference: https://stackoverflow.com/a/51636763/647151

Use statically defined formatter and explain why

This closes #5943.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-04-23 22:11:40 -04:00
exceptionfactory 4f423a59ba
NIFI-9952 Upgraded Jackson BOM to 2.13.2.20220328
- Removed unnecessary references to jackson.version property
- Removed unnecessary dependency management references to Jackson libraries

This closes #5992

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-04-23 08:23:39 -04:00
exceptionfactory 4b7f8cbe1f
NIFI-9949 This closes #5988. Corrected intermittent failures in TestPutSFTP
- Changed SSH server to start and stop after each method
- Replaced queued file with string FlowFile contents
- Refactored TestPutSFTP using JUnit 5

Signed-off-by: Joe Witt <joewitt@apache.org>
2022-04-21 12:23:07 -07:00
exceptionfactory b288810316
NIFI-9943 Added Transform Provider to nifi-xml-processing
- Refactored TransformerFactory references using StandardTransformerProvider

This closes #5986
Signed-off-by: Paul Grey <greyp@apache.org>
2022-04-21 15:14:54 -04:00
Paul Grey 4450f4ce50
NIFI-9941 - fix handling of ReportingTask stop request
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5985
2022-04-21 14:18:09 -04:00
Mark Payne 5a3c7d1fbd
NIFI-9940: Renamed StandardProcessGroupSynchronizer to StandardVersionedComponentSynchronizer. Added synching for processors, input/output ports, connections, etc. Added unit tests.
This closes #5983
2022-04-21 10:28:21 -04:00
exceptionfactory 1e345db649
NIFI-9942 Removed load test from TestPutUDP
- Refactored TestPutUDP using JUnit 5
- Standardized test method naming

This closes #5984
Signed-off-by: Paul Grey <greyp@apache.org>
2022-04-21 09:45:00 -04:00
Ravinarayan Singh 28c7583bdb
fixed error message reporting (#5979) 2022-04-21 09:12:02 -04:00
Mark Payne aeaec170d5 NIFI-9861: Removed BlockListClassLoader in favor of AllowListClassLoader
NIFI-9861: Fixed stateless-processor-tests assembly to ensure that all necessary libraries were included; removed BlockListClassLoader
NIFI-9861: Fixed issue in which we would list .class files as files that we allow through the AllowListClassLoader but didn't allow them.

This closes #5925.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-04-20 22:49:43 +02:00
Lehel 73ee1a9a1e
NIFI-9720 Added QuerySalesforceObject Processor
This closes #5802

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-04-20 13:59:21 -05:00
simonbence 285a8cbc3c
NIFI-9776 Adding the possibility to export flow definition with referenced services (#5859)
* NIFI-9776 Adding the possibility to export flow definition with referenced services

* NIFI-9776 Refining naming based on code review
2022-04-20 10:29:38 -04:00