74531fa1ca
NIFI-8348: Updated system tests to reference the version of jersey client used elsewhere, as it was missed in the previous commit for NIFI-8348 and system tests are failing
2021-04-10 10:33:16 -04:00
365dde2d11
NIFI-7573 Added Azure Active Directory User Group Provider
...
This closes #4367
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-04-09 15:06:14 -07:00
d0045c13b1
NIFI-8341 Support Multi Hosts in AMQP Processors
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4939 .
2021-04-09 17:42:24 +02:00
76f33e42c6
NIFI-7198: Adding optional SSL Context Service to RedisConnectionPoolService
...
This closes #4981
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-04-09 10:36:31 -05:00
bfd964b9c7
NIFI-8176 - Move the timeout check after we process the response from Splunk to make sure we poll for acknowledgement at least once. (No need for flag.)
...
This closes #4824 .
Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
2021-04-09 17:22:56 +02:00
14e6dc3dc6
NIFI-8405: Added debug logging around how long it takes to establish connections/query dns/read and write headers and body when replication requests; added additional timing around Ranger audits and authorizations and monitoring of long-running tasks because those run often and frequently show up in the logs at the same time as the long requests
...
This closes #4983
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-04-08 15:41:44 -05:00
78b69f10dc
NIFI-8404: Added capability to rollback on failure for PublishKafka(Record)_2_0 and _2_6
...
NIFI-8404: Added unit tests for new Failure Strategy property
This closes #4982 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-04-08 17:47:13 +02:00
33ec8c8427
NIFI-8400: Use longs in SystemDiagnostics, add null checks
...
This closes #4980 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-04-08 10:06:26 +02:00
77a73a603b
NIFI-8394 Fix NPE error when NiFi connects to external zookeeper insecurely
...
This closes #4974
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-04-07 11:23:25 -07:00
9da3b1ec01
NIFI-7912 - Added properties to configure DoSFilter timeout and whitelisted addresses
...
- Added nifi.web.request.ip.whitelist property to set DoSFilter.ipWhitelist
- Added nifi.web.request.timeout property to set DoSFilter.maxRequestMs with default of 60 seconds
This closes #4972
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-04-07 07:40:03 -05:00
b79987918a
NIFI-8380: Allow for an extensions.directory property to specify where to place downloaded files. Also fixed an issue that was encountered, when a Source Processor is scheduled for Primary Node Only but more than 1 task is set. In that case, even though only a single task will should be scheduled, an Exception was getting thrown because @OnScheduled methods of Processors were still called. To avoid this, moved the initialization of the dataflow outside of the creation of the dataflow so that initialization can be triggered only when appropriate.
...
NIFI-8380: Removed requirement in validation for working directory and extensions directory to exist; removed auto-creation of directories in validation
NIFI-8380: Fixed a few thrading bugs, so that if we have multiple threads trying to download/unpack extensions, we properly synchronize the unpacking and unpack into the correct sub-directory under the working directory
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4950 .
2021-04-07 13:28:12 +02:00
f4ad658fae
NIFI-8389 - Grok Reader - raw data strategy when no match
...
added allowable value in property
added docs in additionalDetails
This closes #4966 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-04-07 09:11:44 +02:00
4b852ba7c8
NIFI-8390: Handle HBase namespaces in Atlas reporting task
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4977 .
2021-04-07 08:57:29 +02:00
ef60e7e404
NIFI-6714 Corrected OSUtils to use Process.pid interface method
...
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes #4970 .
2021-04-06 19:48:05 -04:00
fa3da2aa1c
There was some minor changes required to stop using deprecated APIs byu accumulo, inspecting the deprecated makes the fixes really simple
...
* writerFactory.createWriter receives an extra parameteres with a map, in the implementation it sends an empty one if not passed.
* tests were relying on getConnector, that is replaced by createAccumuloClient
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4942 .
2021-04-06 11:10:01 +02:00
64b12176b2
NIFI-8388 Upgrade Hazelcast from 4.0.1 to 4.2
...
- Removes dependency on Guava using ByteBuffer methods for serializing revision
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4967 .
2021-04-05 11:46:32 +02:00
f21c1be60f
NIFI-8374 Refactor readOnly to accessMode in ApiModelProperty usages
...
This closes #4945
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2021-04-02 14:22:07 -04:00
87b561f026
[NIFI-8387] - Use the bulletins from the referencing components rathe… ( #4969 )
...
* [NIFI-8387] - Use the bulletins from the referencing components rather than making extraneous http calls to get them.
* Moving the spinner next to the Parameter/Variable lables
This closes #4969
2021-04-02 09:59:34 -04:00
7db1b8d564
NIFI-8386: Ensure that we set (and merge) bulletins when creating AffectedComponent entities and ControllerService Reference entities
...
NIFI-8386: Addressed review feedback: removed unused call to determine permissions, null out bulletins in standalone mode if permissions not allowed. Also fixed automated tests that were failing due to changes
This closes #4955
2021-04-01 14:11:32 -04:00
2edf5514b7
NIFI-8344: If configured to tail a file for some period of time post-rollover, ensure that we only consume lines that are fully written (i.e., end with a newline). Once we stop tailing that file post-rollover, consume any data from that file that has not yet been consumed, up to the end of the file, even if there is no newline.
...
This closes #4937 .
Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
2021-03-31 13:40:08 +02:00
89dda0b6a4
NIFI-8364 This closes #4953 . Set static date for TestQuerySolr
...
- Static date for TestQuerySolr avoids the potential for missing values in facet range queries
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-03-30 16:05:04 -07:00
ee1386db02
NIFI-8317 heap and non_heap usage Prometheus metrics now return -1.0 when undefined.
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4920
2021-03-30 11:06:52 -04:00
02ea103c6b
NIFI-8379 This closes #4949 . Improved reliability and performance of DBCPServiceTest
...
- Removed unreliable H2 network connection test methods
- Removed duplicate and ignored test methods
- Reduced duplication of setting common service properties
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-03-30 07:24:35 -07:00
a5dbf56114
NIFI-8368: If decimal scale > precision, set precision = scale
...
This closes #4938
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-03-29 10:53:46 -05:00
bff3e94c01
NIFI-7172 Trim trailing whitespace from NiFi properties
...
This closes #4854
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-03-25 15:40:35 -05:00
1cd3fbb4eb
NIFI-8288 Removed OkHttpClientUtils to reduce reliance on nifi-security-utils
...
- Added createTrustManager() on SSLContextService
- Removed nifi-security-utils and okhttp dependencies from nifi-web-utils
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes #4869 .
2021-03-25 15:38:24 -04:00
66e2741871
NIFI-5535 NIFI-4713 This closes #4858 . Change metric tagging in DataDogReportingTask
...
Remove low-value default tags
Send processor name as tag instead of as part of metric name
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-03-25 09:59:59 -07:00
54d2222fb0
NIFI-8364 This closes #4935 . Refactored TestQuerySolr and TestGetSolr to reuse embedded SolrClient
...
- Replaced Gson with Jackson for parsing JSON
- Corrected assertion argument ordering
- Simplified relative path determination for EmbeddedSolrServerFactory
- Replaced SimpleDateFormat with java.time.Instant parsing and formatting
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-03-25 09:33:19 -07:00
36e0187648
NIFI-8347 Set Thread Context ClassLoader for provided jetty-server classes
...
- Updated unit test with WebSocket connect method
NIFI-8347 Replaced init() method with ServletContextHandler.setClassLoader()
This closes #4918 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-03-25 15:51:22 +01:00
9df717f69b
NIFI-8283 Value handling in ScanAccumulo processor
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4931 .
2021-03-25 13:00:04 +01:00
0f28702b47
NIFI-8354 ExecuteStreamCommand processor doesn't delete the temp file… ( #4923 )
...
* NIFI-8354 ExecuteStreamCommand processor doesn't delete the temp file if the process start failed
* NIFI-8354 Record the log when delete file failed
This closes #4923
Signed-off-by: Otto Fowler <otto@apache.org>
2021-03-24 12:26:56 -04:00
4473d23ccd
NIFI-8356: Add unit test for LongRunningTaskMonitor.
...
This closes #4925
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-03-24 10:29:59 -05:00
91313a2e75
NIFI-8360: Fixed an overflow issue where we used an integer to store the number of bytes encountered when reading data and searching for a given pattern
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4929 .
2021-03-24 15:30:22 +01:00
74ea3840ac
NIFI-8357: Updated Kafka 2.6 processors to automatically handle recreating Consumer Lease objects when an existing one is poisoned, even if using statically assigned partitions
...
This closes #4926 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-03-24 14:59:13 +01:00
2f08d1f466
NIFI-8357: Updated Kafka 2.0 processors to automatically handle recreating Consumer Lease objects when an existing one is poisoned, even if using statically assigned partitions
...
This closes #4926 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-03-24 14:58:51 +01:00
057b4af482
NIFI-8296 - Use API to retrieve all subjects associated with a schema id for Confluent Schema Registry v5.4.0+
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4872 .
2021-03-24 14:23:41 +01:00
1719e36165
NIFI-8348: upgrade jersey version to one fully compatible with Java 11
2021-03-24 09:18:31 -04:00
7fe8858d22
NIFI-8359 resolving unused import
2021-03-23 14:30:28 -07:00
788abe9eb3
NIFI-8359 removing test that cannot be reliable. Random data doesn't guarantee it will be detected as octet-stream and there is no obvious 'will never match' set of bytes so this test is not reliable or valuable
2021-03-23 17:18:54 -04:00
22fcb594d9
NIFI-8346 Route to failure in PutAzureBlobStorage for auth, IO exceptions
...
This closes #4917
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-03-23 11:28:04 -07:00
321e979673
NIFI-8353: When receiving data via load-balanced connection, throw an Exception (resulting in an ABORT_TRANSACTION status code) when attempting to add received FlowFiles to the FlowFile queue, if the node is not currently connected to cluster. In this case, ensure that we remove the received FlowFiles from the FlowFile Repository and emit a DROP event to the repository (with appropriate details) to coincide with the already-committed Provenance events. Also ensure that when a node is disconnected that we don't keep sending to that node until it reconnects. While testing the fixes via LoadBalancedQueueIT, also noticed that unit test logging was not working properly in nifi-framework-core because of the (erroneous) dependency on logback-class; removed this dependency and updated unit tests that were created that depended on its existence. ( #4924 )
2021-03-23 14:17:54 -04:00
e505e8b42d
NIFI-8352 This closes #4922 . Changed assertServerConnections to loop and sleep while evaluating
...
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-03-23 07:15:48 -07:00
4189a1c1eb
NIFI-8350 This closes 4921. updated worfklow
...
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-03-23 07:09:59 -07:00
e1b9548ab6
NIFI-8344: Introduced new Rollover Tail Period property
...
This closes #4916 .
Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
2021-03-22 18:22:00 +01:00
3508b5b12f
NIFI-7850 Adds nifi.cluster.protocol.is.secure to Docker secure.sh
...
This closes #4896
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-03-19 15:30:23 -07:00
07a4966d10
NIFI-8329 - Updated dependencies with no build failures
...
NIFI-8329 - Removed unnecessary jackson.version from azure bundle to use the global property instead.
NIFI-8329 - Updated jackson/jackson-databind version and removed the 'jackson-databind.version' pom property in favor of 'jackson.version'
Updated dependencies include the following:
- jackson-core
- jackson-databind
- icu4j
- snakeyaml
- spring-integration-mail
- spring-core and framework modules
- activemq-client
- activemq-broker
- xercesImpl
This closes #4911
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-03-19 14:46:33 -05:00
0e659981d4
NIFI-8298 Refactored Kerberos and Socket classes from security-utils to new modules
...
- Created nifi-security-socket-ssl
- Created nifi-security-kerberos
- Removed nifi-security-utils dependency from nifi-processor-utils
- Updated modules to reference new dependencies
- Eliminated unnecessary transitive dependencies on bcprov-jdk15on from over 30 modules
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes #4881 .
2021-03-19 13:12:19 -04:00
4d68509212
NIFI-8308 Remove unnecessary field adjustments from QuestDB row creation
...
This closes #4888 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-03-19 17:55:11 +01:00
fedeafb43f
NIFI-8300 Make non-source processors invalid if scheduled for Primary Node only
...
This closes #4913 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-03-19 10:48:29 -04:00
446401b6f1
NIFI-8328: Allow RestLookupService to use FlowFile attributes in header properties
...
This closes #4908
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-03-19 09:26:00 -05:00
Mark Payne
sjyang18
nabmoh123
Joe Gresock
Timea Barna
Matthew Burgess
Mohammed Nadeem
Nathan Gough
Pierre Villard
Peter Turcsanyi
exceptionfactory
Jose Luis Pedrosa
Rob Fellows
noblenumbat360
Eric Olson
Florimond Manca
Hsin-Ying Lee
Dmitry Ibragimov
Mark Bean
Joe Witt
Denes Arvay
Karthick Narendran
Bence Simon