Commit Graph

6155 Commits

Author SHA1 Message Date
tpalfy 940bc3056c
NIFI-7954 Wrapping HBase_*_ClientService calls in getUgi().doAs() (#4629)
* NIFI-7954 Wrapping HBase_*_ClientService calls in getUgi().doAs() and taking care of TGT renewal.

* NIFI-7954 Simplified SecurityUtil.callWithUgi a little.

* NIFI-7954 Simplified SecurityUtil.callWithUgi more.

* NIFI-7954 Removed unnecessary code.
2020-11-09 09:00:20 -05:00
Pierre Villard 14ec02f21d
NIFI-7981 - add support for enum type in avro schema
This closes #4648

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-11-05 18:19:55 -05:00
Siyao Meng 9a3a659c44
NIFI-7949: Add tag HCFS to HDFS Processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4633.
2020-11-05 13:21:11 +01:00
Bryan Bende 1366d017b8
NIFI-7968 Ensure the status login user is set in UserGroupInformation after creating a UGI from a Subject
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4643.
2020-11-05 13:18:35 +01:00
Bence Simon 59e00c4b6f NIFI-7959 Handling node disconnection in MonitorActivity processor
- Make reporting in clustered scope to dependent of expected cluster state in order to prevent unexpected flow file emission

This closes #4642.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-11-04 10:09:46 -05:00
Dustin Rodrigues 0805670263
NIFI-7978 - update Homebrew getting started instructions
NIFI-7978 Removed a few more OS X references.

This closes #4647

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-11-04 07:20:21 -05:00
Matthew Burgess 71a5735f63
NIFI-7976: Infer ints in JSON when all values fit in integer type
NIFI-7976: Fixed unit test

This closes #4645

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-11-03 15:43:15 -05:00
Bryan Bende d773521ee0
NIFI-1121 Fix Schema Name and Schema Branch properties
This closes #4512.

Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:38:41 -05:00
mtien 42c2cda9a2
NIFI-1121 Fixed a dependent value check error.
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:51 -05:00
Mark Payne 4b9014b959
NIFI-1121: Updated backend to perform appropriate validation. Added tests. Updated documentation writer. Updated dev guide to explain how PropertyDescriptor.Builder#dependsOn affects validation. Updated JavaDocs for PropertyDescriptor.Builder#dependsOn
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:42 -05:00
mtien 535cab3167
NIFI-1121: Added an additional check for hidden properties to account for transitive dependent properties.
- Added a 'dependent' attribute to determine whether or not to save dependent property values

Co-authored-by: Scott Aslan <scottyaslan@gmail.com>
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:33 -05:00
Mark Payne e2e901b6b9
NIFI-1121: Added property dependencies to MergeContent
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:26 -05:00
mtien 4bd9d7b413
NIFI-1121 Show and hide properties that depend on another property.
Co-authored-by: Scott Aslan <scottyaslan@gmail.com>
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:16 -05:00
Mark Payne f7f336a4b0
NIFI-1121: Added API changes for having one Property depend on another
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:08 -05:00
r65535 d8d9aa9cec Added regex support for attribute header selection on HandleHTTPResponse 2020-11-03 12:22:41 -05:00
mtien 8b78277a45
NIFI-7914 Bumped H2 dependency to 1.4.199.
Bumped icu4j dependency to 60.2.
Replaced jackson-mapper-asl dependency with jackson-databind.
Fixed an error comparing key identities in TestKeyService.
Replaced jackson-mapper-asl ObjectMapper with jackson-databind ObjectMapper in LivySessionController.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4640.
2020-11-03 11:46:29 +01:00
sjyang18 80f49eb7bd NIFI-7406 Added PutAzureCosmosRecord processor for Azure Cosmos DB
This closes #4253

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2020-11-03 01:54:33 -06:00
Peter Turcsanyi 2b461bbf29
NIFI-7452: Support adls_gen2_directory in Atlas reporting task
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4636.
2020-10-30 13:50:53 +01:00
Mark Bean a115894984 NIFI-7963: fix package name for TestZooKeeperStateServer
This closes #4631

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2020-10-28 21:09:10 -05:00
Mark Payne 2232e28052
NIFI-7956: This closes #4626. Added option of rolling back session on error instead of routing to failure for PutKudu
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-10-27 21:23:59 -07:00
Mark Payne c610aab3cb NIFI-7953: Updated ConsumeKafka_2_0/ConsumeKafkaRecord_2_0/ConsumeKafka_2_6/ConsumeKafkaRecord_2_6 to allow separating records by key 2020-10-27 19:04:19 -04:00
Mark Payne 74968991d5 NIFI-7952: Allow RecordPath to be used for specifying the Insertion Operation and the data to be inserted into Kudu 2020-10-27 19:03:35 -04:00
mtien bf962f6227 NIFI-7584 Added OIDC logout mechanism.
Added method to validate the OIDC Access Token for the revoke endpoint.
Created a new callback URI of oidc/logoutCallback to handle certain OIDC logout cases.
Changed method to exchange the Authorization Code for a Login Authentication Token.
Added a new method to exchange the AuthN Code for an Access Token.
Changed method to convert OIDC Token to a Login AuthN Token instead of a NiFi JWT.
Created new OidcServiceGroovyTest class.

NIFI-7584-rebase Added test.

NIFI-7584 Fixed a checkstyle issue.

NIFI-7584 Removed a dependency not in use.

NIFI-7584 Made revisions based on PR review.
Refactored revoke endpoint POST request to a private method.
Removed unnecessary dependencies.
Fixed Regex Pattern to search for literal dot character.
Fixed logging the Exception message.
Fixed caught Exception.
Changed timeout value to a static variable.
Changed repeating error messages to a static string.
Reduced sleep duration in unit test.
Refactored cookie generation to private method.

NIFI-7584 Fixed the snapshot version.

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4593.
2020-10-27 17:18:05 -04:00
Joe Witt c70f4ac530
NIFI-7925 removed flow.xml.gz which made its way into the commit - but full build doesnt need nor can we have 2020-10-27 13:19:16 -07:00
Mark Payne bcf625f4c8 NIFI-7920: When node that is connected to cluster is asked to reconnect, ensure that it relinquishes role of Cluster Coordinator and Primary Node 2020-10-27 14:30:13 -04:00
Bryan Bende e7f244f267
NIFI-7923 Catch Throwable instead of Exception in NarAutoLoaderTask so that thread won't die (#4623) 2020-10-27 12:33:08 -04:00
Matthew Burgess 4fee076561
NIFI-7637: Ensure all array elements are compatible in DataTypeUtils.isArrayTypeCompatible() (#4612) 2020-10-27 11:39:28 -04:00
Bryan Bende 77b4abc21e
NIFI-7984 Use UGI doAs in DeleteHDFS (#4622) 2020-10-27 11:24:48 -04:00
Bryan Bende 3612973a47
NIFI-7929 Resolving conflicts after rebasing on top of NIFI-7719
This clsoes #4610.

Signed-off-by: Bryan Bende <bbende@apache.org>
2020-10-27 10:18:06 -04:00
Denes Arvay dc25d4c39e
NIFI-7929: connectionTimeout and readTimeout options are not exposed in the CLI
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-10-27 10:17:52 -04:00
markap14 67d1b73a85
NIFI-7944: When checking if a Component can reference a Controller Service, NiFi would sometimes fail to find that Controller Service because it was looking based on the wrong Process Group. Fixed that. (#4619) 2020-10-27 08:57:05 -04:00
Bryan Bende 6e9bef1428
NIFI-7719 Initial refactoring to NiFiClient in CLI to support additional authentication mechanisms
Added command options for basic auth and bearer token and updated NiFiClientFactory to create the appropriate RequestConfig

Added get-token command for NiFi

Update NiFi Registry CLI commands based on nifi-registry-client 0.8.0-SNAPSHOT

Implementing commands for NiFi and NiFi Registry to obtain a token via SPNEGO and to logout a given token

Add Keberos JAAS classes for using the ticket cache, update GetAccessTokenSpnego methods to fallback to ticket cache when no keytab or password is specified

Use released 0.8.0 version of registry client

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4461.
2020-10-27 13:44:42 +01:00
Pierre Villard 9c1656dbff NIFI-7854 - Add proxy username/password support to SQS processors
This closes #4600.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-10-27 00:22:15 +01:00
Peter Turcsanyi 3dd024fb66
NIFI-7825: Support native library loading via absolute path
Use an AspectJ aspect and agent to intercept the load native library calls
at runtime, copy the native library file to temp folder and proceed with
the newly created file in order to provide classloader isolation.

Remove AspectJ jars from lib directory, move the necessary jar to lib/aspectj subdirectory.

This closes #4540.

Signed-off-by: Bryan Bende <bbende@apache.org>
2020-10-26 16:21:16 -04:00
Fabio Torchetti 9a11d23c83
NIFI-7943 - Add application properties to GetAzureEventHub
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4617.
2020-10-26 19:36:27 +01:00
Bence Simon c8ea7523ef
NIFI-7942 Fixing ordering issue when counter based metrics are added by component
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4621.
2020-10-26 19:31:05 +01:00
Andrew Lim 718b77c7c4
NIFI-7941 Add NiFi Registry mode options and examples to Encrypt-Conf… (#4616)
* NIFI-7941 Add NiFi Registry mode options and examples to Encrypt-Config section of Toolkit Guide

* Update toolkit-guide.adoc

Replaced references to "master key" with "root key"

* Update toolkit-guide.adoc

Corrected a typo. Anchor was "sensistive_property_key_migration". Changed to "sensitive_property_key_migration". Confirmed nothing was referencing the original.
2020-10-26 09:11:58 -04:00
Bence Simon b980a8ea8c NIFI-7549 Adding Hazelcast based DistributedMapCacheClient support
NIFI-7549 Refining documentation; Changing explicit HA mode; Smaller review comments
NIFI-7549 Code review responses about license, documentation and dependencies
NIFI-7549 Fixing issue when explicit HA; Some further review based adjustments
NIFI-7549 Response to code review comments
NIFI-7549 Adding extra serialization test
NIFI-7549 Minor changes based on review comments
NIFI-7549 Adding hook point to the shutdown

This closes #4510.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-10-22 19:22:49 +02:00
Joe Witt d1bfa67927
NIFI-7939 improving version ranges to fix usage of now deprecated github actions commands
self merging given nature of change

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-10-22 07:16:31 -07:00
Denes Arvay d05d0c6240 NIFI-7925: ValidateRecord reports false positive for avro arrays with null elements 2020-10-21 09:17:29 -04:00
Joey Frazee 55cb8d73cb NIFI-7401 Add ZooKeeper client TLS to CuratorLeaderElectionManager
NIFI-7401 Rebased to 1.13.0-SNAPSHOT and simplified tests

NIFI-7401 Added keystore types and changed properties to match nifi.security.*

NIFI-7401 Removed dead code from SecureClientZooKeeperFactory test

NIFI-7401 Renamed bean methods, moved helper code into NiFiProperties

NIFI-7401 Changed connection socket constants to use .class.getName()

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4592.
2020-10-20 11:20:43 -04:00
eylonronen b442df58e2
NIFI-7935 Add reason to flow file on error
This closes #4608

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-10-16 18:44:27 -04:00
Indrajeet Gour 4a370eae8c
NIFI-7933 - Set proper property descriptor for CSV Escape
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4606.
2020-10-16 18:13:29 +02:00
Dan Kim 5b5b9a7800
NIFI-7916 refactor CountText to ignore whitespace words when counting
This closes #4595

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-10-14 09:09:42 -04:00
Pierre Villard 52f1fe54a5
Merge pull request #4597 from apache/dependabot/maven/nifi-nar-bundles/nifi-ranger-bundle/nifi-ranger-plugin/junit-junit-4.13.1
Bump junit from 4.13 to 4.13.1 in /nifi-nar-bundles/nifi-ranger-bundle/nifi-ranger-plugin
2020-10-13 11:32:11 +02:00
Pierre Villard 2d3b171d63
Merge pull request #4598 from apache/dependabot/maven/junit-junit-4.13.1
Bump junit from 4.13 to 4.13.1
2020-10-13 11:31:36 +02:00
dependabot[bot] 8aa001a918
Bump junit from 4.13 to 4.13.1
Bumps [junit](https://github.com/junit-team/junit4) from 4.13 to 4.13.1.
- [Release notes](https://github.com/junit-team/junit4/releases)
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.13.1.md)
- [Commits](https://github.com/junit-team/junit4/compare/r4.13...r4.13.1)

Signed-off-by: dependabot[bot] <support@github.com>
2020-10-12 21:54:52 +00:00
Matthew Burgess 4c235f0405
NIFI-7909: Change DataTypeUtils.toInteger() to use Math.toIntExact()
This closes #4596

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-10-12 17:53:27 -04:00
dependabot[bot] 99a75e833b
Bump junit in /nifi-nar-bundles/nifi-ranger-bundle/nifi-ranger-plugin
Bumps [junit](https://github.com/junit-team/junit4) from 4.13 to 4.13.1.
- [Release notes](https://github.com/junit-team/junit4/releases)
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.13.1.md)
- [Commits](https://github.com/junit-team/junit4/compare/r4.13...r4.13.1)

Signed-off-by: dependabot[bot] <support@github.com>
2020-10-12 19:17:09 +00:00
Pierre Villard c2960998ac
NIFI-7869 - adding a GCP profile for GCS connector (#4565) 2020-10-09 09:42:20 -04:00