Commit Graph

4078 Commits

Author SHA1 Message Date
tpalfy 940bc3056c
NIFI-7954 Wrapping HBase_*_ClientService calls in getUgi().doAs() (#4629)
* NIFI-7954 Wrapping HBase_*_ClientService calls in getUgi().doAs() and taking care of TGT renewal.

* NIFI-7954 Simplified SecurityUtil.callWithUgi a little.

* NIFI-7954 Simplified SecurityUtil.callWithUgi more.

* NIFI-7954 Removed unnecessary code.
2020-11-09 09:00:20 -05:00
Pierre Villard 14ec02f21d
NIFI-7981 - add support for enum type in avro schema
This closes #4648

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-11-05 18:19:55 -05:00
Siyao Meng 9a3a659c44
NIFI-7949: Add tag HCFS to HDFS Processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4633.
2020-11-05 13:21:11 +01:00
Bryan Bende 1366d017b8
NIFI-7968 Ensure the status login user is set in UserGroupInformation after creating a UGI from a Subject
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4643.
2020-11-05 13:18:35 +01:00
Bence Simon 59e00c4b6f NIFI-7959 Handling node disconnection in MonitorActivity processor
- Make reporting in clustered scope to dependent of expected cluster state in order to prevent unexpected flow file emission

This closes #4642.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-11-04 10:09:46 -05:00
Matthew Burgess 71a5735f63
NIFI-7976: Infer ints in JSON when all values fit in integer type
NIFI-7976: Fixed unit test

This closes #4645

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-11-03 15:43:15 -05:00
Bryan Bende d773521ee0
NIFI-1121 Fix Schema Name and Schema Branch properties
This closes #4512.

Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:38:41 -05:00
mtien 42c2cda9a2
NIFI-1121 Fixed a dependent value check error.
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:51 -05:00
Mark Payne 4b9014b959
NIFI-1121: Updated backend to perform appropriate validation. Added tests. Updated documentation writer. Updated dev guide to explain how PropertyDescriptor.Builder#dependsOn affects validation. Updated JavaDocs for PropertyDescriptor.Builder#dependsOn
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:42 -05:00
mtien 535cab3167
NIFI-1121: Added an additional check for hidden properties to account for transitive dependent properties.
- Added a 'dependent' attribute to determine whether or not to save dependent property values

Co-authored-by: Scott Aslan <scottyaslan@gmail.com>
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:33 -05:00
Mark Payne e2e901b6b9
NIFI-1121: Added property dependencies to MergeContent
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:26 -05:00
mtien 4bd9d7b413
NIFI-1121 Show and hide properties that depend on another property.
Co-authored-by: Scott Aslan <scottyaslan@gmail.com>
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:16 -05:00
Mark Payne f7f336a4b0
NIFI-1121: Added API changes for having one Property depend on another
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:08 -05:00
r65535 d8d9aa9cec Added regex support for attribute header selection on HandleHTTPResponse 2020-11-03 12:22:41 -05:00
mtien 8b78277a45
NIFI-7914 Bumped H2 dependency to 1.4.199.
Bumped icu4j dependency to 60.2.
Replaced jackson-mapper-asl dependency with jackson-databind.
Fixed an error comparing key identities in TestKeyService.
Replaced jackson-mapper-asl ObjectMapper with jackson-databind ObjectMapper in LivySessionController.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4640.
2020-11-03 11:46:29 +01:00
sjyang18 80f49eb7bd NIFI-7406 Added PutAzureCosmosRecord processor for Azure Cosmos DB
This closes #4253

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2020-11-03 01:54:33 -06:00
Peter Turcsanyi 2b461bbf29
NIFI-7452: Support adls_gen2_directory in Atlas reporting task
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4636.
2020-10-30 13:50:53 +01:00
Mark Bean a115894984 NIFI-7963: fix package name for TestZooKeeperStateServer
This closes #4631

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2020-10-28 21:09:10 -05:00
Mark Payne 2232e28052
NIFI-7956: This closes #4626. Added option of rolling back session on error instead of routing to failure for PutKudu
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-10-27 21:23:59 -07:00
Mark Payne c610aab3cb NIFI-7953: Updated ConsumeKafka_2_0/ConsumeKafkaRecord_2_0/ConsumeKafka_2_6/ConsumeKafkaRecord_2_6 to allow separating records by key 2020-10-27 19:04:19 -04:00
Mark Payne 74968991d5 NIFI-7952: Allow RecordPath to be used for specifying the Insertion Operation and the data to be inserted into Kudu 2020-10-27 19:03:35 -04:00
mtien bf962f6227 NIFI-7584 Added OIDC logout mechanism.
Added method to validate the OIDC Access Token for the revoke endpoint.
Created a new callback URI of oidc/logoutCallback to handle certain OIDC logout cases.
Changed method to exchange the Authorization Code for a Login Authentication Token.
Added a new method to exchange the AuthN Code for an Access Token.
Changed method to convert OIDC Token to a Login AuthN Token instead of a NiFi JWT.
Created new OidcServiceGroovyTest class.

NIFI-7584-rebase Added test.

NIFI-7584 Fixed a checkstyle issue.

NIFI-7584 Removed a dependency not in use.

NIFI-7584 Made revisions based on PR review.
Refactored revoke endpoint POST request to a private method.
Removed unnecessary dependencies.
Fixed Regex Pattern to search for literal dot character.
Fixed logging the Exception message.
Fixed caught Exception.
Changed timeout value to a static variable.
Changed repeating error messages to a static string.
Reduced sleep duration in unit test.
Refactored cookie generation to private method.

NIFI-7584 Fixed the snapshot version.

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4593.
2020-10-27 17:18:05 -04:00
Mark Payne bcf625f4c8 NIFI-7920: When node that is connected to cluster is asked to reconnect, ensure that it relinquishes role of Cluster Coordinator and Primary Node 2020-10-27 14:30:13 -04:00
Bryan Bende e7f244f267
NIFI-7923 Catch Throwable instead of Exception in NarAutoLoaderTask so that thread won't die (#4623) 2020-10-27 12:33:08 -04:00
Bryan Bende 77b4abc21e
NIFI-7984 Use UGI doAs in DeleteHDFS (#4622) 2020-10-27 11:24:48 -04:00
markap14 67d1b73a85
NIFI-7944: When checking if a Component can reference a Controller Service, NiFi would sometimes fail to find that Controller Service because it was looking based on the wrong Process Group. Fixed that. (#4619) 2020-10-27 08:57:05 -04:00
Pierre Villard 9c1656dbff NIFI-7854 - Add proxy username/password support to SQS processors
This closes #4600.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-10-27 00:22:15 +01:00
Peter Turcsanyi 3dd024fb66
NIFI-7825: Support native library loading via absolute path
Use an AspectJ aspect and agent to intercept the load native library calls
at runtime, copy the native library file to temp folder and proceed with
the newly created file in order to provide classloader isolation.

Remove AspectJ jars from lib directory, move the necessary jar to lib/aspectj subdirectory.

This closes #4540.

Signed-off-by: Bryan Bende <bbende@apache.org>
2020-10-26 16:21:16 -04:00
Fabio Torchetti 9a11d23c83
NIFI-7943 - Add application properties to GetAzureEventHub
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4617.
2020-10-26 19:36:27 +01:00
Bence Simon c8ea7523ef
NIFI-7942 Fixing ordering issue when counter based metrics are added by component
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4621.
2020-10-26 19:31:05 +01:00
Bence Simon b980a8ea8c NIFI-7549 Adding Hazelcast based DistributedMapCacheClient support
NIFI-7549 Refining documentation; Changing explicit HA mode; Smaller review comments
NIFI-7549 Code review responses about license, documentation and dependencies
NIFI-7549 Fixing issue when explicit HA; Some further review based adjustments
NIFI-7549 Response to code review comments
NIFI-7549 Adding extra serialization test
NIFI-7549 Minor changes based on review comments
NIFI-7549 Adding hook point to the shutdown

This closes #4510.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-10-22 19:22:49 +02:00
Denes Arvay d05d0c6240 NIFI-7925: ValidateRecord reports false positive for avro arrays with null elements 2020-10-21 09:17:29 -04:00
Joey Frazee 55cb8d73cb NIFI-7401 Add ZooKeeper client TLS to CuratorLeaderElectionManager
NIFI-7401 Rebased to 1.13.0-SNAPSHOT and simplified tests

NIFI-7401 Added keystore types and changed properties to match nifi.security.*

NIFI-7401 Removed dead code from SecureClientZooKeeperFactory test

NIFI-7401 Renamed bean methods, moved helper code into NiFiProperties

NIFI-7401 Changed connection socket constants to use .class.getName()

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4592.
2020-10-20 11:20:43 -04:00
eylonronen b442df58e2
NIFI-7935 Add reason to flow file on error
This closes #4608

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-10-16 18:44:27 -04:00
Indrajeet Gour 4a370eae8c
NIFI-7933 - Set proper property descriptor for CSV Escape
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4606.
2020-10-16 18:13:29 +02:00
Dan Kim 5b5b9a7800
NIFI-7916 refactor CountText to ignore whitespace words when counting
This closes #4595

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-10-14 09:09:42 -04:00
dependabot[bot] 8aa001a918
Bump junit from 4.13 to 4.13.1
Bumps [junit](https://github.com/junit-team/junit4) from 4.13 to 4.13.1.
- [Release notes](https://github.com/junit-team/junit4/releases)
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.13.1.md)
- [Commits](https://github.com/junit-team/junit4/compare/r4.13...r4.13.1)

Signed-off-by: dependabot[bot] <support@github.com>
2020-10-12 21:54:52 +00:00
Matthew Burgess 4c235f0405
NIFI-7909: Change DataTypeUtils.toInteger() to use Math.toIntExact()
This closes #4596

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-10-12 17:53:27 -04:00
Pierre Villard c2960998ac
NIFI-7869 - adding a GCP profile for GCS connector (#4565) 2020-10-09 09:42:20 -04:00
kpsngh1998 a95ffce6bd
NIFI-7534 Consume MQTT Processor. Support Expression Language for Topic Filter property
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4573.
2020-10-08 15:38:37 +02:00
Andy LoPresto 82d3fd6cd6 NIFI-7898 Fixed unit test which failed with simultaneous services running on localhost:80. 2020-10-08 08:58:14 -04:00
Peter Gyori 668c278e30
NIFI-7815: Enhance the logging in MergeContent processor
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4534.
2020-10-08 11:44:52 +02:00
exceptionfactory efb629e37d
NIFI-7777 Removed support for Expression Language from Password property
Added unit test for no password configured on Zip files

This closes #4572.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-10-07 10:59:50 -07:00
exceptionfactory ea6b01159d
NIFI-7777 Added optional Password property to UnpackContent for decrypting Zip archives
This closes #4572.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-10-07 10:56:23 -07:00
M Tien de39a3d9f4
NIFI-7892 Created a Logout page to inform users of a complete logout when OIDC is configured. (#4579) 2020-10-07 12:17:00 -04:00
Matthew Burgess 325a49556c
NIFI-7592: Allow NiFi to be started without a GUI/REST interface (#4509)
* NIFI-7592: Allow NiFi to be started without a GUI/REST interface

* NIFI-7592: Enable all controller services when starting headless

* NIFI-7592: Marked duplicate dependencies as provided

* NIFI-7592: Incorporated additional review comments
2020-10-07 09:12:16 -04:00
Matthew Burgess 7cc3713389
NIFI-7796: Add Prometheus counters for total bytes sent/received (#4522)
* NIFI-7796: Add Prometheus metrics for total bytes sent/received, fixed read/written metrics

* NIFI-7796: Incorporated review comments
2020-10-06 09:26:27 -04:00
Ofirmgr be72dbb815
rename onConfigured to OnEnabled to make code more clear
This closes #4547

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-10-04 14:55:57 -04:00
Mark Payne 5d37809d30
NIFI-7874: Fixed bug that results in buffering all S3 elements in heap before emitting any of them. Added Batch Size property to ListS3
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4566.
2020-10-02 19:12:40 +02:00
Denes Arvay 6990f0d3a9
NIFI-7685: Add UTF8 support for FetchFTP
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4446
2020-10-01 17:35:46 -04:00