Updated PutHive3Streaming to wrap calls to Hive in UGI.doAs methods
Fixed misleading logging message after the principal has been authenticated with the KDC
When connecting to unsecured Hive 3, a UGI with "simple" auth will be used
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#4108
Updating validation logic to be consistent with other password-based kerberos processors, removing getPassword from KerberosPasswordUser as it was only used from testing
This closes#4062.
Kerberos Password property should not support EL, this includes a change to KerberosProperties which is also used by the HDFS processors (AbstractHadoopProcessor)
Added wiring in a KerberosContext to a TestRunner's MockProcessorInitializationContext
Removed synchronization blocks around KerberosUser.checkTGTAndRelogin, since that method is already synchronized
Updated AbstractHadoopProcessor to have a boolean accessor method to determine if explicit keytab configuration is allowed
Removed synchronization block from HiveConnectionPool's getConnection method (in Hive, Hive_1_1, Hive3 modules), since new TGT ticket acquisition is handled by the KerberosUser implementation. If UGI is used to relogin, synchronization is handled internally by UGI.
Added Kerberos Principal and Kerberos Password properties to Hive, Hive_1_1, and Hive3 components
Hive, Hive_1_1, and Hive3 components now use KerberosUser implementations to authenticate with a KDC
Updated handling of the NIFI_ALLOW_EXPLICIT_KEYTAB environment variable in Hive and Hive3 components. An accessor method has been added that uses Boolean.parseBoolean, which returns true if the environment variable is set to true, and false otherwise (including when the environment variable is unset).
Addressing PR feedback
Addressing PR feedback
This closes#4102.
AbstractHadoopProcessor will always authenticate the principal with a KerberosUser implementation and a UGI will be acquired from the Subject associated with the KerberosUser implementation
AbstractHadoopProcessor's getUserGroupInformation method will now attempt to check the TGT and relogin if a KerberosUser impelmentation is available, otherwise it will return the UGI referenced in the HdfsResource instance
Updated AbstractHadoopProcessor's customValidate method to consider the provided password and updated validation failure explanations when a KerberosCredentialsService is specified together with a principal, password, or keytab
Added toString method override to AbstractKerberosUser
Updated Hive/HBase components to be compatible with the KerberosProperties.validatePrincipalWithKeytabOrPassword method
Fixed null ComponentLog in GetHDFSSequenceFileTest
Added package-protected accessor method (getAllowExplicitKeytabEnvironmentVariable) to AbstractHadoopProcessor for determining if the environment variable "NIFI_ALLOW_EXPLICIT_KEYTAB" has been set
AbstractHadoopProcessor will now only fail validation when the NIFI_ALLOW_EXPLICIT_KEYTAB environment variable is set to false if a keytab is provided to allow the user to specify a principal and password
Added AbstractHadoopProcessorSpec to verify validation of principal/keytab/password/kerberos credential service combinations
This closes#4095.
random ID if not set.
Also add group ID field to ConsumeMQTT processor. Allows consumer to join consumer group at $share/<group_id>/<topic_filter>
add expression language support for the MQTT client ID
Setting client id in publish test fails because it is not a flowfile attribute.
Remove client id and autogenerate it when testing.
Since the evaluation is done in onScheduled, there is no flow file available and we're not using the attributes to make the expression language evaluation. You can change the scope to use the Variable Registry.
Co-Authored-By: Pierre Villard <pierre.villard.fr@gmail.com>
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#3879.
Handle scenario where message might be null.
Update to test case that was failing because adding attributes modified a flow file even if you don't change the contents.
Fixed Style Issues and Updated WritesAttributes.
Added Test Case for Error Message
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#4052
JsonContentConversionExceptionMapper, JsonMappingExceptionMapper, JsonParseExceptionMapper.
Registered the custom ExceptionMappers.
Added unit tests to throw Exception for string port value and sanitize script input. Handled null or empty JsonMappingException reference path.
Added the Apache license to Groovy Test.
Signed-off-by: Andy LoPresto <alopresto@apache.org>
NIFI-6363 Additional fixes.
NIFI-6363 Fix Hadoop compile problem. Add GCP IT instructions.
NIFI-6363 - Removed GCP provider due to dependency conflicts with GRPC processors. Fixed unit test to match master branch after rebase.
NIFI-6363 - Added some docs and experimental tag to the relevant classes.
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#4080.
NIFI-7007: Add additional unit tests that hit non-happy path
NIFI-7007: Use AllowableValue instead of string
NIFI-7007: Add the use of attributes for the update method, statement type and batch statement type
NIFI-7007: Add additional tests, mainly for the use of attributes
NIFI-7007: add some ReadsAttribute properties to the PutCassandraRecord processor
NIFI-7007: additional update keys validation logic
NIFI-7007: fix imports
NIFI-7007: Convert fieldValue to long in separate method
NIFI-7007: Add new style of tests checking actual CQL output
NIFI-7007: add license to new test file
NIFI-7007: add customValidate to check for certain incompatible property combinations
NIFI-7007: remove check on updateMethod and replace Set.of with java 8 compatible replacmenet
NIFI-7007: Add test for failure with empty update method via attributes
NIFI-7007: remove unused variable
NIFI-7007: Fix customValidate that incorrectly invalidated a valid config
Fix Checkstyle
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#3977
Uses `DataTypeUtils.toTimestamp` when writing to Kudu
timestamp (`UNIXTIME_MICROS`) columns. This allows
us to use the `row.addTimestamp` API and get much more
intuitive and predictable timestamp ingest behavior.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#4054.
This patch adds validation to the PutKudu operation type property.
It also improves the description to include the valid values and
adjusts the inputs to be case insensitive.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#4063.
Adds a boolean property to the PutKudu processor to optionally
enable automatic schema drift handling.
If set to true, when fields with names that are not in the target
Kudu table are encountered, the Kudu table will be altered to
include new columns for those fields.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#4053.
Matthew Burgess
Otto Fowler
Andy LoPresto
jstorck
Eduardo Fontes
M Tien
karthik.kadajji
Bryan Bende
Joey Frazee
Justin Miller
Joe Witt
Shawn Weeks
Przemysław Dubaniewicz
Pierre Villard
Yolanda M. Davis
Troy Melhase
MatthewKnight-NG
Wouter de Vries
Zoltan Kornel Torok
Andy LoPresto
Grant Henke
Peter Turcsanyi