Commit Graph

212 Commits

Author SHA1 Message Date
Timea Barna 8491486181
NIFI-9185 Add Avro logical type to SelectHive3QL processor
Modifying unit test to avoid systemdefault timezone usuage

NIFI-9185 Applying review recommendations removing duplicate dependency from pom.xml

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5358
2021-11-08 13:59:46 -05:00
Joe Witt c055895952
NIFI-9344-RC3 prepare for next development iteration 2021-11-03 08:53:37 -07:00
Joe Witt 7fdc07cccd
NIFI-9344-RC3 prepare release nifi-1.15.0-RC3 2021-11-03 08:53:32 -07:00
exceptionfactory 60d6d469bf
NIFI-9342 Upgraded to Netty 3.10.6 and 4.1.69
- Replaced Netty 3.6.9 and 3.7.1 with 3.10.6
- Replaced Netty 4.1 with 4.1.69

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5490.
2021-10-28 09:34:31 +02:00
Denes Arvay d722b820a1
NIFI-9248 Use hive-exec:core instead of the "regular" hive-exec dependency
- Excluded org.apache.hive:hive-exec and added the "core" classifier version
- Added the removed dependencies explicitly

NIFI-9248 Add additional log4j excludes

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5414
2021-10-18 11:55:43 -04:00
exceptionfactory 9d578e63fd
NIFI-9285 Upgraded ZooKeeper transitive dependencies to 3.4.14
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5443.
2021-10-07 05:32:39 -04:00
exceptionfactory 4bcd03024a
NIFI-9283 Excluded Log4j 1.2 and upgraded Log4j2 to 2.14.1
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5440.
2021-10-06 11:17:36 -04:00
Mark Payne 607480c1a0
NIFI-9239: Updated Consume/Publish Kafka processors to support Exactly Once Semantics. Added ExecuteStateless processor/bundle
- A few minor updates to Stateless in order to surface some concepts from the stateless engine up to the caller, such as bulletins, counters, etc.

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5412.
2021-10-04 15:32:21 -04:00
exceptionfactory bb638b13aa
NIFI-9216 Upgraded Derby to 10.14.2.0 in nifi-hive-bundle
This closes #5385

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2021-09-14 15:19:18 -04:00
Mohammed Nadeem acb25c1130
NIFI-9170 Upgrade commons-beanutils to 1.9.4 to mitigate CVE-2019-10086
NIFI-9170 Add two more 1.9.4 references to close out the few things identified by the Maven dependency plugin.

This closes #5351

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2021-09-07 16:24:57 -04:00
Bryan Bende e436381c3a
NIFI-8975 Integrate KerberosUserService into HBase processors/services
This closes #5322

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-27 17:41:58 -05:00
Bryan Bende 21c2fb95d3
NIFI-8973 Implement KerberosUserService API and keytab, password, and ticket cache implementations
NIFI-8974 Integrate KerberosUserService with HDFS processors

NIFI-8980 Integrate KerberosUserService with Kafka 2.6 processors
- Introduced SelfContainerKerberosUserService to restrict which impls can be used with Kafka
- Add variations of KerberosUser doAs that allow setting the context ClassLoader
- Add additional unit tests for configurations

This closes #5277

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-18 19:54:56 -05:00
Matthew Burgess 5ff4974b1f
NIFI-5936: Added DROP provenance event to MockProcessSession.remove() to match real impl
This closes #5302

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2021-08-11 07:16:11 -04:00
Denes Arvay f57685e971 NIFI-8955 Add Max Connection Lifetime property to Hive(_1_1)ConnectionPool CS
This closes #5259.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-08-09 15:15:18 +02:00
exceptionfactory a4375a046f
NIFI-8990 Upgraded Groovy to 2.5.14
- Added dependency management override for Groovy 2.4.21 in Hive Processors
- Upgraded TinkerPop Gremlin to 3.5.1 in graph bundle to depend on Groovy 2.5.14
- Adjusted TlsHelperGroovyTest variable declaration to avoid compiler warnings

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5276
2021-08-04 09:18:38 -04:00
Peter Turcsanyi a239eea8ff
NIFI-8996: Close JDBC statements in PutHive*QL processors.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5280
2021-08-04 09:13:15 -04:00
exceptionfactory d93e9f152c
NIFI-8986 Upgraded Commons Compress to 1.21
- Upgraded direct and transitive dependencies from 1.20 and earlier to 1.21

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5273.
2021-08-03 15:52:45 +02:00
exceptionfactory ba7271c85b
NIFI-8988 Upgraded SnakeYAML to 1.29
- Updated MiNiFi references to use shared parent dependency version
- Updated direct dependencies from 1.28 to 1.29
- Added dependency management declarations where necessary to override transitive versions

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5275.
2021-08-03 10:02:20 +02:00
Joe Witt 97feacc181
NIFI-8767-RC2 prepare for next development iteration 2021-07-10 12:17:09 -07:00
Joe Witt fcbf1d5f97
NIFI-8767-RC2 prepare release nifi-1.14.0-RC2 2021-07-10 12:17:05 -07:00
exceptionfactory f2698a4e1e
NIFI-8723 Upgraded Jackson to 2.12.3
- Updated MiNiFi to reference root jackson.version
- Updated Registry assembly to reference root jackson.version
- Added dependency management in NAR modules to override transitive versions

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5174
2021-06-22 11:21:45 -04:00
exceptionfactory 4df2a79c8a
NIFI-8678 Forced Calcite 1.27.0 and Calcite Avatica 1.6.0 for Hive Bundle
- Overriding Calcite incubating versions avoids dependencies from third party repositories

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5142
2021-06-10 10:02:50 -04:00
Dominik Przybysz f07e17ba74
[NIFI-8610] Do not reuse generic records in convertAvroToORC
[NIFI-8610] Simplify decimal test for convertAvroToORC

Signed-off-by: Arpad Boda <aboda@apache.org>

This closes #1081
2021-06-07 11:26:16 +02:00
exceptionfactory 4888eaf62d
NIFI-8623 Changed repositories from conjars.org to public.nexus.pentaho.org
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5091
2021-05-21 08:07:09 -04:00
exceptionfactory ef794c56b6
NIFI-8398 Updated dependencies to build using Apache Maven 3.8.1
- Updated ParCEFone to 1.2.8
- Added conjars.org HTTPS repository for Hive transitive dependencies

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4989
2021-04-12 14:45:02 -04:00
Mark Payne 7d1d536da6
NIFI-8206: Added identifiesExternalResource() method to PropertyDescriptor.Builder and implemented functionality.
- Updated components to make use of new feature

NIFI-8206: Added a ResourceType of TEXT. This requires that the ResourceReferenceFactory know which types are allowed in order to create the ResourceReference. PropertyValue needs to then have the PropertyDescriptor available to it. This resulted in highlighting many bugs in unit tests where components were not exposing property descriptors via getSupportedPropertyDescriptors() or were evaluating Expression Language using the wrong scope, so fixed many unit tests/components to properly declare Expression Language scope when using it

NIFI-8206: Removed problematic unit test that required directory names with special characters that are not allowed on some operating systems

This closes #4890.

Signed-off-by: Bryan Bende <bbende@apache.org>
2021-04-12 11:48:04 -04:00
exceptionfactory 0e659981d4 NIFI-8298 Refactored Kerberos and Socket classes from security-utils to new modules
- Created nifi-security-socket-ssl
- Created nifi-security-kerberos
- Removed nifi-security-utils dependency from nifi-processor-utils
- Updated modules to reference new dependencies
- Eliminated unnecessary transitive dependencies on bcprov-jdk15on from over 30 modules

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4881.
2021-03-19 13:12:19 -04:00
Joe Witt 88fab00e29
NIFI-7873 merging release branch to latest and updating to 1.14.0-SNAPSHOT 2021-02-15 12:09:32 -07:00
Joe Witt 4afb2ba743
NIFI-7873-RC4 prepare for next development iteration 2021-02-15 12:09:31 -07:00
Joe Witt 487280bee9
NIFI-7873-RC4 prepare release nifi-1.13.0-RC4 2021-02-15 12:09:30 -07:00
Joe Witt d826416217
NIFI-8192 updating Copyright years for nifi things to include 2021
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4802.
2021-02-03 12:56:28 +04:00
Matthew Burgess b9076ca26e NIFI-7989: Add Update Field Names and Record Writer to UpdateHiveTable processors
NIFI-7989: Only rewrite records if a field name doesn't match a table column name exactly
NIFI-7989: Rewrite records for created tables if Update Field Names is set

This closes #4750.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-01-13 23:26:44 +01:00
Matthew Burgess f29d6a6046 NIFI-7989: Add support to UpdateHiveTable for creating external tables
NIFI-7989: Add support for creating partitions, quote identifiers
NIFI-7989: Quote table name when getting description

This closes #4697.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-12-15 22:05:53 +01:00
exceptionfactory cfbcecc4c6
NIFI-7884 Added and applied Distributed File System permissions (#4713) 2020-12-08 15:56:46 -05:00
Arek Burdach 87fd9f968f
NIFI-7958 ConvertAvroToORC - decimal support
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4628
2020-11-23 13:00:54 -05:00
Matthew Burgess edc060bd92 NIFI-7989: Add UpdateHiveTable processors for data drift capability
NIFI-7989: Allow for optional blank line after optional column and partition headers
NIFI-7989: Incorporated review comments
NIFI-7989: Close Statement when finishing processing
NIFI-7989: Remove database name property, update output table attribute

This closes #4653.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-11-18 00:39:07 +01:00
Pierre Villard 14ec02f21d
NIFI-7981 - add support for enum type in avro schema
This closes #4648

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-11-05 18:19:55 -05:00
Indrajeet Gour 4a370eae8c
NIFI-7933 - Set proper property descriptor for CSV Escape
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4606.
2020-10-16 18:13:29 +02:00
Pierre Villard c2960998ac
NIFI-7869 - adding a GCP profile for GCS connector (#4565) 2020-10-09 09:42:20 -04:00
Mohammed Nadeem e2ccfbbacf
NIFI-7859: Support for capturing execution duration of query run as attributes in SelectHiveQL processors
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4560
2020-10-01 09:06:01 -04:00
Pierre Villard 740bfee8f4
NIFI-7833 - Add Ozone support in Hadoop components where appropriate (#4545)
* NIFI-7833 - Add Ozone support in Hadoop components where appropriate

* added fs dependency
2020-09-29 09:06:13 -04:00
Matthew Burgess 7e145142e1
NIFI-7799: Relogin with Kerberos on connect exception in DBCPConnectionPool (#4519) 2020-09-17 13:33:54 -04:00
Matthew Burgess 45470b0984 NIFI-7740: Add Records Per Transaction and Transactions Per Batch properties to PutHive3Streaming
NIFI-7740: Incorporated review comments

NIFI-7740: Restore RecordsEOFException superclass to SerializationError

This closes #4489.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-09-01 17:17:02 +02:00
Joe Witt 8baa5c9940
NIFI-7692 updating for next dev release 1.13.0 2020-08-18 14:48:02 -07:00
Joe Witt fb57bcbc11
NIFI-7692-RC1 prepare for next development iteration 2020-08-13 09:20:39 -07:00
Joe Witt 303d6c59ba
NIFI-7692-RC1 prepare release nifi-1.12.0-RC1 2020-08-13 09:20:36 -07:00
Bence Simon 5c2bfcf7d3 NIFI-7369 Adding decimal support for record handling in order to avoid missing precision when reading in records
Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-06-02 15:13:14 -04:00
Matthew Burgess 53a161234e
NIFI-7448: Fix quoting of DDL table name in PutORC
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4269.
2020-05-14 11:41:35 +02:00
Matthew Burgess b7c81d6007
NIFI-7428: Switch hive.version property to set Hive 3 version
This closes #4259
2020-05-06 19:38:23 -04:00
Joe Witt f694e6464f NIFI-7187 adding missing version strings from accumulo bundle pom
- Removed Cat X JSON.org dep inclusion which seems to not be necessary
- Updated a ton of easier/safer looking deps
- Updated tika due to CVE

This closes #4086

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-03-20 10:07:56 -04:00