Commit Graph

100 Commits

Author SHA1 Message Date
Matthew Burgess 921403ff55 NIFI-5336: Exclude hbase-server from Hive 3 bundle
This closes #2811

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-06-26 05:33:07 -04:00
Andy LoPresto d42a1e8bf4
NIFI-5323-RC1 prepare for next development iteration 2018-06-19 20:02:21 -07:00
Andy LoPresto 99bcd1f88d
NIFI-5323-RC1 prepare release nifi-1.7.0-RC1 2018-06-19 20:02:01 -07:00
Matthew Burgess 97f71fd6c7 NIFI-5044: Applied changes in SelectHiveQL to SelectHive3QL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2799.
2018-06-15 22:42:59 +02:00
Ed B 187417d077 NIFI-5044 SelectHiveQL accept only one statement
SelectHiveQL support only single SELECT statement.
This change adds support for pre- and post- select statements.
It will be useful for configuration queries, i.e. "set tez.queue.name=default", and others.

renamed selectQuery to hqlStatement for better readability

style check correction in comment

removed meaningless examples for post-queries

execute query without result set

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2695
2018-06-15 15:48:16 -04:00
amitt90 8370a56dce NIFI-5309 update the logger message to get the output format from the outputFormat variable
This closes #2793

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-06-14 07:09:23 -04:00
Matthew Burgess da99f873a7
NIFI-4963: Added Hive3 bundle
- Incorporated review comments
- Added more defensive code for PutHive3Streaming error handling

This closes #2755.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-06-13 14:32:58 -04:00
Matthew Burgess 099bfcdf3a NIFI-5121: Added DBCPService API method for passing in flow file attributes when available
This closes #2658

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-11 08:19:16 -04:00
Jeff Storck 3ad3243511 NIFI-5134 Explicitly requesting UGI to relogin before attempting to get a DB connection in HiveConnectionPool
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2667
2018-05-03 09:12:07 -04:00
Koji Kawamura c575a98936 NIFI-5095: Suppress SET property parse failure at Hive processors
Log debug message when ParseException is thrown.

Log warning message if other unknown Exception is thrown.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2645
2018-04-20 11:24:02 -04:00
Matthew Burgess 090e748726 NIFI-4647: Fix support for strings in unions for ConvertAvroToORC
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2644.
2018-04-19 09:49:32 +02:00
Matthew Burgess ae1d3e3945 NIFI-4862: Keep incoming flowfile attributes in outgoing flowfiles from SelectHiveQL
This closes #2605.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-04-09 12:31:05 +09:00
joewitt 59f625d3c2 Merge branch 'NIFI-4995-RC3' as part of NiFi 1.6.0 release process 2018-04-06 16:48:08 -07:00
Pierre Villard 4c787799ff NIFI-4149 - Indicate if EL is evaluated against FFs or not
- take into account input requirement for documentation rendering
- Renamed variable registry scope and added comments
- Doc + change in mock framework to check scope + update of components + UI
2018-04-06 12:40:54 -04:00
Matthew Burgess 7ff38f690d
NIFI-5045: Fixed error code handling in PutHiveQL. This closes #2608 2018-04-05 16:03:42 -04:00
joewitt d511fe3e4b NIFI-4995-RC3 prepare for next development iteration 2018-04-03 08:28:34 -07:00
joewitt f8466cb16d NIFI-4995-RC3 prepare release nifi-1.6.0-RC3 2018-04-03 08:28:15 -07:00
joewitt 7b5bf265a6 NIFI-4995 updating copyright year on all notices 2018-03-26 21:54:10 -04:00
Mark Payne 0b0aebe148
NIFI-4917: Externalize Keytab and Principal configuration from Processors to a Controller Service. This gives us the ability to allow users to interact with those Keytabs/Principals to which they've been given access without allowing them access to all Keytabs and Principals
- Addressed review feedback; rebased against master

This closes #2552.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-03-21 14:23:41 -04:00
Koji Kawamura d3f54994a6 NIFI-4972 - SelectHiveQL to emit FETCH provenance event
SelectHiveQL should emit FETCH instead of CONTENT_MODIFIED when it has
incoming connections.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2543.
2018-03-14 10:17:19 +01:00
Matthew Burgess d4632bdd5d
NIFI-4944: Guard against race condition in Snappy for PutHiveStreaming
NIFI-4944: Removed unnecessary synchronized block, added more comments
This closes #2519
2018-03-12 13:22:40 -04:00
joewitt c71409fb5d
NIFI-4936 trying to quiet down the mvn output a bit so we dont exceed the travis-ci 4MB max
NIFI-4936 updated dependency handling pushing down delcarations where they belong
This closes #2512
2018-03-09 16:34:53 -05:00
joewitt 41ce788812 NIFI-4751 changed to next minor release version snapshot 2018-01-12 15:15:32 -05:00
joewitt 36405e888c NIFI-4751-RC1 prepare for next development iteration 2018-01-08 23:39:49 -07:00
joewitt 46d30c7e92 NIFI-4751-RC1 prepare release nifi-1.5.0-RC1 2018-01-08 23:39:32 -07:00
Jeff Storck 42a1ee011b NIFI-4323 This closes #2360. Wrapped Get/ListHDFS hadoop operations in ugi.doAs calls
NIFI-3472 NIFI-4350 Removed explicit relogin code from HDFS/Hive/HBase components and updated SecurityUtils.loginKerberos to use UGI.loginUserFromKeytab. This brings those components in line with daemon-process-style usage, made possible by NiFi's InstanceClassloader isolation.  Relogin (on ticket expiry/connection failure) can now be properly handled by hadoop-client code implicitly.
NIFI-3472 Added default value (true) for javax.security.auth.useSubjectCredsOnly to bootstrap.conf
NIFI-3472 Added javadoc explaining the removal of explicit relogin threads and usage of UGI.loginUserFromKeytab
Readded Relogin Period property to AbstractHadoopProcessor, and updated its documentation to indicate that it is now a deprecated property
Additional cleanup of code that referenced relogin periods
Marked KerberosTicketRenewer is deprecated

NIFI-3472 Cleaned up imports in TestPutHiveStreaming
2018-01-03 11:31:47 -05:00
Marco Gaido 353fcdda9c NIFI-3660: This closes #2356. Support schema containing a map with an array value in ConvertAvroToORC
Signed-off-by: joewitt <joewitt@apache.org>
2017-12-26 17:46:35 -05:00
Matthew Burgess febb119fac NIFI-4696 - Add Flowfile attribute EL support and per-table concurrency to PutHiveStreaming
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2342.
2017-12-20 12:11:38 +01:00
Koji Kawamura 37ef2839e0 NIFI-4545: Improve Hive processors provenance transit URL
Incorporated review comments:

- Added 'input' to equals() method so that the same table name can appear
as input and output tables.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2239
2017-11-09 12:53:28 -05:00
Matthew Burgess 5cd8a3e729 NIFI-4473: Fixed SelectHiveQL flowfile handling during error conditions
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2247.
2017-11-03 17:03:27 +01:00
Matthew Burgess 1a1e01c568 NIFI-4473 - fixed NPE in parametrized queries
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2215
2017-10-18 10:13:03 -04:00
Matthew Burgess 4edafad6e5 NIFI-4473: Add support for large result sets and normalizing Avro names to SelectHiveQL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2212.
2017-10-18 13:37:31 +02:00
Matt Gilman 6baea8ccff
NIFI-4444:
- Upgrading to Jersey 2.x.
- Updating NOTICE files where necessary.
- Fixing checkstyle issues.

This closes #2206.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-10-12 10:27:02 -07:00
Jeff Storck a57911d3db NIFI-4412-RC2 prepare for next development iteration 2017-09-28 13:45:36 -04:00
Jeff Storck e6508ba7d3 NIFI-4412-RC2 prepare release nifi-1.4.0-RC2 2017-09-28 13:45:21 -04:00
Pierre Villard 9ac88d210a NIFI-4342 - Add EL support to PutHiveStreaming
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2120
2017-09-06 13:33:18 -04:00
Maurizio Colleluori 59a32948ea
NIFI-2923 Added evaluation of attribute expressions for Kerberos principal and keytab
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-21 17:14:28 -04:00
Matt Gilman cc741d2be6
NIFI-3997:
- Bumping to next minor version.
2017-06-08 15:22:51 -04:00
Matt Gilman 6ee12e9b47
NIFI-3997-RC1prepare for next development iteration 2017-06-05 11:07:43 -04:00
Matt Gilman ddb73612bd
NIFI-3997-RC1prepare release nifi-1.3.0-RC1 2017-06-05 11:07:28 -04:00
Matt Burgess fb925fc182 NIFI-3867: Fixed issue with getConnectionURL in HiveConnectionPool using Expression Language
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1847.
2017-05-23 21:28:59 +02:00
Matt Burgess 3353865ce9 NIFI-3867: Add Expression Language support to HiveConnectionPool properties
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1783.
2017-05-15 13:46:29 +02:00
Tim Reardon e9848f4276 NIFI-3881 Fix PutHiveStreaming EL evaluation
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1791
2017-05-12 14:06:32 -04:00
Bryan Bende 3af53419af
NIFI-3770-RC2 prepare for next development iteration 2017-05-05 20:50:28 -04:00
Bryan Bende 3a605af8e0
NIFI-3770-RC2 prepare release nifi-1.2.0-RC2 2017-05-05 20:50:14 -04:00
joewitt 829c02b058 NIFI-3825 fix L&N for jolt, json, and source deps
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1765
2017-05-05 18:43:04 -04:00
joewitt ee8dc7640f
NIFI-3820 added calcite to assembly notice and updated all copyright years for compress, avro, and jetty
This closes #1764.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 14:45:40 -04:00
Koji Kawamura af6f63691c
NIFI-3818: PutHiveStreaming throws IllegalStateException
Changed from async append to sync as it breaks 'recursionSet' check in StandardProcessSession by updating it from multiple threads, resulting IllegalStateException to happen.

This closes #1761.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 13:25:59 -04:00
Bryan Bende 60d88b5a64
NIFI-3724 - Initial commit of Parquet bundle with PutParquet and FetchParquet
- Creating nifi-records-utils to share utility code from record services
- Refactoring Parquet tests to use MockRecorderParser and MockRecordWriter
- Refactoring AbstractPutHDFSRecord to use schema access strategy
- Adding custom validate to AbstractPutHDFSRecord and adding handling of UNION types when writing Records as Avro
- Refactoring project structure to get CS API references out of nifi-commons, introducing nifi-extension-utils under nifi-nar-bundles
- Updating abstract put/fetch processors to obtain the WriteResult and update flow file attributes

This closes #1712.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-01 16:10:35 -04:00
Bryan Rosander 0054a9e35f
NIFI-3755 - Restoring Hive exception handling behavior
This closes #1711.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-04-28 11:45:43 -04:00