4554 Commits

Author SHA1 Message Date
Koji Kawamura
79a7014a95 NIFI-4971: ReportLineageToAtlas complete path can miss one-time lineages
- Separated Hook message de-duplication logic from NiFiAtlasHook to
NotificationSender
- NiFiAtlasHook used to send individual CREATE_ENTITY messages for each entities,
this commit changed it to bundle all new entities into a single
CREATE_ENTITY to preserve entity creation order, so that new DataSet
entities can be referred from new nifi_flow_path entities
- Added more unit tests

This closes #2542

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-15 05:50:26 -04:00
Fokko Driesprong
ed9263811b [NIFI-5190] Bump Apache Parquet to 1.10.0
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2697.
2018-05-15 11:40:11 +02:00
JohannesDaniel
04745a556a NIFI-5189 Schema name is not available for RecordSchema
This closes #2700

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-14 14:35:12 -04:00
Koji Kawamura
03adaeca22 NIFI-5114: Added Basic auth support to WebSocket components
This closes #2652

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-14 14:26:47 -04:00
Otto Fowler
61fe493786 NIFI-5170 Upgrad Grok to version 0.1.9
This closes #2691

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-13 08:40:20 -04:00
Gardella Juan Pablo
64356e0014 NIFI-5049 Fix handling of Phonenix datetime columns
This closes #2625

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-11 09:51:24 -04:00
Matthew Burgess
099bfcdf3a NIFI-5121: Added DBCPService API method for passing in flow file attributes when available
This closes #2658

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-11 08:19:16 -04:00
Koji Kawamura
500a254e3f NIFI-4637: HBase visibility lables
Fixed additional docs directory structure.
2018-05-11 07:30:19 +00:00
Mike Thomsen
0b851910f3 NIFI-4637 Added support for visibility labels to the HBase processors.
NIFI-4637 Removed integration test and updated the hbase client from 1.1.2 to 1.1.13 which is the final version for 1.1.X

NIFI-4637 Fixed EL support issue w/ tests.

NIFI-4637 Added more documentation to DeleteHBaseCells.

NIFI-4637 changed PutHBaseCell/JSON to use dynamic properties instead of a 'default visibility string.'

NIFI-4637 Added changes requested in a code review.

NIFI-4637 Moved pickVisibilityString to a utility class to make testing easier.

NIFI-4637 Added additionalDetails.html for PutHBaseRecord.

NIFI-4637 Added additional documentation and testing.

NIFI-4637 Added documentation for DeleteHBaseCells.

NIFI-4637 Added pickVisibilityLabel support to PutHBaseRecord and updated documentation to reflect that.

NIFI-4637 Reverted version bump to hbase client.

This closes #2518.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-05-11 00:55:40 +00:00
himanshu
6e0be8e641 NIFI-4938 Upgraded org.eclipse.paho.client.mqttv3 dependency version to 1.2.0
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2514.
2018-05-10 11:04:32 +02:00
Mark Bean
def9741a39 NIFI-5180: update JMS additional details to set Destination Type to Required, default 'QUEUE'
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2694.
2018-05-10 10:47:20 +02:00
Matt Gilman
22342a0e0c NIFI-5167:
- Updating how the nf-reporting-task module is injected to the nf-controller-service module.

This closes #2688

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2018-05-09 14:06:19 -04:00
Andy LoPresto
4f1444c0e0 NIFI-4942 This closes #2690. Resolved test failures in JCE limited mode.
Signed-off-by: joewitt <joewitt@apache.org>
2018-05-09 13:52:05 -04:00
Junegunn Choi
02ba4cf2c8 NIFI-5064 - Fixes and improvements to PutKudu processor
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2621.
2018-05-09 19:39:29 +02:00
patricker
0a44bad76e NIFI-5168 - ReplaceText Processor Should Use Single FlowFile Processing Instead of Batch
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2687.
2018-05-09 18:46:35 +02:00
Mark Payne
4700b8653d NIFI-5138: Bug fix to ensure that when we have a CHOICE between two or more REOCRD types that we choose the appropriate RECORD type when creating the Record in the JSON Reader.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2670.
2018-05-09 18:38:04 +02:00
zenfenan
f742a3a6ac NIFI-5156: Updated GCP SDK to latest version
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2680.
2018-05-09 18:19:17 +02:00
Andy LoPresto
7a4990e7fe
NIFI-5146 Only support HTTP or HTTPS operation for NiFi API/UI
- Added logic to check for simultaneous configuration of HTTP and HTTPS connectors in JettyServer.
- Added test logging resources. Added unit tests.
- Refactored shared functionality to generic method which accepts lambdas.
  Fixed unit test with logging side effects.
- Added note about exclusive HTTP/HTTPS behavior to Admin Guide. Fixed typos.

This closes #2683.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2018-05-09 12:14:11 -04:00
Bryan Bende
92b4a3208f NIFI-5136 Ensure processor references are removed from LogRepository and from ProcessScheduler
- Forcing FileSystem statistics thread to be interrupted when HDFS processors are stopped
- Stop creating temp components during import from registry, use bundle info instead

This closes #2668.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-05-08 15:29:37 -04:00
thenatog
2094786ec8
NIFI-5161 - Moved filename escaping method to TlsHelper.java to allow use by the different Tls modes.
Added another test for special characters in the DN/output key filename.
Added a method to escape special characters in the alias name for keys in the truststore. This fixes an error with the TlsToolkit which occurs when extracting keys and writing them to file.

This closes #2684.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-05-07 14:07:20 -07:00
Bryan Bende
a9e85c358c
NIFI-5163 Clearing version control info when creating a template
This closes #2685.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2018-05-07 16:13:26 -04:00
Mark Payne
fb48ae2f88
NIFI-5154: When Processor or Controller Service is added to a Process Group, remove any references from it to any other Controller Service that is not reachable from the newly assigned Process Group
Fixed bug in unit test
Addressed review feedback/addressed issue where if a group is moved inside another group, the descendant processors of the moved group did not have their service references updated properly. Also addressed an issue where if a service is defined in Group A, then Group B lives within Group A and has a processor that references a service at the level of Group A, we allowed user to move Group B outside of Group A (but wouldn't allow the processor to be moved out of scope by itself).
This closes #2678
2018-05-07 16:01:26 -04:00
Mark Payne
8acac9cba5
NIFI-5153: If a node is disconnected due to failure to complete mutable request, the node should be allowed to rejoin
This closes #2677.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-05-07 15:41:38 -04:00
zenfenan
4544f3969d
NIFI-5152: MoveHDFS now works even with no upstream connection
This closes #2681.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-05-07 13:20:56 -04:00
Mark Payne
868808f4b4
NIFI-5150: Fixed bug that caused StandardProcessSession.append() to copy too much data when called on an incoming flowfile
This closes #2676.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-05-07 11:44:52 -04:00
Matt Gilman
5cfa29e48f
NIFI-5135:
- Adding support for applying transforms to user identities and group names.

This closes #2673.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-05-07 11:17:26 -04:00
Koji Kawamura
ff04f8efee NIFI-4980: Typo in ReportAtlasLineage kafka kerberos service name property
This closes #2550

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-07 06:08:59 -04:00
Michal Misiewicz
ed30bb9b78 NIFI-5130 ExecuteInfluxDBQuery processor chunking support
This closes #2666

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-04 17:32:10 -04:00
Jeff Storck
3ad3243511 NIFI-5134 Explicitly requesting UGI to relogin before attempting to get a DB connection in HiveConnectionPool
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2667
2018-05-03 09:12:07 -04:00
Koji Kawamura
2007c207ab NIFI-4393: Handle database specific identifier escape characters
QueryDatabaseTable and GenerateTableFetch processors were not able to
use max value state as expected, if max value column was wrapped with
escape characters. Due to a mis-match between computed state keys
and actual keys used in the managed state. State keys computed by
getStateKey method included escape characters while actual stored keys
did not. Resulted querying the same dataset again and again.

This commit added unwrapIdentifier method to DatabaseAdapter class to
remove database specific escape characters for identifiers such as table
and column names, so that max value state keys are populated correctly
even if identifiers are wrapped with escape characters.

This commit also added new DatabaseAdapter for MySQL, to handle MySQL
specific identifier escape with back-ticks.

This closes #2424

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-03 09:06:05 -04:00
Mark Payne
18ad348107
NIFI-5142: Do not allow a connection's destination to be changed to a funnel if the source is the same funnel. Also fixed some typos in StandardFunnel. This closes #2669 2018-05-02 15:53:52 -04:00
Matthew Burgess
0289ca7114 NIFI-4456: Support multiple JSON objects in JSON record reader/writers
This closes #2640.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-05-02 15:22:01 -04:00
Matt Gilman
4ed6631edc
NIFI-5124:
- Upgrading to the latest version of commons-fileupload.

This closes #2662.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-05-02 08:20:33 -07:00
zenfenan
579078e352 NIFI-5129: AWS Processors now displays proper region names
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2665.
2018-04-29 17:06:10 -07:00
zenfenan
ba84ad1e38 NIFI-5105: Improvements for nifi-aws-bundle
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2664.
2018-04-28 14:03:14 -07:00
Koji Kawamura
6e8b1c8f74 NIFI-5090: Create move target dir dynamically at FetchFTP and FetchSFTP
Support creating target parent directories even if directory listing is disabled.

fixed typo in doc

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2642
2018-04-27 11:48:26 -04:00
Matthew Burgess
159b64b4c8
NIFI-5123: Move SchemaRegistryService to nifi-avro-record-utils
This closes #2661.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-04-26 11:50:01 -04:00
Andy LoPresto
4e4aa54c69
NIFI-5116 Implemented logic to translate nifi.properties file to CLI properties format.
Added unit tests.

This closes #2660.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-04-26 09:59:59 -04:00
sbouchex
3719a6268c NIFI-5120 AbstractListenEventProcessor supports expression language
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2659.
2018-04-26 14:05:53 +02:00
Pierre Villard
23937835f3 NIFI-5092 - Removed local state management for S2S Bulletins RT
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2643
2018-04-25 16:25:00 -04:00
Matt Gilman
af2513adf8 NIFI-1295:
- Adding UI controls for terminating hung threads.
- Showing current number of terminated threads.
- Fixing issue when replicating terminate threads request throughout the cluster.

This closes #2607.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-04-25 16:23:34 -04:00
greencee
fdea876ede NIFI-5060 Updated SubstringAfter record processing to support multi-character search trimming
This closes #2623.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-04-25 15:40:20 -04:00
Jeff Storck
88c079e447
NIFI-4923 Updated nifi-hadoop-libraries-nar, nifi-hdfs-processors, and nifi-hadoop-utils dependency on hadoop-client from 2.7.3 to 3.0.0
This closes #2582.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-04-25 15:34:14 -04:00
Matt Gilman
6938e58c81 NIFI-5066:
- Allowing the enable/disable buttons to be active under the same conditions as the start/stop buttons.

This closes #2633.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-04-25 14:58:29 -04:00
zenfenan
72f8999b15 NIFI-5015: Implemented Azure Queue Storage processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2611.
2018-04-25 20:07:15 +02:00
zenfenan
c118e96238 NIFI-5000 - ListHDFS properly lists files from updated directory path
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2576.
2018-04-25 18:03:44 +02:00
joewitt
ac9944ccee
NIFI-5108 Updated all explicit refs and media nar usage of commons-compress to latest version and updated spring redis client
This closes #2651.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-04-24 15:16:04 -04:00
abhinavrohatgi30
e3f4720797
NIFI-4035 - Adding PutSolrRecord Processor that reads NiFi records and indexes them into Solr as SolrDocuments
Adding Test Cases for PutSolrRecord Processor

Adding PutSolrRecord Processor in the list of Processors

Resolving checkstyle errors

Resolving checkstyle errors in test classes

Adding License information and additional information about the processor

1. Implementing Batch Indexing 2. Changes for nested records 3. Removing MockRecordParser

Fixing bugs with nested records

Updating version of dependencies

Setting Expression Language Scope

This closes #2561.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-04-24 10:50:40 -04:00
patricker
0390c0f196 NIFI-4561 ExecuteSQL returns no FlowFile for some queries
This closes #2243

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-04-24 10:27:44 -04:00
Mike Thomsen
5ca6261de0 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2018-04-23 15:39:20 -04:00