Commit Graph

4389 Commits

Author SHA1 Message Date
Mark Payne 4700b8653d NIFI-5138: Bug fix to ensure that when we have a CHOICE between two or more REOCRD types that we choose the appropriate RECORD type when creating the Record in the JSON Reader.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2670.
2018-05-09 18:38:04 +02:00
zenfenan f742a3a6ac NIFI-5156: Updated GCP SDK to latest version
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2680.
2018-05-09 18:19:17 +02:00
Andy LoPresto 7a4990e7fe
NIFI-5146 Only support HTTP or HTTPS operation for NiFi API/UI
- Added logic to check for simultaneous configuration of HTTP and HTTPS connectors in JettyServer.
- Added test logging resources. Added unit tests.
- Refactored shared functionality to generic method which accepts lambdas.
  Fixed unit test with logging side effects.
- Added note about exclusive HTTP/HTTPS behavior to Admin Guide. Fixed typos.

This closes #2683.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2018-05-09 12:14:11 -04:00
Bryan Bende 92b4a3208f NIFI-5136 Ensure processor references are removed from LogRepository and from ProcessScheduler
- Forcing FileSystem statistics thread to be interrupted when HDFS processors are stopped
- Stop creating temp components during import from registry, use bundle info instead

This closes #2668.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-05-08 15:29:37 -04:00
thenatog 2094786ec8
NIFI-5161 - Moved filename escaping method to TlsHelper.java to allow use by the different Tls modes.
Added another test for special characters in the DN/output key filename.
Added a method to escape special characters in the alias name for keys in the truststore. This fixes an error with the TlsToolkit which occurs when extracting keys and writing them to file.

This closes #2684.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-05-07 14:07:20 -07:00
Bryan Bende a9e85c358c
NIFI-5163 Clearing version control info when creating a template
This closes #2685.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2018-05-07 16:13:26 -04:00
Mark Payne fb48ae2f88
NIFI-5154: When Processor or Controller Service is added to a Process Group, remove any references from it to any other Controller Service that is not reachable from the newly assigned Process Group
Fixed bug in unit test
Addressed review feedback/addressed issue where if a group is moved inside another group, the descendant processors of the moved group did not have their service references updated properly. Also addressed an issue where if a service is defined in Group A, then Group B lives within Group A and has a processor that references a service at the level of Group A, we allowed user to move Group B outside of Group A (but wouldn't allow the processor to be moved out of scope by itself).
This closes #2678
2018-05-07 16:01:26 -04:00
Mark Payne 8acac9cba5
NIFI-5153: If a node is disconnected due to failure to complete mutable request, the node should be allowed to rejoin
This closes #2677.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-05-07 15:41:38 -04:00
zenfenan 4544f3969d
NIFI-5152: MoveHDFS now works even with no upstream connection
This closes #2681.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-05-07 13:20:56 -04:00
Mark Payne 868808f4b4
NIFI-5150: Fixed bug that caused StandardProcessSession.append() to copy too much data when called on an incoming flowfile
This closes #2676.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-05-07 11:44:52 -04:00
Matt Gilman 5cfa29e48f
NIFI-5135:
- Adding support for applying transforms to user identities and group names.

This closes #2673.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-05-07 11:17:26 -04:00
Koji Kawamura ff04f8efee NIFI-4980: Typo in ReportAtlasLineage kafka kerberos service name property
This closes #2550

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-07 06:08:59 -04:00
Michal Misiewicz ed30bb9b78 NIFI-5130 ExecuteInfluxDBQuery processor chunking support
This closes #2666

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-04 17:32:10 -04:00
Jeff Storck 3ad3243511 NIFI-5134 Explicitly requesting UGI to relogin before attempting to get a DB connection in HiveConnectionPool
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2667
2018-05-03 09:12:07 -04:00
Koji Kawamura 2007c207ab NIFI-4393: Handle database specific identifier escape characters
QueryDatabaseTable and GenerateTableFetch processors were not able to
use max value state as expected, if max value column was wrapped with
escape characters. Due to a mis-match between computed state keys
and actual keys used in the managed state. State keys computed by
getStateKey method included escape characters while actual stored keys
did not. Resulted querying the same dataset again and again.

This commit added unwrapIdentifier method to DatabaseAdapter class to
remove database specific escape characters for identifiers such as table
and column names, so that max value state keys are populated correctly
even if identifiers are wrapped with escape characters.

This commit also added new DatabaseAdapter for MySQL, to handle MySQL
specific identifier escape with back-ticks.

This closes #2424

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-03 09:06:05 -04:00
Mark Payne 18ad348107
NIFI-5142: Do not allow a connection's destination to be changed to a funnel if the source is the same funnel. Also fixed some typos in StandardFunnel. This closes #2669 2018-05-02 15:53:52 -04:00
Matthew Burgess 0289ca7114 NIFI-4456: Support multiple JSON objects in JSON record reader/writers
This closes #2640.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-05-02 15:22:01 -04:00
Matt Gilman 4ed6631edc
NIFI-5124:
- Upgrading to the latest version of commons-fileupload.

This closes #2662.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-05-02 08:20:33 -07:00
zenfenan 579078e352 NIFI-5129: AWS Processors now displays proper region names
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2665.
2018-04-29 17:06:10 -07:00
zenfenan ba84ad1e38 NIFI-5105: Improvements for nifi-aws-bundle
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2664.
2018-04-28 14:03:14 -07:00
Koji Kawamura 6e8b1c8f74 NIFI-5090: Create move target dir dynamically at FetchFTP and FetchSFTP
Support creating target parent directories even if directory listing is disabled.

fixed typo in doc

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2642
2018-04-27 11:48:26 -04:00
Matthew Burgess 159b64b4c8
NIFI-5123: Move SchemaRegistryService to nifi-avro-record-utils
This closes #2661.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-04-26 11:50:01 -04:00
Andy LoPresto 4e4aa54c69
NIFI-5116 Implemented logic to translate nifi.properties file to CLI properties format.
Added unit tests.

This closes #2660.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-04-26 09:59:59 -04:00
sbouchex 3719a6268c NIFI-5120 AbstractListenEventProcessor supports expression language
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2659.
2018-04-26 14:05:53 +02:00
Pierre Villard 23937835f3 NIFI-5092 - Removed local state management for S2S Bulletins RT
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2643
2018-04-25 16:25:00 -04:00
Matt Gilman af2513adf8 NIFI-1295:
- Adding UI controls for terminating hung threads.
- Showing current number of terminated threads.
- Fixing issue when replicating terminate threads request throughout the cluster.

This closes #2607.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-04-25 16:23:34 -04:00
greencee fdea876ede NIFI-5060 Updated SubstringAfter record processing to support multi-character search trimming
This closes #2623.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-04-25 15:40:20 -04:00
Jeff Storck 88c079e447
NIFI-4923 Updated nifi-hadoop-libraries-nar, nifi-hdfs-processors, and nifi-hadoop-utils dependency on hadoop-client from 2.7.3 to 3.0.0
This closes #2582.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-04-25 15:34:14 -04:00
Matt Gilman 6938e58c81 NIFI-5066:
- Allowing the enable/disable buttons to be active under the same conditions as the start/stop buttons.

This closes #2633.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-04-25 14:58:29 -04:00
zenfenan 72f8999b15 NIFI-5015: Implemented Azure Queue Storage processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2611.
2018-04-25 20:07:15 +02:00
zenfenan c118e96238 NIFI-5000 - ListHDFS properly lists files from updated directory path
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2576.
2018-04-25 18:03:44 +02:00
joewitt ac9944ccee
NIFI-5108 Updated all explicit refs and media nar usage of commons-compress to latest version and updated spring redis client
This closes #2651.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-04-24 15:16:04 -04:00
abhinavrohatgi30 e3f4720797
NIFI-4035 - Adding PutSolrRecord Processor that reads NiFi records and indexes them into Solr as SolrDocuments
Adding Test Cases for PutSolrRecord Processor

Adding PutSolrRecord Processor in the list of Processors

Resolving checkstyle errors

Resolving checkstyle errors in test classes

Adding License information and additional information about the processor

1. Implementing Batch Indexing 2. Changes for nested records 3. Removing MockRecordParser

Fixing bugs with nested records

Updating version of dependencies

Setting Expression Language Scope

This closes #2561.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-04-24 10:50:40 -04:00
patricker 0390c0f196 NIFI-4561 ExecuteSQL returns no FlowFile for some queries
This closes #2243

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-04-24 10:27:44 -04:00
Mike Thomsen 5ca6261de0 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2018-04-23 15:39:20 -04:00
Otto Fowler 45bc1f1b4f NIFI-3576 Support for QueryInfo relationship, can be used to track no-hits
Squashed commit includes related commit from GitHub user wietze.

This closes #2601

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-04-23 15:38:27 -04:00
Mark Payne 0e736f59fd NIFI-4185: Minor tweaks to XML Record Reader, around documentation and error handling
This closes #2587.
2018-04-23 14:41:17 -04:00
JohannesDaniel d21bd3870b NIFI-4185: Add XML Record Reader 2018-04-23 14:39:18 -04:00
Koji Kawamura 8e4aa6bf22 NIFI-5075: Do not execute Funnels with no outgoing connections
- Added dedicated conditions for Funnels
- Fixed stale Javadoc
- Stopped caching hasNonLoopConnection variable
- Grouped some conditions to isSourceComponent variable for better
readability

This closes #2634.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-04-23 10:37:22 -04:00
Pierre Villard 2799211946 NIFI-5099 - Update flow differences when updating a connection
This closes #2649

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-04-23 07:08:00 -04:00
Matthew Burgess ba32879ec8 NIFI-5082: Added support for custom Oracle timestamp types to Avro conversion
This closes #2638

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-04-22 19:59:48 -04:00
JohannesDaniel 8b6539e3c7 Added provenance reporting
This closes #2650

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-04-22 15:58:06 -04:00
Mark Payne 54eb6bc232
NIFI-5096: Periodically poll ZooKeeper to determine the leader for each registered role in Leader Election. This avoids a condition whereby a node may occasionally fail to receive notification that it is no longer the elected leader.
NIFI-5096: More proactively setting leadership to false when ZooKeeper/Curator ConnectionState changes

This closes #2646
2018-04-20 16:51:02 -04:00
Pierre Villard 262bf011e4 NIFI-5078 - added source/destination connection info in S2SStatusRT
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2632
2018-04-20 11:50:06 -04:00
Koji Kawamura c575a98936 NIFI-5095: Suppress SET property parse failure at Hive processors
Log debug message when ParseException is thrown.

Log warning message if other unknown Exception is thrown.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2645
2018-04-20 11:24:02 -04:00
zenfenan 1dbfcb9445 NIFI-5062: Removed hbase-client dependecy from hbase bundle
This closes #2636

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-04-20 08:50:25 -04:00
Kevin Doran fc902234b6 NIFI-4942 Fixes Travis CI build
- Fixes unit test salt assertion regex
- Adds RAT excludes and reenables console output for unapproved files

NIFI-4942 Temporarily disables tests that are failing on Linux

This closes #2648.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-04-20 05:15:47 +00:00
Alex Moundalexis af12923da4 NIFI-5097 This closes #2647. add client-side HTTP redirect to error index.jsp
Existing web error index page suggests user may have meant /nifi/ and crafts a contextPath link, may as well redirect them automatically. Non-human (i.e. curl/wget) can still retrieve the original source without being redirected.

Signed-off-by: joewitt <joewitt@apache.org>
2018-04-19 15:01:33 -04:00
Matthew Burgess 090e748726 NIFI-4647: Fix support for strings in unions for ConvertAvroToORC
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2644.
2018-04-19 09:49:32 +02:00
JohannesDaniel aa196bc01f NIFI-4516 Added QuerySolr after rebase
This closes #2517

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-04-18 17:47:04 -04:00