Commit Graph

2920 Commits

Author SHA1 Message Date
Bryan Bende 0b0ba1eae3
NIFI-5319 Utilize NiFi Registry 0.2.0 client
This closes #2801.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-06-18 15:37:05 -07:00
Mike Moser d4d9259664 NIFI-5274 avoid rollback on uncaught errors in ReplaceText
forgot to penalize flowfiles sent to failure

NIFI-5274 avoid rollback on StackOverflowError in ReplaceText

add comment

removed penalize on failure, doesn't really apply here

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2767
2018-06-18 12:00:30 -04:00
Ed 568959baf5
NIFI-4906 Add GetHDFSFileInfo
- Updated doc for write attributes
- Removing redundant files
- Updates based on review

Changes:
- changed validators for regex params (3)
- removed TriggerWhenEmpty and TriggerSerially.
- removed unused LISTING_LAG_NANOS
- removed unimplemented customValidate
- removed unsused error handling for building/updating request params
- added MIME_TYPE for content
- suppressed output to original relationship when scheduled
- added full path to original/new FF when not found
- added warning messages for huge attribute values.

This closes #2639.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-06-18 09:33:42 -04:00
Pierre Villard e5322b5d33 NIFI-4982 - Add a DistributedMapCacheLookupService
This closes #2558

Signed-off-by: zenfenan <zenfenan@apache.org>
2018-06-18 18:35:12 +05:30
Dustin Rodrigues 021ee6faf2 NIFI-5317 - support non-ASCII X-Mailer PutEmail header
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2800.
2018-06-18 10:45:56 +02:00
Matthew Burgess 97f71fd6c7 NIFI-5044: Applied changes in SelectHiveQL to SelectHive3QL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2799.
2018-06-15 22:42:59 +02:00
Ed B 187417d077 NIFI-5044 SelectHiveQL accept only one statement
SelectHiveQL support only single SELECT statement.
This change adds support for pre- and post- select statements.
It will be useful for configuration queries, i.e. "set tez.queue.name=default", and others.

renamed selectQuery to hqlStatement for better readability

style check correction in comment

removed meaningless examples for post-queries

execute query without result set

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2695
2018-06-15 15:48:16 -04:00
Dongkyu Hwangbo 9b461027a4 NIFI-5188: DruidTranquilityController does not fully support Druid aggregator
Rollback Druid 0.9.2 to 0.9.1

Fixed checkstyle error

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2696
2018-06-15 09:52:14 -04:00
Koji Kawamura 52d6b9cfa2
NIFI-5257: Expand Couchbase Server integration
- Added CouchbaseMapCacheClient.
- Added CouchbaseKeyValueLookupService.
- Added CouchbaseRecordLookupService.
- Added 'Put Value to Attribute' to GetCouchbaseKey.
- Fixed Get/PutCouchbaseKey relationship descriptions.
- Stop sharing Relationship.Builders.

This closes #2750
2018-06-15 09:14:30 -04:00
Koji Kawamura f05c5e6ea0
NIFI-5054: Add Couchbase user authentication 2018-06-15 08:48:46 -04:00
Dustin Rodrigues 2277b9411e NIFI-5252 - support arbitrary headers in PutEmail processor
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2787.
2018-06-15 14:29:56 +02:00
Matthew Burgess 0886dcb0b4 NIFI-5303: Fixed QueryDatabaseTable to work with DB2
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2788.
2018-06-15 13:56:52 +02:00
zenfenan 3752398943 NIFI-5221 - Added 'Object Tagging' functionalities to S3 Processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2751.
2018-06-15 13:49:13 +02:00
Mike Thomsen be63378a1e NIFI-5214 Added REST LookupService
NIFI-5214 Added support for the new ProxyConfigurationService

NIFI-5214 Integration tests added.

NIFI-5214 Added missing pom.xml and a change from a code review.

NIFI-5214 Added another tag based on code review.

NIFI-5214 Added user-defined header support.

NIFI-5214 Added Basic Auth support.

NIFI-5214 Moved documentation.

NIFI-5214 Fixed checkstyle issues; added changes requested in a review.

NIFI-5214 Added changes requested in a code review.

NIFI-5214 Added verb test and @DynamicProperties

NIFI-5214 Added templated URL support to RestLookupService.

NIFI-5214 Fixed documentation based on code review changes.

NIFI-5214 Changed RestLookupService to use a property descriptor.

NIFI-5214 Updated documentation.

NIFI-5214 Made changes requested in code review.

NIFI-5214 Renamed nifi-standard-web-utils to reflect that it is for tests.

NIFI-5214: Refactor RestLookupService.

1. Added 'Base URL' property to address environment specific part of URL.
2. Removed 'Record Path Property Name' property, because the name of
a resulted record field of a record path can be obtained by field name.
3. Lower cased HTTP method name should be used throughout.
4. Added mimeType require check when body is specified.
5. Added debug log to print HTTP response code.
6. Prepare for NIFI-5287.
7. Fixed that mime.type being used regardless of whether body is
specified or not, caused NullPointerException when 'mime.type' is not
specified when it is not required.
8. Updated documentation.

NIFI-5214 Fixed AWS processors that broke w/ change to TestServer location.

NIFI-5214 Added changes requested in a code review.

Refactored the way to evaluate EL for URL property

- Use PropertyValue instead of PreparedQuery to utilize Variable
Registry.
- Removed BASE_URL because Variable Registry can be used at URL

NIFI-5214 Rebased to use new LookupService method.

This closes #2723.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-06-15 15:18:03 +09:00
Mark Payne d0499eb867
NIFI-5311: When creating a processor, controller service, or reporting task, give the component up to 50 ms to complete validation before returning the DTO. This closes #2795 2018-06-14 13:24:27 -04:00
Matt Gilman fe31a06fdc
NIFI-4907:
- Minor adjustments following PR.
- Avoiding additional find operation when authorizing components when populating component details.
- Requiring access to provenance events when downloading content or submitting a replay as they may provide events details.
- Updating the REST API docs detailing the required permissions.
- Updating the wording in the documentation regarding the provenance and data policies.
- Removed the event attributes from the authorization calls that were verifying access to provenance events.
- Only checking content availability when the user is authorized for the components data.
- Addressing typo in JavaDoc.

This closes #2703
2018-06-14 12:21:47 -04:00
Mark Bean e27798797a
NIFI-4907: add 'view provenance' component policy
whitespace removed for checkstyle
2018-06-14 10:59:17 -04:00
Marco Gaido d98d335497 [NIFI-3242] Avoid double scheduling of a task due to quartz imprecision
This closes #2789.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-06-14 10:10:45 -04:00
amitt90 8370a56dce NIFI-5309 update the logger message to get the output format from the outputFormat variable
This closes #2793

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-06-14 07:09:23 -04:00
Koji Kawamura 4349a8e078 NIFI-5231: CalculateRecordCount should use 'record.count'
This closes #2792

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-06-14 07:02:47 -04:00
Mike Thomsen 0831059d2c NIFI-5287 Made LookupRecord able to take in flowfile attributes and combine them with lookup keys.
NIFI-5287 Removed unneeded property descriptor.

NIFI-5287 Added additional changes from a code review.

This closes #2777.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-06-14 10:54:44 +09:00
thenatog dbf259508c
NIFI-5258 - Changed addHeader to setHeader which stops X-Frame-Options being added twice to responses. Added unit test.
This closes #2791.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-06-13 16:13:53 -07:00
Mark Payne 275b8cbf23
NIFI-5308: Avoid holding Template DOM Nodes in heap. This closes #2790 2018-06-13 16:00:01 -04:00
Matthew Burgess da99f873a7
NIFI-4963: Added Hive3 bundle
- Incorporated review comments
- Added more defensive code for PutHive3Streaming error handling

This closes #2755.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-06-13 14:32:58 -04:00
Matt Gilman 8feac9ae54
NIFI-5237:
- Ensuring the proxy headers are considered when redirecting the user following a OIDC or Knox login exchange.

This closes #2763.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-06-12 12:50:37 -04:00
Pierre Villard 09f9260035 NIFI-5293 - add merge.uuid to MergeRecord processor
This closes #2783

Signed-off-by: zenfenan <zenfenan@apache.org>
2018-06-12 19:16:17 +05:30
Pierre Villard ee18ead16c NIFI-5297 - EL support in ScanAttribute
This closes #2786

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-06-11 18:48:38 -04:00
Otto Fowler 946c2fe6b3 NIFI-5022 This closes #2588. InvokeAWSGatewayApi processor
per review, simplify header building and remove new map
Per review:
- removed apache headers from 3rd party files per https://www.apache.org/legal/src-headers.html#3party
- referenced commit/repo in notice file
- created rat configuration to account for files without headers

Signed-off-by: joewitt <joewitt@apache.org>
2018-06-11 10:49:41 -04:00
Mark Payne e3b0949b6b NIFI-4508: This closes #2784. Update ConsumeAMQP to use basicConsume API instead of basicGet in order to provide better performance
NIFI-4508: Added some additional unit tests to verify behavior

Signed-off-by: joewitt <joewitt@apache.org>
2018-06-11 10:27:28 -04:00
Junegunn Choi 18a4819d51 NIFI-5290 - PutKudu should support unixtime_micros columns
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2781.
2018-06-11 09:40:38 +02:00
Mike Thomsen 1803c15bcd NIFI-5231 Added RecordStats processor.
NIFI-5231 Minor fixes from code review.

NIFI-5231 Added documentation and made record_count a constant.

NIFI-5231 Added changes requested in a code review.

NIFI-5231 Added changes based on code review.

This closes #2737.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-06-11 13:35:01 +09:00
Mike Thomsen 5a39d2a819 NIFI-5288 Quietly convert Java arrays to Lists so the MongoDB API can handle them.
NIFI-5288 Made changes based on code review.

NIFI-5288 Theoretically supports nested arrays.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2778
2018-06-10 22:10:12 -04:00
Aldrin Piri ead3969ab7
NIFI-5006 Updating NOTICEs to 2018
This closes #2779.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-06-10 13:09:13 -07:00
zenfenan 794a642310 NIFI-5284: Added JSON_TYPE support to RunMongoAggregation
This closes #2776

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-06-10 06:45:21 -04:00
zenfenan cf3c666683 NIFI-5286: Updated FasterXML Jackson libraries to 2.9.5
This closes #2775

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-06-09 14:19:45 -04:00
Mark Payne 00a63d17af NIFI-5200: Fixed issue with InputStream being closed when calling ProcessSession.read() twice against sequential Content Claims
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2753
2018-06-08 16:47:28 -04:00
Pierre Villard 05d7b6c6e7 NIFI-4262 - MergeContent - option to add merged uuid in original flow files
This closes #2056

Signed-off-by: zenfenan <zenfenan@apache.org>
2018-06-08 20:54:47 +05:30
Pierre Villard 6e067734d5 NIFI-5264 - Added attribute for validation error message in ValidateCSV
This closes #2769

Signed-off-by: zenfenan <zenfenan@apache.org>
2018-06-08 19:16:20 +05:30
Mark Payne 49228aa5dc NIFI-5281: If value is not valid according to the schema's CHOICE field, JSON Writer should write null value instead of throwing NullPointerException
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2772.
2018-06-08 10:07:43 +02:00
Matthew Burgess 0e09b98b02 NIFI-5143: Initial work to support column values for paging results
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2728.
2018-06-08 09:36:27 +02:00
Mark Payne 729f8aa246
NIFI-5279: Allow components up to 50 milliseconds to complete validation before returning from update request
This closes #2770.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-06-07 16:32:19 -07:00
Matt Gilman 7b9d779a4b NIFI-3217:
- Disabling the resize of SlickGrid when there is an active edit occuring. This will prevent the accidental closure of the current edit.
- Triggering a resize when the active editor closes.

This cloese #2766

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2018-06-07 18:26:13 -04:00
Joe Trite f5364875e8 Squashed commit of the following:
commit 9c31e45d3f
Author: Joe Trite <joetrite@gmail.com>
Date:   Mon Mar 13 07:55:19 2017 -0400

    NIFI-1705 Adding AttributesToCSV processor

commit 5e9afa9ccf
Merge: 3177eb185 74cbfc4b6
Author: Joe Trite <joetrite@gmail.com>
Date:   Sat Mar 4 08:12:39 2017 -0500

    Merge remote-tracking branch 'origin/master'

    # Conflicts:
    #	nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ScanAttribute.java

commit 3177eb1852
Author: Joe Trite <joetrite@gmail.com>
Date:   Sat Mar 4 08:09:06 2017 -0500

    NIFI-3497 Converted properties to use displayName, changed validator on demarcator property and created a method for duplicate code.

commit 74cbfc4b69
Merge: a974f78e0 f8cad0f8c
Author: Joe Trite <joetrite@gmail.com>
Date:   Sat Mar 4 07:47:46 2017 -0500

    Merge branch 'master' into master

commit a974f78e03
Author: Joe Trite <joetrite@gmail.com>
Date:   Sat Mar 4 07:43:02 2017 -0500

    NIFI-3497 Converted properties to use displayName, changed validator on demarcator property and created a method for duplicate code.

commit 1bfaef8e87
Merge: 65ed46de9 89ec68d14
Author: Joe Trite <joetrite@gmail.com>
Date:   Fri Mar 3 08:01:59 2017 -0500

    Merge branch 'master' of https://github.com/joetrite/nifi

commit 65ed46de9a
Author: Joe Trite <joetrite@gmail.com>
Date:   Fri Feb 24 18:09:36 2017 -0500

    NIFI-3497 - fixing Pcontrib issues

commit c5d52cf6f0
Author: Joe Trite <joetrite@gmail.com>
Date:   Thu Feb 23 10:19:01 2017 -0500

    NIFI-3497 - excluding test files

    Adding new test data files to exclude list.

commit b195934113
Author: Joe Trite <joetrite@gmail.com>
Date:   Wed Feb 22 16:48:10 2017 -0500

    NIFI-3497 - New dictionary files for test

    Adding new dictionary files to support metadata dictionary option.

commit e296268f39
Author: Joe Trite <joetrite@gmail.com>
Date:   Wed Feb 22 16:46:13 2017 -0500

    NIFI-3497 test cases for metadata updates

    Adding test cases to support metadata option update.

commit de7e348e62
Author: Joe Trite <joetrite@gmail.com>
Date:   Wed Feb 22 16:36:08 2017 -0500

    NIFI-3497 - Added metadata option

    Added optional to post additional metadata as new attributed if a match is found in the dictionary.

commit 89ec68d14b
Author: Joe Trite <joetrite@gmail.com>
Date:   Fri Feb 24 18:09:36 2017 -0500

    NIFI-3497 - fixing Pcontrib issues

commit d71426037b
Author: Joe Trite <joetrite@gmail.com>
Date:   Thu Feb 23 10:19:01 2017 -0500

    NIFI-3497 - excluding test files

    Adding new test data files to exclude list.

commit a7a7b6ace8
Author: Joe Trite <joetrite@gmail.com>
Date:   Wed Feb 22 16:48:10 2017 -0500

    NIFI-3497 - New dictionary files for test

    Adding new dictionary files to support metadata dictionary option.

commit 8eb54a5019
Author: Joe Trite <joetrite@gmail.com>
Date:   Wed Feb 22 16:46:13 2017 -0500

    NIFI-3497 test cases for metadata updates

    Adding test cases to support metadata option update.

commit f52e1f2a06
Author: Joe Trite <joetrite@gmail.com>
Date:   Wed Feb 22 16:36:08 2017 -0500

    NIFI-3497 - Added metadata option

    Added optional to post additional metadata as new attributed if a match is found in the dictionary.

NIFI-1705 Adding AttributesToCSV processor.

NIFI-1705 Adding AttributesToCSV processor.

NIFI-1705 Adding AttributesToCSV processor.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2711
2018-06-07 14:48:53 -04:00
Matt Gilman c412445a1f NIFI-5280:
- Preventing duplicate bulletins that are registried under different contexts (e.g. Controller and Component).

This closes #2771.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-06-07 13:51:47 -04:00
Mike Thomsen d2fd7e5e7d NIFI-5271 Moved JSON_VALIDATOR to its own maven module.
This closes #2765

Signed-off-by: zenfenan <zenfenan@apache.org>
2018-06-07 20:38:25 +05:30
Mark Payne ff00050266
NIFI-5241: Updated EventSumValue to use synchronized methods instead of many atomic values. This is more efficient and uses less heap. Also noticed that the Logger instance in ProcessorNode was not used so removed it, and in testing this also noticed that the default connection pool size for OkHttpReplicationClient was only 5, which can cause a lot of unnecessary HTTP connections to be created so adjusted the pool size
NIFI-5241: Extended timeout that Jetty uses before closing an active HTTP connection. Because the UI refreshes every 30 seconds by default, and the Jetty connection pool times out every 30 seconds by default, we very frequently saw new HTTP connections being created for the UI refreshes. This resulted in 4 new connections and 4 SSL handshakes occurring every 30 seconds. By extending the timeout, we now see those connections being reused and SSL Handshakes no longer occurring frequently
NIFI-5241: Set Jetty idle timeout to double the amount of time for browser to refresh
NIFI-5241: Fixed synchronization issue with EventSumValue
This closes #2752
2018-06-07 09:15:32 -04:00
Otto Fowler f7f809c3d3 NIFI-4272 support multiple captures when EL is present in replacement value
This closes #2748

Signed-off-by: Mike Moser <mosermw@apache.org>
2018-06-06 17:23:20 +00:00
Marco Gaido eedf1237a1 NIFI-5262 - Retrieve file attributes only once in ListFile
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2754.
2018-06-06 10:10:29 +02:00
Matt Gilman c3af53ce65 NIFI-5263 - Fixing the advice auditing the method for updating controller service referencing components.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2756.
2018-06-06 10:03:42 +02:00
Koji Kawamura 71cf3fd46d NIFI-5268: Fix JostTransformJSON spec validation with EL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2762.
2018-06-06 09:37:02 +02:00
zenfenan 504152eaa1 NIFI-5261: Added JSON_VALIDATOR to StandardValidators
This closes #2758

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-06-05 18:24:47 -04:00
Matt Gilman cb216b79ec NIFI-5208:
- Ensuring nf-storage is injected where necessary.

This closes #2757

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2018-06-05 09:33:25 -04:00
Peter Toth a1794b101e NIFI-5041: Adds SPNEGO authentication to LivySessionController
NIFI-5041: fixes http client version issue

Change-Id: I1b87ec4752ff6e1603025883a72113919aba5dd4

NIFI-5041: fixes Kerberos configuration

Change-Id: I868fdf3ea7cfd28cf415164e420f23bf3f6eefeb

NIFI-5041: adds new NOTICE entries

NIFI-5041: yields processor if no session is available, fixes error handling in session manager thread, fixes error returned in KerberosKeytabSPNegoScheme on authentication failure

Change-Id: I443e063ae21c446980087e5464a4b70373d730f6

NIFI-5041: makes the session manager thread exceptions visible to the users

Change-Id: I33fde5df6933cec2a87a4d82e681d4464f21b459

NIFI-5041: adds special SessionManagerException to identify error occurred on session manager thread

Change-Id: I25a52c025376a0cd238f14bda533d6f5f3e5fb4a

This closes #2630

Signed-off-by: Matthew Burgess <mattyb149@apache.org>
2018-05-31 11:07:11 -04:00
Mike Thomsen 83fccc4beb NIFI-5024 Fixed a checkstyle issue that was missed in the merge of 5024. 2018-05-31 08:35:25 -04:00
Nicolas Sanglard 498fd8f22d NIFI-5024 Resolves deadlock in ExecuteStreamCommand processor
This closes #2594

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-31 06:35:39 -04:00
Bryan Bende f69b720464
NIFI-5148 Refactoring Kerberos auth for Solr processors
- Created resuable KeytabUser and KeytabConfiguration in nifi-security-utils
- Refactored Solr processors to use a KeytabControllerService and no longer rely on JAAS system property
- Wrapped all calls in SolrProcessor onTrigger in a doAs when kerberos is enabled
- Added IT tests against MiniKDC
- This closes #2674
2018-05-30 14:22:28 -04:00
Matt Gilman 9093d280f2 NIFI-5208:
- Removing test for unsupported functionality.

This closes #2740

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2018-05-30 13:41:20 -04:00
Matt Gilman 05a1d63090 NIFI-5208:
- Fixing missing message when a node is disconnected from a cluster.
- Updating endpoints to accept a flag to allow for changes to be made to a disconnected node.
- Updating custom UIs to acknowledge disconnected nodes prior to performing modifications.
2018-05-30 13:41:03 -04:00
Seiji Sogabe 3a248e96de NIFI-5245: SimpleCSVFileLookupService should take account of charset
This closes #2745

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-30 07:10:00 -04:00
Pierre Villard 1d21e3baf8 NIFI-5122 - Add Record Writer for S2S RTs
This closes #2663

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-30 06:58:01 -04:00
zenfenan 1663a6c094 NIFI-5133: Implemented Google Cloud PubSub Processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2724.
2018-05-30 09:47:18 +02:00
Scott Aslan 9aa7e65f70
[NIFI-5215] Upgrade AngularJS to v1.7.0. Also, adding package-lock.json to source control
[NIFI-5215] add package-lock.json to rat excludes
This closes #2721
2018-05-29 13:20:49 -04:00
Mike Thomsen 688f906a73
NIFI-5244 Fixed a bug in MockSchemaRegistry that prevented it from loading using the schema name strategy.
This closes #2742.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-05-29 10:31:14 -04:00
Matthew Burgess 65beec39bb NIFI-5242: Exclude JTS dependency from Elasticsearch client
This closes #2741

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-27 19:00:34 -04:00
Mark Payne eb0b4283e8 NIFI-5222: Prevent validating components multiple times for each update
- Avoid triggering async validation for each update to component when instantiating a template (such as copy/paste or templates). Added debug logging to indicate when and why we are triggering validation; removed unit test that made poor assumptions about the inner workings of the FlowSynchronizer that resulted in failures when we make calls into processors that the unit test doesn't know about"

This closes #2731.
2018-05-25 13:44:15 -04:00
zenfenan 0973c2d8d1 NIFI-543 Added annotation to restrict processor to run only on the primary node
- PR Fix - 'Execution' dropdown will now be shown in all cases
- Annotated ListGCSBucket with PrimaryNodeOnly

This closes #2509.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-05-25 11:50:29 -04:00
Matt Gilman dbef69040f NIFI-5066:
- Fixing predicate for finding all processors/ports that can be disabled.

This closes #2739.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-05-25 11:22:21 -04:00
Matt Gilman 18ec386150 NIFI-5066:
- Ensuring we verify we can enable/disable when appropriate.
2018-05-25 11:22:21 -04:00
Pierre Villard 64cd34016b NIFI-5233 - Add EL support with Variable Registry scope for HBase client service
This closes #2738

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-25 10:37:45 -04:00
Bryan Bende cf6089196f NIFI-5229 Adding a DBCPService implementation that can lookup other DBCPServices dynamically at runtime
This closes #2735

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-25 08:54:03 -04:00
Mark Payne 397e88c858 NIFI-4227: Fixed typo 2018-05-24 15:57:28 -04:00
Pierre Villard be0ed70423 NIFI-4227 - add a ForkRecord processor
Added split/extract modes, unit tests, and additional details

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-05-24 15:55:17 -04:00
veteranbv 3b3d6d4eb2 NIFI-5171: fixed Yandex Jersey issues by adding dependency to POM and modified API call to now detect for languages
Added license to YandexTranslate NOTICE file
Updated to use StringUtils.isBlank for detecting sourceLanguage field being blank and languages to file to align with new logic

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-05-24 14:35:33 -04:00
Mark Payne 1597492fed NIFI-5073: Removed unneeded code for changing ClassLoader since it is now managed by framework
This closes #2653.
2018-05-24 11:30:39 -04:00
zenfenan db259628c7 NIFI-5073: JMSConnectionFactoryProvider now resolves EL Expression from VariableRegistry
- CLIENT_LIB_PATH is updated to include 'dynamicallyModifiesClasspath(true)'
2018-05-24 11:29:47 -04:00
Mike Thomsen 716587d09f NIFI-5169 This closes #2705. Upgrade to JSONPath 2.4
NIFI-5169 Added explicit checking for empty strings in JsonPath expression property.

Signed-off-by: joewitt <joewitt@apache.org>
2018-05-23 16:16:25 -05:00
zenfenan 06d45c3a6e
NIFI-4987: Added TTL to RedisDistributedMapCacheClientService
NIFI-4987: PR Review Fixes - Reverted getAndPutIfAbsent and added TTL setting with a different approach

NIFI-4987: PR Review Fixes - Added TTL to putIfAbsent()

This closes #2726.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-05-23 15:28:39 -04:00
Frederik Petersen d75ba167cd NIFI-5225: Purge event data from event repository when Connectable is removed
This closes #2732.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-05-23 13:39:20 -04:00
Mark Payne cca520aab7
NIFI-5228: Make the storing of file attributes as FlowFile attributes optional for ListFile
NIFI-5228: Addressed review feedback
This closes #2733
2018-05-23 10:50:01 -04:00
Mark Payne 167014441b
NIFI-5191: Revert "NIFI-4631: Use java.nio.file.Files in ListFile to improve performance"
This reverts commit 758e44682f.
2018-05-23 10:50:01 -04:00
Matthew Burgess 23feb8f429 NIFI-5230: Fixed NPE in InvokeScriptedProcessor on customValidate
This closes #2734

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-23 10:00:25 -04:00
Mike Thomsen 3a29c1e4ca NIFI-5052 Added DeleteByQuery ElasticSearch processor.
NIFI-5052 fixed checkstyle issue.

NIFI-5052 Added changes requested in a code review.

NIFI-5052 Fixed a typo.

NIFI-5052 Added changes requested in a code review.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2616
2018-05-23 09:14:04 -04:00
Koji Kawamura f7035f0497 NIFI-4637: Fix ExtensionManager warning on VisibilityLabelService
This closes #2729

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-23 07:34:49 -04:00
Max Viazovskyi dd8b25ab48 NIFI-5109 Reset justElectedPrimaryNode flag right after reelection happen
Read full cluster state and return if no update required

This closes #2657.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-05-23 08:52:49 +09:00
Matt Gilman 0305ca0fb8 NIFI-5186:
- Addressing sort issue discovered during PR.

This closes #2722

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2018-05-22 11:05:35 -04:00
Matt Gilman 568288dbcd NIFI-5186:
- Updating UI to support showing when a component is validating.
2018-05-22 11:05:28 -04:00
Andrew Psaltis 6356d7b9ee NIFI-4988 This closes #2725. Changed the exception handling so that an invliad ZIP is handled. When an invalid zip is processed, the exception is an IllegalArgumentException which was not being handled and thus the session was being rollbacked.
Signed-off-by: joewitt <joewitt@apache.org>
2018-05-22 10:24:57 -04:00
Otto Fowler 6bf6a92abd NIFI-5220: add validation for host and password, and tests
This closes #2727.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-05-22 09:28:30 +09:00
Koji Kawamura 2834fa4ce4 NIFI-4199: Added ProxyConfigurationService
- Added ProxyConfigurationService to manage centralized proxy
configurations
- Adopt ProxyConfigurationService at FTP and HTTP processors

NIFI-4175 - Add HTTP proxy support to *SFTP processors

This closes #2018.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>

NIFI-4199: Add ProxyConfigurationService to SFTP processors

- Fixed check style issue
- Use the same proxy related PropertyDescriptors from FTPTransfer and
SFTPTransfer
- Dropped FlowFile EL evaluation support to make it align with other
processors spec, Now it supports VARIABLE_REGISTRY
- Added ProxyConfigurationService to SFTP processors
- Added SOCKS proxy support to SFTP processors

NIFI-4199: Added ProxyConfigurationService to ElasticsearchHttp processors

- ElasticsearchHttp processors now support SOCKS proxy, too
- Added proxy support to PutElasticsearchHttpRecord
- Moved more common property descriptors to
AbstractElasticsearchHttpProcessor and just return static unmodifiable
property descriptor list at each implementation processors

NIFI-4196 - Expose AWS proxy authentication settings

NIFI-4196 - Fix jUnit errors

This closes #2016.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>

NIFI-4199: Add ProxyConfigService to AWS processors

- Applied ProxyConfigService to S3 processors
- Added proxy support to following processors:
  - PutKinesisFirehose, PutKinesisStream
  - PutDynamoDB, DeleteDynamoDB, GetDynamoDB
  - PutKinesisStream
- All AWS processors support HTTP proxy now

NIFI-4199: Add ProxyConfigService to Azure processors

NIFI-4199: More explicit validation and docs for Proxy spec

- Each processor has different supporting Proxy specs
- Show supported spec to ProxyConfigurationService property doc
- Validate not only Proxy type, but also with Authentication

NIFI-4199: Incorporated review comments

- Fixed TestListS3 property descriptor check
- Separate name and displayName

This closes #2016
This closes #2018
This closes #2704

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-20 17:57:47 -04:00
Matthew Burgess d79216d6b0 NIFI-5217: Allow non-nullable fields to be added to MockRecordParser
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2720
2018-05-18 15:59:12 -04:00
Mark Payne 2afbf96381
NIFI-5204: Ensure that verifyCanStop throws ISE if component is disabled
NIFI-5204: If processor joins cluster and inherits 'disabled' state but is still stopping, ensure that the state becomes disabled when the processor finishes stopping
This closes #2713
2018-05-17 13:44:52 -04:00
Mike Thomsen f5108ea839 NIFI-5197 Fixed a bunch of EL scope bugs.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2712
2018-05-17 09:57:35 -04:00
Matthew Burgess 5613bf4012 NIFI-5202: Fixed timing issue in TestListDatabaseTables
This closes #2709

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-17 07:10:35 -04:00
Jeff Storck 807e1e5c7a NIFI-5175 Updated NiFi compiled on Java 1.8 to run on Java 9
The bootstrap process (RunNiFi) detects Java 9 and adds "--add-modules=java.xml.bind" to the command to start NiFi
Updated OSUtils to detect Java 9 and reflectively invoke the Process.pid() method to get the PID of the NiFi process
Added java debug variable to nifi.sh to allow debugging of the bootstrap process (RunNiFi)

This closes #2708

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-17 07:00:46 -04:00
Mark Payne 604656fe88
NIFI-950: Make component validation asynchronous
NIFI-950: Still seeing some slow response times when instantiating a large template in cluster mode so making some minor tweaks based on the results of CPU profiling
NIFI-5112: Refactored FlowSerializer so that it creates the desired intermediate data model that can be serialized, separate from serializing. This allows us to hold the FlowController's Read Lock only while creating the data model, not while actually serializing the data. Configured Jersey Client in ThreadPoolRequestReplicator not to look for features using the Service Loader for every request. Updated Template object to hold a DOM Node that represents the template contents instead of having to serialize the DTO, then parse the serialized form as a DOM object each time that it needs to be serialized.
NIFI-5112: Change ThreadPoolRequestReplicator to use OkHttp client instead of Jersey Client
NIFI-5111: Ensure that if a node is no longer cluster coordinator, that it clears any stale heartbeats.
NIFI-5110: Notify StandardProcessScheduler when a component is removed so that it will clean up any resource related to component lifecycle.
NIFI-950: Avoid gathering the Status objects for entire flow when we don't need them; removed unnecessary code
NIFI-950: Bug fixes
NIFI-950: Bug fix; added validation status to ProcessorDTO, ControllerServiceDTO, ReportingTaskDTO; updated DebugFlow to allow for pause time to be set in the customValidate method for testing functionality
NIFI-950: Addressing test failures
NIFI-950: Bug fixes
NIFI-950: Addressing review feedback
NIFI-950: Fixed validation logic in mock framework
This closes #2693
2018-05-16 14:39:23 -04:00
Mike Thomsen ea41b41ede NIFI-5145 Made MockPropertyValue.evaluateExpressionLanguage(FlowFile able to accommodate null flowfiles the way live NiFi does.
NIFI-5145 Fixed a mistake with evaluateAttributeExpressions found in a code review.

NIFI-5145 Removed a property descriptor from GetHBase that was a duplication of the one in VisibilityFetchSupport.

NIFI-5145 Added comments in the code that were requested in the review.

NIFI-5145 Fixed check style issue.

NIFI-5145 Fixed a few improperly scoped items in ListFileTransfer that impacted the FTP processor(s).

NIFI-5145 Changed which override gets called based on code review feedback.

NIFI-5145: Reverted changes to ListFileTransfer re ExpressionLanguageScope

NIFI-5145: Removed TODO from MockPropertyValue javadoc

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2672
2018-05-16 13:50:27 -04:00
Mark Payne 0688363d87 NIFI-5200: Fixed bug that caused the wrong InputStream to be closed by StandardProcessSession if calling Session.read() from the callback of another Session.read(); also changed default of the 'allowSessionStreamManagement' flag from 'false' to 'true' as it will provide performance benefits in some cases and NIFI-516 outlined that it should be 'true' initially but ended up being false when the PR was merged.
This closes #2707

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-16 12:27:19 -04:00
James Wing 10254a03c2 NIFI-4015 NIFI-3999 Fix DeleteSQS Issues
* Avoid exception by providing id in DeleteMessageBatchRequestEntry
* Include receipt handle property descriptor for user configuration

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1888.
2018-05-16 18:15:20 +02:00
Matt Gilman c3059939e8 NIFI-5173:
- Removing unnecessary logic in the zoom handler since the zoom event is no longer triggered during onClick.

This closes #2692

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2018-05-16 11:12:16 -04:00
Koji Kawamura de1ad3eb6f NIFI-4198: ElasticsearchHttp processors Proxy settings
- Added Expression Language support with variable registry at proxy user
and password
2018-05-16 14:02:28 +09:00
Arun Manivannan 01a01a3dfc NIFI-4198 *ElasticsearchHttp processors do not expose Proxy settings
This closes #2094.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-05-16 14:01:19 +09:00
Otto Fowler ccedc71c8b NIFI-5077 ExtractGrok support for `keep empty captures`
Support for the new option to keep empty captures.  I did not add to the GrokReader because I am not sure of the effect on
the schema extraction.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2698.
2018-05-15 19:24:10 +02:00
JohannesDaniel 2c8c9374af NIFI-5113: Add XMLRecordSetWriter
This closes #2675.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-05-15 11:07:19 -04:00
Mark Payne 620d446860 NIFI-5194: Ensure that even if calling KafkaConsumer.resume() throws an Exception, that we still release the lock that we are holding
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2701.
2018-05-15 11:52:55 +02:00
Koji Kawamura 79a7014a95 NIFI-4971: ReportLineageToAtlas complete path can miss one-time lineages
- Separated Hook message de-duplication logic from NiFiAtlasHook to
NotificationSender
- NiFiAtlasHook used to send individual CREATE_ENTITY messages for each entities,
this commit changed it to bundle all new entities into a single
CREATE_ENTITY to preserve entity creation order, so that new DataSet
entities can be referred from new nifi_flow_path entities
- Added more unit tests

This closes #2542

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-15 05:50:26 -04:00
Fokko Driesprong ed9263811b [NIFI-5190] Bump Apache Parquet to 1.10.0
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2697.
2018-05-15 11:40:11 +02:00
JohannesDaniel 04745a556a NIFI-5189 Schema name is not available for RecordSchema
This closes #2700

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-14 14:35:12 -04:00
Koji Kawamura 03adaeca22 NIFI-5114: Added Basic auth support to WebSocket components
This closes #2652

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-14 14:26:47 -04:00
Otto Fowler 61fe493786 NIFI-5170 Upgrad Grok to version 0.1.9
This closes #2691

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-13 08:40:20 -04:00
Gardella Juan Pablo 64356e0014 NIFI-5049 Fix handling of Phonenix datetime columns
This closes #2625

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-11 09:51:24 -04:00
Matthew Burgess 099bfcdf3a NIFI-5121: Added DBCPService API method for passing in flow file attributes when available
This closes #2658

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-11 08:19:16 -04:00
Koji Kawamura 500a254e3f NIFI-4637: HBase visibility lables
Fixed additional docs directory structure.
2018-05-11 07:30:19 +00:00
Mike Thomsen 0b851910f3 NIFI-4637 Added support for visibility labels to the HBase processors.
NIFI-4637 Removed integration test and updated the hbase client from 1.1.2 to 1.1.13 which is the final version for 1.1.X

NIFI-4637 Fixed EL support issue w/ tests.

NIFI-4637 Added more documentation to DeleteHBaseCells.

NIFI-4637 changed PutHBaseCell/JSON to use dynamic properties instead of a 'default visibility string.'

NIFI-4637 Added changes requested in a code review.

NIFI-4637 Moved pickVisibilityString to a utility class to make testing easier.

NIFI-4637 Added additionalDetails.html for PutHBaseRecord.

NIFI-4637 Added additional documentation and testing.

NIFI-4637 Added documentation for DeleteHBaseCells.

NIFI-4637 Added pickVisibilityLabel support to PutHBaseRecord and updated documentation to reflect that.

NIFI-4637 Reverted version bump to hbase client.

This closes #2518.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-05-11 00:55:40 +00:00
himanshu 6e0be8e641 NIFI-4938 Upgraded org.eclipse.paho.client.mqttv3 dependency version to 1.2.0
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2514.
2018-05-10 11:04:32 +02:00
Mark Bean def9741a39 NIFI-5180: update JMS additional details to set Destination Type to Required, default 'QUEUE'
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2694.
2018-05-10 10:47:20 +02:00
Matt Gilman 22342a0e0c NIFI-5167:
- Updating how the nf-reporting-task module is injected to the nf-controller-service module.

This closes #2688

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2018-05-09 14:06:19 -04:00
Junegunn Choi 02ba4cf2c8 NIFI-5064 - Fixes and improvements to PutKudu processor
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2621.
2018-05-09 19:39:29 +02:00
patricker 0a44bad76e NIFI-5168 - ReplaceText Processor Should Use Single FlowFile Processing Instead of Batch
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2687.
2018-05-09 18:46:35 +02:00
Mark Payne 4700b8653d NIFI-5138: Bug fix to ensure that when we have a CHOICE between two or more REOCRD types that we choose the appropriate RECORD type when creating the Record in the JSON Reader.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2670.
2018-05-09 18:38:04 +02:00
zenfenan f742a3a6ac NIFI-5156: Updated GCP SDK to latest version
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2680.
2018-05-09 18:19:17 +02:00
Andy LoPresto 7a4990e7fe
NIFI-5146 Only support HTTP or HTTPS operation for NiFi API/UI
- Added logic to check for simultaneous configuration of HTTP and HTTPS connectors in JettyServer.
- Added test logging resources. Added unit tests.
- Refactored shared functionality to generic method which accepts lambdas.
  Fixed unit test with logging side effects.
- Added note about exclusive HTTP/HTTPS behavior to Admin Guide. Fixed typos.

This closes #2683.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2018-05-09 12:14:11 -04:00
Bryan Bende 92b4a3208f NIFI-5136 Ensure processor references are removed from LogRepository and from ProcessScheduler
- Forcing FileSystem statistics thread to be interrupted when HDFS processors are stopped
- Stop creating temp components during import from registry, use bundle info instead

This closes #2668.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-05-08 15:29:37 -04:00
Bryan Bende a9e85c358c
NIFI-5163 Clearing version control info when creating a template
This closes #2685.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2018-05-07 16:13:26 -04:00
Mark Payne fb48ae2f88
NIFI-5154: When Processor or Controller Service is added to a Process Group, remove any references from it to any other Controller Service that is not reachable from the newly assigned Process Group
Fixed bug in unit test
Addressed review feedback/addressed issue where if a group is moved inside another group, the descendant processors of the moved group did not have their service references updated properly. Also addressed an issue where if a service is defined in Group A, then Group B lives within Group A and has a processor that references a service at the level of Group A, we allowed user to move Group B outside of Group A (but wouldn't allow the processor to be moved out of scope by itself).
This closes #2678
2018-05-07 16:01:26 -04:00
Mark Payne 8acac9cba5
NIFI-5153: If a node is disconnected due to failure to complete mutable request, the node should be allowed to rejoin
This closes #2677.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-05-07 15:41:38 -04:00
zenfenan 4544f3969d
NIFI-5152: MoveHDFS now works even with no upstream connection
This closes #2681.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-05-07 13:20:56 -04:00
Mark Payne 868808f4b4
NIFI-5150: Fixed bug that caused StandardProcessSession.append() to copy too much data when called on an incoming flowfile
This closes #2676.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-05-07 11:44:52 -04:00
Matt Gilman 5cfa29e48f
NIFI-5135:
- Adding support for applying transforms to user identities and group names.

This closes #2673.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-05-07 11:17:26 -04:00
Koji Kawamura ff04f8efee NIFI-4980: Typo in ReportAtlasLineage kafka kerberos service name property
This closes #2550

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-07 06:08:59 -04:00
Michal Misiewicz ed30bb9b78 NIFI-5130 ExecuteInfluxDBQuery processor chunking support
This closes #2666

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-04 17:32:10 -04:00
Jeff Storck 3ad3243511 NIFI-5134 Explicitly requesting UGI to relogin before attempting to get a DB connection in HiveConnectionPool
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2667
2018-05-03 09:12:07 -04:00
Koji Kawamura 2007c207ab NIFI-4393: Handle database specific identifier escape characters
QueryDatabaseTable and GenerateTableFetch processors were not able to
use max value state as expected, if max value column was wrapped with
escape characters. Due to a mis-match between computed state keys
and actual keys used in the managed state. State keys computed by
getStateKey method included escape characters while actual stored keys
did not. Resulted querying the same dataset again and again.

This commit added unwrapIdentifier method to DatabaseAdapter class to
remove database specific escape characters for identifiers such as table
and column names, so that max value state keys are populated correctly
even if identifiers are wrapped with escape characters.

This commit also added new DatabaseAdapter for MySQL, to handle MySQL
specific identifier escape with back-ticks.

This closes #2424

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-03 09:06:05 -04:00
Mark Payne 18ad348107
NIFI-5142: Do not allow a connection's destination to be changed to a funnel if the source is the same funnel. Also fixed some typos in StandardFunnel. This closes #2669 2018-05-02 15:53:52 -04:00
Matthew Burgess 0289ca7114 NIFI-4456: Support multiple JSON objects in JSON record reader/writers
This closes #2640.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-05-02 15:22:01 -04:00
Matt Gilman 4ed6631edc
NIFI-5124:
- Upgrading to the latest version of commons-fileupload.

This closes #2662.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-05-02 08:20:33 -07:00
zenfenan 579078e352 NIFI-5129: AWS Processors now displays proper region names
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2665.
2018-04-29 17:06:10 -07:00
zenfenan ba84ad1e38 NIFI-5105: Improvements for nifi-aws-bundle
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2664.
2018-04-28 14:03:14 -07:00
Koji Kawamura 6e8b1c8f74 NIFI-5090: Create move target dir dynamically at FetchFTP and FetchSFTP
Support creating target parent directories even if directory listing is disabled.

fixed typo in doc

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2642
2018-04-27 11:48:26 -04:00
Matthew Burgess 159b64b4c8
NIFI-5123: Move SchemaRegistryService to nifi-avro-record-utils
This closes #2661.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-04-26 11:50:01 -04:00
sbouchex 3719a6268c NIFI-5120 AbstractListenEventProcessor supports expression language
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2659.
2018-04-26 14:05:53 +02:00
Pierre Villard 23937835f3 NIFI-5092 - Removed local state management for S2S Bulletins RT
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2643
2018-04-25 16:25:00 -04:00
Matt Gilman af2513adf8 NIFI-1295:
- Adding UI controls for terminating hung threads.
- Showing current number of terminated threads.
- Fixing issue when replicating terminate threads request throughout the cluster.

This closes #2607.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-04-25 16:23:34 -04:00
Jeff Storck 88c079e447
NIFI-4923 Updated nifi-hadoop-libraries-nar, nifi-hdfs-processors, and nifi-hadoop-utils dependency on hadoop-client from 2.7.3 to 3.0.0
This closes #2582.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-04-25 15:34:14 -04:00
Matt Gilman 6938e58c81 NIFI-5066:
- Allowing the enable/disable buttons to be active under the same conditions as the start/stop buttons.

This closes #2633.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-04-25 14:58:29 -04:00
zenfenan 72f8999b15 NIFI-5015: Implemented Azure Queue Storage processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2611.
2018-04-25 20:07:15 +02:00
zenfenan c118e96238 NIFI-5000 - ListHDFS properly lists files from updated directory path
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2576.
2018-04-25 18:03:44 +02:00
joewitt ac9944ccee
NIFI-5108 Updated all explicit refs and media nar usage of commons-compress to latest version and updated spring redis client
This closes #2651.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-04-24 15:16:04 -04:00
abhinavrohatgi30 e3f4720797
NIFI-4035 - Adding PutSolrRecord Processor that reads NiFi records and indexes them into Solr as SolrDocuments
Adding Test Cases for PutSolrRecord Processor

Adding PutSolrRecord Processor in the list of Processors

Resolving checkstyle errors

Resolving checkstyle errors in test classes

Adding License information and additional information about the processor

1. Implementing Batch Indexing 2. Changes for nested records 3. Removing MockRecordParser

Fixing bugs with nested records

Updating version of dependencies

Setting Expression Language Scope

This closes #2561.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-04-24 10:50:40 -04:00
patricker 0390c0f196 NIFI-4561 ExecuteSQL returns no FlowFile for some queries
This closes #2243

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-04-24 10:27:44 -04:00
Mike Thomsen 5ca6261de0 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2018-04-23 15:39:20 -04:00
Otto Fowler 45bc1f1b4f NIFI-3576 Support for QueryInfo relationship, can be used to track no-hits
Squashed commit includes related commit from GitHub user wietze.

This closes #2601

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-04-23 15:38:27 -04:00
Mark Payne 0e736f59fd NIFI-4185: Minor tweaks to XML Record Reader, around documentation and error handling
This closes #2587.
2018-04-23 14:41:17 -04:00
JohannesDaniel d21bd3870b NIFI-4185: Add XML Record Reader 2018-04-23 14:39:18 -04:00
Koji Kawamura 8e4aa6bf22 NIFI-5075: Do not execute Funnels with no outgoing connections
- Added dedicated conditions for Funnels
- Fixed stale Javadoc
- Stopped caching hasNonLoopConnection variable
- Grouped some conditions to isSourceComponent variable for better
readability

This closes #2634.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-04-23 10:37:22 -04:00
Pierre Villard 2799211946 NIFI-5099 - Update flow differences when updating a connection
This closes #2649

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-04-23 07:08:00 -04:00
Matthew Burgess ba32879ec8 NIFI-5082: Added support for custom Oracle timestamp types to Avro conversion
This closes #2638

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-04-22 19:59:48 -04:00
JohannesDaniel 8b6539e3c7 Added provenance reporting
This closes #2650

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-04-22 15:58:06 -04:00
Mark Payne 54eb6bc232
NIFI-5096: Periodically poll ZooKeeper to determine the leader for each registered role in Leader Election. This avoids a condition whereby a node may occasionally fail to receive notification that it is no longer the elected leader.
NIFI-5096: More proactively setting leadership to false when ZooKeeper/Curator ConnectionState changes

This closes #2646
2018-04-20 16:51:02 -04:00
Pierre Villard 262bf011e4 NIFI-5078 - added source/destination connection info in S2SStatusRT
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2632
2018-04-20 11:50:06 -04:00
Koji Kawamura c575a98936 NIFI-5095: Suppress SET property parse failure at Hive processors
Log debug message when ParseException is thrown.

Log warning message if other unknown Exception is thrown.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2645
2018-04-20 11:24:02 -04:00
zenfenan 1dbfcb9445 NIFI-5062: Removed hbase-client dependecy from hbase bundle
This closes #2636

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-04-20 08:50:25 -04:00
Alex Moundalexis af12923da4 NIFI-5097 This closes #2647. add client-side HTTP redirect to error index.jsp
Existing web error index page suggests user may have meant /nifi/ and crafts a contextPath link, may as well redirect them automatically. Non-human (i.e. curl/wget) can still retrieve the original source without being redirected.

Signed-off-by: joewitt <joewitt@apache.org>
2018-04-19 15:01:33 -04:00
Matthew Burgess 090e748726 NIFI-4647: Fix support for strings in unions for ConvertAvroToORC
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2644.
2018-04-19 09:49:32 +02:00
JohannesDaniel aa196bc01f NIFI-4516 Added QuerySolr after rebase
This closes #2517

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-04-18 17:47:04 -04:00
Joe Percivall a0c9bebe24 NIFI-5085 In InvokeHttp, moving the OkHttp Response object to a try with resources 2018-04-18 13:18:30 -04:00
patricker 82ac815536 NIFI-1706: Extend QueryDatabaseTable to support arbitrary queries
- Only include Maximum Value columns in the type map.
- Squashed commits in the previous PR
- Rebased against the latest master
- Added stop method to GenerateTableFetch so that it refreshes the
column type map when it gets restarted
- Fixed whitespacing around if/for statement
- Updated expressionLanguageSupported value since it is not auto-merged
correctly

This closes #2618.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-04-13 16:30:13 +09:00
Matt Gilman b7272e3f32 NIFI-4997:
- Fixing process group audit advice.
- Setting spring security user in background threads.
- Removing unnecessary overloaded methods.

This closes #2626.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-04-11 16:18:36 -04:00
Pierre Villard 6fbe1515ee NIFI-4809 - Implement a SiteToSiteMetricsReportingTask
Fixed dependency issue by providing a local JSON reader

Rebased + fixed conflict + updated versions in pom + EL scope

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2575
2018-04-11 14:44:30 -04:00
mans2singh 4366c67b27 NIFI-4927 - InfluxDB Query Processor
This closes #2562

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-04-11 06:58:07 -04:00
Matthew Burgess b29304df79 NIFI-4857: Support String<->byte[] conversion for records
This closes #2570.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-04-09 14:53:21 -04:00
Matthew Burgess ae1d3e3945 NIFI-4862: Keep incoming flowfile attributes in outgoing flowfiles from SelectHiveQL
This closes #2605.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-04-09 12:31:05 +09:00
joewitt 59f625d3c2 Merge branch 'NIFI-4995-RC3' as part of NiFi 1.6.0 release process 2018-04-06 16:48:08 -07:00
Vi On b414132324 NIFI-5050 update frontend-maven-plugin
This closes #2613

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2018-04-06 16:07:25 -04:00
Mike Thomsen 63d4bba5a4 NIFI-5047 Fixed a bug in PutMongo that prevented it from working if the mode is insert and the query/query key are not set.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2609.
2018-04-06 21:36:30 +02:00
Andrew Lim 8e10f0bd54 NIFI-5048 Corrected typo in KeytabCredentialsService controller service description
This closes #2612

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2018-04-06 15:17:03 -04:00
Pierre Villard 7cc77937d8 NIFI-4149 - fixed contrib-check errors 2018-04-06 20:38:04 +02:00
Mark Payne 644133dc35 NIFI-4149: Minor tweaks to verbiage
This closes #2205.
2018-04-06 12:42:38 -04:00
Pierre Villard 4c787799ff NIFI-4149 - Indicate if EL is evaluated against FFs or not
- take into account input requirement for documentation rendering
- Renamed variable registry scope and added comments
- Doc + change in mock framework to check scope + update of components + UI
2018-04-06 12:40:54 -04:00
Matthew Burgess 7ff38f690d
NIFI-5045: Fixed error code handling in PutHiveQL. This closes #2608 2018-04-05 16:03:42 -04:00
Marco Gaido c119547222 NIFI-5043: TailFile in Multifile mode should not open new readers in onTrigger
This closes #2606.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-04-05 13:53:01 -04:00
Mike Moser dc9b4cb516
NIFI-3599 Allow back pressure object count and data size to be configurable in nifi.properties. This closes #2497 2018-04-05 13:50:16 -04:00
Koji Kawamura 1913b1e2a8
NIFI-4932: Enable S2S work behind a Reverse Proxy
Adding S2S endpoint Reverse Proxy mapping capability.
Added license header to SVG files.
Incorporated review comments.
Use regex to check property key processing.
Catch AttributeExpressionLanguageParsingException.
This closes #2510
2018-04-03 15:40:28 -04:00
joewitt d511fe3e4b NIFI-4995-RC3 prepare for next development iteration 2018-04-03 08:28:34 -07:00
joewitt f8466cb16d NIFI-4995-RC3 prepare release nifi-1.6.0-RC3 2018-04-03 08:28:15 -07:00
Matt Gilman 364985fb41 NIFI-5034:
- Processing properties and property descriptors in Controller Service referencing components unconditionally.

This closes #2602.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-04-03 10:34:33 -04:00
Matt Gilman 924c935a38 NIFI-5034:
- Ensuring descriptors and properties of referencing components are populated like Processors, Reporting Tasks, etc.
2018-04-03 10:34:33 -04:00
patricker 2279fad673 NIFI-5037 - ConsumeEWS Fails to Read Emtpy Message Body Emails
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2603.
2018-04-03 14:38:38 +02:00
Mark Bean c59b6fdf66 NIFI-4658 set Maximum Number of Entries to required and allow FlowFiles having fragment.count greater than Max Entries property
Signed-off-by: Mike Moser <mosermw@apache.org>

This closes #2559
2018-04-02 20:42:04 -04:00
Bryan Bende b7e1f48133 NIFI-5035 Moving MongoDB services out of standard services
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2599
2018-04-02 12:18:27 -04:00
Mark Payne 94d444abb0 NIFI-4883: Updated ValidateRecord to allow an optional Record Writer to be configured for invalid records that differs from the Record Writer used for writing valid records
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2600.
2018-04-02 18:16:40 +02:00
Matt Gilman bbe79d2260 NIFI-5033:
- Ensuring appropriate response in checkAuthorization when user is null.
- Ensuring the user reference is passed down when applying variable changes.

This closes #2598.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-04-02 10:35:04 -04:00
Bryan Bende 134339c4b7
NIFI-5021 Moving nifi-elasticsearch-client-service-api to nifi-elasticsearch-bundle and creating NAR for the API
This closes #2586.
2018-04-02 09:49:05 -04:00
Arun A K 5d90d14f17 NIFI-5017 - ConvertExcelToCSVProcessor - EL support for rows and columns to skip
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2597.
2018-03-30 11:45:48 +02:00
Matt Gilman c4dcde837e NIFI-5032: This closes #2596.
- Overriding version ranges of transitive dependencies through oauth2-oidc-sdk.

Signed-off-by: joewitt <joewitt@apache.org>
2018-03-29 12:21:40 -04:00
Mark Payne bbbe428e7b NIFI-5030: If ControlRate encounters a FlowFile that cannot be transferred, it should continue processing other FlowFiles that have different group attribute values
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2593.
2018-03-29 10:31:15 +02:00
Scott Aslan c1459825bb
[NIFI-4998] update node and npm version
This closes #2571
2018-03-28 16:49:32 -04:00
Mark Payne 641ba29cc8 NIFI-4993: Accept null user at querying ProvenanceRepository
Pass null user from ReportLineageToAtlas.

Address NullPointer dereferences

This closes #2589.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-28 12:15:05 -04:00
Mark Payne f394c874e1
NIFI-4895: Added backend code to give users the ability to forcibly terminate a processor if the processor does not complete after clicking Stop
NIFI-4895: Addressed review feedback; addressed checkstyle issues
This closes #2555
2018-03-28 12:02:47 -04:00
Ed 9de73fbe1f NIFI-4388: Modules Not Honored
The bug is not reproducible in Jython (it handles modules with every script reload). But Groovy loads JARs and dirs with classes only on setup.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2584
2018-03-27 11:20:40 -04:00
joewitt 7b5bf265a6 NIFI-4995 updating copyright year on all notices 2018-03-26 21:54:10 -04:00