Commit Graph

4624 Commits

Author SHA1 Message Date
Andy LoPresto 8996b7f6d6
NIFI-5193 Added logic to handle complex user filter expressions.
Added unit tests.
Added unit test resources.
Fixed comments.
Refactored XmlSlurper instantiation to keep ignorable whitespace.
Added logic to handle LIP complex user search filter.
Added unit tests.
Added unit test resources.
Removed unnecessary substitution/repopulation logic from encrypt|decryptAuthorizers.
All unit tests pass.
Removed unnecessary substitution/repopulation logic from CET.
Removed unnecessary unit tests.
Removed unnecessary commons-text dependency from pom.xml.

This closes #2797.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-06-19 13:27:47 -04:00
Matthew Burgess aa33989192
NIFI-5266: Sanitize ES parameters in PutElasticsearchHttp processors
Incorporated review comments
Added integration test with JSON character

This closes #2760.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-06-18 17:59:28 -07:00
Bryan Bende 0b0ba1eae3
NIFI-5319 Utilize NiFi Registry 0.2.0 client
This closes #2801.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-06-18 15:37:05 -07:00
Mike Moser d4d9259664 NIFI-5274 avoid rollback on uncaught errors in ReplaceText
forgot to penalize flowfiles sent to failure

NIFI-5274 avoid rollback on StackOverflowError in ReplaceText

add comment

removed penalize on failure, doesn't really apply here

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2767
2018-06-18 12:00:30 -04:00
Ed 568959baf5
NIFI-4906 Add GetHDFSFileInfo
- Updated doc for write attributes
- Removing redundant files
- Updates based on review

Changes:
- changed validators for regex params (3)
- removed TriggerWhenEmpty and TriggerSerially.
- removed unused LISTING_LAG_NANOS
- removed unimplemented customValidate
- removed unsused error handling for building/updating request params
- added MIME_TYPE for content
- suppressed output to original relationship when scheduled
- added full path to original/new FF when not found
- added warning messages for huge attribute values.

This closes #2639.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-06-18 09:33:42 -04:00
Mike Thomsen 2fb213d26c NIFI-5145 Fixed evaluateAttributeExpressions in mockpropertyvalue to make it handle null flowfiles.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2749
2018-06-18 09:14:25 -04:00
Pierre Villard e5322b5d33 NIFI-4982 - Add a DistributedMapCacheLookupService
This closes #2558

Signed-off-by: zenfenan <zenfenan@apache.org>
2018-06-18 18:35:12 +05:30
Dustin Rodrigues 021ee6faf2 NIFI-5317 - support non-ASCII X-Mailer PutEmail header
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2800.
2018-06-18 10:45:56 +02:00
Matthew Burgess 97f71fd6c7 NIFI-5044: Applied changes in SelectHiveQL to SelectHive3QL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2799.
2018-06-15 22:42:59 +02:00
Ed B 187417d077 NIFI-5044 SelectHiveQL accept only one statement
SelectHiveQL support only single SELECT statement.
This change adds support for pre- and post- select statements.
It will be useful for configuration queries, i.e. "set tez.queue.name=default", and others.

renamed selectQuery to hqlStatement for better readability

style check correction in comment

removed meaningless examples for post-queries

execute query without result set

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2695
2018-06-15 15:48:16 -04:00
Dongkyu Hwangbo 9b461027a4 NIFI-5188: DruidTranquilityController does not fully support Druid aggregator
Rollback Druid 0.9.2 to 0.9.1

Fixed checkstyle error

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2696
2018-06-15 09:52:14 -04:00
Koji Kawamura 52d6b9cfa2
NIFI-5257: Expand Couchbase Server integration
- Added CouchbaseMapCacheClient.
- Added CouchbaseKeyValueLookupService.
- Added CouchbaseRecordLookupService.
- Added 'Put Value to Attribute' to GetCouchbaseKey.
- Fixed Get/PutCouchbaseKey relationship descriptions.
- Stop sharing Relationship.Builders.

This closes #2750
2018-06-15 09:14:30 -04:00
Koji Kawamura f05c5e6ea0
NIFI-5054: Add Couchbase user authentication 2018-06-15 08:48:46 -04:00
Dustin Rodrigues 2277b9411e NIFI-5252 - support arbitrary headers in PutEmail processor
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2787.
2018-06-15 14:29:56 +02:00
Matthew Burgess 0886dcb0b4 NIFI-5303: Fixed QueryDatabaseTable to work with DB2
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2788.
2018-06-15 13:56:52 +02:00
Andy LoPresto 90b8e7f9ff NIFI-5209 Removed unused test resources.
Removed RAT exclusion from pom.xml.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2798.
2018-06-15 13:52:33 +02:00
zenfenan 3752398943 NIFI-5221 - Added 'Object Tagging' functionalities to S3 Processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2751.
2018-06-15 13:49:13 +02:00
Mike Thomsen be63378a1e NIFI-5214 Added REST LookupService
NIFI-5214 Added support for the new ProxyConfigurationService

NIFI-5214 Integration tests added.

NIFI-5214 Added missing pom.xml and a change from a code review.

NIFI-5214 Added another tag based on code review.

NIFI-5214 Added user-defined header support.

NIFI-5214 Added Basic Auth support.

NIFI-5214 Moved documentation.

NIFI-5214 Fixed checkstyle issues; added changes requested in a review.

NIFI-5214 Added changes requested in a code review.

NIFI-5214 Added verb test and @DynamicProperties

NIFI-5214 Added templated URL support to RestLookupService.

NIFI-5214 Fixed documentation based on code review changes.

NIFI-5214 Changed RestLookupService to use a property descriptor.

NIFI-5214 Updated documentation.

NIFI-5214 Made changes requested in code review.

NIFI-5214 Renamed nifi-standard-web-utils to reflect that it is for tests.

NIFI-5214: Refactor RestLookupService.

1. Added 'Base URL' property to address environment specific part of URL.
2. Removed 'Record Path Property Name' property, because the name of
a resulted record field of a record path can be obtained by field name.
3. Lower cased HTTP method name should be used throughout.
4. Added mimeType require check when body is specified.
5. Added debug log to print HTTP response code.
6. Prepare for NIFI-5287.
7. Fixed that mime.type being used regardless of whether body is
specified or not, caused NullPointerException when 'mime.type' is not
specified when it is not required.
8. Updated documentation.

NIFI-5214 Fixed AWS processors that broke w/ change to TestServer location.

NIFI-5214 Added changes requested in a code review.

Refactored the way to evaluate EL for URL property

- Use PropertyValue instead of PreparedQuery to utilize Variable
Registry.
- Removed BASE_URL because Variable Registry can be used at URL

NIFI-5214 Rebased to use new LookupService method.

This closes #2723.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-06-15 15:18:03 +09:00
pepov be31c1efd3 NIFI-5249 Dockerfile enhancements: create data repository and log directories, explicit environment variable declarations to replace nifi-env.sh. Add integration tests, remove latest tag, since only one tag can be specified that will be used to build the image.
Cover dockerhub image with the same tests as dockermaven
Fix entrypoint to handle signals properly, also get rid of hardcoded paths

This closes #2747
2018-06-14 15:29:39 -04:00
Mark Payne d0499eb867
NIFI-5311: When creating a processor, controller service, or reporting task, give the component up to 50 ms to complete validation before returning the DTO. This closes #2795 2018-06-14 13:24:27 -04:00
Matt Gilman fe31a06fdc
NIFI-4907:
- Minor adjustments following PR.
- Avoiding additional find operation when authorizing components when populating component details.
- Requiring access to provenance events when downloading content or submitting a replay as they may provide events details.
- Updating the REST API docs detailing the required permissions.
- Updating the wording in the documentation regarding the provenance and data policies.
- Removed the event attributes from the authorization calls that were verifying access to provenance events.
- Only checking content availability when the user is authorized for the components data.
- Addressing typo in JavaDoc.

This closes #2703
2018-06-14 12:21:47 -04:00
Mark Bean e27798797a
NIFI-4907: add 'view provenance' component policy
whitespace removed for checkstyle
2018-06-14 10:59:17 -04:00
Marco Gaido d98d335497 [NIFI-3242] Avoid double scheduling of a task due to quartz imprecision
This closes #2789.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-06-14 10:10:45 -04:00
amitt90 8370a56dce NIFI-5309 update the logger message to get the output format from the outputFormat variable
This closes #2793

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-06-14 07:09:23 -04:00
Koji Kawamura 4349a8e078 NIFI-5231: CalculateRecordCount should use 'record.count'
This closes #2792

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-06-14 07:02:47 -04:00
Mike Thomsen 0831059d2c NIFI-5287 Made LookupRecord able to take in flowfile attributes and combine them with lookup keys.
NIFI-5287 Removed unneeded property descriptor.

NIFI-5287 Added additional changes from a code review.

This closes #2777.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-06-14 10:54:44 +09:00
thenatog dbf259508c
NIFI-5258 - Changed addHeader to setHeader which stops X-Frame-Options being added twice to responses. Added unit test.
This closes #2791.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-06-13 16:13:53 -07:00
Mark Payne 275b8cbf23
NIFI-5308: Avoid holding Template DOM Nodes in heap. This closes #2790 2018-06-13 16:00:01 -04:00
Matthew Burgess da99f873a7
NIFI-4963: Added Hive3 bundle
- Incorporated review comments
- Added more defensive code for PutHive3Streaming error handling

This closes #2755.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-06-13 14:32:58 -04:00
Matt Gilman 8feac9ae54
NIFI-5237:
- Ensuring the proxy headers are considered when redirecting the user following a OIDC or Knox login exchange.

This closes #2763.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-06-12 12:50:37 -04:00
Pierre Villard 09f9260035 NIFI-5293 - add merge.uuid to MergeRecord processor
This closes #2783

Signed-off-by: zenfenan <zenfenan@apache.org>
2018-06-12 19:16:17 +05:30
Pierre Villard ee18ead16c NIFI-5297 - EL support in ScanAttribute
This closes #2786

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-06-11 18:48:38 -04:00
Otto Fowler 946c2fe6b3 NIFI-5022 This closes #2588. InvokeAWSGatewayApi processor
per review, simplify header building and remove new map
Per review:
- removed apache headers from 3rd party files per https://www.apache.org/legal/src-headers.html#3party
- referenced commit/repo in notice file
- created rat configuration to account for files without headers

Signed-off-by: joewitt <joewitt@apache.org>
2018-06-11 10:49:41 -04:00
Mark Payne e3b0949b6b NIFI-4508: This closes #2784. Update ConsumeAMQP to use basicConsume API instead of basicGet in order to provide better performance
NIFI-4508: Added some additional unit tests to verify behavior

Signed-off-by: joewitt <joewitt@apache.org>
2018-06-11 10:27:28 -04:00
Junegunn Choi 18a4819d51 NIFI-5290 - PutKudu should support unixtime_micros columns
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2781.
2018-06-11 09:40:38 +02:00
Mike Thomsen 1803c15bcd NIFI-5231 Added RecordStats processor.
NIFI-5231 Minor fixes from code review.

NIFI-5231 Added documentation and made record_count a constant.

NIFI-5231 Added changes requested in a code review.

NIFI-5231 Added changes based on code review.

This closes #2737.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-06-11 13:35:01 +09:00
Mike Thomsen 5a39d2a819 NIFI-5288 Quietly convert Java arrays to Lists so the MongoDB API can handle them.
NIFI-5288 Made changes based on code review.

NIFI-5288 Theoretically supports nested arrays.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2778
2018-06-10 22:10:12 -04:00
Andy LoPresto d02cd4f909
NIFI-5209 Removed secure hash functionality from ConfigEncryptionTool.
Removed relevant unit tests.

This closes #2761.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2018-06-10 21:54:25 -04:00
Aldrin Piri ead3969ab7
NIFI-5006 Updating NOTICEs to 2018
This closes #2779.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-06-10 13:09:13 -07:00
zenfenan 794a642310 NIFI-5284: Added JSON_TYPE support to RunMongoAggregation
This closes #2776

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-06-10 06:45:21 -04:00
zenfenan cf3c666683 NIFI-5286: Updated FasterXML Jackson libraries to 2.9.5
This closes #2775

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-06-09 14:19:45 -04:00
Mark Payne 00a63d17af NIFI-5200: Fixed issue with InputStream being closed when calling ProcessSession.read() twice against sequential Content Claims
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2753
2018-06-08 16:47:28 -04:00
Scott Aslan 4bccab7e05 [NIFI-5240] adding GPG key for Scott Aslan 2018-06-08 13:50:25 -04:00
Pierre Villard 05d7b6c6e7 NIFI-4262 - MergeContent - option to add merged uuid in original flow files
This closes #2056

Signed-off-by: zenfenan <zenfenan@apache.org>
2018-06-08 20:54:47 +05:30
Pierre Villard 6e067734d5 NIFI-5264 - Added attribute for validation error message in ValidateCSV
This closes #2769

Signed-off-by: zenfenan <zenfenan@apache.org>
2018-06-08 19:16:20 +05:30
Mark Payne 49228aa5dc NIFI-5281: If value is not valid according to the schema's CHOICE field, JSON Writer should write null value instead of throwing NullPointerException
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2772.
2018-06-08 10:07:43 +02:00
Matthew Burgess 0e09b98b02 NIFI-5143: Initial work to support column values for paging results
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2728.
2018-06-08 09:36:27 +02:00
Mark Payne 729f8aa246
NIFI-5279: Allow components up to 50 milliseconds to complete validation before returning from update request
This closes #2770.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-06-07 16:32:19 -07:00
Matt Gilman 7b9d779a4b NIFI-3217:
- Disabling the resize of SlickGrid when there is an active edit occuring. This will prevent the accidental closure of the current edit.
- Triggering a resize when the active editor closes.

This cloese #2766

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2018-06-07 18:26:13 -04:00
Joe Trite f5364875e8 Squashed commit of the following:
commit 9c31e45d3f
Author: Joe Trite <joetrite@gmail.com>
Date:   Mon Mar 13 07:55:19 2017 -0400

    NIFI-1705 Adding AttributesToCSV processor

commit 5e9afa9ccf
Merge: 3177eb185 74cbfc4b6
Author: Joe Trite <joetrite@gmail.com>
Date:   Sat Mar 4 08:12:39 2017 -0500

    Merge remote-tracking branch 'origin/master'

    # Conflicts:
    #	nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ScanAttribute.java

commit 3177eb1852
Author: Joe Trite <joetrite@gmail.com>
Date:   Sat Mar 4 08:09:06 2017 -0500

    NIFI-3497 Converted properties to use displayName, changed validator on demarcator property and created a method for duplicate code.

commit 74cbfc4b69
Merge: a974f78e0 f8cad0f8c
Author: Joe Trite <joetrite@gmail.com>
Date:   Sat Mar 4 07:47:46 2017 -0500

    Merge branch 'master' into master

commit a974f78e03
Author: Joe Trite <joetrite@gmail.com>
Date:   Sat Mar 4 07:43:02 2017 -0500

    NIFI-3497 Converted properties to use displayName, changed validator on demarcator property and created a method for duplicate code.

commit 1bfaef8e87
Merge: 65ed46de9 89ec68d14
Author: Joe Trite <joetrite@gmail.com>
Date:   Fri Mar 3 08:01:59 2017 -0500

    Merge branch 'master' of https://github.com/joetrite/nifi

commit 65ed46de9a
Author: Joe Trite <joetrite@gmail.com>
Date:   Fri Feb 24 18:09:36 2017 -0500

    NIFI-3497 - fixing Pcontrib issues

commit c5d52cf6f0
Author: Joe Trite <joetrite@gmail.com>
Date:   Thu Feb 23 10:19:01 2017 -0500

    NIFI-3497 - excluding test files

    Adding new test data files to exclude list.

commit b195934113
Author: Joe Trite <joetrite@gmail.com>
Date:   Wed Feb 22 16:48:10 2017 -0500

    NIFI-3497 - New dictionary files for test

    Adding new dictionary files to support metadata dictionary option.

commit e296268f39
Author: Joe Trite <joetrite@gmail.com>
Date:   Wed Feb 22 16:46:13 2017 -0500

    NIFI-3497 test cases for metadata updates

    Adding test cases to support metadata option update.

commit de7e348e62
Author: Joe Trite <joetrite@gmail.com>
Date:   Wed Feb 22 16:36:08 2017 -0500

    NIFI-3497 - Added metadata option

    Added optional to post additional metadata as new attributed if a match is found in the dictionary.

commit 89ec68d14b
Author: Joe Trite <joetrite@gmail.com>
Date:   Fri Feb 24 18:09:36 2017 -0500

    NIFI-3497 - fixing Pcontrib issues

commit d71426037b
Author: Joe Trite <joetrite@gmail.com>
Date:   Thu Feb 23 10:19:01 2017 -0500

    NIFI-3497 - excluding test files

    Adding new test data files to exclude list.

commit a7a7b6ace8
Author: Joe Trite <joetrite@gmail.com>
Date:   Wed Feb 22 16:48:10 2017 -0500

    NIFI-3497 - New dictionary files for test

    Adding new dictionary files to support metadata dictionary option.

commit 8eb54a5019
Author: Joe Trite <joetrite@gmail.com>
Date:   Wed Feb 22 16:46:13 2017 -0500

    NIFI-3497 test cases for metadata updates

    Adding test cases to support metadata option update.

commit f52e1f2a06
Author: Joe Trite <joetrite@gmail.com>
Date:   Wed Feb 22 16:36:08 2017 -0500

    NIFI-3497 - Added metadata option

    Added optional to post additional metadata as new attributed if a match is found in the dictionary.

NIFI-1705 Adding AttributesToCSV processor.

NIFI-1705 Adding AttributesToCSV processor.

NIFI-1705 Adding AttributesToCSV processor.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2711
2018-06-07 14:48:53 -04:00