864 Commits

Author SHA1 Message Date
Mark Payne
d1ab17580f NIFI-5465: Set the Idle Timeout on jetty connectors to the same as the Request Timeout. 2018-08-06 10:33:24 -04:00
Matthew Burgess
9742dd2fac NIFI-5471: Push table aliasing to DatabaseAdapter impls
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2926.
2018-07-31 10:14:27 +02:00
Matthew Burgess
692943f016 NIFI-5454: Added EL support and copy.index attribute to DuplicateFlowFile
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2917.
2018-07-26 14:13:14 +02:00
Koji Kawamura
8b9d446118 NIFI-54096: Consolidating new model into existing List processors.
Adding followings:

- Use separate DistributedMapCache for tracking entities to avoid
conflict with existing code
- Added more validation
- Delete listed entities from cache if reset is needed
- Support Local scope
- Added Initial Listing Target

This closes #2876.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-07-17 13:59:57 -04:00
Otto Fowler
b10220439c
NIFI-5337 Syslog 5424 Record Reader and nifi-syslog-utils
- Create nifi-syslog-utils to move syslog parsing functionalty to a central location shared by the processors and serialization/record system.
- Refactor Processors to use these utils
- Update 5424 syslog classes using simple-syslog-5424 to pick up new changes to support this work, as well as keep dependencies/types from bleeding out to the
processors or readers
- Refactor Syslog5424Event and Parser
- Create Syslog5424RecordReader
- per review, handle blank message differently from eof
- name schema per review

This closes #2816.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-07-16 11:50:55 -04:00
Marco Gaido
31bb89514c NIFI-5426: Use NIO.2 API for ListFile 2018-07-13 13:54:27 -04:00
Andy LoPresto
ffbff42421 NIFI-5415 Renamed ListenSyslogGroovyTest to ITListenSyslogGroovy.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2883
2018-07-13 09:57:50 -04:00
patricker
0dd4a91a67 NIFI-5312 QueryDatabaseTable updates state when an SQLException is thrown
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2868
2018-07-10 19:03:14 -04:00
patricker
e13602f752 NIFI-4279 PutDatabaseRecord and ConvertJSONToSQL stream has already been closed error on Oracle
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2874
2018-07-10 15:30:07 -04:00
patricker
624bbab8f0 NIFI-5382 ConvertRecord Add Option To Drop 0 Record Files
NIFI-5382 ConvertRecord Add Option To Drop 0 Record Files

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2853
2018-07-10 12:56:53 -04:00
Andy LoPresto
f60585a9b6
NIFI-5376 Removed deprecation warnings.
Updated Javadoc for SiteToSiteClient#createTransaction() and HttpClient implementation.
Reverted exception listing in method contract for SiteToSiteClient#createTransaction and HttpClient tion of same.
Reverted import ordering in TestSiteToSiteClient.
Reverted exception listing in TestGetHDFSFileInfo, TestListHDFS, and StandardHttpFlowFileServerProtocol.
Restored @SuppressWarnings annotation and removed unnecessary "public static" keywords from inner classes in SiteToSiteClient.

This closes #2841.

Signed-off-by: Joe Witt <joewitt@apache.org>
2018-07-09 20:45:34 -07:00
patricker
3826536546 NIFI-1251 ExecuteSQL Max Rows and Output Batching
NIFI-1251: Fixed doc and checkstyle issues
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2834
2018-07-09 13:36:45 -04:00
Mike Thomsen
790102d8b1 Revert "NIFI-5147 Add CalculateAttributeHash processor"
This reverts commit 7fb931889f37ecec7d14665e58c3332423916b64.
2018-07-02 16:03:28 -04:00
Otto Fowler
7fb931889f NIFI-5147 Add CalculateAttributeHash processor
This closes #2802

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-07-02 06:20:21 -04:00
patricker
a14dbc7dbe NIFI-5339 - Better Time Tracking for ExecuteSQL Durations
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2824.
2018-06-29 16:47:55 +02:00
patricker
c1083dfb62 NIFI-5339 - Better Time Tracking for ExecuteSQL Durations
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2817.
2018-06-29 14:59:27 +02:00
Matthew Burgess
e741ba1ce4 NIFI-5349: Fixed EL handling in Initial Max Value props for DB Fetch processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2822.
2018-06-29 14:51:59 +02:00
Andy LoPresto
ea9b0db2f6
Merge branch 'NIFI-5323-RC1' as part of Apache NiFi 1.7.0 release process. 2018-06-25 12:03:00 -07:00
Otto Fowler
a64680bfa5
NIFI-5325 A Syslog Parser that fully supports RFC 5424 Structured Data
- address review comments
- per review, add property for including body in attributes
- per review, create processor in OnScheduled

This closes #2805.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-06-21 12:38:41 -04:00
Andy LoPresto
d42a1e8bf4
NIFI-5323-RC1 prepare for next development iteration 2018-06-19 20:02:21 -07:00
Andy LoPresto
99bcd1f88d
NIFI-5323-RC1 prepare release nifi-1.7.0-RC1 2018-06-19 20:02:01 -07:00
Bartlomiej Tartanus
235463af7b allow expression language in Schema File property
Update ValidateXml.java

Update ValidateXml.java

NIFI-5192: Added evaluateAttributeExpressions call, error handling, and unit tests
2018-06-19 16:12:52 -04:00
Mark Payne
2760b07770 NIFI-5322: Addressed bug that caused QueryRecord to fail (rollback session instead of routing to 'failure') whenever a record in the incoming FlowFile did not adhere to its schema. This happened because the InputStream for the FlowFile was not properly closed. Also updated the text of the Exception to include information from its 'cause' so user is better able to understand the underlying issue.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2803
2018-06-19 16:01:33 -04:00
Mike Moser
d4d9259664 NIFI-5274 avoid rollback on uncaught errors in ReplaceText
forgot to penalize flowfiles sent to failure

NIFI-5274 avoid rollback on StackOverflowError in ReplaceText

add comment

removed penalize on failure, doesn't really apply here

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2767
2018-06-18 12:00:30 -04:00
Dustin Rodrigues
021ee6faf2 NIFI-5317 - support non-ASCII X-Mailer PutEmail header
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2800.
2018-06-18 10:45:56 +02:00
Dustin Rodrigues
2277b9411e NIFI-5252 - support arbitrary headers in PutEmail processor
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2787.
2018-06-15 14:29:56 +02:00
Matthew Burgess
0886dcb0b4 NIFI-5303: Fixed QueryDatabaseTable to work with DB2
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2788.
2018-06-15 13:56:52 +02:00
Mike Thomsen
be63378a1e NIFI-5214 Added REST LookupService
NIFI-5214 Added support for the new ProxyConfigurationService

NIFI-5214 Integration tests added.

NIFI-5214 Added missing pom.xml and a change from a code review.

NIFI-5214 Added another tag based on code review.

NIFI-5214 Added user-defined header support.

NIFI-5214 Added Basic Auth support.

NIFI-5214 Moved documentation.

NIFI-5214 Fixed checkstyle issues; added changes requested in a review.

NIFI-5214 Added changes requested in a code review.

NIFI-5214 Added verb test and @DynamicProperties

NIFI-5214 Added templated URL support to RestLookupService.

NIFI-5214 Fixed documentation based on code review changes.

NIFI-5214 Changed RestLookupService to use a property descriptor.

NIFI-5214 Updated documentation.

NIFI-5214 Made changes requested in code review.

NIFI-5214 Renamed nifi-standard-web-utils to reflect that it is for tests.

NIFI-5214: Refactor RestLookupService.

1. Added 'Base URL' property to address environment specific part of URL.
2. Removed 'Record Path Property Name' property, because the name of
a resulted record field of a record path can be obtained by field name.
3. Lower cased HTTP method name should be used throughout.
4. Added mimeType require check when body is specified.
5. Added debug log to print HTTP response code.
6. Prepare for NIFI-5287.
7. Fixed that mime.type being used regardless of whether body is
specified or not, caused NullPointerException when 'mime.type' is not
specified when it is not required.
8. Updated documentation.

NIFI-5214 Fixed AWS processors that broke w/ change to TestServer location.

NIFI-5214 Added changes requested in a code review.

Refactored the way to evaluate EL for URL property

- Use PropertyValue instead of PreparedQuery to utilize Variable
Registry.
- Removed BASE_URL because Variable Registry can be used at URL

NIFI-5214 Rebased to use new LookupService method.

This closes #2723.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-06-15 15:18:03 +09:00
Koji Kawamura
4349a8e078 NIFI-5231: CalculateRecordCount should use 'record.count'
This closes #2792

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-06-14 07:02:47 -04:00
Mike Thomsen
0831059d2c NIFI-5287 Made LookupRecord able to take in flowfile attributes and combine them with lookup keys.
NIFI-5287 Removed unneeded property descriptor.

NIFI-5287 Added additional changes from a code review.

This closes #2777.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-06-14 10:54:44 +09:00
Pierre Villard
09f9260035 NIFI-5293 - add merge.uuid to MergeRecord processor
This closes #2783

Signed-off-by: zenfenan <zenfenan@apache.org>
2018-06-12 19:16:17 +05:30
Pierre Villard
ee18ead16c NIFI-5297 - EL support in ScanAttribute
This closes #2786

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-06-11 18:48:38 -04:00
Mike Thomsen
1803c15bcd NIFI-5231 Added RecordStats processor.
NIFI-5231 Minor fixes from code review.

NIFI-5231 Added documentation and made record_count a constant.

NIFI-5231 Added changes requested in a code review.

NIFI-5231 Added changes based on code review.

This closes #2737.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-06-11 13:35:01 +09:00
zenfenan
cf3c666683 NIFI-5286: Updated FasterXML Jackson libraries to 2.9.5
This closes #2775

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-06-09 14:19:45 -04:00
Pierre Villard
05d7b6c6e7 NIFI-4262 - MergeContent - option to add merged uuid in original flow files
This closes #2056

Signed-off-by: zenfenan <zenfenan@apache.org>
2018-06-08 20:54:47 +05:30
Pierre Villard
6e067734d5 NIFI-5264 - Added attribute for validation error message in ValidateCSV
This closes #2769

Signed-off-by: zenfenan <zenfenan@apache.org>
2018-06-08 19:16:20 +05:30
Matthew Burgess
0e09b98b02 NIFI-5143: Initial work to support column values for paging results
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2728.
2018-06-08 09:36:27 +02:00
Joe Trite
f5364875e8 Squashed commit of the following:
commit 9c31e45d3f94bf1e73c87379f315f6559e6a23f4
Author: Joe Trite <joetrite@gmail.com>
Date:   Mon Mar 13 07:55:19 2017 -0400

    NIFI-1705 Adding AttributesToCSV processor

commit 5e9afa9ccf4276c7a2318b761ded77fd514a60d9
Merge: 3177eb185 74cbfc4b6
Author: Joe Trite <joetrite@gmail.com>
Date:   Sat Mar 4 08:12:39 2017 -0500

    Merge remote-tracking branch 'origin/master'

    # Conflicts:
    #	nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ScanAttribute.java

commit 3177eb1852dbc7e70c250c53d038a5bcc5cfa3c5
Author: Joe Trite <joetrite@gmail.com>
Date:   Sat Mar 4 08:09:06 2017 -0500

    NIFI-3497 Converted properties to use displayName, changed validator on demarcator property and created a method for duplicate code.

commit 74cbfc4b69ffde1ddb4442b91e160b66ea3c8b6b
Merge: a974f78e0 f8cad0f8c
Author: Joe Trite <joetrite@gmail.com>
Date:   Sat Mar 4 07:47:46 2017 -0500

    Merge branch 'master' into master

commit a974f78e033885455cadd2cbffd7e387cbabc4d7
Author: Joe Trite <joetrite@gmail.com>
Date:   Sat Mar 4 07:43:02 2017 -0500

    NIFI-3497 Converted properties to use displayName, changed validator on demarcator property and created a method for duplicate code.

commit 1bfaef8e87eaa6af84b1a80cb680e0a15eef2f8d
Merge: 65ed46de9 89ec68d14
Author: Joe Trite <joetrite@gmail.com>
Date:   Fri Mar 3 08:01:59 2017 -0500

    Merge branch 'master' of https://github.com/joetrite/nifi

commit 65ed46de9a00518cfe06ecd69bd4f3bbd8d3e662
Author: Joe Trite <joetrite@gmail.com>
Date:   Fri Feb 24 18:09:36 2017 -0500

    NIFI-3497 - fixing Pcontrib issues

commit c5d52cf6f0f16496d9ef83411770409c0b2f88d4
Author: Joe Trite <joetrite@gmail.com>
Date:   Thu Feb 23 10:19:01 2017 -0500

    NIFI-3497 - excluding test files

    Adding new test data files to exclude list.

commit b1959341138d3b5004979544fcea86ba36fe6ebb
Author: Joe Trite <joetrite@gmail.com>
Date:   Wed Feb 22 16:48:10 2017 -0500

    NIFI-3497 - New dictionary files for test

    Adding new dictionary files to support metadata dictionary option.

commit e296268f39bf213a9e8eaa8298b26556c6efe278
Author: Joe Trite <joetrite@gmail.com>
Date:   Wed Feb 22 16:46:13 2017 -0500

    NIFI-3497 test cases for metadata updates

    Adding test cases to support metadata option update.

commit de7e348e62c0f7c5fbd334106878ca6ac46935af
Author: Joe Trite <joetrite@gmail.com>
Date:   Wed Feb 22 16:36:08 2017 -0500

    NIFI-3497 - Added metadata option

    Added optional to post additional metadata as new attributed if a match is found in the dictionary.

commit 89ec68d14bb34cbe65ff9a4d50ff5321fd4ec0ef
Author: Joe Trite <joetrite@gmail.com>
Date:   Fri Feb 24 18:09:36 2017 -0500

    NIFI-3497 - fixing Pcontrib issues

commit d71426037b142da8ca04dae38952c164d1614806
Author: Joe Trite <joetrite@gmail.com>
Date:   Thu Feb 23 10:19:01 2017 -0500

    NIFI-3497 - excluding test files

    Adding new test data files to exclude list.

commit a7a7b6ace80380416c342809ce95a4f4087bb066
Author: Joe Trite <joetrite@gmail.com>
Date:   Wed Feb 22 16:48:10 2017 -0500

    NIFI-3497 - New dictionary files for test

    Adding new dictionary files to support metadata dictionary option.

commit 8eb54a50193897cf564eb7d222aae35481168af4
Author: Joe Trite <joetrite@gmail.com>
Date:   Wed Feb 22 16:46:13 2017 -0500

    NIFI-3497 test cases for metadata updates

    Adding test cases to support metadata option update.

commit f52e1f2a064b31f87d4165af6075716fa7d55046
Author: Joe Trite <joetrite@gmail.com>
Date:   Wed Feb 22 16:36:08 2017 -0500

    NIFI-3497 - Added metadata option

    Added optional to post additional metadata as new attributed if a match is found in the dictionary.

NIFI-1705 Adding AttributesToCSV processor.

NIFI-1705 Adding AttributesToCSV processor.

NIFI-1705 Adding AttributesToCSV processor.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2711
2018-06-07 14:48:53 -04:00
Otto Fowler
f7f809c3d3 NIFI-4272 support multiple captures when EL is present in replacement value
This closes #2748

Signed-off-by: Mike Moser <mosermw@apache.org>
2018-06-06 17:23:20 +00:00
Marco Gaido
eedf1237a1 NIFI-5262 - Retrieve file attributes only once in ListFile
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2754.
2018-06-06 10:10:29 +02:00
Koji Kawamura
71cf3fd46d NIFI-5268: Fix JostTransformJSON spec validation with EL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2762.
2018-06-06 09:37:02 +02:00
Mike Thomsen
83fccc4beb NIFI-5024 Fixed a checkstyle issue that was missed in the merge of 5024. 2018-05-31 08:35:25 -04:00
Nicolas Sanglard
498fd8f22d NIFI-5024 Resolves deadlock in ExecuteStreamCommand processor
This closes #2594

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-31 06:35:39 -04:00
Matt Gilman
9093d280f2 NIFI-5208:
- Removing test for unsupported functionality.

This closes #2740

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2018-05-30 13:41:20 -04:00
Matt Gilman
05a1d63090 NIFI-5208:
- Fixing missing message when a node is disconnected from a cluster.
- Updating endpoints to accept a flag to allow for changes to be made to a disconnected node.
- Updating custom UIs to acknowledge disconnected nodes prior to performing modifications.
2018-05-30 13:41:03 -04:00
Scott Aslan
9aa7e65f70
[NIFI-5215] Upgrade AngularJS to v1.7.0. Also, adding package-lock.json to source control
[NIFI-5215] add package-lock.json to rat excludes
This closes #2721
2018-05-29 13:20:49 -04:00
zenfenan
0973c2d8d1 NIFI-543 Added annotation to restrict processor to run only on the primary node
- PR Fix - 'Execution' dropdown will now be shown in all cases
- Annotated ListGCSBucket with PrimaryNodeOnly

This closes #2509.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-05-25 11:50:29 -04:00
Mark Payne
397e88c858 NIFI-4227: Fixed typo 2018-05-24 15:57:28 -04:00
Pierre Villard
be0ed70423 NIFI-4227 - add a ForkRecord processor
Added split/extract modes, unit tests, and additional details

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-05-24 15:55:17 -04:00
Mike Thomsen
716587d09f NIFI-5169 This closes #2705. Upgrade to JSONPath 2.4
NIFI-5169 Added explicit checking for empty strings in JsonPath expression property.

Signed-off-by: joewitt <joewitt@apache.org>
2018-05-23 16:16:25 -05:00