4492 Commits

Author SHA1 Message Date
joewitt
7b5bf265a6 NIFI-4995 updating copyright year on all notices 2018-03-26 21:54:10 -04:00
Joe Percivall
56ad14ad1e
NIFI-4977 Adding expression language support to the Sender properties of PutSyslog
This closes #2547.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-03-26 15:49:52 -04:00
Mike Thomsen
aa947e4d3e NIFI-4325 Added new processor that uses the JSON DSL.
NIFI-4325 Cleaned up how ElasticSearch client service builds SSLContext, added query attribute to flowfiles and other changes requested in a code review.

This closes #2113.

Signed-off-by: Joe Percivall <JPercivall@apache.org>
2018-03-26 15:01:37 -04:00
Mike Thomsen
6b34d3bea9 NIFI-4743 - Added configurable null suppression to PutElasticsearchHttpRecord
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2501.
2018-03-26 20:41:21 +02:00
Mark Payne
456c9c8fc0
NIFI-5012: When connecting to cluster, esure that controller services appropriately enabled/disabled
This closes #2579.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-03-26 13:07:58 -04:00
zenfenan
3612fbe522
NIFI-4864: Improvements to PR #2470
This closes #2581.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-03-26 10:49:20 -04:00
Matt Gilman
69a564e4c8
NIFI-5009:
- Fixing required permission for PutParquet.

NIFI-5008:
- Ensuring all restricted components are tagged as such.

This closes #2583.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-03-26 09:47:54 -04:00
Otto Fowler
98c295dfc7
NIFI-5013 create test jar for standard processors
remove unnec TestServer instances

This closes #2580.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2018-03-24 10:18:46 -04:00
Mark Payne
03f4c0fc47 NIFI-5011: Fixed threading bug in TestFileSystemRepository
This closes #2578.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-23 15:46:17 -04:00
Marco Gaido
758e44682f NIFI-4631: Use java.nio.file.Files in ListFile to improve performance
This closes #2565.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-23 11:00:21 -04:00
Mike Thomsen
66590b78f5 NIFI-4989 Made PutMongo able to use nested lookup keys, a query param and multiple lookup keys.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2560.
2018-03-23 14:46:50 +01:00
Derek Straka
95dd1ebffc NIFI-4882: Resolve issue with parsing custom date, time, and timestamp formats in CSV files
Refactor the two existing CSV classes to inherit from an abstract base class that parses the data
Add unit tests for calls to nextRecord that do not coerce types

This closes #2473.

Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-03-22 09:18:00 +09:00
Mark Payne
0b0aebe148
NIFI-4917: Externalize Keytab and Principal configuration from Processors to a Controller Service. This gives us the ability to allow users to interact with those Keytabs/Principals to which they've been given access without allowing them access to all Keytabs and Principals
- Addressed review feedback; rebased against master

This closes #2552.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-03-21 14:23:41 -04:00
sbouchex
9c9215982f NIFI-4999: Added request URI into the "restlistener.request.uri" attribute
This closes #2574.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-21 13:17:17 -04:00
Matt Gilman
98cd9ad531
NIFI-4885:
- Updating the versioning endpoints to account for the granular access restrictions.

This closes #2573.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-03-21 11:45:13 -04:00
joewitt
478e34082d NIFI-4995 release process exposed this item doesn't get updated by versions mechanism so using project version instead 2018-03-20 14:32:00 -04:00
joewitt
e8a8c19db2 Revert "NIFI-4809 - Implement a SiteToSiteMetricsReportingTask"
This reverts commit 930417b9dcabb430df79c683acb9c927aa4f5b8b due to NIFI-4996
2018-03-20 12:58:50 -04:00
Andy LoPresto
d09145e3e3 NIFI-4557 Added Expression Language support for PGP private keyring passphrase in EncryptContent.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2566.
2018-03-20 11:23:45 +01:00
Koji Kawamura
2590c88318 NIFI-4979 - Fix ReportLineageToAtlas documentation errors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2549.
2018-03-19 17:56:13 +01:00
jugi92
78daaf40f8 NIFI-4991 This closes #2564.
nifi.database.directory should be moved out of default location <root-level-nifi-dir>/database_repository to ease the update process.

Signed-off-by: joewitt <joewitt@apache.org>
2018-03-19 11:41:56 -04:00
Mark Payne
39556e3513 NIFI-4902: This closes #2485. Updated ConsumeAMQP, PublishAMQP to use one connection per concurrent task instead of a single connection shared by all concurrent tasks. This offers far better throughput when the network latency is non-trivial. Also refactored to simplify code
Signed-off-by: joewitt <joewitt@apache.org>
2018-03-19 11:40:49 -04:00
Derek Straka
5bdb7cf6e7 NIFI-4912: This closes #2494. Update jackson version to latest stable version (2.9.4)
Signed-off-by: joewitt <joewitt@apache.org>
2018-03-19 10:22:50 -04:00
Internet
1091093133 NIFI-4777: get schema by id even if not latest
This closes #2405.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-19 09:57:38 -04:00
Pierre Villard
4687e0fb35 NIFI-4981 Allow using FF's content as message in PutEmail
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2556.
2018-03-18 11:35:43 -07:00
Mike Moser
c9f1c9789d NIFI-4973 Added license comment to new file RemoteProcessGroupEntityMergerTest.java 2018-03-16 19:15:52 +00:00
Julian Gimbel
9cd0aab696 NIFI-4918: looping over several methods to try and fit the dynamic attribute.
If failed, use first method and throw error if not working.

Array.length can never be less then 0

This closes #2499.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-16 14:40:59 -04:00
Mark Bean
8b2c5b7246 NIFI-4950 Defining behavior for MergeContent when more than 1 FlowFile has the same fragment.index value
Signed-off-by: Mike Moser <mosermw@apache.org>

This closes #2557.
2018-03-16 18:31:43 +00:00
Derek Straka
c056ede6cc NIFI-4951: Update convertToAvroObject to use the DataTypeUtils conversion function
The feature allows users to convert from non-integral types to the correct underlying type.  The
original behavior is maintained; however, now simple conversions take place automatically for some
logical types (date, time, and timestamp).

This closes #2526.

Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-16 14:08:14 -04:00
Matt Gilman
a1c917656e NIFI-4973:
- Fixing RPG port merging.
- Adding unit tests.
- Removing unecessary sorting that wasn't maintained while clustered.

This closes #2551.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-16 13:03:09 -04:00
Marco Gaido
86f162b611 NIFI-4959: Remove flowfiles and close connection for Bad Requests causing IOException
This closes #2544.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-16 12:24:00 -04:00
Mark Payne
3f0aa6e13c Changed some unit tests to Integration Tests because they are creating embedded JMS Brokers (implicitly) and commnicating with it. This becomes brittle in a full build, especially if any other unit test attempts to create a broker implicitly due to the way that the activemq broker implicit creation works. 2018-03-16 12:07:33 -04:00
Koji Kawamura
ada069ce42 NIFI-4978: Fixed ReportLineageToAtlas NPE when unscheduled
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2548.
2018-03-16 10:19:59 +01:00
Mark Payne
7d5f803ce6 Fixed checkstyle violation 2018-03-15 16:16:12 -04:00
Bryan Bende
de71a41bd0 NIFI-4935 Refactoring to support specifying schema branch or schema version when using schema by name strategy
This closes #2523.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-15 16:16:12 -04:00
Pierre Villard
930417b9dc NIFI-4809 - Implement a SiteToSiteMetricsReportingTask
address review comments

NIFI-4809 - Added Record Writer property

added unit tests and additional details doc

review comments

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2430
2018-03-15 15:58:31 -04:00
Mark Payne
844da06344
NIFI-4849: Implemented REST Endpoint and associated backend code to generate a Diagnostics Report for a Processor
Implemented review feedback. Refactored data model to make the API cleaner and delineate more along the lines of what permissions are required in order to see which details
Implementing review feedback
Removed sensitive information from the diagnostics reports
Fixed bug in merging logic for GCDiagnosticsSnapshots
This closes #2468
2018-03-15 15:10:41 -04:00
Mark Payne
867ffdb52e NIFI-4976: If unable to retrieve message content, warn an error but acknowledge message.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2545.
2018-03-14 22:50:34 +01:00
Koji Kawamura
d3f54994a6 NIFI-4972 - SelectHiveQL to emit FETCH provenance event
SelectHiveQL should emit FETCH instead of CONTENT_MODIFIED when it has
incoming connections.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2543.
2018-03-14 10:17:19 +01:00
Jorge Machado
b0e5e1644c NIFI-4821 - Upgrade poi to 3.17
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2534.
2018-03-13 22:59:03 +01:00
Deon Huang
37ad48844a NIFI-4962 FlattenJson processor add unexpected backslash after flatten
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2538.
2018-03-13 22:33:53 +01:00
Pierre Villard
11a7d5c09f NIFI-4955 - Preserve columns ordering with CSV and ValidateRecord
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2528
2018-03-13 17:09:07 -04:00
Pierre Villard
95e543d4cc NIFI-4967 - CSVRecordReader does not read header with specific formats
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2536
2018-03-13 16:54:46 -04:00
Pierre Villard
c75604a564 NIFI-4966 - JacksonCSVRecordReader - NPE with some CSV formats
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2535
2018-03-13 16:49:47 -04:00
Ed
29c2e18832 NIFI-4968: fix printing indefinite log errors
After first failure in customValidate, stop printing logs until any property is changed

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2537
2018-03-13 16:40:49 -04:00
Bryan Bende
9b098f3235
NIFI-4864 - Additional improvements to additonal resource loading
This closes #2470.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-03-13 13:23:28 -04:00
zenfenan
fad152f389
NIFI-4864 Fixing Additional Resources property pointing at a directory won't find new JARs
NIFI-4864: Code refactor and improved additionalUrlsFingerprint implementation
Signed-off-by: Bryan Bende <bbende@apache.org>
2018-03-13 13:23:16 -04:00
Bryan Bende
f1d7518547
NIFI-4969 Fixing error when importing versioned flow with a processor that uses event driven scheduling. This closes #2539 2018-03-13 13:15:24 -04:00
Mark Payne
d14229e440 NIFI-4774: Allow user to choose which write-ahead log implementation should be used by the WriteAheadFlowFileRepository
Removed TODO comment

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2487
2018-03-13 13:03:39 -04:00
Ed
c2616e6fe7
NIFI-4833 Add ScanHBase Processor
- New processor for scanning HBase records based on verious params like range of rowkeys, range of timestamps. Supports result limit and reverse scan.
- Adds Atlas Support for ScanHBase processor
- Fixed not recent version of FF
- Formatting and Style changes
- Single line to multiline if-then statements
- Removed HTML formatting that is not used for doc generation
- Fixed issue with limitRows
- Fixed issue with filter expression
- Refactored "processRows"
- Fixed possible NPE for bulkSize var
- Changed provenance to "receive" to indicate new data from external source.
- Updated min/max timestamp custom validation
- JSON array support
- Removed in-memory caching for records. Now records are being written directly to FF
- Removed unfinished flowfile from session, transfered original to "failure". Test cases update as well

This closes #2478.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-03-13 10:07:07 -04:00
m-hogue
6471f66acd NIFI-3402 - Added etag support to InvokeHTTP
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2150.
2018-03-13 10:00:11 +01:00