Commit Graph

4109 Commits

Author SHA1 Message Date
Mark Payne b1901d5fe0 NIFI-3838: Initial implementation of RecordPath and UpdateRecord processor
NIFI-3838: Updated version from 1.2.0-SNAPSHOT to 1.3.0-SNAPSHOT; removed unneeded value from AttributeExpression.ResultType enum

NIFI-3838: Addressed PR Review feedback

NIFI-3838: Allow for schemas to be merged together for a record; refactored RecordSetWriterFactory so that there is a method to obtain the schema and then the writer is created with that schema. Added additional unit tests

NIFI-3838: Addressed problems with documentation based on PR Review

NIFI-3838: Fixed checkstyle violation

NIFI-3838: Addressed issue of comparing different types of Number objects

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1772
2017-05-12 12:36:52 -04:00
Scott Aslan 1c58e78ce9 [NIFI-3876] trim whitespace processor names before applying ellipsis
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1789.
2017-05-12 16:01:43 +02:00
Matt Gilman dd0306cce7 NIFI-3837: - Being more specific regarding the escaping of back references in evaluated expressions.
Signed-off-by: Matt Burgess <mattyb149@apache.org>

fixed typo in comment

This closes #1781
2017-05-11 22:16:46 -04:00
Koji Kawamura 0b0ac196ea NIFI-3861: AvroTypeUtil used different constant.
Previous fix #1779 refactored the way to check Logical type to use string constants.
One of those refactoring used wrong constant mistakenly in normalizeValue method.

Fortunately, this defect is harmless since even though normalizeValue did not convert int to Time, DataTypeUtils.convertType does the same conversion.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1782
2017-05-11 21:31:07 -04:00
Steve Champagne 6e4db6b11a NIFI-3871: Convert Avro map values
This closes #1787.

- When converting from a raw value to an Avro object, convert the values
  of any Avro map types so that they can be complex types like other
  records.
2017-05-11 20:25:19 -04:00
Aldrin Piri 0f693e9448 NIFI-3802 Add link to ASF HipChat NiFi room in README.
This closes: #1767

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-05-11 09:04:04 +10:00
Matt Gilman cf497c9cf9
NIFI-3865: - Obtaining the appropriate input stream after accessing content. The stream was being consumed into a buffer and was not longer available.
This closes #1780.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-10 15:51:00 -04:00
Tim Reardon dc44e30698 NIFI-3856 PutCloudWatchMetric Stats, Dimensions
PutCloudWatchMetric support for StatisticSets and Dimensions

Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1775.
2017-05-10 12:07:04 -07:00
Tim Reardon 361a58c531 NIFI-3851 Support EL in AWS Endpoint Override URL
Add expression language support to AWS Endpoint Override URL property

Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1769.
2017-05-10 10:37:39 -07:00
Koji Kawamura 72de1cbdef NIFI-3861: Fixed AvroReader nullable logical types issue
- AvroReader did not convert logical types if those are defined with union
- Consolidated createSchema method in AvroSchemaRegistry and AvroTypeUtil as both has identical implementation and mai
ntaining both would be error-prone

This closes #1779.
2017-05-10 10:08:22 -04:00
Andrew Lim b6bdc4a0a8 NIFI-3850 Corrected errors in Admin Guide and made property formatting consistent
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1771
2017-05-09 15:06:23 -04:00
Koji Kawamura 744ecc3d83 NIFI-3832: Fixed AvroRecordSetWriter validation NPE.
This closes #1768.
2017-05-09 14:11:43 -04:00
Bryan Bende 3af53419af
NIFI-3770-RC2 prepare for next development iteration 2017-05-05 20:50:28 -04:00
Bryan Bende 3a605af8e0
NIFI-3770-RC2 prepare release nifi-1.2.0-RC2 2017-05-05 20:50:14 -04:00
joewitt f4f174b899
NIFI-3826 added proper NOTICE entries for apache calcite and maxmind source
This closes #1766.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 20:00:22 -04:00
joewitt 829c02b058 NIFI-3825 fix L&N for jolt, json, and source deps
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1765
2017-05-05 18:43:04 -04:00
Joey Frazee d069bf9c3e NIFI-3815 This closes #1763. Fix build RPM using mvn install -Prpm,generateArchives
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-05 16:12:28 -04:00
Jeremy Dyer 5f65b2561a NIFI-3600 Improve logging and relationship routing for failures in DeleteHDFS
realized that the session should be cloned here because its inside a for loop and the original flow file would be transferred but not be the latest flow file if an error occurred in the for loop

@trixpan at a high level what do you think about this approach?

NIFI-3600: Added unit test

NIFI-3600: Removed the hdfs.error.code attribute
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1595
2017-05-05 15:25:08 -04:00
Matt Gilman f0dfcc180d NIFI-3808:
- Making the selector for the enable/disable controller service state more specific. Previously name was too general and was affect other parts of the application.

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1759
2017-05-05 15:19:28 -04:00
joewitt ee8dc7640f
NIFI-3820 added calcite to assembly notice and updated all copyright years for compress, avro, and jetty
This closes #1764.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 14:45:40 -04:00
Mark Payne 12249e5a08
NIFI-2268: Instead of allowing HandleHttpRequest to run constantly calling BlockingQueue.poll() as fast as possible, call BlockingQueue.poll(2, TimeUnit.MILLISECONDS) to avoid overutilization of CPU when there is no work to be done
This closes #1762.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 13:58:10 -04:00
Matt Gilman e6be5d3276
NIFI-3801:
- Making the version text next to the component name more subtle.

This closes #1760.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-05 13:50:21 -04:00
Koji Kawamura af6f63691c
NIFI-3818: PutHiveStreaming throws IllegalStateException
Changed from async append to sync as it breaks 'recursionSet' check in StandardProcessSession by updating it from multiple threads, resulting IllegalStateException to happen.

This closes #1761.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 13:25:59 -04:00
Scott Aslan 85405dae15
[NIFI-3813] fix broken PG config dialog when user has read only perms on the PG. This closes #1757 2017-05-05 12:37:56 -04:00
Bryan Bende d9410d6404 NIFI-3795 Moved no-git profile to nifi-nar-bundles
Removing buildnumber plugin and build-info-no-git profile from nifi-assembly
and adding build-info-no-git profile to nifi-nar-bundles pom

Signed-off-by: James Wing <jvwing@gmail.com>
2017-05-04 10:09:25 -07:00
Andy LoPresto 7f2f38be52
NIFI-3796
Added test logic to only run POSIX permission set on *nix OS.
Separated missing and unreadable key providers to different tests and run unreadable on POSIX-compliant OS only.
This closes #1751
2017-05-04 12:13:06 -04:00
Andy LoPresto 4f40eca16c
NIFI-3788 Switched Amazon HTTP client instantiation from using null HostnameVerifier (which defaulted to Strict, which cannot handle wildcard certificate hostnames) to DefaultHostnameVerifier, which is fine.
I still want to add unit tests and integration tests, but I ran a flow which had previously caused the reproducible exception and this worked fine (flow showed objects were put in S3, no exceptions, and I verified through AWS Web Console that new objects were present).

This closes #1753.

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-05-04 12:09:52 -04:00
Matt Gilman f3745065b0
NIFI-3800:
- Cleaning up the headers when replicating requests.

This closes #1752.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-04 12:05:24 -04:00
Andre F de Miranda 970ddf8f7e
NIFI-3794 - Expose the control of ListenRELP's CLIENT_AUTH property to DFM
This closes #1746.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-04 10:50:22 -04:00
Yolanda M. Davis b710420f09
NIFI-3793 - switch to reference dependency directly
This closes #1748.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-04 09:08:18 -04:00
Yolanda M. Davis 5dac189601
NIFI-3793 - switch to use project.version
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-04 08:56:06 -04:00
Scott Aslan 98fb58975f [NIFI-3021] add white-space: pre to the #label-value element
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1749.
2017-05-04 07:45:08 +02:00
Bryan Bende 1e75f33677
NIFI-3789 Removing unnecessary intermittent test failure as described in JIRA
This closes #1743.
2017-05-03 14:07:18 -04:00
Mark Payne 9b177fbcba
NIFI-3787: Addressed NPE and ensure that if validation fails due to RuntimeException, that it gets logged. Also clarified documentation for Json Reader services
This closes #1742.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-03 13:13:43 -04:00
Matt Burgess b7c15c360b NIFI-3718: Fixed timezone issue in TestAvroReaderWithEmbeddedSchema
This closes #1738
2017-05-03 11:53:40 -04:00
Bryan Bende e723a2c5c4
NIFI-3776 Correcting links in documentation for SeeAlso and properties referencing controller services. This closes #1739 2017-05-03 11:15:48 -04:00
Matt Gilman ce233bdbbb
NIFI-3782: - Ensuring referenced Controller Services are not included in copy/paste requests when the Processor is located in a sub group.
This closes #1740.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-03 11:04:08 -04:00
Andre F de Miranda 54d47c7f74 NIFI-3726 - Introduces CompareFuzzyHash processor
- Abstract FuzzyhashContent to reduce a bit of code
            duplication

NIFI-3726 - Attempt to address peer review comments

NIFI-3726 - Addresses additional PR comments

NIFI-3726 - Fix checkstyle violations

NIFI-3726 - Fix issues with POMs

NIFI-3726 - Fixes the logging level to error in case of failure due to message too short

NIFI-3726 - Capture ArrayIndexOutOfBoundsException when consuming a hash source

            that matches but lacks a separator AND filename or matches but lacks
            a filename(i.e. ends with trailing separator)

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1692
2017-05-03 09:18:17 -04:00
Sarah Olson 960ef9142d Documentation for admin toolkit
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1736
2017-05-03 09:01:59 -04:00
Mark Payne daa277b067 NIFI-3775: This closes #1735. Ensure that a '0' byte is written out to Hortonworks Encoded Schema Reference header to indicate that GenericRecord is being used
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-02 17:43:26 -04:00
Sarah Olson 16f37763f6
NIFI-3773 Added documentation and screenshots for Component Versioning.
This closes #1734.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-02 17:37:18 -04:00
Andrew Lim 580d65dfde
NIFI-3701 Documentation improvements for 1.x.
This closes # 1733.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-02 16:41:47 -04:00
Jeff Storck 26d90fbccf
NIFI-1833 - Addressed issues from PR review.
NIFI-1833 Moved AbstractListProcessor.java, EntityListing.java, and ListableEntity.java from nifi-standard-processors into nifi-processor-utils
Moved TestAbstractListProcessor.java into nifi-processor-utils
Set nifi-azure-nar's nar dependency back to nifi-standard-services-api-nar
Fixed failing integration tests (ITFetchAzureBlobStorage.java, ITListAzureBlobStorage.java, and ITPutAzureStorageBlob.java) and refactored them to be able to run in parallel

NIFI-1833 Moved security notice info in the additional details documentation into the descriptions of the specific attributes for which those notices are intended
Added displayName usage to properties
Updated exception handling in FetchAzureBlobStorage.java and PutAzureBlobStorage.java to cause flowfiles with Output/InputStreamCallback failures to be routed to the processor's failure relationship
Cleaned up dependencies in pom

NIFI-1833 Removed unnecessary calls to map on Optional in the onTrigger exception handling of FetchAzureBlobStorage.java and PutAzureBlobStorage.java

NIFI-1833 Updates due to nifi-processor-utils being moved under nifi-nar-bundles

This closes #1719.

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-05-02 14:39:46 -04:00
Andrew Grande f30c8169ab
NIFI-1833 - Addressed issues from PR review.
Addressed dependency issues from the review.
Addressed a checkstyle issue.
Review: reworded the descriptions.
Review: implemented the reset condition logic.
Review: dropped static qualifier from method signatures, not required really
Review: removed sys.out, inlined a single method to get access to the ProcessContext.getName()
Switched to HTTPS as per MSFT recommendation. Some DRY. Dropped cruft.
Addressing review suggestions from 4/5
Review: documentation improvements
Review: documentation improvements

This closes #1636.

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-05-02 14:39:37 -04:00
Simon Elliston Ball 3488a169ca
NIFI-1833 - Azure Storage processors
Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-05-02 14:39:16 -04:00
Andy LoPresto 7d242076ce
NIFI-3594 Implemented encrypted provenance repository.
Added src/test/resources/logback-test.xml files resetting log level from DEBUG (in nifi-data-provenance-utils) to WARN because later tests depend on MockComponentLog recording a certain number of messages and this number is different than expected if the log level is DEBUG.

This closes #1686.

Signed-off-by: Bryan Bende, Yolanda M. Davis, and Mark Payne
2017-05-02 13:24:07 -04:00
Yolanda M. Davis fae2e3aa21
NIFI-3765
- added status operation to NodeManager
- updated help text
- add multiple url support with the -u flag
- This closes #1726
2017-05-02 13:07:18 -04:00
Andre F de Miranda eb12b932ab NIFI-3651 - Add logo to README.md
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1721
2017-05-02 10:08:56 -04:00
Koji Kawamura cca5c42095 Incorporated review comments.
- Added description on what session maintenance does.
- Added calling deregister when initial connection attempt fails so that a processor can retry connecting at next onTrigger.

This closes #1597

Signed-off-by: Jeremy Dyer <jeremydyer@apache.org>
2017-05-02 10:02:55 -04:00
Koji Kawamura 0a014b471b NIFI-3609: ConnectWebSocket auto session recovery
- Removed unused disconnect method from WebSocketService interface.
- Added session maintenance background thread at JettyWebSocketClient
  which reconnects sessions those are still referred by ConnectWebSocket
  processor but no longer active.
- Added Session Maintenance Interval property to JettyWebSocketClient.
- Allowed specifying existing session id so that it can be recovered
  transparently.
- Moved test classes to appropriate package.
- Added test cases that verify the same session id can be used after
  WebSocket server restarts.
2017-05-02 10:02:55 -04:00