Commit Graph

3589 Commits

Author SHA1 Message Date
Mark Payne 12249e5a08
NIFI-2268: Instead of allowing HandleHttpRequest to run constantly calling BlockingQueue.poll() as fast as possible, call BlockingQueue.poll(2, TimeUnit.MILLISECONDS) to avoid overutilization of CPU when there is no work to be done
This closes #1762.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 13:58:10 -04:00
Matt Gilman e6be5d3276
NIFI-3801:
- Making the version text next to the component name more subtle.

This closes #1760.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-05 13:50:21 -04:00
Koji Kawamura af6f63691c
NIFI-3818: PutHiveStreaming throws IllegalStateException
Changed from async append to sync as it breaks 'recursionSet' check in StandardProcessSession by updating it from multiple threads, resulting IllegalStateException to happen.

This closes #1761.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 13:25:59 -04:00
Scott Aslan 85405dae15
[NIFI-3813] fix broken PG config dialog when user has read only perms on the PG. This closes #1757 2017-05-05 12:37:56 -04:00
Bryan Bende d9410d6404 NIFI-3795 Moved no-git profile to nifi-nar-bundles
Removing buildnumber plugin and build-info-no-git profile from nifi-assembly
and adding build-info-no-git profile to nifi-nar-bundles pom

Signed-off-by: James Wing <jvwing@gmail.com>
2017-05-04 10:09:25 -07:00
Andy LoPresto 7f2f38be52
NIFI-3796
Added test logic to only run POSIX permission set on *nix OS.
Separated missing and unreadable key providers to different tests and run unreadable on POSIX-compliant OS only.
This closes #1751
2017-05-04 12:13:06 -04:00
Andy LoPresto 4f40eca16c
NIFI-3788 Switched Amazon HTTP client instantiation from using null HostnameVerifier (which defaulted to Strict, which cannot handle wildcard certificate hostnames) to DefaultHostnameVerifier, which is fine.
I still want to add unit tests and integration tests, but I ran a flow which had previously caused the reproducible exception and this worked fine (flow showed objects were put in S3, no exceptions, and I verified through AWS Web Console that new objects were present).

This closes #1753.

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-05-04 12:09:52 -04:00
Matt Gilman f3745065b0
NIFI-3800:
- Cleaning up the headers when replicating requests.

This closes #1752.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-04 12:05:24 -04:00
Andre F de Miranda 970ddf8f7e
NIFI-3794 - Expose the control of ListenRELP's CLIENT_AUTH property to DFM
This closes #1746.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-04 10:50:22 -04:00
Yolanda M. Davis b710420f09
NIFI-3793 - switch to reference dependency directly
This closes #1748.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-04 09:08:18 -04:00
Yolanda M. Davis 5dac189601
NIFI-3793 - switch to use project.version
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-04 08:56:06 -04:00
Scott Aslan 98fb58975f [NIFI-3021] add white-space: pre to the #label-value element
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1749.
2017-05-04 07:45:08 +02:00
Bryan Bende 1e75f33677
NIFI-3789 Removing unnecessary intermittent test failure as described in JIRA
This closes #1743.
2017-05-03 14:07:18 -04:00
Mark Payne 9b177fbcba
NIFI-3787: Addressed NPE and ensure that if validation fails due to RuntimeException, that it gets logged. Also clarified documentation for Json Reader services
This closes #1742.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-03 13:13:43 -04:00
Matt Burgess b7c15c360b NIFI-3718: Fixed timezone issue in TestAvroReaderWithEmbeddedSchema
This closes #1738
2017-05-03 11:53:40 -04:00
Bryan Bende e723a2c5c4
NIFI-3776 Correcting links in documentation for SeeAlso and properties referencing controller services. This closes #1739 2017-05-03 11:15:48 -04:00
Matt Gilman ce233bdbbb
NIFI-3782: - Ensuring referenced Controller Services are not included in copy/paste requests when the Processor is located in a sub group.
This closes #1740.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-03 11:04:08 -04:00
Andre F de Miranda 54d47c7f74 NIFI-3726 - Introduces CompareFuzzyHash processor
- Abstract FuzzyhashContent to reduce a bit of code
            duplication

NIFI-3726 - Attempt to address peer review comments

NIFI-3726 - Addresses additional PR comments

NIFI-3726 - Fix checkstyle violations

NIFI-3726 - Fix issues with POMs

NIFI-3726 - Fixes the logging level to error in case of failure due to message too short

NIFI-3726 - Capture ArrayIndexOutOfBoundsException when consuming a hash source

            that matches but lacks a separator AND filename or matches but lacks
            a filename(i.e. ends with trailing separator)

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1692
2017-05-03 09:18:17 -04:00
Sarah Olson 960ef9142d Documentation for admin toolkit
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1736
2017-05-03 09:01:59 -04:00
Mark Payne daa277b067 NIFI-3775: This closes #1735. Ensure that a '0' byte is written out to Hortonworks Encoded Schema Reference header to indicate that GenericRecord is being used
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-02 17:43:26 -04:00
Sarah Olson 16f37763f6
NIFI-3773 Added documentation and screenshots for Component Versioning.
This closes #1734.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-02 17:37:18 -04:00
Andrew Lim 580d65dfde
NIFI-3701 Documentation improvements for 1.x.
This closes # 1733.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-02 16:41:47 -04:00
Jeff Storck 26d90fbccf
NIFI-1833 - Addressed issues from PR review.
NIFI-1833 Moved AbstractListProcessor.java, EntityListing.java, and ListableEntity.java from nifi-standard-processors into nifi-processor-utils
Moved TestAbstractListProcessor.java into nifi-processor-utils
Set nifi-azure-nar's nar dependency back to nifi-standard-services-api-nar
Fixed failing integration tests (ITFetchAzureBlobStorage.java, ITListAzureBlobStorage.java, and ITPutAzureStorageBlob.java) and refactored them to be able to run in parallel

NIFI-1833 Moved security notice info in the additional details documentation into the descriptions of the specific attributes for which those notices are intended
Added displayName usage to properties
Updated exception handling in FetchAzureBlobStorage.java and PutAzureBlobStorage.java to cause flowfiles with Output/InputStreamCallback failures to be routed to the processor's failure relationship
Cleaned up dependencies in pom

NIFI-1833 Removed unnecessary calls to map on Optional in the onTrigger exception handling of FetchAzureBlobStorage.java and PutAzureBlobStorage.java

NIFI-1833 Updates due to nifi-processor-utils being moved under nifi-nar-bundles

This closes #1719.

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-05-02 14:39:46 -04:00
Andrew Grande f30c8169ab
NIFI-1833 - Addressed issues from PR review.
Addressed dependency issues from the review.
Addressed a checkstyle issue.
Review: reworded the descriptions.
Review: implemented the reset condition logic.
Review: dropped static qualifier from method signatures, not required really
Review: removed sys.out, inlined a single method to get access to the ProcessContext.getName()
Switched to HTTPS as per MSFT recommendation. Some DRY. Dropped cruft.
Addressing review suggestions from 4/5
Review: documentation improvements
Review: documentation improvements

This closes #1636.

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-05-02 14:39:37 -04:00
Simon Elliston Ball 3488a169ca
NIFI-1833 - Azure Storage processors
Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-05-02 14:39:16 -04:00
Andy LoPresto 7d242076ce
NIFI-3594 Implemented encrypted provenance repository.
Added src/test/resources/logback-test.xml files resetting log level from DEBUG (in nifi-data-provenance-utils) to WARN because later tests depend on MockComponentLog recording a certain number of messages and this number is different than expected if the log level is DEBUG.

This closes #1686.

Signed-off-by: Bryan Bende, Yolanda M. Davis, and Mark Payne
2017-05-02 13:24:07 -04:00
Yolanda M. Davis fae2e3aa21
NIFI-3765
- added status operation to NodeManager
- updated help text
- add multiple url support with the -u flag
- This closes #1726
2017-05-02 13:07:18 -04:00
Andre F de Miranda eb12b932ab NIFI-3651 - Add logo to README.md
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1721
2017-05-02 10:08:56 -04:00
Koji Kawamura cca5c42095 Incorporated review comments.
- Added description on what session maintenance does.
- Added calling deregister when initial connection attempt fails so that a processor can retry connecting at next onTrigger.

This closes #1597

Signed-off-by: Jeremy Dyer <jeremydyer@apache.org>
2017-05-02 10:02:55 -04:00
Koji Kawamura 0a014b471b NIFI-3609: ConnectWebSocket auto session recovery
- Removed unused disconnect method from WebSocketService interface.
- Added session maintenance background thread at JettyWebSocketClient
  which reconnects sessions those are still referred by ConnectWebSocket
  processor but no longer active.
- Added Session Maintenance Interval property to JettyWebSocketClient.
- Allowed specifying existing session id so that it can be recovered
  transparently.
- Moved test classes to appropriate package.
- Added test cases that verify the same session id can be used after
  WebSocket server restarts.
2017-05-02 10:02:55 -04:00
Matt Gilman 8fa35294eb
NIFI-3764: - Ensuring that UUIDs are not converted multiple times during template export.
This closes #1730.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-02 09:59:39 -04:00
Matt Gilman 82829927ae NIFI-3762:
- Addressing issue when enabling/disabling controller services when referencing services are not in scope.
- Fixing enabled/disabled icons for referencing services.
- Rendering when a reference cycle is detected.

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1725
2017-05-02 09:43:54 -04:00
Andre F de Miranda 3bfe323a5d NIFI-3769 - Document detail value of 'nifi.provenance.repository.warm.cache.frequency' 2017-05-02 08:44:01 -04:00
Andre F de Miranda be9e317000 This closes #584 (stalled)
This closes #399 (superseded)
2017-05-02 18:48:48 +10:00
Mark Payne 016ae3191e NIFI-3760: This closes #1717. Fixed issue that caused session to continually poll from the same connection when using ProcessSession.get(FlowFileFilter, Set)
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-02 00:31:30 -04:00
Mark Payne 40de1b18d9 NIFI-3739: This closes #1728. Pass the proper InputStream to RecordSetWriterFactory in order to obtain RecordSetWriter; also fix error handling so that we don't kill kafka client if unable to create writer, since we roll back the offsets
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-02 00:21:15 -04:00
Mark Payne b96e402e78 NIFI-3768: This closes #1727. Avoid wrapping FlowFile Attributes Map in an UnmodifiableMap in the StandardFlowFileBuilder
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-01 22:51:05 -04:00
Mark Payne 07989b8460 NIFI-3739: This closes #1695. Added ConsumeKafkaRecord_0_10 and PublishKafkaRecord_0_10 processors 2017-05-01 18:47:51 -04:00
Andy LoPresto 946f4a1a28
NIFI-3721 Added documentation for Encrypted Provenance Repositories to Admin Guide and User Guide.
Added screenshot of encrypted provenance repository contents on disk.
Added note about clearing existing provenance repository when switching to encrypted implementation (see PR 1686 @ https://github.com/apache/nifi/pull/1686#issuecomment-298432578).

This closes #1713.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-01 17:19:25 -04:00
Bryan Bende 60d88b5a64
NIFI-3724 - Initial commit of Parquet bundle with PutParquet and FetchParquet
- Creating nifi-records-utils to share utility code from record services
- Refactoring Parquet tests to use MockRecorderParser and MockRecordWriter
- Refactoring AbstractPutHDFSRecord to use schema access strategy
- Adding custom validate to AbstractPutHDFSRecord and adding handling of UNION types when writing Records as Avro
- Refactoring project structure to get CS API references out of nifi-commons, introducing nifi-extension-utils under nifi-nar-bundles
- Updating abstract put/fetch processors to obtain the WriteResult and update flow file attributes

This closes #1712.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-01 16:10:35 -04:00
Andrew Lim 11b935a27b NIFI-3748 Add component linking functionality to documentation
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1724
2017-05-01 15:37:13 -04:00
Andrew Lim 0a7b9467e9 NIFI-3684 Make docs more explicit about anonymous access to a secured instance
This closes #1722

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2017-05-01 13:13:54 -04:00
Andre F de Miranda da6ad4f3bc
NIFI-3670 - Expose the control of ListenSyslog's CLIENT_AUTH property to DFM
This closes #1720.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-01 10:41:12 -04:00
Andre F de Miranda a9a9b67430 NIFI-2616 - Remove 2 unused classes in IdentifyMimeType identified during PR 2017-05-02 00:25:42 +10:00
Jon Anderson III 25295cdb4d NIFI-2616 Remove reference to non-existent setting in IdentifyMimeType Additional Details
Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-05-02 00:20:04 +10:00
Andre F de Miranda 9ad7802284
NIFI-1939 - Correct issue where ParseSyslog was unable to parse RFC3164 messages containg an IPv6 address as source
NIFI-1939 - Fix typo and adjust ListenSyslog as per PR feedback

This closes #1639.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-01 10:00:33 -04:00
Bryan Bende de67e5f7d5
NIFI-3674 Adding EL support to regex properties and cleaning up property descriptions
This closes #1660.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-01 09:55:17 -04:00
Joe Gresock ea6320d621
NIFI-3674: Implementing SiteToSiteStatusReportingTask
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-01 09:55:10 -04:00
Pierre Villard 7df5c2dc89 NIFI-3607 Allow multi files mode with fixed names
This closes: #1608

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-05-01 09:31:24 +10:00
Bryan Rosander 0054a9e35f
NIFI-3755 - Restoring Hive exception handling behavior
This closes #1711.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-04-28 11:45:43 -04:00