Commit Graph

4147 Commits

Author SHA1 Message Date
Andre F de Miranda da6ad4f3bc
NIFI-3670 - Expose the control of ListenSyslog's CLIENT_AUTH property to DFM
This closes #1720.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-01 10:41:12 -04:00
Andre F de Miranda a9a9b67430 NIFI-2616 - Remove 2 unused classes in IdentifyMimeType identified during PR 2017-05-02 00:25:42 +10:00
Jon Anderson III 25295cdb4d NIFI-2616 Remove reference to non-existent setting in IdentifyMimeType Additional Details
Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-05-02 00:20:04 +10:00
Andre F de Miranda 9ad7802284
NIFI-1939 - Correct issue where ParseSyslog was unable to parse RFC3164 messages containg an IPv6 address as source
NIFI-1939 - Fix typo and adjust ListenSyslog as per PR feedback

This closes #1639.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-01 10:00:33 -04:00
Bryan Bende de67e5f7d5
NIFI-3674 Adding EL support to regex properties and cleaning up property descriptions
This closes #1660.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-01 09:55:17 -04:00
Joe Gresock ea6320d621
NIFI-3674: Implementing SiteToSiteStatusReportingTask
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-01 09:55:10 -04:00
Pierre Villard 7df5c2dc89 NIFI-3607 Allow multi files mode with fixed names
This closes: #1608

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-05-01 09:31:24 +10:00
Bryan Rosander 0054a9e35f
NIFI-3755 - Restoring Hive exception handling behavior
This closes #1711.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-04-28 11:45:43 -04:00
Matt Burgess aa4efb43ca NIFI-3749: Added database filtering to DDL events in CaptureChangeMySQL
This closes #1708.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-28 13:16:01 +09:00
Andrew Lim 2664ea093d NIFI-3756 This closes #1715. Update ConvertAvroToOrc documentation to note that processor does not support unions of nested data
Signed-off-by: joewitt <joewitt@apache.org>
2017-04-28 00:02:03 -04:00
Matt Burgess d61f519326 NIFI-3742: Fixed handling of nulls in DELETE for PutDatabaseRecord
This closes #1709.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-28 11:34:43 +09:00
joewitt 8651d79778 NIFI-3757 grabs latest kafka client 0.10.2.1
This closes #1714.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-28 11:16:35 +09:00
Andrew Lim 6512306fa4 NIFI-3752 Fixed broken links in User Guide
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1710
2017-04-27 15:42:44 -04:00
Joe Percivall 49a9bc88c6 NIFI-3633 This closes #1705. Adding a try with resources so the OkHttp response is properly closed
Signed-off-by: joewitt <joewitt@apache.org>
2017-04-27 15:04:51 -04:00
Mark Payne 726d15d1f7 NIFI-3717 This closes #1707. Added HortonworksSchemaRegistry controller service for schema lookup
Signed-off-by: joewitt <joewitt@apache.org>
2017-04-27 14:56:52 -04:00
Koji Kawamura d9acdb54be NIFI-3415: Add Rollback on Failure.
- Added org.apache.nifi.processor.util.pattern package in nifi-processor-utils containing reusable functions to mix-in 'Rollback on Failure' capability.
- Created a process pattern classes, Put and PutGroup. It will be helpful to standardize Processor implementations.
- Applied Rollback on Failure to PutSQL, PutHiveQL, PutHiveStreaming and PutDatabaseRecord.
- Stop using AbstractProcessor for these processors, as it penalizes FlowFiles being processed when it rollback a process session. If FlowFiles are penalized, it will not be fetched again until penalization expires.
- Yield processor when a failure occurs and RollbackOnFailure is enabled. If we do not penalize nor yield, a failed FlowFile retries too frequently.
- When Rollback on Failure is enabled but processor is not transactional, discontinue when an error occurred after successful processes.
- Fixed existing issues on PutHiveStreaming:
  - Output FlowFile Avro format was corrupted by concatenating multiple Avro files.
  - Output FlowFile records had incorrect values because of reusing GenericRecord instance.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1658
2017-04-27 13:44:56 -04:00
Mark Payne a1bffbcc87 NIFI-3659: This closes #1707. Added Processor to Split a FlowFile consisting of Record-oriented data into multiple FlowFiles, each containing a subset of the original FlowFile's records
Signed-off-by: joewitt <joewitt@apache.org>
2017-04-27 12:51:01 -04:00
Yolanda M. Davis 55b8c7ddad
NIFI-3695
- added proxy dn flag to support providing authorized username for secured environments
- addressed pr comments including fix to ensure proxy info added when getting cluster info, showing cleaner error messaging  and improving help text. Also fixed potential issue with versioning comparison (mismatched lengths)
- Printing response body when requests fails.
- This closes #1697
2017-04-27 12:42:12 -04:00
Koji Kawamura a41a2a9b1a
NIFI-1202: Site-to-Site batch settings.
- Added batchCount, batchSize, batchDuration to limit flow files to be
  included in a single Site-to-Site transaction.
- Added batch throttling logic when StandardRemoteGroupPort transfers
  flow files to a remote input port using the batch limit configurations,
  so that users can limit batch not only for pulling data, but also pushing data.
- Added destination list shuffle to provide better load distribution.
  Previously, the load distribution algorithm produced the same host consecutively.
- Added new batch settings to FlowConfiguration.xsd.
- Added new batch settings to Flow Fingerprint.
- Added new batch settings to Audit.
- Sort ports by name at 'Remote Process Group Ports' dialog.
- Show 'No value set' when a batch configuration is not set
- Updated batch settings tooltip to clearly explain how it works the configuration works differently for input and output ports.
- Updated DTO by separating batch settings to BatchSettingsDTO to indicate count, size and duration are a set of configurations.
- This closes #1306
2017-04-27 10:35:07 -04:00
Matt Burgess cf0e8bb1d3 NIFI-3741: Improved connection error message for CaptureChangeMySQL
This closes #1704.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-27 10:40:35 +09:00
Matt Burgess 097548da9d NIFI-3745: Fixed Table caching / primary key logic in PutDatabaseRecord
This closes #1700.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-27 08:43:13 +09:00
Matt Burgess d66eac2ea1 NIFI-3743: Include RENAME TABLE events in CaptureChangeMySQL
This closes #1701.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-27 08:42:01 +09:00
Matt Burgess da0454d80f NIFI-3746: Fixed DDL event transfer when outside a transaction in CaptureChangeMySQL
This closes #1702.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-27 08:35:34 +09:00
Matt Burgess 24bb8cf95d NIFI-3735: Replaced 'Schema Change Event' references to 'DDL Event'
This closes #1703.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-27 08:26:33 +09:00
Bryan Rosander 97461657b1 NIFI-3744 - PutHiveStreaming cleanup null fixes
This closes #1698.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
2017-04-26 22:54:49 +02:00
Andre F de Miranda 5cacc52cfc NIFI-2661 - Abstract the GeoEnrichIP processor NIFI-2661 - Add accuracy radius to GeoEnrichIP NIFI-2661 - Introduce ISPEnrichIP Processor
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-2661 - Addresses PR comments

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-2661 - Addresses PR comments

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-2661: Fixed typos and reversed logging params

This closes #1650
2017-04-26 09:10:48 -04:00
Matt Burgess 4303e4742a NIFI-3704: Add PutDatabaseRecord processor
NIFI-3704: Incorporated review comments

NIFI-3704: Updated PutDatabaseRecord documentation, including limitations on Update records

NIFI-3704: Added retry paths for transient SQL errors

NIFI-3704: Updated to use renamed RecordReaderFactory and catch SchemaNotFoundException

NIFI-3704: Added better error handling and rollback

This closes #1677.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-26 09:50:50 +09:00
Andy LoPresto ef5bac207e
NIFI-3738 Fixed NPE when ListenSyslog UDP datagram has zero length. Added default constructor to SyslogParser to allow map coercion for test. Added unit test.
NIFI-3738 Fixed NPE when ParseSyslog UDP datagram has zero length.
Added unit test.

NIFI-3738 Added licenses to new unit tests.

This closes #1694.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-04-25 16:38:46 -04:00
Matt Burgess 49a62448ce
NIFI-3734: Add ScriptedReader and ScriptedRecordSetWriter
This closes #1691.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-04-25 10:57:21 -07:00
Bryan Rosander ee4b88620a NIFI-3737 - Reverting problematic pngs
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1693
2017-04-25 12:47:48 -04:00
Bryan Rosander 879fe3efd6 NIFI-3737 - Ran optipng on all png files 2017-04-25 12:28:49 -04:00
Matt Burgess 26ca40c0d9 NIFI-3728: Detect 'truncate table' event, allow exclusion of schema change events in CaptureChangeMySQL
NIFI-3728: Changed 'Include Schema Change Events' to 'Include DDL Events', updated tests

This closes #1690.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-25 23:37:09 +09:00
Matt Burgess 9ed001869b NIFI-2323: Fixed Oracle timestamp handling in DB fetch processors. This closes #1664. 2017-04-25 09:59:43 -04:00
Anders Breindahl 995c7ce2fa
NIFI-2656: replace -k [password] with -K [passwordfile].
This approaches a proper solution on how to hand over the key from
RunNiFi to NiFi. Insofar the password file is pruned as part of the
startup, NiFi processors can't read it.

See also: NIFI-3045.

This closes #1302.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-04-25 00:32:58 -07:00
Yolanda M. Davis c0f0462e8b
NIFI-3695 - created the nifi admin toolkit which includes shell scripts and classes to support notification and basic node management in standalone and clustered nifi.
This closes #1669.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-04-24 23:08:37 -07:00
Matt Burgess ba2bdf8586 NIFI-3730: Allow exclusion of BEGIN/COMMIT event output in CaptureChangeMySQL
This closes #1689.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-25 14:40:24 +09:00
Mark Payne 50ea1083ec NIFI-3682: This closes #1682. Add Schema Access Strategy and Schema Write Strategy Record Readers and Writers; bug fixes.
Signed-off-by: joewitt <joewitt@apache.org>
2017-04-24 17:02:45 -04:00
Scott Aslan 57ccf97c58
[NIFI-3035] disable deep linking when URL .searchParams and URL .search properties are unsupported by browser. This closes #1687 2017-04-21 14:37:27 -04:00
Matt Gilman 0b95ccb90a NIFI-3380:
- Fixing opening of Usage from the Controller Service and Reporting Task table.

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1684
2017-04-21 09:48:24 -04:00
Scott Aslan 2f928490e3
[NIFI-3360] inject nf.ControllerServices into nf.ControllerService. This closes #1685 2017-04-21 08:24:14 -04:00
Bryan Rosander 1c9ea249f7 NIFI-3666 - TlsToolkit using .equals() comparison for parent and root
This closes #1641.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-21 16:24:13 +09:00
Jeremy Dyer 769e874677 Support for websocket multiplexing to all existing websocket
connections (Think chat to all clients instead of individual person).
The core change was a change in WebSocketMessageRouter.java where if a
sessionId is not present the message is sent to all connected clients.
So the key is leaving the sessionId to empty or null to send to all
clients. If the sessionId is specified the message will be sent just to
that session specified.

This closes #1649.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-21 08:21:56 +09:00
ShellyLC 816034bd01 NIFI-3543 This closes #1548. Added support for using EL when defining brokerURL in jms-cf-service
Signed-off-by: joewitt <joewitt@apache.org>
2017-04-19 22:50:48 -07:00
Koji Kawamura 8ce2a1b3a7 NIFI-3657 This closes #1634. Fix HTTP S2S to use local address.
- Fixed SiteInfoProvider and HttpClient to use specified local address with its SiteToSiteRestApiClient
- Removed setupRequestConfig method call from connection and read timeout setter methods at SiteToSiteRestApiClient, because it created config object before local address was set
- Null clear StandardRemoteProcessGroup localAddress when user clears Local Network Interface

Signed-off-by: joewitt <joewitt@apache.org>
2017-04-19 22:19:41 -07:00
Koji Kawamura 0ddb90cd2d NIFI-3166 This closes #1324. Fix SocketRemoteSiteListener NPE.
- Refactored ServerProtocol.sendPeerList method signature to clarify the
  meaning of arguments, and avoid null pointer exception when converting null Integer to int.
- Refactored SocketRemoteSiteListener handleRequest method to make it
  more unit test friendly.
- Added more unit tests.
2017-04-19 22:18:43 -07:00
Mark Payne 0f2ac39f69 NIFI-3273 This closes #1611. Handle the case of trailing NUL bytes in MinimalLockingWriteAheadLog 2017-04-19 22:08:59 -07:00
Mark Payne 0207f21ce4 NIFI-3273: Added nifi-toolkit-flowfile-repo that contains a simple Java class that is capable of recovering a FlowFile Repository manually in the case of an operating system crash that results in trailing 0's being dumped into the edit logs. Also refactored flowfile repo into some independent modules so that it additional capabilities can be added in the future to examine the flowfile repo 2017-04-19 21:44:23 -07:00
Matt Burgess 141334c3c9 NIFI-3718: Fixed TestAvroRecordReader to handle timezone differences
This closes #1683
2017-04-19 17:04:45 -04:00
Mark Payne b93cf7bbdb NIFI-3658: Created processor for converting records between data formats with like schemas
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-3658: Incorporated PR review feedback; added counter; clarified documentation

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1668
2017-04-19 16:12:14 -04:00
Matt Gilman 7e7f9a5deb NIFI-3705:
- Using consistent logic when verifying connection creation. Removed some unecessary checks as verification has been changed to run in cluster and standalone mode.

This closes #1672.
2017-04-19 15:49:11 -04:00