Commit Graph

3431 Commits

Author SHA1 Message Date
Mark Payne a1bffbcc87 NIFI-3659: This closes #1707. Added Processor to Split a FlowFile consisting of Record-oriented data into multiple FlowFiles, each containing a subset of the original FlowFile's records
Signed-off-by: joewitt <joewitt@apache.org>
2017-04-27 12:51:01 -04:00
Yolanda M. Davis 55b8c7ddad
NIFI-3695
- added proxy dn flag to support providing authorized username for secured environments
- addressed pr comments including fix to ensure proxy info added when getting cluster info, showing cleaner error messaging  and improving help text. Also fixed potential issue with versioning comparison (mismatched lengths)
- Printing response body when requests fails.
- This closes #1697
2017-04-27 12:42:12 -04:00
Koji Kawamura a41a2a9b1a
NIFI-1202: Site-to-Site batch settings.
- Added batchCount, batchSize, batchDuration to limit flow files to be
  included in a single Site-to-Site transaction.
- Added batch throttling logic when StandardRemoteGroupPort transfers
  flow files to a remote input port using the batch limit configurations,
  so that users can limit batch not only for pulling data, but also pushing data.
- Added destination list shuffle to provide better load distribution.
  Previously, the load distribution algorithm produced the same host consecutively.
- Added new batch settings to FlowConfiguration.xsd.
- Added new batch settings to Flow Fingerprint.
- Added new batch settings to Audit.
- Sort ports by name at 'Remote Process Group Ports' dialog.
- Show 'No value set' when a batch configuration is not set
- Updated batch settings tooltip to clearly explain how it works the configuration works differently for input and output ports.
- Updated DTO by separating batch settings to BatchSettingsDTO to indicate count, size and duration are a set of configurations.
- This closes #1306
2017-04-27 10:35:07 -04:00
Matt Burgess cf0e8bb1d3 NIFI-3741: Improved connection error message for CaptureChangeMySQL
This closes #1704.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-27 10:40:35 +09:00
Matt Burgess 097548da9d NIFI-3745: Fixed Table caching / primary key logic in PutDatabaseRecord
This closes #1700.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-27 08:43:13 +09:00
Matt Burgess d66eac2ea1 NIFI-3743: Include RENAME TABLE events in CaptureChangeMySQL
This closes #1701.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-27 08:42:01 +09:00
Matt Burgess da0454d80f NIFI-3746: Fixed DDL event transfer when outside a transaction in CaptureChangeMySQL
This closes #1702.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-27 08:35:34 +09:00
Matt Burgess 24bb8cf95d NIFI-3735: Replaced 'Schema Change Event' references to 'DDL Event'
This closes #1703.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-27 08:26:33 +09:00
Bryan Rosander 97461657b1 NIFI-3744 - PutHiveStreaming cleanup null fixes
This closes #1698.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
2017-04-26 22:54:49 +02:00
Andre F de Miranda 5cacc52cfc NIFI-2661 - Abstract the GeoEnrichIP processor NIFI-2661 - Add accuracy radius to GeoEnrichIP NIFI-2661 - Introduce ISPEnrichIP Processor
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-2661 - Addresses PR comments

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-2661 - Addresses PR comments

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-2661: Fixed typos and reversed logging params

This closes #1650
2017-04-26 09:10:48 -04:00
Matt Burgess 4303e4742a NIFI-3704: Add PutDatabaseRecord processor
NIFI-3704: Incorporated review comments

NIFI-3704: Updated PutDatabaseRecord documentation, including limitations on Update records

NIFI-3704: Added retry paths for transient SQL errors

NIFI-3704: Updated to use renamed RecordReaderFactory and catch SchemaNotFoundException

NIFI-3704: Added better error handling and rollback

This closes #1677.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-26 09:50:50 +09:00
Andy LoPresto ef5bac207e
NIFI-3738 Fixed NPE when ListenSyslog UDP datagram has zero length. Added default constructor to SyslogParser to allow map coercion for test. Added unit test.
NIFI-3738 Fixed NPE when ParseSyslog UDP datagram has zero length.
Added unit test.

NIFI-3738 Added licenses to new unit tests.

This closes #1694.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-04-25 16:38:46 -04:00
Matt Burgess 49a62448ce
NIFI-3734: Add ScriptedReader and ScriptedRecordSetWriter
This closes #1691.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-04-25 10:57:21 -07:00
Bryan Rosander ee4b88620a NIFI-3737 - Reverting problematic pngs
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1693
2017-04-25 12:47:48 -04:00
Bryan Rosander 879fe3efd6 NIFI-3737 - Ran optipng on all png files 2017-04-25 12:28:49 -04:00
Matt Burgess 26ca40c0d9 NIFI-3728: Detect 'truncate table' event, allow exclusion of schema change events in CaptureChangeMySQL
NIFI-3728: Changed 'Include Schema Change Events' to 'Include DDL Events', updated tests

This closes #1690.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-25 23:37:09 +09:00
Matt Burgess 9ed001869b NIFI-2323: Fixed Oracle timestamp handling in DB fetch processors. This closes #1664. 2017-04-25 09:59:43 -04:00
Anders Breindahl 995c7ce2fa
NIFI-2656: replace -k [password] with -K [passwordfile].
This approaches a proper solution on how to hand over the key from
RunNiFi to NiFi. Insofar the password file is pruned as part of the
startup, NiFi processors can't read it.

See also: NIFI-3045.

This closes #1302.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-04-25 00:32:58 -07:00
Yolanda M. Davis c0f0462e8b
NIFI-3695 - created the nifi admin toolkit which includes shell scripts and classes to support notification and basic node management in standalone and clustered nifi.
This closes #1669.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-04-24 23:08:37 -07:00
Matt Burgess ba2bdf8586 NIFI-3730: Allow exclusion of BEGIN/COMMIT event output in CaptureChangeMySQL
This closes #1689.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-25 14:40:24 +09:00
Mark Payne 50ea1083ec NIFI-3682: This closes #1682. Add Schema Access Strategy and Schema Write Strategy Record Readers and Writers; bug fixes.
Signed-off-by: joewitt <joewitt@apache.org>
2017-04-24 17:02:45 -04:00
Scott Aslan 57ccf97c58
[NIFI-3035] disable deep linking when URL .searchParams and URL .search properties are unsupported by browser. This closes #1687 2017-04-21 14:37:27 -04:00
Matt Gilman 0b95ccb90a NIFI-3380:
- Fixing opening of Usage from the Controller Service and Reporting Task table.

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1684
2017-04-21 09:48:24 -04:00
Scott Aslan 2f928490e3
[NIFI-3360] inject nf.ControllerServices into nf.ControllerService. This closes #1685 2017-04-21 08:24:14 -04:00
Bryan Rosander 1c9ea249f7 NIFI-3666 - TlsToolkit using .equals() comparison for parent and root
This closes #1641.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-21 16:24:13 +09:00
Jeremy Dyer 769e874677 Support for websocket multiplexing to all existing websocket
connections (Think chat to all clients instead of individual person).
The core change was a change in WebSocketMessageRouter.java where if a
sessionId is not present the message is sent to all connected clients.
So the key is leaving the sessionId to empty or null to send to all
clients. If the sessionId is specified the message will be sent just to
that session specified.

This closes #1649.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-21 08:21:56 +09:00
ShellyLC 816034bd01 NIFI-3543 This closes #1548. Added support for using EL when defining brokerURL in jms-cf-service
Signed-off-by: joewitt <joewitt@apache.org>
2017-04-19 22:50:48 -07:00
Koji Kawamura 8ce2a1b3a7 NIFI-3657 This closes #1634. Fix HTTP S2S to use local address.
- Fixed SiteInfoProvider and HttpClient to use specified local address with its SiteToSiteRestApiClient
- Removed setupRequestConfig method call from connection and read timeout setter methods at SiteToSiteRestApiClient, because it created config object before local address was set
- Null clear StandardRemoteProcessGroup localAddress when user clears Local Network Interface

Signed-off-by: joewitt <joewitt@apache.org>
2017-04-19 22:19:41 -07:00
Koji Kawamura 0ddb90cd2d NIFI-3166 This closes #1324. Fix SocketRemoteSiteListener NPE.
- Refactored ServerProtocol.sendPeerList method signature to clarify the
  meaning of arguments, and avoid null pointer exception when converting null Integer to int.
- Refactored SocketRemoteSiteListener handleRequest method to make it
  more unit test friendly.
- Added more unit tests.
2017-04-19 22:18:43 -07:00
Mark Payne 0f2ac39f69 NIFI-3273 This closes #1611. Handle the case of trailing NUL bytes in MinimalLockingWriteAheadLog 2017-04-19 22:08:59 -07:00
Mark Payne 0207f21ce4 NIFI-3273: Added nifi-toolkit-flowfile-repo that contains a simple Java class that is capable of recovering a FlowFile Repository manually in the case of an operating system crash that results in trailing 0's being dumped into the edit logs. Also refactored flowfile repo into some independent modules so that it additional capabilities can be added in the future to examine the flowfile repo 2017-04-19 21:44:23 -07:00
Matt Burgess 141334c3c9 NIFI-3718: Fixed TestAvroRecordReader to handle timezone differences
This closes #1683
2017-04-19 17:04:45 -04:00
Mark Payne b93cf7bbdb NIFI-3658: Created processor for converting records between data formats with like schemas
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-3658: Incorporated PR review feedback; added counter; clarified documentation

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1668
2017-04-19 16:12:14 -04:00
Matt Gilman 7e7f9a5deb NIFI-3705:
- Using consistent logic when verifying connection creation. Removed some unecessary checks as verification has been changed to run in cluster and standalone mode.

This closes #1672.
2017-04-19 15:49:11 -04:00
ijokarumawak 220a870a1e NIFI-2697: Fix opening Remote Ports dialog.
If there is an unconnected remote port, right clicking a Remote Process Group
then select 'Remote Ports' causes a JS error saying 'Uncaught TypeError'.
Because there are few code paths those don't return the created
transmissionSwitch value from createPortOption function in
nf-remote-process-group-ports.js.

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1679
2017-04-18 16:08:10 -04:00
joewitt 189b4ad63d NIFI-3694 removed the total cap size value as per review feedback 2017-04-18 14:07:38 -04:00
Andre F de Miranda cd260bab9f
NIFI-2747 - Introduce FuzzyHashContent processor
This closes #1619.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-04-18 10:27:15 -04:00
Matt Gilman 7e1ecad738
NIFI-3706: - Removing non-existent resource when converting a legacy authorized users file.
This closes #1673.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-04-18 09:46:21 -04:00
joewitt d08233b9fe NIFI-3694 updated latest logback,jetty,moved servlet-api and jetty-schemas to lib root as needed now
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1678
2017-04-18 07:41:21 -04:00
Matt Burgess edb9c644dd NIFI-3413: Standardize 'write' event to 'insert'
This closes #1675.

Signed-off-by: ijokarumawak <ijokarumawak@apache.org>
2017-04-18 00:55:17 +09:00
Yolanda M. Davis 5f73ba7f2c NIFI-3708 - removed typo from character escape regex
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1674
2017-04-14 22:13:09 -04:00
Scott Aslan deed25656f
[NIFI-3035] use URLSearchParams .toString() to update URL. This closes #1659 2017-04-13 14:32:30 -04:00
Oleg Zhurakousky 68c592ea43 NIFI-1280 added support for RecordSchema in SchemaRegistry
Signed-off-by: Mark Payne <markap14@hotmail.com>
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-1280: Updated SimpleKeyValueSchemaRegistry to make use of new CHOICE RecordFieldType - Update Record Readers to use SchemaRegistry controller service. Moved SchemaRegistry api into its own maven module and added to standard-services-api so that we can properly add dependencies on it. Code cleanup and bug fixes

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-1280: Fixed checkstyle violations and license exclusions for RAT plugin

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-1280: Addressed feedback from PR Review

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-1280: Additional changes/doc to support QueryFlowFile and Record Readers/Writers

This closes #1652
2017-04-11 19:29:35 -04:00
Mark Payne a88d3bfa3c NIFI-1280: Refactoring to make more generic so that other data types can be supported; created InputStreams to content on-demand so that multiple passes can be made over FlowFile content if required. Created new Controller Services for reading and writing specific data types
Signed-off-by: Matt Burgess <mattyb149@apache.org>
2017-04-11 19:29:04 -04:00
Toivo Adams 4d5872a385 NIFI-1280 Create FilterCSVColumns Processor.
Signed-off-by: Matt Burgess <mattyb149@apache.org>
2017-04-11 19:29:04 -04:00
joewitt 52cf9a7953 NIFI-3690 Updated nifi-gcp-nar L&N, cat-x deps
Updated L&N in nifi-gcp-nar and nifi-assembly and fixed nifi-gcp-processors
to avoid jsr305 and json.org cat-x dependencies

Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1665.
2017-04-11 15:56:25 -07:00
Joey Frazee 4a8398bb31 NIFI-3692 This closes #1663. Use hbc-core instead of hbc-twitter4j in social-media bundle to resolve org.json cat-x
Signed-off-by: joewitt <joewitt@apache.org>
2017-04-11 14:25:04 -04:00
Koji Kawamura 7c19796eb2 NIFI-3561: Fix WS port forwarding
Fixed service instance lookup to use local port so that it works when
it is accessed through port forwarding.

This fixes: #1568

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-04-11 21:27:53 +10:00
Pierre Villard 776a00b0be NIFI-2860 Use different properties for JVM Heap Max and Min values
This closes: #1382

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-04-11 21:18:49 +10:00
Joe Percivall 5419891fe7 NIFI-3633 This closes #1610. Adding HttpNotificationService.
Signed-off-by: joewitt <joewitt@apache.org>
2017-04-11 00:37:12 -04:00