- Added detailed description about how the URL property works with
GetHTMLElement
- Added Expression support with URL
- Made URL property dynamic with ModifyHTMLElement and PutHTMLElement,
since it won't be used to alter HTML element and need not to be
specified. Making it a dynamic property let existing processor configuration stays valid
* add exec to RUN_NIFI_CMD
* remove subshell for else
* tested compatible with runit with these changes
This closes#966.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
* Corrected handling of corrupt journal file records that prevents instance startup and loss of records from corrupt files. Specifically, exception handling was expanded to cover failures on records after the first the same as failures on the first record.
* Adjusted log messages to reflect that the remainder or all of the journal will be skipped, not just the current record.
This closes#1485.
H2 and Kafka broker uses the same default port 9092.
If an user is running Kafka broker on the same machine, or run the unit
tests in parallel, DBCPServiceTest can fail since some of its test
methods connects to port 9092.
This closes#1504.
Signed-off-by: Andy LoPresto <alopresto@apache.org>
* Remove function based on JDK source.
* Add new function to find bytes based on RFC3629.
* Add field name to log entry when field is truncated.
Signed-off-by: Mike Moser <mosermw@apache.org>
This closes#1475
* Credentials service with tests
* Abstract processor definitions
* GCS-themed processors and their corresponding tests
Signed-off-by: James Wing <jvwing@gmail.com>
This closes#1482.
use the FileNameFilter when not passing down explit jar paths
Filter out ^. files when reading lists of files from directories
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
- Added configure audits for Transport Protocol, HTTP Proxy Server Host,
Port, User and Password in RemoteProcessGroup configuration
- Added configure audits for enabling/disabling individual remote port
- Added configure audits for Concurrent Tasks and Compressed in Remote
Port configuration
- This closes#1476
* Updated StandardRecordWriter, even though it is now deprecated to consider the encoding behavior of java.io.DataOutputStream.writeUTF() and truncate string values such that the UTF representation will not be longer than that DataOutputStream's 64K UTF format limit.
* Updated the new SchemaRecordWriter class to similarly truncate long Strings that will be written as UTF.
* Add tests to confirm handling of large UTF strings and various edge conditions of UTF string handling.
Signed-off-by: Mike Moser <mosermw@apache.org>
This closes#1469.
- Marked PutKafka Partition Strategy property as deprecated, as Kafka 0.8 client doesn't use 'partitioner.class' as producer property, we don't have to specify it.
- Changed Partition Strategy property from a required one to a dynamic property, so that existing processor config can stay in valid state.
- Fixed partition property to work.
- Route a flow file if it failed to be published due to invalid partition.
This closes#1425
NIFI-2615 Addressing changes from P/R. Specifically, removing .gitignore as it should not be there for a nar. Removed non-used class. Changed name in notice
- Requiring WRITE permissions to the parent resource when attempting to remove a component.
- Updating expired certificates in the REST API integration tests.
This closes#1399.
Signed-off-by: James Wing <jvwing@gmail.com>
- Support counters at Wait/Notify processors so that NiFi flow can be
configured to wait for N signals
- Extract Wait/Notify logics into WaitNotifyProtocol
- Added FragmentAttributes to manage commonly used fragment attributes
- Changed existing split processors to set 'fragment.identifier' and
'fragment.count', so that Wait can use those to wait for all splits
get processed
This closes#1420.
Signed-off-by: Bryan Bende <bbende@apache.org>
Add support in SelectHiveQL to get script content from the Flow File to bring consistency with patterns used for PutHiveQL and support extra query management.
Changed behavior of using Flowfile to match ExecuteSQL. Handle query delimiter when embedded. Added test case for embedded delimiter
Formatting and License Header
PutHiveQL and SelectHiveQL Processor enhancements. Added support for multiple statements in a script. Options for delimiters, quotes, escaping, include header and alternate header.
Add support in SelectHiveQL to get script content from the Flow File to bring consistency with patterns used for PutHiveQL and support extra query management.
Changed behavior of using Flowfile to match ExecuteSQL. Handle query delimiter when embedded. Added test case for embedded delimiter
Removing dead code.
Signed-off-by: Matt Burgess <mattyb149@apache.org>
Comments to Clarify test case.
Signed-off-by: Matt Burgess <mattyb149@apache.org>
Final whitespace/formatting/typo changes
This closes#1316
- Adding additional parameters to be able to limit the size of the provenance response. Specifically, whether the events should be summarized and whether events should be returned incrementally before the query has completed.
- Ensuring the cluster node address is included in provenance events returned.
- Ensuring there is a cluster coordinator before attempting to get the cluster node address.
- Removing exponential back off between provenance requests.
- Ensuring the content viewer url is retrieve before initializing the provenance table.
This closes#1413.
- Using fetch and replace together can provide optimistic locking for
concurrency control.
- Added fetch to get cache entry with its meta data such as revision
number.
- Added replace to update cache only if it has not been updated.
- Added Map Cache protocol version 2 for those new operations.
- Existing operations such as get or put can work with protocol version
1.
This closes#1410.
Signed-off-by: Bryan Bende <bbende@apache.org>
NIFI-3004 Added logic to expire StandardSSLContextService customValidate cache after 5 invocations.
Updated unit test to demonstrate this logic.
This closes#1375.
Signed-off-by: Andy LoPresto <alopresto@apache.org>
- Removing unnecessary authorization check during second phase of connection creation.
- Ensuring that the remote group port returns the correct resource type though not super critical since it is not possible to create policies for remote ports.
This closes#1353.
- Fixing the importUri when replicating a Template upload request. Previously was leveraging getResourceUri which considers proxy headers meant for client responses.
This closes#1334.
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
- Removed host and port field from Peer since the same information is
available in PeerDescription
- Refactored variable names in SocketRemoteSiteListener to improve readability
- Changed how SocketRemoteSiteListener constructs PeerDescription
instance. It used to use hard-coded 'localhost' as hostname, and
getPort() which returns server's port. Since the peer is a remote peer,
i.e the client, it should be client hostname and port.
- Added hostname resolution at DataTransferResource to make s2s.host
value consistent with RAW transport. Without this, RAW uses hostname
while HTTP uses IP address. It will be hard to be used from downstream flows.
- Replaced heavy use of mockito which was difficult to maintain, with
nifi-mock
- Added SiteToSiteAttributes and more assertions in unit tests
This closes#1342.
Signed-off-by: Bryan Bende <bbende@apache.org>
- Ensure that controller services are added to child process groups when creating snippet
- Addressed issue related to modifying higher-level process groups' controller services in snippet after having already visited the process group
This closes#1318
Signed-off-by: jpercivall <JPercivall@apache.org>
NIFI-1582 removing the option to use cluster state
NIFI-1582 addressing Oleg's comments
NIFI-1582 No longer forcing numbers as the init value and adding getStateValue() to EL instead of using 'ATTRIBUTE_state'
NIFI-1582 Removing init state value
NIFI-1582 Adding documentation for the changes to Init State value
This closes#319
NIFI-3133:
- Using getSourceAuthorizable() when accessing flow files and content.
NIFI-3133:
- Decouple local and remote connectable's to avoid ambiguity with self referencing RPGs.
NIFI-3133:
- Addressing comments from the PR.
NIFI-3133:
- Fixed check verifying source/destination when creating a connection.
NIFI-3133:
- Only showing the go to link when the source component is not a remote port.
NIFI-3133:
- Removing unnecessary checking of remote group port authorization since it's handled by the parent RPG.
NIFI-3133:
- Fixing issue showing the connection details dialog when the source component is a RPG.
NIFI-3133:
- Ensuring the local connectable was found.
This closes#1297
Signed-off-by: jpercivall <JPercivall@apache.org>
- Added urls in addition to the existing url, to support multiple target
URLs
- Backward compatibility is provided by returning the first url if
multipe urls are specified, but component accessing the url doesn't
support multiple urls
- UI is not fully updated yet. Following UI components are planned to be updated
by different commits
- Search component: only the first URL is searchable and shown
- Component status: RPG status shows only the first URL
- Component action history: only the first URL is searchable and shown
- Updated Search component to use URLs.
This closes#1208.
Pierre Villard
Andy LoPresto
Koji Kawamura
JJ
patricker
Bryan Rosander
Andre F de Miranda
Hunter Morgan
Simon Elliston Ball
Matt Burgess
Mike Moser
Joe Skora
Selim Namsi
joewitt
Yolanda M. Davis
Scott Aslan
Gene Peters
Otto Fowler
Matt Gilman
Ryan Persaud
Nick Carenza
Oleg Zhurakousky
Giovanni Lanzani
Joe Gresock
James Wing
Ben Schofield
Jeff Storck
Andrew Psaltis
David W. Streever
Andrew Lim
afucs
Joey Frazee
jpercivall
Davy De Waele
Bryan Bende
Randy Gelhausen
Wesley-Lawrence
Mark Payne
Byunghwa Yun
jpercivall