1e5cc070a3
NIFI-1081 Adding option to ExecuteStreamCommand to put output value to an attribute
...
Reviewed and amended (comments,whitespace,and some code readability (discussed in ticket)) by Tony Kurc (tkurc@apache.org )
2015-11-18 23:23:10 -05:00
9e2f6df205
NIFI-1123: Fixing a botched commit.
2015-11-18 21:59:42 -05:00
52b24b93d9
NIFI-1123 Adds expression language support to DeleteAttributesExpression on UpdateAttributes Processor.
...
Reviewed by Tony Kurc (trkurc@gmail.com ) after Aldrin Piri <aldrin@apache.org> did the initial review and actionable comments
2015-11-18 19:10:21 -05:00
ab7940368a
NIFI-1187: Fixing issue of possible assigment reordering causing uninitalized values to be possibly returned
...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-11-18 17:37:47 -05:00
c541c82c35
NIFI-1191 Adding missing tags for ConvertAvroToJSON
2015-11-18 16:38:15 -05:00
69bce2c2db
NIFI-1168: Ensure that processors with only looping
...
connections are scheduled to run, even if the connections have no FlowFiles;
expose these details to processor developers; update documentation
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-11-18 14:53:30 -05:00
773576e041
NIFI-1108 Providing additional annotations on processors to ensure utilization of the InputRequirement annotation.
2015-11-18 13:56:21 -05:00
911e1c5412
NIFI-1108: Updated processors to include the @InputRequirement annotation
...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-11-18 13:47:59 -05:00
d88b6cb6bc
NIFI-1173: Even if FlowFile Queue is empty, it needs to hold onto the Empty Queue request so that subsequent cancel/clear requests can reference it
...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-11-18 10:56:38 -05:00
180ea1ba22
NIFI-1176: Use a smaller internal blocking queue size of only 5000 messages; do not create a new queue every time the processor is scheduled to run
...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-11-18 10:51:22 -05:00
93be753301
NIFI-1181: Ensure that a FlowFile's uuid cannot be modified by processors
...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-11-18 10:35:37 -05:00
2b9b5e008f
NIFI-1175 Exposing minimum properties required to create an HBase connection on the HBaseClientService as an optional alternative to the conf files
2015-11-17 12:01:46 -05:00
453b140d6b
Merge branch 'NIFI-748' of https://github.com/olegz/nifi into NIFI-748
2015-11-17 09:23:10 -05:00
03a54bf2d5
Changes after review
2015-11-16 13:32:17 -06:00
b954ca620e
NIFI-1146 Allow GetKafka to be configured with auto.offset.reset to largest or smallest
2015-11-16 11:59:52 -06:00
15880f9fcc
NIFI-748 addressed PR comments
...
- made DocReader package private
- polished logic in read(..) method to avoid escaping the loop
- added call to sorting logic in LuceneUtil.groupDocsByStorageFileName(..) to ensure that previous behavior and assumptions in read(..) methodd are preserved
- other minor polishing
2015-11-16 08:39:23 -05:00
64369f67f9
NIFI-1160:
...
- Addressing possible NPE when removing a drop request.
- Only updating a connection status once it's been retrieved for the first time.
2015-11-16 08:34:04 -05:00
da9f40b3f0
NIFI-973: Updated main Help page to provide a link to the Getting Started Guide.
...
Reviewed by Aldrin Piri (aldrin@apache.org )
2015-11-15 08:38:51 -05:00
528dab78d6
NIFI-1073: Fixing coverity discovered errors. Resource leaks, and statics
...
Reviewed by Bryan Bende (bbende@apache.org )
2015-11-14 18:56:43 -05:00
01539ed323
NIFI-1163: Providing handling of SSLContext creation in GetHTTP in case of only performing a one-way SSL request and accompanying test to verify the configuration/usage.
...
Reviewed by Tony Kurc (tkurc@apache.org )
2015-11-14 18:47:50 -05:00
f8d26d2059
Merge branch 'master' into NIFI-1156
2015-11-13 15:02:23 -05:00
c82fc18f8e
HTML Parsing Processors Bundle
...
NIFI-1156 HTML Parsing Processors Bundle
2015-11-13 15:01:10 -05:00
a4d93c62c8
NIFI-748 Fixed logic around handling partial query results from provenance repository
...
- Ensured that failures derived form correlating Document to its actual provenance event do fail the entire query and produce partial results with warning messages
- Refactored DocsReader.read() operation.
- Added test to validate two conditions where the such failures could occur
2015-11-13 14:23:31 -05:00
90f6830003
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi
2015-11-13 10:47:05 -05:00
22de23baa6
NIFI-1097: Rewrite PutKafka to use the new producer api
2015-11-13 10:46:46 -05:00
36d00a60f5
NIFI-1155 fixed contrib-check violation
2015-11-13 00:06:12 -05:00
37d6b7350e
NIFI-1155: Refactored StandardFlowFileQueue to update member variables more intelligently, using CAS operations instead of locks. This reduces code complexities because other optimizations that previously existed are no longer needed
...
Signed-off-by: joewitt <joewitt@apache.org>
2015-11-13 00:06:12 -05:00
3ed0949c55
NIFI-1155: Ensure that when poll(FlowFileFilter, Set) is called, we properly update the indicator for whether or not queue is full
...
Signed-off-by: joewitt <joewitt@apache.org>
2015-11-13 00:06:12 -05:00
428b20fc25
NIFI-1103: Add support for long polling in GetSQS processor
2015-11-12 22:03:36 -07:00
e6086420aa
NIFI-1158 Default timestamp based on now
2015-11-12 23:11:43 -05:00
0900fb80c9
NIFI-1153: If no incoming FlowFile, don't try to transfer null
...
Reviewed by Bryan Bende (bbende@apache.org )
Amended based on review (change to a log message) by Tony Kurc (tkurc@apache.org )
2015-11-12 22:41:57 -05:00
33ef59c5ba
NIFI-1127 Adding Kerberos properties to FetchHDFS and ListHDFS.
...
Reviewed by Tony Kurc (tkurc@apache.org )
2015-11-12 22:14:12 -05:00
65bd8c0b1f
NIFI-1109 Updated documentation and cleaned up code
...
Reviewed by Bryan Bende (bbende@apache.org ).
Committed with amendments (for whitespace, a couple misspellings and to change a test method name based on review) by Tony Kurc (tkurc@apache.org )
2015-11-12 21:07:01 -05:00
02102ea1c2
Revert "NIFI-1134". This was removed because they looked like floating classes. But in reality they are used to make jars which we use to test the build. However, we really need the test to work differently but until that is fixed these must stay.
...
This reverts commit 41f3875347
2015-11-12 18:00:40 -05:00
56ad22aea6
NIFI-900: Created Processors for interacting with Microsoft Azure EventHubs
...
Reviewed (with amendments needed for clean merge, whitespace and NOTICEs) by Tony Kurc (tkurc@apache.org )
2015-11-11 23:06:04 -05:00
7a165b62cc
Merge branch 'NIFI-994'
2015-11-11 16:05:40 -05:00
854d203982
Merge branch 'NIFI-1143' of https://github.com/olegz/nifi into NIFI-1143
2015-11-11 14:14:54 -05:00
5baafa156a
NIFI-1143 Fixed race condition which caused intermittent failures
...
Fixed the order of service state check in PropertyDescriptor
Encapsulated the check into private method for readability
Modified and documented test to validate correct behavior.
For more details please see comment in https://issues.apache.org/jira/browse/NIFI-1143
2015-11-11 14:06:08 -05:00
11fcad90d1
Merge branch 'NIFI-1083'
2015-11-11 12:07:15 -05:00
054774fe7b
Merge branch 'NIFI-1061' of https://github.com/olegz/nifi into NIFI-1061
2015-11-11 11:17:25 -05:00
e7c6c7cae9
NIFI-1083: Added a Grouping Regular Expression property for grouping lines of text
2015-11-11 10:19:07 -05:00
a65a29921a
NIFI-1124, NIFI-1062 addressed more PR comments
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-11-11 10:12:30 -05:00
70cc61bb53
NIFI-1124, NIFI-1062 addressed PR comments
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-11-11 10:12:30 -05:00
62e3cfc629
NIFI-1024 NIFI-1062 Fixed PutHDFS processor to properly route failures. Ensured that during put failures the FlowFile is routed to 'failure' relationship. Added validation test Re-enabled previously ignored test.
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-11-11 10:12:30 -05:00
3e538d9007
NIFI-1083 Created a processor that routes lines of text based on different matching and routing strategies
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-11 09:12:47 -05:00
76690a8ee9
NIFI-1061 fixed deadlock caused by DBCPConnectionPool.onConfigured()
...
Current implementation of DBCPConnectionPool was attempting to test if connection could be obtained via dataSource.getConnection().
Such call is naturally a blocking call and the duration of the block is dependent on driver implementation. Some drivers (e.g., Phoenix - https://phoenix.apache.org/installation.html )
attempts numerous retries before failing creating a deadlock when attempt was made to disable DBCPConnectionPool which was still being enabled.
This fix removes the connection test from DBCPConnectionPool.onConfigured() operation returning successfully upon creation of DataSource.
For more details see comments in https://issues.apache.org/jira/browse/NIFI-1061
2015-11-11 08:33:51 -05:00
6c510fae80
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi
2015-11-11 08:16:38 -05:00
d887d8fdf0
NIFI-993 added expression language support to GetHTTP for the filename and URL properties
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-11 08:05:23 -05:00
ba5b1d837c
NIFI-1117 Changed line endings to unix line endings in java source
2015-11-10 18:50:57 -05:00
9756e9f126
NIFI-1117 Added unit tests and default behavior for avro files with no records. Also fixed a findbugs warning in SplitAvro (made inner classes static)
2015-11-10 18:46:17 -05:00
e6644b3747
Merge branch 'NIFI-1141'
2015-11-10 17:17:55 -05:00
7b9c8df6c5
NIFI-994: Fixed issue that could result in data duplication if more than 1 rollover of tailed file has occurred on restart of Processor
2015-11-10 16:53:29 -05:00
6ed5f28f26
NIFI-965 penalized failed flowfiles in S3 Processors. Also discovered a typo in a static final that I corrected
...
Signed-off-by: Mike Moser <mosermw@apache.org>
2015-11-10 15:14:28 -05:00
72008035b1
Merge branch 'NIFI-1000'
2015-11-10 15:02:02 -05:00
8f942fcdcc
NIFI-1000: Fixed checkstyle violation in TestGetJSMQueue use activemq-broker with test scope and activemq-client with default scope rather than pulling in activemq-all; this reduces size of standard nar from 30 MB to 18 MB
2015-11-10 14:31:20 -05:00
53725b5c72
Merge branch 'NIFI-1000' of https://github.com/olegz/nifi into NIFI-1000
2015-11-10 13:34:56 -05:00
8699e35108
NIFI-1000 addressed PR comments
2015-11-10 12:13:16 -05:00
11768cc388
NIFI-1141: Fixed the order of the arguments to the setCurrentContentClaim method
2015-11-10 11:35:14 -05:00
e748fd5848
NIFI-817 Processors for interacting with HBase
...
- Refactoring PutHBaseCell to batch Puts by table
- Adding optional Columns property to GetHBase to return only selected column families or columns
- Making GetHBase cluster friendly by storing state in the distributed cache and a local file
- Adding Initial Time Range property to GetHBase
- Adding Filter Expression property and custom validate to prevent using columns and a filter at the same time
- Creating an HBaseClientService controller service to isolate the HBase client and support multiple versions
- Creating appropriate LICENSE/NOTICE files
- Adding @InputRequirement to processors
- Addressing comments from review, moving hbase client services under standard services
- Making sure result of session.penalize() is assinged to FlowFile variable before transferring
2015-11-10 11:13:51 -05:00
2a90bd501b
NIFI-797 Corrected URL for PutS3 to include correct protocol and endpoint by region.
...
- corrected a missed 'final' on org.apache.nifi.processors.aws.AbstractAWSProcessor.relationships
- added protected method org.apache.nifi.processors.aws.AbstractAWSProcessor.getRegion()
- added protected method org.apache.nifi.processors.aws.s3.AbstractS3Processor.getUrlForObject(String, String)
- explicitly set AWS client protocol to HTTPS, and created a static final field with comments if other protocols may be considered
- added a static final field for the UserAgent
Reviewed by Aldrin Piri <aldrin@apache.org>
2015-11-10 06:37:56 -05:00
ef0be5a5d6
NIFI-1000 Fixed JmsFactory to properly obtain destiniation name
...
Re-enabled JMS Tests that were annotated with @Ignore
2015-11-09 18:35:31 -05:00
b75af5b344
Merge branch 'NIFI-1085'
2015-11-09 16:13:12 -05:00
41f3875347
NIFI-1134
2015-11-09 15:37:40 -05:00
73c1671975
NIFI-1132: Limited number of Lineage Identifiers held to 100 and marked the getLineageIdentifiers() method as deprecated
2015-11-09 12:09:56 -05:00
5f8fdae909
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi
2015-11-09 12:07:45 -05:00
64b1fc983c
NIFI-925 reviewed patch/ builds well/ fixed licensing gaps.
2015-11-08 22:15:11 -05:00
59a49aea12
Revert "NIFI-1082: Ensure that events returned from the provenance repository are ordered such that newest events are provided first"
...
This reverts commit cf8ca3dc2c
2015-11-08 21:46:06 -05:00
440e4c4940
NIFI-1106 fixed platform-specific object separator.
2015-11-08 12:27:41 -05:00
77de51df19
NIFI-1025 Updated aws-sdk and joda-time to resolve AWS issues with Java 1.8 update 60.
...
- Refactored tests - created AbstractS3Test for common utility methods
- Corrected incorrect unit test in TestDeleteS3Object, and adjusted processor documentation to reflect behavior
- moved aws dependency management to root pom
This closes #107
Tested, Reviewed and Amended by Tony Kurc (<tkurc@apache.org>)
2015-11-07 21:09:18 -05:00
99629646fe
NIFI-1122 moved to 0.4.0-SNAPSHOT
2015-11-06 23:41:15 -05:00
0d6e81b54f
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi
2015-11-06 09:16:12 -05:00
ee87f33ce7
Fixing check-style violation in StandardFlowFileQueue
2015-11-06 09:10:10 -05:00
1a388f957e
NIFI-1111: We were subtracting values in the wrong order so that we always would 'yield' for a negative amount of time, which equated to not yielding at all.
2015-11-06 09:03:38 -05:00
98f5a1ab77
NIFI-1110: Fixed bug that caused queue size to become negative when FlowFiles are expired
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-05 12:12:02 -05:00
4ac52bfc0a
Removing nifi-azure-bundle that was incorrectly added to master
2015-11-05 10:42:10 -05:00
f1bd3d471d
NIFI-1112 Fixed APIOperation Annotation on ReportingTaskResource.
...
Reviewed by Tony Kurc (tkurc@apache.org )
2015-11-04 18:31:34 -05:00
618f22e110
NIFI-274 - Fixing TestListenSyslog, fixing default buffer size to be bytes, adding syslog.protocol to attributes
...
- Adding syslog.port to ListenSyslog attributes, logging at warn level when rejecting tcp connections
- Adding @InputRequirement to processors and adding appropriate send and receive provenance events
2015-11-04 18:01:59 -05:00
5611dac3f8
NIFI-274 - added use of Selectors for TCP and UDP connections. Added a max connections to the TCP thread
...
- Added comments and code review changes
- fixed fixbugs bug
2015-11-04 18:00:18 -05:00
9c542432da
NIFI-274 Adding ListenSyslog and PutSyslog to standard processors.
...
- Refactoring connection handling on put side, removing number of buffers from properties and basing it off concurrent tasks for the processor.
- Refactoring some of the TCP handling so it keeps reading from a connection until the client closes it
- Adding an error queue
- Adding a sender field on the syslog event to record the system that sent the message
2015-11-04 17:56:13 -05:00
6add372bc1
NIFI-1085 WebClusterManager fair locking and HeartbeatMonitoringTimerTask scheduling
2015-11-04 12:19:00 -05:00
201eac052b
NIFI-1098 - Ensuring we pull the correct attribute from the servlet context. Reviewed by Tony Kurc (tkurc@apache.org)
2015-11-04 12:11:19 -05:00
fedbe40313
NIFI-1103: Added InputRequirement annotation
2015-11-04 11:11:33 -05:00
2b1d093120
NIFI-1105: Only trigger a processor that requires input to run if data is available for it process
2015-11-04 08:41:10 -05:00
dbf0c7893f
NIFI-730: Completed merging of master
2015-11-02 14:31:28 -05:00
e4cebba3c7
Merge branch 'master' into NIFI-730
2015-11-02 14:02:16 -05:00
a549621267
NIFI-1077 enabling expression language on ConvertCharacterSet input and output
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-02 11:46:17 -05:00
814e8b212c
Merge branch 'NIFI-1082'
2015-11-02 11:29:52 -05:00
abf2c62884
NIFI-1051: Fixed checkstyle violations
2015-11-02 11:01:50 -05:00
7a73867c46
Merge branch 'NIFI-1051' of https://github.com/olegz/nifi into NIFI-1051
2015-11-02 10:51:43 -05:00
2ae49026e8
NIFI-883 Fixing issue HandleHttpRequest had with PrimaryNodeOnly scheduling
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-02 10:32:14 -05:00
5a04021dd7
NIFI-730:
...
- Updating logic now that original is guaranteed to be non null.
- Always reporting 100% once the drop request has completed.
2015-11-02 08:59:11 -05:00
f5727cfb0f
NIFI-730: Ensure that we always populate queue counts when initiating a Drop FlowFile request
2015-11-02 08:36:50 -05:00
ad849c77df
NIFI-1090: Fixed log message that was at info level but should have been debug level
2015-11-01 14:37:01 -05:00
b729bf4c19
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi
2015-11-01 14:16:54 -05:00
6e193dffa3
Merge branch 'NIFI-1088'
2015-11-01 14:16:35 -05:00
cef7b6c736
NIFI-1080 fleshed out InvokeHttp unit tests
...
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-10-31 00:19:36 -04:00
9c148da1c2
NIFI-869: Fixed checkstyle violations
2015-10-30 16:09:50 -04:00
ba72452f66
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi
2015-10-30 16:02:15 -04:00
a3d43d23dc
NIFI-869 Fixed SimpleProcessLogger to log correct messages Ensured that SimpleProcessLogger correctly interprets Throwable as discussed in JIRA Added tests
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-30 15:52:33 -04:00
90aea01350
NIFI-869 Fixed formatting issue Fixed formatting issue with printed error message which only apears when NiFi is cnfigured using Logback. Please see NIFI-869 for more details
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-30 15:52:33 -04:00
eb389cf84c
NIFI-1079 Replacing Name for NULL_VALUE_FOR_EMPTY_STRING as I had accidentally removed it.
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-10-30 14:30:56 -04:00
aef0d8fe9b
NIFI-1079 Check style and documentation based updates recommended by user bbende on Github pull request
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-10-30 14:30:42 -04:00
217b1049cf
NIFI-1079 Added Destination Property to control if JSON goes to Attribute or
...
Content of FlowFile. Added Include Core Attributes Property to control
if FlowFile CoreAttributes are included in the JSON output or not.
Added Null value for Empty String flag to control if empty values in
the JSON are empty string or true NULL values. Added more tests and
minor text refactoring per Github comments
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-10-30 14:30:25 -04:00
19b7a4cc7d
NIFI-1079 Create a new Attribute from the existing FlowFile attributes by taking
...
either all of the existing attributes or a user defined list. The
existing Attributes are converted to JSON and placed in a new Attribute
on the existing FlowFile as Attribute “JSONAttributes”
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-10-30 14:29:29 -04:00
9515b74607
NIFI-1088: Ensure that FlowFile is penalized before routing to failure
2015-10-30 14:25:27 -04:00
bfa9e45079
NIFI-994: Ensure that processor is not valid due to the tail file not yet existing
2015-10-30 11:44:51 -04:00
5c4042bd7c
NIFI-1051 Allowed FileSystemRepository to skip un-readable entries.
...
The exception was caused due to basic file permissions. This fix overrides
'visitFileFailed' method of SimpleFileVisitor to log WARN message and allow
FileSystemRepository to continue.
2015-10-29 16:31:17 -04:00
5cc2b04b91
NIFI-986 Refactoring of action classes from nifi-user-actions to have interfaces in nifi-api, and adding getFlowChanges to EventAccess
...
- Fixing empty java docs and adding sort by id asc to the history query
- Changing userDn to userIdentity in Action and FlowChangeAction
- Modifying NiFiAuditor to always save events locally, and implementing getFlowChanges for ClusteredEventAccess
2015-10-29 16:28:36 -04:00
c4f0cb1c6c
Removing nifi-pcap-bundle/.gitignore
2015-10-29 12:06:25 -04:00
cf8ca3dc2c
NIFI-1082: Ensure that events returned from the provenance repository are ordered such that newest events are provided first
2015-10-28 17:32:51 -04:00
edf238e004
NIFI-730: Do not require a Read Lock in order to obtain backpressure configuration values for FlowFileQueue's
2015-10-28 10:27:58 -04:00
dc4004de64
Merge branch 'NIFI-977'
2015-10-27 16:53:38 -04:00
af19053a7f
Merge branch 'NIFI-447'
2015-10-27 10:42:46 -04:00
07238c7913
NIFI-447 adding replacement strategy to ReplaceText processor
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-27 10:37:43 -04:00
b885f955f4
NIFI-516 adding option to StandardProcessSession.read to close stream
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-26 20:23:13 -04:00
1c1738670c
NIFI-10: Updated FetchHDFS, FetchFileTransfer to use new FETCH provenance event
2015-10-26 17:14:29 -04:00
fc2aa2764c
NIFI-10: Added FETCH and DOWNLOAD Provenance Events; updated FlowController to use DOWNLOAD event instead of SEND whenever a user downloads/views content via Provenance Event
2015-10-26 14:58:50 -04:00
51f564024a
NIFI-10: Added FETCH Provenance Event and updated processors to use this new event type
2015-10-26 14:58:50 -04:00
aec32a277c
NIFI-1070: Added detailed debug-level logging about how FileSystemRepository is choosing to expire archived data
2015-10-26 14:36:03 -04:00
f8c3377c84
NIFI-820:
...
- Creating a context menu item for accessing provenance directly for a selected component.
2015-10-26 10:16:01 -04:00
26edab3185
NIFI-908 Added support for SSL in JMS connections.
...
- Added SSL context to JMS producer and consumer processors
- Tony Kurc Amended patch to check SSL need by scheme and exception consistency
Reviewed by Tony Kurc (tkurc@apache.org )
2015-10-26 00:29:05 -04:00
8d2f9bc64b
NIFI-1056: Do not treat BigDecimal, BigInteger the same way we treat Number's when serializing to Avro because Avro Java Library doesn't support them
...
Reviewed and Unit Test Amended by Tony Kurc (tkurc@apache.org )
2015-10-25 11:31:30 -04:00
385bfbb2c6
NIFI-673: Rebased from master; Added InputRequirement annotation, as it is now merged into master
2015-10-25 11:14:11 -04:00
4e38288062
NIFI-673: Added sftp.listing.user attribute to FlowFiles created by ListSFTP; ensure that FetchSFTP indicates that the username supports Expression Language
2015-10-25 11:13:02 -04:00
b0322d9ffe
NIFI-673: Added Completion Strategy to FetchSFTP
2015-10-25 11:13:02 -04:00
d1d57931bf
NIFI-673: Initial implementation of ListSFTP, FetchSFTP
2015-10-25 11:13:02 -04:00
8a80060851
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi
2015-10-25 11:11:49 -04:00
0636f0e731
NIFI-810: Merged master into branch
2015-10-25 11:02:40 -04:00
f2c4f2d2a1
[PATCH] NIFI-997 Periodically attempt a kerberos relogin in AbstractHadoopProcessor
...
- attempt a relogin based on an interval specified in the processor configuration
- use hadoop's UserGroupInformation.checkTGTAndReloginFromKeytab to determine if a relogin is necessary based on the ticket and do so if needed
- improve code readability with HdfsResources object in AbstractHadoopProcessor
Reviewed and Amended by Tony Kurc (tkurc@apache.org ). This closes #97
2015-10-25 00:49:58 -04:00
d63cd6bd2f
Merge branch 'NIFI-1010'
2015-10-23 14:39:16 -04:00
88fc8d28a0
NIFI-1010: If database driver does not support getting table name for column from ResultSetMetadata, then just use a default name
2015-10-23 10:25:59 -04:00
0fc5d30461
NIFI-1055: Fixed checkstyle violations
2015-10-23 10:08:44 -04:00
5d90c9be07
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi
2015-10-23 09:52:33 -04:00
bd506b1e10
NIFI-972: Added additional unit test; deleted lines that were commented out
2015-10-23 09:52:09 -04:00
a9e5325047
NIFI-972 attribute to indicate rows count and cleanup
...
Signed-off-by: Toivo Adams <toivo.adams@gmail.com>
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-23 09:28:03 -04:00
ba3225fe92
NIFI-972 ExecuteSQL bug in createSchema() create Arvo Schema 1
...
Signed-off-by: Toivo Adams <toivo.adams@gmail.com>
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-23 09:28:03 -04:00
a5a5badb88
NIFI-945 Create a new property (JSON Container) in ConvertAvroToJson, which determines how stream of records is exposed: either as a sequence of single Objects ("none"),
...
writing every Object to a new line, or as an array of Objects.
Let's assume you have an Avro content as stream of records (record1, record2, ...). If JSON container is "none", the converter will expose the records as sequence of
single JSON objects:
record1
record2
...
recordN
Please bear in mind, that the final output is not a valid JSON content. You can then forward this content e.g. to Kafka, where every record will be a single Kafka message.
If JSON container is "array", the output looks like this:
[record1,record2,...,recordN]
It is useful when you want to convert your Avro content to a valid JSON array.
This closes #88
Reviewed and Amended (amendments reviewed by original patch author on github) by Tony Kurc (tkurc@apache.org )
2015-10-21 20:15:29 -04:00
da28b81eec
Merge branch 'NIFI-988'
2015-10-21 10:44:06 -04:00
97441ea0c2
Merge branch 'NIFI-988' of https://github.com/ImpressTV/nifi into NIFI-988
2015-10-21 10:18:05 -04:00
e68fdca517
NIFI-944 Added support and unit tests for escaped characters in ConvertCSVtoAvro processor properties. This closes #87 . Reviewed by Tony Kurc <tkurc@apache.org>
2015-10-21 00:01:38 -04:00
518670dbf6
NIFI-1042 Adds restlistener.remote.source.ip attribute to ListenHTTPServlet. This closes #105
2015-10-19 23:43:44 -04:00
b809031195
NIFI-1046 shell clean up: +variable braces for consistency. Reviewed by Tony Kurc (tkurc@apache.org). This closes #106
2015-10-19 20:36:58 -04:00
9aa716b059
NIFI-1046 shell clean up: quoting prevents globbing & word splitting.
...
Ignoring the unquoted array expansions on L188/190; suspect they ought to be quoted per [SC2068](https://github.com/koalaman/shellcheck/wiki/SC2068 ), but need to confirm desired arguments.
Reviewed by Tony Kurc (tkurc@apache.org )
2015-10-19 20:36:07 -04:00
58d9b02517
NIFI-1046 shell clean up: remove backticks from nifi.sh. Reviewed by Tony Kurc (tkurc@apache.org)
2015-10-19 20:33:52 -04:00
9200542298
NIFI-1016 Naive fix so org.apache.nifi.prioritizer.PriorityAttributePrioritizerTest builds
2015-10-18 20:34:36 -04:00
88b1b844fb
NIFI-747 This closes #104 . PR from Venkatesh Sellappa <VS186031@outlook.com> was modified, then code reviewed by Joe Witt (comments in ticket)
2015-10-18 20:04:24 -04:00
ad73a23aff
NIFI-1016: Use centralized priority attribute name in PriorityAttributePrioritizer. closes #98 .
...
Signed-off-by: joewitt <joewitt@apache.org>
2015-10-18 19:42:46 -04:00
31f0909bd3
NIFI-994: Initial import of TailFile
2015-10-18 19:00:24 -04:00
943ccfedef
NIFI-1035 AbstractKiteProcessor.getSchema leaks file handles. closes #101 .
...
Close file handles when reading schema from resources or paths in AbstractKiteProcessor.getSchema
Signed-off-by: joewitt <joewitt@apache.org>
2015-10-18 18:56:26 -04:00
22924c656b
NIFI-995 TestGetFile.testAttributes() failed in case of a NTFS partition in Linux, because Files.setPosixFilePermissions() did not have any effect on the files and did not throw Exception. closes #95 . Signed off by Tony Kurc (tkurc@apache.org)
2015-10-17 10:46:49 -04:00
9a8d763d8d
NIFI-612 Remove FlowUnmarshaller. Builds, runs, could not find any latent references using reflection or other ways of loading classes without direct references. This closes #103 . Signed off by Tony Kurc <tkurc@apache.org>
2015-10-17 09:43:10 -04:00
570202eb30
NIFI-730:
...
- Fixing capitalization to remain consistent.
2015-10-16 10:47:02 -04:00
a872403831
NIFI-730:
...
- Updating the styling of the drop request status dialog.
- Rendering any errors that may have occurred.
2015-10-16 09:37:57 -04:00
2b4999c018
NIFI-730: do not interrupt swap thread if drop flowfiles is canceled
2015-10-16 09:06:13 -04:00
b8dbd1018c
NIFI-730: Updated details of Provenance Event for when a queue is emptied
2015-10-16 08:53:40 -04:00
39a050d2fd
NIFI-730:
...
- Adding emptying a queue when clustered.
2015-10-14 17:47:06 -04:00
09a3f6dadd
NIFI-730: reordered states for drop flowfile request
2015-10-14 16:32:39 -04:00
98a04eec74
NIFI-730: Set current queue size on drop flowfile request immediately when creating request
2015-10-14 13:48:59 -04:00
84db372538
NIFI-977: Allow SQL Data Types with numerals that are negative
2015-10-14 13:12:10 -04:00
cad0e7cf0f
NIFI-730:
...
- Adding progress bar for showing current status of drop request.
- Allowing the user to cancel the current drop request.
2015-10-14 12:39:19 -04:00
a2ae99f899
NIFI-730: Make cancel request actually cancel
2015-10-14 11:11:48 -04:00
5867193bc1
NIFI-730: Added additional testing logic to unit tests
2015-10-14 09:56:52 -04:00
0af1acaafa
NIFI-730: Return DropFlowFileStatus object when calling cancel
2015-10-14 09:46:21 -04:00
77f7d7524c
NIFI-730: bug fixes and code cleanup for swap manager and flowfile queue
2015-10-14 09:14:15 -04:00
afb76afcd0
NIFI-730: Added error messages if we fail to drop FlowFiles from queue
2015-10-13 15:57:18 -04:00
72ff2a25d5
NIFI-730: Updated queue sizes appropriately during dropping of flowfiles
2015-10-13 15:46:56 -04:00
4b41aaab02
NIFI-730:
...
- Fixing checkstyle violations.
- Wiring endpoints and framework model.
- Lowering the max delay while polling from 8 seconds to 4 seconds.
2015-10-13 13:43:10 -04:00
af78354d84
NIFI-730: Added additional parameters to dropFlowFiles
2015-10-13 12:20:18 -04:00
09d6fe5cdb
NIFI-730:
...
- Fixing checkstyle violations.
- Renaming context menu item for emptying a queue.
2015-10-13 10:12:58 -04:00
ad6af95d07
NIFI-730: Fixed checkstyle violations
2015-10-13 10:09:21 -04:00
9be37914dd
Merge branch 'NIFI-730' of https://git-wip-us.apache.org/repos/asf/nifi into NIFI-730
2015-10-13 10:03:31 -04:00
49a781df2d
NIFI-730: Implemented swapping in and out on-demand by the FlowFileQueue rather than in a background thread
2015-10-13 10:03:03 -04:00
49ee06b0a2
NIFI-774 Ignore integration style test
2015-10-12 19:59:46 -04:00
e0ac7cde37
NIFI-730:
...
- Starting to add support for deleting flow files from a queue by creating endpoints and starting to wire everything together.
- Adding context menu item for initiating the request to drop flow files.
2015-10-12 10:00:54 -04:00
fad81d872d
Merge remote-tracking branch 'yu/NIFI-774'
2015-10-12 09:55:52 -04:00
b8c51dc35d
NIFI-730: Added methods for dropping queued flowfiles; refactored swap manager but have not yet started swapping flowfiles in or out from within the flowfile queue
2015-10-11 10:27:07 -04:00
f798127290
NIFI-1031 resolve reporting task error in flow.xml
...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-10-09 10:53:23 -04:00
b4bfcc1f21
NIFI-1030 made InvokeHttp basic auth password a sensitive value
...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-10-08 15:46:12 -04:00
2583d7869a
NIFI-442:
...
NIFI-828:
- Always selecting the first item in the new component table.
- Enabling adding the selected component by typing Enter.
- Removing the 'filter by' in the new component dialogs and instead just searching every field.
2015-10-07 19:11:55 -04:00
8e2308b78d
NIFI-810: Reworded validation errors pertaining ot upstream connections
2015-10-07 17:53:04 -04:00
ccfb57fe9f
NIFI-810: Addressed several checkstyle violations
2015-10-07 17:48:51 -04:00
b974445ddd
NIFI-810: rebased from master
2015-10-07 17:28:39 -04:00
5ecdb1858e
NIFI-810: Updated the wording on validation errors due to upstream connections
2015-10-07 17:26:14 -04:00
13edcfda2e
NIFI-810:
...
- Adding basic support for preventing connection when appropriate.
- Updating validation when [dis]connecting processors.
2015-10-07 17:26:14 -04:00
4afd8f88f8
NIFI-810: Created RequiresInput annotation and ensure that processors are invalid if connections do not agree
2015-10-07 17:26:14 -04:00
31fba6b333
Bumped nifi-kafka-processors Kafka version
...
This closes #99 .
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-10-05 10:39:22 -04:00
6b2f5ad1f5
NIFI-992 Adding NOTICE to nifi-couchbase-nar
2015-10-01 16:56:49 -04:00
033a1553ab
nifi-992 Improvements based on code review part II.
...
- Penalize or Yield based on ErrorHandlingStrategy.Penalty
- Add Retry relationship to PutCouchbaseKey
- Remove unnecessary try/catch and let the framework handle it
- Change CouchbaseException relation mapping for Fatal from Failure to Retry
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-10-01 14:13:20 -04:00
72eb64e8a4
nifi-992 Improvements based on code review.
...
- Removed checkstyle and contrib-check profile since it's inherit from
top-level pom.
- Consolidate DOC_ID and DOC_ID_EXP into a single DOC_ID property.
- Add capability description on GetCouchbaseKey.
- Fixed documentation spell misses.
- Handle Exceptions accordingly.
- Add 'retry' relationship.
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-09-29 13:48:13 -04:00
2466a24530
nifi-992 Adding nifi-couchbase-bundle.
...
- new CouchbaseClusterControllerService
- new Processors
- GetCouchbaseKey
- PutCouchbaseKey
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-09-28 11:21:42 -04:00
2215bc848b
NIFI-810:
...
- Adding basic support for preventing connection when appropriate.
- Updating validation when [dis]connecting processors.
2015-09-25 17:46:58 -04:00
034ee6de6b
NIFI-810: Created RequiresInput annotation and ensure that processors are invalid if connections do not agree
2015-09-25 11:39:58 -04:00
96764ed6a1
Merge branch 'master' of http://git-wip-us.apache.org/repos/asf/nifi
2015-09-25 09:15:34 -04:00
eb94dab00c
NIFI-604: Fixed checkstyle violation
2015-09-24 11:51:44 -04:00
e53ab391fa
NIFI-641 Add delete capability to the UpdateAttributes processor.
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-09-24 10:46:10 -04:00
e12da7c9a3
NIFI-985: Custom log prefix for LogAttribute processor
...
The LogAttribute processor evaluates the log prefix EL using the current flow file.
Log prefix helps to distinguish the log output of multiple LogAttribute processors and identify the right processor. Log prefix appears in the first and the last log line, followed by the original 50 dashes. If you configure log prefix 'STEP 1: ' the log output looks like this:
STEP 1 : --------------------------------------------------
Standard FlowFile Attributes
Key: 'entryDate'
Value: 'Tue Sep 22 15:13:02 CEST 2015'
Key: 'lineageStartDate'
Value: 'Tue Sep 22 15:13:02 CEST 2015'
Key: 'fileSize'
Value: '9'
FlowFile Attribute Map Content
Key: 'customAttribute'
Value: 'custom value'
STEP 1 : --------------------------------------------------
flow file content...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-09-24 10:31:04 -04:00
24c080788f
NIFI-604: Removed unused import, as it caused checkstyle violation
2015-09-24 10:06:39 -04:00
193dbe6533
Merge branch 'NIFI-604' of https://github.com/rickysaltzer/nifi into NIFI-604
2015-09-23 16:03:46 -04:00
26f80095b7
Merge branch 'master' of http://git-wip-us.apache.org/repos/asf/nifi
2015-09-23 15:31:19 -04:00
f884086fc2
NIFI-713: Fixed checkstyle issue
2015-09-23 15:20:25 -04:00
da56b49b88
Merge branch 'NIFI-713' of https://github.com/rickysaltzer/nifi into NIFI-713
2015-09-23 12:06:50 -04:00
ee7d89cb01
NIFI-988: Test cases for PutDistributedMapCache
2015-09-23 14:32:37 +02:00
6b1328f3f1
NIFI-988: PutDisributedMapCache processor implementation
2015-09-23 14:31:13 +02:00
8615941c89
NIFI-959 Fixing incorrect evaluation for getting the collection and adding a unit test to prevent regression
2015-09-22 15:38:53 -04:00
5764e89293
NIFI-975 Fixing NullPointerException when one of the delimiters is not provided when using the Text strategy
2015-09-22 13:19:15 -04:00
14eaeeb1ee
NIFI-984: Bad class used for logging in TestDetectDuplicate
...
Using TestListenUDP instead of TestDetectDuplicate when creating the logger.
This closes #89
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-09-22 08:51:58 -04:00
d75c7b54e8
NIFI-934:
...
- Added a content viewer for image files.
2015-09-18 18:17:12 -04:00
600f91a262
NIFI-932 Adding hasIncomingConnection() and hasConnection(Relationship) to ProcessContext and updating ExecuteSQL
2015-09-18 17:05:31 -04:00
1fd45958d3
NIFI-876:
...
- Merge branch 'release-nifi-0.3.0-rc1'
2015-09-18 08:48:48 -04:00
624bb5dd74
NIFI-876:
...
- Merge branch 'release-nifi-0.3.0-rc1'
2015-09-18 08:48:40 -04:00
b36841e213
NIFI-919 Adding SplitAvro processor that splits binary datafiles into smaller datafiles, or bare records.
...
- Adding documentation about bare record use, renaming Split Size to Output Size, and adding a test case with 0 records
- Removing validators on properties that have allowable values, using positive integer validator for Output Size, and fixing type on processor description
2015-09-16 10:11:30 -04:00
1007999415
NIFI-912 Initial version of ExtractAvroMetadata processor.
...
- Adding optional ability to extract record count
- Renaming record.count to item.count for clarity, and updating documentation
- Adding a test case with 0 records
- Removing validators from properties that use allowable values
2015-09-16 09:59:50 -04:00
ded74ec94c
NIFI-876 prepare for next development iteration
2015-09-14 21:48:11 -04:00
2ec735e350
NIFI-876 prepare release nifi-0.3.0-RC1
2015-09-14 21:48:00 -04:00
3d4ce34529
NIFI-938: If ResourceClaim is removed while a process has access to its stream, don't delete the claim
2015-09-14 11:47:02 -04:00
992e841027
NIFI-948: Provide capability to register notifications for nifi lifecycle events
2015-09-13 19:55:12 -04:00
4a7d1fe618
NIFI-936: Ensure that if we fail to write a record to the repository that we do not write to that log file again
2015-09-11 14:55:48 -04:00
3780159d01
NIFI-939: Fixed bug in equals and hashCode methods of StandardContentClaim; added unit test to TestStandardProcessSession in order to verify behavior of append() method
2015-09-11 12:50:39 -04:00
eb1d6b554c
Remove `REL_NOT_FOUND`
2015-09-11 18:05:20 +09:00
a83ed34f91
Commit for NIFI-836 to fix broken javadoc @link elements
...
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-09-10 12:05:47 -04:00
f718b4bf46
Modify an error message
2015-09-07 23:34:22 +09:00
d32a32a92d
Remove the process to check if the target key exists or not
2015-09-07 23:32:48 +09:00
d1206fe074
NIFI-926:
...
- Ensuring that nodes are not disconnected when the user attempts to reset a counter that does not exist on that node. This can happen when/if counters are adjusted conditionally.
2015-09-07 08:41:47 -04:00
85534ca860
NIFI-916 Refactoring how the closed flag is used in PersistentProvenanceRepository to ensure proper shutdown
2015-09-06 13:22:07 -04:00
c492a1aaae
Fix a type and modify an error message
2015-09-02 21:54:02 +09:00
0334f04640
Add a relation for "not found" and that is transfered if a target key doesn't exist on S3
2015-09-02 13:48:39 +09:00
213f507f53
Add @Override annotations
2015-09-02 13:02:39 +09:00
6cbc6db860
Remove `getSupportedDynamicPropertyDescriptor`
2015-09-02 06:49:46 +09:00
7e6834937b
Ignore the test suite
2015-09-01 22:36:50 +09:00
d1dbd37629
[NIFI-774] Create a DeleteS3Object Processor
2015-09-01 22:35:11 +09:00
cd2e1424cb
NIFI-911:
...
- Updating default value for Regex so it matches once (?s:^.*$) instead of twice (.*). Matching on .* results in matching for every character and then again for 0 characters.
2015-08-31 14:58:38 -04:00
c72fb201d5
NIFI-713: Infer Hadoop Compression Automatically
...
- Three main types of compression options:
NONE : no compression
AUTOMATIC : infers codec by extension
SPECIFIED : specified codec (e.g. snappy, gzip, bzip, or lz4)
2015-08-31 14:38:15 -04:00
2bb7853001
Merge branch 'NIFI-905'
2015-08-31 11:14:17 -04:00
f8c216b061
NIFI-903: Check for null value before trying to close 'out' OutputStream on StandardRecordWriter.close
2015-08-31 10:50:00 -04:00
3159cec782
NIFI-905: Ensure that when archive threshold is hit, archived data is destroyed and if no archived data exists that Processors aren't blocked from updating content repo
2015-08-28 14:15:35 -04:00
5de37f63d9
NIFI-902: Ensure that if we get an IOException during rollover of WAL, we are able to recover
2015-08-28 10:04:58 -04:00
76b5b38cc4
Merge branch 'NIFI-893'
2015-08-27 17:34:27 -04:00
9cefc4a5a5
NIFI-604: Custom Argument Delimiters ExecuteStreamCommand / ExecuteProcess
...
- Unified the way ExecuteStreamCommand and ExecuteProcess handle arguments
- Argument delimiters can now be specified. Their default being what they were using before (; and space)
2015-08-27 15:26:56 -04:00
fbec28bad9
NIFI-893: Ensure that if session.commit fails, previously 'checkpointed' sessions are rolled back
2015-08-27 15:17:55 -04:00
f34d73e73e
NIFI-895 ensuring the session factory is re-set in the onTrigger method in the case where the runner needed to be stopped
2015-08-26 11:21:08 -04:00
310347fd66
Merge branch 'NIFI-842'
2015-08-24 18:58:56 -04:00
7fb6e884a7
NIFI-866: Add Kerberos Support for Hadoop
...
- Add krb5.conf to nifi.properties
nifi.kerberos.krb5.file | path to krb5.conf
- Connections to secure Hadoop clusters will be determined by their config,
that is, hadoop.security.authentication should be set to kerberos.
- Added two optional arguments to AbstractHadoopProcessor (principal and keytab),
these are only required if the cluster you're connecting to is secured. Both of
these options require the krb5.conf to be present in nifi.properties.
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-08-24 15:36:22 -04:00
3bfe830afc
NIFI-842 Updating org.apache.nifi.NiFi so it sends a started signal to bootstrap which can then be used to make retsart decisions
2015-08-24 12:00:21 -04:00
c7f7704220
Changed line wrap length to 200 and combined adjacent imports into single try-with-resources blocks.
...
This closes #76
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-08-23 23:07:50 -04:00
b30fbe1f3b
NIFI-888 EncryptContent for PGP Leaks File Handles.
...
Closes InputStreams created to read the public keys for PGP encryption and several other
streams involved in PGP encryptiong. This prevents NiFi from leaking file handles on
every validate call or encryption attempt in the EncryptContent processor.
2015-08-23 23:07:26 -04:00
4d4f74cf05
NIFI-889: Return the failure relationshp in the set of allowed relationships
2015-08-22 20:47:45 -04:00
671445f10a
Merge branch 'NIFI-844'
2015-08-21 17:15:39 -04:00
e8248353df
Merge branch 'NIFI-882'
2015-08-21 17:13:43 -04:00
a9c2bf58b8
NIFI-844: Initial import of the ConvertAvroToJSON processor
2015-08-21 17:11:56 -04:00
d351f3ed1b
NIFI-882: When a ProcessGroup is removed from the graph, ensure that the removeXYZ method is called for each component in that Process Group so that resources are appropriately cleaned up
2015-08-21 15:55:04 -04:00
Joseph Percivall
Tony Kurc
Joe Skora
Aldrin Piri
Mark Payne
Bryan Bende
Naveen Madhire
Oleg Zhurakousky
Matt Gilman
Jeremy Dyer
joewitt
Adam Lamar
Mike Moser
Venkatesh Sellappa
Andrew Grande
Luke Williamson
ricky
Toivo Adams
Joe
Andre F de Miranda
Alex Moundalexis
Alan Jackoway
danbress
Mike Moser
Randy Gelhausen
ijokarumawak
Yu ISHIKAWA