37d6b7350e
NIFI-1155: Refactored StandardFlowFileQueue to update member variables more intelligently, using CAS operations instead of locks. This reduces code complexities because other optimizations that previously existed are no longer needed
...
Signed-off-by: joewitt <joewitt@apache.org>
2015-11-13 00:06:12 -05:00
3ed0949c55
NIFI-1155: Ensure that when poll(FlowFileFilter, Set) is called, we properly update the indicator for whether or not queue is full
...
Signed-off-by: joewitt <joewitt@apache.org>
2015-11-13 00:06:12 -05:00
e6086420aa
NIFI-1158 Default timestamp based on now
2015-11-12 23:11:43 -05:00
0900fb80c9
NIFI-1153: If no incoming FlowFile, don't try to transfer null
...
Reviewed by Bryan Bende (bbende@apache.org )
Amended based on review (change to a log message) by Tony Kurc (tkurc@apache.org )
2015-11-12 22:41:57 -05:00
33ef59c5ba
NIFI-1127 Adding Kerberos properties to FetchHDFS and ListHDFS.
...
Reviewed by Tony Kurc (tkurc@apache.org )
2015-11-12 22:14:12 -05:00
65bd8c0b1f
NIFI-1109 Updated documentation and cleaned up code
...
Reviewed by Bryan Bende (bbende@apache.org ).
Committed with amendments (for whitespace, a couple misspellings and to change a test method name based on review) by Tony Kurc (tkurc@apache.org )
2015-11-12 21:07:01 -05:00
02102ea1c2
Revert "NIFI-1134". This was removed because they looked like floating classes. But in reality they are used to make jars which we use to test the build. However, we really need the test to work differently but until that is fixed these must stay.
...
This reverts commit 41f3875347
2015-11-12 18:00:40 -05:00
56ad22aea6
NIFI-900: Created Processors for interacting with Microsoft Azure EventHubs
...
Reviewed (with amendments needed for clean merge, whitespace and NOTICEs) by Tony Kurc (tkurc@apache.org )
2015-11-11 23:06:04 -05:00
7a165b62cc
Merge branch 'NIFI-994'
2015-11-11 16:05:40 -05:00
854d203982
Merge branch 'NIFI-1143' of https://github.com/olegz/nifi into NIFI-1143
2015-11-11 14:14:54 -05:00
5baafa156a
NIFI-1143 Fixed race condition which caused intermittent failures
...
Fixed the order of service state check in PropertyDescriptor
Encapsulated the check into private method for readability
Modified and documented test to validate correct behavior.
For more details please see comment in https://issues.apache.org/jira/browse/NIFI-1143
2015-11-11 14:06:08 -05:00
11fcad90d1
Merge branch 'NIFI-1083'
2015-11-11 12:07:15 -05:00
054774fe7b
Merge branch 'NIFI-1061' of https://github.com/olegz/nifi into NIFI-1061
2015-11-11 11:17:25 -05:00
e7c6c7cae9
NIFI-1083: Added a Grouping Regular Expression property for grouping lines of text
2015-11-11 10:19:07 -05:00
a65a29921a
NIFI-1124, NIFI-1062 addressed more PR comments
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-11-11 10:12:30 -05:00
70cc61bb53
NIFI-1124, NIFI-1062 addressed PR comments
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-11-11 10:12:30 -05:00
62e3cfc629
NIFI-1024 NIFI-1062 Fixed PutHDFS processor to properly route failures. Ensured that during put failures the FlowFile is routed to 'failure' relationship. Added validation test Re-enabled previously ignored test.
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-11-11 10:12:30 -05:00
3e538d9007
NIFI-1083 Created a processor that routes lines of text based on different matching and routing strategies
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-11 09:12:47 -05:00
76690a8ee9
NIFI-1061 fixed deadlock caused by DBCPConnectionPool.onConfigured()
...
Current implementation of DBCPConnectionPool was attempting to test if connection could be obtained via dataSource.getConnection().
Such call is naturally a blocking call and the duration of the block is dependent on driver implementation. Some drivers (e.g., Phoenix - https://phoenix.apache.org/installation.html )
attempts numerous retries before failing creating a deadlock when attempt was made to disable DBCPConnectionPool which was still being enabled.
This fix removes the connection test from DBCPConnectionPool.onConfigured() operation returning successfully upon creation of DataSource.
For more details see comments in https://issues.apache.org/jira/browse/NIFI-1061
2015-11-11 08:33:51 -05:00
6c510fae80
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi
2015-11-11 08:16:38 -05:00
d887d8fdf0
NIFI-993 added expression language support to GetHTTP for the filename and URL properties
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-11 08:05:23 -05:00
ba5b1d837c
NIFI-1117 Changed line endings to unix line endings in java source
2015-11-10 18:50:57 -05:00
9756e9f126
NIFI-1117 Added unit tests and default behavior for avro files with no records. Also fixed a findbugs warning in SplitAvro (made inner classes static)
2015-11-10 18:46:17 -05:00
e6644b3747
Merge branch 'NIFI-1141'
2015-11-10 17:17:55 -05:00
7b9c8df6c5
NIFI-994: Fixed issue that could result in data duplication if more than 1 rollover of tailed file has occurred on restart of Processor
2015-11-10 16:53:29 -05:00
6ed5f28f26
NIFI-965 penalized failed flowfiles in S3 Processors. Also discovered a typo in a static final that I corrected
...
Signed-off-by: Mike Moser <mosermw@apache.org>
2015-11-10 15:14:28 -05:00
72008035b1
Merge branch 'NIFI-1000'
2015-11-10 15:02:02 -05:00
8f942fcdcc
NIFI-1000: Fixed checkstyle violation in TestGetJSMQueue use activemq-broker with test scope and activemq-client with default scope rather than pulling in activemq-all; this reduces size of standard nar from 30 MB to 18 MB
2015-11-10 14:31:20 -05:00
53725b5c72
Merge branch 'NIFI-1000' of https://github.com/olegz/nifi into NIFI-1000
2015-11-10 13:34:56 -05:00
8699e35108
NIFI-1000 addressed PR comments
2015-11-10 12:13:16 -05:00
11768cc388
NIFI-1141: Fixed the order of the arguments to the setCurrentContentClaim method
2015-11-10 11:35:14 -05:00
e748fd5848
NIFI-817 Processors for interacting with HBase
...
- Refactoring PutHBaseCell to batch Puts by table
- Adding optional Columns property to GetHBase to return only selected column families or columns
- Making GetHBase cluster friendly by storing state in the distributed cache and a local file
- Adding Initial Time Range property to GetHBase
- Adding Filter Expression property and custom validate to prevent using columns and a filter at the same time
- Creating an HBaseClientService controller service to isolate the HBase client and support multiple versions
- Creating appropriate LICENSE/NOTICE files
- Adding @InputRequirement to processors
- Addressing comments from review, moving hbase client services under standard services
- Making sure result of session.penalize() is assinged to FlowFile variable before transferring
2015-11-10 11:13:51 -05:00
2a90bd501b
NIFI-797 Corrected URL for PutS3 to include correct protocol and endpoint by region.
...
- corrected a missed 'final' on org.apache.nifi.processors.aws.AbstractAWSProcessor.relationships
- added protected method org.apache.nifi.processors.aws.AbstractAWSProcessor.getRegion()
- added protected method org.apache.nifi.processors.aws.s3.AbstractS3Processor.getUrlForObject(String, String)
- explicitly set AWS client protocol to HTTPS, and created a static final field with comments if other protocols may be considered
- added a static final field for the UserAgent
Reviewed by Aldrin Piri <aldrin@apache.org>
2015-11-10 06:37:56 -05:00
ef0be5a5d6
NIFI-1000 Fixed JmsFactory to properly obtain destiniation name
...
Re-enabled JMS Tests that were annotated with @Ignore
2015-11-09 18:35:31 -05:00
b75af5b344
Merge branch 'NIFI-1085'
2015-11-09 16:13:12 -05:00
41f3875347
NIFI-1134
2015-11-09 15:37:40 -05:00
73c1671975
NIFI-1132: Limited number of Lineage Identifiers held to 100 and marked the getLineageIdentifiers() method as deprecated
2015-11-09 12:09:56 -05:00
5f8fdae909
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi
2015-11-09 12:07:45 -05:00
64b1fc983c
NIFI-925 reviewed patch/ builds well/ fixed licensing gaps.
2015-11-08 22:15:11 -05:00
59a49aea12
Revert "NIFI-1082: Ensure that events returned from the provenance repository are ordered such that newest events are provided first"
...
This reverts commit cf8ca3dc2c
2015-11-08 21:46:06 -05:00
440e4c4940
NIFI-1106 fixed platform-specific object separator.
2015-11-08 12:27:41 -05:00
77de51df19
NIFI-1025 Updated aws-sdk and joda-time to resolve AWS issues with Java 1.8 update 60.
...
- Refactored tests - created AbstractS3Test for common utility methods
- Corrected incorrect unit test in TestDeleteS3Object, and adjusted processor documentation to reflect behavior
- moved aws dependency management to root pom
This closes #107
Tested, Reviewed and Amended by Tony Kurc (<tkurc@apache.org>)
2015-11-07 21:09:18 -05:00
99629646fe
NIFI-1122 moved to 0.4.0-SNAPSHOT
2015-11-06 23:41:15 -05:00
0d6e81b54f
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi
2015-11-06 09:16:12 -05:00
ee87f33ce7
Fixing check-style violation in StandardFlowFileQueue
2015-11-06 09:10:10 -05:00
1a388f957e
NIFI-1111: We were subtracting values in the wrong order so that we always would 'yield' for a negative amount of time, which equated to not yielding at all.
2015-11-06 09:03:38 -05:00
98f5a1ab77
NIFI-1110: Fixed bug that caused queue size to become negative when FlowFiles are expired
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-05 12:12:02 -05:00
4ac52bfc0a
Removing nifi-azure-bundle that was incorrectly added to master
2015-11-05 10:42:10 -05:00
f1bd3d471d
NIFI-1112 Fixed APIOperation Annotation on ReportingTaskResource.
...
Reviewed by Tony Kurc (tkurc@apache.org )
2015-11-04 18:31:34 -05:00
618f22e110
NIFI-274 - Fixing TestListenSyslog, fixing default buffer size to be bytes, adding syslog.protocol to attributes
...
- Adding syslog.port to ListenSyslog attributes, logging at warn level when rejecting tcp connections
- Adding @InputRequirement to processors and adding appropriate send and receive provenance events
2015-11-04 18:01:59 -05:00
5611dac3f8
NIFI-274 - added use of Selectors for TCP and UDP connections. Added a max connections to the TCP thread
...
- Added comments and code review changes
- fixed fixbugs bug
2015-11-04 18:00:18 -05:00
9c542432da
NIFI-274 Adding ListenSyslog and PutSyslog to standard processors.
...
- Refactoring connection handling on put side, removing number of buffers from properties and basing it off concurrent tasks for the processor.
- Refactoring some of the TCP handling so it keeps reading from a connection until the client closes it
- Adding an error queue
- Adding a sender field on the syslog event to record the system that sent the message
2015-11-04 17:56:13 -05:00
6add372bc1
NIFI-1085 WebClusterManager fair locking and HeartbeatMonitoringTimerTask scheduling
2015-11-04 12:19:00 -05:00
201eac052b
NIFI-1098 - Ensuring we pull the correct attribute from the servlet context. Reviewed by Tony Kurc (tkurc@apache.org)
2015-11-04 12:11:19 -05:00
fedbe40313
NIFI-1103: Added InputRequirement annotation
2015-11-04 11:11:33 -05:00
2b1d093120
NIFI-1105: Only trigger a processor that requires input to run if data is available for it process
2015-11-04 08:41:10 -05:00
dbf0c7893f
NIFI-730: Completed merging of master
2015-11-02 14:31:28 -05:00
e4cebba3c7
Merge branch 'master' into NIFI-730
2015-11-02 14:02:16 -05:00
a549621267
NIFI-1077 enabling expression language on ConvertCharacterSet input and output
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-02 11:46:17 -05:00
814e8b212c
Merge branch 'NIFI-1082'
2015-11-02 11:29:52 -05:00
abf2c62884
NIFI-1051: Fixed checkstyle violations
2015-11-02 11:01:50 -05:00
7a73867c46
Merge branch 'NIFI-1051' of https://github.com/olegz/nifi into NIFI-1051
2015-11-02 10:51:43 -05:00
2ae49026e8
NIFI-883 Fixing issue HandleHttpRequest had with PrimaryNodeOnly scheduling
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-02 10:32:14 -05:00
5a04021dd7
NIFI-730:
...
- Updating logic now that original is guaranteed to be non null.
- Always reporting 100% once the drop request has completed.
2015-11-02 08:59:11 -05:00
f5727cfb0f
NIFI-730: Ensure that we always populate queue counts when initiating a Drop FlowFile request
2015-11-02 08:36:50 -05:00
ad849c77df
NIFI-1090: Fixed log message that was at info level but should have been debug level
2015-11-01 14:37:01 -05:00
b729bf4c19
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi
2015-11-01 14:16:54 -05:00
6e193dffa3
Merge branch 'NIFI-1088'
2015-11-01 14:16:35 -05:00
cef7b6c736
NIFI-1080 fleshed out InvokeHttp unit tests
...
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-10-31 00:19:36 -04:00
9c148da1c2
NIFI-869: Fixed checkstyle violations
2015-10-30 16:09:50 -04:00
ba72452f66
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi
2015-10-30 16:02:15 -04:00
a3d43d23dc
NIFI-869 Fixed SimpleProcessLogger to log correct messages Ensured that SimpleProcessLogger correctly interprets Throwable as discussed in JIRA Added tests
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-30 15:52:33 -04:00
90aea01350
NIFI-869 Fixed formatting issue Fixed formatting issue with printed error message which only apears when NiFi is cnfigured using Logback. Please see NIFI-869 for more details
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-30 15:52:33 -04:00
eb389cf84c
NIFI-1079 Replacing Name for NULL_VALUE_FOR_EMPTY_STRING as I had accidentally removed it.
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-10-30 14:30:56 -04:00
aef0d8fe9b
NIFI-1079 Check style and documentation based updates recommended by user bbende on Github pull request
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-10-30 14:30:42 -04:00
217b1049cf
NIFI-1079 Added Destination Property to control if JSON goes to Attribute or
...
Content of FlowFile. Added Include Core Attributes Property to control
if FlowFile CoreAttributes are included in the JSON output or not.
Added Null value for Empty String flag to control if empty values in
the JSON are empty string or true NULL values. Added more tests and
minor text refactoring per Github comments
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-10-30 14:30:25 -04:00
19b7a4cc7d
NIFI-1079 Create a new Attribute from the existing FlowFile attributes by taking
...
either all of the existing attributes or a user defined list. The
existing Attributes are converted to JSON and placed in a new Attribute
on the existing FlowFile as Attribute “JSONAttributes”
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-10-30 14:29:29 -04:00
9515b74607
NIFI-1088: Ensure that FlowFile is penalized before routing to failure
2015-10-30 14:25:27 -04:00
bfa9e45079
NIFI-994: Ensure that processor is not valid due to the tail file not yet existing
2015-10-30 11:44:51 -04:00
5c4042bd7c
NIFI-1051 Allowed FileSystemRepository to skip un-readable entries.
...
The exception was caused due to basic file permissions. This fix overrides
'visitFileFailed' method of SimpleFileVisitor to log WARN message and allow
FileSystemRepository to continue.
2015-10-29 16:31:17 -04:00
5cc2b04b91
NIFI-986 Refactoring of action classes from nifi-user-actions to have interfaces in nifi-api, and adding getFlowChanges to EventAccess
...
- Fixing empty java docs and adding sort by id asc to the history query
- Changing userDn to userIdentity in Action and FlowChangeAction
- Modifying NiFiAuditor to always save events locally, and implementing getFlowChanges for ClusteredEventAccess
2015-10-29 16:28:36 -04:00
c4f0cb1c6c
Removing nifi-pcap-bundle/.gitignore
2015-10-29 12:06:25 -04:00
cf8ca3dc2c
NIFI-1082: Ensure that events returned from the provenance repository are ordered such that newest events are provided first
2015-10-28 17:32:51 -04:00
edf238e004
NIFI-730: Do not require a Read Lock in order to obtain backpressure configuration values for FlowFileQueue's
2015-10-28 10:27:58 -04:00
dc4004de64
Merge branch 'NIFI-977'
2015-10-27 16:53:38 -04:00
af19053a7f
Merge branch 'NIFI-447'
2015-10-27 10:42:46 -04:00
07238c7913
NIFI-447 adding replacement strategy to ReplaceText processor
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-27 10:37:43 -04:00
b885f955f4
NIFI-516 adding option to StandardProcessSession.read to close stream
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-26 20:23:13 -04:00
1c1738670c
NIFI-10: Updated FetchHDFS, FetchFileTransfer to use new FETCH provenance event
2015-10-26 17:14:29 -04:00
fc2aa2764c
NIFI-10: Added FETCH and DOWNLOAD Provenance Events; updated FlowController to use DOWNLOAD event instead of SEND whenever a user downloads/views content via Provenance Event
2015-10-26 14:58:50 -04:00
51f564024a
NIFI-10: Added FETCH Provenance Event and updated processors to use this new event type
2015-10-26 14:58:50 -04:00
aec32a277c
NIFI-1070: Added detailed debug-level logging about how FileSystemRepository is choosing to expire archived data
2015-10-26 14:36:03 -04:00
f8c3377c84
NIFI-820:
...
- Creating a context menu item for accessing provenance directly for a selected component.
2015-10-26 10:16:01 -04:00
26edab3185
NIFI-908 Added support for SSL in JMS connections.
...
- Added SSL context to JMS producer and consumer processors
- Tony Kurc Amended patch to check SSL need by scheme and exception consistency
Reviewed by Tony Kurc (tkurc@apache.org )
2015-10-26 00:29:05 -04:00
8d2f9bc64b
NIFI-1056: Do not treat BigDecimal, BigInteger the same way we treat Number's when serializing to Avro because Avro Java Library doesn't support them
...
Reviewed and Unit Test Amended by Tony Kurc (tkurc@apache.org )
2015-10-25 11:31:30 -04:00
385bfbb2c6
NIFI-673: Rebased from master; Added InputRequirement annotation, as it is now merged into master
2015-10-25 11:14:11 -04:00
4e38288062
NIFI-673: Added sftp.listing.user attribute to FlowFiles created by ListSFTP; ensure that FetchSFTP indicates that the username supports Expression Language
2015-10-25 11:13:02 -04:00
b0322d9ffe
NIFI-673: Added Completion Strategy to FetchSFTP
2015-10-25 11:13:02 -04:00
d1d57931bf
NIFI-673: Initial implementation of ListSFTP, FetchSFTP
2015-10-25 11:13:02 -04:00
8a80060851
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi
2015-10-25 11:11:49 -04:00
Mark Payne
Tony Kurc
Bryan Bende
joewitt
Oleg Zhurakousky
Joseph Percivall
Mike Moser
Venkatesh Sellappa
Andrew Grande
Matt Gilman
Jeremy Dyer
Luke Williamson