Apache
/
nifi
mirror of https://github.com/apache/nifi.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,691 Commits 146 Branches 201 Tags 404 MiB
Commit Graph

332 Commits

Author SHA1 Message Date
Mark Payne e862f7ff03 NIFI-1165: Use FileChannel instead of RandomAccessFile in order to avoid locking files in Windows 
Reviewed by Tony Kurc (tkurc@apache.org)
 2015-11-19 01:01:28 -05:00
Joseph Percivall 1e5cc070a3 NIFI-1081 Adding option to ExecuteStreamCommand to put output value to an attribute 
Reviewed and amended (comments,whitespace,and some code readability (discussed in ticket)) by Tony Kurc (tkurc@apache.org)
 2015-11-18 23:23:10 -05:00
Tony Kurc 9e2f6df205 NIFI-1123: Fixing a botched commit. 2015-11-18 21:59:42 -05:00
Joe Skora 52b24b93d9 NIFI-1123 Adds expression language support to DeleteAttributesExpression on UpdateAttributes Processor. 
Reviewed by Tony Kurc (trkurc@gmail.com) after Aldrin Piri <aldrin@apache.org> did the initial review and actionable comments
 2015-11-18 19:10:21 -05:00
Tony Kurc ab7940368a NIFI-1187: Fixing issue of possible assigment reordering causing uninitalized values to be possibly returned 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2015-11-18 17:37:47 -05:00
Aldrin Piri c541c82c35 NIFI-1191 Adding missing tags for ConvertAvroToJSON 2015-11-18 16:38:15 -05:00
Mark Payne 69bce2c2db NIFI-1168: Ensure that processors with only looping 
connections are scheduled to run, even if the connections have no FlowFiles;
 expose these details to processor developers; update documentation

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2015-11-18 14:53:30 -05:00
Aldrin Piri 773576e041 NIFI-1108 Providing additional annotations on processors to ensure utilization of the InputRequirement annotation. 2015-11-18 13:56:21 -05:00
Mark Payne 911e1c5412 NIFI-1108: Updated processors to include the @InputRequirement annotation 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2015-11-18 13:47:59 -05:00
Mark Payne d88b6cb6bc NIFI-1173: Even if FlowFile Queue is empty, it needs to hold onto the Empty Queue request so that subsequent cancel/clear requests can reference it 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2015-11-18 10:56:38 -05:00
Mark Payne 180ea1ba22 NIFI-1176: Use a smaller internal blocking queue size of only 5000 messages; do not create a new queue every time the processor is scheduled to run 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2015-11-18 10:51:22 -05:00
Mark Payne 93be753301 NIFI-1181: Ensure that a FlowFile's uuid cannot be modified by processors 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2015-11-18 10:35:37 -05:00
Bryan Bende 2b9b5e008f NIFI-1175 Exposing minimum properties required to create an HBase connection on the HBaseClientService as an optional alternative to the conf files 2015-11-17 12:01:46 -05:00
Mark Payne 453b140d6b Merge branch 'NIFI-748' of https://github.com/olegz/nifi into NIFI-748 2015-11-17 09:23:10 -05:00
Naveen Madhire 03a54bf2d5 Changes after review 2015-11-16 13:32:17 -06:00
Naveen Madhire b954ca620e NIFI-1146 Allow GetKafka to be configured with auto.offset.reset to largest or smallest 2015-11-16 11:59:52 -06:00
Oleg Zhurakousky 15880f9fcc NIFI-748 addressed PR comments 
- made DocReader package private
- polished logic in read(..) method to avoid escaping the loop
- added call to sorting logic in LuceneUtil.groupDocsByStorageFileName(..) to ensure that previous behavior and assumptions in read(..) methodd are preserved
- other minor polishing
 2015-11-16 08:39:23 -05:00
Matt Gilman 64369f67f9 NIFI-1160: 
- Addressing possible NPE when removing a drop request.
- Only updating a connection status once it's been retrieved for the first time.
 2015-11-16 08:34:04 -05:00
Mark Payne da9f40b3f0 NIFI-973: Updated main Help page to provide a link to the Getting Started Guide. 
Reviewed by Aldrin Piri (aldrin@apache.org)
 2015-11-15 08:38:51 -05:00
Tony Kurc 528dab78d6 NIFI-1073: Fixing coverity discovered errors. Resource leaks, and statics 
Reviewed by Bryan Bende (bbende@apache.org)
 2015-11-14 18:56:43 -05:00
Aldrin Piri 01539ed323 NIFI-1163: Providing handling of SSLContext creation in GetHTTP in case of only performing a one-way SSL request and accompanying test to verify the configuration/usage. 
Reviewed by Tony Kurc (tkurc@apache.org)
 2015-11-14 18:47:50 -05:00
Oleg Zhurakousky a4d93c62c8 NIFI-748 Fixed logic around handling partial query results from provenance repository 
- Ensured that failures derived form correlating Document to its actual provenance event do fail the entire query and produce partial results with warning messages
- Refactored DocsReader.read() operation.
- Added test to validate two conditions where the such failures could occur
 2015-11-13 14:23:31 -05:00
Mark Payne 90f6830003 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2015-11-13 10:47:05 -05:00
Mark Payne 22de23baa6 NIFI-1097: Rewrite PutKafka to use the new producer api 2015-11-13 10:46:46 -05:00
joewitt 36d00a60f5 NIFI-1155 fixed contrib-check violation 2015-11-13 00:06:12 -05:00
Mark Payne 37d6b7350e NIFI-1155: Refactored StandardFlowFileQueue to update member variables more intelligently, using CAS operations instead of locks. This reduces code complexities because other optimizations that previously existed are no longer needed 
Signed-off-by: joewitt <joewitt@apache.org>
 2015-11-13 00:06:12 -05:00
Mark Payne 3ed0949c55 NIFI-1155: Ensure that when poll(FlowFileFilter, Set) is called, we properly update the indicator for whether or not queue is full 
Signed-off-by: joewitt <joewitt@apache.org>
 2015-11-13 00:06:12 -05:00
Adam Lamar 428b20fc25 NIFI-1103: Add support for long polling in GetSQS processor 2015-11-12 22:03:36 -07:00
Tony Kurc e6086420aa NIFI-1158 Default timestamp based on now 2015-11-12 23:11:43 -05:00
Mark Payne 0900fb80c9 NIFI-1153: If no incoming FlowFile, don't try to transfer null 
Reviewed by Bryan Bende (bbende@apache.org)
Amended based on review (change to a log message) by Tony Kurc (tkurc@apache.org)
 2015-11-12 22:41:57 -05:00
Bryan Bende 33ef59c5ba NIFI-1127 Adding Kerberos properties to FetchHDFS and ListHDFS. 
Reviewed by Tony Kurc (tkurc@apache.org)
 2015-11-12 22:14:12 -05:00
Mark Payne 65bd8c0b1f NIFI-1109 Updated documentation and cleaned up code 
Reviewed by Bryan Bende (bbende@apache.org).
Committed with amendments (for whitespace, a couple misspellings and to change a test method name based on review) by Tony Kurc (tkurc@apache.org)
 2015-11-12 21:07:01 -05:00
joewitt 02102ea1c2 Revert "NIFI-1134". This was removed because they looked like floating classes. But in reality they are used to make jars which we use to test the build. However, we really need the test to work differently but until that is fixed these must stay. 
This reverts commit 41f3875347.
 2015-11-12 18:00:40 -05:00
Mark Payne 56ad22aea6 NIFI-900: Created Processors for interacting with Microsoft Azure EventHubs 
Reviewed (with amendments needed for clean merge, whitespace and NOTICEs) by Tony Kurc (tkurc@apache.org)
 2015-11-11 23:06:04 -05:00
Mark Payne 7a165b62cc Merge branch 'NIFI-994' 2015-11-11 16:05:40 -05:00
Mark Payne 854d203982 Merge branch 'NIFI-1143' of https://github.com/olegz/nifi into NIFI-1143 2015-11-11 14:14:54 -05:00
Oleg Zhurakousky 5baafa156a NIFI-1143 Fixed race condition which caused intermittent failures 
Fixed the order of service state check in PropertyDescriptor
Encapsulated the check into private method for readability
Modified and documented test to validate correct behavior.
For more details please see comment in https://issues.apache.org/jira/browse/NIFI-1143
 2015-11-11 14:06:08 -05:00
Mark Payne 11fcad90d1 Merge branch 'NIFI-1083' 2015-11-11 12:07:15 -05:00
Mark Payne 054774fe7b Merge branch 'NIFI-1061' of https://github.com/olegz/nifi into NIFI-1061 2015-11-11 11:17:25 -05:00
Mark Payne e7c6c7cae9 NIFI-1083: Added a Grouping Regular Expression property for grouping lines of text 2015-11-11 10:19:07 -05:00
Oleg Zhurakousky a65a29921a NIFI-1124, NIFI-1062 addressed more PR comments 
Signed-off-by: Bryan Bende <bbende@apache.org>
 2015-11-11 10:12:30 -05:00
Oleg Zhurakousky 70cc61bb53 NIFI-1124, NIFI-1062 addressed PR comments 
Signed-off-by: Bryan Bende <bbende@apache.org>
 2015-11-11 10:12:30 -05:00
Oleg Zhurakousky 62e3cfc629 NIFI-1024 NIFI-1062 Fixed PutHDFS processor to properly route failures. Ensured that during put failures the FlowFile is routed to 'failure' relationship. Added validation test Re-enabled previously ignored test. 
Signed-off-by: Bryan Bende <bbende@apache.org>
 2015-11-11 10:12:30 -05:00
Joseph Percivall 3e538d9007 NIFI-1083 Created a processor that routes lines of text based on different matching and routing strategies 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-11-11 09:12:47 -05:00
Oleg Zhurakousky 76690a8ee9 NIFI-1061 fixed deadlock caused by DBCPConnectionPool.onConfigured() 
Current implementation of DBCPConnectionPool was attempting to test if connection could be obtained via dataSource.getConnection().
Such call is naturally a blocking call and the duration of the block is dependent on driver implementation. Some drivers (e.g., Phoenix - https://phoenix.apache.org/installation.html)
attempts numerous retries before failing creating a deadlock when attempt was made to disable DBCPConnectionPool which was still being enabled.

This fix removes the connection test from DBCPConnectionPool.onConfigured() operation returning successfully upon creation of DataSource.
For more details see comments in https://issues.apache.org/jira/browse/NIFI-1061
 2015-11-11 08:33:51 -05:00
Mark Payne 6c510fae80 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2015-11-11 08:16:38 -05:00
Joseph Percivall d887d8fdf0 NIFI-993 added expression language support to GetHTTP for the filename and URL properties 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-11-11 08:05:23 -05:00
Tony Kurc ba5b1d837c NIFI-1117 Changed line endings to unix line endings in java source 2015-11-10 18:50:57 -05:00
Tony Kurc 9756e9f126 NIFI-1117 Added unit tests and default behavior for avro files with no records. Also fixed a findbugs warning in SplitAvro (made inner classes static) 2015-11-10 18:46:17 -05:00
Mark Payne e6644b3747 Merge branch 'NIFI-1141' 2015-11-10 17:17:55 -05:00
Mark Payne 7b9c8df6c5 NIFI-994: Fixed issue that could result in data duplication if more than 1 rollover of tailed file has occurred on restart of Processor 2015-11-10 16:53:29 -05:00
Tony Kurc 6ed5f28f26 NIFI-965 penalized failed flowfiles in S3 Processors. Also discovered a typo in a static final that I corrected 
Signed-off-by: Mike Moser <mosermw@apache.org>
 2015-11-10 15:14:28 -05:00
Mark Payne 72008035b1 Merge branch 'NIFI-1000' 2015-11-10 15:02:02 -05:00
Mark Payne 8f942fcdcc NIFI-1000: Fixed checkstyle violation in TestGetJSMQueue use activemq-broker with test scope and activemq-client with default scope rather than pulling in activemq-all; this reduces size of standard nar from 30 MB to 18 MB 2015-11-10 14:31:20 -05:00
Mark Payne 53725b5c72 Merge branch 'NIFI-1000' of https://github.com/olegz/nifi into NIFI-1000 2015-11-10 13:34:56 -05:00
Oleg Zhurakousky 8699e35108 NIFI-1000 addressed PR comments 2015-11-10 12:13:16 -05:00
Mark Payne 11768cc388 NIFI-1141: Fixed the order of the arguments to the setCurrentContentClaim method 2015-11-10 11:35:14 -05:00
Bryan Bende e748fd5848 NIFI-817 Processors for interacting with HBase 
- Refactoring PutHBaseCell to batch Puts by table
- Adding optional Columns property to GetHBase to return only selected column families or columns
- Making GetHBase cluster friendly by storing state in the distributed cache and a local file
- Adding Initial Time Range property to GetHBase
- Adding Filter Expression property and custom validate to prevent using columns and a filter at the same time
- Creating an HBaseClientService controller service to isolate the HBase client and support multiple versions
- Creating appropriate LICENSE/NOTICE files
- Adding @InputRequirement to processors
- Addressing comments from review, moving hbase client services under standard services
- Making sure result of session.penalize() is assinged to FlowFile variable before transferring
 2015-11-10 11:13:51 -05:00
Tony Kurc 2a90bd501b NIFI-797 Corrected URL for PutS3 to include correct protocol and endpoint by region. 
- corrected a missed 'final' on org.apache.nifi.processors.aws.AbstractAWSProcessor.relationships
 - added protected method org.apache.nifi.processors.aws.AbstractAWSProcessor.getRegion()
 - added protected method org.apache.nifi.processors.aws.s3.AbstractS3Processor.getUrlForObject(String, String)
 - explicitly set AWS client protocol to HTTPS, and created a static final field with comments if other protocols may be considered
 - added a static final field for the UserAgent

Reviewed by Aldrin Piri <aldrin@apache.org>
 2015-11-10 06:37:56 -05:00
Oleg Zhurakousky ef0be5a5d6 NIFI-1000 Fixed JmsFactory to properly obtain destiniation name 
Re-enabled JMS Tests that were annotated with @Ignore
 2015-11-09 18:35:31 -05:00
Mike Moser b75af5b344 Merge branch 'NIFI-1085' 2015-11-09 16:13:12 -05:00
joewitt 41f3875347 NIFI-1134 2015-11-09 15:37:40 -05:00
Mark Payne 73c1671975 NIFI-1132: Limited number of Lineage Identifiers held to 100 and marked the getLineageIdentifiers() method as deprecated 2015-11-09 12:09:56 -05:00
Mark Payne 5f8fdae909 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2015-11-09 12:07:45 -05:00
joewitt 64b1fc983c NIFI-925 reviewed patch/ builds well/ fixed licensing gaps. 2015-11-08 22:15:11 -05:00
Mark Payne 59a49aea12 Revert "NIFI-1082: Ensure that events returned from the provenance repository are ordered such that newest events are provided first" 
This reverts commit cf8ca3dc2c.
 2015-11-08 21:46:06 -05:00
Tony Kurc 440e4c4940 NIFI-1106 fixed platform-specific object separator. 2015-11-08 12:27:41 -05:00
Venkatesh Sellappa 77de51df19 NIFI-1025 Updated aws-sdk and joda-time to resolve AWS issues with Java 1.8 update 60. 
- Refactored tests - created AbstractS3Test for common utility methods
 - Corrected incorrect unit test in TestDeleteS3Object, and adjusted processor documentation to reflect behavior
 - moved aws dependency management to root pom

This closes #107

Tested, Reviewed and Amended by Tony Kurc (<tkurc@apache.org>)
 2015-11-07 21:09:18 -05:00
joewitt 99629646fe NIFI-1122 moved to 0.4.0-SNAPSHOT 2015-11-06 23:41:15 -05:00
Mark Payne 0d6e81b54f Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2015-11-06 09:16:12 -05:00
Bryan Bende ee87f33ce7 Fixing check-style violation in StandardFlowFileQueue 2015-11-06 09:10:10 -05:00
Mark Payne 1a388f957e NIFI-1111: We were subtracting values in the wrong order so that we always would 'yield' for a negative amount of time, which equated to not yielding at all. 2015-11-06 09:03:38 -05:00
Mark Payne 98f5a1ab77 NIFI-1110: Fixed bug that caused queue size to become negative when FlowFiles are expired 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-11-05 12:12:02 -05:00
Bryan Bende 4ac52bfc0a Removing nifi-azure-bundle that was incorrectly added to master 2015-11-05 10:42:10 -05:00
Andrew Grande f1bd3d471d NIFI-1112 Fixed APIOperation Annotation on ReportingTaskResource. 
Reviewed by Tony Kurc (tkurc@apache.org)
 2015-11-04 18:31:34 -05:00
Bryan Bende 618f22e110 NIFI-274 - Fixing TestListenSyslog, fixing default buffer size to be bytes, adding syslog.protocol to attributes 
- Adding syslog.port to ListenSyslog attributes, logging at warn level when rejecting tcp connections
         - Adding @InputRequirement to processors and adding appropriate send and receive provenance events
 2015-11-04 18:01:59 -05:00
Tony Kurc 5611dac3f8 NIFI-274 - added use of Selectors for TCP and UDP connections. Added a max connections to the TCP thread 
- Added comments and code review changes
         - fixed fixbugs bug
 2015-11-04 18:00:18 -05:00
Bryan Bende 9c542432da NIFI-274 Adding ListenSyslog and PutSyslog to standard processors. 
- Refactoring connection handling on put side, removing number of buffers from properties and basing it off concurrent tasks for the processor.
- Refactoring some of the TCP handling so it keeps reading from a connection until the client closes it
- Adding an error queue
- Adding a sender field on the syslog event to record the system that sent the message
 2015-11-04 17:56:13 -05:00
Mike Moser 6add372bc1 NIFI-1085 WebClusterManager fair locking and HeartbeatMonitoringTimerTask scheduling 2015-11-04 12:19:00 -05:00
Matt Gilman 201eac052b NIFI-1098 - Ensuring we pull the correct attribute from the servlet context. Reviewed by Tony Kurc (tkurc@apache.org) 2015-11-04 12:11:19 -05:00
Mark Payne fedbe40313 NIFI-1103: Added InputRequirement annotation 2015-11-04 11:11:33 -05:00
Mark Payne 2b1d093120 NIFI-1105: Only trigger a processor that requires input to run if data is available for it process 2015-11-04 08:41:10 -05:00
Mark Payne dbf0c7893f NIFI-730: Completed merging of master 2015-11-02 14:31:28 -05:00
Mark Payne e4cebba3c7 Merge branch 'master' into NIFI-730 2015-11-02 14:02:16 -05:00
Joseph Percivall a549621267 NIFI-1077 enabling expression language on ConvertCharacterSet input and output 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-11-02 11:46:17 -05:00
Mark Payne 814e8b212c Merge branch 'NIFI-1082' 2015-11-02 11:29:52 -05:00
Mark Payne abf2c62884 NIFI-1051: Fixed checkstyle violations 2015-11-02 11:01:50 -05:00
Mark Payne 7a73867c46 Merge branch 'NIFI-1051' of https://github.com/olegz/nifi into NIFI-1051 2015-11-02 10:51:43 -05:00
Joseph Percivall 2ae49026e8 NIFI-883 Fixing issue HandleHttpRequest had with PrimaryNodeOnly scheduling 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-11-02 10:32:14 -05:00
Matt Gilman 5a04021dd7 NIFI-730: 
- Updating logic now that original is guaranteed to be non null.
- Always reporting 100% once the drop request has completed.
 2015-11-02 08:59:11 -05:00
Mark Payne f5727cfb0f NIFI-730: Ensure that we always populate queue counts when initiating a Drop FlowFile request 2015-11-02 08:36:50 -05:00
Mark Payne ad849c77df NIFI-1090: Fixed log message that was at info level but should have been debug level 2015-11-01 14:37:01 -05:00
Mark Payne b729bf4c19 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2015-11-01 14:16:54 -05:00
Mark Payne 6e193dffa3 Merge branch 'NIFI-1088' 2015-11-01 14:16:35 -05:00
Joseph Percivall cef7b6c736 NIFI-1080 fleshed out InvokeHttp unit tests 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2015-10-31 00:19:36 -04:00
Mark Payne 9c148da1c2 NIFI-869: Fixed checkstyle violations 2015-10-30 16:09:50 -04:00
Mark Payne ba72452f66 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2015-10-30 16:02:15 -04:00
Oleg Zhurakousky a3d43d23dc NIFI-869 Fixed SimpleProcessLogger to log correct messages Ensured that SimpleProcessLogger correctly interprets Throwable as discussed in JIRA Added tests 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-10-30 15:52:33 -04:00
Oleg Zhurakousky 90aea01350 NIFI-869 Fixed formatting issue Fixed formatting issue with printed error message which only apears when NiFi is cnfigured using Logback. Please see NIFI-869 for more details 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-10-30 15:52:33 -04:00
Jeremy Dyer eb389cf84c NIFI-1079 Replacing Name for NULL_VALUE_FOR_EMPTY_STRING as I had accidentally removed it. 
Signed-off-by: Bryan Bende <bbende@apache.org>
 2015-10-30 14:30:56 -04:00