Apache
/
nifi
mirror of https://github.com/apache/nifi.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,569 Commits 143 Branches 197 Tags 400 MiB
Commit Graph

288 Commits

Author SHA1 Message Date
Mark Payne e862f7ff03 NIFI-1165: Use FileChannel instead of RandomAccessFile in order to avoid locking files in Windows 
Reviewed by Tony Kurc (tkurc@apache.org)
 2015-11-19 01:01:28 -05:00
Joseph Percivall 1e5cc070a3 NIFI-1081 Adding option to ExecuteStreamCommand to put output value to an attribute 
Reviewed and amended (comments,whitespace,and some code readability (discussed in ticket)) by Tony Kurc (tkurc@apache.org)
 2015-11-18 23:23:10 -05:00
Mark Payne 69bce2c2db NIFI-1168: Ensure that processors with only looping 
connections are scheduled to run, even if the connections have no FlowFiles;
 expose these details to processor developers; update documentation

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2015-11-18 14:53:30 -05:00
Aldrin Piri 773576e041 NIFI-1108 Providing additional annotations on processors to ensure utilization of the InputRequirement annotation. 2015-11-18 13:56:21 -05:00
Mark Payne 911e1c5412 NIFI-1108: Updated processors to include the @InputRequirement annotation 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2015-11-18 13:47:59 -05:00
Tony Kurc 528dab78d6 NIFI-1073: Fixing coverity discovered errors. Resource leaks, and statics 
Reviewed by Bryan Bende (bbende@apache.org)
 2015-11-14 18:56:43 -05:00
Aldrin Piri 01539ed323 NIFI-1163: Providing handling of SSLContext creation in GetHTTP in case of only performing a one-way SSL request and accompanying test to verify the configuration/usage. 
Reviewed by Tony Kurc (tkurc@apache.org)
 2015-11-14 18:47:50 -05:00
Tony Kurc e6086420aa NIFI-1158 Default timestamp based on now 2015-11-12 23:11:43 -05:00
Mark Payne 0900fb80c9 NIFI-1153: If no incoming FlowFile, don't try to transfer null 
Reviewed by Bryan Bende (bbende@apache.org)
Amended based on review (change to a log message) by Tony Kurc (tkurc@apache.org)
 2015-11-12 22:41:57 -05:00
joewitt 02102ea1c2 Revert "NIFI-1134". This was removed because they looked like floating classes. But in reality they are used to make jars which we use to test the build. However, we really need the test to work differently but until that is fixed these must stay. 
This reverts commit 41f3875347.
 2015-11-12 18:00:40 -05:00
Mark Payne 7a165b62cc Merge branch 'NIFI-994' 2015-11-11 16:05:40 -05:00
Mark Payne e7c6c7cae9 NIFI-1083: Added a Grouping Regular Expression property for grouping lines of text 2015-11-11 10:19:07 -05:00
Joseph Percivall 3e538d9007 NIFI-1083 Created a processor that routes lines of text based on different matching and routing strategies 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-11-11 09:12:47 -05:00
Joseph Percivall d887d8fdf0 NIFI-993 added expression language support to GetHTTP for the filename and URL properties 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-11-11 08:05:23 -05:00
Mark Payne 7b9c8df6c5 NIFI-994: Fixed issue that could result in data duplication if more than 1 rollover of tailed file has occurred on restart of Processor 2015-11-10 16:53:29 -05:00
Mark Payne 72008035b1 Merge branch 'NIFI-1000' 2015-11-10 15:02:02 -05:00
Mark Payne 8f942fcdcc NIFI-1000: Fixed checkstyle violation in TestGetJSMQueue use activemq-broker with test scope and activemq-client with default scope rather than pulling in activemq-all; this reduces size of standard nar from 30 MB to 18 MB 2015-11-10 14:31:20 -05:00
Oleg Zhurakousky 8699e35108 NIFI-1000 addressed PR comments 2015-11-10 12:13:16 -05:00
Oleg Zhurakousky ef0be5a5d6 NIFI-1000 Fixed JmsFactory to properly obtain destiniation name 
Re-enabled JMS Tests that were annotated with @Ignore
 2015-11-09 18:35:31 -05:00
joewitt 41f3875347 NIFI-1134 2015-11-09 15:37:40 -05:00
joewitt 99629646fe NIFI-1122 moved to 0.4.0-SNAPSHOT 2015-11-06 23:41:15 -05:00
Bryan Bende 618f22e110 NIFI-274 - Fixing TestListenSyslog, fixing default buffer size to be bytes, adding syslog.protocol to attributes 
- Adding syslog.port to ListenSyslog attributes, logging at warn level when rejecting tcp connections
         - Adding @InputRequirement to processors and adding appropriate send and receive provenance events
 2015-11-04 18:01:59 -05:00
Tony Kurc 5611dac3f8 NIFI-274 - added use of Selectors for TCP and UDP connections. Added a max connections to the TCP thread 
- Added comments and code review changes
         - fixed fixbugs bug
 2015-11-04 18:00:18 -05:00
Bryan Bende 9c542432da NIFI-274 Adding ListenSyslog and PutSyslog to standard processors. 
- Refactoring connection handling on put side, removing number of buffers from properties and basing it off concurrent tasks for the processor.
- Refactoring some of the TCP handling so it keeps reading from a connection until the client closes it
- Adding an error queue
- Adding a sender field on the syslog event to record the system that sent the message
 2015-11-04 17:56:13 -05:00
Mark Payne fedbe40313 NIFI-1103: Added InputRequirement annotation 2015-11-04 11:11:33 -05:00
Joseph Percivall a549621267 NIFI-1077 enabling expression language on ConvertCharacterSet input and output 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-11-02 11:46:17 -05:00
Joseph Percivall 2ae49026e8 NIFI-883 Fixing issue HandleHttpRequest had with PrimaryNodeOnly scheduling 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-11-02 10:32:14 -05:00
Joseph Percivall cef7b6c736 NIFI-1080 fleshed out InvokeHttp unit tests 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2015-10-31 00:19:36 -04:00
Jeremy Dyer eb389cf84c NIFI-1079 Replacing Name for NULL_VALUE_FOR_EMPTY_STRING as I had accidentally removed it. 
Signed-off-by: Bryan Bende <bbende@apache.org>
 2015-10-30 14:30:56 -04:00
Jeremy Dyer aef0d8fe9b NIFI-1079 Check style and documentation based updates recommended by user bbende on Github pull request 
Signed-off-by: Bryan Bende <bbende@apache.org>
 2015-10-30 14:30:42 -04:00
Jeremy Dyer 217b1049cf NIFI-1079 Added Destination Property to control if JSON goes to Attribute or 
Content of FlowFile. Added Include Core Attributes Property to control
if FlowFile CoreAttributes are included in the JSON output or not.
Added Null value for Empty String flag to control if empty values in
the JSON are empty string or true NULL values. Added more tests and
minor text refactoring per Github comments

Signed-off-by: Bryan Bende <bbende@apache.org>
 2015-10-30 14:30:25 -04:00
Jeremy Dyer 19b7a4cc7d NIFI-1079 Create a new Attribute from the existing FlowFile attributes by taking 
either all of the existing attributes or a user defined list. The
existing Attributes are converted to JSON and placed in a new Attribute
on the existing FlowFile as Attribute “JSONAttributes”

Signed-off-by: Bryan Bende <bbende@apache.org>
 2015-10-30 14:29:29 -04:00
Mark Payne bfa9e45079 NIFI-994: Ensure that processor is not valid due to the tail file not yet existing 2015-10-30 11:44:51 -04:00
Mark Payne dc4004de64 Merge branch 'NIFI-977' 2015-10-27 16:53:38 -04:00
Mark Payne af19053a7f Merge branch 'NIFI-447' 2015-10-27 10:42:46 -04:00
Joseph Percivall 07238c7913 NIFI-447 adding replacement strategy to ReplaceText processor 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-10-27 10:37:43 -04:00
Joseph Percivall b885f955f4 NIFI-516 adding option to StandardProcessSession.read to close stream 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-10-26 20:23:13 -04:00
Mark Payne 1c1738670c NIFI-10: Updated FetchHDFS, FetchFileTransfer to use new FETCH provenance event 2015-10-26 17:14:29 -04:00
Mark Payne 51f564024a NIFI-10: Added FETCH Provenance Event and updated processors to use this new event type 2015-10-26 14:58:50 -04:00
Luke Williamson 26edab3185 NIFI-908 Added support for SSL in JMS connections. 
- Added SSL context to JMS producer and consumer processors
 - Tony Kurc Amended patch to check SSL need by scheme and exception consistency
Reviewed by Tony Kurc (tkurc@apache.org)
 2015-10-26 00:29:05 -04:00
Mark Payne 8d2f9bc64b NIFI-1056: Do not treat BigDecimal, BigInteger the same way we treat Number's when serializing to Avro because Avro Java Library doesn't support them 
Reviewed and Unit Test Amended by Tony Kurc (tkurc@apache.org)
 2015-10-25 11:31:30 -04:00
Mark Payne 385bfbb2c6 NIFI-673: Rebased from master; Added InputRequirement annotation, as it is now merged into master 2015-10-25 11:14:11 -04:00
Mark Payne 4e38288062 NIFI-673: Added sftp.listing.user attribute to FlowFiles created by ListSFTP; ensure that FetchSFTP indicates that the username supports Expression Language 2015-10-25 11:13:02 -04:00
Mark Payne b0322d9ffe NIFI-673: Added Completion Strategy to FetchSFTP 2015-10-25 11:13:02 -04:00
Mark Payne d1d57931bf NIFI-673: Initial implementation of ListSFTP, FetchSFTP 2015-10-25 11:13:02 -04:00
Mark Payne 0636f0e731 NIFI-810: Merged master into branch 2015-10-25 11:02:40 -04:00
Mark Payne d63cd6bd2f Merge branch 'NIFI-1010' 2015-10-23 14:39:16 -04:00
Mark Payne 88fc8d28a0 NIFI-1010: If database driver does not support getting table name for column from ResultSetMetadata, then just use a default name 2015-10-23 10:25:59 -04:00
Mark Payne 0fc5d30461 NIFI-1055: Fixed checkstyle violations 2015-10-23 10:08:44 -04:00
Mark Payne bd506b1e10 NIFI-972: Added additional unit test; deleted lines that were commented out 2015-10-23 09:52:09 -04:00
Toivo Adams a9e5325047 NIFI-972 attribute to indicate rows count and cleanup 
Signed-off-by: Toivo Adams <toivo.adams@gmail.com>
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-10-23 09:28:03 -04:00
Toivo Adams ba3225fe92 NIFI-972 ExecuteSQL bug in createSchema() create Arvo Schema 1 
Signed-off-by: Toivo Adams <toivo.adams@gmail.com>
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-10-23 09:28:03 -04:00
Mark Payne da28b81eec Merge branch 'NIFI-988' 2015-10-21 10:44:06 -04:00
Mark Payne 97441ea0c2 Merge branch 'NIFI-988' of https://github.com/ImpressTV/nifi into NIFI-988 2015-10-21 10:18:05 -04:00
Andre F de Miranda 518670dbf6 NIFI-1042 Adds restlistener.remote.source.ip attribute to ListenHTTPServlet. This closes #105 2015-10-19 23:43:44 -04:00
Tony Kurc 88b1b844fb NIFI-747 This closes #104. PR from Venkatesh Sellappa <VS186031@outlook.com> was modified, then code reviewed by Joe Witt (comments in ticket) 2015-10-18 20:04:24 -04:00
Mark Payne 31f0909bd3 NIFI-994: Initial import of TailFile 2015-10-18 19:00:24 -04:00
Joe 22924c656b NIFI-995 TestGetFile.testAttributes() failed in case of a NTFS partition in Linux, because Files.setPosixFilePermissions() did not have any effect on the files and did not throw Exception. closes #95. Signed off by Tony Kurc (tkurc@apache.org) 2015-10-17 10:46:49 -04:00
Mark Payne 84db372538 NIFI-977: Allow SQL Data Types with numerals that are negative 2015-10-14 13:12:10 -04:00
Joseph Percivall b4bfcc1f21 NIFI-1030 made InvokeHttp basic auth password a sensitive value 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2015-10-08 15:46:12 -04:00
Mark Payne ccfb57fe9f NIFI-810: Addressed several checkstyle violations 2015-10-07 17:48:51 -04:00
Mark Payne 4afd8f88f8 NIFI-810: Created RequiresInput annotation and ensure that processors are invalid if connections do not agree 2015-10-07 17:26:14 -04:00
Mark Payne 96764ed6a1 Merge branch 'master' of http://git-wip-us.apache.org/repos/asf/nifi 2015-09-25 09:15:34 -04:00
Mark Payne eb94dab00c NIFI-604: Fixed checkstyle violation 2015-09-24 11:51:44 -04:00
Joe e12da7c9a3 NIFI-985: Custom log prefix for LogAttribute processor 
The LogAttribute processor evaluates the log prefix EL using the current flow file.

Log prefix helps to distinguish the log output of multiple LogAttribute processors and identify the right processor. Log prefix appears in the first and the last log line, followed by the original 50 dashes. If you configure log prefix 'STEP 1: ' the log output looks like this:

STEP 1 :  --------------------------------------------------
Standard FlowFile Attributes
Key: 'entryDate'
        Value: 'Tue Sep 22 15:13:02 CEST 2015'
Key: 'lineageStartDate'
        Value: 'Tue Sep 22 15:13:02 CEST 2015'
Key: 'fileSize'
        Value: '9'
FlowFile Attribute Map Content
Key: 'customAttribute'
        Value: 'custom value'
STEP 1 :  --------------------------------------------------

flow file content...

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2015-09-24 10:31:04 -04:00
Mark Payne 24c080788f NIFI-604: Removed unused import, as it caused checkstyle violation 2015-09-24 10:06:39 -04:00
Mark Payne 193dbe6533 Merge branch 'NIFI-604' of https://github.com/rickysaltzer/nifi into NIFI-604 2015-09-23 16:03:46 -04:00
Joe ee7d89cb01 NIFI-988: Test cases for PutDistributedMapCache 2015-09-23 14:32:37 +02:00
Joe 6b1328f3f1 NIFI-988: PutDisributedMapCache processor implementation 2015-09-23 14:31:13 +02:00
Bryan Bende 5764e89293 NIFI-975 Fixing NullPointerException when one of the delimiters is not provided when using the Text strategy 2015-09-22 13:19:15 -04:00
Joe 14eaeeb1ee NIFI-984: Bad class used for logging in TestDetectDuplicate 
Using TestListenUDP instead of TestDetectDuplicate when creating the logger.

This closes #89

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2015-09-22 08:51:58 -04:00
Bryan Bende 600f91a262 NIFI-932 Adding hasIncomingConnection() and hasConnection(Relationship) to ProcessContext and updating ExecuteSQL 2015-09-18 17:05:31 -04:00
Matt Gilman ded74ec94c NIFI-876 prepare for next development iteration 2015-09-14 21:48:11 -04:00
Matt Gilman 2ec735e350 NIFI-876 prepare release nifi-0.3.0-RC1 2015-09-14 21:48:00 -04:00
Matt Gilman cd2e1424cb NIFI-911: 
- Updating default value for Regex so it matches once (?s:^.*$) instead of twice (.*). Matching on .* results in matching for every character and then again for 0 characters.
 2015-08-31 14:58:38 -04:00
ricky 9cefc4a5a5 NIFI-604: Custom Argument Delimiters ExecuteStreamCommand / ExecuteProcess 
- Unified the way ExecuteStreamCommand and ExecuteProcess handle arguments
- Argument delimiters can now be specified. Their default being what they were using before (; and space)
 2015-08-27 15:26:56 -04:00
Alan Jackoway c7f7704220 Changed line wrap length to 200 and combined adjacent imports into single try-with-resources blocks. 
This closes #76

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2015-08-23 23:07:50 -04:00
Alan Jackoway b30fbe1f3b NIFI-888 EncryptContent for PGP Leaks File Handles. 
Closes InputStreams created to read the public keys for PGP encryption and several other
streams involved in PGP encryptiong. This prevents NiFi from leaking file handles on
every validate call or encryption attempt in the EncryptContent processor.
 2015-08-23 23:07:26 -04:00
Mark Payne 4d4f74cf05 NIFI-889: Return the failure relationshp in the set of allowed relationships 2015-08-22 20:47:45 -04:00
Mark Payne 671445f10a Merge branch 'NIFI-844' 2015-08-21 17:15:39 -04:00
Mark Payne a9c2bf58b8 NIFI-844: Initial import of the ConvertAvroToJSON processor 2015-08-21 17:11:56 -04:00
Mark Payne 82d32230e6 NIFI-881: Added support for Proxy Host and Proxy Port properties to InvokeHTTP 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2015-08-21 11:35:30 -04:00
Bryan Bende d421e3c242 NIFI-846 MonitorActivity not setting start of evaluation period correctly 2015-08-21 11:06:20 -04:00
Mark Payne 28549c2b1e NIFI-853: Made updates to processors 2015-08-19 17:36:29 -04:00
Mark Payne 1a37c95f58 NIFI-293: Added processors ConvertFlatJSONToSQL, PutSQL 2015-08-19 17:36:29 -04:00
Aldrin Piri f044ba5d45 NIFI-808 Providing property to allow exclusion of capture group 0 for any regex expressions as an attribute. 2015-08-19 12:00:25 -04:00
Aldrin Piri 740b09b182 NIFI-796 Adjusting ControlRate documentation to reflect that a long is used instead of an integer. 2015-08-17 11:51:10 -04:00
joewitt aa99884782 NIFI-850 removed nifi parent, updated nifi pom, moved all nifi subdirs up one level, fixed readme. 2015-08-15 13:12:22 -04:00