Commit Graph

301 Commits

Author SHA1 Message Date
Daniel Cave 171b9c4e94 NIFI-1093 - Added settings for processing JSON documents when validating against database columns to fail, log a warning, or ignore unmatched columns. Added tests for new settings.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2016-02-01 11:45:15 -05:00
joewitt 1bf7a585b2 NIFI-1456 This closes #198. reduced tests to 10MB instead of 100MB datasets and resolved IO issues which impact test stability 2016-01-31 21:56:53 -05:00
Richard Miskin 8ef337b65e NIFI-1434 Prevent array index exception in PutEmail
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-01-30 10:56:31 -05:00
Oleg Zhurakousky 0126800201 NIFI-1093 added support for handling non-required columns
Mostly based on contribution from Daniel Cave
2016-01-29 13:41:27 -05:00
jpercivall be27c07c7f NIFI-1382 Created FetchDistributedMapCache processor 2016-01-29 12:12:23 -05:00
Mark Payne a0224f1b4c NIFI-1431: Added documentation about how ETag and LastModified dates are used by the Processor
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-28 16:02:05 -05:00
jpercivall ee30b27cc8 NIFI-1396 Correcting PostHttp chunked encoding functionality 2016-01-28 14:50:22 -05:00
Matt Burgess aa6bc5d505 NIFI-1409: Fix missing transfer on error in ExecuteSql
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-01-27 18:23:32 -05:00
Andy LoPresto 93aac8cff3 NIFI-1365
Added Groovy support for unit tests to pom with skeleton test.
Added Groovy unit tests for OCSPCertificateValidator.
Implemented positive & negative unit tests with cache injection for valid/revoked OCSP certificate.
Modified pom.xml to support Groovy unit tests with custom variable.

mvn clean test -Dgroovy=test

Added local cache injection into Groovy tests for OCSP certificate validation (see NIFI-1324 and NIFI-1364).
Set Java version to 1.7 for Groovy test src/target.
Moved Groovy unit test profile from nifi-web-security to root pom.
Added null check for algorithm argument in PGPUtil.
Changed buffer length check from ">= 0" to "> -1" because it was confusing other developers.
Resolved contrib-check line length issues.
Fixed contrib-check issues in OpenPGPKeyBasedEncryptorTest.
This closes #163

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-26 17:37:59 -05:00
Mark Payne 0e92607466 Merge branch 'NIFI-1412' of https://github.com/mattyb149/nifi into NIFI-1412 2016-01-26 12:30:19 -05:00
Richard Miskin c16c8db459 NIFI-1421 Make namespace ordering deterministic in SplitXML 2016-01-26 05:35:07 +00:00
Bryan Bende b7f7e6ed80 NIFI-1436 This closes #189. Combining stop() and close() into a single method to simplify, and adding checks on stopped flag in the run method of SocketChannelDispatcher and DatagramChannelDispatcher to ensure the run() method exits as soon as possible upon close() being called
NIFI-1436 Adding synchronization on keys set in close() method  based on Selector JavaDoc

Signed-off-by: joewitt <joewitt@apache.org>
2016-01-25 22:50:25 -05:00
joewitt beb92492be NIFI-1421 This closes #184. Added content assertion to test and removed extra space
Signed-off-by: Mark Payne <markap14@hotmail.com>
2016-01-25 16:04:47 -05:00
Mark Payne 4a5fb37b33 Merge branch 'NIFI-1421' of https://github.com/rpmiskin/nifi into NIFI-1421 2016-01-25 15:38:08 -05:00
Matt Burgess 91e7ba84e0 NIFI-1412: Fix error during Avro conversion where no fields are present 2016-01-25 08:35:14 -05:00
Aldrin Piri 8d37af07b9 NIFI-1381 Removing the hardcoded jms:// prefix and instead deferring to the URI specified by the processor properties of PutJMS
Signed-off-by: joewitt <joewitt@apache.org>
2016-01-24 23:10:47 -05:00
jpercivall 0db2dc9fc3 NIFI-1405 Adding option to invokeHttp to control using chunked encoding for http PUT and POST calls
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-24 15:52:05 -05:00
Richard Miskin dfed7e727d NIFI-1421 Added missing copyright notice to new test input 2016-01-23 08:52:54 +00:00
Mark Payne 93e2c52195 NIFI-1415: Do not enfoce that a password or Private Key Passphrase is set
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-22 20:46:14 -05:00
Mark Payne 92e6961b50 NIFI-1416: If FetchSFTP's Remote File has a directory name in it, do not include that as part of the 'filename' attribute but instead add a 'path' attribute
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-22 18:15:19 -05:00
Bryan Bende 1089f0a956 NIFI-1273 Adding ListenRELP processor which includes refactoring code that was previously part of ListenSyslog into a reusable framework for implementing listener processors, as well as back-end code for handling the RELP protocol. Addressing some feedback from review and fixing logging statements Addressing review feedback, added AsyncChannelDispatcher and made ChannelHandlerFactory use generics for the dispatcher This closes #179
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-22 16:28:58 -05:00
Oleg Zhurakousky 3db6fffa68 NIFI-1378: fixed JMS URI validation simplified JmsFactory check for SSL and scheme-less URIs ensured URI validation is handled by ActiveMqConnectionFactory ensured informative error messages are shown to the userm removed unneeded test, cleaned up unused imports
Reviewed by Tony Kurc (tkurc@apache.org). This closes #167
2016-01-21 22:29:50 -05:00
Richard Miskin 68a9375f3e NIFI-1421 Update SplitXML to support namespace declarations 2016-01-21 22:19:15 +00:00
Mark Payne f6ec437bc7 Merge branch 'master' into NIFI-259 2016-01-21 09:32:02 -05:00
Aldrin Piri bc75dd3c17 NIFI-1406 Collecting the number of bytes sent regardless if one or multiple FlowFiles are sent.
Reviewed by Tony Kurc (tkurc@apache.org). This closes #176
2016-01-20 21:08:15 -05:00
Mark Payne 348831b20c Merge branch 'NIFI-1414' 2016-01-20 17:31:34 -05:00
smarthi e1588d4136 [NIFI-1411] - TestJdbcTypesDerby: java.sql.SQLSyntaxErrorException: TYPE 'DATETIME' does not exist 2016-01-20 16:51:58 -05:00
Aldrin Piri 9ebcc9e4fa NIFI-1393 Providing the ability to send using gzip Content-Encoding in PostHTTP if the endpoint server supports it regardless if the processor is configured to send as a FlowFile This closes #175
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-20 16:46:57 -05:00
Mark Payne d696391f76 NIFI-259: Update ZooKeeperStateServer so that the thread starting it doesn't block indefinitely, as this was causing NiFi to not shut down properly; applied patch for NIFI-1415 2016-01-20 11:52:41 -05:00
Mark Payne 7ef8af9eea NIFI-259: Applied patch for NIFI-1414 2016-01-20 09:14:04 -05:00
Mark Payne 1845f6df8d NIFI-1414: Ensure that the DirectoryStream is closed after streaming contents of a directory 2016-01-20 09:12:14 -05:00
Jeremy Dyer 8f688d4921 NIFI-1360 Add expression language support for "MergeContent" "Correlation Attribute Field" property
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-19 10:46:37 -08:00
Mark Payne d6a21537c1 NIFI-259: Do not require the State File anymore, since state management is being used now 2016-01-19 11:03:31 -05:00
smarthi 9e9182aa63 - Unit test creates resources in HOME directory This closes #174
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-19 09:44:03 -05:00
Mark Payne 16dcf4595c Merge branch 'master' into NIFI-259 2016-01-18 08:48:30 -05:00
Mark Payne eba25ecaca NIFI-259, NIFI-1339: Added OnConfigurationRestored annotation, always invoke onPropertyModified even on restart when properties are changed from defaults, as was done previously 2016-01-17 14:40:27 -05:00
Jeremy Dyer ecb81ec113 NIFI-1362 Set mime.type attribute on response FlowFile based on InvokeHTTP response Content-Type
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-15 16:40:36 -05:00
Mark Payne 0cd6f80f36 NIFI-259: Bug fixes 2016-01-13 15:11:53 -05:00
Mark Payne 0151b1eed1 NIFI-259: Fixed checkstyle violations 2016-01-13 13:57:47 -05:00
Mark Payne d39067ede6 NIFI-259: Update GetHBase to use new State Management feature; updated docs; bug fixes 2016-01-13 13:30:05 -05:00
Mark Payne bbce596d74 NIFI-259: Updated GetHBase to use new State Management; bug fixes; updated docs 2016-01-13 12:47:08 -05:00
Andy LoPresto c757c4d6ea NIFI-1385: Resolved test for missing file which was failing on Windows due to differing error message.
Reviewed by Tony Kurc (tkurc@apache.org)
2016-01-12 19:16:29 -05:00
Mark Payne 774c29a4da NIFI-259: Added Stateful annotation as described on ticket 2016-01-12 15:28:35 -05:00
Mark Payne b4bc105ea9 Merge branch 'nifi-1283' of https://github.com/jvwing/nifi 2016-01-12 10:49:04 -05:00
Mark Payne f6b34cbbc5 NIFI-259: Removed bad unit test 2016-01-12 10:40:00 -05:00
Mark Payne e559c68766 NIFI-259: Began updating processors to use new state management 2016-01-12 10:21:47 -05:00
Matt Gilman 44ffddd8b9 NIFI-1324:
- Addressing lengthy lines contrib-check issue.
- This closes #162
2016-01-12 09:42:31 -05:00
Andy LoPresto ffbfffce6d NIFI-1324:
Changed Maven dependencies for BouncyCastle bcprov and bcpg from jdk16:1.46 to jdk15on:1.53 (kept nifi-web-security on jdk16:1.46 because jdk15on:1.53 splits OCSP logic into new module bcpkix).
Added individual unit tests for PGP public keyring validation.
Passes all legacy unit tests.
Added TODOs for customizable brick encryption and refactoring shared code.
Cleaned up magic numbers to constants.
Added unit tests for OpenPGPPasswordBasedEncryptor (internal consistency and legacy file decrypt).
Began refactoring shared encrypt code from OpenPGP* implementations.
Extracted encrypt utility method from OpenPGPPasswordBasedEncryptor to PGPUtil class.
Added test resources (signed and unsigned key-encrypted files).
Added unit tests for OpenPGPKeyBasedEncryptor (internal consistency and external file decrypt).
Changed BC dependency for nifi-web-security to bcprov-jdk15on:1.53 and bcpkix-jdk15on:1.53.
Updated OCSPValidator to use new BC logic for OCSP validation. This code compiles but should be fully audited, as the legacy OCSP validation was not completely implemented.
Added skeleton of OCSP validator unit tests with successful keypair and certificate generation and signing code.
Added further unit tests for issued certificates.
Annotated unimplemented unit tests with note about Groovy integration.
Refactored Jersey call in OCSPCertificateValidator to internal method.
Added toString() to NiFi local OcspRequest.
Implemented positive & negative unit tests with cache injection for valid/revoked OCSP certificate.
Resolved contrib-check issues.
Removed ignored code in unit test.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-12 09:22:51 -05:00
jpercivall 6b54753dbb NIFI-1316 adding option to DetectDuplicate to not cache the entry identifier
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-11 22:59:24 -05:00
James Wing c526656a22 NIFI-1283 Fixing ControllerStatusReportingTask loggers to use fully-qualified class name 2016-01-11 13:39:15 -08:00
Mark Payne d2a969e3d6 NIFI-259: Initial implementation of State Management feature 2016-01-11 16:38:52 -05:00
joewitt 0c68e2c3a8 NIFI-1378 added validator to ensure the JMS URI has a scheme set
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-10 16:07:56 -05:00
Aldrin Piri 317b2f4a4a NIFI-1361 Exposing Content-Type as a processor property for both InvokeHTTP and PostHTTP. This makes use of the mime.type attribute via the EL expression ${mime.type}, making more apparent to the user how this header is derived and allowing the explicit setting of a value. 2016-01-08 10:43:14 -05:00
jpercivall 6c46f22b62 NIFI-1327 NIFI-1353 Updating version of OkHttp and OkHttp-Digest 2016-01-07 15:51:49 -05:00
Aldrin Piri 47453d2c47 NIFI-1145 Providing delays in tests where resources are being initialized to allow heavily taxed environments sufficient time. 2016-01-07 12:05:23 -05:00
Andy LoPresto 8f0e7a1ac5 NIFI-1354
Generated new self-signed RSA-2048 key for FQDN localhost in keystore and exported to truststore.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-06 18:16:37 -05:00
Joe Skora de2dd93f26 NIFI-1300 Corrects inconsistent error behavior in PutJMS and added PutJMS unit testing
Reviewed by Tony Kurc (tkurc@apache.org) This closes #145
2016-01-04 00:26:21 -05:00
gresockj 34bd2061f7 NIFI-1291: Adding BinFiles in nifi-processor-utilities, deprecating version in nifi-standard-processors. This closes #155.
Signed-off-by: joewitt <joewitt@apache.org>
2016-01-03 23:05:28 -05:00
Aldrin Piri c696cb09b6 NIFI-1212 Treating text/csv files as plain text so they may also be displayed. This closes #154.
Signed-off-by: joewitt <joewitt@apache.org>
2016-01-03 16:12:06 -05:00
ricky 236a2b72b6 NIFI-1335: Allow UnpackContent to Handle application/x-tar mime.type
- Add additinal `mime.type` for tar archives (application/x-tar)
- Update `UnpackContent` tests to verify auto-detect format property
2016-01-03 11:42:18 -05:00
joewitt e16a9a0c68 NIFI-1343 added unit tests to verify behavior 2016-01-01 14:08:46 -05:00
Mark Payne 02b325b07e NIFI-1343: Make comparisons case sensitive if configured to do so. This closes #153.
Signed-off-by: joewitt <joewitt@apache.org>
2016-01-01 13:34:07 -05:00
James Harrison 03730adb3f NIFI-1338 - Clarifying documentation for InvokeHTTP
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-31 13:48:00 -05:00
Bryan Bende 3189a13da1 NIFI-1328 Explicitly setting the parameters based on type in PutSQL, falling back to setObject()
This closes #146
2015-12-23 13:08:56 -05:00
Bryan Bende 8d46041a13 NIFI-1319 Updating JdbcCommon to check meta.isSigned(i) to determine if Avro schema should use a long or int
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-12-23 10:03:30 -05:00
joewitt f4ac8d75c5 NIFI-1312-RC1 prepare for next development iteration 2015-12-19 00:41:04 -05:00
joewitt d624ea4866 NIFI-1312-RC1 prepare release nifi-0.4.1-RC1 2015-12-19 00:40:53 -05:00
Mark Payne bd4f31a4c2 NIFI-1279: Take the FlowFile's 1-up id into account for the newest and oldest flowfile prioritizers
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-18 12:08:31 -05:00
joewitt d755e43ec8 NIFI-1122_nifi-0.4.0-RC2prepare for next development iteration 2015-12-08 13:00:10 -05:00
joewitt b66c029090 NIFI-1122_nifi-0.4.0-RC2prepare release nifi-0.4.0-RC2 2015-12-08 12:59:59 -05:00
Mark Payne d1395ea7ee NIFI-1269: Ensure that we roll back session if not transferring FlowFile
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-07 21:18:33 -05:00
Mark Payne 0c7617b686 NIFI-1267: TestListFile attempts to ensure that files that are not readable are skipped but the assumptions made are not valid if running as root; removed this check, since it is not reliable
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-07 20:42:07 -05:00
Mark Payne 46623c2bf3 Merge branch 'NIFI-1261' 2015-12-07 15:13:12 -05:00
Mark Payne bfe6fad502 NIFI-1261: Use consistent file separators for ListFile / TestListFile based on what the Operating System uses 2015-12-07 10:19:49 -05:00
Mark Payne bd3fd870ac NIFI-1263: Windows considers last mod time on both rolled over files to be the same, due to the way that windows computes the lastMod time. On linux and os x it is done differently. This is easily solved by adding a small sleep to the unit test before 'rolling over' the file so that the last mod date is different on all OS's, which yields the same behavior on all OS's 2015-12-07 09:53:33 -05:00
Andy LoPresto f83e6d33c5 NIFI-1242:
Added logic and test resources to debug JCE unlimited strength cryptography policy issues and incorporated into processor property validation.
Excluded test resources from RAT check
Added KeyDerivationFunction enum.
Added kdf property in EncryptContent processor and provided to PasswordBasedEncryptor.
Added logic in PasswordBasedEncryptor to handle variable KDF.
Added unit tests for EncryptContent processor.
Added test resources and excluded from RAT check.

plain.txt: This is a plaintext message.

0s @ 12:20:32 $ openssl enc -aes-256-cbc -e -in plain.txt -out salted_raw.enc -k thisIsABadPassword -p
salt=31DC301A6C7B8A0B
key=CB878A6E167A5B530B8F2BD175E6359E3092AFF7C83274A22A5B421D79E599AC
iv =0C614A72FC06B454B84E035B3FA8F877
0s @ 12:20:44 $ xxd salted_raw.enc
0000000: 5361 6c74 6564 5f5f 31dc 301a 6c7b 8a0b  Salted__1.0.l{..
0000010: 616b c65d f767 504d c085 ba7a c517 d0cb  ak.].gPM...z....
0000020: 7832 211e f573 b6f1 ded2 8f59 88e8 088f  x2!..s.....Y....

0s @ 20:14:00 $ openssl enc -aes-256-cbc -e -in plain.txt -out unsalted_raw.enc -k thisIsABadPassword -p -nosalt
key=711E85689CE7AFF6F410AEA43ABC5446842F685B84879B2E00F977C22B9E9A7D
iv =0C90ABF8ECE84B92BAA2CD448EC760F0
0s @ 20:14:17 $ xxd unsalted_raw.enc
0000000: 70cd 2984 fdbb 0e7c c01b 7206 88b1 6b50  p.)....|..r...kP
0000010: 5eeb e4f3 4036 773b 00ce dd8e 85d8 f90a  ^...@6w;........

This closes #140

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-12-04 19:45:10 -05:00
Mark Payne ee14d8f9dd Merge branch 'NIFI-1249' 2015-12-04 16:38:49 -05:00
Mike Moser 14b3349e8d NIFI-1254 remove Destination Type property from GetJMSQueue and GetJMSTopic
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-12-04 15:11:47 -05:00
Mark Payne f378ee9021 NIFI-1249: Allow Processors to add their own variables to those referencable by Expression Language. Make ReplaceText allow users to reference back-references of regex matches 2015-12-04 13:17:37 -05:00
Mark Payne dce039b54f Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2015-12-03 16:35:33 -05:00
Mark Payne 76914ead03 Merge branch 'NIFI-1246' 2015-12-03 16:35:19 -05:00
Mark Payne 3b7512ebac NIFI-1246: When calculating relative paths, ensure that we have an absolute path 2015-12-03 16:06:03 -05:00
joewitt 3656c883c7 NIFI-1240 removing explicit reference to SUN provider. Not necessary for our use and ties us to Sun or JREs with Sun JCE available. Favoring no-args constructor instantiation of SecureRandom for greater flexibility in choosing from available CSPs. Deprecating the associated public constant for the PRNG.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-12-03 14:46:02 -05:00
Daniel Cave 6bdcd3913f nifi-1247 Correction for check of UnmatchedFieldBehaviour in generateUpdate to match generateInsert correct functionality
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-12-03 14:12:23 -05:00
Mark Payne dae25accb7 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2015-12-03 12:13:33 -05:00
Mark Payne ca79c146fd NIFI-1241: Fixed checkstyle violation that slipped through 2015-12-03 12:13:23 -05:00
Bryan Bende 4653908b22 NIFI-1227 Adding SSL/TLS support to Syslog processors
-Removing connection creation from PutSyslog onScheduled, and reusing constructor in SSLSocketChannel
-Reducing visiblity of inner classes
2015-12-03 11:42:33 -05:00
Mark Payne eceb1d20c7 NIFI-1241: Incorporate Boolean, Numeric, and Decimal data types for PutSQL 2015-12-03 10:02:36 -05:00
Mark Payne 6b75eda9ab NIFI-1244: Incorporate Schema Name property 2015-12-03 09:48:27 -05:00
Andy LoPresto bde270a911 NIFI-1240:
Added explicit reference to Sun Java Cryptographic Service Provider in PasswordBasedEncryptor.
Removed manual seeding of SecureRandom in PasswordBasedEncryptor.

This closes #138.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-12-02 19:37:09 -05:00
Mark Payne 02dc7e82b2 NIFI-1239: Fixed bug that caused the cache to hold 0 elements instead of 100 2015-12-02 16:39:40 -05:00
Bryan Bende 6a8c2dba12 NIFI-1237 Reducing number of messages sent in testBatching() to be less than size of ListenSyslog's internal queue
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-01 22:53:42 -05:00
Tony Kurc 3a7ddc6a35 NIFI-1054: Fixed DOS line endings in xml, java and js source files
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-01 22:49:51 -05:00
Mark Payne e5281f1fc1 NIFI-1221: Support batching of Syslog messages
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-11-30 17:32:49 -05:00
Mark Payne c59087bc3a Merge branch 'NIFI-1049' 2015-11-30 16:02:36 -05:00
Mark Payne 07c619cf40 NIFI-1049: Added unit test to verify that reported bug was fixed; corrected a typo in log message 2015-11-30 15:37:58 -05:00
joewitt 67aed5eb92 NIFI-1224: made TestExecuteSQL about 25 seconds faster. Same logic just less to process
Reviewed and Amended (removed trailing whitespace) by Tony Kurc (tkurc@apache.org)
2015-11-26 00:33:54 -05:00
joewitt a3cb803147 NIFI-1224: reduced size of test set produced and removed ignored memory test
Reviewed and amended (removed unused imports for checkstyle) by Tony Kurc (tkurc@apache.org)
2015-11-25 23:09:49 -05:00
joewitt 6692645209 NIFI-631: addressed environment specific unit test problems
Reviewed by Tony Kurc (tkurc@apache.org)
2015-11-25 22:06:45 -05:00
Joe Skora 226ac64ef9 NIFI-631: Correct the WritesAttributes descriptions.
Reviewed and Ammended (based on comments from dev list) by Tony Kurc (tkurc@apache.org)
2015-11-25 18:03:19 -05:00
Mark Payne ba83bf9c05 NIFI-631: Initial implementation of FetchFile processor
Reviewed by Tony Kurc (tkurc@apache.org)
2015-11-25 18:03:19 -05:00
Joe Skora 4c4d62c61f NIFI-631: Added ListFile processor.
Reviewed by Tony Kurc (tkurc@apache.org)
2015-11-25 18:03:19 -05:00
Bryan Bende 4281a51c83 Merge branch 'NIFI-1208' 2015-11-23 21:35:57 -05:00
Mark Payne 4e2c94d659 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2015-11-23 14:18:50 -05:00
Mark Payne 2516b1dad2 NIFI-1171: Ensure that we pick up changes when files roll over and ensure that we don't pick up the rolled over file multiple times 2015-11-23 14:11:14 -05:00
Bryan Bende 3ffb455903 NIFI-1208 Adding context.yield() to Listen and Put syslog when there is nothing to do in onTrigger() 2015-11-23 10:30:33 -05:00
Joseph Percivall cdd2c4f22c NIFI-1086: Changed behavior on retrieval with no input file to RETRIEVE events, removed @TriggerWhenEmpty
Reviewed by Tony Kurc (tkurc@apache.org)
2015-11-21 19:20:33 -05:00
Bryan Bende f1f67f6395 Fixing one-character typo in syslog attribute name 2015-11-20 10:45:21 -05:00
Joseph Percivall 8c2323dc8d NIFI-1086 Provide refactoring of InvokeHTTP
NIFI-980 Add support for HTTP Digest authentication to InvokeHttp
NIFI-1080 Provide additional InvokeHttp unit tests
NIFI-1133 InvokeHTTP Processor does not save Location header for 3xx responses
NIFI-1009 InvokeHTTP should be able to be scheduled without any incoming connection for GET operations
NIFI-61 Multiple improvements for InvokeHTTP inclusive of providing unique tx.id across clusters, dynamic HTTP header properties

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-11-19 01:40:21 -05:00
Joe Percivall fb335ea282 NIFI-1165: Fix for tests TestRouteText and PutHDFS which did not succeed on Windows
Reviewed by Tony Kurc (tkurc@apache.org)
2015-11-19 01:01:34 -05:00
Mark Payne e862f7ff03 NIFI-1165: Use FileChannel instead of RandomAccessFile in order to avoid locking files in Windows
Reviewed by Tony Kurc (tkurc@apache.org)
2015-11-19 01:01:28 -05:00
Joseph Percivall 1e5cc070a3 NIFI-1081 Adding option to ExecuteStreamCommand to put output value to an attribute
Reviewed and amended (comments,whitespace,and some code readability (discussed in ticket)) by Tony Kurc (tkurc@apache.org)
2015-11-18 23:23:10 -05:00
Mark Payne 69bce2c2db NIFI-1168: Ensure that processors with only looping
connections are scheduled to run, even if the connections have no FlowFiles;
 expose these details to processor developers; update documentation

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-11-18 14:53:30 -05:00
Aldrin Piri 773576e041 NIFI-1108 Providing additional annotations on processors to ensure utilization of the InputRequirement annotation. 2015-11-18 13:56:21 -05:00
Mark Payne 911e1c5412 NIFI-1108: Updated processors to include the @InputRequirement annotation
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-11-18 13:47:59 -05:00
Tony Kurc 528dab78d6 NIFI-1073: Fixing coverity discovered errors. Resource leaks, and statics
Reviewed by Bryan Bende (bbende@apache.org)
2015-11-14 18:56:43 -05:00
Aldrin Piri 01539ed323 NIFI-1163: Providing handling of SSLContext creation in GetHTTP in case of only performing a one-way SSL request and accompanying test to verify the configuration/usage.
Reviewed by Tony Kurc (tkurc@apache.org)
2015-11-14 18:47:50 -05:00
Tony Kurc e6086420aa NIFI-1158 Default timestamp based on now 2015-11-12 23:11:43 -05:00
Mark Payne 0900fb80c9 NIFI-1153: If no incoming FlowFile, don't try to transfer null
Reviewed by Bryan Bende (bbende@apache.org)
Amended based on review (change to a log message) by Tony Kurc (tkurc@apache.org)
2015-11-12 22:41:57 -05:00
joewitt 02102ea1c2 Revert "NIFI-1134". This was removed because they looked like floating classes. But in reality they are used to make jars which we use to test the build. However, we really need the test to work differently but until that is fixed these must stay.
This reverts commit 41f3875347.
2015-11-12 18:00:40 -05:00
Mark Payne 7a165b62cc Merge branch 'NIFI-994' 2015-11-11 16:05:40 -05:00
Mark Payne e7c6c7cae9 NIFI-1083: Added a Grouping Regular Expression property for grouping lines of text 2015-11-11 10:19:07 -05:00
Joseph Percivall 3e538d9007 NIFI-1083 Created a processor that routes lines of text based on different matching and routing strategies
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-11 09:12:47 -05:00
Joseph Percivall d887d8fdf0 NIFI-993 added expression language support to GetHTTP for the filename and URL properties
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-11 08:05:23 -05:00
Mark Payne 7b9c8df6c5 NIFI-994: Fixed issue that could result in data duplication if more than 1 rollover of tailed file has occurred on restart of Processor 2015-11-10 16:53:29 -05:00
Mark Payne 72008035b1 Merge branch 'NIFI-1000' 2015-11-10 15:02:02 -05:00
Mark Payne 8f942fcdcc NIFI-1000: Fixed checkstyle violation in TestGetJSMQueue use activemq-broker with test scope and activemq-client with default scope rather than pulling in activemq-all; this reduces size of standard nar from 30 MB to 18 MB 2015-11-10 14:31:20 -05:00
Oleg Zhurakousky 8699e35108 NIFI-1000 addressed PR comments 2015-11-10 12:13:16 -05:00
Oleg Zhurakousky ef0be5a5d6 NIFI-1000 Fixed JmsFactory to properly obtain destiniation name
Re-enabled JMS Tests that were annotated with @Ignore
2015-11-09 18:35:31 -05:00
joewitt 41f3875347 NIFI-1134 2015-11-09 15:37:40 -05:00
joewitt 99629646fe NIFI-1122 moved to 0.4.0-SNAPSHOT 2015-11-06 23:41:15 -05:00
Bryan Bende 618f22e110 NIFI-274 - Fixing TestListenSyslog, fixing default buffer size to be bytes, adding syslog.protocol to attributes
- Adding syslog.port to ListenSyslog attributes, logging at warn level when rejecting tcp connections
         - Adding @InputRequirement to processors and adding appropriate send and receive provenance events
2015-11-04 18:01:59 -05:00
Tony Kurc 5611dac3f8 NIFI-274 - added use of Selectors for TCP and UDP connections. Added a max connections to the TCP thread
- Added comments and code review changes
         - fixed fixbugs bug
2015-11-04 18:00:18 -05:00
Bryan Bende 9c542432da NIFI-274 Adding ListenSyslog and PutSyslog to standard processors.
- Refactoring connection handling on put side, removing number of buffers from properties and basing it off concurrent tasks for the processor.
- Refactoring some of the TCP handling so it keeps reading from a connection until the client closes it
- Adding an error queue
- Adding a sender field on the syslog event to record the system that sent the message
2015-11-04 17:56:13 -05:00
Mark Payne fedbe40313 NIFI-1103: Added InputRequirement annotation 2015-11-04 11:11:33 -05:00
Joseph Percivall a549621267 NIFI-1077 enabling expression language on ConvertCharacterSet input and output
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-02 11:46:17 -05:00
Joseph Percivall 2ae49026e8 NIFI-883 Fixing issue HandleHttpRequest had with PrimaryNodeOnly scheduling
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-02 10:32:14 -05:00
Joseph Percivall cef7b6c736 NIFI-1080 fleshed out InvokeHttp unit tests
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-10-31 00:19:36 -04:00
Jeremy Dyer eb389cf84c NIFI-1079 Replacing Name for NULL_VALUE_FOR_EMPTY_STRING as I had accidentally removed it.
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-10-30 14:30:56 -04:00
Jeremy Dyer aef0d8fe9b NIFI-1079 Check style and documentation based updates recommended by user bbende on Github pull request
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-10-30 14:30:42 -04:00
Jeremy Dyer 217b1049cf NIFI-1079 Added Destination Property to control if JSON goes to Attribute or
Content of FlowFile. Added Include Core Attributes Property to control
if FlowFile CoreAttributes are included in the JSON output or not.
Added Null value for Empty String flag to control if empty values in
the JSON are empty string or true NULL values. Added more tests and
minor text refactoring per Github comments

Signed-off-by: Bryan Bende <bbende@apache.org>
2015-10-30 14:30:25 -04:00
Jeremy Dyer 19b7a4cc7d NIFI-1079 Create a new Attribute from the existing FlowFile attributes by taking
either all of the existing attributes or a user defined list. The
existing Attributes are converted to JSON and placed in a new Attribute
on the existing FlowFile as Attribute “JSONAttributes”

Signed-off-by: Bryan Bende <bbende@apache.org>
2015-10-30 14:29:29 -04:00
Mark Payne bfa9e45079 NIFI-994: Ensure that processor is not valid due to the tail file not yet existing 2015-10-30 11:44:51 -04:00
Mark Payne dc4004de64 Merge branch 'NIFI-977' 2015-10-27 16:53:38 -04:00
Mark Payne af19053a7f Merge branch 'NIFI-447' 2015-10-27 10:42:46 -04:00
Joseph Percivall 07238c7913 NIFI-447 adding replacement strategy to ReplaceText processor
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-27 10:37:43 -04:00
Joseph Percivall b885f955f4 NIFI-516 adding option to StandardProcessSession.read to close stream
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-26 20:23:13 -04:00
Mark Payne 1c1738670c NIFI-10: Updated FetchHDFS, FetchFileTransfer to use new FETCH provenance event 2015-10-26 17:14:29 -04:00
Mark Payne 51f564024a NIFI-10: Added FETCH Provenance Event and updated processors to use this new event type 2015-10-26 14:58:50 -04:00
Luke Williamson 26edab3185 NIFI-908 Added support for SSL in JMS connections.
- Added SSL context to JMS producer and consumer processors
 - Tony Kurc Amended patch to check SSL need by scheme and exception consistency
Reviewed by Tony Kurc (tkurc@apache.org)
2015-10-26 00:29:05 -04:00