16f1852453
NIFI-259 Updating logger message for AbstractListProcessor to reference State Manager in onTrigger instead of the distributed cache.
2016-02-05 12:55:06 -05:00
498b5023ce
NIFI-1257 NIFI-1259
...
Added utility method to return the maximum acceptable password length for PBE ciphers on JVM with limited strength crypto because BC implementation is undocumented (based on empirical evidence).
Updated EncryptionMethod definitions to accurately reflect need for unlimited strength crypto according to algorithm key length.
Added processor logic to invoke keyed cipher.
Added EncryptContent processor property for raw hex key (always visible until NIFI-1121).
Added validations for KDF (keyed and PBE) and hex key.
Added utility method to return list of valid key lengths for algorithm.
Added description to allowable values for KDF and encryption method in EncryptContent processor.
Added IV read/write to KeyedCipherProvider and changed from interface to abstract class.
Added salt read/write logic to NifiLegacy and OpenSSL cipher providers.
Changed RandomIVPBECipherProvider from interface to abstract class.
Updated strong KDF implementations.
Renamed CipherFactory to CipherProviderFactory.
Added unit test for registered KDF resolution from factory.
Updated default iteration count for PBKDF2 cipher provider.
Implemented Scrypt cipher provider.
Added salt translator from mcrypt format to Java format.
Added unit tests for salt formatting and validation.
Added surefire block to groovy unit test profile to enforce 3072 MB heap for Scrypt test.
Added local Java implementation of Scrypt KDF (and underlying PBKDF2 KDF) from Will Glozer.
Defined interface for KeyedCipherProvider.
Implemented AES implementation for KeyedCipherProvider.
Added Ruby script to test/resources for external compatibility check.
Added key length check to PBKDF2 cipher provider.
Changed default PRF to SHA-512.
Added salt and key length check to PBKDF2 cipher provider.
Added utility method to check key length validity for cipher families.
Added Bcrypt implementation.
Implemented PBKDF2 cipher provider.
Added default constructor with strong choices for PBKDF2 cipher provider.
Implemented NiFiLegacyCipherProvider and added unit tests.
Added key length parameter to PBKDF2 cipher provider.
Added PRF resolution to PBKDF2 cipher provider.
Added RandomIVPBECipherProvider to allow for non-deterministic IVs.
Added new keyed encryption methods and added boolean field for compatibility with new KDFs.
Added CipherFactory.
Improved Javadoc in NiFi legacy cipher provider and OpenSSL cipher provider.
Added KeyedCipherProvider interface.
Added OpenSSL PKCS#5 v1.5 EVP_BytesToKey cipher provider and unit test.
This closes #201 .
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-04 19:40:55 -05:00
29fb9c9393
NIFI-259 Corrected some logic in AbstractListProcessor regarding being elected primary node and improved readability in GetHBase a smidge
2016-02-04 16:07:18 -05:00
88d4d2ce5f
NIFI-259 Corrected GetHttp state managment and added a new unit test
2016-02-03 20:04:39 -05:00
55b77fe03b
NIFI-1223: - Removing conf directory from unit test.
...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-03 16:05:38 -05:00
ef80549d63
Merge branch 'NIFI-1093' of https://github.com/olegz/nifi into NIFI-1093
2016-02-01 11:51:27 -05:00
171b9c4e94
NIFI-1093 - Added settings for processing JSON documents when validating against database columns to fail, log a warning, or ignore unmatched columns. Added tests for new settings.
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2016-02-01 11:45:15 -05:00
1bf7a585b2
NIFI-1456 This closes #198 . reduced tests to 10MB instead of 100MB datasets and resolved IO issues which impact test stability
2016-01-31 21:56:53 -05:00
8ef337b65e
NIFI-1434 Prevent array index exception in PutEmail
...
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-01-30 10:56:31 -05:00
0126800201
NIFI-1093 added support for handling non-required columns
...
Mostly based on contribution from Daniel Cave
2016-01-29 13:41:27 -05:00
be27c07c7f
NIFI-1382 Created FetchDistributedMapCache processor
2016-01-29 12:12:23 -05:00
a0224f1b4c
NIFI-1431: Added documentation about how ETag and LastModified dates are used by the Processor
...
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-28 16:02:05 -05:00
ee30b27cc8
NIFI-1396 Correcting PostHttp chunked encoding functionality
2016-01-28 14:50:22 -05:00
aa6bc5d505
NIFI-1409: Fix missing transfer on error in ExecuteSql
...
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-01-27 18:23:32 -05:00
93aac8cff3
NIFI-1365
...
Added Groovy support for unit tests to pom with skeleton test.
Added Groovy unit tests for OCSPCertificateValidator.
Implemented positive & negative unit tests with cache injection for valid/revoked OCSP certificate.
Modified pom.xml to support Groovy unit tests with custom variable.
mvn clean test -Dgroovy=test
Added local cache injection into Groovy tests for OCSP certificate validation (see NIFI-1324 and NIFI-1364).
Set Java version to 1.7 for Groovy test src/target.
Moved Groovy unit test profile from nifi-web-security to root pom.
Added null check for algorithm argument in PGPUtil.
Changed buffer length check from ">= 0" to "> -1" because it was confusing other developers.
Resolved contrib-check line length issues.
Fixed contrib-check issues in OpenPGPKeyBasedEncryptorTest.
This closes #163
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-26 17:37:59 -05:00
0e92607466
Merge branch 'NIFI-1412' of https://github.com/mattyb149/nifi into NIFI-1412
2016-01-26 12:30:19 -05:00
c16c8db459
NIFI-1421 Make namespace ordering deterministic in SplitXML
2016-01-26 05:35:07 +00:00
b7f7e6ed80
NIFI-1436 This closes #189 . Combining stop() and close() into a single method to simplify, and adding checks on stopped flag in the run method of SocketChannelDispatcher and DatagramChannelDispatcher to ensure the run() method exits as soon as possible upon close() being called
...
NIFI-1436 Adding synchronization on keys set in close() method based on Selector JavaDoc
Signed-off-by: joewitt <joewitt@apache.org>
2016-01-25 22:50:25 -05:00
beb92492be
NIFI-1421 This closes #184 . Added content assertion to test and removed extra space
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2016-01-25 16:04:47 -05:00
4a5fb37b33
Merge branch 'NIFI-1421' of https://github.com/rpmiskin/nifi into NIFI-1421
2016-01-25 15:38:08 -05:00
91e7ba84e0
NIFI-1412: Fix error during Avro conversion where no fields are present
2016-01-25 08:35:14 -05:00
8d37af07b9
NIFI-1381 Removing the hardcoded jms:// prefix and instead deferring to the URI specified by the processor properties of PutJMS
...
Signed-off-by: joewitt <joewitt@apache.org>
2016-01-24 23:10:47 -05:00
0db2dc9fc3
NIFI-1405 Adding option to invokeHttp to control using chunked encoding for http PUT and POST calls
...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-24 15:52:05 -05:00
dfed7e727d
NIFI-1421 Added missing copyright notice to new test input
2016-01-23 08:52:54 +00:00
93e2c52195
NIFI-1415: Do not enfoce that a password or Private Key Passphrase is set
...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-22 20:46:14 -05:00
92e6961b50
NIFI-1416: If FetchSFTP's Remote File has a directory name in it, do not include that as part of the 'filename' attribute but instead add a 'path' attribute
...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-22 18:15:19 -05:00
1089f0a956
NIFI-1273 Adding ListenRELP processor which includes refactoring code that was previously part of ListenSyslog into a reusable framework for implementing listener processors, as well as back-end code for handling the RELP protocol. Addressing some feedback from review and fixing logging statements Addressing review feedback, added AsyncChannelDispatcher and made ChannelHandlerFactory use generics for the dispatcher This closes #179
...
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-22 16:28:58 -05:00
3db6fffa68
NIFI-1378: fixed JMS URI validation simplified JmsFactory check for SSL and scheme-less URIs ensured URI validation is handled by ActiveMqConnectionFactory ensured informative error messages are shown to the userm removed unneeded test, cleaned up unused imports
...
Reviewed by Tony Kurc (tkurc@apache.org ). This closes #167
2016-01-21 22:29:50 -05:00
68a9375f3e
NIFI-1421 Update SplitXML to support namespace declarations
2016-01-21 22:19:15 +00:00
f6ec437bc7
Merge branch 'master' into NIFI-259
2016-01-21 09:32:02 -05:00
bc75dd3c17
NIFI-1406 Collecting the number of bytes sent regardless if one or multiple FlowFiles are sent.
...
Reviewed by Tony Kurc (tkurc@apache.org ). This closes #176
2016-01-20 21:08:15 -05:00
348831b20c
Merge branch 'NIFI-1414'
2016-01-20 17:31:34 -05:00
e1588d4136
[NIFI-1411] - TestJdbcTypesDerby: java.sql.SQLSyntaxErrorException: TYPE 'DATETIME' does not exist
2016-01-20 16:51:58 -05:00
9ebcc9e4fa
NIFI-1393 Providing the ability to send using gzip Content-Encoding in PostHTTP if the endpoint server supports it regardless if the processor is configured to send as a FlowFile This closes #175
...
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-20 16:46:57 -05:00
d696391f76
NIFI-259: Update ZooKeeperStateServer so that the thread starting it doesn't block indefinitely, as this was causing NiFi to not shut down properly; applied patch for NIFI-1415
2016-01-20 11:52:41 -05:00
7ef8af9eea
NIFI-259: Applied patch for NIFI-1414
2016-01-20 09:14:04 -05:00
1845f6df8d
NIFI-1414: Ensure that the DirectoryStream is closed after streaming contents of a directory
2016-01-20 09:12:14 -05:00
8f688d4921
NIFI-1360 Add expression language support for "MergeContent" "Correlation Attribute Field" property
...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-19 10:46:37 -08:00
d6a21537c1
NIFI-259: Do not require the State File anymore, since state management is being used now
2016-01-19 11:03:31 -05:00
9e9182aa63
- Unit test creates resources in HOME directory This closes #174
...
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-19 09:44:03 -05:00
16dcf4595c
Merge branch 'master' into NIFI-259
2016-01-18 08:48:30 -05:00
eba25ecaca
NIFI-259, NIFI-1339: Added OnConfigurationRestored annotation, always invoke onPropertyModified even on restart when properties are changed from defaults, as was done previously
2016-01-17 14:40:27 -05:00
ecb81ec113
NIFI-1362 Set mime.type attribute on response FlowFile based on InvokeHTTP response Content-Type
...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-15 16:40:36 -05:00
0cd6f80f36
NIFI-259: Bug fixes
2016-01-13 15:11:53 -05:00
0151b1eed1
NIFI-259: Fixed checkstyle violations
2016-01-13 13:57:47 -05:00
d39067ede6
NIFI-259: Update GetHBase to use new State Management feature; updated docs; bug fixes
2016-01-13 13:30:05 -05:00
bbce596d74
NIFI-259: Updated GetHBase to use new State Management; bug fixes; updated docs
2016-01-13 12:47:08 -05:00
c757c4d6ea
NIFI-1385: Resolved test for missing file which was failing on Windows due to differing error message.
...
Reviewed by Tony Kurc (tkurc@apache.org )
2016-01-12 19:16:29 -05:00
774c29a4da
NIFI-259: Added Stateful annotation as described on ticket
2016-01-12 15:28:35 -05:00
b4bc105ea9
Merge branch 'nifi-1283' of https://github.com/jvwing/nifi
2016-01-12 10:49:04 -05:00
f6b34cbbc5
NIFI-259: Removed bad unit test
2016-01-12 10:40:00 -05:00
e559c68766
NIFI-259: Began updating processors to use new state management
2016-01-12 10:21:47 -05:00
44ffddd8b9
NIFI-1324:
...
- Addressing lengthy lines contrib-check issue.
- This closes #162
2016-01-12 09:42:31 -05:00
ffbfffce6d
NIFI-1324:
...
Changed Maven dependencies for BouncyCastle bcprov and bcpg from jdk16:1.46 to jdk15on:1.53 (kept nifi-web-security on jdk16:1.46 because jdk15on:1.53 splits OCSP logic into new module bcpkix).
Added individual unit tests for PGP public keyring validation.
Passes all legacy unit tests.
Added TODOs for customizable brick encryption and refactoring shared code.
Cleaned up magic numbers to constants.
Added unit tests for OpenPGPPasswordBasedEncryptor (internal consistency and legacy file decrypt).
Began refactoring shared encrypt code from OpenPGP* implementations.
Extracted encrypt utility method from OpenPGPPasswordBasedEncryptor to PGPUtil class.
Added test resources (signed and unsigned key-encrypted files).
Added unit tests for OpenPGPKeyBasedEncryptor (internal consistency and external file decrypt).
Changed BC dependency for nifi-web-security to bcprov-jdk15on:1.53 and bcpkix-jdk15on:1.53.
Updated OCSPValidator to use new BC logic for OCSP validation. This code compiles but should be fully audited, as the legacy OCSP validation was not completely implemented.
Added skeleton of OCSP validator unit tests with successful keypair and certificate generation and signing code.
Added further unit tests for issued certificates.
Annotated unimplemented unit tests with note about Groovy integration.
Refactored Jersey call in OCSPCertificateValidator to internal method.
Added toString() to NiFi local OcspRequest.
Implemented positive & negative unit tests with cache injection for valid/revoked OCSP certificate.
Resolved contrib-check issues.
Removed ignored code in unit test.
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-12 09:22:51 -05:00
6b54753dbb
NIFI-1316 adding option to DetectDuplicate to not cache the entry identifier
...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-11 22:59:24 -05:00
c526656a22
NIFI-1283 Fixing ControllerStatusReportingTask loggers to use fully-qualified class name
2016-01-11 13:39:15 -08:00
d2a969e3d6
NIFI-259: Initial implementation of State Management feature
2016-01-11 16:38:52 -05:00
0c68e2c3a8
NIFI-1378 added validator to ensure the JMS URI has a scheme set
...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-10 16:07:56 -05:00
317b2f4a4a
NIFI-1361 Exposing Content-Type as a processor property for both InvokeHTTP and PostHTTP. This makes use of the mime.type attribute via the EL expression ${mime.type}, making more apparent to the user how this header is derived and allowing the explicit setting of a value.
2016-01-08 10:43:14 -05:00
6c46f22b62
NIFI-1327 NIFI-1353 Updating version of OkHttp and OkHttp-Digest
2016-01-07 15:51:49 -05:00
47453d2c47
NIFI-1145 Providing delays in tests where resources are being initialized to allow heavily taxed environments sufficient time.
2016-01-07 12:05:23 -05:00
8f0e7a1ac5
NIFI-1354
...
Generated new self-signed RSA-2048 key for FQDN localhost in keystore and exported to truststore.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-06 18:16:37 -05:00
de2dd93f26
NIFI-1300 Corrects inconsistent error behavior in PutJMS and added PutJMS unit testing
...
Reviewed by Tony Kurc (tkurc@apache.org ) This closes #145
2016-01-04 00:26:21 -05:00
34bd2061f7
NIFI-1291: Adding BinFiles in nifi-processor-utilities, deprecating version in nifi-standard-processors. This closes #155 .
...
Signed-off-by: joewitt <joewitt@apache.org>
2016-01-03 23:05:28 -05:00
c696cb09b6
NIFI-1212 Treating text/csv files as plain text so they may also be displayed. This closes #154 .
...
Signed-off-by: joewitt <joewitt@apache.org>
2016-01-03 16:12:06 -05:00
236a2b72b6
NIFI-1335: Allow UnpackContent to Handle application/x-tar mime.type
...
- Add additinal `mime.type` for tar archives (application/x-tar)
- Update `UnpackContent` tests to verify auto-detect format property
2016-01-03 11:42:18 -05:00
e16a9a0c68
NIFI-1343 added unit tests to verify behavior
2016-01-01 14:08:46 -05:00
02b325b07e
NIFI-1343: Make comparisons case sensitive if configured to do so. This closes #153 .
...
Signed-off-by: joewitt <joewitt@apache.org>
2016-01-01 13:34:07 -05:00
03730adb3f
NIFI-1338 - Clarifying documentation for InvokeHTTP
...
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-31 13:48:00 -05:00
3189a13da1
NIFI-1328 Explicitly setting the parameters based on type in PutSQL, falling back to setObject()
...
This closes #146
2015-12-23 13:08:56 -05:00
8d46041a13
NIFI-1319 Updating JdbcCommon to check meta.isSigned(i) to determine if Avro schema should use a long or int
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-12-23 10:03:30 -05:00
f4ac8d75c5
NIFI-1312-RC1 prepare for next development iteration
2015-12-19 00:41:04 -05:00
d624ea4866
NIFI-1312-RC1 prepare release nifi-0.4.1-RC1
2015-12-19 00:40:53 -05:00
bd4f31a4c2
NIFI-1279: Take the FlowFile's 1-up id into account for the newest and oldest flowfile prioritizers
...
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-18 12:08:31 -05:00
d755e43ec8
NIFI-1122_nifi-0.4.0-RC2prepare for next development iteration
2015-12-08 13:00:10 -05:00
b66c029090
NIFI-1122_nifi-0.4.0-RC2prepare release nifi-0.4.0-RC2
2015-12-08 12:59:59 -05:00
d1395ea7ee
NIFI-1269: Ensure that we roll back session if not transferring FlowFile
...
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-07 21:18:33 -05:00
0c7617b686
NIFI-1267: TestListFile attempts to ensure that files that are not readable are skipped but the assumptions made are not valid if running as root; removed this check, since it is not reliable
...
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-07 20:42:07 -05:00
46623c2bf3
Merge branch 'NIFI-1261'
2015-12-07 15:13:12 -05:00
bfe6fad502
NIFI-1261: Use consistent file separators for ListFile / TestListFile based on what the Operating System uses
2015-12-07 10:19:49 -05:00
bd3fd870ac
NIFI-1263: Windows considers last mod time on both rolled over files to be the same, due to the way that windows computes the lastMod time. On linux and os x it is done differently. This is easily solved by adding a small sleep to the unit test before 'rolling over' the file so that the last mod date is different on all OS's, which yields the same behavior on all OS's
2015-12-07 09:53:33 -05:00
f83e6d33c5
NIFI-1242:
...
Added logic and test resources to debug JCE unlimited strength cryptography policy issues and incorporated into processor property validation.
Excluded test resources from RAT check
Added KeyDerivationFunction enum.
Added kdf property in EncryptContent processor and provided to PasswordBasedEncryptor.
Added logic in PasswordBasedEncryptor to handle variable KDF.
Added unit tests for EncryptContent processor.
Added test resources and excluded from RAT check.
plain.txt: This is a plaintext message.
0s @ 12:20:32 $ openssl enc -aes-256-cbc -e -in plain.txt -out salted_raw.enc -k thisIsABadPassword -p
salt=31DC301A6C7B8A0B
key=CB878A6E167A5B530B8F2BD175E6359E3092AFF7C83274A22A5B421D79E599AC
iv =0C614A72FC06B454B84E035B3FA8F877
0s @ 12:20:44 $ xxd salted_raw.enc
0000000: 5361 6c74 6564 5f5f 31dc 301a 6c7b 8a0b Salted__1.0.l{..
0000010: 616b c65d f767 504d c085 ba7a c517 d0cb ak.].gPM...z....
0000020: 7832 211e f573 b6f1 ded2 8f59 88e8 088f x2!..s.....Y....
0s @ 20:14:00 $ openssl enc -aes-256-cbc -e -in plain.txt -out unsalted_raw.enc -k thisIsABadPassword -p -nosalt
key=711E85689CE7AFF6F410AEA43ABC5446842F685B84879B2E00F977C22B9E9A7D
iv =0C90ABF8ECE84B92BAA2CD448EC760F0
0s @ 20:14:17 $ xxd unsalted_raw.enc
0000000: 70cd 2984 fdbb 0e7c c01b 7206 88b1 6b50 p.)....|..r...kP
0000010: 5eeb e4f3 4036 773b 00ce dd8e 85d8 f90a ^...@6w;........
This closes #140
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-12-04 19:45:10 -05:00
ee14d8f9dd
Merge branch 'NIFI-1249'
2015-12-04 16:38:49 -05:00
14b3349e8d
NIFI-1254 remove Destination Type property from GetJMSQueue and GetJMSTopic
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-12-04 15:11:47 -05:00
f378ee9021
NIFI-1249: Allow Processors to add their own variables to those referencable by Expression Language. Make ReplaceText allow users to reference back-references of regex matches
2015-12-04 13:17:37 -05:00
dce039b54f
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi
2015-12-03 16:35:33 -05:00
76914ead03
Merge branch 'NIFI-1246'
2015-12-03 16:35:19 -05:00
3b7512ebac
NIFI-1246: When calculating relative paths, ensure that we have an absolute path
2015-12-03 16:06:03 -05:00
3656c883c7
NIFI-1240 removing explicit reference to SUN provider. Not necessary for our use and ties us to Sun or JREs with Sun JCE available. Favoring no-args constructor instantiation of SecureRandom for greater flexibility in choosing from available CSPs. Deprecating the associated public constant for the PRNG.
...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-12-03 14:46:02 -05:00
6bdcd3913f
nifi-1247 Correction for check of UnmatchedFieldBehaviour in generateUpdate to match generateInsert correct functionality
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-12-03 14:12:23 -05:00
dae25accb7
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi
2015-12-03 12:13:33 -05:00
ca79c146fd
NIFI-1241: Fixed checkstyle violation that slipped through
2015-12-03 12:13:23 -05:00
4653908b22
NIFI-1227 Adding SSL/TLS support to Syslog processors
...
-Removing connection creation from PutSyslog onScheduled, and reusing constructor in SSLSocketChannel
-Reducing visiblity of inner classes
2015-12-03 11:42:33 -05:00
eceb1d20c7
NIFI-1241: Incorporate Boolean, Numeric, and Decimal data types for PutSQL
2015-12-03 10:02:36 -05:00
6b75eda9ab
NIFI-1244: Incorporate Schema Name property
2015-12-03 09:48:27 -05:00
bde270a911
NIFI-1240:
...
Added explicit reference to Sun Java Cryptographic Service Provider in PasswordBasedEncryptor.
Removed manual seeding of SecureRandom in PasswordBasedEncryptor.
This closes #138 .
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-12-02 19:37:09 -05:00
02dc7e82b2
NIFI-1239: Fixed bug that caused the cache to hold 0 elements instead of 100
2015-12-02 16:39:40 -05:00
6a8c2dba12
NIFI-1237 Reducing number of messages sent in testBatching() to be less than size of ListenSyslog's internal queue
...
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-01 22:53:42 -05:00
3a7ddc6a35
NIFI-1054: Fixed DOS line endings in xml, java and js source files
...
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-01 22:49:51 -05:00
e5281f1fc1
NIFI-1221: Support batching of Syslog messages
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-11-30 17:32:49 -05:00
c59087bc3a
Merge branch 'NIFI-1049'
2015-11-30 16:02:36 -05:00
07c619cf40
NIFI-1049: Added unit test to verify that reported bug was fixed; corrected a typo in log message
2015-11-30 15:37:58 -05:00
67aed5eb92
NIFI-1224: made TestExecuteSQL about 25 seconds faster. Same logic just less to process
...
Reviewed and Amended (removed trailing whitespace) by Tony Kurc (tkurc@apache.org )
2015-11-26 00:33:54 -05:00
a3cb803147
NIFI-1224: reduced size of test set produced and removed ignored memory test
...
Reviewed and amended (removed unused imports for checkstyle) by Tony Kurc (tkurc@apache.org )
2015-11-25 23:09:49 -05:00
6692645209
NIFI-631: addressed environment specific unit test problems
...
Reviewed by Tony Kurc (tkurc@apache.org )
2015-11-25 22:06:45 -05:00
226ac64ef9
NIFI-631: Correct the WritesAttributes descriptions.
...
Reviewed and Ammended (based on comments from dev list) by Tony Kurc (tkurc@apache.org )
2015-11-25 18:03:19 -05:00
ba83bf9c05
NIFI-631: Initial implementation of FetchFile processor
...
Reviewed by Tony Kurc (tkurc@apache.org )
2015-11-25 18:03:19 -05:00
4c4d62c61f
NIFI-631: Added ListFile processor.
...
Reviewed by Tony Kurc (tkurc@apache.org )
2015-11-25 18:03:19 -05:00
4281a51c83
Merge branch 'NIFI-1208'
2015-11-23 21:35:57 -05:00
4e2c94d659
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi
2015-11-23 14:18:50 -05:00
2516b1dad2
NIFI-1171: Ensure that we pick up changes when files roll over and ensure that we don't pick up the rolled over file multiple times
2015-11-23 14:11:14 -05:00
3ffb455903
NIFI-1208 Adding context.yield() to Listen and Put syslog when there is nothing to do in onTrigger()
2015-11-23 10:30:33 -05:00
cdd2c4f22c
NIFI-1086: Changed behavior on retrieval with no input file to RETRIEVE events, removed @TriggerWhenEmpty
...
Reviewed by Tony Kurc (tkurc@apache.org )
2015-11-21 19:20:33 -05:00
f1f67f6395
Fixing one-character typo in syslog attribute name
2015-11-20 10:45:21 -05:00
8c2323dc8d
NIFI-1086 Provide refactoring of InvokeHTTP
...
NIFI-980 Add support for HTTP Digest authentication to InvokeHttp
NIFI-1080 Provide additional InvokeHttp unit tests
NIFI-1133 InvokeHTTP Processor does not save Location header for 3xx responses
NIFI-1009 InvokeHTTP should be able to be scheduled without any incoming connection for GET operations
NIFI-61 Multiple improvements for InvokeHTTP inclusive of providing unique tx.id across clusters, dynamic HTTP header properties
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-11-19 01:40:21 -05:00
fb335ea282
NIFI-1165: Fix for tests TestRouteText and PutHDFS which did not succeed on Windows
...
Reviewed by Tony Kurc (tkurc@apache.org )
2015-11-19 01:01:34 -05:00
e862f7ff03
NIFI-1165: Use FileChannel instead of RandomAccessFile in order to avoid locking files in Windows
...
Reviewed by Tony Kurc (tkurc@apache.org )
2015-11-19 01:01:28 -05:00
1e5cc070a3
NIFI-1081 Adding option to ExecuteStreamCommand to put output value to an attribute
...
Reviewed and amended (comments,whitespace,and some code readability (discussed in ticket)) by Tony Kurc (tkurc@apache.org )
2015-11-18 23:23:10 -05:00
69bce2c2db
NIFI-1168: Ensure that processors with only looping
...
connections are scheduled to run, even if the connections have no FlowFiles;
expose these details to processor developers; update documentation
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-11-18 14:53:30 -05:00
773576e041
NIFI-1108 Providing additional annotations on processors to ensure utilization of the InputRequirement annotation.
2015-11-18 13:56:21 -05:00
911e1c5412
NIFI-1108: Updated processors to include the @InputRequirement annotation
...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-11-18 13:47:59 -05:00
528dab78d6
NIFI-1073: Fixing coverity discovered errors. Resource leaks, and statics
...
Reviewed by Bryan Bende (bbende@apache.org )
2015-11-14 18:56:43 -05:00
01539ed323
NIFI-1163: Providing handling of SSLContext creation in GetHTTP in case of only performing a one-way SSL request and accompanying test to verify the configuration/usage.
...
Reviewed by Tony Kurc (tkurc@apache.org )
2015-11-14 18:47:50 -05:00
e6086420aa
NIFI-1158 Default timestamp based on now
2015-11-12 23:11:43 -05:00
0900fb80c9
NIFI-1153: If no incoming FlowFile, don't try to transfer null
...
Reviewed by Bryan Bende (bbende@apache.org )
Amended based on review (change to a log message) by Tony Kurc (tkurc@apache.org )
2015-11-12 22:41:57 -05:00
02102ea1c2
Revert "NIFI-1134". This was removed because they looked like floating classes. But in reality they are used to make jars which we use to test the build. However, we really need the test to work differently but until that is fixed these must stay.
...
This reverts commit 41f3875347
2015-11-12 18:00:40 -05:00
7a165b62cc
Merge branch 'NIFI-994'
2015-11-11 16:05:40 -05:00
e7c6c7cae9
NIFI-1083: Added a Grouping Regular Expression property for grouping lines of text
2015-11-11 10:19:07 -05:00
3e538d9007
NIFI-1083 Created a processor that routes lines of text based on different matching and routing strategies
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-11 09:12:47 -05:00
d887d8fdf0
NIFI-993 added expression language support to GetHTTP for the filename and URL properties
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-11 08:05:23 -05:00
7b9c8df6c5
NIFI-994: Fixed issue that could result in data duplication if more than 1 rollover of tailed file has occurred on restart of Processor
2015-11-10 16:53:29 -05:00
72008035b1
Merge branch 'NIFI-1000'
2015-11-10 15:02:02 -05:00
8f942fcdcc
NIFI-1000: Fixed checkstyle violation in TestGetJSMQueue use activemq-broker with test scope and activemq-client with default scope rather than pulling in activemq-all; this reduces size of standard nar from 30 MB to 18 MB
2015-11-10 14:31:20 -05:00
8699e35108
NIFI-1000 addressed PR comments
2015-11-10 12:13:16 -05:00
ef0be5a5d6
NIFI-1000 Fixed JmsFactory to properly obtain destiniation name
...
Re-enabled JMS Tests that were annotated with @Ignore
2015-11-09 18:35:31 -05:00
41f3875347
NIFI-1134
2015-11-09 15:37:40 -05:00
99629646fe
NIFI-1122 moved to 0.4.0-SNAPSHOT
2015-11-06 23:41:15 -05:00
618f22e110
NIFI-274 - Fixing TestListenSyslog, fixing default buffer size to be bytes, adding syslog.protocol to attributes
...
- Adding syslog.port to ListenSyslog attributes, logging at warn level when rejecting tcp connections
- Adding @InputRequirement to processors and adding appropriate send and receive provenance events
2015-11-04 18:01:59 -05:00
5611dac3f8
NIFI-274 - added use of Selectors for TCP and UDP connections. Added a max connections to the TCP thread
...
- Added comments and code review changes
- fixed fixbugs bug
2015-11-04 18:00:18 -05:00
9c542432da
NIFI-274 Adding ListenSyslog and PutSyslog to standard processors.
...
- Refactoring connection handling on put side, removing number of buffers from properties and basing it off concurrent tasks for the processor.
- Refactoring some of the TCP handling so it keeps reading from a connection until the client closes it
- Adding an error queue
- Adding a sender field on the syslog event to record the system that sent the message
2015-11-04 17:56:13 -05:00
fedbe40313
NIFI-1103: Added InputRequirement annotation
2015-11-04 11:11:33 -05:00
a549621267
NIFI-1077 enabling expression language on ConvertCharacterSet input and output
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-02 11:46:17 -05:00
2ae49026e8
NIFI-883 Fixing issue HandleHttpRequest had with PrimaryNodeOnly scheduling
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-02 10:32:14 -05:00
cef7b6c736
NIFI-1080 fleshed out InvokeHttp unit tests
...
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-10-31 00:19:36 -04:00
eb389cf84c
NIFI-1079 Replacing Name for NULL_VALUE_FOR_EMPTY_STRING as I had accidentally removed it.
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-10-30 14:30:56 -04:00
aef0d8fe9b
NIFI-1079 Check style and documentation based updates recommended by user bbende on Github pull request
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-10-30 14:30:42 -04:00
217b1049cf
NIFI-1079 Added Destination Property to control if JSON goes to Attribute or
...
Content of FlowFile. Added Include Core Attributes Property to control
if FlowFile CoreAttributes are included in the JSON output or not.
Added Null value for Empty String flag to control if empty values in
the JSON are empty string or true NULL values. Added more tests and
minor text refactoring per Github comments
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-10-30 14:30:25 -04:00
19b7a4cc7d
NIFI-1079 Create a new Attribute from the existing FlowFile attributes by taking
...
either all of the existing attributes or a user defined list. The
existing Attributes are converted to JSON and placed in a new Attribute
on the existing FlowFile as Attribute “JSONAttributes”
Signed-off-by: Bryan Bende <bbende@apache.org>
2015-10-30 14:29:29 -04:00
bfa9e45079
NIFI-994: Ensure that processor is not valid due to the tail file not yet existing
2015-10-30 11:44:51 -04:00
dc4004de64
Merge branch 'NIFI-977'
2015-10-27 16:53:38 -04:00
af19053a7f
Merge branch 'NIFI-447'
2015-10-27 10:42:46 -04:00
07238c7913
NIFI-447 adding replacement strategy to ReplaceText processor
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-27 10:37:43 -04:00
b885f955f4
NIFI-516 adding option to StandardProcessSession.read to close stream
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-26 20:23:13 -04:00
1c1738670c
NIFI-10: Updated FetchHDFS, FetchFileTransfer to use new FETCH provenance event
2015-10-26 17:14:29 -04:00
51f564024a
NIFI-10: Added FETCH Provenance Event and updated processors to use this new event type
2015-10-26 14:58:50 -04:00
26edab3185
NIFI-908 Added support for SSL in JMS connections.
...
- Added SSL context to JMS producer and consumer processors
- Tony Kurc Amended patch to check SSL need by scheme and exception consistency
Reviewed by Tony Kurc (tkurc@apache.org )
2015-10-26 00:29:05 -04:00
8d2f9bc64b
NIFI-1056: Do not treat BigDecimal, BigInteger the same way we treat Number's when serializing to Avro because Avro Java Library doesn't support them
...
Reviewed and Unit Test Amended by Tony Kurc (tkurc@apache.org )
2015-10-25 11:31:30 -04:00
385bfbb2c6
NIFI-673: Rebased from master; Added InputRequirement annotation, as it is now merged into master
2015-10-25 11:14:11 -04:00
4e38288062
NIFI-673: Added sftp.listing.user attribute to FlowFiles created by ListSFTP; ensure that FetchSFTP indicates that the username supports Expression Language
2015-10-25 11:13:02 -04:00
b0322d9ffe
NIFI-673: Added Completion Strategy to FetchSFTP
2015-10-25 11:13:02 -04:00
d1d57931bf
NIFI-673: Initial implementation of ListSFTP, FetchSFTP
2015-10-25 11:13:02 -04:00
0636f0e731
NIFI-810: Merged master into branch
2015-10-25 11:02:40 -04:00
d63cd6bd2f
Merge branch 'NIFI-1010'
2015-10-23 14:39:16 -04:00
88fc8d28a0
NIFI-1010: If database driver does not support getting table name for column from ResultSetMetadata, then just use a default name
2015-10-23 10:25:59 -04:00
0fc5d30461
NIFI-1055: Fixed checkstyle violations
2015-10-23 10:08:44 -04:00
bd506b1e10
NIFI-972: Added additional unit test; deleted lines that were commented out
2015-10-23 09:52:09 -04:00
a9e5325047
NIFI-972 attribute to indicate rows count and cleanup
...
Signed-off-by: Toivo Adams <toivo.adams@gmail.com>
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-23 09:28:03 -04:00
ba3225fe92
NIFI-972 ExecuteSQL bug in createSchema() create Arvo Schema 1
...
Signed-off-by: Toivo Adams <toivo.adams@gmail.com>
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-23 09:28:03 -04:00
da28b81eec
Merge branch 'NIFI-988'
2015-10-21 10:44:06 -04:00
97441ea0c2
Merge branch 'NIFI-988' of https://github.com/ImpressTV/nifi into NIFI-988
2015-10-21 10:18:05 -04:00
518670dbf6
NIFI-1042 Adds restlistener.remote.source.ip attribute to ListenHTTPServlet. This closes #105
2015-10-19 23:43:44 -04:00
9200542298
NIFI-1016 Naive fix so org.apache.nifi.prioritizer.PriorityAttributePrioritizerTest builds
2015-10-18 20:34:36 -04:00
88b1b844fb
NIFI-747 This closes #104 . PR from Venkatesh Sellappa <VS186031@outlook.com> was modified, then code reviewed by Joe Witt (comments in ticket)
2015-10-18 20:04:24 -04:00
ad73a23aff
NIFI-1016: Use centralized priority attribute name in PriorityAttributePrioritizer. closes #98 .
...
Signed-off-by: joewitt <joewitt@apache.org>
2015-10-18 19:42:46 -04:00
31f0909bd3
NIFI-994: Initial import of TailFile
2015-10-18 19:00:24 -04:00
22924c656b
NIFI-995 TestGetFile.testAttributes() failed in case of a NTFS partition in Linux, because Files.setPosixFilePermissions() did not have any effect on the files and did not throw Exception. closes #95 . Signed off by Tony Kurc (tkurc@apache.org)
2015-10-17 10:46:49 -04:00
84db372538
NIFI-977: Allow SQL Data Types with numerals that are negative
2015-10-14 13:12:10 -04:00
b4bfcc1f21
NIFI-1030 made InvokeHttp basic auth password a sensitive value
...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-10-08 15:46:12 -04:00
ccfb57fe9f
NIFI-810: Addressed several checkstyle violations
2015-10-07 17:48:51 -04:00
4afd8f88f8
NIFI-810: Created RequiresInput annotation and ensure that processors are invalid if connections do not agree
2015-10-07 17:26:14 -04:00
96764ed6a1
Merge branch 'master' of http://git-wip-us.apache.org/repos/asf/nifi
2015-09-25 09:15:34 -04:00
eb94dab00c
NIFI-604: Fixed checkstyle violation
2015-09-24 11:51:44 -04:00
e12da7c9a3
NIFI-985: Custom log prefix for LogAttribute processor
...
The LogAttribute processor evaluates the log prefix EL using the current flow file.
Log prefix helps to distinguish the log output of multiple LogAttribute processors and identify the right processor. Log prefix appears in the first and the last log line, followed by the original 50 dashes. If you configure log prefix 'STEP 1: ' the log output looks like this:
STEP 1 : --------------------------------------------------
Standard FlowFile Attributes
Key: 'entryDate'
Value: 'Tue Sep 22 15:13:02 CEST 2015'
Key: 'lineageStartDate'
Value: 'Tue Sep 22 15:13:02 CEST 2015'
Key: 'fileSize'
Value: '9'
FlowFile Attribute Map Content
Key: 'customAttribute'
Value: 'custom value'
STEP 1 : --------------------------------------------------
flow file content...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-09-24 10:31:04 -04:00
24c080788f
NIFI-604: Removed unused import, as it caused checkstyle violation
2015-09-24 10:06:39 -04:00
193dbe6533
Merge branch 'NIFI-604' of https://github.com/rickysaltzer/nifi into NIFI-604
2015-09-23 16:03:46 -04:00
ee7d89cb01
NIFI-988: Test cases for PutDistributedMapCache
2015-09-23 14:32:37 +02:00
6b1328f3f1
NIFI-988: PutDisributedMapCache processor implementation
2015-09-23 14:31:13 +02:00
5764e89293
NIFI-975 Fixing NullPointerException when one of the delimiters is not provided when using the Text strategy
2015-09-22 13:19:15 -04:00
14eaeeb1ee
NIFI-984: Bad class used for logging in TestDetectDuplicate
...
Using TestListenUDP instead of TestDetectDuplicate when creating the logger.
This closes #89
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-09-22 08:51:58 -04:00
600f91a262
NIFI-932 Adding hasIncomingConnection() and hasConnection(Relationship) to ProcessContext and updating ExecuteSQL
2015-09-18 17:05:31 -04:00
ded74ec94c
NIFI-876 prepare for next development iteration
2015-09-14 21:48:11 -04:00
2ec735e350
NIFI-876 prepare release nifi-0.3.0-RC1
2015-09-14 21:48:00 -04:00
cd2e1424cb
NIFI-911:
...
- Updating default value for Regex so it matches once (?s:^.*$) instead of twice (.*). Matching on .* results in matching for every character and then again for 0 characters.
2015-08-31 14:58:38 -04:00
9cefc4a5a5
NIFI-604: Custom Argument Delimiters ExecuteStreamCommand / ExecuteProcess
...
- Unified the way ExecuteStreamCommand and ExecuteProcess handle arguments
- Argument delimiters can now be specified. Their default being what they were using before (; and space)
2015-08-27 15:26:56 -04:00
c7f7704220
Changed line wrap length to 200 and combined adjacent imports into single try-with-resources blocks.
...
This closes #76
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-08-23 23:07:50 -04:00
b30fbe1f3b
NIFI-888 EncryptContent for PGP Leaks File Handles.
...
Closes InputStreams created to read the public keys for PGP encryption and several other
streams involved in PGP encryptiong. This prevents NiFi from leaking file handles on
every validate call or encryption attempt in the EncryptContent processor.
2015-08-23 23:07:26 -04:00
4d4f74cf05
NIFI-889: Return the failure relationshp in the set of allowed relationships
2015-08-22 20:47:45 -04:00
671445f10a
Merge branch 'NIFI-844'
2015-08-21 17:15:39 -04:00
a9c2bf58b8
NIFI-844: Initial import of the ConvertAvroToJSON processor
2015-08-21 17:11:56 -04:00
82d32230e6
NIFI-881: Added support for Proxy Host and Proxy Port properties to InvokeHTTP
...
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-08-21 11:35:30 -04:00
d421e3c242
NIFI-846 MonitorActivity not setting start of evaluation period correctly
2015-08-21 11:06:20 -04:00
28549c2b1e
NIFI-853: Made updates to processors
2015-08-19 17:36:29 -04:00
1a37c95f58
NIFI-293: Added processors ConvertFlatJSONToSQL, PutSQL
2015-08-19 17:36:29 -04:00
f044ba5d45
NIFI-808 Providing property to allow exclusion of capture group 0 for any regex expressions as an attribute.
2015-08-19 12:00:25 -04:00
740b09b182
NIFI-796 Adjusting ControlRate documentation to reflect that a long is used instead of an integer.
2015-08-17 11:51:10 -04:00
a837fa5f38
NIFI-801 fixed reference to BSD and name of NOTICE
2015-08-16 22:42:28 -04:00
aa99884782
NIFI-850 removed nifi parent, updated nifi pom, moved all nifi subdirs up one level, fixed readme.
2015-08-15 13:12:22 -04:00
Aldrin Piri
Andy LoPresto
jpercivall
Matt Gilman
Mark Payne
Daniel Cave
joewitt
Richard Miskin
Oleg Zhurakousky
Matt Burgess
Bryan Bende
smarthi
Jeremy Dyer
James Wing
Joe Skora
gresockj
ricky
James Harrison
Mike Moser
Tony Kurc
Luke Williamson
Toivo Adams
Andre F de Miranda
Joe
Alan Jackoway