Commit Graph

538 Commits

Author SHA1 Message Date
Koji Kawamura e84f9a2416 NIFI-4160: SFTPTransfer connection timeout for opening channel.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1991.
2017-07-07 12:31:50 +02:00
Koji Kawamura 50c364a793 NIFI-4162: PutSQL batch update error message should include the cause
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1992.
2017-07-07 12:25:10 +02:00
m-hogue e6b166a3a2 NIFI-552: added regex properties for include and ignore filters in LogAttribute
This closes #1981

Signed-off-by: Tony Kurc <tkurc@apache.org>
2017-07-05 21:21:50 -04:00
Pierre Villard 8b5342dec0 NIFI-3974 - Added charset in LogAttribute
This closes #1967

Signed-off-by: Tony Kurc <tkurc@apache.org>
2017-07-02 12:03:07 -04:00
Pierre Villard 902b6e205d NIFI-4131 - GFT warning message when badly setting relationships
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1954
2017-06-30 15:12:10 -04:00
yuri1969 3bf1d12706
NIFI-4125 Added secure transform feature and configuration to TransformXML processor to mitigate XXE file system leaks.
This closes #1946.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-29 10:40:33 -07:00
Matt Burgess 9bfa7469cb NIFI-4137: Add support for UTF-8 in GetFTP and PutFTP
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1957
2017-06-28 16:41:14 -04:00
Andre F de Miranda 1e7eceee84 NIFI-4126 - Add SupportBatching, SideEffectFree and EventDriven annotations to ExtractGrok
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1950.
2017-06-27 20:56:56 +02:00
Koji Kawamura 91ed96f8c4 NIFI-4040: Continue if SFTP failed getting user home directory
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1898.
2017-06-26 19:09:22 +02:00
Andy LoPresto 253ea2e73b NIFI-4095 Changed minimum capture group count in ExtractText from 1 to 0.
Added unit test and removed obsolete test.
Added custom validation to enforce capture group if "include capture group 0" is false.
2017-06-22 23:31:35 +02:00
Koji Kawamura 5c755c006b
NIFI-4017: Emit provenance event from Notify.
This closes #1890.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-22 16:49:31 -04:00
Pierre Villard f54e146561
NIFI-4085 Add exception in log message when (S)FTP listing fails
This closes #1929.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-22 16:06:18 -04:00
Jeff Storck c99100c934
NIFI-4010 Enables EL on Fetch/List/PutSFTP and List/Fetch/Put/DeleteHDFS processor properties
FetchSFTP/ListSFTP/PutSFTP: Private Key Path
ListHDFS/FetchHDFS/PutHDFS/DeleteHDFS: Hadoop Configuration Resources, Kerberos Principal, Kerberos Keytab, Kerberos Relogin Period

This closes #1148
This closes #1930.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-21 17:14:49 -04:00
Bryan Bende aabd4a25d2 NIFI-4043 Initial commit of nifi-redis-bundle
NIFI-4061 Initial version of RedisStateProvider
- Adding PropertyContext and updating existing contexts to extend it
- Added embedded Redis for unit testing
- Added wrapped StateProvider with NAR ClassLoader in StandardStateManagerProvider
- Updating state-management.xml with config for RedisStateProvider
- Renaming tests that use RedisServer to be IT tests so they don't run all the time

This closes #1918.
2017-06-21 12:47:19 -04:00
Matt Gilman 490e1da5db NIFI-4019:
- Adding support for X-Forwarded-* headers.
- Unrelated code clean up.
- Addressing additional proxying issues.
- Addressing mistyped landing page.
- Handled trailing slashes in proxy headers for context path
2017-06-15 14:17:35 -04:00
Koji Kawamura de1b84e2aa
NIFI-4049: Refactor AtomicDistributedMapCacheClient
To be used with cache engines that does not have revision number.

NIFI-4049: Refactor AtomicDistributedMapCacheClient

Removed old methods completely.

This closes #1904.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-12 10:34:48 -04:00
Koji Kawamura 7761903352 NIFI-4041: Add file.size attribute to ListFTP and ListSFTP
This closes #1899.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-06-12 10:53:09 +09:00
Mark Payne e7dcb6f6c5 NIFI-3921: Allow Record Writers to inherit schema from Record
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1902
2017-06-09 16:13:25 -04:00
Mark Payne d8dc2bb00b NIFI-4047: Addressed issue with inconsistent Lexical policy between used cached schemas and non-cached schemas
This closes #1907

Signed-off-by: Joe Percivall <JPercivall@apache.org>
2017-06-09 12:05:11 -04:00
Matt Gilman cc741d2be6
NIFI-3997:
- Bumping to next minor version.
2017-06-08 15:22:51 -04:00
Matt Gilman 6ee12e9b47
NIFI-3997-RC1prepare for next development iteration 2017-06-05 11:07:43 -04:00
Matt Gilman ddb73612bd
NIFI-3997-RC1prepare release nifi-1.3.0-RC1 2017-06-05 11:07:28 -04:00
Andre F de Miranda 082b99640e NIFI-4001 - Bump ParCEFone version
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1875.
2017-06-03 12:11:38 +02:00
Mark Payne 32314d70fd NIFI-4009: Added support for several key functions in RecordPath
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1881
2017-06-02 14:06:05 -04:00
Pierre Villard 80dfe0257f ExtractGrok - Include exception message when compile fails
This closes: #1845

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-05-27 00:56:25 +10:00
Koji Kawamura 13b59b5621 NIFI-3958: Decimal logical type with undefined precision and scale.
- Oracle NUMBER can return 0 precision and -127 or 0 scale with variable scale NUMBER such as ROWNUM or function result
- Added 'Default Decimal Precision' and 'Default Decimal Scale' property to ExecuteSQL and QueryDatabaseTable to apply default precision and scale if those are unknown
- Coerce BigDecimal scale to field schema logical type, so that BigDecimals having different scale can be written

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1851
2017-05-25 14:37:17 -04:00
Koji Kawamura 4d0667380a NIFI-3404: Improved UX of LookupAttributes.
- Added dependency notice.
- Added EL evaluation at SimpleKeyValueLookupService.
- Updated documentation.
- Updated CommonsConfigurationLookupService to throw LookupFailureException if it fails to get configuration so that error messages can be displayed at each processor bulletin.
- Added calling getConfiguration at OnEnabled of CommonsConfigurationLookupService, so that the service will stay in Enabling state if there is any issue.

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2017-05-25 10:05:32 -05:00
Joey Frazee 46e2420d74 NIFI-3404 Added LookupAttribute processor and lookup controller services
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2017-05-25 10:05:32 -05:00
Andre F de Miranda c07850aec3 NIFI-2702 - Deprecates nifi-standard-bundle/*JMS and document the recommended alternative
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1643.
2017-05-25 08:44:39 +02:00
Mark Payne 08b66b5b6a NIFI-3969: Prevent merging flowfiles prematurely when all bins fill but some are already full and can be processed
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1850.
2017-05-24 19:36:18 +02:00
Andy LoPresto 6d07ebf50c NIFI-3954 This closes #1841. Upgraded version of httpclient library to handle TLS issues.
Added explicit dependency on httpcore for @ThreadSafe annotation.

Signed-off-by: joewitt <joewitt@apache.org>
2017-05-23 23:42:28 -04:00
Mark Payne 6d16fdf170
NIFI-3952: Updated UpdateRecord to pass field-related variables to the Expression Language
This closes #1836.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-22 14:32:45 -04:00
Mark Payne 7f8987471d
NIFI-3946: Update LookupService to take a Map instead of a String for the input
NIFI-3946: Fixed issues where null values were returned instead of empty optionals

This closes #1833.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-22 11:52:41 -04:00
Mark Payne c49933f03d NIFI-3948: This closes #1834. Added flush() method to RecordWriter and call it when writing a single record to OutputStream for PublishKafkaRecord. Also removed no-longer-used class WriteAvroResult
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-19 23:05:04 -04:00
Mark Payne ae9953db64 NIFI-3857: This closes #1825. Added PartitionRecord processor
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-19 02:08:52 -04:00
Mark Payne 9bd0246a96 NIFI-3863: Initial implementation of Lookup Services. Implemented LookupRecord processors. This required some refactoring of RecordSetWriter interface, so refactored that interface and all implementations and references of it 2017-05-19 01:02:41 -04:00
Koji Kawamura 3fd7d3b272 NIFI-3873: Improve Avro logical types readability
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1788
2017-05-17 13:08:00 -04:00
Peter G. Horvath bf15502e19 NIFI-3763 Add new processor to log user defined messages built with NiFi Expression Language
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1737
2017-05-15 14:18:57 -04:00
Koji Kawamura 1811ba5681 NIFI-2624: Avro logical types for ExecuteSQL and QueryDatabaseTable
- Added Logical type support for DECIMAL/NUMBER, DATE, TIME and TIMESTAMP column types.
- Added Logical type 'decimal' to AvroReader so that Avro records with logical types written by ExecuteSQL and QueryDatabaseTable can be consumed by AvroReader.
- Added JdbcCommon.AvroConversionOptions to consolidate conversion options.
- Added 'Use Avro Logical Types' property to ExecuteSQL and QueryDatabaseTable to toggle whether to use Logical types.
- Added 'mime.type' FlowFile attribute as 'application/avro-binary' so that output FlowFiles can be displayed by content viewer.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1798
2017-05-15 14:15:23 -04:00
Mark Payne b1901d5fe0 NIFI-3838: Initial implementation of RecordPath and UpdateRecord processor
NIFI-3838: Updated version from 1.2.0-SNAPSHOT to 1.3.0-SNAPSHOT; removed unneeded value from AttributeExpression.ResultType enum

NIFI-3838: Addressed PR Review feedback

NIFI-3838: Allow for schemas to be merged together for a record; refactored RecordSetWriterFactory so that there is a method to obtain the schema and then the writer is created with that schema. Added additional unit tests

NIFI-3838: Addressed problems with documentation based on PR Review

NIFI-3838: Fixed checkstyle violation

NIFI-3838: Addressed issue of comparing different types of Number objects

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1772
2017-05-12 12:36:52 -04:00
Matt Gilman dd0306cce7 NIFI-3837: - Being more specific regarding the escaping of back references in evaluated expressions.
Signed-off-by: Matt Burgess <mattyb149@apache.org>

fixed typo in comment

This closes #1781
2017-05-11 22:16:46 -04:00
Bryan Bende 3af53419af
NIFI-3770-RC2 prepare for next development iteration 2017-05-05 20:50:28 -04:00
Bryan Bende 3a605af8e0
NIFI-3770-RC2 prepare release nifi-1.2.0-RC2 2017-05-05 20:50:14 -04:00
joewitt f4f174b899
NIFI-3826 added proper NOTICE entries for apache calcite and maxmind source
This closes #1766.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 20:00:22 -04:00
joewitt 829c02b058 NIFI-3825 fix L&N for jolt, json, and source deps
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1765
2017-05-05 18:43:04 -04:00
joewitt ee8dc7640f
NIFI-3820 added calcite to assembly notice and updated all copyright years for compress, avro, and jetty
This closes #1764.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 14:45:40 -04:00
Mark Payne 12249e5a08
NIFI-2268: Instead of allowing HandleHttpRequest to run constantly calling BlockingQueue.poll() as fast as possible, call BlockingQueue.poll(2, TimeUnit.MILLISECONDS) to avoid overutilization of CPU when there is no work to be done
This closes #1762.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 13:58:10 -04:00
Andre F de Miranda 970ddf8f7e
NIFI-3794 - Expose the control of ListenRELP's CLIENT_AUTH property to DFM
This closes #1746.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-04 10:50:22 -04:00
Jeff Storck 26d90fbccf
NIFI-1833 - Addressed issues from PR review.
NIFI-1833 Moved AbstractListProcessor.java, EntityListing.java, and ListableEntity.java from nifi-standard-processors into nifi-processor-utils
Moved TestAbstractListProcessor.java into nifi-processor-utils
Set nifi-azure-nar's nar dependency back to nifi-standard-services-api-nar
Fixed failing integration tests (ITFetchAzureBlobStorage.java, ITListAzureBlobStorage.java, and ITPutAzureStorageBlob.java) and refactored them to be able to run in parallel

NIFI-1833 Moved security notice info in the additional details documentation into the descriptions of the specific attributes for which those notices are intended
Added displayName usage to properties
Updated exception handling in FetchAzureBlobStorage.java and PutAzureBlobStorage.java to cause flowfiles with Output/InputStreamCallback failures to be routed to the processor's failure relationship
Cleaned up dependencies in pom

NIFI-1833 Removed unnecessary calls to map on Optional in the onTrigger exception handling of FetchAzureBlobStorage.java and PutAzureBlobStorage.java

NIFI-1833 Updates due to nifi-processor-utils being moved under nifi-nar-bundles

This closes #1719.

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-05-02 14:39:46 -04:00
Andy LoPresto 7d242076ce
NIFI-3594 Implemented encrypted provenance repository.
Added src/test/resources/logback-test.xml files resetting log level from DEBUG (in nifi-data-provenance-utils) to WARN because later tests depend on MockComponentLog recording a certain number of messages and this number is different than expected if the log level is DEBUG.

This closes #1686.

Signed-off-by: Bryan Bende, Yolanda M. Davis, and Mark Payne
2017-05-02 13:24:07 -04:00