Commit Graph

562 Commits

Author SHA1 Message Date
Pierre Villard 6ff8321cf7
NIFI-4278 - add error message to invalid FFs in ValidateXml
This closes #2069.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-09 21:20:08 -07:00
Matt Burgess 9c4fdd4ef3 NIFI-4192: Add more Avro/metadata merge options to MergeContent
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2067.
2017-08-09 12:55:12 +02:00
Bryan Bende 0029f025f8 NIFI-4152 Initial commit of ListenTCPRecord 2017-08-07 22:44:11 +02:00
Wil Selwood f7f64a572c NIFI-4268: Add fetch tag to FetchFile processor.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2062.
2017-08-07 21:38:55 +02:00
Andy LoPresto 675d989003
NIFI-4139
- Moved key provider interface and implementations from nifi-data-provenance-utils module to nifi-security-utils module.
- Refactored duplicate byte[] concatenation methods from utility classes and removed deprecation warnings from CipherUtility.
- Created KeyProviderFactory to encapsulate key provider instantiation logic.
- Added logic to handle legacy package configuration values for key providers.
- Added unit tests.
- Added resource files for un/limited strength cryptography scenarios.
- Added ASL to test resources.
- Moved legacy FQCN handling logic to CryptUtils.
- Added unit tests to ensure application startup logic handles legacy FQCNs.
- Moved master key extraction/provision out of FBKP.
- Removed nifi-security-utils dependency on nifi-properties-loader module.
- Added unit tests.
2017-08-07 15:21:58 -04:00
Mark Payne 969bbe654c NIFI-4051: Ensure that we use the appropriate Lexical syntax for SQL query validation as we do when running
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2054.
2017-08-04 16:31:56 +02:00
Matt Burgess 2d33582147 NIFI-4253: Add Oracle 12+ database adapter
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2048.
2017-08-03 09:59:29 +02:00
Leah Anderson 70878fe6d6 NIFI-4052 Add DynamicProperty annotation for LookupRecord
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2032.
2017-07-28 17:12:06 +02:00
Matt Burgess dc4006f423 NIFI-3335: Add initial.maxvalue support to GenerateTableFetch
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2039.
2017-07-27 08:43:00 +02:00
Leah Anderson 3371e915cc NIFI-4087 This closes #2026. Fix to allow exclusion of filename from tika criteria. 2017-07-21 16:56:56 -04:00
Andre F de Miranda b0be99036d NIFI-55 - Ensures ListenHTTP logs the source of an expired hold
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1623.
2017-07-20 23:14:32 +02:00
Koji Kawamura b4e0a6e206 NIFI-4205: Avoid duplicated data from TailFile
Before this fix, it is possible that TailFile to produce duplicated data
if an already tailed file has newer timestamp and fewer or the same
amout of data.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2021.
2017-07-20 12:14:05 +02:00
Bryan Bende f87d2a2f57 NIFI-4157 Improvements to PutTCP
Signed-off-by: Bryan Bende <bbende@apache.org>
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1989.
2017-07-19 12:25:42 +02:00
joewitt 59754500d5 NIFI-2829 ignoring unit test that appears broken. Commented on JIRA about the problem observed 2017-07-18 22:23:10 -04:00
Andre F de Miranda a307ea3a00 NIFI-4098 This closes #2013. Adjust ParseCEF to re-use bean validator in order to improve performance 2017-07-18 22:02:39 -04:00
Matt Burgess 844dbe4edb NIFI-4156: Fixed fragment.count in SplitText to equal emitted flow files
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2014.
2017-07-18 11:31:16 +02:00
deonhuang a6e94de0bb NIFI-2829 - Add Date and Time Format Support for PutSQL
Fix unit test for Date and Time type time zone problem
Enhance Time type to record milliseconds

This closes #1983.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-07-18 14:16:10 +09:00
patricker 03bff7c9fc NIFI-2829 Date and Time Format Support for PutSQL
This closes #1524.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-07-18 14:11:37 +09:00
Koji Kawamura 8acee02393 NIFI-1613
- Truncate text data types only.
- Added conversion from a boolean to number.
2017-07-12 17:02:55 -04:00
Koji Kawamura 3844a821f1 NIFI-1613: ConvertJSONToSQL truncates numeric value wrongly.
- Changed boolean value conversion to use Boolean.valueOf.
- Updated comments in source code to reflect current situation more clearly.
- Updated tests those have been added since the original commits were made.
2017-07-12 17:02:55 -04:00
Toivo Adams 3b2e43b75c NIFI-1613 Initial version, try to improve conversion for different SQL types. New test and refactored existing test to reuse DBCP service.
nifi-1613 Adding numeric and Date/time types conversion and test.
2017-07-12 17:02:55 -04:00
Mark Payne b603cb955d NIFI-4060: Initial implementation of MergeRecord
NIFI-4060: Addressed threading issue with RecordBin being updated after it is completed; fixed issue that caused mime.type attribute not to be written properly if all incoming flowfiles already have a different value for that attribute

NIFI-4060: Bug fixes; improved documentation; added a lot of debug information; updated StandardProcessSession to produce more accurate logs in case of a session being committed/rolled back with open input/output streams
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1958
2017-07-12 16:36:48 -04:00
mans2singh 0fd51b4d12 NIFI-4145 Added writes attributes annotation
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1998.
2017-07-10 21:25:26 -07:00
Tony Kurc 2dc45a4dd7 NIFI-4154 Fixing line endings in .java,.html files
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1982.
2017-07-09 12:13:15 -07:00
Koji Kawamura e84f9a2416 NIFI-4160: SFTPTransfer connection timeout for opening channel.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1991.
2017-07-07 12:31:50 +02:00
Koji Kawamura 50c364a793 NIFI-4162: PutSQL batch update error message should include the cause
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1992.
2017-07-07 12:25:10 +02:00
m-hogue e6b166a3a2 NIFI-552: added regex properties for include and ignore filters in LogAttribute
This closes #1981

Signed-off-by: Tony Kurc <tkurc@apache.org>
2017-07-05 21:21:50 -04:00
Pierre Villard 8b5342dec0 NIFI-3974 - Added charset in LogAttribute
This closes #1967

Signed-off-by: Tony Kurc <tkurc@apache.org>
2017-07-02 12:03:07 -04:00
Pierre Villard 902b6e205d NIFI-4131 - GFT warning message when badly setting relationships
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1954
2017-06-30 15:12:10 -04:00
yuri1969 3bf1d12706
NIFI-4125 Added secure transform feature and configuration to TransformXML processor to mitigate XXE file system leaks.
This closes #1946.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-29 10:40:33 -07:00
Matt Burgess 9bfa7469cb NIFI-4137: Add support for UTF-8 in GetFTP and PutFTP
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1957
2017-06-28 16:41:14 -04:00
Andre F de Miranda 1e7eceee84 NIFI-4126 - Add SupportBatching, SideEffectFree and EventDriven annotations to ExtractGrok
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1950.
2017-06-27 20:56:56 +02:00
Koji Kawamura 91ed96f8c4 NIFI-4040: Continue if SFTP failed getting user home directory
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1898.
2017-06-26 19:09:22 +02:00
Andy LoPresto 253ea2e73b NIFI-4095 Changed minimum capture group count in ExtractText from 1 to 0.
Added unit test and removed obsolete test.
Added custom validation to enforce capture group if "include capture group 0" is false.
2017-06-22 23:31:35 +02:00
Koji Kawamura 5c755c006b
NIFI-4017: Emit provenance event from Notify.
This closes #1890.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-22 16:49:31 -04:00
Pierre Villard f54e146561
NIFI-4085 Add exception in log message when (S)FTP listing fails
This closes #1929.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-22 16:06:18 -04:00
Jeff Storck c99100c934
NIFI-4010 Enables EL on Fetch/List/PutSFTP and List/Fetch/Put/DeleteHDFS processor properties
FetchSFTP/ListSFTP/PutSFTP: Private Key Path
ListHDFS/FetchHDFS/PutHDFS/DeleteHDFS: Hadoop Configuration Resources, Kerberos Principal, Kerberos Keytab, Kerberos Relogin Period

This closes #1148
This closes #1930.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-21 17:14:49 -04:00
Bryan Bende aabd4a25d2 NIFI-4043 Initial commit of nifi-redis-bundle
NIFI-4061 Initial version of RedisStateProvider
- Adding PropertyContext and updating existing contexts to extend it
- Added embedded Redis for unit testing
- Added wrapped StateProvider with NAR ClassLoader in StandardStateManagerProvider
- Updating state-management.xml with config for RedisStateProvider
- Renaming tests that use RedisServer to be IT tests so they don't run all the time

This closes #1918.
2017-06-21 12:47:19 -04:00
Matt Gilman 490e1da5db NIFI-4019:
- Adding support for X-Forwarded-* headers.
- Unrelated code clean up.
- Addressing additional proxying issues.
- Addressing mistyped landing page.
- Handled trailing slashes in proxy headers for context path
2017-06-15 14:17:35 -04:00
Koji Kawamura de1b84e2aa
NIFI-4049: Refactor AtomicDistributedMapCacheClient
To be used with cache engines that does not have revision number.

NIFI-4049: Refactor AtomicDistributedMapCacheClient

Removed old methods completely.

This closes #1904.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-12 10:34:48 -04:00
Koji Kawamura 7761903352 NIFI-4041: Add file.size attribute to ListFTP and ListSFTP
This closes #1899.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-06-12 10:53:09 +09:00
Mark Payne e7dcb6f6c5 NIFI-3921: Allow Record Writers to inherit schema from Record
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1902
2017-06-09 16:13:25 -04:00
Mark Payne d8dc2bb00b NIFI-4047: Addressed issue with inconsistent Lexical policy between used cached schemas and non-cached schemas
This closes #1907

Signed-off-by: Joe Percivall <JPercivall@apache.org>
2017-06-09 12:05:11 -04:00
Matt Gilman cc741d2be6
NIFI-3997:
- Bumping to next minor version.
2017-06-08 15:22:51 -04:00
Matt Gilman 6ee12e9b47
NIFI-3997-RC1prepare for next development iteration 2017-06-05 11:07:43 -04:00
Matt Gilman ddb73612bd
NIFI-3997-RC1prepare release nifi-1.3.0-RC1 2017-06-05 11:07:28 -04:00
Andre F de Miranda 082b99640e NIFI-4001 - Bump ParCEFone version
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1875.
2017-06-03 12:11:38 +02:00
Mark Payne 32314d70fd NIFI-4009: Added support for several key functions in RecordPath
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1881
2017-06-02 14:06:05 -04:00
Pierre Villard 80dfe0257f ExtractGrok - Include exception message when compile fails
This closes: #1845

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-05-27 00:56:25 +10:00
Koji Kawamura 13b59b5621 NIFI-3958: Decimal logical type with undefined precision and scale.
- Oracle NUMBER can return 0 precision and -127 or 0 scale with variable scale NUMBER such as ROWNUM or function result
- Added 'Default Decimal Precision' and 'Default Decimal Scale' property to ExecuteSQL and QueryDatabaseTable to apply default precision and scale if those are unknown
- Coerce BigDecimal scale to field schema logical type, so that BigDecimals having different scale can be written

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1851
2017-05-25 14:37:17 -04:00