Commit Graph

885 Commits

Author SHA1 Message Date
Scott Aslan c1459825bb
[NIFI-4998] update node and npm version
This closes #2571
2018-03-28 16:49:32 -04:00
Mark Payne f394c874e1
NIFI-4895: Added backend code to give users the ability to forcibly terminate a processor if the processor does not complete after clicking Stop
NIFI-4895: Addressed review feedback; addressed checkstyle issues
This closes #2555
2018-03-28 12:02:47 -04:00
joewitt 7b5bf265a6 NIFI-4995 updating copyright year on all notices 2018-03-26 21:54:10 -04:00
Joe Percivall 56ad14ad1e
NIFI-4977 Adding expression language support to the Sender properties of PutSyslog
This closes #2547.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-03-26 15:49:52 -04:00
Matt Gilman 69a564e4c8
NIFI-5009:
- Fixing required permission for PutParquet.

NIFI-5008:
- Ensuring all restricted components are tagged as such.

This closes #2583.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-03-26 09:47:54 -04:00
Otto Fowler 98c295dfc7
NIFI-5013 create test jar for standard processors
remove unnec TestServer instances

This closes #2580.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2018-03-24 10:18:46 -04:00
Marco Gaido 758e44682f NIFI-4631: Use java.nio.file.Files in ListFile to improve performance
This closes #2565.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-23 11:00:21 -04:00
sbouchex 9c9215982f NIFI-4999: Added request URI into the "restlistener.request.uri" attribute
This closes #2574.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-21 13:17:17 -04:00
Andy LoPresto d09145e3e3 NIFI-4557 Added Expression Language support for PGP private keyring passphrase in EncryptContent.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2566.
2018-03-20 11:23:45 +01:00
Derek Straka 5bdb7cf6e7 NIFI-4912: This closes #2494. Update jackson version to latest stable version (2.9.4)
Signed-off-by: joewitt <joewitt@apache.org>
2018-03-19 10:22:50 -04:00
Pierre Villard 4687e0fb35 NIFI-4981 Allow using FF's content as message in PutEmail
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2556.
2018-03-18 11:35:43 -07:00
Mark Bean 8b2c5b7246 NIFI-4950 Defining behavior for MergeContent when more than 1 FlowFile has the same fragment.index value
Signed-off-by: Mike Moser <mosermw@apache.org>

This closes #2557.
2018-03-16 18:31:43 +00:00
Marco Gaido 86f162b611 NIFI-4959: Remove flowfiles and close connection for Bad Requests causing IOException
This closes #2544.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-16 12:24:00 -04:00
Bryan Bende de71a41bd0 NIFI-4935 Refactoring to support specifying schema branch or schema version when using schema by name strategy
This closes #2523.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-15 16:16:12 -04:00
Deon Huang 37ad48844a NIFI-4962 FlattenJson processor add unexpected backslash after flatten
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2538.
2018-03-13 22:33:53 +01:00
Pierre Villard 11a7d5c09f NIFI-4955 - Preserve columns ordering with CSV and ValidateRecord
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2528
2018-03-13 17:09:07 -04:00
m-hogue 6471f66acd NIFI-3402 - Added etag support to InvokeHTTP
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2150.
2018-03-13 10:00:11 +01:00
Deon Huang 7aabb99bcc NIFI-4800 Expose the flattenMode as property in FlattenJSON processor
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2530.
2018-03-13 09:28:22 +01:00
Matt Gilman b1217f529b NIFI-4885:
- Introducing more granular restricted component access policies.

This closes #2515.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-12 14:19:42 -04:00
joewitt c71409fb5d
NIFI-4936 trying to quiet down the mvn output a bit so we dont exceed the travis-ci 4MB max
NIFI-4936 updated dependency handling pushing down delcarations where they belong
This closes #2512
2018-03-09 16:34:53 -05:00
Andy LoPresto e9d585a785
NIFI-4870 Upgraded activemq-client and activemq-broker versions to 5.15.3.
- Updated copyright dates in NOTICE files.
- This closes #2469.
2018-03-09 13:59:13 -05:00
Jeff Storck e916594b69 NIFI-4872 Added annotation for specifying scenarios in which components can cause high usage of system resources.
- Initial set of components marked with the HighResourceUsageScenario annotation.
- Added customized descriptions to SystemResourceConsideration annotations for MergeContent, SplitContent, SplitJson, SplitText, and SplitXml.

This closes #2475.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-05 15:32:23 -05:00
Matthew Burgess dd58a376c9 NIFI-4773 - Fixed column type map initialization in QueryDatabaseTable
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2504.
2018-03-02 17:32:14 +01:00
Matthew Burgess 62732cbb88 NIFI-4903 - Fixed timestamp bug with fetch processors using Oracle 12+
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2488.
2018-02-22 20:56:30 +01:00
Pierre Villard ff95be2a11 NIFI-4815 - Add EL support to ExecuteProcess
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2432
2018-02-21 12:47:33 -05:00
Matthew Burgess 2a5e21c11b NIFI-4836 - Allow output of FlowFiles during result set processing in QueryDatabaseTable
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2447.
2018-02-17 15:56:05 +01:00
Andy LoPresto 28067a29fd
NIFI-4869 Added SAX utility method for SplitXML processor.
Added unit tests.
Added test resources to RAT exclude list.
This closes #2466
2018-02-13 15:29:29 -05:00
Matthew Burgess b5ca7adbb9 NIFI-978: Support parameterized statements in ExecuteSQL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2433.
2018-02-12 20:44:31 +01:00
Matthew Burgess 7d6bbce123 NIFI-4830: Fixed logic errors in BLOB/CLOB processing in JdbcCommon
This closes #2459.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-02-09 11:45:12 +09:00
Mark Payne 4428fe28bf
NIFI-4858: Expose Request Timeout from HTTP Context Map and use that to set as the timeout of the 'AsyncContext' in HandleHttpRequest. Otherwise, the request will never timeout (which is OK because the HttpContextMap will handle this). However, Jetty behind the scenes is adding a task to Scheduled Executor for each request with a delay of whatever the timeout is set to. Since it's currently set to Long.MAX_VALUE, that task will never be run and as a result the ExecutorService's queue will grow indefinitely, eventually exhausting the JVM Heap
This closes #2460.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-02-08 17:38:13 -05:00
Mike Moser a2f2ddd6b8 NIFI-4699 Use a filter in PostHTTP to pull flowfiles from queue whose URL is the same
This closes #2412.
2018-02-08 18:36:11 +00:00
Matthew Burgess 16e56ccfca NIFI-4080: Restored customValidate in ValidateCSV for schema property with no EL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2226.
2018-02-08 11:56:56 +01:00
Marco Gaido ba192d22da enable EL on FF level 2018-02-08 11:56:49 +01:00
Matthew Burgess 836c324258 NIFI-4080: Added EL support to fields in ValidateCSV 2018-02-08 11:56:42 +01:00
Matt Gilman f3013d0764
NIFI-4837:
- When Jetty initializes fails, performing a shutdown sequence to ensure all allocated resources are released.

This closes #2455.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-02-07 16:40:23 -05:00
Koji Kawamura e9af6c6ad8 NIFI-4828: Fix MergeContent to process all ready bins
Before this fix, MergeContent only processed the first bin even if there
were multiple bins.

There were two unit tests marked with Ignore those had been
failing because of this.

This closes #2444.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-02-07 15:45:39 -05:00
Koji Kawamura dbbf78f22c
NIFI-4848: Update HttpComponents version
- httpclient 4.5.3 -> 4.5.5
- httpcore 4.4.4 -> 4.4.9
  - ThreadSafe annotation is removed since 4.4.5, HTTPCLIENT-1743.
    Removed the annotation from DebugFlow processor.
- httpasyncclient 4.1.2 -> 4.1.3
- This closes #2453
2018-02-07 08:55:43 -05:00
Deon Huang b5938062a8 NIFI-4395 GenerateTableFetch can't fetch column type by state after instance reboot
Add multiple states recover

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2442
2018-02-02 11:15:46 -05:00
Marco Gaido 37271e8241 NIFI-4790 - support HTTPS Proxy in InvokeHTTP
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2426.
2018-01-24 17:51:28 +01:00
Matthew Burgess 84848f7cbb NIFI-4773: Moved DB Fetch processors' connection code from setup to onTrigger
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2422.
2018-01-23 15:10:34 +01:00
Mark Payne 6f282c6843
NIFI-4803: Ensure that we set the FileCountAttribute of BinManager to the proper value, regardless of the merge strategy
This closes #2423.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-01-22 15:37:31 -05:00
joewitt 41ce788812 NIFI-4751 changed to next minor release version snapshot 2018-01-12 15:15:32 -05:00
joewitt 36405e888c NIFI-4751-RC1 prepare for next development iteration 2018-01-08 23:39:49 -07:00
joewitt 46d30c7e92 NIFI-4751-RC1 prepare release nifi-1.5.0-RC1 2018-01-08 23:39:32 -07:00
Mike Moser 6e7544bd37 NIFI-4697 This closes #2381. clarified some PostHTTP documentation
Signed-off-by: joewitt <joewitt@apache.org>
2018-01-08 21:47:27 -07:00
Mark Payne 953e922d32 NIFI-4749: This closes #2386. Pass the record reader's schema along to the ResultSetRecordSet so that it is able to resolve the schema for Record fields
Signed-off-by: joewitt <joewitt@apache.org>
2018-01-08 19:47:18 -07:00
Andy LoPresto a7f1eb89c2
NIFI-4727 Added CountText processor and unit test.
This closes #2371.

Signed-off-by: Kevin Doran <kdoran.apache@gmail.org>
2018-01-08 14:26:14 -08:00
Mike Thomsen d9866c75e2 NIFI-4649 Added FlattenJson processor.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2307

Replaced star imports, removed unused import

Added explanation for invalid Expression
2018-01-05 18:14:30 -05:00
Mark Payne 3b10a8479e Fixed bug in unit tests that were causing test failures 2018-01-05 09:59:01 -05:00
Matthew Burgess ce4374ee00
NIFI-4738: Fixed logic bug in JdbcCommon for 9-digit unsigned ints. This closes #2373 2018-01-04 15:20:35 -05:00
Pierre Villard 99d767aa44 NIFI-4383 - Fix UpdateRecord when updating arrays elements. This closes #2208.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-01-04 14:30:14 -05:00
joewitt 6ee6b5e57c NIFI-4734 refactored couchbase bundle, jms bundle, removed jms test-lib usage, fixed broken SeeAlso and service links causing docs warns. Corrected L&N in each bundle. This closes #2369.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-01-04 09:16:17 -05:00
Koji Kawamura 1959586389 NIFI-4729: This closes #2366. ConvertJSONToSQL converted boolean values in reverse
Signed-off-by: joewitt <joewitt@apache.org>
2018-01-02 22:01:40 -05:00
Mark Payne c91d99884a NIFI-4717: Several minor bug fixes and performance improvements around record-oriented processors
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2359
2017-12-29 10:43:21 -05:00
Mike Moser c59a967623 NIFI-4504, NIFI-4505 added removeAndGet, removeByPatternAndGet, and keySet methods to MapCache API
cleaned up some warnings on deprecated nifi.stream.io classes

This closes #2284.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-12-29 00:23:09 +09:00
Marco Gaido 2fbe922a2b NIFI-2169: This closes #2343. Cache compiled regexp for RouteText
Signed-off-by: joewitt <joewitt@apache.org>
2017-12-26 18:02:02 -05:00
Norito Agetsuma f04ddcf442 NIFI-4539 - ExtractGrok - Add support returning only named captures
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2233.
2017-12-19 15:49:14 +01:00
Matthew Burgess 612675e428 NIFI-4684 - Added SQL Parameter Attribute Prefix property to ConvertJSONToSQL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2333.
2017-12-19 10:32:52 +01:00
trixpan 44a3ac9eff NIFI-4706 - This closes #2348. Bump ParCEFone version
Signed-off-by: joewitt <joewitt@apache.org>
2017-12-18 10:13:34 -05:00
Josh Anderton f50a07ec88 NIFI-4655: This closes #2347. Fixed NPE in the InvokeHTTP processor caused when an SSLContextService was assigned without keystore properties
- Added check for keystore properties and only initialized keystore when necessary.
- Added TestInvokeHttpTwoWaySSL test class to test with two-way SSL
- Modified TestInvokeHttpSSL to test with one-way SSL

Signed-off-by: joewitt <joewitt@apache.org>
2017-12-18 09:11:53 -05:00
Marco Gaido 463dcd8812 NIFI-4629: This closes #2345. Put flowfiles without the grouping attribute in the default group
Signed-off-by: joewitt <joewitt@apache.org>
2017-12-15 10:35:13 -05:00
Koji Kawamura 57947d64cd NIFI-4694: Make PutSFTP Disable Directory Listing a normal property. This closes #2338. 2017-12-14 15:05:40 -05:00
Mike Thomsen d7347a2dc3 NIFI-4644 Fixed LookupService API to allow for more than String/String lookup pairs. This closes #2304. 2017-12-13 16:04:23 -05:00
Mark Payne 1fc1d38fd8 NIFI-4562: This closes #2331. If an Exception is thrown when merging FlowFiles, ensure that we remove the 'bundle' flowfile before exiting the method
Signed-off-by: joewitt <joewitt@apache.org>
2017-12-08 15:35:36 -05:00
joewitt a774f1df69 NIFI-4673 changed offending tests to be integration tests and fixed travis config to run the build only once and during appropriate script phase instead of install. Reviewed by Bende. 2017-12-06 16:13:42 -05:00
joewitt 7e61c6333c
NIFI-2375 updated to latest Tika across codebase. Fixed
tests and updated L&N for Tika parsers.  Turned unstable grpc test to IT
 test. Updated deps due to Tika testing findings. This closes #2320
2017-12-06 11:53:15 -05:00
joewitt cdc1facf39
NIFI-4664, NIFI-4662, NIFI-4660, NIFI-4659 moved tests which are timing/threading/network dependent and brittle to integration tests and un-ignored tests that are IT. Updated travis to reduce impact on infra and appveyor now skips test runs so is just to prove build works on windows. This closes #2319
squash
2017-12-06 10:53:09 -05:00
Mark Payne 45df23b1e0 NIFI-4607, NIFI-3975, NIFI-4602, NIFI-4606: This closes #2272. Fixed bug in TailFile that caused new Primary Node to not pull current Clustered State when File Location was set to Remote. Fixed bug that caused TailFile to occasionally become 'stuck' when the file it is tailing is renamed and a new file is created with the same name. Removed the 'Rolling Strategy' property from TailFile because it is not actually used in the processor anymore. Deleted MonitorMemoryTest because the unit test was testing the behavior of FlowController more than the behavior of the reporting task itself and in order to do so had a dependency in the pom.xml on nifi-framework-core, which means that it no longer compiles when FlowController is modified.
Signed-off-by: joewitt <joewitt@apache.org>
2017-11-30 11:10:46 -05:00
Matthew Burgess 3b15ed855c NIFI-4559: Add non-zero status relationship to ExecuteStreamCommand
NIFI-4559: Removed Penalize Non-zero Status property and updated doc per review comments
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2246
2017-11-30 10:12:00 -05:00
sbouchex 33281300cd NIFI-4597 This closes #2283. correcting imports, checkstyle issues, and adding validator
Add a property to override HTTP return code in ListenHTTP processor

Signed-off-by: joewitt <joewitt@apache.org>
2017-11-30 10:02:32 -05:00
Matt Gilman 3df3ff6c61 NIFI-4646: This closes #2306.
- Ensuring the reason behind the SchemaNotFoundException is reported when interacting with the configured SchemaRegistry.

Signed-off-by: joewitt <joewitt@apache.org>
2017-11-30 09:02:21 -05:00
Matthew Burgess 16a23f5a0f NIFI-4589: Allow multiple keys in FetchDistributedMapCache, add subMap operation to API
Signed-off-by: Mike Moser <mosermw@apache.org>

This closes #2260.
2017-11-21 20:13:35 +00:00
m-hogue 73702004b9 NIFI-3515: This closes #2257. Added EventDriven annotation to PutFile
Signed-off-by: joewitt <joewitt@apache.org>
2017-11-21 12:40:13 -05:00
Andy LoPresto 5d643edfab
NIFI-4501
- Changed request header handling logic.
- Removed unnecessary Maven dependency.
- This closes #2279
2017-11-17 20:36:55 -05:00
Pierre Villard 9e9c129c21 NIFI-4577 - ValidateCsv - add attributes to indicate number of valid/invalid lines
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2268
2017-11-13 21:48:30 -05:00
Julian Gimbel d0502285f2 NIFI-4568 Added existing but missing property WORKING_DIR
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2250.
2017-11-08 09:50:14 +01:00
Andrew Lim b5d7aecfac NIFI-4560 Added mime.type and record.count Write Attributes to ValidateRecord doc
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2249.
2017-11-03 11:10:00 +01:00
Andrew Lim 9a850c7ed2 NIFI-4552 Add mime.type and record.count Write Attributes to QueryRecord doc
NIFI-4552 minor checkstyle violation

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2242
2017-11-01 14:49:02 -04:00
Andrew Lim fb94e983b6 NIFI-2938 Added Write Attributes to RouteText doc and edited Grouping Regular Expression property description
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2232.
2017-10-30 09:29:52 +01:00
Pierre Villard 68715a0dd4 NIFI-4513 - add EL support for proxy settings in InvokeHttp
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2223
2017-10-27 18:15:27 -04:00
Matthew Burgess 856dedab12 NIFI-4522: Add SQL Statement property to PutSQL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2225.
2017-10-27 09:17:51 +02:00
Matt Gilman 6baea8ccff
NIFI-4444:
- Upgrading to Jersey 2.x.
- Updating NOTICE files where necessary.
- Fixing checkstyle issues.

This closes #2206.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-10-12 10:27:02 -07:00
Bryan Bende 9324a2a742 NIFI-4476 Improving logic for determining when to yield in PutTCP/UDP/Syslog/Splunk
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2204.
2017-10-10 09:02:02 +02:00
Bryan Bende 883c223ced NIFI-4386 Fixing connection handling in FetchFileTransfer
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2203.
2017-10-10 08:50:57 +02:00
patricker 0876cf12b1 NIFI-3432: Handle Multiple Result Sets in ExecuteSQL
NIFI-3432

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #1471
2017-10-05 16:05:22 -04:00
Andy LoPresto d4168f5ff1
NIFI-4297
- Upgraded immediately actionable dependency versions from Meterian report.
- Upgraded jackson-core test dependencies for HBase and Elasticsearch modules.
- Only 3 instances of jackson-core < 2.8.6 (Google Cloud Platform and Spark Receiver modules).
- Upgraded version of poi dependency in nifi-email-processors to 3.16.
- Resolving dependency issues after rebasing against 1.5.0-SNAPSHOT.
- Removed jackson-databind from <dependencyManagement> block in nifi/pom.xml and added explicit reference to ${jackson.version} in all referenced artifacts.
- Removed jackson-mapper-asl from <dependencyManagement> block in nifi/pom.xml and added explicit reference to ${jackson.old.version} in all referenced artifacts.
- Removed Jasypt from <dependencyManagement> and added explicit version in test dependency for legacy compatibility.
- This closes #2084
2017-10-05 15:23:52 -04:00
Pierre Villard ea2c91f9d0 NIFI-4372 Wait processor - recommend prioritizer in documentation. This closes #2139. 2017-10-04 16:02:52 -04:00
Jeff Storck 2694adcca9 Merge branch 'NIFI-4412-RC2' 2017-10-02 13:58:54 -04:00
Jeff Storck a57911d3db NIFI-4412-RC2 prepare for next development iteration 2017-09-28 13:45:36 -04:00
Jeff Storck e6508ba7d3 NIFI-4412-RC2 prepare release nifi-1.4.0-RC2 2017-09-28 13:45:21 -04:00
Pierre Villard bdab3cda0a NIFI-4340 - fix record path evaluation when array is [ null ]
This closes #2122.
2017-09-28 13:39:55 -04:00
m-hogue b7b6d9082c
NIFI-2529: Added support for SSLContextService protocols in HandleHttpRequest
Forced HandleHTTPRequest to use RestrictedSSLContextService and removed extraneous SSL algorithm checks
Throw RuntimeException if the chosen SSL protocol isn't supported by HandleHttpRequest

This closes #1985.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-09-26 18:48:19 -07:00
Andy LoPresto 2c1f5b49e4 NIFI-3116 This closes #2108. Added initial regression test for StringEncryptor to ensure continued functionality during removal of Jasypt.
Added external compatibility regression test for StringEncryptor to ensure continued functionality during removal of Jasypt.
Documents custom salt lengths and iteration counts for each encryption method.
Added (ignored) failing tests for keyed encryption (Jasypt does not support keyed encryption).
Changed StringEncryptor to non-final class and added protected default constructor.
Added failing test for initialization status.
Added utility methods in CipherUtility.
Moved PBE cipher providers (and tests) from nifi-standard-processors to nifi-security-utils module.
Implemented PBE and keyed encryption/decryption logic.
Moved Scrypt unit test back into scrypt package.
Resolved test failures in limited strength cryptographic environment.
Implemented keyed encryption/decryption and enabled unit tests.
Removed Jasypt dependency from production scope (kept in test scope for backward compatibility tests).

Signed-off-by: joewitt <joewitt@apache.org>
2017-09-26 10:55:29 -04:00
Bryan Bende 6eab91923e NIFI-4418 Adding ListenUDPRecord processor. This closes #2173. 2017-09-25 13:19:23 -04:00
Koji Kawamura c6f4421c8e NIFI-2829: Fixed PutSQL time unit test.
The unit test for DATE type used GMT timezone, that causes an assertion error in timezones such as EST (-5).
We need to use local timezone instead of GMT, as Derby and PutSQL uses local timezone.

The unit test failed before as follows:
- Unit test code, passed: '2002-02-02 GMT'
- PutSQL code convertedi it to local: '2002-02-01 EST', and stored as '2002-02-01' in Derby database without timezone info
- Unit test code SELECT the inserted value, passed a GMT calender, then got epoch timestamp, which was '2002-01-31'

Support negative long value for timezones ahead of UTC.

- For timezones such as '+0800', it's possible that a local time e.g. '02:03:04' can be a negative epoch value. This commit changes LONG_PATTERN so that it can accept nevative values.
- Changed time values in unit tests to verify negative epoch values, and avoid using the same digits among different time unit for better readability.

This closes #2082
2017-09-23 17:41:42 -04:00
Deon Huang a29348f2a4 NIFI-4395 - GenerateTableFetch can't fetch column type by state after instance reboot
NIFI-4395: Updated unit test for GenerateTableFetch
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2166
2017-09-22 14:34:00 -04:00
Mark Payne 05b5dd1488 NIFI-4409: Ensure that when record schema is inherited, the schema from teh ResultSet is used instead of the schema from the RecordReader because the schema from the RecordReader mmay not match the actual data
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2169
2017-09-22 12:41:17 -04:00
Bryan Bende 329dbe3a64 NIFI-4405 Adding charset property to GenerateFlowFile
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2168.
2017-09-21 17:10:47 +02:00
Koji Kawamura 7b07eb0577 NIFI-4228: Fix PutDatabaseRecord to ignore unmatched fields
- Unmatched fields were ignored, but the number of prepared statement
  place holders were not correct.
- Added unit test code for generateUpdate.
- Added unit test code with "Ignore Unmatched Columns".

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2165.
2017-09-21 16:11:07 +02:00
Koji Kawamura 1e70e24267 NIFI-4352: Add CLOB and NCLOB support to PutSQL and ExecuteSQL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2145.
2017-09-14 11:02:48 +02:00
Gino Lisignoli 897b8ab601 NIFI-4374: Added buffer size to FTP connections
This closes #2141.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-09-11 23:43:56 +09:00
Joe Percivall 9367c28064 NIFI-2162 Updating OkHttp to 3.8.1 and OkHttp-Digest to 1.13 and refactoring InvokeHttp to adjust for changes
This closes #2004

Signed-off-by: Tony Kurc <tkurc@apache.org>
2017-09-10 22:50:37 -04:00
Mark Payne 9ebf2cfaf1 NIFI-4306: Allow root field to be updated to a child field by adding property with name / and value /child, for instance
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2103.
2017-09-08 18:50:49 +02:00
Koji Kawamura 1f67cbf628 NIFI-4004: Use RecordReaderFactory without FlowFile.
- Removed FlowFile from RecordReaderFactory, RecordSetWriterFactory and SchemaAccessStrategy.
- Renamed variable 'allowableValue' to 'strategy' to represent its meaning better.
- Removed creation of temporal FlowFile to resolve Record Schema from ConsumerLease.

- Removed unnecessary 'InputStream content' argument from
  RecordSetWriterFactory.getSchema method.

This closes #1877.
2017-09-08 12:37:40 -04:00
m-hogue 03e51ee8ac
NIFI-4335: Changed SSLContextService implementations to RestrictedSSLContextService for all Listen* processors
This closes #2131.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-09-07 17:34:00 -07:00
Pierre Villard c10ff574c4 NIFI-4257 - add custom WHERE clause in database fetch processors
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2050
2017-09-07 14:41:26 -04:00
patricker ae30c7f350 NIFI-3484: GenerateTableFetch Should Allow for Right Boundary
fix checkstyle issue, and added unit test showing data duplication issue, removed property

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2091
2017-09-07 11:05:38 -04:00
Pierre Villard 458c987fe3 NIFI-4355 - query execution time as attribute of ExecuteSQL
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2129
2017-09-06 13:29:19 -04:00
Mark Payne bfd6c0aef7 NIFI-4116: Allow fields of Record returned from Lookup Service to be placed into record in the input, instead of requiring that the 'wrapper record' returned from Lookup be included
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2110.
2017-09-05 21:29:16 -07:00
m-hogue d28e61c5dd
NIFI-2528 Added RestrictedSSLContextService interface with implementation. Changed ListenHTTP to allow only new StandardRestrictedSSLContextService.
This closes #1986.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-30 20:17:01 -07:00
Pierre Villard e62417ea6b NIFI-1923 - AttributesToJson regex property
added EL support

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2099
2017-08-30 14:01:27 -04:00
Scott Aslan e2b8be53cb
[NIFI-4327] Parameterize node and npm in poms. This closes #2115 2017-08-28 15:10:24 -04:00
Koji Kawamura e68ff153e8
NIFI-3332: ListXXX to not miss files with the latest processed timestamp
Before this fix, it's possible that ListXXX processors can miss files those have the same timestamp as the one which was the latest processed timestamp at the previous cycle. Since it only used timestamps, it was not possible to determine whether a file is already processed or not.

However, storing every single processed identifier as we used to will not perform well.
Instead, this commit makes ListXXX to store only identifiers those have the latest timestamp at a cycle to minimize the amount of state data to store.

NIFI-3332: ListXXX to not miss files with the latest processed timestamp

- Fixed TestAbstractListProcessor to use appropriate time precision.
  Without this fix, arbitrary test can fail if generated timestamp does
  not have the desired time unit value, e.g. generated '10:51:00' where
  second precision is tested.
- Fixed TestFTP.basicFileList to use millisecond time precision explicitly
  because FakeFtpServer's time precision is in minutes.
- Changed junit dependency scope to 'provided' as it is needed by
  ListProcessorTestWatcher which is shared among different modules.

This closes #1975.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-08-28 11:31:04 -04:00
Koji Kawamura 28ee70222b
NIFI-4069: Make ListXXX work with timestamp precision in seconds or minutes
- Refactored variable names to better represents what those are meant for.
- Added deterministic logic which detects target filesystem timestamp precision and adjust lag time based on it.
- Changed from using System.nanoTime() to System.currentTimeMillis in test because Java File API reports timestamp in milliseconds at the best granularity. Also, System.nanoTime should not be used in mix with epoch milliseconds because it uses arbitrary origin and measured differently.
- Changed TestListFile to use more longer interval between file timestamps those are used by testFilterAge to provide more consistent test result because sleep time can be longer with filesystems whose timestamp in seconds precision.
- Added logging at TestListFile.
- Added TestWatcher to dump state in case assertion fails for further investigation.
- Added Timestamp Precision property so that user can set if auto-detect is not enough
- Adjust timestamps for ages test

This closes #1915.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-08-28 11:31:03 -04:00
Pierre Villard 41984bed13 NIFI-1835 - added text/xml to content viewer
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2098
2017-08-18 12:48:49 -04:00
Pierre Villard c1b99d584d NIFI-4028 - fix cache update when Wait releases flow files
NIFI-4028: Refactored Wait processor.

- Consolidated implementation for the cases of releasableFlowCount is 1 or more, in order to reduce complexity and behavior differences
- Added 'consumed' counter when total counter is used to release incoming FlowFiles
- Fixed method name typo, releaseCandidates

This closes #2055.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-08-17 08:30:07 +09:00
Pierre Villard 9f23ec360a NIFI-3931 - Added EL to properties in SFTP transfer
The properties are already evaluated against EL. Also added an
integration test that is failing without the change.

This closes #1968.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-08-17 08:20:30 +09:00
Arun Manivannan 6df112e4b1 NIFI-4062 Provide an option to disable DTD validation for EvaluateXPath and EvaluateXQuery
This closes #2093.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-08-17 08:07:37 +09:00
Pierre Villard a1706d12f5 NIFI-3281 - fix for (S)FTP processors when using EL against FFs
NIFI-3281 - Review - handle completePendingCommand return and added a unit test for ListFTP

NIFI-3281 - Review - Added flow file for EL evaluation in other methods and added unit test for NIFI-3590

This closes #1974.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-08-15 11:24:48 +09:00
Matt Burgess 50f22162b0
NIFI-4286: Fix NPE in GenerateTableFetch when Partition Size = 0
This closes #2078.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-08-14 15:28:07 -04:00
Mark Payne 451f9cf124 NIFI-4142: This closes #2015. Refactored Record Reader/Writer to allow for reading/writing "raw records". Implemented ValidateRecord. Updated Record Reader to take two parameters for nextRecord: (boolean coerceTypes) and (boolean dropUnknownFields)
Signed-off-by: joewitt <joewitt@apache.org>
2017-08-11 22:01:46 -07:00
Bryan Bende 28d5a70ec0
NIFI-4288 Ignoring tests that intermittently fail in TestPutUDP and TestPutTCP
This closes #2076.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-11 17:27:55 -07:00
Wayne Steel a6e8f0afe3 NIFI-4177 This closes #2002. MergeContent - Tar - Save modification timestamp like Tar does
Signed-off-by: joewitt <joewitt@apache.org>
2017-08-10 19:35:46 -07:00
Pierre Villard 6ff8321cf7
NIFI-4278 - add error message to invalid FFs in ValidateXml
This closes #2069.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-09 21:20:08 -07:00
Matt Burgess 9c4fdd4ef3 NIFI-4192: Add more Avro/metadata merge options to MergeContent
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2067.
2017-08-09 12:55:12 +02:00
Bryan Bende 0029f025f8 NIFI-4152 Initial commit of ListenTCPRecord 2017-08-07 22:44:11 +02:00
Wil Selwood f7f64a572c NIFI-4268: Add fetch tag to FetchFile processor.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2062.
2017-08-07 21:38:55 +02:00
Andy LoPresto 675d989003
NIFI-4139
- Moved key provider interface and implementations from nifi-data-provenance-utils module to nifi-security-utils module.
- Refactored duplicate byte[] concatenation methods from utility classes and removed deprecation warnings from CipherUtility.
- Created KeyProviderFactory to encapsulate key provider instantiation logic.
- Added logic to handle legacy package configuration values for key providers.
- Added unit tests.
- Added resource files for un/limited strength cryptography scenarios.
- Added ASL to test resources.
- Moved legacy FQCN handling logic to CryptUtils.
- Added unit tests to ensure application startup logic handles legacy FQCNs.
- Moved master key extraction/provision out of FBKP.
- Removed nifi-security-utils dependency on nifi-properties-loader module.
- Added unit tests.
2017-08-07 15:21:58 -04:00
Mark Payne 969bbe654c NIFI-4051: Ensure that we use the appropriate Lexical syntax for SQL query validation as we do when running
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2054.
2017-08-04 16:31:56 +02:00
Matt Burgess 2d33582147 NIFI-4253: Add Oracle 12+ database adapter
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2048.
2017-08-03 09:59:29 +02:00
Leah Anderson 70878fe6d6 NIFI-4052 Add DynamicProperty annotation for LookupRecord
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2032.
2017-07-28 17:12:06 +02:00
Matt Burgess dc4006f423 NIFI-3335: Add initial.maxvalue support to GenerateTableFetch
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2039.
2017-07-27 08:43:00 +02:00
Leah Anderson 3371e915cc NIFI-4087 This closes #2026. Fix to allow exclusion of filename from tika criteria. 2017-07-21 16:56:56 -04:00
Andre F de Miranda b0be99036d NIFI-55 - Ensures ListenHTTP logs the source of an expired hold
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1623.
2017-07-20 23:14:32 +02:00
Koji Kawamura b4e0a6e206 NIFI-4205: Avoid duplicated data from TailFile
Before this fix, it is possible that TailFile to produce duplicated data
if an already tailed file has newer timestamp and fewer or the same
amout of data.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2021.
2017-07-20 12:14:05 +02:00
Bryan Bende f87d2a2f57 NIFI-4157 Improvements to PutTCP
Signed-off-by: Bryan Bende <bbende@apache.org>
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1989.
2017-07-19 12:25:42 +02:00
joewitt 59754500d5 NIFI-2829 ignoring unit test that appears broken. Commented on JIRA about the problem observed 2017-07-18 22:23:10 -04:00
Andre F de Miranda a307ea3a00 NIFI-4098 This closes #2013. Adjust ParseCEF to re-use bean validator in order to improve performance 2017-07-18 22:02:39 -04:00
Matt Burgess 844dbe4edb NIFI-4156: Fixed fragment.count in SplitText to equal emitted flow files
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2014.
2017-07-18 11:31:16 +02:00
deonhuang a6e94de0bb NIFI-2829 - Add Date and Time Format Support for PutSQL
Fix unit test for Date and Time type time zone problem
Enhance Time type to record milliseconds

This closes #1983.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-07-18 14:16:10 +09:00
patricker 03bff7c9fc NIFI-2829 Date and Time Format Support for PutSQL
This closes #1524.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-07-18 14:11:37 +09:00
Koji Kawamura 8acee02393 NIFI-1613
- Truncate text data types only.
- Added conversion from a boolean to number.
2017-07-12 17:02:55 -04:00
Koji Kawamura 3844a821f1 NIFI-1613: ConvertJSONToSQL truncates numeric value wrongly.
- Changed boolean value conversion to use Boolean.valueOf.
- Updated comments in source code to reflect current situation more clearly.
- Updated tests those have been added since the original commits were made.
2017-07-12 17:02:55 -04:00
Toivo Adams 3b2e43b75c NIFI-1613 Initial version, try to improve conversion for different SQL types. New test and refactored existing test to reuse DBCP service.
nifi-1613 Adding numeric and Date/time types conversion and test.
2017-07-12 17:02:55 -04:00
Mark Payne b603cb955d NIFI-4060: Initial implementation of MergeRecord
NIFI-4060: Addressed threading issue with RecordBin being updated after it is completed; fixed issue that caused mime.type attribute not to be written properly if all incoming flowfiles already have a different value for that attribute

NIFI-4060: Bug fixes; improved documentation; added a lot of debug information; updated StandardProcessSession to produce more accurate logs in case of a session being committed/rolled back with open input/output streams
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1958
2017-07-12 16:36:48 -04:00
mans2singh 0fd51b4d12 NIFI-4145 Added writes attributes annotation
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1998.
2017-07-10 21:25:26 -07:00
Tony Kurc 2dc45a4dd7 NIFI-4154 Fixing line endings in .java,.html files
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1982.
2017-07-09 12:13:15 -07:00
Koji Kawamura e84f9a2416 NIFI-4160: SFTPTransfer connection timeout for opening channel.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1991.
2017-07-07 12:31:50 +02:00
Koji Kawamura 50c364a793 NIFI-4162: PutSQL batch update error message should include the cause
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1992.
2017-07-07 12:25:10 +02:00
m-hogue e6b166a3a2 NIFI-552: added regex properties for include and ignore filters in LogAttribute
This closes #1981

Signed-off-by: Tony Kurc <tkurc@apache.org>
2017-07-05 21:21:50 -04:00
Pierre Villard 8b5342dec0 NIFI-3974 - Added charset in LogAttribute
This closes #1967

Signed-off-by: Tony Kurc <tkurc@apache.org>
2017-07-02 12:03:07 -04:00
Pierre Villard 902b6e205d NIFI-4131 - GFT warning message when badly setting relationships
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1954
2017-06-30 15:12:10 -04:00
yuri1969 3bf1d12706
NIFI-4125 Added secure transform feature and configuration to TransformXML processor to mitigate XXE file system leaks.
This closes #1946.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-29 10:40:33 -07:00
Matt Burgess 9bfa7469cb NIFI-4137: Add support for UTF-8 in GetFTP and PutFTP
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1957
2017-06-28 16:41:14 -04:00
Andre F de Miranda 1e7eceee84 NIFI-4126 - Add SupportBatching, SideEffectFree and EventDriven annotations to ExtractGrok
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1950.
2017-06-27 20:56:56 +02:00
Koji Kawamura 91ed96f8c4 NIFI-4040: Continue if SFTP failed getting user home directory
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1898.
2017-06-26 19:09:22 +02:00
Andy LoPresto 253ea2e73b NIFI-4095 Changed minimum capture group count in ExtractText from 1 to 0.
Added unit test and removed obsolete test.
Added custom validation to enforce capture group if "include capture group 0" is false.
2017-06-22 23:31:35 +02:00
Koji Kawamura 5c755c006b
NIFI-4017: Emit provenance event from Notify.
This closes #1890.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-22 16:49:31 -04:00
Pierre Villard f54e146561
NIFI-4085 Add exception in log message when (S)FTP listing fails
This closes #1929.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-22 16:06:18 -04:00
Jeff Storck c99100c934
NIFI-4010 Enables EL on Fetch/List/PutSFTP and List/Fetch/Put/DeleteHDFS processor properties
FetchSFTP/ListSFTP/PutSFTP: Private Key Path
ListHDFS/FetchHDFS/PutHDFS/DeleteHDFS: Hadoop Configuration Resources, Kerberos Principal, Kerberos Keytab, Kerberos Relogin Period

This closes #1148
This closes #1930.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-21 17:14:49 -04:00
Bryan Bende aabd4a25d2 NIFI-4043 Initial commit of nifi-redis-bundle
NIFI-4061 Initial version of RedisStateProvider
- Adding PropertyContext and updating existing contexts to extend it
- Added embedded Redis for unit testing
- Added wrapped StateProvider with NAR ClassLoader in StandardStateManagerProvider
- Updating state-management.xml with config for RedisStateProvider
- Renaming tests that use RedisServer to be IT tests so they don't run all the time

This closes #1918.
2017-06-21 12:47:19 -04:00
Matt Gilman 490e1da5db NIFI-4019:
- Adding support for X-Forwarded-* headers.
- Unrelated code clean up.
- Addressing additional proxying issues.
- Addressing mistyped landing page.
- Handled trailing slashes in proxy headers for context path
2017-06-15 14:17:35 -04:00
Koji Kawamura de1b84e2aa
NIFI-4049: Refactor AtomicDistributedMapCacheClient
To be used with cache engines that does not have revision number.

NIFI-4049: Refactor AtomicDistributedMapCacheClient

Removed old methods completely.

This closes #1904.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-12 10:34:48 -04:00
Koji Kawamura 7761903352 NIFI-4041: Add file.size attribute to ListFTP and ListSFTP
This closes #1899.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-06-12 10:53:09 +09:00
Mark Payne e7dcb6f6c5 NIFI-3921: Allow Record Writers to inherit schema from Record
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1902
2017-06-09 16:13:25 -04:00
Mark Payne d8dc2bb00b NIFI-4047: Addressed issue with inconsistent Lexical policy between used cached schemas and non-cached schemas
This closes #1907

Signed-off-by: Joe Percivall <JPercivall@apache.org>
2017-06-09 12:05:11 -04:00
Matt Gilman cc741d2be6
NIFI-3997:
- Bumping to next minor version.
2017-06-08 15:22:51 -04:00
Matt Gilman 6ee12e9b47
NIFI-3997-RC1prepare for next development iteration 2017-06-05 11:07:43 -04:00
Matt Gilman ddb73612bd
NIFI-3997-RC1prepare release nifi-1.3.0-RC1 2017-06-05 11:07:28 -04:00
Andre F de Miranda 082b99640e NIFI-4001 - Bump ParCEFone version
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1875.
2017-06-03 12:11:38 +02:00
Mark Payne 32314d70fd NIFI-4009: Added support for several key functions in RecordPath
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1881
2017-06-02 14:06:05 -04:00
Pierre Villard 80dfe0257f ExtractGrok - Include exception message when compile fails
This closes: #1845

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-05-27 00:56:25 +10:00
Koji Kawamura 13b59b5621 NIFI-3958: Decimal logical type with undefined precision and scale.
- Oracle NUMBER can return 0 precision and -127 or 0 scale with variable scale NUMBER such as ROWNUM or function result
- Added 'Default Decimal Precision' and 'Default Decimal Scale' property to ExecuteSQL and QueryDatabaseTable to apply default precision and scale if those are unknown
- Coerce BigDecimal scale to field schema logical type, so that BigDecimals having different scale can be written

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1851
2017-05-25 14:37:17 -04:00
Koji Kawamura 4d0667380a NIFI-3404: Improved UX of LookupAttributes.
- Added dependency notice.
- Added EL evaluation at SimpleKeyValueLookupService.
- Updated documentation.
- Updated CommonsConfigurationLookupService to throw LookupFailureException if it fails to get configuration so that error messages can be displayed at each processor bulletin.
- Added calling getConfiguration at OnEnabled of CommonsConfigurationLookupService, so that the service will stay in Enabling state if there is any issue.

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2017-05-25 10:05:32 -05:00
Joey Frazee 46e2420d74 NIFI-3404 Added LookupAttribute processor and lookup controller services
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2017-05-25 10:05:32 -05:00
Andre F de Miranda c07850aec3 NIFI-2702 - Deprecates nifi-standard-bundle/*JMS and document the recommended alternative
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1643.
2017-05-25 08:44:39 +02:00
Mark Payne 08b66b5b6a NIFI-3969: Prevent merging flowfiles prematurely when all bins fill but some are already full and can be processed
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1850.
2017-05-24 19:36:18 +02:00
Andy LoPresto 6d07ebf50c NIFI-3954 This closes #1841. Upgraded version of httpclient library to handle TLS issues.
Added explicit dependency on httpcore for @ThreadSafe annotation.

Signed-off-by: joewitt <joewitt@apache.org>
2017-05-23 23:42:28 -04:00
Mark Payne 6d16fdf170
NIFI-3952: Updated UpdateRecord to pass field-related variables to the Expression Language
This closes #1836.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-22 14:32:45 -04:00
Mark Payne 7f8987471d
NIFI-3946: Update LookupService to take a Map instead of a String for the input
NIFI-3946: Fixed issues where null values were returned instead of empty optionals

This closes #1833.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-22 11:52:41 -04:00
Mark Payne c49933f03d NIFI-3948: This closes #1834. Added flush() method to RecordWriter and call it when writing a single record to OutputStream for PublishKafkaRecord. Also removed no-longer-used class WriteAvroResult
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-19 23:05:04 -04:00
Mark Payne ae9953db64 NIFI-3857: This closes #1825. Added PartitionRecord processor
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-19 02:08:52 -04:00
Mark Payne 9bd0246a96 NIFI-3863: Initial implementation of Lookup Services. Implemented LookupRecord processors. This required some refactoring of RecordSetWriter interface, so refactored that interface and all implementations and references of it 2017-05-19 01:02:41 -04:00
Koji Kawamura 3fd7d3b272 NIFI-3873: Improve Avro logical types readability
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1788
2017-05-17 13:08:00 -04:00
Peter G. Horvath bf15502e19 NIFI-3763 Add new processor to log user defined messages built with NiFi Expression Language
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1737
2017-05-15 14:18:57 -04:00
Koji Kawamura 1811ba5681 NIFI-2624: Avro logical types for ExecuteSQL and QueryDatabaseTable
- Added Logical type support for DECIMAL/NUMBER, DATE, TIME and TIMESTAMP column types.
- Added Logical type 'decimal' to AvroReader so that Avro records with logical types written by ExecuteSQL and QueryDatabaseTable can be consumed by AvroReader.
- Added JdbcCommon.AvroConversionOptions to consolidate conversion options.
- Added 'Use Avro Logical Types' property to ExecuteSQL and QueryDatabaseTable to toggle whether to use Logical types.
- Added 'mime.type' FlowFile attribute as 'application/avro-binary' so that output FlowFiles can be displayed by content viewer.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1798
2017-05-15 14:15:23 -04:00
Mark Payne b1901d5fe0 NIFI-3838: Initial implementation of RecordPath and UpdateRecord processor
NIFI-3838: Updated version from 1.2.0-SNAPSHOT to 1.3.0-SNAPSHOT; removed unneeded value from AttributeExpression.ResultType enum

NIFI-3838: Addressed PR Review feedback

NIFI-3838: Allow for schemas to be merged together for a record; refactored RecordSetWriterFactory so that there is a method to obtain the schema and then the writer is created with that schema. Added additional unit tests

NIFI-3838: Addressed problems with documentation based on PR Review

NIFI-3838: Fixed checkstyle violation

NIFI-3838: Addressed issue of comparing different types of Number objects

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1772
2017-05-12 12:36:52 -04:00
Matt Gilman dd0306cce7 NIFI-3837: - Being more specific regarding the escaping of back references in evaluated expressions.
Signed-off-by: Matt Burgess <mattyb149@apache.org>

fixed typo in comment

This closes #1781
2017-05-11 22:16:46 -04:00
Bryan Bende 3af53419af
NIFI-3770-RC2 prepare for next development iteration 2017-05-05 20:50:28 -04:00
Bryan Bende 3a605af8e0
NIFI-3770-RC2 prepare release nifi-1.2.0-RC2 2017-05-05 20:50:14 -04:00
joewitt f4f174b899
NIFI-3826 added proper NOTICE entries for apache calcite and maxmind source
This closes #1766.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 20:00:22 -04:00
joewitt 829c02b058 NIFI-3825 fix L&N for jolt, json, and source deps
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1765
2017-05-05 18:43:04 -04:00
joewitt ee8dc7640f
NIFI-3820 added calcite to assembly notice and updated all copyright years for compress, avro, and jetty
This closes #1764.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 14:45:40 -04:00
Mark Payne 12249e5a08
NIFI-2268: Instead of allowing HandleHttpRequest to run constantly calling BlockingQueue.poll() as fast as possible, call BlockingQueue.poll(2, TimeUnit.MILLISECONDS) to avoid overutilization of CPU when there is no work to be done
This closes #1762.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 13:58:10 -04:00
Andre F de Miranda 970ddf8f7e
NIFI-3794 - Expose the control of ListenRELP's CLIENT_AUTH property to DFM
This closes #1746.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-04 10:50:22 -04:00
Jeff Storck 26d90fbccf
NIFI-1833 - Addressed issues from PR review.
NIFI-1833 Moved AbstractListProcessor.java, EntityListing.java, and ListableEntity.java from nifi-standard-processors into nifi-processor-utils
Moved TestAbstractListProcessor.java into nifi-processor-utils
Set nifi-azure-nar's nar dependency back to nifi-standard-services-api-nar
Fixed failing integration tests (ITFetchAzureBlobStorage.java, ITListAzureBlobStorage.java, and ITPutAzureStorageBlob.java) and refactored them to be able to run in parallel

NIFI-1833 Moved security notice info in the additional details documentation into the descriptions of the specific attributes for which those notices are intended
Added displayName usage to properties
Updated exception handling in FetchAzureBlobStorage.java and PutAzureBlobStorage.java to cause flowfiles with Output/InputStreamCallback failures to be routed to the processor's failure relationship
Cleaned up dependencies in pom

NIFI-1833 Removed unnecessary calls to map on Optional in the onTrigger exception handling of FetchAzureBlobStorage.java and PutAzureBlobStorage.java

NIFI-1833 Updates due to nifi-processor-utils being moved under nifi-nar-bundles

This closes #1719.

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-05-02 14:39:46 -04:00
Andy LoPresto 7d242076ce
NIFI-3594 Implemented encrypted provenance repository.
Added src/test/resources/logback-test.xml files resetting log level from DEBUG (in nifi-data-provenance-utils) to WARN because later tests depend on MockComponentLog recording a certain number of messages and this number is different than expected if the log level is DEBUG.

This closes #1686.

Signed-off-by: Bryan Bende, Yolanda M. Davis, and Mark Payne
2017-05-02 13:24:07 -04:00
Bryan Bende 60d88b5a64
NIFI-3724 - Initial commit of Parquet bundle with PutParquet and FetchParquet
- Creating nifi-records-utils to share utility code from record services
- Refactoring Parquet tests to use MockRecorderParser and MockRecordWriter
- Refactoring AbstractPutHDFSRecord to use schema access strategy
- Adding custom validate to AbstractPutHDFSRecord and adding handling of UNION types when writing Records as Avro
- Refactoring project structure to get CS API references out of nifi-commons, introducing nifi-extension-utils under nifi-nar-bundles
- Updating abstract put/fetch processors to obtain the WriteResult and update flow file attributes

This closes #1712.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-01 16:10:35 -04:00
Andre F de Miranda da6ad4f3bc
NIFI-3670 - Expose the control of ListenSyslog's CLIENT_AUTH property to DFM
This closes #1720.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-01 10:41:12 -04:00
Andre F de Miranda a9a9b67430 NIFI-2616 - Remove 2 unused classes in IdentifyMimeType identified during PR 2017-05-02 00:25:42 +10:00
Jon Anderson III 25295cdb4d NIFI-2616 Remove reference to non-existent setting in IdentifyMimeType Additional Details
Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-05-02 00:20:04 +10:00
Andre F de Miranda 9ad7802284
NIFI-1939 - Correct issue where ParseSyslog was unable to parse RFC3164 messages containg an IPv6 address as source
NIFI-1939 - Fix typo and adjust ListenSyslog as per PR feedback

This closes #1639.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-01 10:00:33 -04:00
Pierre Villard 7df5c2dc89 NIFI-3607 Allow multi files mode with fixed names
This closes: #1608

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-05-01 09:31:24 +10:00
Matt Burgess d61f519326 NIFI-3742: Fixed handling of nulls in DELETE for PutDatabaseRecord
This closes #1709.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-28 11:34:43 +09:00
Koji Kawamura d9acdb54be NIFI-3415: Add Rollback on Failure.
- Added org.apache.nifi.processor.util.pattern package in nifi-processor-utils containing reusable functions to mix-in 'Rollback on Failure' capability.
- Created a process pattern classes, Put and PutGroup. It will be helpful to standardize Processor implementations.
- Applied Rollback on Failure to PutSQL, PutHiveQL, PutHiveStreaming and PutDatabaseRecord.
- Stop using AbstractProcessor for these processors, as it penalizes FlowFiles being processed when it rollback a process session. If FlowFiles are penalized, it will not be fetched again until penalization expires.
- Yield processor when a failure occurs and RollbackOnFailure is enabled. If we do not penalize nor yield, a failed FlowFile retries too frequently.
- When Rollback on Failure is enabled but processor is not transactional, discontinue when an error occurred after successful processes.
- Fixed existing issues on PutHiveStreaming:
  - Output FlowFile Avro format was corrupted by concatenating multiple Avro files.
  - Output FlowFile records had incorrect values because of reusing GenericRecord instance.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1658
2017-04-27 13:44:56 -04:00
Mark Payne a1bffbcc87 NIFI-3659: This closes #1707. Added Processor to Split a FlowFile consisting of Record-oriented data into multiple FlowFiles, each containing a subset of the original FlowFile's records
Signed-off-by: joewitt <joewitt@apache.org>
2017-04-27 12:51:01 -04:00
Matt Burgess 097548da9d NIFI-3745: Fixed Table caching / primary key logic in PutDatabaseRecord
This closes #1700.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-27 08:43:13 +09:00
Matt Burgess 4303e4742a NIFI-3704: Add PutDatabaseRecord processor
NIFI-3704: Incorporated review comments

NIFI-3704: Updated PutDatabaseRecord documentation, including limitations on Update records

NIFI-3704: Added retry paths for transient SQL errors

NIFI-3704: Updated to use renamed RecordReaderFactory and catch SchemaNotFoundException

NIFI-3704: Added better error handling and rollback

This closes #1677.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-04-26 09:50:50 +09:00
Andy LoPresto ef5bac207e
NIFI-3738 Fixed NPE when ListenSyslog UDP datagram has zero length. Added default constructor to SyslogParser to allow map coercion for test. Added unit test.
NIFI-3738 Fixed NPE when ParseSyslog UDP datagram has zero length.
Added unit test.

NIFI-3738 Added licenses to new unit tests.

This closes #1694.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-04-25 16:38:46 -04:00
Matt Burgess 9ed001869b NIFI-2323: Fixed Oracle timestamp handling in DB fetch processors. This closes #1664. 2017-04-25 09:59:43 -04:00
Mark Payne 50ea1083ec NIFI-3682: This closes #1682. Add Schema Access Strategy and Schema Write Strategy Record Readers and Writers; bug fixes.
Signed-off-by: joewitt <joewitt@apache.org>
2017-04-24 17:02:45 -04:00
Mark Payne b93cf7bbdb NIFI-3658: Created processor for converting records between data formats with like schemas
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-3658: Incorporated PR review feedback; added counter; clarified documentation

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1668
2017-04-19 16:12:14 -04:00
Andre F de Miranda cd260bab9f
NIFI-2747 - Introduce FuzzyHashContent processor
This closes #1619.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-04-18 10:27:15 -04:00
joewitt d08233b9fe NIFI-3694 updated latest logback,jetty,moved servlet-api and jetty-schemas to lib root as needed now
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1678
2017-04-18 07:41:21 -04:00
Yolanda M. Davis 5f73ba7f2c NIFI-3708 - removed typo from character escape regex
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1674
2017-04-14 22:13:09 -04:00
Oleg Zhurakousky 68c592ea43 NIFI-1280 added support for RecordSchema in SchemaRegistry
Signed-off-by: Mark Payne <markap14@hotmail.com>
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-1280: Updated SimpleKeyValueSchemaRegistry to make use of new CHOICE RecordFieldType - Update Record Readers to use SchemaRegistry controller service. Moved SchemaRegistry api into its own maven module and added to standard-services-api so that we can properly add dependencies on it. Code cleanup and bug fixes

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-1280: Fixed checkstyle violations and license exclusions for RAT plugin

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-1280: Addressed feedback from PR Review

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-1280: Additional changes/doc to support QueryFlowFile and Record Readers/Writers

This closes #1652
2017-04-11 19:29:35 -04:00
Mark Payne a88d3bfa3c NIFI-1280: Refactoring to make more generic so that other data types can be supported; created InputStreams to content on-demand so that multiple passes can be made over FlowFile content if required. Created new Controller Services for reading and writing specific data types
Signed-off-by: Matt Burgess <mattyb149@apache.org>
2017-04-11 19:29:04 -04:00
Toivo Adams 4d5872a385 NIFI-1280 Create FilterCSVColumns Processor.
Signed-off-by: Matt Burgess <mattyb149@apache.org>
2017-04-11 19:29:04 -04:00
Mark Payne 8a5398eba7 NIFI-3679: This closes #1655. Added Avro Content Viewer
Signed-off-by: joewitt <joewitt@apache.org>
2017-04-11 00:00:09 -04:00
Koji Kawamura 9583ca99c1 NIFI-3414: Added EnforceOrder processor
Use it with FirstInFirstOutPrioritizer, it can enforce original ordering
of 'out-of-order' FlowFiles.

nifi-mock is modified to support FlowFile assertion using Prioritizer.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-3414: Added EnforceOrder processor

Incorporated review comments, added displayNames.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-3414: Added EnforceOrder processor

Incorporate review comments:

- Moved nifi-standard-prioritizers dependency to top level nifi/pom.xml.
- Changed default initial order from 1 to 0.
- Fixed typos.
- Use session.get(batchCount).

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-3414: Added EnforceOrder processor

When a FlowFile is transferred to success, remove attributes previously set when it was transferred to wait or failure.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1496
2017-04-10 14:03:22 -04:00
Pierre Villard 02d38b55bf NIFI-2952 Fixed search value validation in ReplaceText
This closes: #1378

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-04-11 00:00:07 +10:00
Mark Payne a5d630672a NIFI-3635: This closes #1631. Avoid using a static member variable for the 'Grok' object. Code cleanup
Signed-off-by: joewitt <joewitt@apache.org>
2017-04-07 13:22:46 -04:00
Bryan Bende 556f309df0
NIFI-3520 Refactoring instance class loading
- Fixing FlowController to use appropriate class loader when instantiating processor
- Updating ExtensionManager to leverage new flag in MANIFEST from NAR plugin
- Adding ReloadComponent interface and refactoring instance class loading to use it
- Fixing FetchHDFS issue with TDE by using ugi.doAs
- Refactoring nifi-nar-utils so that ExtensionManager only lives in nifi-framework
- Caching temp components found during service loader in ExtensionManager
- Updating authorizables, docs, and fingerprinting to use the cached components
- Introducing a flag on @RequiresInstanceClassLoading to indicate if ancestor resources should be cloned
- Updating developer guide regarding cloneAncestorResources flag
- This closes #1635
2017-04-06 13:51:08 -04:00
joewitt 6a64b3cd9c NIFI-3440 This closes #1638. fixing tests not written for windows to not run on windows 2017-03-31 01:59:35 -04:00
Mark Payne 83fa24b68f NIFI-3613: Fixed threading bug in JoltTransformJSON and adding caching of compiled Transforms
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1630
2017-03-29 10:45:34 -04:00
Pierre Villard ced6708d4b NIFI-3596 - added attributes to GenerateTableFetch processor
Signed-off-by: Matt Burgess <mattyb149@apache.org>

Updated test to check selected column names

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1596
2017-03-27 08:50:55 -04:00
Bryan Bende d90cf846b9 NIFI-3380 Bumping NAR plugin to 1.2.0-SNAPSHOT development to leverage changes from master, adding buildnumber-maven-plugin to nifi-nar-bundles to properly set build info in MANIFEST of NARs
- Refactoring NarDetails to include all info from MANIFEST
- Adding the concept of a Bundle and refactoring NarClassLoaders to pass Bundles to ExtensionManager
- Adding logic to fail start-up when multiple NARs with same coordinates exist, moving Bundle classes to framework API
- Refactoring bundle API to classes and creating BundleCoordinate
- Updating FlowController to use BundleCoordinate

- Updating the UI and DTO model to support showing bundle details that loaded an extension type.
- Adding bundle details for processor canvas node, processor dialogs, controller service dialogs, and reporting task dialogs.
- Updating the formating of the bundle coordinates.
- Addressing text overflow in the configuration/details dialog.
- Fixing self referencing functions.
- Updating extension UI mapping to incorporate bundle coordinates.
- Discovering custom UIs through the supplied bundles.
- Adding verification methods for creating extensions through the rest api.
- Only returning extensions that are common amongst all nodes.
- Rendering the ghost processors using a dotted border.
- Adding bundle details to the flow.xml.
- Loading NiFi build and version details from the framework NAR.
- Removing properties for build and version details.
- Wiring together front end and back end changes.
- Including bundle coordinates in the component data model.
- Wiring together component data model and flow.xml.
- Addressing issue when resolve unvesioned dependent NARs.

Updating unit tests to pass based on framework changes
- Fixing logging of extension types during start up

- Allowing the application to start if there is a compatible bundle found. - Reporting missing bundle when the a compatible bundle is not found. - Fixing table height in new component dialogs.

Fixing chechstyle error and increasing test timeout for TestStandardControllerServiceProvider
- Adding ability to change processor type at runtime
- Adding backend code to change type for controller services

- Cleaning up instance classloaders for temp components.
- Creating a dialog for changing the version of a component.
- Updating the formatting of the component type and bundle throughout.
- Updating the new component dialogs to support selecting source group.
- Cleaning up new component dialogs.
- Cleaning up documentation in the cluster node endpoint.

Adding missing include in nifi-web-ui pom compressor plugin
- Refactoring so ConfigurableComponent provides getLogger() and so the nodes provide the ConfigurableComponent
- Creating LoggableComponent to pass around the component, logger, and coordinate with in the framework

- Finishing clean up following rebase.

Calling lifecycle methods for add and remove when changing versions of a component
- Introducing verifyCanUpdateBundle(coordinate) to ConfiguredComponent, and adding unit tests

- Ensuring documentation is available for all components. Including those of the same type that are loaded from different bundles.

Adding lookup from ClassLoader to Bundle, adding fix for instance class loading to include all parent NARs, and adding additional unit tests for FlowController
- Adding validation to ensure referenced controller services implement the required API
- Fixing template instantiation to look up compatible bundle

- Requiring services/reporting tasks to be disabled/stopped.
- Only supporting a change version option when the item has multiple versions available.
- Limiting the possible new controller services to the applicable API version.
- Showing the implemented API versions for Controller Services.
- Updating the property descriptor tooltip to indicate the required service requirements.
- Introducing version based sorting in the new component dialog, change version dialog, and new controller service dialog.
- Addressing remainder of the issues from recent rebase.

Ensuring bundles have been added to the flow before proposing a flow, and incorporating bundle information into flow fingerprinting
- Refactoring the way missing bundles work to retain the desired bundle if available
- Fixing logger.isDebugEnabled to be logger.isTraceEnabled

- Auditing when user changes the bundle. - Ensuring bundle details are present in templates.

Moving standard prioritizers to framework NAR and refactoring ExtensionManager logic to handle cases where an extension is in a JAR directly in the lib directory

- Ensuring all nodes attempt to instantiate the same template instance when the available bundles may differ. - Fixing the auditing of copy/paste and template instantiation. - Running addtional verification methods when running standalone.

Refactoring controller service invocation handler to allow updating the node used by the invocation handler
- Ensuring the bundles in a proposed flow are compatible with the current instance when the current instance has no flow is going to accept the proposed flow
- Merging whether multiple versions of the component are available
- Setting NAR plugin back to current released version
- Cleaning up DocGenerator to not process multiple times

Addressing incorrect usage of nf.Common. - Using formatType in the new component type dialogs.

Improving error messages when looking for bundles

Addressing comments from PR. - Fixing references to global nf namespace. - Fixing injection of nfProcessGroupConfiguration in nfComponentVersion. - Fixing web api integration tests.

Not rendering unversioned in help documentation. - Ensuring the isExtentionMissing flag is correct after changing the component type.

Adding synchronization in node classes to ensure changing component can't occur when component is running, introducing MissingBundleException for better reporting when a node can't join cluster due to a missing bundle, and bumping NAR plugin to released version 1.2.0

Adding concept of missing components to fingerprinting to ensure nodes agree on missing components when joining a cluster

NIFI-3380: NIFI-3520: - Fixing hive nar dependency. - Marking DBCPService as provided. - Skipping services that require instance classloading and are cobundled with their service API. - Skipping components that require instance classloading and reference service APIs that are cobundled. - Addressing UI issues in the new component dialogs when re-opening with a filter applied.

Fixing checkstyles issue and adding back assume checks to distributed cache server test

Ensuring new component types are sorted correctly when shown initially.

This closes #1585.
2017-03-24 11:06:44 -04:00
Andre F de Miranda a298f1a707 NIFI-3466 - Address non English locales in ParseCEF
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1614.
2017-03-23 09:04:16 +01:00
Matt Burgess d1ebddce98 NIFI-3627: Added removeByPattern() to DistributedMapCache interfaces
NIFI-3627: Updated unit tests that use MapCache interface(s)

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1609.
2017-03-22 19:22:08 +01:00
Matt Burgess 37a1e6d07f NIFI-3585: Add DatabaseAdapter impl for MS SQL 2008
This closes #1584

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2017-03-22 14:57:42 +00:00
Matt Burgess c4d0c0bbd1
NIFI-3626: Add support for DELETE in ConvertJSONToSQL
This closes #1605.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-03-21 17:20:27 -04:00
Andy LoPresto b3b65219a0
NIFI-3361:
- Refactored TestGetHTTPGroovy for componentization & customization of tests.
- Ignored some TestGetHTTPGroovy tests because they cannot connect to a Jetty server running TLSv1 or TLSv1.1 (the new version of Jetty only supports TLSv1.2+). The tests are still valuable to demonstrate GetHTTP's capacity to connect to various TLS protocols, and should not be removed unless necessary.
- Refactored TestPostHTTPGroovy for componentization & customization of tests.
- Ignored some TestPostHTTPGroovy tests because they cannot connect to a Jetty server running TLSv1 or TLSv1.1 (the new version of Jetty only supports TLSv1.2+). The tests are still valuable to demonstrate PostHTTP's capacity to connect to various TLS protocols, and should not be removed unless necessary.
- Restored one of the TestGetHTTPGroovy unit tests (GetHTTP vs. TLSv1/1.1/1.2) by connecting to https://nifi.apache.org which still supports all three protocol versions (and uses JVM cacerts as truststore).
- This closes #1601
2017-03-21 14:48:40 -04:00
Matt Gilman 8e61aa3c43
NIFI-3361:
- Upgrading to the latest Jetty version.
2017-03-21 13:44:26 -04:00
Joe Percivall 5daeba6e03 NIFI-3614 Adding "return" statement to HandleHttpResponse when statusCode is not a number
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1598.
2017-03-20 14:51:56 +01:00
ambud 9424836412 NIFI-3525 Add tablename attribute to QDT flowfiles
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1537.
2017-03-13 14:22:14 -07:00
Peddy ccd11816e4 NIFI-3034 This closes #1323. Update Counter and test case 2017-03-12 16:34:33 -04:00
patricker 18f05856ff NIFI-3481 - Add DB Adapter for MS SQL 2012+
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-3481 Added support for Database Adapter Descriptions

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-3481 Updated Adapter Name

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1510
2017-03-10 11:34:16 -05:00
Andre F de Miranda 5a6373b16c
NIFI-3442 Explicitly support EL on username, password, port.
Introduced basic test unit

This closes #1474.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-03-08 20:56:01 -08:00
Pierre Villard 37e9c5efc7 NIFI-3556 Fixed ValidateCsv schema parsing
Fixed schema parsing on comma boundaries.

This closes #1570.

Signed-off-by: James Wing <jvwing@gmail.com>
2017-03-08 14:57:33 -08:00
Joe Skora 90ff275bb9 NIFI-3517 If HandleHttpResponse cannot write response remove entry from HttpContextMap.
* Add test for failure not clear context map.
* Add handler to remove context map entry if ProcessException occurs during while exporting response.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1567.
2017-03-08 13:16:58 +01:00
Pierre Villard 70175816b3 NIFI-3541 NIFI-3545 - check style violations
Fix for checkstyle issues introduced by:
9e68f02f1f
000414e7ea

This closes #1569.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-03-07 22:17:23 +09:00
Koji Kawamura 000414e7ea
NIFI-3545: Release M FlowFilews once N signals arrive
- Support multiplle incoming FlowFiles to Wait processor, up to Wait
  Buffer Count
- Added Releasable FlowFile Count, which controls how many FlowFiles can
  be released when wait condition is met
- Added special meaning to Notify delta Zero(0) to clear a signal
  counter back to zero

  This closes #1554

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-03-06 09:29:44 -05:00
Mark Payne 7f5c0dfb54 NIFI-3553: If IOException is thrown when completing FTP transfer, do not update the FlowFile to point to the content that was streamed but instead leave FlowFile pointing to original version
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1562.
2017-03-05 15:27:48 +01:00
Pere Urbon-Bayes 33a702500c NIFI-3554 Fixing the jBCrypt dependency name from nifi-standard-processors/pom.xml file, so it uses the expected naming format from the maven repository
Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-03-05 10:15:06 +11:00
Joe Trite f8cad0f8c9
NIFI-1750 Updated ScanAttribute dictionary file processor property description.
This closes #1543.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-03-03 17:36:40 -08:00
Giovanni Lanzani 5990db39ae
NIFI-3459 Fixed documentation for UnpackContent File Filter.
This closes #1558.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-03-02 16:19:11 -08:00
josephxsxn ac5a7960e8
NIFI-3547 Changing level from WARN to INFO for Wait Processor's expected behavior
This closes #1556.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-03-02 15:37:39 -05:00
patricker d689ac561e NIFI-3494: GenerateTableFetch Some JDBC Drivers Have Trouble With Name vs Label
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1517
2017-03-02 11:12:54 -05:00
patricker bf2f04fb5f NIFI-3540: QueryDatabaseTable Failing to Track MS SQL Max Values
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1547
2017-03-02 10:04:09 -05:00
Oleg Zhurakousky 8e17929d6a NIFI-2876 refactored demarcators into a common abstract class
This closes #1214.
2017-02-24 10:06:53 -05:00