Apache
/
nifi
mirror of https://github.com/apache/nifi.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,448 Commits 147 Branches 201 Tags 399 MiB
Commit Graph

2704 Commits

Author SHA1 Message Date
Matt Gilman b7272e3f32 NIFI-4997: 
- Fixing process group audit advice.
- Setting spring security user in background threads.
- Removing unnecessary overloaded methods.

This closes #2626.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-04-11 16:18:36 -04:00
Pierre Villard 6fbe1515ee NIFI-4809 - Implement a SiteToSiteMetricsReportingTask 
Fixed dependency issue by providing a local JSON reader

Rebased + fixed conflict + updated versions in pom + EL scope

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2575
 2018-04-11 14:44:30 -04:00
mans2singh 4366c67b27 NIFI-4927 - InfluxDB Query Processor 
This closes #2562

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
 2018-04-11 06:58:07 -04:00
Matthew Burgess b29304df79 NIFI-4857: Support String<->byte[] conversion for records 
This closes #2570.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-04-09 14:53:21 -04:00
Matthew Burgess ae1d3e3945 NIFI-4862: Keep incoming flowfile attributes in outgoing flowfiles from SelectHiveQL 
This closes #2605.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2018-04-09 12:31:05 +09:00
joewitt 59f625d3c2 Merge branch 'NIFI-4995-RC3' as part of NiFi 1.6.0 release process 2018-04-06 16:48:08 -07:00
Vi On b414132324 NIFI-5050 update frontend-maven-plugin 
This closes #2613

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
 2018-04-06 16:07:25 -04:00
Mike Thomsen 63d4bba5a4 NIFI-5047 Fixed a bug in PutMongo that prevented it from working if the mode is insert and the query/query key are not set. 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2609.
 2018-04-06 21:36:30 +02:00
Andrew Lim 8e10f0bd54 NIFI-5048 Corrected typo in KeytabCredentialsService controller service description 
This closes #2612

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
 2018-04-06 15:17:03 -04:00
Pierre Villard 7cc77937d8 NIFI-4149 - fixed contrib-check errors 2018-04-06 20:38:04 +02:00
Mark Payne 644133dc35 NIFI-4149: Minor tweaks to verbiage 
This closes #2205.
 2018-04-06 12:42:38 -04:00
Pierre Villard 4c787799ff NIFI-4149 - Indicate if EL is evaluated against FFs or not 
- take into account input requirement for documentation rendering
- Renamed variable registry scope and added comments
- Doc + change in mock framework to check scope + update of components + UI
 2018-04-06 12:40:54 -04:00
Matthew Burgess 7ff38f690d
NIFI-5045: Fixed error code handling in PutHiveQL. This closes #2608 2018-04-05 16:03:42 -04:00
Marco Gaido c119547222 NIFI-5043: TailFile in Multifile mode should not open new readers in onTrigger 
This closes #2606.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-04-05 13:53:01 -04:00
Mike Moser dc9b4cb516
NIFI-3599 Allow back pressure object count and data size to be configurable in nifi.properties. This closes #2497 2018-04-05 13:50:16 -04:00
Koji Kawamura 1913b1e2a8
NIFI-4932: Enable S2S work behind a Reverse Proxy 
Adding S2S endpoint Reverse Proxy mapping capability.
Added license header to SVG files.
Incorporated review comments.
Use regex to check property key processing.
Catch AttributeExpressionLanguageParsingException.
This closes #2510
 2018-04-03 15:40:28 -04:00
joewitt d511fe3e4b NIFI-4995-RC3 prepare for next development iteration 2018-04-03 08:28:34 -07:00
joewitt f8466cb16d NIFI-4995-RC3 prepare release nifi-1.6.0-RC3 2018-04-03 08:28:15 -07:00
Matt Gilman 364985fb41 NIFI-5034: 
- Processing properties and property descriptors in Controller Service referencing components unconditionally.

This closes #2602.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-04-03 10:34:33 -04:00
Matt Gilman 924c935a38 NIFI-5034: 
- Ensuring descriptors and properties of referencing components are populated like Processors, Reporting Tasks, etc.
 2018-04-03 10:34:33 -04:00
patricker 2279fad673 NIFI-5037 - ConsumeEWS Fails to Read Emtpy Message Body Emails 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2603.
 2018-04-03 14:38:38 +02:00
Mark Bean c59b6fdf66 NIFI-4658 set Maximum Number of Entries to required and allow FlowFiles having fragment.count greater than Max Entries property 
Signed-off-by: Mike Moser <mosermw@apache.org>

This closes #2559
 2018-04-02 20:42:04 -04:00
Bryan Bende b7e1f48133 NIFI-5035 Moving MongoDB services out of standard services 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2599
 2018-04-02 12:18:27 -04:00
Mark Payne 94d444abb0 NIFI-4883: Updated ValidateRecord to allow an optional Record Writer to be configured for invalid records that differs from the Record Writer used for writing valid records 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2600.
 2018-04-02 18:16:40 +02:00
Matt Gilman bbe79d2260 NIFI-5033: 
- Ensuring appropriate response in checkAuthorization when user is null.
- Ensuring the user reference is passed down when applying variable changes.

This closes #2598.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-04-02 10:35:04 -04:00
Bryan Bende 134339c4b7
NIFI-5021 Moving nifi-elasticsearch-client-service-api to nifi-elasticsearch-bundle and creating NAR for the API 
This closes #2586.
 2018-04-02 09:49:05 -04:00
Arun A K 5d90d14f17 NIFI-5017 - ConvertExcelToCSVProcessor - EL support for rows and columns to skip 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2597.
 2018-03-30 11:45:48 +02:00
Matt Gilman c4dcde837e NIFI-5032: This closes #2596. 
- Overriding version ranges of transitive dependencies through oauth2-oidc-sdk.

Signed-off-by: joewitt <joewitt@apache.org>
 2018-03-29 12:21:40 -04:00
Mark Payne bbbe428e7b NIFI-5030: If ControlRate encounters a FlowFile that cannot be transferred, it should continue processing other FlowFiles that have different group attribute values 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2593.
 2018-03-29 10:31:15 +02:00
Scott Aslan c1459825bb
[NIFI-4998] update node and npm version 
This closes #2571
 2018-03-28 16:49:32 -04:00
Mark Payne 641ba29cc8 NIFI-4993: Accept null user at querying ProvenanceRepository 
Pass null user from ReportLineageToAtlas.

Address NullPointer dereferences

This closes #2589.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-03-28 12:15:05 -04:00
Mark Payne f394c874e1
NIFI-4895: Added backend code to give users the ability to forcibly terminate a processor if the processor does not complete after clicking Stop 
NIFI-4895: Addressed review feedback; addressed checkstyle issues
This closes #2555
 2018-03-28 12:02:47 -04:00
Ed 9de73fbe1f NIFI-4388: Modules Not Honored 
The bug is not reproducible in Jython (it handles modules with every script reload). But Groovy loads JARs and dirs with classes only on setup.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2584
 2018-03-27 11:20:40 -04:00
joewitt 7b5bf265a6 NIFI-4995 updating copyright year on all notices 2018-03-26 21:54:10 -04:00
Joe Percivall 56ad14ad1e
NIFI-4977 Adding expression language support to the Sender properties of PutSyslog 
This closes #2547.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2018-03-26 15:49:52 -04:00
Mike Thomsen aa947e4d3e NIFI-4325 Added new processor that uses the JSON DSL. 
NIFI-4325 Cleaned up how ElasticSearch client service builds SSLContext, added query attribute to flowfiles and other changes requested in a code review.

This closes #2113.

Signed-off-by: Joe Percivall <JPercivall@apache.org>
 2018-03-26 15:01:37 -04:00
Mike Thomsen 6b34d3bea9 NIFI-4743 - Added configurable null suppression to PutElasticsearchHttpRecord 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2501.
 2018-03-26 20:41:21 +02:00
Mark Payne 456c9c8fc0
NIFI-5012: When connecting to cluster, esure that controller services appropriately enabled/disabled 
This closes #2579.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2018-03-26 13:07:58 -04:00
zenfenan 3612fbe522
NIFI-4864: Improvements to PR #2470 
This closes #2581.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2018-03-26 10:49:20 -04:00
Matt Gilman 69a564e4c8
NIFI-5009: 
- Fixing required permission for PutParquet.

NIFI-5008:
- Ensuring all restricted components are tagged as such.

This closes #2583.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2018-03-26 09:47:54 -04:00
Otto Fowler 98c295dfc7
NIFI-5013 create test jar for standard processors 
remove unnec TestServer instances

This closes #2580.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2018-03-24 10:18:46 -04:00
Mark Payne 03f4c0fc47 NIFI-5011: Fixed threading bug in TestFileSystemRepository 
This closes #2578.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-03-23 15:46:17 -04:00
Marco Gaido 758e44682f NIFI-4631: Use java.nio.file.Files in ListFile to improve performance 
This closes #2565.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-03-23 11:00:21 -04:00
Mike Thomsen 66590b78f5 NIFI-4989 Made PutMongo able to use nested lookup keys, a query param and multiple lookup keys. 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2560.
 2018-03-23 14:46:50 +01:00
Derek Straka 95dd1ebffc NIFI-4882: Resolve issue with parsing custom date, time, and timestamp formats in CSV files 
Refactor the two existing CSV classes to inherit from an abstract base class that parses the data
Add unit tests for calls to nextRecord that do not coerce types

This closes #2473.

Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2018-03-22 09:18:00 +09:00
Mark Payne 0b0aebe148
NIFI-4917: Externalize Keytab and Principal configuration from Processors to a Controller Service. This gives us the ability to allow users to interact with those Keytabs/Principals to which they've been given access without allowing them access to all Keytabs and Principals 
- Addressed review feedback; rebased against master

This closes #2552.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2018-03-21 14:23:41 -04:00
sbouchex 9c9215982f NIFI-4999: Added request URI into the "restlistener.request.uri" attribute 
This closes #2574.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-03-21 13:17:17 -04:00
Matt Gilman 98cd9ad531
NIFI-4885: 
- Updating the versioning endpoints to account for the granular access restrictions.

This closes #2573.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2018-03-21 11:45:13 -04:00
joewitt e8a8c19db2 Revert "NIFI-4809 - Implement a SiteToSiteMetricsReportingTask" 
This reverts commit 930417b9dc due to NIFI-4996
 2018-03-20 12:58:50 -04:00
Andy LoPresto d09145e3e3 NIFI-4557 Added Expression Language support for PGP private keyring passphrase in EncryptContent. 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2566.
 2018-03-20 11:23:45 +01:00
Koji Kawamura 2590c88318 NIFI-4979 - Fix ReportLineageToAtlas documentation errors 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2549.
 2018-03-19 17:56:13 +01:00
Mark Payne 39556e3513 NIFI-4902: This closes #2485. Updated ConsumeAMQP, PublishAMQP to use one connection per concurrent task instead of a single connection shared by all concurrent tasks. This offers far better throughput when the network latency is non-trivial. Also refactored to simplify code 
Signed-off-by: joewitt <joewitt@apache.org>
 2018-03-19 11:40:49 -04:00
Derek Straka 5bdb7cf6e7 NIFI-4912: This closes #2494. Update jackson version to latest stable version (2.9.4) 
Signed-off-by: joewitt <joewitt@apache.org>
 2018-03-19 10:22:50 -04:00
Internet 1091093133 NIFI-4777: get schema by id even if not latest 
This closes #2405.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-03-19 09:57:38 -04:00
Pierre Villard 4687e0fb35 NIFI-4981 Allow using FF's content as message in PutEmail 
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2556.
 2018-03-18 11:35:43 -07:00
Mike Moser c9f1c9789d NIFI-4973 Added license comment to new file RemoteProcessGroupEntityMergerTest.java 2018-03-16 19:15:52 +00:00
Julian Gimbel 9cd0aab696 NIFI-4918: looping over several methods to try and fit the dynamic attribute. 
If failed, use first method and throw error if not working.

Array.length can never be less then 0

This closes #2499.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-03-16 14:40:59 -04:00
Mark Bean 8b2c5b7246 NIFI-4950 Defining behavior for MergeContent when more than 1 FlowFile has the same fragment.index value 
Signed-off-by: Mike Moser <mosermw@apache.org>

This closes #2557.
 2018-03-16 18:31:43 +00:00
Derek Straka c056ede6cc NIFI-4951: Update convertToAvroObject to use the DataTypeUtils conversion function 
The feature allows users to convert from non-integral types to the correct underlying type.  The
original behavior is maintained; however, now simple conversions take place automatically for some
logical types (date, time, and timestamp).

This closes #2526.

Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-03-16 14:08:14 -04:00
Matt Gilman a1c917656e NIFI-4973: 
- Fixing RPG port merging.
- Adding unit tests.
- Removing unecessary sorting that wasn't maintained while clustered.

This closes #2551.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-03-16 13:03:09 -04:00
Marco Gaido 86f162b611 NIFI-4959: Remove flowfiles and close connection for Bad Requests causing IOException 
This closes #2544.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-03-16 12:24:00 -04:00
Mark Payne 3f0aa6e13c Changed some unit tests to Integration Tests because they are creating embedded JMS Brokers (implicitly) and commnicating with it. This becomes brittle in a full build, especially if any other unit test attempts to create a broker implicitly due to the way that the activemq broker implicit creation works. 2018-03-16 12:07:33 -04:00
Koji Kawamura ada069ce42 NIFI-4978: Fixed ReportLineageToAtlas NPE when unscheduled 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2548.
 2018-03-16 10:19:59 +01:00
Mark Payne 7d5f803ce6 Fixed checkstyle violation 2018-03-15 16:16:12 -04:00
Bryan Bende de71a41bd0 NIFI-4935 Refactoring to support specifying schema branch or schema version when using schema by name strategy 
This closes #2523.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-03-15 16:16:12 -04:00
Pierre Villard 930417b9dc NIFI-4809 - Implement a SiteToSiteMetricsReportingTask 
address review comments

NIFI-4809 - Added Record Writer property

added unit tests and additional details doc

review comments

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2430
 2018-03-15 15:58:31 -04:00
Mark Payne 844da06344
NIFI-4849: Implemented REST Endpoint and associated backend code to generate a Diagnostics Report for a Processor 
Implemented review feedback. Refactored data model to make the API cleaner and delineate more along the lines of what permissions are required in order to see which details
Implementing review feedback
Removed sensitive information from the diagnostics reports
Fixed bug in merging logic for GCDiagnosticsSnapshots
This closes #2468
 2018-03-15 15:10:41 -04:00
Mark Payne 867ffdb52e NIFI-4976: If unable to retrieve message content, warn an error but acknowledge message. 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2545.
 2018-03-14 22:50:34 +01:00
Koji Kawamura d3f54994a6 NIFI-4972 - SelectHiveQL to emit FETCH provenance event 
SelectHiveQL should emit FETCH instead of CONTENT_MODIFIED when it has
incoming connections.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2543.
 2018-03-14 10:17:19 +01:00
Jorge Machado b0e5e1644c NIFI-4821 - Upgrade poi to 3.17 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2534.
 2018-03-13 22:59:03 +01:00
Deon Huang 37ad48844a NIFI-4962 FlattenJson processor add unexpected backslash after flatten 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2538.
 2018-03-13 22:33:53 +01:00
Pierre Villard 11a7d5c09f NIFI-4955 - Preserve columns ordering with CSV and ValidateRecord 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2528
 2018-03-13 17:09:07 -04:00
Pierre Villard 95e543d4cc NIFI-4967 - CSVRecordReader does not read header with specific formats 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2536
 2018-03-13 16:54:46 -04:00
Pierre Villard c75604a564 NIFI-4966 - JacksonCSVRecordReader - NPE with some CSV formats 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2535
 2018-03-13 16:49:47 -04:00
Ed 29c2e18832 NIFI-4968: fix printing indefinite log errors 
After first failure in customValidate, stop printing logs until any property is changed

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2537
 2018-03-13 16:40:49 -04:00
Bryan Bende 9b098f3235
NIFI-4864 - Additional improvements to additonal resource loading 
This closes #2470.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2018-03-13 13:23:28 -04:00
zenfenan fad152f389
NIFI-4864 Fixing Additional Resources property pointing at a directory won't find new JARs 
NIFI-4864: Code refactor and improved additionalUrlsFingerprint implementation
Signed-off-by: Bryan Bende <bbende@apache.org>
 2018-03-13 13:23:16 -04:00
Bryan Bende f1d7518547
NIFI-4969 Fixing error when importing versioned flow with a processor that uses event driven scheduling. This closes #2539 2018-03-13 13:15:24 -04:00
Mark Payne d14229e440 NIFI-4774: Allow user to choose which write-ahead log implementation should be used by the WriteAheadFlowFileRepository 
Removed TODO comment

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2487
 2018-03-13 13:03:39 -04:00
Ed c2616e6fe7
NIFI-4833 Add ScanHBase Processor 
- New processor for scanning HBase records based on verious params like range of rowkeys, range of timestamps. Supports result limit and reverse scan.
- Adds Atlas Support for ScanHBase processor
- Fixed not recent version of FF
- Formatting and Style changes
- Single line to multiline if-then statements
- Removed HTML formatting that is not used for doc generation
- Fixed issue with limitRows
- Fixed issue with filter expression
- Refactored "processRows"
- Fixed possible NPE for bulkSize var
- Changed provenance to "receive" to indicate new data from external source.
- Updated min/max timestamp custom validation
- JSON array support
- Removed in-memory caching for records. Now records are being written directly to FF
- Removed unfinished flowfile from session, transfered original to "failure". Test cases update as well

This closes #2478.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2018-03-13 10:07:07 -04:00
m-hogue 6471f66acd NIFI-3402 - Added etag support to InvokeHTTP 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2150.
 2018-03-13 10:00:11 +01:00
Deon Huang 7aabb99bcc NIFI-4800 Expose the flattenMode as property in FlattenJSON processor 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2530.
 2018-03-13 09:28:22 +01:00
joewitt d78d95ad6f NIFI-4885 fixing checkstyle issue 2018-03-12 22:22:59 -04:00
Ed 373cf090a4 NIFI-4953 - FetchHBaseRow - update log level for "not found" to DEBUG instead of ERROR 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2527.
 2018-03-12 20:21:40 +01:00
Matt Gilman b1217f529b NIFI-4885: 
- Introducing more granular restricted component access policies.

This closes #2515.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-03-12 14:19:42 -04:00
Matthew Burgess d4632bdd5d
NIFI-4944: Guard against race condition in Snappy for PutHiveStreaming 
NIFI-4944: Removed unnecessary synchronized block, added more comments
This closes #2519
 2018-03-12 13:22:40 -04:00
Matt Gilman 32a8db2253
NIFI-4936: 
- Moving definition of ranger version into root pom.

This closes #2532.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2018-03-12 10:39:54 -04:00
Jorge Machado 06cb354c5e NIFI-4960: fix object setting. This closes #2531. 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-03-12 09:44:23 -04:00
Joe Skora 49c7af03c1 NIFI-3039 Provenance Repository - Fix PurgeOldEvent and Rollover Size Limits 
* Added low water mark for purgeOldEvents() to more quickly purge after a large spike in events.
* Adjusted rollover high water mark to avoid overrunning "nifi.provenance.repository.max.storage.size".
* Adjusted looping logic in mergeJournals() to use ".firstKey()" instead of ".entrySet().iterator().next()" to avoid unnecessary object creation.

Signed-off-by: Mike Moser <mosermw@apache.org>
This closes #1240
 2018-03-09 21:49:13 +00:00
joewitt c71409fb5d
NIFI-4936 trying to quiet down the mvn output a bit so we dont exceed the travis-ci 4MB max 
NIFI-4936 updated dependency handling pushing down delcarations where they belong
This closes #2512
 2018-03-09 16:34:53 -05:00
Andy LoPresto e9d585a785
NIFI-4870 Upgraded activemq-client and activemq-broker versions to 5.15.3. 
- Updated copyright dates in NOTICE files.
- This closes #2469.
 2018-03-09 13:59:13 -05:00
Mike Thomsen 3cdd3f60f6 NIFI-4949 Converted nifi-mongodb-services' unit tests into integration tests so that the @Ignore annotation doesn't have to be removed to make them run. 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2525.
 2018-03-09 09:36:15 +01:00
Kevin Doran 79fa4ad46f
NIFI-4835 Corrects Swagger spec response types in FlowResource 2018-03-08 14:30:31 -05:00
Kevin Doran fdca7d0a01
NIFI-4859 Corrects Swagger Spec VersionedFlowState allowableValues 2018-03-08 14:30:30 -05:00
Pierre Villard 1f8af1bde3 NIFI-4948 - MongoDB Lookup Service throws an exception if there is no match 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2522
 2018-03-08 13:28:41 -05:00
Mike Thomsen dae2b73d94 NIFI-4929 Converted the majority of MongoDB unit tests to integration tests so they can be reliably run with 'mvn -Pintegration-tests integration-test' 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2508.
 2018-03-08 19:23:44 +01:00
gardellajuanpablo 2e4ddbf3b2 NIFI-4893 Cannot convert Avro schemas to Record schemas with default arrays 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-03-06 14:09:39 -05:00
Takanobu Asanuma bfecf0f06c
NIFI-4855: 
- Fix the layout of NiFi API document
- update the sub-title to be more detailed
- This closes #2503
 2018-03-06 10:30:18 -05:00
Matt Gilman 91f40febeb NIFI-4925: 
- Addressing memory leak from lingering authorization results that did not represent actual access attempts. This closes #2511.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-03-05 15:37:31 -05:00
Jeff Storck e916594b69 NIFI-4872 Added annotation for specifying scenarios in which components can cause high usage of system resources. 
- Initial set of components marked with the HighResourceUsageScenario annotation.
- Added customized descriptions to SystemResourceConsideration annotations for MergeContent, SplitContent, SplitJson, SplitText, and SplitXml.

This closes #2475.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-03-05 15:32:23 -05:00
Matthew Burgess dd58a376c9 NIFI-4773 - Fixed column type map initialization in QueryDatabaseTable 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2504.
 2018-03-02 17:32:14 +01:00
Bryan Bende 179e967b47 NIFI-4920 Skipping sensitive properties when updating component properties from versioned component. This closes #2505. 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-03-02 11:15:39 -05:00
Mark Payne 74bb341abc NIFI-2630: Changed name of queue in unit test to be unique in order to avoid getting messages from another test if the other tests fails to properly shutdown the connection. This closes #2458. 2018-03-02 09:37:00 -05:00
Mike Moser 42e6fa42a3 NIFI-2630 Allow PublishJMS to send TextMessages 
- Added configurable character set encoding for JMS TextMessages
- Improved PublishJMS/ConsumeJMS documentation
- Validate character set in property validator instead of OnScheduled
 2018-03-02 09:11:58 -05:00
Pierre Villard c58b025186 NIFI-4916 - ConvertExcelToCSVProcessor inherit parent attributes. This closes #2500. 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-03-02 08:47:32 -05:00
Mark Payne b4a4cc564e Fixed failing unit tests: Changed the queues used to unique names so that one test won't interfere with another; also changed JMSPublisherConsumerTest to JMSPublisherConsumerIT since it is an integration test between the publisher and consumer with ActiveMQ as the broker 2018-03-02 08:44:25 -05:00
Derek Straka 44bc2d41d7 NIFI-4880: Add the ability to map record based on the aliases. This closes #2474 
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-02-28 14:19:53 -05:00
James Wing 77dc186097 NIFI-4876 - Adding Min Object Age to ListS3 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2491.
 2018-02-28 12:01:35 +01:00
Matt Gilman 19b5b60b0f NIFI-3502: 
- Ensuring D3 is available on the summary and users page.

This closes #2495

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
 2018-02-27 08:54:58 -05:00
Mike Thomsen 3d9c470be3 NIFI-4827 Added support for reading queries from the flowfile body to GetMongo. 
NIFI-4827 Added changes from code review.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2443
 2018-02-26 14:19:35 -05:00
mans2singh f7fe2da106 NIFI-4289 - InfluxDB Put processor 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2101.
 2018-02-24 14:46:35 +01:00
Matthew Burgess 62732cbb88 NIFI-4903 - Fixed timestamp bug with fetch processors using Oracle 12+ 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2488.
 2018-02-22 20:56:30 +01:00
Matt Gilman d75f118ae1 NIFI-4894: 
- Ensuring that any proxy paths are retained when querying for bulletins.

This closes #2482

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
 2018-02-21 13:09:59 -05:00
Pierre Villard ff95be2a11 NIFI-4815 - Add EL support to ExecuteProcess 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2432
 2018-02-21 12:47:33 -05:00
Pierre Villard bc37015693 NIFI-4814 - Add distinctive attribute to S2S reporting tasks 
NIFI-4814 - added UUID attribute

NIFI-4814 - added type attribute

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2431
 2018-02-21 11:14:38 -05:00
Matthew Burgess b4b970b876 NIFI-4816: Allow name to be updated for ReportingTasks 
This closes #2452

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
 2018-02-21 09:42:25 -05:00
Mike Thomsen 54b1659704 NIFI-4429 Added GetMongoAggregation to support running Mongo aggregations. 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2180
 2018-02-21 08:58:38 -05:00
Matt Gilman 160bea8bf3
NIFI-4538: 
- Fixing broken unit test.
 2018-02-20 17:14:49 -05:00
Matt Gilman ba9e72ed8b NIFI-3502: 
- Preventing digest() during an ongoing apply() or digest() lifecycle.

This closes #2471

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
 2018-02-20 16:25:12 -05:00
Matt Gilman bf72d05825 NIFI-3502: 
- Ensuring that upon deselection, the monitor palette is updated accordingly.
 2018-02-20 14:51:44 -05:00
Matt Gilman 9cd631f963 NIFI-3502: 
- Upgrading to D3 version 4.
 2018-02-20 14:51:44 -05:00
yuri1969 91e98aa50b
NIFI-4538 - Add Process Group information to... 
...Search results

* Separated the search functionality.
* Added a unit test.
* Added the PG info to UI (a mere draft).
* Introduce the nearest versioned group
* Removed the top level group results in favour of the nearest versioned group.
* This closes #2364
 2018-02-20 12:42:09 -05:00
Mark Payne 9f95a10df9 NIFI-4794: Updated event writers to avoid creating a lot of byte[] by reusing buffers. Also removed synchronization on EventWriter when rolling over the writer and just moved the writing of the header to happen before making the writer available to any other threads. This reduces thread contention during rollover. 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2437
 2018-02-19 09:31:11 -05:00
Mark Payne 0bcb241db3 NIFI-4774: Implementation of SequentialAccessWriteAheadLog and updates to WriteAheadFlowFileRepository to make use of the new implementation instead of MinimalLockingWriteAheadLog. 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2416
 2018-02-19 09:26:01 -05:00
Mark Payne 14fef2de14 NIFI-4772: Refactored how the @OnScheduled methods of processors is invoked/monitored. The new method does away with the two previously created 8-thread thread pools and just uses the Timer-Driven thread pool that is used by other framework tasks. 
NIFI-4772: Introduced a new thread-pool with 2 threads that will be used for monitoring lifecycle task. This means that if all threads in the timer-driven thead pool are blocked by processors that don't complete their @OnScheduled methods, we have a separate thread pool that at least gives us a chance of interrupting those threads

NIFI-4772: Remove unused import
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2403
 2018-02-19 09:19:53 -05:00
Matthew Burgess 2a5e21c11b NIFI-4836 - Allow output of FlowFiles during result set processing in QueryDatabaseTable 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2447.
 2018-02-17 15:56:05 +01:00
zenfenaan 82e36f3c71 NIFI-4819: Added DeleteAzureBlobStorage that handles deletion of blob from an Azure Storage container 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2436.
 2018-02-17 15:49:01 +01:00
Bryan Bende b855d0acae
NIFI-4884 Fixing ordering during import from registry so that we set schedule strategy on a processor before setting scheduling duration. This closes #2472 2018-02-16 09:27:36 -05:00
Mike Thomsen 143d7e6829 NIFI-3538 Added DeleteHBaseRow 
This closes #2294.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2018-02-14 10:04:25 +09:00
Andy LoPresto 28067a29fd
NIFI-4869 Added SAX utility method for SplitXML processor. 
Added unit tests.
Added test resources to RAT exclude list.
This closes #2466
 2018-02-13 15:29:29 -05:00
Matthew Burgess b5ca7adbb9 NIFI-978: Support parameterized statements in ExecuteSQL 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2433.
 2018-02-12 20:44:31 +01:00
Koji Kawamura b4a9f52a4e
NIFI-4866: Fixed HBase performance issue 
HBase_1_1_2_ClientService performance dropped due to accessing HBase
admin upon every FlowFile transfer. Fixed that by getting master address
only once when the service is enabled and a connection is established.

This closes #2464
 2018-02-11 16:14:52 -05:00
Mark Payne 61c6f0305b
NIFI-4863: Bug fixes to the way that we handled Remote Group Ports when changing flow version / reverting local changes 
- Everywhere that we ignore adding remote ports we should ignore removing remote ports as well in flow diffs

This closes #2462.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2018-02-09 15:45:54 -05:00
Bryan Bende 34b678d30d NIFI-4841 Fixing NPE when reverting local changes involving remote group ports. This closes #2454. 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-02-09 10:26:19 -05:00
Matthew Burgess 25e0bbb68d NIFI-4853 - Fixed PutMongoRecord handling of nested records 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2457.
 2018-02-09 15:32:38 +01:00
Matthew Burgess 7d6bbce123 NIFI-4830: Fixed logic errors in BLOB/CLOB processing in JdbcCommon 
This closes #2459.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2018-02-09 11:45:12 +09:00
Mark Payne 4428fe28bf
NIFI-4858: Expose Request Timeout from HTTP Context Map and use that to set as the timeout of the 'AsyncContext' in HandleHttpRequest. Otherwise, the request will never timeout (which is OK because the HttpContextMap will handle this). However, Jetty behind the scenes is adding a task to Scheduled Executor for each request with a delay of whatever the timeout is set to. Since it's currently set to Long.MAX_VALUE, that task will never be run and as a result the ExecutorService's queue will grow indefinitely, eventually exhausting the JVM Heap 
This closes #2460.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2018-02-08 17:38:13 -05:00
Mike Moser a2f2ddd6b8 NIFI-4699 Use a filter in PostHTTP to pull flowfiles from queue whose URL is the same 
This closes #2412.
 2018-02-08 18:36:11 +00:00
Matthew Burgess 16e56ccfca NIFI-4080: Restored customValidate in ValidateCSV for schema property with no EL 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2226.
 2018-02-08 11:56:56 +01:00
Marco Gaido ba192d22da enable EL on FF level 2018-02-08 11:56:49 +01:00
Matthew Burgess 836c324258 NIFI-4080: Added EL support to fields in ValidateCSV 2018-02-08 11:56:42 +01:00
Koji Kawamura 270892f926 NIFI-4846: AvroTypeUtil to support more input types for logical decimal conversion 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2451
 2018-02-07 19:42:26 -05:00
Matt Gilman f3013d0764
NIFI-4837: 
- When Jetty initializes fails, performing a shutdown sequence to ensure all allocated resources are released.

This closes #2455.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2018-02-07 16:40:23 -05:00
Koji Kawamura e9af6c6ad8 NIFI-4828: Fix MergeContent to process all ready bins 
Before this fix, MergeContent only processed the first bin even if there
were multiple bins.

There were two unit tests marked with Ignore those had been
failing because of this.

This closes #2444.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-02-07 15:45:39 -05:00
Mark Payne d7da04209a Fix for unit tests that are causing build failures in certain environments 2018-02-07 15:08:52 -05:00
Koji Kawamura 2b062e211f NIFI-4844: Adjust BigDecimal scale to the target Avro schema 
- Applied the same scale adjustment not only to BigDecimal inputs, but
also to Double values.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2450
 2018-02-07 10:38:45 -05:00
Koji Kawamura dbbf78f22c
NIFI-4848: Update HttpComponents version 
- httpclient 4.5.3 -> 4.5.5
- httpcore 4.4.4 -> 4.4.9
  - ThreadSafe annotation is removed since 4.4.5, HTTPCLIENT-1743.
    Removed the annotation from DebugFlow processor.
- httpasyncclient 4.1.2 -> 4.1.3
- This closes #2453
 2018-02-07 08:55:43 -05:00
Koji Kawamura f16cbd462b
NIFI-4818: Fix transit URL parsing at Hive2JDBC and KafkaTopic for ReportLineageToAtlas 
- Hive2JDBC: Handle connection parameters and multiple host entries
correctly
- KafkaTopic: Handle multiple host entries correctly
- Avoid potential "IllegalStateException: Duplicate key" exception
when NiFiAtlasHook analyzes existing NiFiFlowPath input/output entries
- This closes #2435
 2018-02-07 08:49:35 -05:00
Mark Payne 3ca7c3e7a1
NIFI-4834: Updated AbstractJMSProcessor to use a separate SingleConnectionFactory per concurrent task instead of sharing one across the entire processor. 
This closes #2445.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
 2018-02-02 16:10:07 -05:00
Deon Huang b5938062a8 NIFI-4395 GenerateTableFetch can't fetch column type by state after instance reboot 
Add multiple states recover

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2442
 2018-02-02 11:15:46 -05:00