Commit Graph

51 Commits

Author SHA1 Message Date
Mark Payne 451f9cf124 NIFI-4142: This closes #2015. Refactored Record Reader/Writer to allow for reading/writing "raw records". Implemented ValidateRecord. Updated Record Reader to take two parameters for nextRecord: (boolean coerceTypes) and (boolean dropUnknownFields)
Signed-off-by: joewitt <joewitt@apache.org>
2017-08-11 22:01:46 -07:00
Mark Payne e7dcb6f6c5 NIFI-3921: Allow Record Writers to inherit schema from Record
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1902
2017-06-09 16:13:25 -04:00
Matt Gilman cc741d2be6
NIFI-3997:
- Bumping to next minor version.
2017-06-08 15:22:51 -04:00
Matt Gilman 6ee12e9b47
NIFI-3997-RC1prepare for next development iteration 2017-06-05 11:07:43 -04:00
Matt Gilman ddb73612bd
NIFI-3997-RC1prepare release nifi-1.3.0-RC1 2017-06-05 11:07:28 -04:00
Mark Payne 7f8987471d
NIFI-3946: Update LookupService to take a Map instead of a String for the input
NIFI-3946: Fixed issues where null values were returned instead of empty optionals

This closes #1833.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-22 11:52:41 -04:00
Mark Payne c49933f03d NIFI-3948: This closes #1834. Added flush() method to RecordWriter and call it when writing a single record to OutputStream for PublishKafkaRecord. Also removed no-longer-used class WriteAvroResult
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-19 23:05:04 -04:00
Matt Burgess 9294a26139
NIFI-3938: Added ScriptedLookupService, some refactor for reusable scripting classes
This closes #1828.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-19 11:20:38 -07:00
Mark Payne ae9953db64 NIFI-3857: This closes #1825. Added PartitionRecord processor
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-19 02:08:52 -04:00
Mark Payne 9bd0246a96 NIFI-3863: Initial implementation of Lookup Services. Implemented LookupRecord processors. This required some refactoring of RecordSetWriter interface, so refactored that interface and all implementations and references of it 2017-05-19 01:02:41 -04:00
joewitt 3b98abbf41
NIFI-3940 upgraded to latest jruby which resolves an apparent lgpl bunding issue
This closes #1827.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-18 21:37:25 -07:00
Mark Payne b1901d5fe0 NIFI-3838: Initial implementation of RecordPath and UpdateRecord processor
NIFI-3838: Updated version from 1.2.0-SNAPSHOT to 1.3.0-SNAPSHOT; removed unneeded value from AttributeExpression.ResultType enum

NIFI-3838: Addressed PR Review feedback

NIFI-3838: Allow for schemas to be merged together for a record; refactored RecordSetWriterFactory so that there is a method to obtain the schema and then the writer is created with that schema. Added additional unit tests

NIFI-3838: Addressed problems with documentation based on PR Review

NIFI-3838: Fixed checkstyle violation

NIFI-3838: Addressed issue of comparing different types of Number objects

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1772
2017-05-12 12:36:52 -04:00
Bryan Bende 3af53419af
NIFI-3770-RC2 prepare for next development iteration 2017-05-05 20:50:28 -04:00
Bryan Bende 3a605af8e0
NIFI-3770-RC2 prepare release nifi-1.2.0-RC2 2017-05-05 20:50:14 -04:00
Bryan Bende 60d88b5a64
NIFI-3724 - Initial commit of Parquet bundle with PutParquet and FetchParquet
- Creating nifi-records-utils to share utility code from record services
- Refactoring Parquet tests to use MockRecorderParser and MockRecordWriter
- Refactoring AbstractPutHDFSRecord to use schema access strategy
- Adding custom validate to AbstractPutHDFSRecord and adding handling of UNION types when writing Records as Avro
- Refactoring project structure to get CS API references out of nifi-commons, introducing nifi-extension-utils under nifi-nar-bundles
- Updating abstract put/fetch processors to obtain the WriteResult and update flow file attributes

This closes #1712.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-01 16:10:35 -04:00
Matt Burgess 49a62448ce
NIFI-3734: Add ScriptedReader and ScriptedRecordSetWriter
This closes #1691.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-04-25 10:57:21 -07:00
Andre F de Miranda 84f1fb3959 NIFI-3222 - Allow ExecuteScript,InvokeScriptedProcessor and ScriptedReporting to use ExpressionLanguage to define the scripting engine modules path
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1640
2017-03-31 23:41:02 -04:00
Matt Burgess 7963df89eb NIFI-3288: Add Clojure support for ExecuteScript
This closes: #1402.

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-02-15 02:52:11 +11:00
Matt Burgess 675f4f544c
NIFI-1458: Added ScriptedReportingTask
This closes #1045.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-01-06 11:56:19 -08:00
joewitt 92f17a995b NIFI-3100-rc2 prepare for next development iteration 2016-11-25 23:49:27 -05:00
joewitt 5536f690a8 NIFI-3100-rc2 prepare release nifi-1.1.0-RC2 2016-11-25 23:49:13 -05:00
Koji Kawamura 36796b5440 NIFI-3078: Test InvokeScriptedProcessor with JS.
This closes #1255.
2016-11-22 21:42:36 +01:00
joewitt fb9cbccc38 NIFI-2954 This closes #1244. Moved StandardPropertyValidator to nifi-utils, documented scope/purpose of a few util libs, removed deps from nifi-utils. 2016-11-21 16:30:42 -05:00
Matt Gilman 7f5eabd603
NIFI-3050: Implemented access control logic for restricted components.
- Addressing comments from PR.
- Adding restricted tags to relevant components.
- Showing a restricted icon overlay on the processor node on the canvas. (+1 squashed commit)
Squashed commits:
[f487682] NIFI-3050:
- Introducing a Restricted annotation for components that require elevated privileges to use.
- Updating the new Processor, Controller Service, and Reporting Task dialogs to include these details and prevent unauthorized selection.
- Including the Restricted description in the generated component documentation.
- Updating processor access control integration test to verify restricted component creation.
- Updating the developer, user, and admin guide to include the restricted component policy.

This closes #1247.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-21 12:07:48 -08:00
Peter Mitchell 893fed794c Add Stateful annotation to scripting processors
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-2682: Added text to Stateful description for AbstractScriptProcessor
2016-10-12 13:20:37 -04:00
Matt Burgess 66fe004058 NIFI-2815: Fixed InvokeScriptedProcessor to allow changes to script engine type
This closes #1055.
2016-09-23 19:30:51 +02:00
Joseph Percivall 1fe18a1567 NIFI-2676-rc1 prepare for next development iteration 2016-08-26 11:40:58 -04:00
Joseph Percivall 74d5224783 NIFI-2676-rc1 prepare release nifi-1.0.0-RC1 2016-08-26 11:40:44 -04:00
Matt Burgess a84d3c9873 NIFI-2665: This closes #942. Fixed intermittent validation errors in InvokeScriptedProcessor 2016-08-25 15:34:50 -04:00
joewitt 7d7401add4 NIFI-2574 Changed NiFiProperties to avoid static initializer and updated all references to it. 2016-08-17 00:10:07 -07:00
joewitt 05a99a93cb NIFI-2208 This closes #754. refactored as per comments on JIRA. Reduced API expsosure and tightened lifecycle management. 2016-08-01 14:17:26 -04:00
Yolanda M. Davis 8412d2662a NIFI-2208 - initial commit Custom Property Expression Language support with Variable Registry, includes bug fix for NIFI-2057
This closes #529

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-29 17:10:20 -04:00
Koji Kawamura 53326c7f9b NIFI-2267: A way for Processor to know node type
- Added NodeTypeProvider to expose flowController's isClustered and
  isPrimaryNode so that processor can know if it's running on a cluster
  and if it's a primary node.
- Added mechanism to simulate clustered or not, and primary or not, for testing

This closes #657

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-15 11:38:50 -04:00
joewitt cfaacb1d5c NIFI-1152 NIFI-2117 Fixed standard session impl api adherance, mock session api adherance, corrected code and tests for script processors that had issues due to that process session bug 2016-07-13 08:45:57 -04:00
Matt Gilman ce5330330a NIFI-1781:
- Updating UI according to permissions through out the application.
- Shuffling provenance events, template, and cluster search REST APIs according to resources being authorized.
- Moving template upload controls.
- Removing username where appropriate.
- Addressing issues when authorizing flow configuration actions.
- Code clean up.
2016-07-01 15:10:27 -04:00
Andy LoPresto ae3012a992 NIFI-1918 Ignored failing test because timing is not deterministic and should not be used in unit test.
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #465
2016-05-24 22:43:17 -04:00
Aldrin Piri 1bd2cf0d09 NIFI-1811 Renaming MockProcessorLogger to MockComponentLogger for consistency. Removing unused imports from ExecuteScript causing checkstyle failures. 2016-05-19 14:38:41 -04:00
Pierre Villard 372ffb8aa0 NIFI-1811 Removed ProcessorLog and updated dependent interfaces
This closes #403.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-05-19 13:08:09 -04:00
Matt Burgess c13fb3159f
NIFI-1822 Enabled pooled ExecuteScript engines to facilitate concurrent processing.
Removed unused variable in unit test. (+10 squashed commits)
Squashed commits:
[7c5acc1] NIFI-1822 Removed trailing whitespace to conform with checkstyle rules.
[cb108cd] NIFI-1822 Added ASF License to unit test.
[9264428] NIFI-1822 Removed debugging log statements for script engine queue size.
Added unit tests demonstrating pooled execution timing and thread usage.
[bdbc4ba] NIFI-1822 Renamed reference to MockProcessorContext#setNumThreads to setMaxConcurrentTasks after refactor.
[12bbe82] NIFI-1822 Moved unit test to correct directory.
Added test script resource which generates flowfile and updates attribute with current thread.
Added tests for single run, serial run, and pooled run (not complete).
[4c174c8] NIFI-1822 Added debugging messages to script execution.
[8ab0ce5] NIFI-1822 Added for loop to instantiate multiple script engines in queue.
[8c5ba51] NIFI-1822 Added variable max concurrent task field in MockProcessorContext because it was previously hardcoded to 1. Changed setNumThreads to setMaxConcurrentTasks to maintain naming convention.
[fd9120c] NIFI-1822 Added unit test skeleton for pooled script processor execution.
[23e4f68] NIFI-1822: Allow concurrent execution in ExecuteScript

This closes #443.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-05-17 14:58:00 -07:00
Matt Gilman 2de7f3f884 Updating versions to 1.0.0-SNAPSHOT. 2016-04-04 11:36:20 -04:00
Aldrin Piri e977729b56 NIFI-1634-rc2 prepare for next development iteration 2016-03-23 18:56:34 -04:00
Aldrin Piri 0b9bd20d31 NIFI-1634-rc2 prepare release nifi-0.6.0-RC2 2016-03-23 18:56:22 -04:00
Aldrin Piri 58e50ae39d NIFI-1511 Incorporating Groovy unit tests as part of the main build and providing explicit compile scope on the Groovy dependency for the scripting bundle. This closes #220
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-02-26 10:25:45 -05:00
Tony Kurc c7e24c7569 NIFI-1513: fixed some easy to fix errors
Addressing checkstyle issue.

This closes #221

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-02-25 15:21:40 -05:00
Tony Kurc ad73b5c9d9 NIFI-1379-RC3 prepare for next development iteration 2016-02-12 17:28:10 -05:00
Tony Kurc 8309dba80b NIFI-1379-RC3 prepare release nifi-0.5.0-RC3 2016-02-12 17:27:59 -05:00
Oleg Zhurakousky 8a05f6880a NIFI-1493 This closes #211. removed Executor from InvokeScriptedProcessor
Signed-off-by: joewitt <joewitt@apache.org>
2016-02-09 20:31:57 -05:00
Tony Kurc 303f8eabf1 NIFI-1379: Move to 0.5.0-SNAPSHOT, add tkurc code signing key to keys 2016-02-06 08:49:48 -05:00
Aldrin Piri 4df6512126 Merge branch 'NIFI-259' 2016-02-05 14:09:29 -05:00
Matt Burgess 478226451c NIFI-210: Add support for multiple module paths for scripting 2016-02-03 10:41:47 -05:00