Commit Graph

139 Commits

Author SHA1 Message Date
Mark Payne a88d3bfa3c NIFI-1280: Refactoring to make more generic so that other data types can be supported; created InputStreams to content on-demand so that multiple passes can be made over FlowFile content if required. Created new Controller Services for reading and writing specific data types
Signed-off-by: Matt Burgess <mattyb149@apache.org>
2017-04-11 19:29:04 -04:00
joewitt 52cf9a7953 NIFI-3690 Updated nifi-gcp-nar L&N, cat-x deps
Updated L&N in nifi-gcp-nar and nifi-assembly and fixed nifi-gcp-processors
to avoid jsr305 and json.org cat-x dependencies

Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1665.
2017-04-11 15:56:25 -07:00
Joey Frazee 4a8398bb31 NIFI-3692 This closes #1663. Use hbc-core instead of hbc-twitter4j in social-media bundle to resolve org.json cat-x
Signed-off-by: joewitt <joewitt@apache.org>
2017-04-11 14:25:04 -04:00
Joe Percivall 5419891fe7 NIFI-3633 This closes #1610. Adding HttpNotificationService.
Signed-off-by: joewitt <joewitt@apache.org>
2017-04-11 00:37:12 -04:00
Matt Burgess 8f37ad4512 NIFI-3413: Add GetChangeDataCaptureMySQL processor
NIFI-3413: Incorporated review comments
NIFI-3413: Changed GetChangeDataCaptureMySQL to CaptureChangeMySQL, fixed some bugs
NIFI-3413: Refactored setup() for better error handling, more review comments incorporated
NIFI-3413: Refactored CDC into its own module(s), updated assembly and top-level POMs
NIFI-3413: Added RECEIVE prov event and Server ID property

Signed-off-by: ijokarumawak <ijokarumawak@apache.org>
2017-04-07 00:44:42 +09:00
Puspendu Banerjee 3386839ebc NIFI-3586: Fix for retrieving ProcessID for NiFi under windows.
This closes #1586

Signed-off-by: Tony Kurc <tkurc@apache.org>
2017-04-05 23:09:48 -04:00
kedarchitale 457df93dd6 NIFI-3147 Build processor to parse CCDA into attributes
This closes #1312

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2017-03-29 20:30:57 +00:00
Bryan Bende d90cf846b9 NIFI-3380 Bumping NAR plugin to 1.2.0-SNAPSHOT development to leverage changes from master, adding buildnumber-maven-plugin to nifi-nar-bundles to properly set build info in MANIFEST of NARs
- Refactoring NarDetails to include all info from MANIFEST
- Adding the concept of a Bundle and refactoring NarClassLoaders to pass Bundles to ExtensionManager
- Adding logic to fail start-up when multiple NARs with same coordinates exist, moving Bundle classes to framework API
- Refactoring bundle API to classes and creating BundleCoordinate
- Updating FlowController to use BundleCoordinate

- Updating the UI and DTO model to support showing bundle details that loaded an extension type.
- Adding bundle details for processor canvas node, processor dialogs, controller service dialogs, and reporting task dialogs.
- Updating the formating of the bundle coordinates.
- Addressing text overflow in the configuration/details dialog.
- Fixing self referencing functions.
- Updating extension UI mapping to incorporate bundle coordinates.
- Discovering custom UIs through the supplied bundles.
- Adding verification methods for creating extensions through the rest api.
- Only returning extensions that are common amongst all nodes.
- Rendering the ghost processors using a dotted border.
- Adding bundle details to the flow.xml.
- Loading NiFi build and version details from the framework NAR.
- Removing properties for build and version details.
- Wiring together front end and back end changes.
- Including bundle coordinates in the component data model.
- Wiring together component data model and flow.xml.
- Addressing issue when resolve unvesioned dependent NARs.

Updating unit tests to pass based on framework changes
- Fixing logging of extension types during start up

- Allowing the application to start if there is a compatible bundle found. - Reporting missing bundle when the a compatible bundle is not found. - Fixing table height in new component dialogs.

Fixing chechstyle error and increasing test timeout for TestStandardControllerServiceProvider
- Adding ability to change processor type at runtime
- Adding backend code to change type for controller services

- Cleaning up instance classloaders for temp components.
- Creating a dialog for changing the version of a component.
- Updating the formatting of the component type and bundle throughout.
- Updating the new component dialogs to support selecting source group.
- Cleaning up new component dialogs.
- Cleaning up documentation in the cluster node endpoint.

Adding missing include in nifi-web-ui pom compressor plugin
- Refactoring so ConfigurableComponent provides getLogger() and so the nodes provide the ConfigurableComponent
- Creating LoggableComponent to pass around the component, logger, and coordinate with in the framework

- Finishing clean up following rebase.

Calling lifecycle methods for add and remove when changing versions of a component
- Introducing verifyCanUpdateBundle(coordinate) to ConfiguredComponent, and adding unit tests

- Ensuring documentation is available for all components. Including those of the same type that are loaded from different bundles.

Adding lookup from ClassLoader to Bundle, adding fix for instance class loading to include all parent NARs, and adding additional unit tests for FlowController
- Adding validation to ensure referenced controller services implement the required API
- Fixing template instantiation to look up compatible bundle

- Requiring services/reporting tasks to be disabled/stopped.
- Only supporting a change version option when the item has multiple versions available.
- Limiting the possible new controller services to the applicable API version.
- Showing the implemented API versions for Controller Services.
- Updating the property descriptor tooltip to indicate the required service requirements.
- Introducing version based sorting in the new component dialog, change version dialog, and new controller service dialog.
- Addressing remainder of the issues from recent rebase.

Ensuring bundles have been added to the flow before proposing a flow, and incorporating bundle information into flow fingerprinting
- Refactoring the way missing bundles work to retain the desired bundle if available
- Fixing logger.isDebugEnabled to be logger.isTraceEnabled

- Auditing when user changes the bundle. - Ensuring bundle details are present in templates.

Moving standard prioritizers to framework NAR and refactoring ExtensionManager logic to handle cases where an extension is in a JAR directly in the lib directory

- Ensuring all nodes attempt to instantiate the same template instance when the available bundles may differ. - Fixing the auditing of copy/paste and template instantiation. - Running addtional verification methods when running standalone.

Refactoring controller service invocation handler to allow updating the node used by the invocation handler
- Ensuring the bundles in a proposed flow are compatible with the current instance when the current instance has no flow is going to accept the proposed flow
- Merging whether multiple versions of the component are available
- Setting NAR plugin back to current released version
- Cleaning up DocGenerator to not process multiple times

Addressing incorrect usage of nf.Common. - Using formatType in the new component type dialogs.

Improving error messages when looking for bundles

Addressing comments from PR. - Fixing references to global nf namespace. - Fixing injection of nfProcessGroupConfiguration in nfComponentVersion. - Fixing web api integration tests.

Not rendering unversioned in help documentation. - Ensuring the isExtentionMissing flag is correct after changing the component type.

Adding synchronization in node classes to ensure changing component can't occur when component is running, introducing MissingBundleException for better reporting when a node can't join cluster due to a missing bundle, and bumping NAR plugin to released version 1.2.0

Adding concept of missing components to fingerprinting to ensure nodes agree on missing components when joining a cluster

NIFI-3380: NIFI-3520: - Fixing hive nar dependency. - Marking DBCPService as provided. - Skipping services that require instance classloading and are cobundled with their service API. - Skipping components that require instance classloading and reference service APIs that are cobundled. - Addressing UI issues in the new component dialogs when re-opening with a filter applied.

Fixing checkstyles issue and adding back assume checks to distributed cache server test

Ensuring new component types are sorted correctly when shown initially.

This closes #1585.
2017-03-24 11:06:44 -04:00
Scott Aslan a90a770244
[NIFI-3035] add deep linking URL to canvas components and PGs. This closes #1559 2017-03-22 14:23:02 -04:00
Jeremy Dyer d05727b8c0 NIFI-2613 Apache POI processor for Excel to CSV
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #929.
2017-03-17 10:39:05 -07:00
joewitt 443803e729 NIFI-2564 corrected L&N to reference new BeanUtils dependency. 2017-03-09 22:00:41 -05:00
jpercivall b7f946e847 NIFI-1682 Adding AttributeRollingWindow processor
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1328.
2017-03-06 19:42:25 +01:00
Andre F de Miranda a32e1509b6 NIFI-3238 - Introduce ListenBeats processor and deprecates ListenLumberjack processor
NIFI-3238 - Rewords processor description

NIFI-3238: Removed unused imports

This closes #1418
2017-03-01 21:41:51 -05:00
smarthi dfd04e638b
[NIFI-3507]-Update NiFi-assembly Java Requirement to 1.8
This closes #1531.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-02-23 09:30:47 -05:00
Scott Aslan 364efa96cc
[NIFI-3501] Upgrade D3 to 3.5.17 and allow npm to pull it down at build time. This closes #1519 2017-02-21 10:17:34 -05:00
Oleg Zhurakousky 6a1854c975 NIFI-3354 Added support for simple AVRO/CSV/JSON transformers that utilize external Schema
Added support for simple Key/Value Schema Registry as Controller Service
Added support for registering multiple schemas as dynamic properties of Schema Registry  Controller Service
Added the following 8 processors
- ExtractAvroFieldsViaSchemaRegistry
- TransformAvroToCSVViaSchemaRegistry
- TransformAvroToJsonViaSchemaRegistry
- TransformCSVToAvroViaSchemaRegistry
- TransformCSVToJsonViaSchemaRegistry
- TransformJsonToAvroViaSchemaRegistry
- TransformJsonToCSVViaSchemaRegistry
- UpdateAttributeWithSchemaViaSchemaRegistry

polishing

NIFI-3354 Adding support for HDFS Schema Registry, unions and default values in the Avro Schema and NULL columns in the source CSV

NIFI-3354 Adding support for logicalTypes per the Avro 1.7.7 spec

NIFI-3354 polishing and restructuring CSVUtils

NIFI-3354 renamed processors to address PR comment

NIFI-3354 addressed latest PR comments
- removed HDFS-based ControllerService. It will be migrated into a separate bundle as a true extension.
- removed UpdateAttribute. . . processor
- added mime.type attribute to all Transform* processors

NIFI-3354 added missing L&N entries

This closes pr/1436
2017-02-17 14:32:06 -05:00
Aldrin Piri 76fcf5def1 NIFI-3385 - This closes #1440. Updating copyright statements to reflect the new year. 2017-02-16 00:10:02 -05:00
patricker b7cdc6b382 NIFI-2779 - Add processor to GetEmail Supporting Exchange Web Services
This closes: #1326

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-02-15 23:48:34 +11:00
Matt Burgess 7963df89eb NIFI-3288: Add Clojure support for ExecuteScript
This closes: #1402.

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-02-15 02:52:11 +11:00
Selim Namsi 2ef7c15b5d NIFI-2565: add Grok parser
This closes #1108.

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-02-14 21:11:03 +11:00
Gene Peters 897c70298a NIFI-3449: Adding Google Cloud Storage Bundle
* Credentials service with tests
* Abstract processor definitions
* GCS-themed processors and their corresponding tests

Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1482.
2017-02-12 19:52:38 -08:00
Andre F de Miranda 68c2d7862f NIFI-3434 - Introduce the ability to skip the packaging of zip and tar.gz archive
via maven profile (i.e. -Pdir-only)
2017-02-07 11:42:03 +01:00
Andre F de Miranda 76e31dfd19 NIFI-3015 - This closes #1422. Corrects issue were RPM profile run.as property was not being used
This supersedes #1198
2017-02-06 09:36:52 -05:00
Oleg Zhurakousky f0856565ad NIFI-2615 polishing
- added missing POM entries to nifi POM and assembly POM
- added available port-discovery to tests amongs other minor polishings
2017-01-25 13:22:50 -05:00
Matt Gilman 6170f6442e NIFI-3291:
- Upgrading jQuery and it's plugins.
- Packaging at build time using npm.
2017-01-17 14:54:59 -05:00
joewitt 92f17a995b NIFI-3100-rc2 prepare for next development iteration 2016-11-25 23:49:27 -05:00
joewitt 5536f690a8 NIFI-3100-rc2 prepare release nifi-1.1.0-RC2 2016-11-25 23:49:13 -05:00
joewitt 22143e386d NIFI-2991 This closes #1265. was allowed to restore twitter4j json.org usage until April. Resolution for that will be in NIFI-3089 2016-11-23 10:21:41 -05:00
joewitt fb9cbccc38 NIFI-2954 This closes #1244. Moved StandardPropertyValidator to nifi-utils, documented scope/purpose of a few util libs, removed deps from nifi-utils. 2016-11-21 16:30:42 -05:00
Matt Burgess dba9d5924a NIFI-3011: Added Elasticsearch5 processors
This closes #1233

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-21 15:58:40 -05:00
Koji Kawamura 26a5881d21 NIFI-1002: Added WebSocket support.
NIFI-1002: Added WebSocket support.

- Reflecting review comments
- Added displayName to peroperty descriptors

NIFI-1002: Added WebSocket support.

This closes #1184

- Reflecting review comments:
  - Removed unnecessary use of NarCloseable.withComponentNarLoader.
  - Removed intermediate on memory queue to make it simpler and more
    robust. Received messages in WebSocket layer now will be transferred
to downstream relationships directly.
2016-11-15 19:34:59 -05:00
joewitt b026f0bebe NIFI-2991 removed json.org references as appropriate for build. Twitter proc no longer deployed but reference kept in notice just in case someone uses it
This closes #1230

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-15 18:09:27 -05:00
joewitt 1c9077334f NIFI-2991 removed usage of json lib and components still using it
Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-15 18:09:09 -05:00
Bryan Bende fe59b3415c NIFI-3003 Upgrading hadoop.version to 2.7.3 and fixing TDE issue with PutHDFS, ensuring clean up of instance class loaders, and adding classpath resource property to all HDFS processors
NIFI-3003 Addressing review feedback

NIFI-3003 added minor notice updates

This closes #1219
2016-11-15 16:23:27 -05:00
joewitt 8ad3125168 NIFI-2380 This closes #817. corrected licensing and notice and remove conflicting mail deps 2016-11-14 21:04:33 -05:00
Andre F de Miranda a347ebf87b NIFI-2380 - Introduce ExtractTNEFAttachment - A processor to extract
attachments embedded within a Transport Neutral Encapsulation Format (TNEF) attachment (i.e. winmail.dat)
2016-11-14 21:04:12 -05:00
joewitt 3e892c55ec NIFI-2971 This closes #1172. addressed RPM bundled deps and handling exception for bootstrap cl construction 2016-11-09 11:08:43 -05:00
James Wing a486fefb1e NIFI-2115 Detailed Version Info in About Box
* Java version and vendor
* OS name and version
* Release Tag
* Build revision (commit SHA), branch, and timestamp
* Handles formal releases, ad-hoc builds, and non-release source builds
* Standalone UI presence in About dialog, Summary -> System Diagnostics
* Cluster UI as Versions tab in Cluster dialog
* Reduce About Dialog Content
* Fix Missing Property Display Bugs
* Marking the build time as type string.
* This closes #583
2016-11-03 11:16:20 -04:00
Artem Ermakov 511f2a0c8b [NIFI-2812] Fix creating NiFi RPM
This closes: PR 1047
This resolves: NIFI-2812

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2016-11-03 02:26:20 +11:00
Andre F de Miranda b864d49f66 NIFI-2341 - Introduce ParseCEF processor
This closes #785
2016-10-20 17:16:57 -04:00
Joseph Percivall 1fe18a1567 NIFI-2676-rc1 prepare for next development iteration 2016-08-26 11:40:58 -04:00
Joseph Percivall 74d5224783 NIFI-2676-rc1 prepare release nifi-1.0.0-RC1 2016-08-26 11:40:44 -04:00
jpercivall 6ef1cca18f NIFI-2269 Fixing Datadog licensing 2016-08-25 22:43:00 -04:00
jpercivall e1da9fc11f NIFI-2667 Adding License and Notice information to Media Bundle Nar and Assembly
This closes #950
2016-08-25 22:42:37 -04:00
joewitt 1624dd8e4d NIFI-2666 This closes #947. updated where it says BSD to say MIT for bcrypt 2016-08-25 14:59:46 -04:00
Matt Burgess 3224b1a5dd NIFI-2668: This closes #945. Replaced JSR-107 JCache API JAR w Geronimo version, updated notices 2016-08-25 14:41:08 -04:00
joewitt 1745c1274b NIFI-2608 This closes #930. restructured the ConsumeKafka processor to follow new consumer API. Made nar classloading more precise to support spawned threads NIFI-2660. 2016-08-25 10:09:50 -04:00
Andy LoPresto c638191a47
NIFI-1831 Added internal logic and command-line tool to allow AES-encrypted sensitive configuration values in nifi.properties.
This closes #834.
2016-08-23 20:34:31 -07:00
jpercivall 6e82ec738c NIFI-2269 Fixing minor issues with style check, licensing, etc. 2016-08-22 21:34:51 -04:00
Ramiz 376d3c4ef4 NIFI-2269 Added datadog support
This closes #655

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-22 21:34:51 -04:00