Commit Graph

460 Commits

Author SHA1 Message Date
Pierre Villard ced6708d4b NIFI-3596 - added attributes to GenerateTableFetch processor
Signed-off-by: Matt Burgess <mattyb149@apache.org>

Updated test to check selected column names

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1596
2017-03-27 08:50:55 -04:00
Bryan Bende d90cf846b9 NIFI-3380 Bumping NAR plugin to 1.2.0-SNAPSHOT development to leverage changes from master, adding buildnumber-maven-plugin to nifi-nar-bundles to properly set build info in MANIFEST of NARs
- Refactoring NarDetails to include all info from MANIFEST
- Adding the concept of a Bundle and refactoring NarClassLoaders to pass Bundles to ExtensionManager
- Adding logic to fail start-up when multiple NARs with same coordinates exist, moving Bundle classes to framework API
- Refactoring bundle API to classes and creating BundleCoordinate
- Updating FlowController to use BundleCoordinate

- Updating the UI and DTO model to support showing bundle details that loaded an extension type.
- Adding bundle details for processor canvas node, processor dialogs, controller service dialogs, and reporting task dialogs.
- Updating the formating of the bundle coordinates.
- Addressing text overflow in the configuration/details dialog.
- Fixing self referencing functions.
- Updating extension UI mapping to incorporate bundle coordinates.
- Discovering custom UIs through the supplied bundles.
- Adding verification methods for creating extensions through the rest api.
- Only returning extensions that are common amongst all nodes.
- Rendering the ghost processors using a dotted border.
- Adding bundle details to the flow.xml.
- Loading NiFi build and version details from the framework NAR.
- Removing properties for build and version details.
- Wiring together front end and back end changes.
- Including bundle coordinates in the component data model.
- Wiring together component data model and flow.xml.
- Addressing issue when resolve unvesioned dependent NARs.

Updating unit tests to pass based on framework changes
- Fixing logging of extension types during start up

- Allowing the application to start if there is a compatible bundle found. - Reporting missing bundle when the a compatible bundle is not found. - Fixing table height in new component dialogs.

Fixing chechstyle error and increasing test timeout for TestStandardControllerServiceProvider
- Adding ability to change processor type at runtime
- Adding backend code to change type for controller services

- Cleaning up instance classloaders for temp components.
- Creating a dialog for changing the version of a component.
- Updating the formatting of the component type and bundle throughout.
- Updating the new component dialogs to support selecting source group.
- Cleaning up new component dialogs.
- Cleaning up documentation in the cluster node endpoint.

Adding missing include in nifi-web-ui pom compressor plugin
- Refactoring so ConfigurableComponent provides getLogger() and so the nodes provide the ConfigurableComponent
- Creating LoggableComponent to pass around the component, logger, and coordinate with in the framework

- Finishing clean up following rebase.

Calling lifecycle methods for add and remove when changing versions of a component
- Introducing verifyCanUpdateBundle(coordinate) to ConfiguredComponent, and adding unit tests

- Ensuring documentation is available for all components. Including those of the same type that are loaded from different bundles.

Adding lookup from ClassLoader to Bundle, adding fix for instance class loading to include all parent NARs, and adding additional unit tests for FlowController
- Adding validation to ensure referenced controller services implement the required API
- Fixing template instantiation to look up compatible bundle

- Requiring services/reporting tasks to be disabled/stopped.
- Only supporting a change version option when the item has multiple versions available.
- Limiting the possible new controller services to the applicable API version.
- Showing the implemented API versions for Controller Services.
- Updating the property descriptor tooltip to indicate the required service requirements.
- Introducing version based sorting in the new component dialog, change version dialog, and new controller service dialog.
- Addressing remainder of the issues from recent rebase.

Ensuring bundles have been added to the flow before proposing a flow, and incorporating bundle information into flow fingerprinting
- Refactoring the way missing bundles work to retain the desired bundle if available
- Fixing logger.isDebugEnabled to be logger.isTraceEnabled

- Auditing when user changes the bundle. - Ensuring bundle details are present in templates.

Moving standard prioritizers to framework NAR and refactoring ExtensionManager logic to handle cases where an extension is in a JAR directly in the lib directory

- Ensuring all nodes attempt to instantiate the same template instance when the available bundles may differ. - Fixing the auditing of copy/paste and template instantiation. - Running addtional verification methods when running standalone.

Refactoring controller service invocation handler to allow updating the node used by the invocation handler
- Ensuring the bundles in a proposed flow are compatible with the current instance when the current instance has no flow is going to accept the proposed flow
- Merging whether multiple versions of the component are available
- Setting NAR plugin back to current released version
- Cleaning up DocGenerator to not process multiple times

Addressing incorrect usage of nf.Common. - Using formatType in the new component type dialogs.

Improving error messages when looking for bundles

Addressing comments from PR. - Fixing references to global nf namespace. - Fixing injection of nfProcessGroupConfiguration in nfComponentVersion. - Fixing web api integration tests.

Not rendering unversioned in help documentation. - Ensuring the isExtentionMissing flag is correct after changing the component type.

Adding synchronization in node classes to ensure changing component can't occur when component is running, introducing MissingBundleException for better reporting when a node can't join cluster due to a missing bundle, and bumping NAR plugin to released version 1.2.0

Adding concept of missing components to fingerprinting to ensure nodes agree on missing components when joining a cluster

NIFI-3380: NIFI-3520: - Fixing hive nar dependency. - Marking DBCPService as provided. - Skipping services that require instance classloading and are cobundled with their service API. - Skipping components that require instance classloading and reference service APIs that are cobundled. - Addressing UI issues in the new component dialogs when re-opening with a filter applied.

Fixing checkstyles issue and adding back assume checks to distributed cache server test

Ensuring new component types are sorted correctly when shown initially.

This closes #1585.
2017-03-24 11:06:44 -04:00
Andre F de Miranda a298f1a707 NIFI-3466 - Address non English locales in ParseCEF
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1614.
2017-03-23 09:04:16 +01:00
Matt Burgess d1ebddce98 NIFI-3627: Added removeByPattern() to DistributedMapCache interfaces
NIFI-3627: Updated unit tests that use MapCache interface(s)

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1609.
2017-03-22 19:22:08 +01:00
Matt Burgess 37a1e6d07f NIFI-3585: Add DatabaseAdapter impl for MS SQL 2008
This closes #1584

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2017-03-22 14:57:42 +00:00
Matt Burgess c4d0c0bbd1
NIFI-3626: Add support for DELETE in ConvertJSONToSQL
This closes #1605.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-03-21 17:20:27 -04:00
Andy LoPresto b3b65219a0
NIFI-3361:
- Refactored TestGetHTTPGroovy for componentization & customization of tests.
- Ignored some TestGetHTTPGroovy tests because they cannot connect to a Jetty server running TLSv1 or TLSv1.1 (the new version of Jetty only supports TLSv1.2+). The tests are still valuable to demonstrate GetHTTP's capacity to connect to various TLS protocols, and should not be removed unless necessary.
- Refactored TestPostHTTPGroovy for componentization & customization of tests.
- Ignored some TestPostHTTPGroovy tests because they cannot connect to a Jetty server running TLSv1 or TLSv1.1 (the new version of Jetty only supports TLSv1.2+). The tests are still valuable to demonstrate PostHTTP's capacity to connect to various TLS protocols, and should not be removed unless necessary.
- Restored one of the TestGetHTTPGroovy unit tests (GetHTTP vs. TLSv1/1.1/1.2) by connecting to https://nifi.apache.org which still supports all three protocol versions (and uses JVM cacerts as truststore).
- This closes #1601
2017-03-21 14:48:40 -04:00
Matt Gilman 8e61aa3c43
NIFI-3361:
- Upgrading to the latest Jetty version.
2017-03-21 13:44:26 -04:00
Joe Percivall 5daeba6e03 NIFI-3614 Adding "return" statement to HandleHttpResponse when statusCode is not a number
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1598.
2017-03-20 14:51:56 +01:00
ambud 9424836412 NIFI-3525 Add tablename attribute to QDT flowfiles
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1537.
2017-03-13 14:22:14 -07:00
Peddy ccd11816e4 NIFI-3034 This closes #1323. Update Counter and test case 2017-03-12 16:34:33 -04:00
patricker 18f05856ff NIFI-3481 - Add DB Adapter for MS SQL 2012+
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-3481 Added support for Database Adapter Descriptions

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-3481 Updated Adapter Name

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1510
2017-03-10 11:34:16 -05:00
Andre F de Miranda 5a6373b16c
NIFI-3442 Explicitly support EL on username, password, port.
Introduced basic test unit

This closes #1474.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-03-08 20:56:01 -08:00
Pierre Villard 37e9c5efc7 NIFI-3556 Fixed ValidateCsv schema parsing
Fixed schema parsing on comma boundaries.

This closes #1570.

Signed-off-by: James Wing <jvwing@gmail.com>
2017-03-08 14:57:33 -08:00
Joe Skora 90ff275bb9 NIFI-3517 If HandleHttpResponse cannot write response remove entry from HttpContextMap.
* Add test for failure not clear context map.
* Add handler to remove context map entry if ProcessException occurs during while exporting response.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1567.
2017-03-08 13:16:58 +01:00
Pierre Villard 70175816b3 NIFI-3541 NIFI-3545 - check style violations
Fix for checkstyle issues introduced by:
9e68f02f1f
000414e7ea

This closes #1569.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-03-07 22:17:23 +09:00
Koji Kawamura 000414e7ea
NIFI-3545: Release M FlowFilews once N signals arrive
- Support multiplle incoming FlowFiles to Wait processor, up to Wait
  Buffer Count
- Added Releasable FlowFile Count, which controls how many FlowFiles can
  be released when wait condition is met
- Added special meaning to Notify delta Zero(0) to clear a signal
  counter back to zero

  This closes #1554

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-03-06 09:29:44 -05:00
Mark Payne 7f5c0dfb54 NIFI-3553: If IOException is thrown when completing FTP transfer, do not update the FlowFile to point to the content that was streamed but instead leave FlowFile pointing to original version
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1562.
2017-03-05 15:27:48 +01:00
Pere Urbon-Bayes 33a702500c NIFI-3554 Fixing the jBCrypt dependency name from nifi-standard-processors/pom.xml file, so it uses the expected naming format from the maven repository
Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-03-05 10:15:06 +11:00
Joe Trite f8cad0f8c9
NIFI-1750 Updated ScanAttribute dictionary file processor property description.
This closes #1543.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-03-03 17:36:40 -08:00
Giovanni Lanzani 5990db39ae
NIFI-3459 Fixed documentation for UnpackContent File Filter.
This closes #1558.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-03-02 16:19:11 -08:00
josephxsxn ac5a7960e8
NIFI-3547 Changing level from WARN to INFO for Wait Processor's expected behavior
This closes #1556.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-03-02 15:37:39 -05:00
patricker d689ac561e NIFI-3494: GenerateTableFetch Some JDBC Drivers Have Trouble With Name vs Label
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1517
2017-03-02 11:12:54 -05:00
patricker bf2f04fb5f NIFI-3540: QueryDatabaseTable Failing to Track MS SQL Max Values
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1547
2017-03-02 10:04:09 -05:00
Oleg Zhurakousky 8e17929d6a NIFI-2876 refactored demarcators into a common abstract class
This closes #1214.
2017-02-24 10:06:53 -05:00
Nick Carenza f9c4d3ce3a NIFI-3508: Added support for PATCH identical to that of PUT/POST
This closes #1522.

Signed-off-by: ijokarumawak <ijokarumawak@apache.org>
2017-02-21 16:23:23 +09:00
Oleg Zhurakousky ded18b94db NIFI-3255 removed dependency on session.merge from SplitText
NIFI-3255 addressed PR comments

NIFI-3255 fixed linkage for Split creation

This closes #1394
2017-02-17 12:31:21 -05:00
Koji Kawamura 095c04eda0 NIFI-3213: ListFile do not skip obviously old files
Before this fix, files with the latest timestamp within a listing
iteration are always be held back one cycle no matter how old it is.

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-02-18 00:56:46 +11:00
Aldrin Piri 76fcf5def1 NIFI-3385 - This closes #1440. Updating copyright statements to reflect the new year. 2017-02-16 00:10:02 -05:00
Koji Kawamura a90fa9c285 NIFI-3431: Support batch update in Notify processor
- Added Signal Counter Delta property
- Added Signal Buffer Count property
- Added processor property name and display name
- Changed IOException handling from routing it to failure to throw
  RuntimeException, so that NiFi framework can yield the processor for a while and try again

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1466.
2017-02-15 15:51:31 +01:00
Pierre Villard 71e2061b5d NIFI-2968 This closes #1377. Improved message when ExecuteSQL hits unsupported SQL types 2017-02-14 23:25:29 -05:00
Pierre Villard 6279fd4184 NIFI-3183 This closes #3183. Added command and arguments as FF attributes in ExecuteProcess processor 2017-02-14 22:08:09 -05:00
Koji Kawamura e46f4131f9 NIFI-3452: Add Wait processor Wait Mode property
Ensure back-pressure is active until downstream processing completes.

This closes #1490.
2017-02-14 19:15:27 +01:00
patricker 9cfc13423d NIFI-3455 Large row count paging
This closes #1499.
2017-02-14 19:09:12 +01:00
Andre F de Miranda ca887308af
NIFI-2565 - Add Elastic copyright to notice due to the use of grok patterns during test
This closes #1505.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-02-14 12:58:06 -05:00
Selim Namsi 2ef7c15b5d NIFI-2565: add Grok parser
This closes #1108.

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-02-14 21:11:03 +11:00
Yolanda M. Davis f1d0a6b36b NIFI-3010 - This closes #1348. applied styling changes 2017-02-13 22:38:54 -05:00
Yolanda M. Davis 8d082d1aca NIFI-3010 - Incorporate expression language support for jolt-spec for processor and Advance UI. Upgraded jolt library to latest version for support of modify functions. Also refactored to interact with JoltTransform interface and updated styling 2017-02-13 22:16:22 -05:00
Otto Fowler 5cfba6ace3 NIFI-3454: Use FileNameFilter to make test consistent
use the FileNameFilter when not passing down explit jar paths
Filter out ^. files when reading lists of files from directories

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-02-11 08:54:06 +09:00
Pierre Villard 2e1b87fa43 NIFI-3403 - Fix NPE in InvokeHTTP
Signed-off-by: Mike Moser <mosermw@apache.org>

This closes #1494
2017-02-10 20:59:18 +00:00
Ryan Persaud ea0f5690cc NIFI-3372 Adding capability for PutSQL to handle BIT fields generated by ConvertJSONToSQL.
This closes #1432.
2017-02-07 17:14:37 +01:00
Nick Carenza 46fa6a167a NIFI-3429 Quoting table identifier behind property
This closes #1464.
2017-02-07 16:48:38 +01:00
Koji Kawamura 9a638cc865 NIFI-1125 InvokeHTTP throws NullPointerException
Added null check in onPropertyModified to avoid NPE.

This closes #1477.
2017-02-07 09:06:43 +01:00
Giovanni Lanzani 3d3faada5c NIFI-3408 Add exception class when InvokeHTTP fails 2017-02-07 09:04:48 +01:00
Giovanni Lanzani 31b943d2c1 NIFI-3408 Add exception class when InvokeHTTP fails
This closes #1467.
2017-02-07 08:51:22 +01:00
Joe Gresock 229b20f395 NIFI-3419: Routing flow files causing ZipException to failure in MergeContent
This closes #1454.
2017-02-06 19:34:49 +01:00
Nick Carenza 6466931c23 NIFI-3430 - added support for specifying sql.arg.N.format for timestamp fields according to java.time.format.DateTimeFormatter
This closes #1468.
2017-02-06 18:32:57 +01:00
Pierre Villard adfaffc773 Updated documentation of ListenHTTP 2017-02-03 22:53:29 +00:00
Matt Burgess 6d4901cd26 NIFI-2881: Added EL support to DB Fetch processors
- Allow incoming flowfiles to GenerateTableFetch
- Incorporated review comments/discussions
- Updated documentation, added error attribute to GenerateTableFetch
- Corrected notes for column properties in fetch processors

This closes #1407.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-02-03 08:51:13 +09:00
Andre F de Miranda ea3c294f94
NIFI-819 - Extend GetHTTP to use dynamically add HTTP headers to a request
This closes #1462.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-02-02 09:56:53 -05:00