Commit Graph

2004 Commits

Author SHA1 Message Date
Pierre Villard 614fa6a6c4
NIFI-3528 Added support for keytab/principal to Kafka 0.10 processors
This closes #1606.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-04-06 15:09:49 -04:00
Bryan Bende 556f309df0
NIFI-3520 Refactoring instance class loading
- Fixing FlowController to use appropriate class loader when instantiating processor
- Updating ExtensionManager to leverage new flag in MANIFEST from NAR plugin
- Adding ReloadComponent interface and refactoring instance class loading to use it
- Fixing FetchHDFS issue with TDE by using ugi.doAs
- Refactoring nifi-nar-utils so that ExtensionManager only lives in nifi-framework
- Caching temp components found during service loader in ExtensionManager
- Updating authorizables, docs, and fingerprinting to use the cached components
- Introducing a flag on @RequiresInstanceClassLoading to indicate if ancestor resources should be cloned
- Updating developer guide regarding cloneAncestorResources flag
- This closes #1635
2017-04-06 13:51:08 -04:00
Matt Burgess 8f37ad4512 NIFI-3413: Add GetChangeDataCaptureMySQL processor
NIFI-3413: Incorporated review comments
NIFI-3413: Changed GetChangeDataCaptureMySQL to CaptureChangeMySQL, fixed some bugs
NIFI-3413: Refactored setup() for better error handling, more review comments incorporated
NIFI-3413: Refactored CDC into its own module(s), updated assembly and top-level POMs
NIFI-3413: Added RECEIVE prov event and Server ID property

Signed-off-by: ijokarumawak <ijokarumawak@apache.org>
2017-04-07 00:44:42 +09:00
Matt Gilman f5f54c468c
NIFI-3664: - Updating the unmarshalling to consider the current date when parsing the time.
This closes #1647.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-04-05 13:22:35 -04:00
Matt Gilman a565484ddd
NIFI-2940: - Allowing access to configuration actions through the Controller when the underlying component has been removed.
This closes #1648.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-04-04 17:40:09 -04:00
Koji Kawamura 5e62b4ae72 NIFI-3668: Fix purging expired replicate requests.
This closes #1646.

Newly created async response is added before checking map size nor
purging expired ones. If there are already 100 remaining requests,
the added request will not be executed nor removed.
2017-04-03 09:50:47 -04:00
Andre F de Miranda 84f1fb3959 NIFI-3222 - Allow ExecuteScript,InvokeScriptedProcessor and ScriptedReporting to use ExpressionLanguage to define the scripting engine modules path
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1640
2017-03-31 23:41:02 -04:00
joewitt 6a64b3cd9c NIFI-3440 This closes #1638. fixing tests not written for windows to not run on windows 2017-03-31 01:59:35 -04:00
Koji Kawamura fd92999daf
NIFI-3189: ConsumeKafka 0.9 and 0.10 with downstream backpressure
Currently, NiFi Kafka consumer processors have following issue.

While downstream connections are full, ConsumeKafka is not scheduled to run onTrigger.
It stopps executing poll to tell Kafka server that this client is alive.
Thus, after a while in that situation, Kafka server rebalances the client.
When downstream connections back to normal, although ConsumeKafka is scheduled again,
the client is no longer a part of a consumer group.

If this happens, Kafka client succeeds polling messages when ConsumeKafka processor resumes, but fails to commit offset.
Received messages are already committed into NiFi flow, but since consumer offset is not updated, those will be consumed again, duplicated.

In order to address above issue:

- For ConsumeKafka_0_10, use latest client library

    Above issue has been addressed by KIP-62.
    The latest Kafka consumer poll checks if the client instance is still valid, and rejoin the group if not, before consuming messages.

- For ConsumeKafka (0.9), added manual retention logic using pause/resume

    Kafka client 0.9 doesn't have background thread heartbeat, so similar machanism is added manually.
    Use Kafka pause/resume consumer API to tell Kafka server that the client stops consuming messages but is still alive.
    Another internal thread is used to perform paused poll periodically based on the time passed since the last onTrigger(poll) is executed.

This closes #1527.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-03-30 16:29:40 -04:00
Matt Gilman 282e1a7b1a NIFI-3421: This closes #1620.
- On contextDestroyed, referencing beans created during contextInitialized to prevent successive attempts to create a bean if that bean failed to be created initially.

Signed-off-by: joewitt <joewitt@apache.org>
2017-03-30 15:14:41 -04:00
Matt Gilman e203535163 NIFI-3272 This closes #1616.
- Updating the garbage collection column to use a icon with tooltip to relay all garbage collections.
2017-03-30 14:55:17 -04:00
Mark Payne 3aa1db6ee5 NIFI-3636: Lazily copy FlowFile Attributes Hash Map instead of doing so eagerly.
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-3257: Added additional logging regarding timing information when replicating requests across cluster in order to glean insight as to what is taking so long when replicating some requests

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-3649: Buffer node responses when replicating HTTP Requests up to a maximum buffer size

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-3636: Added unit test to ensure that flowfile attribute maps are copied when appropriate

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-3636: Removed patch file that should not have been in commit

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1612
2017-03-30 09:53:39 -04:00
Andre F de Miranda a7bf683a0d
NIFI-79 - Introduces the ability to set Bulletin Level to NONE 2017-03-30 08:54:24 -04:00
kedarchitale 457df93dd6 NIFI-3147 Build processor to parse CCDA into attributes
This closes #1312

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2017-03-29 20:30:57 +00:00
Mark Payne 83fa24b68f NIFI-3613: Fixed threading bug in JoltTransformJSON and adding caching of compiled Transforms
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1630
2017-03-29 10:45:34 -04:00
Gahan Rakholia 168fc72bba NIFI-3637 Fixed ListHDFS Documentation
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1633
2017-03-29 10:31:56 -04:00
Oleg Zhurakousky c324c9e21c NIFI-3441 This closes #1629. added @Ignore to integration tests due to intermittent failures 2017-03-28 16:17:17 -04:00
James Wing f9b5bcf7bb NIFI-2907 Fix Flow Schema Validation Errors
This closes #1587.
2017-03-28 10:53:16 -04:00
Matt Gilman 4432958a65
NIFI-3643: - Addressing incorrect capitalization in VersionInfoDTO in NiFiVersion.
This closes #1626.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-03-27 16:44:21 -04:00
Matt Gilman f15e0d8e1e
NIFI-3380: - Addressing issues with paths in generated documentation.
This closes #1627.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-03-27 16:43:15 -04:00
Pierre Villard ced6708d4b NIFI-3596 - added attributes to GenerateTableFetch processor
Signed-off-by: Matt Burgess <mattyb149@apache.org>

Updated test to check selected column names

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1596
2017-03-27 08:50:55 -04:00
Andre F de Miranda a26689318d
NIFI-814 - Introduce hidden header with name of Processor to allow those arriving to site via direct reference to know the name of the processor
This closes #1624.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-03-26 11:41:15 -04:00
Bryan Bende f0a1c14f02
NIFI-3380 Fixing parent pom of nifi-standard-prioritizers to point to correct parent after moving under nifi-framework 2017-03-25 11:13:58 -04:00
Bryan Bende d90cf846b9 NIFI-3380 Bumping NAR plugin to 1.2.0-SNAPSHOT development to leverage changes from master, adding buildnumber-maven-plugin to nifi-nar-bundles to properly set build info in MANIFEST of NARs
- Refactoring NarDetails to include all info from MANIFEST
- Adding the concept of a Bundle and refactoring NarClassLoaders to pass Bundles to ExtensionManager
- Adding logic to fail start-up when multiple NARs with same coordinates exist, moving Bundle classes to framework API
- Refactoring bundle API to classes and creating BundleCoordinate
- Updating FlowController to use BundleCoordinate

- Updating the UI and DTO model to support showing bundle details that loaded an extension type.
- Adding bundle details for processor canvas node, processor dialogs, controller service dialogs, and reporting task dialogs.
- Updating the formating of the bundle coordinates.
- Addressing text overflow in the configuration/details dialog.
- Fixing self referencing functions.
- Updating extension UI mapping to incorporate bundle coordinates.
- Discovering custom UIs through the supplied bundles.
- Adding verification methods for creating extensions through the rest api.
- Only returning extensions that are common amongst all nodes.
- Rendering the ghost processors using a dotted border.
- Adding bundle details to the flow.xml.
- Loading NiFi build and version details from the framework NAR.
- Removing properties for build and version details.
- Wiring together front end and back end changes.
- Including bundle coordinates in the component data model.
- Wiring together component data model and flow.xml.
- Addressing issue when resolve unvesioned dependent NARs.

Updating unit tests to pass based on framework changes
- Fixing logging of extension types during start up

- Allowing the application to start if there is a compatible bundle found. - Reporting missing bundle when the a compatible bundle is not found. - Fixing table height in new component dialogs.

Fixing chechstyle error and increasing test timeout for TestStandardControllerServiceProvider
- Adding ability to change processor type at runtime
- Adding backend code to change type for controller services

- Cleaning up instance classloaders for temp components.
- Creating a dialog for changing the version of a component.
- Updating the formatting of the component type and bundle throughout.
- Updating the new component dialogs to support selecting source group.
- Cleaning up new component dialogs.
- Cleaning up documentation in the cluster node endpoint.

Adding missing include in nifi-web-ui pom compressor plugin
- Refactoring so ConfigurableComponent provides getLogger() and so the nodes provide the ConfigurableComponent
- Creating LoggableComponent to pass around the component, logger, and coordinate with in the framework

- Finishing clean up following rebase.

Calling lifecycle methods for add and remove when changing versions of a component
- Introducing verifyCanUpdateBundle(coordinate) to ConfiguredComponent, and adding unit tests

- Ensuring documentation is available for all components. Including those of the same type that are loaded from different bundles.

Adding lookup from ClassLoader to Bundle, adding fix for instance class loading to include all parent NARs, and adding additional unit tests for FlowController
- Adding validation to ensure referenced controller services implement the required API
- Fixing template instantiation to look up compatible bundle

- Requiring services/reporting tasks to be disabled/stopped.
- Only supporting a change version option when the item has multiple versions available.
- Limiting the possible new controller services to the applicable API version.
- Showing the implemented API versions for Controller Services.
- Updating the property descriptor tooltip to indicate the required service requirements.
- Introducing version based sorting in the new component dialog, change version dialog, and new controller service dialog.
- Addressing remainder of the issues from recent rebase.

Ensuring bundles have been added to the flow before proposing a flow, and incorporating bundle information into flow fingerprinting
- Refactoring the way missing bundles work to retain the desired bundle if available
- Fixing logger.isDebugEnabled to be logger.isTraceEnabled

- Auditing when user changes the bundle. - Ensuring bundle details are present in templates.

Moving standard prioritizers to framework NAR and refactoring ExtensionManager logic to handle cases where an extension is in a JAR directly in the lib directory

- Ensuring all nodes attempt to instantiate the same template instance when the available bundles may differ. - Fixing the auditing of copy/paste and template instantiation. - Running addtional verification methods when running standalone.

Refactoring controller service invocation handler to allow updating the node used by the invocation handler
- Ensuring the bundles in a proposed flow are compatible with the current instance when the current instance has no flow is going to accept the proposed flow
- Merging whether multiple versions of the component are available
- Setting NAR plugin back to current released version
- Cleaning up DocGenerator to not process multiple times

Addressing incorrect usage of nf.Common. - Using formatType in the new component type dialogs.

Improving error messages when looking for bundles

Addressing comments from PR. - Fixing references to global nf namespace. - Fixing injection of nfProcessGroupConfiguration in nfComponentVersion. - Fixing web api integration tests.

Not rendering unversioned in help documentation. - Ensuring the isExtentionMissing flag is correct after changing the component type.

Adding synchronization in node classes to ensure changing component can't occur when component is running, introducing MissingBundleException for better reporting when a node can't join cluster due to a missing bundle, and bumping NAR plugin to released version 1.2.0

Adding concept of missing components to fingerprinting to ensure nodes agree on missing components when joining a cluster

NIFI-3380: NIFI-3520: - Fixing hive nar dependency. - Marking DBCPService as provided. - Skipping services that require instance classloading and are cobundled with their service API. - Skipping components that require instance classloading and reference service APIs that are cobundled. - Addressing UI issues in the new component dialogs when re-opening with a filter applied.

Fixing checkstyles issue and adding back assume checks to distributed cache server test

Ensuring new component types are sorted correctly when shown initially.

This closes #1585.
2017-03-24 11:06:44 -04:00
Andre F de Miranda a298f1a707 NIFI-3466 - Address non English locales in ParseCEF
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1614.
2017-03-23 09:04:16 +01:00
Scott Aslan a90a770244
[NIFI-3035] add deep linking URL to canvas components and PGs. This closes #1559 2017-03-22 14:23:02 -04:00
Matt Burgess d1ebddce98 NIFI-3627: Added removeByPattern() to DistributedMapCache interfaces
NIFI-3627: Updated unit tests that use MapCache interface(s)

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1609.
2017-03-22 19:22:08 +01:00
Pierre Villard 0b73715562 NIFI-3565 - Fixed BigDecimal instantiation and unit tests for specific locales
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1571
2017-03-22 13:40:59 -04:00
Matt Burgess 37a1e6d07f NIFI-3585: Add DatabaseAdapter impl for MS SQL 2008
This closes #1584

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2017-03-22 14:57:42 +00:00
Matt Burgess c4d0c0bbd1
NIFI-3626: Add support for DELETE in ConvertJSONToSQL
This closes #1605.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-03-21 17:20:27 -04:00
Scott Aslan 40acd4a6e1
[NIFI-2697] revert transmission switch toggle on failed update request. This closes #1600 2017-03-21 15:17:02 -04:00
Andy LoPresto b3b65219a0
NIFI-3361:
- Refactored TestGetHTTPGroovy for componentization & customization of tests.
- Ignored some TestGetHTTPGroovy tests because they cannot connect to a Jetty server running TLSv1 or TLSv1.1 (the new version of Jetty only supports TLSv1.2+). The tests are still valuable to demonstrate GetHTTP's capacity to connect to various TLS protocols, and should not be removed unless necessary.
- Refactored TestPostHTTPGroovy for componentization & customization of tests.
- Ignored some TestPostHTTPGroovy tests because they cannot connect to a Jetty server running TLSv1 or TLSv1.1 (the new version of Jetty only supports TLSv1.2+). The tests are still valuable to demonstrate PostHTTP's capacity to connect to various TLS protocols, and should not be removed unless necessary.
- Restored one of the TestGetHTTPGroovy unit tests (GetHTTP vs. TLSv1/1.1/1.2) by connecting to https://nifi.apache.org which still supports all three protocol versions (and uses JVM cacerts as truststore).
- This closes #1601
2017-03-21 14:48:40 -04:00
Matt Gilman 8e61aa3c43
NIFI-3361:
- Upgrading to the latest Jetty version.
2017-03-21 13:44:26 -04:00
Puspendu Banerjee 552148e9e7 NIFI-3579: Incorporated Review comments. This closes #1602. 2017-03-20 10:15:24 -04:00
Joe Percivall 5daeba6e03 NIFI-3614 Adding "return" statement to HandleHttpResponse when statusCode is not a number
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1598.
2017-03-20 14:51:56 +01:00
Jeremy Dyer d05727b8c0 NIFI-2613 Apache POI processor for Excel to CSV
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #929.
2017-03-17 10:39:05 -07:00
Matt Gilman 47c6718fe2 NIFI-3615:
- Removing hidden control characters.

This closes #1599

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2017-03-17 11:46:25 -04:00
Matt Gilman 768e89b579 NIFI-3246:
- Preventing the editing of controller services that are defined in an ancestor process group.
- Adding a go to link for users to easily navigate to where services are actually defined.

This closes #1594

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2017-03-14 14:15:14 -04:00
Puspendu Banerjee 78382c66bc NIFI-3579 Fix for Windows FileStore issue
Signed-off-by: Mike Moser <mosermw@apache.org>

This closes #1580
2017-03-14 13:58:06 -04:00
ambud 9424836412 NIFI-3525 Add tablename attribute to QDT flowfiles
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1537.
2017-03-13 14:22:14 -07:00
Joe Gresock bd91390105
NIFI-3582: Fixing PersistentMapCache bug that skipped persistent evictions
This closes #1592.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-03-13 15:08:25 -04:00
Matt Gilman 9d4239be1e NIFI-3245:
- Adding notes to the controller service tables to better define their availability.
- Clarifying support of Safari.

This closes #1593

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2017-03-13 13:48:11 -04:00
Matt Gilman 620c572703 NIFI-3244:
- Updating Controller Service scope header.
- Rendering the name of the parent when possible.
- Fixing the check if the parent Process Group is write-able when the parent is selected on the current canvas.

This closes #1590

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2017-03-13 12:35:46 -04:00
Jeff Storck a61f353051
NIFI-3520 Updated nifi-hdfs-processors POM to depend directly on hadoop-client
- Removed NAR dependency on nifi-hadoop-libraries-nar from nifi-hadoop-nar so that hadoop-client dependencies will be included directly in nifi-hadoop-nar
- Added RequiresInstanceClassLoading annotation to AbstractHadoopProcessor and HiveConnectionPool
- UGI relogins are now performed using doAs
- Added debug-level logging for UGI relogins in KerberosTicketRenewer and AbstractHadoopProcessor

This closes #1539.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-03-13 12:21:49 -04:00
Matt Gilman 9bb31a70d9 NIFI-3243:
- Rendering the selected Process Group in the header if possible.

This closes #1591

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2017-03-13 12:18:22 -04:00
Peddy ccd11816e4 NIFI-3034 This closes #1323. Update Counter and test case 2017-03-12 16:34:33 -04:00
Matt Gilman 977aa6919c
NIFI-3575:
- Verify that when moving a snippet, we do not attempt to move a process group into itself.
- Update the UI to ensure a subsequent move event cannot be triggered while a previous move event is still in progress.

This closes #1582.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-03-10 17:56:57 -08:00
patricker 18f05856ff NIFI-3481 - Add DB Adapter for MS SQL 2012+
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-3481 Added support for Database Adapter Descriptions

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-3481 Updated Adapter Name

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1510
2017-03-10 11:34:16 -05:00
Bryan Rosander cd8eb775e6 NIFI-3574 - PutHiveStreaming UGI fixes
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1578
2017-03-10 10:50:52 -05:00
joewitt 443803e729 NIFI-2564 corrected L&N to reference new BeanUtils dependency. 2017-03-09 22:00:41 -05:00
Joey Frazee c1e47f9b44 NIFI-2564 This closes #854. Add support for HL7 composite component names
Signed-off-by: joewitt <joewitt@apache.org>
2017-03-09 21:50:02 -05:00
Aldrin Piri c9097e7db3
NIFI-3541 Add copying of local network interface for DtoFactory RPG copy. This closes #1574 2017-03-09 08:23:28 -05:00
James Wing 3c862a1515
NIFI-3334 Fix Update Run Duration. This closes #1572 2017-03-09 08:13:46 -05:00
Andre F de Miranda 5a6373b16c
NIFI-3442 Explicitly support EL on username, password, port.
Introduced basic test unit

This closes #1474.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-03-08 20:56:01 -08:00
Andy LoPresto 7f6373239f
NIFI-3313 Added explicit Java runtime argument to default bootstrap.conf to avoid blocking on VM deployment.
This closes #1579.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-03-08 19:24:43 -08:00
Pierre Villard 37e9c5efc7 NIFI-3556 Fixed ValidateCsv schema parsing
Fixed schema parsing on comma boundaries.

This closes #1570.

Signed-off-by: James Wing <jvwing@gmail.com>
2017-03-08 14:57:33 -08:00
Koji Kawamura 27afd2e0cc NIFI-3163: Add new RPG settings to fingerprint
- Added timeout, yieldPeriod, transportProtocol, proxyHost, proxyPort,
  proxyUser and proxyPassword

- Removed unused fingerprint methods

FingerprintFactory has two types of fingerprinting method, from XML
elements and from DTO. However, the ones from DTO are not used by
anywhere. IDE didn't report those private methods unused because
addProcessGroupFingerprint and addSnippetFingerprint call each other,
but those are not used from outside actuallly.

This commit removes those private methods to keep the class clean to
avoid unnecessary code maintenance and tests.

This closes #1332.
2017-03-08 17:31:22 -05:00
Joe Skora 90ff275bb9 NIFI-3517 If HandleHttpResponse cannot write response remove entry from HttpContextMap.
* Add test for failure not clear context map.
* Add handler to remove context map entry if ProcessException occurs during while exporting response.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1567.
2017-03-08 13:16:58 +01:00
Andy LoPresto 1376e5fbd0
NIFI-3483 Added import statement to resolve failing tests.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-03-07 20:07:59 -05:00
Pierre Villard 01979aebec
NIFI-3483 - fixed one unit test relying on locale
This closes #1566.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-03-07 16:41:45 -05:00
Pierre Villard 70175816b3 NIFI-3541 NIFI-3545 - check style violations
Fix for checkstyle issues introduced by:
9e68f02f1f
000414e7ea

This closes #1569.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-03-07 22:17:23 +09:00
jpercivall b7f946e847 NIFI-1682 Adding AttributeRollingWindow processor
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1328.
2017-03-06 19:42:25 +01:00
Matt Gilman 16bde02ed0
NIFI-3541: - Allowing the user to specify the network interface to send/receive data for a Remote Process Group.
This closes #1550.

Signed-off-by: Mark Payne <markap14@hotmail.com>
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-03-06 10:38:15 -05:00
Mark Payne 9e68f02f1f NIFI-3541: Add local network interface capability to site-to-site client and remote group and ports 2017-03-06 10:36:30 -05:00
Koji Kawamura 000414e7ea
NIFI-3545: Release M FlowFilews once N signals arrive
- Support multiplle incoming FlowFiles to Wait processor, up to Wait
  Buffer Count
- Added Releasable FlowFile Count, which controls how many FlowFiles can
  be released when wait condition is met
- Added special meaning to Notify delta Zero(0) to clear a signal
  counter back to zero

  This closes #1554

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-03-06 09:29:44 -05:00
Mark Payne 7f5c0dfb54 NIFI-3553: If IOException is thrown when completing FTP transfer, do not update the FlowFile to point to the content that was streamed but instead leave FlowFile pointing to original version
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1562.
2017-03-05 15:27:48 +01:00
Pere Urbon-Bayes 33a702500c NIFI-3554 Fixing the jBCrypt dependency name from nifi-standard-processors/pom.xml file, so it uses the expected naming format from the maven repository
Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-03-05 10:15:06 +11:00
Joe Trite f8cad0f8c9
NIFI-1750 Updated ScanAttribute dictionary file processor property description.
This closes #1543.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-03-03 17:36:40 -08:00
Scott Aslan 9f669aa944
[NIFI-3550] inject nf.ConnectionConfiguration into the nf.Cannection module during nf.Graph init. This closes #1560 2017-03-03 09:38:29 -05:00
Mark Payne 6c44d6be2e NIFI-3523: Ensure that if we are unable to read Provenance events on startup that we don't prevent NiFi from starting
This closes #1542
2017-03-03 09:18:29 -05:00
Mark Payne 4ed64e7561
NIFI-3548: Fixed bug that caused failed requests to not get removed from 'request map' and also results in that preventing the purging logic, which would then unintentially throw exceptions. This closes #1555 2017-03-03 09:13:28 -05:00
Giovanni Lanzani 5990db39ae
NIFI-3459 Fixed documentation for UnpackContent File Filter.
This closes #1558.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-03-02 16:19:11 -08:00
Andre F de Miranda 7c6f2dc677 NIFI-3544 - Fixes minor typo n ExtractEmailHeaders log message
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1552.
2017-03-02 23:09:04 +01:00
josephxsxn ac5a7960e8
NIFI-3547 Changing level from WARN to INFO for Wait Processor's expected behavior
This closes #1556.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-03-02 15:37:39 -05:00
patricker d689ac561e NIFI-3494: GenerateTableFetch Some JDBC Drivers Have Trouble With Name vs Label
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1517
2017-03-02 11:12:54 -05:00
patricker bf2f04fb5f NIFI-3540: QueryDatabaseTable Failing to Track MS SQL Max Values
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1547
2017-03-02 10:04:09 -05:00
Pierre Villard 0d66b6dcda NIFI-1797 - CreateHadoopSequenceFile processor (including Pierre's forced push)
Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-03-02 22:52:10 +11:00
Pierre Villard dcdfd3dad9 NIFI-1797 - Added compression codec property to CreateHadoopSequenceFile processor
This closes: #1387

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-03-02 22:19:34 +11:00
Andre F de Miranda a32e1509b6 NIFI-3238 - Introduce ListenBeats processor and deprecates ListenLumberjack processor
NIFI-3238 - Rewords processor description

NIFI-3238: Removed unused imports

This closes #1418
2017-03-01 21:41:51 -05:00
Scott Aslan c3dbbb845f
[NIFI-3038] Update cluster summary dialog layout. This closes #1534 2017-02-28 09:47:47 -05:00
Bryan Rosander cfe899d189 NIFI-3530 - Ensuring configuration and ugi information is passed down to Hive in PutHiveStreaming
NIFI-3530 - closing renewer in PutHiveStreaming.cleanup(), making npe less likely in HiveWriter.toString()

This closes #1544
2017-02-27 15:01:59 -05:00
Scott Aslan 1978c986b5
[NIFI-3465] change CS and RT warning tooltip viewports to a visible and connected DOM element. This closes #1545 2017-02-27 13:18:14 -05:00
Scott Aslan d98426cc3d
[NIFI-2941] allow users to click checkbox labels to toggle the checkbox. This closes #1538 2017-02-27 09:57:43 -05:00
Scott Aslan a76641b252
[NIFI-2891] persist selected tab for connection configuration dialog, except for when creating a new connection the dialog will open with the details tab visible since users will need to select a connection relationship. This closes #1535 2017-02-27 09:21:55 -05:00
Oleg Zhurakousky 8e17929d6a NIFI-2876 refactored demarcators into a common abstract class
This closes #1214.
2017-02-24 10:06:53 -05:00
Scott Aslan fba17159a9
[NIFI-3358] remove duplicate calls to nfProcessGroupConfiguration.showConfiguration(). This closes #1540 2017-02-24 08:05:34 -05:00
Scott Aslan 2c374bafaa
[NIFI-3358] update naming conventions to avoid conflicts. This closes #1533 2017-02-23 09:14:33 -05:00
Mark Payne 64379d2007 NIFI-3522: When creating a clone of a FlowFile, ensure that we properly set the 'updated attributes' on the RepositoryRecord to all attributes on the FlowFile itself, the same way we do when calling create(FlowFile)
This closes #1532
2017-02-22 17:16:14 -05:00
Andre F de Miranda cefc02c5fb NIFI-3514 - Ensures ListGCSBucket triggers serially and forbids inputs
This closes #1526.

Signed-off-by: James Wing <jvwing@gmail.com>
2017-02-22 10:24:37 -08:00
Mark Payne 96ed405d70 NIFI-3356: Initial implementation of writeahead provenance repository
- The idea behind NIFI-3356 was to improve the efficiency and throughput of the Provenance Repository, as it is often the bottleneck. While testing the newly designed repository,
  a handful of other, fairly minor, changes were made to improve efficiency as well, as these came to light when testing the new repository:

- Use a BufferedOutputStream within StandardProcessSession (via a ClaimCache abstraction) in order to avoid continually writing to FileOutputStream when writing many small FlowFiles
- Updated threading model of MinimalLockingWriteAheadLog - now performs serialization outside of lock and writes to a 'synchronized' OutputStream
- Change minimum scheduling period for components from 30 microseconds to 1 nanosecond. ScheduledExecutor is very inconsistent with timing of task scheduling. With the bored.yield.duration
  now present, this value doesn't need to be set to 30 microseconds. This was originally done to avoid processors that had no work from dominating the CPU. However, now that we will yield
  when processors have no work, this results in slowing down processors that are able to perform work.
- Allow nifi.properties to specify multiple directories for FlowFile Repository
- If backpressure is engaged while running a batch of sessions, then stop batch processing earlier. This helps FlowFiles to move through the system much more smoothly instead of the
  herky-jerky queuing that we previously saw at very high rates of FlowFiles.
- Added NiFi PID to log message when starting nifi. This was simply an update to the log message that provides helpful information.

NIFI-3356: Fixed bug in ContentClaimWriteCache that resulted in data corruption and fixed bug in RepositoryConfiguration that threw exception if cache warm duration was set to empty string

NIFI-3356: Fixed NPE

NIFI-3356: Added debug-level performance monitoring

NIFI-3356: Updates to unit tests that failed after rebasing against master

NIFI-3356: Incorporated PR review feedback

NIFI-3356: Fixed bug where we would delete index directories that are still in use; also added additional debug logging and a simple util class that can be used to textualize provenance event files - useful in debugging

This closes #1493
2017-02-22 12:40:06 -05:00
Koji Kawamura 8d467f3d1f
NIFI-3505: Fix nifi.sh restart
The 'exec' command added by NIFI-2689 affected restart behavior
negatively as 'exec' command will not execute subsequent commands in the
shell script.
This commit changes 'exec' is added only when 'run' is specified.

This closes #1523.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-02-22 10:47:39 -05:00
Scott Aslan 364efa96cc
[NIFI-3501] Upgrade D3 to 3.5.17 and allow npm to pull it down at build time. This closes #1519 2017-02-21 10:17:34 -05:00
Nick Carenza f9c4d3ce3a NIFI-3508: Added support for PATCH identical to that of PUT/POST
This closes #1522.

Signed-off-by: ijokarumawak <ijokarumawak@apache.org>
2017-02-21 16:23:23 +09:00
Aldrin Piri 4f49095d75 NIFI-3294 Adjusting nifi.sh to invoke nifi-env.sh when running as another user such that properties are preserved across environments.
This closes #1405.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
Signed-off-by: ijokarumawak <ijokarumawak@apache.org>
2017-02-20 11:04:11 +09:00
Pierre Villard a8b8f1530b NIFI-2914 Fixed message error in AbstractMongoProcessor
This closes: #1380

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-02-19 21:25:08 +11:00
Scott Aslan 19302263c3
[NIFI-3181] add operate palette actions to context menu. This closes #1516 2017-02-17 15:40:23 -05:00
Oleg Zhurakousky 6a1854c975 NIFI-3354 Added support for simple AVRO/CSV/JSON transformers that utilize external Schema
Added support for simple Key/Value Schema Registry as Controller Service
Added support for registering multiple schemas as dynamic properties of Schema Registry  Controller Service
Added the following 8 processors
- ExtractAvroFieldsViaSchemaRegistry
- TransformAvroToCSVViaSchemaRegistry
- TransformAvroToJsonViaSchemaRegistry
- TransformCSVToAvroViaSchemaRegistry
- TransformCSVToJsonViaSchemaRegistry
- TransformJsonToAvroViaSchemaRegistry
- TransformJsonToCSVViaSchemaRegistry
- UpdateAttributeWithSchemaViaSchemaRegistry

polishing

NIFI-3354 Adding support for HDFS Schema Registry, unions and default values in the Avro Schema and NULL columns in the source CSV

NIFI-3354 Adding support for logicalTypes per the Avro 1.7.7 spec

NIFI-3354 polishing and restructuring CSVUtils

NIFI-3354 renamed processors to address PR comment

NIFI-3354 addressed latest PR comments
- removed HDFS-based ControllerService. It will be migrated into a separate bundle as a true extension.
- removed UpdateAttribute. . . processor
- added mime.type attribute to all Transform* processors

NIFI-3354 added missing L&N entries

This closes pr/1436
2017-02-17 14:32:06 -05:00
Oleg Zhurakousky ded18b94db NIFI-3255 removed dependency on session.merge from SplitText
NIFI-3255 addressed PR comments

NIFI-3255 fixed linkage for Split creation

This closes #1394
2017-02-17 12:31:21 -05:00
Koji Kawamura 095c04eda0 NIFI-3213: ListFile do not skip obviously old files
Before this fix, files with the latest timestamp within a listing
iteration are always be held back one cycle no matter how old it is.

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-02-18 00:56:46 +11:00
Scott Aslan d9828c240b
[NIFI-3360] resolve clashing variable name for connection and nf.Connection. This closes #1506 2017-02-16 15:54:00 -05:00
Koji Kawamura 1eb98aefee NIFI-3373: Add nifi.flow.configuration.archive.max.count
- Add 'nifi.flow.configuration.archive.max.count' in nifi.properties
- Change default archive limit so that it uses archive max time(30 days)
  and storage (500MB) if no limitation is specified
- Simplified logic to delete old archives

This closes #1460.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-02-16 14:19:39 +09:00
Aldrin Piri 76fcf5def1 NIFI-3385 - This closes #1440. Updating copyright statements to reflect the new year. 2017-02-16 00:10:02 -05:00
JJ 598563277e NIFI-3485: This closes #1514. Updated test to mock out use of InetAdress when looking up domain. There was an issue where unresolvable domains got routed to an ISP page. This should fix that issue. 2017-02-16 00:07:00 -05:00
Jeff Storck 8b90343715
NIFI-3355 Allows NiFi to bind to specific network interfaces, with separate interface lists for HTTP and HTTPS.
This closes #1508.

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-02-15 18:39:26 -05:00
João Henrique Ferreira de Freitas a62523c7f5 NIFI-3014 GetSNMP add textual oid attribute to flowfile
Signed-off-by: João Henrique Ferreira de Freitas <joaohf@gmail.com>
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1196.
2017-02-15 20:20:30 +01:00
Mark Payne 39cfd03397 NIFI-3205: This closes #1336. Ensure that FlowFile Repository is updated with any Transient Content Claims when session rollback occurs 2017-02-15 13:15:56 -05:00
Bryan Bende b207397a11 NIFI-1784 This closes #1349. Initial commit for FetchHBaseRow processor 2017-02-15 12:59:38 -05:00
Koji Kawamura a90fa9c285 NIFI-3431: Support batch update in Notify processor
- Added Signal Counter Delta property
- Added Signal Buffer Count property
- Added processor property name and display name
- Changed IOException handling from routing it to failure to throw
  RuntimeException, so that NiFi framework can yield the processor for a while and try again

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1466.
2017-02-15 15:51:31 +01:00
patricker b7cdc6b382 NIFI-2779 - Add processor to GetEmail Supporting Exchange Web Services
This closes: #1326

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-02-15 23:48:34 +11:00
Pierre Villard 86fb67d55c NIFI-3405 - Add uptime to JVM section in System Diagnostics
renamed column for node connection date in cluster

This closes #1492.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-02-15 16:53:53 +09:00
Pierre Villard 71e2061b5d NIFI-2968 This closes #1377. Improved message when ExecuteSQL hits unsupported SQL types 2017-02-14 23:25:29 -05:00
Pierre Villard 2d58497c2e NIFI-2859 - This closes #1383. Ignore files starting with a dot in ListHDFS 2017-02-14 23:21:44 -05:00
Pierre Villard 6279fd4184 NIFI-3183 This closes #3183. Added command and arguments as FF attributes in ExecuteProcess processor 2017-02-14 22:08:09 -05:00
Andy LoPresto bd88e4335a
Refactored user identity parsing and proxied entity chain formatting.
Added unit tests.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-02-14 13:39:05 -08:00
Koji Kawamura 4bf267c8bb NIFI-3101: This closes #1271. Improve Get/Modify/PutHTMLElement URL
- Added detailed description about how the URL property works with
  GetHTMLElement
- Added Expression support with URL
- Made URL property dynamic with ModifyHTMLElement and PutHTMLElement,
  since it won't be used to alter HTML element and need not to be
  specified. Making it a dynamic property let existing processor configuration stays valid
2017-02-14 15:28:46 -05:00
JJ 3dc7a160ce NIFI-3478: This closes #1507. added tests for GeoEnrichIP and added support for expression language on 'IP Address Attribute' property. 2017-02-14 13:33:27 -05:00
Koji Kawamura e46f4131f9 NIFI-3452: Add Wait processor Wait Mode property
Ensure back-pressure is active until downstream processing completes.

This closes #1490.
2017-02-14 19:15:27 +01:00
patricker 9cfc13423d NIFI-3455 Large row count paging
This closes #1499.
2017-02-14 19:09:12 +01:00
Bryan Rosander 8ffa1703ba NIFI-3389 - Using long string type for attribute name and value in FlowFileSchema
This closes #1446.
2017-02-14 13:04:09 -05:00
Andre F de Miranda ca887308af
NIFI-2565 - Add Elastic copyright to notice due to the use of grok patterns during test
This closes #1505.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-02-14 12:58:06 -05:00
Hunter Morgan df91169512
NIFI-2689 - improve foreground run mode of nifi
* add exec to RUN_NIFI_CMD
* remove subshell for else
* tested compatible with runit with these changes

This closes #966.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-02-14 12:37:31 -05:00
Simon Elliston Ball 0aa4b72678 Added EL support to CSV properties
Clean up checkstyle errors

Added unit tests, fixed typo

This closes #709

Signed-off-by: Matt Burgess <mattyb149@apache.org>
2017-02-14 11:19:35 -05:00
Matt Burgess 7963df89eb NIFI-3288: Add Clojure support for ExecuteScript
This closes: #1402.

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-02-15 02:52:11 +11:00
Mike Moser 690130b063
NIFI-3362 update FlowConfiguration.xsd to allow all current time period units
This closes #1501.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-02-14 10:20:39 -05:00
Joe Skora bd3e0438a3 NIFI-2890 Provenance Repository Corruption (1.x)
* Corrected handling of corrupt journal file records that prevents instance startup and loss of records from corrupt files.  Specifically, exception handling was expanded to cover failures on records after the first the same as failures on the first record.
* Adjusted log messages  to reflect that the remainder or all of the journal will be skipped, not just the current record.

This closes #1485.
2017-02-14 10:02:16 -05:00
Pierre Villard 1e43694145 NIFI-3057 Added provenance events to PutElasticsearch and FetchElasticsearch
This closes: #1370.

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-02-15 01:37:14 +11:00
Selim Namsi 2ef7c15b5d NIFI-2565: add Grok parser
This closes #1108.

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-02-14 21:11:03 +11:00
Andre F de Miranda 1a91ecc379
NIFI-3352 - Address issue where ListenLumberjack was sending long numbers back to clients - instead of int
This closes #1421.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-02-13 23:55:21 -08:00
Koji Kawamura b2290b24a6
NIFI-3107: Unit test failure for DBCPServiceTest
H2 and Kafka broker uses the same default port 9092.
If an user is running Kafka broker on the same machine, or run the unit
tests in parallel, DBCPServiceTest can fail since some of its test
methods connects to port 9092.

This closes #1504.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-02-13 23:09:07 -08:00
joewitt 7e97946c35
NIFI-3412 ignoring tests which depend on timing/network setup
This closes #1503.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-02-13 22:46:44 -08:00
Yolanda M. Davis f1d0a6b36b NIFI-3010 - This closes #1348. applied styling changes 2017-02-13 22:38:54 -05:00
Yolanda M. Davis 8d082d1aca NIFI-3010 - Incorporate expression language support for jolt-spec for processor and Advance UI. Upgraded jolt library to latest version for support of modify functions. Also refactored to interact with JoltTransform interface and updated styling 2017-02-13 22:16:22 -05:00
Joe Skora 41ad032151 NIFI-3055 StandardRecordWriter Can Throw UTFDataFormatException (1.x)
* Remove function based on JDK source.
* Add new function to find bytes based on RFC3629.
* Add field name to log entry when field is truncated.

Signed-off-by: Mike Moser <mosermw@apache.org>
This closes #1475
2017-02-13 20:15:59 +00:00
Scott Aslan 2bc7d52626
[NIFI-3458] increase processor preview hieght to avoid bottom border being cut off. This closes #1498 2017-02-13 09:46:20 -05:00
Scott Aslan 4274481f1d
[NIFI-3450] update upload template dialog browse button. This closes #1495 2017-02-13 09:32:02 -05:00
Scott Aslan 932b5bb91b
[NIFI-3305] update hover text for change color on the operate pallete. This closes #1489 2017-02-13 09:02:54 -05:00
Gene Peters 897c70298a NIFI-3449: Adding Google Cloud Storage Bundle
* Credentials service with tests
* Abstract processor definitions
* GCS-themed processors and their corresponding tests

Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1482.
2017-02-12 19:52:38 -08:00
Scott Aslan febe6da4a2 [NIFI-3459] remove blue higlighting from bread crumbs
This closes #1500.
2017-02-11 15:36:19 +01:00
Otto Fowler 5cfba6ace3 NIFI-3454: Use FileNameFilter to make test consistent
use the FileNameFilter when not passing down explit jar paths
Filter out ^. files when reading lists of files from directories

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-02-11 08:54:06 +09:00
Scott Aslan e4b74b3d71
[NIFI-3410] and styles for incorrect Processor Details dialog relationship description in Summary page. This closes #1488 2017-02-10 17:01:33 -05:00
Scott Aslan bf3b1640fc
[NIFI-3360] taking bottom up approach to modularizing the canvas...wrapping modulaes with UMD and keep the app loading
- taking top down approach from this point and leveraging inversion of control desgin pattern
- This closes #1487
2017-02-10 16:03:23 -05:00
Pierre Villard 2e1b87fa43 NIFI-3403 - Fix NPE in InvokeHTTP
Signed-off-by: Mike Moser <mosermw@apache.org>

This closes #1494
2017-02-10 20:59:18 +00:00
Pierre Villard 4e4d14f86f NIFI-3139 Added host:port list and non empty EL validators in StandardValidators 2017-02-10 11:22:43 -05:00
Matt Gilman 74ecc20f00 NIFI-3291:
- Ensuring slider step is non negative.

This closes #1486
2017-02-09 09:42:36 -05:00
Matt Gilman bc7b17a64c NIFI-3291:
- Fixing slider issue when there is only one event in the lineage graph.
- Fixing scollbar issue when accessing the queue listing.
2017-02-09 09:29:18 -05:00
Matt Gilman f0c6739a1f NIFI-3251:
- Ensuring read permissions to the ControllerService before attempting to check permissions to parent.

This closes #1483
2017-02-08 15:13:20 -05:00
Matt Gilman 83ee7590a6
- Fixing checkstyle violation. 2017-02-08 10:45:34 -05:00
Pierre Villard da5aafdf3f
NIFI-3121 Remove read permissions on proxy resource for Node Identities. This closes #1368 2017-02-07 17:01:51 -05:00
Koji Kawamura 5af6eb17b0
NIFI-3162: Audit RPG and RPG port config changes.
- Added configure audits for Transport Protocol, HTTP Proxy Server Host,
  Port, User and Password in RemoteProcessGroup configuration
- Added configure audits for enabling/disabling individual remote port
- Added configure audits for Concurrent Tasks and Compressed in Remote
  Port configuration
- This closes #1476
2017-02-07 16:22:51 -05:00
Pierre Villard 93150d3efa
NIFI-3447 - PutSplunk - force connection close
This closes #1479.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-02-07 13:37:32 -05:00
Ryan Persaud ea0f5690cc NIFI-3372 Adding capability for PutSQL to handle BIT fields generated by ConvertJSONToSQL.
This closes #1432.
2017-02-07 17:14:37 +01:00
Nick Carenza 46fa6a167a NIFI-3429 Quoting table identifier behind property
This closes #1464.
2017-02-07 16:48:38 +01:00
Oleg Zhurakousky 027fbf48b8 NIFI-3223 added support for expression language
- EXCHANGE
- ROUTING_KEY

This closes #1449.
2017-02-07 15:45:56 +01:00
Koji Kawamura 9a638cc865 NIFI-1125 InvokeHTTP throws NullPointerException
Added null check in onPropertyModified to avoid NPE.

This closes #1477.
2017-02-07 09:06:43 +01:00
Giovanni Lanzani 3d3faada5c NIFI-3408 Add exception class when InvokeHTTP fails 2017-02-07 09:04:48 +01:00
Giovanni Lanzani 31b943d2c1 NIFI-3408 Add exception class when InvokeHTTP fails
This closes #1467.
2017-02-07 08:51:22 +01:00
Joe Gresock 229b20f395 NIFI-3419: Routing flow files causing ZipException to failure in MergeContent
This closes #1454.
2017-02-06 19:34:49 +01:00
Nick Carenza 6466931c23 NIFI-3430 - added support for specifying sql.arg.N.format for timestamp fields according to java.time.format.DateTimeFormatter
This closes #1468.
2017-02-06 18:32:57 +01:00
James Wing e4cee7ee0d NIFI-3427 Fix Provenance DateTime Sorting
This closes #1465.
2017-02-06 18:06:58 +01:00
Pierre Villard adfaffc773 Updated documentation of ListenHTTP 2017-02-03 22:53:29 +00:00
Joe Skora 376af83a3d NIFI-3055 StandardRecordWriter Can Throw UTFDataFormatException
* Updated StandardRecordWriter, even though it is now deprecated to consider the encoding behavior of java.io.DataOutputStream.writeUTF() and truncate string values such that the UTF representation will not be longer than that DataOutputStream's 64K UTF format limit.
* Updated the new SchemaRecordWriter class to similarly truncate long Strings that will be written as UTF.
* Add tests to confirm handling of large UTF strings and various edge conditions of UTF string handling.

Signed-off-by: Mike Moser <mosermw@apache.org>

This closes #1469.
2017-02-03 20:52:32 +00:00
Matt Burgess 2d6d7710c7 NIFI-3426: Add dynamic property support to DBCPConnectionPool
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1461.
2017-02-03 11:09:02 -08:00
Matt Burgess 6d4901cd26 NIFI-2881: Added EL support to DB Fetch processors
- Allow incoming flowfiles to GenerateTableFetch
- Incorporated review comments/discussions
- Updated documentation, added error attribute to GenerateTableFetch
- Corrected notes for column properties in fetch processors

This closes #1407.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-02-03 08:51:13 +09:00
Andre F de Miranda ea3c294f94
NIFI-819 - Extend GetHTTP to use dynamically add HTTP headers to a request
This closes #1462.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-02-02 09:56:53 -05:00
Oleg Zhurakousky 390754c575 NIFI-3179 Added support for default UTF-8 char encoding
removed deprected usage of BAOS and BAIS

This closes #1452.
2017-01-31 15:33:20 +01:00
Oleg Zhurakousky 6518778c11 NIFI-3180 Fixed NPE in TemplateUtils
added null check for ProcessorDTO.getRelationship()
removed deprecated usage of ByteArrayInputStream

This closes #1451.
2017-01-30 22:10:50 +01:00
Ben Schofield d5b139ffd4 NIFI-3418 Add records-per-transaction property to putHiveStreaming processor
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1455

Minor whitespace Checkstyle issue fixed
2017-01-30 11:46:42 -05:00
Pierre Villard 78a0e1e18b NIFI-3290 Reporting task to send bulletins with S2S
This closes #1401
2017-01-30 09:03:35 -05:00
Pierre Villard 89f1bd3189 NIFI-957 Added the possibility to use DefaultSchedule annotation in reporting tasks
This closes #1400
2017-01-30 08:35:39 -05:00
Koji Kawamura 63c763885c NIFI-3363: PutKafka NPE with User-Defined partition
- Marked PutKafka Partition Strategy property as deprecated, as Kafka 0.8 client doesn't use 'partitioner.class' as producer property, we don't have to specify it.
- Changed Partition Strategy property from a required one to a dynamic property, so that existing processor config can stay in valid state.
- Fixed partition property to work.
- Route a flow file if it failed to be published due to invalid partition.

This closes #1425
2017-01-27 12:48:23 -05:00
Matt Gilman f8f66fa22b NIFI-3291:
- Removing dead code.

This closes #1438
2017-01-27 11:50:56 -05:00
Matt Gilman 0950186fbb NIFI-3291:
- Fixing overflow calculation to determine if scrollbars are necessary.
- Fixing styles with jquery ui slider usage.
2017-01-27 11:47:09 -05:00
Jeff Storck bd4ea488bb NIFI-3387 Updated NOT_YET_CONNECTED value in DisconnectionCode enum
This closes #1448.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-01-27 08:44:11 +09:00
Andrew Psaltis 250c4a8941
NIFI-2615 Addressing documentation issues that caused checkstyle failures and the build to fail
This closes #1447.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-01-26 13:05:33 -08:00
Pierre Villard c15111d985 NIFI-3390 Added support for multiple LDAP servers. This closes #1441 2017-01-26 08:59:18 -05:00
Andy LoPresto 506709922b
NIFI-2615 Fixed checkstyle violations and RAT (missing license) issues. 2017-01-25 19:03:13 -08:00
Oleg Zhurakousky f0856565ad NIFI-2615 polishing
- added missing POM entries to nifi POM and assembly POM
- added available port-discovery to tests amongs other minor polishings
2017-01-25 13:22:50 -05:00
Andrew Psaltis 3626abd78a NIFI-2615 Adding GetTCP processor
NIFI-2615 Addressing changes from P/R. Specifically, removing .gitignore as it should not be there for a nar. Removed non-used class. Changed name in notice
2017-01-25 13:22:50 -05:00
Pierre Villard 1a6802a016 NIFI-3295 - ensures node is disconnected before trying to remove it. This closes #1444 2017-01-25 12:24:45 -05:00
Pierre Villard e113511fef NIFI-3393 - replicates cluster/nodes/{id} to cluster coordinator. This closes #1443 2017-01-25 10:25:36 -05:00
Matt Gilman 7340078de2 NIFI-3251: Delete requires WRITE perms on parent
- Requiring WRITE permissions to the parent resource when attempting to remove a component.
- Updating expired certificates in the REST API integration tests.

This closes #1399.

Signed-off-by: James Wing <jvwing@gmail.com>
2017-01-24 13:24:29 -08:00
Mike Moser ddda602620 NIFI-3350 optimized documentation extraction to reduce startup time
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1416.
2017-01-22 14:27:19 -08:00
Matt Gilman 79ca30be4a NIFI-3301: Provenance UI Cursor Styling
- Addressing inconsistent cursor styling and drag behavior in lineage graph.

This closes #1430.

Signed-off-by: James Wing <jvwing@gmail.com>
2017-01-20 18:40:36 -08:00
Scott Aslan dc934cbb8e [NIFI-3359] Modularize all of nifi-web-ui except canvas directory
- Removing shell.jsp from summary.jsp.
- This closes #1428
2017-01-20 16:18:08 -05:00
Koji Kawamura 7f0171ffa2
NIFI-3216: Add N signals to Wait/Notify
- Support counters at Wait/Notify processors so that NiFi flow can be
  configured to wait for N signals
- Extract Wait/Notify logics into WaitNotifyProtocol
- Added FragmentAttributes to manage commonly used fragment attributes
- Changed existing split processors to set 'fragment.identifier' and
  'fragment.count', so that Wait can use those to wait for all splits
get processed

This closes #1420.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-01-19 15:17:59 -05:00
Matt Gilman e62eeb7563 NIFI-3291:
- Fixing styles from our usage of jquery ui autocomplete.
- Fixing horizontal scroll bars.
- Fixing border around content repos.

This closes #1429
2017-01-19 11:13:44 -05:00
David W. Streever bbc714e73b PutHiveQL and SelectHiveQL Processor enhancements. Added support for multiple statements in a script. Options for delimiters, quotes, escaping, include header and alternate header.
Add support in SelectHiveQL to get script content from the Flow File to bring consistency with patterns used for PutHiveQL and support extra query management.

Changed behavior of using Flowfile to match ExecuteSQL.  Handle query delimiter when embedded.  Added test case for embedded delimiter

Formatting and License Header

PutHiveQL and SelectHiveQL Processor enhancements. Added support for multiple statements in a script.  Options for delimiters, quotes, escaping, include header and alternate header.

Add support in SelectHiveQL to get script content from the Flow File to bring consistency with patterns used for PutHiveQL and support extra query management.

Changed behavior of using Flowfile to match ExecuteSQL.  Handle query delimiter when embedded.  Added test case for embedded delimiter

Removing dead code.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

Comments to Clarify test case.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

Final whitespace/formatting/typo changes

This closes #1316
2017-01-18 14:47:31 -05:00
Matt Gilman e925b18fe6 NIFI-1135:
- Adding additional parameters to be able to limit the size of the provenance response. Specifically, whether the events should be summarized and whether events should be returned incrementally before the query has completed.
- Ensuring the cluster node address is included in provenance events returned.
- Ensuring there is a cluster coordinator before attempting to get the cluster node address.
- Removing exponential back off between provenance requests.
- Ensuring the content viewer url is retrieve before initializing the provenance table.

This closes #1413.
2017-01-18 10:34:56 -05:00
Matt Gilman 82cf0c6fa8 NIFI-3291:
- Ensuring we don't see a horizontal scrollbar.
- Ensuring slick-nifi-theme is bundled.

This closes #1408
2017-01-17 15:42:34 -05:00
Matt Gilman 6170f6442e NIFI-3291:
- Upgrading jQuery and it's plugins.
- Packaging at build time using npm.
2017-01-17 14:54:59 -05:00
Andrew Lim ad98ac179f
NIFI-3340 Modify description of the Directory property for PutHDFS to say it will create the directory if it doesn't exist
This closes #1424.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-01-17 13:29:59 -05:00
Koji Kawamura ef54a8ec69
NIFI-3214: Added fetch and replace to DistributedMapCache
- Using fetch and replace together can provide optimistic locking for
  concurrency control.
- Added fetch to get cache entry with its meta data such as revision
  number.
- Added replace to update cache only if it has not been updated.
- Added Map Cache protocol version 2 for those new operations.
- Existing operations such as get or put can work with protocol version
  1.

This closes #1410.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-01-16 12:15:06 -05:00
Andre F de Miranda 8c7539b20a Revert "NIFI-3352 - Address issue where ListenLumberjack was sending long numbers back to clients - instead of int"
This reverts commit 9609dafc34.
2017-01-17 01:53:18 +11:00
afucs 9609dafc34 NIFI-3352 - Address issue where ListenLumberjack was sending long numbers back to clients - instead of int 2017-01-14 22:57:33 +11:00
Jeff Storck def4918af0
NIFI-3330 Replaced two instances of multiple calls to getter methods during request attribute retrieval with a single call saved to a local variable to prevent null pointer exceptions during multiple invocations of a getter method.
This closes #1415.

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-01-13 18:09:28 -05:00
Joe Skora 4d533a99b3 NIFI-2861 ControlRate should accept more than one flow file per execution
* Support multiple files per onTrigger call.

Signed-off-by: Mike Moser <mosermw@apache.org>

This closes #1412.
2017-01-12 21:43:12 +00:00
Pierre Villard 2fbeabb95f
NIFI-3309 ensures that CS are deleted when a process group is deleted
This closes #1411.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-01-12 11:21:46 -05:00
Matt Gilman a794166d21 NIFI-3118 - Sorting the garbage collection stats and content repository entries client side. Opting to not sort server side as the property of the DTO does not allow deterministic sorting. Consequently, order would need to be implemented every time that DTO is (de)serializaed which may happen in a number of places with zero master clustering.
This closes #1337.
2017-01-09 18:09:25 +01:00
James Wing 1195d4c186 NIFI-3153 Updating AWS SDK Version
This closes #1341.
2017-01-09 17:49:17 +01:00
Joey Frazee 81aca7b97d NIFI-3302 Remove 'accepts incoming FlowFile' doc from QDT
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1406
2017-01-09 10:40:29 -05:00
Matt Burgess 675f4f544c
NIFI-1458: Added ScriptedReportingTask
This closes #1045.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-01-06 11:56:19 -08:00
Andy LoPresto 31ec01b5f9
NIFI-3004 Added failing unit test to SSLContextServiceTest to demonstrate that customValidate caches result from previous validation.
NIFI-3004 Added logic to expire StandardSSLContextService customValidate cache after 5 invocations.
Updated unit test to demonstrate this logic.

This closes #1375.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-01-05 17:32:11 -08:00
Pierre Villard 970c46ccfe
NIFI-3004 Improved StandardSSLContextService customValidate
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-01-05 17:32:01 -08:00
Mike Moser 3259b01f8e NIFI-3230 fix ActiveMQ SSL support for simple connect URIs in Get/Put JMS processors
This closes #1344.
2017-01-05 22:19:25 +01:00
Bryan Rosander 474df053d6 NIFI-3276 - FileSystemRepository.getPath() only check exists if necessary
This closes #1388.
2017-01-05 21:44:59 +01:00
jpercivall 273e69f2cb NIFI-3274 Adding WriteAheadLog configuration options to WriteAheadLogLocalStateProvider
This closes #1386.
2017-01-05 19:59:58 +01:00
jpercivall ac42178e9b NIFI-3283 Fixing typo in UpdateAttribute error message
This closes #1396.
2017-01-05 18:51:40 +01:00
Pierre Villard 64bb55ef3c NIFI-2839 Add a confirmation dialog when deleting a Controller Service or Reporting Task. This closes #1384 2017-01-05 09:07:35 -05:00
Matt Gilman 083d4043e0 NIFI-2840:
- Sorting the templates to ensure that the newest templates are listed first.

This closes #1391.
2017-01-04 19:24:37 +01:00
Matt Gilman b1c9f0e764
NIFI-2695: - Providing more granular and meaningful authorization error messages.
This closes #1309.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-01-04 13:06:19 -05:00
Pierre Villard 2c0f1c348e
NIFI-3259 - Process group level option in Ambari reporting task
This closes #1360.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-01-03 16:18:53 -05:00
Pierre Villard 9b47961d1c
NIFI-3231 Added EL support to hostname and port in PutTCP/UDP
This closes #1361.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-01-03 12:09:19 -05:00
Bryan Rosander 1b4729e448
NIFI-3240 - AttributesToJson performance improvements
This closes #1352.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-01-03 10:20:16 -05:00
Pierre Villard be6bcf20ad
NIFI-2944 Check remote input hostname at startup.
Added unit test cases.

This closes #1379.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-01-02 20:25:03 -08:00
Scott Aslan 67cbef5df3
NIFI-2917: disable canvas refresh after ajax error and allow page refresh
This closes #1351.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-01-02 16:32:05 -08:00
Davy De Waele 55f4716f3d NIFI-3266 Added EL support for basePath and port in ListenHTTP
This closes #1373.
2016-12-30 16:35:46 +01:00
Matt Gilman 0d14db72f5 NIFI-3247:
- Removing unnecessary authorization check during second phase of connection creation.
- Ensuring that the remote group port returns the correct resource type though not super critical since it is not possible to create policies for remote ports.

This closes #1353.
2016-12-30 16:05:06 +01:00
jpercivall 8acbe9aa3f
NIFI-1582 Fixing how failures with state are handled and improving flowfile handling
This closes #1371

Signed-off-by: Bryan Rosander <brosander@apache.org>
2016-12-29 15:49:52 -05:00
Pierre Villard 4986b83b83 NIFI-3164 Added NAR loading duration
This closes #1366.

Signed-off-by: James Wing <jvwing@gmail.com>
2016-12-29 10:29:03 -08:00
Pierre Villard 5c095f3a18 NIFI-3164 Added message at startup to log NAR loading
This closes #1365.

Signed-off-by: James Wing <jvwing@gmail.com>
2016-12-28 14:10:30 -08:00
Bryan Rosander 35e8bedcc8 NIFI-3249 - UpdateAttribute performance improvements
This closes #1356

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-12-28 14:54:05 -05:00
Pierre Villard 16898668c2 NIFI-3228 added s3.contenttype in PutS3Object
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1362.
2016-12-28 10:54:21 -08:00
Joe Gresock 2ea8b4122b NIFI-3127: Fixing NPE for empty flow files in SplitText
This closes #1358.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-12-28 11:21:44 +09:00
Matt Gilman 2f3017638f
NIFI-3239:
- Indicating components are not restricted when appropriate.
- Updating unit tests.

This closes #1350.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-12-27 13:06:07 -05:00
Bryan Bende 978f483ba8
NIFI-3250 Fixing logic in StandardFlowFileQueue when migrating flow files to the active queue
This closes #3250.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-12-27 12:52:01 -05:00
Joe Gresock af8ed8b7de
NIFI-3253: Correcting documentation in Notify processor
This closes #1359.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-12-27 12:19:08 -05:00
Matt Gilman 6c5d4084d5 NIFI-3207: Fixed Template upload import URI
- Fixing the importUri when replicating a Template upload request. Previously was leveraging getResourceUri which considers proxy headers meant for client responses.

This closes #1334.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-12-27 15:44:25 +09:00
Bryan Bende 6ca25e10ef NIFI-3220 Add support for basic auth to Solr processors
This closes #1338.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-12-26 15:41:24 +09:00
Scott Aslan 5ea17d30c5 [NIFI-96] add align horizontal and align vertical capability to components on the canvas. This closes #1354 2016-12-22 16:47:34 -05:00
Matt Gilman e65aad8fe6 NIFI-2994: - Setting the default position for remote process groups.
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1303
2016-12-22 14:39:56 -05:00
Matt Gilman acc9e0b45c NIFI-2967:
- Disabling the Add button in the new Processor, Controller Service, and Reporting Task dialog when no components match the filter.
This closes #1340
2016-12-22 11:52:20 -05:00
Koji Kawamura 908e7d3131
NIFI-2585: Add attributes to track s2s host and port
- Removed host and port field from Peer since the same information is
  available in PeerDescription
- Refactored variable names in SocketRemoteSiteListener to improve readability
- Changed how SocketRemoteSiteListener constructs PeerDescription
  instance. It used to use hard-coded 'localhost' as hostname, and
  getPort() which returns server's port. Since the peer is a remote peer,
  i.e the client, it should be client hostname and port.
- Added hostname resolution at DataTransferResource to make s2s.host
  value consistent with RAW transport. Without this, RAW uses hostname
  while HTTP uses IP address. It will be hard to be used from downstream flows.
- Replaced heavy use of mockito which was difficult to maintain, with
  nifi-mock
- Added SiteToSiteAttributes and more assertions in unit tests

This closes #1342.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-12-22 11:00:00 -05:00
Bryan Bende f7d761a28a
NIFI-2585 Moving attributes into loop in AbstractFlowFileServerProtocol, and also updating StandardRemoteGroupPort to apply the same attributes when doing a pull-based site-to-site.
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-12-22 10:59:56 -05:00
Randy Gelhausen 28e5d85493
NIFI-2585: Add attributes to track where a flow file came from when receiving over site-to-site
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-12-22 10:59:55 -05:00
Bryan Rosander 44c9ea0a66 NIFI-3236 - SplitJson performance improvements
This closes #1347.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-12-22 15:22:53 +09:00
Bryan Rosander 4285157675
NIFI-3235 - EvaluateJsonPath performance improvements
This closes #1346.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-12-21 16:29:23 -05:00
Matt Gilman 721c9ee7f0 NIFI-3167: - Ensuring the DELETE button is visible for cluster nodes. - Removing unecessary wrapper element and non breaking spaces.
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1319
2016-12-21 12:02:44 -05:00
Matt Burgess 3b91635398 NIFI-3194: Fixed error handling in PutElasticsearchHttp
Thise closes #1327

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-12-19 14:51:41 -05:00
Scott Aslan aef17f9a8b [NIFI-3170] remove action details if user does not have read perms, also update action details styles to match other dialogs. This closes #1322 2016-12-19 10:07:15 -05:00
Joe Gresock 34627f78b7
NIFI-190: Initial commit of Wait and Notify processors
This closes #1329.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-12-16 10:29:16 -05:00
Wesley-Lawrence 9a5986b0f6 NIFI-3175 Fix a bug in the ValidateCSV processor to handle empty CSV columns
This closes #1311.
2016-12-15 22:43:15 +01:00
Mark Payne ab2d07aa5a
NIFI-3186: Fixed NPE that was overlooked
This closes #1317.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-12-15 12:56:23 -05:00
Mark Payne 5776c4b1f9 NIFI-3173: When a template is created with multiple components in different groups all referencing the same controller service, ensure that controller service is added to the template at a high enough level that all components needing hte service can access it.
- Ensure that controller services are added to child process groups when creating snippet

- Addressed issue related to modifying higher-level process groups' controller services in snippet after having already visited the process group

This closes #1318

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-12-14 14:10:59 -05:00
Byunghwa Yun 0f462a7c49 NIFI-3029: QueryDatabaseTable supports max fragments property
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1213
2016-12-14 10:51:27 -05:00
jpercivall 266bd80bde NIFI-3192 Adding a check if the FlowFile is null to UpdateAttribute
This closes #1325
2016-12-13 11:30:24 -05:00
Scott Aslan 868795cdaa [NIFI-3159] Add ellipsis to long PG name in access policies shell. This closes #1310 2016-12-13 09:41:31 -05:00
Scott Aslan 45e4514a40 [NIFI-3160] reload canvas when updating PG name of current PG. This closes #1315 2016-12-12 11:45:10 -05:00
Scott Aslan 2c84b44f54 [NIFI-3154] display ellipsis when text is sufficiently long in connection details and connection configuration dialogs. This closes #1305 2016-12-08 12:52:01 -05:00
Koji Kawamura d15bd0c00c NIFI-3165: Update RPG revision when RPGPort is updated. This closes #1308 2016-12-08 08:34:07 -05:00
Bryan Rosander 9ce0af22af NIFI-3138 - Adding urls, transport, proxy settings to rpg portion of flow config xsd
This closes #1304.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-12-08 14:37:07 +09:00
Scott Aslan 6b82e25c74 [NIFI-3106] updating hover text for remove counter icon. This closes #1299 2016-12-07 16:42:19 -05:00
jpercivall e36b37692c NIFI-1582 added state to UpdateAttribute as well as updated a few parts that hadn't be touched in years (referenced the 'FlowFileMetadataEnhancer' processor'. Also added a 'NUMBER_VALIDATOR' to StandardValidators
NIFI-1582 removing the option to use cluster state

NIFI-1582 addressing Oleg's comments

NIFI-1582 No longer forcing numbers as the init value and adding getStateValue() to EL instead of using 'ATTRIBUTE_state'

NIFI-1582 Removing init state value

NIFI-1582 Adding documentation for the changes to Init State value

This closes #319
2016-12-06 16:52:02 -05:00
Bryan Rosander 97d2d30423
NIFI-3131 - Qualifying nifi-env in nifi bat scripts, removing popd from toolkit bat scripts
This closes #1293

Signed-off-by: Bryan Rosander <brosander@apache.org>
2016-12-05 23:03:41 -05:00
Bryan Rosander d1f2492dec
NIFI-3132 - Using cygpath and quoting paths correctly in sh scripts when run in cygwin
This closes #1292

Signed-off-by: Bryan Rosander <brosander@apache.org>
2016-12-05 22:16:24 -05:00
Matt Gilman 2d6bba080f NIFI-3133: - Ensuring that Remote Group Ports are always authorized according to their Remote Process Group.
NIFI-3133:
- Using getSourceAuthorizable() when accessing flow files and content.

NIFI-3133:
- Decouple local and remote connectable's to avoid ambiguity with self referencing RPGs.

NIFI-3133:
- Addressing comments from the PR.

NIFI-3133:
- Fixed check verifying source/destination when creating a connection.

NIFI-3133:
- Only showing the go to link when the source component is not a remote port.

NIFI-3133:
- Removing unnecessary checking of remote group port authorization since it's handled by the parent RPG.

NIFI-3133:
- Fixing issue showing the connection details dialog when the source component is a RPG.

NIFI-3133:
- Ensuring the local connectable was found.

This closes #1297

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-12-05 19:45:45 -05:00
Mark Payne fc0d336f36
NIFI-3152: Ensure that we always close the IndexWriter when appropriate in SimpleIndexManager, even if an IOException is thrown when trying to commit the IndexWriter
This closes #1300.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-12-05 16:07:54 -05:00
jpercivall 7633fe35c1 NIFI-3149 Adding proper literal support for number operators ('-' and '+') to ANTLR parser and UI highlighting
This closes #1298
2016-12-05 13:34:54 -05:00
Byunghwa Yun 2b6d6c25d8 NIFI-2866 The Initial Max Value of QueryDatabaseTable won't be case-sensitive
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1212
2016-12-05 12:50:43 -05:00
Mark Payne 8f8b8cdf46 NIFI-3071: Deprecated InputStreams & OutputStream sin org.apache.nifi.stream.io package in favor of using their Java counterparts 2016-12-05 09:29:19 -05:00
Matt Gilman c5ef076786
NIFI-2325:
- Adding support for LDAPS.

This closes #1275.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-12-02 12:41:03 -08:00
Matt Burgess 316cae16d3 NIFI-3140: Restored optional type handling in FetchElasticsearchHttp
This closes #1288

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-12-02 14:10:52 -05:00
Koji Kawamura 7c5bd876bd NIFI-3026: Support multiple remote target URLs
- Added urls in addition to the existing url, to support multiple target
  URLs
- Backward compatibility is provided by returning the first url if
  multipe urls are specified, but component accessing the url doesn't
support multiple urls
- UI is not fully updated yet. Following UI components are planned to be updated
  by different commits
  - Search component: only the first URL is searchable and shown
  - Component status: RPG status shows only the first URL
  - Component action history: only the first URL is searchable and shown
  - Updated Search component to use URLs.

This closes #1208.
2016-12-02 14:01:39 -05:00
Matt Gilman d8d29811f5
NIFI-3135: - Authorizing restricted components on snippet usage.
- Updating REST API docs accordingly.
- Adding some integration tests to exercise the additional restricted component checks.

This closes #1287.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-12-02 12:55:23 -05:00
Matt Burgess 69b23adf1b NIFI-3095: Add EL support to Elasticsearch 2.x (and HTTP) processors
This closes #1276

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-12-02 11:16:23 -05:00
Koji Kawamura 8da38acf31 NIFI-3141: Fixed TailFile ArrayIndexOutOfBounds.
- Added unit test cases to simulate NiFi version update which fails without this fix.
- Added state object migration code, add file.0. prefix to state keys,
  and add length from stored position.

This closes #1289
2016-12-02 07:53:04 -05:00
Bryan Bende 0297352350
NIFI-2017 Fixing failing test PermissionBasedStatusMergerSpec.groovy introduced in previous commit 2016-12-01 14:02:05 -05:00
Mark Payne 02f02b245a NIFI-2017: Fixed typo in nf-port.js that was referencing the wrong variable name to determine whether or not a port is transmitting
Fixed merging logic for root group status. Only consider a port transmitting when there is an active request, not when there is a 'queued' request
This closes #1279
2016-11-30 14:51:28 -05:00
Matt Gilman 1f40f298c0
NIFI-3130: - Loading the temporary instance from the correct type of component.
This closes #1285.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-11-30 14:13:43 -05:00
Mark Payne fff0148a0e NIFI-3129: When adding controller services to a snippet, ensure that we don't add the service multiple times, even when it's referenced by child process groups. This closes #1284 2016-11-30 13:07:51 -05:00
Matt Burgess ba513447d7
NIFI-3076: Fixed handling of 'medium' unsigned integers in JdbcCommon
This closes #1254.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-11-30 13:05:38 -05:00
Matt Gilman bc223fa197
NIFI-3117: - Checking referenced services when performing an action with a snippet that requires it.
- Updating the SnippetManager to automatically expire unused snippets.
- Making the checking of referenced services consistent across component removal.
- Adding checking of referenced services for all nested components in a snippet.
- Updating the REST API docs to clarify when a referenced service is authorized.
- Conditionally authorizing transitive service references.

This closes #1277.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-11-30 12:50:25 -05:00
jpercivall e3c7611347
NIFI-3112 Revert "NIFI-2199 - allows nifi.sh restart through ssh"
This reverts commit 88d125137e.

This closes #1282.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-29 17:32:47 -08:00
Mark Payne 9d6ca6e1e6
NIFI-2975: Only start expiring FlowFiles after FlowFile Repository has been restored
This closes #1274.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-11-29 09:54:15 -05:00
joewitt 92f17a995b NIFI-3100-rc2 prepare for next development iteration 2016-11-25 23:49:27 -05:00
joewitt 5536f690a8 NIFI-3100-rc2 prepare release nifi-1.1.0-RC2 2016-11-25 23:49:13 -05:00
Joey Frazee 617b62ac7a NIFI-3104 This closes #1273. Removed tests derived from HAPI test data 2016-11-25 19:39:36 -05:00
Koji Kawamura f4f60c3610 NIFI-3103: This closes #1272. Exclude localhost.cert from RAT check. 2016-11-25 19:33:48 -05:00
Andy LoPresto 2c3714536f
NIFI-3024 Added key migration for sensitive processor properties contained in flow.xml.gz. (nifi.sensitive.props.key)
This closes #1261.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-23 13:26:18 -08:00
Mark Payne 7ff14f7191 NIFI-3091: Ensure that we set the appropriate size on FlowFiles when modifying them
This closes #1267

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-23 12:07:44 -05:00
Mark Payne 91ff810dba NIFI-3088: Ensure that on recovery of FlowFile Repository, if we find a FlowFile that maps to an unknown queue that we log a warning that the queue is missing and drop the FlowFile, rather than throwing an NPE
This closes #1266

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-23 11:27:14 -05:00
joewitt 22143e386d NIFI-2991 This closes #1265. was allowed to restore twitter4j json.org usage until April. Resolution for that will be in NIFI-3089 2016-11-23 10:21:41 -05:00
Matt Burgess 7fc7494b21 NIFI-3087: This closes #1263. Added unit tests to PutElasticsearch(Http) to illustrate issue 2016-11-23 08:22:34 -05:00
Matt Burgess d3dbac50a8 NIFI-3087: Fixed issue with partial failure responses in PutElasticsearch(Http) 2016-11-23 08:11:49 -05:00
Jeff Storck 066accc274 NIFI-3086 Updated TailFile.java to deal with windows paths that were breaking the regex parsing Fixed documentation misspelling in TailFile.java Fixed assertions in several tests in TestTailFile that were dependent on the received order of flowfiles, this was differing between Windows and MacOS/Linux
This closes #1264

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-23 00:01:12 -05:00
Joseph Percivall 50667ed222 NIFI-3084 Fixing TestPersistentProvenanceRepository and closing a stream leak in RecordReaders.
This closes #1260.
2016-11-22 17:31:47 -05:00
Oleg Zhurakousky 913ae91614 Added posix guard to ignore tests on windows
This closes #1259

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-22 17:11:25 -05:00
Bryan Bende 78fed3ed56
NIFI-3068 Adding @RequiresInstanceClassLoading to PutHDFS to ensure the security configuration of the Hadoop client is fully isolated between instances of PutHDFS 2016-11-22 16:41:44 -05:00
Koji Kawamura 36796b5440 NIFI-3078: Test InvokeScriptedProcessor with JS.
This closes #1255.
2016-11-22 21:42:36 +01:00
Matt Burgess fa5fed9bb5 NIFI-3082: Fixed status code handling in PutElasticsearchHttp
This closes #1258.
2016-11-22 21:32:03 +01:00
Kathik Narayanan 9e884f6120 NiFI-3064 Convert to Avro fails when a MySQL query returns two columns with the same name even with different aliases
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1251
2016-11-22 11:49:25 -05:00
Mark Payne aa99cc6822
NIFI-3079: Ensure that in all cases we increment session's bytesRead count when finished reading from Content Repo
This closes #3079.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-11-22 11:26:22 -05:00
Scott Aslan ead9205458 [NIFI-3075] remove fixed width from combo options and add max-width css style to display ellipsis when options text is too long. This closes #1253 2016-11-21 17:27:03 -05:00
Mike Moser 15af764dd8 NIFI-2996
- validate processors only when they are in STOPPED state
- report validation errors via REST API on processors/services/tasks/ports only when they are in the STOPPED state
- This closes #1192
2016-11-21 17:02:25 -05:00
joewitt fb9cbccc38 NIFI-2954 This closes #1244. Moved StandardPropertyValidator to nifi-utils, documented scope/purpose of a few util libs, removed deps from nifi-utils. 2016-11-21 16:30:42 -05:00
Matt Burgess dba9d5924a NIFI-3011: Added Elasticsearch5 processors
This closes #1233

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-21 15:58:40 -05:00
Matt Gilman 7f5eabd603
NIFI-3050: Implemented access control logic for restricted components.
- Addressing comments from PR.
- Adding restricted tags to relevant components.
- Showing a restricted icon overlay on the processor node on the canvas. (+1 squashed commit)
Squashed commits:
[f487682] NIFI-3050:
- Introducing a Restricted annotation for components that require elevated privileges to use.
- Updating the new Processor, Controller Service, and Reporting Task dialogs to include these details and prevent unauthorized selection.
- Including the Restricted description in the generated component documentation.
- Updating processor access control integration test to verify restricted component creation.
- Updating the developer, user, and admin guide to include the restricted component policy.

This closes #1247.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-21 12:07:48 -08:00
Matt Burgess eb5abf809d
NIFI-2927: Added Validation Query to HiveConnectionPool
This closes #1252.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-11-21 14:21:22 -05:00
Peter Wicks d65f46f7a5 NIFI-2750
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-2750

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-2750 Tweaking Property Retrieval

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1001
2016-11-21 12:58:17 -05:00
Mark Payne 2ee66de1a6 NIFI-3066: Create unit test to reproduce bug that results in IllegalArgumentException: Cannot migrate FlowFiles from a Process Session to itself
NIFI-3066: Ensure that when a Bin is created, it is always created with its own new session

This closes #1245
2016-11-21 11:34:14 -05:00
Joey Frazee e731f09573 NIFI-3061: Bumped azure-eventhubs version to 0.9.0
This closes #1250.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-11-21 10:30:28 +09:00