Commit Graph

1348 Commits

Author SHA1 Message Date
Mark Payne a9d029d74e NIFI-2717: Ensure that when performing site-to-site communications that we always commit the transaction on the remote side before committing it on the local side
This closes #978

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-09-06 10:57:53 +09:00
Joe Gresock 0e64c3ed30 NIFI-2631: Adding page commits and 'Use Versions' to ListS3
This closes #916 and closes #917.

Signed-off-by: James Wing <jvwing@gmail.com>
2016-09-03 16:04:42 -07:00
Jeff Storck 97e2f406da NIFI-2708 Added merging of SiteToSite details
NIFI-2708 Updated to include ports from clientDto for port merging.

This closes #981
2016-09-02 14:55:08 -04:00
Matt Gilman d37480eaff NIFI-2722:
- Updating the component entity as long as the proposed entity is not older than the current one since stats are bundled in the entity too.

This closes #983

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-09-02 22:25:25 +09:00
Scott Aslan 867c84f32c [NIFI-2649] Update CS table upon creation of a CS from the RT or CS properties table
- Ensuring the controller services table to correctly reloaded regardless if its the processor group or controller level controller services. This closes #968
2016-09-01 09:53:12 -04:00
Matt Gilman 96a766464c
NIFI-2678: - Ensuring the most recent version of each component is save for rendering on the canvas.
This closes #979.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-31 15:59:52 -04:00
Matt Gilman 9e10371c8a NIFI-2704:
- Re-using the DataTransferAuthorizable in the DataTransferResource.
- Removing use of the DataTransferResource when obtaining site to site details as it performs additional unnecessary checks.
- Code clean up.
- This closes #971.
2016-08-31 15:09:17 -04:00
Yolanda M. Davis 1bb909ebca NIFI-2703 - Added bootstrap.conf file for running integration tests. This closes #975 2016-08-31 09:28:08 -04:00
Scott Aslan 648ddb6158 [NIFI-2679] User is now able to disable/enable processors that are invalid or stopped through Operate palette. This closes #973 2016-08-31 09:11:46 -04:00
Mark Payne 1213b46632 NIFI-2628: Whenever a flow is inherited from cluster coordinator, save flow immediately to disk
This closes #967
2016-08-30 16:13:45 -04:00
Matt Gilman ba19238757
NIFI-2694: - Addressing instances where phase one objects were being referenced instead of phase two objects. - Code clean up.
This closes #970.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-30 15:50:48 -04:00
Andy LoPresto 8556777bf6
NIFI-2618 Reverted merging PR 955 to master as 1.0.0 requires Java 8. 2016-08-30 09:27:24 -07:00
Andy LoPresto d600b065bb
NIFI-2618 Added logic to unit test to handle default TLS version for Java <= 7.
This closes #955.
2016-08-30 08:25:38 -07:00
jpercivall 1752a57197 Merge branch 'NIFI-2676-rc1' of https://git-wip-us.apache.org/repos/asf/nifi 2016-08-31 14:01:57 +10:00
Koji Kawamura daf4855570 NIFI-2687: RPG Port ID was used instead of RPG ID
It caused "Error: Unable to find remote process group with id 'XXXX'" error with modifying the transmitting toggle switch of an Input or Output port.

This closes #962
2016-08-29 10:14:55 -04:00
Joseph Percivall 1fe18a1567 NIFI-2676-rc1 prepare for next development iteration 2016-08-26 11:40:58 -04:00
Joseph Percivall 74d5224783 NIFI-2676-rc1 prepare release nifi-1.0.0-RC1 2016-08-26 11:40:44 -04:00
Scott Aslan c8ae7fcbe1 [NIFI-2480] remove explicit angular $sanitize to fix multibyte chars in breadcrumbs and flow status bar. All values are escapoed through the angular ng-bind. This closes #957 2016-08-26 11:03:10 -04:00
Koji Kawamura 2ed58912b2 NIFI-2480: Fix for multibyte chars in breadcrumbs. This closes #956
Removed $sanitize from breadcrumbs controller, as it escapes multibyte
chars with numerical reference. Even without $sanitize, html tags can be
escaped when Angular binds the value to text content.
2016-08-26 10:27:27 -04:00
Matt Burgess 0745990c2d NIFI-2604: Added validators and logic for multiple URLs/files/folders for DB driver location
This closes #912

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-25 23:35:41 -04:00
jpercivall 6ef1cca18f NIFI-2269 Fixing Datadog licensing 2016-08-25 22:43:00 -04:00
jpercivall e1da9fc11f NIFI-2667 Adding License and Notice information to Media Bundle Nar and Assembly
This closes #950
2016-08-25 22:42:37 -04:00
joewitt 405252244e NIFI-2670 minor doc edit to remove kafka.key.hex reference 2016-08-25 22:17:07 -04:00
Mark Payne 58e0ce7f92 NIFI-2670: This closes #954. Expose option for how to interpret Kafka Key - hexadeimal encoding or UTF-8 String 2016-08-25 21:39:40 -04:00
Bryan Rosander 102a9a2b74 NIFI-2674 - Always adding flow files to result set flowfiles, catching runtime exceptions while converting to avro
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #953
2016-08-25 21:16:26 -04:00
Oleg Zhurakousky 7879a99206
NIFI-2675 fixed NPE in template generation
This closes #951.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-08-25 16:44:02 -07:00
Bryan Bende 957c120343 NIFI-2664 Moving System.setProperty for krb5.conf to NiFi startup, and removing conflicting property from KerberosProvider config
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #946
2016-08-25 17:26:37 -04:00
Mark Payne f908ae3c3b NIFI-2669: This closes #949. Ensure that if Exception is thrown during Transaction initialization that the underlying client is closed/cleaned up. Also ensure that we generate bulletins when logging error/warn level log messages 2016-08-25 16:39:00 -04:00
Matt Burgess a84d3c9873 NIFI-2665: This closes #942. Fixed intermittent validation errors in InvokeScriptedProcessor 2016-08-25 15:34:50 -04:00
joewitt 1624dd8e4d NIFI-2666 This closes #947. updated where it says BSD to say MIT for bcrypt 2016-08-25 14:59:46 -04:00
joewitt 218408953a NIFI-2668 Added missing license/notice entries for ignite deps in nar 2016-08-25 14:50:38 -04:00
Matt Burgess 3224b1a5dd NIFI-2668: This closes #945. Replaced JSR-107 JCache API JAR w Geronimo version, updated notices 2016-08-25 14:41:08 -04:00
Matt Burgess ad7808f63d NIFI-2641: This closes #928. Add max values as attributes to QueryDatabaseTable 2016-08-25 14:13:55 -04:00
Oleg Zhurakousky 04db806ace NIFI-2614 This closes #944. added support for max.request.size 2016-08-25 13:51:08 -04:00
Matt Gilman a6133d4ce3 NIFI-2635:
- Fixing contrib check issues.
- Clean up pom.
- Addressing issue where reporting task property descriptor using wrong scope.

NIFI-2635:
- Fixing issue with revisions when creating users and user groups.
- Forwarding requests to the coordinator instead of replicating.
- Tweaking verbage in dialog for removing users and groups.

This closes #943
2016-08-25 13:08:35 -04:00
joewitt 1745c1274b NIFI-2608 This closes #930. restructured the ConsumeKafka processor to follow new consumer API. Made nar classloading more precise to support spawned threads NIFI-2660. 2016-08-25 10:09:50 -04:00
Bryan Bende 626e23e0ab Adding ConsumerResource and ConsumerPool for ConsumeKafka
Signed-off-by: joewitt <joewitt@apache.org>
2016-08-25 09:47:26 -04:00
Scott Aslan 56e515f7ae [NIFI-2658] Use bulletinEntity id instead of bulleting id since the ids match but bulletinEntities do not get bulletin objects if user is not authorized. This closes #938 2016-08-25 09:27:41 -04:00
Andre F de Miranda 6fd1c17ca4 NIFI-2662 - Fix checkstyle violations introduced by NIFI-2635
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #940
2016-08-25 08:55:34 -04:00
Peter Wicks e210172d93 PutSQLBinary
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-2591

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-2591

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-2591 - Added Format option for binary data types. Updated unit tests.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #883
2016-08-25 08:41:23 -04:00
Koji Kawamura afb9a0016f NIFI-2659: Fixed RPG Received stats. This closes #939
It was showing the 'Sent' stats as 'Received' stats.
2016-08-25 08:34:17 -04:00
Matt Gilman c2bfc4ef24 NIFI-2635: - Re-using the original request during the second phase of the two phase commit. - Forwarding requests to the coordinator when received by a node.
This closes #933

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-24 22:42:10 -04:00
Matt Gilman 2ceb5c8097
NIFI-2647:
- Referencing correct components when checking if save is required.

This closes #935.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-08-24 15:52:20 -07:00
Andy LoPresto 6331dda8d3 NIFI-2643 Changed null check to !isBlank() for key in RunNiFi.
NIFI-2643 Updated comment in bootstrap.conf template.

This closes #932
2016-08-24 16:20:10 -04:00
Jeff Storck 6475924f53 NIFI-2642 Catches (and ignores) ResourceNotFoundException during the updating of a property value for a controller service when the current value points to a controller service that has been deleted, allowing the assigning of a new controller service to continue.
This closes #931
2016-08-24 15:55:04 -04:00
Mark Payne c2ae7a6d7c NIFI-2605: - Fixing a regression bug where nodes would potentially be elected leader for Cluster Coordinator role when they do not have the correct flow
-  Ensure that we log which node is the cluster coordinator on startup instead of just indicating that there is one. If we later determine that there is none, ensure that we register for the role

This closes #900

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-24 12:30:48 -04:00
Andy LoPresto c638191a47
NIFI-1831 Added internal logic and command-line tool to allow AES-encrypted sensitive configuration values in nifi.properties.
This closes #834.
2016-08-23 20:34:31 -07:00
Matt Burgess ee9bd94082 NIFI-2632: Added fragment attributes to SplitJson and SplitXml
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #919
2016-08-23 16:30:57 -04:00
Matt Burgess 5d1a4f343f
NIFI-2622: Added support for complex types in SelectHiveQL
This closes #922.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-23 15:52:16 -04:00
Andrew Psaltis 0303805c01
NIFI-2620 Adding support for Binary Row Keys for both PutHBaseCell and PutHBaseJSON. This also involved making changes to PutFlowFile and PutColumn to carry around byte[] and not all strings. This closes #914.
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-23 13:25:55 -04:00
Oleg Zhurakousky 4a49587533 NIFI-2610 annotated unstable tests with @Ignore
This closes #918
2016-08-23 13:13:05 -04:00
Matt Burgess fbec3b9c13
NIFI-2623: Fixed support for binary types in SelectHiveQL
This closes #920.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-23 11:43:46 -04:00
Mark Payne 8cc670c8a6 NIFI-2634: Ensure that we check whether or not the Site-to-Site protocol is set when importing template, instead of assuming that it will be
This closes #921

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-23 11:19:44 -04:00
Scott Aslan 3f60eac544 [NIFI-2599] Enhance Status History Dialog. This closes #894 2016-08-23 10:53:10 -04:00
jpercivall 6e82ec738c NIFI-2269 Fixing minor issues with style check, licensing, etc. 2016-08-22 21:34:51 -04:00
Ramiz 376d3c4ef4 NIFI-2269 Added datadog support
This closes #655

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-22 21:34:51 -04:00
Jeff Storck dca3764ed1 NIFI-2626 Fixes jetty server thread leak. This closes #910 2016-08-22 15:58:06 -04:00
Joe Gresock 17dec04939 NIFI-2611: Fixing bugs in UnpackContent with flow file unpackers
Signed-off-by: Mike Moser <mosermw@apache.org>

This closes #905
2016-08-22 14:47:24 -04:00
Matt Gilman 087622eadc NIFI-2606:
- Addressing issue when fingerprinting ReportingTasks and ControllerServices properties with default values.
- Ensuring the flow is saved when templates are created and imported.
- Ensuring default values are included in templates.
- Fixing unit tests.
This closes #908.
2016-08-22 13:35:15 -04:00
Mark Payne 1fcec3747b NIFI-2617: Instead of retrying an infinite number of times to communicate with ZooKeeper, should try only for a short period in CuratorLeaderElectionManager. This is because web requests may be blocked waiting on a determination of which node is cluster coordinator
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #906
2016-08-22 13:24:22 -04:00
Mark Payne 5fab783b50
NIFI-2609: Ensure that we handle URIs for Remote Process Groups that do not have a path of /nifi or /nifi/
This closes #902.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-22 13:21:42 -04:00
Mark Payne 95b5877f5d NIFI-2600: Ensure that we do not close Index Searchers prematurely, even if they are poisoned
This closes #896
2016-08-22 12:03:47 -04:00
Joe Skora a2d3d0c289 Update to support offsets larger than 2 gigabyte. 2016-08-22 11:02:48 -04:00
Matt Burgess b7dc21bd95 NIFI-2547: Fixed checkstyle errors and updated to reflect KerberosProperties changes 2016-08-21 10:26:40 -04:00
ricky 26d362b144 NIFI-2547: Add DeleteHDFS Processor
This processor adds the capability to delete files or
directories inside of HDFS.

Paths supports both static and expression language values,
as well as glob support (e.g. /data/for/2016/07/*).

This processor may be used standalone, as well as part of a
downstream connection.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

Add Glob Matcher with Tests

Also set displayName on properties.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #850
2016-08-21 10:10:21 -04:00
Joe Skora 02071103d0 * Add "Remove All Content" property, related functionality, and tests to ModifyBytes processor to allow deletion of all flow file content.
* Removed @Ignore annotation on class and unnecessary EOL translation of test data.  Because ModifyBytes treat input as binary data, not text, line endings don't matter as long as they byte offsets are calculated correctly.
* Replace validator with .allowableValues.

This closes #890.
2016-08-19 20:05:14 +02:00
Matt Gilman 3378426f35 NIFI-2581: Keeping context menu and tooltips open when refreshing the canvas.
This closes #899.
2016-08-19 19:36:26 +02:00
Scott Aslan a181c7b9d7 [NIFI-2601] Update bower.json to use specific versions. This closes #897 2016-08-19 10:34:48 -04:00
Jeff Storck 31ca8d02f7 NIFI-2488 Fixed BulletinDTOs not getting their node addresses set properly during bulletin merging. This closes #888 2016-08-19 08:32:00 -04:00
Matt Burgess 46b81058c7 NIFI-2602: Fixed NPE in SelectHiveQL when CSV output and null column value
This closes #898.
2016-08-19 09:56:26 +02:00
Matt Gilman 6b5950647a NIFI-2596:
- Adding tooltip for property descriptor on mouse enter of table cell.

This closes #893.
2016-08-18 17:23:30 -04:00
Matt Gilman 57648308db NIFI-2597:
- Updating the correct object when attempting to create a self loop.
2016-08-18 17:15:16 -04:00
Matt Gilman 87161ab96e
NIFI-2595: - Updating ReportingTasks to use ComponentLogger instead of creating Controller level bulletins.
- Making the bulletin responses consistent in that all bulletins will be included but in redacted form as appropriate.
- Fixing broken unit test.

This closes #892.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-18 17:06:28 -04:00
Matt Gilman f3387426a0 NIFI-2587:
- Adding additional authorization for accessing component property history.
2016-08-18 17:05:32 -04:00
Matt Burgess 6874a5d82d NIFI-2593: This closes #891. Fixed handling of nested records/structs in ConvertAvroToORC 2016-08-18 15:15:51 -04:00
Matt Burgess a74bc2c7c7 NIFI-2598: This closes #889. Fixed issue with static init of properties in HiveConnectionPool 2016-08-18 13:38:22 -04:00
Bryan Bende e0e4b3407a NIFI-2574 This clsoes #887. Fixing NPE when using kerberos keytab location from contexts, and cleaning up hadoop/hbase/hive kerberos variables 2016-08-18 12:19:46 -04:00
Scott Aslan 0e90a0f76a [NIFI-2488] update UI to relect changes to ActionDTO and BulletinDTO. This closes #882 2016-08-18 09:38:16 -04:00
Jeff Storck 08735a4f57 NIFI-2488 Wrapping ActionDTO and BulletinDTO in entities to be able to provide readability permission to the client 2016-08-18 09:37:11 -04:00
Mark Payne 567535c3d0 NIFI-2382: Fixed typo in documentation 2016-08-18 09:15:55 -04:00
Mark Payne 91a59a8a5a NIFI-2545: Ensure that when @OnUnscheduled and @OnStopped methods are called that the active thread count takes that thread into account
This closes #836.
2016-08-18 08:33:45 -04:00
Peter Wicks 7bc7b7f683
NIFI-2584 Fixing date format in PutSQL
This closes #878.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-17 16:10:56 -04:00
Matt Gilman 3d1a24ab7d NIFI-2578:
- Ensuring NiFi still attempts to issue a registration request to support 1.x -> 0.x site to site.
2016-08-17 15:35:16 -04:00
Oleg Zhurakousky a0fab15eb7 NIFI-2519 polishing remote address formatting to avoid NPE and other potential exceptions.
This closes #871
2016-08-17 15:13:05 -04:00
Andre F de Miranda e7288af859 NIFI-2519 - ListenSMTP addresses mismatch between smtp.src attribute documentation and attribute.put 2016-08-17 14:58:02 -04:00
Andy LoPresto 0ffdc2eb92
Merge branch 'master' of https://github.com/apache/nifi
(Apologies for merge commits, but it was not possible to rebase the 2574 commits (PR 872)).

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-08-17 10:58:38 -07:00
joewitt b2e468ecf8
NIFI-2574 updated to reflect new accessor methods for nifi properties with kerberos service
This closes #872.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-08-17 10:51:55 -07:00
Peter Wicks a0d1aae603 NIFI-2583
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-2582

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-2582

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #877
2016-08-17 13:20:44 -04:00
Peter Wicks 1470a52b1b NIFI-2582 NIFI-2583
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-2583 Tweaks

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #877
2016-08-17 13:19:56 -04:00
joewitt a5261914fb NIFI-2574 merging latest kerb changes to adjust for NiFiProperties 2016-08-17 09:06:18 -07:00
joewitt 7c99b5e638 NIFI-2574 fixed queue swap threshold setting 2016-08-17 09:05:36 -07:00
Matt Gilman 3f7216ab84 NIFI-2561: - Decoupling kerberos service and spnego principles and keytabs.
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #873
2016-08-17 08:14:05 -04:00
joewitt d9633757a6 NIFI-2574 fix spring context definitions 2016-08-17 03:38:31 -07:00
joewitt 6bf7e7f325 NIFI-2574 merging master with cluster changes to updated NiFiProperties approach 2016-08-17 01:23:54 -07:00
joewitt 7d7401add4 NIFI-2574 Changed NiFiProperties to avoid static initializer and updated all references to it. 2016-08-17 00:10:07 -07:00
Mark Payne e42ea9ad45 NIFI-2566: Refactored to allow just the Leader Election Manager to be responsible for determining who is the Cluster Coordinator
NIFI-2566: Removed storage of cluster roles from heartbeats and NodeConnectionStatus; use LeaderElectionManager to determine roles instead

NIFI-2566: Updated Heartbeats so that if a node is out-of-sync with cluster topology, cluster coordinator will provide updated information back to the nodes

NIFI-2566: Fixed issue that prevented standalone instance from starting by creating a standalone-instance version of the Leader Election Manager. Also added Controller Service enabled/disabled state to fingerprint rather than attempting to update the state when joining the cluster, as the implementation was incorrect and the correct implementation will be a rather significant effort that doesn't have to happen for 1.0.0 release

This closes #866

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-16 20:39:04 -04:00
Matt Gilman b3f36489ad
NIFI-2534: - Refreshing using component IDs. - Code clean up.
This closes #874.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-16 16:05:20 -04:00
Simon Elliston Ball b0122c6a73
NIFI-2465 - InferAvroSchema EL support based on incoming FlowFiles
This closes #863.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-16 12:50:53 -04:00
Scott Aslan 26f5c496d1
NIFI-2534 Fix JS error when stopping a process group
This closes #867.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-16 11:09:41 -04:00
Peter Wicks b22500d0a3
NIFI-2576 Allowing PutSQL to use timestamps in epoch or string format
This closes #869.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-16 10:26:24 -04:00
Matt Burgess 03d3b3961d NIFI-2577: Increased default stripe size in ConvertAvroToORC to 64MB
This closes #870.
2016-08-16 11:53:05 +02:00
Mark Payne 6af768d0a1 NIFI-2551: Addressed a threading issue in the StandardResourceClaimManager and performed some refactoring so that we can ensure thread-safety across different components of the application, such as ProcessSession and WriteAheadFlowFile Repository when interacting with the StandardResourceClaimManager. Update DebugFlow to allow it to write to a FlowFile multiple times, which exposes the concurrency bug. Also avoided calling ContentRepository.remove() from ProcessSession whenever the content is no longer needed, as that can cause problems now that the Resource Claim is backing the content claim. 2016-08-15 17:02:15 -04:00
Matt Burgess ed42f2e3e3 NIFI-2552: Updated Curator for HiveQL processors using Zookeeper quorum in URL
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #865
2016-08-15 16:53:57 -04:00
Matt Gilman 7d8dd27027
NIFI-2554: - Requiring READ permissions on the referenced controller service when creating/updating processors, controller services, and reporting tasks.
- Preventing client side selection of unauthorized controller services unless they were the previously configured value.

This closes #860.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-15 16:46:58 -04:00
Scott Aslan 2a92747046 [NIFI-2559] UI - Controller Service update missing icons
This closes #864
2016-08-15 16:27:12 -04:00
Matt Gilman d45114e48d NIFI-2501: - Ensuring users can access the controller service list regardless of permissions on the corresponding process group or controller. - Better handling the case where the user had write permissions but no read permissions. - Returning to the appropriate controller service listing after going to the usage page.
This closes #835

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-15 15:16:58 -04:00
Oleg Zhurakousky 51c566ffc8
NIFI-2546 fixed NPE in FlowSnippetDTO fixed NPE caused by non-configured ControllerService
This closes #859.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-15 14:55:20 -04:00
Yolanda M. Davis fbd3201157 NIFI-2524 - Fixes to improve handling of missing journal files during rollover/merge execution.Includes:
This closes #840

 Removed partial file check (based on missing first file)
 Added condition to merge if at least one journal files available on disk. If all files are  missing from disk that is considered an error.
 Added retry logic to prevent endless thread execution when encountering errors (such as missing files).
2016-08-15 14:43:38 -04:00
Peter Wicks 44057f097a
NIFI-2356 ConvertJSONToSQL assumes IS_AUTOINCREMENT
This closes #862.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-15 14:17:16 -04:00
Bryan Bende fd0dd51ff5 NIFI-2553 Fixing handling of Paths in HDFS processors
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #843
2016-08-15 08:59:42 -04:00
joewitt 9c7668948c NIFI-2560 This closes #851. fixed property handling and added tests using it 2016-08-14 14:35:25 -04:00
Oleg Zhurakousky b7b56b1cad NIFI-2560 added connection timeout to IMAP/POP3 2016-08-14 14:06:07 -04:00
joewitt d3b96dcac1 NIFI-2519 This closes #856. aligned threading model with subethastmp 2016-08-14 13:32:00 -04:00
Oleg Zhurakousky 48fa76ecff NIFI-2519 Fixed and refactored ListenSMTP processor
- Removed message queueing which could result in data loss
- Fixed life-cycle issues that coudl put processor in an unstable state
- Fixed PropertyDescriptor translation for Time units and Byte sizes
- Fixed broken tests
- Added additional tests

NIFI-2519 added default for SMTP_MAXIMUM_CONNECTIONS

NIFI-2519 addressed PR comments, polishing
- fixed intermittent deadlock on processor stop and added test for it
- the attributes that can not be extracted from the message but available via MessageContext are written into the outgoing FlowFile
- other minor fixes

NIFI-2519 addressed lates PR comments

NIFI-2519 added better messaging when server closes the connection

NIFI-2519 some polishing and additional tests to validate deadlocks

NIFI-2519 address latest PR comments
fixed deadlock condition for when the consumer is stopped while server is distributing messages
fixed MAX message size issue ensuring it is validated
set max connections to SMTPServer
polished pom
added L&N

NIFI-2519 PR comments
- fixed LICENSE
- Added usage of LimitingInputStream
- simplified SmtpConsumer by removing hasMessage operation
2016-08-14 13:31:46 -04:00
Matt Gilman 0855cb9bd4
NIFI-2555:
- Addressing potential NullPointerException when an AccessDeniedException is thrown from an endpoint that isn't subject to the security filter chain.

This closes #846.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-08-12 15:08:00 -07:00
jpercivall 6799bd4919 NIFI-1876 fixing counter URI pattern 2016-08-12 17:23:12 -04:00
Jeff Storck bbdd087d77 NIFI-1876 Implemented PropertyDescriptorDTO merging
NIFI-1876 Updated PermissionsDTO to use boolean primitives instead of Boolean objects for read and write permission
Removed ISE throwing from EntityFactory, it will rely on null checks instead

This closes #694

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-12 17:22:37 -04:00
Matt Gilman e81147c92a NIFI-1876: - Introducing entities for allowable values and process group status.
Signed-off-by: Jeff Storck <jtswork@gmail.com>
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-12 17:22:26 -04:00
Jeff Storck 572dfed78a NIFI-1876 Implements merging of responses to successful requests based on authorization, returning the most restrictive response - Added StandardHttpResponseMergerSpec for testing response merging - Added Permissible interface - Added nifi-api/controller/archive to ProcessGroupEndpointMerger - Removed AbstractMultiEntityEndpoint.java, not used anymore - Implemented reponse merging for GET requests where there are some successful and problematic responses, returning most restrictive one. - Updated nf-settings.js with ControllerConfigurationEntity property rename from controllerConfiguration to component - Implemented merging of status DTOs based on read permission NIFI-2264 Implemented merging of status history results based on readability permissions - Added StatusHistoryEndpointMergerSpec to test merging of status history based on read permission
NIFI-1876 Added merging for labels, funnels, and controller service references.

NIFI-1876 Added Label and Funnel merging to FlowMerger.java

NIFI-1876 Added replication of request for process group controller services
 - Updated merging code for Status and Status History based on read permissions

NIFI-1876 Fixed issue with node status snapshots all looking like they came from one node
Updated ProcessGroupStatusSnapshotDTO to contain status snapshot entities to retain readability permission
Added entity classes for ConnectionStatusSnapshotDTO, PortStatusSnapshotDTO, ProcessGroupStatusSnapshotDTO, ProcessorStatusSnapshotDTO, and RemoteProcessGroupStatusSnapshotDTO
Updated PropertyDescriptorDTO to contain AllowableValueEntity to retain readability permission
Added entity class for AllowableValueDTO
Moved AllowableValueDTO to its own top-level class
Updated DtoFactory to get permissions for status snapshot entities
Updated StatusMerger to merge status snapshot entities

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-12 17:22:17 -04:00
Scott Aslan a9675552d9 [NIFI-2557] Enhance Modal Plugin. This closes #842 2016-08-12 11:40:45 -04:00
Matt Gilman e5e86cf07c NIFI-2548:
- Allowing selection of User/Cluster node when searching and encountering an exact match.

This closes #839

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-08-12 20:41:49 +09:00
Matt Gilman fa639e2596 NIFI-2542: - Ensuring transitive referencing components are able to be returned. - Ensuring we can enable/disable services with transitive referencing components. - Ensuring we cannot enable/disable services with unauthorized referencing components.
NIFI-2543: - Ensuring we have permissions before attempting to reload a controller service.

This closes #837

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-11 20:42:35 -04:00
Mark Payne 25a2fac453
NIFI-2535: Do not include properties that are unset in flow fingerprint. This allows a new property to be added to a processor without affecting the fingerprint, if the value is never set
This closes #829.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-11 10:44:28 -04:00
Mark Payne 76a4a2c48b
NIFI-2544: Created integration tests for clustering and addressed a few minor bugs that were found in doing so
This closes #832.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-11 10:43:38 -04:00
Matt Burgess d544274881
NIFI-2531: Fixed JdbcCommon handling of BigInteger objects for Avro
This closes #823.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-10 15:19:34 -04:00
Matt Burgess 2938a5353a NIFI-1663: Adding attribution for derived work for OrcFlowFileWriter 2016-08-10 14:51:05 -04:00
Mark Payne 85877a73dc NIFI-2540: Exclude from templates the parts of property descriptors that are not necessary. Also ensure that Property Descriptors are not completely removed from Controller Services.
This closes #828
2016-08-10 13:27:20 -04:00
Matt Burgess d9720239f5 NIFI-1663: Add ConvertAvroToORC processor
This closes #727
2016-08-10 12:37:15 -04:00
Joey Frazee bb24312709 NIFI-2142 Added Guava cache for XSLT stylesheets in TransformXml
This closes #609

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-10 11:13:30 -04:00
Matt Burgess 557d6365bf NIFI-2518: Added unit test showing issue
NIFI-2518: Added support for fractional seconds to AbstractDatabaseFetchProcessor

This closes #821
2016-08-10 11:10:17 -04:00
Matt Gilman 219234d001
NIFI-2530: - When no explicit policy is defined, defer to the parent Process Group instead of attempting to match component's within the flow.
This closes #826.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-10 11:06:16 -04:00
joewitt 84cac0daca NIFI-2299 Add standard services api access for scripting processors 2016-08-09 17:15:44 -04:00
Bryan Rosander 2dc094765d NIFI-2516 - Extract version info into parent pom, upgrade to commons-io 2.5
NIFI-2516 - Removing slf4j-api, version from slf4j-log4j12

This closes #809.
2016-08-09 22:43:29 +02:00
Matt Gilman fbb705e461
NIFI-2511: - Ensuring Process Group bulletins are bubbling up as expected.
This closes #822.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-09 16:28:57 -04:00
Tim Reardon 45c31c8305 NIFI-2507 Added Canned ACL support to PutS3Object
This closes #801

Signed-off-by: James Wing <jvwing@gmail.com>
2016-08-09 13:20:52 -07:00
joewitt b2401522ea NIFI-2521 This closes #815. removed test classes with questionable origin and licensing. 2016-08-09 15:46:58 -04:00
Matt Gilman 7a1f749f69 NIFI-2475: - Updating UI to better suggest that component specific administrators do not override higher level administrators.
This closes #813
2016-08-09 15:40:28 -04:00
Matt Burgess 7575e87cb1 NIFI-2520 This closes #818. Added attribution for storm-hive and other Hive dependencies to NOTICEs 2016-08-09 15:27:24 -04:00
Mark Payne 42df02f014 NIFI-2406 This closes #820. Addressed regression introduced in NIFI-2406 where the cluster does not recognize a new Cluster Coordinator when the coordinator is shutdown 2016-08-09 15:19:49 -04:00
Oleg Zhurakousky 18f4150015 NIFI-2515 This closes #814. fixed Kafka serialization/deserialization settings 2016-08-09 14:28:51 -04:00
Andre F de Miranda f6ba922292 NIFI-1965 This closes #819. COnvert DNS_TIMEOUT property to TIME_PERIOD 2016-08-09 14:25:07 -04:00
Matt Gilman ae8045d99b
NIFI-2505: - Updating the user identity field in the ActionDTO.
This closes #800.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-09 13:07:50 -04:00
Matt Gilman aae6325603
NIFI-2237: - Reorganizing the refactored REST endpoints. - Tweaking message text.
This closes #788.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-09 12:04:31 -04:00
Oleg Zhurakousky 5e5c3f7c3b NIFI-2504 This closes #810. fixed template export for RPG
- ensured that RPG and connections to it are treated as special case where instance ID is preserved
2016-08-08 16:54:27 -04:00
Mark Payne ed14bf22e7 NIFI-2493 This closes #798. Do not fingerprint Remote Ports' running state. When synchronizing remote flow with local flow, start/stop remote group ports as appropriate based on the inherited flow 2016-08-08 16:54:27 -04:00
Mark Payne 8752d11f18 NIFI-2495 This closes #808. Ensure that we always close Index Searchers when we're finished with them 2016-08-08 16:54:27 -04:00
Andre F de Miranda d6a2409d71 NIFI-1965 - Implement QueryDNS processor
This closes #496.
2016-08-08 22:27:40 +02:00
Andre F de Miranda 493919922e NIFI-1965 - Rename nifi-geo-bundle to nifi-enrich-bundle 2016-08-08 22:27:22 +02:00
Oleg Zhurakousky 9b647cd538 NIFI-1148 added IMAP/POP3 support added initial set of processors to support consumption of Email via IMAP/IMAPS and POP3 protocols
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-1148 addressed PR comments from @trixpan

NIFI-1148 addressing PR comments

NIFI-1148 addressed PR comments

This closes #710
2016-08-08 11:57:28 -04:00
Mark Payne c1c052af71 NIFI-2406: Ensure that hearbeat monitor continues to run while instance is running. This way if a node sends heartbeat to this node as elected coordinator changes, we notify the node accordingly. Handle Exceptions more gracefully in leader election code. Tweaked some handling of how nodes reconnect to the cluster to ensure more stability with cluster
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #729
2016-08-08 09:15:10 -04:00