Commit Graph

2415 Commits

Author SHA1 Message Date
Matt Burgess b32c70c419 NIFI-1929: Improvements for PutHDFS attribute handling
This closes #486.
2016-06-06 11:58:55 +02:00
Bryan Bende 8d8a9cba79 NIFI-1916 Updating FileAuthorizer to extend AbstractPolicyBasedAuthorizer and adding intial loading of data users, groups, and policies
- Implementing CRUD operations and unit tests for Users
- Implementing CRUD operations and unit tests for Groups
- Implementing CRUD operations and unit tests for AccessPolicies
- Adding support for seeding with an initial admin user
- Fixing delete for user and group so it removes references from policies
- Adding example to authorizations.xml
- Adding back the old users schema in preparation for auto-converting to the new format, and providing the AuthorizationConfigurationContext with access to the root process group id
- Refactoring some of the FileAuthorizer to ensure thread safety
- Adding /groups to policies created for initial admin
- This closes #473
2016-06-03 17:26:22 -04:00
Matt Gilman 806f4d549d NIFI-1951:
- Removing deprecated NiFiWebContext and related classes.
- Adding authorization to Custom UIs.
- Fixing issue when creating ControllerService inline.
- Addressing contentType issue when attempting to clear component state.
- This closes #489
2016-06-03 16:52:02 -04:00
Mark Payne df0e4e7960 NIFI-1897:
- Refactoring to allow requests to be replicated from a node to other nodes
- Renaming cluster node connection/read timeout properties.
- Renaming NCM DN to Cluster Coordinator DN.
- Fixing default values in properties.
- Starting to fix Spring context to load correctly in standalone mode.
- Using the cluster protocol to handle connection failures instead of heartbeats.
- Ensured replicate call is returned from ControllerResource.
- Ensure the appropriate classloader when serializing templates.
- Handling when the flow contents are null.
- This closes #488
2016-06-03 15:29:16 -04:00
Mark Payne cd011731ab NIFI-1960: Update admin guide regarding documentation for clustering
NIFI-1960: Updates to guide as follow-up from PR review
2016-06-03 15:23:39 -04:00
Andre F de Miranda 5a8979150c NIFI-856 - Implements ListenLumberjack Processor
This closes #290.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-06-02 21:58:41 -04:00
jpercivall 54c5c28cbb NIFI-1943 added proper ReadsAttribute to PutFile
This closes #480
2016-06-02 13:04:14 -04:00
Matt Gilman f0f74fe404 NIFI-1928:
- Fixing UI action for bulk delete.
- Removing registration form.
- Fixing default visibility of the anonymous user warning.
2016-06-02 11:56:10 -04:00
Matt Gilman 950e0cfa58 NIFI-1265:
- Upgrading the jspc maven plugin.
2016-06-02 09:49:57 -04:00
jpercivall 4f101294eb NIFI-1265 Fixing InvokeHttp tests.
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-06-02 09:49:57 -04:00
Matt Gilman e587c99855 NIFI-1265:
- Upgrading to Jetty 9.3.
2016-06-02 09:49:57 -04:00
Jeff Storck 3e064101e9 NIFI-1948 Fixed encoding-version attribute not getting created in exported templates. This closes #485 2016-06-02 09:29:11 -04:00
Andy LoPresto 8127314975
NIFI-1919 Added replaceFirst() expression language method which accepts literal or pattern for replacement.
Reverted whitespace changes. (+8 squashed commits)
Squashed commits:
[329755c] NIFI-1919 Reverted import re-organization from IDE.
[cf73c2f] NIFI-1919 Updated expression language guide.
[d9a1455] NIFI-1919 Reverted changes to ReplaceEvaluator.
Added ReplaceFirstEvaluator.
Added replace first logic to Query buildFunctionEvaluator.
Added unit tests.
[e2eb880] NIFI-1919 Added replaceFirst to AttributeExpression lexer and parser grammar definitions.
[11fe913] NIFI-1919 Ignored demonstrative test for replaceAll as it behaves as expected.
[af97be1] NIFI-1919 Changed ReplaceEvaluator to use String#replaceFirst which interprets regex instead of compiling as literal.
Demonstrative unit test now passes but two existing unit tests fail. I am not sure these tests are correct.
[f24f17b] NIFI-1919 Added working unit test to illustrate fix.
[8a0d43b] NIFI-1919 Added Groovy unit test to demonstrate issue.
Added DelegatingMetaClass code to record it (test not complete).

This closes #474.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-06-01 16:39:58 -07:00
Andre F de Miranda bcead3500d NIFI-1945 - Corrects minor issue in "<Hash Attribute Name>" description
This closes #481.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
2016-06-01 21:13:59 +02:00
Oleg Zhurakousky 93795bc7ba NIFI-1915 Fixed '$' escape logic.
Fixed '$' escape logic (if not followed by number) to use Matcher.quoteReplacement(java.lang.String) to
suppress its special meaning so it could be treated as literal.

This closes #466.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
2016-06-01 20:17:59 +02:00
Joe Skora c6af6be44c NIFI-1754 Rollback log messages should include the flowfile filename and UUID to assist in flow management. Incorporates debug logging into StandardProcessSession.rollback() to list Flowfile records retreived from the session.
Reviewed with slight style amendment (see Jira for details) by Tony Kurc (tkurc@apache.org). This closes #478.
2016-06-01 08:43:54 -07:00
jpercivall f47af1ce83 NIFI-1808 Adding proper license information to Nar and provenance reporting to PublishMQTT
NIFI-1808 changing ConsumeMQTT messageArrived() logging

This closes #484
2016-06-01 11:37:06 -04:00
Jeff Storck 893daf567d NIFI-1908 Added encoding-version attribute to TemplateDTO
added scaling of templates upon instantiation for placement on the canvas
added template-0.7.0.xml for live-testing the import of templates

Fixing issue with potentially uninitialized RemoteGroupPorts in copySnippet.

This closes #471
2016-05-30 14:40:52 -04:00
Matt Gilman b075f238a5 NIFI-1564:
- Addressing issue referencing the global jQuery variable in the content viewer.
- This closes #421
2016-05-26 15:23:29 -04:00
Matt Gilman a0ff2f7a9f NIFI-1554:
- Refactoring Cluster endpoints.
- Updating frontend to accomodate clustering endpoints.
- Remove the 'Make Primary' action.
- This closes #470
2016-05-26 14:28:24 -04:00
Matt Gilman 9152a9fdbb NIFI-1800:
- UI style updates to make the components stand out better.
- Reusing controller service table in different contexts (controller, process group, etc).
- This closes #469
2016-05-26 14:15:56 -04:00
Andy LoPresto ae3012a992 NIFI-1918 Ignored failing test because timing is not deterministic and should not be used in unit test.
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #465
2016-05-24 22:43:17 -04:00
Jeff Storck 433db23567
NIFI-1799 Implements auto-scaling flow layout
added utility class to scale positions of components on the canvas,
extracted get/setPosition methods from ProcesGroup, RemoteProcessGroup, Label, and Connectable into new interface Positionable
added interface method for finding all Positionables in a ProcessGroup to the ProcessGroup interface and added implementation to StandardProcessGroup
added test flow for position rescaling
added Spock config to POM and a spec for testing the scaling of Positionables
forced Surefire to use JUnit (TestNG was on classpath and Surefire seems to prioritize that over JUnit),
added check in StandardFlowSynchronizer to scale positions only when flow encoding version is less than 1.0
added spec for StandardFlowfileSynchronizer
updated FlowConfiguration.xsd to allow encoding-version attribute
added new test flow used in StandardFlowSynchronizerSpec

This closes #442.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-05-24 14:26:28 -07:00
jpercivall 7923fd04c3 NIFI-1808 Refactored MQTT processors, and added proper unit and integration tests
This closes #392
2016-05-24 13:18:25 -04:00
info@richards-tech.com 7a4c71fec7 Added mqtt
Fixed attribute name in PutMQTT
2016-05-24 13:17:31 -04:00
jpercivall e1b654655a NIFI-1913 Properly removing flowfile in HandleHttpRequest when the Context Map fails to register the request
This closes #462
2016-05-24 10:54:46 -04:00
Bryan Bende 5df67c5dc2 NIFI-1907 Moving lazy init of SSLContext to StandardSiteToSiteClientConfig rather than the builder
This closes #457.
2016-05-24 09:51:18 -04:00
Bryan Bende 714925358e NIFI-1884
- Defining API for Users, Groups, and Policies
- Updating hashCode and equals methods
- Documenting that returned sets are unmodifiable
- Adding builders and unit tests
- Refactoring update methods in MutableAuthorizer to not take a string id
- Refactoring builders to use constructors for seeding
- Fixing toString() methods and cleaning up constructors that don't need to worry about null sets
- Changing MutableAuthorizer to an abstract class and providing implementation of authorize
- This closes #452
2016-05-24 09:24:11 -04:00
James Wing 9bb6dcb5e5 NIFI-786 AWS credential refactoring and enhancements
Signed-off-by: Joe Skora <jskora@gmail.com>
This closes #244.
2016-05-24 00:27:13 -04:00
Matt Gilman 4dd50c80a4 NIFI-1781:
- Including access policies in the breadcrumb's trail.
- Updating toolbox according to group access policies.
- Updating actions in palette based on selection access policies.
NIFI-1554:
- Introducing authorization during two phase commit.
- Introducing snippet authorization according to the encapsulated components and the action performed.
- This closes #461
2016-05-23 14:15:32 -04:00
Andrew Grande d42ea467ca Reviewed, typos.
This closes #460
2016-05-22 10:58:48 -04:00
Oleg Zhurakousky f40a090073 NIFI-1905 enabled ExecuteProcess to terminate process
NIFI-1905 polishing

NIFI-1905 changed WARN to INFO during shutdown
This closes #456
2016-05-22 10:36:03 -04:00
Mark Payne 4b74e4de74 NIFI-1745: Refactor how revisions are handled at NCM/Distributed to Node. This closes #454 2016-05-20 14:04:24 -04:00
DomenicPuzio afc8c645a1 NIFI-1498: putFile - Flowfiles routed to failure relationship produce INFO level log messages
This closes #445.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-05-19 16:21:45 -04:00
Oleg Zhurakousky d6674fbb8e NIFI-1898 Fixed Flume processor @OnScheduled methods to accept ProcessContext
This closes #455.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-05-19 16:07:57 -04:00
Aldrin Piri 1bd2cf0d09 NIFI-1811 Renaming MockProcessorLogger to MockComponentLogger for consistency. Removing unused imports from ExecuteScript causing checkstyle failures. 2016-05-19 14:38:41 -04:00
Pierre Villard 372ffb8aa0 NIFI-1811 Removed ProcessorLog and updated dependent interfaces
This closes #403.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-05-19 13:08:09 -04:00
Oleg Zhurakousky 2d03489ec5 NIFI-1296, NIFI-1680, NIFI-1764, NIFI-1837, NIFI-1827, NIFI-1699 implemented new Kafka processors that leverage Kafka 0.9 API
- Improved StreamScanner for better performance
- Renamed StreamScanner to StreamDemarcator as suggested by Joe
- Added failure handling logic to ensure both processors can be reset to their initial state (as if they were just started)
- Provided comprehensive test suite to validate various aspects of both Publish and Consume from Kafka
- Added relevant javadocs
- Added initial additionalDetails docs
- Addressed NPE reported by NIFI-1764
- Life-cycle refactoring for the existing PutKafka to ensure producer restart after errors
- Incorporated code changes contributed by Ralph Perko (see NIFI-1837)
- Addressed partition issue in RoundRobinPartitioner discussed in NIFI-1827
- Updated PropertyDescriptor descriptions to reflect their purpose

NIFI-1296 added @Ignore on some Kafka tests to improve test time

NIFI-1296 reworked tests to avoid dependency on embedded Kafka

NIFI-1296 fixed spelling error

NIFI-1296 fixed trailing whitespaces in non-java files

This closes #366
2016-05-19 07:33:03 -04:00
Matt Burgess c13fb3159f
NIFI-1822 Enabled pooled ExecuteScript engines to facilitate concurrent processing.
Removed unused variable in unit test. (+10 squashed commits)
Squashed commits:
[7c5acc1] NIFI-1822 Removed trailing whitespace to conform with checkstyle rules.
[cb108cd] NIFI-1822 Added ASF License to unit test.
[9264428] NIFI-1822 Removed debugging log statements for script engine queue size.
Added unit tests demonstrating pooled execution timing and thread usage.
[bdbc4ba] NIFI-1822 Renamed reference to MockProcessorContext#setNumThreads to setMaxConcurrentTasks after refactor.
[12bbe82] NIFI-1822 Moved unit test to correct directory.
Added test script resource which generates flowfile and updates attribute with current thread.
Added tests for single run, serial run, and pooled run (not complete).
[4c174c8] NIFI-1822 Added debugging messages to script execution.
[8ab0ce5] NIFI-1822 Added for loop to instantiate multiple script engines in queue.
[8c5ba51] NIFI-1822 Added variable max concurrent task field in MockProcessorContext because it was previously hardcoded to 1. Changed setNumThreads to setMaxConcurrentTasks to maintain naming convention.
[fd9120c] NIFI-1822 Added unit test skeleton for pooled script processor execution.
[23e4f68] NIFI-1822: Allow concurrent execution in ExecuteScript

This closes #443.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-05-17 14:58:00 -07:00
Mark Payne a81c204565 NIFI-1892: Marked method arguments as final so that they can be referenced in inner classes 2016-05-17 13:31:50 -04:00
Bryan Bende 17c5496c04 NIFI-1858 Adding site-to-site reporting bundle
This closes #436.
2016-05-17 10:38:47 -04:00
Matt Gilman 8bb56fca68 NIFI-1554:
- Introducing ControllerService and ReportingTask authorization and revision.
- Removing the revision where appropriate.
NIFI-1883
- Restoring referencing components automatically.
- This closes #448
2016-05-17 10:32:55 -04:00
rkarthik29 cfd36c5539 fix for 1864
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-05-17 08:38:26 -04:00
Andy LoPresto 3070bd5e13
Added PGP key to KEYS file.
This closes #450.
2016-05-16 20:25:38 -07:00
Pierre Villard fa390112ce NIFI-1213 Added the possibility to register FlowFile assertions in mock framework
This closes #404
2016-05-16 18:58:15 -04:00
Mark Payne 270944ec69 NIFI-1801: Scope Templates to Process Groups. This closes #446. 2016-05-16 16:12:43 -04:00
James Wing 347b281b2d NIFI-1887 Updating default timeout in Admin Guide
This closes #447.

Signed-off-by: James Wing <jvwing@gmail.com>
2016-05-16 11:57:33 -07:00
Pierre Villard b2ce6f1f96 NIFI-1755 Fixed remote process group status counts by only considering connected remote ports
This closes #347
2016-05-16 13:37:51 -04:00
Mark Payne 87d96c0225 NIFI-1872: Allow sufficient time for embedded server to startup for unit tests 2016-05-13 13:28:38 -04:00
Mark Payne 707aeb4231 NIFI-1872: Ignored unit test that sporadically fails 2016-05-13 11:27:54 -04:00