Commit Graph

2293 Commits

Author SHA1 Message Date
Jeff Storck 433db23567
NIFI-1799 Implements auto-scaling flow layout
added utility class to scale positions of components on the canvas,
extracted get/setPosition methods from ProcesGroup, RemoteProcessGroup, Label, and Connectable into new interface Positionable
added interface method for finding all Positionables in a ProcessGroup to the ProcessGroup interface and added implementation to StandardProcessGroup
added test flow for position rescaling
added Spock config to POM and a spec for testing the scaling of Positionables
forced Surefire to use JUnit (TestNG was on classpath and Surefire seems to prioritize that over JUnit),
added check in StandardFlowSynchronizer to scale positions only when flow encoding version is less than 1.0
added spec for StandardFlowfileSynchronizer
updated FlowConfiguration.xsd to allow encoding-version attribute
added new test flow used in StandardFlowSynchronizerSpec

This closes #442.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-05-24 14:26:28 -07:00
jpercivall 7923fd04c3 NIFI-1808 Refactored MQTT processors, and added proper unit and integration tests
This closes #392
2016-05-24 13:18:25 -04:00
info@richards-tech.com 7a4c71fec7 Added mqtt
Fixed attribute name in PutMQTT
2016-05-24 13:17:31 -04:00
jpercivall e1b654655a NIFI-1913 Properly removing flowfile in HandleHttpRequest when the Context Map fails to register the request
This closes #462
2016-05-24 10:54:46 -04:00
Bryan Bende 5df67c5dc2 NIFI-1907 Moving lazy init of SSLContext to StandardSiteToSiteClientConfig rather than the builder
This closes #457.
2016-05-24 09:51:18 -04:00
Bryan Bende 714925358e NIFI-1884
- Defining API for Users, Groups, and Policies
- Updating hashCode and equals methods
- Documenting that returned sets are unmodifiable
- Adding builders and unit tests
- Refactoring update methods in MutableAuthorizer to not take a string id
- Refactoring builders to use constructors for seeding
- Fixing toString() methods and cleaning up constructors that don't need to worry about null sets
- Changing MutableAuthorizer to an abstract class and providing implementation of authorize
- This closes #452
2016-05-24 09:24:11 -04:00
James Wing 9bb6dcb5e5 NIFI-786 AWS credential refactoring and enhancements
Signed-off-by: Joe Skora <jskora@gmail.com>
This closes #244.
2016-05-24 00:27:13 -04:00
Matt Gilman 4dd50c80a4 NIFI-1781:
- Including access policies in the breadcrumb's trail.
- Updating toolbox according to group access policies.
- Updating actions in palette based on selection access policies.
NIFI-1554:
- Introducing authorization during two phase commit.
- Introducing snippet authorization according to the encapsulated components and the action performed.
- This closes #461
2016-05-23 14:15:32 -04:00
Andrew Grande d42ea467ca Reviewed, typos.
This closes #460
2016-05-22 10:58:48 -04:00
Oleg Zhurakousky f40a090073 NIFI-1905 enabled ExecuteProcess to terminate process
NIFI-1905 polishing

NIFI-1905 changed WARN to INFO during shutdown
This closes #456
2016-05-22 10:36:03 -04:00
Mark Payne 4b74e4de74 NIFI-1745: Refactor how revisions are handled at NCM/Distributed to Node. This closes #454 2016-05-20 14:04:24 -04:00
DomenicPuzio afc8c645a1 NIFI-1498: putFile - Flowfiles routed to failure relationship produce INFO level log messages
This closes #445.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-05-19 16:21:45 -04:00
Oleg Zhurakousky d6674fbb8e NIFI-1898 Fixed Flume processor @OnScheduled methods to accept ProcessContext
This closes #455.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-05-19 16:07:57 -04:00
Aldrin Piri 1bd2cf0d09 NIFI-1811 Renaming MockProcessorLogger to MockComponentLogger for consistency. Removing unused imports from ExecuteScript causing checkstyle failures. 2016-05-19 14:38:41 -04:00
Pierre Villard 372ffb8aa0 NIFI-1811 Removed ProcessorLog and updated dependent interfaces
This closes #403.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-05-19 13:08:09 -04:00
Oleg Zhurakousky 2d03489ec5 NIFI-1296, NIFI-1680, NIFI-1764, NIFI-1837, NIFI-1827, NIFI-1699 implemented new Kafka processors that leverage Kafka 0.9 API
- Improved StreamScanner for better performance
- Renamed StreamScanner to StreamDemarcator as suggested by Joe
- Added failure handling logic to ensure both processors can be reset to their initial state (as if they were just started)
- Provided comprehensive test suite to validate various aspects of both Publish and Consume from Kafka
- Added relevant javadocs
- Added initial additionalDetails docs
- Addressed NPE reported by NIFI-1764
- Life-cycle refactoring for the existing PutKafka to ensure producer restart after errors
- Incorporated code changes contributed by Ralph Perko (see NIFI-1837)
- Addressed partition issue in RoundRobinPartitioner discussed in NIFI-1827
- Updated PropertyDescriptor descriptions to reflect their purpose

NIFI-1296 added @Ignore on some Kafka tests to improve test time

NIFI-1296 reworked tests to avoid dependency on embedded Kafka

NIFI-1296 fixed spelling error

NIFI-1296 fixed trailing whitespaces in non-java files

This closes #366
2016-05-19 07:33:03 -04:00
Matt Burgess c13fb3159f
NIFI-1822 Enabled pooled ExecuteScript engines to facilitate concurrent processing.
Removed unused variable in unit test. (+10 squashed commits)
Squashed commits:
[7c5acc1] NIFI-1822 Removed trailing whitespace to conform with checkstyle rules.
[cb108cd] NIFI-1822 Added ASF License to unit test.
[9264428] NIFI-1822 Removed debugging log statements for script engine queue size.
Added unit tests demonstrating pooled execution timing and thread usage.
[bdbc4ba] NIFI-1822 Renamed reference to MockProcessorContext#setNumThreads to setMaxConcurrentTasks after refactor.
[12bbe82] NIFI-1822 Moved unit test to correct directory.
Added test script resource which generates flowfile and updates attribute with current thread.
Added tests for single run, serial run, and pooled run (not complete).
[4c174c8] NIFI-1822 Added debugging messages to script execution.
[8ab0ce5] NIFI-1822 Added for loop to instantiate multiple script engines in queue.
[8c5ba51] NIFI-1822 Added variable max concurrent task field in MockProcessorContext because it was previously hardcoded to 1. Changed setNumThreads to setMaxConcurrentTasks to maintain naming convention.
[fd9120c] NIFI-1822 Added unit test skeleton for pooled script processor execution.
[23e4f68] NIFI-1822: Allow concurrent execution in ExecuteScript

This closes #443.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-05-17 14:58:00 -07:00
Mark Payne a81c204565 NIFI-1892: Marked method arguments as final so that they can be referenced in inner classes 2016-05-17 13:31:50 -04:00
Bryan Bende 17c5496c04 NIFI-1858 Adding site-to-site reporting bundle
This closes #436.
2016-05-17 10:38:47 -04:00
Matt Gilman 8bb56fca68 NIFI-1554:
- Introducing ControllerService and ReportingTask authorization and revision.
- Removing the revision where appropriate.
NIFI-1883
- Restoring referencing components automatically.
- This closes #448
2016-05-17 10:32:55 -04:00
rkarthik29 cfd36c5539 fix for 1864
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-05-17 08:38:26 -04:00
Andy LoPresto 3070bd5e13
Added PGP key to KEYS file.
This closes #450.
2016-05-16 20:25:38 -07:00
Pierre Villard fa390112ce NIFI-1213 Added the possibility to register FlowFile assertions in mock framework
This closes #404
2016-05-16 18:58:15 -04:00
Mark Payne 270944ec69 NIFI-1801: Scope Templates to Process Groups. This closes #446. 2016-05-16 16:12:43 -04:00
James Wing 347b281b2d NIFI-1887 Updating default timeout in Admin Guide
This closes #447.

Signed-off-by: James Wing <jvwing@gmail.com>
2016-05-16 11:57:33 -07:00
Pierre Villard b2ce6f1f96 NIFI-1755 Fixed remote process group status counts by only considering connected remote ports
This closes #347
2016-05-16 13:37:51 -04:00
Mark Payne 87d96c0225 NIFI-1872: Allow sufficient time for embedded server to startup for unit tests 2016-05-13 13:28:38 -04:00
Mark Payne 707aeb4231 NIFI-1872: Ignored unit test that sporadically fails 2016-05-13 11:27:54 -04:00
Matt Gilman 3cc16d35ed NIFI-1742:
- Addressing issues when creating a new inline controller service.
- Ensuring controller service referencing components are updated.
- Including revisions and status with each component.
- Dynamically updating component and authorization states.
- This closes #435
2016-05-12 12:41:11 -04:00
Mark Payne 687a686b21 NIFI-1872: Ignore failing unit test for now until we can properly address; since it is on master it's best to ignore, since the problem appears to be the lifecycle of the unit test 2016-05-12 11:57:40 -04:00
Scott Aslan 3c3304aff4 [NIFI-1782] new search UX and add angular control of flow status values. This closes #433 2016-05-11 16:39:38 -04:00
Mark Payne 25e7f314b1 NIFI-1800: Tie Controller Services to Process Groups. This closes #431 2016-05-11 14:56:05 -04:00
Pierre Villard b7aa381ab4 NIFI-1826 Updated documentation
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #396
2016-05-11 10:02:11 -04:00
Pierre Villard 7910e98ba1 NIFI-1826 Expression Language: add function to check enumerator
Signed-off-by: Matt Burgess <mattyb149@apache.org>
2016-05-11 10:01:15 -04:00
Koji Kawamura 7bd2c64adb NIFI-1865 Close StreamThrottler when processor stops.
- Also, replaced copyrighted sample texts for tests

Signed-off-by: Matt Burgess <mattyb149@apache.org>
2016-05-10 22:02:15 -04:00
Yolanda M. Davis 1370eefdd7
NIFI-1812 initial commit for nifi-env script, includes updates to linux/windows scripts, logback changes, RunNifi/Shutdownhook changes
This closes #386.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-05-10 16:37:30 -07:00
Pierre Villard 887648c86f NIFI-1863 extended caught exception in cleaning task for HTTPContextMap
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #428
2016-05-10 13:46:40 -04:00
Scott Aslan cb67520129 [NIFI-1707] upgradeable angular components 2016-05-09 15:16:12 -04:00
Andrew Lim dc4f983c7a
NIFI-1862 User Guide corrections/improvements
Made multiple edits to the User Guide documentation for correcting errors (spelling/grammatical) and improving readability.

This closes #427.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-05-09 10:31:33 -07:00
ijokarumawak 5fb27e608f NIFI-1860 Added ContainerRequestFilter to redirect. This closes #422
- redirect /controller to /site-to-site
2016-05-09 08:17:46 -04:00
James Wing 257590dec3 NIFI-1711 Client-side JS for proxy-friendly URLs. This closes #316 2016-05-08 21:32:34 -04:00
Jeremy Dyer 7a5d53b8d3 NIFI-1747 add maven-jar-plugin to create jar as part of the existing nifi-web-api build process
This closes #340.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-05-08 14:31:25 -04:00
Aldrin Piri b4e4f3eed5 Removing erroneously included files. 2016-05-07 15:01:24 -04:00
Pierre Villard 9064b97631 NIFI-1816 Added provenance event SEND in HandleHttpResponse
This closes #389
2016-05-07 11:51:26 -04:00
Aldrin Piri 993d3cd78f NIFI-1859 Using .equals instead of == for default locale check. Adding some additional information for the input and output schema properties.
This closes #418.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-05-06 23:04:11 -04:00
Pierre Villard 63a0d0c211 NIFI-1732 set jetty timeout to Long.MAX_VALUE
This closes #337
2016-05-06 19:38:52 -04:00
jpercivall e5925ff4b3 Fixing contrib check 2016-05-06 18:02:44 -04:00
Pierre Villard e2bbea1c53 NIFI-1814 Extended caught exceptions
This closes #382
2016-05-06 18:00:40 -04:00
Matt Gilman 9db1def6c6 NIFI-1781:
- Incorporating updated styles to reflect component level authorization.
- Updating canvas for new look and feel.
- This closes #417
2016-05-06 15:50:06 -04:00
Mark Payne 04c41c0654 NIFI-1678:
- Started refactoring heartbeating mechanism, using a new package: org.apache.nifi.cluster.coordination
- Added configuration options for ZooKeeper username & password for heartbeat management. Also addressed issue where nodes that were previously disconnected were asked to disconnect upon restart
- Ensure that ACL is set properly when creating heartbeat node. Removed unused ControllerStartupFailureMessage.java
- Changed ZooKeeper ACL's so that container nodes that would not be sensitive are wide open and removed the usage of username & password when communicating with ZooKeeper. This was done specifically because username/password combination is considered a 'testing' feature that should not be used in production and is not supported by Apache Curator
- Refactored CuratorHeartbeatMonitor into an abstract heartbeat monitor that is responsible for processing heartbeats and CuratorHeartbeatMonitor that is responsible for retrieving heartbeat information
- Refactored so that heartbeats are sent to Cluster Coordinator directly instead of to ZooKeeper. ZooKeeper is used to know which node is the cluster coordinator but heartbeats to the Cluster Coordinator provide additional information about the nodes.
- Started refactoring heartbeating mechanism, using a new package: org.apache.nifi.cluster.coordination
- Added configuration options for ZooKeeper username & password for heartbeat management. Also addressed issue where nodes that were previously disconnected were asked to disconnect upon restart
- Changed ZooKeeper ACL's so that container nodes that would not be sensitive are wide open and removed the usage of username & password when communicating with ZooKeeper. This was done specifically because username/password combination is considered a 'testing' feature that should not be used in production and is not supported by Apache Curator

NIFI-1727:
- Refactored logic for merging HTTP Requests that are federated across cluster

NIFI-1745:
- Refactoring how HTTP Requests are replicated to nodes
- Bug fixes and continuing to work on replication refactoring. Still need to handle cluster locking and revisions
- Begin work on RevisionManager
- Resolved some issues that resulted from rebase
- Fixed URIs to align with new URI's that will be used in 1.0.0
- This closes #413
2016-05-06 15:23:12 -04:00