Commit Graph

806 Commits

Author SHA1 Message Date
James Wing 9bb6dcb5e5 NIFI-786 AWS credential refactoring and enhancements
Signed-off-by: Joe Skora <jskora@gmail.com>
This closes #244.
2016-05-24 00:27:13 -04:00
Matt Gilman 4dd50c80a4 NIFI-1781:
- Including access policies in the breadcrumb's trail.
- Updating toolbox according to group access policies.
- Updating actions in palette based on selection access policies.
NIFI-1554:
- Introducing authorization during two phase commit.
- Introducing snippet authorization according to the encapsulated components and the action performed.
- This closes #461
2016-05-23 14:15:32 -04:00
Oleg Zhurakousky f40a090073 NIFI-1905 enabled ExecuteProcess to terminate process
NIFI-1905 polishing

NIFI-1905 changed WARN to INFO during shutdown
This closes #456
2016-05-22 10:36:03 -04:00
Mark Payne 4b74e4de74 NIFI-1745: Refactor how revisions are handled at NCM/Distributed to Node. This closes #454 2016-05-20 14:04:24 -04:00
DomenicPuzio afc8c645a1 NIFI-1498: putFile - Flowfiles routed to failure relationship produce INFO level log messages
This closes #445.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-05-19 16:21:45 -04:00
Oleg Zhurakousky d6674fbb8e NIFI-1898 Fixed Flume processor @OnScheduled methods to accept ProcessContext
This closes #455.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-05-19 16:07:57 -04:00
Aldrin Piri 1bd2cf0d09 NIFI-1811 Renaming MockProcessorLogger to MockComponentLogger for consistency. Removing unused imports from ExecuteScript causing checkstyle failures. 2016-05-19 14:38:41 -04:00
Pierre Villard 372ffb8aa0 NIFI-1811 Removed ProcessorLog and updated dependent interfaces
This closes #403.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-05-19 13:08:09 -04:00
Oleg Zhurakousky 2d03489ec5 NIFI-1296, NIFI-1680, NIFI-1764, NIFI-1837, NIFI-1827, NIFI-1699 implemented new Kafka processors that leverage Kafka 0.9 API
- Improved StreamScanner for better performance
- Renamed StreamScanner to StreamDemarcator as suggested by Joe
- Added failure handling logic to ensure both processors can be reset to their initial state (as if they were just started)
- Provided comprehensive test suite to validate various aspects of both Publish and Consume from Kafka
- Added relevant javadocs
- Added initial additionalDetails docs
- Addressed NPE reported by NIFI-1764
- Life-cycle refactoring for the existing PutKafka to ensure producer restart after errors
- Incorporated code changes contributed by Ralph Perko (see NIFI-1837)
- Addressed partition issue in RoundRobinPartitioner discussed in NIFI-1827
- Updated PropertyDescriptor descriptions to reflect their purpose

NIFI-1296 added @Ignore on some Kafka tests to improve test time

NIFI-1296 reworked tests to avoid dependency on embedded Kafka

NIFI-1296 fixed spelling error

NIFI-1296 fixed trailing whitespaces in non-java files

This closes #366
2016-05-19 07:33:03 -04:00
Matt Burgess c13fb3159f
NIFI-1822 Enabled pooled ExecuteScript engines to facilitate concurrent processing.
Removed unused variable in unit test. (+10 squashed commits)
Squashed commits:
[7c5acc1] NIFI-1822 Removed trailing whitespace to conform with checkstyle rules.
[cb108cd] NIFI-1822 Added ASF License to unit test.
[9264428] NIFI-1822 Removed debugging log statements for script engine queue size.
Added unit tests demonstrating pooled execution timing and thread usage.
[bdbc4ba] NIFI-1822 Renamed reference to MockProcessorContext#setNumThreads to setMaxConcurrentTasks after refactor.
[12bbe82] NIFI-1822 Moved unit test to correct directory.
Added test script resource which generates flowfile and updates attribute with current thread.
Added tests for single run, serial run, and pooled run (not complete).
[4c174c8] NIFI-1822 Added debugging messages to script execution.
[8ab0ce5] NIFI-1822 Added for loop to instantiate multiple script engines in queue.
[8c5ba51] NIFI-1822 Added variable max concurrent task field in MockProcessorContext because it was previously hardcoded to 1. Changed setNumThreads to setMaxConcurrentTasks to maintain naming convention.
[fd9120c] NIFI-1822 Added unit test skeleton for pooled script processor execution.
[23e4f68] NIFI-1822: Allow concurrent execution in ExecuteScript

This closes #443.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-05-17 14:58:00 -07:00
Bryan Bende 17c5496c04 NIFI-1858 Adding site-to-site reporting bundle
This closes #436.
2016-05-17 10:38:47 -04:00
Matt Gilman 8bb56fca68 NIFI-1554:
- Introducing ControllerService and ReportingTask authorization and revision.
- Removing the revision where appropriate.
NIFI-1883
- Restoring referencing components automatically.
- This closes #448
2016-05-17 10:32:55 -04:00
rkarthik29 cfd36c5539 fix for 1864
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-05-17 08:38:26 -04:00
Mark Payne 270944ec69 NIFI-1801: Scope Templates to Process Groups. This closes #446. 2016-05-16 16:12:43 -04:00
Pierre Villard b2ce6f1f96 NIFI-1755 Fixed remote process group status counts by only considering connected remote ports
This closes #347
2016-05-16 13:37:51 -04:00
Mark Payne 87d96c0225 NIFI-1872: Allow sufficient time for embedded server to startup for unit tests 2016-05-13 13:28:38 -04:00
Mark Payne 707aeb4231 NIFI-1872: Ignored unit test that sporadically fails 2016-05-13 11:27:54 -04:00
Matt Gilman 3cc16d35ed NIFI-1742:
- Addressing issues when creating a new inline controller service.
- Ensuring controller service referencing components are updated.
- Including revisions and status with each component.
- Dynamically updating component and authorization states.
- This closes #435
2016-05-12 12:41:11 -04:00
Mark Payne 687a686b21 NIFI-1872: Ignore failing unit test for now until we can properly address; since it is on master it's best to ignore, since the problem appears to be the lifecycle of the unit test 2016-05-12 11:57:40 -04:00
Scott Aslan 3c3304aff4 [NIFI-1782] new search UX and add angular control of flow status values. This closes #433 2016-05-11 16:39:38 -04:00
Mark Payne 25e7f314b1 NIFI-1800: Tie Controller Services to Process Groups. This closes #431 2016-05-11 14:56:05 -04:00
Koji Kawamura 7bd2c64adb NIFI-1865 Close StreamThrottler when processor stops.
- Also, replaced copyrighted sample texts for tests

Signed-off-by: Matt Burgess <mattyb149@apache.org>
2016-05-10 22:02:15 -04:00
Yolanda M. Davis 1370eefdd7
NIFI-1812 initial commit for nifi-env script, includes updates to linux/windows scripts, logback changes, RunNifi/Shutdownhook changes
This closes #386.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-05-10 16:37:30 -07:00
Pierre Villard 887648c86f NIFI-1863 extended caught exception in cleaning task for HTTPContextMap
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #428
2016-05-10 13:46:40 -04:00
Scott Aslan cb67520129 [NIFI-1707] upgradeable angular components 2016-05-09 15:16:12 -04:00
ijokarumawak 5fb27e608f NIFI-1860 Added ContainerRequestFilter to redirect. This closes #422
- redirect /controller to /site-to-site
2016-05-09 08:17:46 -04:00
James Wing 257590dec3 NIFI-1711 Client-side JS for proxy-friendly URLs. This closes #316 2016-05-08 21:32:34 -04:00
Jeremy Dyer 7a5d53b8d3 NIFI-1747 add maven-jar-plugin to create jar as part of the existing nifi-web-api build process
This closes #340.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-05-08 14:31:25 -04:00
Aldrin Piri b4e4f3eed5 Removing erroneously included files. 2016-05-07 15:01:24 -04:00
Pierre Villard 9064b97631 NIFI-1816 Added provenance event SEND in HandleHttpResponse
This closes #389
2016-05-07 11:51:26 -04:00
Aldrin Piri 993d3cd78f NIFI-1859 Using .equals instead of == for default locale check. Adding some additional information for the input and output schema properties.
This closes #418.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-05-06 23:04:11 -04:00
Pierre Villard 63a0d0c211 NIFI-1732 set jetty timeout to Long.MAX_VALUE
This closes #337
2016-05-06 19:38:52 -04:00
jpercivall e5925ff4b3 Fixing contrib check 2016-05-06 18:02:44 -04:00
Pierre Villard e2bbea1c53 NIFI-1814 Extended caught exceptions
This closes #382
2016-05-06 18:00:40 -04:00
Matt Gilman 9db1def6c6 NIFI-1781:
- Incorporating updated styles to reflect component level authorization.
- Updating canvas for new look and feel.
- This closes #417
2016-05-06 15:50:06 -04:00
Mark Payne 04c41c0654 NIFI-1678:
- Started refactoring heartbeating mechanism, using a new package: org.apache.nifi.cluster.coordination
- Added configuration options for ZooKeeper username & password for heartbeat management. Also addressed issue where nodes that were previously disconnected were asked to disconnect upon restart
- Ensure that ACL is set properly when creating heartbeat node. Removed unused ControllerStartupFailureMessage.java
- Changed ZooKeeper ACL's so that container nodes that would not be sensitive are wide open and removed the usage of username & password when communicating with ZooKeeper. This was done specifically because username/password combination is considered a 'testing' feature that should not be used in production and is not supported by Apache Curator
- Refactored CuratorHeartbeatMonitor into an abstract heartbeat monitor that is responsible for processing heartbeats and CuratorHeartbeatMonitor that is responsible for retrieving heartbeat information
- Refactored so that heartbeats are sent to Cluster Coordinator directly instead of to ZooKeeper. ZooKeeper is used to know which node is the cluster coordinator but heartbeats to the Cluster Coordinator provide additional information about the nodes.
- Started refactoring heartbeating mechanism, using a new package: org.apache.nifi.cluster.coordination
- Added configuration options for ZooKeeper username & password for heartbeat management. Also addressed issue where nodes that were previously disconnected were asked to disconnect upon restart
- Changed ZooKeeper ACL's so that container nodes that would not be sensitive are wide open and removed the usage of username & password when communicating with ZooKeeper. This was done specifically because username/password combination is considered a 'testing' feature that should not be used in production and is not supported by Apache Curator

NIFI-1727:
- Refactored logic for merging HTTP Requests that are federated across cluster

NIFI-1745:
- Refactoring how HTTP Requests are replicated to nodes
- Bug fixes and continuing to work on replication refactoring. Still need to handle cluster locking and revisions
- Begin work on RevisionManager
- Resolved some issues that resulted from rebase
- Fixed URIs to align with new URI's that will be used in 1.0.0
- This closes #413
2016-05-06 15:23:12 -04:00
Scott Aslan c527f64230 [NIFI-1762] Remove bower and npm from convenience binaries. This closes #412. 2016-05-04 15:23:11 -04:00
Adam Lamar ddf8c6f8fa NIFI-840: Create ListS3 processor
This closes #238

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-05-04 12:15:45 -04:00
Scott Aslan 2824f5a23c NIFI-1782 cleanup whitespace and code styles
This closes #411
2016-05-03 15:32:49 -04:00
Yolanda M. Davis defb6f5b61 NIFI-361 - Create Processors to mutate JSON data
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #354
2016-05-03 14:38:11 -04:00
Mark Payne e3bdee8b1e NIFI-1824: If attempting to archive content, and there are no claimant counts for it, ensure that the stream is closed. 2016-05-03 14:27:13 -04:00
Scott Aslan 1df8fe44c4 [NIFI-1782] update toolbox, new global menu, and new pallattes...also introduced frontend-maven-plugin to allow bower to manage pulling in 3rd party libs at build time and not shipped with the source. This closes #395 2016-05-03 14:20:46 -04:00
jpercivall 0557095613 NIFI-1028 initial commit of NiFi In Depth documentation
NIFI-1028 addressing review comments
This closes #339
2016-05-03 14:01:45 -04:00
Matt Burgess 106b0fa0fc NIFI-981: Added SelectHiveQL and PutHiveQL processors
This closes #410.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-05-03 13:51:38 -04:00
Devin Fisher 8336014064 NIFI-1560:
- Fixing a copy and paste error
- Looks like when the original coder copied code from AuthenticationStrategy for the ReferralStrategy and did not change this reference for the error case.
- Also addressing the typo in the message.
- This closes #402
2016-05-03 08:00:34 -04:00
Mark Payne 45ca978498 NIFI-1830: Fixed problems in the merging logic for Drop FlowFile Requests. This closes #394 2016-05-02 08:46:50 -04:00
Matt Gilman ff98d823e2 NIFI-1554:
- Populating component entities in the REST API to decouple key fields from the configuration DTOs.
- Added initial support for components in UI when access isn't allowed. Formal styling to come later.
2016-04-29 14:49:14 -04:00
Andy LoPresto a4409d366c
Fixed typo in ScryptCipherProviderGroovyTest Javadoc.
This closes #380.
2016-04-27 16:05:16 -07:00
Aldrin Piri 11cff0dd39
NIFI-1818 Adjusting exceptions to reflect the appropriate repository instantiation that caused the issue.
This closes #385.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-04-27 15:52:54 -07:00
Aldrin Piri 6938c3db54 NIFI-1817 Respecting when run.as is commented out by ensuring the chosen line starts with run.as optionally preceded by whitespace
This closes #383.
2016-04-27 15:22:19 -04:00