Apache
/
nifi
mirror of https://github.com/apache/nifi.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

NIFI-259: Adjusting property description of the connect string to be more specific concerning socket tuple format and the associated default port if one is not specified. Providing minor cleanup and removing unused import.

This commit is contained in:
Aldrin Piri 2016-01-28 18:22:13 -06:00
parent 329e1fe2f4
commit 6902812678
6 changed files with 34 additions and 31 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
nifi-api/src/main/java/org/apache/nifi/annotation/behavior/Stateful.java
View File

@ -50,5 +50,5 @@ public @interface Stateful {
/** /**
* Indicates the Scope(s) associated with the State that is stored and retrieved. * Indicates the Scope(s) associated with the State that is stored and retrieved.
*/ */
Scope[]scopes(); Scope[] scopes();
} }

3
nifi-commons/nifi-properties/src/main/java/org/apache/nifi/util/NiFiProperties.java
View File

@ -1000,9 +1000,6 @@ public class NiFiProperties extends Properties {
return new File(getProperty(STATE_MANAGEMENT_CONFIG_FILE, DEFAULT_STATE_MANAGEMENT_CONFIG_FILE)); return new File(getProperty(STATE_MANAGEMENT_CONFIG_FILE, DEFAULT_STATE_MANAGEMENT_CONFIG_FILE));
} }
/*
* public static final String STATE_MANAGEMENT_MAX_ZOOKEEPER_SERVERS = "nifi.state.management.embedded.zookeeper.max.instances";
*/
public String getLocalStateProviderId() { public String getLocalStateProviderId() {
return getProperty(STATE_MANAGEMENT_LOCAL_PROVIDER_ID); return getProperty(STATE_MANAGEMENT_LOCAL_PROVIDER_ID);
} }

16
nifi-docs/src/main/asciidoc/administration-guide.adoc
View File

@ -400,12 +400,14 @@ may be referenced by their identifiers. By default, the Local State Provider is
persists the data to the _$NIFI_HOME/state_ directory. The default Cluster State Provider is configured to be a `ZooKeeperStateProvider`. persists the data to the _$NIFI_HOME/state_ directory. The default Cluster State Provider is configured to be a `ZooKeeperStateProvider`.
The default ZooKeeper-based provider must have its `Connect String` property populated before it can be used. It is also advisable, The default ZooKeeper-based provider must have its `Connect String` property populated before it can be used. It is also advisable,
if multiple NiFi instances will use the same ZooKeeper instance, that the value of the `Root Node` property be changed. For instance, if multiple NiFi instances will use the same ZooKeeper instance, that the value of the `Root Node` property be changed. For instance,
one might set the value to `/nifi/<team name>/production`. one might set the value to `/nifi/<team name>/production`. A `Connect String` takes the form of comma separated <host>:<port> tuples,
such as my-zk-server1:2181,my-zk-server2:2181,my-zk-server3:2181. In the event a port is not specified for any of the hosts, the ZooKeeper
default of 2181 is assumed.
When adding data to ZooKeeper, there are two options for Access Control: `Open` and `CreatorOnly`. If the `Access Control` property is When adding data to ZooKeeper, there are two options for Access Control: `Open` and `CreatorOnly`. If the `Access Control` property is
set to `Open`, then anyone is allowed to log into ZooKeeper and have full permissions to see, change, delete, or administer the data. set to `Open`, then anyone is allowed to log into ZooKeeper and have full permissions to see, change, delete, or administer the data.
If `CreatorOnly` is specified, then only user that is allowed to read, change, delete, or administer the data is the user that was used If `CreatorOnly` is specified, then only the user that created the data is allowed to read, change, delete, or administer the data.
to create the data. In order to use the `CreatorOnly` option, NiFi must provide some form of authentication. See the <<zk_access_control>> In order to use the `CreatorOnly` option, NiFi must provide some form of authentication. See the <<zk_access_control>>
section below for more information on how to configure authentication. section below for more information on how to configure authentication.
If NiFi is configured to run in a standalone mode, the `cluster-state-provider` element need not be populated in the _state-management.xml_ If NiFi is configured to run in a standalone mode, the `cluster-state-provider` element need not be populated in the _state-management.xml_
@ -427,7 +429,7 @@ instead of the Local State Provider.
[[embedded_zookeeper]] [[embedded_zookeeper]]
=== Embedded ZooKeeper Server === Embedded ZooKeeper Server
As mentioned above, the default State Provider for cluster-wide state is the `ZooKeeperStateProvider`. At the time of this writing, this is the As mentioned above, the default State Provider for cluster-wide state is the `ZooKeeperStateProvider`. At the time of this writing, this is the
only State Provider that exists for handling cluster-wide state. What this means is that NiFi has a dependencies on ZooKeeper in order to only State Provider that exists for handling cluster-wide state. What this means is that NiFi has dependencies on ZooKeeper in order to
behave as a cluster. However, there are many environments in which NiFi is deployed where there is no existing ZooKeeper ensemble being maintained. behave as a cluster. However, there are many environments in which NiFi is deployed where there is no existing ZooKeeper ensemble being maintained.
In order to avoid the burden of forcing administrators to also maintain a separate ZooKeeper instance, NiFi provides the option of starting an In order to avoid the burden of forcing administrators to also maintain a separate ZooKeeper instance, NiFi provides the option of starting an
embedded ZooKeeper server. embedded ZooKeeper server.
@ -439,7 +441,7 @@ running ZooKeeper on 4 nodes provides no more benefit than running on 3 nodes, Z
However, it is up to the administrator to determine the number of nodes most appropriate to the particular deployment of NiFi. However, it is up to the administrator to determine the number of nodes most appropriate to the particular deployment of NiFi.
If the `nifi.state.management.embedded.zookeeper.start` property is set to `true`, the `nifi.state.management.embedded.zookeeper.properties` property If the `nifi.state.management.embedded.zookeeper.start` property is set to `true`, the `nifi.state.management.embedded.zookeeper.properties` property
in _nifi.properties_ also becomes relevant. This specifies the ZooKeeper properties file to use. At a minimum, This properties file needs to be populated in _nifi.properties_ also becomes relevant. This specifies the ZooKeeper properties file to use. At a minimum, this properties file needs to be populated
with the list of ZooKeeper servers. Each of these servers is configured as <hostname>:<client port>[:<leader election port>]. For example, `myhost:2888:3888`. with the list of ZooKeeper servers. Each of these servers is configured as <hostname>:<client port>[:<leader election port>]. For example, `myhost:2888:3888`.
This list of nodes should be the same nodes in the NiFi cluster that have the `nifi.state.management.embedded.zookeeper.start` This list of nodes should be the same nodes in the NiFi cluster that have the `nifi.state.management.embedded.zookeeper.start`
property set to `true`. Also note that because ZooKeeper will be listening on these ports, the firewall may need to be configured to open these ports property set to `true`. Also note that because ZooKeeper will be listening on these ports, the firewall may need to be configured to open these ports
@ -486,7 +488,7 @@ The second option is to use a user name and password. This is configured by spec
for the `ZooKeeperStateProvider` (see the <<state_providers>> section for more information). The important thing to keep in mind here, though, is that ZooKeeper for the `ZooKeeperStateProvider` (see the <<state_providers>> section for more information). The important thing to keep in mind here, though, is that ZooKeeper
will pass around the password in plain text. This means that using a user name and password should not be used unless ZooKeeper is running on localhost as a will pass around the password in plain text. This means that using a user name and password should not be used unless ZooKeeper is running on localhost as a
one-instance cluster, or if communications with ZooKeeper occur only over encrypted communications, such as a VPN or an SSL connection. ZooKeeper will be one-instance cluster, or if communications with ZooKeeper occur only over encrypted communications, such as a VPN or an SSL connection. ZooKeeper will be
providing support for SSL connections in version 0.5.0. providing support for SSL connections in version 3.5.0.
@ -495,7 +497,7 @@ providing support for SSL connections in version 0.5.0.
When NiFi communicates with ZooKeeper, all communications, by default, are non-secure, and anyone who logs into ZooKeeper is able to view and manipulate all When NiFi communicates with ZooKeeper, all communications, by default, are non-secure, and anyone who logs into ZooKeeper is able to view and manipulate all
of the NiFi state that is stored in ZooKeeper. To prevent this, we can use Kerberos to manage the authentication. At this time, ZooKeeper does not provide of the NiFi state that is stored in ZooKeeper. To prevent this, we can use Kerberos to manage the authentication. At this time, ZooKeeper does not provide
support for encryption via SSL. Support for SSL in ZooKeeper is being actively worked and is expected to be available in the 0.5.x version of ZooKeeper. support for encryption via SSL. Support for SSL in ZooKeeper is being actively developed and is expected to be available in the 3.5.x release version.
In order to secure the communications, we need to ensure that both the client and the server support the same configuration. Instructions for configuring the In order to secure the communications, we need to ensure that both the client and the server support the same configuration. Instructions for configuring the
NiFi ZooKeeper client and embedded ZooKeeper server to use Kerberos are provided below. NiFi ZooKeeper client and embedded ZooKeeper server to use Kerberos are provided below.

6
nifi-docs/src/main/asciidoc/developer-guide.adoc
View File

@ -596,10 +596,10 @@ that the instance is clustered and write the code accordingly.
State is stored using the StateManager's `set`, `replace`, `putIfAbsent`, `remove`, and `clear` methods. All of these methods, State is stored using the StateManager's `set`, `replace`, `putIfAbsent`, `remove`, and `clear` methods. All of these methods,
with the exception of `clear` take as the first argument the key to be set. The key that is used is unique only to the same with the exception of `clear` take as the first argument the key to be set. The key that is used is unique only to the same
instance of the component and for the same Scope. That is, if two Processors store a value using the key _My Key_, those Processors instance of the component and for the same Scope. That is, if two Processors store a value using the key _My Key_, those Processors
will not conflict with each other, even if both Processors are of the same type (e.g., both are of type ListFile). Furthermore, will not conflict with each other, even if both Processors are of the same type (e.g., both are of type ListFile) and scope. Furthermore,
if a Processor stores a value with the key of _My Key_ using the `Scope.CLUSTER` scope, and then attempts to retrieve the value if a Processor stores a value with the key of _My Key_ using the `Scope.CLUSTER` scope, and then attempts to retrieve the value
using the `Scope.NODE` scope, the value retrieved will be `null`. Each Processor's state, then, is stored in isolation from other using the `Scope.NODE` scope, the value retrieved will be `null`. Each Processor's state, then, is stored in isolation from other
Processor's state. A unique key can be thought of as a triple of <Processor Instance, Key, Scope>. Processors' state. A unique key can be thought of as a triple of <Processor Instance, Key, Scope>.
It follows, then, that two Processors cannot share the same state. There are, however, some circumstances in which it is very It follows, then, that two Processors cannot share the same state. There are, however, some circumstances in which it is very
necessary to share state between two Processors of different types, or two Processors of the same type. This can be accomplished necessary to share state between two Processors of different types, or two Processors of the same type. This can be accomplished
@ -793,7 +793,7 @@ public final class InvokeHTTP extends AbstractProcessor {
---- ----
=== Documenting Related Components === Documenting Related Components
Often Processors and ControllerServices are related to one another. Sometimes its a put/get relation as in `PutFile` and `GetFile`. Often Processors and ControllerServices are related to one another. Sometimes it is a put/get relation as in `PutFile` and `GetFile`.
Sometimes a Processor uses a ControllerService like `InvokeHTTP` and `StandardSSLContextService`. Sometimes one ControllerService uses another Sometimes a Processor uses a ControllerService like `InvokeHTTP` and `StandardSSLContextService`. Sometimes one ControllerService uses another
like `DistributedMapCacheClientService` and `DistributedMapCacheServer`. Developers of these extension points may relate these like `DistributedMapCacheClientService` and `DistributedMapCacheServer`. Developers of these extension points may relate these
different components using the `SeeAlso` tag. This annotation links these components in the documentation. different components using the `SeeAlso` tag. This annotation links these components in the documentation.

27
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/pom.xml
View File

@ -1,16 +1,19 @@
<?xml version="1.0" encoding="UTF-8"?> <?xml version="1.0" encoding="UTF-8"?>
<!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor <!--
license agreements. See the NOTICE file distributed with this work for additional Licensed to the Apache Software Foundation (ASF) under one or more
information regarding copyright ownership. The ASF licenses this file to contributor license agreements. See the NOTICE file distributed with
You under the Apache License, Version 2.0 (the "License"); you may not use this work for additional information regarding copyright ownership.
this file except in compliance with the License. You may obtain a copy of The ASF licenses this file to You under the Apache License, Version 2.0
the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required (the "License"); you may not use this file except in compliance with
by applicable law or agreed to in writing, software distributed under the the License. You may obtain a copy of the License at
License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS http://www.apache.org/licenses/LICENSE-2.0
OF ANY KIND, either express or implied. See the License for the specific Unless required by applicable law or agreed to in writing, software
language governing permissions and limitations under the License. --> distributed under the License is distributed on an "AS IS" BASIS,
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> See the License for the specific language governing permissions and
limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion> <modelVersion>4.0.0</modelVersion>
<parent> <parent>
<groupId>org.apache.nifi</groupId> <groupId>org.apache.nifi</groupId>

5
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/state/providers/zookeeper/ZooKeeperStateProvider.java
View File

@ -60,7 +60,8 @@ public class ZooKeeperStateProvider extends AbstractStateProvider {
static final PropertyDescriptor CONNECTION_STRING = new PropertyDescriptor.Builder() static final PropertyDescriptor CONNECTION_STRING = new PropertyDescriptor.Builder()
.name("Connect String") .name("Connect String")
.description("The ZooKeeper Connect String to use. This is a comma-separated list of hostnames/IP addresses, such as \"host1, host2, 127.0.0.1, host4, host5\"") .description("The ZooKeeper Connect String to use. This is a comma-separated list of hostname/IP and port tuples, such as \"host1:2181,host2:2181,127.0.0.1:2181\". If a port is not " +
"specified it defaults to the ZooKeeper client port default of 2181")
.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
.required(false) .required(false)
.build(); .build();
@ -216,7 +217,7 @@ public class ZooKeeperStateProvider extends AbstractStateProvider {
private void verifyEnabled() throws IOException { private void verifyEnabled() throws IOException {
if (!isEnabled()) { if (!isEnabled()) {
throw new IOException("Cannot update or retrieve cluster state becuase node is no longer connected to a cluster"); throw new IOException("Cannot update or retrieve cluster state because node is no longer connected to a cluster");
} }
} }