NIFI-4679 Added new content in User Guide related to Registry integration and replaced screenshots and edited text for recent UI changes. This closes #2382

This commit is contained in:
Andrew Lim 2018-01-06 00:32:55 -05:00 committed by Matt Gilman
parent 33b40fb87f
commit acdaeebac7
No known key found for this signature in database
GPG Key ID: DF61EC19432AEE37
53 changed files with 245 additions and 83 deletions

Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 56 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 187 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.8 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 119 KiB

After

Width:  |  Height:  |  Size: 110 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 132 KiB

After

Width:  |  Height:  |  Size: 140 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 147 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 612 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 724 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 751 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 944 B

Binary file not shown.

Before

Width:  |  Height:  |  Size: 263 B

After

Width:  |  Height:  |  Size: 385 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 973 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 996 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 61 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 264 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 194 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 16 KiB

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 112 KiB

After

Width:  |  Height:  |  Size: 169 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 57 KiB

After

Width:  |  Height:  |  Size: 85 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 66 KiB

After

Width:  |  Height:  |  Size: 90 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 65 KiB

After

Width:  |  Height:  |  Size: 90 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 77 KiB

After

Width:  |  Height:  |  Size: 120 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 58 KiB

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 181 KiB

After

Width:  |  Height:  |  Size: 173 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 249 KiB

After

Width:  |  Height:  |  Size: 196 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 138 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 80 KiB

After

Width:  |  Height:  |  Size: 88 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 68 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 48 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 26 KiB

After

Width:  |  Height:  |  Size: 28 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 183 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 187 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 159 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 180 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 143 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 139 KiB

View File

@ -149,11 +149,7 @@ image::nifi-toolbar-components.png["NiFi Components Toolbar"]
The Components Toolbar runs across the top left portion of your screen. It consists of the components you can drag onto the
canvas to build your dataflow. Each component is described in more detail in link:building-dataflow.html[Building a Dataflow].
The Status Bar is under the Components Toolbar. The Status bar provides information about how many Processors exist on the canvas in
each state (Stopped, Running, Invalid, Disabled), how many Remote Process Groups exist on the canvas in each state
(Transmitting, Not Transmitting), the number of threads that are currently active in the flow, the amount of data that currently
exists in the flow, and the timestamp at which all of this information was last refreshed. Additionally, if the instance of NiFi is clustered, the Status bar shows how many nodes
are in the cluster and how many are currently connected.
The Status Bar is under the Components Toolbar. The Status bar provides information about the number of threads that are currently active in the flow, the amount of data that currently exists in the flow, how many Remote Process Groups exist on the canvas in each state (Transmitting, Not Transmitting), how many Processors exist on the canvas in each state (Stopped, Running, Invalid, Disabled), how many versioned Process Groups exist on the canvas in each state (Up to date, Locally modified, Stale, Locally modified and stale, Sync failure) and the timestamp at which all of this information was last refreshed. Additionally, if the instance of NiFi is clustered, the Status bar shows how many nodes are in the cluster and how many are currently connected.
The Operate Palette sits to the left-hand side of the screen. It consists of buttons that are
used by DFMs to manage the flow, as well as by administrators who manage user access
@ -294,13 +290,14 @@ NOTE: For Processors, Ports, Remote Process Groups, Connections and Labels, it i
- *Start* or *Stop*: This option allows the user to start or stop a Processor; the option will be either Start or Stop, depending on the current state of the Processor.
- *Enable* or *Disable*: This option allows the user to enable or enable a Processor; the option will be either Enable or Disable, depending on the current state of the Processor.
- *Status History*: This option opens a graphical representation of the Processor's statistical information over time.
- *Data provenance*: This option displays the NiFi Data Provenance table, with information about data provenance events for the FlowFiles routed through that Processor (see <<data_provenance>>).
- *Upstream connections*: This option allows the user to see and "jump to" upstream connections that are coming into the Processor. This is particularly useful when processors connect into and out of other Process Groups.
- *Downstream connections*: This option allows the user to see and "jump to" downstream connections that are going out of the Processor. This is particularly useful when processors connect into and out of other Process Groups.
- *Usage*: This option takes the user to the Processor's usage documentation.
- *Change color*: This option allows the user to change the color of the Processor, which can make the visual management of large flows easier.
- *View data provenance*: This option displays the NiFi Data Provenance table, with information about data provenance events for the FlowFiles routed through that Processor (see <<data_provenance>>).
- *View status history*: This option opens a graphical representation of the Processor's statistical information over time.
- *View usage*: This option takes the user to the Processor's usage documentation.
- *View connections->Upstream*: This option allows the user to see and "jump to" upstream connections that are coming into the Processor. This is particularly useful when processors connect into and out of other Process Groups.
- *View connections->Downstream*: This option allows the user to see and "jump to" downstream connections that are going out of the Processor. This is particularly useful when processors connect into and out of other Process Groups.
- *Center in view*: This option centers the view of the canvas on the given Processor.
- *Change color*: This option allows the user to change the color of the Processor, which can make the visual management of large flows easier.
- *Create template*: This option allows the user to create a template from the selected Processor.
- *Copy*: This option places a copy of the selected Processor on the clipboard, so that it may be pasted elsewhere on the canvas by right-clicking on the canvas and selecting Paste. The Copy/Paste actions also may be done using the keystrokes Ctrl-C (Command-C) and Ctrl-V (Command-V).
- *Delete*: This option allows the DFM to delete a Processor from the canvas.
@ -348,17 +345,19 @@ image::nifi-process-group-menu.png["Process Group Menu"]
While the options available from the context menu vary, the following options are typically available when you have full privileges to work with the Process Group:
- *Configure*: This option allows the user to establish or change the configuration of the Process Group.
- *Variables*: This option allows the user to create or configure variables within the NiFi UI.
- *Enter group*: This option allows the user to enter the Process Group.
NOTE: It is also possible to double-click on the Process Group to enter it.
- *Start*: This option allows the user to start a Process Group.
- *Stop*: This option allows the user to stop a Process Group.
- *Group*: This option allows the user to create a new Process Group that contains the selected Process Group and any other components selected on the canvas.
- *Status History*: This option opens a graphical representation of the Process Group's statistical information over time.
- *Upstream connections*: This option allows the user to see and "jump to" upstream connections that are coming into the Process Group.
- *Downstream connections*: This option allows the user to see and "jump to" downstream connections that are going out of the Process Group.
- *View status history*: This option opens a graphical representation of the Process Group's statistical information over time.
- *View connections->Upstream*: This option allows the user to see and "jump to" upstream connections that are coming into the Process Group.
- *View connections->Downstream*: This option allows the user to see and "jump to" downstream connections that are going out of the Process Group.
- *Center in view*: This option centers the view of the canvas on the given Process Group.
- *Group*: This option allows the user to create a new Process Group that contains the selected Process Group and any other components selected on the canvas.
- *Create template*: This option allows the user to create a template from the selected Process Group.
- *Copy*: This option places a copy of the selected Process Group on the clipboard, so that it may be pasted elsewhere on the canvas by right-clicking on the canvas and selecting Paste. The Copy/Paste actions also may be done using the keystrokes Ctrl-C (Command-C) and Ctrl-V (Command-V).
- *Delete*: This option allows the DFM to delete a Process Group.
@ -383,16 +382,18 @@ image::nifi-rpg-menu.png["Remote Process Group Menu"]
While the options available from the context menu vary, the following options are typically available when you have full privileges to work with the Remote Process Group:
- *Configure*: This option allows the user to establish or change the configuration of the Remote Process Group.
- *Group*: This option allows the user to create a new Process Group that contains the selected Remote Process Group and any other components selected on the canvas.
- *Remote Ports*: This option allows the user to see input ports and/or output ports that exist on the remote instance of NiFi that the Remote Process Group is connected to. Note that if the Site-to-Site configuration is secure, only the ports that the connecting NiFi has been given accessed to will be visible.
- *Enable transmission*: Makes the transmission of data between NiFi instances active (see <<Remote_Group_Transmission>>).
- *Disable transmission*: Disables the transmission of data between NiFi instances.
- *Status History*: This option opens a graphical representation of the Remote Process Group's statistical information over time.
- *Upstream connections*: This option allows the user to see and "jump to" upstream connections that are coming into the Remote Process Group.
- *Downstream connections*: This option allows the user to see and "jump to" downstream connections that are going out of the Remote Process Group.
- *Refresh*: This option refreshes the view of the status of the remote NiFi instance.
- *Go to*: This option opens a view of the remote NiFi instance in a new tab of the browser. Note that if the Site-to-Site configuration is secure, the user must have access to the remote NiFi instance in order to view it.
- *View status history*: This option opens a graphical representation of the Remote Process Group's statistical information over time.
- *View connections->Upstream*: This option allows the user to see and "jump to" upstream connections that are coming into the Remote Process Group.
- *View connections->Downstream*: This option allows the user to see and "jump to" downstream connections that are going out of the Remote Process Group.
- *Refresh remote*: This option refreshes the view of the status of the remote NiFi instance.
- *Group*: This option allows the user to create a new Process Group that contains the selected Remote Process Group and any other components selected on the canvas.
- *Manage remote ports*: This option allows the user to see input ports and/or output ports that exist on the remote instance of NiFi that the Remote Process Group is connected to. Note that if the Site-to-Site configuration is secure, only the ports that the connecting NiFi has been given accessed to will be visible.
- *Center in view*: This option centers the view of the canvas on the given Remote Process Group.
- *Go to*: This option opens a view of the remote NiFi instance in a new tab of the browser. Note that if the Site-to-Site configuration is secure, the user must have access to the remote NiFi instance in order to view it.
- *Group*: This option allows the user to create a Process Group containing the selected Remote Process Group.
- *Create template*: This option allows the user to create a template from the selected Remote Process Group.
- *Copy*: This option places a copy of the selected Process Group on the clipboard, so that it may be pasted elsewhere on the canvas by right-clicking on the canvas and selecting Paste. The Copy/Paste actions also may be done using the keystrokes Ctrl-C (Command-C) and Ctrl-V (Command-V).
- *Delete*: This option allows the DFM to delete a Remote Process Group from the canvas.
@ -630,7 +631,7 @@ resources that then are not usable by other Processors. This essentially provide
how much of the system's resources should be allocated to this Processor instead of other Processors. This field is available for
most Processors. There are, however, some types of Processors that can only be scheduled with a single Concurrent task.
The ``Run schedule'' dictates how often the Processor should be scheduled to run. The valid values for this field depend on the selected
The "Run schedule" dictates how often the Processor should be scheduled to run. The valid values for this field depend on the selected
Scheduling Strategy (see above). If using the Event driven Scheduling Strategy, this field is not available. When using the Timer driven
Scheduling Strategy, this value is a time duration specified by a number followed by a time unit. For example, `1 second` or `5 mins`.
The default value of `0 sec` means that the Processor should run as often as possible as long as it has data to process. This is true
@ -837,11 +838,11 @@ To add a Controller Service for a reporting task, select Controller Settings fro
image:controller-settings-selection.png["Global Menu - Controller Settings"]
This displays the NiFi Settings window. The window has three tabs: General, Controller Services, and Reporting Tasks. The General tab provides settings for the overall maximum thread counts of the instance.
This displays the NiFi Settings window. The window has four tabs: General, Reporting Task Controller Services, Reporting Tasks and Registry Clients. The General tab provides settings for the overall maximum thread counts of the instance.
image:settings-general-tab.png["Controller Settings General Tab"]
To the right of the General tab is the Controller Services tab. From this tab, the DFM may click the "+" button in the upper-right corner to create a new Controller Service.
To the right of the General tab is the Reporting Task Controller Services tab. From this tab, the DFM may click the "+" button in the upper-right corner to create a new Controller Service.
image:controller-services-tab.png["Controller Services Tab"]
@ -849,36 +850,30 @@ The Add Controller Service window opens. This window is similar to the Add Proce
image:add-controller-service-window.png["Add Controller Service Window"]
Once you have added a Controller Service, you can configure it by clicking the Edit button in the
far-right column. Other buttons in this column include Remove and Access Policies.
Once you have added a Controller Service, you can configure it by clicking the Configure button in the
far-right column. Other buttons in this column include Enable, Remove and Access Policies.
image:controller-services-reporting-tasks-edit-buttons.png["Controller Services Buttons"]
image:controller-services-configure-buttons.png["Controller Services Buttons"]
You can obtain information about Controller Services by clicking the View Details, Usage, and Alerts buttons in the left-hand column.
You can obtain information about Controller Services by clicking the Usage and Alerts buttons in the left-hand column.
image:controller-services-reporting-tasks-info-buttons.png["Controller Services Information Buttons"]
image:controller-services-info-buttons.png["Controller Services Information Buttons"]
When the DFM clicks the Edit button, a Configure Controller Service window opens. It has three tabs: Settings, Properties,
and Comments. This window is similar to the Configure Processor window. The Settings tab provides a place for the DFM
to give the Controller Service a unique name (if desired). It also lists the UUID, Type, Bundle and Support information for the service and provides a list of other components (reporting tasks or other controller services) that reference the service.
When the DFM clicks the Configure button, a Configure Controller Service window opens. It has three tabs: Settings, Properties,and Comments. This window is similar to the Configure Processor window. The Settings tab provides a place for the DFM to give the Controller Service a unique name (if desired). It also lists the UUID, Type, Bundle and Support information for the service and provides a list of other components (reporting tasks or other controller services) that reference the service.
image:configure-controller-service-settings.png["Configure Controller Service Settings"]
The Properties tab lists the various properties that apply to the particular controller service. As with configuring
processors, the DFM may hover over the question mark icons to see more information about each property.
The Properties tab lists the various properties that apply to the particular controller service. As with configuring processors, the DFM may hover over the question mark icons to see more information about each property.
image:configure-controller-service-properties.png["Configure Controller Service Properties"]
The Comments tab is just an open-text field, where the DFM may include comments about the service. After configuring
a Controller Service, click the Apply button to apply the configuration and close the window, or click the Cancel
button to cancel the changes and close the window.
The Comments tab is just an open-text field, where the DFM may include comments about the service. After configuring a Controller Service, click the Apply button to apply the configuration and close the window, or click the Cancel button to cancel the changes and close the window.
[[Controller_Services_for_Dataflows]]
==== Adding Controller Services for Dataflows
To add a Controller Service for a dataflow, you can either right click a Process Group and select Configure, or click
Configure from the Operate Palette.
To add a Controller Service for a dataflow, you can either right click a Process Group and select Configure, or click Configure from the Operate Palette.
image:process-group-configuration-options.png["Process Group Configuration Options"]
@ -892,9 +887,9 @@ Use the following steps to add a Controller Service:
+
image::process-group-configuration-window.png["Process Group Configuration Window"]
2. From the Process Group Configuration page, select the Controller Services tab.
3. Click the Add button to display the Add Controller Service dialog.
3. Click the "+" button to display the Add Controller Service dialog.
4. Select the Controller Service desired, and click Add.
5. Perform any necessary Controller Service configuration tasks by clicking the Edit icon (image:iconEdit.png["Edit"]) in the right-hand column.
5. Perform any necessary Controller Service configuration tasks by clicking the Configure icon (image:iconConfigure.png["Configure"]) in the right-hand column.
[[Enabling_Disabling_Controller_Services]]
@ -919,13 +914,13 @@ The Add Reporting Task window opens. This window is similar to the Add Processor
image:add-reporting-task-window.png["Add Reporting Task Window"]
Once a Reporting Task has been added, the DFM may configure it by clicking the Edit button in the far-right column. Other buttons in this column include Remove and Access Policies.
Once a Reporting Task has been added, the DFM may configure it by clicking the Edit button in the far-right column. Other buttons in this column include Start, Remove, State and Access Policies.
image:controller-services-reporting-tasks-edit-buttons.png["Reporting Tasks Buttons"]
image:reporting-tasks-edit-buttons.png["Reporting Tasks Edit Buttons"]
You can obtain information about Reporting Tasks by clicking the View Details, Usage, and Alerts buttons in the left-hand column.
image:controller-services-reporting-tasks-info-buttons.png["Reporting Tasks Information Buttons"]
image:reporting-tasks-info-buttons.png["Reporting Tasks Information Buttons"]
When the DFM clicks the Edit button, a Configure Reporting Task window opens. It has three tabs: Settings, Properties, and Comments. This window is similar to the Configure Processor window. The Settings tab provides a place for the DFM to give the Reporting Task a unique name (if desired). It also lists the UUID, Type, and Bundle information for the task and provides settings for the task's Scheduling Strategy and Run Schedule (similar to the same settings in a processor). The DFM may hover the mouse over the question mark icons to see more information about each setting.
@ -937,7 +932,7 @@ image:configure-reporting-task-properties.png["Configure Reporting Task Properti
The Comments tab is just an open-text field, where the DFM may include comments about the task. After configuring the Reporting Task, click the Apply button to apply the configuration and close the window, or click the Cancel button to cancel the changes and close the window.
When you want to run the Reporting Task, click the Start button (image:iconStart.png["Start Button"]) in the far-right column of the Reporting Tasks tab.
When you want to run the Reporting Task, click the Start button (image:iconStart.png["Start Button"]).
[[Connecting_Components]]
@ -993,9 +988,9 @@ image:file_expiration_clock.png["File Expiration Indicator"]
===== Back Pressure
NiFi provides two configuration elements for Back Pressure. These thresholds indicate how much data should be
allowed to exist in the queue before the component that is the source of the Connection is no longer scheduled to run.
This allows the system to avoid being overrun with data. The first option provided is the ``Back pressure object threshold.''
This allows the system to avoid being overrun with data. The first option provided is the "Back pressure object threshold."
This is the number of FlowFiles that can be in the queue before back pressure is applied. The second configuration option
is the ``Back pressure data size threshold.'' This specifies the maximum amount of data (in size) that should be queued up before
is the "Back pressure data size threshold." This specifies the maximum amount of data (in size) that should be queued up before
applying back pressure. This value is configured by entering a number followed by a data size (`B` for bytes, `KB` for
kilobytes, `MB` for megabytes, `GB` for gigabytes, or `TB` for terabytes).
@ -1041,11 +1036,11 @@ image:nifi-connection-menu.png["Connection Menu"]
The following options are available:
- *Configure*: This option allows the user to change the configuration of the connection.
- *Status History*: This option opens a graphical representation of the connection's statistical information over time.
- *Bring to front*: This option brings the connection to the front of the canvas if something else (such as another connection) is overlapping it.
- *View status history*: This option opens a graphical representation of the connection's statistical information over time.
- *List queue*: This option lists the queue of FlowFiles that may be waiting to be processed.
- *Go to source*: This option can be useful if there is a long distance between the connection's source and destination components on the canvas. By clicking this option, the view of the canvas will jump to the source of the connection.
- *Go to destination*: Similar to the "Go to source" option, this option changes the view to the destination component on the canvas and can be useful if there is a long distance between two connected components.
- *List queue*: This option lists the queue of FlowFiles that may be waiting to be processed.
- *Bring to front*: This option brings the connection to the front of the canvas if something else (such as another connection) is overlapping it.
- *Empty queue*: This option allows the DFM to clear the queue of FlowFiles that may be waiting to be processed. This option can be especially useful during testing, when the DFM is not concerned about deleting data from the queue. When this option is selected, users must confirm that they want to delete the data in the queue.
- *Delete*: This option allows the DFM to delete a connection between two components. Note that the components on both sides of the connection must be stopped and the connection must be empty before it can be deleted.
@ -1226,7 +1221,7 @@ In order to start a component, the following conditions must be met:
- The component must be enabled.
- The component must have no active tasks. For more information about active tasks, see the ``Anatomy of ...''
- The component must have no active tasks. For more information about active tasks, see the "Anatomy of ..."
sections under <<monitoring>> (<<processor_anatomy>>, <<process_group_anatomy>>, <<remote_group_anatomy>>).
Components can be started by selecting all of the components to start and then clicking the Start button (
@ -1272,7 +1267,7 @@ When it is desirable to re-enable a component, it can be enabled by selecting th
clicking the Enable button (
image:buttonEnable.png["Enable"]
) in the Operate Palette. This is available only when the selected component or components are disabled.
Alternatively, a component can be enabled by checking the checkbox next to the ``Enabled'' option in
Alternatively, a component can be enabled by checking the checkbox next to the "Enabled" option in
the Settings tab of the Processor configuration dialog or the configuration dialog for a Port.
Once enabled, the component's status indicator will change to either Invalid (
@ -1283,7 +1278,7 @@ image:iconStop.png["Stopped"]
A component is then disabled by selecting the component and clicking the Disable button (
image:buttonDisable.png["Disable"]
) in the Operate Palette, or by clearing the checkbox next to the ``Enabled'' option in the Settings tab
) in the Operate Palette, or by clearing the checkbox next to the "Enabled" option in the Settings tab
of the Processor configuration dialog or the configuration dialog for a Port.
Only Ports and Processors can be enabled and disabled.
@ -1297,7 +1292,7 @@ of NiFi. When a Remote Process Group (RPG) is added to the canvas, it is added w
as indicated by the icon (
image:iconTransmissionInactive.png["Transmission Disabled"]
) in the top-left corner. When Transmission is Disabled, it can be enabled by right-clicking on the
RPG and clicking the ``Enable Transmission'' menu item. This will cause all ports for which there is a Connection
RPG and clicking the "Enable Transmission" menu item. This will cause all ports for which there is a Connection
to begin transmitting data. This will cause the status indicator to then change to the Transmission Enabled icon (
image:iconTransmissionActive.png["Transmission Enabled"]
).
@ -1312,7 +1307,7 @@ more information about the problem.
There are times when the DFM may want to either enable or disable transmission for only a specific
Port within the Remote Process Group. This can be accomplished by right-clicking on the Remote Process Group
and choosing the ``Remote ports'' menu item. This provides a configuration dialog from which each Port can be
and choosing the "Remote ports" menu item. This provides a configuration dialog from which each Port can be
configured:
image::remote-group-ports-dialog.png["Remote Process Groups"]
@ -1327,7 +1322,7 @@ NOTE: If a Port that is expected to be shown is not shown in this dialog, ensure
permissions and that the Remote Process Group's flow is current. This can be checked by closing the Port
Configuration Dialog and looking at the bottom-right corner of the Remote Process Group. The date at which
the flow was last refreshed is shown. If the flow appears to be outdated, it can be updated by right-clicking
on the Remote Process Group and selecting ``Refresh flow.'' (See <<remote_group_anatomy>> for more information).
on the Remote Process Group and selecting "Refresh flow." (See <<remote_group_anatomy>> for more information).
Each Port is shown with the Port name, followed by its description, currently configured number of Concurrent
tasks, and whether or not data sent to this port will be compressed. To the left of this information is a switch
@ -1413,12 +1408,12 @@ The image outlines the following elements:
- *Processor Type*: NiFi provides several different types of Processors in order to allow for a wide range
of tasks to be performed. Each type of Processor is designed to perform one specific task. The Processor
type (PutFile, in this example) describes the task that this Processor performs. In this case, the
Processor writes a FlowFile to disk - or ``Puts'' a FlowFile to a File.
Processor writes a FlowFile to disk - or "Puts" a FlowFile to a File.
- *Bulletin Indicator*: When a Processor logs that some event has occurred, it generates a Bulletin to notify
those who are monitoring NiFi via the User Interface. The DFM is able to configure which
bulletins should be displayed in the User Interface by updating the ``Bulletin level'' field in the
``Settings'' tab of the Processor configuration dialog. The default value is `WARN`, which means that only
bulletins should be displayed in the User Interface by updating the "Bulletin level" field in the
"Settings" tab of the Processor configuration dialog. The default value is `WARN`, which means that only
warnings and errors will be displayed in the UI. This icon is not present unless a Bulletin exists for this
Processor. When it is present, hovering over the icon with the mouse will provide a tooltip explaining the
message provided by the Processor as well as the Bulletin level. If the instance of NiFi is clustered,
@ -1440,7 +1435,7 @@ The image outlines the following elements:
the same as the Processor Type. In the example, this value is "Copy to /review".
- *Active Tasks*: The number of tasks that this Processor is currently executing. This number is constrained
by the ``Concurrent tasks'' setting in the ``Scheduling'' tab of the Processor configuration dialog.
by the "Concurrent tasks" setting in the "Scheduling" tab of the Processor configuration dialog.
Here, we can see that the Processor is currently performing one task. If the NiFi instance is clustered,
this value represents the number of tasks that are currently executing across all nodes in the cluster.
@ -1466,7 +1461,7 @@ The image outlines the following elements:
which already existed in the output directory, data was neither read nor written to disk.
** *Out*: The amount of data that the Processor has transferred to its outbound Connections. This does
not include FlowFiles that the Processor removes itself, or FlowFiles that are routed to connections
that are auto-terminated. Like the ``In'' metric above, this value is represented as <count> (<size>)
that are auto-terminated. Like the "In" metric above, this value is represented as <count> (<size>)
where <count> is the number of FlowFiles that have been transferred to outbound Connections and <size>
is the total size of those FlowFiles' content. In this example, all of the Relationships are configured to be
auto-terminated, so no FlowFiles are reported as having been transferred Out.
@ -1476,7 +1471,7 @@ The image outlines the following elements:
instance, if the Processor is scheduled to run with 60 Concurrent tasks, and each of those tasks takes one
second to complete, it is possible that all 60 tasks will be completed in a single second. However, in this
case we will see the Time metric showing that it took 60 seconds, instead of 1 second. This time can be
thought of as ``System Time,'' or said another way, this value is 60 seconds because that's the amount of
thought of as "System Time," or said another way, this value is 60 seconds because that's the amount of
time it would have taken to perform the action if only a single concurrent task were used.
@ -1496,7 +1491,7 @@ The Process Group consists of the following elements:
- *Name*: This is the user-defined name of the Process Group. This name is set when the Process Group
is added to the canvas. The name can later by changed by right-clicking on the Process Group and clicking
the ``Configure'' menu option. In this example, the name of the Process Group is ``Process Group ABC.''
the "Configure" menu option. In this example, the name of the Process Group is "Process Group ABC."
- *Bulletin Indicator*: When a child component of a Process Group emits a bulletin, that bulletin is propagated to
the component's parent Process Group, as well. When any component has an active Bulletin, this indicator will appear,
@ -1507,13 +1502,10 @@ The Process Group consists of the following elements:
NiFi instance is clustered, this value represents the number of tasks that are currently executing
across all nodes in the cluster.
- *Comments*: When the Process Group is added to the canvas, the user is given the option of specifying Comments in order
to provide information about the Process Group. The comments can later be changed by right-clicking on the Process
Group and clicking the ``Configure'' menu option. In this example, the Comments are set to ``Example Process Group.''
- *Statistics*: Process Groups provide statistics about the amount of data that has been processed by the Process Group in
the past 5 minutes as well as the amount of data currently enqueued within the Process Group. The following elements
comprise the ``Statistics'' portion of a Process Group:
comprise the "Statistics" portion of a Process Group:
** *Queued*: The number of FlowFiles currently enqueued within the Process Group.
This field is represented as <count> (<size>) where <count> is the number of FlowFiles that are
currently enqueued in the Process Group and <size> is the total size of those FlowFiles' content. In this example,
@ -1558,7 +1550,9 @@ The Process Group consists of the following elements:
components may or may not be valid. If the Process Group is started, these components will not cause any errors
but will not be started.
- *Version State Counts*: The Version State Counts element provides information about how many versioned process groups are within the Process Group. See <<version_states>> for more information.
- *Comments*: When the Process Group is added to the canvas, the user is given the option of specifying Comments in order to provide information about the Process Group. The comments can later be changed by right-clicking on the Process Group and clicking the "Configure" menu option.
@ -1595,16 +1589,16 @@ image:iconTransmissionInactive.png["Transmission Inactive"]
This URL is entered when the Remote Process Group is added to the canvas and it cannot be changed.
- *Secure Indicator*: This icon indicates whether or not communications with the remote NiFi instance are
secure. If communications with the remote instance are secure, this will be indicated by the ``locked''
secure. If communications with the remote instance are secure, this will be indicated by the "Locked"
icon (
image:iconSecure.png["Secure"]
). If the communications are not secure, this will be indicated by the ``unlocked'' icon (
). If the communications are not secure, this will be indicated by the "Unlocked" icon (
image:iconNotSecure.png["Not Secure"]
). If the communications are secure, this instance of NiFi will not be able to communicate with the
remote instance until an administrator for the remote instance grants access. Whenever the Remote Process
Group is added to the canvas, this will automatically initiate a request to have a user for this instance of NiFi created on the
remote instance. This instance will be unable to communicate with the remote instance until an administrator
on the remote instance adds the user to the system and adds the ``NiFi'' role to the user.
on the remote instance adds the user to the system and adds the "NiFi" role to the user.
In the event that communications are not secure, the Remote Process Group is able to receive data from anyone,
and the data is not encrypted while it is transferred between instances of NiFi.
@ -1620,7 +1614,7 @@ image:iconNotSecure.png["Not Secure"]
in the User Interface is periodically refreshed in the background. This element indicates the time at which that refresh
last happened, or if the information has not been refreshed for a significant amount of time, the value will change to
indicate _Remote flow not current_. NiFi can be triggered to initiate a refresh of this information by right-clicking
on the Remote Process Group and choosing the ``Refresh flow'' menu item.
on the Remote Process Group and choosing the "Refresh flow" menu item.
@ -1675,11 +1669,11 @@ The Summary page also includes the following elements:
by clicking on the heading so that all the currently existing bulletins are shown at the top of the list.
- *Details*: Clicking the Details icon will provide the user with the details of the component. This dialog is the same as the
dialog provided when the user right-clicks on the component and chooses the ``View configuration'' menu item.
dialog provided when the user right-clicks on the component and chooses the "View Configuration" menu item.
- *Go To*: Clicking this button will close the Summary page and take the user directly to the component on the NiFi canvas. This
may change the Process Group that the user is currently in. This icon is not available if the Summary page has been opened
in a new browser tab or window (by clicking the ``Pop Out'' button, as described below).
in a new browser tab or window (by clicking the "Pop Out" button, as described below).
- *Status History*: Clicking the Status History icon will open a new dialog that shows a historical view of the statistics that
are rendered for this component. See the section <<Status_History>> for more information.
@ -1697,7 +1691,7 @@ The Summary page also includes the following elements:
- *Pop-Out*: When monitoring a flow, it is helpful to be able to open the Summary table in a separate browser tab or window. The
Pop-Out button, next to the Close button, will cause the entire Summary dialog to be opened in a new browser tab or window
(depending on the configuration of the browser). Once the page is ``popped out'', the dialog is closed in the original
(depending on the configuration of the browser). Once the page is "popped out", the dialog is closed in the original
browser tab/window. In the new tab/window, the Pop-Out button and the Go-To button will no longer be available.
- *System Diagnostics*: The System Diagnostics window provides information about how the system is performing with respect to
@ -1713,7 +1707,7 @@ The Summary page also includes the following elements:
While the Summary table and the canvas show numeric statistics pertaining to the performance of a component over the
past five minutes, it is often useful to have a view of historical statistics as well. This information is available
by right-clicking on a component and choosing the ``Status History'' menu option or by clicking on the Status History in the Summary page (see <<Summary_Page>>
by right-clicking on a component and choosing the "Status History" menu option or by clicking on the Status History in the Summary page (see <<Summary_Page>>
for more information).
The amount of historical information that is stored is configurable in the NiFi properties but defaults to 24 hours. For specific
@ -1759,6 +1753,174 @@ rectangle over the graph, double-clicking on the selected portion will cause the
vertical direction (i.e., it will select all values in this time range). Clicking on the bottom graph without dragging
will remove the selection.
[[versioning_dataflow]]
== Versioning a DataFlow
When NiFi is connected to a NiFi Registry, dataflows can be version controlled on the process group level. For more information about NiFi Registry usage and configuration, see the documentation at https://nifi.apache.org/docs/nifi-registry-docs/index.html.
=== Connecting to a NiFi Registry
To connect NiFi to a Registry, select Controller Settings from the Global Menu.
image::controller-settings-selection.png["Global Menu - Controller Settings"]
This displays the NiFi Settings window. Select the Registry Clients tab and click the "+" button in the upper-right corner to register a new Registry client.
image::registry-clients-tab.png["Registry Clients Tab"]
In the Add Registry Client window, provide a name and URL.
image::add-registry-client.png["Add Registry Client Dialog"]
Click "Add" to complete the registration.
image::registry-client-added.png["Registry Client Added"]
NOTE: Versioned flows are stored and organized in registry buckets. Bucket Policies and Special Privileges configured by the registry administrator determine which buckets a user can import versioned flows from and which buckets a user can save versioned flows to. Information on Bucket Policies and Special Privileges can be found in the NiFi Registry User Guide (https://nifi.apache.org/docs/nifi-registry-docs/html/user-guide.html).
[[version_states]]
=== Version States
Versioned process groups exist in the following states:
** image:iconUpToDate.png["Up to date"]
*Up to date*: The flow version is the latest.
** image:iconLocallyModified.png["Locally Modified"]
*Locally modified*: Local changes have been made.
** image:iconStale.png["Stale"]
*Stale*: A newer version of the flow is available.
** image:iconLocallyModifiedStale.png["Locally Modified and Stale"]
*Locally modified and stale*: Local changes have been made and a newer version of the flow is available.
** image:iconSyncFailure.png["Sync Failure"]
*Sync failure*: Unable to synchronize the flow with the registry.
Version state information is displayed:
1. Next to the process group name, for the versioned process group itself. Hovering over the state icon displays additional information about the versioned flow.
2. At the bottom of a process group, for the versioned flows contained in the process group.
3. In the Status Bar at the top of the UI, for the versioned flows contained in the root process group.
image::version-states-display.png["Version States Displayed"]
Version state information is also shown in the "Process Groups" tab of the Summary Page.
image::version-state-summary-page.png["Version State in Summary Page"]
NOTE: To see the most recent version states, it may be necessary to right-click on the NiFi canvas and select `Refresh' from the context menu.
=== Import a Versioned Flow
When a NiFi instance is connected to a registry, an "Import" link will appear in the Add Process Group dialog.
image::add-process-group-import.png["Import Process Group"]
Selecting the link will open the Import Version dialog.
image::import-version-dialog.png["Import Version Dialog"]
Connected registries will appear as options in the Registry drop-down menu. For the chosen Registry, buckets the user has access to will appear as options in the Bucket drop-down menu. The names of the flows in the chosen bucket will appear as options in the Name drop-down menu. Select the desired version of the flow to import and select "Import" for the dataflow to be placed on the canvas.
image::versioned-flow-imported.png["Versioned Flow Imported"]
Since the version imported in this example is the latest version (MySQL CDC, Version 3), the state of the versioned process group is "Up to date" (image:iconUpToDate.png["Up To Date Icon"]). If the version imported had been an older version, the state would be "Stale" (image:iconStale.png["Stale Icon"]).
=== Start Version Control
To place a process group under version control, right-click on the process group and in the context menu, select "Version->Start version control".
image::start-version-control.png["Start Version Control"]
In the Save Flow Version window, select a Registry and Bucket and enter a Name for the Flow. If desired, add content for the Description and Comment fields.
image::save-flow-version-dialog.png["Save Flow Version Dialog"]
Select Save and Version 1 of the flow is saved.
image::versioned-process-group.png["Versioned Process Group"]
As the first and latest version of the flow, the state of the versioned process group is "Up to date" (image:iconUpToDate.png["Up To Date Icon"]).
NOTE: The root process group can not be placed under version control.
[[managing_local_changes]]
=== Managing Local Changes
When changes are made to a versioned process group, the state of the component updates to "Locally modified" (image:iconLocallyModified.png["Locally Modified Icon"]). The DFM can show, revert or commit the local changes. These options are available for selection in the context menu when right-clicking on the process group:
image::local-changes-pg-selected.png["Local Changes PG Selected"]
or when right-clicking on the canvas inside the process group:
image::local-changes-pg-inside.png["Local Changes Inside PG"]
The following actions are not considered local changes:
* disabling/enabling processors and controller services
* stopping/starting processors
* modifying sensitive property values
* modifying remote process group URLs
* updating a processor that was referencing a non-existent controller service to reference an externally available controller service
* modifying variables
WARNING: Variables do not support sensitive values and will be included when versioning a Process Group.
==== Show Local Changes
The local changes made to a versioned process group can be viewed in the Show Local Changes dialog by selecting "Version->Show local changes" from the context menu.
image::show-local-changes-dialog.png["Show Local Changes Dialog"]
You can navigate to a component by selecting the "Go To" icon (image:iconGoTo.png["Go To"]) in its row.
NOTE: As described in the <<managing_local_changes>> section, there are exceptions to which actions are reviewable local changes. Additionally, multiple changes to the same property will only appear as one change in the list as the changes are determined by diffing the current state of the process group and the saved version of the process group noted in the Show Local Changes dialog.
==== Revert Local Changes
Revert the local changes made to a versioned process group by selecting "Version->Revert local changes" from the context menu. The Revert Local Changes dialog displays a list of the local changes for the DFM to review and consider prior to initiating the revert. Select "Revert" to remove all changes.
image::revert-local-changes-dialog.png["Revert Local Changes Dialog"]
You can navigate to a component by selecting the "Go To" icon (image:iconGoTo.png["Go To"]) in its row.
NOTE: As described in the <<managing_local_changes>> section, there are exceptions to which actions are revertible local changes. Additionally, multiple changes to the same property will only appear as one change in the list as the changes are determined by diffing the current state of the process group and the saved version of the process group noted in the Revert Local Changes dialog.
==== Commit Local Changes
To commit and save a flow version, select "Version->Commit local changes" from the context menu. In the Save Flow Version dialog, add comments if desired and select "Save".
image::save-flow-version-commit.png["Save Flow Version Commit"]
Local changes can not be committed if the version that has been modified is not the latest version. In this scenario, the version state is "Locally modified and stale" (image:iconLocallyModifiedStale.png["Locally Modified and Stale"]).
=== Change Version
To change the version of a flow, right-click on the versioned process group and select "Version->Change version".
image::change-version.png["Change Version"]
In the Change Version dialog, select the desired version and select "Change":
image::change-version-dialog.png["Change Version Dialog"]
The version of the flow is changed:
image::flow-version-changed.png["Flow Version Changed"]
In the example shown, the versioned flow is upgraded from an older to the newer latest version. However, a versioned flow can also be rollbacked to an older version.
NOTE: For "Change version" to be an available selection, local changes to the process group need to be reverted.
=== Stop Version Control
To stop version control on a flow, right-click on the versioned process group and select "Version->Stop version control":
image:stop-version-control.png["Stop Version Control"]
In the Stop Version Control dialog, select "Disconnect".
image::stop-version-control-dialog.png["Stop Version Control Dialog"]
The removal of the process group from version control is confirmed.
image:disconnect-dialog.png["Disconnect Confirmation Dialog"]
image::process-group-version-control-stopped.png["Version Control Stopped on Process Group"]
=== Nested Versioned Flows
A versioned process group can contain other versioned process groups. However, local changes to a parent process group cannot be reverted or saved if it contains a child process group that also has local changes. The child process group must first be reverted or have its changes committed for those actions to be performed on the parent process group.
[[templates]]
@ -1776,7 +1938,7 @@ received from others can then be imported into an instance of NiFi and dragged o
[[Create_Template]]
=== Creating a Template
To create a Template, select the components that are to be a part of the template, and then click the
``Create Template'' (
"Create Template" (
image:iconNewTemplate.png["Create Template"]
) button in the Operate Palette (See <<User_Interface>> for more information on the Operate Palette).
@ -1801,13 +1963,13 @@ After receiving a Template that has been exported from another NiFi, the first s
the template into this instance of NiFi. You may import templates into any Process Group where you have the appropriate
authorization.
From the Operate Palette, click the ``Upload Template'' (
From the Operate Palette, click the "Upload Template" (
image:iconUploadTemplate.png["Upload Template"]
) button (See <<User_Interface>> for more information on the Operate Palette). This will display the Upload Template
) button (see <<User_Interface>> for more information on the Operate Palette). This will display the Upload Template
dialog. Click the find icon and use the File Selection dialog to choose which template file to upload.
Select the file and click Open.
Clicking the ``Upload'' button will attempt to import the Template into this instance of NiFi.
The Upload Template dialog will update to show ``Success'' or an error message if there was a problem importing the template.
Clicking the "Upload" button will attempt to import the Template into this instance of NiFi.
The Upload Template dialog will update to show "Success" or an error message if there was a problem importing the template.
=== Instantiating a Template
@ -1818,7 +1980,7 @@ image:iconTemplate.png["Template"]
) from the Components Toolbar (see <<User_Interface>>) onto the canvas.
This will present a dialog to choose which Template to add to the canvas. After choosing the Template to add, simply
click the ``Add'' button. The Template will be added to the canvas with the upper-left-hand side of the Template
click the "Add" button. The Template will be added to the canvas with the upper-left-hand side of the Template
being placed wherever the user dropped the Template icon.
This leaves the contents of the newly instantiated Template selected. If there was a mistake, and this Template is no