mirror of
https://github.com/apache/nifi.git
synced 2025-02-09 03:25:04 +00:00
c68f5ec9a2
Added documentation to indicate how to debug Python side of nifi framework, as well as debugging Python processors themselves using VSCode's Remote debugger. This also provides the ability to launch the Controller process in such a way that it will listen to incoming remote debug connections. This closes #7469 Signed-off-by: David Handermann <exceptionfactory@apache.org>
80 lines
3.9 KiB
Markdown
80 lines
3.9 KiB
Markdown
<!--
|
|
Licensed to the Apache Software Foundation (ASF) under one or more
|
|
contributor license agreements. See the NOTICE file distributed with
|
|
this work for additional information regarding copyright ownership.
|
|
The ASF licenses this file to You under the Apache License, Version 2.0
|
|
(the "License"); you may not use this file except in compliance with
|
|
the License. You may obtain a copy of the License at
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
Unless required by applicable law or agreed to in writing, software
|
|
distributed under the License is distributed on an "AS IS" BASIS,
|
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
See the License for the specific language governing permissions and
|
|
limitations under the License.
|
|
-->
|
|
|
|
|
|
# nifi-py4j-bundle module
|
|
|
|
The NiFi Py4J Bundle provides a linkage between NiFi's Java Process and Python.
|
|
Py4J is the library used in order to launch an RPG server that can be used to communicate between
|
|
the Java and Python Processes.
|
|
|
|
See the NiFi Python Developer's Guide for more information about how to build Processors in Python.
|
|
|
|
|
|
# Debugging
|
|
|
|
There are times when it's helpful to enable remote debugging of the Python code. Because NiFi is responsible
|
|
for launching the Python process, how to enable this may not be as straight-forward as when launching a Python
|
|
process yourself. However, NiFi can be told to enable remote debugging when launching the Python process.
|
|
|
|
The manner in which you connect to the Python Process differs by IDE. Here, we will examine how to use
|
|
VSCode's DebugPy.
|
|
|
|
## Debugging Framework
|
|
|
|
The method for debugging the framework and debugging Processors is different. Typically, when performing debugging
|
|
on the Framework itself, it is easiest to have NiFi enable a DebugPy listener when launching the Python process
|
|
that hosts the Controller.
|
|
|
|
To enable remote debugging, NiFi will use `pip` to install the `debugpy` module into the environment used by the main Python process.
|
|
This process is used to discover available Processors and to create Processors. It is not used by Processors themselves.
|
|
|
|
|
|
### Listen for Incoming Connections (Controller)
|
|
The following properties may be added to nifi.properties in order to enable remote debugging
|
|
of the Controller process:
|
|
|
|
`nifi.python.controller.debugpy.enable` : Indicates whether or not DebugPy should be used when launching hte Controller.
|
|
Defaults to `false`. If set to `true`, the Python process that is responsible for discovering and creating Processors
|
|
will be launched using DebugPy.
|
|
|
|
`nifi.python.controller.debugpy.port` : The local port to use. Defaults to `5678`.
|
|
|
|
`nifi.python.controller.debugpy.host` : The hostname to listen on. Defaults to `localhost`.
|
|
|
|
`nifi.python.controller.debugpy.logs.directory` : The directory to write DebugPy logs to. Defaults to `./logs`
|
|
|
|
|
|
Note that these properties do not exist in the nifi.properties by default.
|
|
This is intentional and is due to the fact that during any normal operations, this should not be used.
|
|
This should be used only by developers wanting to debug the NiFi application itself.
|
|
|
|
|
|
### Connecting to the Python Process
|
|
|
|
It is important, however, to note the host and port that the debugger is using.
|
|
When establishing a connection to the remote debugger, the VSCode may be configured with both the local
|
|
directory to use for Python source files, as well as the remote debugger.
|
|
|
|
Generally, the local directory should point to `${NIFI_SOURCE_DIRECTORY}/nifi-nar-bundles/nifi-py4j-bundle/nifi-python-framework/src/main/python/framework`.
|
|
The remote directory, which defaults to `.` should be specified as `./python/framework`.
|
|
|
|
|
|
## Debugging Processors
|
|
|
|
It is also important to enable remote debugging for Processors. We expect Processor developers to be able
|
|
to do this, not just those who are maintaining the NiFi codebase. As a result, instructions for enabling
|
|
remote debugging of Processors has been added to the NiFi Python Developer's Guide.
|