Nifi-Python-Api: A rich Apache NiFi Python Client SDK
- Three layers of Python support for working with Apache NiFi:
- High-level Demos and example scripts
- Mid-level Client SDK for typical complex tasks
- Low-level Client SDKs for the full API implementation of NiFi and selected sub-projects
- Functionality Highlights:
- Detailed documentation of the full SDK at all levels
- CRUD wrappers for common task areas like Processor Groups, Processors, Templates, Registry Clients, Registry Buckets, Registry Flows, etc.
- Convenience functions for inventory tasks, such as recursively retrieving the entire canvas, or a flat list of all Process Groups
- Support for scheduling and purging flows, controller services, and connections
- Support for fetching and updating Variable Registries
- Support for import/export of Versioned Flows from NiFi-Registry
- Docker Compose configurations for testing and deployment
- A scripted deployment of an interactive environment, and a secured configuration, for testing and demonstration purposes
Please see the issue register for more information on current development.
There are several scripts to produce demo environments in nipyapi.demo.*
The mid-level functionality is in nipyapi.canvas / nipyapi.security / nipyapi.templates / nipyapi.versioning
You can access the entire API using the low-level SDKs in nipyapi.nifi / nipyapi.registry
The easiest way to install NiPyApi is with pip:
# in bash pip install nipyapi
You can set the config for your endpoints in the central config file:
# in python import nipyapi nipyapi.config.nifi_config.host = 'http://localhost:8080/nifi-api' nipyapi.config.registry_config.host = 'http://localhost:18080/nifi-registry-api'
Then import a module and execute tasks:
You can use the Docker demos to create a secured interactive console showing many features:
from nipyapi.demo.secured_console import * from nipyapi.demo.console import *
You can also explore the scripts to get ideas for how NiPyAPi can be used to automate your environment.
Please check out the Contribution Guide if you are interested in contributing to the feature set.
Background and Documentation¶
For more information on Apache NiFi, please visit https://nifi.apache.org
For Documentation on this package please visit https://nipyapi.readthedocs.io.
NiFi Version Support¶
Currently we are testing against NiFi versions 1.1.2 - 1.15.0, and NiFi-Registry versions 0.1.0 - 1.15.0.
If you find a version compatibility problem please raise an issue
Python 2.7 or 3.6-9 supported, though other versions may work. We will shortly stop supporting Python2 There are known issues on Python 3.10
Tested on AL2 and OSX 10.14.x - Windows automated testing not attempted
Outside of the standard Python modules, we make use of lxml, DeepDiff, ruamel.yaml and xmltodict in processing, and Docker for demo/tests.