summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorRasmus Luha <rasmus.luha@ut.ee>2025-04-23 23:24:51 +0300
committerRasmus Luha <rasmus.luha@ut.ee>2025-04-23 23:24:51 +0300
commit0c503f6cbcb8c74dbc06b7355744aa637cac7172 (patch)
tree46e04a3741aa8f06255d5cde416b2bc0f12e2b28
parentc341ab0ab1697a26f09cc8f9131ca1d9f158034e (diff)
update README.md, add interactive mode measurement name input
-rw-r--r--README.md42
-rw-r--r--config.py11
-rw-r--r--modules/nifi/core.py10
3 files changed, 53 insertions, 10 deletions
diff --git a/README.md b/README.md
index 87cc65b..259186f 100644
--- a/README.md
+++ b/README.md
@@ -2,18 +2,54 @@
This is a pipeline generator that currently supports the following platforms: `Nifi`, `Telegraf`(TODO).
## Usage
-WIP
- `source venv/bin/activate`
- `pip install -r requirements.txt`
- Setup sutff in `config.py` (database and nifi auth etc)
- Then run `main.py`
+WIP
## Configuration info
-- Can configure under config.py
-- sample VAR-s are given
+Configuration variables are defined in `config.py`, where sample Variables are already given.
WIP
+### Variables
+
+#### Nifi
+
+| Variable Name | Description | Example Value |
+|---------------------|-------------------------------------------------|----------------------------|-
+| `NIFI_HOST` | Base URL of the Apache NiFi instance | `https://127.0.0.1.nip.io` |
+| `NIFI_USER` | Username for authenticating in Nifi | `nifi_username` |
+| `NIFI_PASS` | Username for authenticating in Nifi | `nifi_password` |
+| `NIFI_DEPLOY` | Automatically delpoy pipeline to Nifi | `True` |
+| `INTERACTIVE_MODE` | If False you can parse through api call data | `True` |
+
+In interactive mode you can manually give the API url, credentials and data fields for the pipeline from API response
+If you opt out the interactive mode you have to define needed variables in the config.py file before using the tool.
+
+The following variables have to be defined if interactive mode is tured to `False`
+
+| Variable Name | Description | Example Value |
+|-------------------------------|-------------------------------------------|---------------------------------------------------------------------------------------------|
+| API_URL | API url that returens JSON data | https://api.open-meteo.com/v1/forecast?latitude=58.38&longitude=26.72&current_weather=true" |
+| API_FIELDS | Data fields for pipeline, with json paths | {'temperature': '.current_weather.temperature', 'windspeed': '.current_weather.windspeed'} |
+| API_USERNAME | Api username, if required | Placeholder" |
+| API_PASSWORD | Api username, if required | Placehoder" |
+| PIPELINE_SCHEDULING_PERIOD | How often the pipeline should run | 5 sec |
+| PIPELINE_NAME | Name of the pipeline | test_pipeline.json |
+
+
+#### Database
+Database variales have to be filled out before using the tool as currently there is only influxDB support.
+
+| Variable Name | Description | Example Value |
+|---------------------|------------------------------------------|--------------------------------------------------|
+| `DB_URL` | Url to the influxDB for the pipeline | `http://influxdb:8086/write?db=nifi_weatherData` |
+| `DB_USER` | Database usernmae | `admin` |
+| `DB_PASS` | Database passwod | `admin` |
+
+
## Current Repo structure
```
diff --git a/config.py b/config.py
index 780bc9b..f5d4554 100644
--- a/config.py
+++ b/config.py
@@ -1,13 +1,14 @@
-INTERACTIVE_MODE=False
+INTERACTIVE_MODE=True
## Nifi
-NIFI_USER="lab08nifiuser"
-NIFI_PASS="tartunifi2023"
-
NIFI_HOST="https://127.0.0.1.nip.io"
NIFI_DEPLOY=False
-NIFI_MEASUREMENT_NAME="test_measurementName"
+NIFI_USER="lab08nifiuser"
+NIFI_PASS="tartunifi2023"
+
+#NIFI_MEASUREMENT_NAME="test_measurementName"
+NIFI_MEASUREMENT_NAME=""
## Database
diff --git a/modules/nifi/core.py b/modules/nifi/core.py
index 4b7f748..4ddc268 100644
--- a/modules/nifi/core.py
+++ b/modules/nifi/core.py
@@ -130,8 +130,14 @@ def modify_all_processors(data_values, schedulingPeriod, new_pipeline_name, api_
- ### Processor editing - TODO only from config file currently
- measurements_name = config.NIFI_MEASUREMENT_NAME+" "
+ ### Processor editing
+
+ ## Measurements name defining
+ if config.INTERACTIVE_MODE:
+ measurements_name = str(input("Palun sisesta andmebaasi jaoks vajalik 'measurement' nimi (influxDB): "))
+ else:
+ measurements_name = config.NIFI_MEASUREMENT_NAME+" "
+
if needs_SplitJson:
## SplitJson update