1
0
Fork 0
mirror of https://github.com/sdr-enthusiasts/docker-adsb-ultrafeeder.git synced 2024-12-26 14:22:00 +00:00

Fix and document mlat_config, add universal ULTRAFEEDER_CONFIG parameter and support (#8)

This commit is contained in:
kx1t 2023-04-19 15:28:02 -04:00 committed by GitHub
commit 14cec384eb
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
11 changed files with 346 additions and 94 deletions

1
.env
View file

@ -7,5 +7,6 @@ FEEDER_ALT_M=xx
FEEDER_ALT_FT=xxx
FEEDER_NAME=my_feeder_name
MULTIFEEDER_UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
ADSBX_UUID=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
FEEDER_HEYWHATSTHAT_ID=xxxxxxxxxxxx
FEEDER_HEYWHATSTHAT_ALTS=3000,10000,30000,40000

View file

@ -1,5 +1,6 @@
ignored:
- DL3003
- DL3007
- DL3008
- SC3054
- SC3044

6
.markdownlint.json Normal file
View file

@ -0,0 +1,6 @@
{
"MD024": false,
"MD013": false,
"MD033": false,
"MD029": false
}

View file

@ -129,8 +129,8 @@ docker compose up -d
You should be able to point your web browser at:
* `http://docker.host.ip.addr:9090/` to access the `prometheus` console.
* `http://docker.host.ip.addr:3000/` to access the `grafana` console, use admin/admin as initial credentials, you should be prompted to change the password on first login.
- `http://docker.host.ip.addr:9090/` to access the `prometheus` console.
- `http://docker.host.ip.addr:3000/` to access the `grafana` console, use admin/admin as initial credentials, you should be prompted to change the password on first login.
Remember to change `docker.host.ip.addr` to the IP address of your docker host.
@ -145,7 +145,7 @@ After you have logged into the `grafana` console the following manual steps are
Option | Input
------------- | -------------
Name | readsb
URL | http://prometheus:9090/
URL | `http://prometheus:9090/`
Clicking `Save & Test` should return a green message indicating success. The dashboard can now be imported with the following steps

View file

@ -103,8 +103,8 @@ The sections below describe how to configure each part of the container function
Note:
- to enable a parameter, you can set it to any of `1`, `true`, `on`, `enabled`, `enable`, `yes`, or `y`. In the table below, we'll simply use `true` for convenience.
- to disable a parameter, you can set it to anything else or simply leave it undefined.
* to enable a parameter, you can set it to any of `1`, `true`, `on`, `enabled`, `enable`, `yes`, or `y`. In the table below, we'll simply use `true` for convenience.
* to disable a parameter, you can set it to anything else or simply leave it undefined.
### General Configuration
@ -153,7 +153,7 @@ The following parameters must be set (mandatory) for the container to function:
| `S6_SERVICES_GRACETIME` | Optional, set to 30000 when saving traces / globe_history | | `3000` |
| `LOGLEVEL` | `verbose` (all messages), `error` (errors only), `none` (minimal) | | `verbose` |
`READSB_EXTRA_ARGS` just passes arguments to the commandline, you can check this file for more options for wiedehofp's readsb fork: <https://github.com/wiedehopf/readsb/blob/dev/help.h>
`READSB_EXTRA_ARGS` just passes arguments to the commandline, you can check this file for more options for wiedehopf's readsb fork: <https://github.com/wiedehopf/readsb/blob/dev/help.h>
### Getting ADSB data to the Ultrafeeder
@ -218,7 +218,41 @@ docker exec -it ultrafeeder /usr/local/bin/autogain1090 reset
In addition to (or instead of) connecting to a SDR or hardware device to get ADSB data, the container also supports ingesting data from a TCP port. Here are some parameters that you need to configure if you want to make this happen:
##### Mandatory parameters
##### All-in-One Configuration using `ULTRAFEEDER_CONFIG`
`ULTRAFEEDER_CONFIG` is a new parameter that can be used instead of separately defining `READSB_NET_CONNECTOR`, `MLAT_NET_CONNECTOR`, `MLATHUB_NET_CONNECTOR`/`MLATHUB_CONFIG`. These legacy parameters will still work; however, we wanted to provide a single parameter that enables configuration of incoming and outgoing ADSB data, MLAT-client data, and MLATHUB data.
Note that `ULTRAFEEDER_CONFIG` and `ULTRAFEEDER_NET_CONNECTOR` can be used interchangeably; in this documentation, we'll use `ULTRAFEEDER_CONFIG` as an example.
`ULTRAFEEDER_CONFIG` syntax:
The ULTRAFEEDER_CONFIG parameter can have multiple config strings, separated by a `;`. Please note that the config strings cannot contain `;` or `,` -- undefined things may happen if these characters are present.
```yaml
- ULTRAFEEDER_CONFIG=adsb,host,port,protocol[,uuid=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX][,extra-arguments]
...or...
- ULTRAFEEDER_CONFIG=mlat,host,port[,return_port][,uuid=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX][,extra-arguments]
...or...
- ULTRAFEEDER_CONFIG=mlathub,host,port,protocol[,uuid=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX][,extra-arguments]
```
In the above configuration strings:
* `host` is an IP address. Specify an IP/hostname/containername for incoming or outgoing connections.
* `port` and `return_port` are TCP port numbers
* `protocol` can be one of the following:
* `beast_reduce_out`: Beast-format output with lower data throughput (saves bandwidth and CPU)
* `beast_reduce_plus_out`: Beast-format output with extra data (UUID). This is the preferred format when feeding the "new" aggregator services
* `beast_out`: Beast-format output
* `beast_in`: Beast-format input
* `raw_out`: Raw output
* `raw_in`: Raw input
* `sbs_out`: SBS-format output
* `vrs_out`: SBS-format output
* `uuid=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX` is an optional parameter that sets the UUID for this specific instance. It will override the global `UUID` parameter. This is only needed when you want to send different UUIDs to different aggregators.
* `extra-arguments` can be any additional command line argument you want to pass to readsb, mlathub, or mlat-client. Example: `--net-only`. Please make sure to only once pass in an extra argument for each of the adsb|mlat|mlathub service.
##### Networking parameters
| Environment Variable | Purpose | Default |
|----------------------|---------|---------|
@ -227,7 +261,7 @@ In addition to (or instead of) connecting to a SDR or hardware device to get ADS
| `MLATHOST` | Legacy parameter. IP/Hostname of an MLAT provider (`mlat-client`). Note - using this parameter will not make the MLAT data part of the consolidated mlathub. The preferred way of ingesting MLAT results is using the `mlathub` functionality of the container, see below for details | |
| `MLATPORT` | Legacy parameter used with `MLATHOST`. TCP port number of an MLAT provider (`mlat-client`) | 30105 |
###### Alternate Configuration Method with `READSB_NET_CONNECTOR`
##### Alternate Configuration Method with `READSB_NET_CONNECTOR`
Instead of (or in addition to) using `BEASTHOST`, you can also define ADSB data ingests using the `READSB_NET_CONNECTOR` parameter. This is the preferred way if you have multiple sources or destinations for your ADSB data. This variable allows you to configure incoming and outgoing connections. The variable takes a semicolon (`;`) separated list of `host,port,protocol[,uuid=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX]`, where:
@ -270,7 +304,7 @@ There are many optional parameters relating to the ingestion of data and the gen
| `READSB_NET_RAW_OUTPUT_INTERVAL` | TCP output flush interval in seconds (maximum interval between two network writes of accumulated data). | `--net-ro-interval=<rate>` | `0.05` |
| `READSB_NET_RAW_OUTPUT_SIZE` | TCP output flush size (maximum amount of internally buffered data before writing to network). | `--net-ro-size=<size>` | `1200` |
| `READSB_NET_CONNECTOR_DELAY` | Outbound re-connection delay. | `--net-connector-delay=<seconds>` | `30` |
| `READSB_NET_HEARTBEAT` | TCP heartbeat rate in seconds (0 to disable). | `--net-heartbeat=<rate>` | `60` |
| `READSB_NET_HEARTBEAT` | TCP heartbeat rate in seconds (0 to disable). | `--net-heartbeat=<rate>` | `35` |
| `READSB_NET_RAW_INPUT_PORT` | TCP raw input listen ports. | `--net-ri-port=<ports>` | `30001` |
| `READSB_NET_RAW_OUTPUT_PORT` | TCP raw output listen ports. | `--net-ro-port=<ports>` | `30002` |
| `READSB_NET_SBS_INPUT_PORT` | TCP BaseStation input listen ports. | `--net-sbs-in-port=<ports>` | Unset |
@ -282,6 +316,33 @@ There are many optional parameters relating to the ingestion of data and the gen
| `READSB_JSON_INTERVAL` | Update interval for the webinterface in seconds / interval between aircraft.json writes | `--write-json-every=<sec>` | `1.0` |
| `READSB_JSON_TRACE_INTERVAL` | Per plane interval for json position output and trace interval for globe history | `--json-trace-interval=<sec>` | `15` |
| `READSB_FORWARD_MLAT_SBS` | If set to anthing, it will include MLAT results in the SBS/BaseStation output. This may be desirable if you feed SBS data to applications like [VRS](https://github.com/sdr-enthusiasts/docker-virtualradarserver) or [PlaneFence](https://github.com/kx1t/docker-planefence) | Unset |
| `UUID` | Sets the UUID that is sent on the `beast_reduce_plus` port if no individual UUIDs have been defined with the `READSB_NET_CONNECTOR` parameter. Similarly, it's also used with `mlat-client` (see below) if no individual UUIDs have been set with the `MLAT_CONFIG` parameter. | | unset |
##### MLAT configuration
The Ultrafeeder contains a capability to send MLAT data to MLAT servers to be processed, and to receive the MLAT results and integrate those with an MLAT Hub and the `tar1090` map.
It will create a separate instance of `mlat-client` for each defined MLAT server. The parameters for these `mlat-client` instances is as follows:
```yaml
environment:
...
- MLAT_NET_CONNECTOR=
mlat-server1.com,port1,return_port1,uuid=1234-5678-90123,--extra-argument hello;
mlat-server2.com,port2,return_port2,uuid=5678-9012-34567,--extra-argument hello
...
```
where:
* `mlat-server.com` is the domain name or ip address of the target MLAT server (mandatory)
* `port` is the port (TCP or UDP) of the target MLAT server (mandatory)
* `return_port` is the port at which the MLAT results are made available in BEAST format. We recommend to sequentially number them starting at 39000 (optional)
* `uuid=xxxx` defines a unique user ID for this MLAT server instance. If included, the string must start with `uuid=` (optional)
* `extra-arguments` are any additional command line arguments that you would like to use for this MLAT Client instance (optional)
Note - the three optional parameters (`return_port`, `uuid=`, and `extra-arguments`) can be given in any order.
If no UUID is specified with the `MLAT_CONFIG` parameter, it will use the value of the `UUID` parameter if it exists. If that fails, no UUID parameter is included with `mlat-client`.
### Web Gui (`tar1090`) Configuration
@ -291,7 +352,7 @@ The web interface is rendered to port `80` in the container. This can me mapped
All of the variables below are optional.
Note - due to design limitations of `readsb`, the `tar1090` graphical interface will by default ONLY show MLAT results from the aggregators/MLAT sources that were defined with the `MLAT_NET_CONNECTOR` parameter. If you want to show any additional MLAT results (for example, those from `piaware`), you should add a separate `READSB_NET_CONNECTOR` for them.
Note - due to design limitations of `readsb`, the `tar1090` graphical interface will by default ONLY show MLAT results from the aggregators/MLAT sources that were defined with the `MLAT_NET_CONNECTOR` parameter. If you want to show any additional MLAT results (for example, those from `piaware`), you should add a separate `READSB_NET_CONNECTOR` entry for them.
#### `tar1090` Core Configuration

View file

@ -34,32 +34,27 @@ services:
# --------------------------------------------------
# Sources and Aggregator connections:
# (Note - remove the ones you are not using / feeding)
- READSB_NET_CONNECTOR=
feed.adsb.fi,30004,beast_reduce_plus_out;
in.adsb.lol,30004,beast_reduce_plus_out;
feed.adsb.one,64004,beast_reduce_plus_out;
feed.planespotters.net,30004,beast_reduce_plus_out;
feed.theairtraffic.com,30004,beast_reduce_plus_out
# If you really want to feed ADSBExchange, you can do so by adding this above: feed1.adsbexchange.com,30004,beast_reduce_plus_out
- UUID=${MULTIFEEDER_UUID}
- ULTRAFEEDER_CONFIG=
adsb,feed.adsb.fi,30004,beast_reduce_plus_out;
adsb,in.adsb.lol,30004,beast_reduce_plus_out;
adsb,feed.adsb.one,64004,beast_reduce_plus_out;
adsb,feed.planespotters.net,30004,beast_reduce_plus_out;
adsb,feed.theairtraffic.com,30004,beast_reduce_plus_out;
mlat,feed.adsb.fi,31090,39000;
mlat,in.adsb.lol,31090,39001;
mlat,feed.adsb.one,64006,39002;
mlat,mlat.planespotters.net,31090,39003;
mlat,feed.theairtraffic.com,31090,39004;
mlathub,piaware,30105,beast_in;
mlathub,rbfeeder,30105,beast_in;
mlathub,radarvirtuel,30105,beast_in
# If you really want to feed ADSBExchange, you can do so by adding this above:
# adsb,feed1.adsbexchange.com,30004,beast_reduce_plus_out,uuid=${ADSBX_UUID};
# mlat,feed.adsbexchange.com,31090,39005,uuid=${ADSBX_UUID}
#
# --------------------------------------------------
# MLAT and MLATHUB related parameters:
# MLAT_CONFIG defines who you are feeding MLAT data to, and on what port(s) the results will become available
- MLAT_CONFIG=
feed.adsb.fi,31090,39000;
in.adsb.lol,31090,39001;
feed.adsb.one,64006,39002;
mlat.planespotters.net,31090,39003;
feed.theairtraffic.com,31090,39004
# If you really want to feed ADSBExchange, you can do so by adding this above: feed.adsbexchange.com,31090,39005
- UUID=${MULTIFEEDER_UUID}
- MLAT_USER=${FEEDER_NAME}
# MLATHUB automatically gets MLAT results data from the feeders defined in MLAT_CONFIG,
# and additionally from the Net Connectors defined with MLATHUB_NET_CONNECTOR
- MLATHUB_NET_CONNECTOR=
piaware,30105,beast_in;
rbfeeder,30105,beast_in;
radarvirtuel,30105,beast_in
#
# --------------------------------------------------
# TAR1090 (Map Web Page) parameters:
@ -100,4 +95,4 @@ services:
tmpfs:
- /run:exec,size=256M
- /tmp:size=128M
- /var/log:size=32M
- /var/log:size=32M

View file

@ -1,3 +0,0 @@
{
"MD024": false
}

View file

@ -1,8 +1,6 @@
#!/usr/bin/with-contenv bash
# shellcheck shell=bash disable=SC1091,SC2015,SC2016
source /scripts/common
#---------------------------------------------------------------------------------------------
# Copyright (C) 2023, Ramon F. Kolb (kx1t)
#
@ -20,7 +18,10 @@ source /scripts/common
#---------------------------------------------------------------------------------------------
APPNAME="mlat-client"
echo "[$(date)][${APPNAME}] Started as an s6 service"
echo "[$(date +"%Y-%m-%d %H:%M:%S")][${APPNAME}] Started as an s6 service"
source /scripts/common
source /scripts/interpret_ultrafeeder_config
MLAT_CMD="/usr/bin/mlat-client"
@ -29,35 +30,42 @@ declare -A pid_array
if [[ -z "${MLAT_CONFIG}" ]]
then
echo "[$(date)][${APPNAME}] Warning: MLAT_CONFIG not defined - MLAT will be disabled."
echo "[$(date +"%Y-%m-%d %H:%M:%S")][${APPNAME}] Warning: MLAT_CONFIG not defined - MLAT will be disabled."
sleep infinity
fi
if [[ -z "${MLAT_USER}" ]] && [[ -z "${UUID}" ]]
then
echo "[$(date)][${APPNAME}] ERROR: either UUID or MLAT_USER must be defined - MLAT will be disabled."
echo "[$(date +"%Y-%m-%d %H:%M:%S")][${APPNAME}] ERROR: either UUID or MLAT_USER must be defined - MLAT will be disabled."
sleep infinity
fi
if [[ -z "$LAT$READSB_LAT" ]]; then
echo "[$(date)][${APPNAME}] ERROR: READSB_LAT or LAT must be defined - MLAT will be disabled."
echo "[$(date +"%Y-%m-%d %H:%M:%S")][${APPNAME}] ERROR: READSB_LAT or LAT must be defined - MLAT will be disabled."
sleep infinity
fi
if [[ -z "$LONG$READSB_LON" ]]; then
echo "[$(date)][${APPNAME}] ERROR: READSB_LON or LONG must be defined - MLAT will be disabled."
echo "[$(date +"%Y-%m-%d %H:%M:%S")][${APPNAME}] ERROR: READSB_LON or LONG must be defined - MLAT will be disabled."
sleep infinity
fi
if [[ -z "$ALT$READSB_ALT" ]]; then
echo "[$(date)][${APPNAME}] ERROR: READSB_ALT or ALT must be defined - MLAT will be disabled."
echo "[$(date +"%Y-%m-%d %H:%M:%S")][${APPNAME}] ERROR: READSB_ALT or ALT must be defined - MLAT will be disabled."
sleep infinity
fi
# MLAT_CONFIG has the following format:
# MLAT_CONFIG=mlatserver_1,mlatserver_port_1[,results_port_1[,extra_params_1]];mlatserver_2,mlatserver_port_2[,results_port_2[,extra_params_2]] etc
# MLAT_CONFIG=mlatserver_1,mlatserver_port_1,x1,y1,z1;mlatserver_2,mlatserver_port_2,x2,y2,z2 etc
# where x1,y1,z1; x22,y2,z2; etc are optional and are interpreted as follows:
# if it's a number, we'll assume it's a port number for return messages
# if it starts with 'uuid=', it's a UUID number for that instance
# anything else will be appended as extra parameter(s) to the mlat-client command line
# parse MLAT_CONFIG string into mlat_configs array
readarray -td ";" mlat_configs < <(printf '%s' "${MLAT_CONFIG// /}")
# Strip any extraneous spaces:
MLAT_CONFIG="${MLAT_CONFIG#"${MLAT_CONFIG%%[![:space:]]*}"}" # strip leading space
MLAT_CONFIG="${MLAT_CONFIG//; /;}"
readarray -td ";" mlat_configs < <(printf '%s' "${MLAT_CONFIG}")
# Now loop through the MLAT_CONFIG items and start up an Mlat_client for each of them:
for instance in "${mlat_configs[@]}"
@ -66,15 +74,19 @@ do
# put individual params into the $params array:
readarray -td "," params < <(printf '%s' "${instance}")
uuid_arg=false
extra_args=""
# Check if the params array has values for the mandatory elements:
if [[ -z "${params[0]}" ]] || [[ -z "${params[1]}" ]]
then
echo "[$(date)][${APPNAME}] ERROR -- MLAT_CONFIG is malformed: \"${instance}\". Stopping MLAT execution."
echo "[$(date +"%Y-%m-%d %H:%M:%S")][${APPNAME}] ERROR -- MLAT_CONFIG is malformed: \"${instance}\". Stopping MLAT execution."
# shellcheck disable=SC2046
kill $(ps -s $$ -o pid=)
sleep infinity
fi
# ------------------------------------------------
# Build the MLAT parameter string:
MLAT_PARAM=(--input-type "${MLAT_INPUT_TYPE}")
MLAT_PARAM+=(--input-connect localhost:30005)
@ -98,40 +110,79 @@ do
MLAT_PARAM+=(--alt "${READSB_ALT}")
fi
[[ -n "${MLAT_USER}" ]] && MLAT_PARAM+=(--user "${MLAT_USER}") || MLAT_PARAM+=(--user "${UUID}")
[[ -n "${params[2]}" ]] && MLAT_PARAM+=("--results beast,listen,${params[2]}") || true
[[ -n "${params[3]}" ]] && MLAT_PARAM+=("${params[3]}") || true
if [[ -n "${MLAT_USER}" ]]; then
MLAT_PARAM+=(--user "${MLAT_USER}")
else
rnd="${RANDOM}"
echo "[$(date +"%Y-%m-%d %H:%M:%S")][${APPNAME}] WARNING: MLAT_USER is not set - using random number \"${rnd}\" as MLAT_USER"
MLAT_PARAM+=(--user "${rnd}")
fi
# Now process the rest of the arguments
# The order doesn't matter, we'll do pattern matching:
# If the argument is a number, then it must be the beast_results port
# If the argument starts with "uuid=", then it must be a UUID
# If the argument isn't any of the above, then it's an "extra argument"
for ((i=2; i<${#params[*]}; i++))
do
if [[ -n "${params[i]}" ]] && [[ "${params[i]}" =~ ^[0-9]+$ ]]; then
# It's a number so it must be the return port
MLAT_PARAM+=("--results beast,listen,${params[i]}")
elif header="${params[i]:0:5}" && [[ "${header,,}" == "uuid=" ]]; then
# It's a UUID
MLAT_PARAM+=("--uuid ${params[i]:5}")
uuid_arg=true
else
# It's an Extra Args string
extra_args="$extra_args ${params[i]}"
fi
done
# If no UUID was set in the argument strings, check if we can use the generic UUID parameter
if [[ $uuid_arg == false ]] && [[ -n "${UUID}" ]]; then
MLAT_PARAM+=("--uuid ${UUID}")
fi
# Now add the extra_args, if any:
[[ -n "${extra_args}" ]] && MLAT_PARAM+=("${extra_args}") || true
# ------------------------------------------------
# Create the command exec string:
# shellcheck disable=SC2048,SC2086
execstring="$(echo ${MLAT_CMD} ${MLAT_PARAM[*]} | xargs)"
# Wait a bit until readsb is established...
if ! pgrep readsb >/dev/null
then
echo "[$(date)][mlathub] Delaying start of MLAT client(s) until container is established..."
while ! pgrep readsb >/dev/null
do
sleep 2
done
fi
sleep 10 # sleep a bit so everything is well established - starting readsb may take a bit
# ------------------------------------------------
# Ready to launch, but wait until readsb is established...
if ! pgrep readsb >/dev/null; then
echo "[$(date +"%Y-%m-%d %H:%M:%S")][$APPNAME] Delaying start of MLAT client(s) until container is established..."
while ! pgrep readsb >/dev/null
do
sleep 2
done
fi
sleep 10 # sleep a bit so everything is well established - starting readsb may take a bit
# run this Mlat_client instance in the background:
echo "[$(date)][${APPNAME}] starting: ${MLAT_CMD} ${MLAT_PARAM[*]}"
echo "[$(date +"%Y-%m-%d %H:%M:%S")][${APPNAME}] starting: ${MLAT_CMD} ${MLAT_PARAM[*]}"
if [[ -z "${LOGLEVEL}" ]] || [[ "${LOGLEVEL,,}" == "verbose" ]]; then
{ exec ${execstring} 2>&1 | stdbuf -o0 awk -v app="${APPNAME}" -v inst="${params[0]}" '{print "[" strftime("%Y/%m/%d %H:%M:%S", systime()) "][" app "][" inst "] " $0}'; } &
{ exec ${execstring} 2>&1 | stdbuf -o0 awk -v app="${APPNAME}" -v inst="${params[0]}" '{print "[" strftime("%Y-%m-%d %H:%M:%S", systime()) "][" app "][" inst "] " $0}'; } &
elif [[ "${LOGLEVEL,,}" == "error" ]]; then
{ exec ${execstring} 2>&1 >/dev/null | stdbuf -o0 awk -v app="${APPNAME}" -v inst="${params[0]}" '{print "[" strftime("%Y/%m/%d %H:%M:%S", systime()) "][" app "][" inst "] " $0}'; } &
{ exec ${execstring} 2>&1 >/dev/null | stdbuf -o0 awk -v app="${APPNAME}" -v inst="${params[0]}" '{print "[" strftime("%Y-%m-%d %H:%M:%S", systime()) "][" app "][" inst "] " $0}'; } &
elif [[ "${LOGLEVEL,,}" == "none" ]]; then
{ exec ${execstring} >/dev/null 2>/dev/null; } &
fi
# pid_array is indexed by the PID of each mlat_client and contains the MLAT_PARAMs for that instance
# This is done so we can monitor them and restart them if needed
pid_array[$!]="${MLAT_PARAM[*]}"
sleep 5
done
# All MLAT Clients are up, so set the flag:
# All MLAT Clients are up, so set the flag. This flag is read by MLATHUB as a sign that it's
# safe to start up
touch /run/mlathub_up
# Now iterate over all MLAT-client instances and check if they are still running:
@ -143,14 +194,14 @@ do
then
# it exited - let's restart:
sleep "${RESTARTTIMER}"
[[ "${LOGLEVEL,,}" != "none" ]] && echo "[$(date)][${APPNAME}] MLAT_Client $(awk '{print $6}' <<< "${pid_array[$mlat_pid]}") exited. Attempting to restart." || true
[[ "${LOGLEVEL,,}" != "none" ]] && echo "[$(date +"%Y-%m-%d %H:%M:%S")][${APPNAME}] MLAT_Client $(awk '{print $6}' <<< "${pid_array[$mlat_pid]}") exited. Attempting to restart." || true
# shellcheck disable=SC2086
execstring="$(echo ${MLAT_CMD} ${pid_array[$mlat_pid]} | xargs)"
if [[ -z "${LOGLEVEL}" ]] || [[ "${LOGLEVEL,,}" == "verbose" ]]; then
{ exec ${execstring} 2>&1 | stdbuf -o0 awk -v app="${APPNAME}" -v inst="${params[0]}" '{print "[" strftime("%Y/%m/%d %H:%M:%S", systime()) "][" app "][" inst "] " $0}'; } &
{ exec ${execstring} 2>&1 | stdbuf -o0 awk -v app="${APPNAME}" -v inst="${params[0]}" '{print "[" strftime("%Y-%m-%d %H:%M:%S", systime()) "][" app "][" inst "] " $0}'; } &
elif [[ "${LOGLEVEL,,}" == "error" ]]; then
{ exec ${execstring} 2>&1 >/dev/null | stdbuf -o0 awk -v app="${APPNAME}" -v inst="${params[0]}" '{print "[" strftime("%Y/%m/%d %H:%M:%S", systime()) "][" app "][" inst "] " $0}'; } &
{ exec ${execstring} 2>&1 >/dev/null | stdbuf -o0 awk -v app="${APPNAME}" -v inst="${params[0]}" '{print "[" strftime("%Y-%m-%d %H:%M:%S", systime()) "][" app "][" inst "] " $0}'; } &
elif [[ "${LOGLEVEL,,}" == "none" ]]; then
{ exec ${execstring} >/dev/null 2>/dev/null; } &
fi

View file

@ -1,15 +1,17 @@
#!/usr/bin/with-contenv bash
# shellcheck shell=bash disable=SC1091,SC2016
# shellcheck shell=bash disable=SC1091,SC2015,SC2016
APPNAME="mlathub"
source /scripts/common
source /scripts/interpret_ultrafeeder_config
if [[ -z "${MLAT_CONFIG}" ]] && [[ -z "$MLATHUB_NET_CONNECTOR" ]]; then
echo "[$(date)][mlathub] No MLAT servers have been defined in MLAT_CONFIG and no external sources have been defined in MLATHUB_NET_CONNECTOR - no need to start MLATHUB"
echo "[$(date "+%Y-%m-%d %H:%M:%S")][${APPNAME}] No MLAT servers have been defined in MLAT_CONFIG and no external sources have been defined in MLATHUB_NET_CONNECTOR - no need to start MLATHUB"
sleep infinity
fi
if chk_enabled "${MLATHUB_DISABLED}"; then
echo "[$(date)][mlathub] MLATHUB is disabled."
echo "[$(date "+%Y-%m-%d %H:%M:%S")][${APPNAME}] MLATHUB is disabled."
sleep infinity
fi
@ -43,33 +45,38 @@ do
# put individual params into the $params array:
readarray -td "," params < <(printf '%s' "${instance}")
if [[ -n "${params[2]}" ]]
then
MLATHUB_CMD+=("--net-connector=localhost,${params[2]},beast_in")
(( mlat_result_sources++ )) || true
fi
for ((i=2; i<${#params[*]}; i++))
do
if [[ -n "${params[i]}" ]] && [[ "${params[i]}" =~ ^[0-9]+$ ]]; then
# the parameter is a number and it is assumed that this is the beast_results port
MLATHUB_CMD+=("--net-connector=localhost,${params[i]},beast_in")
(( mlat_result_sources++ )) || true
break
fi
done
done
if (( mlat_result_sources == 0 ))
then
echo "[$(date)][mlathub] No MLAT servers have been defined in MLAT_CONFIG - no need to start MLATHUB"
echo "[$(date "+%Y-%m-%d %H:%M:%S")][${APPNAME}] No MLAT servers have been defined in MLAT_CONFIG - no need to start MLATHUB"
sleep infinity
fi
# removed as it will cause looping:
# handle MLATHUB_ENABLE_ADSB_INGEST (if set, connect to readsb as input)
# if [[ -n "$MLATHUB_ENABLE_ADSB_INGEST" ]]; then
# MLATHUB_CMD+=("--net-connector=localhost,30006,beast_in")
# fi
# Add any additional net_connectors:
if [[ -n "$MLATHUB_NET_CONNECTOR" ]]; then
# Strip any extraneous spaces:
MLATHUB_NET_CONNECTOR="${MLATHUB_NET_CONNECTOR#"${MLATHUB_NET_CONNECTOR%%[![:space:]]*}"}" # strip leading spaces
MLATHUB_NET_CONNECTOR="${MLATHUB_NET_CONNECTOR//; /;}" # strip space after semicolon
IFS=';' read -r -a MLATHUB_NET_CONNECTOR_ARRAY <<< "$MLATHUB_NET_CONNECTOR"
for NET_CONNECTOR_ELEMENT in "${MLATHUB_NET_CONNECTOR_ARRAY[@]}"
do
MLATHUB_CMD+=("--net-connector=${NET_CONNECTOR_ELEMENT// /}")
MLATHUB_CMD+=("--net-connector=${NET_CONNECTOR_ELEMENT}")
done
fi
# Handle ULTRAFEEDER_CONFIG / ULTRAFEEDER_NET_CONNECTOR elements relevant to readsb:
[[ "${#MLATHUB_CONF_ARR[@]}" -gt 0 ]] && MLATHUB_CMD+=("${MLATHUB_CONF_ARR[@]}") || true
if chk_enabled "${READSB_DEBUG}"; then
MLATHUB_CMD+=("--debug=$READSB_DEBUG")
fi
@ -101,7 +108,7 @@ MLATHUB_CMD+=("--net-heartbeat=${READSB_NET_HEARTBEAT:-35}")
if [[ ! -f /run/mlathub_up ]]
then
echo "[$(date)][mlathub] Delaying MLAT hub start until container is established..."
echo "[$(date "+%Y-%m-%d %H:%M:%S")][${APPNAME}] Delaying MLAT hub start until container is established..."
while [[ ! -f /run/mlathub_up ]]
do
sleep 2
@ -109,18 +116,18 @@ then
fi
sleep 5 # sleep a bit so everything is well established
echo "[$(date)][mlathub] Starting MLATHUB..."
echo "[$(date "+%Y-%m-%d %H:%M:%S")][${APPNAME}] Starting MLATHUB..."
# shellcheck disable=SC2086
if [[ -z "${LOGLEVEL}" ]] || [[ "${LOGLEVEL,,}" == "verbose" ]]; then
"${MLATHUB_BIN}" "${MLATHUB_CMD[@]}" $MLATHUB_EXTRA_ARGS 2>&1 | \
mawk -W Interactive '{print "[" strftime("%Y/%m/%d %H:%M:%S", systime()) "][mlathub] " $0}'
mawk -W Interactive '{print "[" strftime("%Y-%m-%d %H:%M:%S", systime()) "][mlathub] " $0}'
elif [[ "${LOGLEVEL,,}" == "error" ]]; then
"${MLATHUB_BIN}" "${MLATHUB_CMD[@]}" $MLATHUB_EXTRA_ARGS 2>&1 >/dev/null | \
mawk -W Interactive '{print "[" strftime("%Y/%m/%d %H:%M:%S", systime()) "][mlathub] " $0}'
mawk -W Interactive '{print "[" strftime("%Y-%m-%d %H:%M:%S", systime()) "][mlathub] " $0}'
elif [[ "${LOGLEVEL,,}" == "none" ]]; then
"${MLATHUB_BIN}" "${MLATHUB_CMD[@]}" $MLATHUB_EXTRA_ARGS >/dev/null 2>/dev/null
fi
echo "[$(date "+%Y-%m-%d %H:%M:%S")][mlathub] The mlathub instance has exited. Restarting in 10 seconds."
echo "[$(date "+%Y-%m-%d %H:%M:%S")][${APPNAME}] The mlathub instance has exited. Restarting in 10 seconds."
sleep 10

View file

@ -1,7 +1,10 @@
#!/usr/bin/with-contenv bash
# shellcheck shell=bash disable=SC2016,SC1091
# shellcheck shell=bash disable=SC2015,SC2016,SC1091
APPNAME="readsb"
source /scripts/common
source /scripts/interpret_ultrafeeder_config
mkdir -p /run/readsb
@ -61,12 +64,12 @@ if ! chk_enabled "$READSB_NET_SBS_DISABLE_REDUCE"; then
fi
if chk_enabled "${READSB_FORWARD_MLAT}"; then
echo "[$(date +"%Y-%m-%d %H:%M:%S")][readsb] WARNING -- READSB_FORWARD_MLAT has been set! Do not feed the output of this container to any aggregators!"
echo "[$(date +"%Y-%m-%d %H:%M:%S")][${APPNAME}] WARNING -- READSB_FORWARD_MLAT has been set! Do not feed the output of this container to any aggregators!"
READSB_CMD+=("--forward-mlat")
fi
if chk_enabled "${READSB_FORWARD_MLAT_SBS}"; then
echo "[$(date +"%Y-%m-%d %H:%M:%S")][readsb] WARNING -- READSB_FORWARD_MLAT_SBS has been set! Do not feed the SBS (BaseStation) output of this container to any aggregators!"
echo "[$(date +"%Y-%m-%d %H:%M:%S")][${APPNAME}] WARNING -- READSB_FORWARD_MLAT_SBS has been set! Do not feed the SBS (BaseStation) output of this container to any aggregators!"
READSB_CMD+=("--forward-mlat-sbs")
fi
@ -93,6 +96,7 @@ if [[ -n "$READSB_MAX_RANGE" ]]; then
fi
# Handle "--mlat"
# shellcheck disable=SC2153
if chk_enabled "$READSB_MLAT"; then
READSB_CMD+=("--mlat")
fi
@ -168,6 +172,9 @@ fi
# Handle "--net-connector=<ip,port,protocol>"
if [[ -n "$READSB_NET_CONNECTOR" ]]; then
# Strip any extraneous spaces:
READSB_NET_CONNECTOR="${READSB_NET_CONNECTOR#"${READSB_NET_CONNECTOR%%[![:space:]]*}"}" # strip leading space
READSB_NET_CONNECTOR="${READSB_NET_CONNECTOR//; /;}"
IFS=';' read -r -a READSB_NET_CONNECTOR_ARRAY <<< "$READSB_NET_CONNECTOR"
for NET_CONNECTOR_ELEMENT in "${READSB_NET_CONNECTOR_ARRAY[@]}"
do
@ -175,13 +182,16 @@ if [[ -n "$READSB_NET_CONNECTOR" ]]; then
done
fi
# Handle ULTRAFEEDER_CONFIG / ULTRAFEEDER_NET_CONNECTOR elements relevant to readsb:
[[ "${#READSB_CONF_ARR[@]}" -gt 0 ]] && READSB_CMD+=("${READSB_CONF_ARR[@]}") || true
# Handle "--net-connector-delay=<seconds>"
if [[ -n "$READSB_NET_CONNECTOR_DELAY" ]]; then
READSB_CMD+=("--net-connector-delay=$READSB_NET_CONNECTOR_DELAY")
fi
# Handle "--net-heartbeat=<rate>"
READSB_CMD+=("--net-heartbeat=${READSB_NET_HEARTBEAT:-30}")
READSB_CMD+=("--net-heartbeat=${READSB_NET_HEARTBEAT:-35}")
# Handle "--net-ri-port=<ports>"
if [[ -n "$READSB_NET_RAW_INPUT_PORT" ]]; then
@ -320,7 +330,7 @@ elif [[ "${LOGLEVEL,,}" == "none" ]]; then
"${READSB_BIN}" "${READSB_CMD[@]}" $READSB_EXTRA_ARGS >/dev/null 2>/dev/null
fi
echo "[$(date "+%Y-%m-%d %H:%M:%S")][readsb] The main readsb instance has exited. Restarting in 10 seconds."
echo "[$(date "+%Y-%m-%d %H:%M:%S")][${APPNAME}] The main readsb instance has exited. Restarting in 10 seconds."
sleep 10

View file

@ -0,0 +1,123 @@
# shellcheck shell=bash disable=SC2015
#
# This scripts should be sourced by the /etc/services.d/xxx/run modules for
# readsb, mlat-client, and mlathub. It interprets the ULTRAFEEDER_CONFIG / ULTRAFEEDER_NET_CONNECTOR parameter
# and makes individual net-connector strings available for these three modules.
#
#---------------------------------------------------------------------------------------------
# Copyright (C) 2023, Ramon F. Kolb (kx1t)
#
# This program is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
# Free Software Foundation, either version 3 of the License, or (at your option)
# any later version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along with this program.
# If not, see <https://www.gnu.org/licenses/>.
#---------------------------------------------------------------------------------------------
#
# Note:
# ULTRAFEEDER_CONFIG and ULTRAFEEDER_NET_CONNECTOR can be used interchangeably
# If both are defined, they will be considered jointly.
#
# Format:
# ULTRAFEEDER_CONFIG=adsb,host,port,protocol[,uuid=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX][,extra-arguments]
# ULTRAFEEDER_CONFIG=mlat,host,port[,return_port][,uuid=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX][,extra-arguments]
# ULTRAFEEDER_CONFIG=mlathub,host,port,protocol[,uuid=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX][,extra-arguments]
#
# The ULTRAFEEDER_CONFIG parameter can have multiple config strings, separated by a `;`
# Please note that the config strings cannot containe `;` or `,` -- undefined things may happen if these characters are present.
#
# In the above configuration strings:
# `host` is an IP address. Specify an IP/hostname/containername for incoming or outgoing connections.
# `port` is a TCP port number
# `protocol` can be one of the following:
# `beast_reduce_out`: Beast-format output with lower data throughput (saves bandwidth and CPU)
# `beast_reduce_plus_out`: Beast-format output with extra data (UUID). This is the preferred format when feeding the "new" aggregator services
# `beast_out`: Beast-format output
# `beast_in`: Beast-format input
# `raw_out`: Raw output
# `raw_in`: Raw input
# `sbs_out`: SBS-format output
# `vrs_out`: SBS-format output
# `uuid=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX` is an optional parameter that sets the UUID for this specific instance.
# It will override the global `UUID` parameter. This is only needed when you want to send different UUIDs to different aggregators.
# `extra-arguments` can be any additional command line argument you want to pass to readsb, mlathub, or mlat-client
# Example: `--net-only`. Please make sure to only once pass in an extra argument for each of the adsb|mlat|mlathub service.
# Do some prep work:
# combine ULTRAFEEDER_CONFIG and ULTRAFEEDER_NET_CONNECTOR
ULTRAFEEDER_CONFIG="${ULTRAFEEDER_CONFIG}${ULTRAFEEDER_CONFIG:+;}${ULTRAFEEDER_NET_CONNECTOR}"
# Strip any extraneous spaces:
ULTRAFEEDER_CONFIG="${ULTRAFEEDER_CONFIG#"${ULTRAFEEDER_CONFIG%%[![:space:]]*}"}" # strip leading space
ULTRAFEEDER_CONFIG="${ULTRAFEEDER_CONFIG//; /;}"
# Now read each entry and parse it:
READSB_CONF_ARR=()
MLATHUB_CONF_ARR=()
readarray -td ";" configs < <(printf '%s' "${ULTRAFEEDER_CONFIG}")
# Now loop through the MLAT_CONFIG items and start up an Mlat_client for each of them:
for instance in "${configs[@]}"
do
[[ -z "${instance}" ]] && continue || true
# put individual params into the $params array:
readarray -td "," param < <(printf '%s' "${instance}")
# Process based on parameter string type in param[0]:
param[0]="${param[0],,}"
case "${param[0]}" in
adsb)
# parse arguments for main readsb
# build net-connector string into $readsb_str
# and add any extra arguments in as individual entries into READSB_CONF_ARR()
readsb_str="${param[1]},${param[2]},${param[3]}"
for ((i=4; i<${#param[*]}; i++))
do
if [[ "${param[i]:0:5}" == "uuid=" ]]; then
readsb_str="${readsb_str},${param[i]}"
elif [[ -n "${param[i]}" ]]; then
READSB_CONF_ARR+=("${param[i]}")
fi
done
# Now add $readsb_str to the READSB_CONF_ARR() as --net-connector argument:
READSB_CONF_ARR+=("--net-connector=${readsb_str}")
;;
mlat)
# parse arguments for the individual MLAT clients.
# In this case, the mlat-client run module will interpret everything
# as long as it is in the MLAT_CONFIG parameter. So we'll just add that here.
MLAT_CONFIG="${MLAT_CONFIG}${MLAT_CONFIG:+;}${instance#*,}"
;;
mlathub)
# parse arguments for mlat_hub
# build net-connector string into $readsb_str
# and add any extra arguments in as individual entries into MLATHUB_CONF_STR()
mlathub_str="${param[1]},${param[2]},${param[3]}"
for ((i=4; i<${#param[*]}; i++))
do
if [[ "${param[i]:0:5}" == "uuid=" ]]; then
mlathub_str="${readsb_str},${param[i]}"
elif [[ -n "${param[i]}" ]]; then
MLATHUB_CONF_ARR+=("${param[i]}")
fi
done
# Now add $readsb_str to the READSB_CONF_STR as --net-connector argument:
MLATHUB_CONF_ARR+=("--net-connector=${mlathub_str}")
;;
*)
# unknown string; warn and skip
echo "[$(date)][${APPNAME}] Warning: skipping unknown ULTRAFEEDER_CONFIG config element \"${instance}\""
esac
done