gpsd support (#97)

As the branch mlat-gps but with some latest changes.
Also rebased on main.

.hadolint.yaml

@@ -4,3 +4,4 @@ ignored:
   - DL3008
   - SC3054
   - SC3044
+  - DL3015

Dockerfile

@@ -43,6 +43,10 @@ RUN TEMP_PACKAGES=() && \
     ln -s /usr/local/bin/mlat-client /usr/bin/mlat-client && \
     popd && \
     rm -rf /git && \
+    # Compile distance binary
+    curl -sSL https://raw.githubusercontent.com/sdr-enthusiasts/docker-adsb-ultrafeeder/main/downloads/distance-in-meters.c -o /distance-in-meters.c && \
+    gcc -static /distance-in-meters.c -o /usr/local/bin/distance -lm -Ofast && \
+    rm -f /distance-in-meters.c && \
     #
     # Clean up and install POST_PACKAGES:
     apt-get remove -q -y "${TEMP_PACKAGES[@]}" && \

README.md

@@ -38,6 +38,9 @@
       - [Configuring the Core Temperature graphs](#configuring-the-core-temperature-graphs)
       - [Reducing Disk IO for Graphs1090](#reducing-disk-io-for-graphs1090)
     - [`timelapse1090` Configuration](#timelapse1090-configuration)
+  - [Updating your location with GPSD](#updating-your-location-with-gpsd)
+    - [Basic Installation and Configuration of your GPS hardware and `gpsd` drivers](#basic-installation-and-configuration-of-your-gps-hardware-and-gpsd-drivers)
+    - [Optional parameters regulating the restart of `mlat-client` when the location changes](#optional-parameters-regulating-the-restart-of-mlat-client-when-the-location-changes)
   - [Web Pages](#web-pages)
   - [Paths](#paths)
   - [Display of Metrix with Grafana and Prometheus/InfluxDB](#display-of-metrix-with-grafana-and-prometheusinfluxdb)
@@ -728,6 +731,79 @@ Legacy: **We recommend AGAINST enabling this feature** as it has been replaced w
 | `TIMELAPSE1090_INTERVAL` | Snapshot interval in seconds                                                                                                    | `10`    |
 | `TIMELAPSE1090_HISTORY`  | Time saved in hours                                                                                                             | `24`    |
 
+## Updating your location with GPSD
+
+This feature enables you to deploy Ultrafeeder while you are moving around. It will read your current longitude/latitude/altitude from a GPS unit that is connected to `gpsd` on your host system, and ensure that the map will show your current location. It will also restart any `mlat-client` instances once it detects that you moved from your previous location.
+
+### Basic Installation and Configuration of your GPS hardware and `gpsd` drivers
+
+The simplest way of getting this to work is to acquire a ["VK163" USB GPS "Mouse"](https://a.co/d/0D7Tj0n), similar to the one in the link. You can connect this mouse to any USB port on your machine.
+
+For this to work, you should install and configure GPSD to work on your host machine. The `DEVICES` parameter is probably correct as shown below, but you may want to double-check that data is received on that device (`cat /dev/ttyACM0`) and adjust it if needed:
+
+```bash
+sudo apt update && sudo apt install -y gpsd
+cat < EOM | sudo tee /etc/default/gpsd
+# Devices gpsd should collect to at boot time.
+# They need to be read/writeable, either by user gpsd or the group dialout.
+DEVICES="/dev/ttyACM0"
+# Other options you want to pass to gpsd
+GPSD_OPTIONS="-G"
+# Automatically hot add/remove USB GPS devices via gpsdctl
+USBAUTO="true"
+EOM
+cat < EOM | sudo tee /lib/systemd/system/gpsd.socket
+[Unit]
+Description=GPS (Global Positioning System) Daemon Sockets
+
+[Socket]
+ListenStream=/run/gpsd.sock
+ListenStream=[::]:2947
+ListenStream=0.0.0.0:2947
+SocketMode=0600
+BindIPv6Only=yes
+
+[Install]
+WantedBy=sockets.target
+EOM
+sudo systemctl daemon-reload
+sudo systemctl restart gpsd gpsd.socket
+```
+
+Then, you can add the following values to `ultrafeeder` service settings in `docker-compose.yml`:
+
+```yaml
+services:
+...
+  ultrafeeder:
+    ...
+    extra_hosts:
+      - "host.docker.internal:host-gateway"
+    ...
+    environment:
+      ULTRAFEEDER-CONFIG=
+        gpsd,host.docker.internal,2947;
+    ...
+```
+
+Finally, restart the container with `docker compose up -d`
+
+This will:
+
+- install and configure `gpsd` (`/etc/default/gpsd`) so it makes GPS data available on the default TCP port 2947 of your host system
+- configure the ultrafeeder docker container to read GPSD data
+- configure the ultrafeeder container so the hostname `host.docker.internal` always resolves to the IP address of the underlying machine (where `gpsd` is running)
+
+### Optional parameters regulating the restart of `mlat-client` when the location changes
+
+The following parameters are all optional and are subject to change. You don't need to set them unless you want to change the default behavior:
+
+| Environment Variable | Purpose | Default |
+| -------------------- | ------- | ------- |
+| `GPSD_MIN_DISTANCE` | Distance (in meters) that your station must move before it's considered moving (maximum 40 meters) | `20` (meters) |
+| `GPSD_MLAT_WAIT` | The wait period (in seconds) your station must be stationary before mlat is started (minimum 90 seconds) | `90` (seconds) |
+| `GPSD_CHECK_INTERVAL` | How often the container checks for updated location information. (minimum 5 seconds) | `30` (seconds) |
+
 ## Web Pages
 
 If you have configured the container as described above, you should be able to browse to the following web pages:

rootfs/etc/s6-overlay/scripts/mlat-client

@@ -1,5 +1,5 @@
 #!/command/with-contenv bash
-# shellcheck shell=bash disable=SC1091,SC2015,SC2016
+# shellcheck shell=bash disable=SC1091,SC2015,SC2016,SC2001
 
 #---------------------------------------------------------------------------------------------
 # Copyright (C) 2023-2024, Ramon F. Kolb (kx1t) and contributors
@@ -51,19 +51,85 @@ then
     exec sleep infinity
 fi
 
-if [[ -z "$LAT$READSB_LAT" ]]; then
+function check_gpsd() {
-    "${s6wrap[@]}" --args echo "ERROR: READSB_LAT or LAT must be defined - MLAT will be disabled."
+    if (( GPSD == 0 )) || ! [[ -f /run/readsb/gpsd.json ]]; then
-    exec sleep infinity
+        return 1
+    fi
+    if new_lat="$(jq -r .lat /run/readsb/gpsd.json)" \
+        && [[ "$new_lat" != "null" ]] \
+        && new_lon="$(jq -r .lon /run/readsb/gpsd.json)" \
+        && [[ "$new_lon" != "null" ]] \
+        && new_alt="$(jq -r .altMSL /run/readsb/gpsd.json)" \
+        && [[ "$new_alt" != "null" ]] \
+
+    then
+        # yay, vars are set and not null
+        return 0
+    else
+        new_lat=""
+        new_lon=""
+        new_alt=""
+        return 1
+    fi
+}
+
+GPSD=0
+if grep -qs "gpsd" <<< "$ULTRAFEEDER_CONFIG" || grep -qs "gpsd" <<< "$ULTRAFEEDER_NET_CONNECTOR"; then
+    GPSD=1
+    LOCATION_PERSIST=/var/globe_history/gpsd_last_location
+    if [[ -f "$LOCATION_PERSIST" ]]; then
+        read new_lat new_lon new_alt < "$LOCATION_PERSIST"
+    fi
+    # initialize "old" location for gpsd movement detection
+    # use zero island as starting point if location persist does not exit
+    old_lat=${new_lat:-0}
+    old_lon=${new_lon:-0}
+
+    # wait for gpsd to continue with startup
+    "${s6wrap[@]}" --args echo "GPSD configured, waiting for gpsd to provide location data"
+    while ! check_gpsd; do
+        sleep "${GPSD_CHECK_INTERVAL:-30}" & wait $!
+    done
+    "${s6wrap[@]}" --args echo "GPSD has provided location data"
+
+    GPSD_MIN_DISTANCE="${GPSD_MIN_DISTANCE:-20}"
+    # enforce gpsd min distance is no larger than 40m
+    if (( GPSD_MIN_DISTANCE > 40 )); then
+        GPSD_MIN_DISTANCE=40
     fi
 
+    GPSD_CHECK_INTERVAL="${GPSD_CHECK_INTERVAL:-30}"
+    if (( GPSD_CHECK_INTERVAL < 5 )); then
+        GPSD_CHECK_INTERVAL=5
+    fi
+
+    # in seconds
+    no_movement_required=${GPSD_MLAT_WAIT:-90}
+    # enforce it to be longer than the checking interval for implementation reasons
+    if (( no_movement_required < GPSD_CHECK_INTERVAL )); then
+        no_movement_required="${GPSD_CHECK_INTERVAL}"
+    fi
+    # enforce 90 second minimum
+    if (( no_movement_required < 90 )); then
+        no_movement_required=90
+    fi
+    # set no_movement to a number higher than the required time of no movement
+    # this way on startup there is no bogus message printed about starting mlat-clients with a new location
+    no_movement=$(( 2 * no_movement_required ))
+else
+    if [[ -z "$LAT$READSB_LAT" ]]; then
+        "${s6wrap[@]}" --args echo "ERROR: READSB_LAT or LAT must be defined or GPSD must be enabled - MLAT will be disabled."
+        exec sleep infinity
+    fi
     if [[ -z "$LONG$READSB_LON" ]]; then
-    "${s6wrap[@]}" --args echo "ERROR: READSB_LON or LONG must be defined - MLAT will be disabled."
+        "${s6wrap[@]}" --args echo "ERROR: READSB_LON or LONG must be defined or GPSD must be enabled - MLAT will be disabled."
         exec sleep infinity
     fi
     if [[ -z "$ALT$READSB_ALT" ]]; then
-    "${s6wrap[@]}" --args echo "ERROR: READSB_ALT or ALT must be defined - MLAT will be disabled."
+        "${s6wrap[@]}" --args echo "ERROR: READSB_ALT or ALT must be defined or GPSD must be enabled - MLAT will be disabled."
         exec sleep infinity
     fi
+fi
 
 # MLAT_CONFIG has the following format:
 # MLAT_CONFIG=mlatserver_1,mlatserver_port_1,x1,y1,z1;mlatserver_2,mlatserver_port_2,x2,y2,z2 etc
@@ -176,24 +242,25 @@ do
     fi
 
     # add LAT/LON/ALT to instance:
-    if [[ -n "${lat_arg}" ]]; then
+
-        MLAT_PARAM+=(--lat "${lat_arg}")
+    if [[ -n "${new_lat:-$lat_arg}" ]]; then
+        MLAT_PARAM+=(--lat "${new_lat:-$lat_arg}")
     elif [[ -n "${LAT}" ]]; then
         MLAT_PARAM+=(--lat "${LAT}")
     elif [[ -n "${READSB_LAT}" ]]; then
         MLAT_PARAM+=(--lat "${READSB_LAT}")
     fi
 
-    if [[ -n "${lon_arg}" ]]; then
+    if [[ -n "${new_lon:-$lon_arg}" ]]; then
-        MLAT_PARAM+=(--lon "${lon_arg}")
+        MLAT_PARAM+=(--lon "${new_lon:-$lon_arg}")
     elif [[ -n "${LONG}" ]]; then
         MLAT_PARAM+=(--lon "${LONG}")
     elif [[ -n "${READSB_LON}" ]]; then
         MLAT_PARAM+=(--lon "${READSB_LON}")
     fi
 
-    if [[ -n "${alt_arg}" ]]; then
+    if [[ -n "${new_alt:-$alt_arg}" ]]; then
-        MLAT_PARAM+=(--alt "${alt_arg}")
+        MLAT_PARAM+=(--alt "${new_alt:-$alt_arg}")
     elif [[ -n "${ALT}" ]]; then
         MLAT_PARAM+=(--alt "${ALT}")
     elif [[ -n "${READSB_ALT}" ]]; then
@@ -217,6 +284,12 @@ do
     # shellcheck disable=SC2048,SC2086
     execstring="$(echo ${MLAT_CMD} ${MLAT_PARAM[*]} | xargs)"
 
+    if (( GPSD == 1 )); then
+        # when GPSD is active, just write the pid array, mlat-client will be started by the gpsd checking logic later
+        # use a long random PID so that it's detected as "not running"
+        pid_array["${RANDOM}${RANDOM}${RANDOM}"]="${MLAT_PARAM[*]}"
+        continue
+    fi
 
     # stagger by 15 second so they don't all start at the same time
     sleep "${MLAT_STARTUP_STAGGER:-15}" & wait $!
@@ -244,12 +317,61 @@ sleep 5 & wait $!
 # Now iterate over all MLAT-client instances and check if they are still running:
 while true
 do
+    if (( GPSD == 1 )); then
+        if ! check_gpsd; then
+            # don't do mlat client restarts if GPSD is configured but not providing a position
+            sleep "${GPSD_CHECK_INTERVAL}" & wait $!
+            continue
+        fi
+
+        distance="$(distance "$old_lat" "$old_lon" "$new_lat" "$new_lon")"
+        if ! [[ -f "$LOCATION_PERSIST" ]]; then
+            echo "$new_lat" "$new_lon" "$new_alt" > "$LOCATION_PERSIST"
+        fi
+        if (( ${distance%%.*} > ${GPSD_MIN_DISTANCE:-20} )); then
+
+            msg="Receiver moved ${distance%%.*} meters"
+
+            # kill the mlat-client instances so they get restarted with the new GPS coords
+            if pkill -f "/usr/bin/python3 /usr/bin/mlat-client" >/dev/null 2>&1; then
+                msg+=" - Stopping all mlat-clients"
+            fi
+
+            "${s6wrap[@]}" --args echo "${msg}"
+
+            old_lat="$new_lat"
+            old_lon="$new_lon"
+
+            # new location means the receiver has moved, sleep a bit and then check again
+            no_movement=0
+            sleep "${GPSD_CHECK_INTERVAL:-30}" & wait $!
+            # as the recevier has moved, mlat-clients are not restarted until there has been no movement for some time
+            # thus we continue skipping the mlat-client restart section of the loop
+            continue
+        else
+            # no movement during the checking interval, allow mlat-clients to be restarted
+            (( no_movement += ${GPSD_CHECK_INTERVAL:-30} )) || true
+            if (( no_movement < no_movement_required )); then
+                msg="Receiver moved less than ${GPSD_MIN_DISTANCE} meters, $(( no_movement_required - no_movement )) seconds remaining before starting mlat-clients"
+                "${s6wrap[@]}" --args echo "${msg}"
+                sleep "${GPSD_CHECK_INTERVAL:-30}" & wait $!
+                continue
+            elif (( no_movement / ${GPSD_CHECK_INTERVAL:-30} == no_movement_required / ${GPSD_CHECK_INTERVAL:-30} )); then
+                "${s6wrap[@]}" --args echo "Receiver stationary - starting all mlat-clients with new location"
+                echo "$new_lat" "$new_lon" "$new_alt" > "$LOCATION_PERSIST"
+            fi
+        fi
+    fi
+
     for mlat_pid in "${!pid_array[@]}"
     do
         if ! kill -0 "${mlat_pid}" >/dev/null 2>&1
         then
             # it exited - let's restart:
+            if [[ ! -f /run/readsb/gpsd.json ]] || [[ "$(jq -r .lat /run/readsb/gpsd.json)" == "null" ]]; then
+                # only sleep for a bit if the restarts aren't caused by GPS movement:
                 sleep "${RESTARTTIMER}" & wait $!
+            fi
             servername="$(awk '{print $4}' <<< "${pid_array[$mlat_pid]}")"
             servername="${servername%%:*}"
 
@@ -257,8 +379,16 @@ do
             # shellcheck disable=SC2086
             execstring="$(echo ${MLAT_CMD} ${pid_array[$mlat_pid]} | xargs)"
 
+            # If GPSD is active, then replace the lat/lon/alt params with the ones from GPSD
+            if (( GPSD == 1 )); then
+                execstring="$(sed "s/^\(.*\s\+--lat\s\+\)[0-9.-]\+\(.*\)$/\1${new_lat}\2/g" <<< "$execstring")"
+                execstring="$(sed "s/^\(.*\s\+--lon\s\+\)[0-9.-]\+\(.*\)$/\1${new_lon}\2/g" <<< "$execstring")"
+                execstring="$(sed "s/^\(.*\s\+--alt\s\+\)[mft0-9.-]\+\(.*\)$/\1${new_alt}m\2/g" <<< "$execstring")"
+            fi
+
             #shellcheck disable=SC2069,SC2086
             if [[ "${LOGLEVEL}" == "verbose" ]]; then
+                "${s6wrap[@]}" --prepend="$(basename "$0")][${servername}" --args echo "Restarting: ${execstring}"
                 "${s6wrap[@]}" --prepend="$(basename "$0")][${servername}" --args ${execstring} &
             elif [[ "${LOGLEVEL}" == "error" ]]; then
                 "${s6wrap[@]}" --ignore=stdout --prepend="$(basename "$0")][${servername}" --args ${execstring} &
@@ -270,5 +400,6 @@ do
             unset "pid_array[${mlat_pid}]"
         fi
     done
-    sleep 10 & wait $!
+
+    sleep "${GPSD_CHECK_INTERVAL:-30}" & wait $!
 done

rootfs/etc/s6-overlay/scripts/readsb

@@ -31,6 +31,14 @@ if ! [[ "$LOGLEVEL" =~ ^(verbose|error|none)$ ]]; then
     LOGLEVEL="verbose"
 fi
 
+LOCATION_PERSIST=/var/globe_history/gpsd_last_location
+if [[ -f "$LOCATION_PERSIST" ]] && { grep -qs "gpsd" <<< "$ULTRAFEEDER_CONFIG" || grep -qs "gpsd" <<< "$ULTRAFEEDER_NET_CONNECTOR"; }; then
+    read LAT LON ALT < "$LOCATION_PERSIST"
+    READSB_LAT=""
+    READSB_LON=""
+    READSB_ALT=""
+fi
+
 # Build the readsb command line based on options
 READSB_BIN="/usr/local/bin/readsb"
 

rootfs/scripts/interpret_ultrafeeder_config

@@ -123,6 +123,15 @@ do
             MLATHUB_CONF_ARR+=("--net-connector=${mlathub_str}")
         ;;
 
+        gpsd)
+            # add gpsd_in parameter to $READSB_CONF_ARR
+            readsb_str="${param[1]},${param[2]}"
+            if [[ -n "${param[3]}" ]] && [[ -n "${param[4]}" ]]; then
+                readsb_str="$readsb_str,${param[3]},${param[4]}"
+            fi
+            READSB_CONF_ARR+=("--net-connector=${readsb_str},gpsd_in")
+        ;;
+
         *)
             # Check if there's anything in ${ULTRAFEEDER_NET_CONNECTOR} -- if not, it's a bad config element. If yes, add it as if it were ADSB
             if [[ -z "${ULTRAFEEDER_NET_CONNECTOR}" ]]; then