c-l/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2025-01-01 08:31:59 +00:00
Code Releases Activity

feat(ml): support dynamic scaling (#12065)

Browse source 
feat(ml): make http keep-alive configurable

Closes #12064
This commit is contained in:
Richard Kojedzinszky 2024-08-29 17:11:49 +02:00 committed by GitHub
parent 9f5a3f1e84
commit f3e176e192
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 20 additions and 15 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
docs/docs/install/environment-variables.md
View file

@ -160,7 +160,7 @@ Redis (Sentinel) URL example JSON before encoding:
## Machine Learning ## Machine Learning
| Variable | Description | Default | Containers | | Variable | Description | Default | Containers |
| :----------------------------------------------- | :-------------------------------------------------------------------------------------------------- | :-----------------------------------: | :--------------- | | :-------------------------------------------------------- | :-------------------------------------------------------------------------------------------------- | :-----------------------------------: | :--------------- |
| `MACHINE_LEARNING_MODEL_TTL` | Inactivity time (s) before a model is unloaded (disabled if \<= 0) | `300` | machine learning | | `MACHINE_LEARNING_MODEL_TTL` | Inactivity time (s) before a model is unloaded (disabled if \<= 0) | `300` | machine learning |
| `MACHINE_LEARNING_MODEL_TTL_POLL_S` | Interval (s) between checks for the model TTL (disabled if \<= 0) | `10` | machine learning | | `MACHINE_LEARNING_MODEL_TTL_POLL_S` | Interval (s) between checks for the model TTL (disabled if \<= 0) | `10` | machine learning |
| `MACHINE_LEARNING_CACHE_FOLDER` | Directory where models are downloaded | `/cache` | machine learning | | `MACHINE_LEARNING_CACHE_FOLDER` | Directory where models are downloaded | `/cache` | machine learning |
@ -168,6 +168,7 @@ Redis (Sentinel) URL example JSON before encoding:
| `MACHINE_LEARNING_MODEL_INTER_OP_THREADS` | Number of parallel model operations | `1` | machine learning | | `MACHINE_LEARNING_MODEL_INTER_OP_THREADS` | Number of parallel model operations | `1` | machine learning |
| `MACHINE_LEARNING_MODEL_INTRA_OP_THREADS` | Number of threads for each model operation | `2` | machine learning | | `MACHINE_LEARNING_MODEL_INTRA_OP_THREADS` | Number of threads for each model operation | `2` | machine learning |
| `MACHINE_LEARNING_WORKERS`<sup>\*2</sup> | Number of worker processes to spawn | `1` | machine learning | | `MACHINE_LEARNING_WORKERS`<sup>\*2</sup> | Number of worker processes to spawn | `1` | machine learning |
| `MACHINE_LEARNING_HTTP_KEEPALIVE_TIMEOUT_S`<sup>\*3</sup> | HTTP Keep-alive time in seconds | `2` | machine learning |
| `MACHINE_LEARNING_WORKER_TIMEOUT` | Maximum time (s) of unresponsiveness before a worker is killed | `120` (`300` if using OpenVINO image) | machine learning | | `MACHINE_LEARNING_WORKER_TIMEOUT` | Maximum time (s) of unresponsiveness before a worker is killed | `120` (`300` if using OpenVINO image) | machine learning |
| `MACHINE_LEARNING_PRELOAD__CLIP` | Name of a CLIP model to be preloaded and kept in cache | | machine learning | | `MACHINE_LEARNING_PRELOAD__CLIP` | Name of a CLIP model to be preloaded and kept in cache | | machine learning |
| `MACHINE_LEARNING_PRELOAD__FACIAL_RECOGNITION` | Name of a facial recognition model to be preloaded and kept in cache | | machine learning | | `MACHINE_LEARNING_PRELOAD__FACIAL_RECOGNITION` | Name of a facial recognition model to be preloaded and kept in cache | | machine learning |
@ -179,6 +180,8 @@ Redis (Sentinel) URL example JSON before encoding:
\*2: Since each process duplicates models in memory, changing this is not recommended unless you have abundant memory to go around. \*2: Since each process duplicates models in memory, changing this is not recommended unless you have abundant memory to go around.
\*3: For scenarios like HPA in K8S. https://github.com/immich-app/immich/discussions/12064
:::info :::info
Other machine learning parameters can be tuned from the admin UI. Other machine learning parameters can be tuned from the admin UI.

2
machine-learning/start.sh
View file

@ -13,6 +13,7 @@ fi
: "${IMMICH_HOST:=[::]}" : "${IMMICH_HOST:=[::]}"
: "${IMMICH_PORT:=3003}" : "${IMMICH_PORT:=3003}"
: "${MACHINE_LEARNING_WORKERS:=1}" : "${MACHINE_LEARNING_WORKERS:=1}"
: "${MACHINE_LEARNING_HTTP_KEEPALIVE_TIMEOUT_S:=2}"
gunicorn app.main:app \ gunicorn app.main:app \
-k app.config.CustomUvicornWorker \ -k app.config.CustomUvicornWorker \
@ -20,4 +21,5 @@ gunicorn app.main:app \
-w "$MACHINE_LEARNING_WORKERS" \ -w "$MACHINE_LEARNING_WORKERS" \
-t "$MACHINE_LEARNING_WORKER_TIMEOUT" \ -t "$MACHINE_LEARNING_WORKER_TIMEOUT" \
--log-config-json log_conf.json \ --log-config-json log_conf.json \
--keep-alive "$MACHINE_LEARNING_HTTP_KEEPALIVE_TIMEOUT_S" \
--graceful-timeout 0 --graceful-timeout 0