1
0
Fork 0

Merge pull request #38 from Salvoxia/dev

Merge features to main
This commit is contained in:
Salvoxia 2024-08-21 12:49:16 +00:00 committed by GitHub
commit 34f4945e4b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 542 additions and 93 deletions

View file

@ -6,6 +6,6 @@ COPY immich_auto_album.py requirements.txt docker/immich_auto_album.sh docker/se
RUN pip install --no-cache-dir -r /script/requirements.txt \
&& chmod +x /script/setup_cron.sh /script/immich_auto_album.sh \
&& rm -rf /tmp/* /var/tmp/* /var/cache/apk/* /var/cache/distfiles/*
ENV IS_DOCKER=1
WORKDIR /script
CMD ["sh", "-c", "/script/setup_cron.sh && crond -f"]

160
README.md
View file

@ -19,7 +19,9 @@ This script is mostly based on the following original script: [REDVM/immich_auto
2. [Usage (Docker)](#docker)
3. [Choosing the correct `root_path`](#choosing-the-correct-root_path)
4. [How It Works (with Examples)](#how-it-works)
5. [Cleaning Up Albums](#cleaning-up-albums)
5. [Automatic Album Sharing](#automatic-album-sharing)
6. [Cleaning Up Albums](#cleaning-up-albums)
7. [Dealing with External Library Changes](#dealing-with-external-library-changes)
## Usage
### Bare Python Script
@ -34,7 +36,9 @@ This script is mostly based on the following original script: [REDVM/immich_auto
```
3. Run the script
```
usage: immich_auto_album.py [-h] [-r ROOT_PATH] [-u] [-a ALBUM_LEVELS] [-s ALBUM_SEPARATOR] [-c CHUNK_SIZE] [-C FETCH_CHUNK_SIZE] [-l {CRITICAL,ERROR,WARNING,INFO,DEBUG}] [-k] [-i IGNORE] [-m {CREATE,CLEANUP,DELETE_ALL}] [-d] root_path api_url api_key
usage: immich_auto_album.py [-h] [-r ROOT_PATH] [-u] [-a ALBUM_LEVELS] [-s ALBUM_SEPARATOR] [-c CHUNK_SIZE] [-C FETCH_CHUNK_SIZE] [-l {CRITICAL,ERROR,WARNING,INFO,DEBUG}] [-k] [-i IGNORE] [-m {CREATE,CLEANUP,DELETE_ALL}] [-d] [-x SHARE_WITH] [-o {viewer,editor}]
[-S {0,1,2}] [-O {False,asc,desc}]
root_path api_url api_key
Create Immich Albums from an external library path based on the top level folders
@ -66,6 +70,16 @@ This script is mostly based on the following original script: [REDVM/immich_auto
Mode for the script to run with. CREATE = Create albums based on folder names and provided arguments; CLEANUP = Create album nmaes based on current images and script arguments, but delete albums if they exist; DELETE_ALL = Delete all
albums. If the mode is anything but CREATE, --unattended does not have any effect. Only performs deletion if -d/--delete-confirm option is set, otherwise only performs a dry-run. (default: CREATE)
-d, --delete-confirm Confirm deletion of albums when running in mode CLEANUP or DELETE_ALL. If this flag is not set, these modes will perform a dry run only. Has no effect in mode CREATE (default: False)
-x SHARE_WITH, --share-with SHARE_WITH
A user name (or email address of an existing user) to share newly created albums with. Sharing only happens if the album was actually created, not if new assets were added to an existing album. If the the share role should be specified by
user, the format <userName>=<shareRole> must be used, where <shareRole> must be one of 'viewer' or 'editor'. May be specified multiple times to share albums with more than one user. (default: None)
-o {viewer,editor}, --share-role {viewer,editor}
The default share role for users newly created albums are shared with. Only effective if --share-with is specified at least once and the share role is not specified within --share-with. (default: viewer)
-S {0,1,2}, --sync-mode {0,1,2}
Synchronization mode to use. Synchronization mode helps synchronizing changes in external libraries structures to Immich after albums have already been created. Possible Modes: 0 = do nothing; 1 = Delete any empty albums; 2 = Trigger
offline asset removal (REQUIRES API KEY OF AN ADMIN USER!) (default: 0)
-O {False,asc,desc}, --album-order {False,asc,desc}
Set sorting order for newly created albums to newest or oldest file first, Immich defaults to newest file first (default: False)
```
__Plain example without optional arguments:__
@ -85,16 +99,20 @@ The environment variables are analoguous to the script's command line arguments.
| ROOT_PATH | yes | A single or a comma separated list of import paths for external libraries in Immich. <br>Refer to [Choosing the correct `root_path`](#choosing-the-correct-root_path).|
| API_URL | yes | The root API URL of immich, e.g. https://immich.mydomain.com/api/ |
| API_KEY | yes | The Immich API Key to use
| CRON_EXPRESSION | yes | A [crontab-style expression](https://crontab.guru/) (e.g. "0 * * * *") to perform album creation on a schedule (e.g. every hour). |
| ALBUM_LEVELS | no | Number of sub-folders or range of sub-folder levels below the root path used for album name creation. Positive numbers start from top of the folder structure, negative numbers from the bottom. Cannot be 0. If a range should be set, the start level and end level must be separated by a comma. <br>Refer to [How it works](#how-it-works) for a detailed explanation and examples. |
| ALBUM_SEPARATOR | no | Separator string to use for compound album names created from nested folders. Only effective if -a is set to a value > 1 (default: " ") |
| CHUNK_SIZE | no | Maximum number of assets to add to an album with a single API call (default: 2000) |
| FETCH_CHUNK_SIZE | no | Maximum number of assets to fetch with a single API call (default: 5000) |
| LOG_LEVEL | no | Log level to use (default: INFO), allowed values: CRITICAL,ERROR,WARNING,INFO,DEBUG |
| CRON_EXPRESSION | yes | A [crontab-style expression](https://crontab.guru/) (e.g. `0 * * * *`) to perform album creation on a schedule (e.g. every hour). |
| ALBUM_LEVELS | no | Number of sub-folders or range of sub-folder levels below the root path used for album name creation. Positive numbers start from top of the folder structure, negative numbers from the bottom. Cannot be `0`. If a range should be set, the start level and end level must be separated by a comma. <br>Refer to [How it works](#how-it-works) for a detailed explanation and examples. |
| ALBUM_SEPARATOR | no | Separator string to use for compound album names created from nested folders. Only effective if `-a` is set to a value `> 1`(default: "` `") |
| CHUNK_SIZE | no | Maximum number of assets to add to an album with a single API call (default: `2000`) |
| FETCH_CHUNK_SIZE | no | Maximum number of assets to fetch with a single API call (default: `5000`) |
| LOG_LEVEL | no | Log level to use (default: INFO), allowed values: `CRITICAL`,`ERROR`,`WARNING`,`INFO`,`DEBUG` |
| INSECURE | no | Set to `true` to disable SSL verification for the Immich API server, useful for self-signed certificates (default: `false`), allowed values: `true`, `false` |
| INSECURE | no | A string containing a list of folders, sub-folder sequences or file names separated by ':' that will be ignored. |
| MODE | no | Mode for the script to run with. <br> __CREATE__ = Create albums based on folder names and provided arguments<br>__CLEANUP__ = Create album nmaes based on current images and script arguments, but delete albums if they exist <br> __DELETE_ALL__ = Delete all albums. <br> If the mode is anything but CREATE, `--unattended` does not have any effect. <br> (default: CREATE). <br>Refer to [Cleaning Up Albums](#cleaning-up-albums). |
| DELETE_CONFIRM | no | Confirm deletion of albums when running in mode CLEANUP or DELETE_ALL. If this flag is not set, these modes will perform a dry run only. Has no effect in mode CREATE (default: False). <br>Refer to [Cleaning Up Albums](#cleaning-up-albums).|
| INSECURE | no | A string containing a list of folders, sub-folder sequences or file names separated by '`:`' that will be ignored. |
| MODE | no | Mode for the script to run with. <br> __`CREATE`__ = Create albums based on folder names and provided arguments<br>__`CLEANUP`__ = Create album nmaes based on current images and script arguments, but delete albums if they exist <br> __`DELETE_ALL`__ = Delete all albums. <br> If the mode is anything but `CREATE`, `--unattended` does not have any effect. <br> (default: `CREATE`). <br>Refer to [Cleaning Up Albums](#cleaning-up-albums). |
| DELETE_CONFIRM | no | Confirm deletion of albums when running in mode `CLEANUP` or `DELETE_ALL`. If this flag is not set, these modes will perform a dry run only. Has no effect in mode `CREATE` (default: `False`). <br>Refer to [Cleaning Up Albums](#cleaning-up-albums).|
| SHARE_WITH | no | A single or a colon (`:`) separated list of existing user names (or email addresses of existing users) to share newly created albums with. If the the share role should be specified by user, the format <userName>=<shareRole> must be used, where <shareRole> must be one of `viewer` or `editor`. May be specified multiple times to share albums with more than one user. (default: None) Sharing only happens if an album is actually created, not if new assets are added to it. <br>Refer to [Automatic Album Sharing](#automatic-album-sharing).|
| SHARE_ROLE | no | The role for users newly created albums are shared with. Only effective if `SHARE_WITH` is not empty and no explicit share role was specified for at least one user. (default: viewer), allowed values: `viewer`, `editor` |
| SYNC_MODE | no | Synchronization mode to use. Synchronization mode helps synchronizing changes in external libraries structures to Immich after albums have already been created. Possible Modes: <br>`0` = do nothing<br>`1` = Delete any empty albums<br>`2` = Trigger offline asset removal (REQUIRES API KEY OF AN ADMIN USER!)<br>(default: `0`)<br>Refer to [Dealing with External Library Changes](#dealing-with-external-library-changes). |
| ALBUM_ORDER | no | Set sorting order for newly created albums to newest (`desc`) or oldest (`asc`) file first, Immich defaults to newest file first, allowed values: `asc`, `desc` |
#### Run the container with Docker
@ -121,8 +139,6 @@ docker run --name immich-folder-album-creator -e TZ="Europe/Berlin" -e CRON_EXPR
Adding the container to Immich's `docker-compose.yml` file:
```yml
version: "3.8"
#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
@ -242,6 +258,52 @@ Albums created for `root_path = /external_libs/photos/Birthdays`:
Since Immich does not support real nested albums ([yet?](https://github.com/immich-app/immich/discussions/2073)), neither does this script.
## Automatic Album Sharing
The scripts support sharing newly created albums with a list of existing users. The sharing role (`viewer` or `editor`) can be specified for all users at once or individually per user.
### Album Sharing Examples (Bare Python Script)
Two arguments control this feature:
- `-o / --share-role`: The default role for users an album is shared with. Allowed values are `viewer` or `editor`. This argument is optional and defaults to `viewer`.
- `-x / --share-with`: Specify once per user to share with. The value should be either the user name or the user's email address as specified in Immich. If the user name is used and it contains blanks ` `, it must be wrapped in double quotes `"`. To override the default share role and specify a role explicitly for this user, the format `<userName>=<shareRole>` must be used (refer to examples below).
To share new albums with users `User A` and `User B` as `viewer`, use the following call:
```bash
python3 ./immich_auto_album.py --share-with "User A" --share-with "User B" /path/to/external/lib https://immich.mydomain.com/api thisIsMyApiKeyCopiedFromImmichWebGui
```
To share new albums with users `User A` and `User B` as `editor`, use the following call:
```bash
python3 ./immich_auto_album.py --share-with "User A" --share-with "User B" --share-role "editor" /path/to/external/lib https://immich.mydomain.com/api thisIsMyApiKeyCopiedFromImmichWebGui
```
To share new albums with users `User A` and a user with mail address `userB@mydomain.com`, but `User A` should be an editor, use the following call:
```bash
python3 ./immich_auto_album.py --share-with "User A=editor" --share-with "userB@mydomain.com" /path/to/external/lib https://immich.mydomain.com/api thisIs
```
### Album Sharing Examples (Docker)
Two environment variables control this feature:
- `SHARE_ROLE`: The default role for users an album is shared with. Allowed values are `viewer` or `editor`. This argument is optional and defaults to `viewer`.
- `SHARE_WITH`: A colon `:` separated list of either names or email addresses (or a mix) of existing users. To override the default share role and specify a role explicitly for each user, the format `<userName>=<shareRole>` must be used (refer to examples below).
To share new albums with users `User A` and `User B` as `viewer`, use the following call:
```bash
docker run -e SHARE_WITH="User A:User B" -e UNATTENDED="1" -e API_URL="https://immich.mydomain.com/api/" -e API_KEY="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" -e ROOT_PATH="/external_libs/photos" salvoxia/immich-folder-album-creator:latest /script/immich_auto_album.sh
```
To share new albums with users `User A` and `User B` as `editor`, use the following call:
```bash
docker run -e SHARE_WITH="User A:User B" -e SHARE_ROLE="editor" -e UNATTENDED="1" -e API_URL="https://immich.mydomain.com/api/" -e API_KEY="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" -e ROOT_PATH="/external_libs/photos" salvoxia/immich-folder-album-creator:latest /script/immich_auto_album.sh
```
To share new albums with users `User A` and a user with mail address `userB@mydomain.com`, but `User A` should be an editor, use the following call:
```bash
docker run -e SHARE_WITH="User A=editor:userB@mydomain.com" -e UNATTENDED="1" -e API_URL="https://immich.mydomain.com/api/" -e API_KEY="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" -e ROOT_PATH="/external_libs/photos" salvoxia/immich-folder-album-creator:latest /script/immich_auto_album.sh
```
## Cleaning Up Albums
The script supports differnt run modes (option `-m`/`--mode` or env variable `MODE` for Docker). The default mode is `CREATE`, which is used to create albums.
@ -254,3 +316,75 @@ To actually delete albums, the option `-d/--delete-confirm` (or env variable `DE
__WARNING ⚠__
Deleting albums cannot be undone! The only option is to let the script run again and create new albums base on the passed arguments and current assets in Immich.
## Dealing with External Library Changes
Due to their nature, external libraries may be changed by the user without Immich having any say in it.
Two examples of this are the user deleting or renaming folders in their external libraries after the script has created albums, or the user moving single files from one folder to another. The script would create new albums from renamed folders or add images to their new album after they have been moved.
Immich itself deals with this by marking images/videos it no longer sees in their original location as "offline". This can lead to albums being completely populated by "offline" files only (if the folder was renamed or deleted) while they exist normally in a new album or with single images being offline in one album, while existing normally in their new albums.
There is an administrative option to trigger deletion of "offline assets" from libraries in the Immich Admin Interface. However, in the first example this might leave behind empty albums if the album only contained offline files.
To clean up this mess this script offers an option called `Sync Mode`. It is an optional argument / environment variable that may have values from 0 to 2.
The following behaviors wil be triggered by the different values:
- `0`: No syncing (default)
- `1`: Delete all empty albums at the end of a run
- `2`: Trigger offline asset removal for all libraires (⚠️ This function requires the API key of an Admin User!)
The complete syncing process consists of two stages:
1. Trigger Offline Asset Removal
This will start a background job for each library to find and delete files from Immich's database that are marked as 'offline'. It is asynchronous and might take some time (depending on the size of the databases and the numver of offline files).
Offline Asset Removal can only be triggered by Administrators, so the API Key used for this sync mode must be one of an Administrator. This might not be the same you use for creating albums!
2. Delete Empty Albums
Step 1 might leave behind empty albums for a different user. The script is capable of removing empty albums at the end of a run to clean up after Offline Asset Removal.
⚠️ It is __not__ possible for the script to distinguish between an album that was left behind empty after Offline Asset Removal and a manually created album with no images added to it! All empty albums of that user will be deleted!
It is up to you whether you want to use the full capabilities Sync Mode offers, parts of it or none.
An example for the Immich `docker-compose.yml` stack when using full Sync Mode might look like this:
```yml
#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#
name: immich
services:
immich-server:
container_name: immich_server
volumes:
- /path/to/my/photos:/external_libs/photos
...
immich-folder-album-creator-offline-asset-removal:
container_name: immich_folder_album_creator
image: salvoxia/immich-folder-album-creator:latest
restart: unless-stopped
environment:
API_URL: http://immich_server:3001/api
# Admin User API Key
API_KEY: yyyyyyyyyyyyyyyyyy
ROOT_PATH: /thisIsADummyPathThatDoesNotExist
CRON_EXPRESSION: "0 * * * *"
TZ: Europe/Berlin
# Regularly trigger offline asset removal
SYNC_MODE: "2"
immich-folder-album-creator:
container_name: immich_folder_album_creator
image: salvoxia/immich-folder-album-creator:latest
restart: unless-stopped
environment:
API_URL: http://immich_server:3001/api
API_KEY: xxxxxxxxxxxxxxxxx
ROOT_PATH: /external_libs/photos
# Run 10 minutes past the hour to give Offline Asset Removal time to work
CRON_EXPRESSION: "10 * * * *"
TZ: Europe/Berlin
# Delete empty albums after each run
SYNC_MODE: "1"
```
The `immich-folder-album-creator-offline-asset-removal` service has the one and only purpose of triggering Offline Asset Removal, since we do not want to create albums for the Admin User. Thus, a dummy `ROOT_PATH` is passed so that the script will never find any assets to create albums from. `SYNC_MODE` is set to `2` to regularly trigger Offline Asset Removal.
The `immich-folder-album-creator` service is the normal service for creating albums from an external library. The cron job triggers 10 minutes past the hour, whereas Offline Asset Removal is trigger at the full hour. This gives the job 10 minutes of time to run before any empty albums are deleted at the end of the run due to `SYNC_MODE` being set to `1`.

View file

@ -5,6 +5,7 @@ oldIFS=$IFS
IFS=','
# disable globbing
set -f
# parse ROOT_PATH CSV
main_root_path=""
additional_root_paths=""
for path in ${ROOT_PATH}; do
@ -16,6 +17,17 @@ for path in ${ROOT_PATH}; do
done
IFS=$oldIFS
# parse semicolon separated root paths and wrap in quotes
oldIFS=$IFS
IFS=':'
# parse SHARE_WITH CSV
share_with_list=""
for share_user in ${SHARE_WITH}; do
share_with_list="-x \"$share_user\" $share_with_list"
done
# reset IFS
IFS=$oldIFS
unattended=
if [ ! -z "$UNATTENDED" ]; then
unattended="-u"
@ -63,5 +75,21 @@ if [ ! -z "$DELETE_CONFIRM" ]; then
args="-d $args"
fi
if [ ! -z "$share_with_list" ]; then
args="$share_with_list $args"
fi
if [ ! -z "$SHARE_ROLE" ]; then
args="-o $SHARE_ROLE $args"
fi
if [ ! -z "$SYNC_MODE" ]; then
args="-S $SYNC_MODE $args"
fi
if [ ! -z "$ALBUM_ORDER" ]; then
args="-O $ALBUM_ORDER $args"
fi
BASEDIR=$(dirname "$0")
echo $args | xargs python3 -u $BASEDIR/immich_auto_album.py

View file

@ -1,8 +1,10 @@
from typing import Tuple
import requests
import argparse
import logging
import sys
import os
import datetime
from collections import defaultdict
import urllib3
@ -24,6 +26,12 @@ SCRIPT_MODE_CLEANUP = "CLEANUP"
# Delete ALL albums
SCRIPT_MODE_DELETE_ALL = "DELETE_ALL"
# Environment variable to check if the script is running inside Docker
ENV_IS_DOCKER = "IS_DOCKER"
# List of allowed share user roles
SHARE_ROLES = ["editor", "viewer"]
parser = argparse.ArgumentParser(description="Create Immich Albums from an external library path based on the top level folders", formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument("root_path", action='append', help="The external libarary's root path in Immich")
@ -40,11 +48,17 @@ parser.add_argument("-k", "--insecure", action="store_true", help="Set to true t
parser.add_argument("-i", "--ignore", default="", type=str, help="A string containing a list of folders, sub-folder sequences or file names separated by ':' that will be ignored.")
parser.add_argument("-m", "--mode", default=SCRIPT_MODE_CREATE, choices=[SCRIPT_MODE_CREATE, SCRIPT_MODE_CLEANUP, SCRIPT_MODE_DELETE_ALL], help="Mode for the script to run with. CREATE = Create albums based on folder names and provided arguments; CLEANUP = Create album nmaes based on current images and script arguments, but delete albums if they exist; DELETE_ALL = Delete all albums. If the mode is anything but CREATE, --unattended does not have any effect. Only performs deletion if -d/--delete-confirm option is set, otherwise only performs a dry-run.")
parser.add_argument("-d", "--delete-confirm", action="store_true", help="Confirm deletion of albums when running in mode "+SCRIPT_MODE_CLEANUP+" or "+SCRIPT_MODE_DELETE_ALL+". If this flag is not set, these modes will perform a dry run only. Has no effect in mode "+SCRIPT_MODE_CREATE)
parser.add_argument("-x", "--share-with", action="append", help="A user name (or email address of an existing user) to share newly created albums with. Sharing only happens if the album was actually created, not if new assets were added to an existing album. If the the share role should be specified by user, the format <userName>=<shareRole> must be used, where <shareRole> must be one of 'viewer' or 'editor'. May be specified multiple times to share albums with more than one user.")
parser.add_argument("-o", "--share-role", default="viewer", choices=['viewer', 'editor'], help="The default share role for users newly created albums are shared with. Only effective if --share-with is specified at least once and the share role is not specified within --share-with.")
parser.add_argument("-S", "--sync-mode", default=0, type=int, choices=[0, 1, 2], help="Synchronization mode to use. Synchronization mode helps synchronizing changes in external libraries structures to Immich after albums have already been created. Possible Modes: 0 = do nothing; 1 = Delete any empty albums; 2 = Trigger offline asset removal (REQUIRES API KEY OF AN ADMIN USER!)")
parser.add_argument("-O", "--album-order", default=False, type=str, choices=[False, 'asc', 'desc'], help="Set sorting order for newly created albums to newest or oldest file first, Immich defaults to newest file first")
args = vars(parser.parse_args())
# set up logger to log in logfmt format
logging.basicConfig(level=args["log_level"], stream=sys.stdout, format='time=%(asctime)s level=%(levelname)s msg=%(message)s')
logging.Formatter.formatTime = (lambda self, record, datefmt=None: datetime.datetime.fromtimestamp(record.created, datetime.timezone.utc).astimezone().isoformat(sep="T",timespec="milliseconds"))
root_paths = args["root_path"]
root_url = args["api_url"]
api_key = args["api_key"]
@ -55,15 +69,21 @@ album_levels = args["album_levels"]
# Album Levels Range handling
album_levels_range_arr = ()
album_level_separator = args["album_separator"]
album_order = args["album_order"]
insecure = args["insecure"]
ignore_albums = args["ignore"]
mode = args["mode"]
delete_confirm = args["delete_confirm"]
share_with = args["share_with"]
share_role = args["share_role"]
sync_mode = args["sync_mode"]
# Override unattended if we're running in destructive mode
if mode != SCRIPT_MODE_CREATE:
unattended = False
is_docker = os.environ.get(ENV_IS_DOCKER, False)
logging.debug("root_path = %s", root_paths)
logging.debug("root_url = %s", root_url)
logging.debug("api_key = %s", api_key)
@ -73,10 +93,15 @@ logging.debug("unattended = %s", unattended)
logging.debug("album_levels = %s", album_levels)
#logging.debug("album_levels_range = %s", album_levels_range)
logging.debug("album_level_separator = %s", album_level_separator)
logging.debug("album_order = %s", album_order)
logging.debug("insecure = %s", insecure)
logging.debug("ignore = %s", ignore_albums)
logging.debug("mode = %s", mode)
logging.debug("delete_confirm = %s", delete_confirm)
logging.debug("is_docker = %s", is_docker)
logging.debug("share_with = %s", share_with)
logging.debug("share_role = %s", share_role)
logging.debug("sync_mode = %d", sync_mode)
# Verify album levels
if is_integer(album_levels) and album_levels == 0:
@ -132,17 +157,51 @@ requests_kwargs = {
'verify' : not insecure
}
# Yield successive n-sized
# chunks from l.
def divide_chunks(l, n):
def divide_chunks(l: list, n: int):
"""Yield successive n-sized chunks from l. """
# looping till length l
for i in range(0, len(l), n):
yield l[i:i + n]
yield l[i:i + n]
def parseSeparatedString(s: str, seprator: str) -> Tuple[str, str]:
"""
Parse a key, value pair, separated by the provided separator.
That's the reverse of ShellArgs.
On the command line (argparse) a declaration will typically look like:
foo=hello
or
foo="hello world"
"""
items = s.split(seprator)
key = items[0].strip() # we remove blanks around keys, as is logical
value = None
if len(items) > 1:
# rejoin the rest:
value = seprator.join(items[1:])
return (key, value)
def parseSeparatedStrings(items: list[str]) -> dict:
"""
Parse a series of key-value pairs and return a dictionary
"""
d = {}
if items:
for item in items:
key, value = parseSeparatedString(item, '=')
d[key] = value
return d
# Create album names from provided path_chunks string array
# based on supplied album_levels argument (either by level range or absolute album levels)
def create_album_name(path_chunks):
def create_album_name(path_chunks: list[str], album_separator: str) -> str:
"""
Create album names from provided path_chunks string array.
The method uses global variables album_levels_range_arr or album_levels to
generate ablum names either by level range or absolute album levels. If multiple
album path chunks are used for album names they are separated by album_separator.
"""
album_name_chunks = ()
logging.debug("path chunks = %s", list(path_chunks))
# Check which path to take: album_levels_range or album_levels
@ -183,11 +242,22 @@ def create_album_name(path_chunks):
if album_name_chunk_size < 0:
album_name_chunks = path_chunks[album_name_chunk_size:]
logging.debug("album_name_chunks = %s", album_name_chunks)
return album_level_separator.join(album_name_chunks)
return album_separator.join(album_name_chunks)
# Fetches assets from the Immich API
# Takes different API versions into account for compatibility
def fetchServerVersion():
def fetchServerVersion() -> dict:
"""
Fetches the API version from the immich server.
If the API endpoint does not yet exist, returns the latest version
before that API endpoint was introduced: 1.105.1
Returns
-------
Dictionary with keys
- major
- minor
- patch
"""
# This API call was only introduced with version 1.106.1, so it will fail
# for older versions.
# Initialize the version with the latest version without this API call
@ -201,49 +271,28 @@ def fetchServerVersion():
logging.info("Detected Immich server version %s.%s.%s or older", version['major'], version['minor'], version['patch'])
return version
# Fetches assets from the Immich API
# Takes different API versions into account for compatibility
def fetchAssets():
if version['major'] == 1 and version['minor'] <= 105:
return fetchAssetsLegacy()
else:
return fetchAssetsMinorV106()
def fetchAssets(isNotInAlbum: bool) -> list:
"""
Fetches assets from the Immich API.
# Fetches assets from the Immich API
# Uses the legacy GET /asset call which only exists up to v1.105.x
def fetchAssetsLegacy():
assets = []
# Initial API call, let's fetch our first chunk
r = requests.get(root_url+'asset?take='+str(number_of_assets_to_fetch_per_request), **requests_kwargs)
assert r.status_code == 200
logging.debug("Received %s assets with chunk 1", len(r.json()))
assets = assets + r.json()
Uses the /search/meta-data call. Much more efficient than the legacy method
since this call allows to filter for assets that are not in an album only.
Parameters
----------
isNotInAlbum : bool
Flag indicating whether to fetch only assets that are not part
of an album or not.
Returns
---------
An array of asset objects
"""
# If we got a full chunk size back, let's perfrom subsequent calls until we get less than a full chunk size
skip = 0
while len(r.json()) == number_of_assets_to_fetch_per_request:
skip += number_of_assets_to_fetch_per_request
r = requests.get(root_url+'asset?take='+str(number_of_assets_to_fetch_per_request)+'&skip='+str(skip), **requests_kwargs)
if skip == number_of_assets_to_fetch_per_request and assets == r.json():
logging.info("Non-chunked Immich API detected, stopping fetching assets since we already got all in our first call")
break
assert r.status_code == 200
logging.debug("Received %s assets with chunk", len(r.json()))
assets = assets + r.json()
return assets
# Fetches assets from the Immich API
# Uses the /search/meta-data call. Much more efficient than the legacy method
# since this call allows to filter for assets that are not in an album only.
def fetchAssetsMinorV106():
assets = []
# prepare request body
body = {}
# only request images that are not in any album if we are running in CREATE mode,
# otherwise we need all images, even if they are part of an album
if mode == SCRIPT_MODE_CREATE:
body['isNotInAlbum'] = 'true'
body['isNotInAlbum'] = isNotInAlbum
# This API call allows a maximum page size of 1000
number_of_assets_to_fetch_per_request_search = min(1000, number_of_assets_to_fetch_per_request)
@ -271,24 +320,34 @@ def fetchAssetsMinorV106():
return assets
# Fetches albums from the Immich API
# Takes different API versions into account for compatibility
def fetchAlbums():
"""Fetches albums from the Immich API"""
apiEndpoint = 'albums'
if version['major'] == 1 and version['minor'] <= 105:
apiEndpoint = 'album'
r = requests.get(root_url+apiEndpoint, **requests_kwargs)
r.raise_for_status()
return r.json()
# Deletes an album identified by album['id']
# Takes different API versions into account for compatibility
# Returns False if the album could not be deleted, otherwise True
def deleteAlbum(album):
def deleteAlbum(album: dict):
"""
Deletes an album identified by album['id']
If the album could not be deleted, logs an error.
Parameters
----------
album : dict
Dictionary with the following keys:
- id
- albumName
Returns
---------
True if the album was deleted, otherwise False
"""
apiEndpoint = 'albums'
if version['major'] == 1 and version['minor'] <= 105:
apiEndpoint = 'album'
logging.debug("Album ID = %s, Album Name = %s", album['id'], album['albumName'])
r = requests.delete(root_url+apiEndpoint+'/'+album['id'], **requests_kwargs)
if r.status_code not in [200, 201]:
@ -296,25 +355,66 @@ def deleteAlbum(album):
return False
return True
# Creates an album with the provided name and returns the ID of the
# created album
def createAlbum(albumName):
def createAlbum(albumName: str, albumOrder: str) -> str:
"""
Creates an album with the provided name and returns the ID of the created album
Parameters
----------
albumName : str
Name of the album to create
albumOrder : str
False or order [asc|desc]
Returns
---------
True if the album was deleted, otherwise False
Raises
----------
Exception if the API call failed
"""
apiEndpoint = 'albums'
if version['major'] == 1 and version['minor'] <= 105:
apiEndpoint = 'album'
data = {
'albumName': albumName,
'description': albumName
}
r = requests.post(root_url+apiEndpoint, json=data, **requests_kwargs)
assert r.status_code in [200, 201]
return r.json()['id']
# Adds the provided assetIds to the provided albumId
def addAssetsToAlbum(albumId, assets):
albumId = r.json()['id']
if albumOrder:
data = {
'order': albumOrder
}
r = requests.patch(root_url+apiEndpoint+f'/{albumId}', json=data, **requests_kwargs)
assert r.status_code in [200, 201]
return albumId
def addAssetsToAlbum(albumId: str, assets: list[str]):
"""
Adds the assets IDs provided in assets to the provided albumId.
If assets if larger than number_of_images_per_request, the list is chunked
and one API call is performed per chunk.
Only logs errors and successes.
Parameters
----------
albumId : str
The ID of the album to add assets to
assets: list[str]
A list of asset IDs to add to the album
"""
apiEndpoint = 'albums'
if version['major'] == 1 and version['minor'] <= 105:
apiEndpoint = 'album'
# Divide our assets into chunks of number_of_images_per_request,
# So the API can cope
assets_chunked = list(divide_chunks(assets, number_of_images_per_request))
@ -339,6 +439,87 @@ def addAssetsToAlbum(albumId, assets):
if cpt > 0:
logging.info("%d new assets added to %s", cpt, album)
def fetchUsers():
"""Queries and returns all users"""
apiEndpoint = 'users'
r = requests.get(root_url+apiEndpoint, **requests_kwargs)
assert r.status_code in [200, 201]
return r.json()
def shareAlbumWithUserAndRole(album_id: str, share_user_ids: list[str], share_role: str):
"""
Shares the album with the provided album_id with all provided share_user_ids
using share_role as a role.
Parameters
----------
album_id : str
The ID of the album to share
share_user_ids: list[str]
IDs of users to share the album with
share_role: str
The share role to use when sharing the album, valid values are
"viewer" or "editor"
Raises
----------
Exception if share_role contains an invalid value
Exception if the API call fails
"""
apiEndpoint = 'albums/'+album_id+'/users'
assert share_role in SHARE_ROLES
# build payload
album_users = []
for share_user_id in share_user_ids:
share_info = dict()
share_info['role'] = share_role
share_info['userId'] = share_user_id
album_users.append(share_info)
data = {
'albumUsers': album_users
}
r = requests.put(root_url+apiEndpoint, json=data, **requests_kwargs)
assert r.status_code in [200, 201]
def fetchLibraries():
"""Queries and returns all libraries"""
apiEndpoint = 'libraries'
r = requests.get(root_url+apiEndpoint, **requests_kwargs)
if r.status_code == 403:
logging.fatal("--sync-mode 2 requires an Admin User API key!")
else:
assert r.status_code in [200, 201]
return r.json()
def triggerOfflineAssetRemoval(libraryId: str):
"""
Triggers removal of offline assets in the library identified by libraryId.
Parameters
----------
libraryId : str
The ID of the library to trigger offline asset removal for
Raises
----------
Exception if the API call fails
"""
apiEndpoint = 'libraries/'+libraryId+'/removeOffline'
r = requests.post(root_url+apiEndpoint, **requests_kwargs)
if r.status_code == 403:
logging.fatal("--sync-mode 2 requires an Admin User API key!")
else:
assert r.status_code == 204
# append trailing slash to all root paths
for i in range(len(root_paths)):
if root_paths[i][-1] != '/':
@ -348,6 +529,12 @@ if root_url[-1] != '/':
root_url = root_url + '/'
version = fetchServerVersion()
# Check version
if version['major'] == 1 and version ['minor'] < 106:
logging.fatal("This script only works with Immich Server v1.106.0 and newer! Update Immich Server or use script version 0.8.1!")
exit(1)
# Special case: Run Mode DELETE_ALL albums
if mode == SCRIPT_MODE_DELETE_ALL:
albums = fetchAlbums()
@ -357,8 +544,12 @@ if mode == SCRIPT_MODE_DELETE_ALL:
album_names = []
for album in albums:
album_names.append(album['albumName'])
print("Would delete the following albums (ALL albums!). Call with --delete-confirm to actually delete albums!")
print("Would delete the following albums (ALL albums!):")
print(album_names)
if is_docker:
print("Run the container with environment variable DELETE_CONFIRM set to 1 to actually delete these albums!")
else:
print("Call with --delete-confirm to actually delete albums!")
exit(0)
cpt = 0
for album in albums:
@ -369,7 +560,12 @@ if mode == SCRIPT_MODE_DELETE_ALL:
exit(0)
logging.info("Requesting all assets")
assets = fetchAssets()
# only request images that are not in any album if we are running in CREATE mode,
# otherwise we need all images, even if they are part of an album
if mode == SCRIPT_MODE_CREATE:
assets = fetchAssets(True)
else:
assets = fetchAssets(False)
logging.info("%d photos found", len(assets))
@ -400,7 +596,7 @@ for asset in assets:
# remove last item from path chunks, which is the file name
del path_chunks[-1]
album_name = create_album_name(path_chunks)
album_name = create_album_name(path_chunks, album_level_separator)
if len(album_name) > 0:
album_to_assets[album_name].append(asset['id'])
else:
@ -412,8 +608,12 @@ logging.info("%d albums identified", len(album_to_assets))
logging.info("Album list: %s", list(album_to_assets.keys()))
if not unattended and mode == SCRIPT_MODE_CREATE:
print("Press enter to create these albums, Ctrl+C to abort")
input()
if is_docker:
print("Check that this is the list of albums you want to create. Run the container with environment variable UNATTENDED set to 1 to actually create these albums.")
exit(0)
else:
print("Press enter to create these albums, Ctrl+C to abort")
input()
album_to_id = {}
@ -427,8 +627,12 @@ logging.info("%d existing albums identified", len(albums))
if mode == SCRIPT_MODE_CLEANUP:
# Delete Confirm check
if not delete_confirm:
print("Would delete the following albums. Call with --delete-confirm to actually delete albums!")
print("Would delete the following albums:")
print(list(album_to_id.keys()))
if is_docker:
print("Run the container with environment variable DELETE_CONFIRM set to 1 to actually delete these albums!")
else:
print(" Call with --delete-confirm to actually delete albums!")
exit(0)
cpt = 0
@ -446,14 +650,73 @@ if mode == SCRIPT_MODE_CLEANUP:
# mode CREATE
logging.info("Creating albums if needed")
cpt = 0
created_albums = dict()
for album in album_to_assets:
if album in album_to_id:
continue
album_to_id[album] = createAlbum(album)
album_id = createAlbum(album, album_order)
album_to_id[album] = album_id
created_albums[album] = album_id
logging.info('Album %s added!', album)
cpt += 1
logging.info("%d albums created", cpt)
logging.info("%d albums created", len(created_albums))
# Share newly created albums with users
if share_with is not None and len(created_albums) > 0:
logging.info("Sharing created albums with users")
share_user_roles = parseSeparatedStrings(share_with)
logging.debug("Share User Roles: %s", share_user_roles)
# Get all users
users = fetchUsers()
logging.debug("Found users: %s", users)
# Initialize dicitionary of share roles to user IDs to share with
roles_for_share_user_ids = dict()
for allowed_role in SHARE_ROLES:
roles_for_share_user_ids[allowed_role] = list()
# Search user IDs of users to share with
for share_user in share_user_roles.keys():
role = share_user_roles[share_user]
# search user ID by name or email
found_user = False
if role == None:
role = share_role
logging.debug("No explicit share role passed for share user %s, using default role %s", share_user, share_role)
elif role not in SHARE_ROLES:
role = share_role
logging.warning("Passed share role %s for user %s is not allowed, defaulting to %s", role, share_user, share_role)
else:
logging.debug("Explicit share role %s passed for share user %s", role, share_user)
for user in users:
# Search by name or mail address
if user['name'] == share_user or user['email'] == share_user:
share_user_id = user['id']
logging.debug("User %s has ID %s", share_user, share_user_id)
roles_for_share_user_ids[role].append(share_user_id)
found_user = True
break
if not found_user:
logging.warning("User %s to share albums with does not exist!", share_user)
shared_album_cnt = 0
# Only try sharing if we found at least one user ID to share with
for share_album in created_albums.keys():
album_shared_successfully = False
for role in roles_for_share_user_ids.keys():
share_user_ids = roles_for_share_user_ids[role]
if len(share_user_ids) > 0:
try:
shareAlbumWithUserAndRole(created_albums[share_album], share_user_ids, role)
logging.debug("Album %s shared with users IDs %s in role: %s)", share_album, share_user_ids, role)
album_shared_successfully = True
except:
logging.warning("Error sharing album %s for users %s in role %s", share_album, share_user_ids, role)
album_shared_successfully = False
if album_shared_successfully:
shared_album_cnt += 1
logging.info("Successfully shared %d/%d albums", shared_album_cnt, len(created_albums))
logging.info("Adding assets to albums")
@ -463,4 +726,28 @@ for album, assets in album_to_assets.items():
id = album_to_id[album]
addAssetsToAlbum(id, assets)
# Perform sync mode action: Delete empty albums
# Attention: Since Offline Asset Removal is an asynchronous job,
# albums affected by it are most likely not empty yet! So this
# might only be effective in the next script run.
if sync_mode == 1:
logging.info("Deleting all empty albums")
albums = fetchAlbums()
emptyAlbumCount = 0
deletedAlbumCount = 0
for album in albums:
if album['assetCount'] == 0:
emptyAlbumCount += 1
logging.info("Deleting empty album %s", album['albumName'])
if deleteAlbum(album):
deletedAlbumCount += 1
logging.info("Successfully deleted %d/%d empty albums!", deletedAlbumCount, emptyAlbumCount)
# Perform sync mode action: Trigger offline asset removal
if sync_mode == 2:
logging.info("Trigger offline asset removal")
libraries = fetchLibraries()
for library in libraries:
triggerOfflineAssetRemoval(library["id"])
logging.info("Done!")