c-l/immich-folder-album-creator
1
0
Fork 0
Code Releases Activity

merge upstream

Browse source
This commit is contained in:
alternak 2024-06-16 17:01:12 +02:00
commit 699cb8b030
2 changed files with 278 additions and 86 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

38
README.md
View file

@ -1,10 +1,15 @@
[![Docker][docker-image]][docker-url]
[docker-image]: https://img.shields.io/docker/pulls/salvoxia/immich-folder-album-creator.svg
[docker-url]: https://hub.docker.com/r/salvoxia/immich-folder-album-creator/
# Immich Folder Album Creator
This is a python script designed to automatically create albums in [Immich](https://immich.app/) from a folder structure mounted into the Immich container.
This is useful for automatically creating and populating albums for external libraries.
Using the provided docker image, the script can simply be added to the Immich compose stack and run along the rest of Immich's containers.
__Current compatibility:__ Immich v1.103.x and below
__Current compatibility:__ Immich v1.106.x and below
## Disclaimer
This script is mostly based on the following original script: [REDVM/immich_auto_album.py](https://gist.github.com/REDVM/d8b3830b2802db881f5b59033cf35702)
@ -39,7 +44,8 @@ options:
Additional external libarary root path in Immich; May be specified multiple times for multiple import paths or external libraries. (default: None)
-u, --unattended Do not ask for user confirmation after identifying albums. Set this flag to run script as a cronjob. (default: False)
-a ALBUM_LEVELS, --album-levels ALBUM_LEVELS
Number of sub-folders below the root path used for album name creation. Positive numbers start from top of the folder structure, negative numbers from the bottom. Cannot be 0. (default: 1)
Number of sub-folders or range of sub-folder levels below the root path used for album name creation. Positive numbers start from top of the folder structure, negative numbers from the bottom. Cannot be 0. If a range should be set, the start level and end level must be separated by a comma
like '<startLevel>,<endLevel>'. If negative levels are used in a range, <startLevel> must be less than or equal to <endLevel>. (default: 1)
-s ALBUM_SEPARATOR, --album-separator ALBUM_SEPARATOR
Separator string to use for compound album names created from nested folders. Only effective if -a is set to a value > 1 (default: )
-c CHUNK_SIZE, --chunk-size CHUNK_SIZE
@ -59,10 +65,11 @@ The environment variables are analoguous to the script's command line arguments.
| Environment varible | Mandatory? | Description |
| :------------------- | :----------- | :------------ |
| ROOT_PATH | yes | A single or a comma separated list of import paths for external libraries in Immich |
| API_URL | yes | The root API URL of immich, e.g. https://immich.mydomain.com/api/ |
| API_KEY | yes | The Immich API Key to use |
| ALBUM_LEVELS | no | Number of sub-folders below the root path used for album name creation. Positive numbers start from top of the folder structure, negative numbers from the bottom. Cannot be 0. Refer to [How it works](#how-it-works) for a detailed explanation|
| ROOT_PATH | yes | A single or a comma separated list of import paths for external libraries in Immich´|
| API_URL | yes | The root API URL of immich, e.g. https://immich.mydomain.com/api/ |
| API_KEY | yes | The Immich API Key to use
| CRON_EXPRESSION | yes | A [crontab-style expression](https://crontab.guru/) (e.g. "0 * * * *") to perform album creation on a schedule (e.g. every hour). |
| ALBUM_LEVELS | no | Number of sub-folders or range of sub-folder levels below the root path used for album name creation. Positive numbers start from top of the folder structure, negative numbers from the bottom. Cannot be 0. If a range should be set, the start level and end level must be separated by a comma. Refer to [How it works](#how-it-works) for a detailed explanation |
| ALBUM_SEPARATOR | no | Separator string to use for compound album names created from nested folders. Only effective if -a is set to a value > 1 (default: " ") |
| CHUNK_SIZE | no | Maximum number of assets to add to an album with a single API call (default: 2000) |
| FETCH_CHUNK_SIZE | no | Maximum number of assets to fetch with a single API call (default: 5000) |
@ -179,7 +186,24 @@ Albums created for `root_path = /external_libs/photos/Birthdays`:
- `Jane` (containing all imags from `Birthdays/Jane`)
- `Skiing 2023`
Note that with negative `album-levels` images from different parent folders will be mixed in the same album if they reside in folders with the same name (see `Vacation` in example above).
### Album Level Ranges
It is possible to specify not just a nunmber for `album-levels`, but a range from level x to level y in the folder structure that should make up an album's name:
`--album-levels="2,3"`
The range is applied to the folder structure beneath `root_path` from the top for positive levels and from the bottom for negative levels.
Suppose the following folder structure for an external library with the script's `root_path` set to `/external_libs/photos`:
```
/external_libs/photos/2020/2020 02 Feb/Vacation
/external_libs/photos/2020/2020 08 Aug/Vacation
```
- `--album-levels="2,3"` will create albums (for this folder structure, this is equal to `--album-levels="-2"`)
- `2020 02 Feb Facation`
- `2020 08 Aug Vacation`
- `--album-levels="2,2"` will create albums (for this folder structure, this is equal to `--album-levels="-2,-2"`)
- `2020 02 Feb`
- `2020 08 Aug`
⚠️ Note that with negative `album-levels` or album level ranges, images from different parent folders will be mixed in the same album if they reside in sub-folders with the same name (see `Vacation` in example above).
Since Immich does not support real nested albums ([yet?](https://github.com/immich-app/immich/discussions/2073)), neither does this script.

326
immich_auto_album.py
View file

@ -1,5 +1,4 @@
import requests
import os
import argparse
import logging
import sys
@ -7,6 +6,14 @@ import datetime
from collections import defaultdict
import urllib3
# Trying to deal with python's isnumeric() function
# not recognizing negative numbers
def is_integer(str):
try:
int(str)
return True
except ValueError:
return False
parser = argparse.ArgumentParser(description="Create Immich Albums from an external library path based on the top level folders", formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument("root_path", action='append', help="The external libarary's root path in Immich")
@ -14,7 +21,7 @@ parser.add_argument("api_url", help="The root API URL of immich, e.g. https://im
parser.add_argument("api_key", help="The Immich API Key to use")
parser.add_argument("-r", "--root-path", action="append", help="Additional external libarary root path in Immich; May be specified multiple times for multiple import paths or external libraries.")
parser.add_argument("-u", "--unattended", action="store_true", help="Do not ask for user confirmation after identifying albums. Set this flag to run script as a cronjob.")
parser.add_argument("-a", "--album-levels", default=1, type=int, help="Number of sub-folders below the root path used for album name creation. Positive numbers start from top of the folder structure, negative numbers from the bottom. Cannot be 0.")
parser.add_argument("-a", "--album-levels", default="1", type=str, help="Number of sub-folders or range of sub-folder levels below the root path used for album name creation. Positive numbers start from top of the folder structure, negative numbers from the bottom. Cannot be 0. If a range should be set, the start level and end level must be separated by a comma like '<startLevel>,<endLevel>'. If negative levels are used in a range, <startLevel> must be less than or equal to <endLevel>.")
parser.add_argument("-s", "--album-separator", default=" ", type=str, help="Separator string to use for compound album names created from nested folders. Only effective if -a is set to a value > 1")
parser.add_argument("-c", "--chunk-size", default=2000, type=int, help="Maximum number of assets to add to an album with a single API call")
parser.add_argument("-C", "--fetch-chunk-size", default=5000, type=int, help="Maximum number of assets to fetch with a single API call")
@ -32,6 +39,8 @@ number_of_images_per_request = args["chunk_size"]
number_of_assets_to_fetch_per_request = args["fetch_chunk_size"]
unattended = args["unattended"]
album_levels = args["album_levels"]
# Album Levels Range handling
album_levels_range_arr = ()
album_level_separator = args["album_separator"]
ignore_ssl = args["ignore_ssl"]
logging.debug("root_path = %s", root_paths)
@ -40,27 +49,52 @@ logging.debug("api_key = %s", api_key)
logging.debug("number_of_images_per_request = %d", number_of_images_per_request)
logging.debug("number_of_assets_to_fetch_per_request = %d", number_of_assets_to_fetch_per_request)
logging.debug("unattended = %s", unattended)
logging.debug("album_levels = %d", album_levels)
logging.debug("album_levels = %s", album_levels)
#logging.debug("album_levels_range = %s", album_levels_range)
logging.debug("album_level_separator = %s", album_level_separator)
logging.debug("ignore_ssl = %s", ignore_ssl)
# Verify album levels
if album_levels == 0:
if is_integer(album_levels) and album_levels == 0:
parser.print_help()
exit(1)
if ignore_ssl:
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
# Yield successive n-sized
# chunks from l.
def divide_chunks(l, n):
# looping till length l
for i in range(0, len(l), n):
yield l[i:i + n]
# Verify album levels range
if not is_integer(album_levels):
album_levels_range_split = album_levels.split(",")
if (len(album_levels_range_split) != 2
or not is_integer(album_levels_range_split[0])
or not is_integer(album_levels_range_split[1])
or int(album_levels_range_split[0]) == 0
or int(album_levels_range_split[1]) == 0
or (int(album_levels_range_split[0]) >= 0 and int(album_levels_range_split[1]) < 0)
or (int(album_levels_range_split[0]) < 0 and int(album_levels_range_split[1]) >= 0)
or (int(album_levels_range_split[0]) < 0 and int(album_levels_range_split[1]) < 0) and int(album_levels_range_split[0]) > int(album_levels_range_split[1])):
logging.error("Invalid album_levels range format! If a range should be set, the start level and end level must be separated by a comma like '<startLevel>,<endLevel>'. If negative levels are used in a range, <startLevel> must be less than or equal to <endLevel>.")
exit(1)
album_levels_range_arr = album_levels_range_split
# Convert to int
album_levels_range_arr[0] = int(album_levels_range_split[0])
album_levels_range_arr[1] = int(album_levels_range_split[1])
# Special case: both levels are negative and end level is -1, which is equivalent to just negative album level of start level
if(album_levels_range_arr[0] < 0 and album_levels_range_arr[1] == -1):
album_levels = album_levels_range_arr[0]
album_levels_range_arr = ()
logging.debug("album_levels is a range with negative start level and end level of -1, converted to album_levels = %d", album_levels)
else:
logging.debug("valid album_levels range argument supplied")
logging.debug("album_levels_start_level = %d", album_levels_range_arr[0])
logging.debug("album_levels_end_level = %d", album_levels_range_arr[1])
# Deduct 1 from album start levels, since album levels start at 1 for user convenience, but arrays start at index 0
if album_levels_range_arr[0] > 0:
album_levels_range_arr[0] -= 1
album_levels_range_arr[1] -= 1
# Request arguments for API calls
requests_kwargs = {
'headers' : {
'x-api-key': api_key,
@ -70,6 +104,195 @@ requests_kwargs = {
'verify' : not ignore_ssl
}
# Yield successive n-sized
# chunks from l.
def divide_chunks(l, n):
# looping till length l
for i in range(0, len(l), n):
yield l[i:i + n]
# Create album names from provided path_chunks string array
# based on supplied album_levels argument (either by level range or absolute album levels)
def create_album_name(path_chunks):
album_name_chunks = ()
logging.debug("path chunks = %s", list(path_chunks))
# Check which path to take: album_levels_range or album_levels
if len(album_levels_range_arr) == 2:
if album_levels_range_arr[0] < 0:
album_levels_start_level_capped = min(len(path_chunks), abs(album_levels_range_arr[0]))
album_levels_end_level_capped = album_levels_range_arr[1]+1
album_levels_start_level_capped *= -1
else:
album_levels_start_level_capped = min(len(path_chunks)-1, album_levels_range_arr[0])
# Add 1 to album_levels_end_level_capped to include the end index, which is what the user intended to. It's not a problem
# if the end index is out of bounds.
album_levels_end_level_capped = min(len(path_chunks)-1, album_levels_range_arr[1]) + 1
logging.debug("album_levels_start_level_capped = %d", album_levels_start_level_capped)
logging.debug("album_levels_end_level_capped = %d", album_levels_end_level_capped)
# album start level is not equal to album end level, so we want a range of levels
if album_levels_start_level_capped is not album_levels_end_level_capped:
# if the end index is out of bounds.
if album_levels_end_level_capped < 0 and abs(album_levels_end_level_capped) >= len(path_chunks):
album_name_chunks = path_chunks[album_levels_start_level_capped:]
else:
album_name_chunks = path_chunks[album_levels_start_level_capped:album_levels_end_level_capped]
# album start and end levels are equal, we want exactly that level
else:
# create on-the-fly array with a single element taken from
album_name_chunks = [path_chunks[album_levels_start_level_capped]]
else:
album_levels_int = int(album_levels)
# either use as many path chunks as we have,
# or the specified album levels
album_name_chunk_size = min(len(path_chunks), abs(album_levels_int))
if album_levels_int < 0:
album_name_chunk_size *= -1
# Copy album name chunks from the path to use as album name
album_name_chunks = path_chunks[:album_name_chunk_size]
if album_name_chunk_size < 0:
album_name_chunks = path_chunks[album_name_chunk_size:]
logging.debug("album_name_chunks = %s", album_name_chunks)
return album_level_separator.join(album_name_chunks)
# Fetches assets from the Immich API
# Takes different API versions into account for compatibility
def fetchServerVersion():
# This API call was only introduced with version 1.106.1, so it will fail
# for older versions.
# Initialize the version with the latest version without this API call
version = {'major': 1, 'minor': 105, "patch": 1}
r = requests.get(root_url+'server-info/version', **requests_kwargs)
assert r.status_code == 200 or r.status_code == 404
if r.status_code == 200:
version = r.json()
logging.info("Detected Immich server version %s.%s.%s", version['major'], version['minor'], version['patch'])
else:
logging.info("Detected Immich server version %s.%s.%s or older", version['major'], version['minor'], version['patch'])
return version
# Fetches assets from the Immich API
# Takes different API versions into account for compatibility
def fetchAssets():
if version['major'] == 1 and version['minor'] <= 105:
return fetchAssetsLegacy()
else:
return fetchAssetsMinorV106()
# Fetches assets from the Immich API
# Uses the legacy GET /asset call which only exists up to v1.105.x
def fetchAssetsLegacy():
assets = []
# Initial API call, let's fetch our first chunk
r = requests.get(root_url+'asset?take='+str(number_of_assets_to_fetch_per_request), **requests_kwargs)
assert r.status_code == 200
logging.debug("Received %s assets with chunk 1", len(r.json()))
assets = assets + r.json()
# If we got a full chunk size back, let's perfrom subsequent calls until we get less than a full chunk size
skip = 0
while len(r.json()) == number_of_assets_to_fetch_per_request:
skip += number_of_assets_to_fetch_per_request
r = requests.get(root_url+'asset?take='+str(number_of_assets_to_fetch_per_request)+'&skip='+str(skip), **requests_kwargs)
if skip == number_of_assets_to_fetch_per_request and assets == r.json():
logging.info("Non-chunked Immich API detected, stopping fetching assets since we already got all in our first call")
break
assert r.status_code == 200
logging.debug("Received %s assets with chunk", len(r.json()))
assets = assets + r.json()
return assets
# Fetches assets from the Immich API
# Uses the /search/meta-data call. Much more efficient than the legacy method
# since this call allows to filter for assets that are not in an album only.
def fetchAssetsMinorV106():
assets = []
# prepare request body
body = {}
body['isNotInAlbum'] = 'true'
# This API call allows a maximum page size of 1000
number_of_assets_to_fetch_per_request_search = min(1000, number_of_assets_to_fetch_per_request)
body['size'] = number_of_assets_to_fetch_per_request_search
# Initial API call, let's fetch our first chunk
page = 1
body['page'] = str(page)
r = requests.post(root_url+'search/metadata', json=body, **requests_kwargs)
r.raise_for_status()
responseJson = r.json()
assetsReceived = responseJson['assets']['items']
logging.debug("Received %s assets with chunk %s", len(assetsReceived), page)
assets = assets + assetsReceived
# If we got a full chunk size back, let's perfrom subsequent calls until we get less than a full chunk size
while len(assetsReceived) == number_of_assets_to_fetch_per_request_search:
page += 1
body['page'] = page
r = requests.post(root_url+'search/metadata', json=body, **requests_kwargs)
assert r.status_code == 200
responseJson = r.json()
assetsReceived = responseJson['assets']['items']
logging.debug("Received %s assets with chunk %s", len(assetsReceived), page)
assets = assets + assetsReceived
return assets
# Fetches assets from the Immich API
# Takes different API versions into account for compatibility
def fetchAlbums():
apiEndpoint = 'albums'
if version['major'] == 1 and version['minor'] <= 105:
apiEndpoint = 'album'
r = requests.get(root_url+apiEndpoint, **requests_kwargs)
r.raise_for_status()
return r.json()
# Creates an album with the provided name and returns the ID of the
# created album
def createAlbum(albumName):
apiEndpoint = 'albums'
if version['major'] == 1 and version['minor'] <= 105:
apiEndpoint = 'album'
data = {
'albumName': albumName,
'description': albumName
}
r = requests.post(root_url+apiEndpoint, json=data, **requests_kwargs)
assert r.status_code in [200, 201]
return r.json()['id']
# Adds the provided assetIds to the provided albumId
def addAssetsToAlbum(albumId, assets):
apiEndpoint = 'albums'
if version['major'] == 1 and version['minor'] <= 105:
apiEndpoint = 'album'
# Divide our assets into chunks of number_of_images_per_request,
# So the API can cope
assets_chunked = list(divide_chunks(assets, number_of_images_per_request))
for assets_chunk in assets_chunked:
data = {'ids':assets_chunk}
r = requests.put(root_url+apiEndpoint+f'/{albumId}/assets', json=data, **requests_kwargs)
if r.status_code not in [200, 201]:
print(album)
print(r.json())
print(data)
continue
assert r.status_code in [200, 201]
response = r.json()
cpt = 0
for res in response:
if not res['success']:
if res['error'] != 'duplicate':
logging.warning("Error adding an asset to an album: %s", res['error'])
else:
cpt += 1
if cpt > 0:
logging.info("%d new assets added to %s", cpt, album)
# append trailing slash to all root paths
for i in range(len(root_paths)):
if root_paths[i][-1] != '/':
@ -78,25 +301,10 @@ for i in range(len(root_paths)):
if root_url[-1] != '/':
root_url = root_url + '/'
logging.info("Requesting all assets")
assets = []
# Initial API call, let's fetch our first chunk
r = requests.get(root_url+'asset?take='+str(number_of_assets_to_fetch_per_request), **requests_kwargs)
assert r.status_code == 200
logging.debug("Received %s assets with chunk 1", len(r.json()))
assets = assets + r.json()
version = fetchServerVersion()
# If we got a full chunk size back, let's perfrom subsequent calls until we get less than a full chunk size
skip = 0
while len(r.json()) == number_of_assets_to_fetch_per_request:
skip += number_of_assets_to_fetch_per_request
r = requests.get(root_url+'asset?take='+str(number_of_assets_to_fetch_per_request)+'&skip='+str(skip), **requests_kwargs)
if skip == number_of_assets_to_fetch_per_request and assets == r.json():
logging.info("Non-chunked Immich API detected, stopping fetching assets since we already got all in our first call")
break
assert r.status_code == 200
logging.debug("Received %s assets with chunk", len(r.json()))
assets = assets + r.json()
logging.info("Requesting all assets")
assets = fetchAssets()
logging.info("%d photos found", len(assets))
@ -116,21 +324,11 @@ for asset in assets:
# remove last item from path chunks, which is the file name
del path_chunks[-1]
album_name_chunks = ()
# either use as many path chunks as we have,
# or the specified album levels
album_name_chunk_size = min(len(path_chunks), album_levels)
if album_levels < 0:
album_name_chunk_size = min(len(path_chunks), abs(album_levels))*-1
# Copy album name chunks from the path to use as album name
album_name_chunks = path_chunks[:album_name_chunk_size]
if album_name_chunk_size < 0:
album_name_chunks = path_chunks[album_name_chunk_size:]
album_name = album_level_separator.join(album_name_chunks)
# Check that the extracted album name is not actually a file name in root_path
album_to_assets[album_name].append(asset['id'])
album_name = create_album_name(path_chunks)
if len(album_name) > 0:
album_to_assets[album_name].append(asset['id'])
else:
logging.warning("Got empty album name for asset path %s, check your album_level settings!", asset_path)
album_to_assets = {k:v for k, v in sorted(album_to_assets.items(), key=(lambda item: item[0]))}
@ -144,9 +342,8 @@ if not unattended:
album_to_id = {}
logging.info("Listing existing albums on immich")
r = requests.get(root_url+'album', **requests_kwargs)
assert r.status_code == 200
albums = r.json()
albums = fetchAlbums()
album_to_id = {album['albumName']:album['id'] for album in albums }
logging.info("%d existing albums identified", len(albums))
@ -156,13 +353,7 @@ cpt = 0
for album in album_to_assets:
if album in album_to_id:
continue
data = {
'albumName': album,
'description': album
}
r = requests.post(root_url+'album', json=data, **requests_kwargs)
assert r.status_code in [200, 201]
album_to_id[album] = r.json()['id']
album_to_id[album] = createAlbum(album)
logging.info('Album %s added!', album)
cpt += 1
logging.info("%d albums created", cpt)
@ -173,29 +364,6 @@ logging.info("Adding assets to albums")
# so we can each time ad all assets to same album, no photo will be duplicated
for album, assets in album_to_assets.items():
id = album_to_id[album]
# Divide our assets into chunks of number_of_images_per_request,
# So the API can cope
assets_chunked = list(divide_chunks(assets, number_of_images_per_request))
for assets_chunk in assets_chunked:
data = {'ids':assets_chunk}
r = requests.put(root_url+f'album/{id}/assets', json=data, **requests_kwargs)
if r.status_code not in [200, 201]:
print(album)
print(r.json())
print(data)
continue
assert r.status_code in [200, 201]
response = r.json()
cpt = 0
for res in response:
if not res['success']:
if res['error'] != 'duplicate':
logging.warning("Error adding an asset to an album: %s", res['error'])
else:
cpt += 1
if cpt > 0:
logging.info("%d new assets added to %s", cpt, album)
addAssetsToAlbum(id, assets)
logging.info("Done!")