c-l/immich-folder-album-creator
1
0
Fork 0
Code Releases Activity

Merge pull request #59 from Salvoxia/enhancement/multiplePathFilters

Browse source 
Enhancement/multiple path filters
This commit is contained in:
Salvoxia 2024-09-27 17:13:01 +00:00 committed by GitHub
commit 79332b4c00
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 156 additions and 117 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

63
README.md
View file

@ -39,8 +39,8 @@ This script is mostly based on the following original script: [REDVM/immich_auto
```
3. Run the script
```
usage: immich_auto_album.py [-h] [-r ROOT_PATH] [-u] [-a ALBUM_LEVELS] [-s ALBUM_SEPARATOR] [-c CHUNK_SIZE] [-C FETCH_CHUNK_SIZE] [-l {CRITICAL,ERROR,WARNING,INFO,DEBUG}] [-k] [-i IGNORE] [-m {CREATE,CLEANUP,DELETE_ALL}] [-d]
[-x SHARE_WITH] [-o {viewer,editor}] [-S {0,1,2}] [-O {False,asc,desc}] [-A] [-f PATH_FILTER]
usage: immich_auto_album.py [-h] [-r ROOT_PATH] [-u] [-a ALBUM_LEVELS] [-s ALBUM_SEPARATOR] [-c CHUNK_SIZE] [-C FETCH_CHUNK_SIZE] [-l {CRITICAL,ERROR,WARNING,INFO,DEBUG}] [-k] [-i IGNORE] [-m {CREATE,CLEANUP,DELETE_ALL}] [-d] [-x SHARE_WITH] [-o {viewer,editor}] [-S {0,1,2}] [-O {False,asc,desc}] [-A]
[-f PATH_FILTER] [--set-album-thumbnail {first,last,random,random-all,random-filtered}]
root_path api_url api_key
Create Immich Albums from an external library path based on the top level folders
@ -56,9 +56,8 @@ This script is mostly based on the following original script: [REDVM/immich_auto
Additional external libarary root path in Immich; May be specified multiple times for multiple import paths or external libraries. (default: None)
-u, --unattended Do not ask for user confirmation after identifying albums. Set this flag to run script as a cronjob. (default: False)
-a ALBUM_LEVELS, --album-levels ALBUM_LEVELS
Number of sub-folders or range of sub-folder levels below the root path used for album name creation. Positive numbers start from top of the folder structure, negative numbers from the bottom. Cannot be
0. If a range should be set, the start level and end level must be separated by a comma like '<startLevel>,<endLevel>'. If negative levels are used in a range, <startLevel> must be less than or equal to
<endLevel>. (default: 1)
Number of sub-folders or range of sub-folder levels below the root path used for album name creation. Positive numbers start from top of the folder structure, negative numbers from the bottom. Cannot be 0. If a range should be set, the start level and end level must be separated by a comma
like '<startLevel>,<endLevel>'. If negative levels are used in a range, <startLevel> must be less than or equal to <endLevel>. (default: 1)
-s ALBUM_SEPARATOR, --album-separator ALBUM_SEPARATOR
Separator string to use for compound album names created from nested folders. Only effective if -a is set to a value > 1 (default: )
-c CHUNK_SIZE, --chunk-size CHUNK_SIZE
@ -69,32 +68,28 @@ This script is mostly based on the following original script: [REDVM/immich_auto
Log level to use (default: INFO)
-k, --insecure Set to true to ignore SSL verification (default: False)
-i IGNORE, --ignore IGNORE
A string containing a list of folders, sub-folder sequences or file names separated by ':' that will be ignored. (default: )
Use either literals or glob-like patterns to ignore assets for album name creation. This filter is evaluated after any values passed with --path-filter. May be specified multiple times. (default: None)
-m {CREATE,CLEANUP,DELETE_ALL}, --mode {CREATE,CLEANUP,DELETE_ALL}
Mode for the script to run with. CREATE = Create albums based on folder names and provided arguments; CLEANUP = Create album nmaes based on current images and script arguments, but delete albums if they
exist; DELETE_ALL = Delete all albums. If the mode is anything but CREATE, --unattended does not have any effect. Only performs deletion if -d/--delete-confirm option is set, otherwise only performs a
dry-run. (default: CREATE)
Mode for the script to run with. CREATE = Create albums based on folder names and provided arguments; CLEANUP = Create album nmaes based on current images and script arguments, but delete albums if they exist; DELETE_ALL = Delete all albums. If the mode is anything but CREATE, --unattended
does not have any effect. Only performs deletion if -d/--delete-confirm option is set, otherwise only performs a dry-run. (default: CREATE)
-d, --delete-confirm Confirm deletion of albums when running in mode CLEANUP or DELETE_ALL. If this flag is not set, these modes will perform a dry run only. Has no effect in mode CREATE (default: False)
-x SHARE_WITH, --share-with SHARE_WITH
A user name (or email address of an existing user) to share newly created albums with. Sharing only happens if the album was actually created, not if new assets were added to an existing album. If the
the share role should be specified by user, the format <userName>=<shareRole> must be used, where <shareRole> must be one of 'viewer' or 'editor'. May be specified multiple times to share albums with
more than one user. (default: None)
A user name (or email address of an existing user) to share newly created albums with. Sharing only happens if the album was actually created, not if new assets were added to an existing album. If the the share role should be specified by user, the format <userName>=<shareRole> must be used,
where <shareRole> must be one of 'viewer' or 'editor'. May be specified multiple times to share albums with more than one user. (default: None)
-o {viewer,editor}, --share-role {viewer,editor}
The default share role for users newly created albums are shared with. Only effective if --share-with is specified at least once and the share role is not specified within --share-with. (default:
viewer)
The default share role for users newly created albums are shared with. Only effective if --share-with is specified at least once and the share role is not specified within --share-with. (default: viewer)
-S {0,1,2}, --sync-mode {0,1,2}
Synchronization mode to use. Synchronization mode helps synchronizing changes in external libraries structures to Immich after albums have already been created. Possible Modes: 0 = do nothing; 1 =
Delete any empty albums; 2 = Trigger offline asset removal (REQUIRES API KEY OF AN ADMIN USER!) (default: 0)
Synchronization mode to use. Synchronization mode helps synchronizing changes in external libraries structures to Immich after albums have already been created. Possible Modes: 0 = do nothing; 1 = Delete any empty albums; 2 = Trigger offline asset removal (REQUIRES API KEY OF AN ADMIN USER!)
(default: 0)
-O {False,asc,desc}, --album-order {False,asc,desc}
Set sorting order for newly created albums to newest or oldest file first, Immich defaults to newest file first (default: False)
-A, --find-assets-in-albums
By default, the script only finds assets that are not assigned to any album yet. Set this option to make the script discover assets that are already part of an album and handle them as usual. (default:
False)
By default, the script only finds assets that are not assigned to any album yet. Set this option to make the script discover assets that are already part of an album and handle them as usual. (default: False)
-f PATH_FILTER, --path-filter PATH_FILTER
Use glob-like patterns to filter assets before album name creation. This filter is evaluated before any values passed with --ignore. (default: )
Use either literals or glob-like patterns to filter assets before album name creation. This filter is evaluated before any values passed with --ignore. May be specified multiple times. (default: None)
--set-album-thumbnail {first,last,random,random-all,random-filtered}
Set first/last/random image as thumbnail for newly created albums or albums assets have been added to. If set to random-filtered, thumbnails are shuffled for all albums whose assets would not be filtered out or ignored by the ignore or path-filter options, even if no assets were added during
the run. If set to random-all, the thumbnails for ALL albums will be shuffled on every run. (default: None)
Set first/last/random image as thumbnail for newly created albums or albums assets have been added to. If set to random-filtered, thumbnails are shuffled for all albums whose assets would not be filtered out or ignored by the ignore or path-filter options, even if no assets were added during
the run. If set to random-all, the thumbnails for ALL albums will be shuffled on every run. (default: None)
```
__Plain example without optional arguments:__
@ -121,7 +116,7 @@ The environment variables are analoguous to the script's command line arguments.
| FETCH_CHUNK_SIZE | no | Maximum number of assets to fetch with a single API call (default: `5000`) |
| LOG_LEVEL | no | Log level to use (default: INFO), allowed values: `CRITICAL`,`ERROR`,`WARNING`,`INFO`,`DEBUG` |
| INSECURE | no | Set to `true` to disable SSL verification for the Immich API server, useful for self-signed certificates (default: `false`), allowed values: `true`, `false` |
| IGNORE | no | A string containing a list of folders, sub-folder sequences or file names separated by '`:`' that will be ignored. |
| IGNORE | no | A colon `:` separated list of literals or glob-style patterns that will cause an image to be ignored if found in its path. |
| MODE | no | Mode for the script to run with. <br> __`CREATE`__ = Create albums based on folder names and provided arguments<br>__`CLEANUP`__ = Create album nmaes based on current images and script arguments, but delete albums if they exist <br> __`DELETE_ALL`__ = Delete all albums. <br> If the mode is anything but `CREATE`, `--unattended` does not have any effect. <br> (default: `CREATE`). <br>Refer to [Cleaning Up Albums](#cleaning-up-albums). |
| DELETE_CONFIRM | no | Confirm deletion of albums when running in mode `CLEANUP` or `DELETE_ALL`. If this flag is not set, these modes will perform a dry run only. Has no effect in mode `CREATE` (default: `False`). <br>Refer to [Cleaning Up Albums](#cleaning-up-albums).|
| SHARE_WITH | no | A single or a colon (`:`) separated list of existing user names (or email addresses of existing users) to share newly created albums with. If the the share role should be specified by user, the format <userName>=<shareRole> must be used, where <shareRole> must be one of `viewer` or `editor`. May be specified multiple times to share albums with more than one user. (default: None) Sharing only happens if an album is actually created, not if new assets are added to it. <br>Refer to [Automatic Album Sharing](#automatic-album-sharing).|
@ -129,7 +124,7 @@ The environment variables are analoguous to the script's command line arguments.
| SYNC_MODE | no | Synchronization mode to use. Synchronization mode helps synchronizing changes in external libraries structures to Immich after albums have already been created. Possible Modes: <br>`0` = do nothing<br>`1` = Delete any empty albums<br>`2` = Trigger offline asset removal (REQUIRES API KEY OF AN ADMIN USER!)<br>(default: `0`)<br>Refer to [Dealing with External Library Changes](#dealing-with-external-library-changes). |
| ALBUM_ORDER | no | Set sorting order for newly created albums to newest (`desc`) or oldest (`asc`) file first, Immich defaults to newest file first, allowed values: `asc`, `desc` |
| FIND_ASSETS_IN_ALBUMS | no | By default, the script only finds assets that are not assigned to any album yet. Set this option to make the script discover assets that are already part of an album and handle them as usual. (default: `False`)<br>Refer to [Assets in Multiple Albums](#assets-in-multiple-albums). |
| PATH_FILTER | no | Use glob-like patterns to filter assets before album name creation. This filter is evaluated before any values passed with --ignore. (default: ``)<br>Refer to [Filtering](#filtering). |
| PATH_FILTER | no | A colon `:` separated list of literals or glob-style patterns to filter assets before album name creation. (default: ``)<br>Refer to [Filtering](#filtering). |
| SET_ALBUM_THUMBNAIL | no | Set first/last/random image as thumbnail (based on image creation timestamp) for newly created albums or albums assets have been added to.<br> Allowed values: `first`,`last`,`random`,`random-filtered`,`random-all`<br>If set to `random-filtered`, thumbnails are shuffled for all albums whose assets would not be filtered out or ignored by the `IGNORE` or `PATH_FILTER` options, even if no assets were added during
the run. If set to random-all, the thumbnails for ALL albums will be shuffled on every run. (default: `None`)<br>Refer to [Setting Album Thumbnails](#setting-album-thumbnails). |
@ -284,16 +279,9 @@ Since Immich does not support real nested albums ([yet?](https://github.com/immi
## Filtering
It is possible filter images by either specifying path patterns to include or keywords which will ignore an image if its path contains any. Two options control this behavior.
It is possible filter images by either specifying keywords or path patterns to either specifically filter for or ignore assets based on their path. Two options control this behavior.
Internally, the script converts literals to glob-patterns that will match a path if the specified literal occurs anywhere in it. Example: `--ignore Feb` is equal to `--ignore **/*Feb*/**`.
### Ignoring Assets
The option `-i / --ignore` or Docker environment variable `IGNORE` accepts a semicolon-separated `:` list of keywords. If an image's path contains that keyword, it will be ignored.
**Example:**
`--ignore "Vacation:Birthday"` will not include any images for which the path **below the root path** contains either `Vacation` or `Birthday`. Albums will not be created for these images and they will not be added to albums.
### Filtering for Assets
The option `-f / ---path-filter` or Docker environment variable `PATH_FILTER` accepts a glob-style pattern to filter for images for which the path **below the root path** matches the provided pattern. **Only** these images will be considered for album creation.
The following wild-cards are supported:
| Pattern | Meaning |
|---------|---------------------------------------------------------------------------------------------|
@ -302,10 +290,19 @@ The following wild-cards are supported:
|`[]` | Matches one character in the brackets, e.g. `[a]` literally matches `a` |
|`[!]` | Matches one character *not* in the brackets, e.h. `[!a]` matches any character **but** `a` |
### Ignoring Assets
The option `-i / --ignore` can be specified multiple times for each literal or glob-style path pattern.
When using Docker, the environment variable `IGNORE` accepts a semicolon-separated `:` list of literals or glob-style patterns. If an image's path **below the root path** matches the pattern, it will be ignored.
### Filtering for Assets
The option `-f / ---path-filter` can be specified multiple times for each literal or glob-style path pattern.
When using Docker, the environment variable `PATH_FILTER` accepts a semicolon-separated `:` of literals or glob-style patterns. If an image's path **below the root path** does **NOT** match the pattern, it will be ignored.
> [!TIP]
> When working with path filters, consider setting the `-A / --find-assets-in-albums` option or Docker environment variable `FIND_ASSETS_IN_ALBUMS` for the script to discover assets that are already part of an album. That way, assets can be added to multiple albums by the script. Refer to the [Assets in Multiple Albums](#assets-in-multiple-albums) section for more information.
**Examples:**
### Filter Examples
Consider the following folder structure:
```
/external_libs/photos/

62
docker/immich_auto_album.sh
View file

@ -12,7 +12,7 @@ for path in ${ROOT_PATH}; do
if [ -z "$main_root_path" ]; then
main_root_path="\"$path\""
else
additional_root_paths="-r \"$path\" $additional_root_paths"
additional_root_paths="--root-path \"$path\" $additional_root_paths"
fi
done
IFS=$oldIFS
@ -22,15 +22,34 @@ oldIFS=$IFS
IFS=':'
# parse SHARE_WITH CSV
share_with_list=""
for share_user in ${SHARE_WITH}; do
share_with_list="-x \"$share_user\" $share_with_list"
done
if [ ! -z "$SHARE_WITH" ]; then
for share_user in ${SHARE_WITH}; do
share_with_list="--share-with \"$share_user\" $share_with_list"
done
fi
# parse PATH_FILTER CSV
path_filter_list=""
if [ ! -z "$PATH_FILTER" ]; then
for path_filter_entry in ${PATH_FILTER}; do
path_filter_list="--path-filter \"$path_filter_entry\" $path_filter_list"
done
fi
# parse IGNORE CSV
ignore_list=""
if [ ! -z "$IGNORE" ]; then
for ignore_entry in ${IGNORE}; do
ignore_list="--ignore \"$ignore_entry\" $ignore_list"
done
fi
# reset IFS
IFS=$oldIFS
unattended=
if [ ! -z "$UNATTENDED" ]; then
unattended="-u"
unattended="--unattended"
fi
args="$unattended $main_root_path $API_URL $API_KEY"
@ -40,39 +59,39 @@ if [ ! -z "$additional_root_paths" ]; then
fi
if [ ! -z "$ALBUM_LEVELS" ]; then
args="-a $ALBUM_LEVELS $args"
args="--album-levels $ALBUM_LEVELS $args"
fi
if [ ! -z "$ALBUM_SEPARATOR" ]; then
args="-s \"$ALBUM_SEPARATOR\" $args"
args="--album-separator \"$ALBUM_SEPARATOR\" $args"
fi
if [ ! -z "$FETCH_CHUNK_SIZE" ]; then
args="-C $FETCH_CHUNK_SIZE $args"
args="--fetch-chunk-size $FETCH_CHUNK_SIZE $args"
fi
if [ ! -z "$CHUNK_SIZE" ]; then
args="-c $CHUNK_SIZE $args"
args="--chunk-size $CHUNK_SIZE $args"
fi
if [ ! -z "$LOG_LEVEL" ]; then
args="-l $LOG_LEVEL $args"
args="--log-level $LOG_LEVEL $args"
fi
if [ "$INSECURE" = "true" ]; then
args="-k $args"
args="--insecure $args"
fi
if [ ! -z "$IGNORE" ]; then
args="-i \"$IGNORE\" $args"
if [ ! -z "$ignore_list" ]; then
args="$ignore_list $args"
fi
if [ ! -z "$MODE" ]; then
args="-m \"$MODE\" $args"
args="--mode \"$MODE\" $args"
fi
if [ ! -z "$DELETE_CONFIRM" ]; then
args="-d $args"
args="--delete-confirm $args"
fi
if [ ! -z "$share_with_list" ]; then
@ -80,23 +99,23 @@ if [ ! -z "$share_with_list" ]; then
fi
if [ ! -z "$SHARE_ROLE" ]; then
args="-o $SHARE_ROLE $args"
args="--share-role $SHARE_ROLE $args"
fi
if [ ! -z "$SYNC_MODE" ]; then
args="-S $SYNC_MODE $args"
args="--sync-mode $SYNC_MODE $args"
fi
if [ ! -z "$ALBUM_ORDER" ]; then
args="-O $ALBUM_ORDER $args"
args="--album-order $ALBUM_ORDER $args"
fi
if [ ! -z "$FIND_ASSETS_IN_ALBUMS" ]; then
args="-A $args"
args="--find-assets-in-albums $args"
fi
if [ ! -z "$PATH_FILTER" ]; then
args="-f \"$PATH_FILTER\" $args"
if [ ! -z "$path_filter_list" ]; then
args="$path_filter_list $args"
fi
if [ ! -z "$SET_ALBUM_THUMBNAIL" ]; then
@ -104,4 +123,5 @@ if [ ! -z "$SET_ALBUM_THUMBNAIL" ]; then
fi
BASEDIR=$(dirname "$0")
echo $args
echo $args | xargs python3 -u $BASEDIR/immich_auto_album.py

148
immich_auto_album.py
View file

@ -79,7 +79,7 @@ parser.add_argument("-c", "--chunk-size", default=2000, type=int, help="Maximum
parser.add_argument("-C", "--fetch-chunk-size", default=5000, type=int, help="Maximum number of assets to fetch with a single API call")
parser.add_argument("-l", "--log-level", default="INFO", choices=['CRITICAL', 'ERROR', 'WARNING', 'INFO', 'DEBUG'], help="Log level to use")
parser.add_argument("-k", "--insecure", action="store_true", help="Set to true to ignore SSL verification")
parser.add_argument("-i", "--ignore", default="", type=str, help="A string containing a list of folders, sub-folder sequences or file names separated by ':' that will be ignored.")
parser.add_argument("-i", "--ignore", action="append", help="Use either literals or glob-like patterns to ignore assets for album name creation. This filter is evaluated after any values passed with --path-filter. May be specified multiple times.")
parser.add_argument("-m", "--mode", default=SCRIPT_MODE_CREATE, choices=[SCRIPT_MODE_CREATE, SCRIPT_MODE_CLEANUP, SCRIPT_MODE_DELETE_ALL], help="Mode for the script to run with. CREATE = Create albums based on folder names and provided arguments; CLEANUP = Create album nmaes based on current images and script arguments, but delete albums if they exist; DELETE_ALL = Delete all albums. If the mode is anything but CREATE, --unattended does not have any effect. Only performs deletion if -d/--delete-confirm option is set, otherwise only performs a dry-run.")
parser.add_argument("-d", "--delete-confirm", action="store_true", help="Confirm deletion of albums when running in mode "+SCRIPT_MODE_CLEANUP+" or "+SCRIPT_MODE_DELETE_ALL+". If this flag is not set, these modes will perform a dry run only. Has no effect in mode "+SCRIPT_MODE_CREATE)
parser.add_argument("-x", "--share-with", action="append", help="A user name (or email address of an existing user) to share newly created albums with. Sharing only happens if the album was actually created, not if new assets were added to an existing album. If the the share role should be specified by user, the format <userName>=<shareRole> must be used, where <shareRole> must be one of 'viewer' or 'editor'. May be specified multiple times to share albums with more than one user.")
@ -87,7 +87,7 @@ parser.add_argument("-o", "--share-role", default="viewer", choices=['viewer', '
parser.add_argument("-S", "--sync-mode", default=0, type=int, choices=[0, 1, 2], help="Synchronization mode to use. Synchronization mode helps synchronizing changes in external libraries structures to Immich after albums have already been created. Possible Modes: 0 = do nothing; 1 = Delete any empty albums; 2 = Trigger offline asset removal (REQUIRES API KEY OF AN ADMIN USER!)")
parser.add_argument("-O", "--album-order", default=False, type=str, choices=[False, 'asc', 'desc'], help="Set sorting order for newly created albums to newest or oldest file first, Immich defaults to newest file first")
parser.add_argument("-A", "--find-assets-in-albums", action="store_true", help="By default, the script only finds assets that are not assigned to any album yet. Set this option to make the script discover assets that are already part of an album and handle them as usual.")
parser.add_argument("-f", "--path-filter", default="", type=str, help="Use glob-like patterns to filter assets before album name creation. This filter is evaluated before any values passed with --ignore.")
parser.add_argument("-f", "--path-filter", action="append", help="Use either literals or glob-like patterns to filter assets before album name creation. This filter is evaluated before any values passed with --ignore. May be specified multiple times.")
parser.add_argument("--set-album-thumbnail", choices=ALBUM_THUMBNAIL_SETTINGS, help="Set first/last/random image as thumbnail for newly created albums or albums assets have been added to. If set to "+ALBUM_THUMBNAIL_RANDOM_FILTERED+", thumbnails are shuffled for all albums whose assets would not be filtered out or ignored by the ignore or path-filter options, even if no assets were added during the run. If set to "+ALBUM_THUMBNAIL_RANDOM_ALL+", the thumbnails for ALL albums will be shuffled on every run.")
@ -152,56 +152,6 @@ if is_integer(album_levels) and album_levels == 0:
parser.print_help()
exit(1)
if insecure:
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
# Verify album levels range
if not is_integer(album_levels):
album_levels_range_split = album_levels.split(",")
if (len(album_levels_range_split) != 2
or not is_integer(album_levels_range_split[0])
or not is_integer(album_levels_range_split[1])
or int(album_levels_range_split[0]) == 0
or int(album_levels_range_split[1]) == 0
or (int(album_levels_range_split[0]) >= 0 and int(album_levels_range_split[1]) < 0)
or (int(album_levels_range_split[0]) < 0 and int(album_levels_range_split[1]) >= 0)
or (int(album_levels_range_split[0]) < 0 and int(album_levels_range_split[1]) < 0) and int(album_levels_range_split[0]) > int(album_levels_range_split[1])):
logging.error("Invalid album_levels range format! If a range should be set, the start level and end level must be separated by a comma like '<startLevel>,<endLevel>'. If negative levels are used in a range, <startLevel> must be less than or equal to <endLevel>.")
exit(1)
album_levels_range_arr = album_levels_range_split
# Convert to int
album_levels_range_arr[0] = int(album_levels_range_split[0])
album_levels_range_arr[1] = int(album_levels_range_split[1])
# Special case: both levels are negative and end level is -1, which is equivalent to just negative album level of start level
if(album_levels_range_arr[0] < 0 and album_levels_range_arr[1] == -1):
album_levels = album_levels_range_arr[0]
album_levels_range_arr = ()
logging.debug("album_levels is a range with negative start level and end level of -1, converted to album_levels = %d", album_levels)
else:
logging.debug("valid album_levels range argument supplied")
logging.debug("album_levels_start_level = %d", album_levels_range_arr[0])
logging.debug("album_levels_end_level = %d", album_levels_range_arr[1])
# Deduct 1 from album start levels, since album levels start at 1 for user convenience, but arrays start at index 0
if album_levels_range_arr[0] > 0:
album_levels_range_arr[0] -= 1
album_levels_range_arr[1] -= 1
if not ignore_albums == "":
ignore_albums = ignore_albums.split(":")
else:
ignore_albums = False
path_filter_regex = False
if path_filter == "":
path_filter = False
else:
# # Check if last porition of glob pattern contains a dot '.'
# path_filter_parsed = path_filter.split('/')
# if not '.' in path_filter_parsed[len(path_filter_parsed)-1]:
# # Include all files
# path_filter += "/*.*"
path_filter_regex = glob_to_re(path_filter)
# Request arguments for API calls
requests_kwargs = {
'headers' : {
@ -212,6 +162,30 @@ requests_kwargs = {
'verify' : not insecure
}
def expand_to_glob(expr: str) -> str:
"""
Expands the passed expression to a glob-style
expression if it doesn't contain neither a slash nor an asterisk.
The resulting glob-style expression matches any path that contains the
original expression anywhere.
Parameters
----------
expr : str
Expression to expand to a GLOB-style expression if not already
one
Returns
---------
The original expression if it contained a slash or an asterisk,
otherwise \\*\\*/\\*\\<expr\\>\\*/\\*\\*
"""
if not '/' in expr and not '*' in expr:
glob_expr = f'**/*{expr}*/**'
logging.debug("expanding %s to %s", expr, glob_expr)
return glob_expr
else:
return expr
def divide_chunks(l: list, n: int):
"""Yield successive n-sized chunks from l. """
# looping till length l
@ -492,20 +466,22 @@ def is_asset_ignored(asset: dict) -> bool:
logging.debug("Identified root_path for asset %s = %s", asset_path, asset_root_path)
if asset_root_path:
# First apply filter, if any
if path_filter:
if not re.fullmatch(path_filter_regex, asset_path.replace(asset_root_path, '')):
if len(path_filter_regex) > 0:
any_match = False
for path_filter_regex_entry in path_filter_regex:
if re.fullmatch(path_filter_regex_entry, asset_path.replace(asset_root_path, '')):
any_match = True
if not any_match:
logging.debug("Ignoring asset %s due to path_filter setting!", asset_path)
is_asset_ignored = True
# Check ignore_albums
ignore = False
if ignore_albums:
for ignore_entry in ignore_albums:
if ignore_entry in asset_path:
ignore = True
# If the asset "survived" the path filter, check if it is in the ignore_albums argument
if not is_asset_ignored and len(ignore_albums_regex) > 0:
for ignore_albums_regex_entry in ignore_albums_regex:
if re.fullmatch(ignore_albums_regex_entry, asset_path.replace(asset_root_path, '')):
is_asset_ignored = True
logging.debug("Ignoring asset %s due to ignore_albums setting!", asset_path)
break
if ignore:
logging.debug("Ignoring asset %s due to ignore_albums setting!", asset_path)
is_asset_ignored = True
return is_asset_ignored
@ -661,6 +637,52 @@ def setAlbumThumbnail(albumId: str, assetId: str):
assert r.status_code == 200
if insecure:
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
# Verify album levels range
if not is_integer(album_levels):
album_levels_range_split = album_levels.split(",")
if (len(album_levels_range_split) != 2
or not is_integer(album_levels_range_split[0])
or not is_integer(album_levels_range_split[1])
or int(album_levels_range_split[0]) == 0
or int(album_levels_range_split[1]) == 0
or (int(album_levels_range_split[0]) >= 0 and int(album_levels_range_split[1]) < 0)
or (int(album_levels_range_split[0]) < 0 and int(album_levels_range_split[1]) >= 0)
or (int(album_levels_range_split[0]) < 0 and int(album_levels_range_split[1]) < 0) and int(album_levels_range_split[0]) > int(album_levels_range_split[1])):
logging.error("Invalid album_levels range format! If a range should be set, the start level and end level must be separated by a comma like '<startLevel>,<endLevel>'. If negative levels are used in a range, <startLevel> must be less than or equal to <endLevel>.")
exit(1)
album_levels_range_arr = album_levels_range_split
# Convert to int
album_levels_range_arr[0] = int(album_levels_range_split[0])
album_levels_range_arr[1] = int(album_levels_range_split[1])
# Special case: both levels are negative and end level is -1, which is equivalent to just negative album level of start level
if(album_levels_range_arr[0] < 0 and album_levels_range_arr[1] == -1):
album_levels = album_levels_range_arr[0]
album_levels_range_arr = ()
logging.debug("album_levels is a range with negative start level and end level of -1, converted to album_levels = %d", album_levels)
else:
logging.debug("valid album_levels range argument supplied")
logging.debug("album_levels_start_level = %d", album_levels_range_arr[0])
logging.debug("album_levels_end_level = %d", album_levels_range_arr[1])
# Deduct 1 from album start levels, since album levels start at 1 for user convenience, but arrays start at index 0
if album_levels_range_arr[0] > 0:
album_levels_range_arr[0] -= 1
album_levels_range_arr[1] -= 1
# Create ignore regular expressions
ignore_albums_regex = []
if ignore_albums:
for ignore_albums_entry in ignore_albums:
ignore_albums_regex.append(glob_to_re(expand_to_glob(ignore_albums_entry)))
# Create path filter regular expressions
path_filter_regex = []
if path_filter:
for path_filter_entry in path_filter:
path_filter_regex.append(glob_to_re(expand_to_glob(path_filter_entry)))
# append trailing slash to all root paths
for i in range(len(root_paths)):
if root_paths[i][-1] != '/':