1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2025-01-10 13:56:47 +01:00
Commit graph

33 commits

Author SHA1 Message Date
Daniel Dietzler
a46366d336
chore(server): introduce proper job status (#7932)
* introduce proper job status

* fix condition for onDone jobs

* fix tests
2024-03-15 09:16:54 -04:00
Mert
ee8e8a0c0f
perf(server): optimize getByIds query (#7918)
* clean up usage

* i'm not updating all these tests

* update tests

* add indices

* add indices to entities

remove index from person entity

add to face entity

fix

* simplify query

* update sql

* missing await

* remove synchronize false
2024-03-14 01:58:09 -04:00
Jonathan Jogenfors
5d377e5b0f
chore(server): eslint await-thenable (#7545)
* await-thenable

* fix library watchers

* moar eslint

* fix test

* fix typo

* try to remove check void return

* fix checksVoidReturn

* move to domain utils

* remove eslint ignores

* chore: cleanup types

* chore: use logger

* fix: e2e

---------

Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-03-05 17:23:06 -05:00
Mert
dc7a329cc8
fix(server): only queue ml / transcoding jobs after thumbnail generation on upload (#7516) 2024-02-28 23:23:21 -05:00
Jonathan Jogenfors
f44fa45aa0
chore(server,cli,web): housekeeping and stricter code style (#6751)
* add unicorn to eslint

* fix lint errors for cli

* fix merge

* fix album name extraction

* Update cli/src/commands/upload.command.ts

Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>

* es2k23

* use lowercase os

* return undefined album name

* fix bug in asset response dto

* auto fix issues

* fix server code style

* es2022 and formatting

* fix compilation error

* fix test

* fix config load

* fix last lint errors

* set string type

* bump ts

* start work on web

* web formatting

* Fix UUIDParamDto as UUIDParamDto

* fix library service lint

* fix web errors

* fix errors

* formatting

* wip

* lints fixed

* web can now start

* alphabetical package json

* rename error

* chore: clean up

---------

Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-02-01 22:18:00 -05:00
Mert
ae7f174948
refactor: rename clip -> smart search (#6713) 2024-01-29 09:51:22 -05:00
Mert
bd87eb309c
feat(server): optimize partial facial recognition (#6634)
* optimize partial facial recognition

* add tests

* use map

* bulk insert faces
2024-01-25 01:27:39 -05:00
Mert
68f52818ae
feat(server): separate face clustering job (#5598)
* separate facial clustering job

* update api

* fixed some tests

* invert clustering

* hdbscan

* update api

* remove commented code

* wip dbscan

* cleanup

removed cluster endpoint

remove commented code

* fixes

updated tests

minor fixes and formatting

fixed queuing

refinements

* scale search range based on library size

* defer non-core faces

* optimizations

removed unused query option

* assign faces individually for correctness

fixed unit tests

remove unused method

* don't select face embedding

update sql

linting

fixed ml typing

* updated job mock

* paginate people query

* select face embeddings because typeorm

* fix setting face detection concurrency

* update sql

formatting

linting

* simplify logic

remove unused imports

* more specific delete signature

* more accurate typing for face stubs

* add migration

formatting

* chore: better typing

* don't select embedding by default

remove unused import

* updated sql

* use normal try/catch

* stricter concurrency typing and enforcement

* update api

* update job concurrency panel to show disabled queues

formatting

* check jobId in queueAll

fix tests

* remove outdated comment

* better facial recognition icon

* wording

wording

formatting

* fixed tests

* fix

* formatting & sql

* try to fix sql check

* more detailed description

* update sql

* formatting

* wording

* update `minFaces` description

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2024-01-18 00:08:48 -05:00
cfitzw
deb1f970a8
feat(server, web): quotas (#4471)
* feat: quotas

* chore: open api

* chore: update status box and upload error message

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2024-01-12 19:43:36 -05:00
Jason Rasmussen
bf1dd36fa9
refactor(server): split api and jobs into separate e2e suites (#6307)
* refactor: domain and infra modules

* refactor(server): e2e tests
2024-01-09 23:04:16 -05:00
Michael Manganiello
4a5b8c3770
feat(server): Enqueue jobs in bulk (#5974)
* feat(server): Enqueue jobs in bulk

The Job Repository now has a `queueAll` method, that enqueues messages
in bulk (using BullMQ's
[`addBulk`](https://docs.bullmq.io/guide/queues/adding-bulks)),
improving performance when many jobs must be enqueued within the same
operation.

Primary change is in `src/domain/job/job.service.ts`, and other services
have been refactored to use `queueAll` when useful.

As a simple local benchmark, triggering a full thumbnail generation
process over a library of ~1,200 assets and ~350 faces went from
**~600ms** to **~250ms**.

* fix: Review feedback
2024-01-01 15:45:42 -05:00
Jason Rasmussen
03eb5ffc5c
refactor(server): simplify config init process (#5702) 2024-01-01 13:16:44 -05:00
Zack Pollard
2e38fa73bf
feat: storage template file move hardening (#5917)
* fix: pgvecto.rs extension breaks typeorm schema:drop command

* fix: parse postgres bigints to javascript number types when selecting data

* feat: verify file size is the same as original asset after copying file for storage template job

* feat: allow disabling of storage template job, defaults to disabled for new instances

* fix: don't allow setting concurrency for storage template migration, can cause race conditions above 1

* feat: add checksum verification when file is copied for storage template job

* fix: extract metadata for assets that aren't visible on timeline
2023-12-29 18:41:33 +00:00
Mert
092a23fd7f
feat(server,ml): remove image tagging (#5903)
* remove image tagging

* updated lock

* fixed tests, improved logging

* be nice

* fixed tests
2023-12-20 20:47:56 -05:00
Mert
cb1201e690
chore(web): update job dashboard (#5745)
* rename clip encoding to smart search

* update job subtitles

* update api

* update smart search job title and subtitle

* fix `getJobName`

* change smart search icon

* formatting

* wording

* update reference to clip

* formatting

* update reference to Encode CLIP
2023-12-16 10:50:46 -06:00
Mert
458257847e
fix(server): disable classification by default (#5708)
* disable classification by default

* fixed tests
2023-12-14 23:34:18 -06:00
Jason Rasmussen
1e99ba8167
feat: use pgvecto.rs (#3605) 2023-12-08 11:15:46 -05:00
Daniel Dietzler
cd375a976e
feat(server): custom library scanning interval (#4390)
* add automatic library scan config options

* add validation

* open api

* use CronJob instead of cron-validator

* fix tests

* catch potential error of the library scan initialization

* better description for input field

* move library scan job initialization to server app service

* fix tests

* add comments to all parameters of cronjob contructor

* make scan a child of a more general library object

* open api

* chore: cleanup

* move cronjob handling to job repoistory

* web: select for common cron expressions

* fix open api

* fix tests

* put scanning settings in nested accordion

* fix system config validation

* refactor, tests

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-10-31 15:19:12 -05:00
Jason Rasmussen
2ea080cacd
refactor: domain repositories (#4403) 2023-10-09 14:25:03 +00:00
Daniel Dietzler
0243570c0b
fix(server): make system config core singleton (#4392)
* make system config core singleton

* refactor

* fix tests

* chore: fix tests

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-10-09 00:51:03 +00:00
Alex
36b21948bf
feat(web): enable websocket (#3765)
* send store event to page

* fix format

* add new asset to existing bucket

* format

* debouncing

* format

* load bucket

* feedback

* feat: listen to deletes and auto-subscribe on all asset grid pages

* feat: auto refresh on person thumbnail

* chore: skip upload event for now

* fix: person thumbnail event

* fix merge

* update handleAssetDeletion with websocket communication

* update info box on mount

* fix test

* fix test

* feat: event for trash asset

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-10-06 15:48:11 -05:00
shenlong
4a8887f37b
feat(server): trash asset (#4015)
* refactor(server): delete assets endpoint

* fix: formatting

* chore: cleanup

* chore: open api

* chore(mobile): replace DeleteAssetDTO with BulkIdsDTOs

* feat: trash an asset

* chore(server): formatting

* chore: open api

* chore: wording

* chore: open-api

* feat(server): add withDeleted to getAssets queries

* WIP: mobile-recycle-bin

* feat(server): recycle-bin to system config

* feat(web): use recycle-bin system config

* chore(server): domain assetcore removed

* chore(server): rename recycle-bin to trash

* chore(web): rename recycle-bin to trash

* chore(server): always send soft deleted assets for getAllByUserId

* chore(web): formatting

* feat(server): permanent delete assets older than trashed period

* feat(web): trash empty placeholder image

* feat(server): empty trash

* feat(web): empty trash

* WIP: mobile-recycle-bin

* refactor(server): empty / restore trash to separate endpoint

* test(server): handle failures

* test(server): fix e2e server-info test

* test(server): deletion test refactor

* feat(mobile): use map settings from server-config to enable / disable map

* feat(mobile): trash asset

* fix(server): operations on assets in trash

* feat(web): show trash statistics

* fix(web): handle trash enabled

* fix(mobile): restore updates from trash

* fix(server): ignore trashed assets for person

* fix(server): add / remove search index when trashed / restored

* chore(web): format

* fix(server): asset service test

* fix(server): include trashed assts for duplicates from uploads

* feat(mobile): no dialog for trash, always dialog for permanent delete

* refactor(mobile): use isar where instead of dart filter

* refactor(mobile): asset provide - handle deletes in single db txn

* chore(mobile): review changes

* feat(web): confirmation before empty trash

* server: review changes

* fix(server): handle library changes

* fix: filter external assets from getting trashed / deleted

* fix(server): empty-bin

* feat: broadcast config update events through ws

* change order of trash button on mobile

* styling

* fix(mobile): do not show trashed toast for local only assets

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-10-06 02:01:14 -05:00
Daniel Dietzler
3053cbd4c8
chore(server): Store generated files (thumbnails, encoded video) in subdirectories (#4112)
* save thumbnails in subdirectories

* migration job, migrate assets and face thumbnails

* fix tests

* directory depth of two instead of three

* cleanup empty dirs after migration

* clean up empty dirs after migration, migrate people without assetId

* add job card for new migration job

* fix removeEmptyDirs race condition because of missing await

* cleanup empty directories after asset deletion

* move ensurePath to storage core

* rename jobs

* remove unnecessary property of IEntityJob

* use updated person getById, minor refactoring

* ensure that directory cleanup doesn't interfere with migration

* better description for job in ui

* fix remove directories when migration is done

* cleanup empty folders at start of migration

* fix: actually persist concurrency setting

* add comment explaining regex

* chore: cleanup

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-09-25 11:07:21 -04:00
Jonathan Jogenfors
acdc66413c
feat(server,web): libraries (#3124)
* feat: libraries

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex <alex.tran1502@gmail.com>
2023-09-20 13:16:33 +02:00
Daniel Dietzler
7173af60e4
chore(server): Improve test coverage! (#3889)
* tests for person service

* tests for auth service

* tests for access core

* improve tests for album service

* fix missing brackets and remove comments

* tests for asset service

* tests for face recognition

* tests for job service

* feedback

* tests for search service (broken)

* fix: disabled search test

* tests for smart-info service

* tests for storage template service

* tests for user service

* fix formatting of untouched files LOL

* attempt to fix formatting

* streamline api utils, add asset api for uploading files

* test upload of assets

* fix formatting

* move test-utils to correct folder

* test add assets to album

* use random bytes instead of test image

* (e2e) test albums with assets

* (e2e) complete tests for album endpoints

* (e2e) tests for asset endpoint

* fix: asset upload/import dto validation

* (e2e) tests for statistics asset endpoint

* fix wrong describe text

* (e2e) tests for people with faces

* (e2e) clean up person tests

* (e2e) tests for partner sharing endpoints

* (e2e) tests for link sharing

* (e2e) tests for the asset time bucket endpoint

* fix minor issues

* remove access.core.spec.ts

* chore: wording

* chore: organize test api files

* chore: fix test describe

* implement feedback

* fix race condition in album tests

---------

Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-09-11 11:56:38 -04:00
Fynn Petersen-Frey
cf9e04c8ec
feat(server): asset entity audit (#3824)
* feat(server): audit log

* feedback

* Insert to database

* migration

* test

* controller/repository/service

* test

* module

* feat(server): implement audit endpoint

* directly return changed assets

* add daily cleanup of audit table

* fix tests

* review feedback

* ci

* refactor(server): audit implementation

* chore: open api

---------

Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
Co-authored-by: Fynn Petersen-Frey <zoodyy@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
2023-08-24 15:28:50 -04:00
Jason Rasmussen
4762fd83d4
fix(server): link live photos after metadata extraction finishes (#3702)
* fix(server): link live photos after metadata extraction finishes

* chore: fix test

---------

Co-authored-by: Alex <alex.tran1502@gmail.com>
2023-08-15 20:34:57 -05:00
Jason Rasmussen
9e085c1071
refactor(server): test fixtures (#3491) 2023-07-31 20:28:07 -05:00
Mert
71a2914f3e
fix(server): transcodes failing due to storage migration happening simultaneously (#3071) 2023-07-05 00:36:16 -05:00
Jason Rasmussen
2feac54382
refactro(server): job dto (#3057) 2023-06-30 23:41:12 -05:00
Covalent
3e804f16df
feat(web,server): add thumbhash support (#2649)
* add thumbhash: server generation and web impl

* move logic to infra & use byta in db

* remove unnecesary logs

* update generated API and simplify thumbhash gen

* fix check errors

* removed unnecessary library and css tag

* style edits

* syntax mistake

* update server test, change thumbhash job name

* fix tests

* Update server/src/domain/asset/response-dto/asset-response.dto.ts

Co-authored-by: Thomas <9749173+uhthomas@users.noreply.github.com>

* add unit test, change migration date

* change to official thumbhash impl

* update call method to not use eval

* "generate missing" looks for thumbhash

* improve queue & improve syntax

* update syntax again

* update tests

* fix thumbhash generation

* consolidate queueing to avoid duplication

* cover all types of incorrect thumbnail cases

* split out jest tasks

* put back thumbnail duration loading for images without thumbhash

* Remove stray package.json

---------

Co-authored-by: Luke McCarthy <mail@lukehmcc.com>
Co-authored-by: Thomas <9749173+uhthomas@users.noreply.github.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
2023-06-17 22:22:31 -05:00
Jason Rasmussen
bff6914a73
chore(server): organize imports (#2779)
* feat: lint rule for organize imports

* chore: organize imports
2023-06-16 19:54:17 +00:00
Jason Rasmussen
8ebac41318
refactor(server)*: tsconfigs (#2689)
* refactor(server): tsconfigs

* chore: dummy commit

* fix: start.sh

* chore: restore original entry scripts
2023-06-08 10:01:07 -05:00
Renamed from server/libs/domain/src/job/job.service.spec.ts (Browse further)