* move markers and style to dedicated map endpoint
* chore: open api
* chore: clean up repos
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* feat(server): user metadata
* add missing method to user mock
* update migration to include cascades
* update sql files
* test: fix e2e
* chore: clean up
---------
Co-authored-by: Daniel Dietzler <mail@ddietzler.dev>
* duplicate detection job, entity, config
* queueing
* job panel, update api
* use embedding in db instead of fetching
* disable concurrency
* only queue visible assets
* handle multiple duplicateIds
* update concurrent queue check
* add provider
* add web placeholder, server endpoint, migration, various fixes
* update sql
* select embedding by default
* rename variable
* simplify
* remove separate entity, handle re-running with different threshold, set default back to 0.02
* fix tests
* add tests
* add index to entity
* formatting
* update asset mock
* fix `upsertJobStatus` signature
* update sql
* formatting
* default to 0.03
* optimize clustering
* use asset's `duplicateId` if present
* update sql
* update tests
* expose admin setting
* refactor
* formatting
* skip if ml is disabled
* debug trash e2e
* remove from web
* remove from sidebar
* test if ml is disabled
* update sql
* separate duplicate detection from clip in config, disable by default for now
* fix doc
* lower minimum `maxDistance`
* update api
* Add and Use Duplicate Detection Feature Flag (#9364)
* Add Duplicate Detection Flag
* Use Duplicate Detection Flag
* Attempt Fixes for Failing Checks
* lower minimum `maxDistance`
* fix tests
---------
Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
* chore: fixes and additions after rebase
* chore: update api (remove new Role enum)
* fix: left join smart search so getAll works without machine learning
* test: trash e2e go back to checking length of assets is zero
* chore: regen api after rebase
* test: fix tests after rebase
* redundant join
---------
Co-authored-by: Nicholas Flamy <30300649+NicholasFlamy@users.noreply.github.com>
Co-authored-by: Zack Pollard <zackpollard@ymail.com>
Co-authored-by: Zack Pollard <zack@futo.org>
* do crop and resize together
* redundant `pipelineColorspace` call
* formatting
* fix rebase
* handle orientation
* remove unused import
* formatting
* use oriented dimensions for half size calculation
* default case for orientation
* simplify orientation code
---------
Co-authored-by: Alex <alex.tran1502@gmail.com>
* feat(server): add `react-mail` as mail template engine and `nodemailer`
* feat(server): add `smtp` related configs to `SystemConfig`
* feat(web): add page for SMTP settings
* feat(server): add `react-email.adapter`
This adapter render the React-Email into HTML and plain/text email.
The output is set as the body of the email.
* feat(server): add `MailRepository` and `MailService`
Allow to use the NestJS-modules-mailer module to send SMTP emails.
This is the base transport for the `NotificationRepository`
* feat(server): register the job dispatcher and Job for async email
This allows to queue email sending jobs for the `EmailService`.
* feat(server): add `NotificationRepository` and `NotificationService`
This act as a middleware to properly route the notification to the right transport.
As POC I've only implemented a simple SMTP transport.
* feat(server): add `welcome` email template
* feat(server): add the first notification on `createUser` in `UserService`
This trigger an event for the `NotificationRepository` that once processes
by using the global config and per-user config will carry the payload to the right notification transport.
* chore: clean up
* chore: clean up web
* fix: type errors"
* fix package lock
* fix mail sending, option to ignore certs
* chore: open api
* chore: clean up
* remove unused import
* feat: email feature flag
* chore: remove unused interface
* small styling
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Daniel Dietzler <mail@ddietzler.dev>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* rename albums_shared_users_users to album_permissions and add readonly column
* disable synchronize on the original join table
* remove unnecessary FK names
* set readonly=true as default for new album shares
* separate and implement album READ and WRITE permission
* expose albumPermissions on the API, deprecate sharedUsers
* generate openapi
* create readonly view on frontend
* ??? move slideshow button out from ellipsis menu so that non-owners can have access too
* correct sharedUsers joins
* add album permission repository
* remove a log
* fix assetCount getting reset when adding users
* fix lint
* add set permission endpoint and UI
* sort users
* remove log
* Revert "??? move slideshow button out from ellipsis menu so that non-owners can have access too"
This reverts commit 1343bfa311.
* rename stuff
* fix db schema annotations
* sql generate
* change readonly default to follow migration
* fix deprecation notice
* change readonly boolean to role enum
* fix joincolumn as primary key
* rename albumUserRepository in album service
* clean up userId and albumId
* add write access to shared link
* fix existing tests
* switch to vitest
* format and fix tests on web
* add new test
* fix one e2e test
* rename new API field to albumUsers
* capitalize serverside enum
* remove unused ReadWrite type
* missed rename from previous commit
* rename to albumUsers in album entity as well
* remove outdated Equals calls
* unnecessary relation
* rename to updateUser in album service
* minor renamery
* move sorting to backend
* rename and separate ALBUM_WRITE as ADD_ASSET and REMOVE_ASSET
* fix tests
* fix "should migrate single moving picture" test failing on European system timezone
* generated changes after merge
* lint fix
* fix correct page to open after removing user from album
* fix e2e tests and some bugs
* rename updateAlbumUser rest endpoint
* add new e2e tests for updateAlbumUser endpoint
* small optimizations
* refactor album e2e test, add new album shared with viewer
* add new test to check if viewer can see the album
* add new e2e tests for readonly share
* failing test: User delete doesn't cascade to UserAlbum entity
* fix: handle deleted users
* use lodash for sort
* add role to addUsersToAlbum endpoint
* add UI for adding editors
* lint fixes
* change role back to editor as DB default
* fix server tests
* redesign user selection modal editor selector
* style tweaks
* fix type error
* Revert "style tweaks"
This reverts commit ab604f4c8f.
* Revert "redesign user selection modal editor selector"
This reverts commit e6f344856c.
* chore: cleanup and improve add user modal
* chore: open api
* small styling
---------
Co-authored-by: mgabor <>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* added "isExternal" to the getLibraryAssetPaths query
* handleQueueAssetRefresh skip "non external" video asset, closes#8562
* correctly implements live photo deletion for external library
* use "external asset" for external library tests
* minor: external library asset checksum is "path hash" not file hash
* renamed to getExternalLibraryAssetPaths and added isExternal where clause
* generated sql
* reverted leftover change
* clean up usage
* i'm not updating all these tests
* update tests
* add indices
* add indices to entities
remove index from person entity
add to face entity
fix
* simplify query
* update sql
* missing await
* remove synchronize false
* feat(server,web): add force delete to immediately remove user
* update wording on force delete confirmation
* fix force delete css
* PR feedback
* cleanup user service delete for force
* adding user status column
* some cleanup and tests
* more test fixes
* run npm run sql:generate
* chore: cleanup and websocket
* chore: linting
* userRepository.restore
* removed bad color class from delete-confirm-dialoge
* additional confirmation for user force delete
* shorten confirmation message
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* commit
* controller/service/repository logic
* use enum
* openapi
* suggest people
* suggest place/camera
* cursor hover
* refactor
* Add try catch
* Remove get people with name service
* Remove deadcode
* people selection
* People placement
* sort people
* Update server/src/domain/repositories/metadata.repository.ts
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* pr feedback
* styling
* done
* open api
* fix test
* use string type
* remmove bad merge
* use correct type
* fix test
* fix lint
* remove unused code
* remove unused code
* pr feedback
* pr feedback
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* basic changes
update version check
set ef_search for clip
* pgvector compatibility
Revert "pgvector compatibility"
This reverts commit 2b66a52aa4097dd27da58138c5288fd87cb9b24a.
pgvector compatibility: minimal edition
pgvector startup check
* update extension at startup
* wording
shortened vector extension variable name
* nightly docker
* fixed version checks
* update tests
add tests for updating extension
remove unnecessary check
* simplify `getRuntimeConfig`
* wording
* reindex on minor version update
* 0.2 upgrade testing
update prod compose
* acquire lock for init
* wip vector down on shutdown
* use upgrade helper
* update image tag
* refine restart check
check error message
* test reindex
testing
upstream fix
formatting
fixed reindexing
* use enum in signature
* fix tests
remove unused code
* add reindexing tests
* update to official 0.2
remove alpha from version name
* add warning test if restart required
* update test image to 0.2.0
* linting and test cleanup
* formatting
* update sql
* wording
* handle setting search path for new and existing databases
* handle new db in reindex check
* fix post-update reindexing
* get dim size
* formatting
* use vbase
* handle different db name
* update sql
* linting
* fix suggested env
* add unicorn to eslint
* fix lint errors for cli
* fix merge
* fix album name extraction
* Update cli/src/commands/upload.command.ts
Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>
* es2k23
* use lowercase os
* return undefined album name
* fix bug in asset response dto
* auto fix issues
* fix server code style
* es2022 and formatting
* fix compilation error
* fix test
* fix config load
* fix last lint errors
* set string type
* bump ts
* start work on web
* web formatting
* Fix UUIDParamDto as UUIDParamDto
* fix library service lint
* fix web errors
* fix errors
* formatting
* wip
* lints fixed
* web can now start
* alphabetical package json
* rename error
* chore: clean up
---------
Co-authored-by: Ben McCann <322311+benmccann@users.noreply.github.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* feat: search peoples
* fix: responsive design
* use existing count
* generate sql file
* fix: tests
* remove visible people
* fix: merge, hide...
* use component
* fix: linter
* chore: regenerate api
* fix: change name when searching for a face
* save search
* remove duplicate
* use enums for query parameters
* fix: increase to 20 for the local search
* use constants
* simplify
* fix: number of people more visible
* fix: merge
* fix: search
* fix: loading spinner position
* pr feedback
* Fix extraction of samsung motionphoto videos
* Refactor binary tag extraction to the repository to consolidate exiftool usage
* format
* fix linting and swap argument orders
* Fix tag name and conditional order
* Add unit test
* Update server test assets submodule
* Remove old motion photo video assets when a new one is extracted
* delete first, then write
* Include motion photo asset uuid's in the filename
If the filenames are not uniquified, then we can't delete old/corrupt ones
* Fix formatting and fix/add tests
* chore: only use new uuid
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* feat(web): onboarding
* feat: openapi
* feat: modulization
* feat: page advancing
* Animation
* Add storage templaete settings
* sql
* more style
* Theme
* information and styling
* hide/show table
* Styling
* Update user property
* fix test
* fix test:
* fix e2e
* test
* Update web/src/lib/components/onboarding-page/onboarding-hello.svelte
Co-authored-by: bo0tzz <git@bo0tzz.me>
* naming
* use System Metadata
* better return type
* onboarding using server metadata
* revert previous changes in user entity
* sql
* test web
* fix test server
* server/web test
* more test
* consolidate color theme change logic
* consolidate save button to storage template
* merge main
* fix web
---------
Co-authored-by: bo0tzz <git@bo0tzz.me>
* feat(server): Enqueue jobs in bulk
The Job Repository now has a `queueAll` method, that enqueues messages
in bulk (using BullMQ's
[`addBulk`](https://docs.bullmq.io/guide/queues/adding-bulks)),
improving performance when many jobs must be enqueued within the same
operation.
Primary change is in `src/domain/job/job.service.ts`, and other services
have been refactored to use `queueAll` when useful.
As a simple local benchmark, triggering a full thumbnail generation
process over a library of ~1,200 assets and ~350 faces went from
**~600ms** to **~250ms**.
* fix: Review feedback
* fix: pgvecto.rs extension breaks typeorm schema:drop command
* fix: parse postgres bigints to javascript number types when selecting data
* feat: verify file size is the same as original asset after copying file for storage template job
* feat: allow disabling of storage template job, defaults to disabled for new instances
* fix: don't allow setting concurrency for storage template migration, can cause race conditions above 1
* feat: add checksum verification when file is copied for storage template job
* fix: extract metadata for assets that aren't visible on timeline
* run migrations after checks
* optional migrations
* only run checks in server and e2e
* re-add migrations for microservices
* refactor
* move e2e init
* remove assert from migration
* update providers
* update microservices app service
* fixed logging
* refactored version check, added unit tests
* more version tests
* don't use mocks for sut
* refactor tests
* suggest image only if postgres is 14, 15 or 16
* review suggestions
* fixed regexp escape
* fix typing
* update migration
Modify Access repository, to evaluate `asset` permissions in bulk.
This is the last set of permission changes, to migrate all of them to
run in bulk!
Queries have been validated to match what they currently generate for single ids.
Queries:
* `activity` owner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "activity" "ActivityEntity"
WHERE
"ActivityEntity"."id" = $1
AND "ActivityEntity"."userId" = $2
)
LIMIT 1
-- After
SELECT "ActivityEntity"."id" AS "ActivityEntity_id"
FROM "activity" "ActivityEntity"
WHERE
"ActivityEntity"."id" IN ($1)
AND "ActivityEntity"."userId" = $2
```
* `activity` album owner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "activity" "ActivityEntity"
LEFT JOIN "albums" "ActivityEntity__ActivityEntity_album"
ON "ActivityEntity__ActivityEntity_album"."id"="ActivityEntity"."albumId"
AND "ActivityEntity__ActivityEntity_album"."deletedAt" IS NULL
WHERE
"ActivityEntity"."id" = $1
AND "ActivityEntity__ActivityEntity_album"."ownerId" = $2
)
LIMIT 1
-- After
SELECT "ActivityEntity"."id" AS "ActivityEntity_id"
FROM "activity" "ActivityEntity"
LEFT JOIN "albums" "ActivityEntity__ActivityEntity_album"
ON "ActivityEntity__ActivityEntity_album"."id"="ActivityEntity"."albumId"
AND "ActivityEntity__ActivityEntity_album"."deletedAt" IS NULL
WHERE
"ActivityEntity"."id" IN ($1)
AND "ActivityEntity__ActivityEntity_album"."ownerId" = $2
```
* `activity` create access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "albums" "AlbumEntity"
LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
WHERE
(
(
"AlbumEntity"."id" = $1
AND "AlbumEntity"."isActivityEnabled" = $2
AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $3
)
OR (
"AlbumEntity"."id" = $4
AND "AlbumEntity"."isActivityEnabled" = $5
AND "AlbumEntity"."ownerId" = $6
)
)
AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1
-- After
SELECT "AlbumEntity"."id" AS "AlbumEntity_id"
FROM "albums" "AlbumEntity"
LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
WHERE
(
(
"AlbumEntity"."id" IN ($1)
AND "AlbumEntity"."isActivityEnabled" = $2
AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $3
)
OR (
"AlbumEntity"."id" IN ($4)
AND "AlbumEntity"."isActivityEnabled" = $5
AND "AlbumEntity"."ownerId" = $6
)
)
AND "AlbumEntity"."deletedAt" IS NULL
```
* feat: unassign person faces
* multiple improvements
* chore: regenerate api
* feat: improve face interactions in photos
* fix: tests
* fix: tests
* optimize
* fix: wrong assignment on complex-multiple re-assignments
* fix: thumbnails with large photos
* fix: complex reassign
* fix: don't send people with faces
* fix: person thumbnail generation
* chore: regenerate api
* add tess
* feat: face box even when zoomed
* fix: change feature photo
* feat: make the blue icon hoverable
* chore: regenerate api
* feat: use websocket
* fix: loading spinner when clicking on the done button
* fix: use the svelte way
* fix: tests
* simplify
* fix: unused vars
* fix: remove unused code
* fix: add migration
* chore: regenerate api
* ci: add unit tests
* chore: regenerate api
* feat: if a new person is created for a face and the server takes more than 15 seconds to generate the person thumbnail, don't wait for it
* reorganize
* chore: regenerate api
* feat: global edit
* pr feedback
* pr feedback
* simplify
* revert test
* fix: face generation
* fix: tests
* fix: face generation
* fix merge
* feat: search names in unmerge face selector modal
* fix: merge face selector
* simplify feature photo generation
* fix: change endpoint
* pr feedback
* chore: fix merge
* chore: fix merge
* fix: tests
* fix: edit & hide buttons
* fix: tests
* feat: show if person is hidden
* feat: rename face to person
* feat: split in new panel
* copy-paste-error
* pr feedback
* fix: feature photo
* do not leak faces
* fix: unmerge modal
* fix: merge modal event
* feat(server): remove duplicates
* fix: title for image thumbnails
* fix: disable side panel when there's no face until next PR
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
Modify Access repository, to evaluate `asset` permissions in bulk.
Queries have been validated to match what they currently generate for single ids.
Queries:
* `asset` album access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "albums" "AlbumEntity"
LEFT JOIN "albums_assets_assets" "AlbumEntity_AlbumEntity__AlbumEntity_assets"
ON "AlbumEntity_AlbumEntity__AlbumEntity_assets"."albumsId"="AlbumEntity"."id"
LEFT JOIN "assets" "AlbumEntity__AlbumEntity_assets"
ON "AlbumEntity__AlbumEntity_assets"."id"="AlbumEntity_AlbumEntity__AlbumEntity_assets"."assetsId"
AND "AlbumEntity__AlbumEntity_assets"."deletedAt" IS NULL
LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
WHERE
(
("AlbumEntity"."ownerId" = $1 AND "AlbumEntity__AlbumEntity_assets"."id" = $2)
OR ("AlbumEntity__AlbumEntity_sharedUsers"."id" = $3 AND "AlbumEntity__AlbumEntity_assets"."id" = $4)
OR ("AlbumEntity"."ownerId" = $5 AND "AlbumEntity__AlbumEntity_assets"."livePhotoVideoId" = $6)
OR ("AlbumEntity__AlbumEntity_sharedUsers"."id" = $7 AND "AlbumEntity__AlbumEntity_assets"."livePhotoVideoId" = $8)
)
AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1
-- After
SELECT
"asset"."id" AS "assetId",
"asset"."livePhotoVideoId" AS "livePhotoVideoId"
FROM "albums" "album"
INNER JOIN "albums_assets_assets" "album_asset"
ON "album_asset"."albumsId"="album"."id"
INNER JOIN "assets" "asset"
ON "asset"."id"="album_asset"."assetsId"
AND "asset"."deletedAt" IS NULL
LEFT JOIN "albums_shared_users_users" "album_sharedUsers"
ON "album_sharedUsers"."albumsId"="album"."id"
LEFT JOIN "users" "sharedUsers"
ON "sharedUsers"."id"="album_sharedUsers"."usersId"
AND "sharedUsers"."deletedAt" IS NULL
WHERE
(
"album"."ownerId" = $1
OR "sharedUsers"."id" = $2
)
AND (
"asset"."id" IN ($3, $4)
OR "asset"."livePhotoVideoId" IN ($5, $6)
)
AND "album"."deletedAt" IS NULL
```
* `asset` owner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "assets" "AssetEntity"
WHERE
"AssetEntity"."id" = $1
AND "AssetEntity"."ownerId" = $2
)
LIMIT 1
-- After
SELECT
"AssetEntity"."id" AS "AssetEntity_id"
FROM "assets" "AssetEntity"
WHERE
"AssetEntity"."id" IN ($1, $2)
AND "AssetEntity"."ownerId" = $3
```
* `asset` partner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "partners" "PartnerEntity"
LEFT JOIN "users" "PartnerEntity__PartnerEntity_sharedWith"
ON "PartnerEntity__PartnerEntity_sharedWith"."id"="PartnerEntity"."sharedWithId"
AND "PartnerEntity__PartnerEntity_sharedWith"."deletedAt" IS NULL
LEFT JOIN "users" "PartnerEntity__PartnerEntity_sharedBy"
ON "PartnerEntity__PartnerEntity_sharedBy"."id"="PartnerEntity"."sharedById"
AND "PartnerEntity__PartnerEntity_sharedBy"."deletedAt" IS NULL
LEFT JOIN "assets" "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"
ON "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"."ownerId"="PartnerEntity__PartnerEntity_sharedBy"."id"
AND "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"."deletedAt" IS NULL
LEFT JOIN "users" "PartnerEntity__sharedBy"
ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
LEFT JOIN "users" "PartnerEntity__sharedWith"
ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
WHERE
"PartnerEntity__PartnerEntity_sharedWith"."id" = $1
AND "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"."id" = $2
)
LIMIT 1
-- After
SELECT
"asset"."id" AS "assetId"
FROM "partners" "partner"
INNER JOIN "users" "sharedBy"
ON "sharedBy"."id"="partner"."sharedById"
AND "sharedBy"."deletedAt" IS NULL
INNER JOIN "assets" "asset"
ON "asset"."ownerId"="sharedBy"."id"
AND "asset"."deletedAt" IS NULL
WHERE
"partner"."sharedWithId" = $1
AND "asset"."id" IN ($2, $3)
```
* `asset` shared link access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "shared_links" "SharedLinkEntity"
LEFT JOIN "albums" "SharedLinkEntity__SharedLinkEntity_album"
ON "SharedLinkEntity__SharedLinkEntity_album"."id"="SharedLinkEntity"."albumId"
AND "SharedLinkEntity__SharedLinkEntity_album"."deletedAt" IS NULL
LEFT JOIN "albums_assets_assets" "760f12c00d97bdcec1ce224d1e3bf449859942b6"
ON "760f12c00d97bdcec1ce224d1e3bf449859942b6"."albumsId"="SharedLinkEntity__SharedLinkEntity_album"."id"
LEFT JOIN "assets" "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"
ON "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."id"="760f12c00d97bdcec1ce224d1e3bf449859942b6"."assetsId"
AND "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."deletedAt" IS NULL
LEFT JOIN "shared_link__asset" "SharedLinkEntity__SharedLinkEntity_assets_SharedLinkEntity"
ON "SharedLinkEntity__SharedLinkEntity_assets_SharedLinkEntity"."sharedLinksId"="SharedLinkEntity"."id"
LEFT JOIN "assets" "SharedLinkEntity__SharedLinkEntity_assets"
ON "SharedLinkEntity__SharedLinkEntity_assets"."id"="SharedLinkEntity__SharedLinkEntity_assets_SharedLinkEntity"."assetsId"
AND "SharedLinkEntity__SharedLinkEntity_assets"."deletedAt" IS NULL
WHERE (
("SharedLinkEntity"."id" = $1 AND "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."id" = $2)
OR ("SharedLinkEntity"."id" = $3 AND "SharedLinkEntity__SharedLinkEntity_assets"."id" = $4)
OR ("SharedLinkEntity"."id" = $5 AND "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."livePhotoVideoId" = $6)
OR ("SharedLinkEntity"."id" = $7 AND "SharedLinkEntity__SharedLinkEntity_assets"."livePhotoVideoId" = $8)
)
)
LIMIT 1
-- After
SELECT
"assets"."id" AS "assetId",
"assets"."livePhotoVideoId" AS "assetLivePhotoVideoId",
"albumAssets"."id" AS "albumAssetId",
"albumAssets"."livePhotoVideoId" AS "albumAssetLivePhotoVideoId"
FROM "shared_links" "sharedLink"
LEFT JOIN "albums" "album"
ON "album"."id"="sharedLink"."albumId"
AND "album"."deletedAt" IS NULL
LEFT JOIN "shared_link__asset" "assets_sharedLink"
ON "assets_sharedLink"."sharedLinksId"="sharedLink"."id"
LEFT JOIN "assets" "assets"
ON "assets"."id"="assets_sharedLink"."assetsId"
AND "assets"."deletedAt" IS NULL
LEFT JOIN "albums_assets_assets" "album_albumAssets"
ON "album_albumAssets"."albumsId"="album"."id"
LEFT JOIN "assets" "albumAssets"
ON "albumAssets"."id"="album_albumAssets"."assetsId"
AND "albumAssets"."deletedAt" IS NULL
WHERE
"sharedLink"."id" = $1
AND (
"assets"."id" IN ($2, $3)
OR "albumAssets"."id" IN ($4, $5)
OR "assets"."livePhotoVideoId" IN ($6, $7)
OR "albumAssets"."livePhotoVideoId" IN ($8, $9)
)
```
* chore: rebase and clean-up
* feat: sync description, add e2e tests
* feat: simplify web code
* chore: unit tests
* fix: linting
* Bug fix with the arrows key
* timezone typeahead filter
timezone typeahead filter
* small stlying
* format fix
* Bug fix in the map selection
Bug fix in the map selection
* Websocket basic
Websocket basic
* Update metadata visualisation through the websocket
* Update timeline
* fix merge
* fix web
* fix web
* maplibre system
* format fix
* format fix
* refactor: clean up
* Fix small bug in the hour/timezone
* Don't diplay modify for readOnly asset
* Add log in case of failure
* Formater + try/catch error
* Remove everything related to websocket
* Revert "Remove everything related to websocket"
This reverts commit 14bcb9e1e4.
* remove notification
* fix test
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
Modify Access repository, to evaluate `authDevice`, `library`, `partner`,
`person`, and `timeline` permissions in bulk.
Queries have been validated to match what they currently generate for
single ids.
As an extra performance improvement, we now use a custom QueryBuilder
for the Partners queries, to avoid the eager relationships that add
unneeded `LEFT JOIN` clauses. We only filter based on the ids present in
the `partners` table, so those joins can be avoided.
Queries:
* `library` owner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "libraries" "LibraryEntity"
WHERE
"LibraryEntity"."id" = $1
AND "LibraryEntity"."ownerId" = $2
AND "LibraryEntity"."deletedAt" IS NULL
)
LIMIT 1
-- After
SELECT "LibraryEntity"."id" AS "LibraryEntity_id"
FROM "libraries" "LibraryEntity"
WHERE
"LibraryEntity"."id" IN ($1, $2)
AND "LibraryEntity"."ownerId" = $3
AND "LibraryEntity"."deletedAt" IS NULL
```
* `library` partner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "partners" "PartnerEntity"
LEFT JOIN "users" "PartnerEntity__sharedBy"
ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
LEFT JOIN "users" "PartnerEntity__sharedWith"
ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
WHERE
"PartnerEntity"."sharedWithId" = $1
AND "PartnerEntity"."sharedById" = $2
)
LIMIT 1
-- After
SELECT
"partner"."sharedById" AS "partner_sharedById",
"partner"."sharedWithId" AS "partner_sharedWithId"
FROM "partners" "partner"
WHERE
"partner"."sharedById" IN ($1, $2)
AND "partner"."sharedWithId" = $3
```
* `authDevice` owner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "user_token" "UserTokenEntity"
WHERE
"UserTokenEntity"."userId" = $1
AND "UserTokenEntity"."id" = $2
)
LIMIT 1
-- After
SELECT "UserTokenEntity"."id" AS "UserTokenEntity_id"
FROM "user_token" "UserTokenEntity"
WHERE
"UserTokenEntity"."userId" = $1
AND "UserTokenEntity"."id" IN ($2, $3)
```
* `timeline` partner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "partners" "PartnerEntity"
LEFT JOIN "users" "PartnerEntity__sharedBy"
ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
LEFT JOIN "users" "PartnerEntity__sharedWith"
ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
WHERE
"PartnerEntity"."sharedWithId" = $1
AND "PartnerEntity"."sharedById" = $2
)
LIMIT 1
-- After
SELECT
"partner"."sharedById" AS "partner_sharedById",
"partner"."sharedWithId" AS "partner_sharedWithId"
FROM "partners" "partner"
WHERE
"partner"."sharedById" IN ($1, $2)
AND "partner"."sharedWithId" = $3
```
* `person` owner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "person" "PersonEntity"
WHERE
"PersonEntity"."id" = $1
AND "PersonEntity"."ownerId" = $2
)
LIMIT 1
-- After
SELECT "PersonEntity"."id" AS "PersonEntity_id"
FROM "person" "PersonEntity"
WHERE
"PersonEntity"."id" IN ($1, $2)
AND "PersonEntity"."ownerId" = $3
```
* `partner` update access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "partners" "PartnerEntity"
LEFT JOIN "users" "PartnerEntity__sharedBy"
ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
LEFT JOIN "users" "PartnerEntity__sharedWith"
ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
WHERE
"PartnerEntity"."sharedWithId" = $1
AND "PartnerEntity"."sharedById" = $2
)
LIMIT 1
-- After
SELECT
"partner"."sharedById" AS "partner_sharedById",
"partner"."sharedWithId" AS "partner_sharedWithId"
FROM "partners" "partner"
WHERE
"partner"."sharedById" IN ($1, $2)
AND "partner"."sharedWithId" = $3
```
* chore(server): Check album permissions in bulk
Modify Access repository, to evaluate `album` permissions in bulk.
Queries have been validated to match what they currently generate for
single ids.
Queries:
* Owner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "albums" "AlbumEntity"
WHERE
"AlbumEntity"."id" = $1
AND "AlbumEntity"."ownerId" = $2
AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1
-- After
SELECT
"AlbumEntity"."id" AS "AlbumEntity_id"
FROM "albums" "AlbumEntity"
WHERE
"AlbumEntity"."id" IN ($1, $2)
AND "AlbumEntity"."ownerId" = $3
AND "AlbumEntity"."deletedAt" IS NULL
```
* Shared link access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "shared_links" "SharedLinkEntity"
WHERE
"SharedLinkEntity"."id" = $1
AND "SharedLinkEntity"."albumId" = $2
)
LIMIT 1
-- After
SELECT
"SharedLinkEntity"."albumId" AS "SharedLinkEntity_albumId",
"SharedLinkEntity"."id" AS "SharedLinkEntity_id"
FROM "shared_links" "SharedLinkEntity"
WHERE
"SharedLinkEntity"."id" = $1
AND "SharedLinkEntity"."albumId" IN ($2, $3)
```
* Shared album access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "albums" "AlbumEntity"
LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
WHERE
"AlbumEntity"."id" = $1
AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $2
AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1
-- After
SELECT
"AlbumEntity"."id" AS "AlbumEntity_id"
FROM "albums" "AlbumEntity"
LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
WHERE
"AlbumEntity"."id" IN ($1, $2)
AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $3
AND "AlbumEntity"."deletedAt" IS NULL
```
* chore(server): Add set utils, avoid double queries for same ids
* chore(server): Review feedback
* feat: add system metadata repository for storing key values for internal usage
* feat: add database entities for geodata
* feat: move reverse geocoding from local-reverse-geocoder to postgresql
* infra: disable synchronization for geodata_places table until typeorm supports earth column
* feat: remove cities override config as we will default all instances to cities500 now
* test: e2e tests don't clear geodata tables on reset
* feat(server): GET /assets endpoint
* chore: open api
* chore: use dumb name
* feat: search by make, model, lens, city, state, country
* chore: open api
* chore: pagination validation and tests
* chore: pr feedback
* Add AssetJobStatus
* fentity
* Add jobStatus field to AssetEntity
* Fix the migration doc paths
* Filter on facesRecognizedAt
* Set facesRecognizedAt field
* Test for facesRecognizedAt
* Done testing
* Adjust FK properties
* Add tests for WithoutProperty.FACES
* chore: non-nullable
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* maplibre on web, custom styles from server
Actually use new vector tile server, custom style.json
support multiple style files, light/dark mode
cleanup, use new map everywhere
send file directly instead of loading first
better light/dark mode switching
remove leaflet
fix mapstyles dto, first draft of map settings
delete and add styles
fix delete default styles
fix tests
only allow one light and one dark style url
revert config core changes
fix server config store
fix tests
move axios fetches to repo
fix package-lock
fix tests
* open api
* add assets to docker container
* web: use mapSettings color for style
* style: add unique ids to map styles
* mobile: use style json for vector / raster
* do not use svelte-material-icons
* add click events to markers, simplify asset detail map
* improve map performance by using asset thumbnails for markers instead of original file
* Remove custom attribution
(by request)
* mobile: update map attribution
* style: map dark mode
* style: map light mode
* zoom level for state
* styling
* overflow gradient
* Limit maxZoom to 14
* mobile: listen for mapStyle changes in MapThumbnail
* mobile: update concurrency
---------
Co-authored-by: shalong-tanwen <139912620+shalong-tanwen@users.noreply.github.com>
Co-authored-by: bo0tzz <git@bo0tzz.me>
Co-authored-by: Alex <alex.tran1502@gmail.com>
* add automatic library scan config options
* add validation
* open api
* use CronJob instead of cron-validator
* fix tests
* catch potential error of the library scan initialization
* better description for input field
* move library scan job initialization to server app service
* fix tests
* add comments to all parameters of cronjob contructor
* make scan a child of a more general library object
* open api
* chore: cleanup
* move cronjob handling to job repoistory
* web: select for common cron expressions
* fix open api
* fix tests
* put scanning settings in nested accordion
* fix system config validation
* refactor, tests
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Add `AlbumRepository` method to retrieve an album's asset ids, with an
optional parameter to only filter by the provided asset ids. With this,
we can now check asset membership using a single query.
When adding or removing assets to an album, checking whether each asset
is already present in the album now requires a single query, instead of
one query per asset.
Related to #4539 performance improvements.
Before:
```
// Asset membership and permissions check (2 queries per asset)
immich_server | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "albums" "AlbumEntity" LEFT JOIN "albums_assets_assets" "AlbumEntity_AlbumEntity__AlbumEntity_assets" ON "AlbumEntity_AlbumEntity__AlbumEntity_assets"."albumsId"="AlbumEntity"."id" LEFT JOIN "assets" "AlbumEntity__AlbumEntity_assets" ON "AlbumEntity__AlbumEntity_assets"."id"="AlbumEntity_AlbumEntity__AlbumEntity_assets"."assetsId" AND ("AlbumEntity__AlbumEntity_assets"."deletedAt" IS NULL) WHERE ( ("AlbumEntity"."id" = $1 AND "AlbumEntity__AlbumEntity_assets"."id" = $2) ) AND ( "AlbumEntity"."deletedAt" IS NULL )) LIMIT 1 -- PARAMETERS: ["3fdf0e58-a1c7-4efe-8288-06e4c3f38df9","b666ae6c-afa8-4d6f-a1ad-7091a0659320"]
immich_server | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["b666ae6c-afa8-4d6f-a1ad-7091a0659320","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
immich_server | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "albums" "AlbumEntity" LEFT JOIN "albums_assets_assets" "AlbumEntity_AlbumEntity__AlbumEntity_assets" ON "AlbumEntity_AlbumEntity__AlbumEntity_assets"."albumsId"="AlbumEntity"."id" LEFT JOIN "assets" "AlbumEntity__AlbumEntity_assets" ON "AlbumEntity__AlbumEntity_assets"."id"="AlbumEntity_AlbumEntity__AlbumEntity_assets"."assetsId" AND ("AlbumEntity__AlbumEntity_assets"."deletedAt" IS NULL) WHERE ( ("AlbumEntity"."id" = $1 AND "AlbumEntity__AlbumEntity_assets"."id" = $2) ) AND ( "AlbumEntity"."deletedAt" IS NULL )) LIMIT 1 -- PARAMETERS: ["3fdf0e58-a1c7-4efe-8288-06e4c3f38df9","c656ab1c-7775-4ff7-b56f-01308c072a76"]
immich_server | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["c656ab1c-7775-4ff7-b56f-01308c072a76","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
immich_server | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "albums" "AlbumEntity" LEFT JOIN "albums_assets_assets" "AlbumEntity_AlbumEntity__AlbumEntity_assets" ON "AlbumEntity_AlbumEntity__AlbumEntity_assets"."albumsId"="AlbumEntity"."id" LEFT JOIN "assets" "AlbumEntity__AlbumEntity_assets" ON "AlbumEntity__AlbumEntity_assets"."id"="AlbumEntity_AlbumEntity__AlbumEntity_assets"."assetsId" AND ("AlbumEntity__AlbumEntity_assets"."deletedAt" IS NULL) WHERE ( ("AlbumEntity"."id" = $1 AND "AlbumEntity__AlbumEntity_assets"."id" = $2) ) AND ( "AlbumEntity"."deletedAt" IS NULL )) LIMIT 1 -- PARAMETERS: ["3fdf0e58-a1c7-4efe-8288-06e4c3f38df9","cf82adb2-1fcc-4f9e-9013-8fc03cc8d3a9"]
immich_server | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["cf82adb2-1fcc-4f9e-9013-8fc03cc8d3a9","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
```
After:
```
// Asset membership check (1 query for all assets)
immich_server | query: SELECT "albums_assets"."assetsId" AS "assetId" FROM "albums_assets_assets" "albums_assets" WHERE "albums_assets"."albumsId" = $1 AND "albums_assets"."assetsId" IN ($2, $3, $4) -- PARAMETERS: ["ca870d76-6311-4e89-bf9a-f5b51ea2452c","b666ae6c-afa8-4d6f-a1ad-7091a0659320","c656ab1c-7775-4ff7-b56f-01308c072a76","cf82adb2-1fcc-4f9e-9013-8fc03cc8d3a9"]
// Permissions check (1 query per asset)
immich_server | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["b666ae6c-afa8-4d6f-a1ad-7091a0659320","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
immich_server | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["c656ab1c-7775-4ff7-b56f-01308c072a76","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
immich_server | query: SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (SELECT 1 FROM "assets" "AssetEntity" WHERE ("AssetEntity"."id" = $1 AND "AssetEntity"."ownerId" = $2)) LIMIT 1 -- PARAMETERS: ["cf82adb2-1fcc-4f9e-9013-8fc03cc8d3a9","6bc60cf1-bd18-4501-a1c2-120b51276fda"]
```
