* fix(server): Split database queries based on PostgreSQL bound params limit
PostgreSQL uses a 16-bit integer to indicate the number of bound
parameters.
This means that the maximum number of parameters for any query is 65535.
Any query that tries to bind more than that (e.g. searching by a list of
IDs) requires splitting the query into multiple chunks.
This change includes refactoring every Repository that runs queries
using a list of ids, and either flattening or merging results.
Fixes#5788, #5997.
Also, potentially a fix for #4648 (at least based on
[this comment](https://github.com/immich-app/immich/issues/4648#issuecomment-1826134027)).
References:
* https://github.com/typeorm/typeorm/issues/7565
* [PostgreSQL message format - Bind](https://www.postgresql.org/docs/15/protocol-message-formats.html#PROTOCOL-MESSAGE-FORMATS-BIND)
* misc: Create Chunked decorator to simplify implementation
* feat: Add ChunkedArray/ChunkedSet decorators
The current `removeAsset` implementation just builds the query but does
not execute it. That also seems to be the reason the `@GenerateSql`
decorator was commented out.
* feat(web): onboarding
* feat: openapi
* feat: modulization
* feat: page advancing
* Animation
* Add storage templaete settings
* sql
* more style
* Theme
* information and styling
* hide/show table
* Styling
* Update user property
* fix test
* fix test:
* fix e2e
* test
* Update web/src/lib/components/onboarding-page/onboarding-hello.svelte
Co-authored-by: bo0tzz <git@bo0tzz.me>
* naming
* use System Metadata
* better return type
* onboarding using server metadata
* revert previous changes in user entity
* sql
* test web
* fix test server
* server/web test
* more test
* consolidate color theme change logic
* consolidate save button to storage template
* merge main
* fix web
---------
Co-authored-by: bo0tzz <git@bo0tzz.me>
* fix(deps): update dependency @nestjs/schedule to v4
* fix: change CronJob in addCronJob to match new type required by nestjs schedule module
---------
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Zack Pollard <zackpollard@ymail.com>
* feat(server): Enqueue jobs in bulk
The Job Repository now has a `queueAll` method, that enqueues messages
in bulk (using BullMQ's
[`addBulk`](https://docs.bullmq.io/guide/queues/adding-bulks)),
improving performance when many jobs must be enqueued within the same
operation.
Primary change is in `src/domain/job/job.service.ts`, and other services
have been refactored to use `queueAll` when useful.
As a simple local benchmark, triggering a full thumbnail generation
process over a library of ~1,200 assets and ~350 faces went from
**~600ms** to **~250ms**.
* fix: Review feedback
* fix: locks need to be acquired and released using the same session
* feat: only allow a single storage template operation at a single time
this has been added to avoid possible rare race conditions where two files are moved at once to the same path with the same name, causing our duplicate iterator to not detect them and therefore both files will have the same name and overwrite eachother
* fix: pgvecto.rs extension breaks typeorm schema:drop command
* fix: parse postgres bigints to javascript number types when selecting data
* feat: verify file size is the same as original asset after copying file for storage template job
* feat: allow disabling of storage template job, defaults to disabled for new instances
* fix: don't allow setting concurrency for storage template migration, can cause race conditions above 1
* feat: add checksum verification when file is copied for storage template job
* fix: extract metadata for assets that aren't visible on timeline
* fix(server): Reduce number of bound parameters in Access queries
According to https://github.com/typeorm/typeorm/issues/7565, the
introduction of bulk queries for permission checks could quickly reach
the limit of 65536 bound parameters allowed by the PostgreSQL
connection.
To avoid reaching that limit, this first change refactors the Access
queries that are expanding the set of ids multiple times. For example,
`asset.checkSharedLinkAccess` expands the ids 4 times, so providing just
~16400 ids is enough to break the query.
Refactored queries:
* activity.checkCreateAccess
```sql
-- Before
SELECT "AlbumEntity"."id" AS "AlbumEntity_id"
FROM "albums" "AlbumEntity"
LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
AND ("AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL)
WHERE
(
(
"AlbumEntity"."id" IN ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
AND "AlbumEntity"."isActivityEnabled" = $11
AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $12
)
OR (
"AlbumEntity"."id" IN ($13, $14, $15, $16, $17, $18, $19, $20, $21, $22)
AND "AlbumEntity"."isActivityEnabled" = $23
AND "AlbumEntity"."ownerId" = $24
)
)
AND "AlbumEntity"."deletedAt" IS NULL
-- After
SELECT "album"."id" AS "album_id"
FROM "albums" "album"
LEFT JOIN "albums_shared_users_users" "album_sharedUsers"
ON "album_sharedUsers"."albumsId"="album"."id"
LEFT JOIN "users" "sharedUsers"
ON "sharedUsers"."id"="album_sharedUsers"."usersId"
AND "sharedUsers"."deletedAt" IS NULL
WHERE
"album"."id" IN ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
AND "album"."isActivityEnabled" = true
AND (
"album"."ownerId" = $11
OR "sharedUsers"."id" = $12
)
AND "album"."deletedAt" IS NULL
```
* asset.checkAlbumAccess
```sql
-- Before
SELECT
"asset"."id" AS "assetId",
"asset"."livePhotoVideoId" AS "livePhotoVideoId"
FROM "albums" "album"
INNER JOIN "albums_assets_assets" "album_asset"
ON "album_asset"."albumsId"="album"."id"
INNER JOIN "assets" "asset"
ON "asset"."id"="album_asset"."assetsId"
AND "asset"."deletedAt" IS NULL
LEFT JOIN "albums_shared_users_users" "album_sharedUsers"
ON "album_sharedUsers"."albumsId"="album"."id"
LEFT JOIN "users" "sharedUsers"
ON "sharedUsers"."id"="album_sharedUsers"."usersId"
AND "sharedUsers"."deletedAt" IS NULL
WHERE
(
"album"."ownerId" = $1
OR "sharedUsers"."id" = $2
)
AND (
"asset"."id" IN ($3, $4, $5, $6, $7, $8, $9, $10, $11, $12)
OR "asset"."livePhotoVideoId" IN ($13, $14, $15, $16, $17, $18, $19, $20, $21, $22)
)
AND "album"."deletedAt" IS NULL
-- After
WITH "assetIds" AS (
SELECT unnest(array[$1, $2, $3, $4, $5, $6, $7, $8, $9, $10])::uuid AS "id"
FROM (SELECT 1 AS dummy_column) "dummy_table"
)
SELECT
"asset"."id" AS "assetId",
"asset"."livePhotoVideoId" AS "livePhotoVideoId"
FROM "albums" "album"
INNER JOIN "albums_assets_assets" "album_asset"
ON "album_asset"."albumsId"="album"."id"
INNER JOIN "assets" "asset"
ON "asset"."id"="album_asset"."assetsId"
AND "asset"."deletedAt" IS NULL
LEFT JOIN "albums_shared_users_users" "album_sharedUsers"
ON "album_sharedUsers"."albumsId"="album"."id"
LEFT JOIN "users" "sharedUsers"
ON "sharedUsers"."id"="album_sharedUsers"."usersId"
AND "sharedUsers"."deletedAt" IS NULL
WHERE
(
"album"."ownerId" = $11
OR "sharedUsers"."id" = $12
)
AND (
"asset"."id" IN (SELECT id FROM "assetIds")
OR "asset"."livePhotoVideoId" IN (SELECT id FROM "assetIds")
)
AND "album"."deletedAt" IS NULL
```
* asset.checkSharedLinkAccess
```sql
-- Before
SELECT
"assets"."id" AS "assetId",
"assets"."livePhotoVideoId" AS "assetLivePhotoVideoId",
"albumAssets"."id" AS "albumAssetId",
"albumAssets"."livePhotoVideoId" AS "albumAssetLivePhotoVideoId"
FROM "shared_links" "sharedLink"
LEFT JOIN "albums" "album"
ON "album"."id"="sharedLink"."albumId"
AND "album"."deletedAt" IS NULL
LEFT JOIN "shared_link__asset" "assets_sharedLink"
ON "assets_sharedLink"."sharedLinksId"="sharedLink"."id"
LEFT JOIN "assets" "assets"
ON "assets"."id"="assets_sharedLink"."assetsId"
AND "assets"."deletedAt" IS NULL
LEFT JOIN "albums_assets_assets" "album_albumAssets"
ON "album_albumAssets"."albumsId"="album"."id"
LEFT JOIN "assets" "albumAssets"
ON "albumAssets"."id"="album_albumAssets"."assetsId"
AND "albumAssets"."deletedAt" IS NULL
WHERE
"sharedLink"."id" = $1
AND (
"assets"."id" IN ($2, $3, $4, $5, $6, $7, $8, $9, $10, $11)
OR "albumAssets"."id" IN ($12, $13, $14, $15, $16, $17, $18, $19, $20, $21)
OR "assets"."livePhotoVideoId" IN ($22, $23, $24, $25, $26, $27, $28, $29, $30, $31)
OR "albumAssets"."livePhotoVideoId" IN ($32, $33, $34, $35, $36, $37, $38, $39, $40, $41)
)
-- After
WITH "assetIds" AS (
SELECT unnest(array[$1, $2, $3, $4, $5, $6, $7, $8, $9, $10])::uuid AS "id"
FROM (SELECT 1 AS dummy_column) "dummy_table"
)
SELECT
"assets"."id" AS "assetId",
"assets"."livePhotoVideoId" AS "assetLivePhotoVideoId",
"albumAssets"."id" AS "albumAssetId",
"albumAssets"."livePhotoVideoId" AS "albumAssetLivePhotoVideoId"
FROM "shared_links" "sharedLink"
LEFT JOIN "albums" "album"
ON "album"."id"="sharedLink"."albumId"
AND "album"."deletedAt" IS NULL
LEFT JOIN "shared_link__asset" "assets_sharedLink"
ON "assets_sharedLink"."sharedLinksId"="sharedLink"."id"
LEFT JOIN "assets" "assets"
ON "assets"."id"="assets_sharedLink"."assetsId"
AND "assets"."deletedAt" IS NULL
LEFT JOIN "albums_assets_assets" "album_albumAssets"
ON "album_albumAssets"."albumsId"="album"."id"
LEFT JOIN "assets" "albumAssets"
ON "albumAssets"."id"="album_albumAssets"."assetsId"
AND "albumAssets"."deletedAt" IS NULL
WHERE
"sharedLink"."id" = $11
AND (
"assets"."id" IN (SELECT id FROM "assetIds")
OR "albumAssets"."id" IN (SELECT id FROM "assetIds")
OR "assets"."livePhotoVideoId" IN (SELECT id FROM "assetIds")
OR "albumAssets"."livePhotoVideoId" IN (SELECT id FROM "assetIds")
)
```
* fix: Use array overlapping instead of CTEs
included all thumbnail metadata. It seems this has to be explicitly disabled.
Refs: #4382
feat. basic metadata e2e test
fix: use tiff thumbnails in first step + e2e fix
fix: revert switch to tiff
feat: test metadata of both webp and jpg
feat: use upload in e2e test
fix: lint
strip metadata with exiftool
use `withIccProfile`
fix e2e
formatting
run jobs in e2e
* run migrations after checks
* optional migrations
* only run checks in server and e2e
* re-add migrations for microservices
* refactor
* move e2e init
* remove assert from migration
* update providers
* update microservices app service
* fixed logging
* refactored version check, added unit tests
* more version tests
* don't use mocks for sut
* refactor tests
* suggest image only if postgres is 14, 15 or 16
* review suggestions
* fixed regexp escape
* fix typing
* update migration
* Hide the person age if it is negative
* Add validation to prevent future birth dates
* Add comment
* Add test, Add birth date validation and update birth date modal
* Add birthDate validation in PersonService and SetBirthDateModal
* Running npm run format:fix
* Generating the migration file propoerly, and Make the birthdate form logic simpler
* Make birthDate type only string
* Adding useLocationPin back
* Allow building and installing cli
* feat: add format fix
* docs: remove cli folder
* feat: use immich scoped package
* feat: rewrite cli readme
* docs: add info on running without building
* cleanup
* chore: remove import functionality from cli
* feat: add logout to cli
* docs: add todo for file format from server
* docs: add compilation step to cli
* fix: success message spacing
* feat: can create albums
* fix: add check step to cli
* fix: typos
* feat: pull file formats from server
* chore: use crawl service from server
* chore: fix lint
* docs: add cli documentation
* chore: rename ignore pattern
* chore: add version number to cli
* feat: use sdk
* fix: cleanup
* feat: album name on windows
* chore: remove skipped asset field
* feat: add more info to server-info command
* chore: cleanup
* wip
* chore: remove unneeded packages
* e2e test can start
* git ignore for geocode in cli
* add cli e2e to github actions
* can do e2e tests in the cli
* simplify e2e test
* cleanup
* set matrix strategy in workflow
* run npm ci in server
* choose different working directory
* check out submodules too
* increase test timeout
* set node version
* cli docker e2e tests
* fix cli docker file
* run cli e2e in correct folder
* set docker context
* correct docker build
* remove cli from dockerignore
* chore: fix docs links
* feat: add cli v2 milestone
* fix: set correct cli date
* remove submodule
* chore: add npmignore
* chore(cli): push to npm
* fix: server e2e
* run npm ci in server
* remove state from e2e
* run npm ci in server
* reshuffle docker compose files
* use new e2e composes in makefile
* increase test timeout to 10 minutes
* make github actions run makefile e2e tests
* cleanup github test names
* assert on server version
* chore: split cli e2e tests into one file per command
* chore: set cli release working dir
* chore: add repo url to npmjs
* chore: bump node setup to v4
* chore: normalize the github url
* check e2e code in lint
* fix lint
* test key login flow
* feat: allow configurable config dir
* fix session service tests
* create missing dir
* cleanup
* bump cli version to 2.0.4
* remove form-data
* feat: allow single files as argument
* add version option
* bump dependencies
* fix lint
* wip use axios as upload
* version bump
* cApiTALiZaTiON
* don't touch package lock
* wip: don't use job queues
* don't use make for cli e2e
* fix server e2e
* chore: remove old gha step
* add npm ci to server
---------
Co-authored-by: Alex <alex.tran1502@gmail.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Modify Access repository, to evaluate `asset` permissions in bulk.
This is the last set of permission changes, to migrate all of them to
run in bulk!
Queries have been validated to match what they currently generate for single ids.
Queries:
* `activity` owner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "activity" "ActivityEntity"
WHERE
"ActivityEntity"."id" = $1
AND "ActivityEntity"."userId" = $2
)
LIMIT 1
-- After
SELECT "ActivityEntity"."id" AS "ActivityEntity_id"
FROM "activity" "ActivityEntity"
WHERE
"ActivityEntity"."id" IN ($1)
AND "ActivityEntity"."userId" = $2
```
* `activity` album owner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "activity" "ActivityEntity"
LEFT JOIN "albums" "ActivityEntity__ActivityEntity_album"
ON "ActivityEntity__ActivityEntity_album"."id"="ActivityEntity"."albumId"
AND "ActivityEntity__ActivityEntity_album"."deletedAt" IS NULL
WHERE
"ActivityEntity"."id" = $1
AND "ActivityEntity__ActivityEntity_album"."ownerId" = $2
)
LIMIT 1
-- After
SELECT "ActivityEntity"."id" AS "ActivityEntity_id"
FROM "activity" "ActivityEntity"
LEFT JOIN "albums" "ActivityEntity__ActivityEntity_album"
ON "ActivityEntity__ActivityEntity_album"."id"="ActivityEntity"."albumId"
AND "ActivityEntity__ActivityEntity_album"."deletedAt" IS NULL
WHERE
"ActivityEntity"."id" IN ($1)
AND "ActivityEntity__ActivityEntity_album"."ownerId" = $2
```
* `activity` create access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "albums" "AlbumEntity"
LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
WHERE
(
(
"AlbumEntity"."id" = $1
AND "AlbumEntity"."isActivityEnabled" = $2
AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $3
)
OR (
"AlbumEntity"."id" = $4
AND "AlbumEntity"."isActivityEnabled" = $5
AND "AlbumEntity"."ownerId" = $6
)
)
AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1
-- After
SELECT "AlbumEntity"."id" AS "AlbumEntity_id"
FROM "albums" "AlbumEntity"
LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
WHERE
(
(
"AlbumEntity"."id" IN ($1)
AND "AlbumEntity"."isActivityEnabled" = $2
AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $3
)
OR (
"AlbumEntity"."id" IN ($4)
AND "AlbumEntity"."isActivityEnabled" = $5
AND "AlbumEntity"."ownerId" = $6
)
)
AND "AlbumEntity"."deletedAt" IS NULL
```
* feat: unassign person faces
* multiple improvements
* chore: regenerate api
* feat: improve face interactions in photos
* fix: tests
* fix: tests
* optimize
* fix: wrong assignment on complex-multiple re-assignments
* fix: thumbnails with large photos
* fix: complex reassign
* fix: don't send people with faces
* fix: person thumbnail generation
* chore: regenerate api
* add tess
* feat: face box even when zoomed
* fix: change feature photo
* feat: make the blue icon hoverable
* chore: regenerate api
* feat: use websocket
* fix: loading spinner when clicking on the done button
* fix: use the svelte way
* fix: tests
* simplify
* fix: unused vars
* fix: remove unused code
* fix: add migration
* chore: regenerate api
* ci: add unit tests
* chore: regenerate api
* feat: if a new person is created for a face and the server takes more than 15 seconds to generate the person thumbnail, don't wait for it
* reorganize
* chore: regenerate api
* feat: global edit
* pr feedback
* pr feedback
* simplify
* revert test
* fix: face generation
* fix: tests
* fix: face generation
* fix merge
* feat: search names in unmerge face selector modal
* fix: merge face selector
* simplify feature photo generation
* fix: change endpoint
* pr feedback
* chore: fix merge
* chore: fix merge
* fix: tests
* fix: edit & hide buttons
* fix: tests
* feat: show if person is hidden
* feat: rename face to person
* feat: split in new panel
* copy-paste-error
* pr feedback
* fix: feature photo
* do not leak faces
* fix: unmerge modal
* fix: merge modal event
* feat(server): remove duplicates
* fix: title for image thumbnails
* fix: disable side panel when there's no face until next PR
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
Modify Access repository, to evaluate `asset` permissions in bulk.
Queries have been validated to match what they currently generate for single ids.
Queries:
* `asset` album access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "albums" "AlbumEntity"
LEFT JOIN "albums_assets_assets" "AlbumEntity_AlbumEntity__AlbumEntity_assets"
ON "AlbumEntity_AlbumEntity__AlbumEntity_assets"."albumsId"="AlbumEntity"."id"
LEFT JOIN "assets" "AlbumEntity__AlbumEntity_assets"
ON "AlbumEntity__AlbumEntity_assets"."id"="AlbumEntity_AlbumEntity__AlbumEntity_assets"."assetsId"
AND "AlbumEntity__AlbumEntity_assets"."deletedAt" IS NULL
LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
WHERE
(
("AlbumEntity"."ownerId" = $1 AND "AlbumEntity__AlbumEntity_assets"."id" = $2)
OR ("AlbumEntity__AlbumEntity_sharedUsers"."id" = $3 AND "AlbumEntity__AlbumEntity_assets"."id" = $4)
OR ("AlbumEntity"."ownerId" = $5 AND "AlbumEntity__AlbumEntity_assets"."livePhotoVideoId" = $6)
OR ("AlbumEntity__AlbumEntity_sharedUsers"."id" = $7 AND "AlbumEntity__AlbumEntity_assets"."livePhotoVideoId" = $8)
)
AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1
-- After
SELECT
"asset"."id" AS "assetId",
"asset"."livePhotoVideoId" AS "livePhotoVideoId"
FROM "albums" "album"
INNER JOIN "albums_assets_assets" "album_asset"
ON "album_asset"."albumsId"="album"."id"
INNER JOIN "assets" "asset"
ON "asset"."id"="album_asset"."assetsId"
AND "asset"."deletedAt" IS NULL
LEFT JOIN "albums_shared_users_users" "album_sharedUsers"
ON "album_sharedUsers"."albumsId"="album"."id"
LEFT JOIN "users" "sharedUsers"
ON "sharedUsers"."id"="album_sharedUsers"."usersId"
AND "sharedUsers"."deletedAt" IS NULL
WHERE
(
"album"."ownerId" = $1
OR "sharedUsers"."id" = $2
)
AND (
"asset"."id" IN ($3, $4)
OR "asset"."livePhotoVideoId" IN ($5, $6)
)
AND "album"."deletedAt" IS NULL
```
* `asset` owner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "assets" "AssetEntity"
WHERE
"AssetEntity"."id" = $1
AND "AssetEntity"."ownerId" = $2
)
LIMIT 1
-- After
SELECT
"AssetEntity"."id" AS "AssetEntity_id"
FROM "assets" "AssetEntity"
WHERE
"AssetEntity"."id" IN ($1, $2)
AND "AssetEntity"."ownerId" = $3
```
* `asset` partner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "partners" "PartnerEntity"
LEFT JOIN "users" "PartnerEntity__PartnerEntity_sharedWith"
ON "PartnerEntity__PartnerEntity_sharedWith"."id"="PartnerEntity"."sharedWithId"
AND "PartnerEntity__PartnerEntity_sharedWith"."deletedAt" IS NULL
LEFT JOIN "users" "PartnerEntity__PartnerEntity_sharedBy"
ON "PartnerEntity__PartnerEntity_sharedBy"."id"="PartnerEntity"."sharedById"
AND "PartnerEntity__PartnerEntity_sharedBy"."deletedAt" IS NULL
LEFT JOIN "assets" "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"
ON "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"."ownerId"="PartnerEntity__PartnerEntity_sharedBy"."id"
AND "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"."deletedAt" IS NULL
LEFT JOIN "users" "PartnerEntity__sharedBy"
ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
LEFT JOIN "users" "PartnerEntity__sharedWith"
ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
WHERE
"PartnerEntity__PartnerEntity_sharedWith"."id" = $1
AND "0aabe9f4a62b794e2c24a074297e534f51a4ac6c"."id" = $2
)
LIMIT 1
-- After
SELECT
"asset"."id" AS "assetId"
FROM "partners" "partner"
INNER JOIN "users" "sharedBy"
ON "sharedBy"."id"="partner"."sharedById"
AND "sharedBy"."deletedAt" IS NULL
INNER JOIN "assets" "asset"
ON "asset"."ownerId"="sharedBy"."id"
AND "asset"."deletedAt" IS NULL
WHERE
"partner"."sharedWithId" = $1
AND "asset"."id" IN ($2, $3)
```
* `asset` shared link access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "shared_links" "SharedLinkEntity"
LEFT JOIN "albums" "SharedLinkEntity__SharedLinkEntity_album"
ON "SharedLinkEntity__SharedLinkEntity_album"."id"="SharedLinkEntity"."albumId"
AND "SharedLinkEntity__SharedLinkEntity_album"."deletedAt" IS NULL
LEFT JOIN "albums_assets_assets" "760f12c00d97bdcec1ce224d1e3bf449859942b6"
ON "760f12c00d97bdcec1ce224d1e3bf449859942b6"."albumsId"="SharedLinkEntity__SharedLinkEntity_album"."id"
LEFT JOIN "assets" "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"
ON "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."id"="760f12c00d97bdcec1ce224d1e3bf449859942b6"."assetsId"
AND "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."deletedAt" IS NULL
LEFT JOIN "shared_link__asset" "SharedLinkEntity__SharedLinkEntity_assets_SharedLinkEntity"
ON "SharedLinkEntity__SharedLinkEntity_assets_SharedLinkEntity"."sharedLinksId"="SharedLinkEntity"."id"
LEFT JOIN "assets" "SharedLinkEntity__SharedLinkEntity_assets"
ON "SharedLinkEntity__SharedLinkEntity_assets"."id"="SharedLinkEntity__SharedLinkEntity_assets_SharedLinkEntity"."assetsId"
AND "SharedLinkEntity__SharedLinkEntity_assets"."deletedAt" IS NULL
WHERE (
("SharedLinkEntity"."id" = $1 AND "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."id" = $2)
OR ("SharedLinkEntity"."id" = $3 AND "SharedLinkEntity__SharedLinkEntity_assets"."id" = $4)
OR ("SharedLinkEntity"."id" = $5 AND "4a35f463ae8c5544ede95c4b6d9ce8c686b6bfe6"."livePhotoVideoId" = $6)
OR ("SharedLinkEntity"."id" = $7 AND "SharedLinkEntity__SharedLinkEntity_assets"."livePhotoVideoId" = $8)
)
)
LIMIT 1
-- After
SELECT
"assets"."id" AS "assetId",
"assets"."livePhotoVideoId" AS "assetLivePhotoVideoId",
"albumAssets"."id" AS "albumAssetId",
"albumAssets"."livePhotoVideoId" AS "albumAssetLivePhotoVideoId"
FROM "shared_links" "sharedLink"
LEFT JOIN "albums" "album"
ON "album"."id"="sharedLink"."albumId"
AND "album"."deletedAt" IS NULL
LEFT JOIN "shared_link__asset" "assets_sharedLink"
ON "assets_sharedLink"."sharedLinksId"="sharedLink"."id"
LEFT JOIN "assets" "assets"
ON "assets"."id"="assets_sharedLink"."assetsId"
AND "assets"."deletedAt" IS NULL
LEFT JOIN "albums_assets_assets" "album_albumAssets"
ON "album_albumAssets"."albumsId"="album"."id"
LEFT JOIN "assets" "albumAssets"
ON "albumAssets"."id"="album_albumAssets"."assetsId"
AND "albumAssets"."deletedAt" IS NULL
WHERE
"sharedLink"."id" = $1
AND (
"assets"."id" IN ($2, $3)
OR "albumAssets"."id" IN ($4, $5)
OR "assets"."livePhotoVideoId" IN ($6, $7)
OR "albumAssets"."livePhotoVideoId" IN ($8, $9)
)
```
* chore: rebase and clean-up
* feat: sync description, add e2e tests
* feat: simplify web code
* chore: unit tests
* fix: linting
* Bug fix with the arrows key
* timezone typeahead filter
timezone typeahead filter
* small stlying
* format fix
* Bug fix in the map selection
Bug fix in the map selection
* Websocket basic
Websocket basic
* Update metadata visualisation through the websocket
* Update timeline
* fix merge
* fix web
* fix web
* maplibre system
* format fix
* format fix
* refactor: clean up
* Fix small bug in the hour/timezone
* Don't diplay modify for readOnly asset
* Add log in case of failure
* Formater + try/catch error
* Remove everything related to websocket
* Revert "Remove everything related to websocket"
This reverts commit 14bcb9e1e4.
* remove notification
* fix test
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
Modify Access repository, to evaluate `authDevice`, `library`, `partner`,
`person`, and `timeline` permissions in bulk.
Queries have been validated to match what they currently generate for
single ids.
As an extra performance improvement, we now use a custom QueryBuilder
for the Partners queries, to avoid the eager relationships that add
unneeded `LEFT JOIN` clauses. We only filter based on the ids present in
the `partners` table, so those joins can be avoided.
Queries:
* `library` owner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "libraries" "LibraryEntity"
WHERE
"LibraryEntity"."id" = $1
AND "LibraryEntity"."ownerId" = $2
AND "LibraryEntity"."deletedAt" IS NULL
)
LIMIT 1
-- After
SELECT "LibraryEntity"."id" AS "LibraryEntity_id"
FROM "libraries" "LibraryEntity"
WHERE
"LibraryEntity"."id" IN ($1, $2)
AND "LibraryEntity"."ownerId" = $3
AND "LibraryEntity"."deletedAt" IS NULL
```
* `library` partner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "partners" "PartnerEntity"
LEFT JOIN "users" "PartnerEntity__sharedBy"
ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
LEFT JOIN "users" "PartnerEntity__sharedWith"
ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
WHERE
"PartnerEntity"."sharedWithId" = $1
AND "PartnerEntity"."sharedById" = $2
)
LIMIT 1
-- After
SELECT
"partner"."sharedById" AS "partner_sharedById",
"partner"."sharedWithId" AS "partner_sharedWithId"
FROM "partners" "partner"
WHERE
"partner"."sharedById" IN ($1, $2)
AND "partner"."sharedWithId" = $3
```
* `authDevice` owner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "user_token" "UserTokenEntity"
WHERE
"UserTokenEntity"."userId" = $1
AND "UserTokenEntity"."id" = $2
)
LIMIT 1
-- After
SELECT "UserTokenEntity"."id" AS "UserTokenEntity_id"
FROM "user_token" "UserTokenEntity"
WHERE
"UserTokenEntity"."userId" = $1
AND "UserTokenEntity"."id" IN ($2, $3)
```
* `timeline` partner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "partners" "PartnerEntity"
LEFT JOIN "users" "PartnerEntity__sharedBy"
ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
LEFT JOIN "users" "PartnerEntity__sharedWith"
ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
WHERE
"PartnerEntity"."sharedWithId" = $1
AND "PartnerEntity"."sharedById" = $2
)
LIMIT 1
-- After
SELECT
"partner"."sharedById" AS "partner_sharedById",
"partner"."sharedWithId" AS "partner_sharedWithId"
FROM "partners" "partner"
WHERE
"partner"."sharedById" IN ($1, $2)
AND "partner"."sharedWithId" = $3
```
* `person` owner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "person" "PersonEntity"
WHERE
"PersonEntity"."id" = $1
AND "PersonEntity"."ownerId" = $2
)
LIMIT 1
-- After
SELECT "PersonEntity"."id" AS "PersonEntity_id"
FROM "person" "PersonEntity"
WHERE
"PersonEntity"."id" IN ($1, $2)
AND "PersonEntity"."ownerId" = $3
```
* `partner` update access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "partners" "PartnerEntity"
LEFT JOIN "users" "PartnerEntity__sharedBy"
ON "PartnerEntity__sharedBy"."id"="PartnerEntity"."sharedById"
AND "PartnerEntity__sharedBy"."deletedAt" IS NULL
LEFT JOIN "users" "PartnerEntity__sharedWith"
ON "PartnerEntity__sharedWith"."id"="PartnerEntity"."sharedWithId"
AND "PartnerEntity__sharedWith"."deletedAt" IS NULL
WHERE
"PartnerEntity"."sharedWithId" = $1
AND "PartnerEntity"."sharedById" = $2
)
LIMIT 1
-- After
SELECT
"partner"."sharedById" AS "partner_sharedById",
"partner"."sharedWithId" AS "partner_sharedWithId"
FROM "partners" "partner"
WHERE
"partner"."sharedById" IN ($1, $2)
AND "partner"."sharedWithId" = $3
```
* chore(server): Check album permissions in bulk
Modify Access repository, to evaluate `album` permissions in bulk.
Queries have been validated to match what they currently generate for
single ids.
Queries:
* Owner access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "albums" "AlbumEntity"
WHERE
"AlbumEntity"."id" = $1
AND "AlbumEntity"."ownerId" = $2
AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1
-- After
SELECT
"AlbumEntity"."id" AS "AlbumEntity_id"
FROM "albums" "AlbumEntity"
WHERE
"AlbumEntity"."id" IN ($1, $2)
AND "AlbumEntity"."ownerId" = $3
AND "AlbumEntity"."deletedAt" IS NULL
```
* Shared link access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "shared_links" "SharedLinkEntity"
WHERE
"SharedLinkEntity"."id" = $1
AND "SharedLinkEntity"."albumId" = $2
)
LIMIT 1
-- After
SELECT
"SharedLinkEntity"."albumId" AS "SharedLinkEntity_albumId",
"SharedLinkEntity"."id" AS "SharedLinkEntity_id"
FROM "shared_links" "SharedLinkEntity"
WHERE
"SharedLinkEntity"."id" = $1
AND "SharedLinkEntity"."albumId" IN ($2, $3)
```
* Shared album access:
```sql
-- Before
SELECT 1 AS "row_exists" FROM (SELECT 1 AS dummy_column) "dummy_table" WHERE EXISTS (
SELECT 1
FROM "albums" "AlbumEntity"
LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
WHERE
"AlbumEntity"."id" = $1
AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $2
AND "AlbumEntity"."deletedAt" IS NULL
)
LIMIT 1
-- After
SELECT
"AlbumEntity"."id" AS "AlbumEntity_id"
FROM "albums" "AlbumEntity"
LEFT JOIN "albums_shared_users_users" "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."albumsId"="AlbumEntity"."id"
LEFT JOIN "users" "AlbumEntity__AlbumEntity_sharedUsers"
ON "AlbumEntity__AlbumEntity_sharedUsers"."id"="AlbumEntity_AlbumEntity__AlbumEntity_sharedUsers"."usersId"
AND "AlbumEntity__AlbumEntity_sharedUsers"."deletedAt" IS NULL
WHERE
"AlbumEntity"."id" IN ($1, $2)
AND "AlbumEntity__AlbumEntity_sharedUsers"."id" = $3
AND "AlbumEntity"."deletedAt" IS NULL
```
* chore(server): Add set utils, avoid double queries for same ids
* chore(server): Review feedback
