* duplicate detection job, entity, config
* queueing
* job panel, update api
* use embedding in db instead of fetching
* disable concurrency
* only queue visible assets
* handle multiple duplicateIds
* update concurrent queue check
* add provider
* add web placeholder, server endpoint, migration, various fixes
* update sql
* select embedding by default
* rename variable
* simplify
* remove separate entity, handle re-running with different threshold, set default back to 0.02
* fix tests
* add tests
* add index to entity
* formatting
* update asset mock
* fix `upsertJobStatus` signature
* update sql
* formatting
* default to 0.03
* optimize clustering
* use asset's `duplicateId` if present
* update sql
* update tests
* expose admin setting
* refactor
* formatting
* skip if ml is disabled
* debug trash e2e
* remove from web
* remove from sidebar
* test if ml is disabled
* update sql
* separate duplicate detection from clip in config, disable by default for now
* fix doc
* lower minimum `maxDistance`
* update api
* Add and Use Duplicate Detection Feature Flag (#9364)
* Add Duplicate Detection Flag
* Use Duplicate Detection Flag
* Attempt Fixes for Failing Checks
* lower minimum `maxDistance`
* fix tests
---------
Co-authored-by: mertalev <101130780+mertalev@users.noreply.github.com>
* chore: fixes and additions after rebase
* chore: update api (remove new Role enum)
* fix: left join smart search so getAll works without machine learning
* test: trash e2e go back to checking length of assets is zero
* chore: regen api after rebase
* test: fix tests after rebase
* redundant join
---------
Co-authored-by: Nicholas Flamy <30300649+NicholasFlamy@users.noreply.github.com>
Co-authored-by: Zack Pollard <zackpollard@ymail.com>
Co-authored-by: Zack Pollard <zack@futo.org>
* feat(server, web): include pictures of shared albums on map
* run prettier
* re-create api clients
* implement suggestions from code review
* shared from partner -> shared from partners
* rename to 'include shared partner assets'
* chore: fix tsc error in server and prettier in web
* fix: include assets shared via owner albums
---------
Co-authored-by: Zack Pollard <zackpollard@ymail.com>
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* refactor: remove isReadOnly and isExternal usages
* chore: open api
* fix: linting
* remove mobile isReadOnly dependency
---------
Co-authored-by: Alex Tran <alex.tran1502@gmail.com>
* ignore non external assets in external libraries during syncUsage
* only update storage usage if asset is from internal libraries
* update storage usage on motion photo video asset creation
* updated metadata service tests
* added a test
* simplified syncUsage condition
* check for library type upload instead of not external
* fixed broken sql
---------
Co-authored-by: Jason Rasmussen <jrasm91@gmail.com>
* added "isExternal" to the getLibraryAssetPaths query
* handleQueueAssetRefresh skip "non external" video asset, closes#8562
* correctly implements live photo deletion for external library
* use "external asset" for external library tests
* minor: external library asset checksum is "path hash" not file hash
* renamed to getExternalLibraryAssetPaths and added isExternal where clause
* generated sql
* reverted leftover change
* delete thumbnail and other generated files even for readonly asset
* updated test
* don't delete sidecar file for readonly file
* fixed test
* improved external detection
