1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2025-01-10 13:56:47 +01:00
immich/server/src/services/database.service.ts

159 lines
6.8 KiB
TypeScript
Raw Normal View History

import { Inject, Injectable } from '@nestjs/common';
import {
DatabaseExtension,
DatabaseLock,
IDatabaseRepository,
VectorExtension,
VectorIndex,
extName,
} from 'src/interfaces/database.interface';
2024-03-21 04:15:09 +01:00
import { ImmichLogger } from 'src/utils/logger';
import { Version, VersionType } from 'src/utils/version';
@Injectable()
export class DatabaseService {
private logger = new ImmichLogger(DatabaseService.name);
private vectorExt: VectorExtension;
feat(server): add postgres major version check (#6213) * feat(server): Throw error when PostgreSQL version is not within the supported versions The pgvecto.rs extension, though not distributed, can be built for PostgreSQL 12 and 13. An installation of PostgreSQL 12 with the pgvecto.rs extensions installed will not be caught by immich. This causes immich to attempt to run the database migrations without having a proper environment. With assertPostgresql the server will throw an error if the PostgreSQL version is not within the supported range. * Replaced assertion with lesser than comparison As requested by @zackpollard * Changed the comparison to use the minPostgresVersion variable. If we define one we might as well use it. makes changing the versioning later easier * Added two new tests, modified two existing tests `should return if minimum supported PostgreSQL and vectors version are installed`: Check if init returns properly and that getPostgresVersion is called twice `should thrown an error if PostgreSQL version is below minimum supported version`: Checks if the init function correctly returns an error `should suggest image with postgres ${major} if database is ${major}`: Modified to set MockResolvedValue instead of MockResolvedValueOnce. With the new check we get the PostgreSQL version twice. So it needs to be set during the entire test. `should not suggest image if postgres version is not in 14, 15 or 16`: Modified the bounds to [14, 18]. Because values below 14 now will not get called. Also Modified to call `getPostgresVersion.MockResolvedValueOnce` for twice, because it gets called twice. * Fixed two mistakes in the jest functions from previous commit #2abcb60 `should thrown an error if PostgreSQL version is below minimum supported version`: The regex function I wrote mistakingly used the negate function which check that the error *did not* contain the phrase "PostgreSQL". Which is the opposite `should not suggest image if postgres version is not in 14, 15 or 16`: confused bounds for a normal javascript array. Changed the test to only check for values above 16. As values below 14 will get thrown out by test `should return if minimum supported PostgreSQL and vectors version are installed` I apologise for the mistakes in my previous commit. * Format fix --------- Co-authored-by: max <wak@vanling.net>
2024-01-07 01:24:09 +01:00
minPostgresVersion = 14;
minVectorsVersion = new Version(0, 2, 0);
vectorsVersionPin = VersionType.MINOR;
minVectorVersion = new Version(0, 5, 0);
vectorVersionPin = VersionType.MAJOR;
constructor(@Inject(IDatabaseRepository) private databaseRepository: IDatabaseRepository) {
this.vectorExt = this.databaseRepository.getPreferredVectorExtension();
}
async init() {
feat(server): add postgres major version check (#6213) * feat(server): Throw error when PostgreSQL version is not within the supported versions The pgvecto.rs extension, though not distributed, can be built for PostgreSQL 12 and 13. An installation of PostgreSQL 12 with the pgvecto.rs extensions installed will not be caught by immich. This causes immich to attempt to run the database migrations without having a proper environment. With assertPostgresql the server will throw an error if the PostgreSQL version is not within the supported range. * Replaced assertion with lesser than comparison As requested by @zackpollard * Changed the comparison to use the minPostgresVersion variable. If we define one we might as well use it. makes changing the versioning later easier * Added two new tests, modified two existing tests `should return if minimum supported PostgreSQL and vectors version are installed`: Check if init returns properly and that getPostgresVersion is called twice `should thrown an error if PostgreSQL version is below minimum supported version`: Checks if the init function correctly returns an error `should suggest image with postgres ${major} if database is ${major}`: Modified to set MockResolvedValue instead of MockResolvedValueOnce. With the new check we get the PostgreSQL version twice. So it needs to be set during the entire test. `should not suggest image if postgres version is not in 14, 15 or 16`: Modified the bounds to [14, 18]. Because values below 14 now will not get called. Also Modified to call `getPostgresVersion.MockResolvedValueOnce` for twice, because it gets called twice. * Fixed two mistakes in the jest functions from previous commit #2abcb60 `should thrown an error if PostgreSQL version is below minimum supported version`: The regex function I wrote mistakingly used the negate function which check that the error *did not* contain the phrase "PostgreSQL". Which is the opposite `should not suggest image if postgres version is not in 14, 15 or 16`: confused bounds for a normal javascript array. Changed the test to only check for values above 16. As values below 14 will get thrown out by test `should return if minimum supported PostgreSQL and vectors version are installed` I apologise for the mistakes in my previous commit. * Format fix --------- Co-authored-by: max <wak@vanling.net>
2024-01-07 01:24:09 +01:00
await this.assertPostgresql();
await this.databaseRepository.withLock(DatabaseLock.Migrations, async () => {
await this.createVectorExtension();
await this.updateVectorExtension();
await this.assertVectorExtension();
try {
if (await this.databaseRepository.shouldReindex(VectorIndex.CLIP)) {
await this.databaseRepository.reindex(VectorIndex.CLIP);
}
if (await this.databaseRepository.shouldReindex(VectorIndex.FACE)) {
await this.databaseRepository.reindex(VectorIndex.FACE);
}
} catch (error) {
this.logger.warn(
'Could not run vector reindexing checks. If the extension was updated, please restart the Postgres instance.',
);
throw error;
}
await this.databaseRepository.runMigrations();
});
}
private async assertPostgresql() {
const { major } = await this.databaseRepository.getPostgresVersion();
if (major < this.minPostgresVersion) {
throw new Error(`
The PostgreSQL version is ${major}, which is older than the minimum supported version ${this.minPostgresVersion}.
Please upgrade to this version or later.`);
}
}
private async createVectorExtension() {
try {
await this.databaseRepository.createExtension(this.vectorExt);
} catch (error) {
const otherExt =
this.vectorExt === DatabaseExtension.VECTORS ? DatabaseExtension.VECTOR : DatabaseExtension.VECTORS;
this.logger.fatal(`
Failed to activate ${extName[this.vectorExt]} extension.
Please ensure the Postgres instance has ${extName[this.vectorExt]} installed.
If the Postgres instance already has ${extName[this.vectorExt]} installed, Immich may not have the necessary permissions to activate it.
In this case, please run 'CREATE EXTENSION IF NOT EXISTS ${this.vectorExt}' manually as a superuser.
See https://immich.app/docs/guides/database-queries for how to query the database.
Alternatively, if your Postgres instance has ${extName[otherExt]}, you may use this instead by setting the environment variable 'DB_VECTOR_EXTENSION=${extName[otherExt]}'.
Note that switching between the two extensions after a successful startup is not supported.
The exception is if your version of Immich prior to upgrading was 1.90.2 or earlier.
In this case, you may set either extension now, but you will not be able to switch to the other extension following a successful startup.
`);
throw error;
}
}
private async updateVectorExtension() {
const [version, availableVersion] = await Promise.all([
this.databaseRepository.getExtensionVersion(this.vectorExt),
this.databaseRepository.getAvailableExtensionVersion(this.vectorExt),
]);
if (version == null) {
throw new Error(`Unexpected: ${extName[this.vectorExt]} extension is not installed.`);
}
if (availableVersion == null) {
return;
}
const maxVersion = this.vectorExt === DatabaseExtension.VECTOR ? this.vectorVersionPin : this.vectorsVersionPin;
const isNewer = availableVersion.isNewerThan(version);
if (isNewer == null || isNewer > maxVersion) {
return;
}
try {
this.logger.log(`Updating ${extName[this.vectorExt]} extension to ${availableVersion}`);
const { restartRequired } = await this.databaseRepository.updateVectorExtension(this.vectorExt, availableVersion);
if (restartRequired) {
this.logger.warn(`
The ${extName[this.vectorExt]} extension has been updated to ${availableVersion}.
Please restart the Postgres instance to complete the update.`);
}
} catch (error) {
this.logger.warn(`
The ${extName[this.vectorExt]} extension version is ${version}, but ${availableVersion} is available.
Immich attempted to update the extension, but failed to do so.
This may be because Immich does not have the necessary permissions to update the extension.
Please run 'ALTER EXTENSION ${this.vectorExt} UPDATE' manually as a superuser.
See https://immich.app/docs/guides/database-queries for how to query the database.`);
this.logger.error(error);
}
}
feat(server): add postgres major version check (#6213) * feat(server): Throw error when PostgreSQL version is not within the supported versions The pgvecto.rs extension, though not distributed, can be built for PostgreSQL 12 and 13. An installation of PostgreSQL 12 with the pgvecto.rs extensions installed will not be caught by immich. This causes immich to attempt to run the database migrations without having a proper environment. With assertPostgresql the server will throw an error if the PostgreSQL version is not within the supported range. * Replaced assertion with lesser than comparison As requested by @zackpollard * Changed the comparison to use the minPostgresVersion variable. If we define one we might as well use it. makes changing the versioning later easier * Added two new tests, modified two existing tests `should return if minimum supported PostgreSQL and vectors version are installed`: Check if init returns properly and that getPostgresVersion is called twice `should thrown an error if PostgreSQL version is below minimum supported version`: Checks if the init function correctly returns an error `should suggest image with postgres ${major} if database is ${major}`: Modified to set MockResolvedValue instead of MockResolvedValueOnce. With the new check we get the PostgreSQL version twice. So it needs to be set during the entire test. `should not suggest image if postgres version is not in 14, 15 or 16`: Modified the bounds to [14, 18]. Because values below 14 now will not get called. Also Modified to call `getPostgresVersion.MockResolvedValueOnce` for twice, because it gets called twice. * Fixed two mistakes in the jest functions from previous commit #2abcb60 `should thrown an error if PostgreSQL version is below minimum supported version`: The regex function I wrote mistakingly used the negate function which check that the error *did not* contain the phrase "PostgreSQL". Which is the opposite `should not suggest image if postgres version is not in 14, 15 or 16`: confused bounds for a normal javascript array. Changed the test to only check for values above 16. As values below 14 will get thrown out by test `should return if minimum supported PostgreSQL and vectors version are installed` I apologise for the mistakes in my previous commit. * Format fix --------- Co-authored-by: max <wak@vanling.net>
2024-01-07 01:24:09 +01:00
private async assertVectorExtension() {
const version = await this.databaseRepository.getExtensionVersion(this.vectorExt);
if (version == null) {
throw new Error(`Unexpected: The ${extName[this.vectorExt]} extension is not installed.`);
}
if (version.isEqual(new Version(0, 0, 0))) {
feat(server): add postgres major version check (#6213) * feat(server): Throw error when PostgreSQL version is not within the supported versions The pgvecto.rs extension, though not distributed, can be built for PostgreSQL 12 and 13. An installation of PostgreSQL 12 with the pgvecto.rs extensions installed will not be caught by immich. This causes immich to attempt to run the database migrations without having a proper environment. With assertPostgresql the server will throw an error if the PostgreSQL version is not within the supported range. * Replaced assertion with lesser than comparison As requested by @zackpollard * Changed the comparison to use the minPostgresVersion variable. If we define one we might as well use it. makes changing the versioning later easier * Added two new tests, modified two existing tests `should return if minimum supported PostgreSQL and vectors version are installed`: Check if init returns properly and that getPostgresVersion is called twice `should thrown an error if PostgreSQL version is below minimum supported version`: Checks if the init function correctly returns an error `should suggest image with postgres ${major} if database is ${major}`: Modified to set MockResolvedValue instead of MockResolvedValueOnce. With the new check we get the PostgreSQL version twice. So it needs to be set during the entire test. `should not suggest image if postgres version is not in 14, 15 or 16`: Modified the bounds to [14, 18]. Because values below 14 now will not get called. Also Modified to call `getPostgresVersion.MockResolvedValueOnce` for twice, because it gets called twice. * Fixed two mistakes in the jest functions from previous commit #2abcb60 `should thrown an error if PostgreSQL version is below minimum supported version`: The regex function I wrote mistakingly used the negate function which check that the error *did not* contain the phrase "PostgreSQL". Which is the opposite `should not suggest image if postgres version is not in 14, 15 or 16`: confused bounds for a normal javascript array. Changed the test to only check for values above 16. As values below 14 will get thrown out by test `should return if minimum supported PostgreSQL and vectors version are installed` I apologise for the mistakes in my previous commit. * Format fix --------- Co-authored-by: max <wak@vanling.net>
2024-01-07 01:24:09 +01:00
throw new Error(`
The ${extName[this.vectorExt]} extension version is ${version}, which means it is a nightly release.
Please run 'DROP EXTENSION IF EXISTS ${this.vectorExt}' and switch to a release version.
See https://immich.app/docs/guides/database-queries for how to query the database.`);
}
const minVersion = this.vectorExt === DatabaseExtension.VECTOR ? this.minVectorVersion : this.minVectorsVersion;
const maxVersion = this.vectorExt === DatabaseExtension.VECTOR ? this.vectorVersionPin : this.vectorsVersionPin;
if (version.isOlderThan(minVersion) || version.isNewerThan(minVersion) > maxVersion) {
const allowedReleaseType = maxVersion === VersionType.MAJOR ? '' : ` ${VersionType[maxVersion].toLowerCase()}`;
const releases =
maxVersion === VersionType.EQUAL
? minVersion.toString()
: `${minVersion} and later${allowedReleaseType} releases`;
throw new Error(`
The ${extName[this.vectorExt]} extension version is ${version}, but Immich only supports ${releases}.
If the Postgres instance already has a compatible version installed, Immich may not have the necessary permissions to activate it.
In this case, please run 'ALTER EXTENSION UPDATE ${this.vectorExt}' manually as a superuser.
See https://immich.app/docs/guides/database-queries for how to query the database.
Otherwise, please update the version of ${extName[this.vectorExt]} in the Postgres instance to a compatible version.`);
feat(server): add postgres major version check (#6213) * feat(server): Throw error when PostgreSQL version is not within the supported versions The pgvecto.rs extension, though not distributed, can be built for PostgreSQL 12 and 13. An installation of PostgreSQL 12 with the pgvecto.rs extensions installed will not be caught by immich. This causes immich to attempt to run the database migrations without having a proper environment. With assertPostgresql the server will throw an error if the PostgreSQL version is not within the supported range. * Replaced assertion with lesser than comparison As requested by @zackpollard * Changed the comparison to use the minPostgresVersion variable. If we define one we might as well use it. makes changing the versioning later easier * Added two new tests, modified two existing tests `should return if minimum supported PostgreSQL and vectors version are installed`: Check if init returns properly and that getPostgresVersion is called twice `should thrown an error if PostgreSQL version is below minimum supported version`: Checks if the init function correctly returns an error `should suggest image with postgres ${major} if database is ${major}`: Modified to set MockResolvedValue instead of MockResolvedValueOnce. With the new check we get the PostgreSQL version twice. So it needs to be set during the entire test. `should not suggest image if postgres version is not in 14, 15 or 16`: Modified the bounds to [14, 18]. Because values below 14 now will not get called. Also Modified to call `getPostgresVersion.MockResolvedValueOnce` for twice, because it gets called twice. * Fixed two mistakes in the jest functions from previous commit #2abcb60 `should thrown an error if PostgreSQL version is below minimum supported version`: The regex function I wrote mistakingly used the negate function which check that the error *did not* contain the phrase "PostgreSQL". Which is the opposite `should not suggest image if postgres version is not in 14, 15 or 16`: confused bounds for a normal javascript array. Changed the test to only check for values above 16. As values below 14 will get thrown out by test `should return if minimum supported PostgreSQL and vectors version are installed` I apologise for the mistakes in my previous commit. * Format fix --------- Co-authored-by: max <wak@vanling.net>
2024-01-07 01:24:09 +01:00
}
}
}