1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2024-12-29 15:11:58 +00:00
High performance self-hosted photo and video management solution.
Find a file
Alex a8220172f8
WIP refactor container and queuing system (#206)
* refactor microservices to machine-learning

* Update tGithub issue template with correct task syntax

* Added microservices container

* Communicate between service based on queue system

* added dependency

* Fixed problem with having to import BullQueue into the individual service

* Added todo

* refactor server into monorepo with microservices

* refactor database and entity to library

* added simple migration

* Move migrations and database config to library

* Migration works in library

* Cosmetic change in logging message

* added user dto

* Fixed issue with testing not able to find the shared library

* Clean up library mapping path

* Added webp generator to microservices

* Update Github Action build latest

* Fixed issue NPM cannot install due to conflict witl Bull Queue

* format project with prettier

* Modified docker-compose file

* Add GH Action for Staging build:

* Fixed GH action job name

* Modified GH Action to only build & push latest when pushing to main

* Added Test 2e2 Github Action

* Added Test 2e2 Github Action

* Implemented microservice to extract exif

* Added cronjob to scan and generate webp thumbnail  at midnight

* Refactor to ireduce hit time to database when running microservices

* Added error handling to asset services that handle read file from disk

* Added video transcoding queue to process one video at a time

* Fixed loading spinner on web while loading covering the info panel

* Add mechanism to show new release announcement to web and mobile app (#209)

* Added changelog page

* Fixed issues based on PR comments

* Fixed issue with video transcoding run on the server

* Change entry point content for backward combatibility when starting up server

* Added announcement box

* Added error handling to failed silently when the app version checking is not able to make the request to GITHUB

* Added new version announcement overlay

* Update message

* Added messages

* Added logic to check and show announcement

* Add method to handle saving new version

* Added button to dimiss the acknowledge message

* Up version for deployment to the app store
2022-06-11 16:12:06 -05:00
.github WIP refactor container and queuing system (#206) 2022-06-11 16:12:06 -05:00
design Update readme 2022-05-21 18:49:23 -05:00
docker WIP refactor container and queuing system (#206) 2022-06-11 16:12:06 -05:00
fastlane Added required setup for f-droid (#88) 2022-03-29 14:13:47 -05:00
machine-learning WIP refactor container and queuing system (#206) 2022-06-11 16:12:06 -05:00
mobile WIP refactor container and queuing system (#206) 2022-06-11 16:12:06 -05:00
server WIP refactor container and queuing system (#206) 2022-06-11 16:12:06 -05:00
web WIP refactor container and queuing system (#206) 2022-06-11 16:12:06 -05:00
.gitignore Implemented getting correct disk info for the mounted directory (#72) 2022-03-23 14:53:45 -05:00
LICENSE Update Readme 2022-02-11 22:35:14 -06:00
Makefile Fix user e2e tests (#194) 2022-06-06 11:16:03 -05:00
NOTES.md Add web interface with admin functionality (#167) 2022-05-21 02:23:55 -05:00
PR_CHECKLIST.md Deploy 1.8.0_12-dev (#132) 2022-04-29 13:10:42 -05:00
README.md Fix typos and grammar, plus change Linux to Unix (#180) 2022-05-23 11:58:00 -05:00

License: MIT Star on Github Android Build iOS Build Build Status



Immich

Self-hosted photo and video backup solution directly from your mobile phone.

Loading ~4000 images/videos

Screenshots

Mobile client

Web client

Note

!! NOT READY FOR PRODUCTION! DO NOT USE TO STORE YOUR ASSETS !!

This project is under heavy development, there will be continuous functions, features and api changes.

Features

  • Upload and view assets (videos/images).
  • Auto Backup.
  • Download asset to local device.
  • Multi-user supported.
  • Quick navigation with drag scroll bar.
  • Support HEIC/HEIF Backup.
  • Extract and display EXIF info.
  • Real-time render from multi-device upload event.
  • Image Tagging/Classification based on ImageNet dataset
  • Object detection based on COCO SSD.
  • Search assets based on tags and exif data (lens, make, model, orientation)
  • [Optional] Reverse geocoding using Mapbox (Generous free-tier of 100,000 search/month)
  • Show asset's location information on map (OpenStreetMap).
  • Show curated places on the search page
  • Show curated objects on the search page
  • Shared album with users on the same server
  • Selective backup - albums can be included and excluded during the backup process.
  • Web interface is available for administrative tasks (creating new users) and viewing assets on the server - additional features are coming.

System Requirement

OS: Preferred unix-based operating system (Ubuntu, Debian, MacOS...etc).

I haven't tested with Docker for Windows as well as WSL on Windows

Raspberry Pi can be used but microservices container has to be comment out in docker-compose since TensorFlow has not been supported in Docker image on arm64v7 yet.

RAM: At least 2GB, preffered 4GB.

Core: At least 2 cores, preffered 4 cores.

Getting Started

You can use docker compose for development and testing out the application, there are several services that compose Immich:

  1. NestJs - Backend of the application
  2. SvelteKit - Web frontend of the application
  3. PostgreSQL - Main database of the application
  4. Redis - For sharing websocket instance between docker instances and background tasks message queue.
  5. Nginx - Load balancing and optimized file uploading.
  6. TensorFlow - Object Detection and Image Classification.

Step 1: Populate .env file

Navigate to docker directory and run

cp .env.example .env

Then populate the value in there.

Notice that if set ENABLE_MAPBOX to true, you will have to provide MAPBOX_KEY for the server to run.

Pay attention to the key UPLOAD_LOCATION, this directory must exist and is owned by the user that run the docker-compose command below.

Example

###################################################################################
# Database
###################################################################################
DB_USERNAME=postgres
DB_PASSWORD=postgres
DB_DATABASE_NAME=immich

###################################################################################
# Upload File Config
###################################################################################
UPLOAD_LOCATION=<put-the-path-of-the-upload-folder-here>

###################################################################################
# JWT SECRET
###################################################################################
JWT_SECRET=randomstringthatissolongandpowerfulthatnoonecanguess

###################################################################################
# MAPBOX
####################################################################################
# ENABLE_MAPBOX is either true of false -> if true, you have to provide MAPBOX_KEY
ENABLE_MAPBOX=false
MAPBOX_KEY=

###################################################################################
# WEB
###################################################################################
# This is the URL of your vm/server where you host Immich, so that the web frontend
# know where can it make the request to.
# For example: If your server IP address is 10.1.11.50, the environment variable will
# be VITE_SERVER_ENDPOINT=http://10.1.11.50:2283
VITE_SERVER_ENDPOINT=http://192.168.1.216:2283

Step 2: Start the server

To start, run

docker-compose -f ./docker/docker-compose.yml up 

If you have a few thousand photos/videos, I suggest running docker-compose with scaling option for the immich_server container to handle high I/O load when using fast scrolling.

docker-compose -f ./docker/docker-compose.yml up --scale immich-server=5 

To update docker-compose with newest image (if you have started the docker-compose previously)

docker-compose -f ./docker/docker-compose.yml pull && docker-compose -f ./docker/docker-compose.yml up

The server will be running at http://your-ip:2283 through Nginx

Step 3: Register User

Access the web interface at http://your-ip:2285 to register an admin account.

Additional accounts on the server can be created by the admin account.

Step 4: Run mobile app

The app is distributed on several platforms below.

F-Droid

You can get the app on F-droid by clicking the image below.

Get it on F-Droid

Android

Get the app on Google Play Store here

The App version might be lagging behind the latest release due to the review process.

iOS

Get the app on Apple AppStore here:

The App version might be lagging behind the latest release due to the review process.

Development

The development environment can be started from the root of the project after populating the .env file with the command:

make dev # required Makefile installed on the system.

All servers and web container are hot reload for quick feedback loop.

Support

If you like the app, find it helpful, and want to support me to offset the cost of publishing to AppStores, you can sponsor the project with Github Sponsor, or a one time donation with the Buy Me a coffee link below.

"Buy Me A Coffee"

This is also a meaningful way to give me motivation and encouragement to continue working on the app.

Cheers! 🎉

Known Issue

TensorFlow Build Issue

This is a known issue on RaspberryPi 4 arm64-v7 and incorrect Promox setup

TensorFlow doesn't run with older CPU architecture, it requires a CPU with AVX and AVX2 instruction set. If you encounter the error illegal instruction core dump when running the docker-compose command above, check for your CPU flags with the command and make sure you see AVX and AVX2:

more /proc/cpuinfo | grep flags

If you are running virtualization in Promox, the VM doesn't have the flag enabled.

You need to change the CPU type from kvm64 to host under VMs hardware tab.

Hardware > Processors > Edit > Advanced > Type (dropdown menu) > host

Otherwise you can:

  • edit docker-compose.yml file and comment the whole immich_microservices service which will disable machine learning features like object detection and image classification
  • switch to a different VM/desktop with different architecture.