Commit graph

15 commits

Author SHA1 Message Date
Vítor Vasconcellos b261164ca6
Fix pnpm prep on Linux (#1904)
* Fix pnpm-prep on linux

* update archive-wasm to 1.6.1
2023-12-20 09:12:49 +00:00
Ericson "Fogo" Soares 7c90bcb95b
[ENG-1479] AI Prototype (#1845)
* First draft on image labeling

* Fixing execution providers for other OSs

* Better error handling and shutdown

* Working with shallow media processor

* bruh

* Fix warnings

* Now hooked to media processor job

* Link desktop app with libonnxruntime to avoid TLS error during startup

* Be able to change models on runtime
Revert to use labels table instead of tags

* A bug on a model-less inference

* Show AI labels on Inspector
 - Change yolo inference to use half precision
 - Add labels api to core

* Remove LD_PRELOAD

* Fix race condition on model executor shutdown

* Don't load all images in memory moron

* Embeed yolo model in prod build
 - Change yolo model path to new one relative to executable

* Disable volume watcher on linux, it was crashing the app
 - Invalidate labels when they are updated

* Rust fmt

* Minor changes

* Gate onnxruntime linking to the ai-models feature

* Add build script to sd-server to handle onnxruntime linking workaround

* Move AI stuff to its own crate and normalize deps

* Rust fmt

* Don't regenerate labels unless asked to

* Now blazingly fast

* Bad merge

* Fix

* Fix

* Add backend logic to download extra yolo models

* Add models api route
 - Add api call to get available model version
 - Add api call to change the model version

* Improve new model download logic
 - Add frontend to change image labeler model

* Fix new model downloader

* Fix model select width

* invalidate labels count after media_processor generates a new output

* Rename AI crate and first draft on download notifications

* fix types

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-12-19 09:28:57 +00:00
Vítor Vasconcellos f3ad0d7513
Pin rust to 1.73 (Workaround Internal Compiler Error) (#1796)
* Pin rust to 1.73
- Clippy fixes

* Don't install cargo-watch on CI
2023-11-17 22:47:06 +00:00
Vítor Vasconcellos 2e6e00bc6d
[ENG-1184, ENG-1286, ENG-1330] Rework native dependencies (+ deb fixes) (#1685)
* Almost working

* Downgrade libplacebo
 - FFMpeg 6.0 uses some now removed deprecated functions

* Use -Oz for zimg

* Fix CI script to run the new ffmpeg build script

* Fix heif step name + Ignore docker cache while building in CI

* Fix Opencl build on linux

* Fix adding incorrect -target argument to linker
 - Update zig for windows target

* Disable opengl for ffmpeg, it only uses it as an outdev, not for processing
 - Disable opengl and directx for libplacebo, ffmpeg only supports vulkan when using it
 - Add WIN32_LEAN_AND_MEAN to global cflags to optimize windows api usage
 - Fix 99-heif.sh incorrect bsdtar flag

* Remove WIN32_LEAN_AND_MEAN from global CFLAGS as that was breaking OpenCL build
 - Fix Dockerfile step for cleaning up the out dir
 - Improve licensing handling

* x86_64 windows and linux builds are working

* Fix aarch64 build for windows and linux

* Fix symbol visibility in linux builds
 - Fix soxr failing to download due to sourcefourge
 - Only patch zimg on windows targets
 - Tell cmake to hide libheif symbols

* Fix Linux .so rpath
 - Add lzo dependency
 - Publish source for the built libs
 - Add warning for missing nasm in tauri.mjs
 - Remove ffmpeg install from setup.sh
 - Add download logic for our linux ffmpeg bundle in preprep.mjs

* Remove jobs, docker doesn't support this

* Fix typing

* Change ffmpeg references to native deps
 - Rename FFMpeg.framework to Spacedrive.framework
 - Centralize the macOS native deps build with the windows and linux one
 - Change the preprep script to only download our native deps
 - Remove old macOS ffmpeg build scripts

* Compress native deps before creating github artifact
 - The zip implementation for github artifact does not mantain symlinks and permissions
 - Remove conditional protoc, it is now always included

* Don't strip dylibs, it was breaking them
 - Only download macOS Framework for darwin targets
 - Fix preprep script
 - Improve README.md for native-deps
 - Fix not finding native-deps src

* Attempt to fix macOS dylib

* Fix macOS dylibs
 - Replace lld.ld64 with apple's own linker
 - Add stages for building apple's compiler tools to use instead of LLVM ones

* Ensure sourced file exists

* All targets should build now
 - Fix environment sourcing in build.sh
 - Some minor improvements to cc.sh
 - Fix incorrect flag in zlib.sh
 - Improve how -f[...] flags are passed to compiler and linker
 - Add more stack hardening flags

* We now can support macOS 11.0 on arm64

* Improve macOS Framework generation
 - Remove installed unused deps
 - Improve cleanup and organization logic in Dockerfile last step
 - Move libav* .dll.a to .lib to fix missing files in windows target
 - Remove apple tools from /srv folder after installation to prevent their files from being copied by other stage steps
 - Create all the necessary symlinks for the macOS targets while building
 - Remove symlink logic for macOS target from preprep.mjs

* Remove native-deps from spacedrive repo
 - It now resides in https://github.com/spacedriveapp/native-deps
 - Modify preprep script to dowload native-deps from new location
 - Remove Github API code from scripts (not needed anymore)
 - Add flock.mjs to allow running tauri.mjs cleanup as soon as cargo finishes building in linux

* Handle flock not present in system
 - Allow macOS to try using flock

* Fix preprep on macOS

* Add script that patch deb to fix errors and warnings raised by lintian

* Fix ctrl+c/ctrl+v typo

* Remove gstreamer1.0-gtk3 from deb dependencies

* eval is evil

* Handle tauri build release with an explicit target in fix-deb.sh

* Preserve environment variables when re-executing fix-deb with sudo

* Only execute fix-deb.sh when building a deb bundle

* Improvements fix-deb.sh

* Improve setup.sh (Add experiemental alpine support)
2023-11-17 19:20:14 +00:00
Ericson "Fogo" Soares 614ca0d811
Remove < 21 node restriction (#1728) 2023-11-04 03:44:07 +00:00
Vítor Vasconcellos 3d3f333b64
Fix Windows CI (#1695)
* Add Nasm to setup.ps1
 - Nasm is now required to compile the `openssl-sys` crate

* Run Clippy CI when setup or prep scripts change

* Ensure nasm is also installed in Windows CI
 - Add nasm to setup.sh
 - Remove openssl-dev from setup.sh

* Ensure nasm is installed in macOS too

* Dumb ctrl+c/ctrl+v mistake

* 's/==/=/'
2023-10-29 00:25:26 +00:00
Julian Braha cdd635a0ba
Add installation of libvips for arch systems to setup script (#1688)
Add libvips package for arch systems to setup script
2023-10-27 20:31:30 +00:00
Vítor Vasconcellos f7277d602f
Frontend misc fixes and deps update (#1650)
Dependencies overhaul
 - Update dependencies for all projects (except Mobile-only deps)
 - Remove unused dependencies from all projects (except Mobile-only deps)
 - Fix Storybook failing to import sd/ui style
 - Add Node 21 as not supported due to sass-loader not working on it yet
 - Add work-around for new rook version requiring webpack specific global property
 - Fix landing dev not working due to missing default env value on dev
 - Fix some incorrect uses of phosphor-icons non server side icons on server components on landing
 - Fix some incorrect uses of phosphor-icons server side icon on client components on landing
 - Fix landing fail to build on dev due to always required a Github Token to get the latest release
 - Fix new Next.js version not suporting Response.redirect due to immutable Headers
 - Add Gitlab as social link for teams page
 - Update Vítor's team photo
 - Add Vítor's twitter link
 - Fix some warning due to missing useEffect dependencies
 - Remove test-files dir
 - Fix QuickPreview unblurred buttons in Linux
 - Formatting
2023-10-24 07:51:58 +00:00
devqore 5b09c615c9
Improve error message with rustup not found (#1659)
* Improve error message with rustup not found

* Improve error message with rustup not found on the mobile devices

* Fix `err` function in setup.sh

Fix `err` function printing repeated error lines

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-10-22 15:42:37 +00:00
Gedeon d52b4f9f22
really small typo fix (#1631) 2023-10-19 18:32:32 +00:00
Vítor Vasconcellos 3b686946e6
Tauri 1.5.2 + normalize and update frontend dev deps (#1579)
* Normalize & update front-end dev-deps + Tauri 1.5.2

* format code
2023-10-14 04:27:09 +00:00
Brendan Allan 49cc098f32
[ENG-927, ENG-735, ENG-766] Fix Updater & Tauri 1.5 (#1361)
* custom updater with toasts

* new state management + updated router route

* tauri-specific update route

* ref

* update in prod only

* change 'Install' to 'Update'

* fix tsconfig

* desktop tauri

* remove tauri patch

* tauri 1.5

* tauri 1.5

* use tauri script

* native-deps

* Rework preprep and tauri script to better support tauri 1.5

* Update to tauri 1.5.1
 - Update workspace and apps/desktop dependencies
 - Fix mustache import, @types/mustache is not compatible with ES imports
 - Replace arm64 with aarch64 in machineID, they should be treated the same and this simplyfies the code

* Fix tauri updater not building due to missing key
 - Fix dmg background not being found
 - Generate an adhoc key for tauri updater with it is enabled and the user is doing a prod build

* Fix ctrl+c/ctrl+v typo

* Normalie @tanstack/react-query version through workspace
 - Use undici in scripts instead of global fetch
 - Fix typecheck

* Fix linux prod and dev builds
 - Improve error handling in tauri.mjs

* Normalize dev deps in workspace
 - Improve linux shared libs setup

* Fix CI and server docker

* Fix windows
 - Remove superfluous envvar

* Attempt to fix server, mobile, deb and release updater

* Attempt to fix deb and mobile again
 - Fix type on deb dependency
 - Enable release deb for aarch64-unknown-linux-gnu

* Github doesn't have arm runners
 - Fix typo in server Dockerfile

* Publish deb and updater artifacts

* remove version from asset name

* update commands

* log release

* Some logs on updater errors

* show updater errors on frontend

* fix desktop ui caching

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
2023-10-10 07:30:56 +00:00
Vítor Vasconcellos d75a6e9b12
[ENG-1054] libheif for Linux (plus fixes) + Webp update (#1405)
* Move postinstall script to a preprep script
 - Fix libheif crate failing to build with our libheif
 - Rework CI due to postinstall to preprep changes

* Linux heif build script + Update webp

* Fix ctrl+c/ctrl+v bug

* Improve libheif linux script
 - Add support for linux aarch64
 - Add CI workflow to build libheif for linux
 - Some other misc fixes

* Fix libheif CI requires sudo

* Fix wrong path for libheif build.rs override in Windows

* Fix wrong path manipulations in libheif build script

* 🤦

* Use ubuntu-latest in libheif action
 - Specify glib version in target triple to support old distros
 - Fix libheif artifact publishing

* Fix artifact upload path again

* Add musl support for libheif
 - Remove unused files from libheif artifact
 - Add setup logic for libheif in postinstall script

* Build libheif for linux as a shared lib

* Fix meson not building the correct arch
 - Add logic to get git branch from githubs CI env vars

* libheif finnaly works on linux
 - Make spacedrive binary rpath point to where appimage and deb expects our libs to be
 - Add some logic to tauri.js to convince tauri to bundle our shared libs
 - Work-around appimage bundling step breaking sometimes
 - Add logic to handle sigint in tauri.js to ensure we clean up after ourselves
 - Rename postinstall.mjs to setup.mjs
 - Add logic to setup.mjs to point our dev build to our shared libs in linux

* Fix windows dekstop dev
 - Rename setup.mjs to preprep.mjs

* test cache-factory

* Fix preprep script not parsing the cross compilation target triple and always using the host info to download dependencies
 - Fix action env vars not being correctly passed
 - Remove possibility to pass multiple targests to rust action

* Don't compile mobile crates on desktop targets

* Remove cache-factory pull_request trigger

* remove patched tauri cli

* Use git plumbing command to get remote branch name
 - Fallback to reading .git/HEAD if remote name was not retrieved

* fix type

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-10-03 17:31:33 +00:00
Vítor Vasconcellos f71711d09a
Change docs and CI scripts to use pnpm tauri command (#1421)
* Change docs and CI scripts to use `pnpm tauri` command
 - Fix documentation and some scripts to use the new tauri subcommand following https://github.com/spacedriveapp/spacedrive/pull/1411 changes
 - Format/Lint some files

* Document new envvar to lauch devtools alongside the desktop app
2023-10-03 08:10:55 +00:00
Vítor Vasconcellos bd0a7ff434
[ENG-265] Improve setup scripts (#1368)
* Setup script revamp

* Move toml to dev dep + CI runs the postinstall in debug mode

* Fix windows CI

* chmod +x scripts/setup.sh

* Fix eslint and ts CI

* Remove binstall + Fix trying to read input in CI

* Doesn't need to check pnpm and rust in CI

* Run postinstall script for Clippy CI

* Attempt to fix windows CI not running postinstall
 - Ignore cache when running postinstall on CI

* commited generated config.toml by mistake

* Pass GITHUB_TOKEN to `pnpm i`

* Update archive-wasm + Increase minimum node version to 18.17

* CI: Move rust setup after post-install script

* Revert: CI: Move rust setup after post-install script

* Fix CI, generate dummy cargo config.toml to fix prisma generation

* Fix windows CI

* CI: Fix wrong command

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-09-28 10:03:46 +00:00