Commit graph

85 commits

Author SHA1 Message Date
Vítor Vasconcellos b4159d89f8
Package & distribute frontend bundle when doing releases for Linux (#2500)
* Package & distribute frontend bundle when doing releases for Linux

* Only check for tar.xz frontend bundle

* debug

* Fix sd-desktop
2024-06-04 23:19:41 +00:00
Consoli f4a79a2fcd
Faster debug builds (#2483)
* Config dev profile for faster debug builds

* Remove redundant changes to Cargo.toml in CI

* Forgot Windows

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2024-05-14 05:59:52 +00:00
Vítor Vasconcellos e797b02e65
Media metadata extraction & Thumbnailer rework (#2285)
* initial ffprobe commit

* Working slim down version ffprobe

* Auto format ffprobe and deps source

* Remove show_pixel_formats logic
- Fix do_bitexact incorrect check in main after last changes
- Fix some clangd warning

* Remove show_* and print_format options and their respective logic

* Rework ffprobe into simple_ffprobe
- Simplify ffprobe logic into a simple program that gather and print a media file metadata

* Reduce the amount of ffmpeg log messages while generating thumbnails

* Fix completly wrong comments

* mend

* Start modeling ffmpeg extracted metadata on schema
 - Start porting ffprobe code to rust
 - Rename some references to media_data to exif_data

* Finish modeling media info data
 - Add MediaProgram, MediaStream, MediaCodec, MediaVideoProps, MediaAudioProps, MediaSubtitleProps to Schema
 - Fix simple_ffproble to use its custom print_codec, instead of ffmpeg's impl

* Add relation between MediaInfo and FilePath
 - Remove shared properties from MediaInfo and related structs
 - Implement Iterator for FFmpegDict

* Fix and update schema

* Data models and start populating MediaInfo in rust

* Finish populating media info, chapters and program

* Improve FFmpegFormatContext data raw pointer access
 - Implement stream data gathering

* Impl FFmpegCodecContext, retrieve codec information
 - Improve some unsafe pointer uses
 - Impl from FFmpegFormatContext to MediaInfo conversion

* Fix FFmpegDict Drop

* Fix some crago warnings

* Impl retrieval of video props
 - Fix C char* to Rust String convertion

* Impl retrieval of audio and subtitle props
 - Fill props for MediaCodec

* Remove simple_ffprobe now that the Rust impl is done

* Fix schema to match actually retrieved media info
 - Fix import some FFmpeg constants instead of directly using values

* Rework movie_decoder
 - Re-implement create_scale_string and add support anamorphic video
 - Improve C pointer access for FFmpegFormatContext and FFmpegCodecContext
 - Use newer FFmpeg abstractions in movie_decoder

* Fix incorrect props when initializing MovieDecoder

* Remove unecessary lifetimes

* Added more native wrappers for some FFmpeg native objects used in movie_decoder

* Remove FFmpegPacket
 - Some more improvements to movie_decoder

* WIP

* Some small fixes

* More fixes
Rename movie_decoder to frame_decoder
Remove more references to film_strips

* fmt

* Fix duplicate migration for job error changes

* fix rebase

* Solving segfaults, fuck C lang

Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>

* Update rust to version 1.77
 - Pin rust version with rust-toolchain.toml
 - Change from dtolnay/rust-toolchain to IronCoreLabs/rust-toolchain for rust-toolchain support
 - Remove unused function and imports
 - Replace most CString uses with new c literal string

* More segfault solving and other minor fixes

Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>

* Fix ffmpeg rotation filter breaking portrait video thumbnails #2150
 - Plus some other misc fixes

* Auto format

* Retrieve video/audio metadata on frontend

* Auto format

* First draft on ffmpeg data save on db

Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>

* Fix some incorrect changes to prisma schema

* Some fixes for the FFmpegData schema
 - Expand logic to save FFmpegData to db

* A ton of things

Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>

* Integrating ffmpeg media data in jobs and API

* Rspc can't BigInt

* 🙄

* Add initial ffmpeg metadata entries to Inspector
 - Fix ephemeral metadata api to match the files metadata api call

* Fix Inspector not showing ffmpeg metadata

* Add bitrate, start time and chapters video metadata to Inspector
- Fix backend BigInt conversion incorrectly using i32 instead of u32
- Change FFmpegFormatContext/FFmpegMetaData bit_rate to i64
- Rename byteSize to humanizeSize
- Expand humanizeSize logic to allow handling bits and Binary units
- Move capitalize to @sd/client utils

* Solving some issues

* Fix ffmpeg probe getting incorrect stream id and breaking database unique constraint
 - Fix humanizeSize breaking when receiving floating numbers
 - Fix incorrect equality in StatCard
 - Fix unhandled error in Dialog when trying to remove an unknown dialog

* fmt

* small improvements
 - Remove some unecessary recursion_limit directive
 - Remove unused app_image releated functions
 - Fix metadata query enabled flag

* Add migration for ffmpeg media data

* Fix cypress test

* Requested changes

* Implement feedback
 - Update locale keys for all languages
 - Add pnpm command to update all language keys

* Fix thumb reactivity in non indexed locations

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
2024-05-09 02:20:28 +00:00
Vítor Vasconcellos 0f405caf24
Fix CI (#2461)
Use native-tls-vendored for reqwest
 - Update prisma-client-rust
 - Fix mock_instance test
 - Clippy + fmt
 - Improve error handling when uninstall i386 libs in setup-system action
2024-05-08 05:04:25 +00:00
Vítor Vasconcellos 44478207e7
Remove AppImage logic and build system (#2446) 2024-05-04 22:27:14 +00:00
Utku 3bed56d4d9
Alpha 0.2.12 (#2382)
* pnpm

* alpha 0.2.12

* make pnpm version non strict
2024-04-24 14:25:22 +00:00
Utku 6c5e80fffb
Replace Appimage with deb (#2345)
* replace appimage with deb

* Update pnpm to 9.0.2 and lock CI version

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2024-04-17 21:09:12 +00:00
Vítor Vasconcellos 4fc7ba6275
Update pnpm to version 9 (#2341)
* Update pnpm to version 9

* Don't force python v3.11 in CI

* Ensure Cypress is installed

* ¯\_(ツ)_/¯
2024-04-17 06:16:18 +00:00
Vítor Vasconcellos 56cee3c64d
Fix CI (due to python 3.12 update) + Fix migration CI check (#2306)
* Fix migration check not always failing when it should on main
 - Fix check due to Github using `refs/heads/gh-readonly-queue/main/` as ref when merging a PR from merge-queue to main
 - Add migration due to changes made in #2161

* Fix missing migration error message being showed when it shouldn't
 - Filter out empty migrations files from check testing if a migration is missing

* Prevent inner grep from failling the whole command
 - Fix last grep outputting gibberish when not receiving any file

* Fix xargs and grep command on macOS

* auto format

* Configure python 3.11 version on Github CI
 - Python 3.12 is the default on CI now, which breaks node-gyp and most of the native dependencies installations
 - Fix _new_migrations having lead and trailing spaces

* Remove migration, already included in #2302
2024-04-10 16:32:46 +00:00
Vítor Vasconcellos 11555c583a
CI now checks if a migration is missing (#2295)
* CI now checks if a migration is missing
 - Fail CI on main and tag releases if a new migration should be created due to changes in the db schema
 - Add error msg to schema.prisma file if a migration is not created to reflect changes made to db schema, but do NOT fail action

* Change schema to test CI changes

* Add debug to prisma generate CI script

* Removem prisma cache to test

* Fix grep for migrations

* Revert changes made for testing

* Only cache prisma data on main or cache-factory
2024-04-06 04:31:22 +00:00
Vítor Vasconcellos 99ed9006cf
Make Cypress tests more resilient (#2249)
* Replace macos-14 with ubuntu runner in cypress
- Replace macos-14 with macos-13 runner in clippy
 - macos14 runner has weaker hardware than other runners: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories

* Remove pre-installed incompatible packages from ubuntu runner

https://github.com/actions/runner-images/issues/9546#issuecomment-2014940361

* Use ubuntu-20.04 for Cypress CI

* Adjust Cypress CI to follow common steps for linux env

* Debug preprep script

* Fix copy/paste typo

* Install playwright native dependencies

* Update release.yml

* Add  Remove 32-bit libs to setup-system

* Change Cypress CI to macos-13
 - Some small test fixes
 - Fix onboarding test must skip testing default location screen if no default locations os available

* Back to ubuntu for Cypress and back to macos-14 to clippy
- Attempt to fix empty default locations on ubuntu CI runner

* Oops

* Install playwright native deps

* Replace path regexs with consts
 - Click on share minimum when doing a fastOnboarding

* Try macos-14 again
2024-03-28 22:45:25 +00:00
Utku 9a1cdf1dca
Fix all the weird eslint warnings (#2245)
* actually fix the weird lint errors

* Prettier auto-format

---------

Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2024-03-26 20:46:44 +00:00
Vítor Vasconcellos 51acd0fd8e
Make mobile tests more resilient (#2236)
* Attempt at making mobile tests more resilient

* Forgot shell property

* Remove debug flag from run-maestro-tests.sh

* Fix kill command

* Improve device emulator launch code
 - Remove maestro version lock (expo is not using it anymore on their repo)
 - Use some of the same command expo uses to ensure device emulator launched

* ...

* ...

* Make grep timeout less strict

* Increase timeouts

* Downgrade macOS and Xcode

Following Expo changes:
cb7c90dac4
2024-03-25 19:42:37 +00:00
Brendan Allan f91c6fa2fb
separate bundle and archive exts in artifact publisher (#2181)
* separate bundle and archive exts in artifact publisher

* actually build the damn action ffs
2024-03-08 05:31:27 +00:00
Vítor Vasconcellos 5a103fb21b
[ENG-1472] Frontend bug fixes and dependencies update (#2137)
* Fix onboarding sometimes not redirecting to Explorer on prod builds
 - Fix failure trying to compile landing prod builds outside Vercel
 - Fix Server docker failing to restart due to some incorrecting logic for creating the unprivileged users erroring out
 - Fix Storybook failing to build due to updates to Vite
 - Update Storybook dependencies
 - Remove unused Inter font dependency
 - Fix some incorrect references to NodeJS types inside web code
 - Fix $libraryId index using the incorrect redirect function
 - Improve error handling for the $libraryId index route
 - Fix Prism not being correctly loaded, and failing to register its plugins
 - Improve Prism loading error handling
 - Small improvement to the text highlight logic
 - Fix SCSS deprecation for untyped hsl values
 - Fix library query cache incorrectly saving empty values, which lead to the onboarding redirect bug
 - Patch contentLayer to fix an error during the final part of it's build process
 - Update most dev dependencies
 - Update publish-artifact to be compatible with new @actions/artifact
 - Fix issue with new vite-plugin-solid failing to build our .ts files due to the removal of the typescript plugin
 - Fix pnpm overrides not applying due to incorrect placement in package.json
 - Replace deprecated react-tsparticles and updated three used by the Bubbles background in the landing page
 - Rework Bubbles background to be compatible with new @tsparticles/react
 - Update @sd/config dependencies
 - Update @sd/scripts dependencies

* Implement suggestions
 - Replace mobile JS node setup with custom setup-pnpm action
 - Handle GITHUB_SECRET default value in code and throw a warning when it is not set
 - Fix pnpm now resolving the correct node version when building Spacedrive server docker
 - Add missing getent command to spacedrive server docker
 - Fix typo in entrypoint.sh
 - Implement a more robust check if the user is already in a group
 - Fix adduser failing due to missing default group
 - Disconnect IntersectionObserver on component unmount
 - Improve prism import comment

* Implement more suggestions
 - Pin genent version to latest stable release of UClibc
 - Add checksum checks for all ADD clauses in Spacedrive server Dockerfile

* Increase Maestro timeout to reduce CI failures due to slow simulator startup

* Dowgrade maestro to workaround CI timeout

* Improvements to the script that run maestro mobile tests
 - Increase the amount of retries for a maestro test run to 6
 - Increase Maestro driver startup timeout to 2 minutes

* Let run-maestro-tests.sh decide how to run itself

* ¯\_(ツ)_/¯
2024-02-28 23:52:24 +00:00
Vítor Vasconcellos 3bdcc05c2d
Fix mobile CI + Some small CI improvements (#2113)
Fix mobile CI
 - Use rust envvars in all workflows
 - Use rust envvars and mold when build sd-server docker
2024-02-21 14:18:15 +00:00
Vítor Vasconcellos 393a907b57
Update github actions due to nodejs 16 deprecation (#2107)
* Update github actions due to nodejs 16 deprecation
 - Replace archived actions-rs/clippy-check with maintained fork actions-rs-plus/clippy-check
 - Replace redhat-actions/push-to-registry with updated fork Eusebiotrigo/push-to-registry
 - Point redhat-actions/buildah-build and softprops/action-gh-release to current master to fix nodejs deprecation

* Build the correct ios core rust arch for CI runs

* Build ios app for the same arch as the host in Mobile CI

* Some changes to try and make cache-factory faster and avoid failing so much

* Add trigger to run cache-factory on pull requests when there are changes to itself

* Attempt to fix sed usage on macOS

* Don't treat warning as errors

* Fix windows

* Fix windows 2

* Use target ad cache key for rust to differentiate between macOS x86_64 and arm64

* Use faster/better linkers to compile for macOS, Linux and Windows

* Fix missing shell in action

* Fix typo

* Fix missing shell in action 2

* Fix mold download
 - Replace bsdtar with plain tar

* Fix permission denied when extracting mold

* Remove zld

* Don't restore cache for rustfmt
 - Remove target symlink to C:/ in an attempt to speed-up windows CI

* Fix typo

* Restore target symlink on windows
 - Removing it didn't make CI faster

* Run Mobile on macos-14
2024-02-21 09:27:40 +00:00
Brendan Allan c45f6d7bfa
separate cloud sync into sd-core-cloud-sync (#2025)
* separate cloud sync into sd-core-cloud-sync

* ci
2024-01-31 08:16:54 +00:00
Oscar Beaumont 949707c7f9
Vite upgrades (#1911)
* Upgrade Vite + use SWC React plugin

* Upgrade to type module

* lazy load Sentry

* Lazy load prism

* fix

* Lazy load some of the icons

* fix types

* Fix eslint config

* run lint --fix

* Upgrade Turbo

* Turborepo not happy

* Upgrade Typescript

* Stop complaining Turborepo
2024-01-02 06:26:28 +00:00
Vítor Vasconcellos f3ad0d7513
Pin rust to 1.73 (Workaround Internal Compiler Error) (#1796)
* Pin rust to 1.73
- Clippy fixes

* Don't install cargo-watch on CI
2023-11-17 22:47:06 +00:00
Brendan Allan 0ef65fce2d
[ENG-1423] Multiple tabs (#1777)
* tabs w/ multiple router instances

* fix router switching

* keybinds

* manual history tracking

* eslint

* remove scroll restoration

* fix tab removal

* route title + tab create delay

* typescript

* put tab list up top

* Remove import + show close button only if tabs length more than 1

* lint

* unify blur across whole top bar

* add to keybindings page, tauri drag region, and tooltip

* fix blur

* more drag regions

* merge moment

---------

Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
2023-11-16 15:14:47 +00:00
Vítor Vasconcellos 3d3f333b64
Fix Windows CI (#1695)
* Add Nasm to setup.ps1
 - Nasm is now required to compile the `openssl-sys` crate

* Run Clippy CI when setup or prep scripts change

* Ensure nasm is also installed in Windows CI
 - Add nasm to setup.sh
 - Remove openssl-dev from setup.sh

* Ensure nasm is installed in macOS too

* Dumb ctrl+c/ctrl+v mistake

* 's/==/=/'
2023-10-29 00:25:26 +00:00
Vítor Vasconcellos f7277d602f
Frontend misc fixes and deps update (#1650)
Dependencies overhaul
 - Update dependencies for all projects (except Mobile-only deps)
 - Remove unused dependencies from all projects (except Mobile-only deps)
 - Fix Storybook failing to import sd/ui style
 - Add Node 21 as not supported due to sass-loader not working on it yet
 - Add work-around for new rook version requiring webpack specific global property
 - Fix landing dev not working due to missing default env value on dev
 - Fix some incorrect uses of phosphor-icons non server side icons on server components on landing
 - Fix some incorrect uses of phosphor-icons server side icon on client components on landing
 - Fix landing fail to build on dev due to always required a Github Token to get the latest release
 - Fix new Next.js version not suporting Response.redirect due to immutable Headers
 - Add Gitlab as social link for teams page
 - Update Vítor's team photo
 - Add Vítor's twitter link
 - Fix some warning due to missing useEffect dependencies
 - Remove test-files dir
 - Fix QuickPreview unblurred buttons in Linux
 - Formatting
2023-10-24 07:51:58 +00:00
Brendan Allan 5d350472a9
[ENG-1314] Re-enable cache factory on all main commits (#1608)
re-enable cache factory on all main commits
2023-10-18 04:22:41 +00:00
Brendan Allan b27c5ac49f
Smarter release workflow (#1525)
* js-based publish workflow

* don't create release if not using tag

* rename

* update lockfile

* remove unnecessary tsconfigs

* replace old publish action

* dumb

* esmodule moment

* glob files

* lockfile

* build app bundle

* Fix artifact upload hopefully

* upload artifacts from root

* skill issue ffs

* Use copy instead of move

* hopefully last one

* artifcats

* make .artifacts -_-

* ffs

* Make cp recursive, so it work with dirs

* build

* separate updater & standalone targets

* create draft release

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-10-13 08:56:54 +00:00
Brendan Allan 49cc098f32
[ENG-927, ENG-735, ENG-766] Fix Updater & Tauri 1.5 (#1361)
* custom updater with toasts

* new state management + updated router route

* tauri-specific update route

* ref

* update in prod only

* change 'Install' to 'Update'

* fix tsconfig

* desktop tauri

* remove tauri patch

* tauri 1.5

* tauri 1.5

* use tauri script

* native-deps

* Rework preprep and tauri script to better support tauri 1.5

* Update to tauri 1.5.1
 - Update workspace and apps/desktop dependencies
 - Fix mustache import, @types/mustache is not compatible with ES imports
 - Replace arm64 with aarch64 in machineID, they should be treated the same and this simplyfies the code

* Fix tauri updater not building due to missing key
 - Fix dmg background not being found
 - Generate an adhoc key for tauri updater with it is enabled and the user is doing a prod build

* Fix ctrl+c/ctrl+v typo

* Normalie @tanstack/react-query version through workspace
 - Use undici in scripts instead of global fetch
 - Fix typecheck

* Fix linux prod and dev builds
 - Improve error handling in tauri.mjs

* Normalize dev deps in workspace
 - Improve linux shared libs setup

* Fix CI and server docker

* Fix windows
 - Remove superfluous envvar

* Attempt to fix server, mobile, deb and release updater

* Attempt to fix deb and mobile again
 - Fix type on deb dependency
 - Enable release deb for aarch64-unknown-linux-gnu

* Github doesn't have arm runners
 - Fix typo in server Dockerfile

* Publish deb and updater artifacts

* remove version from asset name

* update commands

* log release

* Some logs on updater errors

* show updater errors on frontend

* fix desktop ui caching

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
2023-10-10 07:30:56 +00:00
Vítor Vasconcellos d75a6e9b12
[ENG-1054] libheif for Linux (plus fixes) + Webp update (#1405)
* Move postinstall script to a preprep script
 - Fix libheif crate failing to build with our libheif
 - Rework CI due to postinstall to preprep changes

* Linux heif build script + Update webp

* Fix ctrl+c/ctrl+v bug

* Improve libheif linux script
 - Add support for linux aarch64
 - Add CI workflow to build libheif for linux
 - Some other misc fixes

* Fix libheif CI requires sudo

* Fix wrong path for libheif build.rs override in Windows

* Fix wrong path manipulations in libheif build script

* 🤦

* Use ubuntu-latest in libheif action
 - Specify glib version in target triple to support old distros
 - Fix libheif artifact publishing

* Fix artifact upload path again

* Add musl support for libheif
 - Remove unused files from libheif artifact
 - Add setup logic for libheif in postinstall script

* Build libheif for linux as a shared lib

* Fix meson not building the correct arch
 - Add logic to get git branch from githubs CI env vars

* libheif finnaly works on linux
 - Make spacedrive binary rpath point to where appimage and deb expects our libs to be
 - Add some logic to tauri.js to convince tauri to bundle our shared libs
 - Work-around appimage bundling step breaking sometimes
 - Add logic to handle sigint in tauri.js to ensure we clean up after ourselves
 - Rename postinstall.mjs to setup.mjs
 - Add logic to setup.mjs to point our dev build to our shared libs in linux

* Fix windows dekstop dev
 - Rename setup.mjs to preprep.mjs

* test cache-factory

* Fix preprep script not parsing the cross compilation target triple and always using the host info to download dependencies
 - Fix action env vars not being correctly passed
 - Remove possibility to pass multiple targests to rust action

* Don't compile mobile crates on desktop targets

* Remove cache-factory pull_request trigger

* remove patched tauri cli

* Use git plumbing command to get remote branch name
 - Fallback to reading .git/HEAD if remote name was not retrieved

* fix type

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-10-03 17:31:33 +00:00
Vítor Vasconcellos f71711d09a
Change docs and CI scripts to use pnpm tauri command (#1421)
* Change docs and CI scripts to use `pnpm tauri` command
 - Fix documentation and some scripts to use the new tauri subcommand following https://github.com/spacedriveapp/spacedrive/pull/1411 changes
 - Format/Lint some files

* Document new envvar to lauch devtools alongside the desktop app
2023-10-03 08:10:55 +00:00
Vítor Vasconcellos bd0a7ff434
[ENG-265] Improve setup scripts (#1368)
* Setup script revamp

* Move toml to dev dep + CI runs the postinstall in debug mode

* Fix windows CI

* chmod +x scripts/setup.sh

* Fix eslint and ts CI

* Remove binstall + Fix trying to read input in CI

* Doesn't need to check pnpm and rust in CI

* Run postinstall script for Clippy CI

* Attempt to fix windows CI not running postinstall
 - Ignore cache when running postinstall on CI

* commited generated config.toml by mistake

* Pass GITHUB_TOKEN to `pnpm i`

* Update archive-wasm + Increase minimum node version to 18.17

* CI: Move rust setup after post-install script

* Revert: CI: Move rust setup after post-install script

* Fix CI, generate dummy cargo config.toml to fix prisma generation

* Fix windows CI

* CI: Fix wrong command

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-09-28 10:03:46 +00:00
Utku 5550ef2d87
Fixes release action (#1277)
* fix action

* updater artificats
2023-08-30 17:58:17 +00:00
Vítor Vasconcellos 1b28343304
Fix Server CI running out of disk space (#1222) 2023-08-16 21:01:39 +00:00
Vítor Vasconcellos 34d4074d5f
[ENG-937] Work-around running out of space in GitHub CI for cache factory (#1158)
* Attmept to workaround Linux and Windows CI runners out-of-space problems

* Don't mix cmd and powershell

* bruh

* Attempt to fix prisma cache

* No need to cd, as the step is already in the correct cwd

* Leave enough room in root to install the necessary system packages

* Attempt to fix macOS cache being ovewritten

* Fix dumb mistake

* Up linux root free size to 6GB because
 - The cache action compress the data to root and it need more free space

* Remove pull_request trigger from cache_factory

---------

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-08-16 08:17:29 +00:00
Utku 0975d135f8
fix action (#1023) 2023-06-26 15:00:12 +00:00
Vítor Vasconcellos 417d5d3be3
[ENG-811] Fix server Dockerfile + create CI workflow to build and publish docker image (#1012)
* Fix server docker build failing due to apt locking error
 - Replace ffmpeg dep with only the libav* and related required lib (reduce size)
 - Add warning to tauri build command when DMG background is missing
 - Minor rust fmt

* Enable assets feature for server Dockerfile
 - Fix web app not using the correct address in prod
 - Add build step for web assets in server Dockerfile

* Enable repository to be defined in server's Dockerfile
 - Enable server's release workflow to specify which repository to build to allow external PRs to work

* Fix pnpm docker cache not working
2023-06-25 15:16:11 +00:00
Utku debcb1daed
[ENG-734] Hide releases (#946)
* add a dummy check

* informative description

* change retention to 1 day
2023-06-15 05:30:10 +00:00
Oscar Beaumont a20f6262fd
[ENG-659] Spacedrive On Jamie's NAS (#866)
* serve dem

* 0% chance of working

* With a side of Vite plz

* no CI for now

* Clippy plz work

* undo

* bruh

* plz work
2023-05-29 10:47:59 +00:00
Brendan Allan 6c82f1f2c8
update prisma cache (#878) 2023-05-29 09:11:29 +00:00
Vítor Vasconcellos b169538716
Some misc fixes (#836)
* Fix setup-system breaking in macOS due to unavailable envvar
 - Remove unused docker build-arg from ffmpeg workflow
 - Fix Overview Explorer extending below inspector
 - Fix some left behind formatting problems

* Some more formatting
2023-05-20 04:36:35 +00:00
Brendan Allan 99f4df1487
macos signing (#835)
* add certificate envs

more signing envs

* signing identity

install api key in release workflow

sign ffmpeg libraries

* update all envs

* update docker build action

* update fmmpeg scripts

* remove lib codesigning from setup script

* declare signing identity arg

* codesign ffmpeg in setup

* stupid

* qutoes
2023-05-20 03:20:14 +00:00
Vítor Vasconcellos cf482f5621
[ENG-270, ENG-213] Stabilize release pipeline (#768)
* Attempt at building our own ffmpeg dylibs for macOS
 - Create dockerfile to setup osxcross and required host dependencies for cross-compiling ffmpeg for macOS
 - Create script to setup required macOS dependencies and build ffmpeg for both x86_64 and aarch64

* Improve dockerfile to better use docker's cache
 - Attempt at fixing arm64 build (still not there yet)

* ARM64 sucessfully compiles \o/
 - Use clang for everything (e.g. linker, assembler, ...)
 - Remove -lc++ because it is now handled internally thanks to a patch
 - Apply 3 open PRs to osxcross as external patches to solve some of our problems
 - PR 180 reduces the macports dependencies size by prefering non universal libs for x88
 - PR 314 adds the -lc++ flag for macOS SDKs >= 11.1
 - PR 379 fix macports pulling incorrects deps for SDK11 and adds support for SDK13 (future proofing, as it does nothing for us rigth now)

* Fix relink and copy logic in ffmpeg build script
 - Ensure ffmpeg and dependencies dylibs are correctly relinked and copied over to the output directory
 - Copy includes from ffmpeg and dependencies to allow for dev building against these dylibs
 - Set up symlinks for all dylibs to the common `lib` directory
 - Remove superfluous linker and compiler options
 - Add `-headerpad_max_install_names` linker option to accommodate the required relink changes
 - Remove `--disable-fft` for the arm64 build, as it doesn't seem to be required nowadays
 - Allow `swresample` and `avdevices` to be compiled as they seem to be expected to exist by `ffmpeg-sys-next`
 - Disable ffmpeg program compilation
 - Fix dylibs id to match their new expected location
 - Fix ffmpeg dylibs inter-linkage not being updated
 - Add custom patch for osxcross macports to load already downloaded files from cache
 - Remove the requirement for specifying the darwin version in the Dockerfile
 - Enable docker cache for macports
 - Move arm min macOS version ARG to the top

* Attempt at building ffmpeg with github actions
 - Split osxcross setup into it's own Dockerfile to allow exporting it as a image to Dockerhub
 - Create a github action to build ffmpeg
 - Add osxcross patch for PR 372 for future proofing (Add support for macOS SDK 13.1)

* Remove incompatible step name

* Change FFMPEG_VERSION to string to avoid yaml weird type casting

* Start to use our ffmpeg build in the macOS
 - Adjust the macOS setup script to download our ffmpeg build, also replace installing protobuf compiler from brew with grabbing it from it's own repository
 - Some misc improvement to the *nix setup script
 - Fix ffmpeg build script not copying static libs (for when we may want to statically link ffmpeg)
 - Fix ffmpeg build script not taking into acount possible libraries that are multiple paths deep in /lib
 - Update CI to actions/checkout@v3, v2 is deprecated
 - Update ffmpeg build CI to not xz the artifacts, as github already zips it anyway

* Fix macOS setup not finding the ffmpeg artifact due to a jq script mistake
 - Comment out embeeding static libs in ffmpeg artifact for now, as they take too much space
 - Remove incorrect dockerfile settings from editorconfig

* Fix protobuf download in macOS
 - Fix Framework directory creation not following script's dir but cwd
 - Improve download logic to avoid being rate limited on failures
 - Return to using xz for ffmpeg build, as it saves quite a bit of space

* Replace `OSTYPE` with `$(uname)` for a more standardized way to check which os is running
 - Fix mobile check using `==` instead of `=` in test command

* Fix syntax error in bash 3.x (which is the default in macOS)
 - Fix jq not downloading all deps and being placed in the incorrect directory

* Fix incorrect install_name_tool arguments
 - Remove leading './' when referencing ffmpeg libs during build

* Fix protoc not being executable
 - Fix some error messages

* jq throw input when it is invalid
 - Enable debug log in setup script when running in CI

* Add a dev.js build script for desktop app
 - Remove dll copy from setup-system.ps1, now done by dev.js
 - Minor rework to build.js to improve signals and stdio handling

* Configure cargo to use FFMPEG_DIR and PROTOC envvars on macOS

* Pass GITHUB_TOKEN to macOS setup script in CI

* Replace simple curl with gh_curl in github request

* Fix some ctrl+c/ctrl+v typos

* Add missing ffmpeg deps
 - Fix missing symlinks for ffmpeg libs in the output_dir/libs
 - Change arm64 minimum macOS target to 11.2 due to libbrotli only being available precompiled to this version
 - Add more options to ffmpeg configure in an attempt to reduce its size + improve performance

* Rework ffmpeg build script to create a FFMpeg.framework instead of loose dylibs
 - Disable unused postproc and enabled missing required protocols in ffmpeg configuration
 - Adjust Dockerfile and setup-script to handle the new FFMpeg.framework

* Fix setup script incorrect linking logic for FFMpeg.framework

* Forgot to remote the leading path in the lib/header linking logic for FFMpeg.framework

* Enable size optimization for ffmpeg
 - Remove Frameworks directory before setting it up in setup-system.sh

* Revert lib id to use `@executable_path/../Frameworks/` instead of `@rpath`
 - `@rpath` breaks dev builds
 - Add logic to create less versioned versions of dylibs in setup-system script, due to it being required for compiling ffmpeg-sys-next (not required during runtime, just for compilation)
 - Add scripts/deps to gitignore

* Fix ffmpeg build script not copying unversioned symlinks to FFMpeg.framework
 - Remove unecessary unversioned symlink logic from setup-system
 - Add new build scripts for dev and prod that setup the environment and tauri to correctly point to FFMpeg.framework
 - Rework windows setup-system script to follow the same logic as in macOS setup-system

* Attempt at fixing windows setup script
 - Add powershell editorconfig

* Attempt to fix Windows CI

* Fix some mistakes in the build scripts
 - Replace toml lib with something more used and maybe better?
 - Attempt to fix Windows CI

* Fix some more problems with the build script

* Another attempt at fixing windows CI

* Some more fixes for the windows setup script

* Add build.js step to release CI
 - Implement patch only mode for build.js when running under CI
 - Implement appeding extra env in cargo config in env.js
 - Only run windows dlls copy in dev in env.js
 - Another attempt at fixing windows CI

* Maybe this will make the windows CI happy?

* Windows CI why dont you work, whyyy?

* Try to apease the Windows CI Gods
 - Disable updater build in macOS, due to tauri-apps/tauri#3933

* Fix build.js extra env vars
 - Uncomment parts of setup-system.ps1 to check if it keeps working

* Uncomment another section and see if windows CI will complain

* uncomment some more of setup-system.ps1

* Use Start-Process for robocopy instead of executing it directly
 - Only raise an error if robocopy exists with a status code >=8
 - Revert build.js macOS specific bundles change
 - Disable updater for now

* Remove updater from cargo.toml

* Add -PassThru to Start-Process to ensure we get and Process obj back

* Comment out more of updater

* Improve documentation for both of the docker containers
 - Fix a harcoded install_name_tool call in the ffmpeg-build-macos.sh script

* Configure tauri to copy ffmpeg dlls to windows prod version
 - Fix windows dlls copy on dev
 - Stop replacing tauri.conf.json, except on CI, for prod builds and use a tauri.conf.patch.json for any required changes
 - Don't unset macOSPrivateApi on prod build, we need it for the transparent background
 - Fix dev.js not exiting when the spawned command was over

* Fix windows not find protoc
 - Fix missing import in env.js

* Fix Windows Dlls copy logic for prod build
 - Fix dev script failing due to missing env
 - Implement error handler to spawned process

* Format + Fix pnpm-lock

* Fix video thumb generation failing on Windows due to bad path to CString conversion logic

* Fix mobile build
 - Greatly improve windows setup-script error handling
 - Install LLVM with winget instead of downloading exe from github

* Improve CI a bit
 - Fix mobile CI not using github token to avoid being rate limited

* Fix pnpm not accesible to actions/setup-node

* Skip pnpm check on CI

* Fix pnpm skip not working
 - Fix pnpm cache check missing a step

* Only lock action to major versions

* CI really doesn't like running robocopy directly

* Attempt to build our own patched tauri bundler

* Trigger new workflow

* Fix attempt for patched tauri action tests
 - Disable cache for patched tauri action
 - Enable package publishing for patched tauri action

* Attempt fix patched tauri publish

* Remove commented code from patched tauri workflow
 - Replace deprecated `actions-rs/toolchain` with `dtolnay/rust-toolchain`

* Fix url patching logic for publishing step of patched tauri package

* Fix prepublishOnly command for patched tauri package

* Adjust patched tauri package package.json

* Also patch the specific os/arch sub packages

* Instruct napi to stop doing github releases

* Patch package.json befor build

* Re-enable updater
 - Patch tauri cli to workaround a tauri bug that prevented use from bundling frameworks in the updater
 - Update tauri/api to 1.3 and tauri/cli 1.3.1
 - Add target triple in the release artifacts name
 - Use ubuntu-20.04 instead of ubuntu-latest
 - Configure release workflow to build to all supported platforms
 - Replace `tauri-apps/tauri-action` with simply calling `pnpm desktop build`
 - Simplify tauri-patched-cli-js to only build the required macOS binaries
 - Unify build.js and dev.js in a tauri.js build script, which now acts as an all around wrapper for the tauri/cli

* Restore incorrectly removed step from `tauri-patched-cli-js`

* Disable auto-run of tauri-patched-cli-js
 - Update setup-system.sh with the latest tauri-patched-cli-js artifacts

* Forgot to mkdir bin

* Fix pnpm-lock.yaml not being up to date
 - Remove pnpm deps caching

* Fix variable name conflict

* Enable the updater build conditionally
 - Enable release workflow to test it

* Fix artifacts not being published
 - Change macOS on arm64 minimumSystemVersion to 11.2
 - Commented out rust cache to check if this is the culprit to the low memory problem on windows runner
 - Add some debug log to tauri.js

* Fix release workflow artifact uploading

* Specify which bundles to build on each platform
 - Attempt fix apple arm build
 - Don't include internal deb archives in linux updater artifact

* Attempt to fix apple arm build
 - Fix dmg background not applying

* Fix incorrect semver usage

* Patch swift-rs to see if that fix apple arm64 release build

* Update swift-rs submodule
 - Remove unused deps lodash.merge

* Configure so only release workflow runs without cache
 - Improve tauri.js logs

* Ignore `crates/swift-rs`

* Revert "Ignore `crates/swift-rs`"
- Remove `crates/swift-rs` before running `cargo fmt`

This reverts commit 851bd84373.

* Github CI input/output are always string
 - Attempt to fix Windows Clippy

* Fix CI syntax error

* Fix mobile CI
 - Disable pnpm deps caching
 - Disable rust targets caching on Windows
 - Configure cache factory to run for all targets

* Remove `crates/swift-rs`

* Attempt fix Mobile CI
 - Enable cache factory run in pull_request to test it
 - Specific more path for the CI run to ignore
 - Specify path that will trigger the cache factory run
 - Some cleanup in the setup-pnpm action

* Remove restore-cache options
 - Add prefix key to cache-factory
 - Fix LLVM cache-hit check
 - Add libheif to FFMpeg.framework

* Remove submodule clone from actions

* Add fake deps to ffmpeg build dockerfile
 - Fake install deps that aren't really necessary to build ffmpeg
 - Fail ffmpeg build script if it can't find a macports deps
 - Copy libheif headers to Framework

* Remove automatic trigger for relase and pull request trigger for cache-factory actions

* Remove pnpm setup from Cache factory
 - Cache factory is exclusivly for rust deps caching, no need to setup Node and pnpm with it

* Re-enabled fail-fast in Github CI
2023-05-15 18:02:56 +00:00
Utku f2ff107de0
bump pnpm (#798)
* bump pnpm

* bump ci pnpm version

* almost forgot a pnpm ci lol

* Update pnpm version installed during windows setup-script

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-05-05 22:14:49 +00:00
Vítor Vasconcellos c7d182e12b
Fix Windows CI + format (#767)
* Attempt to fix Windows CI:
- Ensure version 15 of LLVM is installed in Windows CI (due to` ffmpeg-sys-next`)
- Use `GITHUB_TOKEN` in setup-system.ps1 to avoid rate-limiting by github api in Windows CI
- Add `yaml` settings to `.editorconfig`
- Create a symbolic link from `.prettierignore` to `.eslintignore`
- Move `eslint` root to package root config
- Enable `fileNesting` to reduce dotfile clutter in vscode
- Pass `--strict` to apps/releases `lint` command to avoid making interactive queries and hanging `pnpm lint`
- Format with `prettier` and `eslint`
- Remove empty interface/hooks/useMediaQuery.ts

* Replace unmaintained `actions-rs/toolchain` with `dtolnay/rust-toolchain`
 - Change setup-system.ps1 to avoid installing pnpm in CI
 - Change `pnpm/action-setup` to lock only o major versions of the action
2023-04-27 07:32:15 +00:00
Brendan Allan 7565b211d1
Releases with signatures (#752)
* package releases with signatures

* only include necessary stuff in generated artifacts

* Publish updater binaries

* proper glob for updater binaries

* use separate paths for unix and windows updaters

* make .app executable

* brendan dumb
2023-04-22 00:56:59 -07:00
Brendan Allan 46db121a09
Fix cache workflow (#746)
* use proper comparison

* Revert "use proper comparison"

This reverts commit 0271ea3d5c.

* bruh
2023-04-21 11:57:17 -07:00
Brendan Allan b5a3e05c9a
Update server (#744)
* release workflow

* tauriScript

* pnpm deps

* unify rust cache keys

* bruh

* basic manual updater stuff

* move updater override

* use existing publishing action

* use debug builds for desktop

* cache release builds too

* use release builds instead

* add workflow dispatch trigger

* remove branch from release workflow trigger
2023-04-21 10:02:31 -07:00
Vítor Vasconcellos a6a57bbd50
Fix pnpm format command failing with some files (#622)
* Fix `pnpm format` command failure
 - Update prettier and plugins dependencies
 - Add auto-install-peers to .npmrc
 - Fix formating of previously failing files
 - Some basic security mitigations (`pnpm audit --fix`)

* setup-system.sh: Don't do a full system update on ArchLinux

* Revert "Some basic security mitigations (`pnpm audit --fix`)"
2023-03-19 16:47:10 +00:00
Brendan Allan 31df51501e
CI Cache Factory (#593)
* introduce cache factory

run cache factory on all branches

add names

use single quotes

add setup and prisma generation

add pnpm install

don't install cargo-watch in ci

make composite setup script

use env var for windows ci

undo all powershell stuff

* better windows ci

* don't install cargo-watch in ci

* only run cache factory on main

* i'm silly

* debug only ci

* check runner instead of matrix in setup action

* don't save cache output most of the time

* no more os-specific features

* cache entire workspace

* only cache on main again

* test instead of buidl in cache factory

* smaller ci + testing

* don't individually test each crate

* remove unnecessary needs

* smaller formatting and more clippy

* undo rustfmt changes

* disable tests
2023-03-06 06:10:55 -08:00
Brendan Allan dad2072d7b
generate + use ModelSyncData enum (#592)
* generate + use ModelSyncData enum

* cache prisma clients on sync-generator
2023-03-04 22:12:37 -08:00
maxichrome ec4352a30a cleanup(don't hate me): prettier format everything 2023-01-19 19:32:35 -08:00
Brendan Allan 676e512c4a
Beginning of sync stuff (#502) 2023-01-04 23:11:55 -08:00