Commit graph

14 commits

Author SHA1 Message Date
Oscar Beaumont b015763a6f
More P2P docs (#2492)
* Remove relay

* restructure p2p

* wip

* cleanup webrtc

* split up P2P docs

* wip

* more wip

* the fork has moved

* finish local network discovery

* Document the relay system

* be less stupid

* a

* remote ip from deploy script

* remove debug from deploy script

* Explain relay setup and usage

* Physical pain

* fix

* error handling for relay setup

* Listeners Relay state + merge it into NLM state

* `node_remote_identity`

* redo libraries hook

* toggle relay active in settings

* Dedicated network settings page

* Stablise P2P debug page

* warning for rspc remote

* Linear links in docs

* fix p2p settings switches

* fix typescript errors on general page

* fix ipv6 listener status

* discovery method in UI

* Remove p2p debug menu on the sidebar

* wip

* lol

* wat

* fix

* another attempt at fixing library hook

* fix

* Remove sync from sidebar

* fix load library code

* I hate this

* Detect connections over the relay

* fix

* fixes

* a

* fix mDNS

* a bunch o fixes

* a bunch of state management fixes

* Metadata sync on connection

* skill issue

* fix markdown

* Clippy cleanup

* Backport #2380

* Update interface/locales/en/common.json

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* Update docs/developers/p2p/local-network-discovery.mdx

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* Update docs/developers/p2p/local-network-discovery.mdx

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* Update docs/developers/p2p/relay.mdx

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* Update docs/developers/p2p/relay.mdx

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* Update docs/developers/p2p/relay.mdx

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* Update docs/developers/p2p/relay.mdx

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* Update docs/developers/p2p/relay.mdx

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* Update docs/developers/p2p/sd_p2p.mdx

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* Update docs/developers/p2p/sd_p2p.mdx

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* Update docs/developers/p2p/sd_p2p_proto.mdx

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* Update docs/developers/p2p/overview.mdx

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* Update docs/developers/p2p/overview.mdx

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* Update docs/developers/p2p/relay.mdx

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* Update docs/developers/p2p/sd_p2p_proto.mdx

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* Update docs/developers/p2p/sd_p2p_proto.mdx

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* Update docs/developers/p2p/transport-layer.mdx

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* Update docs/developers/p2p/sd_p2p_proto.mdx

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* Update docs/developers/p2p/local-network-discovery.mdx

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* Update docs/developers/p2p/sd_p2p_proto.mdx

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>

* a

* Cleaning binario section

* cleanup Docker message

* idk

* Idempotent listeners

* Manual peers working????

* minor fixes

* crazy idea - don't panic in the event loop

* fixes

* debug

* debug

* LAN badge in network settings

* Use `dns_lookup` instead of `tokio::net::lookup_host`

* fix

* bruh sandwich

* proper dialing

* a

* remove logs

* fix

* Small cleanup

* manual peers state on connected device

* a

* Fix manual discovery state + give it a badge

* Clippy improvements

* flip discovery priority

* Add `addrs` to debug query

* connection candidates in debug

* Fix state

* Clippppppppppppy

* Manual discovery badge

* Flesh out ping example

* Usage guide

* `sd_p2p_proto` examples

* More discovery docs

* More docs work

* docs docs docs and more docs

* PONG

* rename

---------

Co-authored-by: Matthew Yung <117509016+myung03@users.noreply.github.com>
2024-05-30 21:48:12 +08:00
Vítor Vasconcellos d639ed49ac
Fix server release CI (#2342)
* Fix server release CI

* Forgot sudo

* Fix netavark download

* Fix netavark installation path

* Install passt

* Fix install passt

* Forgot sudo

* Install python in as build deps while building server docker container
2024-04-17 15:16:51 +00:00
Vítor Vasconcellos 4fc7ba6275
Update pnpm to version 9 (#2341)
* Update pnpm to version 9

* Don't force python v3.11 in CI

* Ensure Cypress is installed

* ¯\_(ツ)_/¯
2024-04-17 06:16:18 +00:00
Vítor Vasconcellos 5a103fb21b
[ENG-1472] Frontend bug fixes and dependencies update (#2137)
* Fix onboarding sometimes not redirecting to Explorer on prod builds
 - Fix failure trying to compile landing prod builds outside Vercel
 - Fix Server docker failing to restart due to some incorrecting logic for creating the unprivileged users erroring out
 - Fix Storybook failing to build due to updates to Vite
 - Update Storybook dependencies
 - Remove unused Inter font dependency
 - Fix some incorrect references to NodeJS types inside web code
 - Fix $libraryId index using the incorrect redirect function
 - Improve error handling for the $libraryId index route
 - Fix Prism not being correctly loaded, and failing to register its plugins
 - Improve Prism loading error handling
 - Small improvement to the text highlight logic
 - Fix SCSS deprecation for untyped hsl values
 - Fix library query cache incorrectly saving empty values, which lead to the onboarding redirect bug
 - Patch contentLayer to fix an error during the final part of it's build process
 - Update most dev dependencies
 - Update publish-artifact to be compatible with new @actions/artifact
 - Fix issue with new vite-plugin-solid failing to build our .ts files due to the removal of the typescript plugin
 - Fix pnpm overrides not applying due to incorrect placement in package.json
 - Replace deprecated react-tsparticles and updated three used by the Bubbles background in the landing page
 - Rework Bubbles background to be compatible with new @tsparticles/react
 - Update @sd/config dependencies
 - Update @sd/scripts dependencies

* Implement suggestions
 - Replace mobile JS node setup with custom setup-pnpm action
 - Handle GITHUB_SECRET default value in code and throw a warning when it is not set
 - Fix pnpm now resolving the correct node version when building Spacedrive server docker
 - Add missing getent command to spacedrive server docker
 - Fix typo in entrypoint.sh
 - Implement a more robust check if the user is already in a group
 - Fix adduser failing due to missing default group
 - Disconnect IntersectionObserver on component unmount
 - Improve prism import comment

* Implement more suggestions
 - Pin genent version to latest stable release of UClibc
 - Add checksum checks for all ADD clauses in Spacedrive server Dockerfile

* Increase Maestro timeout to reduce CI failures due to slow simulator startup

* Dowgrade maestro to workaround CI timeout

* Improvements to the script that run maestro mobile tests
 - Increase the amount of retries for a maestro test run to 6
 - Increase Maestro driver startup timeout to 2 minutes

* Let run-maestro-tests.sh decide how to run itself

* ¯\_(ツ)_/¯
2024-02-28 23:52:24 +00:00
Vítor Vasconcellos 3bdcc05c2d
Fix mobile CI + Some small CI improvements (#2113)
Fix mobile CI
 - Use rust envvars in all workflows
 - Use rust envvars and mold when build sd-server docker
2024-02-21 14:18:15 +00:00
Vítor Vasconcellos 72efcc9f62
[ENG-1595] Fix some problems with the AI system (#2063)
* Fix some problems with the AI system
 - Fix downloading model using an internal rust string representation as path for the model file
 - Fix Linux loading onnx shared lib from a hardcoded path
 - Fix App should not crash when the AI system fails to start
 - Fix sd-server failing to start due to onnxruntime incorrect linking
 - Some extra clippy auto fixes

* Use latest ort

* Fix dangling sd_ai reference
 - Use entrypoint.sh to initilize container

* Fix server Dockerfile
 - Fix cargo warning

* Workaround intro video breaking onboarding for the web version

* Fix rebase
2024-02-12 17:45:17 +00:00
Vítor Vasconcellos 2e17b6fe84
Small improvements to sd-server (#1822)
* Fix web spacedrive
 - Replace useOperatingSystem from App.tsx to directly using the 'browser' value, as we can't access usePlatform yet and the frontend crashes due to it.
 - Remove libav* deps from server docker, they are not needed anymore
 - Replace debian with distroless for the runtime base image in server docker

* Fix wrong distroless base image

* Copy missing shared lib

* Add comment explaining why use the debug tag for distroless

* Add missing SpacedriveInterfaceRoot in web/App.tsx
2023-11-25 01:23:10 +00:00
Vítor Vasconcellos f7277d602f
Frontend misc fixes and deps update (#1650)
Dependencies overhaul
 - Update dependencies for all projects (except Mobile-only deps)
 - Remove unused dependencies from all projects (except Mobile-only deps)
 - Fix Storybook failing to import sd/ui style
 - Add Node 21 as not supported due to sass-loader not working on it yet
 - Add work-around for new rook version requiring webpack specific global property
 - Fix landing dev not working due to missing default env value on dev
 - Fix some incorrect uses of phosphor-icons non server side icons on server components on landing
 - Fix some incorrect uses of phosphor-icons server side icon on client components on landing
 - Fix landing fail to build on dev due to always required a Github Token to get the latest release
 - Fix new Next.js version not suporting Response.redirect due to immutable Headers
 - Add Gitlab as social link for teams page
 - Update Vítor's team photo
 - Add Vítor's twitter link
 - Fix some warning due to missing useEffect dependencies
 - Remove test-files dir
 - Fix QuickPreview unblurred buttons in Linux
 - Formatting
2023-10-24 07:51:58 +00:00
Brendan Allan 49cc098f32
[ENG-927, ENG-735, ENG-766] Fix Updater & Tauri 1.5 (#1361)
* custom updater with toasts

* new state management + updated router route

* tauri-specific update route

* ref

* update in prod only

* change 'Install' to 'Update'

* fix tsconfig

* desktop tauri

* remove tauri patch

* tauri 1.5

* tauri 1.5

* use tauri script

* native-deps

* Rework preprep and tauri script to better support tauri 1.5

* Update to tauri 1.5.1
 - Update workspace and apps/desktop dependencies
 - Fix mustache import, @types/mustache is not compatible with ES imports
 - Replace arm64 with aarch64 in machineID, they should be treated the same and this simplyfies the code

* Fix tauri updater not building due to missing key
 - Fix dmg background not being found
 - Generate an adhoc key for tauri updater with it is enabled and the user is doing a prod build

* Fix ctrl+c/ctrl+v typo

* Normalie @tanstack/react-query version through workspace
 - Use undici in scripts instead of global fetch
 - Fix typecheck

* Fix linux prod and dev builds
 - Improve error handling in tauri.mjs

* Normalize dev deps in workspace
 - Improve linux shared libs setup

* Fix CI and server docker

* Fix windows
 - Remove superfluous envvar

* Attempt to fix server, mobile, deb and release updater

* Attempt to fix deb and mobile again
 - Fix type on deb dependency
 - Enable release deb for aarch64-unknown-linux-gnu

* Github doesn't have arm runners
 - Fix typo in server Dockerfile

* Publish deb and updater artifacts

* remove version from asset name

* update commands

* log release

* Some logs on updater errors

* show updater errors on frontend

* fix desktop ui caching

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
2023-10-10 07:30:56 +00:00
Vítor Vasconcellos bd0a7ff434
[ENG-265] Improve setup scripts (#1368)
* Setup script revamp

* Move toml to dev dep + CI runs the postinstall in debug mode

* Fix windows CI

* chmod +x scripts/setup.sh

* Fix eslint and ts CI

* Remove binstall + Fix trying to read input in CI

* Doesn't need to check pnpm and rust in CI

* Run postinstall script for Clippy CI

* Attempt to fix windows CI not running postinstall
 - Ignore cache when running postinstall on CI

* commited generated config.toml by mistake

* Pass GITHUB_TOKEN to `pnpm i`

* Update archive-wasm + Increase minimum node version to 18.17

* CI: Move rust setup after post-install script

* Revert: CI: Move rust setup after post-install script

* Fix CI, generate dummy cargo config.toml to fix prisma generation

* Fix windows CI

* CI: Fix wrong command

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-09-28 10:03:46 +00:00
Brendan Allan b8e81ad7b6
Increase ram for server release (#1354)
* js has 4gib of ram now

* define env var properly

---------

Co-authored-by: brxken128 <77554505+brxken128@users.noreply.github.com>
2023-09-18 09:36:31 +00:00
Vítor Vasconcellos c86a728a1a
Update server Dockerfile due to crate name change (#1211)
* Update server Dockerfile due to crate name change

* Forgot that the bin name changed too
2023-08-12 21:00:16 +00:00
Vítor Vasconcellos 417d5d3be3
[ENG-811] Fix server Dockerfile + create CI workflow to build and publish docker image (#1012)
* Fix server docker build failing due to apt locking error
 - Replace ffmpeg dep with only the libav* and related required lib (reduce size)
 - Add warning to tauri build command when DMG background is missing
 - Minor rust fmt

* Enable assets feature for server Dockerfile
 - Fix web app not using the correct address in prod
 - Add build step for web assets in server Dockerfile

* Enable repository to be defined in server's Dockerfile
 - Enable server's release workflow to specify which repository to build to allow external PRs to work

* Fix pnpm docker cache not working
2023-06-25 15:16:11 +00:00
Vítor Vasconcellos 1c7855ded6
[ENG-630] Windows ffmpeg + libheif custom build (#871)
* Initial Windows ffmpeg + libheif custom build

* Add build steps for most of ffmpeg deps

* FFmpeg deps and libheif

* Fix libheif build

* Fix libvpx and dlfcn + attempt to fix rav1e

* Rework the whole ffmpeg-windows build system
 - New system based on https://github.com/BtbN/FFmpeg-Builds
 - Add new ffmpeg-windows workflow
 - Rename macos ffmpeg workflow
 - Adapt macos setupt script due to above name change

* Forgot to update update the workflow name

* Strip all libs from debug symbols

* Add docs

* Add libde265 deps, required by libheif
 - Make x265, svtav1 and dav1d as shared deps (used by both ffmpeg and libheif)

* Add missing libheif to Linux setup script

* Fix libx265 build script

* Forgot to point x265 ninja install to the correct directory

* Remove libaom and libsvt-av1
 - dav1d and rav1d are our default AV1 decoders/encoders
 - Quote subshell executions
 - Make libweb shared

* Forgot to remove libaom and libsvt-av1 build steps

* Fix typo

* Try force webp to link against static libs

* Revert libwebp to a static build

* Dumb typo

* Modify windows script to download our ffmpeg build (WIP)

* Fix dlls output folder structure

* Fix dumb mistake

* Remove unused ffbuild_enabled

* Enable core's heif feature on Windows
 - Fix windows ffmpeg build not including the headers
 - Fix windows setupt script incorrect download loagic
 - Implement build_arg to pass which repo ref to use when cloning

* Fix windows setup script

* Fix workflow artifact path
 - Make ffmpeg-windows dockerfile respect the FFMPEG_VERSION env

* Fix Windows setup script incorrect logic for downloading ffmpeg builds

* Error out when workflow_runs is empty

* Fix dumb mistake

* Manually define ffmpeg version for windows script

* Fix ffmpeg windows build extract logic

* Fix prop access in windows setup script

* Revert back to a web request because nightly.link does a redirect before serving the artifact content

* Fix windows setup script

* Do not use nightly.link in Github CI

* Fix windows setup script

* Should finally fix window setup script
 - Update ffmpeg-windows deps
 - Should fix ffmpeg-windows failing to build due to mingw changes in new base image

* Fix libxz failing to build due to doxygen

* Fix windows setup-script not executing till the end

* Fix LASTEXITCODE not defined

* Remove libjxl, deps are not being compiled

* Fix dll and lib copy logic

* Move final copy dll logic to external script

* Use main for libjxl

* Change brotli from stable to main
 - Attempt to fix libjxl

* Attempt fix lib copy again

* Split copy_dll logic to avoid cache burst in docker

* Missing file

* Change how to export build files from shared deps

* Replace rsync with cp

* Fix copy

* Fix dir not existing

* Fix pkgconfig

* Remove superflous files from exported ffmpeg for windows
 - Adjust dav1d to not build tools and examples
 - Adjust windows setup-script to point linker to the libs directory

* Fix dav1d meson config args

* Fix dumb mistake

* WORK PLZ

* Fix .lib file location
 - Strip all dlls

* Formatter
2023-06-10 15:23:37 +00:00