Commit graph

288 commits

Author SHA1 Message Date
Oscar Beaumont f67b7e807a
[ENG-1291] enable/disable P2P & configurable port (#1609)
* it works pog

* UI for when networking is disabled

* Fix Clippy issues

* clippy tings
2023-10-18 03:52:54 +00:00
Oscar Beaumont 1b856848e8
[ENG-1310, ENG-1300] Spacedrop better testing + fix zero-sized files (#1606)
* unit tests save lives

* Support for zero-sized files

* fix: clippy

---------

Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
2023-10-17 06:12:38 +00:00
Oscar Beaumont a7ad24c641
[ENG-625] Spacedrop improvements (#1447)
* Fist fight with the abstraction

* Prevent opening multiple Spacedrop dialogues

* Infinite renders = infinite power

* Comment be gone

* Spacedrop multiple files

* correlate connection id's

* fixes

* thanks Typescript

* Clippy errors + fix cfg's from main
2023-10-17 04:31:42 +00:00
Ericson "Fogo" Soares bfe5a65949
[ENG-1267] Move thumbnails generation away from media processor (#1585)
* New vscode task to start developing

* Updating db in case of library updates just in case

* Done

* Forgot to remove some debug logs

* Rust fmt

* Saving thumbnails processing state on app shutdown

* bruh
2023-10-17 02:09:36 +00:00
Ericson "Fogo" Soares 20fde557d4
[ENG-1225] Put brakes on thumbnail generation for ephemeral locations (#1523)
* Done

* Small nitpick
2023-10-12 04:02:27 +00:00
jake 078490cdd5
[ENG-1143] Media View sort by date taken (#1390)
* clippy -_-

* migrations

* alter the migration so it just renames the `dimensions` field (no db re-creation required)

* remove confusing semver addition for `libheif-sys`

* remove warning on the migration as it's just a rename

* add sort opts for resolution and date image was truly taken

* major serde ckeanup & add epoch_time and pixel_count

* rename, cleanup et optimise

* clippy

* ignore this mess

* bindings

* add explanation to schema

* comment out dt test

* better comment and WIP time

* cleanup rust

* failed timezone attempt

* remove image resolution as a sort by option

* update schema (and rename the dimensions table instead of dropping it)

* just show raw date

* add comments and update bindings

* fix migration hopefully

* fix broken migration

---------

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Jamie Pine <ijamespine@me.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-10-11 00:58:59 +00:00
Vítor Vasconcellos 6fd8bd4ad8
[ENG-1220] Fix pdf thumb crashing app (#1460)
Fix pdf thumb crashing app
2023-10-11 00:47:36 +00:00
Brendan Allan 49cc098f32
[ENG-927, ENG-735, ENG-766] Fix Updater & Tauri 1.5 (#1361)
* custom updater with toasts

* new state management + updated router route

* tauri-specific update route

* ref

* update in prod only

* change 'Install' to 'Update'

* fix tsconfig

* desktop tauri

* remove tauri patch

* tauri 1.5

* tauri 1.5

* use tauri script

* native-deps

* Rework preprep and tauri script to better support tauri 1.5

* Update to tauri 1.5.1
 - Update workspace and apps/desktop dependencies
 - Fix mustache import, @types/mustache is not compatible with ES imports
 - Replace arm64 with aarch64 in machineID, they should be treated the same and this simplyfies the code

* Fix tauri updater not building due to missing key
 - Fix dmg background not being found
 - Generate an adhoc key for tauri updater with it is enabled and the user is doing a prod build

* Fix ctrl+c/ctrl+v typo

* Normalie @tanstack/react-query version through workspace
 - Use undici in scripts instead of global fetch
 - Fix typecheck

* Fix linux prod and dev builds
 - Improve error handling in tauri.mjs

* Normalize dev deps in workspace
 - Improve linux shared libs setup

* Fix CI and server docker

* Fix windows
 - Remove superfluous envvar

* Attempt to fix server, mobile, deb and release updater

* Attempt to fix deb and mobile again
 - Fix type on deb dependency
 - Enable release deb for aarch64-unknown-linux-gnu

* Github doesn't have arm runners
 - Fix typo in server Dockerfile

* Publish deb and updater artifacts

* remove version from asset name

* update commands

* log release

* Some logs on updater errors

* show updater errors on frontend

* fix desktop ui caching

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
2023-10-10 07:30:56 +00:00
jake fb8af0832b
[ENG-1096] Image Conversions (#1364)
* derive `Type` on `*Extension`

* update `image`, `webp` and add `rspc::Error` support to the `sd-images` crate

* rename function for verbosity

* cleanup clippy allows and use scaling fn for svg conversions

* remove dead comment

* specify `generic` image extensions and add fn to enumerate all compatible extensions

* re-exports and inline hot provided method

* another inline on a provided method

* `formatter` -> `handler` and return an explicit error if the extension isn't supported, and add `convert_image()`

* `get_ext()` inlined helper fn

* add mutation for conversion and query for getting all supported extensions

* fix `svg` scaling and generation

* update bindings

* rm old const

* temporary test for all exts

* verbosity and no manual drop

* remove dbg test

* big enum and remove `cur` extension as `image` doesn't support it

* add huge extension enum

* hopefully get path from file path id and location id

* main merge fix

* chore: update toml and lockfile

* security: update webp

* comment entire crypto crate for build times

* add bincode, huge enum and custom ser/de

* Revert "comment entire crypto crate for build times"

This reverts commit 458addaad6.

* lockfile

* theoretically working ser/de

* add svg scaling

* fix pdf matching bug and use options

* move image scaling fn

* major consts overhaul with good ser/de

* disable heif checking for now due to new trait api

* make the generic handler truly generic

* fix pdf types

* fix pdf scaling so that it's 120dpi A4 paper size by default - height/aspect res are maintained

* heavy cleanup

* note to myself

* bindings & update core to use the scaling fn from the images crate (reducing code dupes)

* move vulnerable `aovec` to be a dev dep so it stays out of release builds (it's only used in tests)

* remoev exif crate from the core as it's handled in a sep. crate

* silence a clippy warning and update lockfile

* lower the maximum file size a slight amount

* bindings and minor dep bumps

* Some improvements

---------

Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
2023-10-10 02:57:37 +00:00
Jamie Pine ed751a46d0
[ENG-944] Sidebar UX Improvements (#1270)
* - added local section to sidebar
- added spacedrop screen, showing local peers
- added placeholder network screen
-removed unused swift package
- created a watcher for system volumes to invalidate ui when drives are added/removed

* clouds

* fix more imports

* see more

* open location if volume is location

* gen assets

* remove log

* [ENG-939, ENG-1173] PDF Thumbnails (#1242)

* sd-pdf

* Process PDF blocking render inside a spawn_blocking
 - Load a single global Pdfium instance

* Migrate pdf thumb logic to sd-images
 - Replace block_in_place with spawn_blocking
 - Only load LibHeif once
 - Allow thumbnailer (both indexed and non-indexed locations) to process documents
 - Disable loading pdf viewer in Inspection in favour of loading it's thumbnail

* Try to load pdfium lib from absolute path

* Revert removed import due to rebase

* Small nitpick and some warnings

---------

Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>

* [ENG-888] Media view should show current folder downward (#1437)

* Done but ugly

* layout

* Now with a select

---------

Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>

* add cool folder thing to inspector + stuff

* fix text color

* fix lock

* fix typescript

* fix ts

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-10-09 10:11:23 +00:00
Vítor Vasconcellos 556cf1df63
[ENG-939, ENG-1173] PDF Thumbnails (#1242)
* sd-pdf

* Process PDF blocking render inside a spawn_blocking
 - Load a single global Pdfium instance

* Migrate pdf thumb logic to sd-images
 - Replace block_in_place with spawn_blocking
 - Only load LibHeif once
 - Allow thumbnailer (both indexed and non-indexed locations) to process documents
 - Disable loading pdf viewer in Inspection in favour of loading it's thumbnail

* Try to load pdfium lib from absolute path

* Revert removed import due to rebase

* Small nitpick and some warnings

---------

Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
2023-10-06 20:41:22 +00:00
Vítor Vasconcellos d75a6e9b12
[ENG-1054] libheif for Linux (plus fixes) + Webp update (#1405)
* Move postinstall script to a preprep script
 - Fix libheif crate failing to build with our libheif
 - Rework CI due to postinstall to preprep changes

* Linux heif build script + Update webp

* Fix ctrl+c/ctrl+v bug

* Improve libheif linux script
 - Add support for linux aarch64
 - Add CI workflow to build libheif for linux
 - Some other misc fixes

* Fix libheif CI requires sudo

* Fix wrong path for libheif build.rs override in Windows

* Fix wrong path manipulations in libheif build script

* 🤦

* Use ubuntu-latest in libheif action
 - Specify glib version in target triple to support old distros
 - Fix libheif artifact publishing

* Fix artifact upload path again

* Add musl support for libheif
 - Remove unused files from libheif artifact
 - Add setup logic for libheif in postinstall script

* Build libheif for linux as a shared lib

* Fix meson not building the correct arch
 - Add logic to get git branch from githubs CI env vars

* libheif finnaly works on linux
 - Make spacedrive binary rpath point to where appimage and deb expects our libs to be
 - Add some logic to tauri.js to convince tauri to bundle our shared libs
 - Work-around appimage bundling step breaking sometimes
 - Add logic to handle sigint in tauri.js to ensure we clean up after ourselves
 - Rename postinstall.mjs to setup.mjs
 - Add logic to setup.mjs to point our dev build to our shared libs in linux

* Fix windows dekstop dev
 - Rename setup.mjs to preprep.mjs

* test cache-factory

* Fix preprep script not parsing the cross compilation target triple and always using the host info to download dependencies
 - Fix action env vars not being correctly passed
 - Remove possibility to pass multiple targests to rust action

* Don't compile mobile crates on desktop targets

* Remove cache-factory pull_request trigger

* remove patched tauri cli

* Use git plumbing command to get remote branch name
 - Fallback to reading .git/HEAD if remote name was not retrieved

* fix type

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-10-03 17:31:33 +00:00
jake 1694ff0f3b
[ENG-1165] Add a security policy and notices to the appropriate READMEs (#1409)
* add security policy

* add a link to it in the main and crypto readmes
2023-09-30 08:31:38 +00:00
Ericson "Fogo" Soares 54c1223256
[ENG-1081] Ignore the case of extensions when matching (#1394)
Done
2023-09-28 17:04:07 +00:00
jake d7c4b1ff1d
Feature/target gate HEIF on Linux (#1312)
* feature/target gate the heif stuff

* more gating

* make clippy happy

* `--cfg linux_heif` override and dep bump

* disable `libheif` on windows temporarily too
2023-09-25 11:18:45 +00:00
Oscar Beaumont ae4c3d0e6b
Fix Android bridge + drop IOS and Android projects (#1336)
* Goodbye Android Project, you won't be missed

* fix ignore ignoring Kotlin code

* Backport IOS `Podfile` + move Rust crates into `modules/sd-core`

* Workaround for issue in OpenSSL build script

* Fix IOS

* Migrate IOS bridge into Expo Native Module

* Optimistic update for library creation

* useFrameworks static

* add key to infoplist

* more keys

* fix `Podfile`

* Remove IOS project from repo

* fix mobile ci

* woops wrong place

* clean package.json

* Make Android's Rust code build again

* Add back in `if_watch` patch for IOS

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-09-13 10:07:26 +00:00
Oscar Beaumont 94d8f61de9
Update Rust dependencies (#1314)
* Update most of Rust dependencies

* downgrade notify

* a

* fix Linux?

* Android is painful

* update build.gradle

* The lord Clippy has spoken

* Bruh

---------

Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Ericson "Fogo" Soares <ericson.ds999@gmail.com>
2023-09-12 14:03:25 +00:00
jake f421b6883f
Fix builds and make String -> PlusCode validation more generic (Google support) (#1326)
fix builds and make string -> pc matching more generic, so it works with google's format too
2023-09-11 11:35:09 +00:00
jake e2a0878e7a
[ENG-1068] Plus codes / Open Location Code support (#1324)
* bring in rand deps

* SUPER crude pluscode attempt

* working pluscodes!

* some cleanup

* `encodeURIComponent` for URL escaping

---------

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2023-09-11 04:34:24 +00:00
jake d435141496
[ENG-1084] Correctly format Orientation EXIF data in the UI (#1313)
* impl `Display` and a custom serde serializer

* remove the + as rust already shortens coords to 8SF

* remove manual serde impl and string conversion

* add `formatOrientation` to the `MediaData` part of the inspector

* make requested changes

---------

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-09-08 12:21:32 +00:00
jake 7edcbb15d6
[ENG-621][ENG-1074] Trait-based image conversion overhaul (#1307)
* sd-images crate which will support raw/dng, bmp, etc

* more work on the image formatter

* re-work `sd-images`, add svg support, r/g/b and r/g/b/a HEIF image support (will all be async again soon)

* remove `ImageFormatter`, add note about r/g/b/(a) heif impl

* implement the image formatter

* rename the conversion trait and minor cleanups

* isolate heif feature and major cleanup

* very untested raw support

* change fn name to `from_path` (a lot more idiomatic)

* clean up orientation fixing

* heif is no longer forbidden (linux has good heif)

also all extensions are correctly matched in lowercase

* fix builds, ext matching, feature gating

* attempt to fix svg handling?

* raw attempt, quite a few errors

* add comment

* new (untested) attempt

* remove `raw` stuff for now

* replace `sd-svg` with a `ToImage` `SvgHandler` impl

* add some simple math to appropriately scale thumbnails (and bmp/ico support)

* add comments regarding how the math works for image thumbs

* rename the trait to `ImageHandler`
2023-09-07 14:08:17 +00:00
Oscar Beaumont 5a0d0396d0
[ENG-566] Stream files through P2P (#1282)
* Add range to `SpaceblockRequest`

* Determine if file should be served from P2P

* Feature flag serveFromP2P + fix `ServeFrom` logic

* wip: yeet files over P2P

* Just work, argh

* restore backend feature flags after restart

* cleanup
2023-09-06 07:22:04 +00:00
jake a92d6c2faf
[ENG-1039] Apply hard clippy lints to the entire ffmpeg crate (#1281)
* apply clippy lints to the entire ffmpeg crate in hopes of making it safer

* wording

* revert the prisma changes

* fix typo

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>

* fix another typo

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>

* remove so many `#[must_use]`s

* fix bad merge and hopefully clippy

* clippy please work i beg

* make HEIF_EXTENSIONS always available

---------

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-09-05 07:35:26 +00:00
jake 57ac28526e
[ENG-560] P2P Credits (#1297)
add a README with appropriate p2p credits
2023-09-05 07:23:07 +00:00
Ericson "Fogo" Soares 3b8541ef54
[ENG-1028] Media data for ephemeral locations (#1287)
Introducing new getEphemeralMediaData
Fixing some minor stuff
Running pnpm format

Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
2023-09-04 08:28:45 +00:00
Vítor Vasconcellos b0170b9dba
[ENG-904] Thumbnail for svg (#1220)
* Initial implementation for svg rendering

* Remove unused errors
 - Round up size before allocating Pixmap
 - Re-order some operations

* Finish integrating with Thumbnailer
 - Fix svg thumbnail size
 - Fix incorrect color space while converting tiny_skia::Pixmap to image

* Fix Clippy warns

* Feedback + sd-heif async

* Update implementation to match recent changes to the Thumbnailer
 - Change sd-heif and sd-svg to receive the file data, instead of opening the file internally
 - Update changes to make Thumbnailer and sd-heif more async

* Minor import improvement

* Add missing cfg to gate use of sd_heif only to plataforms that have it enabled
2023-08-31 23:01:45 +00:00
Ericson "Fogo" Soares 39549ef74c
[ENG-953] Zero-byte files occasionally receive a random thumbnail (#1280)
* Better handling empty files

* windows bruh
2023-08-31 01:42:54 +00:00
Oscar Beaumont e591516aff
[ENG-401] Spacedrop betterer (#1273)
* Spacedrop progress on `useP2PEvents`

* Spacedrop Tauri file picker for saving

* Prompt for save location when closing dialog

* quick maths

* Spacedrop progress

* Spacedrop cancellation

* Move accept/cancel dialog to toast

* Toast when rejecting transfer

* yeet

* `useEffect` more like `useBug`
2023-08-30 08:07:23 +00:00
Oscar Beaumont d758977d82
Improve Sync + P2P Integration (#1265)
* Big bruh moment

* whoops

* Less stackoverflowy debug

* stuff

* Fix flawed P2P mDNS instance advertisements

* do sync when connecting with peer

* Sync after pairing

* resync_part2 all the time

* Invalidate all the things

* Invalidate whole React Query on sync event

* emit_messages_flag

* emit_messages_flag

* Backend feature flags + "emitSyncEvents" feature

* Patch `confirm` type cause Tauri cringe

* clippy

* idk but plz work

* bruh

* Fix ComLink bug

* remove log

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-08-29 17:54:58 +00:00
jake c674b7107a
[ENG-288, ENG-601] Media data (image) extraction, thumbnail orientation fix (#1099)
* basic layout

* lock

* add codec to image

* very messy wip

* rm that

* lock and toml

* working perfect exif extraction

* formatting

* migration and formatting

* mostly working

* fix

* set date created properly

* fix tsc

* working media data creation

* fix bad main merge? sorry brendan

* schema, migrations, bindings

* working exif data extraction

* why won't it work

* update migrations

* fix bad merge

* final cleanup

* cleanup migrations

* remove test (that was purely used for testing my code)

* working media data pulling, correct thumbnail orientation

* slightly optimise thumbnail rotation

* rename location to prevent specta clashes

* further improvements (location parsing is still broken)

* fix coordinate parsing i think

* rspc add some todos regarding final cleanup

* further thoughts

* major upgrades

* Some improved handling of errors and async calls

* accordion component

* heavily wip flash refactor

* fix builds all because of a (terrible) merge/move

* annoying missing newline

* i really hate exif

* remove dead code

* further flash progress :D

* docs(media-data): clarification

* minor cleanup

* cleanup and some async removal

* fix location parsing

* remove async (will do proper impl for async eventually) and major cleanup

* big W

* clippy and `FlashMode::Unknown` instead of `Invalid`

* add `NIKON` gps ref support

* comments about gps refs

* commit the submodule -_-

* major cleanup & composite image support

* remove old test image

* major cleanup and easier to use API

* remove old consts

* move `ExifReader` to dedicated module

* Media Data Extractor job and shallow job

* Extracting media data on watcher

* report no exif data on file gracefully

* cleanup errors and doctests

* Merging thumbnailer and media data extractor

* Job name and some strings in UI

* remove reliance on `sd-prisma` within the media data crate

* rename query to be more specific

* custom serializer for `MediaTime`

* tweak to format utc -> naive correctly

* generate migrations

* comment out duration in mobile

* delete test-assets folder

* all optional fields

* fix migrations again

* make requested name changes

* make further requested changes

* remove erroneous files from another wip branch

* updates procedures

* use strings where appropriate

* regen pnpm-lock

* add base layouts for video and audio media data

* use appropriate data types in schema and add audio media data framework

* make requested changes

* general cleanup and renaming of enum

* cleanup media data api

* rename media metadata type

* finishing touches

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-08-29 17:02:55 +00:00
Aditya 08ba4f917a
Extending QuickPreview functionality with additional filetype support (#1231)
* added some files `standard` mime type

* Used `TEXTViewer` Component to show Code Preview

* Update Thumb.tsx

* added `prismjs`

* removed unnecessary comment

* `CODEViewer` Component for Syntax Highlighting

* formatting

* using **Atom** Theme for `Prism`

* merge text/code viewers & bg-app-focus for prism

currently calling onError and onLoad without an Event argument
that should change but i'm not really sure what to do there

* removed unused imports

* Update index.ts

* `TEXTViewer` to `TextViewer_`

* `TextViewer_` to `TextViewer`

* Don't highlight normal TextFiles

* clean code

* `TEXTViewer` to `TextViewer`

* using tailwind classes more

* doing things correctly.

* installed `prismjs` in interface

* using own scroller

* Update Thumb.tsx

* Add an AbortController to the fetch request
 - Fix onError and onLoad calls
 - Format code

* Fix onError being called when request was aborted due to re-render
 - Fix Compoenent re-rendering loop due to circular reference in useEffect
 - Remove unused imports

* Improve text file serving and code syntax highlight
 - Implement way to identify text files in file-ext crate
 - Do not depend only on the file extension to identify text files in custom_uri
 - Import more prismjs language rules files
 - Add line numbers to TextViewer when rendering code

* Clippy and prettier

* Fix reading zero byte data to Vec
 - Improve empty file handling

* Expand code highlight to more file types
 - Fix 10MB when it should be 10KB
 - Add supported for more code and config files extensions to sd-file-ext
 - Add comlink and vite-plugin-comlink for easy js worker integration
 - Move Prismjs logic to a Worker, because larger files (1000+ lines) where causing the UI to hang
 - Replace line-number prismjs plugin with our own implementation

* Fix uppercase extension name

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: pr <pineapplerind.info@gmail.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-08-29 10:47:04 +00:00
Oscar Beaumont 7bf0c0ae4a
[ENG-994] Minor P2P fixes (#1238)
* mdns advertisement on detection

* fix bug in P2P discovery

* Add `P2PEvent::ExpirePeer`

* React properly

* Consistent tracing versions

* better logger config

* Reduce excessive logging from P2P

* Fix panic log hook

* Remove `dbg`

* Fix bug in thumbnailer remover

* wip: Connected peers in UI

* `Sync` is large pain

* init services after library load

* improve error handling of logger

* Sync protocol shutdown

* Add "updater" feature

* fix commands bindings

* nlm state debug query

* nlm debug status

* Make TS happy

* Add `RemoteIdentity` to libraries debug info

* Improve debug data presentation

* Among us level sus

* minor fix panic hook

* Fix EOF issue

* fix instance not being added on pairing

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Ericson "Fogo" Soares <ericson.ds999@gmail.com>
2023-08-27 06:41:26 +00:00
Vítor Vasconcellos b199c03c1e
Always enable tauri update + Misc Clippy warns fix (#1250)
Always enable tauri update
 - Fix some misc clippy warnings
2023-08-25 22:08:34 +00:00
Jamie Pine b8a9d91d8a
[ENG-1001] Config Kind (#1248)
* config

* cleanup

* cleanup

* clsx
2023-08-25 16:27:54 +00:00
Vítor Vasconcellos 7a7d01307e
Formatting (Prettier + Rustfmt 1.72) (#1246)
Formatting
2023-08-24 20:34:52 +00:00
Ericson "Fogo" Soares 28d106a2d5
[ENG-862, ENG-921] Ephemeral locations (#1092)
* Some initial drafts

* Finising the first draft on non-indexed locations

* Minor tweaks

* Fix warnings

* Adding date_created and date_modified to non indexed path entries

* Add id and path properties to NonIndexedPathItem

* Working ephemeral location (hardcoded home for now)

* Fix UI for ephemeral locations

* Fix windows

* Passing ephemeral thumbnails to thumbnails remover

* Indexing rules for ephemeral paths walking

* Animate Location button when path text overflow it's size

* Fix Linux not showing all volumes

* Fix Linux
 - Add some missing no_os_protected rules for macOS
 - Improve ephemeral location names

* Remove unecessary import

* Fix Mobile

* Improve resizing behaviour for ephemeral location topbar path button
 - Improve Search View (Replace custom empty component with Explorer's emptyNotice )
 - Improve how TopBar children positioning

* Hide EphemeralSection if there is no volume or home
 - Disable Ephemeral topbar path button animation when text is not overflowing

* minor fixes

* Introducing ordering for ephemeral paths

* TS Format

* Ephemeral locations UI fixes
 - Fix indexed Locations having no metadata
 - Remove date indexed/accessed options for sorting Ephemeral locations
 - Remove empty three dots from SideBar element when no settings is linked

* Add tooltip to add location button in ephemeral locations

* Fix indexed Locations selecting other folder/files in Ephemeral location

* Minor fixes

* Fix app breaking due to wrong logic to get item full path in Explorer

* Revert some recent changes to Thumb.tsx

* Fix original not loading for overview items
 - Fix QuickPreview name broken for overview items

* Improve imports

* Revert replace useEffect with useLayoutEffect for locked logic in ListView
It was causing the component to full reload when clicking a header to sort per column

* Changes from feedback

* Hide some unused Inspector metadata fields on NonIndexedPaths
 - Merge formatDate functions while retaining original behaviour

* Use tauri api for getting user home

* Change ThumbType to a string enum to allow for string comparisons

* Improve ObjectKind typing

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2023-08-23 17:26:07 +00:00
Brendan Allan 98f763134e
lock libheif-sys version (#1183) 2023-08-07 10:12:20 +00:00
Oscar Beaumont 9e919f9909
Library "state" (#1160)
* `Library` -> `LoadedLibrary`

* Move `ThumbnailRemoverActor` onto `NodeServices`
It should either be on `Library` or `NodeServices` none of this in-between stuff.

* small > big files

* prune imports

* isolate cringe

* it helps to test the code

* `Vec` -> `HashMap` - faster lookups

* mpscrr

* minor fixes

* fix language

* `!Sync` :(

* lets goooooo

* removed a comma

* fixes

* fix desktop

* lol, no one uses it

* Clippy
2023-08-05 14:42:21 +00:00
Oscar Beaumont 6c9e08540f
Networked sync (#1133)
* temp remove discriminator from tunnel

* i'm lost :(

* Working `Tunnel` + move sync stuff back into p2p

* sync module

* `NetworkedSyncManager`

* Move sync stuff off `P2PManager`

* Test `SyncMessage`

* Library edit + delete hooks

* stable `identity` column

* fix

* plz work

* p2p is hard

* sync identities post pairing

* Remove `p384`

* a couple improvements

* comment out ReqRes

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-08-02 11:49:50 +00:00
John Smith 58b7a96860
fix: unknown feature proc_macro_span_shrink (#1143)
fix: https://github.com/rust-lang/rust/issues/113152
2023-08-02 06:02:06 +00:00
Vítor Vasconcellos 7e4ed2b547
[ENG-767, ENG-916] Improvements and fixes for Open/Open With (#1082)
* Normalize PATH and XDG envvars on Linux

* Fix normalize_xdg_environment

* Replace custom Desktop file parsers with Glib

* Fix AppImage env influencing external apps
 - Normalize GStream plugin path

* Fix macos pulling linux deps

* Attempt to fix gnome apps failing to launch
 - Fix incorrect logic in `normalize_pathlist`

* Ensure empty envvars are not set

* Revert AppImage ovewritten GTK specific variables
 - `normalize_pathlist` now prefers entries with less priority when dealing with repeated entries, this is not compatible with the default behavior, but it is a more sane approach IMHO

* Remove 32-bit libs from release CI build host

* Remove 32-bit from github runner 2 attempt
 - Remove deprecated vscode config

* Remove libc6-i386

* [ENG-916] Implement `Open With` logic compatible with macOS < 12

* Add some missing gstreamer deps in Linux

* Replace opener with Glib API

* Fix reveal opening file instead of dir
 - Improve Open With logic in Windows
 - Expose functions to test if app is in a flatpak, snap or appimage

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-24 17:36:00 +00:00
Brendan Allan 043b607ad4
CRDTOperation receiving (#1122)
* operation receive + compare

* cleanup + deduplication

* operation receive + compare

* cleanup + deduplication

* sync route + operation grouping

* tag assign sync

* proper relation support in sync debug page

* migration

* separate core-sync + utils crates

* separate p2p event loop from manager

* cleanup library handling

* clippy

* feature gate sync messages properly

* make migration not add required field
2023-07-24 15:26:00 +00:00
Vítor Vasconcellos 8d6a060343
[ENG-914] Fix compilation in Linux arm64 (#1104)
Remove unecessary casts in movie_decoder that were breaking compilation in linux arm64

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-07-17 17:08:09 +00:00
Brendan Allan 0a1d721e58
Use codegen for ModelSyncType exec (#1098)
* use codegen for ModelSyncType exec

* folder client format

* hande create operation properly
2023-07-15 04:22:04 +00:00
Oscar Beaumont cf39f8dbcc
[ENG-759] P2P Cleanup (#1062)
* less stupid name

* yeet

* awaiting futures is kinda important lol

* no-async

* more proto stuff

* cleanup

* move it round

* None of my homies like broadcast

* minor

* do the shuffle

* restore after force push

* reusing `sysinfo::System` as intended

* fix lol

* remove `node_id` from `Volume`

* Remove `node_local_id` from `Library`

* Remove `Job` to `Node` relation

* feature flags be like

* press save 4head

* remove `Location` -> `Node` relation

* `volume.rs` to `volume/mod.rs`

* yes

* add `Instance` model and deprecate `Node` model

* pairing is better now

* Pretty code

* thinking in code

* wip

* What if Observables but in Rust?!

* Observables aren't it + `useP2PEvents` hook

* more around some jsx

* Trade offer: bad code for working pairing?

* Unit test pairing protocol

* fix everything up

* corrections

* Have you got a moment for our lord and saviour Clippy

* tsc --fixMyCode

* Prisma being wacky

* Config files being f'ed up

* broken config after migrating

* Zed being Zed

* Argh

* cliipzy

* rewind

* Fix truncate logic

* wip: instances in peer metadata

* Rethink instance ids

* fix

* whoops

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-12 06:23:30 +00:00
Niklas Wojtkowiak c877c03b63
Add functionality to some Overview Catrgories (#1090)
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-12 06:01:30 +00:00
Vítor Vasconcellos caaacc8f92
[ENG-866, ENG-535] Thumbnailer fixes (double-free, thumbnails for portrait videos) (#1059)
* Fix double-free during MovieDecoder Drop
 - Fix broken thumbnails for portrait videos
 - Fix some inconsistencies between our MovieDecoder impl and ffmpegthumbnailer impl
 - Make sd-desktop-macos use the correct macOS deployment target

* fmt

---------

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-06-30 11:00:04 +00:00
Ericson "Fogo" Soares dcd071d229
[ENG-850] Wrongly labelled .ts files (as video) (#1052)
* Fixing conflits of mts code files showing as video

* Introducing a reidentify_objects param on fullRescan

* Fix job output metadata in case of completed with errors
2023-06-29 02:58:33 +00:00
jake 3b86c99d74
[ENG-828-FIX-2] Actually generate thumbnails (#1033)
damn binary operations
2023-06-27 10:55:43 +00:00
jake 9e206b3cc9
[ENG-828-FIX] Make trivial changes to the previous ENG-828 merge (#1032)
remove old error and cleanup
2023-06-27 10:05:47 +00:00
jake 0dae7f7a63
[ENG-828, ENG-633] Fix corrupt videos force crashing the app (#1028)
* fix corrupt videos crashing the app

* fix typo

* self-proclaimed genius
2023-06-27 05:04:40 +00:00
Oscar Beaumont 0f7a669e5a
[ENG-732] Pairing by library (#943)
* library keypair

* allow opening stream to non-connected node + more

* library identity

* fix types

* fix maybe undefined

* don't forgor migrations file

* library manager inside p2p manager

* rename

* `NodeInformation` struct

* node info exchange

* fill in info

* streamify tunnel

* use tunnel for p2p events

* libp2p is annoying + stop leaking private key's

* Clippy cleanup
2023-06-19 05:13:30 +00:00
Brendan Allan d683d22c82
Remove owned sync (#967)
* remove owned sync + cleanup

* no more atomic records

* byebye owned

* remove owned ops from schema
2023-06-18 23:45:06 +00:00
Oscar Beaumont fd1afd8500
[ENG-568] Spacedrop Progress (#956)
* `Transfer` struct

* progress moment

* fixes
2023-06-18 08:40:51 +00:00
Oscar Beaumont 3d224e02ad
[ENG-675] Spacedrop is 0kb (#942)
* fix sd-web

* pain

* unit tests are pog

* Ignore `sdserver_data2`

* Update mod.rs
2023-06-15 00:32:54 +00:00
Ericson "Fogo" Soares e693c7a542
[ENG-381] Update context menufs actions to support multiple source (#927)
* Generalizing filesystem jobs to accept multiple files at once

* Some small changes and fixing front

* Enable rename route to replicate Finder's behavior

* Assign tag to multiple objects at once

* Remove from recents accepting multiple files

* Fixing some warnings

* Adding multiple files feature to Open and Open With

* Conditional stuff for macos

* Generating commands.ts and other minor warnings

* Rust fmt

* TS typecheck

* Rust format and TS typecheck

* Requested changes

* Requested changes

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-06-14 23:00:28 +00:00
Oscar Beaumont 049966d848
[ENG-674] fix invalidation (#891)
* reenable the invalidate requests system

* a tester for the invalidate system

* watch locations in Library::load

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-05-31 08:55:46 +00:00
Oscar Beaumont e4696c7303
[ENG-667] Fix Clippy CI (#887)
* plz just work

* no shot

* just disable Tauri notifications

* Tauri is weird AF
2023-05-30 17:11:37 +00:00
Oscar Beaumont 5f71e0c1f6
[ENG-546] Improve error handling (#802)
* log to disk

* remove some unwraps

* panicless

* some p2p error handling

* clippy moment

* Fix `<ErrorBoundary />`

* open logs button

* 39 to 0

* fix types

* update deps and comment out broken tests

* clippy

* more clippy

* upgrade rimraf - https://github.com/isaacs/rimraf/issues/259

* regen broken lockfile

* update `notify` and update `commands.ts`

* more clippy (pls work)

* allow deprecated for p2p (hopefully temporary)

* a

* upgrade deps for p2p

* do betterer

* do it correctly

* remove unused import

* improve core startup error + bc keypair

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: brxken128 <77554505+brxken128@users.noreply.github.com>
2023-05-30 09:24:05 +00:00
Brendan Allan 6710f05251
move prisma codegen into sd-prisma (#876) 2023-05-29 08:37:58 +00:00
Utku dbec988147
[ENG-362, ENG-476, ENG-503] Next.js (#817)
* update landing packages & tailwind

* move images

* refactor to nextjs

* doc stuff

* rename markdown to mdx

* working docs

* Add math

* fix pnpm-lock

* Code highlighting

* fix types

* fix more stuff

* Notice component and removed slot

* delete editorconfig eslint rule

* pnpm-lock

* some fixes

* fixed types (pls)

* bump ui's typescript too

* commit next-env.d.ts

* fix doc links

* Use next/head instead of react-helmet & rehype-external-links

* fixes

* fix unsub from list

* add trim() to docs.excerpt

* trim doc title too

* fix titles

* replace <a> to Link

* hide .contentlayer & .next from vscode search
2023-05-21 13:11:03 +00:00
jake fb95c6fec2
HEIF Fixes and Improvements (#838)
* limit HEIF to macos-only because of ubuntu's *archaic* `libheif` version

* drop png entirely, go from raw `[r, g, b...]` -> `DynamicImage`
2023-05-20 22:08:04 +00:00
Jamie Pine 7e4c58a91c
[ENG-603] Support HEIC images (#834)
* working HEIC thumbnails

* better error handling

* better error handling and some cleanup

* fix type of maximum size, and clippy

* better extension support, WIP quick preview, better image resolution

* remove unnecessary x86_64 macos rustflags

* add correct rustflags to setup script

* add fedora libheif deps

* debian libheif deps

* arch libheif too

* add sd-heif as a dep and feature gate it

* enable aforementioned feature in tauri only

* add URI support for heif/heic (quick preview still won't work)

* correct feature gating on everything

* dedicated sd-heif crate

---------

Co-authored-by: brxken128 <77554505+brxken128@users.noreply.github.com>
2023-05-20 06:40:37 +00:00
jake 1aa8826398
SPDX license and workspace inheritence for re-used metadata (#833)
use workspace inheritence for license, edition and repo URL
2023-05-20 01:52:55 +00:00
Oscar Beaumont 01e8716b94
[ENG-559] Spacedrop move to spaceblock (#824)
* Move Spacedrop to Spaceblock

* bruh

* unit test moment

* plz work

* usize != u16 man

* tsc --quite

* clippy
2023-05-18 19:02:44 +00:00
Oscar Beaumont dd3a0ff689
P2P Stream ids (#630)
inbound stream id
2023-05-18 06:54:20 +00:00
Jamie Pine e8895c1a05
[ENG-581, ENG-597, ENG-600] Overview categories (#779)
* added base UI for categories on overview

* update core

* cleanup ui

* Inspector default view if nothing is selected, explorer takes child components, hidden menu tweak if no items are sm:flex

* wip

* somewhat functional

* scroll

* category fixes

* clean category bar

* added config store + made toolbar available on all explorer screens

* clean up overview.tsx

* added counts

* fix inspector bug

* add support for favorites
+ add book extension support

* refactor into smaller components

* Some small rust nitpicks

* fix camel case location_type

* Rust fmt

* fix typescript CI

---------

Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Co-authored-by: nikec <nikec.job@gmail.com>
Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-05-17 20:34:05 +00:00
Ericson "Fogo" Soares dda7516980
[ENG-536] Improve indexer for huge locations (#781)
* WIP materialized_path abstraction revamp

* Optimizing indexer rules loading

* Using a better serialize impl for indexer rules

* New interruptable and faster Walker

* WIP new indexer

* WIP first success compiling after breaking the world

* Fixing some warnings

* Handling some lifetime issues in the walker

* New job completed with errors feature

* Introducing completed with errors to indexer
Removing IOError variant from JobError and using FileIOError instead

* Rust fmt

* Adding missing job status

* Better ergonomics to IsolatedFilePathData
Conversions from db's file_path kinds
Keeping original's relative path data to better conversion to OS's path
First unit tests

* Testing and fixing parent method

* Some error handling

* Rust fmt

* Some small fixes

* Fixing indexer rules decoding

* Bunch of small fixes

* Rust fmt

* Fixing indexer rules

* Updating frontend to new materialized_path format

* Trying to fix windows CI

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-05-16 14:15:51 +00:00
Vítor Vasconcellos cf482f5621
[ENG-270, ENG-213] Stabilize release pipeline (#768)
* Attempt at building our own ffmpeg dylibs for macOS
 - Create dockerfile to setup osxcross and required host dependencies for cross-compiling ffmpeg for macOS
 - Create script to setup required macOS dependencies and build ffmpeg for both x86_64 and aarch64

* Improve dockerfile to better use docker's cache
 - Attempt at fixing arm64 build (still not there yet)

* ARM64 sucessfully compiles \o/
 - Use clang for everything (e.g. linker, assembler, ...)
 - Remove -lc++ because it is now handled internally thanks to a patch
 - Apply 3 open PRs to osxcross as external patches to solve some of our problems
 - PR 180 reduces the macports dependencies size by prefering non universal libs for x88
 - PR 314 adds the -lc++ flag for macOS SDKs >= 11.1
 - PR 379 fix macports pulling incorrects deps for SDK11 and adds support for SDK13 (future proofing, as it does nothing for us rigth now)

* Fix relink and copy logic in ffmpeg build script
 - Ensure ffmpeg and dependencies dylibs are correctly relinked and copied over to the output directory
 - Copy includes from ffmpeg and dependencies to allow for dev building against these dylibs
 - Set up symlinks for all dylibs to the common `lib` directory
 - Remove superfluous linker and compiler options
 - Add `-headerpad_max_install_names` linker option to accommodate the required relink changes
 - Remove `--disable-fft` for the arm64 build, as it doesn't seem to be required nowadays
 - Allow `swresample` and `avdevices` to be compiled as they seem to be expected to exist by `ffmpeg-sys-next`
 - Disable ffmpeg program compilation
 - Fix dylibs id to match their new expected location
 - Fix ffmpeg dylibs inter-linkage not being updated
 - Add custom patch for osxcross macports to load already downloaded files from cache
 - Remove the requirement for specifying the darwin version in the Dockerfile
 - Enable docker cache for macports
 - Move arm min macOS version ARG to the top

* Attempt at building ffmpeg with github actions
 - Split osxcross setup into it's own Dockerfile to allow exporting it as a image to Dockerhub
 - Create a github action to build ffmpeg
 - Add osxcross patch for PR 372 for future proofing (Add support for macOS SDK 13.1)

* Remove incompatible step name

* Change FFMPEG_VERSION to string to avoid yaml weird type casting

* Start to use our ffmpeg build in the macOS
 - Adjust the macOS setup script to download our ffmpeg build, also replace installing protobuf compiler from brew with grabbing it from it's own repository
 - Some misc improvement to the *nix setup script
 - Fix ffmpeg build script not copying static libs (for when we may want to statically link ffmpeg)
 - Fix ffmpeg build script not taking into acount possible libraries that are multiple paths deep in /lib
 - Update CI to actions/checkout@v3, v2 is deprecated
 - Update ffmpeg build CI to not xz the artifacts, as github already zips it anyway

* Fix macOS setup not finding the ffmpeg artifact due to a jq script mistake
 - Comment out embeeding static libs in ffmpeg artifact for now, as they take too much space
 - Remove incorrect dockerfile settings from editorconfig

* Fix protobuf download in macOS
 - Fix Framework directory creation not following script's dir but cwd
 - Improve download logic to avoid being rate limited on failures
 - Return to using xz for ffmpeg build, as it saves quite a bit of space

* Replace `OSTYPE` with `$(uname)` for a more standardized way to check which os is running
 - Fix mobile check using `==` instead of `=` in test command

* Fix syntax error in bash 3.x (which is the default in macOS)
 - Fix jq not downloading all deps and being placed in the incorrect directory

* Fix incorrect install_name_tool arguments
 - Remove leading './' when referencing ffmpeg libs during build

* Fix protoc not being executable
 - Fix some error messages

* jq throw input when it is invalid
 - Enable debug log in setup script when running in CI

* Add a dev.js build script for desktop app
 - Remove dll copy from setup-system.ps1, now done by dev.js
 - Minor rework to build.js to improve signals and stdio handling

* Configure cargo to use FFMPEG_DIR and PROTOC envvars on macOS

* Pass GITHUB_TOKEN to macOS setup script in CI

* Replace simple curl with gh_curl in github request

* Fix some ctrl+c/ctrl+v typos

* Add missing ffmpeg deps
 - Fix missing symlinks for ffmpeg libs in the output_dir/libs
 - Change arm64 minimum macOS target to 11.2 due to libbrotli only being available precompiled to this version
 - Add more options to ffmpeg configure in an attempt to reduce its size + improve performance

* Rework ffmpeg build script to create a FFMpeg.framework instead of loose dylibs
 - Disable unused postproc and enabled missing required protocols in ffmpeg configuration
 - Adjust Dockerfile and setup-script to handle the new FFMpeg.framework

* Fix setup script incorrect linking logic for FFMpeg.framework

* Forgot to remote the leading path in the lib/header linking logic for FFMpeg.framework

* Enable size optimization for ffmpeg
 - Remove Frameworks directory before setting it up in setup-system.sh

* Revert lib id to use `@executable_path/../Frameworks/` instead of `@rpath`
 - `@rpath` breaks dev builds
 - Add logic to create less versioned versions of dylibs in setup-system script, due to it being required for compiling ffmpeg-sys-next (not required during runtime, just for compilation)
 - Add scripts/deps to gitignore

* Fix ffmpeg build script not copying unversioned symlinks to FFMpeg.framework
 - Remove unecessary unversioned symlink logic from setup-system
 - Add new build scripts for dev and prod that setup the environment and tauri to correctly point to FFMpeg.framework
 - Rework windows setup-system script to follow the same logic as in macOS setup-system

* Attempt at fixing windows setup script
 - Add powershell editorconfig

* Attempt to fix Windows CI

* Fix some mistakes in the build scripts
 - Replace toml lib with something more used and maybe better?
 - Attempt to fix Windows CI

* Fix some more problems with the build script

* Another attempt at fixing windows CI

* Some more fixes for the windows setup script

* Add build.js step to release CI
 - Implement patch only mode for build.js when running under CI
 - Implement appeding extra env in cargo config in env.js
 - Only run windows dlls copy in dev in env.js
 - Another attempt at fixing windows CI

* Maybe this will make the windows CI happy?

* Windows CI why dont you work, whyyy?

* Try to apease the Windows CI Gods
 - Disable updater build in macOS, due to tauri-apps/tauri#3933

* Fix build.js extra env vars
 - Uncomment parts of setup-system.ps1 to check if it keeps working

* Uncomment another section and see if windows CI will complain

* uncomment some more of setup-system.ps1

* Use Start-Process for robocopy instead of executing it directly
 - Only raise an error if robocopy exists with a status code >=8
 - Revert build.js macOS specific bundles change
 - Disable updater for now

* Remove updater from cargo.toml

* Add -PassThru to Start-Process to ensure we get and Process obj back

* Comment out more of updater

* Improve documentation for both of the docker containers
 - Fix a harcoded install_name_tool call in the ffmpeg-build-macos.sh script

* Configure tauri to copy ffmpeg dlls to windows prod version
 - Fix windows dlls copy on dev
 - Stop replacing tauri.conf.json, except on CI, for prod builds and use a tauri.conf.patch.json for any required changes
 - Don't unset macOSPrivateApi on prod build, we need it for the transparent background
 - Fix dev.js not exiting when the spawned command was over

* Fix windows not find protoc
 - Fix missing import in env.js

* Fix Windows Dlls copy logic for prod build
 - Fix dev script failing due to missing env
 - Implement error handler to spawned process

* Format + Fix pnpm-lock

* Fix video thumb generation failing on Windows due to bad path to CString conversion logic

* Fix mobile build
 - Greatly improve windows setup-script error handling
 - Install LLVM with winget instead of downloading exe from github

* Improve CI a bit
 - Fix mobile CI not using github token to avoid being rate limited

* Fix pnpm not accesible to actions/setup-node

* Skip pnpm check on CI

* Fix pnpm skip not working
 - Fix pnpm cache check missing a step

* Only lock action to major versions

* CI really doesn't like running robocopy directly

* Attempt to build our own patched tauri bundler

* Trigger new workflow

* Fix attempt for patched tauri action tests
 - Disable cache for patched tauri action
 - Enable package publishing for patched tauri action

* Attempt fix patched tauri publish

* Remove commented code from patched tauri workflow
 - Replace deprecated `actions-rs/toolchain` with `dtolnay/rust-toolchain`

* Fix url patching logic for publishing step of patched tauri package

* Fix prepublishOnly command for patched tauri package

* Adjust patched tauri package package.json

* Also patch the specific os/arch sub packages

* Instruct napi to stop doing github releases

* Patch package.json befor build

* Re-enable updater
 - Patch tauri cli to workaround a tauri bug that prevented use from bundling frameworks in the updater
 - Update tauri/api to 1.3 and tauri/cli 1.3.1
 - Add target triple in the release artifacts name
 - Use ubuntu-20.04 instead of ubuntu-latest
 - Configure release workflow to build to all supported platforms
 - Replace `tauri-apps/tauri-action` with simply calling `pnpm desktop build`
 - Simplify tauri-patched-cli-js to only build the required macOS binaries
 - Unify build.js and dev.js in a tauri.js build script, which now acts as an all around wrapper for the tauri/cli

* Restore incorrectly removed step from `tauri-patched-cli-js`

* Disable auto-run of tauri-patched-cli-js
 - Update setup-system.sh with the latest tauri-patched-cli-js artifacts

* Forgot to mkdir bin

* Fix pnpm-lock.yaml not being up to date
 - Remove pnpm deps caching

* Fix variable name conflict

* Enable the updater build conditionally
 - Enable release workflow to test it

* Fix artifacts not being published
 - Change macOS on arm64 minimumSystemVersion to 11.2
 - Commented out rust cache to check if this is the culprit to the low memory problem on windows runner
 - Add some debug log to tauri.js

* Fix release workflow artifact uploading

* Specify which bundles to build on each platform
 - Attempt fix apple arm build
 - Don't include internal deb archives in linux updater artifact

* Attempt to fix apple arm build
 - Fix dmg background not applying

* Fix incorrect semver usage

* Patch swift-rs to see if that fix apple arm64 release build

* Update swift-rs submodule
 - Remove unused deps lodash.merge

* Configure so only release workflow runs without cache
 - Improve tauri.js logs

* Ignore `crates/swift-rs`

* Revert "Ignore `crates/swift-rs`"
- Remove `crates/swift-rs` before running `cargo fmt`

This reverts commit 851bd84373.

* Github CI input/output are always string
 - Attempt to fix Windows Clippy

* Fix CI syntax error

* Fix mobile CI
 - Disable pnpm deps caching
 - Disable rust targets caching on Windows
 - Configure cache factory to run for all targets

* Remove `crates/swift-rs`

* Attempt fix Mobile CI
 - Enable cache factory run in pull_request to test it
 - Specific more path for the CI run to ignore
 - Specify path that will trigger the cache factory run
 - Some cleanup in the setup-pnpm action

* Remove restore-cache options
 - Add prefix key to cache-factory
 - Fix LLVM cache-hit check
 - Add libheif to FFMpeg.framework

* Remove submodule clone from actions

* Add fake deps to ffmpeg build dockerfile
 - Fake install deps that aren't really necessary to build ffmpeg
 - Fail ffmpeg build script if it can't find a macports deps
 - Copy libheif headers to Framework

* Remove automatic trigger for relase and pull request trigger for cache-factory actions

* Remove pnpm setup from Cache factory
 - Cache factory is exclusivly for rust deps caching, no need to setup Node and pnpm with it

* Re-enabled fail-fast in Github CI
2023-05-15 18:02:56 +00:00
jake 207c82bfeb
[ENG-590] Uppercase file extensions are "unknown" (#801)
match against the lowercase extension
2023-05-07 17:58:19 +00:00
Oscar Beaumont 2571e3b275
Upgrade rspc TS (#775)
* finally

* should be working?

* fix types

* fix types

* Wouldn't it be nice if Metro would just work

* idk

* try harder Metro

* potentially fix bundling issues

* idk, maybe fix it?

* fix metro

* update podfile.lock

* bruh

* bruhz

* tailwind is drunk again

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-05-04 06:21:42 +00:00
Brendan Allan 7cee2b2651
update prisma (#773)
* update prisma

* idk
2023-04-30 23:03:23 -07:00
Brendan Allan e1c626c0fa
Remove all field name string literals (#772)
remove all field name string literals
2023-04-28 21:59:11 -07:00
Brendan Allan 82845a8388
Upgrade Prisma (#770)
upgrade prisma
2023-04-28 17:56:08 +00:00
Oscar Beaumont 3c440783b3
Spacedrop (#761)
* A hint of file drop

* backport from #671

* accept/reject Spacedrop working

* final cleanup

* Rename spacedrop.tsx to Spacedrop.tsx

* Update index.tsx
2023-04-27 04:41:33 +00:00
Oscar Beaumont 85d0ddf6d3
rspc 0.1.3 prep (#756)
* Upgrade rspc + Specta + PCR

* fix jsonrpc executor

* fix invalidate excessive logging

* fix `Protected` type

* fix mobile

* clippy

* lol
2023-04-25 15:14:52 +08:00
jake 37b8d64642
[ENG-490] Add support for late key manager setup (#740)
* support for late key manager setup and remove password from onboarding

* mobile

* delete master-password.tsx

* remove secure temp keystore

* remove more km_config artifacts

* fix builds

* Fix most of the UI, there still some minor bugs

* Workaround portalled closing Popovers

* Restore search in core/src/api/mod.rs

* Fix Password eye icon closing Popover on click

* Fix keymanager setup submit button spinner not centered
 - Fix keymanager setup missing logic for disabling form
 - Adjust keymanager tab roundness to match app style

* Restore .gitignore

* Restore .prettierignore

* fixerino tests

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-04-22 09:10:21 +00:00
jake 4315cee147
[ENG-516] Prevent copies/cuts to the same source/dest (#748)
* add basic UI protection to stop cut/copies to the same path

* add rust src/dest protections

* fix pasting logic

* fix path canonicalization

* skip paths instead of overwriting them

* Using non-blocking ops and fixing some warnings

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-04-22 02:11:59 +00:00
Brendan Allan 469a3831dc
Explorer search (#738)
* basic file path search

* cleanup

* render explorer in search page

* update materialized path docs
2023-04-21 14:49:46 +00:00
jake 288f4947e1
remove int-enum (#721) 2023-04-19 01:33:32 +00:00
Vítor Vasconcellos 03eb27e91d
[ENG-469] Make Prettier and ESLint work together (#706)
* Make Prettier and ESLint work together
- Resolve conflicts between Prettier and ESLint regarding indentation and Tailwind rules order
- Add `.editorconfig` to standardize basic formatting options across tools and editors
- Add `.gitattributes` to hide `pnpm-lock.yaml` in `git diff` output
- Include EditorConfig in the recommended extensions for VSCode
- Replace some instances of `pnpm exec <command>` with `pnpm <command>`
- Remove superfluous Tauri config for Linux

* Revert Prettier changes (it was working correctly before)
 - Update ESLint to read Tailwind config from absolute path
 - Remove redundant Prettier dependency from subprojects
 - Specify the source folder for the lint script in subprojects

* use mobile's tailwind config with eslint

* pnpm format + pnpm lint:fix

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-04-14 21:21:21 +00:00
jake 39e92cc7e7
update to ffmpeg 6.0.0 (#697) 2023-04-13 14:10:23 +00:00
Oscar Beaumont 3d6afe5447
[ENG-414] P2P Shutdown (#688)
* `Event::Shutdown` in P2P system

* Mdns shutdown broadcast + await shutdown

* Fix mdns service name & prevent poll after shutdown
2023-04-11 15:05:50 +00:00
Oscar Beaumont 4c0e341e55
Add MetadataManager into sd-p2p (#686)
* Add `MetadataManager` into sd-p2p

* Add `update_metadata` helper to P2P manager

* Praise Clippy our lord and saviour

* change mdns advertisement log level

* Casually broke Linux
2023-04-11 10:20:14 +08:00
Ericson "Fogo" Soares b346e7ac52
Eng 372 location awareness improvements (#612)
* Complying with a pedantic Clippy

* Solving duplicated directories events
When creating a directory through MacOS's Finder, for some reason
FSEvents receives 2 Create Folder events that we have to handle

* Handling moving to trash bin and restoring on Mac
Still missing the feature to restore a directory and its children

* Now handling creation of empty files on MacOS

* Enabling restore of directories and its children

* Now working: moving a directory to another
inside the same location

* Now Indexer also remove file_paths not on fs

* Enabling multiple file moves on location watcher

* Fix Windows conditional compilation issues

* Fixing cas_id generation and bumping some deps

* Many Windows specific improvs and some refactors

* Rust fmt

* Using conditional compilation on extract inode function

* Linux fixes and some MaterializedPath improvements

* Rust fmt again

* Introducing tick behavior on location watchers

* Making LastFilePathIdManager more atomic

* Some vscode launch configs for lldb debugger

* Simplifying some lifetimes

* Making all watchers more consistent
2023-03-31 04:59:33 +00:00
Oscar Beaumont c2666f7f63
Sync over p2p (#606)
* sync no networkey

* sync technically wired up

* fix p2p endianness for sync payload

* some sync debug stuff

* feature gate message production

* bump lockfile

* execute queries when sync messages disabled

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-03-21 06:03:08 -07:00
Brendan Allan 145f87cdab
Data Router (#609)
* data router time

* update pnpm.lock
2023-03-15 16:11:47 +00:00
jake e12475a4e1
[ENG-430] *Nix setup script improvements (#602)
* add `protobuf` deps to arch and fedora

* update webkit2gtk package name for fedora >= 37

* revert last change as fedora 37/38 can still use the old name

This reverts commit cc9b2d7053.

* shfmt

* re-introduce webkit changes (and fix my bad bash)

* remove duplicate webkit dep

* update mac setup logic

* remove ffmpeg versioning (we don't use it on any other system)

* check that RPMFusion is enabled

* more verbose protobuf variable names

* add comments/todos and remove unnecessary 64 bit declaration

* fix incorrect command & use latest ffmpeg on macos

* update deps & remove comments
2023-03-15 16:11:15 +00:00
jake 171f19bc71
[ENG-429] Crypto organization/refactor (#607)
* update comment

* key manager feature gating

* update deps & more feature gating

* fix keyring feature gating

* add an `exhaustive_read` function for crypto/stream

* restructure STREAM module

* move tests+shared utils

* clean up stream decryption

* further cleanup

* impl to `GenericArray` for `Nonce`

* update examples

* update refs & `use`s

* fix `Nonce` -> `GenericArray` conversions

* better `Protected` conversions + remove `Password` type

* a work of art

* finishing touches

* some API changes

* rename `StreamX` to `X`

* fix everything else

* separate `primitives` from `types`

* update imports & fix build
2023-03-15 16:09:36 +00:00
Brendan Allan 53229809ca
update swift-rs (#605) 2023-03-13 07:53:28 -07:00
Oscar Beaumont 1978e2ff48
Eng 432 refactor mdns service/state (#604)
clean up mdns state and refactor advertisement system
2023-03-13 18:05:46 +08:00
Oscar Beaumont 0c25239c53
Typesafe stream handling (#603)
* typesafe stream handling
Now it's impossible for your to respond to a broadcast stream. Previously this would just fail due to the TCP connection having been closed by the sender.

* remove connected_peers state from `SpaceTime`

* use `OutEvent` instead of `tokio::spawn` for `SpaceTime` events

* Polling for events > tokio::mpsc

* extend max length of sync messages to ~4GB + broadcast done message

* Fix "actOS" operating system decode error
2023-03-13 16:58:35 +08:00
Ericson "Fogo" Soares 4a6b057872
[ENG-371] Subpath shallow re indexing (#599)
* Introducing sub path indexing for IndexerJob

* Introducing shallow indexer job

* Some feature flags to avoid warnings at pnpm prep

* Many fixes at shallow indexer job

* Changing how we implement the bookkeeping for file_path ids
Now we're account for independent locations, and also integrating
better with the LibraryContext, instead of using a static global

* Making indexer job reentrant

* Introducing shallow file identifier job
And a bunch of minor refactors

* Rust fmt

* Removing light scan from getExplorerData query
Light scan is a mutation, so we can call it on useEffect function from
the Explorer component in the frontend, when a location_id or the explorer
path changes

* Handling job early finish on init for identifier

* Only invalidate query if we have orphan paths

* Introducing ShalowThumbnailerJob

* Clippy warnings about `into_iter()`

* Naming scheme for Prisma's selects and includes

* Invalidating getExplorerData at thumbnailer

* Small mistakes lol

* Some nitpicks with pnpm prep

* Rust fmt

* Changing indexer's walk log to `trace!`

* Not deleting all file_paths on location fullRescan

* TS typecheck

* Removing `file_path` selection just with id
2023-03-10 16:25:58 +00:00
Oscar Beaumont 7c5f760fbd
Basic Spacedrop working with rspc! (#601) 2023-03-09 06:23:17 -08:00
Jamie Pine f98d7ad6e0
[ENG-408] Dynamic Object icons (#600)
* improved object icons

* remove redundant control flow

* added support for base icon overrides by extension
+ deleted old icons folder and moved official icons into there

* fixed folder icon error

* removed temp nodes example

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-03-09 11:08:10 +00:00
Oscar Beaumont d21a521cbe
[ENG-407] Spacedrop backend (#598)
* format Rust

* Spacedrop a string

* Praise thee Clippy, lord of the Rust

* add protobuf to Mac and Linux CI

* plz GH Actions have Chocolatey
2023-03-09 11:27:45 +08:00
Oscar Beaumont f746530289
P2P Core (#597)
* implement space time

* I love it when my code works
2023-03-09 03:18:33 +08:00
Brendan Allan dad2072d7b
generate + use ModelSyncData enum (#592)
* generate + use ModelSyncData enum

* cache prisma clients on sync-generator
2023-03-04 22:12:37 -08:00
Brendan Allan 29a3123df5
restrict all swift stuff to macos only (#590)
restrict all swift stuff to macos only
2023-03-03 21:29:52 -08:00
Brendan Allan d89240d9e2
Colocate native code + reorganise (#589)
colocate native code + all rust in crates/
2023-03-03 09:22:22 +00:00
Ericson "Fogo" Soares fe215566d6
Fix unique constraint on file path table (#585)
* Complying with a pedantic Clippy

* Some task runners for vscode

* Solving duplicated directories events
When creating a directory through MacOS's Finder, for some reason
FSEvents receives 2 Create Folder events that we have to handle

* Damn rustfmt
2023-03-02 02:36:49 +00:00
Jamie Pine 3f44d6f521
[ENG-363] Spacedrop UI + Misc Improvements (#568)
* begin spacedrop ui + misc ui improvements

* better 404 xox

* Update extensions.rs

I think I prefer Container

* added DragRegion component,  ot tested cuz im on my fone

* Update DragRegion.tsx

fix import

* added dummy drop items

* better dummy data

* added clouds & search bar

* added action buttons to spacedrop items

* customize subtle button

* added support for apng, thanks luka big pants

* use relative path in sidebar

* use BYTES const

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-02-24 08:12:21 +00:00
jake 4cc582e08c
[ENG-134] OSS Licenses and Projects (#575)
* add `deps-generator` crate

* move `clap` types to separate file

* idiomatic filtering

* experimentally update ci

* add *really* basic dependency page

* remove `license_id` from `License`

* add bare minimum JSON files

* Revert "experimentally update ci"

This reverts commit c04897d902.

* re-insert comments

* Using reqwest blocking feature to avoid asyncness

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-02-20 05:58:55 +00:00
jake c7dbc784cd
[ENG-361] Crypto crate docs and tests (#572)
* add hashing tests

* add encryption/decryption tests

* remove `reason`

* add file header tests (preview media deserialization is broken)

* fix keyslot reading bug

* add sd-crypto testing to ci

* add tests/constants for all hashing algorthms and param levels

* add blake3-kdf tests

* use `const` arrays for storing expected output

* test for `5MiB` `encrypt_streams` and `decrypt_streams`

* add invalid/mismatched nonce tests

* update `primitives` docs

* remove erroneous `,`

* grammar tweaks

* add errors to `#[should_panic]`

* cleanup `stream` tests

* cleanup hashing tests a little

* function docs
2023-02-16 11:42:30 +00:00
Utku b856f15b22
Tailwind Prettier Plugin (#557)
* add tailwind prettier plugin + run prettier

* add eslint tailwind plugin and eslint fix to turbo

* Ran eslint fix + fixed every other lint issues

* twStyle and lint stuff

* remove auto lint as it's buggy

* update eslint & add twStyle to tailwind regex

* Fix auto lint + config inconsistencies

* Add tailwind config to prettier config

* run lint:fix

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-02-16 07:04:19 +00:00
Oscar Beaumont a9fceae819
improvements to spacedrive:// custom URI protocol (#550)
* fix `spacedrive://` custom protocol on Windows
(hopefully)

* custom protocol using `http::Response` + fix broken web

* import patches before App on web

* use `http::Request` for input to `handle_custom_uri`

* break  into dedicated file + error handling

* serving files via custom protocol

* cargo fmt because vscode did cringe

* lru cache to reduce video chunk request time

* add  helper to JS

* clippy be like

* remove duplicate Open buttons in context menu

* fix Linux 🙏

* no shot

* fix Windows custom URI passing (hopefully)

* better fix for custom uri on Linux

* upgrade Tauri for  feature

* switch url replacement order

* prevent React dev tools script being added in prod to desktop

* remove React devtools from html

* upgrade Tauri; required upgrading rspc, Axum, PCR

* pass typecheck + less cringe bigint

* clippy is love, clippy is life

* Typecheck plz

* fix bigint to number conversion

* use httpz + localhost server for Linux

* clippy be right

* Remove console.log

* [wip] proper auth

* fix Linux sidebar padding

* Secure Axum server with random

* Extracting app setup specific to linux to a different file

* remove outdated comment

* Some tweaks on cursom_uri.rs

* file_path_with_location doesn't need to be a named include

* fix typo

* factually wrong comment

* Change `unwrap` to `expect`

* bruh

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-02-14 13:27:11 +08:00
Jamie Pine f47a2d58e5
[ENG 239] Onboarding Flow & Location Settings (#529)
* begin better onboarding

* added input and altered text

* better router & text + database icon

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>

* work on privacy screen + radio buttons

* fix video extension bug and alter screens

* add pending schema and location manager helper

* functional onboarding

* added secure temp store and started creating library loading screen

* fix secure temp keystore + api

* better onboarding

* added location settings and some overview concept, all WIP

* fix switch

* prep

* fix location router

* added backend settings

* attempted to fix form

* begin indexer rules editor, plus tweaks

* indexer rules coming soon

* fix onboarding img size

* cleanup

* clone is needed here, but clippy no like

* sike

* whole bunch of fixes

* clippy + ts

* Removing some TODOs from api/libraries.rs and fixing db size calculation

* moved object kind to client, added half functionality for appearance settings

* fix RadioGroup helper

* fix type issues

* cargo fmt

* fix creating library error handling + invalidate location list on update

* forgot to switch back to onError

* Invalidating getStatistics query on library creation and introducing the concept of waiting for a job on FileCopierJob

* F* cargo fmt

* fix RadioGroup interactivity

* wipe all migrations

* put back COLLATE NOCASE on extension columns

* update core.ts

* remove unused device component

* fix typeerror in mobile

---------

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-02-10 14:08:13 -08:00
jake 5b78bbbb7d
[ENG-358] Malicious client protection for the KM (#562)
* add `queue_check`, `mount_check` and `unlock_check`

* make `unlock_check` marginally more readable

* clippy

* clean up & rename functions

* remove unlock check on `keys.list`

* use american spelling of `favorite`

* use `then_some` (thanks clippy)

* more american spelling
2023-02-09 09:30:26 +00:00
jake d1b6263ae7
[ENG-355] Keychain integration (and some typesafety) (#558)
* update crypto MSRV

* rename `keychain` to `keyring`

* make a start on the keymanager unlock refactor/keychain integration

* update routes

* update bindings

* add const identifiers

* add UI/front-end support for unlocking KM with OS keychains

* remove SK from lib creation dialog

* update query name

* add keyring functions

* attempt to update `change_master_password()` to use the keychain

* cleanup, fix master password change ui, better secret key in keyring detection

* cleanup TS a little

* add route for getting secret key from keyring

* update bindings

* update var names + show secret key in keys settings

* add `react-qr-code` and option to view the secret key (if it's in the OS keyring)

* allow copying SK to clipboard

* add `key_type` so we're not reliant on specific UUIDs for root/verification key handling

* clippy

* fix mobile typecheck

* fix typecheck, fix typo and tweak balloon hash parameters

* minor cleanup + typo fix

* use newtype structs

* WIP type refactoring (major readability boost!)

* update `use`

* add tokio `sync` feature

* too many structs? idk

* more cleanup

* add `generate` and `Nonce`

* `Nonce` and `Key` typesafety (beautiful)

* clippy + cleanup

* update code & examples

* fix bug & remove `ProtectedVec` as it looked out of place

* use `Key`

* add a query invalidation to make the UI extremely responsive

* ci pls work

* remove `keyringHasSk` route
2023-02-07 12:03:12 +00:00
Brendan Allan d1aa927203
Merge pull request #553 from spacedriveapp/non-unique-objects
Store CAS ids on file paths
2023-02-01 12:11:03 +08:00
jake 9fb01fc9d2
[ENG-341] OS Keychains (#554)
* super barebones keychain start

* working+unfinished linux keychain

* add `copy` to `Identifier` and add `delete()`

* add generic errors + code cleanup

* cleanup code & add support for apple keychains

* remove `users` dep

* use uppercase UUID and tweak apple `account` variable

* revert uppercase change and clean up linux labels

* code cleanup & add a readme to the crypto crate

* remove useless `map`

* correctly handle keyring not supported errors

* add `Send` to `Box<dyn Keyring>`

* remove useless `cfg`s

* only return `Ok()` if we support keychains (pls work CI)

* clippy

* Minor fixes

---------

Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
2023-02-01 00:11:03 +00:00
Ericson Fogo Soares dd1db0338e A bunch of minor fixes 2023-01-31 20:02:23 -03:00
Brendan Allan 494e87d115 Merge branch 'main' into non-unique-objects 2023-01-30 12:11:35 +08:00
Brendan Allan bf362eadd6 store cas ids on file paths 2023-01-28 11:42:24 +08:00
brxken128 6874c23c01
make requested changes 2023-01-26 10:39:18 +00:00
brxken128 2abf75defd
update things to use workspace tokio 2023-01-25 22:03:10 +00:00
brxken128 3f2a62feb4
Merge branch 'main' into eng-348-async-crypto 2023-01-25 22:02:23 +00:00
brxken128 a11d81549f
use a tokio::sync::Mutex just to be safe 2023-01-25 18:20:29 +00:00
brxken128 a3b1c8f5b8
tell everything to use workspace tokio + add tokio git patch 2023-01-25 14:36:29 +00:00
brxken128 7623bf9a6c
change fs and macros tokio features to dev-deps 2023-01-25 14:03:00 +00:00
brxken128 fa147f3e87
add correct features to tokio dep 2023-01-25 12:47:26 +00:00
Brendan Allan 3c7c05cc87 update shared sync functions to use new system 2023-01-25 13:17:01 +08:00
brxken128 2381798790
fix stream reading due to read()
Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
2023-01-24 23:28:16 +00:00
brxken128 c75e983fd4
bring in Salt, Key and EncryptedKey type aliases 2023-01-24 17:01:50 +00:00
brxken128 546bf643b2
add ProtectedVec type alias 2023-01-24 16:47:41 +00:00
brxken128 3ed106e27b
clippy 2023-01-24 16:35:31 +00:00
brxken128 3fd547ab2d
disable benchmarks as they need async support 2023-01-24 14:25:42 +00:00
brxken128 f13c247aec
update examples 2023-01-24 14:25:19 +00:00
brxken128 371ac93bcf
async crypto! 2023-01-24 14:19:27 +00:00
brxken128 87b75cfe62
return to match/for statements for async iterators (until rust gets better support) 2023-01-24 14:04:18 +00:00
brxken128 b929d36c89
more async crypto 2023-01-24 13:50:46 +00:00
brxken128 b077e16f00
mostly async keymanager 2023-01-24 13:16:47 +00:00
brxken128 64d92c2132
async stream encryption and decryption 2023-01-24 12:49:54 +00:00
brxken128 2d2f45be3d
Merge branch 'main' into eng-345-cut-copy-duplicate-actions 2023-01-23 11:03:27 +00:00
Ericson Fogo Soares 5e30cf6354 Removing some unnecessary clones and other small fixes 2023-01-20 18:22:17 -03:00
Brendan Allan e4e80fa603
Refactor routes (#535)
* refactor routes

* use default export for screens

* use "route" relative routing

* don't use absolute import for root screens

* no absolute importing folders!!!

* fine we can absolute import folders
2023-01-20 12:40:36 -08:00
Brendan Allan 24de617b92
Fix webkit reload (#534)
* wrap WKWebView in UnsafePointer

* formatting

* more formatting
2023-01-20 00:25:12 -08:00
maxichrome ec4352a30a cleanup(don't hate me): prettier format everything 2023-01-19 19:32:35 -08:00
jake 0a31e7f8ce
[ENG-257] File erasure and deletion (#524)
* add helper functions for getting context-menu related filesystem info

* update comments and such

* add a barebones delete job

* add file deletion route

* add (empty but working) delete file dialog

* add super experimental file erase function

* update function doc

* add erase files route

* add erase files job

* tweak file erasing in crypto crate

* bindings

* add erase files dialog

* add missing `?`

* fix erase file form trickery

* remove unused imports

* terrible attempt at recursion for file erase

* IT WORKS

* code cleanup + clippy

* async just dropped (not for erase yet)

* async erase just dropped (+async crypto crate)

* add "nice data" for jobs
2023-01-18 19:55:14 -08:00
jake 0ef7f64bbb
[ENG-328] Keymanager mounting queue (#514)
* add `get_queue` and `is_queued` to KM and router

* mark keys as being mounted/finished mounting, use a `DashSet` instead of vec

also includes temporary `dbg!()`s for manual valdiation that it works

* remove dbg and fix build (works great)

* mark key manager as unlocking correctly

* cleanup keys api

* add `KeyAlreadyQueued` error

* use react mounting queue

* fully prevent against multiple mount jobs of the same key with the rust queue

* show keymanager as unlocking in the UI

* handle query invalidation for `isKeyManagerUnlocking` with a closure (semi-temporary)

* revert changing text (doesn't work where the keymanager is unlocked from, only the other unlock screen)

* clippy

* add `useRef` for mounting queue within react

* pass `ref.current` to `Key` instead
2023-01-17 06:19:54 -08:00
Brendan Allan 8e479f736f
Eng 332 auto generate sync ids (#523)
* generate sync ids and use model name from ModelActions

* cleanup AddLocationDialog imports

* rustfmt
2023-01-15 07:25:11 -08:00
Brendan Allan ae94ada4f8
Begin form abstraction (#515) 2023-01-12 07:26:59 -08:00
jake 0d554e9d06
[ENG-331] StoredKey overhaul (#513)
* add wip storedkey versioning

* storedkey versioning! (not pretty, but it never will be)

* add version to `StoredKey` and re-gen migrations to handle serde

* use `serde` for interacting with the DB + handle errors
2023-01-11 05:15:08 -08:00
jake 4536710586
Detect SD-encrypted files and display them as such (#511)
* add encrypted file magic bytes/extension

* use archive icon for encrypted files temporarily

* use pascal case for encrypted extensions

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-01-11 00:40:49 -08:00
jake fbd8b6d199
[ENG-327] Crypto refactor (#510)
* fix secret key showing as `Some()` when it's empty

* rename function

* use `uuid` feature of `rspc`  so clippy works with `--all-features`

* remove dead error

* update clippy rules

* more idiomatic + cleaner hashing, that handles secrets better

* remove dead code + minor refactor of the KM

* cleanup`KeyManager::get_key()`

* cleanup keys router

* variable inlining, idiomatic rust, etc

* clean up encrypt/decrypt jobs

* embed preview media into encrypted files

* even more idiomatic rust

* fix previous file extension code (no amount of idiomatic rust can fix bad patterns)

* hopefully make large serialization slightly more efficient

* remove dead code + old tests

* inline key derivation (not so sure about this one)

* use `then` instead of `if`

* stop using expensive clones

* add `Protected::into_inner()` to prevent `.expose().clone()` patterns

* use `tokio::fs::File` where possible

* add and use `JobError::MissingData`

* use `|| null` for the secret key

* use `block_in_place` for large r/w sync tasks
2023-01-10 12:36:28 -08:00
jake c1b2b1f5ee
[ENG-323] Key manager onboarding fixes (#506)
* add hash with secret functions

* refactor KM to make the secret key both optional, and user-settable

* update library create function and create library dialog

* fix annoying MPC typo

* fix ordering of key manager creation and seeding

* overhaul master password changing UI to support the potentially empty secret key

* patch keymanager unlocking for potentially empty SK

* make create library dialog text more fitting

* remove dead code

* add autogeneration, view/hide, copy to clipboard icons

* cleanup older code

* make onboarding screen look a tad better

* minor UI tweaks

* patch backup restoration UI

* add pw generation to MP change dialog

* make sure passwords are same before submitting lib creation

* UI fixes and allow setting algorithm/hashing algorithm for master password

* make sure KM is re-seeded on SD load and prevent loading duplicate keys into the keystore

* remove old comment

* small organization

* update encryptfiledialog

* remove dead KM code

* cleanup some TS

* move `PasswordMeter` to separate file

* refactor master password change dialog

* cleanup the TS and improve string -> hashingalgorithm conversions

* add `key` values to selectoption

* capitalise `BLAKE3`

* fix sorting for `getHashingAlgorithmString`

* remove onboarding route

* Revert "add `key` values to selectoption"

This reverts commit 10e6456f22.

* add `key` to `SelectOptionKeyList`

* fix using rspc errors directly

* Update packages/interface/src/components/onboarding/Onboarding.tsx

Co-authored-by: Brendan Allan <brendonovich@outlook.com>

* use `navigator.clipboard.writeText`

* move `OnboardingConfig` to crypto crate

* move `zxcvbn` options outside of component

* further cleanup

* fix mobile typecheck

* fix keylist typo

* improve `SelectOptionKeyList` further

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-01-06 17:03:22 +00:00
Brendan Allan 676e512c4a
Beginning of sync stuff (#502) 2023-01-04 23:11:55 -08:00
jake 22bc77a39e
[ENG-319] Balloon hashing (#489)
* add key attribute to `Key` in `ListOfKeys`

* add balloon hashing function with untailored parameters

* add ser/de rules for blake3-balloon (and change argon2id's)

* fix benchmark

* use `to_bytes`, `from_bytes` and `from_reader`

* cleanup code

* add blake3-balloon options to the UI and fix library sync/automount enable bug

* cleanup some serialization code

* fix hashing algorithm deserialization

* clean up header serialization + more idiomatic master key decryption

* clippy

* add generic ser/de error to crypto crate

* fix `Display` and crypto cli

* move crypto cli to cli app

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-01-04 09:57:02 +00:00
jake 4569e45e47
[ENG-312] Crypto details CLI tool (#500)
* add untested crypto details tool

* clean up output a little

* clippy

* update crate details and `clap`
2023-01-04 00:15:12 +08:00
Oscar Beaumont 3964d44ce5
Integrate KeyManager with library creation flow (#491)
* integrate keymanager with library creation flow

* final fixes

* fix clippy recommendations

* minor fixes on library create dialog

* reenable `panic!` for no key + fix secret handling code

* prevent user setting secret , instead hardcode it

* clean library manager default key selection

* bring back wrongly removed `keys.onboarding` resolver

* fix types in `CreateLibraryDialog`
2023-01-04 00:13:12 +08:00