Commit graph

303 commits

Author SHA1 Message Date
Oscar Beaumont ae4c3d0e6b
Fix Android bridge + drop IOS and Android projects (#1336)
* Goodbye Android Project, you won't be missed

* fix ignore ignoring Kotlin code

* Backport IOS `Podfile` + move Rust crates into `modules/sd-core`

* Workaround for issue in OpenSSL build script

* Fix IOS

* Migrate IOS bridge into Expo Native Module

* Optimistic update for library creation

* useFrameworks static

* add key to infoplist

* more keys

* fix `Podfile`

* Remove IOS project from repo

* fix mobile ci

* woops wrong place

* clean package.json

* Make Android's Rust code build again

* Add back in `if_watch` patch for IOS

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-09-13 10:07:26 +00:00
Oscar Beaumont 94d8f61de9
Update Rust dependencies (#1314)
* Update most of Rust dependencies

* downgrade notify

* a

* fix Linux?

* Android is painful

* update build.gradle

* The lord Clippy has spoken

* Bruh

---------

Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Ericson "Fogo" Soares <ericson.ds999@gmail.com>
2023-09-12 14:03:25 +00:00
jake e2a0878e7a
[ENG-1068] Plus codes / Open Location Code support (#1324)
* bring in rand deps

* SUPER crude pluscode attempt

* working pluscodes!

* some cleanup

* `encodeURIComponent` for URL escaping

---------

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2023-09-11 04:34:24 +00:00
jake 7edcbb15d6
[ENG-621][ENG-1074] Trait-based image conversion overhaul (#1307)
* sd-images crate which will support raw/dng, bmp, etc

* more work on the image formatter

* re-work `sd-images`, add svg support, r/g/b and r/g/b/a HEIF image support (will all be async again soon)

* remove `ImageFormatter`, add note about r/g/b/(a) heif impl

* implement the image formatter

* rename the conversion trait and minor cleanups

* isolate heif feature and major cleanup

* very untested raw support

* change fn name to `from_path` (a lot more idiomatic)

* clean up orientation fixing

* heif is no longer forbidden (linux has good heif)

also all extensions are correctly matched in lowercase

* fix builds, ext matching, feature gating

* attempt to fix svg handling?

* raw attempt, quite a few errors

* add comment

* new (untested) attempt

* remove `raw` stuff for now

* replace `sd-svg` with a `ToImage` `SvgHandler` impl

* add some simple math to appropriately scale thumbnails (and bmp/ico support)

* add comments regarding how the math works for image thumbs

* rename the trait to `ImageHandler`
2023-09-07 14:08:17 +00:00
ameer2468 5860016789
[ENG-934] EXIF UI (#1305)
* Media data UI

* Make `MediaTime` adjacently tagged

* cleanup ts

* don't destructure accordion props

* Large bruh

* round location coords

---------

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-09-06 13:51:15 +00:00
Oscar Beaumont 65ea570d0b
[ENG-1055] Fix previewing large files (#1304)
* Proper streaming from disk

* Handle `If-Range`

* fix bad merge

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-09-06 08:06:29 +00:00
Oscar Beaumont 5a0d0396d0
[ENG-566] Stream files through P2P (#1282)
* Add range to `SpaceblockRequest`

* Determine if file should be served from P2P

* Feature flag serveFromP2P + fix `ServeFrom` logic

* wip: yeet files over P2P

* Just work, argh

* restore backend feature flags after restart

* cleanup
2023-09-06 07:22:04 +00:00
Vítor Vasconcellos b0170b9dba
[ENG-904] Thumbnail for svg (#1220)
* Initial implementation for svg rendering

* Remove unused errors
 - Round up size before allocating Pixmap
 - Re-order some operations

* Finish integrating with Thumbnailer
 - Fix svg thumbnail size
 - Fix incorrect color space while converting tiny_skia::Pixmap to image

* Fix Clippy warns

* Feedback + sd-heif async

* Update implementation to match recent changes to the Thumbnailer
 - Change sd-heif and sd-svg to receive the file data, instead of opening the file internally
 - Update changes to make Thumbnailer and sd-heif more async

* Minor import improvement

* Add missing cfg to gate use of sd_heif only to plataforms that have it enabled
2023-08-31 23:01:45 +00:00
Oscar Beaumont f821bb9a0f
Stop App Freezing (#1257)
* Drop Tauri custom URI handler
Me no likely but it has to be done.

* fix app startup with location

* fix "Add Location" button on web

* Serve correct content range

* Backport changes from 08ba4f91

* none of my homies like panics

* minor fixes

* fmt with new Rust version
2023-08-31 06:54:40 +00:00
Oscar Beaumont d758977d82
Improve Sync + P2P Integration (#1265)
* Big bruh moment

* whoops

* Less stackoverflowy debug

* stuff

* Fix flawed P2P mDNS instance advertisements

* do sync when connecting with peer

* Sync after pairing

* resync_part2 all the time

* Invalidate all the things

* Invalidate whole React Query on sync event

* emit_messages_flag

* emit_messages_flag

* Backend feature flags + "emitSyncEvents" feature

* Patch `confirm` type cause Tauri cringe

* clippy

* idk but plz work

* bruh

* Fix ComLink bug

* remove log

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-08-29 17:54:58 +00:00
jake c674b7107a
[ENG-288, ENG-601] Media data (image) extraction, thumbnail orientation fix (#1099)
* basic layout

* lock

* add codec to image

* very messy wip

* rm that

* lock and toml

* working perfect exif extraction

* formatting

* migration and formatting

* mostly working

* fix

* set date created properly

* fix tsc

* working media data creation

* fix bad main merge? sorry brendan

* schema, migrations, bindings

* working exif data extraction

* why won't it work

* update migrations

* fix bad merge

* final cleanup

* cleanup migrations

* remove test (that was purely used for testing my code)

* working media data pulling, correct thumbnail orientation

* slightly optimise thumbnail rotation

* rename location to prevent specta clashes

* further improvements (location parsing is still broken)

* fix coordinate parsing i think

* rspc add some todos regarding final cleanup

* further thoughts

* major upgrades

* Some improved handling of errors and async calls

* accordion component

* heavily wip flash refactor

* fix builds all because of a (terrible) merge/move

* annoying missing newline

* i really hate exif

* remove dead code

* further flash progress :D

* docs(media-data): clarification

* minor cleanup

* cleanup and some async removal

* fix location parsing

* remove async (will do proper impl for async eventually) and major cleanup

* big W

* clippy and `FlashMode::Unknown` instead of `Invalid`

* add `NIKON` gps ref support

* comments about gps refs

* commit the submodule -_-

* major cleanup & composite image support

* remove old test image

* major cleanup and easier to use API

* remove old consts

* move `ExifReader` to dedicated module

* Media Data Extractor job and shallow job

* Extracting media data on watcher

* report no exif data on file gracefully

* cleanup errors and doctests

* Merging thumbnailer and media data extractor

* Job name and some strings in UI

* remove reliance on `sd-prisma` within the media data crate

* rename query to be more specific

* custom serializer for `MediaTime`

* tweak to format utc -> naive correctly

* generate migrations

* comment out duration in mobile

* delete test-assets folder

* all optional fields

* fix migrations again

* make requested name changes

* make further requested changes

* remove erroneous files from another wip branch

* updates procedures

* use strings where appropriate

* regen pnpm-lock

* add base layouts for video and audio media data

* use appropriate data types in schema and add audio media data framework

* make requested changes

* general cleanup and renaming of enum

* cleanup media data api

* rename media metadata type

* finishing touches

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-08-29 17:02:55 +00:00
Oscar Beaumont 7bf0c0ae4a
[ENG-994] Minor P2P fixes (#1238)
* mdns advertisement on detection

* fix bug in P2P discovery

* Add `P2PEvent::ExpirePeer`

* React properly

* Consistent tracing versions

* better logger config

* Reduce excessive logging from P2P

* Fix panic log hook

* Remove `dbg`

* Fix bug in thumbnailer remover

* wip: Connected peers in UI

* `Sync` is large pain

* init services after library load

* improve error handling of logger

* Sync protocol shutdown

* Add "updater" feature

* fix commands bindings

* nlm state debug query

* nlm debug status

* Make TS happy

* Add `RemoteIdentity` to libraries debug info

* Improve debug data presentation

* Among us level sus

* minor fix panic hook

* Fix EOF issue

* fix instance not being added on pairing

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Ericson "Fogo" Soares <ericson.ds999@gmail.com>
2023-08-27 06:41:26 +00:00
Brendan Allan fecb81d192
Fix PCR formatting (#1243)
use newer commit of pcr
2023-08-24 06:02:39 +00:00
jake 6acc47ef09
[ENG-980] Barebones, essential telemetry (#1237)
* rename everything to "full" telemetry and re-word onboarding

* impl the `ping` event every 4.5 mins, or on path change

* fix debug popover

* update mobile

* remove callback (and cargo lock update?)

* add to mobile, remove eslint ignore, remove console log

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-08-23 06:28:25 +00:00
Oscar Beaumont bcbbe58141
[ENG-974] DB Backup prototype (#1216)
* DB Backup prototype

* Put backups behind feature flag

* Warning for data folder

* nit

* Clippy

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-08-17 05:37:10 +00:00
Jamie Pine a253431d0f
[ENG-887] Category empty screens (#1056)
* begin

* default category ui + schema

* yes

* better light icons

* new text icon

* fix schema

* fix merge error

* fix optional

* re-export icons

* tweaks

---------

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2023-08-12 16:30:03 +00:00
Brendan Allan 8f1afbf164
Isolate sync p2p logic (#1198)
* remove p2p from sd-core-sync and isolate p2p handling

* re-enable tests and stop being silly

* move sync protocol to modules

* don't emit messages

* cleanup

* formatting
2023-08-11 08:32:20 +00:00
Brendan Allan 98f763134e
lock libheif-sys version (#1183) 2023-08-07 10:12:20 +00:00
Brendan Allan 00e4aa9e8f
Sync ingesting (#1181)
* sync + ingest refactor

* fix Event enum description

* actually do sync over network

* re-enable heif

* remove comment
2023-08-07 07:52:43 +00:00
Oscar Beaumont 9e919f9909
Library "state" (#1160)
* `Library` -> `LoadedLibrary`

* Move `ThumbnailRemoverActor` onto `NodeServices`
It should either be on `Library` or `NodeServices` none of this in-between stuff.

* small > big files

* prune imports

* isolate cringe

* it helps to test the code

* `Vec` -> `HashMap` - faster lookups

* mpscrr

* minor fixes

* fix language

* `!Sync` :(

* lets goooooo

* removed a comma

* fixes

* fix desktop

* lol, no one uses it

* Clippy
2023-08-05 14:42:21 +00:00
Oscar Beaumont 6c9e08540f
Networked sync (#1133)
* temp remove discriminator from tunnel

* i'm lost :(

* Working `Tunnel` + move sync stuff back into p2p

* sync module

* `NetworkedSyncManager`

* Move sync stuff off `P2PManager`

* Test `SyncMessage`

* Library edit + delete hooks

* stable `identity` column

* fix

* plz work

* p2p is hard

* sync identities post pairing

* Remove `p384`

* a couple improvements

* comment out ReqRes

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-08-02 11:49:50 +00:00
Oscar Beaumont 13a3fbe486
Cleanup core (#1140)
* `NodeContext` -> `SharedContext`

* Cleanup context init

* Goodbye `Arc`'s

* Cleanup context

* rename

* minor

* abstract `Platform` into it's own file

* More `Arc`'s go goodbye

* parallelise `categories.list`

* parallelise `categories.list`

* Unused code in `libraries.rs`

* whoops

* a cheeky rename

* bruh

* :-(
2023-08-02 08:06:54 +00:00
jake 7215e40a3b
[ENG-926] Prevent thumbnail destruction and fix the remover (#1127)
* fix(core): thumbnail removal

* chore(core): add todo

* New actor on steroids

* Improving thumbnail remover actor

* Ignoring errors from files that doesn't exist

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-07-25 14:21:50 +00:00
Vítor Vasconcellos 7e4ed2b547
[ENG-767, ENG-916] Improvements and fixes for Open/Open With (#1082)
* Normalize PATH and XDG envvars on Linux

* Fix normalize_xdg_environment

* Replace custom Desktop file parsers with Glib

* Fix AppImage env influencing external apps
 - Normalize GStream plugin path

* Fix macos pulling linux deps

* Attempt to fix gnome apps failing to launch
 - Fix incorrect logic in `normalize_pathlist`

* Ensure empty envvars are not set

* Revert AppImage ovewritten GTK specific variables
 - `normalize_pathlist` now prefers entries with less priority when dealing with repeated entries, this is not compatible with the default behavior, but it is a more sane approach IMHO

* Remove 32-bit libs from release CI build host

* Remove 32-bit from github runner 2 attempt
 - Remove deprecated vscode config

* Remove libc6-i386

* [ENG-916] Implement `Open With` logic compatible with macOS < 12

* Add some missing gstreamer deps in Linux

* Replace opener with Glib API

* Fix reveal opening file instead of dir
 - Improve Open With logic in Windows
 - Expose functions to test if app is in a flatpak, snap or appimage

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-24 17:36:00 +00:00
Brendan Allan 043b607ad4
CRDTOperation receiving (#1122)
* operation receive + compare

* cleanup + deduplication

* operation receive + compare

* cleanup + deduplication

* sync route + operation grouping

* tag assign sync

* proper relation support in sync debug page

* migration

* separate core-sync + utils crates

* separate p2p event loop from manager

* cleanup library handling

* clippy

* feature gate sync messages properly

* make migration not add required field
2023-07-24 15:26:00 +00:00
Oscar Beaumont 23b0c403be
[ENG-925] - Arc<Library>/Arc<NodeContext> (#1124)
* `Arc<Library>` + `Arc<NodeContext>`

* fix specta version

* remove unneccessary derefs

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-24 10:59:57 +00:00
Brendan Allan 0a1d721e58
Use codegen for ModelSyncType exec (#1098)
* use codegen for ModelSyncType exec

* folder client format

* hande create operation properly
2023-07-15 04:22:04 +00:00
Oscar Beaumont cf39f8dbcc
[ENG-759] P2P Cleanup (#1062)
* less stupid name

* yeet

* awaiting futures is kinda important lol

* no-async

* more proto stuff

* cleanup

* move it round

* None of my homies like broadcast

* minor

* do the shuffle

* restore after force push

* reusing `sysinfo::System` as intended

* fix lol

* remove `node_id` from `Volume`

* Remove `node_local_id` from `Library`

* Remove `Job` to `Node` relation

* feature flags be like

* press save 4head

* remove `Location` -> `Node` relation

* `volume.rs` to `volume/mod.rs`

* yes

* add `Instance` model and deprecate `Node` model

* pairing is better now

* Pretty code

* thinking in code

* wip

* What if Observables but in Rust?!

* Observables aren't it + `useP2PEvents` hook

* more around some jsx

* Trade offer: bad code for working pairing?

* Unit test pairing protocol

* fix everything up

* corrections

* Have you got a moment for our lord and saviour Clippy

* tsc --fixMyCode

* Prisma being wacky

* Config files being f'ed up

* broken config after migrating

* Zed being Zed

* Argh

* cliipzy

* rewind

* Fix truncate logic

* wip: instances in peer metadata

* Rethink instance ids

* fix

* whoops

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-12 06:23:30 +00:00
Brendan Allan 07c00a8b7f
[ENG-846] Preferences (#1047)
* preferences table + trait

* cleaner implementation

* router + ts type

* preference test

* per-column list view preferences

* preference read + write

* add some docs

* migration + docs

* remove ts expect error
2023-07-12 04:32:53 +00:00
Vítor Vasconcellos f83f29c720
[ENG-765] Reported Total capacity and Free space are wrong (#1066)
* Attempt at fixing stats

* Fix macOS disk stats retrieve logic
 - Ignore mounted dmgs when calculation disk total/free size
 - Only take into account disk mounted by macOS

* macos only import

* Fix Linux

* Replace byte-size with a custom implementation that supports BigInt

* Fix NaN in Statistics

* clippy

* fmt

* Move linux get_volumes to a specilized function
 - Fix ZFS handling
 - Improve handling of disk symlinks and multiple mounts

* Fix macOS
2023-07-05 18:22:56 +00:00
Vítor Vasconcellos caaacc8f92
[ENG-866, ENG-535] Thumbnailer fixes (double-free, thumbnails for portrait videos) (#1059)
* Fix double-free during MovieDecoder Drop
 - Fix broken thumbnails for portrait videos
 - Fix some inconsistencies between our MovieDecoder impl and ffmpegthumbnailer impl
 - Make sd-desktop-macos use the correct macOS deployment target

* fmt

---------

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-06-30 11:00:04 +00:00
Brendan Allan ff5fce28a4
Cleanup empty notice (#1042)
* cleanup empty notice + move tag assign options

* typescript
2023-06-28 06:59:29 +00:00
Brendan Allan 06de379169
[ENG-816, ENG-821] Re-implement reveal in finder + ContextMenu overhaul (#1029)
* mostly there

* native opening working

* more

* cleanup

* reorganise

* remove unnecessary import

* uncomment some stuff

* spacing

* store quickview ref inside provider

* fix linting

* clippy

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-06-27 15:34:53 +00:00
Oscar Beaumont f86b6c8b52
[ENG-823] Fix some stuff (#1024)
* SQLite journal bug

* Remove "job.report" indexes thing

* resume button - job not found?

* panic hook with logging

* it did

* large pain

* fix `check_nested_location`

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-06-27 05:12:04 +00:00
Oscar Beaumont 0f7a669e5a
[ENG-732] Pairing by library (#943)
* library keypair

* allow opening stream to non-connected node + more

* library identity

* fix types

* fix maybe undefined

* don't forgor migrations file

* library manager inside p2p manager

* rename

* `NodeInformation` struct

* node info exchange

* fill in info

* streamify tunnel

* use tunnel for p2p events

* libp2p is annoying + stop leaking private key's

* Clippy cleanup
2023-06-19 05:13:30 +00:00
Oscar Beaumont fd1afd8500
[ENG-568] Spacedrop Progress (#956)
* `Transfer` struct

* progress moment

* fixes
2023-06-18 08:40:51 +00:00
Vítor Vasconcellos 4078c360b4
[ENG-594] Open With Windows + fixes (#945)
* Windows `Open With` WIP
 - Listing applications capable of hanling a file type is working
 - Openning a file with a selected application is failing with unspecified error HRESULT(0x80004005) for some reason

* Fix file not opening due to COM not being initialized
 - Fix `no apps available` style

* Remove unwrap

* Fix `Open With` due to changes in main

* Fix macOS `Open With`

* Fix Windows `Open With` due to changes in main
 - Sort linux `Open With` entries, to ensure consistent app order

* Fix macOS again

* Update core.ts

* Fix windows CI being rate limited

* Clippy

* Fix CoUninitialize not being called

* minor formatting

* Implement feedback
 - Improve performance of listing apps that can handle a certain file type in Linux

* Fix broken feedback change
 - Small perf improvement to windows crate

* Some improvements to windows crate
2023-06-17 05:23:45 +00:00
Oscar Beaumont 4987b7039e
Fix main (#952)
fml
2023-06-15 09:06:13 +00:00
Ericson "Fogo" Soares e693c7a542
[ENG-381] Update context menufs actions to support multiple source (#927)
* Generalizing filesystem jobs to accept multiple files at once

* Some small changes and fixing front

* Enable rename route to replicate Finder's behavior

* Assign tag to multiple objects at once

* Remove from recents accepting multiple files

* Fixing some warnings

* Adding multiple files feature to Open and Open With

* Conditional stuff for macos

* Generating commands.ts and other minor warnings

* Rust fmt

* TS typecheck

* Rust format and TS typecheck

* Requested changes

* Requested changes

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-06-14 23:00:28 +00:00
Vítor Vasconcellos e6d0e6098c
[ENG-593] Implement Open With for Linux (#939)
* Implement `Open With` for Linux

* Fix TS typechecker failing and Rust formatting

* Take args by ref

---------

Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
2023-06-14 00:54:43 +00:00
Jamie Pine 8938ae6802
[ENG-708] Thumbnail sharding (#925)
* first phase, basic sharding

* improved API for sharding using a "thumbnailKey"

* clean up param handling for custom_uri

* added version manager with migrations for the thumbnail directory

* remove redundant hash of a hash, silly

* fix mobile

* fix clippy

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-06-08 07:13:45 +00:00
Brendan Allan fd236a1b57
[ENG-679] Reserve ids for built in indexer rules (#909)
* indexer rules pub ids

* should work?

* better migrator

* errors

* debugging

* maybe?

* double migrate

* please

* maybe fix?

* update lockfile

* SD_ACCEPT_DATA_LOSS message

* put tracing back

* dumb

* fix system indexer rule ui
2023-06-06 14:42:52 +00:00
Vítor Vasconcellos 350b7e39db
[ENG-683] Fix tauri allow list (#897)
* Fix Tauri drag not working
 - Fix Mpeg videos compleatly breaking QuickPreview
 - Fix Dev build open data folder button not working
 - Fix MP4 videos freezing in QuickPreview on MacOS

* Fix React inter-component state error
2023-06-02 00:09:19 +00:00
Oscar Beaumont ae138b2358
[ENG-672] Find and remove RESYNC log (#892)
large bruh moment
2023-05-31 09:59:03 +00:00
Oscar Beaumont 49814116f4
[ENG-671] Fix mobile (#888)
fix mobile
2023-05-30 17:43:32 +00:00
Oscar Beaumont e4696c7303
[ENG-667] Fix Clippy CI (#887)
* plz just work

* no shot

* just disable Tauri notifications

* Tauri is weird AF
2023-05-30 17:11:37 +00:00
Brendan Allan e70fd4eba5
[ENG-662] Don't use job system for shallow operations (#886)
* make it work

* use functions instead of jobs for shallow stuff

* use generalised AbortOnDrop

* simplify naming

* error stuff + alphabetical rspc types

* log errors

* don't run CI on push

* remove ephemeral

* private Worker::new

* remove comments
2023-05-30 17:07:41 +00:00
Oscar Beaumont 5f71e0c1f6
[ENG-546] Improve error handling (#802)
* log to disk

* remove some unwraps

* panicless

* some p2p error handling

* clippy moment

* Fix `<ErrorBoundary />`

* open logs button

* 39 to 0

* fix types

* update deps and comment out broken tests

* clippy

* more clippy

* upgrade rimraf - https://github.com/isaacs/rimraf/issues/259

* regen broken lockfile

* update `notify` and update `commands.ts`

* more clippy (pls work)

* allow deprecated for p2p (hopefully temporary)

* a

* upgrade deps for p2p

* do betterer

* do it correctly

* remove unused import

* improve core startup error + bc keypair

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: brxken128 <77554505+brxken128@users.noreply.github.com>
2023-05-30 09:24:05 +00:00
Oscar Beaumont a20f6262fd
[ENG-659] Spacedrive On Jamie's NAS (#866)
* serve dem

* 0% chance of working

* With a side of Vite plz

* no CI for now

* Clippy plz work

* undo

* bruh

* plz work
2023-05-29 10:47:59 +00:00
Brendan Allan 6710f05251
move prisma codegen into sd-prisma (#876) 2023-05-29 08:37:58 +00:00
Brendan Allan da2e78dc61
Use search.objects for recents query (#850)
* separate search + filter + ordering

* typesafe categories

* make ordering great again

* fix ts

* eslint
2023-05-23 12:58:45 +00:00
Brendan Allan 9448413048
update pcr (#843) 2023-05-21 07:19:26 +00:00
Brendan Allan b0e95fc59f
Replace getRecent with search.paths (#832)
replace getRecent with search.paths

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-05-20 22:11:43 +00:00
jake fb95c6fec2
HEIF Fixes and Improvements (#838)
* limit HEIF to macos-only because of ubuntu's *archaic* `libheif` version

* drop png entirely, go from raw `[r, g, b...]` -> `DynamicImage`
2023-05-20 22:08:04 +00:00
Jamie Pine 7e4c58a91c
[ENG-603] Support HEIC images (#834)
* working HEIC thumbnails

* better error handling

* better error handling and some cleanup

* fix type of maximum size, and clippy

* better extension support, WIP quick preview, better image resolution

* remove unnecessary x86_64 macos rustflags

* add correct rustflags to setup script

* add fedora libheif deps

* debian libheif deps

* arch libheif too

* add sd-heif as a dep and feature gate it

* enable aforementioned feature in tauri only

* add URI support for heif/heic (quick preview still won't work)

* correct feature gating on everything

* dedicated sd-heif crate

---------

Co-authored-by: brxken128 <77554505+brxken128@users.noreply.github.com>
2023-05-20 06:40:37 +00:00
Oscar Beaumont dd3a0ff689
P2P Stream ids (#630)
inbound stream id
2023-05-18 06:54:20 +00:00
Jamie Pine e8895c1a05
[ENG-581, ENG-597, ENG-600] Overview categories (#779)
* added base UI for categories on overview

* update core

* cleanup ui

* Inspector default view if nothing is selected, explorer takes child components, hidden menu tweak if no items are sm:flex

* wip

* somewhat functional

* scroll

* category fixes

* clean category bar

* added config store + made toolbar available on all explorer screens

* clean up overview.tsx

* added counts

* fix inspector bug

* add support for favorites
+ add book extension support

* refactor into smaller components

* Some small rust nitpicks

* fix camel case location_type

* Rust fmt

* fix typescript CI

---------

Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Co-authored-by: nikec <nikec.job@gmail.com>
Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-05-17 20:34:05 +00:00
Brendan Allan 04deb621e9
Replace locations.getExplorerData with search.paths (#812)
* retire locations.getExplorerData from interface

* regen core.ts

* replace tags.getExplorerData with search.objects

* types out the wazooo

* better optional halding

* ternary-ify

---------

Co-authored-by: Jamie Pine <ijamespine@me.com>
2023-05-16 08:43:59 +00:00
Vítor Vasconcellos cf482f5621
[ENG-270, ENG-213] Stabilize release pipeline (#768)
* Attempt at building our own ffmpeg dylibs for macOS
 - Create dockerfile to setup osxcross and required host dependencies for cross-compiling ffmpeg for macOS
 - Create script to setup required macOS dependencies and build ffmpeg for both x86_64 and aarch64

* Improve dockerfile to better use docker's cache
 - Attempt at fixing arm64 build (still not there yet)

* ARM64 sucessfully compiles \o/
 - Use clang for everything (e.g. linker, assembler, ...)
 - Remove -lc++ because it is now handled internally thanks to a patch
 - Apply 3 open PRs to osxcross as external patches to solve some of our problems
 - PR 180 reduces the macports dependencies size by prefering non universal libs for x88
 - PR 314 adds the -lc++ flag for macOS SDKs >= 11.1
 - PR 379 fix macports pulling incorrects deps for SDK11 and adds support for SDK13 (future proofing, as it does nothing for us rigth now)

* Fix relink and copy logic in ffmpeg build script
 - Ensure ffmpeg and dependencies dylibs are correctly relinked and copied over to the output directory
 - Copy includes from ffmpeg and dependencies to allow for dev building against these dylibs
 - Set up symlinks for all dylibs to the common `lib` directory
 - Remove superfluous linker and compiler options
 - Add `-headerpad_max_install_names` linker option to accommodate the required relink changes
 - Remove `--disable-fft` for the arm64 build, as it doesn't seem to be required nowadays
 - Allow `swresample` and `avdevices` to be compiled as they seem to be expected to exist by `ffmpeg-sys-next`
 - Disable ffmpeg program compilation
 - Fix dylibs id to match their new expected location
 - Fix ffmpeg dylibs inter-linkage not being updated
 - Add custom patch for osxcross macports to load already downloaded files from cache
 - Remove the requirement for specifying the darwin version in the Dockerfile
 - Enable docker cache for macports
 - Move arm min macOS version ARG to the top

* Attempt at building ffmpeg with github actions
 - Split osxcross setup into it's own Dockerfile to allow exporting it as a image to Dockerhub
 - Create a github action to build ffmpeg
 - Add osxcross patch for PR 372 for future proofing (Add support for macOS SDK 13.1)

* Remove incompatible step name

* Change FFMPEG_VERSION to string to avoid yaml weird type casting

* Start to use our ffmpeg build in the macOS
 - Adjust the macOS setup script to download our ffmpeg build, also replace installing protobuf compiler from brew with grabbing it from it's own repository
 - Some misc improvement to the *nix setup script
 - Fix ffmpeg build script not copying static libs (for when we may want to statically link ffmpeg)
 - Fix ffmpeg build script not taking into acount possible libraries that are multiple paths deep in /lib
 - Update CI to actions/checkout@v3, v2 is deprecated
 - Update ffmpeg build CI to not xz the artifacts, as github already zips it anyway

* Fix macOS setup not finding the ffmpeg artifact due to a jq script mistake
 - Comment out embeeding static libs in ffmpeg artifact for now, as they take too much space
 - Remove incorrect dockerfile settings from editorconfig

* Fix protobuf download in macOS
 - Fix Framework directory creation not following script's dir but cwd
 - Improve download logic to avoid being rate limited on failures
 - Return to using xz for ffmpeg build, as it saves quite a bit of space

* Replace `OSTYPE` with `$(uname)` for a more standardized way to check which os is running
 - Fix mobile check using `==` instead of `=` in test command

* Fix syntax error in bash 3.x (which is the default in macOS)
 - Fix jq not downloading all deps and being placed in the incorrect directory

* Fix incorrect install_name_tool arguments
 - Remove leading './' when referencing ffmpeg libs during build

* Fix protoc not being executable
 - Fix some error messages

* jq throw input when it is invalid
 - Enable debug log in setup script when running in CI

* Add a dev.js build script for desktop app
 - Remove dll copy from setup-system.ps1, now done by dev.js
 - Minor rework to build.js to improve signals and stdio handling

* Configure cargo to use FFMPEG_DIR and PROTOC envvars on macOS

* Pass GITHUB_TOKEN to macOS setup script in CI

* Replace simple curl with gh_curl in github request

* Fix some ctrl+c/ctrl+v typos

* Add missing ffmpeg deps
 - Fix missing symlinks for ffmpeg libs in the output_dir/libs
 - Change arm64 minimum macOS target to 11.2 due to libbrotli only being available precompiled to this version
 - Add more options to ffmpeg configure in an attempt to reduce its size + improve performance

* Rework ffmpeg build script to create a FFMpeg.framework instead of loose dylibs
 - Disable unused postproc and enabled missing required protocols in ffmpeg configuration
 - Adjust Dockerfile and setup-script to handle the new FFMpeg.framework

* Fix setup script incorrect linking logic for FFMpeg.framework

* Forgot to remote the leading path in the lib/header linking logic for FFMpeg.framework

* Enable size optimization for ffmpeg
 - Remove Frameworks directory before setting it up in setup-system.sh

* Revert lib id to use `@executable_path/../Frameworks/` instead of `@rpath`
 - `@rpath` breaks dev builds
 - Add logic to create less versioned versions of dylibs in setup-system script, due to it being required for compiling ffmpeg-sys-next (not required during runtime, just for compilation)
 - Add scripts/deps to gitignore

* Fix ffmpeg build script not copying unversioned symlinks to FFMpeg.framework
 - Remove unecessary unversioned symlink logic from setup-system
 - Add new build scripts for dev and prod that setup the environment and tauri to correctly point to FFMpeg.framework
 - Rework windows setup-system script to follow the same logic as in macOS setup-system

* Attempt at fixing windows setup script
 - Add powershell editorconfig

* Attempt to fix Windows CI

* Fix some mistakes in the build scripts
 - Replace toml lib with something more used and maybe better?
 - Attempt to fix Windows CI

* Fix some more problems with the build script

* Another attempt at fixing windows CI

* Some more fixes for the windows setup script

* Add build.js step to release CI
 - Implement patch only mode for build.js when running under CI
 - Implement appeding extra env in cargo config in env.js
 - Only run windows dlls copy in dev in env.js
 - Another attempt at fixing windows CI

* Maybe this will make the windows CI happy?

* Windows CI why dont you work, whyyy?

* Try to apease the Windows CI Gods
 - Disable updater build in macOS, due to tauri-apps/tauri#3933

* Fix build.js extra env vars
 - Uncomment parts of setup-system.ps1 to check if it keeps working

* Uncomment another section and see if windows CI will complain

* uncomment some more of setup-system.ps1

* Use Start-Process for robocopy instead of executing it directly
 - Only raise an error if robocopy exists with a status code >=8
 - Revert build.js macOS specific bundles change
 - Disable updater for now

* Remove updater from cargo.toml

* Add -PassThru to Start-Process to ensure we get and Process obj back

* Comment out more of updater

* Improve documentation for both of the docker containers
 - Fix a harcoded install_name_tool call in the ffmpeg-build-macos.sh script

* Configure tauri to copy ffmpeg dlls to windows prod version
 - Fix windows dlls copy on dev
 - Stop replacing tauri.conf.json, except on CI, for prod builds and use a tauri.conf.patch.json for any required changes
 - Don't unset macOSPrivateApi on prod build, we need it for the transparent background
 - Fix dev.js not exiting when the spawned command was over

* Fix windows not find protoc
 - Fix missing import in env.js

* Fix Windows Dlls copy logic for prod build
 - Fix dev script failing due to missing env
 - Implement error handler to spawned process

* Format + Fix pnpm-lock

* Fix video thumb generation failing on Windows due to bad path to CString conversion logic

* Fix mobile build
 - Greatly improve windows setup-script error handling
 - Install LLVM with winget instead of downloading exe from github

* Improve CI a bit
 - Fix mobile CI not using github token to avoid being rate limited

* Fix pnpm not accesible to actions/setup-node

* Skip pnpm check on CI

* Fix pnpm skip not working
 - Fix pnpm cache check missing a step

* Only lock action to major versions

* CI really doesn't like running robocopy directly

* Attempt to build our own patched tauri bundler

* Trigger new workflow

* Fix attempt for patched tauri action tests
 - Disable cache for patched tauri action
 - Enable package publishing for patched tauri action

* Attempt fix patched tauri publish

* Remove commented code from patched tauri workflow
 - Replace deprecated `actions-rs/toolchain` with `dtolnay/rust-toolchain`

* Fix url patching logic for publishing step of patched tauri package

* Fix prepublishOnly command for patched tauri package

* Adjust patched tauri package package.json

* Also patch the specific os/arch sub packages

* Instruct napi to stop doing github releases

* Patch package.json befor build

* Re-enable updater
 - Patch tauri cli to workaround a tauri bug that prevented use from bundling frameworks in the updater
 - Update tauri/api to 1.3 and tauri/cli 1.3.1
 - Add target triple in the release artifacts name
 - Use ubuntu-20.04 instead of ubuntu-latest
 - Configure release workflow to build to all supported platforms
 - Replace `tauri-apps/tauri-action` with simply calling `pnpm desktop build`
 - Simplify tauri-patched-cli-js to only build the required macOS binaries
 - Unify build.js and dev.js in a tauri.js build script, which now acts as an all around wrapper for the tauri/cli

* Restore incorrectly removed step from `tauri-patched-cli-js`

* Disable auto-run of tauri-patched-cli-js
 - Update setup-system.sh with the latest tauri-patched-cli-js artifacts

* Forgot to mkdir bin

* Fix pnpm-lock.yaml not being up to date
 - Remove pnpm deps caching

* Fix variable name conflict

* Enable the updater build conditionally
 - Enable release workflow to test it

* Fix artifacts not being published
 - Change macOS on arm64 minimumSystemVersion to 11.2
 - Commented out rust cache to check if this is the culprit to the low memory problem on windows runner
 - Add some debug log to tauri.js

* Fix release workflow artifact uploading

* Specify which bundles to build on each platform
 - Attempt fix apple arm build
 - Don't include internal deb archives in linux updater artifact

* Attempt to fix apple arm build
 - Fix dmg background not applying

* Fix incorrect semver usage

* Patch swift-rs to see if that fix apple arm64 release build

* Update swift-rs submodule
 - Remove unused deps lodash.merge

* Configure so only release workflow runs without cache
 - Improve tauri.js logs

* Ignore `crates/swift-rs`

* Revert "Ignore `crates/swift-rs`"
- Remove `crates/swift-rs` before running `cargo fmt`

This reverts commit 851bd84373.

* Github CI input/output are always string
 - Attempt to fix Windows Clippy

* Fix CI syntax error

* Fix mobile CI
 - Disable pnpm deps caching
 - Disable rust targets caching on Windows
 - Configure cache factory to run for all targets

* Remove `crates/swift-rs`

* Attempt fix Mobile CI
 - Enable cache factory run in pull_request to test it
 - Specific more path for the CI run to ignore
 - Specify path that will trigger the cache factory run
 - Some cleanup in the setup-pnpm action

* Remove restore-cache options
 - Add prefix key to cache-factory
 - Fix LLVM cache-hit check
 - Add libheif to FFMpeg.framework

* Remove submodule clone from actions

* Add fake deps to ffmpeg build dockerfile
 - Fake install deps that aren't really necessary to build ffmpeg
 - Fail ffmpeg build script if it can't find a macports deps
 - Copy libheif headers to Framework

* Remove automatic trigger for relase and pull request trigger for cache-factory actions

* Remove pnpm setup from Cache factory
 - Cache factory is exclusivly for rust deps caching, no need to setup Node and pnpm with it

* Re-enabled fail-fast in Github CI
2023-05-15 18:02:56 +00:00
jake 9b2788ed63
deps: remove ffmpeg-next from the core (#808)
remove ffmpeg-next from the core
2023-05-10 04:44:27 +00:00
Brendan Allan 05ff5ed800
Eng 510 open with in context menu (#803)
* somewhat

* proper macos support

* formatting
2023-05-08 09:22:24 +00:00
Oscar Beaumont 29f922ce18
Fix subscriptions (#799)
* fix em

* more fixes
2023-05-06 05:45:33 +00:00
Oscar Beaumont 0d9c1b8329
Fix some stuff (#794)
* Update Tauri

* fix

* fix the entire damn app
2 times in two weeks lol
2023-05-05 16:08:00 +00:00
Oscar Beaumont 2571e3b275
Upgrade rspc TS (#775)
* finally

* should be working?

* fix types

* fix types

* Wouldn't it be nice if Metro would just work

* idk

* try harder Metro

* potentially fix bundling issues

* idk, maybe fix it?

* fix metro

* update podfile.lock

* bruh

* bruhz

* tailwind is drunk again

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-05-04 06:21:42 +00:00
Brendan Allan b620e1e174
paginated location explorer (#774)
* paginated location explorer

* remove props from GridView

* formatting
2023-05-01 09:45:25 +00:00
Brendan Allan 7cee2b2651
update prisma (#773)
* update prisma

* idk
2023-04-30 23:03:23 -07:00
Brendan Allan 82845a8388
Upgrade Prisma (#770)
upgrade prisma
2023-04-28 17:56:08 +00:00
Brendan Allan 3570c5f3d8
File Opening (#762)
* file opening

* eslint disable in gererated commands

* bruh?

* Fixing materialized_path joining and some warnings

* no unwraps

* formatting

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-04-28 06:55:00 -07:00
Oscar Beaumont 9c62df9947 fix type exporting 2023-04-25 23:51:48 +08:00
Oscar Beaumont a4b84d31cc fix desktop 2023-04-25 22:47:41 +08:00
Oscar Beaumont fed3f2211f
Rspc v3 syntax (#758)
just fixed a couple of lines
2023-04-25 17:12:42 +08:00
Oscar Beaumont 85d0ddf6d3
rspc 0.1.3 prep (#756)
* Upgrade rspc + Specta + PCR

* fix jsonrpc executor

* fix invalidate excessive logging

* fix `Protected` type

* fix mobile

* clippy

* lol
2023-04-25 15:14:52 +08:00
Brendan Allan b5a3e05c9a
Update server (#744)
* release workflow

* tauriScript

* pnpm deps

* unify rust cache keys

* bruh

* basic manual updater stuff

* move updater override

* use existing publishing action

* use debug builds for desktop

* cache release builds too

* use release builds instead

* add workflow dispatch trigger

* remove branch from release workflow trigger
2023-04-21 10:02:31 -07:00
Brendan Allan 469a3831dc
Explorer search (#738)
* basic file path search

* cleanup

* render explorer in search page

* update materialized path docs
2023-04-21 14:49:46 +00:00
jake 288f4947e1
remove int-enum (#721) 2023-04-19 01:33:32 +00:00
Vítor Vasconcellos 3fbc2b909c
[ENG-461] Location creation fails for '/' (root) and 'C:' ([A-Z] drives) (#710)
* Fix issue with root locations failing to be added
 - Add `normpath` dependency to the core
 - Improve automatic name resolution for new locations

* Add `MetadataExt` trait to extend the `created` and `modified` methods of `Metadata`
- Prevent failure in the Indexer job caused by paths without associated created or modified times; assume current time for both when unavailable
- Rename possible location in root path from `/` to `Root`

* Improve hadling of location path on Windows
 - Normalize location's path to ensure a valid format on Windows
 - Improve error handling when adding a path
 - Avoid consuming symlinks on Linux by not normalizing location's path
 - Improve location naming logic
2023-04-18 06:12:02 +00:00
Oscar Beaumont 2dd3d6f921
Job system improvements v2 (#707)
* Add `JobInitData` trait + change `StatefulJob::name` to a constant

* `StatefulJob::new`

* Move `invalidate_query` into `StatefulJob::finalize`

* Cleanup `spawn_job` signature

* Remove `queue_job` and run jobs from the `StatefulJob::finalize` method

* `StatefulJob::queue_jobs`

* `JobManager::ingest` return `Result`

* Remove `jobs.isRunning`

* Invalidation system direct push + batching invalidate

* Look ma, only a single clippy warning!!!!

* Error handling for JobManager

* Rust fmt

* Introducing Job hierarchy to enable job enqueuing

* Rust fmt again 🙄

* core.ts

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-04-15 14:43:04 +08:00
jake 39e92cc7e7
update to ffmpeg 6.0.0 (#697) 2023-04-13 14:10:23 +00:00
Oscar Beaumont 4c0e341e55
Add MetadataManager into sd-p2p (#686)
* Add `MetadataManager` into sd-p2p

* Add `update_metadata` helper to P2P manager

* Praise Clippy our lord and saviour

* change mdns advertisement log level

* Casually broke Linux
2023-04-11 10:20:14 +08:00
Vítor Vasconcellos 3304e8f6ce
QuickPreview Component (Needs test on MacOS) (#665)
* Add QuickPreview Component
 - Improve the handling of Range requests
 - Implement logic to answer HEAD and OPTIONS methods
 - Handle CORS pre-flight requests
 - Expand accepted file types
 - Improve error handling of invalid Range requests

* Fix linter errors
 - Add `use std::cmp::min` to custom_uri (Required on MacOS & Windows)
 - Improve logic for retrieving file information in QuickPreview.tsx

* More linter errors

* Simplify `QuickPreview` by extracting the logic for choosing the file preview tag to a `FilePreview` component
 - Fix the typo in `QuickPreview` props name
 - Remove the unused `handleMedia` ref
 - Move the remaining `QuickPreview` logic to the `transitions` callback
 - Simplify the `cors` return type in `custom_uri.rs`

* Refactor range handling in `handle_file` function
 - Move range handling logic to the initialization of the `range` variable
 - Replace `if let` with `match` to reduce code duplication
 - Don't export FilePreview
 - Export QuickPreviewProps

* Fix typo in `RangeNotSatisfiable` error message
 - Remove redundant variables

* Fixing cas_id generation on watcher
Some improvements on watcher file creation

* Rust fmt

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: Jamie Pine <ijamespine@me.com>
2023-04-05 21:15:13 -07:00
Utku 54a2eee827
Mobile CI with Tests (#631)
* stuff

* stuff (but for android)

* test mobile ci

* move pnpm up

* install ndk

* add ios & cleanup

* onboarding flow test

* test ci

* fixes, cleanup, caches

* why you do this cargo

* fix pnpm-lock

* add path to build rust script?

* ci is fun

* yolo

* fix broken flow..

* fix pnpm

* probably not gonna work

* test x2

* use real branch of pcr

* android emulator and try ios x2

* Use react native architectures, instead of all.

* override architecture to speed up android build

* protoc & build android on macos too

* fix java ndk

* android gradle

* disable ios for now

* use simulator sdk & debug configuration on ios build

* cleanup

* avd test

* fix avd settings.

* only build for x86_64 on ci

* Fix ios build

* Add IOS testing

* maestro script

* ios release build & wait for library creation

* clean up and disable android for now

* fix pnpm-lock

* Add concurrency to cancel previous runs

* fix pnpm-lock

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-04-03 08:12:28 +00:00
Ericson "Fogo" Soares b346e7ac52
Eng 372 location awareness improvements (#612)
* Complying with a pedantic Clippy

* Solving duplicated directories events
When creating a directory through MacOS's Finder, for some reason
FSEvents receives 2 Create Folder events that we have to handle

* Handling moving to trash bin and restoring on Mac
Still missing the feature to restore a directory and its children

* Now handling creation of empty files on MacOS

* Enabling restore of directories and its children

* Now working: moving a directory to another
inside the same location

* Now Indexer also remove file_paths not on fs

* Enabling multiple file moves on location watcher

* Fix Windows conditional compilation issues

* Fixing cas_id generation and bumping some deps

* Many Windows specific improvs and some refactors

* Rust fmt

* Using conditional compilation on extract inode function

* Linux fixes and some MaterializedPath improvements

* Rust fmt again

* Introducing tick behavior on location watchers

* Making LastFilePathIdManager more atomic

* Some vscode launch configs for lldb debugger

* Simplifying some lifetimes

* Making all watchers more consistent
2023-03-31 04:59:33 +00:00
Vítor Vasconcellos c84e28189c
Fix thumbnails and file previews on Linux (#635)
* Fix thumbnails and file previews on Linux
 - Restructure auth_middleware to use axum's primitives to handle the auth tokens
 - Fix incorrect token loading in App.tsx
 - Add required dependencies for video playback to the Linux setup script
 - Minor improvements to the Linux setup script, make it less error prone
 - Fix a minor bug in the script_failure function in the Linux setup script

* Use a more idiomatic rust syntax in auth_middleware
2023-03-28 21:19:11 +00:00
Utku 902a3b5ba1
Onboarding, Spacedrop & Location Settings Screen & Styled API (#596)
* fix wrong current lib logic

* add delete lib dialog to LibraryGeneralSettings

* add delete lib to mobile LibraryGeneralSettings too

* onboarding screens

* move zxcvbn to @sd/client

* get started screen and bloom

* merge fix

* move generatePassword back to interface

* add useZodForm to mobile and match react-hook-form versions

* new lib screen

* Implement styled api

* create lib screen and some tweaks

* password input

* fix password meter comp

* new library style tweaks

* Fix remove password bug (interface)

* master password screen

* privacy screen

* creating lib screen

* hexagons are cool

* Expo 48

* keyboard handling

* fix P2P on IOS

* fix types

* asset script

* new icons

* Spacedrop screen

* Fix mobile asset imports

* fix import cycle warning

* Edit Location Settings screen and style changes on other setting screens

* fix library creating bug? hopefully lol

* move PasswordMeter to interface

---------

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2023-03-17 02:00:02 +00:00
jake 171f19bc71
[ENG-429] Crypto organization/refactor (#607)
* update comment

* key manager feature gating

* update deps & more feature gating

* fix keyring feature gating

* add an `exhaustive_read` function for crypto/stream

* restructure STREAM module

* move tests+shared utils

* clean up stream decryption

* further cleanup

* impl to `GenericArray` for `Nonce`

* update examples

* update refs & `use`s

* fix `Nonce` -> `GenericArray` conversions

* better `Protected` conversions + remove `Password` type

* a work of art

* finishing touches

* some API changes

* rename `StreamX` to `X`

* fix everything else

* separate `primitives` from `types`

* update imports & fix build
2023-03-15 16:09:36 +00:00
Brendan Allan 53229809ca
update swift-rs (#605) 2023-03-13 07:53:28 -07:00
Oscar Beaumont f746530289
P2P Core (#597)
* implement space time

* I love it when my code works
2023-03-09 03:18:33 +08:00
Brendan Allan 31df51501e
CI Cache Factory (#593)
* introduce cache factory

run cache factory on all branches

add names

use single quotes

add setup and prisma generation

add pnpm install

don't install cargo-watch in ci

make composite setup script

use env var for windows ci

undo all powershell stuff

* better windows ci

* don't install cargo-watch in ci

* only run cache factory on main

* i'm silly

* debug only ci

* check runner instead of matrix in setup action

* don't save cache output most of the time

* no more os-specific features

* cache entire workspace

* only cache on main again

* test instead of buidl in cache factory

* smaller ci + testing

* don't individually test each crate

* remove unnecessary needs

* smaller formatting and more clippy

* undo rustfmt changes

* disable tests
2023-03-06 06:10:55 -08:00
Brendan Allan d89240d9e2
Colocate native code + reorganise (#589)
colocate native code + all rust in crates/
2023-03-03 09:22:22 +00:00
jake 4cc582e08c
[ENG-134] OSS Licenses and Projects (#575)
* add `deps-generator` crate

* move `clap` types to separate file

* idiomatic filtering

* experimentally update ci

* add *really* basic dependency page

* remove `license_id` from `License`

* add bare minimum JSON files

* Revert "experimentally update ci"

This reverts commit c04897d902.

* re-insert comments

* Using reqwest blocking feature to avoid asyncness

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-02-20 05:58:55 +00:00
Oscar Beaumont a9fceae819
improvements to spacedrive:// custom URI protocol (#550)
* fix `spacedrive://` custom protocol on Windows
(hopefully)

* custom protocol using `http::Response` + fix broken web

* import patches before App on web

* use `http::Request` for input to `handle_custom_uri`

* break  into dedicated file + error handling

* serving files via custom protocol

* cargo fmt because vscode did cringe

* lru cache to reduce video chunk request time

* add  helper to JS

* clippy be like

* remove duplicate Open buttons in context menu

* fix Linux 🙏

* no shot

* fix Windows custom URI passing (hopefully)

* better fix for custom uri on Linux

* upgrade Tauri for  feature

* switch url replacement order

* prevent React dev tools script being added in prod to desktop

* remove React devtools from html

* upgrade Tauri; required upgrading rspc, Axum, PCR

* pass typecheck + less cringe bigint

* clippy is love, clippy is life

* Typecheck plz

* fix bigint to number conversion

* use httpz + localhost server for Linux

* clippy be right

* Remove console.log

* [wip] proper auth

* fix Linux sidebar padding

* Secure Axum server with random

* Extracting app setup specific to linux to a different file

* remove outdated comment

* Some tweaks on cursom_uri.rs

* file_path_with_location doesn't need to be a named include

* fix typo

* factually wrong comment

* Change `unwrap` to `expect`

* bruh

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-02-14 13:27:11 +08:00
Jamie Pine f47a2d58e5
[ENG 239] Onboarding Flow & Location Settings (#529)
* begin better onboarding

* added input and altered text

* better router & text + database icon

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>

* work on privacy screen + radio buttons

* fix video extension bug and alter screens

* add pending schema and location manager helper

* functional onboarding

* added secure temp store and started creating library loading screen

* fix secure temp keystore + api

* better onboarding

* added location settings and some overview concept, all WIP

* fix switch

* prep

* fix location router

* added backend settings

* attempted to fix form

* begin indexer rules editor, plus tweaks

* indexer rules coming soon

* fix onboarding img size

* cleanup

* clone is needed here, but clippy no like

* sike

* whole bunch of fixes

* clippy + ts

* Removing some TODOs from api/libraries.rs and fixing db size calculation

* moved object kind to client, added half functionality for appearance settings

* fix RadioGroup helper

* fix type issues

* cargo fmt

* fix creating library error handling + invalidate location list on update

* forgot to switch back to onError

* Invalidating getStatistics query on library creation and introducing the concept of waiting for a job on FileCopierJob

* F* cargo fmt

* fix RadioGroup interactivity

* wipe all migrations

* put back COLLATE NOCASE on extension columns

* update core.ts

* remove unused device component

* fix typeerror in mobile

---------

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-02-10 14:08:13 -08:00
jake d1b6263ae7
[ENG-355] Keychain integration (and some typesafety) (#558)
* update crypto MSRV

* rename `keychain` to `keyring`

* make a start on the keymanager unlock refactor/keychain integration

* update routes

* update bindings

* add const identifiers

* add UI/front-end support for unlocking KM with OS keychains

* remove SK from lib creation dialog

* update query name

* add keyring functions

* attempt to update `change_master_password()` to use the keychain

* cleanup, fix master password change ui, better secret key in keyring detection

* cleanup TS a little

* add route for getting secret key from keyring

* update bindings

* update var names + show secret key in keys settings

* add `react-qr-code` and option to view the secret key (if it's in the OS keyring)

* allow copying SK to clipboard

* add `key_type` so we're not reliant on specific UUIDs for root/verification key handling

* clippy

* fix mobile typecheck

* fix typecheck, fix typo and tweak balloon hash parameters

* minor cleanup + typo fix

* use newtype structs

* WIP type refactoring (major readability boost!)

* update `use`

* add tokio `sync` feature

* too many structs? idk

* more cleanup

* add `generate` and `Nonce`

* `Nonce` and `Key` typesafety (beautiful)

* clippy + cleanup

* update code & examples

* fix bug & remove `ProtectedVec` as it looked out of place

* use `Key`

* add a query invalidation to make the UI extremely responsive

* ci pls work

* remove `keyringHasSk` route
2023-02-07 12:03:12 +00:00
jake 9fb01fc9d2
[ENG-341] OS Keychains (#554)
* super barebones keychain start

* working+unfinished linux keychain

* add `copy` to `Identifier` and add `delete()`

* add generic errors + code cleanup

* cleanup code & add support for apple keychains

* remove `users` dep

* use uppercase UUID and tweak apple `account` variable

* revert uppercase change and clean up linux labels

* code cleanup & add a readme to the crypto crate

* remove useless `map`

* correctly handle keyring not supported errors

* add `Send` to `Box<dyn Keyring>`

* remove useless `cfg`s

* only return `Ok()` if we support keychains (pls work CI)

* clippy

* Minor fixes

---------

Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
2023-02-01 00:11:03 +00:00
brxken128 c3c51afcb8 remove tokio git patch and update to 1.25.0 2023-01-31 23:38:33 +08:00
brxken128 3f2a62feb4
Merge branch 'main' into eng-348-async-crypto 2023-01-25 22:02:23 +00:00
brxken128 dbe9b9157c
use spacedriveapp/tokio 2023-01-25 15:00:24 +00:00
brxken128 a3b1c8f5b8
tell everything to use workspace tokio + add tokio git patch 2023-01-25 14:36:29 +00:00
brxken128 d8b6789cb7
update cli to use async crypto 2023-01-24 16:19:30 +00:00
brxken128 3fd547ab2d
disable benchmarks as they need async support 2023-01-24 14:25:42 +00:00
Brendan Allan 3885a2cd84 sparate android + ios build targets 2023-01-21 14:31:17 +08:00
jake 0a31e7f8ce
[ENG-257] File erasure and deletion (#524)
* add helper functions for getting context-menu related filesystem info

* update comments and such

* add a barebones delete job

* add file deletion route

* add (empty but working) delete file dialog

* add super experimental file erase function

* update function doc

* add erase files route

* add erase files job

* tweak file erasing in crypto crate

* bindings

* add erase files dialog

* add missing `?`

* fix erase file form trickery

* remove unused imports

* terrible attempt at recursion for file erase

* IT WORKS

* code cleanup + clippy

* async just dropped (not for erase yet)

* async erase just dropped (+async crypto crate)

* add "nice data" for jobs
2023-01-18 19:55:14 -08:00
Brendan Allan 1041cb6e3c
use latest swift-rs (#526)
* use latest swift-rs

* update cargo lock
2023-01-17 06:37:32 -08:00
Ericson "Fogo" Soares 31b5c4b1db
Merge pull request #517 from spacedriveapp/eng-304-too-many-files-macos-unhappy-with-prod
[ENG-304] "Too many files" macOS unhappy with prod!
2023-01-16 20:03:45 -03:00
Brendan Allan ae94ada4f8
Begin form abstraction (#515) 2023-01-12 07:26:59 -08:00
Ericson Fogo Soares cb8820fe47 Checking MacOS file system events using FSEvents instead od Kqueue 2023-01-11 18:57:41 -03:00
Brendan Allan de3a979127 update prisma 2023-01-06 01:19:19 +08:00
Brendan Allan 58f5d8bb4d
update prisma (#505) 2023-01-05 08:05:33 -08:00
Brendan Allan 676e512c4a
Beginning of sync stuff (#502) 2023-01-04 23:11:55 -08:00
jake 2673a233c2
[ENG-320] Key manager bugfixes (#501)
* force key updates once a new key is mounted

* set KM decryption type if a single key is mounted

* prevent the key viewer dialog from changing size based on encryption algorithm

* fix spacing for the key mounter slider and remove unfitting text

* bump rspc version (fixes a lot of KM concurrency bugs)

* add personal reference for rspc override

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2023-01-05 12:52:34 +08:00
jake 22bc77a39e
[ENG-319] Balloon hashing (#489)
* add key attribute to `Key` in `ListOfKeys`

* add balloon hashing function with untailored parameters

* add ser/de rules for blake3-balloon (and change argon2id's)

* fix benchmark

* use `to_bytes`, `from_bytes` and `from_reader`

* cleanup code

* add blake3-balloon options to the UI and fix library sync/automount enable bug

* cleanup some serialization code

* fix hashing algorithm deserialization

* clean up header serialization + more idiomatic master key decryption

* clippy

* add generic ser/de error to crypto crate

* fix `Display` and crypto cli

* move crypto cli to cli app

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-01-04 09:57:02 +00:00
jake 4569e45e47
[ENG-312] Crypto details CLI tool (#500)
* add untested crypto details tool

* clean up output a little

* clippy

* update crate details and `clap`
2023-01-04 00:15:12 +08:00
jake db5b401238
[ENG-318] Implement a KDF to derive new keys (#487)
* add blake3, `derive_key()` and clean up code

* fix a couple of things from the previous commit

* add context strings for root/file key derivation

* add salt to schema

* update refs of `salt` to `content_salt` within the keyslot

* cleanup code and add kdf salt to the keyslot

* rename salt to content salt in examples

* cleanup header code + remove dead code

* implement key derivation for keyslots

* gen new migrations that contain a salt column

* keymanager refactor (code is very idiomatic now) - needs thorough testing

* further cleanup

* clippy

* add a master password context string

* use key derivation for deriving keys from the root key

* update to use new code and remove `match` from `en/decrypt_bytes()`

* clippy

* use less unwraps in library manager code

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2022-12-30 20:09:44 +00:00
Ericson "Fogo" Soares bcbcd260d4
ENG 223 Location Awareness (#468)
* Introducing location online/offline checks and location relink

* Some initial drafts for location watchers

* Location metadata relink and add library

* Many improvements at job system
Now using prisma batching at identifier job
Removing blocking I/O from extension subcrate
Implementing lazy generation of thumbnails
New current directory identifier job to be used on light rescans

* Some optimizations on identifier and object validator jobs

* merge jamie's identifier PR

* fully repaired identifier job

* properly hooked up object kind

* inspector fix

* fix video badge

* small improvements to libraries settings

* identifier and inspector improvements

* fix feature flags and hook up context menu location utilities

* BETTER CONTEXT MENU x100

* test-files

* style tweaks

* new icon designs

* manifest

* fix thumbnails on web

* media data

* New Location Watcher and some minor fixes

* disable broken media_data extractor, wip

* wip

* function name fix

* Fixing pnpm prep and some warnings

* Solving a race condition beetween indexer job and FS event handlerSome other minor warnings

* Generating thumbnails on watcher

* Remove event handler on watcher

* Some initial works on modify events and other small fixes

* File update event

* Trying to be more generic with used events and some tests to validate our assumptions

* Turning on location metadata file

* Introducing core unit tests on CI pipeline

* Submiting new unit test assumptions to validate on windows CI

* Fixing unit tests

* Fixing unit tests again

* Fixing unit tests

* Fixing unit tests for macos

* Fixing unit tests for macos again

* New structure for platform dependent event handling
Implementing event handlers for Linux and MacOS

* minor fixes + rustfmt + clippy

* Windows event handling

* Introducing a feature gate to only use location watching on desktop app for now

* Putting more stuff behind feature gates to avoid warnings

* Adding feature to cargo test on CI

* Changing some debug logs to trace logs and removing Jamie specific stuff

* Make location removal from manager less async

* fix build when "location-watcher" feature disabled

* fix types + clippy

* make location manager non-static

* remove uses of `to_string_lossy`

* more invalidate_query calls

* Expose `library_ctx` directly to avoid needless clones

* New materialized_path handling for directories

* Removing cascade delete between file_path and object
- Some other minor stuff

* remove unused `CurrentDirFileIdentifierJob`

Co-authored-by: Jamie Pine <ijamespine@me.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2022-12-31 00:53:24 +08:00
jake 0eb0376d5b
[ENG-308] Crypto optimizations (#470)
* add benchmarks for encryption/hashing

* add more benchmarking sizes

* general performance improvements

* massive performance gain with minimal security impact

* update `sd-crypto` author info
2022-12-08 12:08:35 +00:00
brxken128 4449bcb143
code cleanup and use hex for secret key encoding 2022-12-07 15:00:14 +00:00
jake 2baf16d982
[ENG-307] Key manager features (#467)
* working key management in settings page

* sync with library button

* fix `rspc` feature and add passphrase generation

* untested key manager rework

* trying to return values from mutations

* update library manager and remove settiong master PW

* update bindings

* set static secret key/master password

* prompt user for master password if correct one hasn't been provided yet

* add `hasMasterPassword` route

* add `clearMasterPassword` route + remove dead code

* tweak `set_master_password()` and add dedicated error

* tweak UI, fix `few hooks than expected`, add unmount+lock button

* remove old comment

* fmt

* clippy

* move static key/password setting so it doesn't fail sometimes

* add dedicated `get_key()` and remove keys from memory

* add `getKey` route

* update bindings

* use `const` instead of `let`

* comment updates

* update schema to remove salt

* add string parse error

* generate passphrase within key manager

* generate new migrations

* feature gate serde support in `crypto` crate

* fmt

* more specific error types

* foramatting

* add locking mechanism to keysettings page (not working?)

* fix react hook issues

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>

* remove empty onclick

* add keymanager dropdown menu

* working key backup functionality

* add experimental master password changing support

* update bindings

* semi-working change master password dialog

* use gear/lock icons to clean up key manager UI

* make settings button functional

* make buttons uniform and format code

* fix double base64 encode

* add change master password dialog and secret key dialog

* code cleanup

* restore backup dialog

* change UI wording

* make a start on restoring from a backup

* potentially working keystore restore

* don't overwrite verification key if one is set

* working backup restore + fix master password changing

* fix typo in static password/verification key check logic

* change wording to make UI clearer

* disable mount button if key is empty

* handle errors+remove type annotations

* show total imported keys on backup restoration

* add zxcvbn package

* change input border colour based on zxcvbn score

* clippy and formatting

* password strength meter

* remove nbsp

* add button type to stop early form submission

* use `react-hook-form` for backup restoration dialog

* more `react-hook-form` stuff

* attempt to fix password meter

* small cleanup

* Fix password meter

* update colours

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2022-12-02 10:18:21 +00:00
jake a403224b3b
[ENG-262] Key Manager Integration (#450)
* add keys router

* make progress on keymanager integration

* make name non-negotiable

* hyphenate encryption algorithm names

* Revert "make name non-negotiable"

This reverts commit 9c0f51329e.

* add some more keymanager queries

* add master password and default key routes

* add newly registered key to db + fmt

* clippy, formatting and `updateKeyName` route

* add automount to schema+automount keys where `true`

* update bindings

* working key add/mount

* working keylist

* mounted keys show first

* cleanup code

* add comments, code cleanup, more functions

* unmount all keys button

* comment and keymanager `clear_master_password()`

* add no keys available message

* fix unmount button

* use dashmap for concurrency

* fix missing keylist issue and add invalidate query macro

* set correct RSPC types

* statically set master password (TEMPORARILY)

* add remove key function within the keymanager

* key dropdown menu and impl

* formatting

* allow `option_if_let_else`

* add comment about key stats

* add additional comment

* rpsc error handling for the keys route

* fix rspc errors with an impl

* crypto crate errors to `sd-crypto::Error`

* remove `map_err`

* use custom result type

* cargo fmt

* clippy

* fix builds

* remove `Error::MutexLock`

* fix unnecessary unwrap

* mutex error handling (buggy for some reason)

* clean default key logic

* fix default key clearing

* allow a key to be removed without bugs

* implement requested changes

* use a single `useMemo`

* update schema with defaults

* re-generate migrations

* use rust enums in TS

* remove dead code

* remove mutate expansion

* read key list from keymanager, not prisma

* add "Default" key marker and cleanup TS

* rustfmt

* remove dead code
2022-11-05 04:18:01 -07:00
Brendan Allan b713c896a0
More Sync Stuff (#438)
* broken sync example

* move to react

* filter owned ops more

* update deps

* working sync

* relation operations

* fix pnpm lock

* please rustfmt

* fix api

* compare strings properly

* please clippy

* feature gate logging

* use bleeding rspc in example

* use bleeding rspc

* use generated rspc client
2022-10-27 23:12:28 -07:00
Brendan Allan 5ceb27c95f
Update Prisma (#441) 2022-10-27 01:56:18 -07:00
Jamie Pine 3a3ec6251f Merge branch 'main'
Conflicts:
	packages/interface/src/screens/settings/library/LibraryGeneralSettings.tsx
2022-10-23 18:43:27 -07:00
jake 0c7aed5f86
[ENG-258] Key Manager and Crypto Crate Improvements (#423)
* add base keymanager structs/functions

* change md/pvm `new()` to prevent useless re-hashing

* update lockfile

* update keymanager

* cleanup code, make things easier to understand

* move md and pvm construction to the header

* update pvm ser/de

* update metadata ser/de

* additional API changes, update example

* formatting and clippy

* update examples

* move `impl` to associated files

* formatting+clippy

* add more keymanager functionality

* add docs

* wrap `master_password` as an option

* add `sd-crypto` crate as a dependency to `core`

* add key manager to `LibraryContext`

* rename `id` -> `uuid`

* add more keymanager functions

* add `set_master_password()` function

* function to see if keymanager has master password

* update schema

* add default bool to schema

* populate keystore on SD startup

* clippy+formatting

* implement requested changes

* remove unwrap

* fmt, clippy, remove ser/de derives

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2022-10-23 03:47:41 -07:00
Brendan Allan 61cb749c07 remove openssl from core 2022-10-22 05:43:39 +08:00
Jamie Pine c4226f0e7a Merge branch 'eng-259-identify-objectkind'
Conflicts:
	apps/landing/package.json
	apps/landing/vite.config.ts
	packages/interface/src/AppLayout.tsx
	packages/interface/src/components/explorer/Explorer.tsx
	packages/interface/src/components/explorer/ExplorerTopBar.tsx
	packages/ui/src/ContextMenu.tsx
	pnpm-lock.yaml
2022-10-20 22:49:15 -07:00
Oscar Beaumont aae3da9b8a 🎉 Fix core on Android 2022-10-18 20:57:17 +08:00
Oscar Beaumont 1a8581d44f upgrade rspc + fix reactivity + add in disabled Normi code 2022-10-18 04:45:37 +08:00
jake 4cd2dde35c
CAS ID Improvements (#413)
* remove `ring` dependency and use `sha2` instead

* use BLAKE3 and include full file checksum

* update schema comments
2022-10-13 15:31:47 -07:00
jake b5c571541e
[ENG-256] Crypto Crate Refinements (#410)
* add experimental preview media support

* clippy+formatting

* add support for mk/pvm nonces

* fix cursor positioning+size deserialization bug

* clippy+formatting

* cleanup code and add `decrypt_preview_media()`

* update comment

* remove `Mode` from `Keyslot`

* use little-endian `usize` for media length

* remove memory objects

* remove mode enum

* add `.zeroize()` to `Protected<>`

* remove `Mode` ser/de

* refactor, remove `Mode`, add helper functions

* formatting

* remove unused error

* update comments/docs

* comments, fix large files, insane performance

* doc updates and fix decrypt zeroize

* revert to 1048576 byte `BLOCK_SIZE`

* `Keyslot` and `PreviewMedia` constructors now handle encryption

* add metadata item with associated functions

* update comments, clean up useless items

* add metadata support within the header

* remove stray 128 in length calcs

* include metadata in header ser/de

* fmt+clippy

* rework keyslot decryption

* formatting

* api changes, code cleanup

* docs for `stream.rs`

* massive documentation re-write

* mark expensive doc test as ignore

* minor api tweaks

* add examples

* formatting+linting

* finalise documentation and add more examples

* formatting
2022-10-13 15:25:42 -07:00
Oscar Beaumont c0b51bcd4a update rspc to version with fixed httpz rev
Main was broken because it was pulling httpz from main and I have been refactoring it.
2022-10-13 15:03:43 +08:00
Jamie Pine b94d2ed736
(hotfix) p2p preventing compile 2022-10-08 11:30:17 -07:00
jake d236b28f84
[ENG-250] Crypto Library (#400)
* add crypto crate with some functionality

* formatting

* add `argon2id` parameter levels

* add descriptive comments

* add stream decryption objects

* add `StreamEncryptor` struct

* add `StreamDecryptor`

* general cleanup

* add `thiserror` and error handling

* add header structs

* add basic serialization functionality

* advance serialization

* finish serialization

* clean up serialization and use `impl`

* finalise deserialization

* add stream helper functions and remove old code

* add AAD creation and retrieval

* add important comment

* add `ChaCha20Rng` as a CSPRNG

* cleanup and crate-wide clippy lints

* apply nursery lints

* add in-memory encryption objects

* rename `utils` to `objects`

* move (de)serialization rules to separate file

* add header-write helper function

* add password hash helper function

* add `decrypt_master_key` function

* cleanup, formatting, linting

* move keyslots to separate file, and rename them

* add basic comments

* remove `secrecy` dependency and import `protected`

* add `to_array` helper function

* `sd_crypto` -> `sd-crypto`

* remove manual drops

* add clippy allows

* add `new()` for `Keyslot` and `FileHeader`

* remove license

* zeroize read buffer on error

* magic bytes are now `ballapp`

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2022-10-07 07:31:40 -07:00
Jamie Pine 900ee618ce
Merge branch 'main' of https://github.com/spacedriveapp/spacedrive 2022-10-06 18:10:08 -07:00
Jamie Pine e402fc52b7
sd extensions crate
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2022-10-06 18:10:03 -07:00
Oscar Beaumont 75c4680b43 Merge remote-tracking branch 'origin/main' into merge-p2p-code 2022-10-07 05:56:05 +08:00
Oscar Beaumont d7fddc83f7 fix major bugs in p2p PR 2022-10-07 05:51:40 +08:00
Oscar Beaumont 598ba38a9d merge in P2P code 2022-10-07 05:32:45 +08:00
Oscar Beaumont dec2697740 Fix mobile + upgrade Expo + even newer rspc 2022-10-06 21:07:46 +08:00
Oscar Beaumont 20523a94c9 upgrade workspace dependencies 2022-10-06 14:03:58 +08:00
Brendan Allan e5359f042d please formatter 2022-10-05 12:14:04 +08:00
Brendan Allan 45ff212e97 merge main into update-rspc-and-more 2022-10-05 12:01:50 +08:00
Brendan Allan b827e024ba update lockfile 2022-10-05 10:20:04 +08:00
Brendan Allan a634727e1b merge meain into update-rspc-and-more 2022-10-05 10:11:52 +08:00
Brendan Allan 162f3d7782
Improve CI Speed (#398)
* use different cache

* cache prisma client in rustfmt

* cleanup prettier config

* build server image in sub-action

* server image + prisma client custom actions

* remove lefthoook checks that are in ci

* fix build-server-image

* rename Clippy job

* rename build core job

* use custom rust caching action

* fix rust cache shared key

* cleanup custom action job names

* remove platform check in clippy ci

* setup pnpm in clippy ci

* update mobile package name

* make ios compiling conditional

* use pnpm cache

* run clippy in release

* force use release for all jobs
2022-10-04 18:45:54 -07:00
Brendan Allan 723058e060 merge main into update-rspc-and-more 2022-10-05 08:58:03 +08:00
Brendan Allan 0821ef575a update pcr version 2022-10-05 04:20:59 +08:00
Brendan Allan ed06e3051e
[ENG-84] Sync library (#394)
* new sql lib stuff

* add sync stuff + rename all crates

* build sd-core

* add sync/example/dist to source

* fix sync example in monorepop

* appease clippy

* update lockfile

* update commit hooks

* fix typescript

* fix typescript build

* please rustfmt
2022-10-04 05:25:12 -07:00
Brendan Allan 66dd9aa28b
move prisma schema and migrations to /core (#392)
* move prisma schema and migrations to /core

* fix prisma updates
2022-10-04 02:32:46 -07:00
Oscar Beaumont 60e228ec94 upgrade rspc 2022-10-02 01:16:11 +08:00
Jamie Pine 317ed2d7b0
removed stray dependency causing build issue 2022-09-29 21:24:42 -07:00
Jamie Pine 3e8c33d1c0
Added crates folder (#389)
* restructure core for crates folder

- improve naming consistency
- added placeholder files for near future code
- added docs

* - removed all typescript from core
- moved rspc bindings to client

* remove useless git ignore & rustfmt
2022-09-29 21:02:29 -07:00