Commit graph

268 commits

Author SHA1 Message Date
Ericson "Fogo" Soares 6b17409a72 Fixing unit tests for MacOS 2023-01-13 17:02:57 -03:00
Ericson "Fogo" Soares 4638f104be MacOS file system event handling using FSEvents backend 2023-01-13 15:55:16 -03:00
Ericson Fogo Soares a18b2f8c8b Trying to figure out why events didn't matched 2023-01-11 19:46:34 -03:00
Ericson Fogo Soares cb8820fe47 Checking MacOS file system events using FSEvents instead od Kqueue 2023-01-11 18:57:41 -03:00
jake 0d554e9d06
[ENG-331] StoredKey overhaul (#513)
* add wip storedkey versioning

* storedkey versioning! (not pretty, but it never will be)

* add version to `StoredKey` and re-gen migrations to handle serde

* use `serde` for interacting with the DB + handle errors
2023-01-11 05:15:08 -08:00
jake fbd8b6d199
[ENG-327] Crypto refactor (#510)
* fix secret key showing as `Some()` when it's empty

* rename function

* use `uuid` feature of `rspc`  so clippy works with `--all-features`

* remove dead error

* update clippy rules

* more idiomatic + cleaner hashing, that handles secrets better

* remove dead code + minor refactor of the KM

* cleanup`KeyManager::get_key()`

* cleanup keys router

* variable inlining, idiomatic rust, etc

* clean up encrypt/decrypt jobs

* embed preview media into encrypted files

* even more idiomatic rust

* fix previous file extension code (no amount of idiomatic rust can fix bad patterns)

* hopefully make large serialization slightly more efficient

* remove dead code + old tests

* inline key derivation (not so sure about this one)

* use `then` instead of `if`

* stop using expensive clones

* add `Protected::into_inner()` to prevent `.expose().clone()` patterns

* use `tokio::fs::File` where possible

* add and use `JobError::MissingData`

* use `|| null` for the secret key

* use `block_in_place` for large r/w sync tasks
2023-01-10 12:36:28 -08:00
jake c1b2b1f5ee
[ENG-323] Key manager onboarding fixes (#506)
* add hash with secret functions

* refactor KM to make the secret key both optional, and user-settable

* update library create function and create library dialog

* fix annoying MPC typo

* fix ordering of key manager creation and seeding

* overhaul master password changing UI to support the potentially empty secret key

* patch keymanager unlocking for potentially empty SK

* make create library dialog text more fitting

* remove dead code

* add autogeneration, view/hide, copy to clipboard icons

* cleanup older code

* make onboarding screen look a tad better

* minor UI tweaks

* patch backup restoration UI

* add pw generation to MP change dialog

* make sure passwords are same before submitting lib creation

* UI fixes and allow setting algorithm/hashing algorithm for master password

* make sure KM is re-seeded on SD load and prevent loading duplicate keys into the keystore

* remove old comment

* small organization

* update encryptfiledialog

* remove dead KM code

* cleanup some TS

* move `PasswordMeter` to separate file

* refactor master password change dialog

* cleanup the TS and improve string -> hashingalgorithm conversions

* add `key` values to selectoption

* capitalise `BLAKE3`

* fix sorting for `getHashingAlgorithmString`

* remove onboarding route

* Revert "add `key` values to selectoption"

This reverts commit 10e6456f22.

* add `key` to `SelectOptionKeyList`

* fix using rspc errors directly

* Update packages/interface/src/components/onboarding/Onboarding.tsx

Co-authored-by: Brendan Allan <brendonovich@outlook.com>

* use `navigator.clipboard.writeText`

* move `OnboardingConfig` to crypto crate

* move `zxcvbn` options outside of component

* further cleanup

* fix mobile typecheck

* fix keylist typo

* improve `SelectOptionKeyList` further

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-01-06 17:03:22 +00:00
jake 746ac76803
[ENG-325] Local paths for the context menu (#507)
* use `path_id` instead of `object_id` for finding local file paths

* fix bug

* make automounting better, UX-wise

* find file_paths correctly

* use `obj_id` for fetching metadata/preview media

* stop using `expect()` during jobs
2023-01-06 16:26:25 +00:00
Brendan Allan 58f5d8bb4d
update prisma (#505) 2023-01-05 08:05:33 -08:00
Brendan Allan 676e512c4a
Beginning of sync stuff (#502) 2023-01-04 23:11:55 -08:00
jake 22bc77a39e
[ENG-319] Balloon hashing (#489)
* add key attribute to `Key` in `ListOfKeys`

* add balloon hashing function with untailored parameters

* add ser/de rules for blake3-balloon (and change argon2id's)

* fix benchmark

* use `to_bytes`, `from_bytes` and `from_reader`

* cleanup code

* add blake3-balloon options to the UI and fix library sync/automount enable bug

* cleanup some serialization code

* fix hashing algorithm deserialization

* clean up header serialization + more idiomatic master key decryption

* clippy

* add generic ser/de error to crypto crate

* fix `Display` and crypto cli

* move crypto cli to cli app

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-01-04 09:57:02 +00:00
Oscar Beaumont 3964d44ce5
Integrate KeyManager with library creation flow (#491)
* integrate keymanager with library creation flow

* final fixes

* fix clippy recommendations

* minor fixes on library create dialog

* reenable `panic!` for no key + fix secret handling code

* prevent user setting secret , instead hardcode it

* clean library manager default key selection

* bring back wrongly removed `keys.onboarding` resolver

* fix types in `CreateLibraryDialog`
2023-01-04 00:13:12 +08:00
jake db5b401238
[ENG-318] Implement a KDF to derive new keys (#487)
* add blake3, `derive_key()` and clean up code

* fix a couple of things from the previous commit

* add context strings for root/file key derivation

* add salt to schema

* update refs of `salt` to `content_salt` within the keyslot

* cleanup code and add kdf salt to the keyslot

* rename salt to content salt in examples

* cleanup header code + remove dead code

* implement key derivation for keyslots

* gen new migrations that contain a salt column

* keymanager refactor (code is very idiomatic now) - needs thorough testing

* further cleanup

* clippy

* add a master password context string

* use key derivation for deriving keys from the root key

* update to use new code and remove `match` from `en/decrypt_bytes()`

* clippy

* use less unwraps in library manager code

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2022-12-30 20:09:44 +00:00
Ericson "Fogo" Soares bcbcd260d4
ENG 223 Location Awareness (#468)
* Introducing location online/offline checks and location relink

* Some initial drafts for location watchers

* Location metadata relink and add library

* Many improvements at job system
Now using prisma batching at identifier job
Removing blocking I/O from extension subcrate
Implementing lazy generation of thumbnails
New current directory identifier job to be used on light rescans

* Some optimizations on identifier and object validator jobs

* merge jamie's identifier PR

* fully repaired identifier job

* properly hooked up object kind

* inspector fix

* fix video badge

* small improvements to libraries settings

* identifier and inspector improvements

* fix feature flags and hook up context menu location utilities

* BETTER CONTEXT MENU x100

* test-files

* style tweaks

* new icon designs

* manifest

* fix thumbnails on web

* media data

* New Location Watcher and some minor fixes

* disable broken media_data extractor, wip

* wip

* function name fix

* Fixing pnpm prep and some warnings

* Solving a race condition beetween indexer job and FS event handlerSome other minor warnings

* Generating thumbnails on watcher

* Remove event handler on watcher

* Some initial works on modify events and other small fixes

* File update event

* Trying to be more generic with used events and some tests to validate our assumptions

* Turning on location metadata file

* Introducing core unit tests on CI pipeline

* Submiting new unit test assumptions to validate on windows CI

* Fixing unit tests

* Fixing unit tests again

* Fixing unit tests

* Fixing unit tests for macos

* Fixing unit tests for macos again

* New structure for platform dependent event handling
Implementing event handlers for Linux and MacOS

* minor fixes + rustfmt + clippy

* Windows event handling

* Introducing a feature gate to only use location watching on desktop app for now

* Putting more stuff behind feature gates to avoid warnings

* Adding feature to cargo test on CI

* Changing some debug logs to trace logs and removing Jamie specific stuff

* Make location removal from manager less async

* fix build when "location-watcher" feature disabled

* fix types + clippy

* make location manager non-static

* remove uses of `to_string_lossy`

* more invalidate_query calls

* Expose `library_ctx` directly to avoid needless clones

* New materialized_path handling for directories

* Removing cascade delete between file_path and object
- Some other minor stuff

* remove unused `CurrentDirFileIdentifierJob`

Co-authored-by: Jamie Pine <ijamespine@me.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2022-12-31 00:53:24 +08:00
jake bbca699e33
[ENG-316] Decrypt with Password (#484)
* update generated key on slider click

* add ui for decrypting with password+saving to key manager

* update rust code to support password decryption and saving it to the library

* add decrypt with password UI functionality

* add content salt to key viewer

* disable key manager decryption if no keys are mounted

* clippy
2022-12-21 13:04:52 +08:00
jake 78a5754f4e
[ENG-315] Key viewer/general improvements (#483)
* add encryption+hashing algorithm to key viewer dialog

* abstract `save()` and `open()` with `Platform`

* add automount to `StoredKey` and offer option to enable/disable it for keys

* fix missing statement exec

* general tidy up

* add sync to library button and cleanup default setting code

* use alert dialogs for unsupported `open`/`save` operations (web)

* tweak alert message
2022-12-13 09:48:14 +08:00
jake 3ce8c74a0d
[ENG-313] Key auto-generation and viewing (#478)
* add generic dialog for keys settings

* revert artifact failed key viewing attempt

* move `Select` key list component

* rename dialog

* remove unused imports and add new select option for *all* keys

* add WIP but broken key viewer dialog

* cleanup code and fix key viewer dialog

* add clipboard icon and copy functionality

* generalise the `AlertDialog` and refactor `BackupRestoreDialog` to use it

* use new alert dialog in place of JS/tauri alerts

* use generic alerts everywhere and bring generic alert props/default state

* make `SelectOptionKeyList` generic for mounted/unmounted keys (with the use of `map` for the latter)

* add clipboard to generic alert dialog + clean up

* fix accent colour button for backup restoration

* remove unneeded props from components

* add slider+automount button

* tweak password gen function

* add password autogeneration

* clippy

* tweak password generation

* use `crypto-random-string` and drop rust password generation

* add default TEMPORARY keymanager pass/secret key to library creation screen

* make key automounting functional

* clean up key viewer

* change dialog name

* remove slider as that wasn't even being used?

* make requested changes and hide key viewer if no keys are in the key manager

* prevent automount and library sync from being enabled simultaneously

* include `memoryOnly` in key

* mark keys as memoryOnly
2022-12-12 17:13:52 +00:00
jake 8ee2d18053
[ENG-309] Envelope encryption for the KM's root key (#482)
* use master key to encrypt the root key (breaking)

* introduce breaking changes to the keymanager (this breaks master password changing, backup restoration, etc)

* change MP -> RK

* minor tweaks + comment out broken features temporarily

* experimental new master password changing

* add a `write_storedkey_to_db` helper function

* general cleanup and fixes

* potentially patch backup restoration

* uncomment+clean up restoration in api

* strip whitespaces from user inputted secret key, and fix MPC bug

* clippy

* add memory only flag to `StoredKey`
2022-12-12 15:04:08 +00:00
jake f1292d6e51
Crypto Hotfixes (#477)
* fix annoying typo that was breaking master password changing under certain conditions

* move `Select` boxes out of `react-hook-form` so they work

* add `DummyKey` and rework the UI a little

* fix bug where user could select a hashing algorithm (it did nothing)

* fix bug where encryption algorithm would be inherited from the key, not the user's choice
2022-12-09 22:05:32 +08:00
jake 5651ac3f69
[ENG-251] Crypto Jobs (#475)
* add base crypto job

* leave hashing to the KM and not keyslots

* add latest header object versions and update examples

* add a very experimental (local file!) encrypt job

* working file encryption with default key

* working file decryption

* non-working encrypt file dialog

* general code cleanup

* comments and correct feature locking

* fix the `Dialog`

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>

* WIP encrypt dialog

* massive encrypt dialog overhaul

* add generic explorer alert dialog

* better key naming/numbering, alerts for unsupported scenarios

* cleanup + decrypt file dialog

* remove debugging stuff

* remove dead code & fix a bug with key select being empty

* clean up dialog UI massively

* add support for setting output path

* add correct button type to prevent early dialog submits

* add "nice" job data for encrypt/decrypt

* tweak functionality of key setting in encrypt dialog

* include full file metadata if selected, and lay out groundwork for preview media inclusion

* add comment about not restoring metadata/pvm just yet

* general cleanup

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
2022-12-07 15:16:13 +00:00
jake 2baf16d982
[ENG-307] Key manager features (#467)
* working key management in settings page

* sync with library button

* fix `rspc` feature and add passphrase generation

* untested key manager rework

* trying to return values from mutations

* update library manager and remove settiong master PW

* update bindings

* set static secret key/master password

* prompt user for master password if correct one hasn't been provided yet

* add `hasMasterPassword` route

* add `clearMasterPassword` route + remove dead code

* tweak `set_master_password()` and add dedicated error

* tweak UI, fix `few hooks than expected`, add unmount+lock button

* remove old comment

* fmt

* clippy

* move static key/password setting so it doesn't fail sometimes

* add dedicated `get_key()` and remove keys from memory

* add `getKey` route

* update bindings

* use `const` instead of `let`

* comment updates

* update schema to remove salt

* add string parse error

* generate passphrase within key manager

* generate new migrations

* feature gate serde support in `crypto` crate

* fmt

* more specific error types

* foramatting

* add locking mechanism to keysettings page (not working?)

* fix react hook issues

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>

* remove empty onclick

* add keymanager dropdown menu

* working key backup functionality

* add experimental master password changing support

* update bindings

* semi-working change master password dialog

* use gear/lock icons to clean up key manager UI

* make settings button functional

* make buttons uniform and format code

* fix double base64 encode

* add change master password dialog and secret key dialog

* code cleanup

* restore backup dialog

* change UI wording

* make a start on restoring from a backup

* potentially working keystore restore

* don't overwrite verification key if one is set

* working backup restore + fix master password changing

* fix typo in static password/verification key check logic

* change wording to make UI clearer

* disable mount button if key is empty

* handle errors+remove type annotations

* show total imported keys on backup restoration

* add zxcvbn package

* change input border colour based on zxcvbn score

* clippy and formatting

* password strength meter

* remove nbsp

* add button type to stop early form submission

* use `react-hook-form` for backup restoration dialog

* more `react-hook-form` stuff

* attempt to fix password meter

* small cleanup

* Fix password meter

* update colours

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2022-12-02 10:18:21 +00:00
jake a403224b3b
[ENG-262] Key Manager Integration (#450)
* add keys router

* make progress on keymanager integration

* make name non-negotiable

* hyphenate encryption algorithm names

* Revert "make name non-negotiable"

This reverts commit 9c0f51329e.

* add some more keymanager queries

* add master password and default key routes

* add newly registered key to db + fmt

* clippy, formatting and `updateKeyName` route

* add automount to schema+automount keys where `true`

* update bindings

* working key add/mount

* working keylist

* mounted keys show first

* cleanup code

* add comments, code cleanup, more functions

* unmount all keys button

* comment and keymanager `clear_master_password()`

* add no keys available message

* fix unmount button

* use dashmap for concurrency

* fix missing keylist issue and add invalidate query macro

* set correct RSPC types

* statically set master password (TEMPORARILY)

* add remove key function within the keymanager

* key dropdown menu and impl

* formatting

* allow `option_if_let_else`

* add comment about key stats

* add additional comment

* rpsc error handling for the keys route

* fix rspc errors with an impl

* crypto crate errors to `sd-crypto::Error`

* remove `map_err`

* use custom result type

* cargo fmt

* clippy

* fix builds

* remove `Error::MutexLock`

* fix unnecessary unwrap

* mutex error handling (buggy for some reason)

* clean default key logic

* fix default key clearing

* allow a key to be removed without bugs

* implement requested changes

* use a single `useMemo`

* update schema with defaults

* re-generate migrations

* use rust enums in TS

* remove dead code

* remove mutate expansion

* read key list from keymanager, not prisma

* add "Default" key marker and cleanup TS

* rustfmt

* remove dead code
2022-11-05 04:18:01 -07:00
Utku 76e3d0b9a2
Mobile - File Import (#443)
* Fix File modal and Stats

* Add disabled and loading state for dialog

* Close library drowdown with drawer

* catch Rust panics on mobile to prevent UB

* Update packages

* Move create lib dialog to container

* Create library on onboarding

* Cleanup metro config & update packages

* onClose for Dialog & update library cache

* Fix pods, downgrade react-native-svg

* Fix lib switching, organization, start import modal

* Add device size info

* Revert merge + version upgrade/fix

* Create Location & Remove placeholder data

* Create default modal component

* Check if the location already exists

* Add media-library + prettier

* fix build

* Fix Xcode shellScript too

* More small fixes

* don't export bindings on mobile devices

* Explorer store + cleanup

* Explorer comp. & add flashlist

* [WIP] Files in Locations & new file thumb

* clean merge

* Fix imports

* Fix core on mobile

* Add platform context to mobile

* Refactor libraryStore

* Add thumb url path to platform context

* Try fixing app startup

* Add zip and video to filethumb

* Delete bindings.ts from mobile

* Remove heroicons from mobile too

* useForwardedRef hook

* Media Library permission stuff

* Clean import modal

* remove valtio-persist from @sd/client

* prevent Sentry capturing all events

* refactor `@sd/client` to make it better for mobile

* fix mobile splashscreen
Just trust me bro

* fix mobile draw active style

* use custom valtioPersist for current library state

* remove mobile lockfile
It's now in the pnpm workspace so it shared the main one.

* finally remove valtio-persist

* remove 'mobile' from Platform as it's in interface

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2022-11-01 21:32:56 +08:00
Brendan Allan b713c896a0
More Sync Stuff (#438)
* broken sync example

* move to react

* filter owned ops more

* update deps

* working sync

* relation operations

* fix pnpm lock

* please rustfmt

* fix api

* compare strings properly

* please clippy

* feature gate logging

* use bleeding rspc in example

* use bleeding rspc

* use generated rspc client
2022-10-27 23:12:28 -07:00
Jamie Pine 0725459fc0
Polish Icon Usage (#437) 2022-10-25 18:47:09 -07:00
Oscar Beaumont b099d47fec fix database migrations 2022-10-25 06:53:24 +10:00
Jamie Pine 01c29dadf4 Merge branch 'spacedrive-but-themable' of github.com:spacedriveapp/spacedrive into spacedrive-but-themable 2022-10-24 01:18:05 -07:00
Jamie Pine 2d8590c616 job indicator + styling 2022-10-24 01:17:10 -07:00
Oscar Beaumont 2b81acc30e debug mode + minor fixes 2022-10-24 17:48:52 +10:00
Jamie Pine 3a3ec6251f Merge branch 'main'
Conflicts:
	packages/interface/src/screens/settings/library/LibraryGeneralSettings.tsx
2022-10-23 18:43:27 -07:00
jake 0c7aed5f86
[ENG-258] Key Manager and Crypto Crate Improvements (#423)
* add base keymanager structs/functions

* change md/pvm `new()` to prevent useless re-hashing

* update lockfile

* update keymanager

* cleanup code, make things easier to understand

* move md and pvm construction to the header

* update pvm ser/de

* update metadata ser/de

* additional API changes, update example

* formatting and clippy

* update examples

* move `impl` to associated files

* formatting+clippy

* add more keymanager functionality

* add docs

* wrap `master_password` as an option

* add `sd-crypto` crate as a dependency to `core`

* add key manager to `LibraryContext`

* rename `id` -> `uuid`

* add more keymanager functions

* add `set_master_password()` function

* function to see if keymanager has master password

* update schema

* add default bool to schema

* populate keystore on SD startup

* clippy+formatting

* implement requested changes

* remove unwrap

* fmt, clippy, remove ser/de derives

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2022-10-23 03:47:41 -07:00
Brendan Allan 61cb749c07 remove openssl from core 2022-10-22 05:43:39 +08:00
Jamie Pine c4226f0e7a Merge branch 'eng-259-identify-objectkind'
Conflicts:
	apps/landing/package.json
	apps/landing/vite.config.ts
	packages/interface/src/AppLayout.tsx
	packages/interface/src/components/explorer/Explorer.tsx
	packages/interface/src/components/explorer/ExplorerTopBar.tsx
	packages/ui/src/ContextMenu.tsx
	pnpm-lock.yaml
2022-10-20 22:49:15 -07:00
Oscar Beaumont 727341080a Merge remote-tracking branch 'origin/main' into fix-reactivity 2022-10-18 22:16:15 +08:00
Oscar Beaumont 4ab1e65059 Merge remote-tracking branch 'origin/main' into fix-reactivity 2022-10-18 22:09:33 +08:00
Oscar Beaumont aae3da9b8a 🎉 Fix core on Android 2022-10-18 20:57:17 +08:00
brxken128 bec33b5bd7 re-import cas.rs from eng-245 2022-10-18 13:44:31 +01:00
jake 3f2b4cda38
[ENG-245] Object Validator Job (#414)
* use `.to_hex()` for blake3 hashes

* move file hashing function+fix large stack alloc

* init validator job

* change branch name

* add object validator job

* formatting

Co-authored-by: Jamie Pine <ijamespine@me.com>
2022-10-18 05:39:33 -07:00
Oscar Beaumont 1a8581d44f upgrade rspc + fix reactivity + add in disabled Normi code 2022-10-18 04:45:37 +08:00
Jamie Pine 4acb8cc594
Merge branch 'main' of https://github.com/spacedriveapp/spacedrive 2022-10-17 01:36:05 -07:00
jake 4cd2dde35c
CAS ID Improvements (#413)
* remove `ring` dependency and use `sha2` instead

* use BLAKE3 and include full file checksum

* update schema comments
2022-10-13 15:31:47 -07:00
Jamie Pine 900ee618ce
Merge branch 'main' of https://github.com/spacedriveapp/spacedrive 2022-10-06 18:10:08 -07:00
Jamie Pine e402fc52b7
sd extensions crate
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2022-10-06 18:10:03 -07:00
Oscar Beaumont 75c4680b43 Merge remote-tracking branch 'origin/main' into merge-p2p-code 2022-10-07 05:56:05 +08:00
Oscar Beaumont d7fddc83f7 fix major bugs in p2p PR 2022-10-07 05:51:40 +08:00
Oscar Beaumont 598ba38a9d merge in P2P code 2022-10-07 05:32:45 +08:00
Jamie Pine ddad55860b
tests for magic bytes 2022-10-06 09:49:14 -07:00
Oscar Beaumont dec2697740 Fix mobile + upgrade Expo + even newer rspc 2022-10-06 21:07:46 +08:00
Brendan Allan f1b55f1101 merge main into update-rspc-and-more 2022-10-06 18:34:18 +08:00
Oscar Beaumont a5e81a898e move arg structs into the builder closure 2022-10-06 13:30:57 +08:00
Jamie Pine 5ab432ae46
Magic bytes 🪄
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2022-10-05 13:16:22 -07:00
Brendan Allan e5359f042d please formatter 2022-10-05 12:14:04 +08:00
Brendan Allan 45ff212e97 merge main into update-rspc-and-more 2022-10-05 12:01:50 +08:00
Jamie Pine d27ff3cd80
Object Kind (#397)
* add temp custom icons and tweak styles

* tweaks

* - added Object type structs
- added extension enums with variant macros
- improved explorer rendering
- added database migration hotfix

* - added dev only data folder
- improved explorer image loading

* macros do rule !

* remove baselining

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2022-10-04 20:34:41 -07:00
Brendan Allan 723058e060 merge main into update-rspc-and-more 2022-10-05 08:58:03 +08:00
Jamie Pine cbd58ee438
Rename Files to Objects (#396)
* backend rename complete

* finalize Object refactor frontend
+ remove some exhaustive dep suppression
+ FIX INVALIDATE QUERY BUG
2022-10-04 09:16:49 -07:00
Brendan Allan ed06e3051e
[ENG-84] Sync library (#394)
* new sql lib stuff

* add sync stuff + rename all crates

* build sd-core

* add sync/example/dist to source

* fix sync example in monorepop

* appease clippy

* update lockfile

* update commit hooks

* fix typescript

* fix typescript build

* please rustfmt
2022-10-04 05:25:12 -07:00
Brendan Allan 66dd9aa28b
move prisma schema and migrations to /core (#392)
* move prisma schema and migrations to /core

* fix prisma updates
2022-10-04 02:32:46 -07:00
Oscar Beaumont 60e228ec94 upgrade rspc 2022-10-02 01:16:11 +08:00
Jamie Pine 317ed2d7b0
removed stray dependency causing build issue 2022-09-29 21:24:42 -07:00
Jamie Pine 3e8c33d1c0
Added crates folder (#389)
* restructure core for crates folder

- improve naming consistency
- added placeholder files for near future code
- added docs

* - removed all typescript from core
- moved rspc bindings to client

* remove useless git ignore & rustfmt
2022-09-29 21:02:29 -07:00
Oscar Beaumont bb40110e6e
Eng 243 documentation on landing (#386)
* Updated roadmap

* Color timeline to show where we are up to

* roadmap improvements

* roadmap changes

* docs init

* progress on docs

* support metadata (name & ordering)

* fix prod build

* fix lock

* fix perender

* docs progress

* added "slots" to markdown parser

* refactor to support sections

* more docs progress

* fixes

* added empty docs

* killer opening paragraph

* fix Heroicon imports & page rendering

* tweaks

* next doc

* fix build

* fix ssg failure

* fix prod build for REAL

it was the bubbles

* fix next doc api

* it's the bubbles

* wip: docs mobile

* doc nav tweak

* doc menu ui

* fix next doc in pre-render

* react-borger-menu

* fix build

* fix height and padding

* mobile docs nav styles

* oooh super cool transition

* fix button state glitch + big headings

* refactor layout

* docs mobile nav bar style

* fix min width & height

* Merge remote-tracking branch 'refs/remotes/origin/main'

Conflicts:
	docs/architecture/spaces.md
	docs/developers/architecture/spaces.md

* placeholder docs

* correct pre styles + env setup guide

* search coming soon

* docs cleanup

* resolved cargo

* reorder docs + add image

Co-authored-by: Lu Hill <email@lewie.me>
Co-authored-by: Jamie Pine <ijamespine@me.com>
2022-09-27 19:48:42 -07:00
Brendan Allan fe497ce104 make rust 1.64 requirement more concrete 2022-09-27 12:28:32 +08:00
Ericson "Fogo" Soares 6fd620087b
Video thumbnails (#376)
* Preparing some scaffolding for video thumbnails

* Implemented thumbnail generation for videos
* Propagating errors of `Node` creation
* Using ffmpeg feature gate

* Introducing ffmpegthumbnailer-rs as a subcrate on core

* - rename to thumbnailer
- fix explorer thumbnail bug
- add more supported video types
- re-fix explorer performance

* remove nested licence

Co-authored-by: Jamie Pine <ijamespine@me.com>
2022-09-24 01:21:13 -07:00
Brendan Allan 1bf315c4db
Use feature gated database branch of PCR (#387)
use feature gating branch of pcr
2022-09-22 23:40:15 -07:00
Brendan Allan 5e30331dad typo fixes 2022-09-23 13:46:23 +08:00
Brendan Allan 6e07435a15
Use PCR for migrations (#385)
* use pcr migrations branch

* use 0.6.2 branch with migrations

* use latest prisma stuff

* allow force reset of db in dev

* remove .spacedrive file

* update rspc in apps/server

* use rspc 0.0.5 in all crates

* add os to prisma client cache key

* add runner os to clippy prisma cache
2022-09-22 00:50:21 -07:00
Oscar Beaumont ef59aa40e8 Merge remote-tracking branch 'origin/main' into eng-237-explorer-performance-2 2022-09-15 12:45:05 +08:00
Oscar Beaumont dd04625a4f properly shutdown the core when using server 2022-09-14 19:02:11 +08:00
Oscar Beaumont a275971643 fix clippy warning 2022-09-12 23:50:02 +08:00
Oscar Beaumont 5dd45f2b93 onboarding UI flow 2022-09-12 23:43:12 +08:00
Oscar Beaumont 29862313e6 Merge remote-tracking branch 'origin/main' into eng-237-explorer-performance-2 2022-09-09 15:30:17 +08:00
Oscar Beaumont b6a2c98ae5 assorted minor changes 2022-09-09 07:57:35 +08:00
Oscar Beaumont 2fda5b9ca6
Better db migrations (#381)
* use prisma migration engine for migrations

* squash migrations + remove excessive logging
2022-09-08 15:57:07 -07:00
Oscar Beaumont ba74af9762 merge changes 2022-09-09 06:32:17 +08:00
Oscar Beaumont 3caf844a9e fix the production build issue + cleanup frontend 2022-09-09 06:29:47 +08:00
Jamie Pine 5eeabb3443
Merge branch 'eng-237-explorer-performance-2' of https://github.com/spacedriveapp/spacedrive into eng-237-explorer-performance-2 2022-09-08 15:01:08 -07:00
Jamie Pine 21482a4730
job manager tweaks 2022-09-08 15:01:05 -07:00
Ericson Fogo Soares ae7aa2bf9b Changing job metadata to serde_json::Value 2022-09-08 17:20:04 -03:00
Jamie Pine d540887a4e
job manager wip 2022-09-07 20:04:13 -07:00
Jamie Pine ed83b0d64c Merge remote-tracking branch 'origin/eng-237-explorer-performance-2' into eng-237-explorer-performance-2 2022-09-07 19:45:53 -07:00
Ericson Fogo Soares 0f92dd7af1 A wrong premise about report on a running job 2022-09-07 23:43:39 -03:00
Jamie Pine 0130f5a6f4 Merge remote-tracking branch 'origin/eng-237-explorer-performance-2' into eng-237-explorer-performance-2 2022-09-07 19:25:43 -07:00
Ericson Fogo Soares 752c4ad45c Missing job error from serde_json::Error 2022-09-07 23:23:04 -03:00
Jamie Pine 4535fddc91 Merge remote-tracking branch 'origin/eng-237-explorer-performance-2' into eng-237-explorer-performance-2 2022-09-07 19:12:49 -07:00
Ericson Fogo Soares a4b6b7b5c9 Implementing metadata for jobs 2022-09-07 23:08:11 -03:00
Jamie Pine 6b1e8c3088
hotfix for identifier orphan query 2022-09-07 14:05:33 -07:00
Utku Bakir b3a586afff Rust useless conversion 2022-09-07 11:56:51 +03:00
Jamie Pine 98f900b99e
(fix) identifier job: better error handling 2022-09-06 23:21:58 -07:00
Brendan Allan b6f1a941b2 cleanup identifier job 2022-09-07 10:15:55 +08:00
Jamie Pine 4196a9823f
fix indexer lag (reimplement debounced progress event for jobs) 2022-09-06 17:44:48 -07:00
Brendan Allan 459a44b7ac utilise include instead of with for type safety 2022-09-07 04:09:57 +08:00
Brendan Allan 3775c33d97 rename getAll to list 2022-09-07 00:21:41 +08:00
Brendan Allan eacb035283 fix everything after rebase 2022-09-06 22:34:39 +08:00
Oscar Beaumont 4427065658 Upgraded rspc with libraries + upgrade PCR 0.6.0 2022-09-06 22:08:19 +08:00
Jamie Pine 9dcef71614
replaced interface store with Valtio 2022-09-04 18:20:22 -07:00
Jamie Pine 05bcb6e1b0
Assignable Tags (#370)
* Tag queries + Identifier bug fix

- added empty pages for docs
- added non-functional "New Library" button to sidebar
- to accomplish above, moved dialog logic to component folder
- fixed Identifier bug where location id was not considered in the orphan files query, meaning it would attempt to identify many non-existent paths and fail to create the legitimate ones

* (fix) maintain file item aspect ratio in grid view
(fix) get tag query + get all tags

* codegen

* resurrected context menu

* fix window flash

* remove location based thumb sorting + update react on landing

* add is_archived to location

* improved context menu

* assign tag + refactor explorer store

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>

* assign/unassign tags from context menu

* fix lint

* keep context menu open on tag change

* fix brendan feedback + merge main

* fix type

* codegen

* style changes

* revert windows size and xcode proj

* remove outdated doc

* coming sooooooon

* updated release notes

* release notes

* dashes > dots

* clean up docs

* more docs!

* restore entitlements

* remove unused util

* regenerated migrations post merge
- some additional error handling added to migration runner, needs more work!

* refactor explorer to support tags

* fix error and revert explorer bg color

* put it in a box they said

* revert location id store removal

* upgrade hero icons + style tweaks

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
2022-09-04 13:00:24 -07:00
Ericson "Fogo" Soares 25c111a8bf
Introducing Indexer rules (#363)
* Introducing indexer rules for the indexer job
* Solving a minor race condition at `Worker::spawn`
* Many refactors at database accesses with prisma
* Documenting some features of indexer rules

* Updating to new prisma rev

* Updating photo

* Some documentation tweaks

* fix note editing

* Implementing some requested changes
* Fixing unit tests on walk function
* Using rsps::Error::with_cause instead of new

* Introducing a compoung unique key between `node_id` and `local_path` at `locations` table

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2022-09-01 16:38:26 -07:00
Oscar Beaumont cc6e921ca0 new invalidate_query + add IOS sim to setup script 2022-08-31 18:42:06 +08:00
Oscar Beaumont fdfb2f00d5 remove phantom file + update rspc bindings
A PR must have been merged incorrect introducing this file which is not used.
2022-08-30 10:50:08 +08:00