Commit graph

97 commits

Author SHA1 Message Date
Brendan Allan de3a979127 update prisma 2023-01-06 01:19:19 +08:00
Brendan Allan 58f5d8bb4d
update prisma (#505) 2023-01-05 08:05:33 -08:00
Brendan Allan 676e512c4a
Beginning of sync stuff (#502) 2023-01-04 23:11:55 -08:00
jake 2673a233c2
[ENG-320] Key manager bugfixes (#501)
* force key updates once a new key is mounted

* set KM decryption type if a single key is mounted

* prevent the key viewer dialog from changing size based on encryption algorithm

* fix spacing for the key mounter slider and remove unfitting text

* bump rspc version (fixes a lot of KM concurrency bugs)

* add personal reference for rspc override

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2023-01-05 12:52:34 +08:00
jake 22bc77a39e
[ENG-319] Balloon hashing (#489)
* add key attribute to `Key` in `ListOfKeys`

* add balloon hashing function with untailored parameters

* add ser/de rules for blake3-balloon (and change argon2id's)

* fix benchmark

* use `to_bytes`, `from_bytes` and `from_reader`

* cleanup code

* add blake3-balloon options to the UI and fix library sync/automount enable bug

* cleanup some serialization code

* fix hashing algorithm deserialization

* clean up header serialization + more idiomatic master key decryption

* clippy

* add generic ser/de error to crypto crate

* fix `Display` and crypto cli

* move crypto cli to cli app

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-01-04 09:57:02 +00:00
jake 4569e45e47
[ENG-312] Crypto details CLI tool (#500)
* add untested crypto details tool

* clean up output a little

* clippy

* update crate details and `clap`
2023-01-04 00:15:12 +08:00
jake db5b401238
[ENG-318] Implement a KDF to derive new keys (#487)
* add blake3, `derive_key()` and clean up code

* fix a couple of things from the previous commit

* add context strings for root/file key derivation

* add salt to schema

* update refs of `salt` to `content_salt` within the keyslot

* cleanup code and add kdf salt to the keyslot

* rename salt to content salt in examples

* cleanup header code + remove dead code

* implement key derivation for keyslots

* gen new migrations that contain a salt column

* keymanager refactor (code is very idiomatic now) - needs thorough testing

* further cleanup

* clippy

* add a master password context string

* use key derivation for deriving keys from the root key

* update to use new code and remove `match` from `en/decrypt_bytes()`

* clippy

* use less unwraps in library manager code

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2022-12-30 20:09:44 +00:00
Ericson "Fogo" Soares bcbcd260d4
ENG 223 Location Awareness (#468)
* Introducing location online/offline checks and location relink

* Some initial drafts for location watchers

* Location metadata relink and add library

* Many improvements at job system
Now using prisma batching at identifier job
Removing blocking I/O from extension subcrate
Implementing lazy generation of thumbnails
New current directory identifier job to be used on light rescans

* Some optimizations on identifier and object validator jobs

* merge jamie's identifier PR

* fully repaired identifier job

* properly hooked up object kind

* inspector fix

* fix video badge

* small improvements to libraries settings

* identifier and inspector improvements

* fix feature flags and hook up context menu location utilities

* BETTER CONTEXT MENU x100

* test-files

* style tweaks

* new icon designs

* manifest

* fix thumbnails on web

* media data

* New Location Watcher and some minor fixes

* disable broken media_data extractor, wip

* wip

* function name fix

* Fixing pnpm prep and some warnings

* Solving a race condition beetween indexer job and FS event handlerSome other minor warnings

* Generating thumbnails on watcher

* Remove event handler on watcher

* Some initial works on modify events and other small fixes

* File update event

* Trying to be more generic with used events and some tests to validate our assumptions

* Turning on location metadata file

* Introducing core unit tests on CI pipeline

* Submiting new unit test assumptions to validate on windows CI

* Fixing unit tests

* Fixing unit tests again

* Fixing unit tests

* Fixing unit tests for macos

* Fixing unit tests for macos again

* New structure for platform dependent event handling
Implementing event handlers for Linux and MacOS

* minor fixes + rustfmt + clippy

* Windows event handling

* Introducing a feature gate to only use location watching on desktop app for now

* Putting more stuff behind feature gates to avoid warnings

* Adding feature to cargo test on CI

* Changing some debug logs to trace logs and removing Jamie specific stuff

* Make location removal from manager less async

* fix build when "location-watcher" feature disabled

* fix types + clippy

* make location manager non-static

* remove uses of `to_string_lossy`

* more invalidate_query calls

* Expose `library_ctx` directly to avoid needless clones

* New materialized_path handling for directories

* Removing cascade delete between file_path and object
- Some other minor stuff

* remove unused `CurrentDirFileIdentifierJob`

Co-authored-by: Jamie Pine <ijamespine@me.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2022-12-31 00:53:24 +08:00
jake 0eb0376d5b
[ENG-308] Crypto optimizations (#470)
* add benchmarks for encryption/hashing

* add more benchmarking sizes

* general performance improvements

* massive performance gain with minimal security impact

* update `sd-crypto` author info
2022-12-08 12:08:35 +00:00
brxken128 4449bcb143
code cleanup and use hex for secret key encoding 2022-12-07 15:00:14 +00:00
jake 2baf16d982
[ENG-307] Key manager features (#467)
* working key management in settings page

* sync with library button

* fix `rspc` feature and add passphrase generation

* untested key manager rework

* trying to return values from mutations

* update library manager and remove settiong master PW

* update bindings

* set static secret key/master password

* prompt user for master password if correct one hasn't been provided yet

* add `hasMasterPassword` route

* add `clearMasterPassword` route + remove dead code

* tweak `set_master_password()` and add dedicated error

* tweak UI, fix `few hooks than expected`, add unmount+lock button

* remove old comment

* fmt

* clippy

* move static key/password setting so it doesn't fail sometimes

* add dedicated `get_key()` and remove keys from memory

* add `getKey` route

* update bindings

* use `const` instead of `let`

* comment updates

* update schema to remove salt

* add string parse error

* generate passphrase within key manager

* generate new migrations

* feature gate serde support in `crypto` crate

* fmt

* more specific error types

* foramatting

* add locking mechanism to keysettings page (not working?)

* fix react hook issues

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>

* remove empty onclick

* add keymanager dropdown menu

* working key backup functionality

* add experimental master password changing support

* update bindings

* semi-working change master password dialog

* use gear/lock icons to clean up key manager UI

* make settings button functional

* make buttons uniform and format code

* fix double base64 encode

* add change master password dialog and secret key dialog

* code cleanup

* restore backup dialog

* change UI wording

* make a start on restoring from a backup

* potentially working keystore restore

* don't overwrite verification key if one is set

* working backup restore + fix master password changing

* fix typo in static password/verification key check logic

* change wording to make UI clearer

* disable mount button if key is empty

* handle errors+remove type annotations

* show total imported keys on backup restoration

* add zxcvbn package

* change input border colour based on zxcvbn score

* clippy and formatting

* password strength meter

* remove nbsp

* add button type to stop early form submission

* use `react-hook-form` for backup restoration dialog

* more `react-hook-form` stuff

* attempt to fix password meter

* small cleanup

* Fix password meter

* update colours

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2022-12-02 10:18:21 +00:00
jake a403224b3b
[ENG-262] Key Manager Integration (#450)
* add keys router

* make progress on keymanager integration

* make name non-negotiable

* hyphenate encryption algorithm names

* Revert "make name non-negotiable"

This reverts commit 9c0f51329e.

* add some more keymanager queries

* add master password and default key routes

* add newly registered key to db + fmt

* clippy, formatting and `updateKeyName` route

* add automount to schema+automount keys where `true`

* update bindings

* working key add/mount

* working keylist

* mounted keys show first

* cleanup code

* add comments, code cleanup, more functions

* unmount all keys button

* comment and keymanager `clear_master_password()`

* add no keys available message

* fix unmount button

* use dashmap for concurrency

* fix missing keylist issue and add invalidate query macro

* set correct RSPC types

* statically set master password (TEMPORARILY)

* add remove key function within the keymanager

* key dropdown menu and impl

* formatting

* allow `option_if_let_else`

* add comment about key stats

* add additional comment

* rpsc error handling for the keys route

* fix rspc errors with an impl

* crypto crate errors to `sd-crypto::Error`

* remove `map_err`

* use custom result type

* cargo fmt

* clippy

* fix builds

* remove `Error::MutexLock`

* fix unnecessary unwrap

* mutex error handling (buggy for some reason)

* clean default key logic

* fix default key clearing

* allow a key to be removed without bugs

* implement requested changes

* use a single `useMemo`

* update schema with defaults

* re-generate migrations

* use rust enums in TS

* remove dead code

* remove mutate expansion

* read key list from keymanager, not prisma

* add "Default" key marker and cleanup TS

* rustfmt

* remove dead code
2022-11-05 04:18:01 -07:00
Brendan Allan b713c896a0
More Sync Stuff (#438)
* broken sync example

* move to react

* filter owned ops more

* update deps

* working sync

* relation operations

* fix pnpm lock

* please rustfmt

* fix api

* compare strings properly

* please clippy

* feature gate logging

* use bleeding rspc in example

* use bleeding rspc

* use generated rspc client
2022-10-27 23:12:28 -07:00
Brendan Allan 5ceb27c95f
Update Prisma (#441) 2022-10-27 01:56:18 -07:00
Jamie Pine 3a3ec6251f Merge branch 'main'
Conflicts:
	packages/interface/src/screens/settings/library/LibraryGeneralSettings.tsx
2022-10-23 18:43:27 -07:00
jake 0c7aed5f86
[ENG-258] Key Manager and Crypto Crate Improvements (#423)
* add base keymanager structs/functions

* change md/pvm `new()` to prevent useless re-hashing

* update lockfile

* update keymanager

* cleanup code, make things easier to understand

* move md and pvm construction to the header

* update pvm ser/de

* update metadata ser/de

* additional API changes, update example

* formatting and clippy

* update examples

* move `impl` to associated files

* formatting+clippy

* add more keymanager functionality

* add docs

* wrap `master_password` as an option

* add `sd-crypto` crate as a dependency to `core`

* add key manager to `LibraryContext`

* rename `id` -> `uuid`

* add more keymanager functions

* add `set_master_password()` function

* function to see if keymanager has master password

* update schema

* add default bool to schema

* populate keystore on SD startup

* clippy+formatting

* implement requested changes

* remove unwrap

* fmt, clippy, remove ser/de derives

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2022-10-23 03:47:41 -07:00
Brendan Allan 61cb749c07 remove openssl from core 2022-10-22 05:43:39 +08:00
Jamie Pine c4226f0e7a Merge branch 'eng-259-identify-objectkind'
Conflicts:
	apps/landing/package.json
	apps/landing/vite.config.ts
	packages/interface/src/AppLayout.tsx
	packages/interface/src/components/explorer/Explorer.tsx
	packages/interface/src/components/explorer/ExplorerTopBar.tsx
	packages/ui/src/ContextMenu.tsx
	pnpm-lock.yaml
2022-10-20 22:49:15 -07:00
Oscar Beaumont aae3da9b8a 🎉 Fix core on Android 2022-10-18 20:57:17 +08:00
Oscar Beaumont 1a8581d44f upgrade rspc + fix reactivity + add in disabled Normi code 2022-10-18 04:45:37 +08:00
jake 4cd2dde35c
CAS ID Improvements (#413)
* remove `ring` dependency and use `sha2` instead

* use BLAKE3 and include full file checksum

* update schema comments
2022-10-13 15:31:47 -07:00
jake b5c571541e
[ENG-256] Crypto Crate Refinements (#410)
* add experimental preview media support

* clippy+formatting

* add support for mk/pvm nonces

* fix cursor positioning+size deserialization bug

* clippy+formatting

* cleanup code and add `decrypt_preview_media()`

* update comment

* remove `Mode` from `Keyslot`

* use little-endian `usize` for media length

* remove memory objects

* remove mode enum

* add `.zeroize()` to `Protected<>`

* remove `Mode` ser/de

* refactor, remove `Mode`, add helper functions

* formatting

* remove unused error

* update comments/docs

* comments, fix large files, insane performance

* doc updates and fix decrypt zeroize

* revert to 1048576 byte `BLOCK_SIZE`

* `Keyslot` and `PreviewMedia` constructors now handle encryption

* add metadata item with associated functions

* update comments, clean up useless items

* add metadata support within the header

* remove stray 128 in length calcs

* include metadata in header ser/de

* fmt+clippy

* rework keyslot decryption

* formatting

* api changes, code cleanup

* docs for `stream.rs`

* massive documentation re-write

* mark expensive doc test as ignore

* minor api tweaks

* add examples

* formatting+linting

* finalise documentation and add more examples

* formatting
2022-10-13 15:25:42 -07:00
Oscar Beaumont c0b51bcd4a update rspc to version with fixed httpz rev
Main was broken because it was pulling httpz from main and I have been refactoring it.
2022-10-13 15:03:43 +08:00
Jamie Pine b94d2ed736
(hotfix) p2p preventing compile 2022-10-08 11:30:17 -07:00
jake d236b28f84
[ENG-250] Crypto Library (#400)
* add crypto crate with some functionality

* formatting

* add `argon2id` parameter levels

* add descriptive comments

* add stream decryption objects

* add `StreamEncryptor` struct

* add `StreamDecryptor`

* general cleanup

* add `thiserror` and error handling

* add header structs

* add basic serialization functionality

* advance serialization

* finish serialization

* clean up serialization and use `impl`

* finalise deserialization

* add stream helper functions and remove old code

* add AAD creation and retrieval

* add important comment

* add `ChaCha20Rng` as a CSPRNG

* cleanup and crate-wide clippy lints

* apply nursery lints

* add in-memory encryption objects

* rename `utils` to `objects`

* move (de)serialization rules to separate file

* add header-write helper function

* add password hash helper function

* add `decrypt_master_key` function

* cleanup, formatting, linting

* move keyslots to separate file, and rename them

* add basic comments

* remove `secrecy` dependency and import `protected`

* add `to_array` helper function

* `sd_crypto` -> `sd-crypto`

* remove manual drops

* add clippy allows

* add `new()` for `Keyslot` and `FileHeader`

* remove license

* zeroize read buffer on error

* magic bytes are now `ballapp`

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2022-10-07 07:31:40 -07:00
Jamie Pine 900ee618ce
Merge branch 'main' of https://github.com/spacedriveapp/spacedrive 2022-10-06 18:10:08 -07:00
Jamie Pine e402fc52b7
sd extensions crate
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2022-10-06 18:10:03 -07:00
Oscar Beaumont 75c4680b43 Merge remote-tracking branch 'origin/main' into merge-p2p-code 2022-10-07 05:56:05 +08:00
Oscar Beaumont d7fddc83f7 fix major bugs in p2p PR 2022-10-07 05:51:40 +08:00
Oscar Beaumont 598ba38a9d merge in P2P code 2022-10-07 05:32:45 +08:00
Oscar Beaumont dec2697740 Fix mobile + upgrade Expo + even newer rspc 2022-10-06 21:07:46 +08:00
Oscar Beaumont 20523a94c9 upgrade workspace dependencies 2022-10-06 14:03:58 +08:00
Brendan Allan e5359f042d please formatter 2022-10-05 12:14:04 +08:00
Brendan Allan 45ff212e97 merge main into update-rspc-and-more 2022-10-05 12:01:50 +08:00
Brendan Allan b827e024ba update lockfile 2022-10-05 10:20:04 +08:00
Brendan Allan a634727e1b merge meain into update-rspc-and-more 2022-10-05 10:11:52 +08:00
Brendan Allan 162f3d7782
Improve CI Speed (#398)
* use different cache

* cache prisma client in rustfmt

* cleanup prettier config

* build server image in sub-action

* server image + prisma client custom actions

* remove lefthoook checks that are in ci

* fix build-server-image

* rename Clippy job

* rename build core job

* use custom rust caching action

* fix rust cache shared key

* cleanup custom action job names

* remove platform check in clippy ci

* setup pnpm in clippy ci

* update mobile package name

* make ios compiling conditional

* use pnpm cache

* run clippy in release

* force use release for all jobs
2022-10-04 18:45:54 -07:00
Brendan Allan 723058e060 merge main into update-rspc-and-more 2022-10-05 08:58:03 +08:00
Brendan Allan 0821ef575a update pcr version 2022-10-05 04:20:59 +08:00
Brendan Allan ed06e3051e
[ENG-84] Sync library (#394)
* new sql lib stuff

* add sync stuff + rename all crates

* build sd-core

* add sync/example/dist to source

* fix sync example in monorepop

* appease clippy

* update lockfile

* update commit hooks

* fix typescript

* fix typescript build

* please rustfmt
2022-10-04 05:25:12 -07:00
Brendan Allan 66dd9aa28b
move prisma schema and migrations to /core (#392)
* move prisma schema and migrations to /core

* fix prisma updates
2022-10-04 02:32:46 -07:00
Oscar Beaumont 60e228ec94 upgrade rspc 2022-10-02 01:16:11 +08:00
Jamie Pine 317ed2d7b0
removed stray dependency causing build issue 2022-09-29 21:24:42 -07:00
Jamie Pine 3e8c33d1c0
Added crates folder (#389)
* restructure core for crates folder

- improve naming consistency
- added placeholder files for near future code
- added docs

* - removed all typescript from core
- moved rspc bindings to client

* remove useless git ignore & rustfmt
2022-09-29 21:02:29 -07:00
Oscar Beaumont bb40110e6e
Eng 243 documentation on landing (#386)
* Updated roadmap

* Color timeline to show where we are up to

* roadmap improvements

* roadmap changes

* docs init

* progress on docs

* support metadata (name & ordering)

* fix prod build

* fix lock

* fix perender

* docs progress

* added "slots" to markdown parser

* refactor to support sections

* more docs progress

* fixes

* added empty docs

* killer opening paragraph

* fix Heroicon imports & page rendering

* tweaks

* next doc

* fix build

* fix ssg failure

* fix prod build for REAL

it was the bubbles

* fix next doc api

* it's the bubbles

* wip: docs mobile

* doc nav tweak

* doc menu ui

* fix next doc in pre-render

* react-borger-menu

* fix build

* fix height and padding

* mobile docs nav styles

* oooh super cool transition

* fix button state glitch + big headings

* refactor layout

* docs mobile nav bar style

* fix min width & height

* Merge remote-tracking branch 'refs/remotes/origin/main'

Conflicts:
	docs/architecture/spaces.md
	docs/developers/architecture/spaces.md

* placeholder docs

* correct pre styles + env setup guide

* search coming soon

* docs cleanup

* resolved cargo

* reorder docs + add image

Co-authored-by: Lu Hill <email@lewie.me>
Co-authored-by: Jamie Pine <ijamespine@me.com>
2022-09-27 19:48:42 -07:00
Brendan Allan fe497ce104 make rust 1.64 requirement more concrete 2022-09-27 12:28:32 +08:00
Ericson "Fogo" Soares 6fd620087b
Video thumbnails (#376)
* Preparing some scaffolding for video thumbnails

* Implemented thumbnail generation for videos
* Propagating errors of `Node` creation
* Using ffmpeg feature gate

* Introducing ffmpegthumbnailer-rs as a subcrate on core

* - rename to thumbnailer
- fix explorer thumbnail bug
- add more supported video types
- re-fix explorer performance

* remove nested licence

Co-authored-by: Jamie Pine <ijamespine@me.com>
2022-09-24 01:21:13 -07:00
Brendan Allan 1bf315c4db
Use feature gated database branch of PCR (#387)
use feature gating branch of pcr
2022-09-22 23:40:15 -07:00
Brendan Allan 6e07435a15
Use PCR for migrations (#385)
* use pcr migrations branch

* use 0.6.2 branch with migrations

* use latest prisma stuff

* allow force reset of db in dev

* remove .spacedrive file

* update rspc in apps/server

* use rspc 0.0.5 in all crates

* add os to prisma client cache key

* add runner os to clippy prisma cache
2022-09-22 00:50:21 -07:00
maxichrome 119b8176f2 Merge remote-tracking branch 'origin/main' into use-rust-shortcuts 2022-09-14 15:59:59 -05:00