Commit graph

288 commits

Author SHA1 Message Date
jake db5b401238
[ENG-318] Implement a KDF to derive new keys (#487)
* add blake3, `derive_key()` and clean up code

* fix a couple of things from the previous commit

* add context strings for root/file key derivation

* add salt to schema

* update refs of `salt` to `content_salt` within the keyslot

* cleanup code and add kdf salt to the keyslot

* rename salt to content salt in examples

* cleanup header code + remove dead code

* implement key derivation for keyslots

* gen new migrations that contain a salt column

* keymanager refactor (code is very idiomatic now) - needs thorough testing

* further cleanup

* clippy

* add a master password context string

* use key derivation for deriving keys from the root key

* update to use new code and remove `match` from `en/decrypt_bytes()`

* clippy

* use less unwraps in library manager code

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2022-12-30 20:09:44 +00:00
Ericson "Fogo" Soares bcbcd260d4
ENG 223 Location Awareness (#468)
* Introducing location online/offline checks and location relink

* Some initial drafts for location watchers

* Location metadata relink and add library

* Many improvements at job system
Now using prisma batching at identifier job
Removing blocking I/O from extension subcrate
Implementing lazy generation of thumbnails
New current directory identifier job to be used on light rescans

* Some optimizations on identifier and object validator jobs

* merge jamie's identifier PR

* fully repaired identifier job

* properly hooked up object kind

* inspector fix

* fix video badge

* small improvements to libraries settings

* identifier and inspector improvements

* fix feature flags and hook up context menu location utilities

* BETTER CONTEXT MENU x100

* test-files

* style tweaks

* new icon designs

* manifest

* fix thumbnails on web

* media data

* New Location Watcher and some minor fixes

* disable broken media_data extractor, wip

* wip

* function name fix

* Fixing pnpm prep and some warnings

* Solving a race condition beetween indexer job and FS event handlerSome other minor warnings

* Generating thumbnails on watcher

* Remove event handler on watcher

* Some initial works on modify events and other small fixes

* File update event

* Trying to be more generic with used events and some tests to validate our assumptions

* Turning on location metadata file

* Introducing core unit tests on CI pipeline

* Submiting new unit test assumptions to validate on windows CI

* Fixing unit tests

* Fixing unit tests again

* Fixing unit tests

* Fixing unit tests for macos

* Fixing unit tests for macos again

* New structure for platform dependent event handling
Implementing event handlers for Linux and MacOS

* minor fixes + rustfmt + clippy

* Windows event handling

* Introducing a feature gate to only use location watching on desktop app for now

* Putting more stuff behind feature gates to avoid warnings

* Adding feature to cargo test on CI

* Changing some debug logs to trace logs and removing Jamie specific stuff

* Make location removal from manager less async

* fix build when "location-watcher" feature disabled

* fix types + clippy

* make location manager non-static

* remove uses of `to_string_lossy`

* more invalidate_query calls

* Expose `library_ctx` directly to avoid needless clones

* New materialized_path handling for directories

* Removing cascade delete between file_path and object
- Some other minor stuff

* remove unused `CurrentDirFileIdentifierJob`

Co-authored-by: Jamie Pine <ijamespine@me.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2022-12-31 00:53:24 +08:00
jake bbca699e33
[ENG-316] Decrypt with Password (#484)
* update generated key on slider click

* add ui for decrypting with password+saving to key manager

* update rust code to support password decryption and saving it to the library

* add decrypt with password UI functionality

* add content salt to key viewer

* disable key manager decryption if no keys are mounted

* clippy
2022-12-21 13:04:52 +08:00
jake 78a5754f4e
[ENG-315] Key viewer/general improvements (#483)
* add encryption+hashing algorithm to key viewer dialog

* abstract `save()` and `open()` with `Platform`

* add automount to `StoredKey` and offer option to enable/disable it for keys

* fix missing statement exec

* general tidy up

* add sync to library button and cleanup default setting code

* use alert dialogs for unsupported `open`/`save` operations (web)

* tweak alert message
2022-12-13 09:48:14 +08:00
jake 8ee2d18053
[ENG-309] Envelope encryption for the KM's root key (#482)
* use master key to encrypt the root key (breaking)

* introduce breaking changes to the keymanager (this breaks master password changing, backup restoration, etc)

* change MP -> RK

* minor tweaks + comment out broken features temporarily

* experimental new master password changing

* add a `write_storedkey_to_db` helper function

* general cleanup and fixes

* potentially patch backup restoration

* uncomment+clean up restoration in api

* strip whitespaces from user inputted secret key, and fix MPC bug

* clippy

* add memory only flag to `StoredKey`
2022-12-12 15:04:08 +00:00
jake f1292d6e51
Crypto Hotfixes (#477)
* fix annoying typo that was breaking master password changing under certain conditions

* move `Select` boxes out of `react-hook-form` so they work

* add `DummyKey` and rework the UI a little

* fix bug where user could select a hashing algorithm (it did nothing)

* fix bug where encryption algorithm would be inherited from the key, not the user's choice
2022-12-09 22:05:32 +08:00
jake 0eb0376d5b
[ENG-308] Crypto optimizations (#470)
* add benchmarks for encryption/hashing

* add more benchmarking sizes

* general performance improvements

* massive performance gain with minimal security impact

* update `sd-crypto` author info
2022-12-08 12:08:35 +00:00
jake 70b063b002
[ENG-311] Hex secret key encoding (#476) 2022-12-07 15:17:14 +00:00
jake 5651ac3f69
[ENG-251] Crypto Jobs (#475)
* add base crypto job

* leave hashing to the KM and not keyslots

* add latest header object versions and update examples

* add a very experimental (local file!) encrypt job

* working file encryption with default key

* working file decryption

* non-working encrypt file dialog

* general code cleanup

* comments and correct feature locking

* fix the `Dialog`

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>

* WIP encrypt dialog

* massive encrypt dialog overhaul

* add generic explorer alert dialog

* better key naming/numbering, alerts for unsupported scenarios

* cleanup + decrypt file dialog

* remove debugging stuff

* remove dead code & fix a bug with key select being empty

* clean up dialog UI massively

* add support for setting output path

* add correct button type to prevent early dialog submits

* add "nice" job data for encrypt/decrypt

* tweak functionality of key setting in encrypt dialog

* include full file metadata if selected, and lay out groundwork for preview media inclusion

* add comment about not restoring metadata/pvm just yet

* general cleanup

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
2022-12-07 15:16:13 +00:00
brxken128 bc4e8cb9c8
add clippy allow so Protected can work correctly 2022-12-07 15:04:57 +00:00
brxken128 4449bcb143
code cleanup and use hex for secret key encoding 2022-12-07 15:00:14 +00:00
jake 2baf16d982
[ENG-307] Key manager features (#467)
* working key management in settings page

* sync with library button

* fix `rspc` feature and add passphrase generation

* untested key manager rework

* trying to return values from mutations

* update library manager and remove settiong master PW

* update bindings

* set static secret key/master password

* prompt user for master password if correct one hasn't been provided yet

* add `hasMasterPassword` route

* add `clearMasterPassword` route + remove dead code

* tweak `set_master_password()` and add dedicated error

* tweak UI, fix `few hooks than expected`, add unmount+lock button

* remove old comment

* fmt

* clippy

* move static key/password setting so it doesn't fail sometimes

* add dedicated `get_key()` and remove keys from memory

* add `getKey` route

* update bindings

* use `const` instead of `let`

* comment updates

* update schema to remove salt

* add string parse error

* generate passphrase within key manager

* generate new migrations

* feature gate serde support in `crypto` crate

* fmt

* more specific error types

* foramatting

* add locking mechanism to keysettings page (not working?)

* fix react hook issues

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>

* remove empty onclick

* add keymanager dropdown menu

* working key backup functionality

* add experimental master password changing support

* update bindings

* semi-working change master password dialog

* use gear/lock icons to clean up key manager UI

* make settings button functional

* make buttons uniform and format code

* fix double base64 encode

* add change master password dialog and secret key dialog

* code cleanup

* restore backup dialog

* change UI wording

* make a start on restoring from a backup

* potentially working keystore restore

* don't overwrite verification key if one is set

* working backup restore + fix master password changing

* fix typo in static password/verification key check logic

* change wording to make UI clearer

* disable mount button if key is empty

* handle errors+remove type annotations

* show total imported keys on backup restoration

* add zxcvbn package

* change input border colour based on zxcvbn score

* clippy and formatting

* password strength meter

* remove nbsp

* add button type to stop early form submission

* use `react-hook-form` for backup restoration dialog

* more `react-hook-form` stuff

* attempt to fix password meter

* small cleanup

* Fix password meter

* update colours

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2022-12-02 10:18:21 +00:00
jake a403224b3b
[ENG-262] Key Manager Integration (#450)
* add keys router

* make progress on keymanager integration

* make name non-negotiable

* hyphenate encryption algorithm names

* Revert "make name non-negotiable"

This reverts commit 9c0f51329e.

* add some more keymanager queries

* add master password and default key routes

* add newly registered key to db + fmt

* clippy, formatting and `updateKeyName` route

* add automount to schema+automount keys where `true`

* update bindings

* working key add/mount

* working keylist

* mounted keys show first

* cleanup code

* add comments, code cleanup, more functions

* unmount all keys button

* comment and keymanager `clear_master_password()`

* add no keys available message

* fix unmount button

* use dashmap for concurrency

* fix missing keylist issue and add invalidate query macro

* set correct RSPC types

* statically set master password (TEMPORARILY)

* add remove key function within the keymanager

* key dropdown menu and impl

* formatting

* allow `option_if_let_else`

* add comment about key stats

* add additional comment

* rpsc error handling for the keys route

* fix rspc errors with an impl

* crypto crate errors to `sd-crypto::Error`

* remove `map_err`

* use custom result type

* cargo fmt

* clippy

* fix builds

* remove `Error::MutexLock`

* fix unnecessary unwrap

* mutex error handling (buggy for some reason)

* clean default key logic

* fix default key clearing

* allow a key to be removed without bugs

* implement requested changes

* use a single `useMemo`

* update schema with defaults

* re-generate migrations

* use rust enums in TS

* remove dead code

* remove mutate expansion

* read key list from keymanager, not prisma

* add "Default" key marker and cleanup TS

* rustfmt

* remove dead code
2022-11-05 04:18:01 -07:00
jake 92a3de4973
Fix metadata/preview media sizes for 32-bit targets (#446)
replace `usize` with `u64` for md/pvm lengths

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2022-10-29 02:07:10 -07:00
Brendan Allan b713c896a0
More Sync Stuff (#438)
* broken sync example

* move to react

* filter owned ops more

* update deps

* working sync

* relation operations

* fix pnpm lock

* please rustfmt

* fix api

* compare strings properly

* please clippy

* feature gate logging

* use bleeding rspc in example

* use bleeding rspc

* use generated rspc client
2022-10-27 23:12:28 -07:00
maxichrome 9d72de352b
Improve setup script (#436)
* Improve bash setup script (less silent!)

* use proper FFmpeg capitalization in relevant docs

* Fix typo in Homebrew error message

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
2022-10-25 18:08:50 -07:00
Jamie Pine 3a3ec6251f Merge branch 'main'
Conflicts:
	packages/interface/src/screens/settings/library/LibraryGeneralSettings.tsx
2022-10-23 18:43:27 -07:00
jake 0c7aed5f86
[ENG-258] Key Manager and Crypto Crate Improvements (#423)
* add base keymanager structs/functions

* change md/pvm `new()` to prevent useless re-hashing

* update lockfile

* update keymanager

* cleanup code, make things easier to understand

* move md and pvm construction to the header

* update pvm ser/de

* update metadata ser/de

* additional API changes, update example

* formatting and clippy

* update examples

* move `impl` to associated files

* formatting+clippy

* add more keymanager functionality

* add docs

* wrap `master_password` as an option

* add `sd-crypto` crate as a dependency to `core`

* add key manager to `LibraryContext`

* rename `id` -> `uuid`

* add more keymanager functions

* add `set_master_password()` function

* function to see if keymanager has master password

* update schema

* add default bool to schema

* populate keystore on SD startup

* clippy+formatting

* implement requested changes

* remove unwrap

* fmt, clippy, remove ser/de derives

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2022-10-23 03:47:41 -07:00
Jamie Pine 6ecdc6a02f finish theming 2022-10-23 03:24:46 -07:00
Jamie Pine c4226f0e7a Merge branch 'eng-259-identify-objectkind'
Conflicts:
	apps/landing/package.json
	apps/landing/vite.config.ts
	packages/interface/src/AppLayout.tsx
	packages/interface/src/components/explorer/Explorer.tsx
	packages/interface/src/components/explorer/ExplorerTopBar.tsx
	packages/ui/src/ContextMenu.tsx
	pnpm-lock.yaml
2022-10-20 22:49:15 -07:00
Brendan Allan 22330e6e71 sync docs 2022-10-19 14:44:09 +08:00
Jamie Pine 4acb8cc594
Merge branch 'main' of https://github.com/spacedriveapp/spacedrive 2022-10-17 01:36:05 -07:00
jake b5c571541e
[ENG-256] Crypto Crate Refinements (#410)
* add experimental preview media support

* clippy+formatting

* add support for mk/pvm nonces

* fix cursor positioning+size deserialization bug

* clippy+formatting

* cleanup code and add `decrypt_preview_media()`

* update comment

* remove `Mode` from `Keyslot`

* use little-endian `usize` for media length

* remove memory objects

* remove mode enum

* add `.zeroize()` to `Protected<>`

* remove `Mode` ser/de

* refactor, remove `Mode`, add helper functions

* formatting

* remove unused error

* update comments/docs

* comments, fix large files, insane performance

* doc updates and fix decrypt zeroize

* revert to 1048576 byte `BLOCK_SIZE`

* `Keyslot` and `PreviewMedia` constructors now handle encryption

* add metadata item with associated functions

* update comments, clean up useless items

* add metadata support within the header

* remove stray 128 in length calcs

* include metadata in header ser/de

* fmt+clippy

* rework keyslot decryption

* formatting

* api changes, code cleanup

* docs for `stream.rs`

* massive documentation re-write

* mark expensive doc test as ignore

* minor api tweaks

* add examples

* formatting+linting

* finalise documentation and add more examples

* formatting
2022-10-13 15:25:42 -07:00
Jamie Pine b94d2ed736
(hotfix) p2p preventing compile 2022-10-08 11:30:17 -07:00
jake d236b28f84
[ENG-250] Crypto Library (#400)
* add crypto crate with some functionality

* formatting

* add `argon2id` parameter levels

* add descriptive comments

* add stream decryption objects

* add `StreamEncryptor` struct

* add `StreamDecryptor`

* general cleanup

* add `thiserror` and error handling

* add header structs

* add basic serialization functionality

* advance serialization

* finish serialization

* clean up serialization and use `impl`

* finalise deserialization

* add stream helper functions and remove old code

* add AAD creation and retrieval

* add important comment

* add `ChaCha20Rng` as a CSPRNG

* cleanup and crate-wide clippy lints

* apply nursery lints

* add in-memory encryption objects

* rename `utils` to `objects`

* move (de)serialization rules to separate file

* add header-write helper function

* add password hash helper function

* add `decrypt_master_key` function

* cleanup, formatting, linting

* move keyslots to separate file, and rename them

* add basic comments

* remove `secrecy` dependency and import `protected`

* add `to_array` helper function

* `sd_crypto` -> `sd-crypto`

* remove manual drops

* add clippy allows

* add `new()` for `Keyslot` and `FileHeader`

* remove license

* zeroize read buffer on error

* magic bytes are now `ballapp`

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2022-10-07 07:31:40 -07:00
Jamie Pine 900ee618ce
Merge branch 'main' of https://github.com/spacedriveapp/spacedrive 2022-10-06 18:10:08 -07:00
Jamie Pine e402fc52b7
sd extensions crate
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2022-10-06 18:10:03 -07:00
Oscar Beaumont aacdf33216 merge duplicate prettier config and format repo with it 2022-10-07 07:45:15 +08:00
Oscar Beaumont fe4da9048a cargo fmt + fix clippy lints 2022-10-07 07:39:59 +08:00
Oscar Beaumont 75c4680b43 Merge remote-tracking branch 'origin/main' into merge-p2p-code 2022-10-07 05:56:05 +08:00
Oscar Beaumont d7fddc83f7 fix major bugs in p2p PR 2022-10-07 05:51:40 +08:00
Oscar Beaumont 598ba38a9d merge in P2P code 2022-10-07 05:32:45 +08:00
Oscar Beaumont 20523a94c9 upgrade workspace dependencies 2022-10-06 14:03:58 +08:00
Brendan Allan 723058e060 merge main into update-rspc-and-more 2022-10-05 08:58:03 +08:00
Brendan Allan ed06e3051e
[ENG-84] Sync library (#394)
* new sql lib stuff

* add sync stuff + rename all crates

* build sd-core

* add sync/example/dist to source

* fix sync example in monorepop

* appease clippy

* update lockfile

* update commit hooks

* fix typescript

* fix typescript build

* please rustfmt
2022-10-04 05:25:12 -07:00
Brendan Allan 66dd9aa28b
move prisma schema and migrations to /core (#392)
* move prisma schema and migrations to /core

* fix prisma updates
2022-10-04 02:32:46 -07:00
Oscar Beaumont 60e228ec94 upgrade rspc 2022-10-02 01:16:11 +08:00
Jamie Pine 3e8c33d1c0
Added crates folder (#389)
* restructure core for crates folder

- improve naming consistency
- added placeholder files for near future code
- added docs

* - removed all typescript from core
- moved rspc bindings to client

* remove useless git ignore & rustfmt
2022-09-29 21:02:29 -07:00