Commit graph

62 commits

Author SHA1 Message Date
Brendan Allan fd39dc3a3d
Replace Location.local_path with path (#571)
* replace Location.local_path with path

* Using more references to avoid unneeded moves and removing unneeded errors variants

* remove unnecessary stuff

* location id checks

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-02-20 05:14:47 +00:00
Jamie Pine f47a2d58e5
[ENG 239] Onboarding Flow & Location Settings (#529)
* begin better onboarding

* added input and altered text

* better router & text + database icon

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>

* work on privacy screen + radio buttons

* fix video extension bug and alter screens

* add pending schema and location manager helper

* functional onboarding

* added secure temp store and started creating library loading screen

* fix secure temp keystore + api

* better onboarding

* added location settings and some overview concept, all WIP

* fix switch

* prep

* fix location router

* added backend settings

* attempted to fix form

* begin indexer rules editor, plus tweaks

* indexer rules coming soon

* fix onboarding img size

* cleanup

* clone is needed here, but clippy no like

* sike

* whole bunch of fixes

* clippy + ts

* Removing some TODOs from api/libraries.rs and fixing db size calculation

* moved object kind to client, added half functionality for appearance settings

* fix RadioGroup helper

* fix type issues

* cargo fmt

* fix creating library error handling + invalidate location list on update

* forgot to switch back to onError

* Invalidating getStatistics query on library creation and introducing the concept of waiting for a job on FileCopierJob

* F* cargo fmt

* fix RadioGroup interactivity

* wipe all migrations

* put back COLLATE NOCASE on extension columns

* update core.ts

* remove unused device component

* fix typeerror in mobile

---------

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-02-10 14:08:13 -08:00
Brendan Allan 1436d779b8
Move location online status out of database (#560)
* move location online status out of database

* remove useLibrarySubscription

* remove old code
2023-02-07 05:58:47 -08:00
jake d1b6263ae7
[ENG-355] Keychain integration (and some typesafety) (#558)
* update crypto MSRV

* rename `keychain` to `keyring`

* make a start on the keymanager unlock refactor/keychain integration

* update routes

* update bindings

* add const identifiers

* add UI/front-end support for unlocking KM with OS keychains

* remove SK from lib creation dialog

* update query name

* add keyring functions

* attempt to update `change_master_password()` to use the keychain

* cleanup, fix master password change ui, better secret key in keyring detection

* cleanup TS a little

* add route for getting secret key from keyring

* update bindings

* update var names + show secret key in keys settings

* add `react-qr-code` and option to view the secret key (if it's in the OS keyring)

* allow copying SK to clipboard

* add `key_type` so we're not reliant on specific UUIDs for root/verification key handling

* clippy

* fix mobile typecheck

* fix typecheck, fix typo and tweak balloon hash parameters

* minor cleanup + typo fix

* use newtype structs

* WIP type refactoring (major readability boost!)

* update `use`

* add tokio `sync` feature

* too many structs? idk

* more cleanup

* add `generate` and `Nonce`

* `Nonce` and `Key` typesafety (beautiful)

* clippy + cleanup

* update code & examples

* fix bug & remove `ProtectedVec` as it looked out of place

* use `Key`

* add a query invalidation to make the UI extremely responsive

* ci pls work

* remove `keyringHasSk` route
2023-02-07 12:03:12 +00:00
Brendan Allan 95ba2e1246 remove temp_cas_id 2023-02-01 11:23:59 +08:00
Brendan Allan bf362eadd6 store cas ids on file paths 2023-01-28 11:42:24 +08:00
Brendan Allan 8e479f736f
Eng 332 auto generate sync ids (#523)
* generate sync ids and use model name from ModelActions

* cleanup AddLocationDialog imports

* rustfmt
2023-01-15 07:25:11 -08:00
jake 0d554e9d06
[ENG-331] StoredKey overhaul (#513)
* add wip storedkey versioning

* storedkey versioning! (not pretty, but it never will be)

* add version to `StoredKey` and re-gen migrations to handle serde

* use `serde` for interacting with the DB + handle errors
2023-01-11 05:15:08 -08:00
Brendan Allan 58f5d8bb4d
update prisma (#505) 2023-01-05 08:05:33 -08:00
Brendan Allan 676e512c4a
Beginning of sync stuff (#502) 2023-01-04 23:11:55 -08:00
jake db5b401238
[ENG-318] Implement a KDF to derive new keys (#487)
* add blake3, `derive_key()` and clean up code

* fix a couple of things from the previous commit

* add context strings for root/file key derivation

* add salt to schema

* update refs of `salt` to `content_salt` within the keyslot

* cleanup code and add kdf salt to the keyslot

* rename salt to content salt in examples

* cleanup header code + remove dead code

* implement key derivation for keyslots

* gen new migrations that contain a salt column

* keymanager refactor (code is very idiomatic now) - needs thorough testing

* further cleanup

* clippy

* add a master password context string

* use key derivation for deriving keys from the root key

* update to use new code and remove `match` from `en/decrypt_bytes()`

* clippy

* use less unwraps in library manager code

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2022-12-30 20:09:44 +00:00
Ericson "Fogo" Soares bcbcd260d4
ENG 223 Location Awareness (#468)
* Introducing location online/offline checks and location relink

* Some initial drafts for location watchers

* Location metadata relink and add library

* Many improvements at job system
Now using prisma batching at identifier job
Removing blocking I/O from extension subcrate
Implementing lazy generation of thumbnails
New current directory identifier job to be used on light rescans

* Some optimizations on identifier and object validator jobs

* merge jamie's identifier PR

* fully repaired identifier job

* properly hooked up object kind

* inspector fix

* fix video badge

* small improvements to libraries settings

* identifier and inspector improvements

* fix feature flags and hook up context menu location utilities

* BETTER CONTEXT MENU x100

* test-files

* style tweaks

* new icon designs

* manifest

* fix thumbnails on web

* media data

* New Location Watcher and some minor fixes

* disable broken media_data extractor, wip

* wip

* function name fix

* Fixing pnpm prep and some warnings

* Solving a race condition beetween indexer job and FS event handlerSome other minor warnings

* Generating thumbnails on watcher

* Remove event handler on watcher

* Some initial works on modify events and other small fixes

* File update event

* Trying to be more generic with used events and some tests to validate our assumptions

* Turning on location metadata file

* Introducing core unit tests on CI pipeline

* Submiting new unit test assumptions to validate on windows CI

* Fixing unit tests

* Fixing unit tests again

* Fixing unit tests

* Fixing unit tests for macos

* Fixing unit tests for macos again

* New structure for platform dependent event handling
Implementing event handlers for Linux and MacOS

* minor fixes + rustfmt + clippy

* Windows event handling

* Introducing a feature gate to only use location watching on desktop app for now

* Putting more stuff behind feature gates to avoid warnings

* Adding feature to cargo test on CI

* Changing some debug logs to trace logs and removing Jamie specific stuff

* Make location removal from manager less async

* fix build when "location-watcher" feature disabled

* fix types + clippy

* make location manager non-static

* remove uses of `to_string_lossy`

* more invalidate_query calls

* Expose `library_ctx` directly to avoid needless clones

* New materialized_path handling for directories

* Removing cascade delete between file_path and object
- Some other minor stuff

* remove unused `CurrentDirFileIdentifierJob`

Co-authored-by: Jamie Pine <ijamespine@me.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2022-12-31 00:53:24 +08:00
jake 2baf16d982
[ENG-307] Key manager features (#467)
* working key management in settings page

* sync with library button

* fix `rspc` feature and add passphrase generation

* untested key manager rework

* trying to return values from mutations

* update library manager and remove settiong master PW

* update bindings

* set static secret key/master password

* prompt user for master password if correct one hasn't been provided yet

* add `hasMasterPassword` route

* add `clearMasterPassword` route + remove dead code

* tweak `set_master_password()` and add dedicated error

* tweak UI, fix `few hooks than expected`, add unmount+lock button

* remove old comment

* fmt

* clippy

* move static key/password setting so it doesn't fail sometimes

* add dedicated `get_key()` and remove keys from memory

* add `getKey` route

* update bindings

* use `const` instead of `let`

* comment updates

* update schema to remove salt

* add string parse error

* generate passphrase within key manager

* generate new migrations

* feature gate serde support in `crypto` crate

* fmt

* more specific error types

* foramatting

* add locking mechanism to keysettings page (not working?)

* fix react hook issues

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>

* remove empty onclick

* add keymanager dropdown menu

* working key backup functionality

* add experimental master password changing support

* update bindings

* semi-working change master password dialog

* use gear/lock icons to clean up key manager UI

* make settings button functional

* make buttons uniform and format code

* fix double base64 encode

* add change master password dialog and secret key dialog

* code cleanup

* restore backup dialog

* change UI wording

* make a start on restoring from a backup

* potentially working keystore restore

* don't overwrite verification key if one is set

* working backup restore + fix master password changing

* fix typo in static password/verification key check logic

* change wording to make UI clearer

* disable mount button if key is empty

* handle errors+remove type annotations

* show total imported keys on backup restoration

* add zxcvbn package

* change input border colour based on zxcvbn score

* clippy and formatting

* password strength meter

* remove nbsp

* add button type to stop early form submission

* use `react-hook-form` for backup restoration dialog

* more `react-hook-form` stuff

* attempt to fix password meter

* small cleanup

* Fix password meter

* update colours

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2022-12-02 10:18:21 +00:00
jake a403224b3b
[ENG-262] Key Manager Integration (#450)
* add keys router

* make progress on keymanager integration

* make name non-negotiable

* hyphenate encryption algorithm names

* Revert "make name non-negotiable"

This reverts commit 9c0f51329e.

* add some more keymanager queries

* add master password and default key routes

* add newly registered key to db + fmt

* clippy, formatting and `updateKeyName` route

* add automount to schema+automount keys where `true`

* update bindings

* working key add/mount

* working keylist

* mounted keys show first

* cleanup code

* add comments, code cleanup, more functions

* unmount all keys button

* comment and keymanager `clear_master_password()`

* add no keys available message

* fix unmount button

* use dashmap for concurrency

* fix missing keylist issue and add invalidate query macro

* set correct RSPC types

* statically set master password (TEMPORARILY)

* add remove key function within the keymanager

* key dropdown menu and impl

* formatting

* allow `option_if_let_else`

* add comment about key stats

* add additional comment

* rpsc error handling for the keys route

* fix rspc errors with an impl

* crypto crate errors to `sd-crypto::Error`

* remove `map_err`

* use custom result type

* cargo fmt

* clippy

* fix builds

* remove `Error::MutexLock`

* fix unnecessary unwrap

* mutex error handling (buggy for some reason)

* clean default key logic

* fix default key clearing

* allow a key to be removed without bugs

* implement requested changes

* use a single `useMemo`

* update schema with defaults

* re-generate migrations

* use rust enums in TS

* remove dead code

* remove mutate expansion

* read key list from keymanager, not prisma

* add "Default" key marker and cleanup TS

* rustfmt

* remove dead code
2022-11-05 04:18:01 -07:00
Oscar Beaumont b099d47fec fix database migrations 2022-10-25 06:53:24 +10:00
jake 0c7aed5f86
[ENG-258] Key Manager and Crypto Crate Improvements (#423)
* add base keymanager structs/functions

* change md/pvm `new()` to prevent useless re-hashing

* update lockfile

* update keymanager

* cleanup code, make things easier to understand

* move md and pvm construction to the header

* update pvm ser/de

* update metadata ser/de

* additional API changes, update example

* formatting and clippy

* update examples

* move `impl` to associated files

* formatting+clippy

* add more keymanager functionality

* add docs

* wrap `master_password` as an option

* add `sd-crypto` crate as a dependency to `core`

* add key manager to `LibraryContext`

* rename `id` -> `uuid`

* add more keymanager functions

* add `set_master_password()` function

* function to see if keymanager has master password

* update schema

* add default bool to schema

* populate keystore on SD startup

* clippy+formatting

* implement requested changes

* remove unwrap

* fmt, clippy, remove ser/de derives

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2022-10-23 03:47:41 -07:00
jake 4cd2dde35c
CAS ID Improvements (#413)
* remove `ring` dependency and use `sha2` instead

* use BLAKE3 and include full file checksum

* update schema comments
2022-10-13 15:31:47 -07:00
Jamie Pine cbd58ee438
Rename Files to Objects (#396)
* backend rename complete

* finalize Object refactor frontend
+ remove some exhaustive dep suppression
+ FIX INVALIDATE QUERY BUG
2022-10-04 09:16:49 -07:00
Brendan Allan 66dd9aa28b
move prisma schema and migrations to /core (#392)
* move prisma schema and migrations to /core

* fix prisma updates
2022-10-04 02:32:46 -07:00
Jamie Pine 3e8c33d1c0
Added crates folder (#389)
* restructure core for crates folder

- improve naming consistency
- added placeholder files for near future code
- added docs

* - removed all typescript from core
- moved rspc bindings to client

* remove useless git ignore & rustfmt
2022-09-29 21:02:29 -07:00
Brendan Allan 1bf315c4db
Use feature gated database branch of PCR (#387)
use feature gating branch of pcr
2022-09-22 23:40:15 -07:00
Brendan Allan 6e07435a15
Use PCR for migrations (#385)
* use pcr migrations branch

* use 0.6.2 branch with migrations

* use latest prisma stuff

* allow force reset of db in dev

* remove .spacedrive file

* update rspc in apps/server

* use rspc 0.0.5 in all crates

* add os to prisma client cache key

* add runner os to clippy prisma cache
2022-09-22 00:50:21 -07:00
Oscar Beaumont 5dd45f2b93 onboarding UI flow 2022-09-12 23:43:12 +08:00
Oscar Beaumont 29862313e6 Merge remote-tracking branch 'origin/main' into eng-237-explorer-performance-2 2022-09-09 15:30:17 +08:00
Oscar Beaumont 2fda5b9ca6
Better db migrations (#381)
* use prisma migration engine for migrations

* squash migrations + remove excessive logging
2022-09-08 15:57:07 -07:00
Ericson Fogo Soares a4b6b7b5c9 Implementing metadata for jobs 2022-09-07 23:08:11 -03:00
Brendan Allan 459a44b7ac utilise include instead of with for type safety 2022-09-07 04:09:57 +08:00
Oscar Beaumont 4427065658 Upgraded rspc with libraries + upgrade PCR 0.6.0 2022-09-06 22:08:19 +08:00
Jamie Pine 05bcb6e1b0
Assignable Tags (#370)
* Tag queries + Identifier bug fix

- added empty pages for docs
- added non-functional "New Library" button to sidebar
- to accomplish above, moved dialog logic to component folder
- fixed Identifier bug where location id was not considered in the orphan files query, meaning it would attempt to identify many non-existent paths and fail to create the legitimate ones

* (fix) maintain file item aspect ratio in grid view
(fix) get tag query + get all tags

* codegen

* resurrected context menu

* fix window flash

* remove location based thumb sorting + update react on landing

* add is_archived to location

* improved context menu

* assign tag + refactor explorer store

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>

* assign/unassign tags from context menu

* fix lint

* keep context menu open on tag change

* fix brendan feedback + merge main

* fix type

* codegen

* style changes

* revert windows size and xcode proj

* remove outdated doc

* coming sooooooon

* updated release notes

* release notes

* dashes > dots

* clean up docs

* more docs!

* restore entitlements

* remove unused util

* regenerated migrations post merge
- some additional error handling added to migration runner, needs more work!

* refactor explorer to support tags

* fix error and revert explorer bg color

* put it in a box they said

* revert location id store removal

* upgrade hero icons + style tweaks

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
2022-09-04 13:00:24 -07:00
Ericson "Fogo" Soares 25c111a8bf
Introducing Indexer rules (#363)
* Introducing indexer rules for the indexer job
* Solving a minor race condition at `Worker::spawn`
* Many refactors at database accesses with prisma
* Documenting some features of indexer rules

* Updating to new prisma rev

* Updating photo

* Some documentation tweaks

* fix note editing

* Implementing some requested changes
* Fixing unit tests on walk function
* Using rsps::Error::with_cause instead of new

* Introducing a compoung unique key between `node_id` and `local_path` at `locations` table

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2022-09-01 16:38:26 -07:00
Brendan Allan 2c375ce27a use latest prisma-client-rust 2022-08-19 07:41:08 +08:00
Oscar Beaumont ea92af57f8
Move Rust backend to rspc (#345)
* move Rust backend to rspc

* move server to Axum + remove parts of old bridge

* move frontend over to @rspc/client

* move core to rspc + update deps

* fix Typescript errors + upgrade deps

* document invalidate_query! macro

* general cleanup + upgrade to PCR 0.6.0

* prisma error handling

* upgrade to rspc 0.0.4

* update vite-plugin-ssr

* fix typescript

* fix builds

* put landing page app name back

* hardcode ffmpeg version on Windows

* rename 'command' to 'mutation' to line up with react-query terminology

* upgrade rspc to v0.0.5 + fix types

* use shared ffmpeg binaries

* general Typescript cleanup

* fix clippy workflow failing due to tauri proc-macro

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2022-08-03 08:36:03 -07:00
Ericson "Fogo" Soares 85e5eec993
Resumable Jobs + Lefthook Integration (#344)
* Introducing Lefthook for git hooks automation
* TypeScript typechecking and linting
* Rust formatting and linting
* Spellchecking (also corrected spell errors in some files)
* Check links in md files

* Introducing resumable jobs
* Abstractions to pause and resume jobs automatically when application exits and is started
* Changing database to use Bytes for UUID fields
* Changing uuid fields on core to use uuid::Uuid instead of String
* Updating some dependencies and introducing msg_pack serialization to save job state on database

* Fixing some clippy warnings

* Fixing a regression introduced on identifier job, doing too much db accesses concurrently
2022-07-27 00:06:34 -07:00
Jamie Pine 9961c49759
Bunch 'O Features (#336)
* CRUD for tags.

* Implement tags query in open.rs and make some changes to CRUD.

* Tag update.

* Hopefully working get tags.

* added node config + spaces schema

* add missing routes

* begin tag ui

* renaming query names to better fit convention

* tags progress

* tag edit

* fix delete tag description

Co-authored-by: xPolar <polar@polar.blue>
2022-07-17 20:45:04 -07:00
Oscar Beaumont c685ce5fe9
Library manager (#258) 2022-07-10 19:05:24 -07:00
Jamie Pine 2cc3f3d95d
Location Settings (#302)
* Delete locations

+ Show online status

* style tweaks

* tweaks

* location rescan button

* fix location delete db locking bug

* opting to remove self referencial relation on file_path

* correct query

* consolodate migration

* consolodate migrations
2022-06-24 06:26:45 -07:00
xPolar 77713bab7e
Notes on files. (#253)
* Temp fix for build.rs

* Note implementation

* Notes work now, need to iron out some things.

* Finish file notes.

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>

* Remove lodash debounce

* Fix pnpm locl

Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
2022-06-20 21:14:05 -07:00
maxichrome aa0fb6c2d1
Merge branch 'main' into jamie-unsorted-changes 2022-05-29 02:39:29 -05:00
Polar 026b559d77 various grammar and typo fixes. 2022-05-27 14:54:56 -07:00
Brendan Allan d164db7fc3 cleanup + update to prisma-client-rust 0.5.0
still waiting to rebase for changes to identifier
2022-05-26 16:17:46 +08:00
Jamie Pine 5175dcbbfb database fixes 2022-05-22 22:28:33 -07:00
Jamie Pine 363bdaeacd remove migration history (#173), fix incorect name 2022-05-22 22:02:18 -07:00
maxichrome f5e0666d11
Format existing code with new conventions 2022-05-22 23:24:43 -05:00
maxichrome a0d0938001
format using tabs 2022-05-22 23:07:35 -05:00
Jamie Pine 86dd403933 improved sync schema 2022-05-22 20:42:41 -07:00
Jamie Pine 19e833ba16 removed redundant schema column + some demo data (unfinished) 2022-05-21 18:48:51 -07:00
Jamie Pine dd518b914f add name to jobs 2022-05-11 20:09:25 -07:00
Jamie Pine 9870d58b35 much 2022-05-10 09:03:20 -07:00
Jamie Pine 181c51e76c better statistics 2022-04-27 20:42:28 -07:00
Jamie Pine eee8efb99a added semi functional statistics, working macos traffic lights 2022-04-27 19:15:55 -07:00