Commit graph

14 commits

Author SHA1 Message Date
Ericson "Fogo" Soares 6b760b0b2a
[ENG-1628] Write new indexer with the task system (#2161)
* Moving file-path-helper to a sub-crate on core

* Parallel walker tested and working

* Change inner core crate names to sd-core-* naming scheme

* Moving stuff around

* Save and Update tasks for the new indexer job

* Some initial drafts on the new job system

* More drafts on job system

* Changing walker task to a more explicit state machine
Also drafting more of job system

* More drafting on job resume

* Completed the draft on job system inner workings

* New job context abstraction to decouple library stuff from job system

* Properly use composition on task dispatcher

* First draft on indexer job

* Job serialization

* Handling ancestors in the distributed walker for indexer

* Saving computed directories sizes on a location to db

* Enable a way to check if jobs are running in a location

* Progress reporting on indexer job

* Reorganizing modules

* Shallow indexer

* Rust fmt

* Attempting windows CI fix

* Attempting windows CI fix again

* Attempting windows CI fix again
2024-04-08 14:32:44 +00:00
jake 355ac191b8
[ENG-440] Crypto Refactor (#2115)
* rebase: `crates/crypto` into current `main`

* refactor: remove `mnemonic` module

* feat: disable secure erase temporarily

* fix: tsc

* fix: tsc due to unused import

* fix: remove `cli` crypto info

* deps: update

* chore: remove dead comment

* refactor: remove `bincode` feature

* refactor: give `keyring` a dedicated feature so it's not reliant on `sys` as well

* fix: remove `aes-gcm` as it's no longer supported

* refactor: remove dead comment

* fix: update `keyring` imports

* refactor: change tests to `aes-256-gcm`

* feat: make `Key` a `Box<>` internally to ensure it's heap allocated (and fix tests)

* chore: clippy

* fix: hashing tests now that `const` keys aren't available

this will be cleaned up with test vectors and `include_bytes!()`

* chore: clippy

* refactor: remove dead code

* test: bring back `encrypt_with_invalid_nonce` test

* fix: secret service keyring

* fix: `zbus` build issues

* doc: update comment for clearer reasoning

* fix: cargo fmt

* fix: use bytes directly

* deps: update lockfile

* fix: secret service keyring

* fix: comment out windows keyring for now

* fix: use session keyring if no keyring backend

* fix: completely remove keyring module if no keyring is available for that OS

* fix: clippy

* fix: move iimport to correct conditional compilation

* fix: fmt
2024-03-07 08:00:37 +00:00
Ericson "Fogo" Soares 7c90bcb95b
[ENG-1479] AI Prototype (#1845)
* First draft on image labeling

* Fixing execution providers for other OSs

* Better error handling and shutdown

* Working with shallow media processor

* bruh

* Fix warnings

* Now hooked to media processor job

* Link desktop app with libonnxruntime to avoid TLS error during startup

* Be able to change models on runtime
Revert to use labels table instead of tags

* A bug on a model-less inference

* Show AI labels on Inspector
 - Change yolo inference to use half precision
 - Add labels api to core

* Remove LD_PRELOAD

* Fix race condition on model executor shutdown

* Don't load all images in memory moron

* Embeed yolo model in prod build
 - Change yolo model path to new one relative to executable

* Disable volume watcher on linux, it was crashing the app
 - Invalidate labels when they are updated

* Rust fmt

* Minor changes

* Gate onnxruntime linking to the ai-models feature

* Add build script to sd-server to handle onnxruntime linking workaround

* Move AI stuff to its own crate and normalize deps

* Rust fmt

* Don't regenerate labels unless asked to

* Now blazingly fast

* Bad merge

* Fix

* Fix

* Add backend logic to download extra yolo models

* Add models api route
 - Add api call to get available model version
 - Add api call to change the model version

* Improve new model download logic
 - Add frontend to change image labeler model

* Fix new model downloader

* Fix model select width

* invalidate labels count after media_processor generates a new output

* Rename AI crate and first draft on download notifications

* fix types

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-12-19 09:28:57 +00:00
Oscar Beaumont 6941912891
[ENG-1399] Upgrade Rust workspace + IPv6 mDNS (#1729)
Upgrade Rust workspace + IPv6 mDNS
2023-11-04 10:20:22 +00:00
Oscar Beaumont 94d8f61de9
Update Rust dependencies (#1314)
* Update most of Rust dependencies

* downgrade notify

* a

* fix Linux?

* Android is painful

* update build.gradle

* The lord Clippy has spoken

* Bruh

---------

Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Ericson "Fogo" Soares <ericson.ds999@gmail.com>
2023-09-12 14:03:25 +00:00
Brendan Allan 00e4aa9e8f
Sync ingesting (#1181)
* sync + ingest refactor

* fix Event enum description

* actually do sync over network

* re-enable heif

* remove comment
2023-08-07 07:52:43 +00:00
Vítor Vasconcellos 7e4ed2b547
[ENG-767, ENG-916] Improvements and fixes for Open/Open With (#1082)
* Normalize PATH and XDG envvars on Linux

* Fix normalize_xdg_environment

* Replace custom Desktop file parsers with Glib

* Fix AppImage env influencing external apps
 - Normalize GStream plugin path

* Fix macos pulling linux deps

* Attempt to fix gnome apps failing to launch
 - Fix incorrect logic in `normalize_pathlist`

* Ensure empty envvars are not set

* Revert AppImage ovewritten GTK specific variables
 - `normalize_pathlist` now prefers entries with less priority when dealing with repeated entries, this is not compatible with the default behavior, but it is a more sane approach IMHO

* Remove 32-bit libs from release CI build host

* Remove 32-bit from github runner 2 attempt
 - Remove deprecated vscode config

* Remove libc6-i386

* [ENG-916] Implement `Open With` logic compatible with macOS < 12

* Add some missing gstreamer deps in Linux

* Replace opener with Glib API

* Fix reveal opening file instead of dir
 - Improve Open With logic in Windows
 - Expose functions to test if app is in a flatpak, snap or appimage

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-24 17:36:00 +00:00
Ericson "Fogo" Soares e693c7a542
[ENG-381] Update context menufs actions to support multiple source (#927)
* Generalizing filesystem jobs to accept multiple files at once

* Some small changes and fixing front

* Enable rename route to replicate Finder's behavior

* Assign tag to multiple objects at once

* Remove from recents accepting multiple files

* Fixing some warnings

* Adding multiple files feature to Open and Open With

* Conditional stuff for macos

* Generating commands.ts and other minor warnings

* Rust fmt

* TS typecheck

* Rust format and TS typecheck

* Requested changes

* Requested changes

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-06-14 23:00:28 +00:00
Oscar Beaumont e4696c7303
[ENG-667] Fix Clippy CI (#887)
* plz just work

* no shot

* just disable Tauri notifications

* Tauri is weird AF
2023-05-30 17:11:37 +00:00
jake 1aa8826398
SPDX license and workspace inheritence for re-used metadata (#833)
use workspace inheritence for license, edition and repo URL
2023-05-20 01:52:55 +00:00
jake d1b6263ae7
[ENG-355] Keychain integration (and some typesafety) (#558)
* update crypto MSRV

* rename `keychain` to `keyring`

* make a start on the keymanager unlock refactor/keychain integration

* update routes

* update bindings

* add const identifiers

* add UI/front-end support for unlocking KM with OS keychains

* remove SK from lib creation dialog

* update query name

* add keyring functions

* attempt to update `change_master_password()` to use the keychain

* cleanup, fix master password change ui, better secret key in keyring detection

* cleanup TS a little

* add route for getting secret key from keyring

* update bindings

* update var names + show secret key in keys settings

* add `react-qr-code` and option to view the secret key (if it's in the OS keyring)

* allow copying SK to clipboard

* add `key_type` so we're not reliant on specific UUIDs for root/verification key handling

* clippy

* fix mobile typecheck

* fix typecheck, fix typo and tweak balloon hash parameters

* minor cleanup + typo fix

* use newtype structs

* WIP type refactoring (major readability boost!)

* update `use`

* add tokio `sync` feature

* too many structs? idk

* more cleanup

* add `generate` and `Nonce`

* `Nonce` and `Key` typesafety (beautiful)

* clippy + cleanup

* update code & examples

* fix bug & remove `ProtectedVec` as it looked out of place

* use `Key`

* add a query invalidation to make the UI extremely responsive

* ci pls work

* remove `keyringHasSk` route
2023-02-07 12:03:12 +00:00
brxken128 2abf75defd
update things to use workspace tokio 2023-01-25 22:03:10 +00:00
brxken128 d8b6789cb7
update cli to use async crypto 2023-01-24 16:19:30 +00:00
jake 22bc77a39e
[ENG-319] Balloon hashing (#489)
* add key attribute to `Key` in `ListOfKeys`

* add balloon hashing function with untailored parameters

* add ser/de rules for blake3-balloon (and change argon2id's)

* fix benchmark

* use `to_bytes`, `from_bytes` and `from_reader`

* cleanup code

* add blake3-balloon options to the UI and fix library sync/automount enable bug

* cleanup some serialization code

* fix hashing algorithm deserialization

* clean up header serialization + more idiomatic master key decryption

* clippy

* add generic ser/de error to crypto crate

* fix `Display` and crypto cli

* move crypto cli to cli app

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-01-04 09:57:02 +00:00