* initial ffprobe commit
* Working slim down version ffprobe
* Auto format ffprobe and deps source
* Remove show_pixel_formats logic
- Fix do_bitexact incorrect check in main after last changes
- Fix some clangd warning
* Remove show_* and print_format options and their respective logic
* Rework ffprobe into simple_ffprobe
- Simplify ffprobe logic into a simple program that gather and print a media file metadata
* Reduce the amount of ffmpeg log messages while generating thumbnails
* Fix completly wrong comments
* mend
* Start modeling ffmpeg extracted metadata on schema
- Start porting ffprobe code to rust
- Rename some references to media_data to exif_data
* Finish modeling media info data
- Add MediaProgram, MediaStream, MediaCodec, MediaVideoProps, MediaAudioProps, MediaSubtitleProps to Schema
- Fix simple_ffproble to use its custom print_codec, instead of ffmpeg's impl
* Add relation between MediaInfo and FilePath
- Remove shared properties from MediaInfo and related structs
- Implement Iterator for FFmpegDict
* Fix and update schema
* Data models and start populating MediaInfo in rust
* Finish populating media info, chapters and program
* Improve FFmpegFormatContext data raw pointer access
- Implement stream data gathering
* Impl FFmpegCodecContext, retrieve codec information
- Improve some unsafe pointer uses
- Impl from FFmpegFormatContext to MediaInfo conversion
* Fix FFmpegDict Drop
* Fix some crago warnings
* Impl retrieval of video props
- Fix C char* to Rust String convertion
* Impl retrieval of audio and subtitle props
- Fill props for MediaCodec
* Remove simple_ffprobe now that the Rust impl is done
* Fix schema to match actually retrieved media info
- Fix import some FFmpeg constants instead of directly using values
* Rework movie_decoder
- Re-implement create_scale_string and add support anamorphic video
- Improve C pointer access for FFmpegFormatContext and FFmpegCodecContext
- Use newer FFmpeg abstractions in movie_decoder
* Fix incorrect props when initializing MovieDecoder
* Remove unecessary lifetimes
* Added more native wrappers for some FFmpeg native objects used in movie_decoder
* Remove FFmpegPacket
- Some more improvements to movie_decoder
* WIP
* Some small fixes
* More fixes
Rename movie_decoder to frame_decoder
Remove more references to film_strips
* fmt
* Fix duplicate migration for job error changes
* fix rebase
* Solving segfaults, fuck C lang
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Update rust to version 1.77
- Pin rust version with rust-toolchain.toml
- Change from dtolnay/rust-toolchain to IronCoreLabs/rust-toolchain for rust-toolchain support
- Remove unused function and imports
- Replace most CString uses with new c literal string
* More segfault solving and other minor fixes
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Fix ffmpeg rotation filter breaking portrait video thumbnails #2150
- Plus some other misc fixes
* Auto format
* Retrieve video/audio metadata on frontend
* Auto format
* First draft on ffmpeg data save on db
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Fix some incorrect changes to prisma schema
* Some fixes for the FFmpegData schema
- Expand logic to save FFmpegData to db
* A ton of things
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Integrating ffmpeg media data in jobs and API
* Rspc can't BigInt
* 🙄
* Add initial ffmpeg metadata entries to Inspector
- Fix ephemeral metadata api to match the files metadata api call
* Fix Inspector not showing ffmpeg metadata
* Add bitrate, start time and chapters video metadata to Inspector
- Fix backend BigInt conversion incorrectly using i32 instead of u32
- Change FFmpegFormatContext/FFmpegMetaData bit_rate to i64
- Rename byteSize to humanizeSize
- Expand humanizeSize logic to allow handling bits and Binary units
- Move capitalize to @sd/client utils
* Solving some issues
* Fix ffmpeg probe getting incorrect stream id and breaking database unique constraint
- Fix humanizeSize breaking when receiving floating numbers
- Fix incorrect equality in StatCard
- Fix unhandled error in Dialog when trying to remove an unknown dialog
* fmt
* small improvements
- Remove some unecessary recursion_limit directive
- Remove unused app_image releated functions
- Fix metadata query enabled flag
* Add migration for ffmpeg media data
* Fix cypress test
* Requested changes
* Implement feedback
- Update locale keys for all languages
- Add pnpm command to update all language keys
* Fix thumb reactivity in non indexed locations
---------
Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Most of it
* Cleanup menu
* fixes
* Fixed linux compilation, still not working though, as the app panic on start
* Add dialog permissions
* fix search keybind
* Fix linux
∙ - Disable linux-ipc-protocol feature, as that is causing panics due to trying to access some WebKit internal structures outside the main thread
* Update to Ubuntu 22.04, new tauri doesn't support ubuntu 20.04
> - Add note to download page about deb's distro support
* Remove gen/schemas
* Update linux dependencies in setup.sh
* Fix linux deps again
- Enable rt-tokio-crypto-rust feature in secret-service so clippy stops complaining
* Clippy + auto fmt
* Fix cache-factory
- Create a devtools feature for desktop app, which is enabled by default on dev builds
* Fix minor error in patchTauri.mjs
* Fix some envvars that have been renamed on tauri v2
* Dont change the secrets
* Add missing linux dependency
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Couple of fixes
- Increase `pnpm prep` connection timeout to 5min, to better handle downloading native deps under flaky network conditions
- Fix `cargo test` and cache-factory CI
- Clippy and fmt
* rebase: `crates/crypto` into current `main`
* refactor: remove `mnemonic` module
* feat: disable secure erase temporarily
* fix: tsc
* fix: tsc due to unused import
* fix: remove `cli` crypto info
* deps: update
* chore: remove dead comment
* refactor: remove `bincode` feature
* refactor: give `keyring` a dedicated feature so it's not reliant on `sys` as well
* fix: remove `aes-gcm` as it's no longer supported
* refactor: remove dead comment
* fix: update `keyring` imports
* refactor: change tests to `aes-256-gcm`
* feat: make `Key` a `Box<>` internally to ensure it's heap allocated (and fix tests)
* chore: clippy
* fix: hashing tests now that `const` keys aren't available
this will be cleaned up with test vectors and `include_bytes!()`
* chore: clippy
* refactor: remove dead code
* test: bring back `encrypt_with_invalid_nonce` test
* fix: secret service keyring
* fix: `zbus` build issues
* doc: update comment for clearer reasoning
* fix: cargo fmt
* fix: use bytes directly
* deps: update lockfile
* fix: secret service keyring
* fix: comment out windows keyring for now
* fix: use session keyring if no keyring backend
* fix: completely remove keyring module if no keyring is available for that OS
* fix: clippy
* fix: move iimport to correct conditional compilation
* fix: fmt
* First draft on image labeling
* Fixing execution providers for other OSs
* Better error handling and shutdown
* Working with shallow media processor
* bruh
* Fix warnings
* Now hooked to media processor job
* Link desktop app with libonnxruntime to avoid TLS error during startup
* Be able to change models on runtime
Revert to use labels table instead of tags
* A bug on a model-less inference
* Show AI labels on Inspector
- Change yolo inference to use half precision
- Add labels api to core
* Remove LD_PRELOAD
* Fix race condition on model executor shutdown
* Don't load all images in memory moron
* Embeed yolo model in prod build
- Change yolo model path to new one relative to executable
* Disable volume watcher on linux, it was crashing the app
- Invalidate labels when they are updated
* Rust fmt
* Minor changes
* Gate onnxruntime linking to the ai-models feature
* Add build script to sd-server to handle onnxruntime linking workaround
* Move AI stuff to its own crate and normalize deps
* Rust fmt
* Don't regenerate labels unless asked to
* Now blazingly fast
* Bad merge
* Fix
* Fix
* Add backend logic to download extra yolo models
* Add models api route
- Add api call to get available model version
- Add api call to change the model version
* Improve new model download logic
- Add frontend to change image labeler model
* Fix new model downloader
* Fix model select width
* invalidate labels count after media_processor generates a new output
* Rename AI crate and first draft on download notifications
* fix types
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
* basic declared actor manager
* put actors in separate file
* clippy
* hopefully clean up some clippy warnings
---------
Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
* sd-pdf
* Process PDF blocking render inside a spawn_blocking
- Load a single global Pdfium instance
* Migrate pdf thumb logic to sd-images
- Replace block_in_place with spawn_blocking
- Only load LibHeif once
- Allow thumbnailer (both indexed and non-indexed locations) to process documents
- Disable loading pdf viewer in Inspection in favour of loading it's thumbnail
* Try to load pdfium lib from absolute path
* Revert removed import due to rebase
* Small nitpick and some warnings
---------
Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
* Generalizing filesystem jobs to accept multiple files at once
* Some small changes and fixing front
* Enable rename route to replicate Finder's behavior
* Assign tag to multiple objects at once
* Remove from recents accepting multiple files
* Fixing some warnings
* Adding multiple files feature to Open and Open With
* Conditional stuff for macos
* Generating commands.ts and other minor warnings
* Rust fmt
* TS typecheck
* Rust format and TS typecheck
* Requested changes
* Requested changes
---------
Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
* support for late key manager setup and remove password from onboarding
* mobile
* delete master-password.tsx
* remove secure temp keystore
* remove more km_config artifacts
* fix builds
* Fix most of the UI, there still some minor bugs
* Workaround portalled closing Popovers
* Restore search in core/src/api/mod.rs
* Fix Password eye icon closing Popover on click
* Fix keymanager setup submit button spinner not centered
- Fix keymanager setup missing logic for disabling form
- Adjust keymanager tab roundness to match app style
* Restore .gitignore
* Restore .prettierignore
* fixerino tests
---------
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
* Introducing sub path indexing for IndexerJob
* Introducing shallow indexer job
* Some feature flags to avoid warnings at pnpm prep
* Many fixes at shallow indexer job
* Changing how we implement the bookkeeping for file_path ids
Now we're account for independent locations, and also integrating
better with the LibraryContext, instead of using a static global
* Making indexer job reentrant
* Introducing shallow file identifier job
And a bunch of minor refactors
* Rust fmt
* Removing light scan from getExplorerData query
Light scan is a mutation, so we can call it on useEffect function from
the Explorer component in the frontend, when a location_id or the explorer
path changes
* Handling job early finish on init for identifier
* Only invalidate query if we have orphan paths
* Introducing ShalowThumbnailerJob
* Clippy warnings about `into_iter()`
* Naming scheme for Prisma's selects and includes
* Invalidating getExplorerData at thumbnailer
* Small mistakes lol
* Some nitpicks with pnpm prep
* Rust fmt
* Changing indexer's walk log to `trace!`
* Not deleting all file_paths on location fullRescan
* TS typecheck
* Removing `file_path` selection just with id
* Complying with a pedantic Clippy
* Some task runners for vscode
* Solving duplicated directories events
When creating a directory through MacOS's Finder, for some reason
FSEvents receives 2 Create Folder events that we have to handle
* Damn rustfmt
* fix `spacedrive://` custom protocol on Windows
(hopefully)
* custom protocol using `http::Response` + fix broken web
* import patches before App on web
* use `http::Request` for input to `handle_custom_uri`
* break into dedicated file + error handling
* serving files via custom protocol
* cargo fmt because vscode did cringe
* lru cache to reduce video chunk request time
* add helper to JS
* clippy be like
* remove duplicate Open buttons in context menu
* fix Linux 🙏
* no shot
* fix Windows custom URI passing (hopefully)
* better fix for custom uri on Linux
* upgrade Tauri for feature
* switch url replacement order
* prevent React dev tools script being added in prod to desktop
* remove React devtools from html
* upgrade Tauri; required upgrading rspc, Axum, PCR
* pass typecheck + less cringe bigint
* clippy is love, clippy is life
* Typecheck plz
* fix bigint to number conversion
* use httpz + localhost server for Linux
* clippy be right
* Remove console.log
* [wip] proper auth
* fix Linux sidebar padding
* Secure Axum server with random
* Extracting app setup specific to linux to a different file
* remove outdated comment
* Some tweaks on cursom_uri.rs
* file_path_with_location doesn't need to be a named include
* fix typo
* factually wrong comment
* Change `unwrap` to `expect`
* bruh
---------
Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
* begin better onboarding
* added input and altered text
* better router & text + database icon
Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
* work on privacy screen + radio buttons
* fix video extension bug and alter screens
* add pending schema and location manager helper
* functional onboarding
* added secure temp store and started creating library loading screen
* fix secure temp keystore + api
* better onboarding
* added location settings and some overview concept, all WIP
* fix switch
* prep
* fix location router
* added backend settings
* attempted to fix form
* begin indexer rules editor, plus tweaks
* indexer rules coming soon
* fix onboarding img size
* cleanup
* clone is needed here, but clippy no like
* sike
* whole bunch of fixes
* clippy + ts
* Removing some TODOs from api/libraries.rs and fixing db size calculation
* moved object kind to client, added half functionality for appearance settings
* fix RadioGroup helper
* fix type issues
* cargo fmt
* fix creating library error handling + invalidate location list on update
* forgot to switch back to onError
* Invalidating getStatistics query on library creation and introducing the concept of waiting for a job on FileCopierJob
* F* cargo fmt
* fix RadioGroup interactivity
* wipe all migrations
* put back COLLATE NOCASE on extension columns
* update core.ts
* remove unused device component
* fix typeerror in mobile
---------
Co-authored-by: maxichrome <maxichrome@users.noreply.github.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
* add `queue_check`, `mount_check` and `unlock_check`
* make `unlock_check` marginally more readable
* clippy
* clean up & rename functions
* remove unlock check on `keys.list`
* use american spelling of `favorite`
* use `then_some` (thanks clippy)
* more american spelling
* update crypto MSRV
* rename `keychain` to `keyring`
* make a start on the keymanager unlock refactor/keychain integration
* update routes
* update bindings
* add const identifiers
* add UI/front-end support for unlocking KM with OS keychains
* remove SK from lib creation dialog
* update query name
* add keyring functions
* attempt to update `change_master_password()` to use the keychain
* cleanup, fix master password change ui, better secret key in keyring detection
* cleanup TS a little
* add route for getting secret key from keyring
* update bindings
* update var names + show secret key in keys settings
* add `react-qr-code` and option to view the secret key (if it's in the OS keyring)
* allow copying SK to clipboard
* add `key_type` so we're not reliant on specific UUIDs for root/verification key handling
* clippy
* fix mobile typecheck
* fix typecheck, fix typo and tweak balloon hash parameters
* minor cleanup + typo fix
* use newtype structs
* WIP type refactoring (major readability boost!)
* update `use`
* add tokio `sync` feature
* too many structs? idk
* more cleanup
* add `generate` and `Nonce`
* `Nonce` and `Key` typesafety (beautiful)
* clippy + cleanup
* update code & examples
* fix bug & remove `ProtectedVec` as it looked out of place
* use `Key`
* add a query invalidation to make the UI extremely responsive
* ci pls work
* remove `keyringHasSk` route
* super barebones keychain start
* working+unfinished linux keychain
* add `copy` to `Identifier` and add `delete()`
* add generic errors + code cleanup
* cleanup code & add support for apple keychains
* remove `users` dep
* use uppercase UUID and tweak apple `account` variable
* revert uppercase change and clean up linux labels
* code cleanup & add a readme to the crypto crate
* remove useless `map`
* correctly handle keyring not supported errors
* add `Send` to `Box<dyn Keyring>`
* remove useless `cfg`s
* only return `Ok()` if we support keychains (pls work CI)
* clippy
* Minor fixes
---------
Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
* add `get_queue` and `is_queued` to KM and router
* mark keys as being mounted/finished mounting, use a `DashSet` instead of vec
also includes temporary `dbg!()`s for manual valdiation that it works
* remove dbg and fix build (works great)
* mark key manager as unlocking correctly
* cleanup keys api
* add `KeyAlreadyQueued` error
* use react mounting queue
* fully prevent against multiple mount jobs of the same key with the rust queue
* show keymanager as unlocking in the UI
* handle query invalidation for `isKeyManagerUnlocking` with a closure (semi-temporary)
* revert changing text (doesn't work where the keymanager is unlocked from, only the other unlock screen)
* clippy
* add `useRef` for mounting queue within react
* pass `ref.current` to `Key` instead
* add wip storedkey versioning
* storedkey versioning! (not pretty, but it never will be)
* add version to `StoredKey` and re-gen migrations to handle serde
* use `serde` for interacting with the DB + handle errors
* fix secret key showing as `Some()` when it's empty
* rename function
* use `uuid` feature of `rspc` so clippy works with `--all-features`
* remove dead error
* update clippy rules
* more idiomatic + cleaner hashing, that handles secrets better
* remove dead code + minor refactor of the KM
* cleanup`KeyManager::get_key()`
* cleanup keys router
* variable inlining, idiomatic rust, etc
* clean up encrypt/decrypt jobs
* embed preview media into encrypted files
* even more idiomatic rust
* fix previous file extension code (no amount of idiomatic rust can fix bad patterns)
* hopefully make large serialization slightly more efficient
* remove dead code + old tests
* inline key derivation (not so sure about this one)
* use `then` instead of `if`
* stop using expensive clones
* add `Protected::into_inner()` to prevent `.expose().clone()` patterns
* use `tokio::fs::File` where possible
* add and use `JobError::MissingData`
* use `|| null` for the secret key
* use `block_in_place` for large r/w sync tasks
* add hash with secret functions
* refactor KM to make the secret key both optional, and user-settable
* update library create function and create library dialog
* fix annoying MPC typo
* fix ordering of key manager creation and seeding
* overhaul master password changing UI to support the potentially empty secret key
* patch keymanager unlocking for potentially empty SK
* make create library dialog text more fitting
* remove dead code
* add autogeneration, view/hide, copy to clipboard icons
* cleanup older code
* make onboarding screen look a tad better
* minor UI tweaks
* patch backup restoration UI
* add pw generation to MP change dialog
* make sure passwords are same before submitting lib creation
* UI fixes and allow setting algorithm/hashing algorithm for master password
* make sure KM is re-seeded on SD load and prevent loading duplicate keys into the keystore
* remove old comment
* small organization
* update encryptfiledialog
* remove dead KM code
* cleanup some TS
* move `PasswordMeter` to separate file
* refactor master password change dialog
* cleanup the TS and improve string -> hashingalgorithm conversions
* add `key` values to selectoption
* capitalise `BLAKE3`
* fix sorting for `getHashingAlgorithmString`
* remove onboarding route
* Revert "add `key` values to selectoption"
This reverts commit 10e6456f22.
* add `key` to `SelectOptionKeyList`
* fix using rspc errors directly
* Update packages/interface/src/components/onboarding/Onboarding.tsx
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
* use `navigator.clipboard.writeText`
* move `OnboardingConfig` to crypto crate
* move `zxcvbn` options outside of component
* further cleanup
* fix mobile typecheck
* fix keylist typo
* improve `SelectOptionKeyList` further
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
