* First draft on image labeling
* Fixing execution providers for other OSs
* Better error handling and shutdown
* Working with shallow media processor
* bruh
* Fix warnings
* Now hooked to media processor job
* Link desktop app with libonnxruntime to avoid TLS error during startup
* Be able to change models on runtime
Revert to use labels table instead of tags
* A bug on a model-less inference
* Show AI labels on Inspector
- Change yolo inference to use half precision
- Add labels api to core
* Remove LD_PRELOAD
* Fix race condition on model executor shutdown
* Don't load all images in memory moron
* Embeed yolo model in prod build
- Change yolo model path to new one relative to executable
* Disable volume watcher on linux, it was crashing the app
- Invalidate labels when they are updated
* Rust fmt
* Minor changes
* Gate onnxruntime linking to the ai-models feature
* Add build script to sd-server to handle onnxruntime linking workaround
* Move AI stuff to its own crate and normalize deps
* Rust fmt
* Don't regenerate labels unless asked to
* Now blazingly fast
* Bad merge
* Fix
* Fix
* Add backend logic to download extra yolo models
* Add models api route
- Add api call to get available model version
- Add api call to change the model version
* Improve new model download logic
- Add frontend to change image labeler model
* Fix new model downloader
* Fix model select width
* invalidate labels count after media_processor generates a new output
* Rename AI crate and first draft on download notifications
* fix types
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
* search options start
* small progress
* more
* bunch of stuff
* semi functioning filters
* cleanup setup api
* progress
* remove filters
* hooked up to query epic moment
* fix
* move db stuff to specific modules
* in/notIn for some fields
* generate ts
* big gains
* working filter options for locations, tags and kind
* working search query
* perfect fixed filters
* saved searches lol
* merge error
* saved searches via api
* better routing
* [ENG-1338] Fix fresh Spacedrive install failing to start due to attempting to query a nonexistent Library (#1649)
Fix Spacedrive failing to start due to attempting to query a nonexistent Library
- Rename useShoudRedirect to useRedirectToNewLocations
- Improve behaviour for the immedite redirection after adding a new location
* Show hidden files false by default (#1652)
bool
* fix remove filter in list
* tweaks
* fix nav buttons
* unify MediaData search handling
* cleanup saved search writing
* Add left top bar portals for tags and search + fixed media view on tags
* added search to filter dropdown
* render cycle improvements
* hotfix
* wip
* Refactor with Brendan, but this is a WIP and the search query no longer works
Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>
* progress
* fix location/$id page
* fix tags too
Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>
* 3rd refactor lol
epic style
* half-done with enum-ification of SearchFilterArgs
* broken fixed filters but working inNotIn filters
* search name + extension kinda working
* hidden filter
* fixed filters working??
* deferred search value
* extensions works
* filtered search items mostly working
* tweaks
* stacked approach working for non-search filters
* move to Explorer/Search
* actually use filterArgs in queries
things actually work properly now
* added new icons from Mint
* goof
* cleanup types, filters and mutation logic
* actually use search value
* remove overview from sidebar
* don't shrink LibrariesDropdown ga
* remove overview from sidebar and default to /network
---------
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>
* clippy -_-
* migrations
* alter the migration so it just renames the `dimensions` field (no db re-creation required)
* remove confusing semver addition for `libheif-sys`
* remove warning on the migration as it's just a rename
* add sort opts for resolution and date image was truly taken
* major serde ckeanup & add epoch_time and pixel_count
* rename, cleanup et optimise
* clippy
* ignore this mess
* bindings
* add explanation to schema
* comment out dt test
* better comment and WIP time
* cleanup rust
* failed timezone attempt
* remove image resolution as a sort by option
* update schema (and rename the dimensions table instead of dropping it)
* just show raw date
* add comments and update bindings
* fix migration hopefully
* fix broken migration
---------
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Jamie Pine <ijamespine@me.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
* basic layout
* lock
* add codec to image
* very messy wip
* rm that
* lock and toml
* working perfect exif extraction
* formatting
* migration and formatting
* mostly working
* fix
* set date created properly
* fix tsc
* working media data creation
* fix bad main merge? sorry brendan
* schema, migrations, bindings
* working exif data extraction
* why won't it work
* update migrations
* fix bad merge
* final cleanup
* cleanup migrations
* remove test (that was purely used for testing my code)
* working media data pulling, correct thumbnail orientation
* slightly optimise thumbnail rotation
* rename location to prevent specta clashes
* further improvements (location parsing is still broken)
* fix coordinate parsing i think
* rspc add some todos regarding final cleanup
* further thoughts
* major upgrades
* Some improved handling of errors and async calls
* accordion component
* heavily wip flash refactor
* fix builds all because of a (terrible) merge/move
* annoying missing newline
* i really hate exif
* remove dead code
* further flash progress :D
* docs(media-data): clarification
* minor cleanup
* cleanup and some async removal
* fix location parsing
* remove async (will do proper impl for async eventually) and major cleanup
* big W
* clippy and `FlashMode::Unknown` instead of `Invalid`
* add `NIKON` gps ref support
* comments about gps refs
* commit the submodule -_-
* major cleanup & composite image support
* remove old test image
* major cleanup and easier to use API
* remove old consts
* move `ExifReader` to dedicated module
* Media Data Extractor job and shallow job
* Extracting media data on watcher
* report no exif data on file gracefully
* cleanup errors and doctests
* Merging thumbnailer and media data extractor
* Job name and some strings in UI
* remove reliance on `sd-prisma` within the media data crate
* rename query to be more specific
* custom serializer for `MediaTime`
* tweak to format utc -> naive correctly
* generate migrations
* comment out duration in mobile
* delete test-assets folder
* all optional fields
* fix migrations again
* make requested name changes
* make further requested changes
* remove erroneous files from another wip branch
* updates procedures
* use strings where appropriate
* regen pnpm-lock
* add base layouts for video and audio media data
* use appropriate data types in schema and add audio media data framework
* make requested changes
* general cleanup and renaming of enum
* cleanup media data api
* rename media metadata type
* finishing touches
---------
Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
* less stupid name
* yeet
* awaiting futures is kinda important lol
* no-async
* more proto stuff
* cleanup
* move it round
* None of my homies like broadcast
* minor
* do the shuffle
* restore after force push
* reusing `sysinfo::System` as intended
* fix lol
* remove `node_id` from `Volume`
* Remove `node_local_id` from `Library`
* Remove `Job` to `Node` relation
* feature flags be like
* press save 4head
* remove `Location` -> `Node` relation
* `volume.rs` to `volume/mod.rs`
* yes
* add `Instance` model and deprecate `Node` model
* pairing is better now
* Pretty code
* thinking in code
* wip
* What if Observables but in Rust?!
* Observables aren't it + `useP2PEvents` hook
* more around some jsx
* Trade offer: bad code for working pairing?
* Unit test pairing protocol
* fix everything up
* corrections
* Have you got a moment for our lord and saviour Clippy
* tsc --fixMyCode
* Prisma being wacky
* Config files being f'ed up
* broken config after migrating
* Zed being Zed
* Argh
* cliipzy
* rewind
* Fix truncate logic
* wip: instances in peer metadata
* Rethink instance ids
* fix
* whoops
---------
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
* [ENG-779] Finalize UI
This is one branch with a variety of UI changes
add tag select mode bar without functionality
fix group job status
add notice icon with info to stat icons
add WIP notice to media view
add modal before add location with greyed out clouds
remove disappearing add location button
add WIP spacedrop page
bring back limited key manager UI
add options bar on search page without functionality
Add greyed out encrypt library button or setting
See more button on locations
Show locations on node screen
Fix overview category left padding
* key manager placeholder
* stat info
* nodes screen
* location click yay
* fix size in bytes
Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>
* small ui improvements
* sh*tty see more button
* last touches
* fix merge boo boo
* Fix mobile
- Move `getItemObject`, `getItemFilePath`, `getItemLocation`, `getExplorerItemData` to @sd/core to allow mobile to use them
* Formatting
* Normalize displayed file size between all screens
- Replace every use of internal formatBytes with byte-size dep
---------
Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
* theoretically
* just please don't fail man
* bruh
* go away serde
* fix migrations
* squash migrations
* plz for the love of all that is good work
* disable some node fields
* fix core types
* remove unused stuff
* regenerate migrations
* Fix error boundry
* undo stuff
* migration
---------
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
Co-authored-by: James Pine <ijamespine@me.com>
* refactor job manager ui
* huge improvements to job ui api
* improve indexer errors
* minor improvements
* make icon bigger + improve styling
* Update useJobInfo.tsx
better
* improve job status reporting
* fix job indexer backend for ui responsiveness
* attempt at debugging job.getRunning slow invalidation during indexer's walk phase
* remove progress debounce, invalidate has its own throttle layer
* hotfix ghost jobs
* basic pause/resume
* pause functionality immaculate
* pause resume working for first job in group, testable on indexer phase two
* WIP
- refactored job manager
- added better job api
* fix merge issues
* add throttle to job update events and correct index
* improve front end job data handling
* move subscription to job
* wip active job indicator
* minor tweak
* Isolated subscriptions for job events + cleanup
Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
* mutable ctx
* plz let me build rspc typesafe errors Jamie
* fix merge
* working job reporting
* fix thumbnail text
* faster tick speed
* fix error
---------
Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
* Introducing many rules on a single indexer rule
* Rust fmt
* Some doc fixes and small stuff
* Small warnings
* Making front typecheks
* Migration fix
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
* fix extension case sensitivity
* job time estimation wip
* update schema
* use `DateTime` to handle estimated completion time
* use a date with `dayjs`
* remove old migrations
* update migrations
* remove dead code
* Quick tweaks
* unused import
---------
Co-authored-by: brxken128 <77554505+brxken128@users.noreply.github.com>
Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
* WIP materialized_path abstraction revamp
* Optimizing indexer rules loading
* Using a better serialize impl for indexer rules
* New interruptable and faster Walker
* WIP new indexer
* WIP first success compiling after breaking the world
* Fixing some warnings
* Handling some lifetime issues in the walker
* New job completed with errors feature
* Introducing completed with errors to indexer
Removing IOError variant from JobError and using FileIOError instead
* Rust fmt
* Adding missing job status
* Better ergonomics to IsolatedFilePathData
Conversions from db's file_path kinds
Keeping original's relative path data to better conversion to OS's path
First unit tests
* Testing and fixing parent method
* Some error handling
* Rust fmt
* Some small fixes
* Fixing indexer rules decoding
* Bunch of small fixes
* Rust fmt
* Fixing indexer rules
* Updating frontend to new materialized_path format
* Trying to fix windows CI
---------
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
* give file paths pub_id and use it primarily
* fix object indexing
* file path location + sync
* remove future plan
* use id instead of pub_id for more indexer queries
* use chain_optional_iter
---------
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
* Implement multiple Glob matches in the same indexer rule
- Replace matching logic to use GlobSet instead of simple Glob
- Add `No OS protected` indexer rule to avoid indexing OS protected files
- Enable `No OS protected` indexer rule by default
* Rust fmt
* Reduce `No OS protected` rule Glob list by using OR matches
* Add globs for android and ios files in `No OS protected` rule
* Add some more unix special path to be ignored
* Add a new property to IndexerRule to enable it by default when adding a new location:
- Modify the Prisma schema to add the default property to the IndexerRule model.
- Adjust the IndexerRule struct's create and save logic to consider the new property.
- Adjust AddLocationDialog to properly load the default indexer rules from the backend.
- Minorly improve IndexerRuleEditor to avoid adding duplicate entries to its assigned form field.
- Add editorconfig entries for SQL and Prisma types.
* rust fmt
* Add Windows users special folders and files to `No OS protected` rule
* Construct `No OS protected` globs from string vec
- Don't repeat windows globs that require both a root and C: version
* Apply review feedback and add error handling for `IndexerRuleEditor`
- Revert manual changes made to init migration
- Create external migration that adds a `default` prop in `IndexerRule` model
- Remove redundant `useMemo`
- Replace destructuring of `useQuery`
- Provide feedback to user when a error occurs while querying indexer rules
* useMemo only for `indexerRulesIds` and not the whole `defaultValues` object
- Remove logic to accpet unix root path for windows indexer `No OS protected` rule
* Complying with a pedantic Clippy
* Solving duplicated directories events
When creating a directory through MacOS's Finder, for some reason
FSEvents receives 2 Create Folder events that we have to handle
* Handling moving to trash bin and restoring on Mac
Still missing the feature to restore a directory and its children
* Now handling creation of empty files on MacOS
* Enabling restore of directories and its children
* Now working: moving a directory to another
inside the same location
* Now Indexer also remove file_paths not on fs
* Enabling multiple file moves on location watcher
* Fix Windows conditional compilation issues
* Fixing cas_id generation and bumping some deps
* Many Windows specific improvs and some refactors
* Rust fmt
* Using conditional compilation on extract inode function
* Linux fixes and some MaterializedPath improvements
* Rust fmt again
* Introducing tick behavior on location watchers
* Making LastFilePathIdManager more atomic
* Some vscode launch configs for lldb debugger
* Simplifying some lifetimes
* Making all watchers more consistent
