* First draft on task system usage, still missing job system
* Scan location roughly working, a ton of stuff to fix yet
* Updating some deps due to crashes and bugs
* Exposing non critical errors to frontend
* Getting active job reports from job system
* Using boxed opaque type to avoid a downcast issue with generics
* Task system issues discovered on race conditions
* Enable debug
* Fix job report in the job manager
* Fix race condition on steal tasks
* Fixed race condition on task suspend
* Some fixes on job progress reporting and save
* Fixed many race conditions and a hard deadlock
Also some progress report polishing
* Ignore .ts and .mts video files for now
* Some better logs
* bruh
* Internal deadlocks and excess of communication in the task system
- Also better logs
* Bunch of fixes and optimizations
* WIP at fixing file identifier
* Fixed file identifier job
- still need to work on its progress report frontend
* A bunch of polishing
* Fixed some bugs and did more polishing
* Cleanup
* Bridging old and new job systems
* A ton of fixes
* A bunch of bugs related to shutdown and resume
* Indexer and watcher bugs
* Log normalizing
* Fixing CI
* Change error! to warn! on non critical errors log
* Fix redirect to new location
* Type annotation
* Bogus merge resolution on cargo lock
* initial ffprobe commit
* Working slim down version ffprobe
* Auto format ffprobe and deps source
* Remove show_pixel_formats logic
- Fix do_bitexact incorrect check in main after last changes
- Fix some clangd warning
* Remove show_* and print_format options and their respective logic
* Rework ffprobe into simple_ffprobe
- Simplify ffprobe logic into a simple program that gather and print a media file metadata
* Reduce the amount of ffmpeg log messages while generating thumbnails
* Fix completly wrong comments
* mend
* Start modeling ffmpeg extracted metadata on schema
- Start porting ffprobe code to rust
- Rename some references to media_data to exif_data
* Finish modeling media info data
- Add MediaProgram, MediaStream, MediaCodec, MediaVideoProps, MediaAudioProps, MediaSubtitleProps to Schema
- Fix simple_ffproble to use its custom print_codec, instead of ffmpeg's impl
* Add relation between MediaInfo and FilePath
- Remove shared properties from MediaInfo and related structs
- Implement Iterator for FFmpegDict
* Fix and update schema
* Data models and start populating MediaInfo in rust
* Finish populating media info, chapters and program
* Improve FFmpegFormatContext data raw pointer access
- Implement stream data gathering
* Impl FFmpegCodecContext, retrieve codec information
- Improve some unsafe pointer uses
- Impl from FFmpegFormatContext to MediaInfo conversion
* Fix FFmpegDict Drop
* Fix some crago warnings
* Impl retrieval of video props
- Fix C char* to Rust String convertion
* Impl retrieval of audio and subtitle props
- Fill props for MediaCodec
* Remove simple_ffprobe now that the Rust impl is done
* Fix schema to match actually retrieved media info
- Fix import some FFmpeg constants instead of directly using values
* Rework movie_decoder
- Re-implement create_scale_string and add support anamorphic video
- Improve C pointer access for FFmpegFormatContext and FFmpegCodecContext
- Use newer FFmpeg abstractions in movie_decoder
* Fix incorrect props when initializing MovieDecoder
* Remove unecessary lifetimes
* Added more native wrappers for some FFmpeg native objects used in movie_decoder
* Remove FFmpegPacket
- Some more improvements to movie_decoder
* WIP
* Some small fixes
* More fixes
Rename movie_decoder to frame_decoder
Remove more references to film_strips
* fmt
* Fix duplicate migration for job error changes
* fix rebase
* Solving segfaults, fuck C lang
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Update rust to version 1.77
- Pin rust version with rust-toolchain.toml
- Change from dtolnay/rust-toolchain to IronCoreLabs/rust-toolchain for rust-toolchain support
- Remove unused function and imports
- Replace most CString uses with new c literal string
* More segfault solving and other minor fixes
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Fix ffmpeg rotation filter breaking portrait video thumbnails #2150
- Plus some other misc fixes
* Auto format
* Retrieve video/audio metadata on frontend
* Auto format
* First draft on ffmpeg data save on db
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Fix some incorrect changes to prisma schema
* Some fixes for the FFmpegData schema
- Expand logic to save FFmpegData to db
* A ton of things
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Integrating ffmpeg media data in jobs and API
* Rspc can't BigInt
* 🙄
* Add initial ffmpeg metadata entries to Inspector
- Fix ephemeral metadata api to match the files metadata api call
* Fix Inspector not showing ffmpeg metadata
* Add bitrate, start time and chapters video metadata to Inspector
- Fix backend BigInt conversion incorrectly using i32 instead of u32
- Change FFmpegFormatContext/FFmpegMetaData bit_rate to i64
- Rename byteSize to humanizeSize
- Expand humanizeSize logic to allow handling bits and Binary units
- Move capitalize to @sd/client utils
* Solving some issues
* Fix ffmpeg probe getting incorrect stream id and breaking database unique constraint
- Fix humanizeSize breaking when receiving floating numbers
- Fix incorrect equality in StatCard
- Fix unhandled error in Dialog when trying to remove an unknown dialog
* fmt
* small improvements
- Remove some unecessary recursion_limit directive
- Remove unused app_image releated functions
- Fix metadata query enabled flag
* Add migration for ffmpeg media data
* Fix cypress test
* Requested changes
* Implement feedback
- Update locale keys for all languages
- Add pnpm command to update all language keys
* Fix thumb reactivity in non indexed locations
---------
Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Most of it
* Cleanup menu
* fixes
* Fixed linux compilation, still not working though, as the app panic on start
* Add dialog permissions
* fix search keybind
* Fix linux
∙ - Disable linux-ipc-protocol feature, as that is causing panics due to trying to access some WebKit internal structures outside the main thread
* Update to Ubuntu 22.04, new tauri doesn't support ubuntu 20.04
> - Add note to download page about deb's distro support
* Remove gen/schemas
* Update linux dependencies in setup.sh
* Fix linux deps again
- Enable rt-tokio-crypto-rust feature in secret-service so clippy stops complaining
* Clippy + auto fmt
* Fix cache-factory
- Create a devtools feature for desktop app, which is enabled by default on dev builds
* Fix minor error in patchTauri.mjs
* Fix some envvars that have been renamed on tauri v2
* Dont change the secrets
* Add missing linux dependency
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
* Bunch of new warns due to a stronger clippy config
* Moving old processing stuff to a new namespace
Also fixing a bunch of typos through the entire codebase.
* Rustfmt
* First draft on image labeling
* Fixing execution providers for other OSs
* Better error handling and shutdown
* Working with shallow media processor
* bruh
* Fix warnings
* Now hooked to media processor job
* Link desktop app with libonnxruntime to avoid TLS error during startup
* Be able to change models on runtime
Revert to use labels table instead of tags
* A bug on a model-less inference
* Show AI labels on Inspector
- Change yolo inference to use half precision
- Add labels api to core
* Remove LD_PRELOAD
* Fix race condition on model executor shutdown
* Don't load all images in memory moron
* Embeed yolo model in prod build
- Change yolo model path to new one relative to executable
* Disable volume watcher on linux, it was crashing the app
- Invalidate labels when they are updated
* Rust fmt
* Minor changes
* Gate onnxruntime linking to the ai-models feature
* Add build script to sd-server to handle onnxruntime linking workaround
* Move AI stuff to its own crate and normalize deps
* Rust fmt
* Don't regenerate labels unless asked to
* Now blazingly fast
* Bad merge
* Fix
* Fix
* Add backend logic to download extra yolo models
* Add models api route
- Add api call to get available model version
- Add api call to change the model version
* Improve new model download logic
- Add frontend to change image labeler model
* Fix new model downloader
* Fix model select width
* invalidate labels count after media_processor generates a new output
* Rename AI crate and first draft on download notifications
* fix types
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
* basic FDA start
* FDA checks and prompt
* lockfile
* misc clippy
* Some small warnings
---------
Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
* clippy -_-
* migrations
* alter the migration so it just renames the `dimensions` field (no db re-creation required)
* remove confusing semver addition for `libheif-sys`
* remove warning on the migration as it's just a rename
* add sort opts for resolution and date image was truly taken
* major serde ckeanup & add epoch_time and pixel_count
* rename, cleanup et optimise
* clippy
* ignore this mess
* bindings
* add explanation to schema
* comment out dt test
* better comment and WIP time
* cleanup rust
* failed timezone attempt
* remove image resolution as a sort by option
* update schema (and rename the dimensions table instead of dropping it)
* just show raw date
* add comments and update bindings
* fix migration hopefully
* fix broken migration
---------
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Jamie Pine <ijamespine@me.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
* derive `Type` on `*Extension`
* update `image`, `webp` and add `rspc::Error` support to the `sd-images` crate
* rename function for verbosity
* cleanup clippy allows and use scaling fn for svg conversions
* remove dead comment
* specify `generic` image extensions and add fn to enumerate all compatible extensions
* re-exports and inline hot provided method
* another inline on a provided method
* `formatter` -> `handler` and return an explicit error if the extension isn't supported, and add `convert_image()`
* `get_ext()` inlined helper fn
* add mutation for conversion and query for getting all supported extensions
* fix `svg` scaling and generation
* update bindings
* rm old const
* temporary test for all exts
* verbosity and no manual drop
* remove dbg test
* big enum and remove `cur` extension as `image` doesn't support it
* add huge extension enum
* hopefully get path from file path id and location id
* main merge fix
* chore: update toml and lockfile
* security: update webp
* comment entire crypto crate for build times
* add bincode, huge enum and custom ser/de
* Revert "comment entire crypto crate for build times"
This reverts commit 458addaad6.
* lockfile
* theoretically working ser/de
* add svg scaling
* fix pdf matching bug and use options
* move image scaling fn
* major consts overhaul with good ser/de
* disable heif checking for now due to new trait api
* make the generic handler truly generic
* fix pdf types
* fix pdf scaling so that it's 120dpi A4 paper size by default - height/aspect res are maintained
* heavy cleanup
* note to myself
* bindings & update core to use the scaling fn from the images crate (reducing code dupes)
* move vulnerable `aovec` to be a dev dep so it stays out of release builds (it's only used in tests)
* remoev exif crate from the core as it's handled in a sep. crate
* silence a clippy warning and update lockfile
* lower the maximum file size a slight amount
* bindings and minor dep bumps
* Some improvements
---------
Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
* bring in rand deps
* SUPER crude pluscode attempt
* working pluscodes!
* some cleanup
* `encodeURIComponent` for URL escaping
---------
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
* impl `Display` and a custom serde serializer
* remove the + as rust already shortens coords to 8SF
* remove manual serde impl and string conversion
* add `formatOrientation` to the `MediaData` part of the inspector
* make requested changes
---------
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
* sd-images crate which will support raw/dng, bmp, etc
* more work on the image formatter
* re-work `sd-images`, add svg support, r/g/b and r/g/b/a HEIF image support (will all be async again soon)
* remove `ImageFormatter`, add note about r/g/b/(a) heif impl
* implement the image formatter
* rename the conversion trait and minor cleanups
* isolate heif feature and major cleanup
* very untested raw support
* change fn name to `from_path` (a lot more idiomatic)
* clean up orientation fixing
* heif is no longer forbidden (linux has good heif)
also all extensions are correctly matched in lowercase
* fix builds, ext matching, feature gating
* attempt to fix svg handling?
* raw attempt, quite a few errors
* add comment
* new (untested) attempt
* remove `raw` stuff for now
* replace `sd-svg` with a `ToImage` `SvgHandler` impl
* add some simple math to appropriately scale thumbnails (and bmp/ico support)
* add comments regarding how the math works for image thumbs
* rename the trait to `ImageHandler`
* basic layout
* lock
* add codec to image
* very messy wip
* rm that
* lock and toml
* working perfect exif extraction
* formatting
* migration and formatting
* mostly working
* fix
* set date created properly
* fix tsc
* working media data creation
* fix bad main merge? sorry brendan
* schema, migrations, bindings
* working exif data extraction
* why won't it work
* update migrations
* fix bad merge
* final cleanup
* cleanup migrations
* remove test (that was purely used for testing my code)
* working media data pulling, correct thumbnail orientation
* slightly optimise thumbnail rotation
* rename location to prevent specta clashes
* further improvements (location parsing is still broken)
* fix coordinate parsing i think
* rspc add some todos regarding final cleanup
* further thoughts
* major upgrades
* Some improved handling of errors and async calls
* accordion component
* heavily wip flash refactor
* fix builds all because of a (terrible) merge/move
* annoying missing newline
* i really hate exif
* remove dead code
* further flash progress :D
* docs(media-data): clarification
* minor cleanup
* cleanup and some async removal
* fix location parsing
* remove async (will do proper impl for async eventually) and major cleanup
* big W
* clippy and `FlashMode::Unknown` instead of `Invalid`
* add `NIKON` gps ref support
* comments about gps refs
* commit the submodule -_-
* major cleanup & composite image support
* remove old test image
* major cleanup and easier to use API
* remove old consts
* move `ExifReader` to dedicated module
* Media Data Extractor job and shallow job
* Extracting media data on watcher
* report no exif data on file gracefully
* cleanup errors and doctests
* Merging thumbnailer and media data extractor
* Job name and some strings in UI
* remove reliance on `sd-prisma` within the media data crate
* rename query to be more specific
* custom serializer for `MediaTime`
* tweak to format utc -> naive correctly
* generate migrations
* comment out duration in mobile
* delete test-assets folder
* all optional fields
* fix migrations again
* make requested name changes
* make further requested changes
* remove erroneous files from another wip branch
* updates procedures
* use strings where appropriate
* regen pnpm-lock
* add base layouts for video and audio media data
* use appropriate data types in schema and add audio media data framework
* make requested changes
* general cleanup and renaming of enum
* cleanup media data api
* rename media metadata type
* finishing touches
---------
Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>