spacedrive/crates/media-metadata/Cargo.toml

31 lines
780 B
TOML
Raw Permalink Normal View History

[ENG-288, ENG-601] Media data (image) extraction, thumbnail orientation fix (#1099) * basic layout * lock * add codec to image * very messy wip * rm that * lock and toml * working perfect exif extraction * formatting * migration and formatting * mostly working * fix * set date created properly * fix tsc * working media data creation * fix bad main merge? sorry brendan * schema, migrations, bindings * working exif data extraction * why won't it work * update migrations * fix bad merge * final cleanup * cleanup migrations * remove test (that was purely used for testing my code) * working media data pulling, correct thumbnail orientation * slightly optimise thumbnail rotation * rename location to prevent specta clashes * further improvements (location parsing is still broken) * fix coordinate parsing i think * rspc add some todos regarding final cleanup * further thoughts * major upgrades * Some improved handling of errors and async calls * accordion component * heavily wip flash refactor * fix builds all because of a (terrible) merge/move * annoying missing newline * i really hate exif * remove dead code * further flash progress :D * docs(media-data): clarification * minor cleanup * cleanup and some async removal * fix location parsing * remove async (will do proper impl for async eventually) and major cleanup * big W * clippy and `FlashMode::Unknown` instead of `Invalid` * add `NIKON` gps ref support * comments about gps refs * commit the submodule -_- * major cleanup & composite image support * remove old test image * major cleanup and easier to use API * remove old consts * move `ExifReader` to dedicated module * Media Data Extractor job and shallow job * Extracting media data on watcher * report no exif data on file gracefully * cleanup errors and doctests * Merging thumbnailer and media data extractor * Job name and some strings in UI * remove reliance on `sd-prisma` within the media data crate * rename query to be more specific * custom serializer for `MediaTime` * tweak to format utc -> naive correctly * generate migrations * comment out duration in mobile * delete test-assets folder * all optional fields * fix migrations again * make requested name changes * make further requested changes * remove erroneous files from another wip branch * updates procedures * use strings where appropriate * regen pnpm-lock * add base layouts for video and audio media data * use appropriate data types in schema and add audio media data framework * make requested changes * general cleanup and renaming of enum * cleanup media data api * rename media metadata type * finishing touches --------- Co-authored-by: Ericson Soares <ericson.ds999@gmail.com> Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com> Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com> Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me> Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com> Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-08-29 17:02:55 +00:00
[package]
name = "sd-media-metadata"
version = "0.1.0"
Media metadata extraction & Thumbnailer rework (#2285) * initial ffprobe commit * Working slim down version ffprobe * Auto format ffprobe and deps source * Remove show_pixel_formats logic - Fix do_bitexact incorrect check in main after last changes - Fix some clangd warning * Remove show_* and print_format options and their respective logic * Rework ffprobe into simple_ffprobe - Simplify ffprobe logic into a simple program that gather and print a media file metadata * Reduce the amount of ffmpeg log messages while generating thumbnails * Fix completly wrong comments * mend * Start modeling ffmpeg extracted metadata on schema - Start porting ffprobe code to rust - Rename some references to media_data to exif_data * Finish modeling media info data - Add MediaProgram, MediaStream, MediaCodec, MediaVideoProps, MediaAudioProps, MediaSubtitleProps to Schema - Fix simple_ffproble to use its custom print_codec, instead of ffmpeg's impl * Add relation between MediaInfo and FilePath - Remove shared properties from MediaInfo and related structs - Implement Iterator for FFmpegDict * Fix and update schema * Data models and start populating MediaInfo in rust * Finish populating media info, chapters and program * Improve FFmpegFormatContext data raw pointer access - Implement stream data gathering * Impl FFmpegCodecContext, retrieve codec information - Improve some unsafe pointer uses - Impl from FFmpegFormatContext to MediaInfo conversion * Fix FFmpegDict Drop * Fix some crago warnings * Impl retrieval of video props - Fix C char* to Rust String convertion * Impl retrieval of audio and subtitle props - Fill props for MediaCodec * Remove simple_ffprobe now that the Rust impl is done * Fix schema to match actually retrieved media info - Fix import some FFmpeg constants instead of directly using values * Rework movie_decoder - Re-implement create_scale_string and add support anamorphic video - Improve C pointer access for FFmpegFormatContext and FFmpegCodecContext - Use newer FFmpeg abstractions in movie_decoder * Fix incorrect props when initializing MovieDecoder * Remove unecessary lifetimes * Added more native wrappers for some FFmpeg native objects used in movie_decoder * Remove FFmpegPacket - Some more improvements to movie_decoder * WIP * Some small fixes * More fixes Rename movie_decoder to frame_decoder Remove more references to film_strips * fmt * Fix duplicate migration for job error changes * fix rebase * Solving segfaults, fuck C lang Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Update rust to version 1.77 - Pin rust version with rust-toolchain.toml - Change from dtolnay/rust-toolchain to IronCoreLabs/rust-toolchain for rust-toolchain support - Remove unused function and imports - Replace most CString uses with new c literal string * More segfault solving and other minor fixes Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Fix ffmpeg rotation filter breaking portrait video thumbnails #2150 - Plus some other misc fixes * Auto format * Retrieve video/audio metadata on frontend * Auto format * First draft on ffmpeg data save on db Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Fix some incorrect changes to prisma schema * Some fixes for the FFmpegData schema - Expand logic to save FFmpegData to db * A ton of things Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Integrating ffmpeg media data in jobs and API * Rspc can't BigInt * 🙄 * Add initial ffmpeg metadata entries to Inspector - Fix ephemeral metadata api to match the files metadata api call * Fix Inspector not showing ffmpeg metadata * Add bitrate, start time and chapters video metadata to Inspector - Fix backend BigInt conversion incorrectly using i32 instead of u32 - Change FFmpegFormatContext/FFmpegMetaData bit_rate to i64 - Rename byteSize to humanizeSize - Expand humanizeSize logic to allow handling bits and Binary units - Move capitalize to @sd/client utils * Solving some issues * Fix ffmpeg probe getting incorrect stream id and breaking database unique constraint - Fix humanizeSize breaking when receiving floating numbers - Fix incorrect equality in StatCard - Fix unhandled error in Dialog when trying to remove an unknown dialog * fmt * small improvements - Remove some unecessary recursion_limit directive - Remove unused app_image releated functions - Fix metadata query enabled flag * Add migration for ffmpeg media data * Fix cypress test * Requested changes * Implement feedback - Update locale keys for all languages - Add pnpm command to update all language keys * Fix thumb reactivity in non indexed locations --------- Co-authored-by: Ericson Soares <ericson.ds999@gmail.com> Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
2024-05-09 02:20:28 +00:00
authors = [
"Jake Robinson <jake@spacedrive.com>",
"Vítor Vasconcellos <vitor@spacedrive.com>",
"Ericson Soares <ericson@spacedrive.com>",
]
[ENG-288, ENG-601] Media data (image) extraction, thumbnail orientation fix (#1099) * basic layout * lock * add codec to image * very messy wip * rm that * lock and toml * working perfect exif extraction * formatting * migration and formatting * mostly working * fix * set date created properly * fix tsc * working media data creation * fix bad main merge? sorry brendan * schema, migrations, bindings * working exif data extraction * why won't it work * update migrations * fix bad merge * final cleanup * cleanup migrations * remove test (that was purely used for testing my code) * working media data pulling, correct thumbnail orientation * slightly optimise thumbnail rotation * rename location to prevent specta clashes * further improvements (location parsing is still broken) * fix coordinate parsing i think * rspc add some todos regarding final cleanup * further thoughts * major upgrades * Some improved handling of errors and async calls * accordion component * heavily wip flash refactor * fix builds all because of a (terrible) merge/move * annoying missing newline * i really hate exif * remove dead code * further flash progress :D * docs(media-data): clarification * minor cleanup * cleanup and some async removal * fix location parsing * remove async (will do proper impl for async eventually) and major cleanup * big W * clippy and `FlashMode::Unknown` instead of `Invalid` * add `NIKON` gps ref support * comments about gps refs * commit the submodule -_- * major cleanup & composite image support * remove old test image * major cleanup and easier to use API * remove old consts * move `ExifReader` to dedicated module * Media Data Extractor job and shallow job * Extracting media data on watcher * report no exif data on file gracefully * cleanup errors and doctests * Merging thumbnailer and media data extractor * Job name and some strings in UI * remove reliance on `sd-prisma` within the media data crate * rename query to be more specific * custom serializer for `MediaTime` * tweak to format utc -> naive correctly * generate migrations * comment out duration in mobile * delete test-assets folder * all optional fields * fix migrations again * make requested name changes * make further requested changes * remove erroneous files from another wip branch * updates procedures * use strings where appropriate * regen pnpm-lock * add base layouts for video and audio media data * use appropriate data types in schema and add audio media data framework * make requested changes * general cleanup and renaming of enum * cleanup media data api * rename media metadata type * finishing touches --------- Co-authored-by: Ericson Soares <ericson.ds999@gmail.com> Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com> Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com> Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me> Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com> Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-08-29 17:02:55 +00:00
edition = "2021"
Media metadata extraction & Thumbnailer rework (#2285) * initial ffprobe commit * Working slim down version ffprobe * Auto format ffprobe and deps source * Remove show_pixel_formats logic - Fix do_bitexact incorrect check in main after last changes - Fix some clangd warning * Remove show_* and print_format options and their respective logic * Rework ffprobe into simple_ffprobe - Simplify ffprobe logic into a simple program that gather and print a media file metadata * Reduce the amount of ffmpeg log messages while generating thumbnails * Fix completly wrong comments * mend * Start modeling ffmpeg extracted metadata on schema - Start porting ffprobe code to rust - Rename some references to media_data to exif_data * Finish modeling media info data - Add MediaProgram, MediaStream, MediaCodec, MediaVideoProps, MediaAudioProps, MediaSubtitleProps to Schema - Fix simple_ffproble to use its custom print_codec, instead of ffmpeg's impl * Add relation between MediaInfo and FilePath - Remove shared properties from MediaInfo and related structs - Implement Iterator for FFmpegDict * Fix and update schema * Data models and start populating MediaInfo in rust * Finish populating media info, chapters and program * Improve FFmpegFormatContext data raw pointer access - Implement stream data gathering * Impl FFmpegCodecContext, retrieve codec information - Improve some unsafe pointer uses - Impl from FFmpegFormatContext to MediaInfo conversion * Fix FFmpegDict Drop * Fix some crago warnings * Impl retrieval of video props - Fix C char* to Rust String convertion * Impl retrieval of audio and subtitle props - Fill props for MediaCodec * Remove simple_ffprobe now that the Rust impl is done * Fix schema to match actually retrieved media info - Fix import some FFmpeg constants instead of directly using values * Rework movie_decoder - Re-implement create_scale_string and add support anamorphic video - Improve C pointer access for FFmpegFormatContext and FFmpegCodecContext - Use newer FFmpeg abstractions in movie_decoder * Fix incorrect props when initializing MovieDecoder * Remove unecessary lifetimes * Added more native wrappers for some FFmpeg native objects used in movie_decoder * Remove FFmpegPacket - Some more improvements to movie_decoder * WIP * Some small fixes * More fixes Rename movie_decoder to frame_decoder Remove more references to film_strips * fmt * Fix duplicate migration for job error changes * fix rebase * Solving segfaults, fuck C lang Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Update rust to version 1.77 - Pin rust version with rust-toolchain.toml - Change from dtolnay/rust-toolchain to IronCoreLabs/rust-toolchain for rust-toolchain support - Remove unused function and imports - Replace most CString uses with new c literal string * More segfault solving and other minor fixes Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Fix ffmpeg rotation filter breaking portrait video thumbnails #2150 - Plus some other misc fixes * Auto format * Retrieve video/audio metadata on frontend * Auto format * First draft on ffmpeg data save on db Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Fix some incorrect changes to prisma schema * Some fixes for the FFmpegData schema - Expand logic to save FFmpegData to db * A ton of things Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Integrating ffmpeg media data in jobs and API * Rspc can't BigInt * 🙄 * Add initial ffmpeg metadata entries to Inspector - Fix ephemeral metadata api to match the files metadata api call * Fix Inspector not showing ffmpeg metadata * Add bitrate, start time and chapters video metadata to Inspector - Fix backend BigInt conversion incorrectly using i32 instead of u32 - Change FFmpegFormatContext/FFmpegMetaData bit_rate to i64 - Rename byteSize to humanizeSize - Expand humanizeSize logic to allow handling bits and Binary units - Move capitalize to @sd/client utils * Solving some issues * Fix ffmpeg probe getting incorrect stream id and breaking database unique constraint - Fix humanizeSize breaking when receiving floating numbers - Fix incorrect equality in StatCard - Fix unhandled error in Dialog when trying to remove an unknown dialog * fmt * small improvements - Remove some unecessary recursion_limit directive - Remove unused app_image releated functions - Fix metadata query enabled flag * Add migration for ffmpeg media data * Fix cypress test * Requested changes * Implement feedback - Update locale keys for all languages - Add pnpm command to update all language keys * Fix thumb reactivity in non indexed locations --------- Co-authored-by: Ericson Soares <ericson.ds999@gmail.com> Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
2024-05-09 02:20:28 +00:00
[features]
ffmpeg = ["dep:sd-ffmpeg"]
[ENG-288, ENG-601] Media data (image) extraction, thumbnail orientation fix (#1099) * basic layout * lock * add codec to image * very messy wip * rm that * lock and toml * working perfect exif extraction * formatting * migration and formatting * mostly working * fix * set date created properly * fix tsc * working media data creation * fix bad main merge? sorry brendan * schema, migrations, bindings * working exif data extraction * why won't it work * update migrations * fix bad merge * final cleanup * cleanup migrations * remove test (that was purely used for testing my code) * working media data pulling, correct thumbnail orientation * slightly optimise thumbnail rotation * rename location to prevent specta clashes * further improvements (location parsing is still broken) * fix coordinate parsing i think * rspc add some todos regarding final cleanup * further thoughts * major upgrades * Some improved handling of errors and async calls * accordion component * heavily wip flash refactor * fix builds all because of a (terrible) merge/move * annoying missing newline * i really hate exif * remove dead code * further flash progress :D * docs(media-data): clarification * minor cleanup * cleanup and some async removal * fix location parsing * remove async (will do proper impl for async eventually) and major cleanup * big W * clippy and `FlashMode::Unknown` instead of `Invalid` * add `NIKON` gps ref support * comments about gps refs * commit the submodule -_- * major cleanup & composite image support * remove old test image * major cleanup and easier to use API * remove old consts * move `ExifReader` to dedicated module * Media Data Extractor job and shallow job * Extracting media data on watcher * report no exif data on file gracefully * cleanup errors and doctests * Merging thumbnailer and media data extractor * Job name and some strings in UI * remove reliance on `sd-prisma` within the media data crate * rename query to be more specific * custom serializer for `MediaTime` * tweak to format utc -> naive correctly * generate migrations * comment out duration in mobile * delete test-assets folder * all optional fields * fix migrations again * make requested name changes * make further requested changes * remove erroneous files from another wip branch * updates procedures * use strings where appropriate * regen pnpm-lock * add base layouts for video and audio media data * use appropriate data types in schema and add audio media data framework * make requested changes * general cleanup and renaming of enum * cleanup media data api * rename media metadata type * finishing touches --------- Co-authored-by: Ericson Soares <ericson.ds999@gmail.com> Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com> Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com> Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me> Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com> Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-08-29 17:02:55 +00:00
[dependencies]
# Spacedrive Sub-crates
Media metadata extraction & Thumbnailer rework (#2285) * initial ffprobe commit * Working slim down version ffprobe * Auto format ffprobe and deps source * Remove show_pixel_formats logic - Fix do_bitexact incorrect check in main after last changes - Fix some clangd warning * Remove show_* and print_format options and their respective logic * Rework ffprobe into simple_ffprobe - Simplify ffprobe logic into a simple program that gather and print a media file metadata * Reduce the amount of ffmpeg log messages while generating thumbnails * Fix completly wrong comments * mend * Start modeling ffmpeg extracted metadata on schema - Start porting ffprobe code to rust - Rename some references to media_data to exif_data * Finish modeling media info data - Add MediaProgram, MediaStream, MediaCodec, MediaVideoProps, MediaAudioProps, MediaSubtitleProps to Schema - Fix simple_ffproble to use its custom print_codec, instead of ffmpeg's impl * Add relation between MediaInfo and FilePath - Remove shared properties from MediaInfo and related structs - Implement Iterator for FFmpegDict * Fix and update schema * Data models and start populating MediaInfo in rust * Finish populating media info, chapters and program * Improve FFmpegFormatContext data raw pointer access - Implement stream data gathering * Impl FFmpegCodecContext, retrieve codec information - Improve some unsafe pointer uses - Impl from FFmpegFormatContext to MediaInfo conversion * Fix FFmpegDict Drop * Fix some crago warnings * Impl retrieval of video props - Fix C char* to Rust String convertion * Impl retrieval of audio and subtitle props - Fill props for MediaCodec * Remove simple_ffprobe now that the Rust impl is done * Fix schema to match actually retrieved media info - Fix import some FFmpeg constants instead of directly using values * Rework movie_decoder - Re-implement create_scale_string and add support anamorphic video - Improve C pointer access for FFmpegFormatContext and FFmpegCodecContext - Use newer FFmpeg abstractions in movie_decoder * Fix incorrect props when initializing MovieDecoder * Remove unecessary lifetimes * Added more native wrappers for some FFmpeg native objects used in movie_decoder * Remove FFmpegPacket - Some more improvements to movie_decoder * WIP * Some small fixes * More fixes Rename movie_decoder to frame_decoder Remove more references to film_strips * fmt * Fix duplicate migration for job error changes * fix rebase * Solving segfaults, fuck C lang Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Update rust to version 1.77 - Pin rust version with rust-toolchain.toml - Change from dtolnay/rust-toolchain to IronCoreLabs/rust-toolchain for rust-toolchain support - Remove unused function and imports - Replace most CString uses with new c literal string * More segfault solving and other minor fixes Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Fix ffmpeg rotation filter breaking portrait video thumbnails #2150 - Plus some other misc fixes * Auto format * Retrieve video/audio metadata on frontend * Auto format * First draft on ffmpeg data save on db Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Fix some incorrect changes to prisma schema * Some fixes for the FFmpegData schema - Expand logic to save FFmpegData to db * A ton of things Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Integrating ffmpeg media data in jobs and API * Rspc can't BigInt * 🙄 * Add initial ffmpeg metadata entries to Inspector - Fix ephemeral metadata api to match the files metadata api call * Fix Inspector not showing ffmpeg metadata * Add bitrate, start time and chapters video metadata to Inspector - Fix backend BigInt conversion incorrectly using i32 instead of u32 - Change FFmpegFormatContext/FFmpegMetaData bit_rate to i64 - Rename byteSize to humanizeSize - Expand humanizeSize logic to allow handling bits and Binary units - Move capitalize to @sd/client utils * Solving some issues * Fix ffmpeg probe getting incorrect stream id and breaking database unique constraint - Fix humanizeSize breaking when receiving floating numbers - Fix incorrect equality in StatCard - Fix unhandled error in Dialog when trying to remove an unknown dialog * fmt * small improvements - Remove some unecessary recursion_limit directive - Remove unused app_image releated functions - Fix metadata query enabled flag * Add migration for ffmpeg media data * Fix cypress test * Requested changes * Implement feedback - Update locale keys for all languages - Add pnpm command to update all language keys * Fix thumb reactivity in non indexed locations --------- Co-authored-by: Ericson Soares <ericson.ds999@gmail.com> Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
2024-05-09 02:20:28 +00:00
sd-ffmpeg = { path = "../ffmpeg", optional = true }
sd-utils = { path = "../utils" }
# Workspace dependencies
[ENG-1479] AI Prototype (#1845) * First draft on image labeling * Fixing execution providers for other OSs * Better error handling and shutdown * Working with shallow media processor * bruh * Fix warnings * Now hooked to media processor job * Link desktop app with libonnxruntime to avoid TLS error during startup * Be able to change models on runtime Revert to use labels table instead of tags * A bug on a model-less inference * Show AI labels on Inspector - Change yolo inference to use half precision - Add labels api to core * Remove LD_PRELOAD * Fix race condition on model executor shutdown * Don't load all images in memory moron * Embeed yolo model in prod build - Change yolo model path to new one relative to executable * Disable volume watcher on linux, it was crashing the app - Invalidate labels when they are updated * Rust fmt * Minor changes * Gate onnxruntime linking to the ai-models feature * Add build script to sd-server to handle onnxruntime linking workaround * Move AI stuff to its own crate and normalize deps * Rust fmt * Don't regenerate labels unless asked to * Now blazingly fast * Bad merge * Fix * Fix * Add backend logic to download extra yolo models * Add models api route - Add api call to get available model version - Add api call to change the model version * Improve new model download logic - Add frontend to change image labeler model * Fix new model downloader * Fix model select width * invalidate labels count after media_processor generates a new output * Rename AI crate and first draft on download notifications * fix types --------- Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com> Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-12-19 09:28:57 +00:00
chrono = { workspace = true, features = ["serde"] }
image = { workspace = true }
rand = { workspace = true }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
[ENG-288, ENG-601] Media data (image) extraction, thumbnail orientation fix (#1099) * basic layout * lock * add codec to image * very messy wip * rm that * lock and toml * working perfect exif extraction * formatting * migration and formatting * mostly working * fix * set date created properly * fix tsc * working media data creation * fix bad main merge? sorry brendan * schema, migrations, bindings * working exif data extraction * why won't it work * update migrations * fix bad merge * final cleanup * cleanup migrations * remove test (that was purely used for testing my code) * working media data pulling, correct thumbnail orientation * slightly optimise thumbnail rotation * rename location to prevent specta clashes * further improvements (location parsing is still broken) * fix coordinate parsing i think * rspc add some todos regarding final cleanup * further thoughts * major upgrades * Some improved handling of errors and async calls * accordion component * heavily wip flash refactor * fix builds all because of a (terrible) merge/move * annoying missing newline * i really hate exif * remove dead code * further flash progress :D * docs(media-data): clarification * minor cleanup * cleanup and some async removal * fix location parsing * remove async (will do proper impl for async eventually) and major cleanup * big W * clippy and `FlashMode::Unknown` instead of `Invalid` * add `NIKON` gps ref support * comments about gps refs * commit the submodule -_- * major cleanup & composite image support * remove old test image * major cleanup and easier to use API * remove old consts * move `ExifReader` to dedicated module * Media Data Extractor job and shallow job * Extracting media data on watcher * report no exif data on file gracefully * cleanup errors and doctests * Merging thumbnailer and media data extractor * Job name and some strings in UI * remove reliance on `sd-prisma` within the media data crate * rename query to be more specific * custom serializer for `MediaTime` * tweak to format utc -> naive correctly * generate migrations * comment out duration in mobile * delete test-assets folder * all optional fields * fix migrations again * make requested name changes * make further requested changes * remove erroneous files from another wip branch * updates procedures * use strings where appropriate * regen pnpm-lock * add base layouts for video and audio media data * use appropriate data types in schema and add audio media data framework * make requested changes * general cleanup and renaming of enum * cleanup media data api * rename media metadata type * finishing touches --------- Co-authored-by: Ericson Soares <ericson.ds999@gmail.com> Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com> Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com> Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me> Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com> Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-08-29 17:02:55 +00:00
specta = { workspace = true, features = ["chrono"] }
[ENG-1479] AI Prototype (#1845) * First draft on image labeling * Fixing execution providers for other OSs * Better error handling and shutdown * Working with shallow media processor * bruh * Fix warnings * Now hooked to media processor job * Link desktop app with libonnxruntime to avoid TLS error during startup * Be able to change models on runtime Revert to use labels table instead of tags * A bug on a model-less inference * Show AI labels on Inspector - Change yolo inference to use half precision - Add labels api to core * Remove LD_PRELOAD * Fix race condition on model executor shutdown * Don't load all images in memory moron * Embeed yolo model in prod build - Change yolo model path to new one relative to executable * Disable volume watcher on linux, it was crashing the app - Invalidate labels when they are updated * Rust fmt * Minor changes * Gate onnxruntime linking to the ai-models feature * Add build script to sd-server to handle onnxruntime linking workaround * Move AI stuff to its own crate and normalize deps * Rust fmt * Don't regenerate labels unless asked to * Now blazingly fast * Bad merge * Fix * Fix * Add backend logic to download extra yolo models * Add models api route - Add api call to get available model version - Add api call to change the model version * Improve new model download logic - Add frontend to change image labeler model * Fix new model downloader * Fix model select width * invalidate labels count after media_processor generates a new output * Rename AI crate and first draft on download notifications * fix types --------- Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com> Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-12-19 09:28:57 +00:00
thiserror = { workspace = true }
Media metadata extraction & Thumbnailer rework (#2285) * initial ffprobe commit * Working slim down version ffprobe * Auto format ffprobe and deps source * Remove show_pixel_formats logic - Fix do_bitexact incorrect check in main after last changes - Fix some clangd warning * Remove show_* and print_format options and their respective logic * Rework ffprobe into simple_ffprobe - Simplify ffprobe logic into a simple program that gather and print a media file metadata * Reduce the amount of ffmpeg log messages while generating thumbnails * Fix completly wrong comments * mend * Start modeling ffmpeg extracted metadata on schema - Start porting ffprobe code to rust - Rename some references to media_data to exif_data * Finish modeling media info data - Add MediaProgram, MediaStream, MediaCodec, MediaVideoProps, MediaAudioProps, MediaSubtitleProps to Schema - Fix simple_ffproble to use its custom print_codec, instead of ffmpeg's impl * Add relation between MediaInfo and FilePath - Remove shared properties from MediaInfo and related structs - Implement Iterator for FFmpegDict * Fix and update schema * Data models and start populating MediaInfo in rust * Finish populating media info, chapters and program * Improve FFmpegFormatContext data raw pointer access - Implement stream data gathering * Impl FFmpegCodecContext, retrieve codec information - Improve some unsafe pointer uses - Impl from FFmpegFormatContext to MediaInfo conversion * Fix FFmpegDict Drop * Fix some crago warnings * Impl retrieval of video props - Fix C char* to Rust String convertion * Impl retrieval of audio and subtitle props - Fill props for MediaCodec * Remove simple_ffprobe now that the Rust impl is done * Fix schema to match actually retrieved media info - Fix import some FFmpeg constants instead of directly using values * Rework movie_decoder - Re-implement create_scale_string and add support anamorphic video - Improve C pointer access for FFmpegFormatContext and FFmpegCodecContext - Use newer FFmpeg abstractions in movie_decoder * Fix incorrect props when initializing MovieDecoder * Remove unecessary lifetimes * Added more native wrappers for some FFmpeg native objects used in movie_decoder * Remove FFmpegPacket - Some more improvements to movie_decoder * WIP * Some small fixes * More fixes Rename movie_decoder to frame_decoder Remove more references to film_strips * fmt * Fix duplicate migration for job error changes * fix rebase * Solving segfaults, fuck C lang Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Update rust to version 1.77 - Pin rust version with rust-toolchain.toml - Change from dtolnay/rust-toolchain to IronCoreLabs/rust-toolchain for rust-toolchain support - Remove unused function and imports - Replace most CString uses with new c literal string * More segfault solving and other minor fixes Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Fix ffmpeg rotation filter breaking portrait video thumbnails #2150 - Plus some other misc fixes * Auto format * Retrieve video/audio metadata on frontend * Auto format * First draft on ffmpeg data save on db Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Fix some incorrect changes to prisma schema * Some fixes for the FFmpegData schema - Expand logic to save FFmpegData to db * A ton of things Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Integrating ffmpeg media data in jobs and API * Rspc can't BigInt * 🙄 * Add initial ffmpeg metadata entries to Inspector - Fix ephemeral metadata api to match the files metadata api call * Fix Inspector not showing ffmpeg metadata * Add bitrate, start time and chapters video metadata to Inspector - Fix backend BigInt conversion incorrectly using i32 instead of u32 - Change FFmpegFormatContext/FFmpegMetaData bit_rate to i64 - Rename byteSize to humanizeSize - Expand humanizeSize logic to allow handling bits and Binary units - Move capitalize to @sd/client utils * Solving some issues * Fix ffmpeg probe getting incorrect stream id and breaking database unique constraint - Fix humanizeSize breaking when receiving floating numbers - Fix incorrect equality in StatCard - Fix unhandled error in Dialog when trying to remove an unknown dialog * fmt * small improvements - Remove some unecessary recursion_limit directive - Remove unused app_image releated functions - Fix metadata query enabled flag * Add migration for ffmpeg media data * Fix cypress test * Requested changes * Implement feedback - Update locale keys for all languages - Add pnpm command to update all language keys * Fix thumb reactivity in non indexed locations --------- Co-authored-by: Ericson Soares <ericson.ds999@gmail.com> Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
2024-05-09 02:20:28 +00:00
tokio = { workspace = true }
[ENG-1479] AI Prototype (#1845) * First draft on image labeling * Fixing execution providers for other OSs * Better error handling and shutdown * Working with shallow media processor * bruh * Fix warnings * Now hooked to media processor job * Link desktop app with libonnxruntime to avoid TLS error during startup * Be able to change models on runtime Revert to use labels table instead of tags * A bug on a model-less inference * Show AI labels on Inspector - Change yolo inference to use half precision - Add labels api to core * Remove LD_PRELOAD * Fix race condition on model executor shutdown * Don't load all images in memory moron * Embeed yolo model in prod build - Change yolo model path to new one relative to executable * Disable volume watcher on linux, it was crashing the app - Invalidate labels when they are updated * Rust fmt * Minor changes * Gate onnxruntime linking to the ai-models feature * Add build script to sd-server to handle onnxruntime linking workaround * Move AI stuff to its own crate and normalize deps * Rust fmt * Don't regenerate labels unless asked to * Now blazingly fast * Bad merge * Fix * Fix * Add backend logic to download extra yolo models * Add models api route - Add api call to get available model version - Add api call to change the model version * Improve new model download logic - Add frontend to change image labeler model * Fix new model downloader * Fix model select width * invalidate labels count after media_processor generates a new output * Rename AI crate and first draft on download notifications * fix types --------- Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com> Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-12-19 09:28:57 +00:00
# Specific Media Metadata dependencies
[ENG-1479] AI Prototype (#1845) * First draft on image labeling * Fixing execution providers for other OSs * Better error handling and shutdown * Working with shallow media processor * bruh * Fix warnings * Now hooked to media processor job * Link desktop app with libonnxruntime to avoid TLS error during startup * Be able to change models on runtime Revert to use labels table instead of tags * A bug on a model-less inference * Show AI labels on Inspector - Change yolo inference to use half precision - Add labels api to core * Remove LD_PRELOAD * Fix race condition on model executor shutdown * Don't load all images in memory moron * Embeed yolo model in prod build - Change yolo model path to new one relative to executable * Disable volume watcher on linux, it was crashing the app - Invalidate labels when they are updated * Rust fmt * Minor changes * Gate onnxruntime linking to the ai-models feature * Add build script to sd-server to handle onnxruntime linking workaround * Move AI stuff to its own crate and normalize deps * Rust fmt * Don't regenerate labels unless asked to * Now blazingly fast * Bad merge * Fix * Fix * Add backend logic to download extra yolo models * Add models api route - Add api call to get available model version - Add api call to change the model version * Improve new model download logic - Add frontend to change image labeler model * Fix new model downloader * Fix model select width * invalidate labels count after media_processor generates a new output * Rename AI crate and first draft on download notifications * fix types --------- Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com> Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-12-19 09:28:57 +00:00
kamadak-exif = "0.5.5"