spacedrive/crates/ai/Cargo.toml

80 lines
2.5 KiB
TOML
Raw Normal View History

[ENG-1479] AI Prototype (#1845) * First draft on image labeling * Fixing execution providers for other OSs * Better error handling and shutdown * Working with shallow media processor * bruh * Fix warnings * Now hooked to media processor job * Link desktop app with libonnxruntime to avoid TLS error during startup * Be able to change models on runtime Revert to use labels table instead of tags * A bug on a model-less inference * Show AI labels on Inspector - Change yolo inference to use half precision - Add labels api to core * Remove LD_PRELOAD * Fix race condition on model executor shutdown * Don't load all images in memory moron * Embeed yolo model in prod build - Change yolo model path to new one relative to executable * Disable volume watcher on linux, it was crashing the app - Invalidate labels when they are updated * Rust fmt * Minor changes * Gate onnxruntime linking to the ai-models feature * Add build script to sd-server to handle onnxruntime linking workaround * Move AI stuff to its own crate and normalize deps * Rust fmt * Don't regenerate labels unless asked to * Now blazingly fast * Bad merge * Fix * Fix * Add backend logic to download extra yolo models * Add models api route - Add api call to get available model version - Add api call to change the model version * Improve new model download logic - Add frontend to change image labeler model * Fix new model downloader * Fix model select width * invalidate labels count after media_processor generates a new output * Rename AI crate and first draft on download notifications * fix types --------- Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com> Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-12-19 09:28:57 +00:00
[package]
name = "sd-ai"
version = "0.1.0"
authors = ["Ericson Soares <ericson@spacedrive.com>"]
readme = "README.md"
description = "A simple library to generate video thumbnails using ffmpeg with the webp format"
Media metadata extraction & Thumbnailer rework (#2285) * initial ffprobe commit * Working slim down version ffprobe * Auto format ffprobe and deps source * Remove show_pixel_formats logic - Fix do_bitexact incorrect check in main after last changes - Fix some clangd warning * Remove show_* and print_format options and their respective logic * Rework ffprobe into simple_ffprobe - Simplify ffprobe logic into a simple program that gather and print a media file metadata * Reduce the amount of ffmpeg log messages while generating thumbnails * Fix completly wrong comments * mend * Start modeling ffmpeg extracted metadata on schema - Start porting ffprobe code to rust - Rename some references to media_data to exif_data * Finish modeling media info data - Add MediaProgram, MediaStream, MediaCodec, MediaVideoProps, MediaAudioProps, MediaSubtitleProps to Schema - Fix simple_ffproble to use its custom print_codec, instead of ffmpeg's impl * Add relation between MediaInfo and FilePath - Remove shared properties from MediaInfo and related structs - Implement Iterator for FFmpegDict * Fix and update schema * Data models and start populating MediaInfo in rust * Finish populating media info, chapters and program * Improve FFmpegFormatContext data raw pointer access - Implement stream data gathering * Impl FFmpegCodecContext, retrieve codec information - Improve some unsafe pointer uses - Impl from FFmpegFormatContext to MediaInfo conversion * Fix FFmpegDict Drop * Fix some crago warnings * Impl retrieval of video props - Fix C char* to Rust String convertion * Impl retrieval of audio and subtitle props - Fill props for MediaCodec * Remove simple_ffprobe now that the Rust impl is done * Fix schema to match actually retrieved media info - Fix import some FFmpeg constants instead of directly using values * Rework movie_decoder - Re-implement create_scale_string and add support anamorphic video - Improve C pointer access for FFmpegFormatContext and FFmpegCodecContext - Use newer FFmpeg abstractions in movie_decoder * Fix incorrect props when initializing MovieDecoder * Remove unecessary lifetimes * Added more native wrappers for some FFmpeg native objects used in movie_decoder * Remove FFmpegPacket - Some more improvements to movie_decoder * WIP * Some small fixes * More fixes Rename movie_decoder to frame_decoder Remove more references to film_strips * fmt * Fix duplicate migration for job error changes * fix rebase * Solving segfaults, fuck C lang Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Update rust to version 1.77 - Pin rust version with rust-toolchain.toml - Change from dtolnay/rust-toolchain to IronCoreLabs/rust-toolchain for rust-toolchain support - Remove unused function and imports - Replace most CString uses with new c literal string * More segfault solving and other minor fixes Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Fix ffmpeg rotation filter breaking portrait video thumbnails #2150 - Plus some other misc fixes * Auto format * Retrieve video/audio metadata on frontend * Auto format * First draft on ffmpeg data save on db Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Fix some incorrect changes to prisma schema * Some fixes for the FFmpegData schema - Expand logic to save FFmpegData to db * A ton of things Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com> * Integrating ffmpeg media data in jobs and API * Rspc can't BigInt * 🙄 * Add initial ffmpeg metadata entries to Inspector - Fix ephemeral metadata api to match the files metadata api call * Fix Inspector not showing ffmpeg metadata * Add bitrate, start time and chapters video metadata to Inspector - Fix backend BigInt conversion incorrectly using i32 instead of u32 - Change FFmpegFormatContext/FFmpegMetaData bit_rate to i64 - Rename byteSize to humanizeSize - Expand humanizeSize logic to allow handling bits and Binary units - Move capitalize to @sd/client utils * Solving some issues * Fix ffmpeg probe getting incorrect stream id and breaking database unique constraint - Fix humanizeSize breaking when receiving floating numbers - Fix incorrect equality in StatCard - Fix unhandled error in Dialog when trying to remove an unknown dialog * fmt * small improvements - Remove some unecessary recursion_limit directive - Remove unused app_image releated functions - Fix metadata query enabled flag * Add migration for ffmpeg media data * Fix cypress test * Requested changes * Implement feedback - Update locale keys for all languages - Add pnpm command to update all language keys * Fix thumb reactivity in non indexed locations --------- Co-authored-by: Ericson Soares <ericson.ds999@gmail.com> Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
2024-05-09 02:20:28 +00:00
rust-version = "1.75"
[ENG-1479] AI Prototype (#1845) * First draft on image labeling * Fixing execution providers for other OSs * Better error handling and shutdown * Working with shallow media processor * bruh * Fix warnings * Now hooked to media processor job * Link desktop app with libonnxruntime to avoid TLS error during startup * Be able to change models on runtime Revert to use labels table instead of tags * A bug on a model-less inference * Show AI labels on Inspector - Change yolo inference to use half precision - Add labels api to core * Remove LD_PRELOAD * Fix race condition on model executor shutdown * Don't load all images in memory moron * Embeed yolo model in prod build - Change yolo model path to new one relative to executable * Disable volume watcher on linux, it was crashing the app - Invalidate labels when they are updated * Rust fmt * Minor changes * Gate onnxruntime linking to the ai-models feature * Add build script to sd-server to handle onnxruntime linking workaround * Move AI stuff to its own crate and normalize deps * Rust fmt * Don't regenerate labels unless asked to * Now blazingly fast * Bad merge * Fix * Fix * Add backend logic to download extra yolo models * Add models api route - Add api call to get available model version - Add api call to change the model version * Improve new model download logic - Add frontend to change image labeler model * Fix new model downloader * Fix model select width * invalidate labels count after media_processor generates a new output * Rename AI crate and first draft on download notifications * fix types --------- Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com> Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-12-19 09:28:57 +00:00
license = { workspace = true }
repository = { workspace = true }
edition = { workspace = true }
[dependencies]
# Inner Core Sub-crates
sd-core-file-path-helper = { path = "../../core/crates/file-path-helper" }
sd-core-prisma-helpers = { path = "../../core/crates/prisma-helpers" }
sd-core-sync = { path = "../../core/crates/sync" }
# Spacedrive Sub-crates
sd-prisma = { path = "../prisma" }
sd-sync = { path = "../sync" }
[ENG-1479] AI Prototype (#1845) * First draft on image labeling * Fixing execution providers for other OSs * Better error handling and shutdown * Working with shallow media processor * bruh * Fix warnings * Now hooked to media processor job * Link desktop app with libonnxruntime to avoid TLS error during startup * Be able to change models on runtime Revert to use labels table instead of tags * A bug on a model-less inference * Show AI labels on Inspector - Change yolo inference to use half precision - Add labels api to core * Remove LD_PRELOAD * Fix race condition on model executor shutdown * Don't load all images in memory moron * Embeed yolo model in prod build - Change yolo model path to new one relative to executable * Disable volume watcher on linux, it was crashing the app - Invalidate labels when they are updated * Rust fmt * Minor changes * Gate onnxruntime linking to the ai-models feature * Add build script to sd-server to handle onnxruntime linking workaround * Move AI stuff to its own crate and normalize deps * Rust fmt * Don't regenerate labels unless asked to * Now blazingly fast * Bad merge * Fix * Fix * Add backend logic to download extra yolo models * Add models api route - Add api call to get available model version - Add api call to change the model version * Improve new model download logic - Add frontend to change image labeler model * Fix new model downloader * Fix model select width * invalidate labels count after media_processor generates a new output * Rename AI crate and first draft on download notifications * fix types --------- Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com> Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-12-19 09:28:57 +00:00
sd-utils = { path = "../utils" }
async-channel = { workspace = true }
chrono = { workspace = true, features = ["serde"] }
futures = { workspace = true }
futures-concurrency = { workspace = true }
image = { workspace = true }
once_cell = { workspace = true }
prisma-client-rust = { workspace = true }
reqwest = { workspace = true, features = ["stream", "native-tls-vendored"] }
rmp-serde = { workspace = true }
rmpv = { workspace = true }
[ENG-1479] AI Prototype (#1845) * First draft on image labeling * Fixing execution providers for other OSs * Better error handling and shutdown * Working with shallow media processor * bruh * Fix warnings * Now hooked to media processor job * Link desktop app with libonnxruntime to avoid TLS error during startup * Be able to change models on runtime Revert to use labels table instead of tags * A bug on a model-less inference * Show AI labels on Inspector - Change yolo inference to use half precision - Add labels api to core * Remove LD_PRELOAD * Fix race condition on model executor shutdown * Don't load all images in memory moron * Embeed yolo model in prod build - Change yolo model path to new one relative to executable * Disable volume watcher on linux, it was crashing the app - Invalidate labels when they are updated * Rust fmt * Minor changes * Gate onnxruntime linking to the ai-models feature * Add build script to sd-server to handle onnxruntime linking workaround * Move AI stuff to its own crate and normalize deps * Rust fmt * Don't regenerate labels unless asked to * Now blazingly fast * Bad merge * Fix * Fix * Add backend logic to download extra yolo models * Add models api route - Add api call to get available model version - Add api call to change the model version * Improve new model download logic - Add frontend to change image labeler model * Fix new model downloader * Fix model select width * invalidate labels count after media_processor generates a new output * Rename AI crate and first draft on download notifications * fix types --------- Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com> Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-12-19 09:28:57 +00:00
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
[ENG-1479] AI Prototype (#1845) * First draft on image labeling * Fixing execution providers for other OSs * Better error handling and shutdown * Working with shallow media processor * bruh * Fix warnings * Now hooked to media processor job * Link desktop app with libonnxruntime to avoid TLS error during startup * Be able to change models on runtime Revert to use labels table instead of tags * A bug on a model-less inference * Show AI labels on Inspector - Change yolo inference to use half precision - Add labels api to core * Remove LD_PRELOAD * Fix race condition on model executor shutdown * Don't load all images in memory moron * Embeed yolo model in prod build - Change yolo model path to new one relative to executable * Disable volume watcher on linux, it was crashing the app - Invalidate labels when they are updated * Rust fmt * Minor changes * Gate onnxruntime linking to the ai-models feature * Add build script to sd-server to handle onnxruntime linking workaround * Move AI stuff to its own crate and normalize deps * Rust fmt * Don't regenerate labels unless asked to * Now blazingly fast * Bad merge * Fix * Fix * Add backend logic to download extra yolo models * Add models api route - Add api call to get available model version - Add api call to change the model version * Improve new model download logic - Add frontend to change image labeler model * Fix new model downloader * Fix model select width * invalidate labels count after media_processor generates a new output * Rename AI crate and first draft on download notifications * fix types --------- Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com> Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-12-19 09:28:57 +00:00
thiserror = { workspace = true }
tokio = { workspace = true, features = ["fs"] }
tokio-stream = { workspace = true }
tracing = { workspace = true }
uuid = { workspace = true, features = ["v4", "serde"] }
# Note: half and ndarray version must be the same as used in ort
[ENG-1479] AI Prototype (#1845) * First draft on image labeling * Fixing execution providers for other OSs * Better error handling and shutdown * Working with shallow media processor * bruh * Fix warnings * Now hooked to media processor job * Link desktop app with libonnxruntime to avoid TLS error during startup * Be able to change models on runtime Revert to use labels table instead of tags * A bug on a model-less inference * Show AI labels on Inspector - Change yolo inference to use half precision - Add labels api to core * Remove LD_PRELOAD * Fix race condition on model executor shutdown * Don't load all images in memory moron * Embeed yolo model in prod build - Change yolo model path to new one relative to executable * Disable volume watcher on linux, it was crashing the app - Invalidate labels when they are updated * Rust fmt * Minor changes * Gate onnxruntime linking to the ai-models feature * Add build script to sd-server to handle onnxruntime linking workaround * Move AI stuff to its own crate and normalize deps * Rust fmt * Don't regenerate labels unless asked to * Now blazingly fast * Bad merge * Fix * Fix * Add backend logic to download extra yolo models * Add models api route - Add api call to get available model version - Add api call to change the model version * Improve new model download logic - Add frontend to change image labeler model * Fix new model downloader * Fix model select width * invalidate labels count after media_processor generates a new output * Rename AI crate and first draft on download notifications * fix types --------- Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com> Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-12-19 09:28:57 +00:00
half = { version = "2.1", features = ['num-traits'] }
ndarray = "0.15"
url = '2.5.0'
[ENG-1479] AI Prototype (#1845) * First draft on image labeling * Fixing execution providers for other OSs * Better error handling and shutdown * Working with shallow media processor * bruh * Fix warnings * Now hooked to media processor job * Link desktop app with libonnxruntime to avoid TLS error during startup * Be able to change models on runtime Revert to use labels table instead of tags * A bug on a model-less inference * Show AI labels on Inspector - Change yolo inference to use half precision - Add labels api to core * Remove LD_PRELOAD * Fix race condition on model executor shutdown * Don't load all images in memory moron * Embeed yolo model in prod build - Change yolo model path to new one relative to executable * Disable volume watcher on linux, it was crashing the app - Invalidate labels when they are updated * Rust fmt * Minor changes * Gate onnxruntime linking to the ai-models feature * Add build script to sd-server to handle onnxruntime linking workaround * Move AI stuff to its own crate and normalize deps * Rust fmt * Don't regenerate labels unless asked to * Now blazingly fast * Bad merge * Fix * Fix * Add backend logic to download extra yolo models * Add models api route - Add api call to get available model version - Add api call to change the model version * Improve new model download logic - Add frontend to change image labeler model * Fix new model downloader * Fix model select width * invalidate labels count after media_processor generates a new output * Rename AI crate and first draft on download notifications * fix types --------- Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com> Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-12-19 09:28:57 +00:00
# Microsoft does not provide a release for osx-gpu. See: https://github.com/microsoft/onnxruntime/releases
# "gpu" means CUDA or TensorRT EP. Thus, the ort crate cannot download them at build time.
# Ref: https://github.com/pykeio/ort/blob/d7defd1862969b4b44f7f3f4b9c72263690bd67b/build.rs#L148
[target.'cfg(target_os = "windows")'.dependencies]
ort = { version = "=2.0.0-rc.0", default-features = false, features = [
[ENG-1479] AI Prototype (#1845) * First draft on image labeling * Fixing execution providers for other OSs * Better error handling and shutdown * Working with shallow media processor * bruh * Fix warnings * Now hooked to media processor job * Link desktop app with libonnxruntime to avoid TLS error during startup * Be able to change models on runtime Revert to use labels table instead of tags * A bug on a model-less inference * Show AI labels on Inspector - Change yolo inference to use half precision - Add labels api to core * Remove LD_PRELOAD * Fix race condition on model executor shutdown * Don't load all images in memory moron * Embeed yolo model in prod build - Change yolo model path to new one relative to executable * Disable volume watcher on linux, it was crashing the app - Invalidate labels when they are updated * Rust fmt * Minor changes * Gate onnxruntime linking to the ai-models feature * Add build script to sd-server to handle onnxruntime linking workaround * Move AI stuff to its own crate and normalize deps * Rust fmt * Don't regenerate labels unless asked to * Now blazingly fast * Bad merge * Fix * Fix * Add backend logic to download extra yolo models * Add models api route - Add api call to get available model version - Add api call to change the model version * Improve new model download logic - Add frontend to change image labeler model * Fix new model downloader * Fix model select width * invalidate labels count after media_processor generates a new output * Rename AI crate and first draft on download notifications * fix types --------- Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com> Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-12-19 09:28:57 +00:00
"ndarray",
"half",
"load-dynamic",
"directml",
] }
[target.'cfg(target_os = "linux")'.dependencies]
ort = { version = "=2.0.0-rc.0", default-features = false, features = [
[ENG-1479] AI Prototype (#1845) * First draft on image labeling * Fixing execution providers for other OSs * Better error handling and shutdown * Working with shallow media processor * bruh * Fix warnings * Now hooked to media processor job * Link desktop app with libonnxruntime to avoid TLS error during startup * Be able to change models on runtime Revert to use labels table instead of tags * A bug on a model-less inference * Show AI labels on Inspector - Change yolo inference to use half precision - Add labels api to core * Remove LD_PRELOAD * Fix race condition on model executor shutdown * Don't load all images in memory moron * Embeed yolo model in prod build - Change yolo model path to new one relative to executable * Disable volume watcher on linux, it was crashing the app - Invalidate labels when they are updated * Rust fmt * Minor changes * Gate onnxruntime linking to the ai-models feature * Add build script to sd-server to handle onnxruntime linking workaround * Move AI stuff to its own crate and normalize deps * Rust fmt * Don't regenerate labels unless asked to * Now blazingly fast * Bad merge * Fix * Fix * Add backend logic to download extra yolo models * Add models api route - Add api call to get available model version - Add api call to change the model version * Improve new model download logic - Add frontend to change image labeler model * Fix new model downloader * Fix model select width * invalidate labels count after media_processor generates a new output * Rename AI crate and first draft on download notifications * fix types --------- Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com> Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-12-19 09:28:57 +00:00
"ndarray",
"half",
"xnnpack",
] }
# [target.'cfg(target_os = "android")'.dependencies]
# ort = { version = "=2.0.0-rc.0", default-features = false, features = [
[ENG-1479] AI Prototype (#1845) * First draft on image labeling * Fixing execution providers for other OSs * Better error handling and shutdown * Working with shallow media processor * bruh * Fix warnings * Now hooked to media processor job * Link desktop app with libonnxruntime to avoid TLS error during startup * Be able to change models on runtime Revert to use labels table instead of tags * A bug on a model-less inference * Show AI labels on Inspector - Change yolo inference to use half precision - Add labels api to core * Remove LD_PRELOAD * Fix race condition on model executor shutdown * Don't load all images in memory moron * Embeed yolo model in prod build - Change yolo model path to new one relative to executable * Disable volume watcher on linux, it was crashing the app - Invalidate labels when they are updated * Rust fmt * Minor changes * Gate onnxruntime linking to the ai-models feature * Add build script to sd-server to handle onnxruntime linking workaround * Move AI stuff to its own crate and normalize deps * Rust fmt * Don't regenerate labels unless asked to * Now blazingly fast * Bad merge * Fix * Fix * Add backend logic to download extra yolo models * Add models api route - Add api call to get available model version - Add api call to change the model version * Improve new model download logic - Add frontend to change image labeler model * Fix new model downloader * Fix model select width * invalidate labels count after media_processor generates a new output * Rename AI crate and first draft on download notifications * fix types --------- Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com> Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-12-19 09:28:57 +00:00
# "half",
# "load-dynamic",
# "qnn",
# "nnapi",
# "xnnpack",
# "acl",
# "armnn",
# ] }
[target.'cfg(any(target_os = "macos", target_os = "ios"))'.dependencies]
ort = { version = "=2.0.0-rc.0", features = [
[ENG-1479] AI Prototype (#1845) * First draft on image labeling * Fixing execution providers for other OSs * Better error handling and shutdown * Working with shallow media processor * bruh * Fix warnings * Now hooked to media processor job * Link desktop app with libonnxruntime to avoid TLS error during startup * Be able to change models on runtime Revert to use labels table instead of tags * A bug on a model-less inference * Show AI labels on Inspector - Change yolo inference to use half precision - Add labels api to core * Remove LD_PRELOAD * Fix race condition on model executor shutdown * Don't load all images in memory moron * Embeed yolo model in prod build - Change yolo model path to new one relative to executable * Disable volume watcher on linux, it was crashing the app - Invalidate labels when they are updated * Rust fmt * Minor changes * Gate onnxruntime linking to the ai-models feature * Add build script to sd-server to handle onnxruntime linking workaround * Move AI stuff to its own crate and normalize deps * Rust fmt * Don't regenerate labels unless asked to * Now blazingly fast * Bad merge * Fix * Fix * Add backend logic to download extra yolo models * Add models api route - Add api call to get available model version - Add api call to change the model version * Improve new model download logic - Add frontend to change image labeler model * Fix new model downloader * Fix model select width * invalidate labels count after media_processor generates a new output * Rename AI crate and first draft on download notifications * fix types --------- Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com> Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-12-19 09:28:57 +00:00
"ndarray",
"half",
"load-dynamic",
"coreml",
"xnnpack",
] }