* initial ffprobe commit
* Working slim down version ffprobe
* Auto format ffprobe and deps source
* Remove show_pixel_formats logic
- Fix do_bitexact incorrect check in main after last changes
- Fix some clangd warning
* Remove show_* and print_format options and their respective logic
* Rework ffprobe into simple_ffprobe
- Simplify ffprobe logic into a simple program that gather and print a media file metadata
* Reduce the amount of ffmpeg log messages while generating thumbnails
* Fix completly wrong comments
* mend
* Start modeling ffmpeg extracted metadata on schema
- Start porting ffprobe code to rust
- Rename some references to media_data to exif_data
* Finish modeling media info data
- Add MediaProgram, MediaStream, MediaCodec, MediaVideoProps, MediaAudioProps, MediaSubtitleProps to Schema
- Fix simple_ffproble to use its custom print_codec, instead of ffmpeg's impl
* Add relation between MediaInfo and FilePath
- Remove shared properties from MediaInfo and related structs
- Implement Iterator for FFmpegDict
* Fix and update schema
* Data models and start populating MediaInfo in rust
* Finish populating media info, chapters and program
* Improve FFmpegFormatContext data raw pointer access
- Implement stream data gathering
* Impl FFmpegCodecContext, retrieve codec information
- Improve some unsafe pointer uses
- Impl from FFmpegFormatContext to MediaInfo conversion
* Fix FFmpegDict Drop
* Fix some crago warnings
* Impl retrieval of video props
- Fix C char* to Rust String convertion
* Impl retrieval of audio and subtitle props
- Fill props for MediaCodec
* Remove simple_ffprobe now that the Rust impl is done
* Fix schema to match actually retrieved media info
- Fix import some FFmpeg constants instead of directly using values
* Rework movie_decoder
- Re-implement create_scale_string and add support anamorphic video
- Improve C pointer access for FFmpegFormatContext and FFmpegCodecContext
- Use newer FFmpeg abstractions in movie_decoder
* Fix incorrect props when initializing MovieDecoder
* Remove unecessary lifetimes
* Added more native wrappers for some FFmpeg native objects used in movie_decoder
* Remove FFmpegPacket
- Some more improvements to movie_decoder
* WIP
* Some small fixes
* More fixes
Rename movie_decoder to frame_decoder
Remove more references to film_strips
* fmt
* Fix duplicate migration for job error changes
* fix rebase
* Solving segfaults, fuck C lang
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Update rust to version 1.77
- Pin rust version with rust-toolchain.toml
- Change from dtolnay/rust-toolchain to IronCoreLabs/rust-toolchain for rust-toolchain support
- Remove unused function and imports
- Replace most CString uses with new c literal string
* More segfault solving and other minor fixes
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Fix ffmpeg rotation filter breaking portrait video thumbnails #2150
- Plus some other misc fixes
* Auto format
* Retrieve video/audio metadata on frontend
* Auto format
* First draft on ffmpeg data save on db
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Fix some incorrect changes to prisma schema
* Some fixes for the FFmpegData schema
- Expand logic to save FFmpegData to db
* A ton of things
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Integrating ffmpeg media data in jobs and API
* Rspc can't BigInt
* 🙄
* Add initial ffmpeg metadata entries to Inspector
- Fix ephemeral metadata api to match the files metadata api call
* Fix Inspector not showing ffmpeg metadata
* Add bitrate, start time and chapters video metadata to Inspector
- Fix backend BigInt conversion incorrectly using i32 instead of u32
- Change FFmpegFormatContext/FFmpegMetaData bit_rate to i64
- Rename byteSize to humanizeSize
- Expand humanizeSize logic to allow handling bits and Binary units
- Move capitalize to @sd/client utils
* Solving some issues
* Fix ffmpeg probe getting incorrect stream id and breaking database unique constraint
- Fix humanizeSize breaking when receiving floating numbers
- Fix incorrect equality in StatCard
- Fix unhandled error in Dialog when trying to remove an unknown dialog
* fmt
* small improvements
- Remove some unecessary recursion_limit directive
- Remove unused app_image releated functions
- Fix metadata query enabled flag
* Add migration for ffmpeg media data
* Fix cypress test
* Requested changes
* Implement feedback
- Update locale keys for all languages
- Add pnpm command to update all language keys
* Fix thumb reactivity in non indexed locations
---------
Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
* Moving file-path-helper to a sub-crate on core
* Parallel walker tested and working
* Change inner core crate names to sd-core-* naming scheme
* Moving stuff around
* Save and Update tasks for the new indexer job
* Some initial drafts on the new job system
* More drafts on job system
* Changing walker task to a more explicit state machine
Also drafting more of job system
* More drafting on job resume
* Completed the draft on job system inner workings
* New job context abstraction to decouple library stuff from job system
* Properly use composition on task dispatcher
* First draft on indexer job
* Job serialization
* Handling ancestors in the distributed walker for indexer
* Saving computed directories sizes on a location to db
* Enable a way to check if jobs are running in a location
* Progress reporting on indexer job
* Reorganizing modules
* Shallow indexer
* Rust fmt
* Attempting windows CI fix
* Attempting windows CI fix again
* Attempting windows CI fix again
* wip for iDevices
* Working HardwareModel Info for iOS
* wip
* Merge 'main' into 'mob-hw-info-overview'
* Half-Working `get_volume()`
* Objective c bridge to talk to FS
* Working objc bridge
The bridge works now, and we can now access the iOS file system using the native objective-c APIs instead for proper values, including on the simulator.
* Isolate `icrate` for `ios` deployments only
* Working Stats for Android
* Clean Up + `pnpm format`
* Fix to FSInfoResult Type
Due to the RNFS fork change, I had to change the types to make it so it doesn't fail building and CI.
* iOS Device Name Fix
* version & microphonePermission text & eslint
* remove polyfill as hermes supports intl now
* why do we have solid on mobile?
* cleanup
* add solid back =_=
* pnpm lock
* we hate relative paths here
* android config
* open file logic
* visual tweaks
---------
Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
* Update github actions due to nodejs 16 deprecation
- Replace archived actions-rs/clippy-check with maintained fork actions-rs-plus/clippy-check
- Replace redhat-actions/push-to-registry with updated fork Eusebiotrigo/push-to-registry
- Point redhat-actions/buildah-build and softprops/action-gh-release to current master to fix nodejs deprecation
* Build the correct ios core rust arch for CI runs
* Build ios app for the same arch as the host in Mobile CI
* Some changes to try and make cache-factory faster and avoid failing so much
* Add trigger to run cache-factory on pull requests when there are changes to itself
* Attempt to fix sed usage on macOS
* Don't treat warning as errors
* Fix windows
* Fix windows 2
* Use target ad cache key for rust to differentiate between macOS x86_64 and arm64
* Use faster/better linkers to compile for macOS, Linux and Windows
* Fix missing shell in action
* Fix typo
* Fix missing shell in action 2
* Fix mold download
- Replace bsdtar with plain tar
* Fix permission denied when extracting mold
* Remove zld
* Don't restore cache for rustfmt
- Remove target symlink to C:/ in an attempt to speed-up windows CI
* Fix typo
* Restore target symlink on windows
- Removing it didn't make CI faster
* Run Mobile on macos-14
* New Android Build Script
* Clean up + Works for CI now
* Simplify android build.sh
- Fix /var/home/vitor fallback for Linux systems
- Run a single cargo ndk for all targets (not parallel build, but a bit faster)
- Fix android target s/x86/x86_64/
- Format setup.sh
- Minor improvements to rust mobile targets installation step in setup.sh
* Add notice to CONTRIBUTING that only Java <= 17 is supported for building android
- Make prettier ignore some mobile build artifacts
* When in CI, Fix build android core for host architecture
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
* fix: wait until libraries have loaded before checking the count
* fix: include new core version in lockfile
* fix: moved value
* fix: check the `libraries/` data dir manually
* Clear localstorage via localStoarge.clear() (#2040)
only clear localstorage in desktop using js
---------
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
* feat: clear localstorage on start if no libraries are found
* chore: add a check to see if the node is for desktop or server use
this will prevent desktop/app localstorage from being cleared if it's a server being hosted
* fix: add secondary localstorage directory for macos
for some reason the app/onboarding progress is kept in `Library/WebKit/Spacedrive` (at least in dev?) so it also needs clearing
* feat: delete cache directories also
* chore: iterate over paths
* feat: add support for windows `localStorage` by using `data_local_dir`
* docs/style: better comments and logs
* feat: remove both `AppData` local and roaming on Windows
* fix: mobile builds
* chore: tweak error message for windows
* First draft on image labeling
* Fixing execution providers for other OSs
* Better error handling and shutdown
* Working with shallow media processor
* bruh
* Fix warnings
* Now hooked to media processor job
* Link desktop app with libonnxruntime to avoid TLS error during startup
* Be able to change models on runtime
Revert to use labels table instead of tags
* A bug on a model-less inference
* Show AI labels on Inspector
- Change yolo inference to use half precision
- Add labels api to core
* Remove LD_PRELOAD
* Fix race condition on model executor shutdown
* Don't load all images in memory moron
* Embeed yolo model in prod build
- Change yolo model path to new one relative to executable
* Disable volume watcher on linux, it was crashing the app
- Invalidate labels when they are updated
* Rust fmt
* Minor changes
* Gate onnxruntime linking to the ai-models feature
* Add build script to sd-server to handle onnxruntime linking workaround
* Move AI stuff to its own crate and normalize deps
* Rust fmt
* Don't regenerate labels unless asked to
* Now blazingly fast
* Bad merge
* Fix
* Fix
* Add backend logic to download extra yolo models
* Add models api route
- Add api call to get available model version
- Add api call to change the model version
* Improve new model download logic
- Add frontend to change image labeler model
* Fix new model downloader
* Fix model select width
* invalidate labels count after media_processor generates a new output
* Rename AI crate and first draft on download notifications
* fix types
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
* Almost working
* Downgrade libplacebo
- FFMpeg 6.0 uses some now removed deprecated functions
* Use -Oz for zimg
* Fix CI script to run the new ffmpeg build script
* Fix heif step name + Ignore docker cache while building in CI
* Fix Opencl build on linux
* Fix adding incorrect -target argument to linker
- Update zig for windows target
* Disable opengl for ffmpeg, it only uses it as an outdev, not for processing
- Disable opengl and directx for libplacebo, ffmpeg only supports vulkan when using it
- Add WIN32_LEAN_AND_MEAN to global cflags to optimize windows api usage
- Fix 99-heif.sh incorrect bsdtar flag
* Remove WIN32_LEAN_AND_MEAN from global CFLAGS as that was breaking OpenCL build
- Fix Dockerfile step for cleaning up the out dir
- Improve licensing handling
* x86_64 windows and linux builds are working
* Fix aarch64 build for windows and linux
* Fix symbol visibility in linux builds
- Fix soxr failing to download due to sourcefourge
- Only patch zimg on windows targets
- Tell cmake to hide libheif symbols
* Fix Linux .so rpath
- Add lzo dependency
- Publish source for the built libs
- Add warning for missing nasm in tauri.mjs
- Remove ffmpeg install from setup.sh
- Add download logic for our linux ffmpeg bundle in preprep.mjs
* Remove jobs, docker doesn't support this
* Fix typing
* Change ffmpeg references to native deps
- Rename FFMpeg.framework to Spacedrive.framework
- Centralize the macOS native deps build with the windows and linux one
- Change the preprep script to only download our native deps
- Remove old macOS ffmpeg build scripts
* Compress native deps before creating github artifact
- The zip implementation for github artifact does not mantain symlinks and permissions
- Remove conditional protoc, it is now always included
* Don't strip dylibs, it was breaking them
- Only download macOS Framework for darwin targets
- Fix preprep script
- Improve README.md for native-deps
- Fix not finding native-deps src
* Attempt to fix macOS dylib
* Fix macOS dylibs
- Replace lld.ld64 with apple's own linker
- Add stages for building apple's compiler tools to use instead of LLVM ones
* Ensure sourced file exists
* All targets should build now
- Fix environment sourcing in build.sh
- Some minor improvements to cc.sh
- Fix incorrect flag in zlib.sh
- Improve how -f[...] flags are passed to compiler and linker
- Add more stack hardening flags
* We now can support macOS 11.0 on arm64
* Improve macOS Framework generation
- Remove installed unused deps
- Improve cleanup and organization logic in Dockerfile last step
- Move libav* .dll.a to .lib to fix missing files in windows target
- Remove apple tools from /srv folder after installation to prevent their files from being copied by other stage steps
- Create all the necessary symlinks for the macOS targets while building
- Remove symlink logic for macOS target from preprep.mjs
* Remove native-deps from spacedrive repo
- It now resides in https://github.com/spacedriveapp/native-deps
- Modify preprep script to dowload native-deps from new location
- Remove Github API code from scripts (not needed anymore)
- Add flock.mjs to allow running tauri.mjs cleanup as soon as cargo finishes building in linux
* Handle flock not present in system
- Allow macOS to try using flock
* Fix preprep on macOS
* Add script that patch deb to fix errors and warnings raised by lintian
* Fix ctrl+c/ctrl+v typo
* Remove gstreamer1.0-gtk3 from deb dependencies
* eval is evil
* Handle tauri build release with an explicit target in fix-deb.sh
* Preserve environment variables when re-executing fix-deb with sudo
* Only execute fix-deb.sh when building a deb bundle
* Improvements fix-deb.sh
* Improve setup.sh (Add experiemental alpine support)
* attempts
* expo stop using spaces pls
* attempt
* you used to work without this sir
* fixed
* use serde instead of specta
* remove `serde(default`) and update bindings
* fix mobile
* fix types
---------
Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
* Vendor openssl dependency
* Remove bindgen, seems like windows don't like it
- Add OPENSSL_RUST_USE_NASM to force windows to use nasm
* Add native-tls-vendored to reqwest
- Shouldn't change much as the openssl crates override was already dealing with the vendoring anyway
* custom updater with toasts
* new state management + updated router route
* tauri-specific update route
* ref
* update in prod only
* change 'Install' to 'Update'
* fix tsconfig
* desktop tauri
* remove tauri patch
* tauri 1.5
* tauri 1.5
* use tauri script
* native-deps
* Rework preprep and tauri script to better support tauri 1.5
* Update to tauri 1.5.1
- Update workspace and apps/desktop dependencies
- Fix mustache import, @types/mustache is not compatible with ES imports
- Replace arm64 with aarch64 in machineID, they should be treated the same and this simplyfies the code
* Fix tauri updater not building due to missing key
- Fix dmg background not being found
- Generate an adhoc key for tauri updater with it is enabled and the user is doing a prod build
* Fix ctrl+c/ctrl+v typo
* Normalie @tanstack/react-query version through workspace
- Use undici in scripts instead of global fetch
- Fix typecheck
* Fix linux prod and dev builds
- Improve error handling in tauri.mjs
* Normalize dev deps in workspace
- Improve linux shared libs setup
* Fix CI and server docker
* Fix windows
- Remove superfluous envvar
* Attempt to fix server, mobile, deb and release updater
* Attempt to fix deb and mobile again
- Fix type on deb dependency
- Enable release deb for aarch64-unknown-linux-gnu
* Github doesn't have arm runners
- Fix typo in server Dockerfile
* Publish deb and updater artifacts
* remove version from asset name
* update commands
* log release
* Some logs on updater errors
* show updater errors on frontend
* fix desktop ui caching
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
* Move postinstall script to a preprep script
- Fix libheif crate failing to build with our libheif
- Rework CI due to postinstall to preprep changes
* Linux heif build script + Update webp
* Fix ctrl+c/ctrl+v bug
* Improve libheif linux script
- Add support for linux aarch64
- Add CI workflow to build libheif for linux
- Some other misc fixes
* Fix libheif CI requires sudo
* Fix wrong path for libheif build.rs override in Windows
* Fix wrong path manipulations in libheif build script
* 🤦
* Use ubuntu-latest in libheif action
- Specify glib version in target triple to support old distros
- Fix libheif artifact publishing
* Fix artifact upload path again
* Add musl support for libheif
- Remove unused files from libheif artifact
- Add setup logic for libheif in postinstall script
* Build libheif for linux as a shared lib
* Fix meson not building the correct arch
- Add logic to get git branch from githubs CI env vars
* libheif finnaly works on linux
- Make spacedrive binary rpath point to where appimage and deb expects our libs to be
- Add some logic to tauri.js to convince tauri to bundle our shared libs
- Work-around appimage bundling step breaking sometimes
- Add logic to handle sigint in tauri.js to ensure we clean up after ourselves
- Rename postinstall.mjs to setup.mjs
- Add logic to setup.mjs to point our dev build to our shared libs in linux
* Fix windows dekstop dev
- Rename setup.mjs to preprep.mjs
* test cache-factory
* Fix preprep script not parsing the cross compilation target triple and always using the host info to download dependencies
- Fix action env vars not being correctly passed
- Remove possibility to pass multiple targests to rust action
* Don't compile mobile crates on desktop targets
* Remove cache-factory pull_request trigger
* remove patched tauri cli
* Use git plumbing command to get remote branch name
- Fallback to reading .git/HEAD if remote name was not retrieved
* fix type
---------
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
