* First draft on image labeling
* Fixing execution providers for other OSs
* Better error handling and shutdown
* Working with shallow media processor
* bruh
* Fix warnings
* Now hooked to media processor job
* Link desktop app with libonnxruntime to avoid TLS error during startup
* Be able to change models on runtime
Revert to use labels table instead of tags
* A bug on a model-less inference
* Show AI labels on Inspector
- Change yolo inference to use half precision
- Add labels api to core
* Remove LD_PRELOAD
* Fix race condition on model executor shutdown
* Don't load all images in memory moron
* Embeed yolo model in prod build
- Change yolo model path to new one relative to executable
* Disable volume watcher on linux, it was crashing the app
- Invalidate labels when they are updated
* Rust fmt
* Minor changes
* Gate onnxruntime linking to the ai-models feature
* Add build script to sd-server to handle onnxruntime linking workaround
* Move AI stuff to its own crate and normalize deps
* Rust fmt
* Don't regenerate labels unless asked to
* Now blazingly fast
* Bad merge
* Fix
* Fix
* Add backend logic to download extra yolo models
* Add models api route
- Add api call to get available model version
- Add api call to change the model version
* Improve new model download logic
- Add frontend to change image labeler model
* Fix new model downloader
* Fix model select width
* invalidate labels count after media_processor generates a new output
* Rename AI crate and first draft on download notifications
* fix types
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
* basic declared actor manager
* put actors in separate file
* clippy
* hopefully clean up some clippy warnings
---------
Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
* fix horizontal pdf rendering
* don't rotate certain formats (heif*)
* return `u32` from scaling function
* add `avifs` as a valid extension
* add avifs as a valid MIME type
* re-categorize `avifs` as a video (sequence)
* Fix mime-types
* Add Hif extension
- Cannon uses a special extension for heif in its cameras: https://github.com/digipres/digipres.github.io/blob/master/_sources/registries/fdd/fddXML/fdd000525.xml#L213-L221
* hif is actually a heic
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
* fda wip
* clippy
* add tauri invoke fns for FDA
* fda wip
* clippy
* add tauri invoke fns for FDA
* wip
* fda wip
* clippy
* add tauri invoke fns for FDA
* wip
* wip
* wip fda
* remove imports
* hopefully improve FDA
* execute only on macos
* ts
* ts
* Update Platform.tsx
* Update AddLocationButton.tsx
* remove console log
* fix: fda and add unit tests
* temp commit for Jake
* add fda state and keybind handling (so the frontend is kept up to date)
* update FDA
* update imports
* testing purposes
* Jakes work
* fix fda checks
* work in progress (but not working)
* remove dead files
* attempt #2
* !!!temporarily enable devtools in prod
* remove alert
* show FDA screen but don't require it
* add an FDA button to general client settings
* Update AddLocationButton.tsx
* remove dead code
* unused dep
* old errors
* remove import
* dead code
* dead code + typesafety
* eslint
* remove fda dialog references
* remove mp4 vid
* hopefully fix onboarding for non-macos OSes
* shorter nav
---------
Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
* Almost working
* Downgrade libplacebo
- FFMpeg 6.0 uses some now removed deprecated functions
* Use -Oz for zimg
* Fix CI script to run the new ffmpeg build script
* Fix heif step name + Ignore docker cache while building in CI
* Fix Opencl build on linux
* Fix adding incorrect -target argument to linker
- Update zig for windows target
* Disable opengl for ffmpeg, it only uses it as an outdev, not for processing
- Disable opengl and directx for libplacebo, ffmpeg only supports vulkan when using it
- Add WIN32_LEAN_AND_MEAN to global cflags to optimize windows api usage
- Fix 99-heif.sh incorrect bsdtar flag
* Remove WIN32_LEAN_AND_MEAN from global CFLAGS as that was breaking OpenCL build
- Fix Dockerfile step for cleaning up the out dir
- Improve licensing handling
* x86_64 windows and linux builds are working
* Fix aarch64 build for windows and linux
* Fix symbol visibility in linux builds
- Fix soxr failing to download due to sourcefourge
- Only patch zimg on windows targets
- Tell cmake to hide libheif symbols
* Fix Linux .so rpath
- Add lzo dependency
- Publish source for the built libs
- Add warning for missing nasm in tauri.mjs
- Remove ffmpeg install from setup.sh
- Add download logic for our linux ffmpeg bundle in preprep.mjs
* Remove jobs, docker doesn't support this
* Fix typing
* Change ffmpeg references to native deps
- Rename FFMpeg.framework to Spacedrive.framework
- Centralize the macOS native deps build with the windows and linux one
- Change the preprep script to only download our native deps
- Remove old macOS ffmpeg build scripts
* Compress native deps before creating github artifact
- The zip implementation for github artifact does not mantain symlinks and permissions
- Remove conditional protoc, it is now always included
* Don't strip dylibs, it was breaking them
- Only download macOS Framework for darwin targets
- Fix preprep script
- Improve README.md for native-deps
- Fix not finding native-deps src
* Attempt to fix macOS dylib
* Fix macOS dylibs
- Replace lld.ld64 with apple's own linker
- Add stages for building apple's compiler tools to use instead of LLVM ones
* Ensure sourced file exists
* All targets should build now
- Fix environment sourcing in build.sh
- Some minor improvements to cc.sh
- Fix incorrect flag in zlib.sh
- Improve how -f[...] flags are passed to compiler and linker
- Add more stack hardening flags
* We now can support macOS 11.0 on arm64
* Improve macOS Framework generation
- Remove installed unused deps
- Improve cleanup and organization logic in Dockerfile last step
- Move libav* .dll.a to .lib to fix missing files in windows target
- Remove apple tools from /srv folder after installation to prevent their files from being copied by other stage steps
- Create all the necessary symlinks for the macOS targets while building
- Remove symlink logic for macOS target from preprep.mjs
* Remove native-deps from spacedrive repo
- It now resides in https://github.com/spacedriveapp/native-deps
- Modify preprep script to dowload native-deps from new location
- Remove Github API code from scripts (not needed anymore)
- Add flock.mjs to allow running tauri.mjs cleanup as soon as cargo finishes building in linux
* Handle flock not present in system
- Allow macOS to try using flock
* Fix preprep on macOS
* Add script that patch deb to fix errors and warnings raised by lintian
* Fix ctrl+c/ctrl+v typo
* Remove gstreamer1.0-gtk3 from deb dependencies
* eval is evil
* Handle tauri build release with an explicit target in fix-deb.sh
* Preserve environment variables when re-executing fix-deb with sudo
* Only execute fix-deb.sh when building a deb bundle
* Improvements fix-deb.sh
* Improve setup.sh (Add experiemental alpine support)
* basic FDA start
* FDA checks and prompt
* lockfile
* misc clippy
* Some small warnings
---------
Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
* let me create pr
* a whole lotta changes
* split `p2p_manager.rs` into smaller files
* the arcpocalypse is over
* minor generic cleanup
* wip removing 'MetadataManager'
* more wip
* wip: i am changing branch
* discovery2 -> discovery
* make it somewhat compile
* more wip
* wip: reassembling manager stream
* state more goodly
* wip
* more wip
* removing generic from sd_p2p::Manager
* reassemble networked libraries
* wip: hooking back up mDNS
* multi-flume wip
* contain bad code to a single file
* p2p_manager_actor + split handlers into file per operation
* cleanup after restructure
* cleaning up more
* wip: reenable resync
* wip: remote identity in connection payload
* track connected clients (required for `service.rs`)
* a big ass iterator
* working towards finishing `service.rs`
* service shutdown
* hook up listen channel in service
* fix address resolution
* merge nlm stuff into LibrariesService
* finish library to service mapping
* less footguns in p2p - seal `PeerId`
* fix previous pr
* p2p state rspc query
* send node events to the frontend
* minor
* wip
* more worky, less crashy
* make spacedrop work + debug state
* fix mdns expiry
* clippy
* other clippy
* remove broadcast type
* merge conflict issue
* let me create pr
* a whole lotta changes
* split `p2p_manager.rs` into smaller files
* the arcpocalypse is over
* minor generic cleanup
* wip removing 'MetadataManager'
* more wip
* wip: i am changing branch
* discovery2 -> discovery
* make it somewhat compile
* more wip
* wip: reassembling manager stream
* state more goodly
* wip
* more wip
* removing generic from sd_p2p::Manager
* reassemble networked libraries
* wip: hooking back up mDNS
* multi-flume wip
* contain bad code to a single file
* p2p_manager_actor + split handlers into file per operation
* cleanup after restructure
* cleaning up more
* wip: reenable resync
* wip: remote identity in connection payload
* track connected clients (required for `service.rs`)
* a big ass iterator
* working towards finishing `service.rs`
* service shutdown
* hook up listen channel in service
* fix address resolution
* merge nlm stuff into LibrariesService
* finish library to service mapping
* less footguns in p2p - seal `PeerId`
* fix previous pr
* p2p state rspc query
* send node events to the frontend
* minor
* wip
* more worky, less crashy
* make spacedrop work + debug state
* fix mdns expiry
* clippy
* other clippy
* remove old tests
* add tests back
---------
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
* Renaming error type on ffmpeg subcrate
* Version manager overhaul
* Reworked thumbnail actor
* Updating sharding scheme
* New migration system for thumbnails
* Updating search to new actor
* Updating custom_uri to new thumbnail improvements
* Updating library to new thumbnail stuff
* LibraryId type alias
* Updating indexer to new thumbnail actor
* Updating watcher to new thumbnail actor
* Update location manager to use LibraryId type alias
* Updating location metadata to new LibraryId type alias
* New LocationPubId type alias
* Updating ephemeral walker to new thumbnail stuff
* Updating media processor to new thumbnail actor
* New thumbnailer actor state manager
* Introducing the concept of job phases
* Segregating the thumbnailer actor worker fn
* Fixes on job pausing
* Processing batches with progress reporting
* Updated actor
* Updated media processor
* Small tweaks
* Updating non indexed walker
* Changing a UI string
Dependencies overhaul
- Update dependencies for all projects (except Mobile-only deps)
- Remove unused dependencies from all projects (except Mobile-only deps)
- Fix Storybook failing to import sd/ui style
- Add Node 21 as not supported due to sass-loader not working on it yet
- Add work-around for new rook version requiring webpack specific global property
- Fix landing dev not working due to missing default env value on dev
- Fix some incorrect uses of phosphor-icons non server side icons on server components on landing
- Fix some incorrect uses of phosphor-icons server side icon on client components on landing
- Fix landing fail to build on dev due to always required a Github Token to get the latest release
- Fix new Next.js version not suporting Response.redirect due to immutable Headers
- Add Gitlab as social link for teams page
- Update Vítor's team photo
- Add Vítor's twitter link
- Fix some warning due to missing useEffect dependencies
- Remove test-files dir
- Fix QuickPreview unblurred buttons in Linux
- Formatting
* New vscode task to start developing
* Updating db in case of library updates just in case
* Done
* Forgot to remove some debug logs
* Rust fmt
* Saving thumbnails processing state on app shutdown
* bruh
* clippy -_-
* migrations
* alter the migration so it just renames the `dimensions` field (no db re-creation required)
* remove confusing semver addition for `libheif-sys`
* remove warning on the migration as it's just a rename
* add sort opts for resolution and date image was truly taken
* major serde ckeanup & add epoch_time and pixel_count
* rename, cleanup et optimise
* clippy
* ignore this mess
* bindings
* add explanation to schema
* comment out dt test
* better comment and WIP time
* cleanup rust
* failed timezone attempt
* remove image resolution as a sort by option
* update schema (and rename the dimensions table instead of dropping it)
* just show raw date
* add comments and update bindings
* fix migration hopefully
* fix broken migration
---------
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Jamie Pine <ijamespine@me.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
* custom updater with toasts
* new state management + updated router route
* tauri-specific update route
* ref
* update in prod only
* change 'Install' to 'Update'
* fix tsconfig
* desktop tauri
* remove tauri patch
* tauri 1.5
* tauri 1.5
* use tauri script
* native-deps
* Rework preprep and tauri script to better support tauri 1.5
* Update to tauri 1.5.1
- Update workspace and apps/desktop dependencies
- Fix mustache import, @types/mustache is not compatible with ES imports
- Replace arm64 with aarch64 in machineID, they should be treated the same and this simplyfies the code
* Fix tauri updater not building due to missing key
- Fix dmg background not being found
- Generate an adhoc key for tauri updater with it is enabled and the user is doing a prod build
* Fix ctrl+c/ctrl+v typo
* Normalie @tanstack/react-query version through workspace
- Use undici in scripts instead of global fetch
- Fix typecheck
* Fix linux prod and dev builds
- Improve error handling in tauri.mjs
* Normalize dev deps in workspace
- Improve linux shared libs setup
* Fix CI and server docker
* Fix windows
- Remove superfluous envvar
* Attempt to fix server, mobile, deb and release updater
* Attempt to fix deb and mobile again
- Fix type on deb dependency
- Enable release deb for aarch64-unknown-linux-gnu
* Github doesn't have arm runners
- Fix typo in server Dockerfile
* Publish deb and updater artifacts
* remove version from asset name
* update commands
* log release
* Some logs on updater errors
* show updater errors on frontend
* fix desktop ui caching
---------
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
* derive `Type` on `*Extension`
* update `image`, `webp` and add `rspc::Error` support to the `sd-images` crate
* rename function for verbosity
* cleanup clippy allows and use scaling fn for svg conversions
* remove dead comment
* specify `generic` image extensions and add fn to enumerate all compatible extensions
* re-exports and inline hot provided method
* another inline on a provided method
* `formatter` -> `handler` and return an explicit error if the extension isn't supported, and add `convert_image()`
* `get_ext()` inlined helper fn
* add mutation for conversion and query for getting all supported extensions
* fix `svg` scaling and generation
* update bindings
* rm old const
* temporary test for all exts
* verbosity and no manual drop
* remove dbg test
* big enum and remove `cur` extension as `image` doesn't support it
* add huge extension enum
* hopefully get path from file path id and location id
* main merge fix
* chore: update toml and lockfile
* security: update webp
* comment entire crypto crate for build times
* add bincode, huge enum and custom ser/de
* Revert "comment entire crypto crate for build times"
This reverts commit 458addaad6.
* lockfile
* theoretically working ser/de
* add svg scaling
* fix pdf matching bug and use options
* move image scaling fn
* major consts overhaul with good ser/de
* disable heif checking for now due to new trait api
* make the generic handler truly generic
* fix pdf types
* fix pdf scaling so that it's 120dpi A4 paper size by default - height/aspect res are maintained
* heavy cleanup
* note to myself
* bindings & update core to use the scaling fn from the images crate (reducing code dupes)
* move vulnerable `aovec` to be a dev dep so it stays out of release builds (it's only used in tests)
* remoev exif crate from the core as it's handled in a sep. crate
* silence a clippy warning and update lockfile
* lower the maximum file size a slight amount
* bindings and minor dep bumps
* Some improvements
---------
Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
* - added local section to sidebar
- added spacedrop screen, showing local peers
- added placeholder network screen
-removed unused swift package
- created a watcher for system volumes to invalidate ui when drives are added/removed
* clouds
* fix more imports
* see more
* open location if volume is location
* gen assets
* remove log
* [ENG-939, ENG-1173] PDF Thumbnails (#1242)
* sd-pdf
* Process PDF blocking render inside a spawn_blocking
- Load a single global Pdfium instance
* Migrate pdf thumb logic to sd-images
- Replace block_in_place with spawn_blocking
- Only load LibHeif once
- Allow thumbnailer (both indexed and non-indexed locations) to process documents
- Disable loading pdf viewer in Inspection in favour of loading it's thumbnail
* Try to load pdfium lib from absolute path
* Revert removed import due to rebase
* Small nitpick and some warnings
---------
Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
* [ENG-888] Media view should show current folder downward (#1437)
* Done but ugly
* layout
* Now with a select
---------
Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
* add cool folder thing to inspector + stuff
* fix text color
* fix lock
* fix typescript
* fix ts
---------
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
* sd-pdf
* Process PDF blocking render inside a spawn_blocking
- Load a single global Pdfium instance
* Migrate pdf thumb logic to sd-images
- Replace block_in_place with spawn_blocking
- Only load LibHeif once
- Allow thumbnailer (both indexed and non-indexed locations) to process documents
- Disable loading pdf viewer in Inspection in favour of loading it's thumbnail
* Try to load pdfium lib from absolute path
* Revert removed import due to rebase
* Small nitpick and some warnings
---------
Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
* Move postinstall script to a preprep script
- Fix libheif crate failing to build with our libheif
- Rework CI due to postinstall to preprep changes
* Linux heif build script + Update webp
* Fix ctrl+c/ctrl+v bug
* Improve libheif linux script
- Add support for linux aarch64
- Add CI workflow to build libheif for linux
- Some other misc fixes
* Fix libheif CI requires sudo
* Fix wrong path for libheif build.rs override in Windows
* Fix wrong path manipulations in libheif build script
* 🤦
* Use ubuntu-latest in libheif action
- Specify glib version in target triple to support old distros
- Fix libheif artifact publishing
* Fix artifact upload path again
* Add musl support for libheif
- Remove unused files from libheif artifact
- Add setup logic for libheif in postinstall script
* Build libheif for linux as a shared lib
* Fix meson not building the correct arch
- Add logic to get git branch from githubs CI env vars
* libheif finnaly works on linux
- Make spacedrive binary rpath point to where appimage and deb expects our libs to be
- Add some logic to tauri.js to convince tauri to bundle our shared libs
- Work-around appimage bundling step breaking sometimes
- Add logic to handle sigint in tauri.js to ensure we clean up after ourselves
- Rename postinstall.mjs to setup.mjs
- Add logic to setup.mjs to point our dev build to our shared libs in linux
* Fix windows dekstop dev
- Rename setup.mjs to preprep.mjs
* test cache-factory
* Fix preprep script not parsing the cross compilation target triple and always using the host info to download dependencies
- Fix action env vars not being correctly passed
- Remove possibility to pass multiple targests to rust action
* Don't compile mobile crates on desktop targets
* Remove cache-factory pull_request trigger
* remove patched tauri cli
* Use git plumbing command to get remote branch name
- Fallback to reading .git/HEAD if remote name was not retrieved
* fix type
---------
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
* feature/target gate the heif stuff
* more gating
* make clippy happy
* `--cfg linux_heif` override and dep bump
* disable `libheif` on windows temporarily too
* bring in rand deps
* SUPER crude pluscode attempt
* working pluscodes!
* some cleanup
* `encodeURIComponent` for URL escaping
---------
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
* impl `Display` and a custom serde serializer
* remove the + as rust already shortens coords to 8SF
* remove manual serde impl and string conversion
* add `formatOrientation` to the `MediaData` part of the inspector
* make requested changes
---------
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
* sd-images crate which will support raw/dng, bmp, etc
* more work on the image formatter
* re-work `sd-images`, add svg support, r/g/b and r/g/b/a HEIF image support (will all be async again soon)
* remove `ImageFormatter`, add note about r/g/b/(a) heif impl
* implement the image formatter
* rename the conversion trait and minor cleanups
* isolate heif feature and major cleanup
* very untested raw support
* change fn name to `from_path` (a lot more idiomatic)
* clean up orientation fixing
* heif is no longer forbidden (linux has good heif)
also all extensions are correctly matched in lowercase
* fix builds, ext matching, feature gating
* attempt to fix svg handling?
* raw attempt, quite a few errors
* add comment
* new (untested) attempt
* remove `raw` stuff for now
* replace `sd-svg` with a `ToImage` `SvgHandler` impl
* add some simple math to appropriately scale thumbnails (and bmp/ico support)
* add comments regarding how the math works for image thumbs
* rename the trait to `ImageHandler`
* Add range to `SpaceblockRequest`
* Determine if file should be served from P2P
* Feature flag serveFromP2P + fix `ServeFrom` logic
* wip: yeet files over P2P
* Just work, argh
* restore backend feature flags after restart
* cleanup
* apply clippy lints to the entire ffmpeg crate in hopes of making it safer
* wording
* revert the prisma changes
* fix typo
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
* fix another typo
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
* remove so many `#[must_use]`s
* fix bad merge and hopefully clippy
* clippy please work i beg
* make HEIF_EXTENSIONS always available
---------
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
