Commit graph

14 commits

Author SHA1 Message Date
jake 471db4a691
Clippy the entire repo (#2195)
* fix(clippy): for the entire repo

* fix: make function non-const fn
2024-03-12 11:27:23 +00:00
Ericson "Fogo" Soares 68ca2382fd
[ENG-1627] Move all jobs and actors to an "Old" namespace (#2157)
* Bunch of new warns due to a stronger clippy config

* Moving old processing stuff to a new namespace
Also fixing a bunch of typos through the entire codebase.

* Rustfmt
2024-03-04 18:57:35 +00:00
Ericson "Fogo" Soares 7c90bcb95b
[ENG-1479] AI Prototype (#1845)
* First draft on image labeling

* Fixing execution providers for other OSs

* Better error handling and shutdown

* Working with shallow media processor

* bruh

* Fix warnings

* Now hooked to media processor job

* Link desktop app with libonnxruntime to avoid TLS error during startup

* Be able to change models on runtime
Revert to use labels table instead of tags

* A bug on a model-less inference

* Show AI labels on Inspector
 - Change yolo inference to use half precision
 - Add labels api to core

* Remove LD_PRELOAD

* Fix race condition on model executor shutdown

* Don't load all images in memory moron

* Embeed yolo model in prod build
 - Change yolo model path to new one relative to executable

* Disable volume watcher on linux, it was crashing the app
 - Invalidate labels when they are updated

* Rust fmt

* Minor changes

* Gate onnxruntime linking to the ai-models feature

* Add build script to sd-server to handle onnxruntime linking workaround

* Move AI stuff to its own crate and normalize deps

* Rust fmt

* Don't regenerate labels unless asked to

* Now blazingly fast

* Bad merge

* Fix

* Fix

* Add backend logic to download extra yolo models

* Add models api route
 - Add api call to get available model version
 - Add api call to change the model version

* Improve new model download logic
 - Add frontend to change image labeler model

* Fix new model downloader

* Fix model select width

* invalidate labels count after media_processor generates a new output

* Rename AI crate and first draft on download notifications

* fix types

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-12-19 09:28:57 +00:00
jake 8e8e355b76
[ENG-1026] Full Disk Access (#1778)
* basic FDA start

* FDA checks and prompt

* lockfile

* misc clippy

* Some small warnings

---------

Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
2023-11-15 19:59:50 +00:00
Oscar Beaumont 6941912891
[ENG-1399] Upgrade Rust workspace + IPv6 mDNS (#1729)
Upgrade Rust workspace + IPv6 mDNS
2023-11-04 10:20:22 +00:00
jake 078490cdd5
[ENG-1143] Media View sort by date taken (#1390)
* clippy -_-

* migrations

* alter the migration so it just renames the `dimensions` field (no db re-creation required)

* remove confusing semver addition for `libheif-sys`

* remove warning on the migration as it's just a rename

* add sort opts for resolution and date image was truly taken

* major serde ckeanup & add epoch_time and pixel_count

* rename, cleanup et optimise

* clippy

* ignore this mess

* bindings

* add explanation to schema

* comment out dt test

* better comment and WIP time

* cleanup rust

* failed timezone attempt

* remove image resolution as a sort by option

* update schema (and rename the dimensions table instead of dropping it)

* just show raw date

* add comments and update bindings

* fix migration hopefully

* fix broken migration

---------

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Jamie Pine <ijamespine@me.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-10-11 00:58:59 +00:00
jake fb8af0832b
[ENG-1096] Image Conversions (#1364)
* derive `Type` on `*Extension`

* update `image`, `webp` and add `rspc::Error` support to the `sd-images` crate

* rename function for verbosity

* cleanup clippy allows and use scaling fn for svg conversions

* remove dead comment

* specify `generic` image extensions and add fn to enumerate all compatible extensions

* re-exports and inline hot provided method

* another inline on a provided method

* `formatter` -> `handler` and return an explicit error if the extension isn't supported, and add `convert_image()`

* `get_ext()` inlined helper fn

* add mutation for conversion and query for getting all supported extensions

* fix `svg` scaling and generation

* update bindings

* rm old const

* temporary test for all exts

* verbosity and no manual drop

* remove dbg test

* big enum and remove `cur` extension as `image` doesn't support it

* add huge extension enum

* hopefully get path from file path id and location id

* main merge fix

* chore: update toml and lockfile

* security: update webp

* comment entire crypto crate for build times

* add bincode, huge enum and custom ser/de

* Revert "comment entire crypto crate for build times"

This reverts commit 458addaad6.

* lockfile

* theoretically working ser/de

* add svg scaling

* fix pdf matching bug and use options

* move image scaling fn

* major consts overhaul with good ser/de

* disable heif checking for now due to new trait api

* make the generic handler truly generic

* fix pdf types

* fix pdf scaling so that it's 120dpi A4 paper size by default - height/aspect res are maintained

* heavy cleanup

* note to myself

* bindings & update core to use the scaling fn from the images crate (reducing code dupes)

* move vulnerable `aovec` to be a dev dep so it stays out of release builds (it's only used in tests)

* remoev exif crate from the core as it's handled in a sep. crate

* silence a clippy warning and update lockfile

* lower the maximum file size a slight amount

* bindings and minor dep bumps

* Some improvements

---------

Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
2023-10-10 02:57:37 +00:00
Oscar Beaumont 94d8f61de9
Update Rust dependencies (#1314)
* Update most of Rust dependencies

* downgrade notify

* a

* fix Linux?

* Android is painful

* update build.gradle

* The lord Clippy has spoken

* Bruh

---------

Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Ericson "Fogo" Soares <ericson.ds999@gmail.com>
2023-09-12 14:03:25 +00:00
jake f421b6883f
Fix builds and make String -> PlusCode validation more generic (Google support) (#1326)
fix builds and make string -> pc matching more generic, so it works with google's format too
2023-09-11 11:35:09 +00:00
jake e2a0878e7a
[ENG-1068] Plus codes / Open Location Code support (#1324)
* bring in rand deps

* SUPER crude pluscode attempt

* working pluscodes!

* some cleanup

* `encodeURIComponent` for URL escaping

---------

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2023-09-11 04:34:24 +00:00
jake d435141496
[ENG-1084] Correctly format Orientation EXIF data in the UI (#1313)
* impl `Display` and a custom serde serializer

* remove the + as rust already shortens coords to 8SF

* remove manual serde impl and string conversion

* add `formatOrientation` to the `MediaData` part of the inspector

* make requested changes

---------

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-09-08 12:21:32 +00:00
jake 7edcbb15d6
[ENG-621][ENG-1074] Trait-based image conversion overhaul (#1307)
* sd-images crate which will support raw/dng, bmp, etc

* more work on the image formatter

* re-work `sd-images`, add svg support, r/g/b and r/g/b/a HEIF image support (will all be async again soon)

* remove `ImageFormatter`, add note about r/g/b/(a) heif impl

* implement the image formatter

* rename the conversion trait and minor cleanups

* isolate heif feature and major cleanup

* very untested raw support

* change fn name to `from_path` (a lot more idiomatic)

* clean up orientation fixing

* heif is no longer forbidden (linux has good heif)

also all extensions are correctly matched in lowercase

* fix builds, ext matching, feature gating

* attempt to fix svg handling?

* raw attempt, quite a few errors

* add comment

* new (untested) attempt

* remove `raw` stuff for now

* replace `sd-svg` with a `ToImage` `SvgHandler` impl

* add some simple math to appropriately scale thumbnails (and bmp/ico support)

* add comments regarding how the math works for image thumbs

* rename the trait to `ImageHandler`
2023-09-07 14:08:17 +00:00
Ericson "Fogo" Soares 3b8541ef54
[ENG-1028] Media data for ephemeral locations (#1287)
Introducing new getEphemeralMediaData
Fixing some minor stuff
Running pnpm format

Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
2023-09-04 08:28:45 +00:00
jake c674b7107a
[ENG-288, ENG-601] Media data (image) extraction, thumbnail orientation fix (#1099)
* basic layout

* lock

* add codec to image

* very messy wip

* rm that

* lock and toml

* working perfect exif extraction

* formatting

* migration and formatting

* mostly working

* fix

* set date created properly

* fix tsc

* working media data creation

* fix bad main merge? sorry brendan

* schema, migrations, bindings

* working exif data extraction

* why won't it work

* update migrations

* fix bad merge

* final cleanup

* cleanup migrations

* remove test (that was purely used for testing my code)

* working media data pulling, correct thumbnail orientation

* slightly optimise thumbnail rotation

* rename location to prevent specta clashes

* further improvements (location parsing is still broken)

* fix coordinate parsing i think

* rspc add some todos regarding final cleanup

* further thoughts

* major upgrades

* Some improved handling of errors and async calls

* accordion component

* heavily wip flash refactor

* fix builds all because of a (terrible) merge/move

* annoying missing newline

* i really hate exif

* remove dead code

* further flash progress :D

* docs(media-data): clarification

* minor cleanup

* cleanup and some async removal

* fix location parsing

* remove async (will do proper impl for async eventually) and major cleanup

* big W

* clippy and `FlashMode::Unknown` instead of `Invalid`

* add `NIKON` gps ref support

* comments about gps refs

* commit the submodule -_-

* major cleanup & composite image support

* remove old test image

* major cleanup and easier to use API

* remove old consts

* move `ExifReader` to dedicated module

* Media Data Extractor job and shallow job

* Extracting media data on watcher

* report no exif data on file gracefully

* cleanup errors and doctests

* Merging thumbnailer and media data extractor

* Job name and some strings in UI

* remove reliance on `sd-prisma` within the media data crate

* rename query to be more specific

* custom serializer for `MediaTime`

* tweak to format utc -> naive correctly

* generate migrations

* comment out duration in mobile

* delete test-assets folder

* all optional fields

* fix migrations again

* make requested name changes

* make further requested changes

* remove erroneous files from another wip branch

* updates procedures

* use strings where appropriate

* regen pnpm-lock

* add base layouts for video and audio media data

* use appropriate data types in schema and add audio media data framework

* make requested changes

* general cleanup and renaming of enum

* cleanup media data api

* rename media metadata type

* finishing touches

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-08-29 17:02:55 +00:00