Commit graph

740 commits

Author SHA1 Message Date
Oscar Beaumont f67b7e807a
[ENG-1291] enable/disable P2P & configurable port (#1609)
* it works pog

* UI for when networking is disabled

* Fix Clippy issues

* clippy tings
2023-10-18 03:52:54 +00:00
Oscar Beaumont 1b856848e8
[ENG-1310, ENG-1300] Spacedrop better testing + fix zero-sized files (#1606)
* unit tests save lives

* Support for zero-sized files

* fix: clippy

---------

Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
2023-10-17 06:12:38 +00:00
Oscar Beaumont a7ad24c641
[ENG-625] Spacedrop improvements (#1447)
* Fist fight with the abstraction

* Prevent opening multiple Spacedrop dialogues

* Infinite renders = infinite power

* Comment be gone

* Spacedrop multiple files

* correlate connection id's

* fixes

* thanks Typescript

* Clippy errors + fix cfg's from main
2023-10-17 04:31:42 +00:00
Jamie Pine 3709ab4883
[END-1309] Fix explorer query invalidation (#1604)
fix
2023-10-17 03:54:41 +00:00
Ericson "Fogo" Soares bfe5a65949
[ENG-1267] Move thumbnails generation away from media processor (#1585)
* New vscode task to start developing

* Updating db in case of library updates just in case

* Done

* Forgot to remove some debug logs

* Rust fmt

* Saving thumbnails processing state on app shutdown

* bruh
2023-10-17 02:09:36 +00:00
Vítor Vasconcellos 65f13bf204
[ENG-1268] Fix custom indexer rule crashing location scanning (#1584)
Fix custom indexer rule crashing location scanning
2023-10-14 06:33:37 +00:00
Oscar Beaumont 39744b5c55
Reduce production logs + button on error page for logs (#1565)
* reduce prod logs

* Open logs dir button on error page
2023-10-13 08:50:01 +00:00
Brendan Allan 25fa1f0974
[ENG-1255] Feedback popover with new API (#1532)
* feedback popover with new api

* allow unauthed feedback

* add login information

* allow no auth

* fix hooks

* remove emoji on reset

* show cancel on left of login button
2023-10-13 08:08:07 +00:00
Ericson "Fogo" Soares 996361f081
[ENG-1160] Opening a location while indexing causes a lot of jobs to be spawned (#1554)
Solved, but missing a frontend error message
2023-10-13 02:47:13 +00:00
Brendan Allan eb208a9e17
ffmpeg error (#1556) 2023-10-13 10:30:03 +08:00
Ericson "Fogo" Soares 20fde557d4
[ENG-1225] Put brakes on thumbnail generation for ephemeral locations (#1523)
* Done

* Small nitpick
2023-10-12 04:02:27 +00:00
Brendan Allan 287ccad11f
Wrong invalidate args (#1493)
wrong invalidate args
2023-10-11 09:29:29 +00:00
Ericson "Fogo" Soares bc0f4787bd
[ENG-1222] udevadm doesn't exist in Docker container (#1471)
* Changing how we watch for volumes

* Small nitpick
2023-10-11 05:59:15 +00:00
jake 078490cdd5
[ENG-1143] Media View sort by date taken (#1390)
* clippy -_-

* migrations

* alter the migration so it just renames the `dimensions` field (no db re-creation required)

* remove confusing semver addition for `libheif-sys`

* remove warning on the migration as it's just a rename

* add sort opts for resolution and date image was truly taken

* major serde ckeanup & add epoch_time and pixel_count

* rename, cleanup et optimise

* clippy

* ignore this mess

* bindings

* add explanation to schema

* comment out dt test

* better comment and WIP time

* cleanup rust

* failed timezone attempt

* remove image resolution as a sort by option

* update schema (and rename the dimensions table instead of dropping it)

* just show raw date

* add comments and update bindings

* fix migration hopefully

* fix broken migration

---------

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Jamie Pine <ijamespine@me.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-10-11 00:58:59 +00:00
Vítor Vasconcellos 9ca22733ad
[ENG-1086, ENG-1087] Add error and loading indicator to thumbnail (#1448)
* Add onLoad and onError listeners to FileThumb
 - Show toast when Original file fails loading in QuickPreview
 - Formatting

* Fix close toast not working when QuickPreview was open

* Fix panic when serving unsupported files

* Formatting & lint
2023-10-11 00:52:21 +00:00
Jamie Pine 01dbc2caf7
[ENG-864] Create directory (#1458)
* folder

* wrote function

* Abstracting duplicate on file name

* Spliting between ephemeral and indexed

* Now with more type safety

* Forgot to prep

* location + path based

* bruh

* link frontend + error toast

* strip main separator

* dumb

* bruh

* create directory

Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>

* make some reactivity happen

---------

Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>
2023-10-11 00:47:56 +00:00
jake fb8af0832b
[ENG-1096] Image Conversions (#1364)
* derive `Type` on `*Extension`

* update `image`, `webp` and add `rspc::Error` support to the `sd-images` crate

* rename function for verbosity

* cleanup clippy allows and use scaling fn for svg conversions

* remove dead comment

* specify `generic` image extensions and add fn to enumerate all compatible extensions

* re-exports and inline hot provided method

* another inline on a provided method

* `formatter` -> `handler` and return an explicit error if the extension isn't supported, and add `convert_image()`

* `get_ext()` inlined helper fn

* add mutation for conversion and query for getting all supported extensions

* fix `svg` scaling and generation

* update bindings

* rm old const

* temporary test for all exts

* verbosity and no manual drop

* remove dbg test

* big enum and remove `cur` extension as `image` doesn't support it

* add huge extension enum

* hopefully get path from file path id and location id

* main merge fix

* chore: update toml and lockfile

* security: update webp

* comment entire crypto crate for build times

* add bincode, huge enum and custom ser/de

* Revert "comment entire crypto crate for build times"

This reverts commit 458addaad6.

* lockfile

* theoretically working ser/de

* add svg scaling

* fix pdf matching bug and use options

* move image scaling fn

* major consts overhaul with good ser/de

* disable heif checking for now due to new trait api

* make the generic handler truly generic

* fix pdf types

* fix pdf scaling so that it's 120dpi A4 paper size by default - height/aspect res are maintained

* heavy cleanup

* note to myself

* bindings & update core to use the scaling fn from the images crate (reducing code dupes)

* move vulnerable `aovec` to be a dev dep so it stays out of release builds (it's only used in tests)

* remoev exif crate from the core as it's handled in a sep. crate

* silence a clippy warning and update lockfile

* lower the maximum file size a slight amount

* bindings and minor dep bumps

* Some improvements

---------

Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
2023-10-10 02:57:37 +00:00
nikec 0d837d2317
[ENG-1138] Fix list view thumb flicker (#1436)
* rename

* don't forget the table

* Update RenameTextBox.tsx
2023-10-09 13:44:31 +00:00
Jamie Pine ed751a46d0
[ENG-944] Sidebar UX Improvements (#1270)
* - added local section to sidebar
- added spacedrop screen, showing local peers
- added placeholder network screen
-removed unused swift package
- created a watcher for system volumes to invalidate ui when drives are added/removed

* clouds

* fix more imports

* see more

* open location if volume is location

* gen assets

* remove log

* [ENG-939, ENG-1173] PDF Thumbnails (#1242)

* sd-pdf

* Process PDF blocking render inside a spawn_blocking
 - Load a single global Pdfium instance

* Migrate pdf thumb logic to sd-images
 - Replace block_in_place with spawn_blocking
 - Only load LibHeif once
 - Allow thumbnailer (both indexed and non-indexed locations) to process documents
 - Disable loading pdf viewer in Inspection in favour of loading it's thumbnail

* Try to load pdfium lib from absolute path

* Revert removed import due to rebase

* Small nitpick and some warnings

---------

Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>

* [ENG-888] Media view should show current folder downward (#1437)

* Done but ugly

* layout

* Now with a select

---------

Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>

* add cool folder thing to inspector + stuff

* fix text color

* fix lock

* fix typescript

* fix ts

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-10-09 10:11:23 +00:00
jake 968e37afcd
[ENG-1176] Append (x) to the end of files instead of overwriting (#1425)
* append ` (x)` to files when duplicating, renaming or pasting instead of overwriting

* cleanup commented code

* fix renames

* rustfmt

* remove unused item

* Small tech debts and some nitpicks

* Bug with appending number on duplicates

* A bug on my new impl

---------

Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
2023-10-07 11:59:58 +00:00
Ericson "Fogo" Soares 97a86a9fbb
[ENG-888] Media view should show current folder downward (#1437)
* Done but ugly

* layout

* Now with a select

---------

Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
2023-10-06 20:46:26 +00:00
Vítor Vasconcellos 556cf1df63
[ENG-939, ENG-1173] PDF Thumbnails (#1242)
* sd-pdf

* Process PDF blocking render inside a spawn_blocking
 - Load a single global Pdfium instance

* Migrate pdf thumb logic to sd-images
 - Replace block_in_place with spawn_blocking
 - Only load LibHeif once
 - Allow thumbnailer (both indexed and non-indexed locations) to process documents
 - Disable loading pdf viewer in Inspection in favour of loading it's thumbnail

* Try to load pdfium lib from absolute path

* Revert removed import due to rebase

* Small nitpick and some warnings

---------

Co-authored-by: Ericson Fogo Soares <ericson.ds999@gmail.com>
2023-10-06 20:41:22 +00:00
Ericson "Fogo" Soares 774e5b7839
[ENG-1124 | ENG-1154] Ephemeral location bug: Opening files | Path should always be in present in Inspector (#1401)
* New functions to open ephemeral files

* Review items for ephemeral files

* Open and OpenWith context menu for ephemeral paths

* Some warnings

* Fixing inspector

* Fixing windows

* Format

* Messy rebase

* Fix macos

* Cargo fmt

* Removing devtools as it can be opened with keybind

* Fix macos

* Separating ext for ephemeral files on inspector

* Fixing bad rebase

* Removing rename button from quickpreview for ephemeral files

* Quick Preview for ephemeral files

* Requested changes
2023-10-05 07:21:37 +00:00
Oscar Beaumont 9c6e418e0f
[ENG-982] Can't load original after renaming (#1430)
* we need new invalidation system

* clippy

---------

Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
2023-10-05 04:35:11 +00:00
Vítor Vasconcellos d75a6e9b12
[ENG-1054] libheif for Linux (plus fixes) + Webp update (#1405)
* Move postinstall script to a preprep script
 - Fix libheif crate failing to build with our libheif
 - Rework CI due to postinstall to preprep changes

* Linux heif build script + Update webp

* Fix ctrl+c/ctrl+v bug

* Improve libheif linux script
 - Add support for linux aarch64
 - Add CI workflow to build libheif for linux
 - Some other misc fixes

* Fix libheif CI requires sudo

* Fix wrong path for libheif build.rs override in Windows

* Fix wrong path manipulations in libheif build script

* 🤦

* Use ubuntu-latest in libheif action
 - Specify glib version in target triple to support old distros
 - Fix libheif artifact publishing

* Fix artifact upload path again

* Add musl support for libheif
 - Remove unused files from libheif artifact
 - Add setup logic for libheif in postinstall script

* Build libheif for linux as a shared lib

* Fix meson not building the correct arch
 - Add logic to get git branch from githubs CI env vars

* libheif finnaly works on linux
 - Make spacedrive binary rpath point to where appimage and deb expects our libs to be
 - Add some logic to tauri.js to convince tauri to bundle our shared libs
 - Work-around appimage bundling step breaking sometimes
 - Add logic to handle sigint in tauri.js to ensure we clean up after ourselves
 - Rename postinstall.mjs to setup.mjs
 - Add logic to setup.mjs to point our dev build to our shared libs in linux

* Fix windows dekstop dev
 - Rename setup.mjs to preprep.mjs

* test cache-factory

* Fix preprep script not parsing the cross compilation target triple and always using the host info to download dependencies
 - Fix action env vars not being correctly passed
 - Remove possibility to pass multiple targests to rust action

* Don't compile mobile crates on desktop targets

* Remove cache-factory pull_request trigger

* remove patched tauri cli

* Use git plumbing command to get remote branch name
 - Fallback to reading .git/HEAD if remote name was not retrieved

* fix type

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-10-03 17:31:33 +00:00
Oscar Beaumont d6d5f1505c
[ENG-1141] Show running jobs first (#1400)
* Fix Rust panics when opening HEIC in `QuickPreview`

* Sort it

* improve sorting
2023-10-03 16:26:59 +00:00
Brendan Allan 3786e93a78
Remove client secret from apps (#1423)
* remove client secret from apps

* dumb
2023-10-03 14:45:19 +00:00
Brendan Allan 861cfb8b40
Use actual oauth clients + ship ids and secrets (#1415)
* use actual oauth clients + ship ids and secrets

* re-enable features

* use prod as default api url
2023-10-02 09:36:43 +00:00
Oscar Beaumont 02f03f5351
Improve QuickPreview (#1350)
* Handle large text files

* wip

* nit

* Fix syntax highlighting

* Requiring an API call for my dev builds, no shot

* backend for line counting + wip frontend

* 600 lines is too much for this file, ngl

* wip: `LimitedByLinesBody` & some more restructuring

* Virtualised list for QuickPreview

* yeet bad ideas

* general cleanup + hack to fix broken toml

* fix

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-09-29 04:58:29 +00:00
nikec cb320f2d04
[ENG-1131] File path hidden filter (#1398)
hidden filter
2023-09-28 18:52:35 +00:00
Ericson "Fogo" Soares 54c1223256
[ENG-1081] Ignore the case of extensions when matching (#1394)
Done
2023-09-28 17:04:07 +00:00
Brendan Allan 3109601fda
Desktop + Web auth flow (#1378)
* initial login session route

* auth works!

* auth on web is dummy easy

* load api url from envs

* SpacedriveAccount + oauth

* use verification url

* heif

* better env handling

* error handling

* handle config write error

* add logout config write error
2023-09-28 16:55:46 +00:00
Vítor Vasconcellos bd0a7ff434
[ENG-265] Improve setup scripts (#1368)
* Setup script revamp

* Move toml to dev dep + CI runs the postinstall in debug mode

* Fix windows CI

* chmod +x scripts/setup.sh

* Fix eslint and ts CI

* Remove binstall + Fix trying to read input in CI

* Doesn't need to check pnpm and rust in CI

* Run postinstall script for Clippy CI

* Attempt to fix windows CI not running postinstall
 - Ignore cache when running postinstall on CI

* commited generated config.toml by mistake

* Pass GITHUB_TOKEN to `pnpm i`

* Update archive-wasm + Increase minimum node version to 18.17

* CI: Move rust setup after post-install script

* Revert: CI: Move rust setup after post-install script

* Fix CI, generate dummy cargo config.toml to fix prisma generation

* Fix windows CI

* CI: Fix wrong command

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-09-28 10:03:46 +00:00
Ericson "Fogo" Soares bee583e836
[ENG-1140] Bind an entire location to a single device number (#1386)
Removing device from file_path
2023-09-26 19:54:09 +00:00
Ericson "Fogo" Soares 052028a9c7
[ENG-1139] Walker wrongly marking files for update (#1373)
* Small warnings

* Solved another bug and more logs

* Rust fmt

* Even more logs
2023-09-23 14:43:54 +00:00
Brendan Allan 3f1ea3df10
[ENG-1132] Disable pagination for ordering by file path size (#1365)
* disable pagination for order by size

* disable 'take' when paginating on size
2023-09-22 10:12:30 +00:00
Ericson "Fogo" Soares f888054441
[ENG-1135] Locations: Total size query and return to Frontend (#1366)
* Done

* Fix size text to take its space

---------

Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
2023-09-22 10:11:59 +00:00
Ericson "Fogo" Soares 3cd55bb551
[ENG-812] Folder size calculation (#1362)
* Fix warnings

* Working when doing a full location indexing

* Directory size calculation for sub_path indexing
Also for shallow indexer
Now shallow indexer dispatches sub_path scans when it finds a new folder

* Directory size calculations in the watcher

* Pass directory sizes to sync system
2023-09-21 13:11:25 +00:00
Ericson "Fogo" Soares 5005c50e23
[ENG-1116] Distinct job manager for library (#1363)
Done
2023-09-21 11:10:56 +00:00
Oscar Beaumont 1cae4fab53
[ENG-1060] Fix the PlatformProvider bug (#1353)
* Life is a lie and nothing makes sense

* consistent version

* Web uses the relative alias resolver it would seem

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-09-15 12:02:50 +00:00
nikec 42c6c358c6
[ENG-1113] Toggleable list columns (#1352)
* Toggleable list columns

* type
2023-09-15 11:03:28 +00:00
Brendan Allan e5cab646c4
[ENG-1022] Keep track of hidden files (#1328)
* allow indexing + viewing hidden files

* generate db migration

* add 'Show hidden files' control to OptionsPanel

* clippy

* handle file renames

* use macos file attributes along with unix heuristic

* handle hidden ephemeral paths

* use un-deprecated st_flags getter

* tweak for uniformity

* formatting?

---------

Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
2023-09-14 07:05:45 +00:00
Ericson "Fogo" Soares 9076709314
[ENG-1076] Opening a sub dir closes the app (#1349)
Fix
2023-09-14 02:52:50 +00:00
Ericson "Fogo" Soares 7da0fdef0d
[ENG-1107] file_path unlinked from object (#1345)
Done
2023-09-13 20:06:53 +00:00
Oscar Beaumont 94d8f61de9
Update Rust dependencies (#1314)
* Update most of Rust dependencies

* downgrade notify

* a

* fix Linux?

* Android is painful

* update build.gradle

* The lord Clippy has spoken

* Bruh

---------

Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Ericson "Fogo" Soares <ericson.ds999@gmail.com>
2023-09-12 14:03:25 +00:00
Ericson "Fogo" Soares 5101ad35e5
[ENG-736] Watcher should check if a file has open write handles before processing it (#1319)
* Handling temporary files events on MacOS

* Debouncing create and update events

* Take files being updated for a long period
of time into account

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-09-12 03:23:47 +00:00
Ericson "Fogo" Soares cf39ae830b
[ENG-874] Windows needs reindex after renaming a file (#1303)
* Done

* Rust FMT

* Better debounce create and update events

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-09-12 03:22:22 +00:00
Brendan Allan 4b60ff2e08
[ENG-1023] Change location path from settings page (#1301)
* update location path in db

* remove + add location watcher if path is changed

---------

Co-authored-by: Ericson "Fogo" Soares <ericson.ds999@gmail.com>
2023-09-08 11:46:16 +00:00
jake 7edcbb15d6
[ENG-621][ENG-1074] Trait-based image conversion overhaul (#1307)
* sd-images crate which will support raw/dng, bmp, etc

* more work on the image formatter

* re-work `sd-images`, add svg support, r/g/b and r/g/b/a HEIF image support (will all be async again soon)

* remove `ImageFormatter`, add note about r/g/b/(a) heif impl

* implement the image formatter

* rename the conversion trait and minor cleanups

* isolate heif feature and major cleanup

* very untested raw support

* change fn name to `from_path` (a lot more idiomatic)

* clean up orientation fixing

* heif is no longer forbidden (linux has good heif)

also all extensions are correctly matched in lowercase

* fix builds, ext matching, feature gating

* attempt to fix svg handling?

* raw attempt, quite a few errors

* add comment

* new (untested) attempt

* remove `raw` stuff for now

* replace `sd-svg` with a `ToImage` `SvgHandler` impl

* add some simple math to appropriately scale thumbnails (and bmp/ico support)

* add comments regarding how the math works for image thumbs

* rename the trait to `ImageHandler`
2023-09-07 14:08:17 +00:00
Ericson "Fogo" Soares f8033d1842
[ENG-941] Jobs running in another library do not appear in the job manager (#1306)
* Fetching job reports from all libraries
Some clippy warnings
pnpm format

* Reverting expects to unwraps
2023-09-07 04:15:31 +00:00
ameer2468 5860016789
[ENG-934] EXIF UI (#1305)
* Media data UI

* Make `MediaTime` adjacently tagged

* cleanup ts

* don't destructure accordion props

* Large bruh

* round location coords

---------

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-09-06 13:51:15 +00:00
Oscar Beaumont 65ea570d0b
[ENG-1055] Fix previewing large files (#1304)
* Proper streaming from disk

* Handle `If-Range`

* fix bad merge

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-09-06 08:06:29 +00:00
Oscar Beaumont 5a0d0396d0
[ENG-566] Stream files through P2P (#1282)
* Add range to `SpaceblockRequest`

* Determine if file should be served from P2P

* Feature flag serveFromP2P + fix `ServeFrom` logic

* wip: yeet files over P2P

* Just work, argh

* restore backend feature flags after restart

* cleanup
2023-09-06 07:22:04 +00:00
Oscar Beaumont 251703104c
[ENG-1036] Fix library rename (#1300)
* Fix Library Rename + `RwLock` on `LibraryConfig`

* Input width fix

* cleanup

---------

Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
2023-09-06 01:01:18 +00:00
Brendan Allan 2d1ce9af03
[ENG-1078] Fix pagination (#1299)
* fix 'load more' breaking

* paginate all paginated queries by model id

* arrays start at 0 stupid
2023-09-05 08:11:04 +00:00
jake a92d6c2faf
[ENG-1039] Apply hard clippy lints to the entire ffmpeg crate (#1281)
* apply clippy lints to the entire ffmpeg crate in hopes of making it safer

* wording

* revert the prisma changes

* fix typo

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>

* fix another typo

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>

* remove so many `#[must_use]`s

* fix bad merge and hopefully clippy

* clippy please work i beg

* make HEIF_EXTENSIONS always available

---------

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-09-05 07:35:26 +00:00
Ericson "Fogo" Soares e8191e4a22
[ENG-198] Cleaning up after removing a location (#1296)
Done
2023-09-05 05:20:46 +00:00
Brendan Allan e7fbdb479c
[ENG-999] Order aware pagination (#1283)
* correct types

* remove optional override

* handle group_directories properly

* throw errors if is_dir is null

* disable size ordering

* usePathsInfiniteQuery

* implement for objects too

* cleanup
2023-09-04 12:38:09 +00:00
Ericson "Fogo" Soares 3b8541ef54
[ENG-1028] Media data for ephemeral locations (#1287)
Introducing new getEphemeralMediaData
Fixing some minor stuff
Running pnpm format

Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
2023-09-04 08:28:45 +00:00
Ericson "Fogo" Soares 7ace99cf1c
[ENG-1063] Actually regenerate thumbnails on Regen Thumbnails button (#1288)
Done

Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
2023-09-02 23:15:41 +00:00
Vítor Vasconcellos b0170b9dba
[ENG-904] Thumbnail for svg (#1220)
* Initial implementation for svg rendering

* Remove unused errors
 - Round up size before allocating Pixmap
 - Re-order some operations

* Finish integrating with Thumbnailer
 - Fix svg thumbnail size
 - Fix incorrect color space while converting tiny_skia::Pixmap to image

* Fix Clippy warns

* Feedback + sd-heif async

* Update implementation to match recent changes to the Thumbnailer
 - Change sd-heif and sd-svg to receive the file data, instead of opening the file internally
 - Update changes to make Thumbnailer and sd-heif more async

* Minor import improvement

* Add missing cfg to gate use of sd_heif only to plataforms that have it enabled
2023-08-31 23:01:45 +00:00
Oscar Beaumont f821bb9a0f
Stop App Freezing (#1257)
* Drop Tauri custom URI handler
Me no likely but it has to be done.

* fix app startup with location

* fix "Add Location" button on web

* Serve correct content range

* Backport changes from 08ba4f91

* none of my homies like panics

* minor fixes

* fmt with new Rust version
2023-08-31 06:54:40 +00:00
Ericson "Fogo" Soares 39549ef74c
[ENG-953] Zero-byte files occasionally receive a random thumbnail (#1280)
* Better handling empty files

* windows bruh
2023-08-31 01:42:54 +00:00
Oscar Beaumont e591516aff
[ENG-401] Spacedrop betterer (#1273)
* Spacedrop progress on `useP2PEvents`

* Spacedrop Tauri file picker for saving

* Prompt for save location when closing dialog

* quick maths

* Spacedrop progress

* Spacedrop cancellation

* Move accept/cancel dialog to toast

* Toast when rejecting transfer

* yeet

* `useEffect` more like `useBug`
2023-08-30 08:07:23 +00:00
Vítor Vasconcellos cb6a356287
[ENG-1027] Fix build on Windows (#1271)
* Fix Vite relativeAliasResolver on Windows
 - Add Strawberry Perl installation to setup-system.ps1 (stop rust-analyzer from crashing on Windows)
 - Remove unused imports from desktop windows crate
 - Add todo for missing sync test match cases

* Use path.join instead of doing string concatenation with path.sep by hand
2023-08-30 03:26:13 +00:00
Oscar Beaumont d758977d82
Improve Sync + P2P Integration (#1265)
* Big bruh moment

* whoops

* Less stackoverflowy debug

* stuff

* Fix flawed P2P mDNS instance advertisements

* do sync when connecting with peer

* Sync after pairing

* resync_part2 all the time

* Invalidate all the things

* Invalidate whole React Query on sync event

* emit_messages_flag

* emit_messages_flag

* Backend feature flags + "emitSyncEvents" feature

* Patch `confirm` type cause Tauri cringe

* clippy

* idk but plz work

* bruh

* Fix ComLink bug

* remove log

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-08-29 17:54:58 +00:00
jake c674b7107a
[ENG-288, ENG-601] Media data (image) extraction, thumbnail orientation fix (#1099)
* basic layout

* lock

* add codec to image

* very messy wip

* rm that

* lock and toml

* working perfect exif extraction

* formatting

* migration and formatting

* mostly working

* fix

* set date created properly

* fix tsc

* working media data creation

* fix bad main merge? sorry brendan

* schema, migrations, bindings

* working exif data extraction

* why won't it work

* update migrations

* fix bad merge

* final cleanup

* cleanup migrations

* remove test (that was purely used for testing my code)

* working media data pulling, correct thumbnail orientation

* slightly optimise thumbnail rotation

* rename location to prevent specta clashes

* further improvements (location parsing is still broken)

* fix coordinate parsing i think

* rspc add some todos regarding final cleanup

* further thoughts

* major upgrades

* Some improved handling of errors and async calls

* accordion component

* heavily wip flash refactor

* fix builds all because of a (terrible) merge/move

* annoying missing newline

* i really hate exif

* remove dead code

* further flash progress :D

* docs(media-data): clarification

* minor cleanup

* cleanup and some async removal

* fix location parsing

* remove async (will do proper impl for async eventually) and major cleanup

* big W

* clippy and `FlashMode::Unknown` instead of `Invalid`

* add `NIKON` gps ref support

* comments about gps refs

* commit the submodule -_-

* major cleanup & composite image support

* remove old test image

* major cleanup and easier to use API

* remove old consts

* move `ExifReader` to dedicated module

* Media Data Extractor job and shallow job

* Extracting media data on watcher

* report no exif data on file gracefully

* cleanup errors and doctests

* Merging thumbnailer and media data extractor

* Job name and some strings in UI

* remove reliance on `sd-prisma` within the media data crate

* rename query to be more specific

* custom serializer for `MediaTime`

* tweak to format utc -> naive correctly

* generate migrations

* comment out duration in mobile

* delete test-assets folder

* all optional fields

* fix migrations again

* make requested name changes

* make further requested changes

* remove erroneous files from another wip branch

* updates procedures

* use strings where appropriate

* regen pnpm-lock

* add base layouts for video and audio media data

* use appropriate data types in schema and add audio media data framework

* make requested changes

* general cleanup and renaming of enum

* cleanup media data api

* rename media metadata type

* finishing touches

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-08-29 17:02:55 +00:00
Aditya 08ba4f917a
Extending QuickPreview functionality with additional filetype support (#1231)
* added some files `standard` mime type

* Used `TEXTViewer` Component to show Code Preview

* Update Thumb.tsx

* added `prismjs`

* removed unnecessary comment

* `CODEViewer` Component for Syntax Highlighting

* formatting

* using **Atom** Theme for `Prism`

* merge text/code viewers & bg-app-focus for prism

currently calling onError and onLoad without an Event argument
that should change but i'm not really sure what to do there

* removed unused imports

* Update index.ts

* `TEXTViewer` to `TextViewer_`

* `TextViewer_` to `TextViewer`

* Don't highlight normal TextFiles

* clean code

* `TEXTViewer` to `TextViewer`

* using tailwind classes more

* doing things correctly.

* installed `prismjs` in interface

* using own scroller

* Update Thumb.tsx

* Add an AbortController to the fetch request
 - Fix onError and onLoad calls
 - Format code

* Fix onError being called when request was aborted due to re-render
 - Fix Compoenent re-rendering loop due to circular reference in useEffect
 - Remove unused imports

* Improve text file serving and code syntax highlight
 - Implement way to identify text files in file-ext crate
 - Do not depend only on the file extension to identify text files in custom_uri
 - Import more prismjs language rules files
 - Add line numbers to TextViewer when rendering code

* Clippy and prettier

* Fix reading zero byte data to Vec
 - Improve empty file handling

* Expand code highlight to more file types
 - Fix 10MB when it should be 10KB
 - Add supported for more code and config files extensions to sd-file-ext
 - Add comlink and vite-plugin-comlink for easy js worker integration
 - Move Prismjs logic to a Worker, because larger files (1000+ lines) where causing the UI to hang
 - Replace line-number prismjs plugin with our own implementation

* Fix uppercase extension name

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: pr <pineapplerind.info@gmail.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-08-29 10:47:04 +00:00
Brendan Allan 5a8c5af964
[ENG-1009] Fetch item count to increase scrollbar size in explorer (#1258)
* search.objectsCount

* provide count to useExplorer and react-table

* use row count if custom count not provided

* remove placeholder items from list view
2023-08-28 08:28:52 +00:00
Oscar Beaumont 7bf0c0ae4a
[ENG-994] Minor P2P fixes (#1238)
* mdns advertisement on detection

* fix bug in P2P discovery

* Add `P2PEvent::ExpirePeer`

* React properly

* Consistent tracing versions

* better logger config

* Reduce excessive logging from P2P

* Fix panic log hook

* Remove `dbg`

* Fix bug in thumbnailer remover

* wip: Connected peers in UI

* `Sync` is large pain

* init services after library load

* improve error handling of logger

* Sync protocol shutdown

* Add "updater" feature

* fix commands bindings

* nlm state debug query

* nlm debug status

* Make TS happy

* Add `RemoteIdentity` to libraries debug info

* Improve debug data presentation

* Among us level sus

* minor fix panic hook

* Fix EOF issue

* fix instance not being added on pairing

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Ericson "Fogo" Soares <ericson.ds999@gmail.com>
2023-08-27 06:41:26 +00:00
Brendan Allan 09857de4c8
[ENG-1004] Add offset pagination to search queries (#1255)
fix undefined cursors
2023-08-27 04:45:29 +00:00
Jamie Pine 0def8fb048
Revert "[ENG-1004] Add offset pagination capability to search queries… (#1254)
Revert "[ENG-1004] Add offset pagination capability to search queries (#1251)"

This reverts commit a5a25b1439.
2023-08-26 21:19:01 +00:00
Brendan Allan a5a25b1439
[ENG-1004] Add offset pagination capability to search queries (#1251)
Add offset pagination capability to search queries
2023-08-26 17:03:22 +00:00
Vítor Vasconcellos b199c03c1e
Always enable tauri update + Misc Clippy warns fix (#1250)
Always enable tauri update
 - Fix some misc clippy warnings
2023-08-25 22:08:34 +00:00
Brendan Allan f387f2f076
[ENG-992] Add search.pathsCount route (#1244)
search.pathsCount route

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-08-24 21:04:50 +00:00
Vítor Vasconcellos 7a7d01307e
Formatting (Prettier + Rustfmt 1.72) (#1246)
Formatting
2023-08-24 20:34:52 +00:00
Ericson "Fogo" Soares 28d106a2d5
[ENG-862, ENG-921] Ephemeral locations (#1092)
* Some initial drafts

* Finising the first draft on non-indexed locations

* Minor tweaks

* Fix warnings

* Adding date_created and date_modified to non indexed path entries

* Add id and path properties to NonIndexedPathItem

* Working ephemeral location (hardcoded home for now)

* Fix UI for ephemeral locations

* Fix windows

* Passing ephemeral thumbnails to thumbnails remover

* Indexing rules for ephemeral paths walking

* Animate Location button when path text overflow it's size

* Fix Linux not showing all volumes

* Fix Linux
 - Add some missing no_os_protected rules for macOS
 - Improve ephemeral location names

* Remove unecessary import

* Fix Mobile

* Improve resizing behaviour for ephemeral location topbar path button
 - Improve Search View (Replace custom empty component with Explorer's emptyNotice )
 - Improve how TopBar children positioning

* Hide EphemeralSection if there is no volume or home
 - Disable Ephemeral topbar path button animation when text is not overflowing

* minor fixes

* Introducing ordering for ephemeral paths

* TS Format

* Ephemeral locations UI fixes
 - Fix indexed Locations having no metadata
 - Remove date indexed/accessed options for sorting Ephemeral locations
 - Remove empty three dots from SideBar element when no settings is linked

* Add tooltip to add location button in ephemeral locations

* Fix indexed Locations selecting other folder/files in Ephemeral location

* Minor fixes

* Fix app breaking due to wrong logic to get item full path in Explorer

* Revert some recent changes to Thumb.tsx

* Fix original not loading for overview items
 - Fix QuickPreview name broken for overview items

* Improve imports

* Revert replace useEffect with useLayoutEffect for locked logic in ListView
It was causing the component to full reload when clicking a header to sort per column

* Changes from feedback

* Hide some unused Inspector metadata fields on NonIndexedPaths
 - Merge formatDate functions while retaining original behaviour

* Use tauri api for getting user home

* Change ThumbType to a string enum to allow for string comparisons

* Improve ObjectKind typing

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2023-08-23 17:26:07 +00:00
ameer2468 fe51d54075
[ENG-607] Explorer persist options (#1189)
* Structs for views

* hook up new preferences types to preferences system

* wip - draft

* laters useEffect, hi explorer context :D

* fix types

* wip

* hm

* preferences wip

* tweaks

* use search::SortOrder

* Refactor explorer settings to use useExplorer and useExplorerSettings (#1226)

refactor to use useExplorer and useExplorerSettings

* preferences

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-08-18 08:19:54 +00:00
Oscar Beaumont bcbbe58141
[ENG-974] DB Backup prototype (#1216)
* DB Backup prototype

* Put backups behind feature flag

* Warning for data folder

* nit

* Clippy

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-08-17 05:37:10 +00:00
Ericson "Fogo" Soares e0b72c41da
[ENG-876] Show folders first in explorer (#1223)
Grouping directories before files
2023-08-16 20:49:02 +00:00
nikec 9c0aec8167
[ENG-300] Explorer multi-select (#1197)
* grid

* Improved multi-select, grid list & view offset. Added gap option & app frame.

* List view multi-select

* Include multi-select in overview, fix page ref type

* Add gap to options panel

* Fix drag

* Update categories z-index

* going pretty well

* fix a couple bugs

* fix another bug :)

* minor improvements

* Separate grid activeItem

* extra comments

* um akshully don't ref during render

* show thumbnails yay

* cleanup

* Clean up

* Fix ranges

* here it is

* fix cols drag

* don't enforce selecto context

* explorer view selectable

* Update index.tsx

* Context menu support for multi-select (#1187)

* here it is

* stopPropagation

* cut copy multiple

---------

Co-authored-by: nikec <nikec.job@gmail.com>

* explorer view selectable

* Update index.tsx

* items Map

* fix renamable

* Update inspector

* Hide tag assign if empty

* fix merge

* cleanup

* fix un-rendered drag select

* fix double click quick preview

* update thumbnail

* mostly handle multiple select in keybindings

* fix ts

* remove another todo

* move useItemAs hooks to @sd/client

* fix thumb controls

* multi-select double click

* cleaner?

* smaller gap

---------

Co-authored-by: Jamie Pine <ijamespine@me.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Ericson "Fogo" Soares <ericson.ds999@gmail.com>
2023-08-15 08:23:41 +00:00
Ericson "Fogo" Soares fc95b19506
[ENG-644] Deleting a single file should be background job (#1214)
Deleting without job for single file
2023-08-14 06:01:19 +00:00
Ericson "Fogo" Soares 4a3acfa1c8
[ENG-835] Duplicating an empty directory doesn't duplicate nested empty directories (#1213)
* Bunch of warnings

* Solving the issue

* sd_core_sync::New

* dumb

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-08-13 18:34:46 +00:00
Jamie Pine a253431d0f
[ENG-887] Category empty screens (#1056)
* begin

* default category ui + schema

* yes

* better light icons

* new text icon

* fix schema

* fix merge error

* fix optional

* re-export icons

* tweaks

---------

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2023-08-12 16:30:03 +00:00
Brendan Allan 160b5e0cf8
[ENG-968] Cache job progress events (#1203)
* cache job report events in context

* hoist report state to JobManager

---------

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-08-12 07:41:18 +00:00
Brendan Allan 8f1afbf164
Isolate sync p2p logic (#1198)
* remove p2p from sd-core-sync and isolate p2p handling

* re-enable tests and stop being silly

* move sync protocol to modules

* don't emit messages

* cleanup

* formatting
2023-08-11 08:32:20 +00:00
Aditya 9ed2ceb14b
[ENG-954] Quick Preview for text file (#1184)
* Quick Preview for text file

Implemented a Quick Preview feature using an iframe to display the content of text files in the browser, enhancing the user experience with easy access to file information.

* made corners rounded

* desabled scroller in inspector

* Text Viewer's Implementation

Implemented a TEXT Viewer like PDF Viewer, which shows content of text files

* Dark Scrollbar & Padding Enhancements

Implemented dark scrollbar to match theme and added horizontal padding for improved text preview.

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-08-10 18:52:57 +00:00
Brendan Allan 9883a191a4
bruh (#1182) 2023-08-07 09:45:40 +00:00
Oscar Beaumont ae3d4c1c37
Library "states" continuation (#1180)
* Remove `NodeServices`

* minor

* fix

* cleanup services

* do the shuffle

* `LoadedLibrary` -> `Library`
Yeah this is probs better

* fixes

* bruh

* lol

* get_all_libraries -> get_all

* job::manager::Actor

* `ctx` to `node` for consistency

* _

* cleanup

* do arc higher up

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-08-07 09:26:10 +00:00
Brendan Allan 00e4aa9e8f
Sync ingesting (#1181)
* sync + ingest refactor

* fix Event enum description

* actually do sync over network

* re-enable heif

* remove comment
2023-08-07 07:52:43 +00:00
Oscar Beaumont 9e919f9909
Library "state" (#1160)
* `Library` -> `LoadedLibrary`

* Move `ThumbnailRemoverActor` onto `NodeServices`
It should either be on `Library` or `NodeServices` none of this in-between stuff.

* small > big files

* prune imports

* isolate cringe

* it helps to test the code

* `Vec` -> `HashMap` - faster lookups

* mpscrr

* minor fixes

* fix language

* `!Sync` :(

* lets goooooo

* removed a comma

* fixes

* fix desktop

* lol, no one uses it

* Clippy
2023-08-05 14:42:21 +00:00
Aditya ac5d7eb4f8
[ENG-848] Implemented "Copy as Path" functionality (#1139)
* #1119 Issue: Implemented "Copy as Path" functionality

I addressed issue #1119 by successfully implementing the "Copy as Path" functionality within the "More Actions" section of the Context Menu. This feature enhances user experience and offers a convenient way to copy file paths. While considering the valuable suggestion of potentially associating this option with the "Alt" key in the future, I opted to make it accessible through the "More Actions" section for now. All tests have passed smoothly

* Extend "Copy as Path" functionality to Overview, Search, and tags explorer

This commit extends the "Copy as Path" functionality to three additional screens in the SpaceDrive application: Overview (under categories), Search, and tags explorer. Users can now conveniently copy file paths from these screens, enhancing their overall experience with the application. The implementation has undergone thorough testing and is ready for review and integration into the project.

* Enhanced File Path Retrieval with RSPC Integration

The code now uses rspc to obtain the filePath, and we simply call it using useLibraryQuery from the frontend to retrieve the filePath. This RSPC-powered approach has successfully replaced the previous, less efficient method of file path retrieval. As a result, the entire process has become streamlined and optimized.

* Merge branch 'eng-848-feature-copy-as-path' of https://github.com/Raghav-45/spacedrive into eng-848-feature-copy-as-path

* Enhanced File Path Retrieval with RSPC Integration

The code now uses RSPC to obtain the file path, and we simply call it using useLibraryQuery from the frontend to retrieve the filePath. This RSPC-powered approach has successfully replaced the previous, less efficient method of file path retrieval. As a result, the entire process has become streamlined and optimized.

In the previous file path retrieval method, we relied on different file access techniques, which often led to slower & non-optimal responses

* Formatted coder with Prettier

* Refactor: Update file paths to '@sd/client'

* doing things with useLibraryMutation instead of useLibraryQuery

doing things with useLibraryMutation instead of useLibraryQuery, so that we only fetch file path only when needed

* Replace locationIdToPath api route with getPath
 - Resolve full file object path on the backend instead of the front-end
 - Use a query instead of a mutation
 - Export rspc queryClients to allow non-reactive usage
 - Error handling
 - Formatting

* Update Object's context menu to use  query

* Move onClick logic inside the CopyAsPath Element

* Improved a Little

- removed unused imports
- formatted document with prettier config which is present in code files `.prettierrc.js`
-

* Update Items.tsx

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: nikec <nikec.job@gmail.com>
2023-08-04 10:26:34 +00:00
Oscar Beaumont 6c9e08540f
Networked sync (#1133)
* temp remove discriminator from tunnel

* i'm lost :(

* Working `Tunnel` + move sync stuff back into p2p

* sync module

* `NetworkedSyncManager`

* Move sync stuff off `P2PManager`

* Test `SyncMessage`

* Library edit + delete hooks

* stable `identity` column

* fix

* plz work

* p2p is hard

* sync identities post pairing

* Remove `p384`

* a couple improvements

* comment out ReqRes

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-08-02 11:49:50 +00:00
Oscar Beaumont 13a3fbe486
Cleanup core (#1140)
* `NodeContext` -> `SharedContext`

* Cleanup context init

* Goodbye `Arc`'s

* Cleanup context

* rename

* minor

* abstract `Platform` into it's own file

* More `Arc`'s go goodbye

* parallelise `categories.list`

* parallelise `categories.list`

* Unused code in `libraries.rs`

* whoops

* a cheeky rename

* bruh

* :-(
2023-08-02 08:06:54 +00:00
Ericson "Fogo" Soares 26e3b53b21
Small fix due to how sharding works (#1138) 2023-07-28 03:46:44 +00:00
Ericson "Fogo" Soares aec0c94d62
[ENG-794] Library deletion hangs (#1137)
Solving deadlock on library deletion
Also improving some minor stuff on thumbnails remover actor
2023-07-27 23:18:45 +00:00
jake 7215e40a3b
[ENG-926] Prevent thumbnail destruction and fix the remover (#1127)
* fix(core): thumbnail removal

* chore(core): add todo

* New actor on steroids

* Improving thumbnail remover actor

* Ignoring errors from files that doesn't exist

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-07-25 14:21:50 +00:00
Ericson "Fogo" Soares 9283e74d85
[ENG-875] Re-index & Re-thumb should be smart (#1130)
* New sub_path rescan rspc route

* Changing some rescan call sites to use new route
2023-07-25 12:47:56 +00:00
Brendan Allan 447069388b
Push libraries into vec after loading (#1129)
push libraries into vec after loading
2023-07-24 17:17:02 +00:00
Brendan Allan 043b607ad4
CRDTOperation receiving (#1122)
* operation receive + compare

* cleanup + deduplication

* operation receive + compare

* cleanup + deduplication

* sync route + operation grouping

* tag assign sync

* proper relation support in sync debug page

* migration

* separate core-sync + utils crates

* separate p2p event loop from manager

* cleanup library handling

* clippy

* feature gate sync messages properly

* make migration not add required field
2023-07-24 15:26:00 +00:00
Oscar Beaumont 23b0c403be
[ENG-925] - Arc<Library>/Arc<NodeContext> (#1124)
* `Arc<Library>` + `Arc<NodeContext>`

* fix specta version

* remove unneccessary derefs

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-24 10:59:57 +00:00
Ericson "Fogo" Soares 03e71e98a4
[ENG-778] Location clean up job (#1123)
* Introducing thumbnail remover actor
Also tweaking the orphan remover actor

* Rust fmt
2023-07-20 15:17:54 +00:00
Oscar Beaumont 53ab3178c2
[ENG-906] Initial library sync (#1095)
* ffs

* typo

* yeet library data over p2p

* fix a bunch of edge cases

* report complete status on responder

* better log

* fix types

* mobile debug screen

* mobile + P2P is a mess

* feature flag mobile p2p pairing

* wrong one

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-17 11:53:25 +00:00
Brendan Allan 0a1d721e58
Use codegen for ModelSyncType exec (#1098)
* use codegen for ModelSyncType exec

* folder client format

* hande create operation properly
2023-07-15 04:22:04 +00:00
Brendan Allan f9b32f5a4a
SQLite connection limit (#1111)
connection_limit=1
2023-07-15 03:56:00 +00:00
Oscar Beaumont 1a5d2898c2
[ENG-537] Notification system (#805)
* `Notification` struct

* Manage dem notifications

* a dash of rspc

* `read` field

* `handle_notification_callback`

* some Prisma

* TS-centric notifications backend

* `useNotifications` hook

* Notification test buttons
2023-07-12 07:43:27 +00:00
Oscar Beaumont cf39f8dbcc
[ENG-759] P2P Cleanup (#1062)
* less stupid name

* yeet

* awaiting futures is kinda important lol

* no-async

* more proto stuff

* cleanup

* move it round

* None of my homies like broadcast

* minor

* do the shuffle

* restore after force push

* reusing `sysinfo::System` as intended

* fix lol

* remove `node_id` from `Volume`

* Remove `node_local_id` from `Library`

* Remove `Job` to `Node` relation

* feature flags be like

* press save 4head

* remove `Location` -> `Node` relation

* `volume.rs` to `volume/mod.rs`

* yes

* add `Instance` model and deprecate `Node` model

* pairing is better now

* Pretty code

* thinking in code

* wip

* What if Observables but in Rust?!

* Observables aren't it + `useP2PEvents` hook

* more around some jsx

* Trade offer: bad code for working pairing?

* Unit test pairing protocol

* fix everything up

* corrections

* Have you got a moment for our lord and saviour Clippy

* tsc --fixMyCode

* Prisma being wacky

* Config files being f'ed up

* broken config after migrating

* Zed being Zed

* Argh

* cliipzy

* rewind

* Fix truncate logic

* wip: instances in peer metadata

* Rethink instance ids

* fix

* whoops

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-12 06:23:30 +00:00
Niklas Wojtkowiak c877c03b63
Add functionality to some Overview Catrgories (#1090)
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-07-12 06:01:30 +00:00
Brendan Allan 07c00a8b7f
[ENG-846] Preferences (#1047)
* preferences table + trait

* cleaner implementation

* router + ts type

* preference test

* per-column list view preferences

* preference read + write

* add some docs

* migration + docs

* remove ts expect error
2023-07-12 04:32:53 +00:00
Ericson "Fogo" Soares 7cd00dab49
[ENG-884] Update file_path metadata (#1077)
* Partitioned paths to be updated on walker

* Updating file_paths in indexer

* Properly tracking modified date on renames

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-07-11 15:14:11 +00:00
nikec 9d800eec31
[ENG-885] Move online locations to lib store (#1074)
* Enable trace logging for invalidate query

* Fixing quick rescan on explorer
And annotating wrong quick rescan usage on top bar reload button

* Move online locations to lib store

* Fix topbar reload path

* fix mobile

* Remove unused imports + format
 - Adjust `arraysEqual` arguments type

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-07-05 22:27:38 +00:00
Vítor Vasconcellos f83f29c720
[ENG-765] Reported Total capacity and Free space are wrong (#1066)
* Attempt at fixing stats

* Fix macOS disk stats retrieve logic
 - Ignore mounted dmgs when calculation disk total/free size
 - Only take into account disk mounted by macOS

* macos only import

* Fix Linux

* Replace byte-size with a custom implementation that supports BigInt

* Fix NaN in Statistics

* clippy

* fmt

* Move linux get_volumes to a specilized function
 - Fix ZFS handling
 - Improve handling of disk symlinks and multiple mounts

* Fix macOS
2023-07-05 18:22:56 +00:00
Matthias Berchtold 52c5d1d2e9
Indexer rule No Git (#1048)
feat: no git indexer

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-07-03 11:45:36 +00:00
Brendan Allan f1239d166a
Search filter cleanup (#1063)
cheeky cleanup
2023-07-03 03:02:57 +00:00
Ericson "Fogo" Soares dcd071d229
[ENG-850] Wrongly labelled .ts files (as video) (#1052)
* Fixing conflits of mts code files showing as video

* Introducing a reidentify_objects param on fullRescan

* Fix job output metadata in case of completed with errors
2023-06-29 02:58:33 +00:00
Ericson "Fogo" Soares 5d49c8a74a
[ENG-823] Fixing bad handling of dots in directory name (#1051)
* Fixing bad handling of dots in directory name

* fix ci not running on rust

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-06-28 21:41:55 +00:00
Ericson "Fogo" Soares 1dce6380fe
Job System and Watcher improvements (#1030)
* Trying to make watcher more resilient between creates and updates

* More checks on create vs update

* Rust fmt

* PCR rev

* Windows CI fix

* Some indexer improvements

* Builder pattern for jobs and reports

* fix list view invalid date

* fix progress bar

* Normalizing job metadata output

* fixed job metadata

* Removing inner report struct from identifier job

---------

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Jamie Pine <ijamespine@me.com>
2023-06-28 16:30:19 +00:00
Brendan Allan 444a6d23c0
[ENG-806] Better library name handling (#1043)
better library name handling
2023-06-28 09:11:24 +00:00
Utku bc100f2505
Fix job resuming/pausing and cancelling (#1035)
why this works?
2023-06-27 19:26:59 +00:00
Brendan Allan 06de379169
[ENG-816, ENG-821] Re-implement reveal in finder + ContextMenu overhaul (#1029)
* mostly there

* native opening working

* more

* cleanup

* reorganise

* remove unnecessary import

* uncomment some stuff

* spacing

* store quickview ref inside provider

* fix linting

* clippy

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-06-27 15:34:53 +00:00
Oscar Beaumont d5ddfcc667
Job System - Simplify data (#1017)
* yeet

* destructure `JobState` when passed to `finalize`.
Let him cook.

* `impl Job for JobInitData` instead of it being `Job::Init`
The pan is on high heat and the 3 sh-meats are coming out.

* Remove `JobInitData` & `IntoJob` & `Job::new`

* clean up a little

* Remove `Job.stateful_job` in favor of `Job.state.init`
Yeah this isn't great but lets get these changes working and then think again about it.

* Properly call `StatefulJob.finalize`

* she'll be right

* fix `initialize_resumable_job`

* builder syntax for creating jobs

* Have Rust gone too far?

* Minor fixes

* fixes

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-06-27 07:15:49 +00:00
Oscar Beaumont f86b6c8b52
[ENG-823] Fix some stuff (#1024)
* SQLite journal bug

* Remove "job.report" indexes thing

* resume button - job not found?

* panic hook with logging

* it did

* large pain

* fix `check_nested_location`

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-06-27 05:12:04 +00:00
Oscar Beaumont ac5f6323a5
Job System - Better Progress (#1015)
yeet

Co-authored-by: Ericson "Fogo" Soares <ericson.ds999@gmail.com>
2023-06-26 17:49:55 +00:00
Gedeon 5a790ac996
[ENG-803] Set date upon tag creation (#1021)
* set date_created to tag

* update tag date modified when edited

* use `FIxedOffset` for tag `date_created`

* remove dead code and use fixedoffset

* discard duplicate code

* remove dead code & add current time to field

* remove unused code

* no longer needed in data

* set creation time for tags

* fix tag creation time

* fix & finalize timestamps

* `update_many` was not needed here

---------

Co-authored-by: brxken128 <77554505+brxken128@users.noreply.github.com>
2023-06-25 13:44:21 +00:00
Ericson "Fogo" Soares 697dfffe37
[ENG-670 / ENG-774] Resumable and Pausable jobs / Queued jobs don't show in the job manager (#961)
* Fixing some warnings

* Optimizing job workers

* Removing the delay from getting job reports

* remove some commented out stuff

* Trying to optimize job report route

* fix thread 'tokio-runtime-worker' panicked at 'attempt to subtract with overflow'

* fix progress bar

* Now pause works from the UI

* Fix

* Now the worker set job report to paused

* show errors for job pause/resume

* bunch 'o ui fixes

* order location in the sidebar

* fix some text

* fix clear all jobs

* fix clear jobs & job group ui

* show queued jobs

* object validator job text

* make cancel button work

* better executable logo

* Now cancel button works instantly

* disable resume button

* remove disabled props from pause/resume buttons

* remove large comments

* show paused progress

---------

Co-authored-by: James Pine <ijamespine@me.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-06-24 07:23:30 +00:00
Oscar Beaumont b980a6f630
[ENG-792] App no starty (#1004)
* lul, message parsing with a response is cringe

* deadlocks hurt me physically

* super secret commit with many secret secrets

---------

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-06-23 06:22:24 +00:00
Ericson "Fogo" Soares 187ec933f4
[ENG-770 / ENG-773] Copy/paste / Explorer context menu duplicate (#992)
* Fixing copy, cut and paste

* Bug on duplicating files without extensions

* Fix paste only ignoring the location origin when validating relative paths
 - Add Error handling to all context menu actions
 - Hide FS actions in Overview and MediaView
 - Remove redundant useExplorerSearchParams
 - Add default schema to the Zod param hooks

* Hide FS actions in all Explorers but location ones
 - Fix react warnings in RenameTextBox
 - Fix ReactTextBox not selecting the whole file name, instead of just up to file extension
 - Fix react warning due to inputing missing onChange

* basic show jobs for cut, copy and delete in job manager

* tweaks

* Fixing some warnings and minor tweaks

* Rust fmt

* Fix cut between locations on Linux

* Change how routes retrieve route params
 - Replace useZodRouteParams with useLoader
 - Define all schemas in the router file instead of in the route Component
 - Parse schema in loader to avoid requiring the route Components having to import the schema from the router file

* Remove default argument from useZodRouteParams

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: James Pine <ijamespine@me.com>
2023-06-22 23:41:29 +00:00
Brendan Allan 32824a88be
[ENG-782] Seeded tags (#994)
* seeded tags infra

* dumb

* here's my seed

---------

Co-authored-by: James Pine <ijamespine@me.com>
Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-06-22 12:08:38 +00:00
jake f5d7a99a70
[ENG-737] Location deletion hanging (#978)
* fix concurrency issue with lib deletion

* clear only completed jobs

* slight fixes

* experimental but faster i think

* lower chunk size
2023-06-22 10:49:21 +00:00
Oscar Beaumont 7198775ea7
[ENG-783] rename node (#993)
rename node + hide some stuff
2023-06-22 08:35:09 +00:00
Jamie Pine e8d3ad6005
[ENG-779] Finalize UI (#986)
* [ENG-779] Finalize UI
This is one branch with a variety of UI changes

add tag select mode bar without functionality 

fix group job status 

add notice icon with info to stat icons

add WIP notice to media view 

add modal before add location with greyed out clouds

remove disappearing add location button

add WIP spacedrop page 

bring back limited key manager UI 

add options bar on search page without functionality 

Add greyed out encrypt library button or setting

See more button on locations

Show locations on node screen

Fix overview category left padding

* key manager placeholder

* stat info

* nodes screen

* location click yay

* fix size in bytes

Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>

* small ui improvements

* sh*tty see more button

* last touches

* fix merge boo boo

* Fix mobile
 - Move `getItemObject`, `getItemFilePath`, `getItemLocation`, `getExplorerItemData` to @sd/core to allow mobile to use them

* Formatting

* Normalize displayed file size between all screens
 - Replace every use of internal formatBytes with byte-size dep

---------

Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-06-22 06:34:45 +00:00
Ericson "Fogo" Soares de85f00efc
[ENG-768 / ENG-769] Regen thumbnails / Generate checksums (#983)
* Making sub_path api for jobs harder to misuse

* re-enable context menu options

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-06-21 12:55:19 +00:00
Brendan Allan c6786f0c3f
Better tag unassign (#989)
better tag unassign
2023-06-21 12:22:20 +00:00
jake aa57b153a7
Remove useless clone from mutation (#988)
* remove useless clone from mutation

* fix clippy too
2023-06-21 10:53:49 +00:00
jake ccaf8ad1c6
Fix tag removal (#985)
fix tag removal
2023-06-21 08:47:59 +00:00
Oscar Beaumont 37bf788ca5
[ENG-685] Sqlite Multithreaded (#917)
* pragma moment

* Try socket_timeout?

* bye

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-06-20 20:09:52 +00:00
Ericson "Fogo" Soares 437f18081a
[ENG-743 / ENG-744] Duplicate item / Delete explorer item (#977)
* Fixing a bunch of small bugs related to FS

* cleanup

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-06-20 09:44:13 +00:00
Brendan Allan 92f2f185d7
Add COLLATE NOCASE back to migrations (#975)
* reintroduce collate nocase

* whoops
2023-06-20 09:01:55 +00:00
Brendan Allan 5df37d3a2d
Do all category logic on server (#972)
* do all category logic on server

* cleanup
2023-06-19 12:21:32 +00:00
Brendan Allan fdda0dd280
Fix categories queries (#971)
* more stringent hidden filter check

* add missing migration case

* don't do sync stuff when sync disabled

* remove old stuff
2023-06-19 11:28:07 +00:00
Oscar Beaumont 43647479a9
ordering for create library is wrong (#970) 2023-06-19 09:01:56 +00:00
Oscar Beaumont 0f7a669e5a
[ENG-732] Pairing by library (#943)
* library keypair

* allow opening stream to non-connected node + more

* library identity

* fix types

* fix maybe undefined

* don't forgor migrations file

* library manager inside p2p manager

* rename

* `NodeInformation` struct

* node info exchange

* fill in info

* streamify tunnel

* use tunnel for p2p events

* libp2p is annoying + stop leaking private key's

* Clippy cleanup
2023-06-19 05:13:30 +00:00
Brendan Allan 6975ebb664
Replace some default fields (#968)
replace some default fields
2023-06-19 00:48:38 +00:00
Brendan Allan d683d22c82
Remove owned sync (#967)
* remove owned sync + cleanup

* no more atomic records

* byebye owned

* remove owned ops from schema
2023-06-18 23:45:06 +00:00
Brendan Allan 2d703f2648
Make indexer rules optional (#965)
* optional indexer rules

* lockfile
2023-06-18 22:13:55 +00:00
Oscar Beaumont fd1afd8500
[ENG-568] Spacedrop Progress (#956)
* `Transfer` struct

* progress moment

* fixes
2023-06-18 08:40:51 +00:00
Jamie Pine 3198a9bdb6
[ENG-747] Indexer rule for Apple Photos .photoslibrary (#963)
photo library indexer rule + memorized video playback volume
2023-06-17 10:33:04 +00:00
Brendan Allan 019459194d
Make Job table all optional (#958)
optional time
2023-06-16 09:20:53 +00:00
Oscar Beaumont af23ef69d3
[ENG-745] App Startup Issue (#957)
* theoretically

* just please don't fail man

* bruh

* go away serde

* fix migrations

* squash migrations

* plz for the love of all that is good work

* disable some node fields

* fix core types

* remove unused stuff

* regenerate migrations

* Fix error boundry

* undo stuff

* migration

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
Co-authored-by: James Pine <ijamespine@me.com>
2023-06-16 07:53:21 +00:00
Jamie Pine dd464a2953
[ENG-686, ENG-670, ENG-669] Improved Job Manager (#907)
* refactor job manager ui

* huge improvements to job ui api

* improve indexer errors

* minor improvements

* make icon bigger + improve styling

* Update useJobInfo.tsx

better

* improve job status reporting

* fix job indexer backend for ui responsiveness

* attempt at debugging job.getRunning slow invalidation during indexer's walk phase

* remove progress debounce, invalidate has its own throttle layer

* hotfix ghost jobs

* basic pause/resume

* pause functionality immaculate

* pause resume working for first job in group, testable on indexer phase two

* WIP
- refactored job manager
- added better job api

* fix merge issues

* add throttle to job update events and correct index

* improve front end job data handling

* move subscription to job

* wip active job indicator

* minor tweak

* Isolated subscriptions for job events + cleanup

Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>

* mutable ctx

* plz let me build rspc typesafe errors Jamie

* fix merge

* working job reporting

* fix thumbnail text

* faster tick speed

* fix error

---------

Co-authored-by: Brendan Allan <Brendonovich@users.noreply.github.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2023-06-16 03:49:02 +00:00
Brendan Allan 01565b0130
remove duplicate new function (#955)
stupid
2023-06-16 01:49:11 +00:00
Brendan Allan 9a25373a94
Optional filepath + object fields (#947)
* rust changes

* fix ts

* formatting

* linter's a little baby

* mere

* address review comments
2023-06-16 00:22:11 +00:00
Ericson "Fogo" Soares 6ab6670102
[ENG-645] Location watcher doesn't always work (#954)
Watcher bugs:
When a file was in the location root directory, it failed to fetch its
parent directory from DB, as we don't store the location root directory
2023-06-15 23:21:09 +00:00
Oscar Beaumont 3d224e02ad
[ENG-675] Spacedrop is 0kb (#942)
* fix sd-web

* pain

* unit tests are pog

* Ignore `sdserver_data2`

* Update mod.rs
2023-06-15 00:32:54 +00:00
Brendan Allan 0725000017
Optional Tag fields (#949)
* optional tag fields

* update comment
2023-06-15 00:29:29 +00:00
Ericson "Fogo" Soares e693c7a542
[ENG-381] Update context menufs actions to support multiple source (#927)
* Generalizing filesystem jobs to accept multiple files at once

* Some small changes and fixing front

* Enable rename route to replicate Finder's behavior

* Assign tag to multiple objects at once

* Remove from recents accepting multiple files

* Fixing some warnings

* Adding multiple files feature to Open and Open With

* Conditional stuff for macos

* Generating commands.ts and other minor warnings

* Rust fmt

* TS typecheck

* Rust format and TS typecheck

* Requested changes

* Requested changes

---------

Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
2023-06-14 23:00:28 +00:00
Oscar Beaumont 1889a767e8
[ENG-723] Feature flags on frontend (#941)
* fixes

* refactor client

* feature flag store

* move `locations.online` subscription into context

* `useDiscoveredPeers` hook

* remove it

* bruh

* Valtio is cringe

* mount p2p events provider
2023-06-14 00:48:54 +00:00
Oscar Beaumont 923a0ceee1
[ENG-698] Stop leaking pk (#934)
stop leaking pk

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-06-12 18:03:12 +00:00
Brendan Allan 7148209343
[ENG 709] Make all location fields optional (#930)
* in progress

* make all location fields optional

* generate migration

* fix formatting

* formatting
2023-06-12 17:52:51 +00:00
Oscar Beaumont 4dc9ab53be
[ENG-703] Fix up errors (#919)
* wip

* temp

* rspc logging be like

* pass through inner errors
2023-06-12 06:10:00 +00:00
Vítor Vasconcellos 1c7855ded6
[ENG-630] Windows ffmpeg + libheif custom build (#871)
* Initial Windows ffmpeg + libheif custom build

* Add build steps for most of ffmpeg deps

* FFmpeg deps and libheif

* Fix libheif build

* Fix libvpx and dlfcn + attempt to fix rav1e

* Rework the whole ffmpeg-windows build system
 - New system based on https://github.com/BtbN/FFmpeg-Builds
 - Add new ffmpeg-windows workflow
 - Rename macos ffmpeg workflow
 - Adapt macos setupt script due to above name change

* Forgot to update update the workflow name

* Strip all libs from debug symbols

* Add docs

* Add libde265 deps, required by libheif
 - Make x265, svtav1 and dav1d as shared deps (used by both ffmpeg and libheif)

* Add missing libheif to Linux setup script

* Fix libx265 build script

* Forgot to point x265 ninja install to the correct directory

* Remove libaom and libsvt-av1
 - dav1d and rav1d are our default AV1 decoders/encoders
 - Quote subshell executions
 - Make libweb shared

* Forgot to remove libaom and libsvt-av1 build steps

* Fix typo

* Try force webp to link against static libs

* Revert libwebp to a static build

* Dumb typo

* Modify windows script to download our ffmpeg build (WIP)

* Fix dlls output folder structure

* Fix dumb mistake

* Remove unused ffbuild_enabled

* Enable core's heif feature on Windows
 - Fix windows ffmpeg build not including the headers
 - Fix windows setupt script incorrect download loagic
 - Implement build_arg to pass which repo ref to use when cloning

* Fix windows setup script

* Fix workflow artifact path
 - Make ffmpeg-windows dockerfile respect the FFMPEG_VERSION env

* Fix Windows setup script incorrect logic for downloading ffmpeg builds

* Error out when workflow_runs is empty

* Fix dumb mistake

* Manually define ffmpeg version for windows script

* Fix ffmpeg windows build extract logic

* Fix prop access in windows setup script

* Revert back to a web request because nightly.link does a redirect before serving the artifact content

* Fix windows setup script

* Do not use nightly.link in Github CI

* Fix windows setup script

* Should finally fix window setup script
 - Update ffmpeg-windows deps
 - Should fix ffmpeg-windows failing to build due to mingw changes in new base image

* Fix libxz failing to build due to doxygen

* Fix windows setup-script not executing till the end

* Fix LASTEXITCODE not defined

* Remove libjxl, deps are not being compiled

* Fix dll and lib copy logic

* Move final copy dll logic to external script

* Use main for libjxl

* Change brotli from stable to main
 - Attempt to fix libjxl

* Attempt fix lib copy again

* Split copy_dll logic to avoid cache burst in docker

* Missing file

* Change how to export build files from shared deps

* Replace rsync with cp

* Fix copy

* Fix dir not existing

* Fix pkgconfig

* Remove superflous files from exported ffmpeg for windows
 - Adjust dav1d to not build tools and examples
 - Adjust windows setup-script to point linker to the libs directory

* Fix dav1d meson config args

* Fix dumb mistake

* WORK PLZ

* Fix .lib file location
 - Strip all dlls

* Formatter
2023-06-10 15:23:37 +00:00
jake 277f9ce6cb
[ENG-691] Comment/remove crypto code from main (#932)
* comment out a bunch

* re-comment encrypt and decryot

* no more crypto in main

* remove keys settings icon

* add `disabled` to sidebar link

* bingo
2023-06-10 12:14:50 +00:00
Brendan Allan 0b9aa35b18
Fix default system rules (#929)
* setup default rules properly

* run rules seeder on library load
2023-06-10 08:17:00 +00:00
Oscar Beaumont e2dec80a51
fix main (#928) 2023-06-08 11:26:13 +00:00
Jamie Pine 8938ae6802
[ENG-708] Thumbnail sharding (#925)
* first phase, basic sharding

* improved API for sharding using a "thumbnailKey"

* clean up param handling for custom_uri

* added version manager with migrations for the thumbnail directory

* remove redundant hash of a hash, silly

* fix mobile

* fix clippy

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-06-08 07:13:45 +00:00
Brendan Allan fd236a1b57
[ENG-679] Reserve ids for built in indexer rules (#909)
* indexer rules pub ids

* should work?

* better migrator

* errors

* debugging

* maybe?

* double migrate

* please

* maybe fix?

* update lockfile

* SD_ACCEPT_DATA_LOSS message

* put tracing back

* dumb

* fix system indexer rule ui
2023-06-06 14:42:52 +00:00
Oscar Beaumont 007e089fcf
[ENG-697] Fix rename library (#916)
* random stuff

* How have we had a deadlock for 2 months lol
2023-06-06 11:41:39 +00:00
Oscar Beaumont 41933c8b80
[ENG-694] Remove Spacedrop (#914)
* goodbye Spacedrop

* fix startup error escaping

* fix error fallback being cringe with long error

* backwards compatibility for early adopters
2023-06-06 10:56:31 +00:00
Brendan Allan 73144faa30
Fix prod builds (#912)
move AbortOnDrop
2023-06-05 21:06:56 +00:00
Vítor Vasconcellos 350b7e39db
[ENG-683] Fix tauri allow list (#897)
* Fix Tauri drag not working
 - Fix Mpeg videos compleatly breaking QuickPreview
 - Fix Dev build open data folder button not working
 - Fix MP4 videos freezing in QuickPreview on MacOS

* Fix React inter-component state error
2023-06-02 00:09:19 +00:00
Brendan Allan 9584dbd924
[ENG-684] Gracefully fail if location not available on startup (#899)
gracefully fail if location not available on startup
2023-06-01 11:29:09 +00:00
Brendan Allan 6daed1df1a
[ENG-688] Invalidate search.paths in shallow operations (#901)
invalidate search.paths in shallow stuff
2023-06-01 11:28:50 +00:00
Brendan Allan 52ab4ced3b
[ENG-689] Delete jobs during resume that don't exist in code (#902)
delete jobs during resume that don't exist in code
2023-06-01 11:28:07 +00:00
Brendan Allan d173cbcfaf
[ENG-678] Hidden object filtering (#895)
object hidden filter
2023-05-31 10:56:36 +00:00
Oscar Beaumont 049966d848
[ENG-674] fix invalidation (#891)
* reenable the invalidate requests system

* a tester for the invalidate system

* watch locations in Library::load

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-05-31 08:55:46 +00:00
Brendan Allan a2400b38d0
skip thumbnailing *ts video extensions (#890) 2023-05-31 07:11:44 +00:00
Oscar Beaumont e4696c7303
[ENG-667] Fix Clippy CI (#887)
* plz just work

* no shot

* just disable Tauri notifications

* Tauri is weird AF
2023-05-30 17:11:37 +00:00
Brendan Allan e70fd4eba5
[ENG-662] Don't use job system for shallow operations (#886)
* make it work

* use functions instead of jobs for shallow stuff

* use generalised AbortOnDrop

* simplify naming

* error stuff + alphabetical rspc types

* log errors

* don't run CI on push

* remove ephemeral

* private Worker::new

* remove comments
2023-05-30 17:07:41 +00:00
Oscar Beaumont 5f71e0c1f6
[ENG-546] Improve error handling (#802)
* log to disk

* remove some unwraps

* panicless

* some p2p error handling

* clippy moment

* Fix `<ErrorBoundary />`

* open logs button

* 39 to 0

* fix types

* update deps and comment out broken tests

* clippy

* more clippy

* upgrade rimraf - https://github.com/isaacs/rimraf/issues/259

* regen broken lockfile

* update `notify` and update `commands.ts`

* more clippy (pls work)

* allow deprecated for p2p (hopefully temporary)

* a

* upgrade deps for p2p

* do betterer

* do it correctly

* remove unused import

* improve core startup error + bc keypair

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: brxken128 <77554505+brxken128@users.noreply.github.com>
2023-05-30 09:24:05 +00:00
Brendan Allan 8226d5e9f0
[ENG-588] Fix total capacity on macOS (#882)
* correct macos root volume handling

* DiskType enum
2023-05-30 06:25:35 +00:00
Brendan Allan 6710f05251
move prisma codegen into sd-prisma (#876) 2023-05-29 08:37:58 +00:00
Ericson "Fogo" Soares b86a30ae8a
[ENG-613] Indexer rule whitelist/blacklist per rule (#847)
* Introducing many rules on a single indexer rule

* Rust fmt

* Some doc fixes and small stuff

* Small warnings

* Making front typecheks

* Migration fix

---------

Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-05-25 19:27:31 +00:00
Vítor Vasconcellos 0676c40331
[ENG-651] Fix PDF rendering breaking app on macOS (#854)
* Fix macOS PDF rendering
 - Fix app crashing due to PDF rendering receiving empty URLs
 - Attempt fix PDF rendering empty PDFs due to it not supporting range requests

* Fix dumb change from `data` to `src` in `<object>`
 - Fix QuickPreview not closing with space bar
 - Fix double-click simultaneously renaming and opening file
 - Minor improvements to QuickPreview header
 - Fix Button inside Button react error in QuickPreview
 - Don't render thumb without a valid source

* ExternalObject events must not influence the link state
 - More macOS PDF range changes

* Use `<iframe>` instead of `<embed>` or `<object>` to load pdf in macOS
 - Revert removing range support for macOS pdf type
 - Rename `ExternalObject` to `PDFViewer`
 - Fix `AddLocationDialog` sometimes requiring multiple confimations on first load

* `Accept-Ranges: none` Header response as it breaks linux video playback
 - Extract location id from from `ExplorerItemData`, to allow rendering original versions on `Overview`

* Format

* cargo fmt
2023-05-25 06:34:18 +00:00
Jamie Pine ea46e7736a
[ENG-635] Job time estimation (#842)
* fix extension case sensitivity

* job time estimation wip

* update schema

* use `DateTime` to handle estimated completion time

* use a date with `dayjs`

* remove old migrations

* update migrations

* remove dead code

* Quick tweaks

* unused import

---------

Co-authored-by: brxken128 <77554505+brxken128@users.noreply.github.com>
Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
2023-05-23 22:21:52 +00:00
Brendan Allan da2e78dc61
Use search.objects for recents query (#850)
* separate search + filter + ordering

* typesafe categories

* make ordering great again

* fix ts

* eslint
2023-05-23 12:58:45 +00:00
Jamie Pine 7e285dc4da
[END-634] Coerce file extensions to lowercase (#841)
fix extension case sensitivity
2023-05-21 09:45:00 +00:00
Jamie Pine d55df77455
[ENG-631] TopBar improvements & misc fixes (#837)
* fix things

* added back/forward buttons to settings

* split top bar context into left and right

* hook up path

* fix background jobs hidden from job manager

* core

* fix type + quick preview transition

* fix selected item color contrast

* fix close button on quick preview

* clean up job ui for light theme

* Improve media view overscan

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-05-21 00:17:27 +00:00
Brendan Allan b0e95fc59f
Replace getRecent with search.paths (#832)
replace getRecent with search.paths

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
2023-05-20 22:11:43 +00:00
jake fb95c6fec2
HEIF Fixes and Improvements (#838)
* limit HEIF to macos-only because of ubuntu's *archaic* `libheif` version

* drop png entirely, go from raw `[r, g, b...]` -> `DynamicImage`
2023-05-20 22:08:04 +00:00
Jamie Pine 7e4c58a91c
[ENG-603] Support HEIC images (#834)
* working HEIC thumbnails

* better error handling

* better error handling and some cleanup

* fix type of maximum size, and clippy

* better extension support, WIP quick preview, better image resolution

* remove unnecessary x86_64 macos rustflags

* add correct rustflags to setup script

* add fedora libheif deps

* debian libheif deps

* arch libheif too

* add sd-heif as a dep and feature gate it

* enable aforementioned feature in tauri only

* add URI support for heif/heic (quick preview still won't work)

* correct feature gating on everything

* dedicated sd-heif crate

---------

Co-authored-by: brxken128 <77554505+brxken128@users.noreply.github.com>
2023-05-20 06:40:37 +00:00
jake 1aa8826398
SPDX license and workspace inheritence for re-used metadata (#833)
use workspace inheritence for license, edition and repo URL
2023-05-20 01:52:55 +00:00
Jamie Pine 3880428596
[ENG-624] Explorer order by (#831)
added order by + fixed explorer padding situation
2023-05-19 15:51:17 +00:00
Oscar Beaumont 01e8716b94
[ENG-559] Spacedrop move to spaceblock (#824)
* Move Spacedrop to Spaceblock

* bruh

* unit test moment

* plz work

* usize != u16 man

* tsc --quite

* clippy
2023-05-18 19:02:44 +00:00
Jamie Pine fb2a4b0137
[ENG-662] Overview infinite scroll (#826)
* overview infinite scroll, quick preview position fix, movie icon

* fix name

* gen assets
2023-05-17 23:11:33 +00:00
Jamie Pine e8895c1a05
[ENG-581, ENG-597, ENG-600] Overview categories (#779)
* added base UI for categories on overview

* update core

* cleanup ui

* Inspector default view if nothing is selected, explorer takes child components, hidden menu tweak if no items are sm:flex

* wip

* somewhat functional

* scroll

* category fixes

* clean category bar

* added config store + made toolbar available on all explorer screens

* clean up overview.tsx

* added counts

* fix inspector bug

* add support for favorites
+ add book extension support

* refactor into smaller components

* Some small rust nitpicks

* fix camel case location_type

* Rust fmt

* fix typescript CI

---------

Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Co-authored-by: nikec <nikec.job@gmail.com>
Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-05-17 20:34:05 +00:00
Ericson "Fogo" Soares dda7516980
[ENG-536] Improve indexer for huge locations (#781)
* WIP materialized_path abstraction revamp

* Optimizing indexer rules loading

* Using a better serialize impl for indexer rules

* New interruptable and faster Walker

* WIP new indexer

* WIP first success compiling after breaking the world

* Fixing some warnings

* Handling some lifetime issues in the walker

* New job completed with errors feature

* Introducing completed with errors to indexer
Removing IOError variant from JobError and using FileIOError instead

* Rust fmt

* Adding missing job status

* Better ergonomics to IsolatedFilePathData
Conversions from db's file_path kinds
Keeping original's relative path data to better conversion to OS's path
First unit tests

* Testing and fixing parent method

* Some error handling

* Rust fmt

* Some small fixes

* Fixing indexer rules decoding

* Bunch of small fixes

* Rust fmt

* Fixing indexer rules

* Updating frontend to new materialized_path format

* Trying to fix windows CI

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-05-16 14:15:51 +00:00
Brendan Allan 04deb621e9
Replace locations.getExplorerData with search.paths (#812)
* retire locations.getExplorerData from interface

* regen core.ts

* replace tags.getExplorerData with search.objects

* types out the wazooo

* better optional halding

* ternary-ify

---------

Co-authored-by: Jamie Pine <ijamespine@me.com>
2023-05-16 08:43:59 +00:00
Brendan Allan 1cccecdf8d
Orphan Object Remover (#821)
* orphan remover task

* loop to do multiple passes

* struct-ify
2023-05-16 01:12:00 +00:00
jake 54010345af
[ENG-609] Remove file from recent files (#811)
allow removing a file from recents
2023-05-12 19:14:31 +00:00
Brendan Allan 3c1696f833
whoops (#810) 2023-05-10 21:13:08 +00:00
jake 9b2788ed63
deps: remove ffmpeg-next from the core (#808)
remove ffmpeg-next from the core
2023-05-10 04:44:27 +00:00
ameer2468 0a4c1f41e7
Job Manager improvement & refactor (#783)
* d

* wip

* Nit: margin and padding tweak

* UI design tweaks

* Update Job.tsx

* Improve UI

* [WIP] - Refactor job manager

* remove invalidate explorer event on thumb generation. the event above performs atomic updates on the front end when new thumbnails are generated, now just need to make that work

* prettier formatting + removed unused imports

* UI tweaks

* progress bar width adjustment

* tweaks

* fix em

* fix thumbnail generation

* fix progress bar

* fix time

---------

Co-authored-by: Jamie Pine <ijamespine@me.com>
Co-authored-by: Brendan Allan <brendonovich@outlook.com>
Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2023-05-08 22:31:49 +00:00
Brendan Allan 05ff5ed800
Eng 510 open with in context menu (#803)
* somewhat

* proper macos support

* formatting
2023-05-08 09:22:24 +00:00
jake 818bc1ecc9
[ENG-592] Prevent libraries with blank names/no names from being created (#800)
* prevent libraries having all null or empty names

* stop onboarding continuation if name field is empty

* prevent libraries with entirely empty names from being created in TS

* remove trim from schema

* add comment explaining regex, and `.trim()`
2023-05-06 07:32:32 +00:00
jake 4f327db611
source has_thumbnail correctly for the recent file query (#797)
* source `has_thumbnail` correctly

* correctly check if thumbnail exists in watcher

* regen bindings

* add helper fn for getting thumbnail directory and cleanup old thumbnails if the watcher creates a new one
2023-05-05 22:01:34 +00:00
nikec a5b59b7684
[ENG-575] Infinite scroll (#785)
Infinite scroll
2023-05-04 19:08:38 +00:00
Oscar Beaumont 2571e3b275
Upgrade rspc TS (#775)
* finally

* should be working?

* fix types

* fix types

* Wouldn't it be nice if Metro would just work

* idk

* try harder Metro

* potentially fix bundling issues

* idk, maybe fix it?

* fix metro

* update podfile.lock

* bruh

* bruhz

* tailwind is drunk again

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
2023-05-04 06:21:42 +00:00
ameer2468 1db757c115
ENG-287: Overview recent files (#780)
* experimental recent file querying

* working recent files

* fix querying and make amount returned customisable

* experimental recent file querying

* working recent files

* fix querying and make amount returned customisable

* return `FilePath`s from `files.getRecent`

* return `ExplorerItem` from `files.getRecent`

* Overview recents

* commands.ts

* cleanup

---------

Co-authored-by: brxken128 <77554505+brxken128@users.noreply.github.com>
2023-05-03 17:06:09 +00:00
Brendan Allan b620e1e174
paginated location explorer (#774)
* paginated location explorer

* remove props from GridView

* formatting
2023-05-01 09:45:25 +00:00
Brendan Allan 7cee2b2651
update prisma (#773)
* update prisma

* idk
2023-04-30 23:03:23 -07:00
Vítor Vasconcellos 90a3509469
[ENG-471, ENG-472] Add dry_run parameter to some API routes and related UI changes (#750)
* Add dry_run for `location.create` and `location.addLibrary`

* Add dry_run to IndexerRuleCreateArgs
- Add invalidate_query to create and edit api method of the location route
- Adjust some code to use the new dry_run property

* `AddLocationDialog` and `IndexerRuleEditor` now validate with backend without user interaction
 - Create `useCallbackToWatchForm` to make it easier to watch form changes with an async function that also executes during component mount
 - `method` is now part of the `addLocationDialog` form schema, and a hidden input
 - Add an `index.ts` for hooks and components

* Fix mobile

* Remove redundant type definition

* Add `useCallbackToWatchForm` as an additional hook to `react-hooks/exhaustive-deps`
 - Improve `useCallbackToWatchForm` argument handling, to avoid duplicated references to form

* Fix unnecessary form value updates in `addLocationDialog`
 - Move reset `method` on `path` change logic from the superfluous `useEffect` to `useCallbackToWatchForm` in `addLocationDialog`
 - Improve core API debug logs for `dry_run`
 - Fix incorrect file name for `useCallbackToWatchForm` hook
 - Improve the documentation for `useCallbackToWatchForm`
 - Fix `useCallbackToWatchForm` not ignoring callback errors
 - Fix `useCallbackToWatchForm` ignoring returned Promise from `onWatch`
 - Add basic validation for the `form` argument in `useCallbackToWatchForm`

* Remove superflous if checks
 - generate new core.ts

* Remove DryRunError

* Remove unused import

---------

Co-authored-by: Brendan Allan <brendonovich@outlook.com>
2023-04-29 10:49:51 +00:00
Brendan Allan e1c626c0fa
Remove all field name string literals (#772)
remove all field name string literals
2023-04-28 21:59:11 -07:00
Brendan Allan 3570c5f3d8
File Opening (#762)
* file opening

* eslint disable in gererated commands

* bruh?

* Fixing materialized_path joining and some warnings

* no unwraps

* formatting

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-04-28 06:55:00 -07:00
Oscar Beaumont 3c440783b3
Spacedrop (#761)
* A hint of file drop

* backport from #671

* accept/reject Spacedrop working

* final cleanup

* Rename spacedrop.tsx to Spacedrop.tsx

* Update index.tsx
2023-04-27 04:41:33 +00:00
Brendan Allan 58e0542d8e
Give file paths pub_id and use it primarily (#754)
* give file paths pub_id and use it primarily

* fix object indexing

* file path location + sync

* remove future plan

* use id instead of pub_id for more indexer queries

* use chain_optional_iter

---------

Co-authored-by: Oscar Beaumont <oscar@otbeaumont.me>
2023-04-25 16:53:36 +00:00
Oscar Beaumont 9c62df9947 fix type exporting 2023-04-25 23:51:48 +08:00
Oscar Beaumont a4b84d31cc fix desktop 2023-04-25 22:47:41 +08:00
Oscar Beaumont fed3f2211f
Rspc v3 syntax (#758)
just fixed a couple of lines
2023-04-25 17:12:42 +08:00
Oscar Beaumont 85d0ddf6d3
rspc 0.1.3 prep (#756)
* Upgrade rspc + Specta + PCR

* fix jsonrpc executor

* fix invalidate excessive logging

* fix `Protected` type

* fix mobile

* clippy

* lol
2023-04-25 15:14:52 +08:00
jake 37b8d64642
[ENG-490] Add support for late key manager setup (#740)
* support for late key manager setup and remove password from onboarding

* mobile

* delete master-password.tsx

* remove secure temp keystore

* remove more km_config artifacts

* fix builds

* Fix most of the UI, there still some minor bugs

* Workaround portalled closing Popovers

* Restore search in core/src/api/mod.rs

* Fix Password eye icon closing Popover on click

* Fix keymanager setup submit button spinner not centered
 - Fix keymanager setup missing logic for disabling form
 - Adjust keymanager tab roundness to match app style

* Restore .gitignore

* Restore .prettierignore

* fixerino tests

---------

Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>
Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
2023-04-22 09:10:21 +00:00
Ericson "Fogo" Soares fa5671f614
[Eng 495] Better job report fields (#726)
* Changing action column on job table

* Introducing action and is_background info

* Requested changes
2023-04-22 06:17:50 +00:00
Vítor Vasconcellos 24a3aedd7b
[ENG-384] Indexer rules editor (#723)
* WIP Create Indexer Rule UI + Backend api fix

* Complete IndexerEditor UI for creating new rules
- WIP testing to ensure all rules are parsed correctly
- Add utilities to satisfy the TypeScript typechecker
- Introduce a utility function to facilitate extracting information from an RSPCError
- Modify AddLocationDialog to utilize the aforementioned utility function

* Validation and submit logic is now functional
- Reorganize UI for improved user flow
- Implement validation and error messaging, replace some bare inputs with form inputs, and fix styling issues
- Resolve issues with post-processing of rules during submission
- Wrap editable portion of component in a `FormProvider` due to the `Form` being in a `Portal`
- Add specialized `ErrorMessage` component to `@sd/ui` library
- Update `AddLocationDialog` and `FormField` to utilize the new `ErrorMessage` component

* Fix submit not waiting for confirmation from backend
 - Reset form after submit
 - Disable form while submiting
 - Update form related dependencies

* Implement deleting an indexer rule
 - Modify indexer rule api route to disallow deleting default rules

* Fix form reset on error
 - Minor style adjustments
2023-04-22 04:30:11 +00:00
jake 4315cee147
[ENG-516] Prevent copies/cuts to the same source/dest (#748)
* add basic UI protection to stop cut/copies to the same path

* add rust src/dest protections

* fix pasting logic

* fix path canonicalization

* skip paths instead of overwriting them

* Using non-blocking ops and fixing some warnings

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-04-22 02:11:59 +00:00
ameer2468 ee531a81fb
[ENG-473] - Job Manager UI Improvements (#745)
* add support for deleting a specific job

* wip

* Update JobManager.tsx

* Job manager design tweaks + popover open state ignore prop

* tweaks

---------

Co-authored-by: jake <77554505+brxken128@users.noreply.github.com>
2023-04-21 17:08:32 -07:00
Brendan Allan 469a3831dc
Explorer search (#738)
* basic file path search

* cleanup

* render explorer in search page

* update materialized path docs
2023-04-21 14:49:46 +00:00
nikec a97145348e
[ENG-295] Explorer media view (#730)
* remove unused onScroll

* Add open media file with double click

* Hide file thumb in inspector

* Enable media layout

* Add media layout options

* Add media view

* Fix vertical images

* Fix vertical square aspect

* query

* Update Location.tsx

* Some optimizations on explorer data query

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-04-21 02:15:22 +00:00
Oscar Beaumont ce6d2d2fda
Eng 494 adding a location does not update the UI (#728)
* `sd_init.json` support

* bruh

* Add `sd_init.json` to developer docs

* Ran cargo clippy --fix to fix warnings

* Dafuq, cargo clippy --fix messed up cargo fmt

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-04-20 22:05:57 +00:00
Ericson "Fogo" Soares f684660a90
Checking if new location is nested or already exists (#716)
* Checking if new location is nested or already exists

* Also checking for children locations on check nested

* Rust fmt
2023-04-19 05:03:28 +00:00
jake 288f4947e1
remove int-enum (#721) 2023-04-19 01:33:32 +00:00
Ericson "Fogo" Soares 7a792de090
[Eng 493] Job manager status showing incorrect (#722)
* Fixing path report for indexer on job manager

* Fixing warnings on migrator tests
2023-04-19 01:05:30 +00:00
Utku 7612458d76
Prisma Schema Comments (#720)
* add comments to prisma schema

* fix

* Update schema.prisma
2023-04-18 15:49:34 +00:00
Oscar Beaumont daadbf592a
Eng 488 fix app migrations system (#719)
* improve app level migration system

* basic unit tests for app migrator
We kinda don't want this going wrong so extra barrier is good.

* fix migrator tests
2023-04-18 08:51:01 +00:00
Vítor Vasconcellos 80fe8f98f1
[ENG-286] Set default indexer rules for location (#690)
* Implement multiple Glob matches in the same indexer rule
 - Replace matching logic to use GlobSet instead of simple Glob
 - Add `No OS protected` indexer rule to avoid indexing OS protected files
 - Enable `No OS protected` indexer rule by default

* Rust fmt

* Reduce `No OS protected` rule Glob list by using OR matches

* Add globs for android and ios files in `No OS protected` rule

* Add some more unix special path to be ignored

* Add a new property to IndexerRule to enable it by default when adding a new location:
- Modify the Prisma schema to add the default property to the IndexerRule model.
- Adjust the IndexerRule struct's create and save logic to consider the new property.
- Adjust AddLocationDialog to properly load the default indexer rules from the backend.
- Minorly improve IndexerRuleEditor to avoid adding duplicate entries to its assigned form field.
- Add editorconfig entries for SQL and Prisma types.

* rust fmt

* Add Windows users special folders and files to `No OS protected` rule

* Construct `No OS protected` globs from string vec
 - Don't repeat windows globs that require both a root and C: version

* Apply review feedback and add error handling for `IndexerRuleEditor`
- Revert manual changes made to init migration
- Create external migration that adds a `default` prop in `IndexerRule` model
- Remove redundant `useMemo`
- Replace destructuring of `useQuery`
- Provide feedback to user when a error occurs while querying indexer rules

* useMemo only for `indexerRulesIds` and not the whole `defaultValues` object
 - Remove logic to accpet unix root path for windows indexer `No OS protected` rule
2023-04-18 06:25:57 +00:00
Oscar Beaumont 8ea6a1c329
Automatic job progress (#717)
Remove time tracker task + sender level debouncing
2023-04-18 06:24:06 +00:00
Vítor Vasconcellos 3fbc2b909c
[ENG-461] Location creation fails for '/' (root) and 'C:' ([A-Z] drives) (#710)
* Fix issue with root locations failing to be added
 - Add `normpath` dependency to the core
 - Improve automatic name resolution for new locations

* Add `MetadataExt` trait to extend the `created` and `modified` methods of `Metadata`
- Prevent failure in the Indexer job caused by paths without associated created or modified times; assume current time for both when unavailable
- Rename possible location in root path from `/` to `Root`

* Improve hadling of location path on Windows
 - Normalize location's path to ensure a valid format on Windows
 - Improve error handling when adding a path
 - Avoid consuming symlinks on Linux by not normalizing location's path
 - Improve location naming logic
2023-04-18 06:12:02 +00:00
Brendan Allan 98d846827d
job parent id fix (#714) 2023-04-17 10:26:27 -07:00
Oscar Beaumont 2dd3d6f921
Job system improvements v2 (#707)
* Add `JobInitData` trait + change `StatefulJob::name` to a constant

* `StatefulJob::new`

* Move `invalidate_query` into `StatefulJob::finalize`

* Cleanup `spawn_job` signature

* Remove `queue_job` and run jobs from the `StatefulJob::finalize` method

* `StatefulJob::queue_jobs`

* `JobManager::ingest` return `Result`

* Remove `jobs.isRunning`

* Invalidation system direct push + batching invalidate

* Look ma, only a single clippy warning!!!!

* Error handling for JobManager

* Rust fmt

* Introducing Job hierarchy to enable job enqueuing

* Rust fmt again 🙄

* core.ts

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
2023-04-15 14:43:04 +08:00
Brendan Allan 8a525f6cc8 Revert "Give FilePath a pub_id (#691)"
This reverts commit 3c494ea96e.
2023-04-14 10:50:37 +08:00
jake 39e92cc7e7
update to ffmpeg 6.0.0 (#697) 2023-04-13 14:10:23 +00:00
Brendan Allan 3c494ea96e
Give FilePath a pub_id (#691)
* checkpoint - indexer parent ids need fixing

* use pub_id for file paths

* Removing LastFilePathIdManager and fixing some loose ends

---------

Co-authored-by: Ericson "Fogo" Soares <ericson.ds999@gmail.com>
2023-04-12 21:58:23 -07:00
Ericson "Fogo" Soares 1e05226efb
ENG-462 Object size is 0 when it shouldn't be (#702)
* Fixing file size not being collected from system

* Rust fmt
2023-04-13 01:06:27 +00:00
Vítor Vasconcellos 8224c7f146
ENG-460 Location creation fails for some writable directories (#696)
Remove read-only check when creating a location
- Revert location creation if writing the `.spacedrive` file or updating the library location manager fails
2023-04-12 12:51:32 +00:00
nikec 04c8a96943
[Desktop] Renamable files (#675)
* Add rename file option

* make Ericson's changes

* Prevent arrow selecting when renaming, start renaming with Enter option

* fix rename

---------

Co-authored-by: Jamie Pine <32987599+jamiepine@users.noreply.github.com>
Co-authored-by: Jamie Pine <ijamespine@me.com>
2023-04-11 20:43:19 -07:00
Oscar Beaumont 3d6afe5447
[ENG-414] P2P Shutdown (#688)
* `Event::Shutdown` in P2P system

* Mdns shutdown broadcast + await shutdown

* Fix mdns service name & prevent poll after shutdown
2023-04-11 15:05:50 +00:00
Oscar Beaumont 4c0e341e55
Add MetadataManager into sd-p2p (#686)
* Add `MetadataManager` into sd-p2p

* Add `update_metadata` helper to P2P manager

* Praise Clippy our lord and saviour

* change mdns advertisement log level

* Casually broke Linux
2023-04-11 10:20:14 +08:00
Vítor Vasconcellos 3304e8f6ce
QuickPreview Component (Needs test on MacOS) (#665)
* Add QuickPreview Component
 - Improve the handling of Range requests
 - Implement logic to answer HEAD and OPTIONS methods
 - Handle CORS pre-flight requests
 - Expand accepted file types
 - Improve error handling of invalid Range requests

* Fix linter errors
 - Add `use std::cmp::min` to custom_uri (Required on MacOS & Windows)
 - Improve logic for retrieving file information in QuickPreview.tsx

* More linter errors

* Simplify `QuickPreview` by extracting the logic for choosing the file preview tag to a `FilePreview` component
 - Fix the typo in `QuickPreview` props name
 - Remove the unused `handleMedia` ref
 - Move the remaining `QuickPreview` logic to the `transitions` callback
 - Simplify the `cors` return type in `custom_uri.rs`

* Refactor range handling in `handle_file` function
 - Move range handling logic to the initialization of the `range` variable
 - Replace `if let` with `match` to reduce code duplication
 - Don't export FilePreview
 - Export QuickPreviewProps

* Fix typo in `RangeNotSatisfiable` error message
 - Remove redundant variables

* Fixing cas_id generation on watcher
Some improvements on watcher file creation

* Rust fmt

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: Jamie Pine <ijamespine@me.com>
2023-04-05 21:15:13 -07:00
Brendan Allan 071af22b42
fix syncing (#679) 2023-04-05 19:47:10 -07:00
Vítor Vasconcellos b711fe8b27
Fix cas_id sample hashing logic (#672) 2023-04-04 13:59:19 +00:00
Brendan Allan ce9be10cdb fix core 2023-04-04 15:12:09 +08:00
Brendan Allan d44a70562b
some more sync impls (#669)
a lil more sync impls
2023-04-03 22:45:00 -07:00