Working Location-Watcher system for iOS app (#2073)

* Working External Storage Locked Locations

We can now add Downloads (A location locked by MANGE_EXTERNAL_STORAGE) on Android.

* Navigate to added page

Adding location now navigates to the added location explorer.

* Way simpler solution

Found a way simpler solution, that doesn't require query calls.

* Clean up

Remove unused import calls.

* Attempt to get Location Watcher working

Well, Location watcher doesn't want to run. But I wanted to push the `xcrun` command to the README.

Also, locations now actually show files, but break on displaying recursive folders (folder content in a folder).

* Quick clean up + Working full rescan

locations.fullRescan works now on mobile with 0 issues. It can recognize orphan paths and when new files are added.

But, Location-Watcher for some reason doesn't want to run automatically when the file system change occurs.

* Working iOS Watcher

Location Watcher now works for iOS when files are created. However, it can't understand the file delete event yet.

* Functional watcher for iOS

The watcher system is now fully functional on iOS, with a few bugs to patch regarding the delete function freezing the system up.

* Update ios.rs

* See Logs on Android

Finally figured out how to see the logs from Android.

* Update CONTRIBUTING.md

* Updated to Stable & Working

* Working Android Build + Explorer working again

* wip

* wip

* Squashed commit of the following:

commit a7bc3b908d
Merge: 201913a1 2899aa3f
Author: Arnab Chakraborty <11457760+Rocky43007@users.noreply.github.com>
Date:   Tue Feb 6 16:18:42 2024 -0500

    Merge branch 'main' of https://github.com/Rocky43007/spacedrive

commit 2899aa3fa5
Author: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Date:   Tue Feb 6 19:30:53 2024 +0300

    [ENG-1594] Change online to connected (#2060)

    : This is a combination of 3 commits.

    Change online to connected

    remove offline

    json

commit 48634c22aa
Author: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Date:   Tue Feb 6 17:49:16 2024 +0300

    [MOB-54] UI Fixes (#2059)

    * UI fixes - rive animation - SD version in settings - and more

    * twStyle

commit 4e70246467
Author: Jesse Rodrigo <39565615+JSSRDRG@users.noreply.github.com>
Date:   Tue Feb 6 11:41:05 2024 +0100

    Dutch locale (#2054)

    * nl locale

    * add nl entry

    * improve some wording

commit bb0d0af6a0
Author: Brendan Allan <brendonovich@outlook.com>
Date:   Tue Feb 6 17:13:47 2024 +0800

    [ENG-1548] use in-memory instances when sending messages to cloud (#2057)

    * use in-memory instances when sending messages to cloud

    * comments

commit 2d0c340e58
Author: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Date:   Tue Feb 6 06:03:33 2024 +0300

    [MOB-47] Location screen and header updates (#2056)

    * Location screen and header updates

    * use tw sizing

    * remove un-necessary prop

    * Nit: change name

commit 58f9305965
Author: Arnab Chakraborty <11457760+Rocky43007@users.noreply.github.com>
Date:   Mon Feb 5 07:06:47 2024 -0500

    Update to Expo 50 and Fix to Rive Crashing (#2049)

    * Update Mobile App to Expo SDK 50
    + Fix to Rive Crashing

    * Added `metro-react-native-babel-transformer` to fix CI

commit 2ff1ffcb9c
Author: Utku <74243531+utkubakir@users.noreply.github.com>
Date:   Sun Feb 4 23:52:26 2024 +0300

    Fix Chinese language (#2050)

    * fix chinese

    * remove console.log

* Squashed commit of the following:

commit bda9a1b6ee
Author: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Date:   Wed Feb 7 20:25:04 2024 +0300

    [MOB-55] Video animation for onboarding on mobile and desktop (#2065)

    * video animation for onboarding on mobile and desktop

    run assets gen

    cleanup

    declare mp4 type

    * update metro config to transform video files from sd assets

    * test ci without native video exclude

    * casing?

    * remove to add back again due to github

    * add videos back

    * versions

    * no need to transform

    ---------

    Co-authored-by: Utku Bakir <74243531+utkubakir@users.noreply.github.com>

commit da2841b37a
Author: Utku <74243531+utkubakir@users.noreply.github.com>
Date:   Wed Feb 7 16:47:55 2024 +0300

    More translations (#2051)

    * translations

    * more translation keys

    * all the translations

* Added Cleaning Script

* Prep for PR

* Clean up + `cargo fmt`

* Update mod.rs

* Squashed commit of the following:

commit 2dc233f1b4
Author: Utku <74243531+utkubakir@users.noreply.github.com>
Date:   Fri Feb 9 18:42:42 2024 +0300

    Update readme & contributing guide & language stuff (#2071)

    * updates

    * keep common errors

    * fix selector being empty for english

    * sort by label

    * update contributing

    * update ndk and docs

    * Update CONTRIBUTING.md

commit 177b2a23d6
Author: Brendan Allan <brendonovich@outlook.com>
Date:   Fri Feb 9 21:20:51 2024 +0800

    sync support for labels (#2070)

    * more sync support for file paths + saved searches

    * sync support for labels

    * update sync prisma generator to support more than tags

    * workey

    * don't do illegal db migration

    * use name as label id in explorer

commit 6f28d8ec28
Author: Brendan Allan <brendonovich@outlook.com>
Date:   Fri Feb 9 16:17:04 2024 +0800

    More sync support for file paths + saved searches (#2067)

    more sync support for file paths + saved searches

* Update build-rust.sh

Fix script so it doesn't build debug apps always now.

* Add Tests for iOS

* Update android.rs

PR 1812 was closed, therefore changing the message to mention the branch instead.

* Change `--debug` to `--release`

Oops

* Remove debug cargo crate for `notify-rs`

* Spelling Fix on `android.rs`

* Squashed commit of the following:

commit dba85ebac3
Author: Ericson "Fogo" Soares <ericson.ds999@gmail.com>
Date:   Mon Feb 26 16:45:58 2024 -0300

    [ENG-1513] Better integration between Jobs and processing Actors (#1974)

    * First draft on new task system

    * Removing save to disk from task system

    * Bunch of concurrency issues

    * Solving Future impl issue when pausing tasks

    * Fix cancel and abort

    * Bunch of fixes on pause, suspend, resume, cancel and abort
    Also better error handling on task completion for the user

    * New capabilities to return an output on a task

    * Introducing a simple way to linear backoff on failed steal

    * Sample actor where tasks can dispatch more tasks

    * Rustfmt

    * Steal test to make sure

    * Stale deps cleanup

    * Removing unused utils

    * Initial lib docs

    * Docs ok

    * Memory cleanup on idle

    ---------

    Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>

commit 53713a9f59
Author: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Date:   Mon Feb 26 18:53:37 2024 +0300

    [ENG-1625] Spacedrop UI correct hover condition & spacing (#2127)

    * improve spacedrop ui with correct hover & spacing

    * remove

commit 6f27504e5f
Author: Matteo Galiazzo <50683509+gekoxyz@users.noreply.github.com>
Date:   Mon Feb 26 16:20:41 2024 +0100

    added it locale, added it entry (#2066)

    * added it locale, added it entry

    * Apply suggestions from code review

    Co-authored-by: Matteo Martellini <matteo@mercxry.me>

    * add missing keys and a readme about the script

    ---------

    Co-authored-by: Utku <74243531+utkubakir@users.noreply.github.com>
    Co-authored-by: Matteo Martellini <matteo@mercxry.me>

commit 28328034f0
Author: jake <77554505+brxken128@users.noreply.github.com>
Date:   Mon Feb 26 15:17:28 2024 +0000

    Clean-up MacOS window closing behaviour code (#2124)

    * fix: delete dead/unused file

    * refactor: add the window event handler with the rest of them

    * refactor: formatting

commit aa0b4abf85
Author: Oscar Beaumont <oscar@otbeaumont.me>
Date:   Mon Feb 26 15:23:48 2024 +0800

    rspc over P2P (#2112)

    * wip: rspc over p2p

    * wip

    * rspc over P2P

    * Cleanup + error handling

    * slight cleanup

    * Using Hyper for HTTP streaming + websockets

commit f7a7b00e37
Author: Utku <74243531+utkubakir@users.noreply.github.com>
Date:   Fri Feb 23 11:26:58 2024 -0500

    Fix android thumbs (#2121)

    * replace react-native-fs with an active fork

    * time sink

    * fix

commit 6358c574c9
Author: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Date:   Fri Feb 23 00:28:35 2024 +0300

    Mob: cleanup warning (#2122)

    Update Categories.tsx

commit a4b7296b45
Author: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Date:   Thu Feb 22 16:08:41 2024 +0300

    MOB: Settings paddings (#2120)

    padding tweaks

commit d007b55763
Author: nikec <43032218+niikeec@users.noreply.github.com>
Date:   Thu Feb 22 13:53:02 2024 +0100

    [ENG-1619] Add spacedrop to the context menu (#2119)

    add spacedrop to context menu

commit dd0acad2e7
Author: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Date:   Thu Feb 22 15:47:08 2024 +0300

    [ENG-1618] Spacedrop UI (#2118)

    * spacedrop ui update

    * i18n and description

    * glitch: remove !

    * already in progress i18n

    * more i18n

commit a17fb910ed
Author: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Date:   Thu Feb 22 15:20:31 2024 +0300

    [MOB-62] Spacing & padding tweaks (#2117)

    Spacing & padding tweaks

commit 6a32752243
Author: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Date:   Thu Feb 22 03:15:36 2024 -0300

    Fix core test and CI breaking (#2116)

    Fix core test passing inverted arguments to sync_db_entry macro

commit da4f038669
Author: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Date:   Wed Feb 21 17:19:40 2024 +0300

    Mob: better visually width fitting for categories (#2114)

    * Visually width fitting for categories

    * remove padding

commit 3bdcc05c2d
Author: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Date:   Wed Feb 21 11:18:15 2024 -0300

    Fix mobile CI + Some small CI improvements (#2113)

    Fix mobile CI
     - Use rust envvars in all workflows
     - Use rust envvars and mold when build sd-server docker

commit b638fc2177
Author: Utku <74243531+utkubakir@users.noreply.github.com>
Date:   Wed Feb 21 08:26:05 2024 -0500

    [MOB-37, MOB-38, MOB-39] Preview for PDF, Text and Media files (#2098)

    * version & microphonePermission text & eslint

    * remove polyfill as hermes supports intl now

    * why do we have solid on mobile?

    * cleanup

    * add solid back =_=

    * pnpm lock

    * we hate relative paths here

    * android config

    * open file logic

    * visual tweaks

    ---------

    Co-authored-by: ameer2468 <33054370+ameer2468@users.noreply.github.com>

commit c533d12df0
Author: Brendan Allan <brendonovich@outlook.com>
Date:   Wed Feb 21 22:42:10 2024 +1100

    media data sync (#2102)

    * basic sync operation backfill

    * media data sync

    * sync entry helpers

    * fix sync generator

    * nicer

    * re-add key_id

commit 393a907b57
Author: Vítor Vasconcellos <vasconcellos.dev@gmail.com>
Date:   Wed Feb 21 06:27:40 2024 -0300

    Update github actions due to nodejs 16 deprecation (#2107)

    * Update github actions due to nodejs 16 deprecation
     - Replace archived actions-rs/clippy-check with maintained fork actions-rs-plus/clippy-check
     - Replace redhat-actions/push-to-registry with updated fork Eusebiotrigo/push-to-registry
     - Point redhat-actions/buildah-build and softprops/action-gh-release to current master to fix nodejs deprecation

    * Build the correct ios core rust arch for CI runs

    * Build ios app for the same arch as the host in Mobile CI

    * Some changes to try and make cache-factory faster and avoid failing so much

    * Add trigger to run cache-factory on pull requests when there are changes to itself

    * Attempt to fix sed usage on macOS

    * Don't treat warning as errors

    * Fix windows

    * Fix windows 2

    * Use target ad cache key for rust to differentiate between macOS x86_64 and arm64

    * Use faster/better linkers to compile for macOS, Linux and Windows

    * Fix missing shell in action

    * Fix typo

    * Fix missing shell in action 2

    * Fix mold download
     - Replace bsdtar with plain tar

    * Fix permission denied when extracting mold

    * Remove zld

    * Don't restore cache for rustfmt
     - Remove target symlink to C:/ in an attempt to speed-up windows CI

    * Fix typo

    * Restore target symlink on windows
     - Removing it didn't make CI faster

    * Run Mobile on macos-14

commit 519b1b6c46
Author: Oscar Beaumont <oscar@otbeaumont.me>
Date:   Wed Feb 21 16:13:40 2024 +0800

    Fix P2P not working for libraries (#2031)

    * P2P Debug route

    * Remove legacy peer to peer pairing process

    * Fix error typo

    * Sync instances with cloud

    * Upgrade deps + extended instance data

    * Create instance with extended metadata

    * Auto sync instances

    * Actually `.await`

    * bruh

    * sync library info

    * this isn't gonna work

    * only sleep cloud receiver when no more messages (#1985)

    * [ENG-1567] Fix renaming (#1986)

    fix rename

    * only sleep cloud receiver when no more messages

    * use in memory instances during cloud receive (#1995)

    * use in memory instances during cloud receive

    * is_empty

    ---------

    Co-authored-by: nikec <43032218+niikeec@users.noreply.github.com>

    * fix type error

    * wip

    * make mdns mdns better

    * rebuild state

    * Add hooks + listeners + discovered state

    * Split into crates

    * wip fixing core + wip merging Spacetime into `sd-p2p2`

    * `SmartLockGuard` + `Listener`

    * Make `sd-core` compile

    * Reenable all operation receivers

    * Fix all broken code within `sd-core`

    * minor fixes found in review

    * Bring in `libp2p` + restructure `sd-p2p` for the gazillion-th time

    * whoops

    * Compile no matter the (runtime) cost

    * fixing merge issues

    * wip

    * a

    * b

    * C

    * Handle port betterer

    * c

    * Migrate node config

    * a

    * no crash on startup

    * wip

    * a

    * jdfhskjfsg

    * a

    * fix discovery

    * a bunch of fixes

    * getting Spacedrop working

    * I don't get why it no worky

    * debug example

    * a

    * wip

    * wip

    * removing logging from stream impl

    * wip: shit is fucked

    * Redo quic integration  + Spacedrop working

    * Fix shutdown - deadlocks + shutdown peers

    * Add Prisma migrations

    * Fix shutdown

    * a

    * fix

    * cleanup

    * The lord clippy hath spoken

    * disable P2P settings for now

    ---------

    Co-authored-by: Brendan Allan <brendonovich@outlook.com>
    Co-authored-by: nikec <43032218+niikeec@users.noreply.github.com>

commit af8dbf7789
Author: Julian Braha <julianbraha@gmail.com>
Date:   Tue Feb 20 20:06:05 2024 +0000

    Improve error handling by using decode::Error instead of io::Result (#2078)

    Use decode::Error instead of io::Result

commit 84dadffa81
Author: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Date:   Tue Feb 20 22:59:11 2024 +0300

    [MOB-59] Empty UI for locations and tags screen (#2110)

    Empty UI for locations and tags screen

commit 9fc38d866a
Author: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Date:   Tue Feb 20 22:47:14 2024 +0300

    [MOB-60] locations & tags query invalidation on updates (#2111)

    Trigger UI updates on location adding/delete and tags

    lint

    name

commit abd5ecbe8d
Author: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Date:   Tue Feb 20 13:05:53 2024 +0300

    mousedown fix (#2108)

    quick mistake fix

commit 9bc1a472a8
Author: Brendan Allan <brendonovich@outlook.com>
Date:   Tue Feb 20 20:22:34 2024 +1100

    Basic sync operation backfill (#2101)

    * basic sync operation backfill

    * no changes

commit 2a283479e6
Author: Arnab Chakraborty <11457760+Rocky43007@users.noreply.github.com>
Date:   Tue Feb 20 01:33:52 2024 -0500

    New Android Build Script (#2096)

    * New Android Build Script

    * Clean up + Works for CI now

    * Simplify android build.sh
     - Fix /var/home/vitor fallback for Linux systems
     - Run a single cargo ndk for all targets (not parallel build, but a bit faster)
     - Fix android target s/x86/x86_64/
     - Format setup.sh
     - Minor improvements to rust mobile targets installation step in setup.sh

    * Add notice to CONTRIBUTING that only Java <= 17 is supported for building android
     - Make prettier ignore some mobile build artifacts

    * When in CI, Fix build android core for host architecture

    ---------

    Co-authored-by: Vítor Vasconcellos <vasconcellos.dev@gmail.com>

commit 19b224370e
Author: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Date:   Mon Feb 19 23:45:29 2024 +0300

    [ENG-1615] bg intro video fixed (#2104)

    * video intro bg

    * test hsl

    * test video bg

    * run tests

    * comment

    * mob intro

    * git glitch

    * git

    * webm type

commit 43360601da
Author: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Date:   Mon Feb 19 19:12:11 2024 +0300

    [MOB-58] Settings routes new design & more (#2103)

    * wip: redesigning settings pages

    * Edit location redesign & more

    * right actions

    * cleanup

commit e8450821df
Author: ameer2468 <33054370+ameer2468@users.noreply.github.com>
Date:   Mon Feb 19 18:23:20 2024 +0300

    [ENG-1612] Fix mouse nav forwards and backwards (#2105)

    Fix mouse nav forwards and backwards

* Clean up commented code

* Clean up + Remove `LocationOnboarding.tsx`
This commit is contained in:
Arnab Chakraborty 2024-02-29 11:14:50 -05:00 committed by GitHub
parent 2ccb83c690
commit 4757676d8e
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
18 changed files with 854 additions and 162 deletions

View file

@ -84,5 +84,9 @@
".eslintrc.js": ".eslintcache",
"package.json": "package-lock.json, yarn.lock, pnpm-lock.yaml, pnpm-workspace.yaml, .pnp.cjs, .pnp.loader.mjs",
"tsconfig.json": "tsconfig.*.json"
}
},
"rust-analyzer.linkedProjects": [],
"rust-analyzer.cargo.extraEnv": {},
"rust-analyzer.check.targets": null,
"rust-analyzer.showUnlinkedFileNotification": false
}

14
Cargo.lock generated
View file

@ -3865,9 +3865,9 @@ dependencies = [
[[package]]
name = "inotify"
version = "0.9.6"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8069d3ec154eb856955c1c0fbffefbf5f3c40a104ec912d4797314c1801abff"
checksum = "fdd168d97690d0b8c412d6b6c10360277f4d7ee495c5d0d5d5fe0854923255cc"
dependencies = [
"bitflags 1.3.2",
"inotify-sys",
@ -5357,19 +5357,19 @@ dependencies = [
[[package]]
name = "notify"
version = "5.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "729f63e1ca555a43fe3efa4f3efdf4801c479da85b432242a7b726f353c88486"
version = "6.1.1"
source = "git+https://github.com/notify-rs/notify.git?rev=c3929ed114fbb0bc7457a9a498260461596b00ca#c3929ed114fbb0bc7457a9a498260461596b00ca"
dependencies = [
"bitflags 1.3.2",
"bitflags 2.4.1",
"filetime",
"fsevent-sys",
"inotify",
"kqueue",
"libc",
"log",
"mio",
"walkdir",
"windows-sys 0.45.0",
"windows-sys 0.48.0",
]
[[package]]

View file

@ -11,7 +11,6 @@ edition = { workspace = true }
[dependencies]
sd-core = { path = "../../../core", features = [
"ffmpeg",
"location-watcher",
"heif",
] }
sd-fda = { path = "../../../crates/fda" }

View file

@ -0,0 +1,36 @@
#!/bin/bash
# Check if the correct number of arguments is provided
if [ "$#" -ne 1 ]; then
echo "Usage: $0 [android|ios]"
exit 1
fi
# Set the target folder based on the first argument
TARGET_FOLDER="target"
# Check if the target folder exists
if [ ! -d "$TARGET_FOLDER" ]; then
echo "Target folder '$TARGET_FOLDER' not found."
exit 1
fi
# Set the keyword based on the first argument
KEYWORD=""
if [ "$1" == "android" ]; then
KEYWORD="android"
elif [ "$1" == "ios" ]; then
KEYWORD="ios"
else
echo "Invalid argument. Please provide either 'android' or 'ios'."
exit 1
fi
# Delete files based on the target folder and keyword
echo "Deleting files in '$TARGET_FOLDER' with keyword '$KEYWORD' in folder names..."
# Find and delete files in folders containing the specified keyword
find "$TARGET_FOLDER" -type d -name "*$KEYWORD*" -exec rm -r {} \;
# End of the script
echo "Files deleted successfully."

View file

@ -28,71 +28,71 @@ export default function TabNavigator() {
labelStyle: Style;
testID: string;
}[] = [
{
name: 'OverviewStack',
component: OverviewStack,
icon: (
<TabBarButton
resourceName="tabs"
animationName="animate"
artboardName="overview"
style={{ width: 28 }}
active={activeIndex === 0}
/>
),
label: 'Overview',
labelStyle: tw`text-[10px] font-semibold`,
testID: 'overview-tab'
},
{
name: 'NetworkStack',
component: NetworkStack,
icon: (
<TabBarButton
resourceName="tabs"
animationName="animate"
artboardName="network"
style={{ width: 18, maxHeight: 23 }}
active={activeIndex === 1}
/>
),
label: 'Network',
labelStyle: tw`text-[10px] font-semibold`,
testID: 'network-tab'
},
{
name: 'BrowseStack',
component: BrowseStack,
icon: (
<TabBarButton
resourceName="tabs"
animationName="animate"
artboardName="browse"
style={{ width: 20 }}
active={activeIndex === 2}
/>
),
label: 'Browse',
labelStyle: tw`text-[10px] font-semibold`,
testID: 'browse-tab'
},
{
name: 'SettingsStack',
component: SettingsStack,
icon: (
<TabBarButton
resourceName="tabs"
animationName="animate"
artboardName="settings"
style={{ width: 19 }}
active={activeIndex === 3}
/>
),
label: 'Settings',
labelStyle: tw`text-[10px] font-semibold`,
testID: 'settings-tab'
}
];
{
name: 'OverviewStack',
component: OverviewStack,
icon: (
<TabBarButton
resourceName="tabs"
animationName="animate"
artboardName="overview"
style={{ width: 28 }}
active={activeIndex === 0}
/>
),
label: 'Overview',
labelStyle: tw`text-[10px] font-semibold`,
testID: 'overview-tab'
},
{
name: 'NetworkStack',
component: NetworkStack,
icon: (
<TabBarButton
resourceName="tabs"
animationName="animate"
artboardName="network"
style={{ width: 18, maxHeight: 23 }}
active={activeIndex === 1}
/>
),
label: 'Network',
labelStyle: tw`text-[10px] font-semibold`,
testID: 'network-tab'
},
{
name: 'BrowseStack',
component: BrowseStack,
icon: (
<TabBarButton
resourceName="tabs"
animationName="animate"
artboardName="browse"
style={{ width: 20 }}
active={activeIndex === 2}
/>
),
label: 'Browse',
labelStyle: tw`text-[10px] font-semibold`,
testID: 'browse-tab'
},
{
name: 'SettingsStack',
component: SettingsStack,
icon: (
<TabBarButton
resourceName="tabs"
animationName="animate"
artboardName="settings"
style={{ width: 19 }}
active={activeIndex === 3}
/>
),
label: 'Settings',
labelStyle: tw`text-[10px] font-semibold`,
testID: 'settings-tab'
}
];
return (
<Tab.Navigator
id="tab"

View file

@ -6,7 +6,6 @@ import SearchScreen from '~/screens/Search';
import TabNavigator, { TabParamList } from './TabNavigator';
const Stack = createStackNavigator<RootStackParamList>();
// This is the main navigator we nest everything under.
export default function RootNavigator() {
return (
@ -38,6 +37,6 @@ export type RootStackScreenProps<Screen extends keyof RootStackParamList> = Stac
declare global {
// eslint-disable-next-line @typescript-eslint/no-namespace
namespace ReactNavigation {
interface RootParamList extends RootStackParamList {}
interface RootParamList extends RootStackParamList { }
}
}

View file

@ -25,7 +25,7 @@ export default function LocationScreen({ navigation, route }: BrowseStackScreenP
take: 100
}
]);
const pathsItemsReferences = useMemo(() => paths.data?.items ?? [], [paths.data]);
useNodes(paths.data?.nodes);
const pathsItems = useCache(pathsItemsReferences);
@ -52,5 +52,5 @@ export default function LocationScreen({ navigation, route }: BrowseStackScreenP
getExplorerStore().path = path ?? '';
}, [id, path]);
return <Explorer items={pathsItems} />;
return <Explorer items={pathsItems} />
}

View file

@ -22,11 +22,10 @@ const AboutScreen = () => {
<View style={tw.style('flex flex-col')}>
<Text style={tw.style('text-2xl font-bold text-white')}>
Spacedrive{' '}
{`for ${
Platform.OS === 'android'
? Platform.OS[0]?.toUpperCase() + Platform.OS.slice(1)
: Platform.OS[0] + Platform.OS.slice(1).toUpperCase()
}`}
{`for ${Platform.OS === 'android'
? Platform.OS[0]?.toUpperCase() + Platform.OS.slice(1)
: Platform.OS[0] + Platform.OS.slice(1).toUpperCase()
}`}
</Text>
<Text style={tw.style('mt-1 text-sm text-ink-dull')}>
The file manager from the future.

View file

@ -13,7 +13,6 @@ ai-models = ["sd-core/ai"]
[dependencies]
sd-core = { path = "../../core", features = [
"ffmpeg",
"location-watcher",
"heif",
] }

View file

@ -14,7 +14,6 @@ default = []
mobile = []
# This feature controls whether the Spacedrive Core contains functionality which requires FFmpeg.
ffmpeg = ["dep:sd-ffmpeg"]
location-watcher = ["dep:notify"]
heif = ["sd-images/heif"]
ai = ["dep:sd-ai"]
@ -48,7 +47,6 @@ sd-sync = { path = "../crates/sync" }
sd-utils = { path = "../crates/utils" }
sd-cloud-api = { version = "0.1.0", path = "../crates/cloud-api" }
# Workspace dependencies
async-channel = { workspace = true }
async-trait = { workspace = true }
@ -112,9 +110,9 @@ http-range = "0.1.5"
int-enum = "0.5.0"
itertools = "0.12.0"
mini-moka = "0.10.2"
notify = { version = "=5.2.0", default-features = false, features = [
"macos_fsevent",
], optional = true }
notify = { git="https://github.com/notify-rs/notify.git", rev="c3929ed114fbb0bc7457a9a498260461596b00ca", default-features = false, features = [
"macos_fsevent",
] }
rmpv = "^1.0.1"
serde-hashkey = "0.4.5"
serde_repr = "0.1"

View file

@ -366,7 +366,6 @@ pub(crate) fn mount() -> AlphaRouter<Ctx> {
pub location_id: location::id::Type,
pub reidentify_objects: bool,
}
R.with2(library()).mutation(
|(node, library),
FullRescanArgs {

View file

@ -206,6 +206,8 @@ impl Node {
"info"
};
// let level = "debug"; // Exists for now to debug the location manager
std::env::set_var(
"RUST_LOG",
format!("info,sd_core={level},sd_p2p=debug,sd_core::location::manager=info,sd_ai={level}"),

View file

@ -20,16 +20,13 @@ use tokio::sync::{
broadcast::{self, Receiver},
oneshot, RwLock,
};
use tracing::error;
use tracing::{debug, error};
#[cfg(feature = "location-watcher")]
use tokio::sync::mpsc;
use uuid::Uuid;
#[cfg(feature = "location-watcher")]
mod watcher;
#[cfg(feature = "location-watcher")]
mod helpers;
#[derive(Clone, Copy, Debug)]
@ -67,22 +64,18 @@ pub struct WatcherManagementMessage {
#[derive(Error, Debug)]
pub enum LocationManagerError {
#[cfg(feature = "location-watcher")]
#[error("Unable to send location management message to location manager actor: (error: {0})")]
ActorSendLocationError(#[from] mpsc::error::SendError<LocationManagementMessage>),
#[cfg(feature = "location-watcher")]
#[error("Unable to send path to be ignored by watcher actor: (error: {0})")]
ActorIgnorePathError(#[from] mpsc::error::SendError<watcher::IgnorePath>),
#[cfg(feature = "location-watcher")]
#[error("Unable to watcher management message to watcher manager actor: (error: {0})")]
ActorIgnorePathMessageError(#[from] mpsc::error::SendError<WatcherManagementMessage>),
#[error("Unable to receive actor response: (error: {0})")]
ActorResponseError(#[from] oneshot::error::RecvError),
#[cfg(feature = "location-watcher")]
#[error("Watcher error: (error: {0})")]
WatcherError(#[from] notify::Error),
@ -119,11 +112,10 @@ type OnlineLocations = BTreeSet<Vec<u8>>;
#[must_use = "'LocationManagerActor::start' must be used to start the actor"]
pub struct LocationManagerActor {
#[cfg(feature = "location-watcher")]
location_management_rx: mpsc::Receiver<LocationManagementMessage>,
#[cfg(feature = "location-watcher")]
watcher_management_rx: mpsc::Receiver<WatcherManagementMessage>,
#[cfg(feature = "location-watcher")]
stop_rx: oneshot::Receiver<()>,
}
@ -178,25 +170,21 @@ impl LocationManagerActor {
}
});
#[cfg(feature = "location-watcher")]
tokio::spawn(Locations::run_locations_checker(
self.location_management_rx,
self.watcher_management_rx,
self.stop_rx,
node,
));
#[cfg(not(feature = "location-watcher"))]
tracing::warn!("Location watcher is disabled, locations will not be checked");
}
}
pub struct Locations {
online_locations: RwLock<OnlineLocations>,
pub online_tx: broadcast::Sender<OnlineLocations>,
#[cfg(feature = "location-watcher")]
location_management_tx: mpsc::Sender<LocationManagementMessage>,
#[cfg(feature = "location-watcher")]
watcher_management_tx: mpsc::Sender<WatcherManagementMessage>,
stop_tx: Option<oneshot::Sender<()>>,
}
@ -205,11 +193,11 @@ impl Locations {
pub fn new() -> (Self, LocationManagerActor) {
let online_tx = broadcast::channel(16).0;
#[cfg(feature = "location-watcher")]
{
let (location_management_tx, location_management_rx) = mpsc::channel(128);
let (watcher_management_tx, watcher_management_rx) = mpsc::channel(128);
let (stop_tx, stop_rx) = oneshot::channel();
debug!("Starting location manager actor");
(
Self {
@ -226,19 +214,6 @@ impl Locations {
},
)
}
#[cfg(not(feature = "location-watcher"))]
{
tracing::warn!("Location watcher is disabled, locations will not be checked");
(
Self {
online_tx,
online_locations: Default::default(),
stop_tx: None,
},
LocationManagerActor {},
)
}
}
#[inline]
@ -249,9 +224,9 @@ impl Locations {
library: Arc<Library>,
action: ManagementMessageAction,
) -> Result<(), LocationManagerError> {
#[cfg(feature = "location-watcher")]
{
let (tx, rx) = oneshot::channel();
debug!("Sending location management message to location manager actor: {action:?}");
self.location_management_tx
.send(LocationManagementMessage {
@ -264,9 +239,6 @@ impl Locations {
rx.await?
}
#[cfg(not(feature = "location-watcher"))]
Ok(())
}
#[inline]
@ -277,10 +249,11 @@ impl Locations {
library: Arc<Library>,
action: WatcherManagementMessageAction,
) -> Result<(), LocationManagerError> {
#[cfg(feature = "location-watcher")]
{
let (tx, rx) = oneshot::channel();
debug!("Sending watcher management message to location manager actor: {action:?}");
self.watcher_management_tx
.send(WatcherManagementMessage {
location_id,
@ -292,9 +265,6 @@ impl Locations {
rx.await?
}
#[cfg(not(feature = "location-watcher"))]
Ok(())
}
pub async fn add(
@ -377,7 +347,6 @@ impl Locations {
})
}
#[cfg(feature = "location-watcher")]
async fn run_locations_checker(
mut location_management_rx: mpsc::Receiver<LocationManagementMessage>,
mut watcher_management_rx: mpsc::Receiver<WatcherManagementMessage>,
@ -388,7 +357,7 @@ impl Locations {
use futures::stream::{FuturesUnordered, StreamExt};
use tokio::select;
use tracing::{info, warn};
use tracing::warn;
use helpers::{
check_online, drop_location, get_location, handle_ignore_path_request,
@ -430,6 +399,8 @@ impl Locations {
(location_id, library.id),
watcher
);
debug!("Location {location_id} is online, watching it");
// info!("Locations watched: {:#?}", locations_watched);
} else {
locations_unwatched.insert(
(location_id, library.id),
@ -578,7 +549,7 @@ impl Locations {
}
_ = &mut stop_rx => {
info!("Stopping location manager");
debug!("Stopping location manager");
break;
}
}
@ -642,14 +613,12 @@ pub struct StopWatcherGuard<'m> {
impl Drop for StopWatcherGuard<'_> {
fn drop(&mut self) {
if cfg!(feature = "location-watcher") {
// FIXME: change this Drop to async drop in the future
if let Err(e) = block_on(self.manager.reinit_watcher(
self.location_id,
self.library.take().expect("library should be set"),
)) {
error!("Failed to reinit watcher on stop watcher guard drop: {e}");
}
// FIXME: change this Drop to async drop in the future
if let Err(e) = block_on(self.manager.reinit_watcher(
self.location_id,
self.library.take().expect("library should be set"),
)) {
error!("Failed to reinit watcher on stop watcher guard drop: {e}");
}
}
}
@ -664,18 +633,16 @@ pub struct IgnoreEventsForPathGuard<'m> {
impl Drop for IgnoreEventsForPathGuard<'_> {
fn drop(&mut self) {
if cfg!(feature = "location-watcher") {
// FIXME: change this Drop to async drop in the future
if let Err(e) = block_on(self.manager.watcher_management_message(
self.location_id,
self.library.take().expect("library should be set"),
WatcherManagementMessageAction::IgnoreEventsForPath {
path: self.path.take().expect("path should be set"),
ignore: false,
},
)) {
error!("Failed to un-ignore path on watcher guard drop: {e}");
}
// FIXME: change this Drop to async drop in the future
if let Err(e) = block_on(self.manager.watcher_management_message(
self.location_id,
self.library.take().expect("library should be set"),
WatcherManagementMessageAction::IgnoreEventsForPath {
path: self.path.take().expect("path should be set"),
ignore: false,
},
)) {
error!("Failed to un-ignore path on watcher guard drop: {e}");
}
}
}

View file

@ -0,0 +1,271 @@
//! Android file system watcher implementation.
//! TODO: Still being worked on by @Rocky43007 on Branch Rocky43007:location-watcher-test-3
//! DO NOT TOUCH FOR NOW
use crate::{invalidate_query, library::Library, location::manager::LocationManagerError, Node};
use sd_prisma::prisma::location;
use sd_utils::error::FileIOError;
use std::{
collections::{BTreeMap, HashMap},
path::{Path, PathBuf},
sync::Arc,
};
use async_trait::async_trait;
use notify::{
event::{CreateKind, DataChange, ModifyKind, RenameMode},
Event,
};
use tokio::{fs, time::Instant};
use tracing::{error, info, trace};
use super::{
utils::{create_dir, recalculate_directories_size, remove, rename, update_file},
EventHandler, HUNDRED_MILLIS, ONE_SECOND,
};
#[derive(Debug)]
pub(super) struct AndroidEventHandler<'lib> {
location_id: location::id::Type,
library: &'lib Arc<Library>,
node: &'lib Arc<Node>,
last_events_eviction_check: Instant,
rename_from: HashMap<PathBuf, Instant>,
recently_renamed_from: BTreeMap<PathBuf, Instant>,
files_to_update: HashMap<PathBuf, Instant>,
reincident_to_update_files: HashMap<PathBuf, Instant>,
to_recalculate_size: HashMap<PathBuf, Instant>,
path_and_instant_buffer: Vec<(PathBuf, Instant)>,
}
#[async_trait]
impl<'lib> EventHandler<'lib> for AndroidEventHandler<'lib> {
fn new(
location_id: location::id::Type,
library: &'lib Arc<Library>,
node: &'lib Arc<Node>,
) -> Self {
Self {
location_id,
library,
node,
last_events_eviction_check: Instant::now(),
rename_from: HashMap::new(),
recently_renamed_from: BTreeMap::new(),
files_to_update: HashMap::new(),
reincident_to_update_files: HashMap::new(),
to_recalculate_size: HashMap::new(),
path_and_instant_buffer: Vec::new(),
}
}
async fn handle_event(&mut self, event: Event) -> Result<(), LocationManagerError> {
info!("Received Android event: {:#?}", event);
// let Event {
// kind, mut paths, ..
// } = event;
// match kind {
// EventKind::Create(CreateKind::File)
// | EventKind::Modify(ModifyKind::Data(DataChange::Any)) => {
// // When we receive a create, modify data or metadata events of the abore kinds
// // we just mark the file to be updated in a near future
// // each consecutive event of these kinds that we receive for the same file
// // we just store the path again in the map below, with a new instant
// // that effectively resets the timer for the file to be updated
// let path = paths.remove(0);
// if self.files_to_update.contains_key(&path) {
// if let Some(old_instant) =
// self.files_to_update.insert(path.clone(), Instant::now())
// {
// self.reincident_to_update_files
// .entry(path)
// .or_insert(old_instant);
// }
// } else {
// self.files_to_update.insert(path, Instant::now());
// }
// }
// EventKind::Create(CreateKind::Folder) => {
// let path = &paths[0];
// // Don't need to dispatch a recalculate directory event as `create_dir` dispatches
// // a `scan_location_sub_path` function, which recalculates the size already
// create_dir(
// self.location_id,
// path,
// &fs::metadata(path)
// .await
// .map_err(|e| FileIOError::from((path, e)))?,
// self.node,
// self.library,
// )
// .await?;
// }
// EventKind::Modify(ModifyKind::Name(RenameMode::From)) => {
// // Just in case we can't garantee that we receive the Rename From event before the
// // Rename Both event. Just a safeguard
// if self.recently_renamed_from.remove(&paths[0]).is_none() {
// self.rename_from.insert(paths.remove(0), Instant::now());
// }
// }
// EventKind::Modify(ModifyKind::Name(RenameMode::Both)) => {
// let from_path = &paths[0];
// let to_path = &paths[1];
// self.rename_from.remove(from_path);
// rename(
// self.location_id,
// to_path,
// from_path,
// fs::metadata(to_path)
// .await
// .map_err(|e| FileIOError::from((to_path, e)))?,
// self.library,
// )
// .await?;
// self.recently_renamed_from
// .insert(paths.swap_remove(0), Instant::now());
// }
// EventKind::Remove(_) => {
// let path = paths.remove(0);
// if let Some(parent) = path.parent() {
// if parent != Path::new("") {
// self.to_recalculate_size
// .insert(parent.to_path_buf(), Instant::now());
// }
// }
// remove(self.location_id, &path, self.library).await?;
// }
// other_event_kind => {
// trace!("Other Linux event that we don't handle for now: {other_event_kind:#?}");
// }
// }
Ok(())
}
async fn tick(&mut self) {
if self.last_events_eviction_check.elapsed() > HUNDRED_MILLIS {
if let Err(e) = self.handle_to_update_eviction().await {
error!("Error while handling recently created or update files eviction: {e:#?}");
}
if let Err(e) = self.handle_rename_from_eviction().await {
error!("Failed to remove file_path: {e:#?}");
}
self.recently_renamed_from
.retain(|_, instant| instant.elapsed() < HUNDRED_MILLIS);
if !self.to_recalculate_size.is_empty() {
if let Err(e) = recalculate_directories_size(
&mut self.to_recalculate_size,
&mut self.path_and_instant_buffer,
self.location_id,
self.library,
)
.await
{
error!("Failed to recalculate directories size: {e:#?}");
}
}
self.last_events_eviction_check = Instant::now();
}
}
}
impl AndroidEventHandler<'_> {
async fn handle_to_update_eviction(&mut self) -> Result<(), LocationManagerError> {
self.path_and_instant_buffer.clear();
let mut should_invalidate = false;
for (path, created_at) in self.files_to_update.drain() {
if created_at.elapsed() < HUNDRED_MILLIS * 5 {
self.path_and_instant_buffer.push((path, created_at));
} else {
if let Some(parent) = path.parent() {
if parent != Path::new("") {
self.to_recalculate_size
.insert(parent.to_path_buf(), Instant::now());
}
}
self.reincident_to_update_files.remove(&path);
update_file(self.location_id, &path, self.node, self.library).await?;
should_invalidate = true;
}
}
self.files_to_update
.extend(self.path_and_instant_buffer.drain(..));
self.path_and_instant_buffer.clear();
// We have to check if we have any reincident files to update and update them after a bigger
// timeout, this way we keep track of files being update frequently enough to bypass our
// eviction check above
for (path, created_at) in self.reincident_to_update_files.drain() {
if created_at.elapsed() < ONE_SECOND * 10 {
self.path_and_instant_buffer.push((path, created_at));
} else {
if let Some(parent) = path.parent() {
if parent != Path::new("") {
self.to_recalculate_size
.insert(parent.to_path_buf(), Instant::now());
}
}
self.files_to_update.remove(&path);
update_file(self.location_id, &path, self.node, self.library).await?;
should_invalidate = true;
}
}
if should_invalidate {
invalidate_query!(self.library, "search.paths");
}
self.reincident_to_update_files
.extend(self.path_and_instant_buffer.drain(..));
Ok(())
}
async fn handle_rename_from_eviction(&mut self) -> Result<(), LocationManagerError> {
self.path_and_instant_buffer.clear();
let mut should_invalidate = false;
for (path, instant) in self.rename_from.drain() {
if instant.elapsed() > HUNDRED_MILLIS {
if let Some(parent) = path.parent() {
if parent != Path::new("") {
self.to_recalculate_size
.insert(parent.to_path_buf(), Instant::now());
}
}
remove(self.location_id, &path, self.library).await?;
should_invalidate = true;
trace!("Removed file_path due timeout: {}", path.display());
} else {
self.path_and_instant_buffer.push((path, instant));
}
}
if should_invalidate {
invalidate_query!(self.library, "search.paths");
}
for (path, instant) in self.path_and_instant_buffer.drain(..) {
self.rename_from.insert(path, instant);
}
Ok(())
}
}

View file

@ -0,0 +1,394 @@
//! iOS file system watcher implementation.
use crate::{invalidate_query, library::Library, location::manager::LocationManagerError, Node};
use sd_file_path_helper::{check_file_path_exists, get_inode, FilePathError, IsolatedFilePathData};
use sd_prisma::prisma::location;
use sd_utils::error::FileIOError;
use std::{
collections::HashMap,
path::{Path, PathBuf},
sync::Arc,
};
use async_trait::async_trait;
use notify::{
event::{CreateKind, DataChange, MetadataKind, ModifyKind, RenameMode},
Event, EventKind,
};
use tokio::{fs, io, time::Instant};
use tracing::{debug, error, trace, warn};
use super::{
utils::{
create_dir, create_file, extract_inode_from_path, extract_location_path,
recalculate_directories_size, remove, rename, update_file,
},
EventHandler, INode, InstantAndPath, HUNDRED_MILLIS, ONE_SECOND,
};
#[derive(Debug)]
pub(super) struct IosEventHandler<'lib> {
location_id: location::id::Type,
library: &'lib Arc<Library>,
node: &'lib Arc<Node>,
files_to_update: HashMap<PathBuf, Instant>,
reincident_to_update_files: HashMap<PathBuf, Instant>,
last_events_eviction_check: Instant,
latest_created_dir: Option<PathBuf>,
old_paths_map: HashMap<INode, InstantAndPath>,
new_paths_map: HashMap<INode, InstantAndPath>,
paths_map_buffer: Vec<(INode, InstantAndPath)>,
to_recalculate_size: HashMap<PathBuf, Instant>,
path_and_instant_buffer: Vec<(PathBuf, Instant)>,
}
#[async_trait]
impl<'lib> EventHandler<'lib> for IosEventHandler<'lib> {
fn new(
location_id: location::id::Type,
library: &'lib Arc<Library>,
node: &'lib Arc<Node>,
) -> Self
where
Self: Sized,
{
Self {
location_id,
library,
node,
files_to_update: HashMap::new(),
reincident_to_update_files: HashMap::new(),
last_events_eviction_check: Instant::now(),
latest_created_dir: None,
old_paths_map: HashMap::new(),
new_paths_map: HashMap::new(),
paths_map_buffer: Vec::new(),
to_recalculate_size: HashMap::new(),
path_and_instant_buffer: Vec::new(),
}
}
async fn handle_event(&mut self, event: Event) -> Result<(), LocationManagerError> {
let Event {
kind, mut paths, ..
} = event;
match kind {
EventKind::Create(CreateKind::Folder) => {
let path = &paths[0];
create_dir(
self.location_id,
path,
&fs::metadata(path)
.await
.map_err(|e| FileIOError::from((path, e)))?,
self.node,
self.library,
)
.await?;
self.latest_created_dir = Some(paths.remove(0));
}
EventKind::Create(CreateKind::File)
| EventKind::Modify(ModifyKind::Data(DataChange::Content))
| EventKind::Modify(ModifyKind::Metadata(
MetadataKind::WriteTime | MetadataKind::Extended,
)) => {
// When we receive a create, modify data or metadata events of the abore kinds
// we just mark the file to be updated in a near future
// each consecutive event of these kinds that we receive for the same file
// we just store the path again in the map below, with a new instant
// that effectively resets the timer for the file to be updated <- Copied from macos.rs
let path = paths.remove(0);
if self.files_to_update.contains_key(&path) {
if let Some(old_instant) =
self.files_to_update.insert(path.clone(), Instant::now())
{
self.reincident_to_update_files
.entry(path)
.or_insert(old_instant);
}
} else {
self.files_to_update.insert(path, Instant::now());
}
}
EventKind::Modify(ModifyKind::Name(RenameMode::Any)) => {
self.handle_single_rename_event(paths.remove(0)).await?;
}
// For some reason, iOS doesn't have a Delete Event, so the vent type comes up as this.
// Delete Event
EventKind::Modify(ModifyKind::Metadata(MetadataKind::Any)) => {
debug!("File has been deleted: {:#?}", paths);
let path = paths.remove(0);
if let Some(parent) = path.parent() {
if parent != Path::new("") {
self.to_recalculate_size
.insert(parent.to_path_buf(), Instant::now());
}
}
remove(self.location_id, &path, self.library).await?; //FIXME: Find out why this freezes the watcher
}
other_event_kind => {
trace!("Other iOS event that we don't handle for now: {other_event_kind:#?}");
}
}
Ok(())
}
async fn tick(&mut self) {
if self.last_events_eviction_check.elapsed() > HUNDRED_MILLIS {
if let Err(e) = self.handle_to_update_eviction().await {
error!("Error while handling recently created or update files eviction: {e:#?}");
}
// Cleaning out recently renamed files that are older than 100 milliseconds
if let Err(e) = self.handle_rename_create_eviction().await {
error!("Failed to create file_path on iOS : {e:#?}");
}
if let Err(e) = self.handle_rename_remove_eviction().await {
error!("Failed to remove file_path: {e:#?}");
}
if !self.to_recalculate_size.is_empty() {
if let Err(e) = recalculate_directories_size(
&mut self.to_recalculate_size,
&mut self.path_and_instant_buffer,
self.location_id,
self.library,
)
.await
{
error!("Failed to recalculate directories size: {e:#?}");
}
}
self.last_events_eviction_check = Instant::now();
}
}
}
impl IosEventHandler<'_> {
async fn handle_to_update_eviction(&mut self) -> Result<(), LocationManagerError> {
self.path_and_instant_buffer.clear();
let mut should_invalidate = false;
for (path, created_at) in self.files_to_update.drain() {
if created_at.elapsed() < HUNDRED_MILLIS * 5 {
self.path_and_instant_buffer.push((path, created_at));
} else {
if let Some(parent) = path.parent() {
if parent != Path::new("") {
self.to_recalculate_size
.insert(parent.to_path_buf(), Instant::now());
}
}
self.reincident_to_update_files.remove(&path);
update_file(self.location_id, &path, self.node, self.library).await?;
should_invalidate = true;
}
}
self.files_to_update
.extend(self.path_and_instant_buffer.drain(..));
self.path_and_instant_buffer.clear();
// We have to check if we have any reincident files to update and update them after a bigger
// timeout, this way we keep track of files being update frequently enough to bypass our
// eviction check above
for (path, created_at) in self.reincident_to_update_files.drain() {
if created_at.elapsed() < ONE_SECOND * 10 {
self.path_and_instant_buffer.push((path, created_at));
} else {
if let Some(parent) = path.parent() {
if parent != Path::new("") {
self.to_recalculate_size
.insert(parent.to_path_buf(), Instant::now());
}
}
self.files_to_update.remove(&path);
update_file(self.location_id, &path, self.node, self.library).await?;
should_invalidate = true;
}
}
if should_invalidate {
invalidate_query!(self.library, "search.paths");
}
self.reincident_to_update_files
.extend(self.path_and_instant_buffer.drain(..));
Ok(())
}
async fn handle_rename_create_eviction(&mut self) -> Result<(), LocationManagerError> {
// Just to make sure that our buffer is clean
self.paths_map_buffer.clear();
let mut should_invalidate = false;
for (inode, (instant, path)) in self.new_paths_map.drain() {
if instant.elapsed() > HUNDRED_MILLIS {
if !self.files_to_update.contains_key(&path) {
let metadata = fs::metadata(&path)
.await
.map_err(|e| FileIOError::from((&path, e)))?;
if metadata.is_dir() {
// Don't need to dispatch a recalculate directory event as `create_dir` dispatches
// a `scan_location_sub_path` function, which recalculates the size already
create_dir(self.location_id, &path, &metadata, self.node, self.library)
.await?;
} else {
if let Some(parent) = path.parent() {
if parent != Path::new("") {
self.to_recalculate_size
.insert(parent.to_path_buf(), Instant::now());
}
}
create_file(self.location_id, &path, &metadata, self.node, self.library)
.await?;
}
trace!("Created file_path due timeout: {}", path.display());
should_invalidate = true;
}
} else {
self.paths_map_buffer.push((inode, (instant, path)));
}
}
if should_invalidate {
invalidate_query!(self.library, "search.paths");
}
self.new_paths_map.extend(self.paths_map_buffer.drain(..));
Ok(())
}
async fn handle_rename_remove_eviction(&mut self) -> Result<(), LocationManagerError> {
// Just to make sure that our buffer is clean
self.paths_map_buffer.clear();
let mut should_invalidate = false;
for (inode, (instant, path)) in self.old_paths_map.drain() {
if instant.elapsed() > HUNDRED_MILLIS {
if let Some(parent) = path.parent() {
if parent != Path::new("") {
self.to_recalculate_size
.insert(parent.to_path_buf(), Instant::now());
}
}
remove(self.location_id, &path, self.library).await?;
trace!("Removed file_path due timeout: {}", path.display());
should_invalidate = true;
} else {
self.paths_map_buffer.push((inode, (instant, path)));
}
}
if should_invalidate {
invalidate_query!(self.library, "search.paths");
}
self.old_paths_map.extend(self.paths_map_buffer.drain(..));
Ok(())
}
async fn handle_single_rename_event(
&mut self,
path: PathBuf, // this is used internally only once, so we can use just PathBuf
) -> Result<(), LocationManagerError> {
match fs::metadata(&path).await {
Ok(meta) => {
// File or directory exists, so this can be a "new path" to an actual rename/move or a creation
trace!("Path exists: {}", path.display());
let inode = get_inode(&meta);
let location_path = extract_location_path(self.location_id, self.library).await?;
if !check_file_path_exists::<FilePathError>(
&IsolatedFilePathData::new(
self.location_id,
&location_path,
&path,
meta.is_dir(),
)?,
&self.library.db,
)
.await?
{
if let Some((_, old_path)) = self.old_paths_map.remove(&inode) {
trace!(
"Got a match new -> old: {} -> {}",
path.display(),
old_path.display()
);
// We found a new path for this old path, so we can rename it
rename(self.location_id, &path, &old_path, meta, self.library).await?;
} else {
trace!("No match for new path yet: {}", path.display());
self.new_paths_map.insert(inode, (Instant::now(), path));
}
} else {
warn!(
"Received rename event for a file that already exists in the database: {}",
path.display()
);
}
}
Err(e) if e.kind() == io::ErrorKind::NotFound => {
// File or directory does not exist in the filesystem, if it exists in the database,
// then we try pairing it with the old path from our map
trace!("Path doesn't exists: {}", path.display());
let inode =
match extract_inode_from_path(self.location_id, &path, self.library).await {
Ok(inode) => inode,
Err(LocationManagerError::FilePath(FilePathError::NotFound(_))) => {
// temporary file, we can ignore it
return Ok(());
}
Err(e) => return Err(e),
};
if let Some((_, new_path)) = self.new_paths_map.remove(&inode) {
trace!(
"Got a match old -> new: {} -> {}",
path.display(),
new_path.display()
);
// We found a new path for this old path, so we can rename it
rename(
self.location_id,
&new_path,
&path,
fs::metadata(&new_path)
.await
.map_err(|e| FileIOError::from((&new_path, e)))?,
self.library,
)
.await?;
} else {
trace!("No match for old path yet: {}", path.display());
// We didn't find a new path for this old path, so we store ir for later
self.old_paths_map.insert(inode, (Instant::now(), path));
}
}
Err(e) => return Err(FileIOError::from((path, e)).into()),
}
Ok(())
}
}

View file

@ -24,6 +24,8 @@ use uuid::Uuid;
use super::LocationManagerError;
mod android;
mod ios;
mod linux;
mod macos;
mod windows;
@ -41,6 +43,12 @@ type Handler<'lib> = macos::MacOsEventHandler<'lib>;
#[cfg(target_os = "windows")]
type Handler<'lib> = windows::WindowsEventHandler<'lib>;
#[cfg(target_os = "android")]
type Handler<'lib> = android::AndroidEventHandler<'lib>;
#[cfg(target_os = "ios")]
type Handler<'lib> = ios::IosEventHandler<'lib>;
pub(super) type IgnorePath = (PathBuf, bool);
type INode = u64;
@ -142,12 +150,12 @@ impl LocationWatcher {
let mut handler_interval = interval_at(Instant::now() + HUNDRED_MILLIS, HUNDRED_MILLIS);
// In case of doubt check: https://docs.rs/tokio/latest/tokio/time/enum.MissedTickBehavior.html
handler_interval.set_missed_tick_behavior(MissedTickBehavior::Delay);
loop {
select! {
Some(event) = events_rx.recv() => {
match event {
Ok(event) => {
debug!("[Debug - handle_watch_events] Received event: {:#?}", event);
if let Err(e) = Self::handle_single_event(
location_id,
location_pub_id,
@ -197,6 +205,7 @@ impl LocationWatcher {
_library: &'lib Library,
ignore_paths: &HashSet<PathBuf>,
) -> Result<(), LocationManagerError> {
debug!("Event: {:#?}", event);
if !check_event(&event, ignore_paths) {
return Ok(());
}
@ -215,6 +224,8 @@ impl LocationWatcher {
return Ok(());
}
// debug!("Handling event: {:#?}", event);
event_handler.handle_event(event).await
}
@ -232,6 +243,7 @@ impl LocationWatcher {
pub(super) fn watch(&mut self) {
let path = &self.path;
debug!("Start watching location: (path: {path})");
if let Err(e) = self
.watcher
@ -368,7 +380,7 @@ mod tests {
use tracing::{debug, error};
// use tracing_test::traced_test;
#[cfg(target_os = "macos")]
#[cfg(any(target_os = "macos", target_os = "ios"))]
use notify::event::DataChange;
#[cfg(target_os = "linux")]
@ -447,7 +459,7 @@ mod tests {
#[cfg(target_os = "windows")]
expect_event(events_rx, &file_path, EventKind::Modify(ModifyKind::Any)).await;
#[cfg(target_os = "macos")]
#[cfg(any(target_os = "macos", target_os = "ios"))]
expect_event(
events_rx,
&file_path,
@ -487,7 +499,7 @@ mod tests {
#[cfg(target_os = "windows")]
expect_event(events_rx, &dir_path, EventKind::Create(CreateKind::Any)).await;
#[cfg(target_os = "macos")]
#[cfg(any(target_os = "macos", target_os = "ios"))]
expect_event(events_rx, &dir_path, EventKind::Create(CreateKind::Folder)).await;
#[cfg(target_os = "linux")]
@ -528,7 +540,7 @@ mod tests {
#[cfg(target_os = "windows")]
expect_event(events_rx, &file_path, EventKind::Modify(ModifyKind::Any)).await;
#[cfg(target_os = "macos")]
#[cfg(any(target_os = "macos", target_os = "ios"))]
expect_event(
events_rx,
&file_path,
@ -577,7 +589,7 @@ mod tests {
)
.await;
#[cfg(target_os = "macos")]
#[cfg(any(target_os = "macos", target_os = "ios"))]
expect_event(
events_rx,
&file_path,
@ -628,7 +640,7 @@ mod tests {
)
.await;
#[cfg(target_os = "macos")]
#[cfg(any(target_os = "macos", target_os = "ios"))]
expect_event(
events_rx,
&dir_path,
@ -676,6 +688,14 @@ mod tests {
#[cfg(target_os = "linux")]
expect_event(events_rx, &file_path, EventKind::Remove(RemoveKind::File)).await;
#[cfg(target_os = "ios")]
expect_event(
events_rx,
&file_path,
EventKind::Modify(ModifyKind::Metadata(MetadataKind::Any)),
)
.await;
debug!("Unwatching root directory: {}", root_dir.path().display());
if let Err(e) = watcher.unwatch(root_dir.path()) {
error!("Failed to unwatch root directory: {e:#?}");
@ -723,6 +743,14 @@ mod tests {
#[cfg(target_os = "linux")]
expect_event(events_rx, &dir_path, EventKind::Remove(RemoveKind::Folder)).await;
#[cfg(target_os = "ios")]
expect_event(
events_rx,
&file_path,
EventKind::Modify(ModifyKind::Metadata(MetadataKind::Any)),
)
.await;
debug!("Unwatching root directory: {}", root_dir.path().display());
if let Err(e) = watcher.unwatch(root_dir.path()) {
error!("Failed to unwatch root directory: {e:#?}");

View file

@ -21,7 +21,6 @@ use sd_utils::{
uuid_to_bytes,
};
#[cfg(feature = "location-watcher")]
use sd_file_path_helper::IsolatedFilePathDataParts;
use std::{
@ -1037,7 +1036,6 @@ pub async fn get_location_path_from_location_id(
})
}
#[cfg(feature = "location-watcher")]
pub async fn create_file_path(
crate::location::Library { db, sync, .. }: &crate::location::Library,
IsolatedFilePathDataParts {

View file

@ -28,7 +28,6 @@ pub fn media_data_image_to_query(
})
}
#[cfg(feature = "location-watcher")]
pub fn media_data_image_to_query_params(
mdi: ImageMetadata,
) -> (Vec<(&'static str, serde_json::Value)>, Vec<SetParam>) {