Software-Mac/spacedrive
Software-Mac/spacedrive
1
0
Fork 0
mirror of https://github.com/spacedriveapp/spacedrive synced 2024-06-28 02:13:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

Media metadata extraction & Thumbnailer rework (#2285)

Browse source 
* initial ffprobe commit

* Working slim down version ffprobe

* Auto format ffprobe and deps source

* Remove show_pixel_formats logic
- Fix do_bitexact incorrect check in main after last changes
- Fix some clangd warning

* Remove show_* and print_format options and their respective logic

* Rework ffprobe into simple_ffprobe
- Simplify ffprobe logic into a simple program that gather and print a media file metadata

* Reduce the amount of ffmpeg log messages while generating thumbnails

* Fix completly wrong comments

* mend

* Start modeling ffmpeg extracted metadata on schema
 - Start porting ffprobe code to rust
 - Rename some references to media_data to exif_data

* Finish modeling media info data
 - Add MediaProgram, MediaStream, MediaCodec, MediaVideoProps, MediaAudioProps, MediaSubtitleProps to Schema
 - Fix simple_ffproble to use its custom print_codec, instead of ffmpeg's impl

* Add relation between MediaInfo and FilePath
 - Remove shared properties from MediaInfo and related structs
 - Implement Iterator for FFmpegDict

* Fix and update schema

* Data models and start populating MediaInfo in rust

* Finish populating media info, chapters and program

* Improve FFmpegFormatContext data raw pointer access
 - Implement stream data gathering

* Impl FFmpegCodecContext, retrieve codec information
 - Improve some unsafe pointer uses
 - Impl from FFmpegFormatContext to MediaInfo conversion

* Fix FFmpegDict Drop

* Fix some crago warnings

* Impl retrieval of video props
 - Fix C char* to Rust String convertion

* Impl retrieval of audio and subtitle props
 - Fill props for MediaCodec

* Remove simple_ffprobe now that the Rust impl is done

* Fix schema to match actually retrieved media info
 - Fix import some FFmpeg constants instead of directly using values

* Rework movie_decoder
 - Re-implement create_scale_string and add support anamorphic video
 - Improve C pointer access for FFmpegFormatContext and FFmpegCodecContext
 - Use newer FFmpeg abstractions in movie_decoder

* Fix incorrect props when initializing MovieDecoder

* Remove unecessary lifetimes

* Added more native wrappers for some FFmpeg native objects used in movie_decoder

* Remove FFmpegPacket
 - Some more improvements to movie_decoder

* WIP

* Some small fixes

* More fixes
Rename movie_decoder to frame_decoder
Remove more references to film_strips

* fmt

* Fix duplicate migration for job error changes

* fix rebase

* Solving segfaults, fuck C lang

Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>

* Update rust to version 1.77
 - Pin rust version with rust-toolchain.toml
 - Change from dtolnay/rust-toolchain to IronCoreLabs/rust-toolchain for rust-toolchain support
 - Remove unused function and imports
 - Replace most CString uses with new c literal string

* More segfault solving and other minor fixes

Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>

* Fix ffmpeg rotation filter breaking portrait video thumbnails #2150
 - Plus some other misc fixes

* Auto format

* Retrieve video/audio metadata on frontend

* Auto format

* First draft on ffmpeg data save on db

Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>

* Fix some incorrect changes to prisma schema

* Some fixes for the FFmpegData schema
 - Expand logic to save FFmpegData to db

* A ton of things

Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>

* Integrating ffmpeg media data in jobs and API

* Rspc can't BigInt

* 🙄

* Add initial ffmpeg metadata entries to Inspector
 - Fix ephemeral metadata api to match the files metadata api call

* Fix Inspector not showing ffmpeg metadata

* Add bitrate, start time and chapters video metadata to Inspector
- Fix backend BigInt conversion incorrectly using i32 instead of u32
- Change FFmpegFormatContext/FFmpegMetaData bit_rate to i64
- Rename byteSize to humanizeSize
- Expand humanizeSize logic to allow handling bits and Binary units
- Move capitalize to @sd/client utils

* Solving some issues

* Fix ffmpeg probe getting incorrect stream id and breaking database unique constraint
 - Fix humanizeSize breaking when receiving floating numbers
 - Fix incorrect equality in StatCard
 - Fix unhandled error in Dialog when trying to remove an unknown dialog

* fmt

* small improvements
 - Remove some unecessary recursion_limit directive
 - Remove unused app_image releated functions
 - Fix metadata query enabled flag

* Add migration for ffmpeg media data

* Fix cypress test

* Requested changes

* Implement feedback
 - Update locale keys for all languages
 - Add pnpm command to update all language keys

* Fix thumb reactivity in non indexed locations

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
This commit is contained in:
Vítor Vasconcellos 2024-05-08 23:20:28 -03:00 committed by GitHub
parent 853f0d4185
commit e797b02e65
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
132 changed files with 4961 additions and 2548 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
.cspell/project_words.txt
View file

@ -11,9 +11,11 @@ codegen
Condvar
dashmap
davidmytton
dayjs
deel
elon
encryptor
Exif
Flac
graps
haden
@ -59,7 +61,9 @@ storedkey
stringly
thumbstrips
tobiaslutke
tokio
typecheck
uuid
vdfs
vijay
zacharysmith

3
.github/actions/setup-rust/action.yaml vendored
View file

@ -17,10 +17,9 @@ runs:
steps:
- name: Install Rust
id: toolchain
uses: dtolnay/rust-toolchain@stable
uses: IronCoreLabs/rust-toolchain@v1
with:
target: ${{ inputs.target }}
toolchain: '1.75'
components: clippy, rustfmt
- name: Cache Rust Dependencies

3
.vscode/launch.json vendored
View file

@ -12,8 +12,7 @@
"args": [
"build",
"--manifest-path=./apps/desktop/src-tauri/Cargo.toml",
"--no-default-features",
"--features=ai-models"
"--no-default-features"
],
"problemMatcher": "$rustc"
},

3
.vscode/tasks.json vendored
View file

@ -56,8 +56,7 @@
"command": "run",
"args": [
"--manifest-path=./apps/desktop/src-tauri/Cargo.toml",
"--no-default-features",
"--features=ai-models"
"--no-default-features"
],
"env": {
"RUST_BACKTRACE": "short"

2
CONTRIBUTING.md
View file

@ -89,7 +89,7 @@ To run the landing page:
If you encounter any issues, ensure that you are using the following versions of Rust, Node and Pnpm:
- Rust version: **1.75**
- Rust version: **1.78**
- Node version: **18.18**
- Pnpm version: **9.0.6**

13
Cargo.lock generated
View file

@ -1673,9 +1673,9 @@ dependencies = [
[[package]]
name = "chrono"
version = "0.4.31"
version = "0.4.38"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f2c685bad3eb3d45a01354cedb7d5faa66194d1d58ba6e267a8de788f79db38"
checksum = "a21f936df1771bf62b77f047b726c4625ff2e8aa607c01ec06e5a05bd8463401"
dependencies = [
"android-tzdata",
"iana-time-zone",
@ -1683,7 +1683,7 @@ dependencies = [
"num-traits",
"serde",
"wasm-bindgen",
"windows-targets 0.48.5",
"windows-targets 0.52.5",
]
[[package]]
@ -9054,7 +9054,11 @@ dependencies = [
name = "sd-ffmpeg"
version = "0.1.0"
dependencies = [
"chrono",
"ffmpeg-sys-next",
"image",
"libc",
"sd-utils",
"tempfile",
"thiserror",
"tokio",
@ -9101,10 +9105,13 @@ dependencies = [
"kamadak-exif",
"rand 0.8.5",
"rand_chacha 0.3.1",
"sd-ffmpeg",
"sd-utils",
"serde",
"serde_json",
"specta",
"thiserror",
"tokio",
]
[[package]]

23
Cargo.toml
View file

@ -22,25 +22,20 @@ repository = "https://github.com/spacedriveapp/spacedrive"
[workspace.dependencies]
# First party dependencies
prisma-client-rust = { git = "https://github.com/spacedriveapp/prisma-client-rust", rev = "528ab1cd02c25a1b183c0a8bc44e28954fdd0bfd", features = [
"specta",
"sqlite-create-many",
"migrations",
"specta",
"sqlite",
"sqlite-create-many",
], default-features = false }
prisma-client-rust-cli = { git = "https://github.com/spacedriveapp/prisma-client-rust", rev = "528ab1cd02c25a1b183c0a8bc44e28954fdd0bfd", features = [
"specta",
"sqlite-create-many",
"migrations",
"specta",
"sqlite",
"sqlite-create-many",
], default-features = false }
prisma-client-rust-sdk = { git = "https://github.com/spacedriveapp/prisma-client-rust", rev = "528ab1cd02c25a1b183c0a8bc44e28954fdd0bfd", features = [
"sqlite",
], default-features = false }
tracing = "0.1.40"
tracing-subscriber = "0.3.18"
tracing-appender = "0.2.3"
rspc = { version = "0.1.4" }
specta = { version = "=2.0.0-rc.11" }
tauri-specta = { version = "=2.0.0-rc.8" }
@ -54,7 +49,7 @@ async-trait = "0.1.77"
axum = "=0.6.20"
base64 = "0.21.5"
blake3 = "1.5.0"
chrono = "0.4.31"
chrono = "0.4.38"
clap = "4.4.7"
futures = "0.3.30"
futures-concurrency = "7.4.3"
@ -64,6 +59,7 @@ http = "0.2.9"
image = "0.24.7"
itertools = "0.12.0"
lending-stream = "1.0.0"
libc = "0.2"
normpath = "1.1.1"
once_cell = "1.18.0"
pin-project-lite = "0.2.13"
@ -83,13 +79,14 @@ thiserror = "1.0.50"
tokio = "1.36.0"
tokio-stream = "0.1.14"
tokio-util = "0.7.10"
tracing = "0.1.40"
tracing-subscriber = "0.3.18"
tracing-appender = "0.2.3"
tracing-test = "^0.2.4"
uhlc = "=0.5.2"
uuid = "1.5.0"
webp = "0.2.6"
[workspace.dev-dependencies]
tracing-test = { version = "^0.2.4" }
[patch.crates-io]
# Proper IOS Support
if-watch = { git = "https://github.com/oscartbeaumont/if-watch.git", rev = "a92c17d3f85c1c6fb0afeeaf6c2b24d0b147e8c3" }

2
apps/desktop/crates/linux/Cargo.toml
View file

@ -7,7 +7,7 @@ edition = { workspace = true }
[dependencies]
tokio = { workspace = true, features = ["fs"] }
libc = "0.2"
libc = { workspace = true }
[target.'cfg(target_os = "linux")'.dependencies]
# WARNING: gtk should follow the same version used by tauri

21
apps/desktop/crates/linux/src/app_info.rs
View file

@ -2,31 +2,14 @@ use std::path::Path;
use gtk::{
gio::{
content_type_guess,
prelude::AppInfoExt,
prelude::{AppLaunchContextExt, FileExt},
AppInfo, AppLaunchContext, DesktopAppInfo, File as GioFile, ResourceError,
content_type_guess, prelude::AppInfoExt, prelude::FileExt, AppInfo, AppLaunchContext,
DesktopAppInfo, File as GioFile, ResourceError,
},
glib::error::Error as GlibError,
prelude::IsA,
};
use tokio::fs::File;
use tokio::io::AsyncReadExt;
use crate::env::remove_prefix_from_pathlist;
fn remove_prefix_from_env_in_ctx(
ctx: &impl IsA<AppLaunchContext>,
env_name: &str,
prefix: &impl AsRef<Path>,
) {
if let Some(value) = remove_prefix_from_pathlist(env_name, prefix) {
ctx.setenv(env_name, value);
} else {
ctx.unsetenv(env_name);
}
}
thread_local! {
static LAUNCH_CTX: AppLaunchContext = {
// TODO: Display supports requires GDK, which can only run on the main thread

37
apps/desktop/crates/linux/src/env.rs
View file

@ -1,29 +1,13 @@
use std::{
collections::HashSet,
env,
ffi::{CStr, OsStr, OsString},
ffi::{CStr, OsStr},
mem,
os::unix::ffi::OsStrExt,
path::{Path, PathBuf},
path::PathBuf,
ptr,
};
fn version(version_str: &str) -> i32 {
let mut version_parts: Vec<i32> = version_str
.split('.')
.take(4) // Take up to 4 components
.map(|part| part.parse().unwrap_or(0))
.collect();
// Pad with zeros if needed
version_parts.resize_with(4, Default::default);
(version_parts[0] * 1_000_000_000)
+ (version_parts[1] * 1_000_000)
+ (version_parts[2] * 1_000)
+ version_parts[3]
}
pub fn get_current_user_home() -> Option<PathBuf> {
use libc::{getpwuid_r, getuid, passwd, ERANGE};
@ -193,23 +177,6 @@ pub fn normalize_environment() {
.expect("PATH must be successfully normalized");
}
pub(crate) fn remove_prefix_from_pathlist(
env_name: &str,
prefix: &impl AsRef<Path>,
) -> Option<OsString> {
env::var_os(env_name).and_then(|value| {
let mut dirs = env::split_paths(&value)
.filter(|dir| !(dir.as_os_str().is_empty() || dir.starts_with(prefix)))
.peekable();
if dirs.peek().is_none() {
None
} else {
Some(env::join_paths(dirs).expect("Should not fail because we are only filtering a pathlist retrieved from the environmnet"))
}
})
}
// Check if snap by looking if SNAP is set and not empty and that the SNAP directory exists
pub fn is_snap() -> bool {
if let Some(snap) = std::env::var_os("SNAP") {

2
apps/desktop/crates/windows/Cargo.toml
View file

@ -8,7 +8,7 @@ edition = { workspace = true }
[dependencies]
normpath = { workspace = true }
thiserror = { workspace = true }
libc = "0.2"
libc = { workspace = true }
[target.'cfg(target_os = "windows")'.dependencies.windows]
version = "0.51"

2
apps/mobile/modules/sd-core/android/crate/Cargo.toml
View file

@ -1,7 +1,7 @@
[package]
name = "sd-mobile-android"
version = "0.1.0"
rust-version = "1.64.0"
rust-version = "1.64"
license = { workspace = true }
repository = { workspace = true }
edition = { workspace = true }

2
apps/mobile/modules/sd-core/core/Cargo.toml
View file

@ -1,7 +1,7 @@
[package]
name = "sd-mobile-core"
version = "0.1.0"
rust-version = "1.64.0"
rust-version = "1.64"
license = { workspace = true }
repository = { workspace = true }
edition = { workspace = true }

2
apps/mobile/modules/sd-core/ios/crate/Cargo.toml
View file

@ -1,7 +1,7 @@
[package]
name = "sd-mobile-ios"
version = "0.1.0"
rust-version = "1.64.0"
rust-version = "1.64"
license = { workspace = true }
repository = { workspace = true }
edition = { workspace = true }

10
apps/mobile/src/components/drawer/DrawerLocations.tsx
View file

@ -2,7 +2,13 @@ import { DrawerNavigationHelpers } from '@react-navigation/drawer/lib/typescript
import { useNavigation } from '@react-navigation/native';
import { useRef } from 'react';
import { Pressable, Text, View } from 'react-native';
import { arraysEqual, byteSize, Location, useLibraryQuery, useOnlineLocations } from '@sd/client';
import {
arraysEqual,
humanizeSize,
Location,
useLibraryQuery,
useOnlineLocations
} from '@sd/client';
import { ModalRef } from '~/components/layout/Modal';
import { tw, twStyle } from '~/lib/tailwind';
@ -45,7 +51,7 @@ const DrawerLocationItem: React.FC<DrawerLocationItemProps> = ({
</View>
<View style={tw`rounded-md border border-app-lightborder bg-app-box px-1 py-0.5`}>
<Text style={tw`text-[11px] font-medium text-ink-dull`} numberOfLines={1}>
{`${byteSize(location.size_in_bytes)}`}
{`${humanizeSize(location.size_in_bytes)}`}
</Text>
</View>
</View>

4
apps/mobile/src/components/locations/GridLocation.tsx
View file

@ -1,6 +1,6 @@
import { DotsThreeOutlineVertical } from 'phosphor-react-native';
import { Pressable, Text, View } from 'react-native';
import { arraysEqual, byteSize, Location, useOnlineLocations } from '@sd/client';
import { arraysEqual, humanizeSize, Location, useOnlineLocations } from '@sd/client';
import { tw, twStyle } from '~/lib/tailwind';
import FolderIcon from '../icons/FolderIcon';
@ -47,7 +47,7 @@ const GridLocation: React.FC<GridLocationProps> = ({ location, modalRef }: GridL
</Text>
</View>
<Text style={tw`text-left text-[13px] font-bold text-ink-dull`} numberOfLines={1}>
{`${byteSize(location.size_in_bytes)}`}
{`${humanizeSize(location.size_in_bytes)}`}
</Text>
</Card>
);

14
apps/mobile/src/components/locations/ListLocation.tsx
View file

@ -1,9 +1,9 @@
import { useNavigation } from '@react-navigation/native';
import { Location, arraysEqual, byteSize, useOnlineLocations } from '@sd/client';
import { DotsThreeVertical } from 'phosphor-react-native';
import { useRef } from 'react';
import { Pressable, Text, View } from 'react-native';
import { Swipeable } from 'react-native-gesture-handler';
import { arraysEqual, humanizeSize, Location, useOnlineLocations } from '@sd/client';
import { tw, twStyle } from '~/lib/tailwind';
import { SettingsStackScreenProps } from '~/navigation/tabs/SettingsStack';
@ -62,22 +62,16 @@ const ListLocation = ({ location }: ListLocationProps) => {
</View>
</View>
<View style={tw`flex-row items-center gap-3`}>
<View
style={tw`rounded-md border border-app-box bg-app p-1.5`}
>
<View style={tw`rounded-md border border-app-box bg-app p-1.5`}>
<Text
style={tw`text-left text-xs font-medium text-ink-dull`}
numberOfLines={1}
>
{`${byteSize(location.size_in_bytes)}`}
{`${humanizeSize(location.size_in_bytes)}`}
</Text>
</View>
<Pressable hitSlop={24} onPress={() => swipeRef.current?.openRight()}>
<DotsThreeVertical
weight="bold"
size={20}
color={tw.color('ink-dull')}
/>
<DotsThreeVertical weight="bold" size={20} color={tw.color('ink-dull')} />
</Pressable>
</View>
</Card>

16
apps/mobile/src/components/modal/inspector/ActionsModal.tsx
View file

@ -1,10 +1,3 @@
import {
byteSize,
getIndexedItemFilePath,
getItemObject,
useLibraryMutation,
useLibraryQuery
} from '@sd/client';
import dayjs from 'dayjs';
import {
Copy,
@ -20,6 +13,13 @@ import {
import { PropsWithChildren, useRef } from 'react';
import { Pressable, Text, View, ViewStyle } from 'react-native';
import FileViewer from 'react-native-file-viewer';
import {
getIndexedItemFilePath,
getItemObject,
humanizeSize,
useLibraryMutation,
useLibraryQuery
} from '@sd/client';
import FileThumb from '~/components/explorer/FileThumb';
import FavoriteButton from '~/components/explorer/sections/FavoriteButton';
import InfoTagPills from '~/components/explorer/sections/InfoTagPills';
@ -119,7 +119,7 @@ export const ActionsModal = () => {
</Text>
<View style={tw`flex flex-row`}>
<Text style={tw`text-xs text-ink-faint`}>
{`${byteSize(filePath?.size_in_bytes_bytes)}`},
{`${humanizeSize(filePath?.size_in_bytes_bytes)}`},
</Text>
<Text style={tw`text-xs text-ink-faint`}>
{' '}

21
apps/mobile/src/components/modal/inspector/FileInfoModal.tsx
View file

@ -2,7 +2,7 @@ import dayjs from 'dayjs';
import { Barcode, CaretLeft, Clock, Cube, Icon, SealCheck, Snowflake } from 'phosphor-react-native';
import { forwardRef } from 'react';
import { Pressable, Text, View } from 'react-native';
import { byteSize, getItemFilePath, getItemObject, type ExplorerItem } from '@sd/client';
import { getItemFilePath, humanizeSize, type ExplorerItem } from '@sd/client';
import FileThumb from '~/components/explorer/FileThumb';
import InfoTagPills from '~/components/explorer/sections/InfoTagPills';
import { Modal, ModalScrollView, type ModalRef } from '~/components/layout/Modal';
@ -39,18 +39,9 @@ type FileInfoModalProps = {
const FileInfoModal = forwardRef<ModalRef, FileInfoModalProps>((props, ref) => {
const { data } = props;
const modalRef = useForwardedRef(ref);
const item = data?.item;
const objectData = data && getItemObject(data);
const filePathData = data && getItemFilePath(data);
// const fullObjectData = useLibraryQuery(['files.get', objectData?.id || -1], {
// enabled: objectData?.id !== undefined
// });
return (
<Modal
ref={modalRef}
@ -82,16 +73,8 @@ const FileInfoModal = forwardRef<ModalRef, FileInfoModalProps>((props, ref) => {
<MetaItem
title="Size"
icon={Cube}
value={`${byteSize(filePathData?.size_in_bytes_bytes)}`}
value={`${humanizeSize(filePathData?.size_in_bytes_bytes)}`}
/>
{/* Duration */}
{/* {fullObjectData.data?.media_data?.duration && (
<MetaItem
title="Duration"
value={fullObjectData.data.media_data.duration}
icon={Clock}
/>
)} */}
{/* Created */}
{data.type !== 'SpacedropPeer' && (
<MetaItem

4
apps/mobile/src/components/overview/OverviewStats.tsx
View file

@ -4,7 +4,7 @@ import { UseQueryResult } from '@tanstack/react-query';
import { useEffect, useState } from 'react';
import { Platform, Text, View } from 'react-native';
import { ClassInput } from 'twrnc/dist/esm/types';
import { byteSize, Statistics, StatisticsResponse, useLibraryContext } from '@sd/client';
import { humanizeSize, Statistics, StatisticsResponse, useLibraryContext } from '@sd/client';
import useCounter from '~/hooks/useCounter';
import { tw, twStyle } from '~/lib/tailwind';
@ -26,7 +26,7 @@ interface StatItemProps {
}
const StatItem = ({ title, bytes, isLoading, style }: StatItemProps) => {
const { value, unit } = byteSize(bytes);
const { value, unit } = humanizeSize(bytes);
const count = useCounter({ name: title, end: value });

10
apps/mobile/src/components/overview/StatCard.tsx
View file

@ -1,7 +1,7 @@
import { useEffect, useMemo, useState } from 'react';
import { Text, View } from 'react-native';
import { AnimatedCircularProgress } from 'react-native-circular-progress';
import { byteSize } from '@sd/client';
import { humanizeSize } from '@sd/client';
import { tw } from '~/lib/tailwind';
import { Icon, IconName } from '../icons/Icon';
@ -20,12 +20,12 @@ const StatCard = ({ icon, name, connectionType, ...stats }: StatCardProps) => {
const [mounted, setMounted] = useState(false);
const { totalSpace, freeSpace, usedSpaceSpace } = useMemo(() => {
const totalSpace = byteSize(stats.totalSpace);
const freeSpace = stats.freeSpace == null ? totalSpace : byteSize(stats.freeSpace);
const totalSpace = humanizeSize(stats.totalSpace);
const freeSpace = stats.freeSpace == null ? totalSpace : humanizeSize(stats.freeSpace);
return {
totalSpace,
freeSpace,
usedSpaceSpace: byteSize(totalSpace.original - freeSpace.original)
usedSpaceSpace: humanizeSize(totalSpace.original - freeSpace.original)
};
}, [stats]);
@ -34,7 +34,7 @@ const StatCard = ({ icon, name, connectionType, ...stats }: StatCardProps) => {
}, []);
const progress = useMemo(() => {
if (!mounted || totalSpace.original === 0n) return 0;
if (!mounted || totalSpace.original === 0) return 0;
return Math.floor((usedSpaceSpace.value / totalSpace.value) * 100);
}, [mounted, totalSpace, usedSpaceSpace]);

16
apps/mobile/src/stores/searchStore.ts
View file

@ -1,12 +1,20 @@
import { SearchFilterArgs } from '@sd/client';
import { proxy, useSnapshot } from 'valtio';
import { SearchFilterArgs } from '@sd/client';
import { IconName } from '~/components/icons/Icon';
export type SearchFilters = 'locations' | 'tags' | 'name' | 'extension' | 'hidden' | 'kind';
export type SortOptionsType = {
by: 'none' | 'name' | 'sizeInBytes' | 'dateIndexed' | 'dateCreated' | 'dateModified' | 'dateAccessed' | 'dateTaken';
by:
| 'none'
| 'name'
| 'sizeInBytes'
| 'dateIndexed'
| 'dateCreated'
| 'dateModified'
| 'dateAccessed'
| 'dateTaken';
direction: 'Asc' | 'Desc';
}
};
export interface FilterItem {
id: number;
@ -38,7 +46,7 @@ interface State {
filters: Filters;
sort: SortOptionsType;
appliedFilters: Partial<Filters>;
mergedFilters: SearchFilterArgs[],
mergedFilters: SearchFilterArgs[];
disableActionButtons: boolean;
}

20
core/Cargo.toml
View file

@ -2,8 +2,8 @@
name = "sd-core"
version = "0.2.14"
description = "Virtual distributed filesystem engine that powers Spacedrive."
authors = ["Spacedrive Technology Inc."]
rust-version = "1.75.0"
authors = ["Spacedrive Technology Inc <support@spacedrive.com>"]
rust-version = "1.78"
license = { workspace = true }
repository = { workspace = true }
edition = { workspace = true }
@ -13,7 +13,7 @@ default = []
# This feature allows features to be disabled when the Core is running on mobile.
mobile = []
# This feature controls whether the Spacedrive Core contains functionality which requires FFmpeg.
ffmpeg = ["dep:sd-ffmpeg"]
ffmpeg = ["dep:sd-ffmpeg", "sd-media-metadata/ffmpeg"]
heif = ["sd-images/heif"]
ai = ["dep:sd-ai"]
crypto = ["dep:sd-crypto"]
@ -25,11 +25,10 @@ sd-core-heavy-lifting = { path = "./crates/heavy-lifting" }
sd-core-indexer-rules = { path = "./crates/indexer-rules" }
sd-core-prisma-helpers = { path = "./crates/prisma-helpers" }
sd-core-sync = { path = "./crates/sync" }
# Spacedrive Sub-crates
sd-actors = { version = "0.1.0", path = "../crates/actors" }
sd-actors = { path = "../crates/actors", version = "0.1.0" }
sd-ai = { path = "../crates/ai", optional = true }
sd-cloud-api = { version = "0.1.0", path = "../crates/cloud-api" }
sd-cloud-api = { path = "../crates/cloud-api", version = "0.1.0" }
sd-crypto = { path = "../crates/crypto", features = [
"sys",
"tokio",
@ -61,6 +60,7 @@ futures = { workspace = true }
futures-concurrency = { workspace = true }
image = { workspace = true }
itertools = { workspace = true }
libc = { workspace = true }
normpath = { workspace = true, features = ["localization"] }
once_cell = { workspace = true }
pin-project-lite = { workspace = true }
@ -101,7 +101,6 @@ tracing-subscriber = { workspace = true, features = ["env-filter"] }
uuid = { workspace = true, features = ["v4", "serde"] }
webp = { workspace = true }
# Specific Core dependencies
async-recursion = "1.0.5"
async-stream = "0.3.5"
@ -118,7 +117,6 @@ http-body = "0.4.5"
http-range = "0.1.5"
hyper = { version = "=0.14.28", features = ["http1", "server", "client"] }
int-enum = "0.5.0"
libc = "0.2.153"
mini-moka = "0.10.2"
notify = { git = "https://github.com/notify-rs/notify.git", rev = "c3929ed114fbb0bc7457a9a498260461596b00ca", default-features = false, features = [
"macos_fsevent",
@ -160,6 +158,8 @@ icrate = { version = "0.1.0", features = [
] }
[dev-dependencies]
tracing-test = { workspace.dev-dependencies = true }
aovec = "1.1.0"
# Workspace dependencies
globset = { workspace = true }
tracing-test = { workspace = true }
# Specific Core dependencies
aovec = "1.1.0"

2
core/crates/file-path-helper/Cargo.toml
View file

@ -3,7 +3,7 @@ name = "sd-core-file-path-helper"
version = "0.1.0"
authors = ["Ericson Soares <ericson@spacedrive.com>"]
readme = "README.md"
rust-version = "1.75.0"
rust-version = "1.75"
license = { workspace = true }
repository = { workspace = true }
edition = { workspace = true }

7
core/crates/heavy-lifting/Cargo.toml
View file

@ -14,15 +14,13 @@ sd-core-file-path-helper = { path = "../file-path-helper" }
sd-core-indexer-rules = { path = "../indexer-rules" }
sd-core-prisma-helpers = { path = "../prisma-helpers" }
sd-core-sync = { path = "../sync" }
# Sub-crates
sd-file-ext = { path = "../../../crates/file-ext" }
sd-prisma = { path = "../../../crates/prisma" }
sd-sync = { path = "../../../crates/sync" }
sd-task-system = { path = "../../../crates/task-system" }
sd-utils = { path = "../../../crates/utils" }
# Workspace dependencies
async-channel = { workspace = true }
async-trait = { workspace = true }
blake3 = { workspace = true }
@ -47,7 +45,6 @@ tokio-stream = { workspace = true, features = ["fs"] }
tracing = { workspace = true }
uuid = { workspace = true, features = ["v4", "serde"] }
[dev-dependencies]
tempfile = { workspace = true }
tracing-test = { workspace.dev-dependencies = true }
tracing-test = { workspace = true }

4
core/crates/heavy-lifting/src/job_system/job.rs
View file

@ -8,8 +8,8 @@ use sd_task_system::{
};
use std::{
collections::VecDeque,
hash::{DefaultHasher, Hash, Hasher},
collections::{hash_map::DefaultHasher, VecDeque},
hash::{Hash, Hasher},
marker::PhantomData,
pin::pin,
sync::Arc,

5
core/crates/indexer-rules/Cargo.toml
View file

@ -1,7 +1,10 @@
[package]
name = "sd-core-indexer-rules"
version = "0.1.0"
authors = ["Ericson Soares <ericson@spacedrive.com>"]
authors = [
"Ericson Soares <ericson@spacedrive.com>",
"Vítor Vasconcellos <vitor@spacedrive.com>",
]
license = { workspace = true }
repository = { workspace = true }
edition = { workspace = true }

30
core/crates/prisma-helpers/src/lib.rs
View file

@ -1,3 +1,4 @@
#![recursion_limit = "256"]
#![warn(
clippy::all,
clippy::pedantic,
@ -139,7 +140,7 @@ file_path::select!(file_path_to_full_path {
// File Path includes!
file_path::include!(file_path_with_object {
object: include {
media_data: select {
exif_data: select {
resolution
media_date
media_location
@ -162,7 +163,7 @@ object::select!(object_for_file_identifier {
object::include!(object_with_file_paths {
file_paths: include {
object: include {
media_data: select {
exif_data: select {
resolution
media_date
media_location
@ -172,6 +173,31 @@ object::include!(object_with_file_paths {
copyright
exif_version
}
ffmpeg_data: include {
chapters
programs: include {
streams: include {
codec: include {
audio_props
video_props
}
}
}
}
}
}
});
object::include!(object_with_media_data {
exif_data
ffmpeg_data: include {
chapters
programs: include {
streams: include {
codec: include {
audio_props
video_props
}
}
}
}
});

14
core/crates/sync/src/backfill.rs
View file

@ -2,7 +2,7 @@ use std::future::Future;
use sd_prisma::{
prisma::{
crdt_operation, file_path, label, label_on_object, location, media_data, object, tag,
crdt_operation, exif_data, file_path, label, label_on_object, location, object, tag,
tag_on_object, PrismaClient, SortOrder,
},
prisma_sync,
@ -163,11 +163,11 @@ pub async fn backfill_operations(db: &PrismaClient, sync: &crate::Manager, insta
paginate(
|cursor| {
db.media_data()
.find_many(vec![media_data::id::gt(cursor)])
.order_by(media_data::id::order(SortOrder::Asc))
db.exif_data()
.find_many(vec![exif_data::id::gt(cursor)])
.order_by(exif_data::id::order(SortOrder::Asc))
.take(1000)
.include(media_data::include!({
.include(exif_data::include!({
object: select { pub_id }
}))
.exec()
@ -179,10 +179,10 @@ pub async fn backfill_operations(db: &PrismaClient, sync: &crate::Manager, insta
media_datas
.into_iter()
.flat_map(|md| {
use media_data::*;
use exif_data::*;
sync.shared_create(
prisma_sync::media_data::SyncId {
prisma_sync::exif_data::SyncId {
object: prisma_sync::object::SyncId {
pub_id: md.object.pub_id,
},

52
core/prisma/migrations/20240409202941_rename_media_data_to_exif_data/migration.sql Normal file
View file

@ -0,0 +1,52 @@
-- CreateTable
CREATE TABLE "exif_data" (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"resolution" BLOB,
"media_date" BLOB,
"media_location" BLOB,
"camera_data" BLOB,
"artist" TEXT,
"description" TEXT,
"copyright" TEXT,
"exif_version" TEXT,
"epoch_time" BIGINT,
"object_id" INTEGER NOT NULL,
CONSTRAINT "exif_data_object_id_fkey" FOREIGN KEY ("object_id") REFERENCES "object" ("id") ON DELETE CASCADE ON UPDATE CASCADE
);
-- CopyData
INSERT INTO "exif_data" (
"id",
"resolution",
"media_date",
"media_location",
"camera_data",
"artist",
"description",
"copyright",
"exif_version",
"epoch_time",
"object_id"
)
SELECT
"id",
"resolution",
"media_date",
"media_location",
"camera_data",
"artist",
"description",
"copyright",
"exif_version",
"epoch_time",
"object_id"
FROM
"media_data";
-- DropTable
PRAGMA foreign_keys=off;
DROP TABLE "media_data";
PRAGMA foreign_keys=on;
-- CreateIndex
CREATE UNIQUE INDEX "exif_data_object_id_key" ON "exif_data"("object_id");

128
core/prisma/migrations/20240508074615_ffmpeg_media_data/migration.sql Normal file
View file

@ -0,0 +1,128 @@
-- CreateTable
CREATE TABLE "ffmpeg_data" (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"formats" TEXT NOT NULL,
"bit_rate" BLOB NOT NULL,
"duration" BLOB,
"start_time" BLOB,
"title" TEXT,
"creation_time" DATETIME,
"date" DATETIME,
"album_artist" TEXT,
"disc" TEXT,
"track" TEXT,
"album" TEXT,
"artist" TEXT,
"metadata" BLOB,
"object_id" INTEGER NOT NULL,
CONSTRAINT "ffmpeg_data_object_id_fkey" FOREIGN KEY ("object_id") REFERENCES "object" ("id") ON DELETE CASCADE ON UPDATE CASCADE
);
-- CreateTable
CREATE TABLE "ffmpeg_media_chapter" (
"chapter_id" INTEGER NOT NULL,
"start" BLOB NOT NULL,
"end" BLOB NOT NULL,
"time_base_den" INTEGER NOT NULL,
"time_base_num" INTEGER NOT NULL,
"title" TEXT,
"metadata" BLOB,
"ffmpeg_data_id" INTEGER NOT NULL,
PRIMARY KEY ("ffmpeg_data_id", "chapter_id"),
CONSTRAINT "ffmpeg_media_chapter_ffmpeg_data_id_fkey" FOREIGN KEY ("ffmpeg_data_id") REFERENCES "ffmpeg_data" ("id") ON DELETE CASCADE ON UPDATE CASCADE
);
-- CreateTable
CREATE TABLE "ffmpeg_media_program" (
"program_id" INTEGER NOT NULL,
"name" TEXT,
"metadata" BLOB,
"ffmpeg_data_id" INTEGER NOT NULL,
PRIMARY KEY ("ffmpeg_data_id", "program_id"),
CONSTRAINT "ffmpeg_media_program_ffmpeg_data_id_fkey" FOREIGN KEY ("ffmpeg_data_id") REFERENCES "ffmpeg_data" ("id") ON DELETE CASCADE ON UPDATE CASCADE
);
-- CreateTable
CREATE TABLE "ffmpeg_media_stream" (
"stream_id" INTEGER NOT NULL,
"name" TEXT,
"aspect_ratio_num" INTEGER NOT NULL,
"aspect_ratio_den" INTEGER NOT NULL,
"frames_per_second_num" INTEGER NOT NULL,
"frames_per_second_den" INTEGER NOT NULL,
"time_base_real_den" INTEGER NOT NULL,
"time_base_real_num" INTEGER NOT NULL,
"dispositions" TEXT,
"title" TEXT,
"encoder" TEXT,
"language" TEXT,
"duration" BLOB,
"metadata" BLOB,
"program_id" INTEGER NOT NULL,
"ffmpeg_data_id" INTEGER NOT NULL,
PRIMARY KEY ("ffmpeg_data_id", "program_id", "stream_id"),
CONSTRAINT "ffmpeg_media_stream_ffmpeg_data_id_program_id_fkey" FOREIGN KEY ("ffmpeg_data_id", "program_id") REFERENCES "ffmpeg_media_program" ("ffmpeg_data_id", "program_id") ON DELETE CASCADE ON UPDATE CASCADE
);
-- CreateTable
CREATE TABLE "ffmpeg_media_codec" (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"kind" TEXT,
"sub_kind" TEXT,
"tag" TEXT,
"name" TEXT,
"profile" TEXT,
"bit_rate" INTEGER NOT NULL,
"stream_id" INTEGER NOT NULL,
"program_id" INTEGER NOT NULL,
"ffmpeg_data_id" INTEGER NOT NULL,
CONSTRAINT "ffmpeg_media_codec_ffmpeg_data_id_program_id_stream_id_fkey" FOREIGN KEY ("ffmpeg_data_id", "program_id", "stream_id") REFERENCES "ffmpeg_media_stream" ("ffmpeg_data_id", "program_id", "stream_id") ON DELETE CASCADE ON UPDATE CASCADE
);
-- CreateTable
CREATE TABLE "ffmpeg_media_video_props" (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"pixel_format" TEXT,
"color_range" TEXT,
"bits_per_channel" INTEGER,
"color_space" TEXT,
"color_primaries" TEXT,
"color_transfer" TEXT,
"field_order" TEXT,
"chroma_location" TEXT,
"width" INTEGER NOT NULL,
"height" INTEGER NOT NULL,
"aspect_ratio_num" INTEGER,
"aspect_ratio_Den" INTEGER,
"properties" TEXT,
"codec_id" INTEGER NOT NULL,
CONSTRAINT "ffmpeg_media_video_props_codec_id_fkey" FOREIGN KEY ("codec_id") REFERENCES "ffmpeg_media_codec" ("id") ON DELETE CASCADE ON UPDATE CASCADE
);
-- CreateTable
CREATE TABLE "ffmpeg_media_audio_props" (
"id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"delay" INTEGER NOT NULL,
"padding" INTEGER NOT NULL,
"sample_rate" INTEGER,
"sample_format" TEXT,
"bit_per_sample" INTEGER,
"channel_layout" TEXT,
"codec_id" INTEGER NOT NULL,
CONSTRAINT "ffmpeg_media_audio_props_codec_id_fkey" FOREIGN KEY ("codec_id") REFERENCES "ffmpeg_media_codec" ("id") ON DELETE CASCADE ON UPDATE CASCADE
);
-- CreateIndex
CREATE UNIQUE INDEX "ffmpeg_data_object_id_key" ON "ffmpeg_data"("object_id");
-- CreateIndex
CREATE UNIQUE INDEX "ffmpeg_media_codec_ffmpeg_data_id_program_id_stream_id_key" ON "ffmpeg_media_codec"("ffmpeg_data_id", "program_id", "stream_id");
-- CreateIndex
CREATE UNIQUE INDEX "ffmpeg_media_video_props_codec_id_key" ON "ffmpeg_media_video_props"("codec_id");
-- CreateIndex
CREATE UNIQUE INDEX "ffmpeg_media_audio_props_codec_id_key" ON "ffmpeg_media_audio_props"("codec_id");

209
core/prisma/schema.prisma
View file

@ -231,33 +231,23 @@ model Object {
date_created DateTime?
date_accessed DateTime?
tags TagOnObject[]
labels LabelOnObject[]
albums ObjectInAlbum[]
spaces ObjectInSpace[]
file_paths FilePath[]
tags TagOnObject[]
labels LabelOnObject[]
albums ObjectInAlbum[]
spaces ObjectInSpace[]
file_paths FilePath[]
// comments Comment[]
media_data MediaData?
exif_data ExifData?
ffmpeg_data FfmpegData?
// key Key? @relation(fields: [key_id], references: [id])
@@map("object")
}
// if there is a conflicting cas_id, the conficting file should be updated to have a larger cas_id as
//the field is unique, however this record is kept to tell the indexer (upon discovering this CAS) that
//there is alternate versions of the file and to check by a full integrity hash to define for which to associate with.
// @brendan: nah this probably won't fly
// model FileConflict {
// original_object_id Int @unique
// detactched_object_id Int @unique
// @@map("file_conflict")
// }
// keys allow us to know exactly which files can be decrypted with a given key
// they can be "mounted" to a client, and then used to decrypt files automatically
/// @shared(id: uuid)
// // keys allow us to know exactly which files can be decrypted with a given key
// // they can be "mounted" to a client, and then used to decrypt files automatically
// /// @shared(id: uuid)
// model Key {
// id Int @id @default(autoincrement())
// // uuid to identify the key
@ -298,7 +288,7 @@ model Object {
// }
/// @shared(id: object, modelId: 4)
model MediaData {
model ExifData {
id Int @id @default(autoincrement())
resolution Bytes?
@ -315,17 +305,164 @@ model MediaData {
// (e.g. we can't get `MediaDate::Utc(2023-09-26T22:04:37+01:00)` from `1695758677` as we don't store the TZ)
epoch_time BigInt? // time since unix epoch
// video-specific
// duration Int?
// fps Int?
// streams Int?
// video_codec String? // eg: "h264, h265, av1"
// audio_codec String? // eg: "opus"
object_id Int @unique
object Object @relation(fields: [object_id], references: [id], onDelete: Cascade)
@@map("media_data")
@@map("exif_data")
}
model FfmpegData {
id Int @id @default(autoincrement())
// Internal FFmpeg properties
formats String
bit_rate Bytes // Actually a i64 in the backend
duration Bytes? // Actually a i64 in the backend
start_time Bytes? // Actually a i64 in the backend
chapters FfmpegMediaChapter[]
programs FfmpegMediaProgram[]
// Metadata for search
title String?
creation_time DateTime?
date DateTime?
album_artist String?
disc String?
track String?
album String?
artist String?
metadata Bytes?
object Object @relation(fields: [object_id], references: [id], onDelete: Cascade)
object_id Int @unique
@@map("ffmpeg_data")
}
model FfmpegMediaChapter {
chapter_id Int
start Bytes // Actually a i64 in the backend
end Bytes // Actually a i64 in the backend
time_base_den Int
time_base_num Int
// Metadata for search
title String?
metadata Bytes?
ffmpeg_data FfmpegData @relation(fields: [ffmpeg_data_id], references: [id], onDelete: Cascade)
ffmpeg_data_id Int
@@id(name: "likeId", [ffmpeg_data_id, chapter_id])
@@map("ffmpeg_media_chapter")
}
model FfmpegMediaProgram {
program_id Int
streams FfmpegMediaStream[]
// Metadata for search
name String?
metadata Bytes?
ffmpeg_data FfmpegData @relation(fields: [ffmpeg_data_id], references: [id], onDelete: Cascade)
ffmpeg_data_id Int
@@id(name: "likeId", [ffmpeg_data_id, program_id])
@@map("ffmpeg_media_program")
}
model FfmpegMediaStream {
stream_id Int
name String?
codec FfmpegMediaCodec?
aspect_ratio_num Int
aspect_ratio_den Int
frames_per_second_num Int
frames_per_second_den Int
time_base_real_den Int
time_base_real_num Int
dispositions String?
// Metadata for search
title String?
encoder String?
language String?
duration Bytes? // Actually a i64 in the backend
metadata Bytes?
program FfmpegMediaProgram @relation(fields: [ffmpeg_data_id, program_id], references: [ffmpeg_data_id, program_id], onDelete: Cascade)
program_id Int
ffmpeg_data_id Int
@@id(name: "likeId", [ffmpeg_data_id, program_id, stream_id])
@@map("ffmpeg_media_stream")
}
model FfmpegMediaCodec {
id Int @id @default(autoincrement())
kind String?
sub_kind String?
tag String?
name String?
profile String?
bit_rate Int
video_props FfmpegMediaVideoProps?
audio_props FfmpegMediaAudioProps?
stream FfmpegMediaStream @relation(fields: [ffmpeg_data_id, program_id, stream_id], references: [ffmpeg_data_id, program_id, stream_id], onDelete: Cascade)
stream_id Int
program_id Int
ffmpeg_data_id Int
@@unique([ffmpeg_data_id, program_id, stream_id])
@@map("ffmpeg_media_codec")
}
model FfmpegMediaVideoProps {
id Int @id @default(autoincrement())
pixel_format String?
color_range String?
bits_per_channel Int?
color_space String?
color_primaries String?
color_transfer String?
field_order String?
chroma_location String?
width Int
height Int
aspect_ratio_num Int?
aspect_ratio_Den Int?
properties String?
codec FfmpegMediaCodec @relation(fields: [codec_id], references: [id], onDelete: Cascade)
codec_id Int @unique
@@map("ffmpeg_media_video_props")
}
model FfmpegMediaAudioProps {
id Int @id @default(autoincrement())
delay Int
padding Int
sample_rate Int?
sample_format String?
bit_per_sample Int?
channel_layout String?
codec FfmpegMediaCodec @relation(fields: [codec_id], references: [id], onDelete: Cascade)
codec_id Int @unique
@@map("ffmpeg_media_audio_props")
}
//// Tag ////
@ -478,20 +615,6 @@ model ObjectInAlbum {
@@map("object_in_album")
}
//// Comment ////
// model Comment {
// id Int @id @default(autoincrement())
// pub_id Bytes @unique
// content String
// date_created DateTime @default(now())
// date_modified DateTime @default(now())
// object_id Int?
// object Object? @relation(fields: [object_id], references: [id])
// @@map("comment")
// }
//// Indexer Rules ////
model IndexerRule {

85
core/src/api/ephemeral_files.rs
View file

@ -1,19 +1,22 @@
use crate::{
api::{files::create_file, utils::library},
api::{
files::{create_file, MediaData},
utils::library,
},
invalidate_query,
library::Library,
object::{
fs::{error::FileSystemJobsError, find_available_filename_for_duplicate},
media::media_data_extractor::{
can_extract_media_data_for_image, extract_media_data, MediaDataError,
},
media::exif_metadata_extractor::{can_extract_exif_data_for_image, extract_exif_data},
},
};
use sd_core_file_path_helper::IsolatedFilePathData;
use sd_file_ext::extensions::ImageExtension;
use sd_media_metadata::MediaMetadata;
use sd_file_ext::{
extensions::{Extension, ImageExtension},
kind::ObjectKind,
};
use sd_media_metadata::FFmpegMetadata;
use sd_utils::error::FileIOError;
use std::{ffi::OsStr, path::PathBuf, str::FromStr};
@ -50,30 +53,56 @@ pub(crate) fn mount() -> AlphaRouter<Ctx> {
R.router()
.procedure("getMediaData", {
R.query(|_, full_path: PathBuf| async move {
let Some(extension) = full_path.extension().and_then(|ext| ext.to_str()) else {
return Ok(None);
};
let kind: Option<ObjectKind> = Extension::resolve_conflicting(&full_path, false)
.await
.map(Into::into);
match kind {
Some(v) if v == ObjectKind::Image => {
let Some(extension) = full_path.extension().and_then(|ext| ext.to_str())
else {
return Ok(None);
};
// TODO(fogodev): change this when we have media data for audio and videos
let image_extension = ImageExtension::from_str(extension).map_err(|e| {
error!("Failed to parse image extension: {e:#?}");
rspc::Error::new(ErrorCode::BadRequest, "Invalid image extension".to_string())
})?;
let image_extension = ImageExtension::from_str(extension).map_err(|e| {
error!("Failed to parse image extension: {e:#?}");
rspc::Error::new(
ErrorCode::BadRequest,
"Invalid image extension".to_string(),
)
})?;
if !can_extract_media_data_for_image(&image_extension) {
return Ok(None);
}
if !can_extract_exif_data_for_image(&image_extension) {
return Ok(None);
}
match extract_media_data(full_path.clone()).await {
Ok(img_media_data) => Ok(Some(MediaMetadata::Image(Box::new(img_media_data)))),
Err(MediaDataError::MediaData(sd_media_metadata::Error::NoExifDataOnPath(
_,
))) => Ok(None),
Err(e) => Err(rspc::Error::with_cause(
ErrorCode::InternalServerError,
"Failed to extract media data".to_string(),
e,
)),
let exif_data = extract_exif_data(full_path)
.await
.map_err(|e| {
rspc::Error::with_cause(
ErrorCode::InternalServerError,
"Failed to extract media data".to_string(),
e,
)
})?
.map(MediaData::Exif);
Ok(exif_data)
}
Some(v) if v == ObjectKind::Audio || v == ObjectKind::Video => {
let ffmpeg_data = MediaData::FFmpeg(
FFmpegMetadata::from_path(full_path).await.map_err(|e| {
error!("{e:#?}");
rspc::Error::with_cause(
ErrorCode::InternalServerError,
e.to_string(),
e,
)
})?,
);
Ok(Some(ffmpeg_data))
}
_ => Ok(None), // No media data
}
})
})

27
core/src/api/files.rs
View file

@ -9,7 +9,7 @@ use crate::{
old_copy::OldFileCopierJobInit, old_cut::OldFileCutterJobInit,
old_delete::OldFileDeleterJobInit, old_erase::OldFileEraserJobInit,
},
media::media_data_image_from_prisma_data,
media::{exif_media_data_from_prisma_data, ffmpeg_data_from_prisma_data},
},
old_job::Job,
};
@ -17,11 +17,12 @@ use crate::{
use sd_core_file_path_helper::{FilePathError, IsolatedFilePathData};
use sd_core_prisma_helpers::{
file_path_to_isolate, file_path_to_isolate_with_id, object_with_file_paths,
object_with_media_data,
};
use sd_file_ext::kind::ObjectKind;
use sd_images::ConvertibleExtension;
use sd_media_metadata::MediaMetadata;
use sd_media_metadata::{ExifMetadata, FFmpegMetadata};
use sd_prisma::{
prisma::{file_path, location, object},
prisma_sync,
@ -59,6 +60,12 @@ enum FileCreateContextTypes {
Text,
}
#[derive(Serialize, Type)]
pub(crate) enum MediaData {
Exif(ExifMetadata),
FFmpeg(FFmpegMetadata),
}
pub(crate) fn mount() -> AlphaRouter<Ctx> {
R.router()
.procedure("get", {
@ -114,17 +121,23 @@ pub(crate) fn mount() -> AlphaRouter<Ctx> {
.db
.object()
.find_unique(object::id::equals(args))
.select(object::select!({ id kind media_data }))
.include(object_with_media_data::include())
.exec()
.await?
.and_then(|obj| {
Some(match obj.kind {
Some(v) if v == ObjectKind::Image as i32 => {
MediaMetadata::Image(Box::new(
media_data_image_from_prisma_data(obj.media_data?).ok()?,
Some(v) if v == ObjectKind::Image as i32 => MediaData::Exif(
exif_media_data_from_prisma_data(obj.exif_data?),
),
Some(v)
if v == ObjectKind::Audio as i32
|| v == ObjectKind::Video as i32 =>
{
MediaData::FFmpeg(ffmpeg_data_from_prisma_data(
obj.ffmpeg_data?,
))
}
_ => return None, // TODO(brxken128): audio and video
_ => return None, // No media data
})
})
.ok_or_else(|| {

10
core/src/api/search/media_data.rs → core/src/api/search/exif_data.rs
View file

@ -1,4 +1,4 @@
use sd_prisma::prisma::{self, media_data};
use sd_prisma::prisma::{self, exif_data};
use serde::{Deserialize, Serialize};
use specta::Type;
@ -7,11 +7,11 @@ use super::utils::*;
#[derive(Serialize, Deserialize, Type, Debug, Clone)]
#[serde(rename_all = "camelCase", tag = "field", content = "value")]
pub enum MediaDataOrder {
pub enum ExifDataOrder {
EpochTime(SortOrder),
}
impl MediaDataOrder {
impl ExifDataOrder {
pub fn get_sort_order(&self) -> prisma::SortOrder {
(*match self {
Self::EpochTime(v) => v,
@ -19,9 +19,9 @@ impl MediaDataOrder {
.into()
}
pub fn into_param(self) -> media_data::OrderByWithRelationParam {
pub fn into_param(self) -> exif_data::OrderByWithRelationParam {
let dir = self.get_sort_order();
use media_data::*;
use exif_data::*;
match self {
Self::EpochTime(_) => epoch_time::order(dir),
}

2
core/src/api/search/mod.rs
View file

@ -19,8 +19,8 @@ use rspc::{alpha::AlphaRouter, ErrorCode};
use serde::{Deserialize, Serialize};
use specta::Type;
pub mod exif_data;
pub mod file_path;
pub mod media_data;
pub mod object;
pub mod saved;
mod utils;

6
core/src/api/search/object.rs
View file

@ -8,7 +8,7 @@ use serde::{Deserialize, Serialize};
use specta::Type;
use super::{
media_data::*,
exif_data::*,
utils::{self, *},
};
@ -61,7 +61,7 @@ impl ObjectCursor {
pub enum ObjectOrder {
DateAccessed(SortOrder),
Kind(SortOrder),
MediaData(Box<MediaDataOrder>),
MediaData(Box<ExifDataOrder>),
}
impl ObjectOrder {
@ -81,7 +81,7 @@ impl ObjectOrder {
match self {
Self::DateAccessed(_) => date_accessed::order(dir),
Self::Kind(_) => kind::order(dir),
Self::MediaData(v) => media_data::order(vec![v.into_param()]),
Self::MediaData(v) => exif_data::order(vec![v.into_param()]),
}
}
}

1
core/src/lib.rs
View file

@ -1,3 +1,4 @@
#![recursion_limit = "256"]
#![warn(clippy::unwrap_used, clippy::panic)]
use crate::{

239
core/src/location/manager/watcher/utils.rs
View file

@ -8,8 +8,12 @@ use crate::{
},
object::{
media::{
media_data_extractor::{can_extract_media_data_for_image, extract_media_data},
media_data_image_to_query_params,
exif_data_image_to_query_params,
exif_metadata_extractor::{can_extract_exif_data_for_image, extract_exif_data},
ffmpeg_metadata_extractor::{
can_extract_ffmpeg_data_for_audio, can_extract_ffmpeg_data_for_video,
extract_ffmpeg_data, save_ffmpeg_data,
},
old_thumbnail::get_indexed_thumbnail_path,
},
old_file_identifier::FileMetadata,
@ -26,9 +30,12 @@ use sd_core_file_path_helper::{
};
use sd_core_prisma_helpers::file_path_with_object;
use sd_file_ext::{extensions::ImageExtension, kind::ObjectKind};
use sd_file_ext::{
extensions::{AudioExtension, ImageExtension, VideoExtension},
kind::ObjectKind,
};
use sd_prisma::{
prisma::{file_path, location, media_data, object},
prisma::{exif_data, file_path, location, object},
prisma_sync,
};
use sd_sync::OperationFactory;
@ -312,63 +319,100 @@ async fn inner_create_file(
)
.await?;
if !extension.is_empty() && matches!(kind, ObjectKind::Image | ObjectKind::Video) {
if !extension.is_empty()
&& matches!(
kind,
ObjectKind::Image | ObjectKind::Video | ObjectKind::Audio
) {
// Running in a detached task as thumbnail generation can take a while and we don't want to block the watcher
if matches!(kind, ObjectKind::Image | ObjectKind::Video) {
if let Some(cas_id) = cas_id {
spawn({
let extension = extension.clone();
let path = path.to_path_buf();
let node = node.clone();
let library_id = *library_id;
if let Some(cas_id) = cas_id {
spawn({
let extension = extension.clone();
let path = path.to_path_buf();
let node = node.clone();
let library_id = *library_id;
async move {
if let Err(e) = node
.thumbnailer
.generate_single_indexed_thumbnail(&extension, cas_id, path, library_id)
.await
{
error!("Failed to generate thumbnail in the watcher: {e:#?}");
async move {
if let Err(e) = node
.thumbnailer
.generate_single_indexed_thumbnail(&extension, cas_id, path, library_id)
.await
{
error!("Failed to generate thumbnail in the watcher: {e:#?}");
}
}
}
});
});
}
}
// TODO: Currently we only extract media data for images, remove this if later
if matches!(kind, ObjectKind::Image) {
if let Ok(image_extension) = ImageExtension::from_str(&extension) {
if can_extract_media_data_for_image(&image_extension) {
if let Ok(media_data) = extract_media_data(path)
.await
.map_err(|e| error!("Failed to extract media data: {e:#?}"))
{
let (sync_params, db_params) = media_data_image_to_query_params(media_data);
match kind {
ObjectKind::Image => {
if let Ok(image_extension) = ImageExtension::from_str(&extension) {
if can_extract_exif_data_for_image(&image_extension) {
if let Ok(Some(exif_data)) = extract_exif_data(path)
.await
.map_err(|e| error!("Failed to extract media data: {e:#?}"))
{
let (sync_params, db_params) =
exif_data_image_to_query_params(exif_data);
sync.write_ops(
db,
(
sync.shared_create(
prisma_sync::media_data::SyncId {
object: prisma_sync::object::SyncId {
pub_id: object_pub_id.clone(),
sync.write_ops(
db,
(
sync.shared_create(
prisma_sync::exif_data::SyncId {
object: prisma_sync::object::SyncId {
pub_id: object_pub_id.clone(),
},
},
},
sync_params,
),
db.media_data().upsert(
media_data::object_id::equals(object_id),
media_data::create(
object::id::equals(object_id),
db_params.clone(),
sync_params,
),
db.exif_data().upsert(
exif_data::object_id::equals(object_id),
exif_data::create(
object::id::equals(object_id),
db_params.clone(),
),
db_params,
),
db_params,
),
),
)
.await?;
)
.await?;
}
}
}
}
ObjectKind::Audio => {
if let Ok(audio_extension) = AudioExtension::from_str(&extension) {
if can_extract_ffmpeg_data_for_audio(&audio_extension) {
if let Ok(ffmpeg_data) = extract_ffmpeg_data(path)
.await
.map_err(|e| error!("Failed to extract media data: {e:#?}"))
{
save_ffmpeg_data([(ffmpeg_data, object_id)], db).await?;
}
}
}
}
ObjectKind::Video => {
if let Ok(video_extension) = VideoExtension::from_str(&extension) {
if can_extract_ffmpeg_data_for_video(&video_extension) {
if let Ok(ffmpeg_data) = extract_ffmpeg_data(path)
.await
.map_err(|e| error!("Failed to extract media data: {e:#?}"))
{
save_ffmpeg_data([(ffmpeg_data, object_id)], db).await?;
}
}
}
}
_ => {
// Do nothing
}
}
}
@ -680,43 +724,74 @@ async fn inner_update_file(
}
}
// TODO: Change this if to include ObjectKind::Video in the future
if let Some(ext) = &file_path.extension {
if let Ok(image_extension) = ImageExtension::from_str(ext) {
if can_extract_media_data_for_image(&image_extension)
&& matches!(kind, ObjectKind::Image)
{
if let Ok(media_data) = extract_media_data(full_path)
.await
.map_err(|e| error!("Failed to extract media data: {e:#?}"))
{
let (sync_params, db_params) =
media_data_image_to_query_params(media_data);
if let Some(extension) = &file_path.extension {
match kind {
ObjectKind::Image => {
if let Ok(image_extension) = ImageExtension::from_str(extension) {
if can_extract_exif_data_for_image(&image_extension) {
if let Ok(Some(exif_data)) = extract_exif_data(full_path)
.await
.map_err(|e| error!("Failed to extract media data: {e:#?}"))
{
let (sync_params, db_params) =
exif_data_image_to_query_params(exif_data);
sync.write_ops(
db,
(
sync.shared_create(
prisma_sync::media_data::SyncId {
object: prisma_sync::object::SyncId {
pub_id: object.pub_id.clone(),
},
},
sync_params,
),
db.media_data().upsert(
media_data::object_id::equals(object.id),
media_data::create(
object::id::equals(object.id),
db_params.clone(),
sync.write_ops(
db,
(
sync.shared_create(
prisma_sync::exif_data::SyncId {
object: prisma_sync::object::SyncId {
pub_id: object.pub_id.clone(),
},
},
sync_params,
),
db.exif_data().upsert(
exif_data::object_id::equals(object.id),
exif_data::create(
object::id::equals(object.id),
db_params.clone(),
),
db_params,
),
),
db_params,
),
),
)
.await?;
)
.await?;
}
}
}
}
ObjectKind::Audio => {
if let Ok(audio_extension) = AudioExtension::from_str(extension) {
if can_extract_ffmpeg_data_for_audio(&audio_extension) {
if let Ok(ffmpeg_data) = extract_ffmpeg_data(full_path)
.await
.map_err(|e| error!("Failed to extract media data: {e:#?}"))
{
save_ffmpeg_data([(ffmpeg_data, object.id)], db).await?;
}
}
}
}
ObjectKind::Video => {
if let Ok(video_extension) = VideoExtension::from_str(extension) {
if can_extract_ffmpeg_data_for_video(&video_extension) {
if let Ok(ffmpeg_data) = extract_ffmpeg_data(full_path)
.await
.map_err(|e| error!("Failed to extract media data: {e:#?}"))
{
save_ffmpeg_data([(ffmpeg_data, object.id)], db).await?;
}
}
}
}
_ => {
// Do nothing
}
}
}
}

16
core/src/location/non_indexed.rs
View file

@ -198,7 +198,7 @@ pub async fn walk(
}
};
let thumbnail_key = if should_generate_thumbnail {
let (thumbnail_key, has_created_thumbnail) = if should_generate_thumbnail {
if let Ok(cas_id) =
generate_cas_id(&path, entry.metadata.len())
.await
@ -221,12 +221,18 @@ pub async fn walk(
));
}
Some(get_ephemeral_thumb_key(&cas_id))
(
Some(get_ephemeral_thumb_key(&cas_id)),
library
.thumbnail_exists(&node, &cas_id)
.await
.map_err(NonIndexedLocationError::from)?,
)
} else {
None
(None, false)
}
} else {
None
(None, false)
};
tx.send(Ok(ExplorerItem::NonIndexedPath {
@ -242,7 +248,7 @@ pub async fn walk(
date_modified: entry.metadata.modified_or_now().into(),
size_in_bytes_bytes: entry.metadata.len().to_be_bytes().to_vec(),
},
has_created_thumbnail: false,
has_created_thumbnail,
}))
.await?;
}

89
core/src/object/media/media_data_extractor.rs → core/src/object/media/exif_metadata_extractor.rs
View file

@ -4,9 +4,8 @@ use sd_core_file_path_helper::IsolatedFilePathData;
use sd_core_prisma_helpers::file_path_for_media_processor;
use sd_file_ext::extensions::{Extension, ImageExtension, ALL_IMAGE_EXTENSIONS};
use sd_media_metadata::ImageMetadata;
use sd_prisma::prisma::{location, media_data, PrismaClient};
use sd_utils::error::FileIOError;
use sd_media_metadata::ExifMetadata;
use sd_prisma::prisma::{exif_data, location, PrismaClient};
use std::{collections::HashSet, path::Path};
@ -14,26 +13,21 @@ use futures_concurrency::future::Join;
use once_cell::sync::Lazy;
use serde::{Deserialize, Serialize};
use thiserror::Error;
use tokio::task::spawn_blocking;
use tracing::error;
use super::media_data_image_to_query;
use super::exif_data_image_to_query;
#[derive(Error, Debug)]
pub enum MediaDataError {
pub enum ExifDataError {
// Internal errors
#[error("database error: {0}")]
Database(#[from] prisma_client_rust::QueryError),
#[error(transparent)]
FileIO(#[from] FileIOError),
#[error(transparent)]
MediaData(#[from] sd_media_metadata::Error),
#[error("failed to join tokio task: {0}")]
TokioJoinHandle(#[from] tokio::task::JoinError),
}
#[derive(Serialize, Deserialize, Default, Debug)]
pub struct OldMediaDataExtractorMetadata {
pub struct OldExifDataExtractorMetadata {
pub extracted: u32,
pub skipped: u32,
}
@ -42,12 +36,12 @@ pub(super) static FILTERED_IMAGE_EXTENSIONS: Lazy<Vec<Extension>> = Lazy::new(||
ALL_IMAGE_EXTENSIONS
.iter()
.cloned()
.filter(can_extract_media_data_for_image)
.filter(can_extract_exif_data_for_image)
.map(Extension::Image)
.collect()
});
pub const fn can_extract_media_data_for_image(image_extension: &ImageExtension) -> bool {
pub const fn can_extract_exif_data_for_image(image_extension: &ImageExtension) -> bool {
use ImageExtension::*;
matches!(
image_extension,
@ -55,13 +49,10 @@ pub const fn can_extract_media_data_for_image(image_extension: &ImageExtension)
)
}
pub async fn extract_media_data(path: impl AsRef<Path>) -> Result<ImageMetadata, MediaDataError> {
let path = path.as_ref().to_path_buf();
// Running in a separated blocking thread due to MediaData blocking behavior (due to sync exif lib)
spawn_blocking(|| ImageMetadata::from_path(path))
.await?
.map_err(Into::into)
pub async fn extract_exif_data(
path: impl AsRef<Path> + Send,
) -> Result<Option<ExifMetadata>, ExifDataError> {
ExifMetadata::from_path(path).await.map_err(Into::into)
}
pub async fn process(
@ -70,46 +61,46 @@ pub async fn process(
location_path: impl AsRef<Path>,
db: &PrismaClient,
ctx_update_fn: &impl Fn(usize),
) -> Result<(OldMediaDataExtractorMetadata, JobRunErrors), MediaDataError> {
let mut run_metadata = OldMediaDataExtractorMetadata::default();
) -> Result<(OldExifDataExtractorMetadata, JobRunErrors), ExifDataError> {
let mut run_metadata = OldExifDataExtractorMetadata::default();
if files_paths.is_empty() {
return Ok((run_metadata, JobRunErrors::default()));
}
let location_path = location_path.as_ref();
let objects_already_with_media_data = db
.media_data()
.find_many(vec![media_data::object_id::in_vec(
let objects_already_with_exif_data = db
.exif_data()
.find_many(vec![exif_data::object_id::in_vec(
files_paths
.iter()
.filter_map(|file_path| file_path.object_id)
.collect(),
)])
.select(media_data::select!({ object_id }))
.select(exif_data::select!({ object_id }))
.exec()
.await?;
if files_paths.len() == objects_already_with_media_data.len() {
if files_paths.len() == objects_already_with_exif_data.len() {
// All files already have media data, skipping
run_metadata.skipped = files_paths.len() as u32;
return Ok((run_metadata, JobRunErrors::default()));
}
let objects_already_with_media_data = objects_already_with_media_data
let objects_already_with_exif_data = objects_already_with_exif_data
.into_iter()
.map(|media_data| media_data.object_id)
.map(|exif_data| exif_data.object_id)
.collect::<HashSet<_>>();
run_metadata.skipped = objects_already_with_media_data.len() as u32;
run_metadata.skipped = objects_already_with_exif_data.len() as u32;
let (media_datas, errors) = {
let maybe_media_data = files_paths
let (exif_datas, errors) = {
let maybe_exif_data = files_paths
.iter()
.enumerate()
.filter_map(|(idx, file_path)| {
file_path.object_id.and_then(|object_id| {
(!objects_already_with_media_data.contains(&object_id))
(!objects_already_with_exif_data.contains(&object_id))
.then_some((idx, file_path, object_id))
})
})
@ -120,7 +111,7 @@ pub async fn process(
.map(|iso_file_path| (idx, location_path.join(iso_file_path), object_id))
})
.map(|(idx, path, object_id)| async move {
let res = extract_media_data(&path).await;
let res = extract_exif_data(&path).await;
ctx_update_fn(idx + 1);
(res, path, object_id)
})
@ -128,37 +119,31 @@ pub async fn process(
.join()
.await;
let total_media_data = maybe_media_data.len();
let total_exif_data = maybe_exif_data.len();
maybe_media_data.into_iter().fold(
// In the good case, all media data were extracted
(Vec::with_capacity(total_media_data), Vec::new()),
|(mut media_datas, mut errors), (maybe_media_data, path, object_id)| {
match maybe_media_data {
Ok(media_data) => media_datas.push((media_data, object_id)),
Err(MediaDataError::MediaData(sd_media_metadata::Error::NoExifDataOnPath(
_,
))) => {
maybe_exif_data.into_iter().fold(
// In the good case, all exif data were extracted
(Vec::with_capacity(total_exif_data), Vec::new()),
|(mut exif_datas, mut errors), (maybe_exif_data, path, object_id)| {
match maybe_exif_data {
Ok(Some(exif_data)) => exif_datas.push((exif_data, object_id)),
Ok(None) => {
// No exif data on path, skipping
run_metadata.skipped += 1;
}
Err(e) => errors.push((e, path)),
}
(media_datas, errors)
(exif_datas, errors)
},
)
};
let created = db
.media_data()
.exif_data()
.create_many(
media_datas
exif_datas
.into_iter()
.filter_map(|(media_data, object_id)| {
media_data_image_to_query(media_data, object_id)
.map_err(|e| error!("{e:#?}"))
.ok()
})
.map(|(exif_data, object_id)| exif_data_image_to_query(exif_data, object_id))
.collect(),
)
.skip_duplicates()

660
core/src/object/media/ffmpeg_metadata_extractor.rs Normal file
View file

@ -0,0 +1,660 @@
use crate::old_job::JobRunErrors;
use prisma_client_rust::QueryError;
use sd_core_file_path_helper::IsolatedFilePathData;
use sd_core_prisma_helpers::file_path_for_media_processor;
use sd_file_ext::extensions::{
AudioExtension, Extension, VideoExtension, ALL_AUDIO_EXTENSIONS, ALL_VIDEO_EXTENSIONS,
};
use sd_media_metadata::{
ffmpeg::{
audio_props::AudioProps,
chapter::Chapter,
codec::{Codec, Props},
metadata::Metadata,
program::Program,
stream::Stream,
video_props::VideoProps,
},
FFmpegMetadata,
};
use sd_prisma::prisma::{
ffmpeg_data, ffmpeg_media_audio_props, ffmpeg_media_chapter, ffmpeg_media_codec,
ffmpeg_media_program, ffmpeg_media_stream, ffmpeg_media_video_props, location, object,
PrismaClient,
};
use sd_utils::db::ffmpeg_data_field_to_db;
use std::{
collections::{HashMap, HashSet},
path::Path,
};
use futures_concurrency::future::{Join, TryJoin};
use once_cell::sync::Lazy;
use serde::{Deserialize, Serialize};
use thiserror::Error;
use tracing::error;
#[derive(Error, Debug)]
pub enum FFmpegDataError {
// Internal errors
#[error("database error: {0}")]
Database(#[from] prisma_client_rust::QueryError),
#[error(transparent)]
MediaData(#[from] sd_media_metadata::Error),
}
#[derive(Serialize, Deserialize, Default, Debug)]
pub struct OldFFmpegDataExtractorMetadata {
pub extracted: u32,
pub skipped: u32,
}
pub(super) static FILTERED_AUDIO_AND_VIDEO_EXTENSIONS: Lazy<Vec<Extension>> = Lazy::new(|| {
ALL_AUDIO_EXTENSIONS
.iter()
.copied()
.filter(can_extract_ffmpeg_data_for_audio)
.map(Extension::Audio)
.chain(
ALL_VIDEO_EXTENSIONS
.iter()
.copied()
.filter(can_extract_ffmpeg_data_for_video)
.map(Extension::Video),
)
.collect()
});
pub const fn can_extract_ffmpeg_data_for_audio(audio_extension: &AudioExtension) -> bool {
use AudioExtension::*;
// TODO: Remove from here any extension which ffmpeg can't extract metadata from
matches!(
audio_extension,
Mp3 | Mp2
| M4a | Wav | Aiff
| Aif | Flac | Ogg
| Oga | Opus | Wma
| Amr | Aac | Wv
| Voc | Tta | Loas
| Caf | Aptx | Adts
| Ast | Mid
)
}
pub const fn can_extract_ffmpeg_data_for_video(video_extension: &VideoExtension) -> bool {
use VideoExtension::*;
// TODO: Remove from here any extension which ffmpeg can't extract metadata from
matches!(
video_extension,
Avi | Avifs
| Qt | Mov | Swf
| Mjpeg | Ts | Mts
| Mpeg | Mxf | M2v
| Mpg | Mpe | M2ts
| Flv | Wm | _3gp
| M4v | Wmv | Asf
| Mp4 | Webm | Mkv
| Vob | Ogv | Wtv
| Hevc | F4v
)
}
pub async fn extract_ffmpeg_data(
path: impl AsRef<Path> + Send,
) -> Result<FFmpegMetadata, FFmpegDataError> {
FFmpegMetadata::from_path(path).await.map_err(Into::into)
}
pub async fn process(
files_paths: &[file_path_for_media_processor::Data],
location_id: location::id::Type,
location_path: impl AsRef<Path> + Send,
db: &PrismaClient,
ctx_update_fn: &impl Fn(usize),
) -> Result<(OldFFmpegDataExtractorMetadata, JobRunErrors), FFmpegDataError> {
let mut run_metadata = OldFFmpegDataExtractorMetadata::default();
if files_paths.is_empty() {
return Ok((run_metadata, JobRunErrors::default()));
}
let location_path = location_path.as_ref();
let objects_already_with_ffmpeg_data = db
.ffmpeg_data()
.find_many(vec![ffmpeg_data::object_id::in_vec(
files_paths
.iter()
.filter_map(|file_path| file_path.object_id)
.collect(),
)])
.select(ffmpeg_data::select!({ object_id }))
.exec()
.await?;
if files_paths.len() == objects_already_with_ffmpeg_data.len() {
// All files already have media data, skipping
run_metadata.skipped = files_paths.len() as u32;
return Ok((run_metadata, JobRunErrors::default()));
}
let objects_already_with_ffmpeg_data = objects_already_with_ffmpeg_data
.into_iter()
.map(|ffmpeg_data| ffmpeg_data.object_id)
.collect::<HashSet<_>>();
run_metadata.skipped = objects_already_with_ffmpeg_data.len() as u32;
let mut errors = vec![];
let ffmpeg_datas = files_paths
.iter()
.enumerate()
.filter_map(|(idx, file_path)| {
file_path.object_id.and_then(|object_id| {
(!objects_already_with_ffmpeg_data.contains(&object_id))
.then_some((idx, file_path, object_id))
})
})
.filter_map(|(idx, file_path, object_id)| {
IsolatedFilePathData::try_from((location_id, file_path))
.map_err(|e| error!("{e:#?}"))
.ok()
.map(|iso_file_path| (idx, location_path.join(iso_file_path), object_id))
})
.map(|(idx, path, object_id)| async move {
let res = extract_ffmpeg_data(&path).await;
ctx_update_fn(idx + 1);
(res, path, object_id)
})
.collect::<Vec<_>>()
.join()
.await
.into_iter()
.filter_map(|(res, path, object_id)| {
res.map(|ffmpeg_data| (ffmpeg_data, object_id))
.map_err(|e| errors.push((e, path)))
.ok()
})
.collect::<Vec<_>>();
let created = save_ffmpeg_data(ffmpeg_datas, db).await?;
run_metadata.extracted = created as u32;
run_metadata.skipped += errors.len() as u32;
Ok((
run_metadata,
errors
.into_iter()
.map(|(e, path)| format!("Couldn't process file: \"{}\"; Error: {e}", path.display()))
.collect::<Vec<_>>()
.into(),
))
}
pub async fn save_ffmpeg_data(
ffmpeg_datas: impl IntoIterator<Item = (FFmpegMetadata, object::id::Type)>,
db: &PrismaClient,
) -> Result<u32, QueryError> {
ffmpeg_datas
.into_iter()
.map(
move |(
FFmpegMetadata {
formats,
duration,
start_time,
bit_rate,
chapters,
programs,
metadata,
},
object_id,
)| {
db._transaction()
.with_timeout(30 * 1000)
.run(move |db| async move {
let data_id = create_ffmpeg_data(
formats, bit_rate, duration, start_time, metadata, object_id, &db,
)
.await?;
create_ffmpeg_chapters(data_id, chapters, &db).await?;
let streams = create_ffmpeg_programs(data_id, programs, &db).await?;
let codecs = create_ffmpeg_streams(data_id, streams, &db).await?;
let (audio_props, video_props) =
create_ffmpeg_codecs(data_id, codecs, &db).await?;
(
create_ffmpeg_audio_props(audio_props, &db),
create_ffmpeg_video_props(video_props, &db),
)
.try_join()
.await
.map(|_| ())
})
},
)
.collect::<Vec<_>>()
.try_join()
.await
.map(|created| created.len() as u32)
}
async fn create_ffmpeg_data(
formats: Vec<String>,
bit_rate: (u32, u32),
duration: Option<(u32, u32)>,
start_time: Option<(u32, u32)>,
metadata: Metadata,
object_id: i32,
db: &PrismaClient,
) -> Result<ffmpeg_data::id::Type, QueryError> {
db.ffmpeg_data()
.create(
formats.join(","),
ffmpeg_data_field_to_db((bit_rate.0 as i64) << 32 | bit_rate.1 as i64),
object::id::equals(object_id),
vec![
ffmpeg_data::duration::set(
duration.map(|(a, b)| ffmpeg_data_field_to_db((a as i64) << 32 | b as i64)),
),
ffmpeg_data::start_time::set(
start_time.map(|(a, b)| ffmpeg_data_field_to_db((a as i64) << 32 | b as i64)),
),
ffmpeg_data::metadata::set(
serde_json::to_vec(&metadata)
.map_err(|err| {
error!("Error reading FFmpegData metadata: {err:#?}");
err
})
.ok(),
),
],
)
.select(ffmpeg_data::select!({ id }))
.exec()
.await
.map(|data| data.id)
}
async fn create_ffmpeg_chapters(
ffmpeg_data_id: ffmpeg_data::id::Type,
chapters: Vec<Chapter>,
db: &PrismaClient,
) -> Result<(), QueryError> {
db.ffmpeg_media_chapter()
.create_many(
chapters
.into_iter()
.map(
|Chapter {
id: chapter_id,
start: (start_high, start_low),
end: (end_high, end_low),
time_base_den,
time_base_num,
metadata,
}| ffmpeg_media_chapter::CreateUnchecked {
chapter_id,
start: ffmpeg_data_field_to_db(
(start_high as i64) << 32 | start_low as i64,
),
end: ffmpeg_data_field_to_db((end_high as i64) << 32 | end_low as i64),
time_base_den,
time_base_num,
ffmpeg_data_id,
_params: vec![ffmpeg_media_chapter::metadata::set(
serde_json::to_vec(&metadata)
.map_err(|err| {
error!("Error reading FFmpegMediaChapter metadata: {err:#?}");
err
})
.ok(),
)],
},
)
.collect(),
)
.exec()
.await
.map(|_| ())
}
async fn create_ffmpeg_programs(
data_id: i32,
programs: Vec<Program>,
db: &PrismaClient,
) -> Result<Vec<(ffmpeg_media_program::program_id::Type, Vec<Stream>)>, QueryError> {
let (creates, streams_by_program_id) =
programs
.into_iter()
.map(
|Program {
id: program_id,
name,
metadata,
streams,
}| {
(
ffmpeg_media_program::CreateUnchecked {
program_id,
ffmpeg_data_id: data_id,
_params: vec![
ffmpeg_media_program::name::set(name.clone()),
ffmpeg_media_program::metadata::set(
serde_json::to_vec(&metadata)
.map_err(|err| {
error!("Error reading FFmpegMediaProgram metadata: {err:#?}");
err
})
.ok(),
),
],
},
(program_id, streams),
)
},
)
.unzip::<_, _, Vec<_>, Vec<_>>();
db.ffmpeg_media_program()
.create_many(creates)
.exec()
.await
.map(|_| streams_by_program_id)
}
async fn create_ffmpeg_streams(
ffmpeg_data_id: ffmpeg_data::id::Type,
streams: Vec<(ffmpeg_media_program::program_id::Type, Vec<Stream>)>,
db: &PrismaClient,
) -> Result<
Vec<(
ffmpeg_media_program::program_id::Type,
ffmpeg_media_stream::stream_id::Type,
Codec,
)>,
QueryError,
> {
let (creates, maybe_codecs) = streams
.into_iter()
.flat_map(|(program_id, streams)| {
streams.into_iter().map(
move |Stream {
id: stream_id,
name,
codec: maybe_codec,
aspect_ratio_num,
aspect_ratio_den,
frames_per_second_num,
frames_per_second_den,
time_base_real_den,
time_base_real_num,
dispositions,
metadata,
}| {
(
ffmpeg_media_stream::CreateUnchecked {
stream_id,
aspect_ratio_num,
aspect_ratio_den,
frames_per_second_num,
frames_per_second_den,
time_base_real_den,
time_base_real_num,
program_id,
ffmpeg_data_id,
_params: vec![
ffmpeg_media_stream::name::set(name),
ffmpeg_media_stream::dispositions::set(
(!dispositions.is_empty()).then_some(dispositions.join(",")),
),
ffmpeg_media_stream::title::set(metadata.title.clone()),
ffmpeg_media_stream::encoder::set(metadata.encoder.clone()),
ffmpeg_media_stream::language::set(metadata.language.clone()),
ffmpeg_media_stream::metadata::set(
serde_json::to_vec(&metadata)
.map_err(|err| {
error!("Error reading FFmpegMediaStream metadata: {err:#?}");
err
})
.ok(),
),
],
},
maybe_codec.map(|codec| (program_id, stream_id, codec)),
)
},
)
})
.unzip::<_, _, Vec<_>, Vec<_>>();
db.ffmpeg_media_stream()
.create_many(creates)
.exec()
.await
.map(|_| maybe_codecs.into_iter().flatten().collect())
}
async fn create_ffmpeg_codecs(
ffmpeg_data_id: ffmpeg_data::id::Type,
codecs: Vec<(
ffmpeg_media_program::program_id::Type,
ffmpeg_media_stream::stream_id::Type,
Codec,
)>,
db: &PrismaClient,
) -> Result<
(
Vec<(ffmpeg_media_codec::id::Type, AudioProps)>,
Vec<(ffmpeg_media_codec::id::Type, VideoProps)>,
),
QueryError,
> {
let expected_creates = codecs.len();
let (creates, mut audio_props, mut video_props) = codecs.into_iter().enumerate().fold(
(
Vec::with_capacity(expected_creates),
HashMap::with_capacity(expected_creates),
HashMap::with_capacity(expected_creates),
),
|(mut creates, mut audio_props, mut video_props),
(
idx,
(
program_id,
stream_id,
Codec {
kind,
sub_kind,
tag,
name,
profile,
bit_rate,
props: maybe_props,
},
),
)| {
creates.push(ffmpeg_media_codec::CreateUnchecked {
bit_rate,
stream_id,
program_id,
ffmpeg_data_id,
_params: vec![
ffmpeg_media_codec::kind::set(kind),
ffmpeg_media_codec::sub_kind::set(sub_kind),
ffmpeg_media_codec::tag::set(tag),
ffmpeg_media_codec::name::set(name),
ffmpeg_media_codec::profile::set(profile),
],
});
if let Some(props) = maybe_props {
match props {
Props::Audio(props) => {
audio_props.insert(idx, props);
}
Props::Video(props) => {
video_props.insert(idx, props);
}
Props::Subtitle(_) => {
// We don't care about subtitles props for now :D
}
}
}
(creates, audio_props, video_props)
},
);
let created_ids = creates
.into_iter()
.map(
|ffmpeg_media_codec::CreateUnchecked {
bit_rate,
stream_id,
program_id,
ffmpeg_data_id,
_params,
}| {
db.ffmpeg_media_codec()
.create_unchecked(bit_rate, stream_id, program_id, ffmpeg_data_id, _params)
.select(ffmpeg_media_codec::select!({ id }))
.exec()
},
)
.collect::<Vec<_>>()
.try_join()
.await?;
assert_eq!(
created_ids.len(),
expected_creates,
"Not all codecs were created and our invariant is broken!"
);
debug_assert!(
created_ids
.windows(2)
.all(|window| window[0].id < window[1].id),
"Codecs were created in a different order than we expected, our invariant is broken!"
);
Ok(created_ids.into_iter().enumerate().fold(
(
Vec::with_capacity(audio_props.len()),
Vec::with_capacity(video_props.len()),
),
|(mut a_props, mut v_props), (idx, codec_data)| {
if let Some(audio_props) = audio_props.remove(&idx) {
a_props.push((codec_data.id, audio_props));
} else if let Some(video_props) = video_props.remove(&idx) {
v_props.push((codec_data.id, video_props));
}
(a_props, v_props)
},
))
}
async fn create_ffmpeg_audio_props(
audio_props: Vec<(ffmpeg_media_codec::id::Type, AudioProps)>,
db: &PrismaClient,
) -> Result<(), QueryError> {
db.ffmpeg_media_audio_props()
.create_many(
audio_props
.into_iter()
.map(
|(
codec_id,
AudioProps {
delay,
padding,
sample_rate,
sample_format,
bit_per_sample,
channel_layout,
},
)| ffmpeg_media_audio_props::CreateUnchecked {
delay,
padding,
codec_id,
_params: vec![
ffmpeg_media_audio_props::sample_rate::set(sample_rate),
ffmpeg_media_audio_props::sample_format::set(sample_format),
ffmpeg_media_audio_props::bit_per_sample::set(bit_per_sample),
ffmpeg_media_audio_props::channel_layout::set(channel_layout),
],
},
)
.collect(),
)
.exec()
.await
.map(|_| ())
}
async fn create_ffmpeg_video_props(
video_props: Vec<(ffmpeg_media_codec::id::Type, VideoProps)>,
db: &PrismaClient,
) -> Result<(), QueryError> {
db.ffmpeg_media_video_props()
.create_many(
video_props
.into_iter()
.map(
|(
codec_id,
VideoProps {
pixel_format,
color_range,
bits_per_channel,
color_space,
color_primaries,
color_transfer,
field_order,
chroma_location,
width,
height,
aspect_ratio_num,
aspect_ratio_den,
properties,
},
)| {
ffmpeg_media_video_props::CreateUnchecked {
width,
height,
codec_id,
_params: vec![
ffmpeg_media_video_props::pixel_format::set(pixel_format),
ffmpeg_media_video_props::color_range::set(color_range),
ffmpeg_media_video_props::bits_per_channel::set(bits_per_channel),
ffmpeg_media_video_props::color_space::set(color_space),
ffmpeg_media_video_props::color_primaries::set(color_primaries),
ffmpeg_media_video_props::color_transfer::set(color_transfer),
ffmpeg_media_video_props::field_order::set(field_order),
ffmpeg_media_video_props::chroma_location::set(chroma_location),
ffmpeg_media_video_props::aspect_ratio_num::set(aspect_ratio_num),
ffmpeg_media_video_props::aspect_ratio_den::set(aspect_ratio_den),
ffmpeg_media_video_props::properties::set(Some(
properties.join(","),
)),
],
}
},
)
.collect(),
)
.exec()
.await
.map(|_| ())
}

239
core/src/object/media/mod.rs
View file

@ -1,18 +1,29 @@
pub mod media_data_extractor;
use sd_core_prisma_helpers::object_with_media_data;
use sd_media_metadata::{
ffmpeg::{
audio_props::AudioProps,
chapter::Chapter,
codec::{Codec, Props},
program::Program,
stream::Stream,
video_props::VideoProps,
},
ExifMetadata, FFmpegMetadata,
};
use sd_prisma::prisma::{
exif_data::*, ffmpeg_media_audio_props, ffmpeg_media_chapter, ffmpeg_media_video_props,
};
pub mod exif_metadata_extractor;
pub mod ffmpeg_metadata_extractor;
pub mod old_media_processor;
pub mod old_thumbnail;
pub use old_media_processor::OldMediaProcessorJobInit;
use sd_media_metadata::ImageMetadata;
use sd_prisma::prisma::media_data::*;
use sd_utils::db::ffmpeg_data_field_from_db;
use self::media_data_extractor::MediaDataError;
pub fn media_data_image_to_query(
mdi: ImageMetadata,
object_id: object_id::Type,
) -> Result<CreateUnchecked, MediaDataError> {
Ok(CreateUnchecked {
pub fn exif_data_image_to_query(mdi: ExifMetadata, object_id: object_id::Type) -> CreateUnchecked {
CreateUnchecked {
object_id,
_params: vec![
camera_data::set(serde_json::to_vec(&mdi.camera_data).ok()),
@ -25,11 +36,11 @@ pub fn media_data_image_to_query(
exif_version::set(mdi.exif_version),
epoch_time::set(mdi.date_taken.map(|x| x.unix_timestamp())),
],
})
}
}
pub fn media_data_image_to_query_params(
mdi: ImageMetadata,
pub fn exif_data_image_to_query_params(
mdi: ExifMetadata,
) -> (Vec<(&'static str, rmpv::Value)>, Vec<SetParam>) {
use sd_sync::option_sync_db_entry;
use sd_utils::chain_optional_iter;
@ -50,10 +61,8 @@ pub fn media_data_image_to_query_params(
.unzip()
}
pub fn media_data_image_from_prisma_data(
data: sd_prisma::prisma::media_data::Data,
) -> Result<ImageMetadata, MediaDataError> {
Ok(ImageMetadata {
pub fn exif_media_data_from_prisma_data(data: sd_prisma::prisma::exif_data::Data) -> ExifMetadata {
ExifMetadata {
camera_data: from_slice_option_to_option(data.camera_data).unwrap_or_default(),
date_taken: from_slice_option_to_option(data.media_date).unwrap_or_default(),
resolution: from_slice_option_to_option(data.resolution).unwrap_or_default(),
@ -62,7 +71,201 @@ pub fn media_data_image_from_prisma_data(
description: data.description,
copyright: data.copyright,
exif_version: data.exif_version,
})
}
}
pub fn ffmpeg_data_from_prisma_data(
object_with_media_data::ffmpeg_data::Data {
formats,
duration,
start_time,
bit_rate,
metadata,
chapters,
programs,
..
}: object_with_media_data::ffmpeg_data::Data,
) -> FFmpegMetadata {
FFmpegMetadata {
formats: formats.split(',').map(String::from).collect::<Vec<_>>(),
duration: duration.map(|duration| {
let duration = ffmpeg_data_field_from_db(&duration);
((duration >> 32) as u32, duration as u32)
}),
start_time: start_time.map(|start_time| {
let start_time = ffmpeg_data_field_from_db(&start_time);
((start_time >> 32) as u32, start_time as u32)
}),
bit_rate: {
let bit_rate = ffmpeg_data_field_from_db(&bit_rate);
((bit_rate >> 32) as u32, bit_rate as u32)
},
chapters: chapters
.into_iter()
.map(
|ffmpeg_media_chapter::Data {
chapter_id,
start,
end,
time_base_den,
time_base_num,
metadata,
..
}| Chapter {
id: chapter_id,
start: {
let start = ffmpeg_data_field_from_db(&start);
((start >> 32) as u32, start as u32)
},
end: {
let end = ffmpeg_data_field_from_db(&end);
((end >> 32) as u32, end as u32)
},
time_base_den,
time_base_num,
metadata: from_slice_option_to_option(metadata).unwrap_or_default(),
},
)
.collect(),
programs: programs
.into_iter()
.map(
|object_with_media_data::ffmpeg_data::programs::Data {
program_id,
name,
metadata,
streams,
..
}| Program {
id: program_id,
name,
streams: streams
.into_iter()
.map(
|object_with_media_data::ffmpeg_data::programs::streams::Data {
stream_id,
name,
aspect_ratio_num,
aspect_ratio_den,
frames_per_second_num,
frames_per_second_den,
time_base_real_den,
time_base_real_num,
dispositions,
metadata,
codec,
..
}| {
Stream {
id: stream_id,
name,
codec: codec.map(
|object_with_media_data::ffmpeg_data::programs::streams::codec::Data{
kind,
sub_kind,
tag,
name,
profile,
bit_rate,
audio_props,
video_props,
..
}| Codec {
kind,
sub_kind,
tag,
name,
profile,
bit_rate,
props: match (audio_props, video_props) {
(
Some(ffmpeg_media_audio_props::Data {
delay,
padding,
sample_rate,
sample_format,
bit_per_sample,
channel_layout,
..
}),
None,
) => Some(Props::Audio(AudioProps {
delay,
padding,
sample_rate,
sample_format,
bit_per_sample,
channel_layout,
})),
(
None,
Some(ffmpeg_media_video_props::Data {
pixel_format,
color_range,
bits_per_channel,
color_space,
color_primaries,
color_transfer,
field_order,
chroma_location,
width,
height,
aspect_ratio_num,
aspect_ratio_den,
properties,
..
}),
) => Some(Props::Video(VideoProps {
pixel_format,
color_range,
bits_per_channel,
color_space,
color_primaries,
color_transfer,
field_order,
chroma_location,
width,
height,
aspect_ratio_num,
aspect_ratio_den,
properties: properties
.map(|dispositions| {
dispositions
.split(',')
.map(String::from)
.collect::<Vec<_>>()
})
.unwrap_or_default(),
})),
_ => None,
},
}
),
aspect_ratio_num,
aspect_ratio_den,
frames_per_second_num,
frames_per_second_den,
time_base_real_den,
time_base_real_num,
dispositions: dispositions
.map(|dispositions| {
dispositions
.split(',')
.map(String::from)
.collect::<Vec<_>>()
})
.unwrap_or_default(),
metadata: from_slice_option_to_option(metadata).unwrap_or_default(),
}
},
)
.collect(),
metadata: from_slice_option_to_option(metadata).unwrap_or_default(),
},
)
.collect(),
metadata: from_slice_option_to_option(metadata).unwrap_or_default(),
}
}
#[must_use]

69
core/src/object/media/old_media_processor/job.rs
View file

@ -2,6 +2,7 @@ use crate::{
invalidate_query,
library::Library,
location::ScanState,
object::media::ffmpeg_metadata_extractor,
old_job::{
CurrentStep, JobError, JobInitOutput, JobReportUpdate, JobResult, JobStepOutput,
StatefulJob, WorkerContext,
@ -45,9 +46,10 @@ use tokio::time::sleep;
use tracing::{debug, error, info, trace, warn};
use super::{
media_data_extractor,
exif_metadata_extractor,
old_thumbnail::{self, GenerateThumbnailArgs},
process, BatchToProcess, MediaProcessorError, OldMediaProcessorMetadata,
process_audio_and_video, process_images, BatchToProcess, MediaProcessorError,
OldMediaProcessorMetadata,
};
const BATCH_SIZE: usize = 10;
@ -84,7 +86,8 @@ pub struct OldMediaProcessorJobData {
#[derive(Debug, Serialize, Deserialize)]
pub enum OldMediaProcessorJobStep {
ExtractMediaData(Vec<file_path_for_media_processor::Data>),
ExtractImageMediaData(Vec<file_path_for_media_processor::Data>),
ExtractAudioAndVideoMediaData(Vec<file_path_for_media_processor::Data>),
WaitThumbnails(usize),
#[cfg(feature = "ai")]
WaitLabels(usize),
@ -176,7 +179,10 @@ impl StatefulJob for OldMediaProcessorJobInit {
None
};
let file_paths = get_files_for_media_data_extraction(db, &iso_file_path).await?;
let file_paths_to_extract_exif_data =
get_files_for_image_media_data_extraction(db, &iso_file_path).await?;
let file_paths_to_extract_ffmpeg_data =
get_files_for_audio_and_video_media_data_extraction(db, &iso_file_path).await?;
#[cfg(feature = "ai")]
let file_paths_for_labeling =
@ -202,14 +208,23 @@ impl StatefulJob for OldMediaProcessorJobInit {
(uuid::Uuid::new_v4(), None)
};
let total_files = file_paths.len();
let total_files =
file_paths_to_extract_exif_data.len() + file_paths_to_extract_ffmpeg_data.len();
let chunked_files = file_paths
let chunked_files = file_paths_to_extract_exif_data
.into_iter()
.chunks(BATCH_SIZE)
.into_iter()
.map(|chunk| chunk.collect::<Vec<_>>())
.map(OldMediaProcessorJobStep::ExtractMediaData)
.map(OldMediaProcessorJobStep::ExtractImageMediaData)
.chain(
file_paths_to_extract_ffmpeg_data
.into_iter()
.chunks(BATCH_SIZE)
.into_iter()
.map(|chunk| chunk.collect::<Vec<_>>())
.map(OldMediaProcessorJobStep::ExtractAudioAndVideoMediaData),
)
.chain(
[(thumbs_to_process_count > 0).then_some(
OldMediaProcessorJobStep::WaitThumbnails(thumbs_to_process_count as usize),
@ -272,7 +287,7 @@ impl StatefulJob for OldMediaProcessorJobInit {
_: &Self::RunMetadata,
) -> Result<JobStepOutput<Self::Step, Self::RunMetadata>, JobError> {
match step {
OldMediaProcessorJobStep::ExtractMediaData(file_paths) => process(
OldMediaProcessorJobStep::ExtractImageMediaData(file_paths) => process_images(
file_paths,
self.location.id,
&data.location_path,
@ -287,6 +302,23 @@ impl StatefulJob for OldMediaProcessorJobInit {
.map(Into::into)
.map_err(Into::into),
OldMediaProcessorJobStep::ExtractAudioAndVideoMediaData(file_paths) => {
process_audio_and_video(
file_paths,
self.location.id,
&data.location_path,
&ctx.library.db,
&|completed_count| {
ctx.progress(vec![JobReportUpdate::CompletedTaskCount(
step_number * BATCH_SIZE + completed_count,
)]);
},
)
.await
.map(Into::into)
.map_err(Into::into)
}
OldMediaProcessorJobStep::WaitThumbnails(total_thumbs) => {
ctx.progress(vec![
JobReportUpdate::TaskCount(*total_thumbs),
@ -417,7 +449,7 @@ impl StatefulJob for OldMediaProcessorJobInit {
.display()
);
if run_metadata.media_data.extracted > 0 {
if run_metadata.exif_data.extracted > 0 || run_metadata.ffmpeg_data.extracted > 0 {
invalidate_query!(ctx.library, "search.paths");
}
@ -512,14 +544,27 @@ async fn dispatch_thumbnails_for_processing(
Ok(thumbs_count as u32)
}
async fn get_files_for_media_data_extraction(
async fn get_files_for_image_media_data_extraction(
db: &PrismaClient,
parent_iso_file_path: &IsolatedFilePathData<'_>,
) -> Result<Vec<file_path_for_media_processor::Data>, MediaProcessorError> {
get_all_children_files_by_extensions(
db,
parent_iso_file_path,
&media_data_extractor::FILTERED_IMAGE_EXTENSIONS,
&exif_metadata_extractor::FILTERED_IMAGE_EXTENSIONS,
)
.await
.map_err(Into::into)
}
async fn get_files_for_audio_and_video_media_data_extraction(
db: &PrismaClient,
parent_iso_file_path: &IsolatedFilePathData<'_>,
) -> Result<Vec<file_path_for_media_processor::Data>, MediaProcessorError> {
get_all_children_files_by_extensions(
db,
parent_iso_file_path,
&ffmpeg_metadata_extractor::FILTERED_AUDIO_AND_VIDEO_EXTENSIONS,
)
.await
.map_err(Into::into)
@ -546,7 +591,7 @@ async fn get_files_for_labeling(
ORDER BY materialized_path ASC",
// Ordering by materialized_path so we can prioritize processing the first files
// in the above part of the directories tree
&media_data_extractor::FILTERED_IMAGE_EXTENSIONS
&exif_metadata_extractor::FILTERED_IMAGE_EXTENSIONS
.iter()
.map(|ext| format!("LOWER('{ext}')"))
.collect::<Vec<_>>()

57
core/src/object/media/old_media_processor/mod.rs
View file

@ -12,7 +12,8 @@ use thiserror::Error;
use tracing::error;
use super::{
media_data_extractor::{self, MediaDataError, OldMediaDataExtractorMetadata},
exif_metadata_extractor::{self, ExifDataError, OldExifDataExtractorMetadata},
ffmpeg_metadata_extractor::{self, FFmpegDataError, OldFFmpegDataExtractorMetadata},
old_thumbnail::{self, BatchToProcess, ThumbnailerError},
};
@ -35,20 +36,35 @@ pub enum MediaProcessorError {
#[error(transparent)]
Thumbnailer(#[from] ThumbnailerError),
#[error(transparent)]
MediaDataExtractor(#[from] MediaDataError),
ExifMediaDataExtractor(#[from] ExifDataError),
#[error(transparent)]
FFmpegDataExtractor(#[from] FFmpegDataError),
}
#[derive(Debug, Serialize, Deserialize, Default)]
pub struct OldMediaProcessorMetadata {
media_data: OldMediaDataExtractorMetadata,
exif_data: OldExifDataExtractorMetadata,
ffmpeg_data: OldFFmpegDataExtractorMetadata,
thumbs_processed: u32,
labels_extracted: u32,
}
impl From<OldMediaDataExtractorMetadata> for OldMediaProcessorMetadata {
fn from(media_data: OldMediaDataExtractorMetadata) -> Self {
impl From<OldExifDataExtractorMetadata> for OldMediaProcessorMetadata {
fn from(exif_data: OldExifDataExtractorMetadata) -> Self {
Self {
media_data,
exif_data,
ffmpeg_data: Default::default(),
thumbs_processed: 0,
labels_extracted: 0,
}
}
}
impl From<OldFFmpegDataExtractorMetadata> for OldMediaProcessorMetadata {
fn from(ffmpeg_data: OldFFmpegDataExtractorMetadata) -> Self {
Self {
exif_data: Default::default(),
ffmpeg_data,
thumbs_processed: 0,
labels_extracted: 0,
}
@ -57,24 +73,37 @@ impl From<OldMediaDataExtractorMetadata> for OldMediaProcessorMetadata {
impl JobRunMetadata for OldMediaProcessorMetadata {
fn update(&mut self, new_data: Self) {
self.media_data.extracted += new_data.media_data.extracted;
self.media_data.skipped += new_data.media_data.skipped;
self.exif_data.extracted += new_data.exif_data.extracted;
self.exif_data.skipped += new_data.exif_data.skipped;
self.ffmpeg_data.extracted += new_data.ffmpeg_data.extracted;
self.ffmpeg_data.skipped += new_data.ffmpeg_data.skipped;
self.thumbs_processed += new_data.thumbs_processed;
self.labels_extracted += new_data.labels_extracted;
}
}
pub async fn process(
pub async fn process_images(
files_paths: &[file_path_for_media_processor::Data],
location_id: location::id::Type,
location_path: impl AsRef<Path>,
location_path: impl AsRef<Path> + Send,
db: &PrismaClient,
ctx_update_fn: &impl Fn(usize),
) -> Result<(OldMediaProcessorMetadata, JobRunErrors), MediaProcessorError> {
// Add here new kinds of media processing if necessary in the future
media_data_extractor::process(files_paths, location_id, location_path, db, ctx_update_fn)
exif_metadata_extractor::process(files_paths, location_id, location_path, db, ctx_update_fn)
.await
.map(|(media_data, errors)| (media_data.into(), errors))
.map(|(exif_extraction_metadata, errors)| (exif_extraction_metadata.into(), errors))
.map_err(Into::into)
}
pub async fn process_audio_and_video(
files_paths: &[file_path_for_media_processor::Data],
location_id: location::id::Type,
location_path: impl AsRef<Path> + Send,
db: &PrismaClient,
ctx_update_fn: &impl Fn(usize),
) -> Result<(OldMediaProcessorMetadata, JobRunErrors), MediaProcessorError> {
ffmpeg_metadata_extractor::process(files_paths, location_id, location_path, db, ctx_update_fn)
.await
.map(|(ffmpeg_extraction_metadata, errors)| (ffmpeg_extraction_metadata.into(), errors))
.map_err(Into::into)
}

69
core/src/object/media/old_media_processor/shallow.rs
View file

@ -1,7 +1,6 @@
use crate::{
invalidate_query,
library::Library,
object::media::old_thumbnail::GenerateThumbnailArgs,
old_job::{JobError, JobRunMetadata},
Node,
};
@ -32,8 +31,8 @@ use tracing::{debug, error};
use futures::StreamExt;
use super::{
media_data_extractor::{self, process},
old_thumbnail::{self, BatchToProcess},
exif_metadata_extractor, ffmpeg_metadata_extractor,
old_thumbnail::{self, BatchToProcess, GenerateThumbnailArgs},
MediaProcessorError, OldMediaProcessorMetadata,
};
@ -92,7 +91,10 @@ pub async fn old_shallow(
)
.await?;
let file_paths = get_files_for_media_data_extraction(db, &iso_file_path).await?;
let file_paths_to_extract_exif_data =
get_files_for_exif_media_data_extraction(db, &iso_file_path).await?;
let file_paths_to_extract_ffmpeg_data =
get_files_for_ffmpeg_media_data_extraction(db, &iso_file_path).await?;
#[cfg(feature = "ai")]
let file_paths_for_labelling =
@ -101,9 +103,17 @@ pub async fn old_shallow(
#[cfg(feature = "ai")]
let has_labels = !file_paths_for_labelling.is_empty();
let total_files = file_paths.len();
let total_files =
file_paths_to_extract_exif_data.len() + file_paths_to_extract_ffmpeg_data.len();
let chunked_files = file_paths
let chunked_files_to_extract_exif_data = file_paths_to_extract_exif_data
.into_iter()
.chunks(BATCH_SIZE)
.into_iter()
.map(Iterator::collect)
.collect::<Vec<Vec<_>>>();
let chunked_files_to_extract_ffmpeg_data = file_paths_to_extract_ffmpeg_data
.into_iter()
.chunks(BATCH_SIZE)
.into_iter()
@ -112,7 +122,7 @@ pub async fn old_shallow(
debug!(
"Preparing to process {total_files} files in {} chunks",
chunked_files.len()
chunked_files_to_extract_exif_data.len() + chunked_files_to_extract_ffmpeg_data.len()
);
#[cfg(feature = "ai")]
@ -131,21 +141,35 @@ pub async fn old_shallow(
let mut run_metadata = OldMediaProcessorMetadata::default();
for files in chunked_files {
let (more_run_metadata, errors) = process(&files, location.id, &location_path, db, &|_| {})
.await
.map_err(MediaProcessorError::from)?;
for files in chunked_files_to_extract_exif_data {
let (more_run_metadata, errors) =
exif_metadata_extractor::process(&files, location.id, &location_path, db, &|_| {})
.await
.map_err(MediaProcessorError::from)?;
run_metadata.update(more_run_metadata.into());
if !errors.is_empty() {
error!("Errors processing chunk of media data shallow extraction:\n{errors}");
error!("Errors processing chunk of image media data shallow extraction:\n{errors}");
}
}
for files in chunked_files_to_extract_ffmpeg_data {
let (more_run_metadata, errors) =
ffmpeg_metadata_extractor::process(&files, location.id, &location_path, db, &|_| {})
.await
.map_err(MediaProcessorError::from)?;
run_metadata.update(more_run_metadata.into());
if !errors.is_empty() {
error!("Errors processing chunk of audio or video media data shallow extraction:\n{errors}");
}
}
debug!("Media shallow processor run metadata: {run_metadata:?}");
if run_metadata.media_data.extracted > 0 {
if run_metadata.exif_data.extracted > 0 || run_metadata.ffmpeg_data.extracted > 0 {
invalidate_query!(library, "search.paths");
invalidate_query!(library, "search.objects");
}
@ -183,14 +207,27 @@ pub async fn old_shallow(
Ok(())
}
async fn get_files_for_media_data_extraction(
async fn get_files_for_exif_media_data_extraction(
db: &PrismaClient,
parent_iso_file_path: &IsolatedFilePathData<'_>,
) -> Result<Vec<file_path_for_media_processor::Data>, MediaProcessorError> {
get_files_by_extensions(
db,
parent_iso_file_path,
&media_data_extractor::FILTERED_IMAGE_EXTENSIONS,
&exif_metadata_extractor::FILTERED_IMAGE_EXTENSIONS,
)
.await
.map_err(Into::into)
}
async fn get_files_for_ffmpeg_media_data_extraction(
db: &PrismaClient,
parent_iso_file_path: &IsolatedFilePathData<'_>,
) -> Result<Vec<file_path_for_media_processor::Data>, MediaProcessorError> {
get_files_by_extensions(
db,
parent_iso_file_path,
&ffmpeg_metadata_extractor::FILTERED_AUDIO_AND_VIDEO_EXTENSIONS,
)
.await
.map_err(Into::into)
@ -214,7 +251,7 @@ async fn get_files_for_labeling(
AND LOWER(extension) IN ({})
AND materialized_path = {{}}
{}",
&media_data_extractor::FILTERED_IMAGE_EXTENSIONS
&exif_metadata_extractor::FILTERED_IMAGE_EXTENSIONS
.iter()
.map(|ext| format!("LOWER('{ext}')"))
.collect::<Vec<_>>()

19
core/src/object/media/old_thumbnail/process.rs
View file

@ -2,7 +2,7 @@ use crate::api::CoreEvent;
use sd_file_ext::extensions::{DocumentExtension, ImageExtension};
use sd_images::{format_image, scale_dimensions, ConvertibleExtension};
use sd_media_metadata::image::Orientation;
use sd_media_metadata::exif::Orientation;
use sd_prisma::prisma::location;
use sd_utils::error::FileIOError;
@ -467,12 +467,17 @@ async fn generate_image_thumbnail(
#[cfg(feature = "ffmpeg")]
async fn generate_video_thumbnail(
file_path: impl AsRef<Path>,
output_path: impl AsRef<Path>,
file_path: impl AsRef<Path> + Send,
output_path: impl AsRef<Path> + Send,
) -> Result<(), ThumbnailerError> {
use sd_ffmpeg::to_thumbnail;
use sd_ffmpeg::{to_thumbnail, ThumbnailSize};
to_thumbnail(file_path, output_path, 256, TARGET_QUALITY)
.await
.map_err(Into::into)
to_thumbnail(
file_path,
output_path,
ThumbnailSize::Scale(256),
TARGET_QUALITY,
)
.await
.map_err(Into::into)
}

2
crates/ai/Cargo.toml
View file

@ -4,7 +4,7 @@ version = "0.1.0"
authors = ["Ericson Soares <ericson@spacedrive.com>"]
readme = "README.md"
description = "A simple library to generate video thumbnails using ffmpeg with the webp format"
rust-version = "1.75.0"
rust-version = "1.75"
license = { workspace = true }
repository = { workspace = true }
edition = { workspace = true }

2
crates/crypto/Cargo.toml
View file

@ -1,6 +1,6 @@
[package]
name = "sd-crypto"
rust-version = "1.72.0"
rust-version = "1.72"
version = "0.0.0"
authors = ["Jake Robinson <jake@spacedrive.com>"]
description = """

13
crates/ffmpeg/Cargo.toml
View file

@ -1,15 +1,23 @@
[package]
name = "sd-ffmpeg"
version = "0.1.0"
authors = ["Ericson Soares <ericson.ds999@gmail.com>"]
authors = [
"Ericson Soares <ericson@spacedrive.com>",
"Vítor Vasconcellos <vitor@spacedrive.com>",
]
readme = "README.md"
description = "A simple library to generate video thumbnails using ffmpeg with the webp format"
rust-version = "1.64.0"
rust-version = "1.78"
license = { workspace = true }
repository = { workspace = true }
edition = { workspace = true }
[dependencies]
sd-utils = { path = "../utils" }
chrono = { workspace = true, features = ["serde"] }
image = { workspace = true }
libc = { workspace = true }
thiserror = { workspace = true }
tokio = { workspace = true, features = ["fs", "rt"] }
tracing = { workspace = true }
@ -17,7 +25,6 @@ webp = { workspace = true }
ffmpeg-sys-next = "6.0.1"
[dev-dependencies]
tempfile = { workspace = true }
tokio = { workspace = true, features = ["fs", "rt", "macros"] }

1
crates/ffmpeg/README.md
View file

@ -29,7 +29,6 @@ async fn main() -> Result<(), ThumbnailerError> {
let thumbnailer = ThumbnailerBuilder::new()
.width_and_height(420, 315)
.seek_percentage(0.25)?
.with_film_strip(false)
.quality(80.0)?
.build();

460
crates/ffmpeg/src/codec_ctx.rs Normal file
View file

@ -0,0 +1,460 @@
use crate::{
error::{Error, FFmpegError},
model::{FFmpegAudioProps, FFmpegCodec, FFmpegProps, FFmpegSubtitleProps, FFmpegVideoProps},
utils::check_error,
};
use std::{
ffi::{CStr, CString},
ptr,
};
use ffmpeg_sys_next::{
av_bprint_finalize, av_bprint_init, av_channel_layout_describe_bprint, av_chroma_location_name,
av_color_primaries_name, av_color_range_name, av_color_space_name, av_color_transfer_name,
av_fourcc_make_string, av_get_bits_per_sample, av_get_bytes_per_sample,
av_get_media_type_string, av_get_pix_fmt_name, av_get_sample_fmt_name, av_pix_fmt_desc_get,
av_reduce, avcodec_alloc_context3, avcodec_flush_buffers, avcodec_free_context,
avcodec_get_name, avcodec_open2, avcodec_parameters_to_context, avcodec_profile_name,
avcodec_receive_frame, avcodec_send_packet, AVBPrint, AVChromaLocation, AVCodec,
AVCodecContext, AVCodecParameters, AVColorPrimaries, AVColorRange, AVColorSpace,
AVColorTransferCharacteristic, AVFieldOrder, AVFrame, AVMediaType, AVPacket, AVPixelFormat,
AVRational, AVSampleFormat, AVERROR, AVERROR_EOF, AV_FOURCC_MAX_STRING_SIZE,
FF_CODEC_PROPERTY_CLOSED_CAPTIONS, FF_CODEC_PROPERTY_FILM_GRAIN, FF_CODEC_PROPERTY_LOSSLESS,
};
use libc::EAGAIN;
pub struct FFmpegCodecContext(*mut AVCodecContext);
impl FFmpegCodecContext {
pub(crate) fn new() -> Result<Self, Error> {
let ptr = unsafe { avcodec_alloc_context3(ptr::null_mut()) };
if ptr.is_null() {
Err(FFmpegError::VideoCodecAllocation)?;
}
Ok(Self(ptr))
}
pub(crate) fn as_ref(&self) -> &AVCodecContext {
unsafe { self.0.as_ref() }.expect("initialized on struct creation")
}
pub(crate) fn as_mut(&mut self) -> &mut AVCodecContext {
unsafe { self.0.as_mut() }.expect("initialized on struct creation")
}
pub(crate) fn parameters_to_context(
&mut self,
codec_params: &AVCodecParameters,
) -> Result<&Self, Error> {
check_error(
unsafe { avcodec_parameters_to_context(self.as_mut(), codec_params) },
"Fail to fill the codec context with codec parameters",
)?;
Ok(self)
}
pub(crate) fn open2(&mut self, video_codec: &AVCodec) -> Result<&Self, Error> {
check_error(
unsafe { avcodec_open2(self.as_mut(), video_codec, ptr::null_mut()) },
"Failed to open video codec",
)?;
Ok(self)
}
pub(crate) fn flush(&mut self) {
unsafe { avcodec_flush_buffers(self.as_mut()) };
}
pub(crate) fn send_packet(&mut self, packet: *mut AVPacket) -> Result<bool, FFmpegError> {
match unsafe { avcodec_send_packet(self.as_mut(), packet) } {
AVERROR_EOF => Ok(false),
ret if ret == AVERROR(EAGAIN) => Err(FFmpegError::Again),
ret if ret < 0 => Err(FFmpegError::from(ret)),
_ => Ok(true),
}
}
pub(crate) fn receive_frame(&mut self, frame: *mut AVFrame) -> Result<bool, FFmpegError> {
match unsafe { avcodec_receive_frame(self.as_mut(), frame) } {
AVERROR_EOF => Ok(false),
ret if ret == AVERROR(EAGAIN) => Err(FFmpegError::Again),
ret if ret < 0 => Err(FFmpegError::from(ret)),
_ => Ok(true),
}
}
fn kind(&self) -> (Option<String>, Option<String>) {
let kind = unsafe { av_get_media_type_string(self.as_ref().codec_type).as_ref() }
.map(|media_type| unsafe { CStr::from_ptr(media_type) });
let sub_kind = unsafe { self.as_ref().codec.as_ref() }
.and_then(|codec| unsafe { codec.name.as_ref() })
.map(|name| unsafe { CStr::from_ptr(name) })
.and_then(|sub_kind| {
if let Some(kind) = kind {
if kind == sub_kind {
return None;
}
}
Some(String::from_utf8_lossy(sub_kind.to_bytes()).to_string())
});
(
kind.map(|cstr| String::from_utf8_lossy(cstr.to_bytes()).to_string()),
sub_kind,
)
}
fn name(&self) -> Option<String> {
unsafe { avcodec_get_name(self.as_ref().codec_id).as_ref() }.map(|codec_name| {
let cstr = unsafe { CStr::from_ptr(codec_name) };
String::from_utf8_lossy(cstr.to_bytes()).to_string()
})
}
fn profile(&self) -> Option<String> {
if self.as_ref().profile == 0 {
None
} else {
unsafe { avcodec_profile_name(self.as_ref().codec_id, self.as_ref().profile).as_ref() }
.map(|profile| {
let cstr = unsafe { CStr::from_ptr(profile) };
String::from_utf8_lossy(cstr.to_bytes()).to_string()
})
}
}
fn tag(&self) -> Option<String> {
if self.as_ref().codec_tag != 0 {
CString::new(vec![
0;
usize::try_from(AV_FOURCC_MAX_STRING_SIZE).expect(
"AV_FOURCC_MAX_STRING_SIZE is 32, must fit in an usize"
)
])
.ok()
.map(|buffer| {
let tag = unsafe {
CString::from_raw(av_fourcc_make_string(
buffer.into_raw(),
self.as_ref().codec_tag,
))
};
String::from_utf8_lossy(tag.as_bytes()).to_string()
})
} else {
None
}
}
fn bit_rate(&self) -> i32 {
// TODO: use i64 instead of i32 when rspc supports it
let ctx = self.as_ref();
match self.as_ref().codec_type {
AVMediaType::AVMEDIA_TYPE_VIDEO
| AVMediaType::AVMEDIA_TYPE_DATA
| AVMediaType::AVMEDIA_TYPE_SUBTITLE
| AVMediaType::AVMEDIA_TYPE_ATTACHMENT => ctx.bit_rate.try_into().unwrap_or_default(),
AVMediaType::AVMEDIA_TYPE_AUDIO => {
let bits_per_sample = unsafe { av_get_bits_per_sample(ctx.codec_id) };
if bits_per_sample != 0 {
let bit_rate = ctx.sample_rate * ctx.ch_layout.nb_channels;
if bit_rate <= i32::MAX / bits_per_sample {
return bit_rate * (bits_per_sample);
}
}
ctx.bit_rate.try_into().unwrap_or_default()
}
_ => 0,
}
}
fn video_props(&self) -> Option<FFmpegVideoProps> {
let ctx = self.as_ref();
if ctx.codec_type != AVMediaType::AVMEDIA_TYPE_VIDEO {
return None;
}
let pixel_format = extract_pixel_format(ctx);
let bits_per_channel = extract_bits_per_channel(ctx);
let color_range = extract_color_range(ctx);
let (color_space, color_primaries, color_transfer) = extract_colors(ctx);
// Field Order
let field_order = extract_field_order(ctx);
// Chroma Sample Location
let chroma_location = extract_chroma_location(ctx);
let width = ctx.width;
let height = ctx.height;
let (aspect_ratio_num, aspect_ratio_den) = extract_aspect_ratio(ctx, width, height);
let mut properties = vec![];
if ctx.properties & (FF_CODEC_PROPERTY_LOSSLESS.unsigned_abs()) != 0 {
properties.push("Closed Captions".to_string());
}
if ctx.properties & (FF_CODEC_PROPERTY_CLOSED_CAPTIONS.unsigned_abs()) != 0 {
properties.push("Film Grain".to_string());
}
if ctx.properties & (FF_CODEC_PROPERTY_FILM_GRAIN.unsigned_abs()) != 0 {
properties.push("lossless".to_string());
}
Some(FFmpegVideoProps {
pixel_format,
color_range,
bits_per_channel,
color_space,
color_primaries,
color_transfer,
field_order,
chroma_location,
width,
height,
aspect_ratio_num,
aspect_ratio_den,
properties,
})
}
fn audio_props(&self) -> Option<FFmpegAudioProps> {
let ctx = self.as_ref();
if ctx.codec_type != AVMediaType::AVMEDIA_TYPE_AUDIO {
return None;
}
let sample_rate = if ctx.sample_rate > 0 {
Some(ctx.sample_rate)
} else {
None
};
let mut bprint = AVBPrint {
str_: ptr::null_mut(),
len: 0,
size: 0,
size_max: 0,
reserved_internal_buffer: [0; 1],
reserved_padding: [0; 1000],
};
unsafe {
av_bprint_init(&mut bprint, 0, u32::MAX /* AV_BPRINT_SIZE_UNLIMITED */);
};
let mut channel_layout = ptr::null_mut();
let channel_layout =
if unsafe { av_channel_layout_describe_bprint(&ctx.ch_layout, &mut bprint) } < 0
|| unsafe { av_bprint_finalize(&mut bprint, &mut channel_layout) } < 0
|| channel_layout.is_null()
{
None
} else {
let cstr = unsafe { CStr::from_ptr(channel_layout) };
Some(String::from_utf8_lossy(cstr.to_bytes()).to_string())
};
let sample_format = if ctx.sample_fmt == AVSampleFormat::AV_SAMPLE_FMT_NONE {
None
} else {
unsafe { av_get_sample_fmt_name(ctx.sample_fmt).as_ref() }.map(|sample_fmt| {
let cstr = unsafe { CStr::from_ptr(sample_fmt) };
String::from_utf8_lossy(cstr.to_bytes()).to_string()
})
};
let bit_per_sample = if ctx.bits_per_raw_sample > 0
&& ctx.bits_per_raw_sample != unsafe { av_get_bytes_per_sample(ctx.sample_fmt) } * 8
{
Some(ctx.bits_per_raw_sample)
} else {
None
};
Some(FFmpegAudioProps {
delay: ctx.initial_padding,
padding: ctx.trailing_padding,
sample_rate,
sample_format,
bit_per_sample,
channel_layout,
})
}
fn subtitle_props(&self) -> Option<FFmpegSubtitleProps> {
if self.as_ref().codec_type != AVMediaType::AVMEDIA_TYPE_SUBTITLE {
return None;
}
Some(FFmpegSubtitleProps {
width: self.as_ref().width,
height: self.as_ref().height,
})
}
fn props(&self) -> Option<FFmpegProps> {
match self.as_ref().codec_type {
AVMediaType::AVMEDIA_TYPE_VIDEO => self.video_props().map(FFmpegProps::Video),
AVMediaType::AVMEDIA_TYPE_AUDIO => self.audio_props().map(FFmpegProps::Audio),
AVMediaType::AVMEDIA_TYPE_SUBTITLE => self.subtitle_props().map(FFmpegProps::Subtitle),
_ => None,
}
}
}
fn extract_aspect_ratio(
ctx: &AVCodecContext,
width: i32,
height: i32,
) -> (Option<i32>, Option<i32>) {
if ctx.sample_aspect_ratio.num == 0 {
(None, None)
} else {
let mut display_aspect_ratio = AVRational { num: 0, den: 0 };
let num = i64::from(width * ctx.sample_aspect_ratio.num);
let den = i64::from(height * ctx.sample_aspect_ratio.den);
let max = 1024 * 1024;
unsafe {
av_reduce(
&mut display_aspect_ratio.num,
&mut display_aspect_ratio.den,
num,
den,
max,
);
}
(
Some(display_aspect_ratio.num),
Some(display_aspect_ratio.den),
)
}
}
fn extract_chroma_location(ctx: &AVCodecContext) -> Option<String> {
if ctx.chroma_sample_location == AVChromaLocation::AVCHROMA_LOC_UNSPECIFIED {
None
} else {
unsafe { av_chroma_location_name(ctx.chroma_sample_location).as_ref() }.map(
|chroma_location| {
let cstr = unsafe { CStr::from_ptr(chroma_location) };
String::from_utf8_lossy(cstr.to_bytes()).to_string()
},
)
}
}
fn extract_field_order(ctx: &AVCodecContext) -> Option<String> {
if ctx.field_order == AVFieldOrder::AV_FIELD_UNKNOWN {
None
} else {
Some(
(match ctx.field_order {
AVFieldOrder::AV_FIELD_TT => "top first",
AVFieldOrder::AV_FIELD_BB => "bottom first",
AVFieldOrder::AV_FIELD_TB => "top coded first (swapped)",
AVFieldOrder::AV_FIELD_BT => "bottom coded first (swapped)",
_ => "progressive",
})
.to_string(),
)
}
}
fn extract_colors(ctx: &AVCodecContext) -> (Option<String>, Option<String>, Option<String>) {
if ctx.colorspace == AVColorSpace::AVCOL_SPC_UNSPECIFIED
&& ctx.color_primaries == AVColorPrimaries::AVCOL_PRI_UNSPECIFIED
&& ctx.color_trc == AVColorTransferCharacteristic::AVCOL_TRC_UNSPECIFIED
{
(None, None, None)
} else {
let color_space =
unsafe { av_color_space_name(ctx.colorspace).as_ref() }.map(|color_space| {
let cstr = unsafe { CStr::from_ptr(color_space) };
String::from_utf8_lossy(cstr.to_bytes()).to_string()
});
let color_primaries = unsafe { av_color_primaries_name(ctx.color_primaries).as_ref() }.map(
|color_primaries| {
let cstr = unsafe { CStr::from_ptr(color_primaries) };
String::from_utf8_lossy(cstr.to_bytes()).to_string()
},
);
let color_transfer =
unsafe { av_color_transfer_name(ctx.color_trc).as_ref() }.map(|color_transfer| {
let cstr = unsafe { CStr::from_ptr(color_transfer) };
String::from_utf8_lossy(cstr.to_bytes()).to_string()
});
(color_space, color_primaries, color_transfer)
}
}
fn extract_color_range(ctx: &AVCodecContext) -> Option<String> {
if ctx.color_range == AVColorRange::AVCOL_RANGE_UNSPECIFIED {
None
} else {
unsafe { av_color_range_name(ctx.color_range).as_ref() }.map(|color_range| {
let cstr = unsafe { CStr::from_ptr(color_range) };
String::from_utf8_lossy(cstr.to_bytes()).to_string()
})
}
}
fn extract_bits_per_channel(ctx: &AVCodecContext) -> Option<i32> {
if ctx.bits_per_raw_sample == 0 || ctx.pix_fmt == AVPixelFormat::AV_PIX_FMT_NONE {
None
} else {
unsafe { av_pix_fmt_desc_get(ctx.pix_fmt).as_ref() }.and_then(|pix_fmt_desc| {
let comp = pix_fmt_desc.comp[0];
if ctx.bits_per_raw_sample < comp.depth {
Some(ctx.bits_per_raw_sample)
} else {
None
}
})
}
}
fn extract_pixel_format(ctx: &AVCodecContext) -> Option<String> {
if ctx.pix_fmt == AVPixelFormat::AV_PIX_FMT_NONE {
None
} else {
unsafe { av_get_pix_fmt_name(ctx.pix_fmt).as_ref() }.map(|pixel_format| {
let cstr = unsafe { CStr::from_ptr(pixel_format) };
String::from_utf8_lossy(cstr.to_bytes()).to_string()
})
}
}
impl Drop for FFmpegCodecContext {
fn drop(&mut self) {
if !self.0.is_null() {
unsafe { avcodec_free_context(&mut self.0) };
self.0 = ptr::null_mut();
}
}
}
impl From<&FFmpegCodecContext> for FFmpegCodec {
fn from(ctx: &FFmpegCodecContext) -> Self {
let (kind, sub_kind) = ctx.kind();
Self {
kind,
sub_kind,
name: ctx.name(),
profile: ctx.profile(),
tag: ctx.tag(),
bit_rate: ctx.bit_rate(),
props: ctx.props(),
}
}
}

181
crates/ffmpeg/src/dict.rs Normal file
View file

@ -0,0 +1,181 @@
use crate::{error::Error, model::FFmpegMetadata, utils::check_error};
use std::{ffi::CStr, ptr};
use chrono::DateTime;
use ffmpeg_sys_next::{
av_dict_free, av_dict_get, av_dict_iterate, av_dict_set, AVDictionary, AVDictionaryEntry,
AV_DICT_MATCH_CASE,
};
#[derive(Debug)]
pub struct FFmpegDictionary {
dict: *mut AVDictionary,
managed: bool,
}
impl FFmpegDictionary {
pub(crate) fn new(av_dict: Option<&mut AVDictionary>) -> Self {
av_dict.map_or_else(
|| Self {
dict: ptr::null_mut(),
managed: true,
},
|ptr| Self {
dict: ptr,
managed: false,
},
)
}
pub(crate) fn get(&self, key: &CStr) -> Option<String> {
if self.dict.is_null() {
return None;
}
unsafe { av_dict_get(self.dict, key.as_ptr(), ptr::null(), AV_DICT_MATCH_CASE).as_ref() }
.and_then(|entry| unsafe { entry.value.as_ref() })
.map(|value| {
let cstr = unsafe { CStr::from_ptr(value) };
String::from_utf8_lossy(cstr.to_bytes()).to_string()
})
}
pub(crate) fn remove(&mut self, key: &CStr) -> Result<(), Error> {
check_error(
unsafe {
av_dict_set(
&mut self.dict,
key.as_ptr(),
ptr::null(),
AV_DICT_MATCH_CASE,
)
},
"Fail to set dictionary key-value pair",
)?;
Ok(())
}
}
impl Drop for FFmpegDictionary {
fn drop(&mut self) {
if self.managed && !self.dict.is_null() {
unsafe { av_dict_free(&mut self.dict) };
self.dict = ptr::null_mut();
}
}
}
impl<'a> IntoIterator for &'a FFmpegDictionary {
type Item = (String, Option<String>);
type IntoIter = FFmpegDictIter<'a>;
#[inline]
fn into_iter(self) -> FFmpegDictIter<'a> {
FFmpegDictIter {
dict: self.dict,
prev: ptr::null(),
_lifetime: std::marker::PhantomData,
}
}
}
pub struct FFmpegDictIter<'a> {
dict: *mut AVDictionary,
prev: *const AVDictionaryEntry,
_lifetime: std::marker::PhantomData<&'a ()>,
}
impl<'a> Iterator for FFmpegDictIter<'a> {
type Item = (String, Option<String>);
fn next(&mut self) -> Option<(String, Option<String>)> {
unsafe { av_dict_iterate(self.dict, self.prev).as_ref() }.and_then(|prev| {
self.prev = prev;
let key = unsafe { prev.key.as_ref() }.map(|key| unsafe { CStr::from_ptr(key) });
let value =
unsafe { prev.value.as_ref() }.map(|value| unsafe { CStr::from_ptr(value) });
match (key, value) {
(None, _) => None,
(Some(key), None) => {
Some((String::from_utf8_lossy(key.to_bytes()).to_string(), None))
}
(Some(key), Some(value)) => Some((
String::from_utf8_lossy(key.to_bytes()).to_string(),
Some(String::from_utf8_lossy(value.to_bytes()).to_string()),
)),
}
})
}
}
impl From<&FFmpegDictionary> for FFmpegMetadata {
fn from(dict: &FFmpegDictionary) -> Self {
let mut media_metadata = Self::default();
for (key, value) in dict {
if let Some(value) = value {
match key.as_str() {
"album" => media_metadata.album = Some(value.clone()),
"album_artist" => media_metadata.album_artist = Some(value.clone()),
"artist" => media_metadata.artist = Some(value.clone()),
"comment" => media_metadata.comment = Some(value.clone()),
"composer" => media_metadata.composer = Some(value.clone()),
"copyright" => media_metadata.copyright = Some(value.clone()),
"creation_time" => {
if let Ok(creation_time) = DateTime::parse_from_rfc2822(&value) {
media_metadata.creation_time = Some(creation_time.into());
} else if let Ok(creation_time) = DateTime::parse_from_rfc3339(&value) {
media_metadata.creation_time = Some(creation_time.into());
}
}
"date" => {
if let Ok(date) = DateTime::parse_from_rfc2822(&value) {
media_metadata.date = Some(date.into());
} else if let Ok(date) = DateTime::parse_from_rfc3339(&value) {
media_metadata.date = Some(date.into());
}
}
"disc" => {
if let Ok(disc) = value.parse() {
media_metadata.disc = Some(disc);
}
}
"encoder" => media_metadata.encoder = Some(value.clone()),
"encoded_by" => media_metadata.encoded_by = Some(value.clone()),
"filename" => media_metadata.filename = Some(value.clone()),
"genre" => media_metadata.genre = Some(value.clone()),
"language" => media_metadata.language = Some(value.clone()),
"performer" => media_metadata.performer = Some(value.clone()),
"publisher" => media_metadata.publisher = Some(value.clone()),
"service_name" => media_metadata.service_name = Some(value.clone()),
"service_provider" => media_metadata.service_provider = Some(value.clone()),
"title" => media_metadata.title = Some(value.clone()),
"track" => {
if let Ok(track) = value.parse() {
media_metadata.track = Some(track);
}
}
"variant_bitrate" => {
if let Ok(variant_bit_rate) = value.parse() {
media_metadata.variant_bit_rate = Some(variant_bit_rate);
}
}
_ => {
media_metadata.custom.insert(key.clone(), value.clone());
}
}
}
}
media_metadata
}
}
impl From<FFmpegDictionary> for FFmpegMetadata {
fn from(dict: FFmpegDictionary) -> Self {
(&dict).into()
}
}

55
crates/ffmpeg/src/error.rs
View file

@ -1,5 +1,9 @@
use std::path::PathBuf;
use std::{ffi::c_int, num::TryFromIntError};
use sd_utils::error::FileIOError;
use std::{
ffi::{c_int, NulError},
num::TryFromIntError,
path::{Path, PathBuf},
};
use thiserror::Error;
use tokio::task::JoinError;
@ -16,43 +20,48 @@ use ffmpeg_sys_next::{
/// Error type for the library.
#[derive(Error, Debug)]
pub enum Error {
#[error("I/O Error: {0}")]
Io(#[from] std::io::Error),
#[error("Background task failed: {0}")]
BackgroundTaskFailed(#[from] JoinError),
#[error("the video is most likely corrupt and will be skipped: <path='{}'>", .0.display())]
CorruptVideo(Box<Path>),
#[error("Received an invalid quality, expected range [0.0, 100.0], received: {0}")]
InvalidQuality(f32),
#[error("Received an invalid seek percentage: {0}")]
InvalidSeekPercentage(f32),
#[error("Error while casting an integer to another integer type")]
IntCastError(#[from] TryFromIntError),
#[error("Duration for video stream is unavailable")]
NoVideoDuration,
#[error("Failed to allocate C data: {0}")]
NulError(#[from] NulError),
#[error("Path conversion error: Path: {0:#?}")]
PathConversion(PathBuf),
#[error("FFmpeg internal error: {0}")]
Ffmpeg(#[from] FfmpegError),
FFmpeg(#[from] FFmpegError),
#[error("FFmpeg internal error: {0}; Reason: {1}")]
FfmpegWithReason(FfmpegError, String),
FFmpegWithReason(FFmpegError, String),
#[error("Failed to decode video frame")]
FrameDecodeError,
#[error("Failed to seek video")]
SeekError,
#[error("Seek not allowed")]
SeekNotAllowed,
#[error("Received an invalid seek percentage: {0}")]
InvalidSeekPercentage(f32),
#[error("Received an invalid quality, expected range [0.0, 100.0], received: {0}")]
InvalidQuality(f32),
#[error("Background task failed: {0}")]
BackgroundTaskFailed(#[from] JoinError),
#[error("The video is most likely corrupt and will be skipped")]
CorruptVideo,
#[error("Error while casting an integer to another integer type")]
IntCastError(#[from] TryFromIntError),
#[error(transparent)]
FileIO(#[from] FileIOError),
}
/// Enum to represent possible errors from `FFmpeg` library
///
/// Extracted from <https://ffmpeg.org/doxygen/trunk/group__lavu__error.html>
#[derive(Error, Debug)]
pub enum FfmpegError {
pub enum FFmpegError {
#[error("Bitstream filter not found")]
BitstreamFilterNotFound,
#[error("Internal bug, also see AVERROR_BUG2")]
InternalBug,
#[error("Buffer too small")]
BufferTooSmall,
#[error("Context allocation error")]
ContextAllocation,
#[error("Decoder not found")]
DecoderNotFound,
#[error("Demuxer not found")]
@ -69,6 +78,8 @@ pub enum FfmpegError {
FilterNotFound,
#[error("Invalid data found when processing input")]
InvalidData,
#[error("Internal bug, also see AVERROR_BUG2")]
InternalBug,
#[error("Muxer not found")]
MuxerNotFound,
#[error("Option not found")]
@ -111,9 +122,13 @@ pub enum FfmpegError {
FilterGraphAllocation,
#[error("Codec Open Error")]
CodecOpen,
#[error("Data not found")]
NullError,
#[error("Resource temporarily unavailable")]
Again,
}
impl From<c_int> for FfmpegError {
impl From<c_int> for FFmpegError {
fn from(code: c_int) -> Self {
match code {
AVERROR_BSF_NOT_FOUND => Self::BitstreamFilterNotFound,

693
crates/ffmpeg/src/film_strip.rs
View file

@ -1,693 +0,0 @@
use crate::video_frame::VideoFrame;
static FILM_STRIP_4: [u8; 4 * 4 * 3] = [
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 107, 107, 107, 135, 135, 135, 55, 55, 55, 0, 0, 0,
159, 159, 159, 195, 195, 195, 82, 82, 82, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
];
static FILM_STRIP_8: [u8; 8 * 8 * 3] = [
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 32, 32, 32, 55, 55, 55, 58,
58, 58, 58, 58, 58, 52, 52, 52, 1, 1, 1, 0, 0, 0, 2, 2, 2, 133, 133, 133, 208, 208, 208, 219,
219, 219, 219, 219, 219, 203, 203, 203, 26, 26, 26, 0, 0, 0, 2, 2, 2, 158, 158, 158, 240, 240,
240, 251, 251, 251, 251, 251, 251, 235, 235, 235, 31, 31, 31, 0, 0, 0, 0, 0, 0, 70, 70, 70,
115, 115, 115, 121, 121, 121, 121, 121, 121, 110, 110, 110, 2, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0,
];
static FILM_STRIP_16: [u8; 16 * 16 * 3] = [
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, 9, 9, 12, 12, 12, 13, 13, 13, 13, 13, 13, 13,
13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 4, 4, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 56, 56, 56, 89, 89, 89, 114, 114, 114, 124, 124, 124, 128, 128, 128, 128, 128, 128,
128, 128, 128, 128, 128, 128, 122, 122, 122, 109, 109, 109, 19, 19, 19, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 9, 9, 9, 89, 89, 89, 140, 140, 140, 175, 175, 175, 190, 190, 190, 194, 194, 194,
194, 194, 194, 194, 194, 194, 193, 193, 193, 187, 187, 187, 168, 168, 168, 64, 64, 64, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 12, 12, 12, 113, 113, 113, 175, 175, 175, 214, 214, 214, 231, 231,
231, 235, 235, 235, 236, 236, 236, 236, 236, 236, 235, 235, 235, 228, 228, 228, 207, 207, 207,
80, 80, 80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 13, 13, 13, 123, 123, 123, 188, 188, 188, 229,
229, 229, 245, 245, 245, 250, 250, 250, 251, 251, 251, 251, 251, 251, 249, 249, 249, 243, 243,
243, 221, 221, 221, 86, 86, 86, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 12, 12, 12, 120, 120, 120,
184, 184, 184, 224, 224, 224, 241, 241, 241, 245, 245, 245, 246, 246, 246, 246, 246, 246, 245,
245, 245, 238, 238, 238, 217, 217, 217, 85, 85, 85, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1,
1, 103, 103, 103, 160, 160, 160, 198, 198, 198, 214, 214, 214, 218, 218, 218, 220, 220, 220,
220, 220, 220, 218, 218, 218, 212, 212, 212, 191, 191, 191, 34, 34, 34, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 26, 26, 26, 32, 32, 32, 35, 35, 35, 36, 36, 36, 36, 36, 36, 36,
36, 36, 36, 36, 36, 35, 35, 35, 10, 10, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
];
static FILM_STRIP_32: [u8; 32 * 32 * 3] = [
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 11, 11,
23, 23, 23, 28, 28, 28, 32, 32, 32, 34, 34, 34, 36, 36, 36, 37, 37, 37, 37, 37, 37, 38, 38, 38,
38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 37, 37, 37, 37, 37, 37, 35, 35, 35,
29, 29, 29, 3, 3, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 28, 28, 28, 54, 54, 54, 69, 69, 69, 83, 83, 83, 93, 93, 93,
101, 101, 101, 105, 105, 105, 108, 108, 108, 109, 109, 109, 111, 111, 111, 110, 110, 110, 110,
110, 110, 110, 110, 110, 110, 110, 110, 110, 110, 110, 109, 109, 109, 107, 107, 107, 103, 103,
103, 97, 97, 97, 88, 88, 88, 13, 13, 13, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 11, 11, 54, 54, 54, 74, 74, 74, 93, 93, 93, 110, 110,
110, 124, 124, 124, 133, 133, 133, 139, 139, 139, 143, 143, 143, 144, 144, 144, 145, 145, 145,
145, 145, 145, 145, 145, 145, 145, 145, 145, 146, 146, 146, 145, 145, 145, 144, 144, 144, 141,
141, 141, 136, 136, 136, 129, 129, 129, 118, 118, 118, 88, 88, 88, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 23, 23, 23, 69, 69, 69, 93, 93,
93, 117, 117, 117, 138, 138, 138, 154, 154, 154, 165, 165, 165, 172, 172, 172, 176, 176, 176,
178, 178, 178, 179, 179, 179, 179, 179, 179, 179, 179, 179, 179, 179, 179, 179, 179, 179, 178,
178, 178, 177, 177, 177, 174, 174, 174, 170, 170, 170, 161, 161, 161, 146, 146, 146, 128, 128,
128, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
28, 28, 28, 82, 82, 82, 110, 110, 110, 138, 138, 138, 162, 162, 162, 180, 180, 180, 192, 192,
192, 200, 200, 200, 204, 204, 204, 206, 206, 206, 207, 207, 207, 207, 207, 207, 207, 207, 207,
207, 207, 207, 207, 207, 207, 207, 207, 207, 205, 205, 205, 202, 202, 202, 197, 197, 197, 187,
187, 187, 172, 172, 172, 151, 151, 151, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 32, 32, 32, 92, 92, 92, 124, 124, 124, 154, 154, 154, 180,
180, 180, 199, 199, 199, 212, 212, 212, 220, 220, 220, 225, 225, 225, 226, 226, 226, 227, 227,
227, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 227, 227, 227, 226, 226, 226,
223, 223, 223, 217, 217, 217, 207, 207, 207, 191, 191, 191, 168, 168, 168, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 34, 34, 34, 100, 100, 100,
134, 134, 134, 165, 165, 165, 192, 192, 192, 212, 212, 212, 226, 226, 226, 234, 234, 234, 238,
238, 238, 240, 240, 240, 241, 241, 241, 241, 241, 241, 241, 241, 241, 241, 241, 241, 241, 241,
241, 241, 241, 241, 240, 240, 240, 236, 236, 236, 230, 230, 230, 220, 220, 220, 203, 203, 203,
180, 180, 180, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 36, 36, 36, 104, 104, 104, 138, 138, 138, 171, 171, 171, 199, 199, 199, 219, 219, 219,
233, 233, 233, 240, 240, 240, 245, 245, 245, 247, 247, 247, 248, 248, 248, 248, 248, 248, 248,
248, 248, 248, 248, 248, 248, 248, 248, 247, 247, 247, 246, 246, 246, 243, 243, 243, 237, 237,
237, 227, 227, 227, 210, 210, 210, 186, 186, 186, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 36, 36, 36, 105, 105, 105, 140, 140, 140, 173,
173, 173, 201, 201, 201, 222, 222, 222, 235, 235, 235, 243, 243, 243, 248, 248, 248, 250, 250,
250, 251, 251, 251, 251, 251, 251, 251, 251, 251, 251, 251, 251, 251, 251, 251, 250, 250, 250,
249, 249, 249, 246, 246, 246, 240, 240, 240, 229, 229, 229, 212, 212, 212, 188, 188, 188, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 36, 36, 36,
104, 104, 104, 138, 138, 138, 171, 171, 171, 199, 199, 199, 219, 219, 219, 233, 233, 233, 240,
240, 240, 245, 245, 245, 247, 247, 247, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248,
248, 248, 248, 248, 247, 247, 247, 246, 246, 246, 243, 243, 243, 237, 237, 237, 227, 227, 227,
210, 210, 210, 186, 186, 186, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 34, 34, 34, 100, 100, 100, 134, 134, 134, 165, 165, 165, 192, 192, 192,
212, 212, 212, 226, 226, 226, 234, 234, 234, 238, 238, 238, 240, 240, 240, 241, 241, 241, 241,
241, 241, 241, 241, 241, 241, 241, 241, 241, 241, 241, 241, 241, 241, 240, 240, 240, 236, 236,
236, 230, 230, 230, 220, 220, 220, 203, 203, 203, 180, 180, 180, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 19, 19, 19, 92, 92, 92, 124, 124,
124, 154, 154, 154, 180, 180, 180, 200, 200, 200, 212, 212, 212, 220, 220, 220, 225, 225, 225,
226, 226, 226, 227, 227, 227, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, 227,
227, 227, 226, 226, 226, 223, 223, 223, 217, 217, 217, 207, 207, 207, 191, 191, 191, 146, 146,
146, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 59, 59, 59, 110, 110, 110, 138, 138, 138, 162, 162, 162, 180, 180, 180, 193, 193, 193,
200, 200, 200, 204, 204, 204, 206, 206, 206, 207, 207, 207, 208, 208, 208, 208, 208, 208, 208,
208, 208, 208, 208, 208, 207, 207, 207, 205, 205, 205, 203, 203, 203, 197, 197, 197, 187, 187,
187, 172, 172, 172, 27, 27, 27, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 27, 27, 27, 58, 58, 58, 69, 69, 69, 77, 77, 77,
83, 83, 83, 86, 86, 86, 88, 88, 88, 89, 89, 89, 89, 89, 89, 90, 90, 90, 90, 90, 90, 90, 90, 90,
90, 90, 90, 89, 89, 89, 88, 88, 88, 87, 87, 87, 85, 85, 85, 70, 70, 70, 8, 8, 8, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0,
];
static FILM_STRIP_64: [u8; 64 * 64 * 3] = [
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16, 16, 16, 34, 34, 34, 47,
47, 47, 54, 54, 54, 59, 59, 59, 64, 64, 64, 68, 68, 68, 72, 72, 72, 75, 75, 75, 77, 77, 77, 79,
79, 79, 81, 81, 81, 82, 82, 82, 82, 82, 82, 83, 83, 83, 83, 83, 83, 84, 84, 84, 84, 84, 84, 84,
84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, 84,
84, 84, 83, 83, 83, 83, 83, 83, 82, 82, 82, 82, 82, 82, 81, 81, 81, 79, 79, 79, 77, 77, 77, 72,
72, 72, 57, 57, 57, 30, 30, 30, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 30, 30, 30, 46, 46,
46, 52, 52, 52, 59, 59, 59, 66, 66, 66, 72, 72, 72, 78, 78, 78, 82, 82, 82, 87, 87, 87, 90, 90,
90, 93, 93, 93, 95, 95, 95, 97, 97, 97, 98, 98, 98, 99, 99, 99, 100, 100, 100, 100, 100, 100,
101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101,
101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 101, 100, 100, 100, 100, 100,
100, 99, 99, 99, 98, 98, 98, 97, 97, 97, 95, 95, 95, 93, 93, 93, 90, 90, 90, 87, 87, 87, 82,
82, 82, 61, 61, 61, 8, 8, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 30, 30, 30, 47, 47, 47, 55, 55, 55, 63, 63, 63,
71, 71, 71, 78, 78, 78, 86, 86, 86, 92, 92, 92, 98, 98, 98, 103, 103, 103, 107, 107, 107, 110,
110, 110, 112, 112, 112, 114, 114, 114, 116, 116, 116, 117, 117, 117, 118, 118, 118, 118, 118,
118, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119,
119, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, 118, 118, 118, 118,
118, 118, 117, 117, 117, 116, 116, 116, 114, 114, 114, 113, 113, 113, 110, 110, 110, 107, 107,
107, 103, 103, 103, 98, 98, 98, 92, 92, 92, 67, 67, 67, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16, 16, 16, 46, 46, 46, 54, 54,
54, 64, 64, 64, 73, 73, 73, 82, 82, 82, 91, 91, 91, 99, 99, 99, 106, 106, 106, 113, 113, 113,
118, 118, 118, 123, 123, 123, 126, 126, 126, 129, 129, 129, 131, 131, 131, 133, 133, 133, 134,
134, 134, 135, 135, 135, 135, 135, 135, 136, 136, 136, 136, 136, 136, 136, 136, 136, 136, 136,
136, 136, 136, 136, 136, 136, 136, 136, 136, 136, 136, 136, 136, 136, 136, 136, 136, 136, 136,
136, 136, 136, 135, 135, 135, 135, 135, 135, 134, 134, 134, 133, 133, 133, 131, 131, 131, 129,
129, 129, 126, 126, 126, 123, 123, 123, 118, 118, 118, 113, 113, 113, 106, 106, 106, 99, 99,
99, 41, 41, 41, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 34, 34, 34, 52, 52, 52, 62, 62, 62, 73, 73, 73, 83, 83, 83, 94, 94, 94, 104,
104, 104, 113, 113, 113, 121, 121, 121, 128, 128, 128, 134, 134, 134, 139, 139, 139, 143, 143,
143, 146, 146, 146, 149, 149, 149, 150, 150, 150, 152, 152, 152, 152, 152, 152, 153, 153, 153,
153, 153, 153, 154, 154, 154, 154, 154, 154, 154, 154, 154, 154, 154, 154, 154, 154, 154, 154,
154, 154, 154, 154, 154, 154, 154, 154, 154, 154, 154, 153, 153, 153, 153, 153, 153, 153, 153,
153, 152, 152, 152, 150, 150, 150, 149, 149, 149, 146, 146, 146, 143, 143, 143, 139, 139, 139,
134, 134, 134, 128, 128, 128, 121, 121, 121, 113, 113, 113, 82, 82, 82, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 47, 47, 47, 59, 59, 59,
71, 71, 71, 82, 82, 82, 94, 94, 94, 105, 105, 105, 116, 116, 116, 126, 126, 126, 135, 135, 135,
143, 143, 143, 150, 150, 150, 155, 155, 155, 159, 159, 159, 163, 163, 163, 165, 165, 165, 167,
167, 167, 169, 169, 169, 169, 169, 169, 170, 170, 170, 170, 170, 170, 171, 171, 171, 171, 171,
171, 171, 171, 171, 171, 171, 171, 171, 171, 171, 171, 171, 171, 171, 171, 171, 171, 171, 171,
171, 171, 171, 170, 170, 170, 170, 170, 170, 169, 169, 169, 169, 169, 169, 167, 167, 167, 165,
165, 165, 163, 163, 163, 160, 160, 160, 155, 155, 155, 150, 150, 150, 143, 143, 143, 135, 135,
135, 126, 126, 126, 111, 111, 111, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 54, 54, 54, 66, 66, 66, 78, 78, 78, 91, 91, 91, 104, 104, 104,
116, 116, 116, 128, 128, 128, 139, 139, 139, 148, 148, 148, 157, 157, 157, 164, 164, 164, 169,
169, 169, 174, 174, 174, 178, 178, 178, 180, 180, 180, 182, 182, 182, 183, 183, 183, 184, 184,
184, 185, 185, 185, 185, 185, 185, 186, 186, 186, 186, 186, 186, 186, 186, 186, 186, 186, 186,
186, 186, 186, 186, 186, 186, 186, 186, 186, 186, 186, 186, 186, 186, 186, 186, 186, 186, 185,
185, 185, 184, 184, 184, 184, 184, 184, 182, 182, 182, 180, 180, 180, 178, 178, 178, 174, 174,
174, 170, 170, 170, 164, 164, 164, 157, 157, 157, 148, 148, 148, 139, 139, 139, 128, 128, 128,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
59, 59, 59, 72, 72, 72, 85, 85, 85, 99, 99, 99, 113, 113, 113, 126, 126, 126, 139, 139, 139,
150, 150, 150, 161, 161, 161, 169, 169, 169, 177, 177, 177, 183, 183, 183, 188, 188, 188, 191,
191, 191, 194, 194, 194, 196, 196, 196, 197, 197, 197, 198, 198, 198, 199, 199, 199, 199, 199,
199, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200,
200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, 199, 199, 199, 198, 198, 198, 198,
198, 198, 196, 196, 196, 194, 194, 194, 191, 191, 191, 188, 188, 188, 183, 183, 183, 177, 177,
177, 169, 169, 169, 161, 161, 161, 150, 150, 150, 139, 139, 139, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 64, 64, 64, 77, 77, 77, 92, 92,
92, 106, 106, 106, 121, 121, 121, 135, 135, 135, 149, 149, 149, 161, 161, 161, 172, 172, 172,
181, 181, 181, 189, 189, 189, 195, 195, 195, 200, 200, 200, 204, 204, 204, 207, 207, 207, 209,
209, 209, 210, 210, 210, 211, 211, 211, 212, 212, 212, 212, 212, 212, 213, 213, 213, 213, 213,
213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213,
213, 213, 213, 213, 213, 213, 212, 212, 212, 211, 211, 211, 210, 210, 210, 209, 209, 209, 207,
207, 207, 204, 204, 204, 200, 200, 200, 195, 195, 195, 189, 189, 189, 181, 181, 181, 172, 172,
172, 161, 161, 161, 149, 149, 149, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 68, 68, 68, 82, 82, 82, 97, 97, 97, 113, 113, 113, 128, 128,
128, 143, 143, 143, 157, 157, 157, 170, 170, 170, 181, 181, 181, 190, 190, 190, 199, 199, 199,
205, 205, 205, 210, 210, 210, 214, 214, 214, 217, 217, 217, 219, 219, 219, 220, 220, 220, 221,
221, 221, 222, 222, 222, 222, 222, 222, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223,
223, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223,
222, 222, 222, 221, 221, 221, 220, 220, 220, 219, 219, 219, 217, 217, 217, 214, 214, 214, 210,
210, 210, 205, 205, 205, 199, 199, 199, 191, 191, 191, 181, 181, 181, 170, 170, 170, 157, 157,
157, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 72, 72, 72, 87, 87, 87, 102, 102, 102, 118, 118, 118, 134, 134, 134, 150, 150, 150, 164,
164, 164, 177, 177, 177, 189, 189, 189, 198, 198, 198, 207, 207, 207, 213, 213, 213, 218, 218,
218, 222, 222, 222, 225, 225, 225, 227, 227, 227, 229, 229, 229, 229, 229, 229, 230, 230, 230,
231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231,
231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 230, 230, 230, 230, 230,
230, 229, 229, 229, 227, 227, 227, 225, 225, 225, 222, 222, 222, 218, 218, 218, 213, 213, 213,
207, 207, 207, 198, 198, 198, 189, 189, 189, 177, 177, 177, 164, 164, 164, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 75, 75, 75, 90, 90, 90,
106, 106, 106, 123, 123, 123, 139, 139, 139, 155, 155, 155, 170, 170, 170, 183, 183, 183, 195,
195, 195, 205, 205, 205, 213, 213, 213, 220, 220, 220, 225, 225, 225, 229, 229, 229, 232, 232,
232, 234, 234, 234, 236, 236, 236, 236, 236, 236, 237, 237, 237, 238, 238, 238, 238, 238, 238,
238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238,
238, 238, 238, 238, 238, 238, 238, 238, 237, 237, 237, 237, 237, 237, 236, 236, 236, 234, 234,
234, 232, 232, 232, 229, 229, 229, 225, 225, 225, 220, 220, 220, 213, 213, 213, 205, 205, 205,
195, 195, 195, 183, 183, 183, 170, 170, 170, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 77, 77, 77, 93, 93, 93, 109, 109, 109, 126, 126, 126,
143, 143, 143, 159, 159, 159, 174, 174, 174, 188, 188, 188, 200, 200, 200, 210, 210, 210, 218,
218, 218, 225, 225, 225, 230, 230, 230, 234, 234, 234, 237, 237, 237, 239, 239, 239, 241, 241,
241, 242, 242, 242, 242, 242, 242, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243,
243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243,
243, 243, 242, 242, 242, 242, 242, 242, 241, 241, 241, 239, 239, 239, 237, 237, 237, 234, 234,
234, 230, 230, 230, 225, 225, 225, 218, 218, 218, 210, 210, 210, 200, 200, 200, 188, 188, 188,
174, 174, 174, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 78, 78, 78, 94, 94, 94, 111, 111, 111, 128, 128, 128, 145, 145, 145, 162, 162, 162,
177, 177, 177, 191, 191, 191, 203, 203, 203, 213, 213, 213, 221, 221, 221, 228, 228, 228, 233,
233, 233, 237, 237, 237, 240, 240, 240, 242, 242, 242, 244, 244, 244, 245, 245, 245, 245, 245,
245, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246,
246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 245, 245, 245, 245,
245, 245, 244, 244, 244, 242, 242, 242, 240, 240, 240, 237, 237, 237, 233, 233, 233, 228, 228,
228, 221, 221, 221, 213, 213, 213, 203, 203, 203, 191, 191, 191, 177, 177, 177, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 80, 80, 80, 96,
96, 96, 113, 113, 113, 130, 130, 130, 147, 147, 147, 164, 164, 164, 179, 179, 179, 193, 193,
193, 205, 205, 205, 215, 215, 215, 224, 224, 224, 230, 230, 230, 236, 236, 236, 239, 239, 239,
242, 242, 242, 244, 244, 244, 246, 246, 246, 247, 247, 247, 247, 247, 247, 248, 248, 248, 248,
248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248,
248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 247, 247, 247, 247, 247, 247, 246, 246, 246,
244, 244, 244, 242, 242, 242, 240, 240, 240, 236, 236, 236, 230, 230, 230, 224, 224, 224, 215,
215, 215, 205, 205, 205, 193, 193, 193, 179, 179, 179, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 80, 80, 80, 96, 96, 96, 113, 113, 113,
131, 131, 131, 148, 148, 148, 165, 165, 165, 180, 180, 180, 194, 194, 194, 206, 206, 206, 217,
217, 217, 225, 225, 225, 232, 232, 232, 237, 237, 237, 241, 241, 241, 244, 244, 244, 246, 246,
246, 248, 248, 248, 249, 249, 249, 249, 249, 249, 250, 250, 250, 250, 250, 250, 250, 250, 250,
250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250,
250, 250, 250, 250, 250, 249, 249, 249, 249, 249, 249, 248, 248, 248, 246, 246, 246, 244, 244,
244, 241, 241, 241, 237, 237, 237, 232, 232, 232, 225, 225, 225, 217, 217, 217, 206, 206, 206,
194, 194, 194, 180, 180, 180, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 80, 80, 80, 96, 96, 96, 113, 113, 113, 131, 131, 131, 148, 148, 148,
165, 165, 165, 180, 180, 180, 194, 194, 194, 206, 206, 206, 217, 217, 217, 225, 225, 225, 232,
232, 232, 237, 237, 237, 241, 241, 241, 244, 244, 244, 246, 246, 246, 248, 248, 248, 249, 249,
249, 249, 249, 249, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250,
250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 249,
249, 249, 249, 249, 249, 248, 248, 248, 246, 246, 246, 244, 244, 244, 241, 241, 241, 237, 237,
237, 232, 232, 232, 225, 225, 225, 217, 217, 217, 206, 206, 206, 194, 194, 194, 180, 180, 180,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
80, 80, 80, 96, 96, 96, 113, 113, 113, 130, 130, 130, 147, 147, 147, 164, 164, 164, 179, 179,
179, 193, 193, 193, 205, 205, 205, 215, 215, 215, 224, 224, 224, 230, 230, 230, 236, 236, 236,
239, 239, 239, 242, 242, 242, 244, 244, 244, 246, 246, 246, 247, 247, 247, 247, 247, 247, 248,
248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248,
248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 248, 247, 247, 247, 247, 247, 247,
246, 246, 246, 244, 244, 244, 242, 242, 242, 240, 240, 240, 236, 236, 236, 230, 230, 230, 224,
224, 224, 215, 215, 215, 205, 205, 205, 193, 193, 193, 179, 179, 179, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 78, 78, 78, 94, 94, 94,
111, 111, 111, 128, 128, 128, 145, 145, 145, 162, 162, 162, 177, 177, 177, 191, 191, 191, 203,
203, 203, 213, 213, 213, 221, 221, 221, 228, 228, 228, 233, 233, 233, 237, 237, 237, 240, 240,
240, 242, 242, 242, 244, 244, 244, 245, 245, 245, 245, 245, 245, 246, 246, 246, 246, 246, 246,
246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246, 246,
246, 246, 246, 246, 246, 246, 246, 246, 245, 245, 245, 245, 245, 245, 244, 244, 244, 242, 242,
242, 240, 240, 240, 237, 237, 237, 233, 233, 233, 228, 228, 228, 221, 221, 221, 213, 213, 213,
203, 203, 203, 191, 191, 191, 177, 177, 177, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 77, 77, 77, 93, 93, 93, 109, 109, 109, 126, 126, 126,
143, 143, 143, 159, 159, 159, 174, 174, 174, 188, 188, 188, 200, 200, 200, 210, 210, 210, 218,
218, 218, 225, 225, 225, 230, 230, 230, 234, 234, 234, 237, 237, 237, 239, 239, 239, 241, 241,
241, 242, 242, 242, 242, 242, 242, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243,
243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243, 243,
243, 243, 242, 242, 242, 242, 242, 242, 241, 241, 241, 239, 239, 239, 237, 237, 237, 234, 234,
234, 230, 230, 230, 225, 225, 225, 218, 218, 218, 210, 210, 210, 200, 200, 200, 188, 188, 188,
174, 174, 174, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 72, 72, 72, 90, 90, 90, 106, 106, 106, 123, 123, 123, 139, 139, 139, 155, 155, 155,
170, 170, 170, 183, 183, 183, 195, 195, 195, 205, 205, 205, 213, 213, 213, 220, 220, 220, 225,
225, 225, 229, 229, 229, 232, 232, 232, 234, 234, 234, 236, 236, 236, 236, 236, 236, 237, 237,
237, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238,
238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 238, 237, 237, 237, 237,
237, 237, 236, 236, 236, 234, 234, 234, 232, 232, 232, 229, 229, 229, 225, 225, 225, 220, 220,
220, 213, 213, 213, 205, 205, 205, 195, 195, 195, 183, 183, 183, 163, 163, 163, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 57, 57, 57, 87,
87, 87, 102, 102, 102, 118, 118, 118, 134, 134, 134, 150, 150, 150, 164, 164, 164, 177, 177,
177, 189, 189, 189, 198, 198, 198, 207, 207, 207, 213, 213, 213, 218, 218, 218, 222, 222, 222,
225, 225, 225, 227, 227, 227, 229, 229, 229, 229, 229, 229, 230, 230, 230, 231, 231, 231, 231,
231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 231,
231, 231, 231, 231, 231, 231, 231, 231, 231, 231, 230, 230, 230, 230, 230, 230, 229, 229, 229,
227, 227, 227, 225, 225, 225, 222, 222, 222, 218, 218, 218, 213, 213, 213, 207, 207, 207, 198,
198, 198, 189, 189, 189, 177, 177, 177, 129, 129, 129, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 30, 30, 30, 82, 82, 82, 97, 97, 97, 113,
113, 113, 128, 128, 128, 143, 143, 143, 157, 157, 157, 170, 170, 170, 181, 181, 181, 191, 191,
191, 199, 199, 199, 205, 205, 205, 210, 210, 210, 214, 214, 214, 217, 217, 217, 219, 219, 219,
220, 220, 220, 221, 221, 221, 222, 222, 222, 222, 222, 222, 223, 223, 223, 223, 223, 223, 223,
223, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223, 223,
223, 223, 223, 223, 222, 222, 222, 221, 221, 221, 221, 221, 221, 219, 219, 219, 217, 217, 217,
214, 214, 214, 210, 210, 210, 205, 205, 205, 199, 199, 199, 191, 191, 191, 181, 181, 181, 170,
170, 170, 71, 71, 71, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 61, 61, 61, 92, 92, 92, 106, 106, 106, 121, 121, 121, 135, 135,
135, 149, 149, 149, 161, 161, 161, 172, 172, 172, 181, 181, 181, 189, 189, 189, 195, 195, 195,
200, 200, 200, 204, 204, 204, 207, 207, 207, 209, 209, 209, 210, 210, 210, 211, 211, 211, 212,
212, 212, 212, 212, 212, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213,
213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 213, 212, 212, 212,
211, 211, 211, 210, 210, 210, 209, 209, 209, 207, 207, 207, 204, 204, 204, 200, 200, 200, 195,
195, 195, 189, 189, 189, 181, 181, 181, 172, 172, 172, 126, 126, 126, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 8, 8,
67, 67, 67, 99, 99, 99, 113, 113, 113, 126, 126, 126, 139, 139, 139, 151, 151, 151, 161, 161,
161, 170, 170, 170, 177, 177, 177, 184, 184, 184, 188, 188, 188, 192, 192, 192, 195, 195, 195,
197, 197, 197, 198, 198, 198, 199, 199, 199, 200, 200, 200, 200, 200, 200, 201, 201, 201, 201,
201, 201, 201, 201, 201, 201, 201, 201, 201, 201, 201, 201, 201, 201, 201, 201, 201, 201, 201,
201, 201, 201, 201, 200, 200, 200, 200, 200, 200, 199, 199, 199, 198, 198, 198, 197, 197, 197,
195, 195, 195, 192, 192, 192, 189, 189, 189, 184, 184, 184, 178, 178, 178, 170, 170, 170, 126,
126, 126, 18, 18, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 41, 41, 41, 82, 82, 82, 111, 111, 111,
128, 128, 128, 139, 139, 139, 149, 149, 149, 157, 157, 157, 164, 164, 164, 170, 170, 170, 175,
175, 175, 178, 178, 178, 181, 181, 181, 183, 183, 183, 184, 184, 184, 185, 185, 185, 186, 186,
186, 186, 186, 186, 187, 187, 187, 187, 187, 187, 187, 187, 187, 187, 187, 187, 187, 187, 187,
187, 187, 187, 187, 187, 187, 187, 187, 187, 187, 187, 187, 186, 186, 186, 186, 186, 186, 185,
185, 185, 184, 184, 184, 183, 183, 183, 181, 181, 181, 178, 178, 178, 175, 175, 175, 163, 163,
163, 129, 129, 129, 71, 71, 71, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0,
];
struct FilmStrip {
width: u32,
height: u32,
strip: Option<&'static [u8]>,
}
pub fn film_strip_filter(video_frame: &mut VideoFrame) {
let FilmStrip {
width,
height,
strip,
} = determine_film_strip(video_frame.width);
if let Some(strip) = strip {
let mut frame_index = 0;
let mut film_hole_index = 0;
let offset = ((video_frame.width * 3) - 3) as usize;
for i in 0..(video_frame.height as usize) {
for j in (0..(width as usize * 3)).step_by(3) {
let current_stripe_index = film_hole_index + j;
video_frame.data[frame_index + j] = strip[current_stripe_index];
video_frame.data[frame_index + j + 1] = strip[current_stripe_index + 1];
video_frame.data[frame_index + j + 2] = strip[current_stripe_index + 2];
video_frame.data[frame_index + offset - j] = strip[current_stripe_index];
video_frame.data[frame_index + offset - j + 1] = strip[current_stripe_index + 1];
video_frame.data[frame_index + offset - j + 2] = strip[current_stripe_index + 2];
}
frame_index += video_frame.line_size as usize;
film_hole_index = (i % height as usize) * width as usize * 3;
}
}
}
fn determine_film_strip(video_width: u32) -> FilmStrip {
match video_width {
// We consider that the smallest film strip is 4, doubling it for each side, we have 8 pixels
0..=8 => FilmStrip {
width: 0,
height: 0,
strip: None,
},
9..=96 => FilmStrip {
width: 4,
height: 4,
strip: Some(&FILM_STRIP_4),
},
97..=192 => FilmStrip {
width: 8,
height: 8,
strip: Some(&FILM_STRIP_8),
},
193..=384 => FilmStrip {
width: 16,
height: 16,
strip: Some(&FILM_STRIP_16),
},
385..=768 => FilmStrip {
width: 32,
height: 32,
strip: Some(&FILM_STRIP_32),
},
_ => FilmStrip {
width: 64,
height: 64,
strip: Some(&FILM_STRIP_64),
},
}
}

256
crates/ffmpeg/src/filter_graph.rs Normal file
View file

@ -0,0 +1,256 @@
use std::{
ffi::{CStr, CString},
ptr,
};
use crate::{
codec_ctx::FFmpegCodecContext, error::FFmpegError, frame_decoder::ThumbnailSize,
utils::check_error, Error,
};
use ffmpeg_sys_next::{
avfilter_get_by_name, avfilter_graph_alloc, avfilter_graph_config,
avfilter_graph_create_filter, avfilter_graph_free, avfilter_link, AVFilterContext,
AVFilterGraph, AVRational,
};
pub struct FFmpegFilterGraph(*mut AVFilterGraph);
impl<'a> FFmpegFilterGraph {
pub(crate) fn new() -> Result<Self, FFmpegError> {
let ptr = unsafe { avfilter_graph_alloc() };
if ptr.is_null() {
return Err(FFmpegError::FrameAllocation);
}
Ok(Self(ptr))
}
fn link(
src: *mut AVFilterContext,
src_pad: u32,
dst: *mut AVFilterContext,
dst_pad: u32,
error: &str,
) -> Result<(), Error> {
check_error(unsafe { avfilter_link(src, src_pad, dst, dst_pad) }, error)?;
Ok(())
}
pub(crate) fn thumbnail_graph(
size: Option<ThumbnailSize>,
time_base: &AVRational,
codec_ctx: &FFmpegCodecContext,
interlaced_frame: bool,
pixel_aspect_ratio: AVRational,
maintain_aspect_ratio: bool,
) -> Result<(Self, &'a mut AVFilterContext, &'a mut AVFilterContext), Error> {
let mut filter_graph = Self::new()?;
let args = format!(
"video_size={}x{}:pix_fmt={}:time_base={}/{}:pixel_aspect={}/{}",
codec_ctx.as_ref().width,
codec_ctx.as_ref().height,
// AVPixelFormat is an i32 enum, so it's safe to cast it to i32
codec_ctx.as_ref().pix_fmt as i32,
time_base.num,
time_base.den,
codec_ctx.as_ref().sample_aspect_ratio.num,
i32::max(codec_ctx.as_ref().sample_aspect_ratio.den, 1)
);
let mut filter_source = ptr::null_mut();
filter_graph.setup_filter(
&mut filter_source,
c"buffer",
c"thumb_buffer",
Some(CString::new(args)?.as_c_str()),
"Failed to create filter source",
)?;
let filter_source_ctx = unsafe { filter_source.as_mut() }.ok_or(FFmpegError::NullError)?;
let mut filter_sink = ptr::null_mut();
filter_graph.setup_filter(
&mut filter_sink,
c"buffersink",
c"thumb_buffersink",
None,
"Failed to create filter sink",
)?;
let filter_sink_ctx = unsafe { filter_sink.as_mut() }.ok_or(FFmpegError::NullError)?;
let mut yadif_filter = ptr::null_mut();
if interlaced_frame {
filter_graph.setup_filter(
&mut yadif_filter,
c"yadif",
c"thumb_deint",
Some(c"deint=1"),
"Failed to create de-interlace filter",
)?;
}
let mut scale_filter = ptr::null_mut();
filter_graph.setup_filter(
&mut scale_filter,
c"scale",
c"thumb_scale",
Some(
CString::new(thumb_scale_filter_args(
size,
codec_ctx,
pixel_aspect_ratio,
maintain_aspect_ratio,
))?
.as_c_str(),
),
"Failed to create scale filter",
)?;
let mut format_filter = ptr::null_mut();
filter_graph.setup_filter(
&mut format_filter,
c"format",
c"thumb_format",
Some(c"pix_fmts=rgb24"),
"Failed to create format filter",
)?;
Self::link(
format_filter,
0,
filter_sink_ctx,
0,
"Failed to link final filter",
)?;
Self::link(
scale_filter,
0,
format_filter,
0,
"Failed to link scale filter",
)?;
if !yadif_filter.is_null() {
Self::link(
yadif_filter,
0,
scale_filter,
0,
"Failed to link yadif filter",
)?;
}
Self::link(
filter_source_ctx,
0,
if yadif_filter.is_null() {
scale_filter
} else {
yadif_filter
},
0,
"Failed to link source filter",
)?;
filter_graph.config()?;
Ok((filter_graph, filter_source_ctx, filter_sink_ctx))
}
pub(crate) fn as_mut(&mut self) -> &mut AVFilterGraph {
unsafe { self.0.as_mut() }.expect("initialized on struct creation")
}
fn setup_filter(
&mut self,
filter_ctx: *mut *mut AVFilterContext,
filter_name: &CStr,
filter_setup_name: &CStr,
args: Option<&CStr>,
error_message: &str,
) -> Result<(), Error> {
check_error(
unsafe {
avfilter_graph_create_filter(
filter_ctx,
avfilter_get_by_name(filter_name.as_ptr()),
filter_setup_name.as_ptr(),
args.map_or(ptr::null(), CStr::as_ptr),
ptr::null_mut(),
self.as_mut(),
)
},
error_message,
)
}
fn config(&mut self) -> Result<&mut Self, Error> {
check_error(
unsafe { avfilter_graph_config(self.as_mut(), ptr::null_mut()) },
"Failed to configure filter graph",
)?;
Ok(self)
}
}
impl Drop for FFmpegFilterGraph {
fn drop(&mut self) {
if !self.0.is_null() {
unsafe { avfilter_graph_free(&mut self.0) };
self.0 = ptr::null_mut();
}
}
}
fn thumb_scale_filter_args(
size: Option<ThumbnailSize>,
codec_ctx: &FFmpegCodecContext,
pixel_aspect_ratio: AVRational,
maintain_aspect_ratio: bool,
) -> String {
let (width, height) = match size {
Some(ThumbnailSize::Dimensions { width, height }) => (width, Some(height)),
Some(ThumbnailSize::Scale(width)) => (width, None),
None => return "w=0:h=0".to_string(),
};
let mut scale = String::new();
if let Some(height) = height {
scale.push_str(&format!("w={width}:h={height}"));
if maintain_aspect_ratio {
scale.push_str(":force_original_aspect_ratio=decrease");
}
} else if !maintain_aspect_ratio {
scale.push_str(&format!("w={width}:h={width}"));
} else {
let size = width;
let mut width = codec_ctx.as_ref().width.unsigned_abs();
let mut height = codec_ctx.as_ref().height.unsigned_abs();
// if the pixel aspect ratio is defined and is not 1, we have an anamorphic stream
if pixel_aspect_ratio.num != 0 && pixel_aspect_ratio.num != pixel_aspect_ratio.den {
width = (width * pixel_aspect_ratio.num.unsigned_abs())
/ pixel_aspect_ratio.den.unsigned_abs();
if size != 0 {
if height > width {
width = (width * size) / height;
height = size;
} else {
height = (height * size) / width;
width = size;
}
}
scale.push_str(&format!("w={width}:h={height}"));
} else if height > width {
scale.push_str(&format!("w=-1:h={}", if size == 0 { height } else { size }));
} else {
scale.push_str(&format!("h=-1:w={}", if size == 0 { width } else { size }));
}
}
scale
}

427
crates/ffmpeg/src/format_ctx.rs Normal file
View file

@ -0,0 +1,427 @@
use crate::{
codec_ctx::FFmpegCodecContext,
dict::FFmpegDictionary,
error::{Error, FFmpegError},
model::{FFmpegChapter, FFmpegMediaData, FFmpegMetadata, FFmpegProgram, FFmpegStream},
utils::check_error,
};
use ffmpeg_sys_next::{
av_cmp_q, av_display_rotation_get, av_read_frame, av_reduce, av_stream_get_side_data,
avformat_close_input, avformat_find_stream_info, avformat_open_input, AVChapter, AVCodecID,
AVDictionary, AVFormatContext, AVMediaType, AVPacket, AVPacketSideDataType, AVRational,
AVStream, AV_DISPOSITION_ATTACHED_PIC, AV_DISPOSITION_CAPTIONS, AV_DISPOSITION_CLEAN_EFFECTS,
AV_DISPOSITION_COMMENT, AV_DISPOSITION_DEFAULT, AV_DISPOSITION_DEPENDENT,
AV_DISPOSITION_DESCRIPTIONS, AV_DISPOSITION_DUB, AV_DISPOSITION_FORCED,
AV_DISPOSITION_HEARING_IMPAIRED, AV_DISPOSITION_KARAOKE, AV_DISPOSITION_LYRICS,
AV_DISPOSITION_METADATA, AV_DISPOSITION_NON_DIEGETIC, AV_DISPOSITION_ORIGINAL,
AV_DISPOSITION_STILL_IMAGE, AV_DISPOSITION_TIMED_THUMBNAILS, AV_DISPOSITION_VISUAL_IMPAIRED,
AV_NOPTS_VALUE,
};
use std::{collections::HashSet, ffi::CStr, ptr};
fn extract_name_and_convert_metadata(
metadata: *mut AVDictionary,
) -> (FFmpegMetadata, Option<String>) {
let mut metadata = FFmpegDictionary::new(unsafe { metadata.as_mut() });
let name = metadata.get(c"name");
if name.is_some() {
let _ = metadata.remove(c"name");
}
(metadata.into(), name)
}
#[derive(Debug)]
pub struct FFmpegFormatContext(*mut AVFormatContext);
impl FFmpegFormatContext {
pub(crate) fn open_file(filename: &CStr) -> Result<Self, Error> {
let mut ptr = ptr::null_mut();
check_error(
unsafe {
avformat_open_input(&mut ptr, filename.as_ptr(), ptr::null(), ptr::null_mut())
},
"Fail to open an input stream and read the header",
)
.map(|()| Self(ptr))
}
pub(crate) fn as_ref(&self) -> &AVFormatContext {
unsafe { self.0.as_ref() }.expect("initialized on struct creation")
}
pub(crate) fn as_mut(&mut self) -> &mut AVFormatContext {
unsafe { self.0.as_mut() }.expect("initialized on struct creation")
}
pub(crate) fn duration(&self) -> Option<i64> {
let duration = self.as_ref().duration;
if duration == AV_NOPTS_VALUE {
return None;
}
Some(duration)
}
pub(crate) fn stream(&self, index: u32) -> Option<&mut AVStream> {
let streams = self.as_ref().streams;
if streams.is_null() {
return None;
}
let Ok(index) = isize::try_from(index) else {
return None;
};
unsafe { (*(streams.offset(index))).as_mut() }
}
pub(crate) fn get_stream_rotation_angle(&self, index: u32) -> f64 {
let Some(stream) = self.stream(index) else {
return 0.0;
};
/*
* This side data contains a 3x3 transformation matrix describing an affine transformation
* that needs to be applied to the decoded video frames for correct presentation.
*
* See libavutil/display.h for a detailed description of the data.
* https://github.com/FFmpeg/FFmpeg/blob/n6.1.1/libavutil/display.h#L32-L71
*
* The pointer conversion is due to the fact that av_stream_get_side_data is a generic function that has no prior
* knowledge of the type of the side data it is retrieving.
*/
#[allow(clippy::cast_ptr_alignment)]
let matrix = (unsafe {
av_stream_get_side_data(
stream,
AVPacketSideDataType::AV_PKT_DATA_DISPLAYMATRIX,
ptr::null_mut(),
)
} as *const i32);
if matrix.is_null() {
0.0
} else {
unsafe { av_display_rotation_get(matrix) }
}
}
pub(crate) fn read_frame(&mut self, packet: *mut AVPacket) -> Result<&mut Self, Error> {
check_error(
unsafe { av_read_frame(self.as_mut(), packet) },
"Fail to read the next frame of a media file",
)?;
Ok(self)
}
pub(crate) fn find_stream_info(&mut self) -> Result<&mut Self, Error> {
check_error(
unsafe { avformat_find_stream_info(self.as_mut(), ptr::null_mut()) },
"Fail to read packets of a media file to get stream information",
)?;
Ok(self)
}
pub(crate) fn find_preferred_video_stream(
&self,
prefer_embedded_metadata: bool,
) -> Result<(bool, &mut AVStream), Error> {
let mut video_streams = vec![];
let mut embedded_data_streams = vec![];
'outer: for stream_idx in 0..self.as_ref().nb_streams {
let Some(stream) = self.stream(stream_idx) else {
continue;
};
let Some((codec_type, codec_id)) = unsafe { stream.codecpar.as_ref() }
.map(|codec_params| (codec_params.codec_type, codec_params.codec_id))
else {
continue;
};
if codec_type != AVMediaType::AVMEDIA_TYPE_VIDEO {
continue;
}
if !prefer_embedded_metadata
|| !(codec_id == AVCodecID::AV_CODEC_ID_MJPEG
|| codec_id == AVCodecID::AV_CODEC_ID_PNG)
{
video_streams.push(stream_idx);
continue;
}
if let Some(metadata) = unsafe { stream.metadata.as_mut() }
.map(|metadata| FFmpegDictionary::new(Some(metadata)))
{
for (key, value) in &metadata {
if let Some(value) = value {
if key == "filename" && value.starts_with("cover.") {
embedded_data_streams.insert(0, stream_idx);
continue 'outer;
}
}
}
}
embedded_data_streams.push(stream_idx);
}
if prefer_embedded_metadata && !embedded_data_streams.is_empty() {
for stream_index in embedded_data_streams {
if let Some(stream) = self.stream(stream_index) {
return Ok((true, stream));
}
}
}
for stream_index in video_streams {
if let Some(stream) = self.stream(stream_index) {
return Ok((false, stream));
}
}
Err(FFmpegError::StreamNotFound)?
}
fn formats(&self) -> Vec<String> {
unsafe { self.as_ref().iformat.as_ref() }
.and_then(|format| unsafe { format.name.as_ref() })
.map(|name| {
let cstr = unsafe { CStr::from_ptr(name) };
String::from_utf8_lossy(cstr.to_bytes())
.split(',')
.map(|entry| entry.trim().to_string())
.filter(|entry| !entry.is_empty())
.collect()
})
.unwrap_or(vec![])
}
fn start_time(&self) -> Option<i64> {
let start_time = self.as_ref().start_time;
if start_time == AV_NOPTS_VALUE {
return None;
}
Some(start_time)
}
fn bit_rate(&self) -> i64 {
self.as_ref().bit_rate
}
fn chapters(&self) -> Vec<FFmpegChapter> {
let chapters_ptr = self.as_ref().chapters;
(!chapters_ptr.is_null())
.then(|| {
(0..isize::try_from(self.as_ref().nb_chapters).unwrap_or(0))
.filter_map(|id| unsafe { (*(chapters_ptr.offset(id))).as_ref() })
.map(Into::into)
.collect()
})
.unwrap_or(vec![])
}
fn programs(&self) -> Vec<FFmpegProgram> {
let mut visited_streams: HashSet<u32> = HashSet::new();
let programs_ptr = self.as_ref().programs;
let mut programs = (!programs_ptr.is_null())
.then(|| {
(0..isize::try_from(self.as_ref().nb_programs).unwrap_or(0))
.filter_map(|id| unsafe { (*(programs_ptr.offset(id))).as_ref() })
.map(|program| {
let (metadata, name) = extract_name_and_convert_metadata(program.metadata);
let streams = (0..isize::try_from(program.nb_stream_indexes).unwrap_or(0))
.filter_map(|index| unsafe {
program.stream_index.offset(index).as_ref()
})
.copied()
.filter_map(|stream_index| {
visited_streams.insert(stream_index);
self.stream(stream_index)
})
.map(|stream| (&*stream).into())
.collect::<Vec<FFmpegStream>>();
FFmpegProgram {
id: program.id,
name,
streams,
metadata,
}
})
.collect::<Vec<FFmpegProgram>>()
})
.unwrap_or(vec![]);
let unvisited_streams = (0..self.as_ref().nb_streams)
.filter(|i| !visited_streams.contains(i))
.filter_map(|i| self.stream(i).map(|stream| (&*stream).into()))
.collect::<Vec<FFmpegStream>>();
if !unvisited_streams.is_empty() {
if let Ok(id) = i32::try_from(programs.len()) {
// Create an empty program to hold unvisited streams if there are any
programs.push(FFmpegProgram {
id,
name: Some("No Program".to_string()),
streams: unvisited_streams,
metadata: FFmpegMetadata::default(),
});
}
}
programs
}
fn metadata(&self) -> FFmpegMetadata {
let fmt_ctx = self.as_ref();
unsafe { fmt_ctx.metadata.as_mut() }.map_or_else(FFmpegMetadata::default, |metadata| {
FFmpegDictionary::new(Some(metadata)).into()
})
}
}
impl Drop for FFmpegFormatContext {
fn drop(&mut self) {
if !self.0.is_null() {
unsafe { avformat_close_input(&mut self.0) };
self.0 = ptr::null_mut();
}
}
}
impl From<&FFmpegFormatContext> for FFmpegMediaData {
fn from(ctx: &FFmpegFormatContext) -> Self {
Self {
formats: ctx.formats(),
duration: ctx.duration(),
start_time: ctx.start_time(),
bit_rate: ctx.bit_rate(),
chapters: ctx.chapters(),
programs: ctx.programs(),
metadata: ctx.metadata(),
}
}
}
impl From<&AVChapter> for FFmpegChapter {
fn from(
AVChapter {
id,
time_base,
start,
end,
metadata,
}: &AVChapter,
) -> Self {
Self {
// NOTICE: chapter.id is a i64, but I think it will be extremely rare to have a chapter id that doesn't fit in a i32
id: *id,
start: *start,
end: *end,
time_base_num: time_base.num,
time_base_den: time_base.den,
metadata: unsafe { metadata.as_mut() }
.map_or_else(FFmpegMetadata::default, |metadata| {
FFmpegDictionary::new(Some(metadata)).into()
}),
}
}
}
impl From<&AVStream> for FFmpegStream {
fn from(stream: &AVStream) -> Self {
let (metadata, name) = extract_name_and_convert_metadata(stream.metadata);
let aspect_ratio = unsafe { stream.codecpar.as_ref() }
.and_then(|codecpar| {
if stream.sample_aspect_ratio.num != 0
&& unsafe { av_cmp_q(stream.sample_aspect_ratio, codecpar.sample_aspect_ratio) }
!= 0
{
let mut display_aspect_ratio = AVRational { num: 0, den: 0 };
let num = i64::from(codecpar.width * codecpar.sample_aspect_ratio.num);
let den = i64::from(codecpar.height * codecpar.sample_aspect_ratio.den);
let max = 1024 * 1024;
unsafe {
av_reduce(
&mut display_aspect_ratio.num,
&mut display_aspect_ratio.den,
num,
den,
max,
);
}
Some(display_aspect_ratio)
} else {
None
}
})
.unwrap_or(stream.sample_aspect_ratio);
let dispositions = [
(AV_DISPOSITION_DEFAULT, "default"),
(AV_DISPOSITION_DUB, "dub"),
(AV_DISPOSITION_ORIGINAL, "original"),
(AV_DISPOSITION_COMMENT, "comment"),
(AV_DISPOSITION_LYRICS, "lyrics"),
(AV_DISPOSITION_KARAOKE, "karaoke"),
(AV_DISPOSITION_FORCED, "forced"),
(AV_DISPOSITION_HEARING_IMPAIRED, "hearing impaired"),
(AV_DISPOSITION_VISUAL_IMPAIRED, "visual impaired"),
(AV_DISPOSITION_CLEAN_EFFECTS, "clean effects"),
(AV_DISPOSITION_ATTACHED_PIC, "attached pic"),
(AV_DISPOSITION_TIMED_THUMBNAILS, "timed thumbnails"),
(AV_DISPOSITION_CAPTIONS, "captions"),
(AV_DISPOSITION_DESCRIPTIONS, "descriptions"),
(AV_DISPOSITION_METADATA, "metadata"),
(AV_DISPOSITION_DEPENDENT, "dependent"),
(AV_DISPOSITION_STILL_IMAGE, "still image"),
(AV_DISPOSITION_NON_DIEGETIC, "non-diegetic"),
]
.iter()
.filter_map(|&(flag, name)| {
if stream.disposition & flag != 0 {
Some(name.to_string())
} else {
None
}
})
.collect::<Vec<String>>();
let codec = unsafe { stream.codecpar.as_ref() }.and_then(|codec_params| {
FFmpegCodecContext::new()
.and_then(|mut codec| {
codec.parameters_to_context(codec_params)?;
Ok(codec)
})
.map(|codec| (&codec).into())
.ok()
});
Self {
id: stream.index,
name,
codec,
aspect_ratio_num: aspect_ratio.num,
aspect_ratio_den: aspect_ratio.den,
frames_per_second_num: stream.avg_frame_rate.num,
frames_per_second_den: stream.avg_frame_rate.den,
time_base_real_num: stream.time_base.num,
time_base_real_den: stream.time_base.den,
dispositions,
metadata,
}
}
}

290
crates/ffmpeg/src/frame_decoder.rs Normal file
View file

@ -0,0 +1,290 @@
use crate::{
codec_ctx::FFmpegCodecContext,
error::{Error, FFmpegError},
filter_graph::FFmpegFilterGraph,
format_ctx::FFmpegFormatContext,
utils::{check_error, from_path},
video_frame::FFmpegFrame,
};
use std::{path::Path, ptr};
use ffmpeg_sys_next::{
av_buffersink_get_frame, av_buffersrc_write_frame, av_frame_alloc,
av_guess_sample_aspect_ratio, av_packet_alloc, av_packet_free, av_packet_unref, av_seek_frame,
avcodec_find_decoder, AVPacket, AVRational, AVStream, AVERROR, AVPROBE_SCORE_MAX,
AV_FRAME_FLAG_INTERLACED, AV_FRAME_FLAG_KEY, AV_TIME_BASE, EAGAIN,
};
#[derive(Debug, Clone, Copy)]
pub enum ThumbnailSize {
Scale(u32),
Dimensions { width: u32, height: u32 },
}
#[derive(Debug)]
pub struct VideoFrame {
pub data: Vec<u8>,
pub width: u32,
pub height: u32,
pub rotation: f64,
}
pub struct FrameDecoder {
format_ctx: FFmpegFormatContext,
preferred_stream_id: u32,
codec_ctx: FFmpegCodecContext,
frame: FFmpegFrame,
packet: *mut AVPacket,
embedded: bool,
allow_seek: bool,
}
impl FrameDecoder {
pub(crate) fn new(
filename: impl AsRef<Path>,
allow_seek: bool,
prefer_embedded: bool,
) -> Result<Self, Error> {
let filename = filename.as_ref();
let mut format_context = FFmpegFormatContext::open_file(from_path(filename)?.as_c_str())?;
format_context.find_stream_info()?;
// This needs to remain at 100 or the app will force crash if it comes
// across a video with subtitles or any type of corruption.
if format_context.as_ref().probe_score != AVPROBE_SCORE_MAX {
return Err(Error::CorruptVideo(
filename.to_path_buf().into_boxed_path(),
));
}
let (embedded, video_stream) =
format_context.find_preferred_video_stream(prefer_embedded)?;
let preferred_stream_id = u32::try_from(video_stream.index)?;
let video_codec = unsafe { video_stream.codecpar.as_ref() }
.and_then(|codecpar| unsafe { avcodec_find_decoder(codecpar.codec_id).as_ref() })
.ok_or(FFmpegError::DecoderNotFound)?;
let mut video_codec_context = FFmpegCodecContext::new()?;
video_codec_context.parameters_to_context(
unsafe { video_stream.codecpar.as_ref() }.ok_or(FFmpegError::NullError)?,
)?;
video_codec_context.as_mut().workaround_bugs = 1;
video_codec_context.open2(video_codec)?;
let frame = unsafe { av_frame_alloc() };
if frame.is_null() {
Err(FFmpegError::FrameAllocation)?;
}
Ok(Self {
format_ctx: format_context,
preferred_stream_id,
codec_ctx: video_codec_context,
frame: FFmpegFrame::new()?,
packet: ptr::null_mut(),
allow_seek,
embedded,
})
}
pub(crate) fn use_embedded(&mut self) -> bool {
self.embedded
}
pub(crate) fn decode_video_frame(&mut self) -> Result<(), Error> {
let mut frame_finished = false;
while !frame_finished && self.find_packet_for_stream() {
frame_finished = self.decode_packet()?;
}
if !frame_finished {
return Err(Error::FrameDecodeError);
}
Ok(())
}
pub(crate) fn seek(&mut self, seconds: i64) -> Result<(), Error> {
if !self.allow_seek {
return Ok(());
}
let timestamp = i64::from(AV_TIME_BASE).checked_mul(seconds).unwrap_or(0);
check_error(
unsafe { av_seek_frame(self.format_ctx.as_mut(), -1, timestamp, 0) },
"Seeking video failed",
)?;
self.codec_ctx.flush();
let mut got_frame = false;
for _ in 0..200 {
got_frame = false;
let mut count = 0;
while !got_frame && count < 20 {
self.find_packet_for_stream();
got_frame = self.decode_packet().unwrap_or(false);
count += 1;
}
if got_frame && self.frame.as_ref().flags & AV_FRAME_FLAG_KEY != 0 {
break;
}
}
if got_frame {
Ok(())
} else {
Err(Error::SeekError)
}
}
pub(crate) fn get_scaled_video_frame(
&mut self,
size: Option<ThumbnailSize>,
maintain_aspect_ratio: bool,
) -> Result<VideoFrame, Error> {
let (time_base, stream_ptr) = self
.format_ctx
.stream(self.preferred_stream_id)
.map(|stream| -> (AVRational, *mut AVStream) { (stream.time_base, stream) })
.ok_or(FFmpegError::NullError)?;
let pixel_aspect_ratio = unsafe {
av_guess_sample_aspect_ratio(self.format_ctx.as_mut(), stream_ptr, self.frame.as_mut())
};
let (_guard, filter_source, filter_sink) = FFmpegFilterGraph::thumbnail_graph(
size,
&time_base,
&self.codec_ctx,
(self.frame.as_mut().flags & AV_FRAME_FLAG_INTERLACED) != 0,
pixel_aspect_ratio,
maintain_aspect_ratio,
)?;
let mut new_frame = FFmpegFrame::new()?;
let mut get_frame_errno = 0;
for _ in 0..10 {
check_error(
unsafe { av_buffersrc_write_frame(filter_source, self.frame.as_ref()) },
"Failed to write frame to filter graph",
)?;
get_frame_errno = unsafe { av_buffersink_get_frame(filter_sink, new_frame.as_mut()) };
if get_frame_errno != AVERROR(EAGAIN) {
break;
}
self.decode_video_frame()?;
}
check_error(get_frame_errno, "Failed to get buffer from filter")?;
let width = new_frame.as_ref().width.unsigned_abs();
let height = new_frame.as_ref().height.unsigned_abs();
let line_size = usize::try_from(new_frame.as_ref().linesize[0])?;
let mut data = Vec::with_capacity(line_size * usize::try_from(height)?);
data.extend_from_slice(unsafe {
std::slice::from_raw_parts(new_frame.as_ref().data[0], data.capacity())
});
Ok(VideoFrame {
data,
width,
height,
rotation: self
.format_ctx
.get_stream_rotation_angle(self.preferred_stream_id)
.round(),
})
}
pub fn get_duration_secs(&self) -> Option<f64> {
self.format_ctx.duration().map(|duration| {
let av_time_base = i64::from(AV_TIME_BASE);
#[allow(clippy::cast_precision_loss)]
{
// SAFETY: the duration would need to be humongous for this cast to f64 to cause problems
(duration / av_time_base) as f64
+ ((duration % av_time_base) as f64 / f64::from(AV_TIME_BASE))
}
})
}
fn reset_packet(&mut self) {
if self.packet.is_null() {
self.packet = unsafe { av_packet_alloc() };
} else {
unsafe { av_packet_unref(self.packet) }
}
}
fn is_packet_for_stream(&self) -> Option<&mut AVPacket> {
let packet = (unsafe { self.packet.as_mut() })?;
let packet_stream_id = u32::try_from(packet.stream_index).ok()?;
if packet_stream_id == self.preferred_stream_id {
Some(packet)
} else {
None
}
}
fn find_packet_for_stream(&mut self) -> bool {
self.reset_packet();
while self.format_ctx.read_frame(self.packet).is_ok() {
if self.is_packet_for_stream().is_some() {
return true;
}
self.reset_packet();
}
false
}
fn decode_packet(&mut self) -> Result<bool, Error> {
let Some(packet) = self.is_packet_for_stream() else {
return Ok(false);
};
if match self.codec_ctx.send_packet(packet) {
Ok(b) => b,
Err(FFmpegError::Again) => true,
Err(e) => {
return Err(Error::FFmpegWithReason(
e,
"Failed to send packet to decoder".to_string(),
))
}
} {
match self.codec_ctx.receive_frame(self.frame.as_mut()) {
Ok(ok) => Ok(ok),
Err(FFmpegError::Again) => Ok(false),
Err(e) => Err(Error::FFmpegWithReason(
e,
"Failed to receive frame from decoder".to_string(),
)),
}
} else {
Ok(false)
}
}
}
impl Drop for FrameDecoder {
fn drop(&mut self) {
unsafe {
av_packet_free(&mut self.packet);
}
}
}

105
crates/ffmpeg/src/lib.rs
View file

@ -1,30 +1,95 @@
use crate::{
film_strip::film_strip_filter,
movie_decoder::{MovieDecoder, ThumbnailSize},
video_frame::VideoFrame,
};
#![warn(
clippy::all,
clippy::pedantic,
clippy::correctness,
clippy::perf,
clippy::style,
clippy::suspicious,
clippy::complexity,
clippy::nursery,
clippy::unwrap_used,
unused_qualifications,
rust_2018_idioms,
trivial_casts,
trivial_numeric_casts,
unused_allocation,
clippy::unnecessary_cast,
clippy::cast_lossless,
clippy::cast_possible_truncation,
clippy::cast_possible_wrap,
clippy::cast_precision_loss,
clippy::cast_sign_loss,
clippy::dbg_macro,
clippy::deprecated_cfg_attr,
clippy::separated_literal_suffix,
deprecated
)]
#![forbid(deprecated_in_future)]
#![allow(clippy::missing_errors_doc, clippy::module_name_repetitions)]
use crate::{format_ctx::FFmpegFormatContext, frame_decoder::FrameDecoder, utils::from_path};
use std::path::Path;
use ffmpeg_sys_next::{av_log_set_level, AV_LOG_FATAL};
mod codec_ctx;
mod dict;
mod error;
mod film_strip;
mod movie_decoder;
mod filter_graph;
mod format_ctx;
mod frame_decoder;
pub mod model;
mod thumbnailer;
mod utils;
mod video_frame;
pub use error::Error;
pub use thumbnailer::{Thumbnailer, ThumbnailerBuilder};
pub use frame_decoder::ThumbnailSize;
pub use model::FFmpegMediaData;
pub use thumbnailer::ThumbnailerBuilder;
use tokio::task::spawn_blocking;
/// Helper function to generate retrieve media data from from a video/audio file
pub async fn probe(filename: impl AsRef<Path> + Send) -> Result<FFmpegMediaData, Error> {
// Reduce the amount of logs generated by FFmpeg
unsafe { av_log_set_level(AV_LOG_FATAL) };
// Dictionary to store format options
// let mut format_opts = FFmpegDict::new(None);
// Some MPEGTS specific option (copied from ffprobe)
// let scan_all_pmts = c"scan_all_pmts";
// format_opts.set(scan_all_pmts, c"1")?;
// Open an input stream, read the header and allocate the format context
spawn_blocking({
let filename = filename.as_ref().to_path_buf();
move || {
let mut fmt_ctx = FFmpegFormatContext::open_file(from_path(filename)?.as_c_str())?;
// // Reset MPEGTS specific option
// format_opts.remove(scan_all_pmts)?;
// Read packets of media file to get stream information.
fmt_ctx.find_stream_info()?;
Ok((&fmt_ctx).into())
}
})
.await?
}
/// Helper function to generate a thumbnail file from a video file with reasonable defaults
pub async fn to_thumbnail(
video_file_path: impl AsRef<Path>,
output_thumbnail_path: impl AsRef<Path>,
size: u32,
video_file_path: impl AsRef<Path> + Send,
output_thumbnail_path: impl AsRef<Path> + Send,
size: ThumbnailSize,
quality: f32,
) -> Result<(), Error> {
// Reduce the amount of logs generated by FFmpeg
unsafe { av_log_set_level(AV_LOG_FATAL) };
ThumbnailerBuilder::new()
.with_film_strip(false)
.size(size)
.quality(quality)?
.build()
@ -32,20 +97,6 @@ pub async fn to_thumbnail(
.await
}
/// Helper function to generate a thumbnail bytes from a video file with reasonable defaults
pub async fn to_webp_bytes(
video_file_path: impl AsRef<Path>,
size: u32,
quality: f32,
) -> Result<Vec<u8>, Error> {
ThumbnailerBuilder::new()
.size(size)
.quality(quality)?
.build()
.process_to_webp_bytes(video_file_path)
.await
}
#[cfg(test)]
mod tests {
use super::*;
@ -93,7 +144,7 @@ mod tests {
];
for (input, output) in video_file_path.iter().zip(actual_webp_files.iter()) {
if let Err(e) = to_thumbnail(input, output, 128, 100.0).await {
if let Err(e) = to_thumbnail(input, output, ThumbnailSize::Scale(128), 100.0).await {
eprintln!("Error: {e}; Input: {}", input.display());
panic!("{}", e);
}

124
crates/ffmpeg/src/model.rs Normal file
View file

@ -0,0 +1,124 @@
use std::collections::HashMap;
use chrono::{DateTime, Utc};
#[derive(Debug)]
pub struct FFmpegMediaData {
pub formats: Vec<String>,
pub duration: Option<i64>,
pub start_time: Option<i64>,
pub bit_rate: i64,
pub chapters: Vec<FFmpegChapter>,
pub programs: Vec<FFmpegProgram>,
pub metadata: FFmpegMetadata,
}
#[derive(Debug)]
pub struct FFmpegChapter {
pub id: i64,
pub start: i64,
pub end: i64,
pub time_base_den: i32,
pub time_base_num: i32,
pub metadata: FFmpegMetadata,
}
#[derive(Default, Debug)]
pub struct FFmpegMetadata {
pub album: Option<String>,
pub album_artist: Option<String>,
pub artist: Option<String>,
pub comment: Option<String>,
pub composer: Option<String>,
pub copyright: Option<String>,
pub creation_time: Option<DateTime<Utc>>,
pub date: Option<DateTime<Utc>>,
pub disc: Option<u32>,
pub encoder: Option<String>,
pub encoded_by: Option<String>,
pub filename: Option<String>,
pub genre: Option<String>,
pub language: Option<String>,
pub performer: Option<String>,
pub publisher: Option<String>,
pub service_name: Option<String>,
pub service_provider: Option<String>,
pub title: Option<String>,
pub track: Option<u32>,
pub variant_bit_rate: Option<u32>,
pub custom: HashMap<String, String>,
}
#[derive(Debug)]
pub struct FFmpegProgram {
pub id: i32,
pub name: Option<String>,
pub streams: Vec<FFmpegStream>,
pub metadata: FFmpegMetadata,
}
#[derive(Debug)]
pub struct FFmpegStream {
pub id: i32,
pub name: Option<String>,
pub codec: Option<FFmpegCodec>,
pub aspect_ratio_num: i32,
pub aspect_ratio_den: i32,
pub frames_per_second_num: i32,
pub frames_per_second_den: i32,
pub time_base_real_den: i32,
pub time_base_real_num: i32,
pub dispositions: Vec<String>,
pub metadata: FFmpegMetadata,
}
#[derive(Debug)]
pub struct FFmpegCodec {
pub kind: Option<String>,
pub sub_kind: Option<String>,
pub tag: Option<String>,
pub name: Option<String>,
pub profile: Option<String>,
pub bit_rate: i32,
pub props: Option<FFmpegProps>,
}
#[derive(Debug)]
pub enum FFmpegProps {
Video(FFmpegVideoProps),
Audio(FFmpegAudioProps),
Subtitle(FFmpegSubtitleProps),
}
#[derive(Debug)]
pub struct FFmpegVideoProps {
pub pixel_format: Option<String>,
pub color_range: Option<String>,
pub bits_per_channel: Option<i32>,
pub color_space: Option<String>,
pub color_primaries: Option<String>,
pub color_transfer: Option<String>,
pub field_order: Option<String>,
pub chroma_location: Option<String>,
pub width: i32,
pub height: i32,
pub aspect_ratio_num: Option<i32>,
pub aspect_ratio_den: Option<i32>,
pub properties: Vec<String>,
}
#[derive(Debug)]
pub struct FFmpegAudioProps {
pub delay: i32,
pub padding: i32,
pub sample_rate: Option<i32>,
pub sample_format: Option<String>,
pub bit_per_sample: Option<i32>,
pub channel_layout: Option<String>,
}
#[derive(Debug)]
pub struct FFmpegSubtitleProps {
pub width: i32,
pub height: i32,
}

750
crates/ffmpeg/src/movie_decoder.rs
View file

@ -1,750 +0,0 @@
use crate::{
error::{Error, FfmpegError},
utils::from_path,
video_frame::{FfmpegFrame, FrameSource, VideoFrame},
};
use ffmpeg_sys_next::{
av_buffersink_get_frame, av_buffersrc_write_frame, av_dict_get, av_display_rotation_get,
av_frame_alloc, av_frame_free, av_packet_alloc, av_packet_free, av_packet_unref, av_read_frame,
av_seek_frame, av_stream_get_side_data, avcodec_alloc_context3, avcodec_find_decoder,
avcodec_flush_buffers, avcodec_free_context, avcodec_open2, avcodec_parameters_to_context,
avcodec_receive_frame, avcodec_send_packet, avfilter_get_by_name, avfilter_graph_alloc,
avfilter_graph_config, avfilter_graph_create_filter, avfilter_graph_free, avfilter_link,
avformat_close_input, avformat_find_stream_info, avformat_open_input, AVCodec, AVCodecContext,
AVCodecID, AVFilterContext, AVFilterGraph, AVFormatContext, AVFrame, AVMediaType, AVPacket,
AVPacketSideDataType, AVRational, AVStream, AVERROR, AVERROR_EOF, AVPROBE_SCORE_MAX,
AV_DICT_IGNORE_SUFFIX, AV_TIME_BASE, EAGAIN,
};
use std::{
ffi::{CStr, CString},
fmt::Write,
path::Path,
time::Duration,
};
#[derive(Debug, Clone, Copy)]
pub enum ThumbnailSize {
Dimensions { width: u32, height: u32 },
Size(u32),
}
pub struct MovieDecoder {
video_stream_index: i32,
format_context: *mut AVFormatContext,
video_codec_context: *mut AVCodecContext,
video_codec: *const AVCodec,
filter_graph: *mut AVFilterGraph,
filter_source: *mut AVFilterContext,
filter_sink: *mut AVFilterContext,
video_stream: *mut AVStream,
frame: *mut AVFrame,
packet: *mut AVPacket,
allow_seek: bool,
use_embedded_data: bool,
}
impl MovieDecoder {
pub(crate) fn new(
filename: impl AsRef<Path>,
prefer_embedded_metadata: bool,
) -> Result<Self, Error> {
let filename = filename.as_ref();
let input_file = if filename == Path::new("-") {
Path::new("pipe:")
} else {
filename
};
let allow_seek = filename != Path::new("-")
&& !filename.starts_with("rsts://")
&& !filename.starts_with("udp://");
let mut decoder = Self {
video_stream_index: -1,
format_context: std::ptr::null_mut(),
video_codec_context: std::ptr::null_mut(),
video_codec: std::ptr::null_mut(),
filter_graph: std::ptr::null_mut(),
filter_source: std::ptr::null_mut(),
filter_sink: std::ptr::null_mut(),
video_stream: std::ptr::null_mut(),
frame: std::ptr::null_mut(),
packet: std::ptr::null_mut(),
allow_seek,
use_embedded_data: false,
};
unsafe {
let input_file_cstring = from_path(input_file)?;
match avformat_open_input(
&mut decoder.format_context,
input_file_cstring.as_ptr(),
std::ptr::null_mut(),
std::ptr::null_mut(),
) {
0 => {
check_error(
avformat_find_stream_info(decoder.format_context, std::ptr::null_mut()),
"Failed to get stream info",
)?;
}
e => {
return Err(Error::FfmpegWithReason(
FfmpegError::from(e),
"Failed to open input".to_string(),
))
}
}
}
unsafe {
// This needs to remain at 100 or the app will force crash if it comes
// across a video with subtitles or any type of corruption.
if (*decoder.format_context).probe_score != AVPROBE_SCORE_MAX {
return Err(Error::CorruptVideo);
}
}
decoder.initialize_video(prefer_embedded_metadata)?;
decoder.frame = unsafe { av_frame_alloc() };
if decoder.frame.is_null() {
return Err(FfmpegError::FrameAllocation.into());
}
Ok(decoder)
}
pub(crate) fn decode_video_frame(&mut self) -> Result<(), Error> {
let mut frame_finished = false;
while !frame_finished && self.get_video_packet() {
frame_finished = self.decode_video_packet()?;
}
if !frame_finished {
return Err(Error::FrameDecodeError);
}
Ok(())
}
pub(crate) const fn embedded_metadata_is_available(&self) -> bool {
self.use_embedded_data
}
pub(crate) fn seek(&mut self, seconds: i64) -> Result<(), Error> {
if !self.allow_seek {
return Ok(());
}
let timestamp = i64::from(AV_TIME_BASE).checked_mul(seconds).unwrap_or(0);
check_error(
unsafe { av_seek_frame(self.format_context, -1, timestamp, 0) },
"Seeking video failed",
)?;
unsafe { avcodec_flush_buffers(self.video_codec_context) };
let mut key_frame_attempts = 0;
let mut got_frame;
loop {
let mut count = 0;
got_frame = false;
while !got_frame && count < 20 {
self.get_video_packet();
got_frame = self.decode_video_packet().unwrap_or(false);
count += 1;
}
key_frame_attempts += 1;
if !((!got_frame || unsafe { (*self.frame).key_frame } == 0)
&& key_frame_attempts < 200)
{
break;
}
}
if !got_frame {
return Err(Error::SeekError);
}
Ok(())
}
pub(crate) fn get_scaled_video_frame(
&mut self,
scaled_size: Option<ThumbnailSize>,
maintain_aspect_ratio: bool,
video_frame: &mut VideoFrame,
) -> Result<(), Error> {
self.initialize_filter_graph(
unsafe {
&(*(*(*self.format_context)
.streams
.offset(self.video_stream_index as isize)))
.time_base
},
scaled_size,
maintain_aspect_ratio,
)?;
check_error(
unsafe { av_buffersrc_write_frame(self.filter_source, self.frame) },
"Failed to write frame to filter graph",
)?;
let mut new_frame = FfmpegFrame::new()?;
let mut attempts = 0;
let mut ret = unsafe { av_buffersink_get_frame(self.filter_sink, new_frame.as_mut_ptr()) };
while ret == AVERROR(EAGAIN) && attempts < 10 {
self.decode_video_frame()?;
check_error(
unsafe { av_buffersrc_write_frame(self.filter_source, self.frame) },
"Failed to write frame to filter graph",
)?;
ret = unsafe { av_buffersink_get_frame(self.filter_sink, new_frame.as_mut_ptr()) };
attempts += 1;
}
if ret < 0 {
return Err(Error::FfmpegWithReason(
FfmpegError::from(ret),
"Failed to get buffer from filter".to_string(),
));
}
// SAFETY: these should always be positive, so clippy doesn't need to alert on them
#[allow(clippy::cast_sign_loss)]
{
video_frame.width = unsafe { (*new_frame.as_mut_ptr()).width as u32 };
video_frame.height = unsafe { (*new_frame.as_mut_ptr()).height as u32 };
video_frame.line_size = unsafe { (*new_frame.as_mut_ptr()).linesize[0] as u32 };
}
video_frame.source = if self.use_embedded_data {
Some(FrameSource::Metadata)
} else {
Some(FrameSource::VideoStream)
};
let frame_data_size = video_frame.line_size as usize * video_frame.height as usize;
match video_frame.data.capacity() {
0 => {
video_frame.data = Vec::with_capacity(frame_data_size);
}
c if c < frame_data_size => {
video_frame.data.reserve_exact(frame_data_size - c);
video_frame.data.clear();
}
c if c > frame_data_size => {
video_frame.data.shrink_to(frame_data_size);
video_frame.data.clear();
}
_ => {
video_frame.data.clear();
}
}
video_frame.data.extend_from_slice(unsafe {
std::slice::from_raw_parts((*new_frame.as_mut_ptr()).data[0], frame_data_size)
});
if !self.filter_graph.is_null() {
unsafe { avfilter_graph_free(&mut self.filter_graph) };
self.filter_graph = std::ptr::null_mut();
}
Ok(())
}
// SAFETY: this should always be positive, so clippy doesn't need to alert on them
#[allow(clippy::cast_sign_loss)]
pub fn get_video_duration(&self) -> Duration {
Duration::from_secs(unsafe { (*self.format_context).duration as u64 / AV_TIME_BASE as u64 })
}
fn initialize_video(&mut self, prefer_embedded_metadata: bool) -> Result<(), Error> {
self.find_preferred_video_stream(prefer_embedded_metadata)?;
self.video_stream = unsafe {
*(*self.format_context)
.streams
.offset(self.video_stream_index as isize)
};
self.video_codec =
unsafe { avcodec_find_decoder((*(*self.video_stream).codecpar).codec_id) };
if self.video_codec.is_null() {
return Err(FfmpegError::DecoderNotFound.into());
}
self.video_codec_context = unsafe { avcodec_alloc_context3(self.video_codec) };
if self.video_codec_context.is_null() {
return Err(FfmpegError::VideoCodecAllocation.into());
}
check_error(
unsafe {
avcodec_parameters_to_context(
self.video_codec_context,
(*self.video_stream).codecpar,
)
},
"Failed to get parameters from context",
)?;
unsafe { (*self.video_codec_context).workaround_bugs = 1 };
check_error(
unsafe {
avcodec_open2(
self.video_codec_context,
self.video_codec,
std::ptr::null_mut(),
)
},
"Failed to open video codec",
)
}
fn find_preferred_video_stream(&mut self, prefer_embedded_metadata: bool) -> Result<(), Error> {
let mut video_streams = vec![];
let mut embedded_data_streams = vec![];
let empty_cstring = CString::new("").unwrap();
for stream_idx in 0..(unsafe { (*self.format_context).nb_streams.try_into()? }) {
let stream = unsafe { *(*self.format_context).streams.offset(stream_idx as isize) };
let codec_params = unsafe { (*stream).codecpar };
if unsafe { (*codec_params).codec_type } == AVMediaType::AVMEDIA_TYPE_VIDEO {
let codec_id = unsafe { (*codec_params).codec_id };
if !prefer_embedded_metadata
|| !(codec_id == AVCodecID::AV_CODEC_ID_MJPEG
|| codec_id == AVCodecID::AV_CODEC_ID_PNG)
{
video_streams.push(stream_idx);
continue;
}
if unsafe { !(*stream).metadata.is_null() } {
let mut tag = std::ptr::null_mut();
loop {
tag = unsafe {
av_dict_get(
(*stream).metadata,
empty_cstring.as_ptr(),
tag,
AV_DICT_IGNORE_SUFFIX,
)
};
if tag.is_null() {
break;
}
// WARNING: NEVER use CString with foreign raw pointer (causes double-free)
let key = unsafe { CStr::from_ptr((*tag).key) }.to_str();
if let Ok(key) = key {
let value = unsafe { CStr::from_ptr((*tag).value) }.to_str();
if let Ok(value) = value {
if key == "filename" && value == "cover." {
embedded_data_streams.insert(0, stream_idx);
continue;
}
}
}
}
}
embedded_data_streams.push(stream_idx);
}
}
self.use_embedded_data = false;
if prefer_embedded_metadata && !embedded_data_streams.is_empty() {
self.use_embedded_data = true;
self.video_stream_index = embedded_data_streams[0];
Ok(())
} else if !video_streams.is_empty() {
self.video_stream_index = video_streams[0];
Ok(())
} else {
Err(FfmpegError::StreamNotFound.into())
}
}
fn get_video_packet(&mut self) -> bool {
let mut frames_available = true;
let mut frame_decoded = false;
if !self.packet.is_null() {
unsafe {
av_packet_unref(self.packet);
av_packet_free(&mut self.packet);
}
}
self.packet = unsafe { av_packet_alloc() };
while frames_available && !frame_decoded {
frames_available = unsafe { av_read_frame(self.format_context, self.packet) >= 0 };
if frames_available {
frame_decoded = unsafe { (*self.packet).stream_index } == self.video_stream_index;
if !frame_decoded {
unsafe { av_packet_unref(self.packet) };
}
}
}
frame_decoded
}
fn decode_video_packet(&self) -> Result<bool, Error> {
if unsafe { (*self.packet).stream_index } != self.video_stream_index {
return Ok(false);
}
let ret = unsafe { avcodec_send_packet(self.video_codec_context, self.packet) };
if ret != AVERROR(EAGAIN) {
if ret == AVERROR_EOF {
return Ok(false);
} else if ret < 0 {
return Err(Error::FfmpegWithReason(
FfmpegError::from(ret),
"Failed to send packet to decoder".to_string(),
));
}
}
match unsafe { avcodec_receive_frame(self.video_codec_context, self.frame) } {
0 => Ok(true),
e if e != AVERROR(EAGAIN) => Err(Error::FfmpegWithReason(
FfmpegError::from(e),
"Failed to receive frame from decoder".to_string(),
)),
_ => Ok(false),
}
}
#[allow(clippy::too_many_lines)]
fn initialize_filter_graph(
&mut self,
timebase: &AVRational,
scaled_size: Option<ThumbnailSize>,
maintain_aspect_ratio: bool,
) -> Result<(), Error> {
unsafe { self.filter_graph = avfilter_graph_alloc() };
if self.filter_graph.is_null() {
return Err(FfmpegError::FilterGraphAllocation.into());
}
let args = unsafe {
format!(
"video_size={}x{}:pix_fmt={}:time_base={}/{}:pixel_aspect={}/{}",
(*self.video_codec_context).width,
(*self.video_codec_context).height,
(*self.video_codec_context).pix_fmt as i32,
timebase.num,
timebase.den,
(*self.video_codec_context).sample_aspect_ratio.num,
i32::max((*self.video_codec_context).sample_aspect_ratio.den, 1)
)
};
setup_filter(
&mut self.filter_source,
"buffer",
"thumb_buffer",
&args,
self.filter_graph,
"Failed to create filter source",
)?;
setup_filter_without_args(
&mut self.filter_sink,
"buffersink",
"thumb_buffersink",
self.filter_graph,
"Failed to create filter sink",
)?;
let mut yadif_filter = std::ptr::null_mut();
if unsafe { (*self.frame).interlaced_frame } != 0 {
setup_filter(
&mut yadif_filter,
"yadif",
"thumb_deint",
"deint=1",
self.filter_graph,
"Failed to create de-interlace filter",
)?;
}
let mut scale_filter = std::ptr::null_mut();
setup_filter(
&mut scale_filter,
"scale",
"thumb_scale",
&Self::create_scale_string(scaled_size, maintain_aspect_ratio),
self.filter_graph,
"Failed to create scale filter",
)?;
let mut format_filter = std::ptr::null_mut();
setup_filter(
&mut format_filter,
"format",
"thumb_format",
"pix_fmts=rgb24",
self.filter_graph,
"Failed to create format filter",
)?;
let mut rotate_filter = std::ptr::null_mut();
let rotation = self.get_stream_rotation();
if rotation == 3 {
setup_filter(
&mut rotate_filter,
"rotate",
"thumb_rotate",
"PI",
self.filter_graph,
"Failed to create rotate filter",
)?;
} else if rotation != -1 {
setup_filter(
&mut rotate_filter,
"transpose",
"thumb_transpose",
&rotation.to_string(),
self.filter_graph,
"Failed to create transpose filter",
)?;
}
check_error(
unsafe {
avfilter_link(
if rotate_filter.is_null() {
format_filter
} else {
rotate_filter
},
0,
self.filter_sink,
0,
)
},
"Failed to link final filter",
)?;
if !rotate_filter.is_null() {
check_error(
unsafe { avfilter_link(format_filter, 0, rotate_filter, 0) },
"Failed to link format filter",
)?;
}
check_error(
unsafe { avfilter_link(scale_filter, 0, format_filter, 0) },
"Failed to link scale filter",
)?;
if !yadif_filter.is_null() {
check_error(
unsafe { avfilter_link(yadif_filter, 0, scale_filter, 0) },
"Failed to link yadif filter",
)?;
}
check_error(
unsafe {
avfilter_link(
self.filter_source,
0,
if yadif_filter.is_null() {
scale_filter
} else {
yadif_filter
},
0,
)
},
"Failed to link source filter",
)?;
check_error(
unsafe { avfilter_graph_config(self.filter_graph, std::ptr::null_mut()) },
"Failed to configure filter graph",
)?;
Ok(())
}
fn create_scale_string(size: Option<ThumbnailSize>, maintain_aspect_ratio: bool) -> String {
let mut scaled_width;
let mut scaled_height = -1;
if size.is_none() {
return "w=0:h=0".to_string();
}
let size = size.expect("Size should have been checked for None");
#[allow(clippy::cast_possible_wrap)]
match size {
ThumbnailSize::Dimensions { width, height } => {
scaled_width = width as i32;
scaled_height = height as i32;
}
ThumbnailSize::Size(width) => {
scaled_width = width as i32;
}
}
if scaled_width <= 0 {
scaled_width = -1;
}
if scaled_height <= 0 {
scaled_height = -1;
}
let mut scale = String::new();
write!(scale, "w={scaled_width}:h={scaled_height}")
.expect("Write of const string should work");
if maintain_aspect_ratio {
write!(scale, ":force_original_aspect_ratio=decrease")
.expect("Write of const string should work");
}
// TODO: Handle anamorphic videos
scale
}
#[allow(clippy::cast_ptr_alignment)]
fn get_stream_rotation(&self) -> i32 {
let matrix = unsafe {
av_stream_get_side_data(
self.video_stream,
AVPacketSideDataType::AV_PKT_DATA_DISPLAYMATRIX,
std::ptr::null_mut(),
)
} as *const i32;
if !matrix.is_null() {
let angle = (unsafe { av_display_rotation_get(matrix) }).round();
if angle < -135.0 {
return 3;
} else if angle > 45.0 && angle < 135.0 {
return 2;
} else if angle < -45.0 && angle > -135.0 {
return 1;
}
}
-1
}
}
impl Drop for MovieDecoder {
fn drop(&mut self) {
if !self.video_codec_context.is_null() {
unsafe {
avcodec_free_context(&mut self.video_codec_context);
}
self.video_codec = std::ptr::null_mut();
}
if !self.format_context.is_null() {
unsafe {
avformat_close_input(&mut self.format_context);
}
self.format_context = std::ptr::null_mut();
}
if !self.packet.is_null() {
unsafe {
av_packet_unref(self.packet);
av_packet_free(&mut self.packet);
}
self.packet = std::ptr::null_mut();
}
if !self.frame.is_null() {
unsafe {
av_frame_free(&mut self.frame);
self.frame = std::ptr::null_mut();
}
self.frame = std::ptr::null_mut();
}
self.video_stream_index = -1;
}
}
fn check_error(return_code: i32, error_message: &str) -> Result<(), Error> {
if return_code < 0 {
Err(Error::FfmpegWithReason(
FfmpegError::from(return_code),
error_message.to_string(),
))
} else {
Ok(())
}
}
fn setup_filter(
filter_ctx: *mut *mut AVFilterContext,
filter_name: &str,
filter_setup_name: &str,
args: &str,
graph_ctx: *mut AVFilterGraph,
error_message: &str,
) -> Result<(), Error> {
let filter_name_cstr = CString::new(filter_name).expect("CString from str");
let filter_setup_name_cstr = CString::new(filter_setup_name).expect("CString from str");
let args_cstr = CString::new(args).expect("CString from str");
check_error(
unsafe {
avfilter_graph_create_filter(
filter_ctx,
avfilter_get_by_name(filter_name_cstr.as_ptr()),
filter_setup_name_cstr.as_ptr(),
args_cstr.as_ptr(),
std::ptr::null_mut(),
graph_ctx,
)
},
error_message,
)
}
fn setup_filter_without_args(
filter_ctx: *mut *mut AVFilterContext,
filter_name: &str,
filter_setup_name: &str,
graph_ctx: *mut AVFilterGraph,
error_message: &str,
) -> Result<(), Error> {
let filter_name_cstr = CString::new(filter_name).unwrap();
let filter_setup_name_cstr = CString::new(filter_setup_name).unwrap();
check_error(
unsafe {
avfilter_graph_create_filter(
filter_ctx,
avfilter_get_by_name(filter_name_cstr.as_ptr()),
filter_setup_name_cstr.as_ptr(),
std::ptr::null_mut(),
std::ptr::null_mut(),
graph_ctx,
)
},
error_message,
)
}

153
crates/ffmpeg/src/thumbnailer.rs
View file

@ -1,6 +1,9 @@
use crate::{film_strip_filter, Error, MovieDecoder, ThumbnailSize, VideoFrame};
use crate::{frame_decoder::ThumbnailSize, Error, FrameDecoder};
use std::{io, ops::Deref, path::Path};
use image::{imageops, DynamicImage, RgbImage};
use sd_utils::error::FileIOError;
use tokio::{fs, task::spawn_blocking};
use tracing::error;
use webp::Encoder;
@ -14,79 +17,114 @@ pub struct Thumbnailer {
impl Thumbnailer {
/// Processes an video input file and write to file system a thumbnail with webp format
pub async fn process(
pub(crate) async fn process(
&self,
video_file_path: impl AsRef<Path>,
output_thumbnail_path: impl AsRef<Path>,
video_file_path: impl AsRef<Path> + Send,
output_thumbnail_path: impl AsRef<Path> + Send,
) -> Result<(), Error> {
let path = output_thumbnail_path.as_ref().parent().ok_or_else(|| {
io::Error::new(
io::ErrorKind::InvalidInput,
"Cannot determine parent directory",
)
let output_thumbnail_path = output_thumbnail_path.as_ref();
let path = output_thumbnail_path.parent().ok_or_else(|| {
FileIOError::from((
output_thumbnail_path,
io::Error::new(
io::ErrorKind::InvalidInput,
"Cannot determine parent directory",
),
))
})?;
fs::create_dir_all(path).await?;
fs::create_dir_all(path)
.await
.map_err(|e| FileIOError::from((path, e)))?;
fs::write(
output_thumbnail_path,
&*self.process_to_webp_bytes(video_file_path).await?,
)
.await
.map_err(Into::into)
.map_err(|e| FileIOError::from((output_thumbnail_path, e)).into())
}
/// Processes an video input file and returns a webp encoded thumbnail as bytes
pub async fn process_to_webp_bytes(
async fn process_to_webp_bytes(
&self,
video_file_path: impl AsRef<Path>,
video_file_path: impl AsRef<Path> + Send,
) -> Result<Vec<u8>, Error> {
let video_file_path = video_file_path.as_ref().to_path_buf();
let prefer_embedded_metadata = self.builder.prefer_embedded_metadata;
let seek_percentage = self.builder.seek_percentage;
let size = self.builder.size;
let maintain_aspect_ratio = self.builder.maintain_aspect_ratio;
let with_film_strip = self.builder.with_film_strip;
let quality = self.builder.quality;
spawn_blocking(move || -> Result<Vec<u8>, Error> {
let mut decoder = MovieDecoder::new(video_file_path.clone(), prefer_embedded_metadata)?;
// We actually have to decode a frame to get some metadata before we can start decoding for real
decoder.decode_video_frame()?;
spawn_blocking({
let video_file_path = video_file_path.as_ref().to_path_buf();
move || -> Result<Vec<u8>, Error> {
let mut decoder = FrameDecoder::new(
&video_file_path,
// TODO: allow_seek should be false for remote files
true,
prefer_embedded_metadata,
)?;
#[allow(clippy::cast_possible_truncation)]
#[allow(clippy::cast_precision_loss)]
if !decoder.embedded_metadata_is_available() {
let result = decoder.seek(
(decoder.get_video_duration().as_secs() as f64 * f64::from(seek_percentage))
.round() as i64,
// We actually have to decode a frame to get some metadata before we can start decoding for real
decoder.decode_video_frame()?;
if !decoder.use_embedded() {
let result = decoder
.get_duration_secs()
.ok_or(Error::NoVideoDuration)
.and_then(|duration| {
decoder.seek(
#[allow(clippy::cast_possible_truncation)]
{
// This conversion is ok because we don't worry much about precision here
(duration * f64::from(seek_percentage)).round() as i64
},
)
});
if let Err(err) = result {
error!(
"Failed to seek {}: {err:#?}",
video_file_path.to_string_lossy()
);
// Seeking failed, try first frame again
// Re-instantiating decoder to avoid possible segfault
// https://github.com/dirkvdb/ffmpegthumbnailer/commit/da292ccb51a526ebc833f851a388ca308d747289
decoder =
FrameDecoder::new(&video_file_path, false, prefer_embedded_metadata)?;
decoder.decode_video_frame()?;
}
}
let video_frame =
decoder.get_scaled_video_frame(Some(size), maintain_aspect_ratio)?;
let mut image = DynamicImage::ImageRgb8(
RgbImage::from_raw(video_frame.width, video_frame.height, video_frame.data)
.ok_or(Error::CorruptVideo(video_file_path.into_boxed_path()))?,
);
if let Err(err) = result {
error!("Failed to seek: {err:#?}");
// seeking failed, try the first frame again
decoder = MovieDecoder::new(video_file_path, prefer_embedded_metadata)?;
decoder.decode_video_frame()?;
}
}
let image = if video_frame.rotation < -135.0 {
imageops::rotate180_in_place(&mut image);
image
} else if video_frame.rotation > 45.0 && video_frame.rotation < 135.0 {
image.rotate270()
} else if video_frame.rotation < -45.0 && video_frame.rotation > -135.0 {
image.rotate90()
} else {
image
};
let mut video_frame = VideoFrame::default();
decoder.get_scaled_video_frame(Some(size), maintain_aspect_ratio, &mut video_frame)?;
if with_film_strip {
film_strip_filter(&mut video_frame);
}
// Type WebPMemory is !Send, which makes the Future in this function !Send,
// this make us `deref` to have a `&[u8]` and then `to_owned` to make a Vec<u8>
// which implies on a unwanted clone...
Ok(
Encoder::from_rgb(&video_frame.data, video_frame.width, video_frame.height)
// Type WebPMemory is !Send, which makes the Future in this function !Send,
// this make us `deref` to have a `&[u8]` and then `to_owned` to make a Vec<u8>
// which implies on a unwanted clone...
Ok(Encoder::from_image(&image)
.expect("Should not fail as the underlining DynamicImage is an RgbImage")
.encode(quality)
.deref()
.to_vec(),
)
.to_vec())
}
})
.await?
}
@ -102,18 +140,16 @@ pub struct ThumbnailerBuilder {
seek_percentage: f32,
quality: f32,
prefer_embedded_metadata: bool,
with_film_strip: bool,
}
impl Default for ThumbnailerBuilder {
fn default() -> Self {
Self {
maintain_aspect_ratio: true,
size: ThumbnailSize::Size(128),
size: ThumbnailSize::Scale(128),
seek_percentage: 0.1,
quality: 80.0,
prefer_embedded_metadata: true,
with_film_strip: true,
}
}
}
@ -125,7 +161,6 @@ impl ThumbnailerBuilder {
/// - `seek_percentage`: 10%
/// - `quality`: 80
/// - `prefer_embedded_metadata`: true
/// - `with_film_strip`: true
pub fn new() -> Self {
Self::default()
}
@ -137,14 +172,8 @@ impl ThumbnailerBuilder {
}
/// To set a thumbnail size, respecting or not its aspect ratio, according to `maintain_aspect_ratio` value
pub const fn size(mut self, size: u32) -> Self {
self.size = ThumbnailSize::Size(size);
self
}
/// To specify width and height of the thumbnail
pub const fn width_and_height(mut self, width: u32, height: u32) -> Self {
self.size = ThumbnailSize::Dimensions { width, height };
pub const fn size(mut self, size: ThumbnailSize) -> Self {
self.size = size;
self
}
@ -173,12 +202,6 @@ impl ThumbnailerBuilder {
self
}
/// If `with_film_strip` is true, a film strip will be added to the thumbnail borders
pub const fn with_film_strip(mut self, with_film_strip: bool) -> Self {
self.with_film_strip = with_film_strip;
self
}
/// Builds a `Thumbnailer` struct
#[must_use]
pub const fn build(self) -> Thumbnailer {

13
crates/ffmpeg/src/utils.rs
View file

@ -1,4 +1,4 @@
use crate::error::Error;
use crate::error::{Error, FFmpegError};
use std::ffi::CString;
use std::path::Path;
@ -19,3 +19,14 @@ pub fn from_path(path: impl AsRef<Path>) -> Result<CString, Error> {
.ok_or(Error::PathConversion(path.to_path_buf()))
}
}
pub fn check_error(return_code: i32, error_message: &str) -> Result<(), Error> {
if return_code < 0 {
Err(Error::FFmpegWithReason(
FFmpegError::from(return_code),
error_message.to_string(),
))
} else {
Ok(())
}
}

49
crates/ffmpeg/src/video_frame.rs
View file

@ -1,44 +1,31 @@
use crate::error::FfmpegError;
use crate::error::FFmpegError;
use ffmpeg_sys_next::{av_frame_alloc, av_frame_free, AVFrame};
#[derive(Debug)]
pub enum FrameSource {
VideoStream,
Metadata,
}
pub struct FFmpegFrame(*mut AVFrame);
#[derive(Debug, Default)]
pub struct VideoFrame {
pub width: u32,
pub height: u32,
pub line_size: u32,
pub data: Vec<u8>,
pub source: Option<FrameSource>,
}
pub struct FfmpegFrame {
data: *mut AVFrame,
}
impl FfmpegFrame {
pub fn new() -> Result<Self, FfmpegError> {
let data = unsafe { av_frame_alloc() };
if data.is_null() {
return Err(FfmpegError::FrameAllocation);
impl FFmpegFrame {
pub(crate) fn new() -> Result<Self, FFmpegError> {
let ptr = unsafe { av_frame_alloc() };
if ptr.is_null() {
return Err(FFmpegError::FrameAllocation);
}
Ok(Self { data })
Ok(Self(ptr))
}
pub fn as_mut_ptr(&mut self) -> *mut AVFrame {
self.data
pub(crate) fn as_ref(&self) -> &AVFrame {
unsafe { self.0.as_ref() }.expect("initialized on struct creation")
}
pub(crate) fn as_mut(&mut self) -> &mut AVFrame {
unsafe { self.0.as_mut() }.expect("initialized on struct creation")
}
}
impl Drop for FfmpegFrame {
impl Drop for FFmpegFrame {
fn drop(&mut self) {
if !self.data.is_null() {
unsafe { av_frame_free(&mut self.data) };
self.data = std::ptr::null_mut();
if !self.0.is_null() {
unsafe { av_frame_free(&mut self.0) };
self.0 = std::ptr::null_mut();
}
}
}

2
crates/file-ext/src/extensions.rs
View file

@ -99,7 +99,7 @@ extension_category_enum! {
// audio extensions
extension_category_enum! {
AudioExtension _ALL_AUDIO_EXTENSIONS {
AudioExtension ALL_AUDIO_EXTENSIONS {
Mp3 = [0x49, 0x44, 0x33],
Mp2 = [0xFF, 0xFB] | [0xFF, 0xFD],
M4a = [0x66, 0x74, 0x79, 0x70, 0x4D, 0x34, 0x41, 0x20] + 4,

13
crates/media-metadata/Cargo.toml
View file

@ -1,10 +1,20 @@
[package]
name = "sd-media-metadata"
version = "0.0.0"
authors = ["Jake Robinson <jake@spacedrive.com>"]
authors = [
"Jake Robinson <jake@spacedrive.com>",
"Vítor Vasconcellos <vitor@spacedrive.com>",
"Ericson Soares <ericson@spacedrive.com>",
]
edition = "2021"
[features]
ffmpeg = ["dep:sd-ffmpeg"]
[dependencies]
sd-ffmpeg = { path = "../ffmpeg", optional = true }
sd-utils = { path = "../utils" }
chrono = { workspace = true, features = ["serde"] }
image = { workspace = true }
rand = { workspace = true }
@ -13,6 +23,7 @@ serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
specta = { workspace = true, features = ["chrono"] }
thiserror = { workspace = true }
tokio = { workspace = true }
kamadak-exif = "0.5.5"

19
crates/media-metadata/src/audio.rs
View file

@ -1,19 +0,0 @@
use std::path::Path;
use crate::Result;
#[derive(
Default, Clone, PartialEq, Eq, Debug, serde::Serialize, serde::Deserialize, specta::Type,
)]
pub struct AudioMetadata {
duration: Option<i32>, // can't use `Duration` due to bigint
audio_codec: Option<String>,
}
impl AudioMetadata {
#[allow(clippy::missing_errors_doc)]
#[allow(clippy::missing_panics_doc)]
pub fn from_path(_path: impl AsRef<Path>) -> Result<Self> {
todo!()
}
}

26
crates/media-metadata/src/error.rs
View file

@ -1,31 +1,29 @@
use std::{
num::ParseFloatError,
path::{Path, PathBuf},
};
use sd_utils::error::FileIOError;
#[derive(Debug, thiserror::Error)]
pub enum Error {
#[error("there was an i/o error {0} at {}", .1.display())]
Io(std::io::Error, Box<Path>),
#[error("error from the exif crate: {0}")]
Exif(#[from] exif::Error),
#[cfg(feature = "ffmpeg")]
#[error("error from the ffmpeg crate: {0}")]
FFmpeg(#[from] sd_ffmpeg::Error),
#[cfg(not(feature = "ffmpeg"))]
#[error("ffmpeg not available")]
NoFFmpeg,
#[error("there was an error while parsing time with chrono: {0}")]
Chrono(#[from] chrono::ParseError),
#[error("there was an error while converting between types")]
Conversion,
#[error("there was an error while parsing the location of an image")]
MediaLocationParse,
#[error("there was an error while parsing a float")]
FloatParse(#[from] ParseFloatError),
#[error("there was an error while initializing the exif reader")]
Init,
#[error("the file provided at ({0}) contains no exif data")]
NoExifDataOnPath(PathBuf),
#[error("the slice provided contains no exif data")]
NoExifDataOnSlice,
#[error("serde error {0}")]
Serde(#[from] serde_json::Error),
#[error("failed to join tokio task: {0}")]
TokioJoinHandle(#[from] tokio::task::JoinError),
#[error(transparent)]
FileIO(#[from] FileIOError),
}
pub type Result<T> = std::result::Result<T, Error>;

0
crates/media-metadata/src/image/composite.rs → crates/media-metadata/src/exif/composite.rs
View file

14
crates/media-metadata/src/image/consts.rs → crates/media-metadata/src/exif/consts.rs
View file

@ -7,10 +7,10 @@ use exif::Tag;
/// ```
/// use sd_media_metadata::image::DMS_DIVISION;
///
/// let latitude = [53_f64, 19_f64, 35.11_f64]; // in DMS
/// let latitude = [53.0, 19.0, 35.11]; // in DMS
/// latitude.iter().zip(DMS_DIVISION.iter());
/// ```
pub const DMS_DIVISION: [f64; 3] = [1_f64, 60_f64, 3600_f64];
pub const DMS_DIVISION: [f64; 3] = [1.0, 60.0, 3600.0];
/// The amount of significant figures we wish to retain after the decimal point.
///
@ -18,7 +18,7 @@ pub const DMS_DIVISION: [f64; 3] = [1_f64, 60_f64, 3600_f64];
/// applications.
///
/// This is calculated with `10^n`, where `n` is the desired amount of SFs.
pub const DECIMAL_SF: f64 = 100_000_000_f64;
pub const DECIMAL_SF: f64 = 100_000_000.0;
/// All possible time tags, to be zipped with [`OFFSET_TAGS`]
pub const TIME_TAGS: [Tag; 3] = [Tag::DateTime, Tag::DateTimeOriginal, Tag::DateTimeDigitized];
@ -31,19 +31,19 @@ pub const OFFSET_TAGS: [Tag; 3] = [
];
/// The Earth's maximum latitude (can also be negative, depending on if you're North or South of the Equator).
pub const LAT_MAX_POS: f64 = 90_f64;
pub const LAT_MAX_POS: f64 = 90.0;
/// The Earth's maximum longitude (can also be negative depending on if you're East or West of the Prime meridian).
///
/// The negative value of this is known as the anti-meridian, and when combined they make a 360 degree circle around the Earth.
pub const LONG_MAX_POS: f64 = 180_f64;
pub const LONG_MAX_POS: f64 = 180.0;
/// 125km. This is the Kármán line + a 25km additional padding just to be safe.
pub const ALT_MAX_HEIGHT: i32 = 125_000_i32;
pub const ALT_MAX_HEIGHT: i32 = 125_000;
/// -1km. This should be adequate for even the Dead Sea on the Israeli border,
/// the lowest point on land (and much deeper).
pub const ALT_MIN_HEIGHT: i32 = -1000_i32;
pub const ALT_MIN_HEIGHT: i32 = -1000;
/// The maximum degrees that a direction can be (as a bearing, starting from 0 degrees)
pub const DIRECTION_MAX: i32 = 360;

4
crates/media-metadata/src/image/datetime.rs → crates/media-metadata/src/exif/datetime.rs
View file

@ -55,10 +55,10 @@ impl MediaDate {
///
/// This is for search ordering/sorting
#[must_use]
pub fn unix_timestamp(&self) -> i64 {
pub const fn unix_timestamp(&self) -> i64 {
match self {
Self::Utc(t) => t.timestamp(),
Self::Naive(t) => t.timestamp(),
Self::Naive(t) => t.and_utc().timestamp(),
}
}
}

0
crates/media-metadata/src/image/flash/consts.rs → crates/media-metadata/src/exif/flash/consts.rs
View file

2
crates/media-metadata/src/image/flash/data.rs → crates/media-metadata/src/exif/flash/data.rs
View file

@ -1,7 +1,7 @@
use exif::Tag;
use super::FlashValue;
use crate::image::{flash::consts::FLASH_MODES, ExifReader};
use crate::exif::{flash::consts::FLASH_MODES, ExifReader};
#[derive(
Default, Clone, Debug, Eq, PartialEq, serde::Serialize, serde::Deserialize, specta::Type,

0
crates/media-metadata/src/image/flash/mod.rs → crates/media-metadata/src/exif/flash/mod.rs
View file

0
crates/media-metadata/src/image/flash/values.rs → crates/media-metadata/src/exif/flash/values.rs
View file

2
crates/media-metadata/src/image/geographic/location.rs → crates/media-metadata/src/exif/geographic/location.rs
View file

@ -1,5 +1,5 @@
use crate::{
image::{
exif::{
consts::{
ALT_MAX_HEIGHT, ALT_MIN_HEIGHT, DECIMAL_SF, DIRECTION_MAX, DMS_DIVISION, LAT_MAX_POS,
LONG_MAX_POS,

0
crates/media-metadata/src/image/geographic/mod.rs → crates/media-metadata/src/exif/geographic/mod.rs
View file

2
crates/media-metadata/src/image/geographic/pluscodes.rs → crates/media-metadata/src/exif/geographic/pluscodes.rs
View file

@ -1,5 +1,5 @@
use crate::{
image::consts::{PLUSCODE_DIGITS, PLUSCODE_GRID_SIZE},
exif::consts::{PLUSCODE_DIGITS, PLUSCODE_GRID_SIZE},
Error,
};
use std::{

158
crates/media-metadata/src/exif/mod.rs Normal file
View file

@ -0,0 +1,158 @@
use std::path::Path;
use exif::Tag;
use sd_utils::error::FileIOError;
use tokio::task::spawn_blocking;
mod composite;
mod consts;
mod datetime;
mod flash;
mod geographic;
mod orientation;
mod profile;
mod reader;
mod resolution;
pub use composite::Composite;
pub use consts::DMS_DIVISION;
pub use datetime::MediaDate;
pub use flash::{Flash, FlashMode, FlashValue};
pub use geographic::{MediaLocation, PlusCode};
pub use orientation::Orientation;
pub use profile::ColorProfile;
pub use reader::ExifReader;
pub use resolution::Resolution;
use crate::{Error, Result};
#[derive(Default, Clone, Debug, PartialEq, serde::Serialize, serde::Deserialize, specta::Type)]
pub struct ExifMetadata {
pub resolution: Resolution,
pub date_taken: Option<MediaDate>,
pub location: Option<MediaLocation>,
pub camera_data: CameraData,
pub artist: Option<String>,
pub description: Option<String>,
pub copyright: Option<String>,
pub exif_version: Option<String>,
}
impl ExifMetadata {
pub async fn from_path(path: impl AsRef<Path> + Send) -> Result<Option<Self>> {
match spawn_blocking({
let path = path.as_ref().to_owned();
move || ExifReader::from_path(path).map(|reader| Self::from_reader(&reader))
})
.await?
{
Ok(data) => Ok(Some(data)),
Err(Error::Exif(
exif::Error::NotFound(_)
| exif::Error::NotSupported(_)
| exif::Error::BlankValue(_),
)) => Ok(None),
Err(Error::Exif(exif::Error::Io(e))) => Err(FileIOError::from((path, e)).into()),
Err(e) => Err(e),
}
}
pub fn from_slice(bytes: &[u8]) -> Result<Option<Self>> {
let res = ExifReader::from_slice(bytes).map(|reader| Self::from_reader(&reader));
if matches!(
res,
Err(Error::Exif(
exif::Error::NotFound(_)
| exif::Error::NotSupported(_)
| exif::Error::BlankValue(_)
))
) {
return Ok(None);
}
res.map(Some)
}
#[allow(clippy::field_reassign_with_default)]
fn from_reader(reader: &ExifReader) -> Self {
Self {
resolution: Resolution::from_reader(reader),
date_taken: MediaDate::from_reader(reader),
location: MediaLocation::from_exif_reader(reader).ok(),
camera_data: CameraData {
device_make: reader.get_tag(Tag::Make),
device_model: reader.get_tag(Tag::Model),
color_space: reader.get_tag(Tag::ColorSpace),
color_profile: ColorProfile::from_reader(reader),
focal_length: reader.get_tag(Tag::FocalLength),
shutter_speed: reader.get_tag(Tag::ShutterSpeedValue),
flash: Flash::from_reader(reader),
orientation: Orientation::from_reader(reader).unwrap_or_default(),
lens_make: reader.get_tag(Tag::LensMake),
lens_model: reader.get_tag(Tag::LensModel),
bit_depth: reader.get_tag::<String>(Tag::BitsPerSample).map_or_else(
|| {
reader
.get_tag::<String>(Tag::CompressedBitsPerPixel)
.unwrap_or_default()
.parse()
.ok()
},
|x| x.parse::<i32>().ok(),
),
zoom: reader
.get_tag(Tag::DigitalZoomRatio)
.map(|x: String| x.replace("unused", "1").parse().ok())
.unwrap_or_default(),
iso: reader.get_tag(Tag::PhotographicSensitivity),
software: reader.get_tag(Tag::Software),
serial_number: reader.get_tag(Tag::BodySerialNumber),
lens_serial_number: reader.get_tag(Tag::LensSerialNumber),
contrast: reader.get_tag(Tag::Contrast),
saturation: reader.get_tag(Tag::Saturation),
sharpness: reader.get_tag(Tag::Sharpness),
composite: Composite::from_reader(reader),
},
artist: reader.get_tag(Tag::Artist),
description: reader.get_tag(Tag::ImageDescription),
copyright: reader.get_tag(Tag::Copyright),
exif_version: reader.get_tag(Tag::ExifVersion),
}
}
}
#[derive(Default, Clone, PartialEq, Debug, serde::Serialize, serde::Deserialize, specta::Type)]
pub struct CameraData {
pub device_make: Option<String>,
pub device_model: Option<String>,
pub color_space: Option<String>,
pub color_profile: Option<ColorProfile>,
pub focal_length: Option<f64>,
pub shutter_speed: Option<f64>,
pub flash: Option<Flash>,
pub orientation: Orientation,
pub lens_make: Option<String>,
pub lens_model: Option<String>,
pub bit_depth: Option<i32>,
pub zoom: Option<f64>,
pub iso: Option<i32>,
pub software: Option<String>,
pub serial_number: Option<String>,
pub lens_serial_number: Option<String>,
pub contrast: Option<i32>,
pub saturation: Option<i32>,
pub sharpness: Option<i32>,
pub composite: Option<Composite>,
}
// TODO(brxken128): more exif spec reading so we can source color spaces correctly too
// pub enum ImageColorSpace {
// Rgb,
// RgbP,
// SRgb,
// Cmyk,
// DciP3,
// Wiz,
// Biz,
// }

0
crates/media-metadata/src/image/orientation.rs → crates/media-metadata/src/exif/orientation.rs
View file

0
crates/media-metadata/src/image/profile.rs → crates/media-metadata/src/exif/profile.rs
View file

17
crates/media-metadata/src/image/reader.rs → crates/media-metadata/src/exif/reader.rs
View file

@ -1,3 +1,5 @@
use crate::Result;
use std::{
fs::File,
io::{BufReader, Cursor},
@ -6,8 +8,7 @@ use std::{
};
use exif::{Exif, In, Tag};
use crate::{Error, Result};
use sd_utils::error::FileIOError;
/// An [`ExifReader`]. This can get exif tags from images (either files or slices).
pub struct ExifReader(Exif);
@ -16,19 +17,17 @@ impl ExifReader {
pub fn from_path(path: impl AsRef<Path>) -> Result<Self> {
exif::Reader::new()
.read_from_container(&mut BufReader::new(
File::open(&path)
.map_err(|e| Error::Io(e, path.as_ref().to_path_buf().into_boxed_path()))?,
File::open(&path).map_err(|e| FileIOError::from((path, e)))?,
))
.map_or_else(
|_| Err(Error::NoExifDataOnPath(path.as_ref().to_path_buf())),
|reader| Ok(Self(reader)),
)
.map(Self)
.map_err(Into::into)
}
pub fn from_slice(slice: &[u8]) -> Result<Self> {
exif::Reader::new()
.read_from_container(&mut Cursor::new(slice))
.map_or_else(|_| Err(Error::NoExifDataOnSlice), |reader| Ok(Self(reader)))
.map(Self)
.map_err(Into::into)
}
/// A helper function which gets the target `Tag` as `T`, provided `T` impls `FromStr`.

0
crates/media-metadata/src/image/resolution.rs → crates/media-metadata/src/exif/resolution.rs
View file

12
crates/media-metadata/src/ffmpeg/audio_props.rs Normal file
View file

@ -0,0 +1,12 @@
use serde::{Deserialize, Serialize};
use specta::Type;
#[derive(Debug, Serialize, Deserialize, Type)]
pub struct AudioProps {
pub delay: i32,
pub padding: i32,
pub sample_rate: Option<i32>,
pub sample_format: Option<String>,
pub bit_per_sample: Option<i32>,
pub channel_layout: Option<String>,
}

14
crates/media-metadata/src/ffmpeg/chapter.rs Normal file
View file

@ -0,0 +1,14 @@
use serde::{Deserialize, Serialize};
use specta::Type;
use super::metadata::Metadata;
#[derive(Debug, Serialize, Deserialize, Type)]
pub struct Chapter {
pub id: i32,
pub start: (u32, u32),
pub end: (u32, u32),
pub time_base_den: i32,
pub time_base_num: i32,
pub metadata: Metadata,
}

22
crates/media-metadata/src/ffmpeg/codec.rs Normal file
View file

@ -0,0 +1,22 @@
use serde::{Deserialize, Serialize};
use specta::Type;
use super::{audio_props::AudioProps, subtitle_props::SubtitleProps, video_props::VideoProps};
#[derive(Debug, Serialize, Deserialize, Type)]
pub struct Codec {
pub kind: Option<String>,
pub sub_kind: Option<String>,
pub tag: Option<String>,
pub name: Option<String>,
pub profile: Option<String>,
pub bit_rate: i32,
pub props: Option<Props>,
}
#[derive(Debug, Serialize, Deserialize, Type)]
pub enum Props {
Video(VideoProps),
Audio(AudioProps),
Subtitle(SubtitleProps),
}

31
crates/media-metadata/src/ffmpeg/metadata.rs Normal file
View file

@ -0,0 +1,31 @@
use std::collections::HashMap;
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use specta::Type;
#[derive(Default, Debug, Serialize, Deserialize, Type)]
pub struct Metadata {
pub album: Option<String>,
pub album_artist: Option<String>,
pub artist: Option<String>,
pub comment: Option<String>,
pub composer: Option<String>,
pub copyright: Option<String>,
pub creation_time: Option<DateTime<Utc>>,
pub date: Option<DateTime<Utc>>,
pub disc: Option<u32>,
pub encoder: Option<String>,
pub encoded_by: Option<String>,
pub filename: Option<String>,
pub genre: Option<String>,
pub language: Option<String>,
pub performer: Option<String>,
pub publisher: Option<String>,
pub service_name: Option<String>,
pub service_provider: Option<String>,
pub title: Option<String>,
pub track: Option<u32>,
pub variant_bit_rate: Option<u32>,
pub custom: HashMap<String, String>,
}

342
crates/media-metadata/src/ffmpeg/mod.rs Normal file
View file

@ -0,0 +1,342 @@
use crate::Result;
use std::path::Path;
use serde::{Deserialize, Serialize};
use specta::Type;
pub mod audio_props;
pub mod chapter;
pub mod codec;
pub mod metadata;
pub mod program;
pub mod stream;
pub mod subtitle_props;
pub mod video_props;
use chapter::Chapter;
use metadata::Metadata;
use program::Program;
#[derive(Debug, Serialize, Deserialize, Type)]
pub struct FFmpegMetadata {
pub formats: Vec<String>,
pub duration: Option<(u32, u32)>,
pub start_time: Option<(u32, u32)>,
pub bit_rate: (u32, u32),
pub chapters: Vec<Chapter>,
pub programs: Vec<Program>,
pub metadata: Metadata,
}
impl FFmpegMetadata {
pub async fn from_path(path: impl AsRef<Path> + Send) -> Result<Self> {
#[cfg(not(feature = "ffmpeg"))]
{
let _ = path;
Err(crate::Error::NoFFmpeg)
}
#[cfg(feature = "ffmpeg")]
{
sd_ffmpeg::probe(path)
.await
.map(Into::into)
.map_err(Into::into)
}
}
}
#[cfg(feature = "ffmpeg")]
mod extract_data {
use sd_ffmpeg::model::{
FFmpegAudioProps, FFmpegChapter, FFmpegCodec, FFmpegMediaData, FFmpegMetadata,
FFmpegProgram, FFmpegProps, FFmpegStream, FFmpegSubtitleProps, FFmpegVideoProps,
};
impl From<FFmpegMediaData> for super::FFmpegMetadata {
fn from(
FFmpegMediaData {
formats,
duration,
start_time,
bit_rate,
chapters,
programs,
metadata,
}: FFmpegMediaData,
) -> Self {
Self {
formats,
duration: duration.map(|duration| {
#[allow(clippy::cast_possible_truncation)]
{
// SAFETY: We're splitting in (high, low) parts, so we're not going to lose data on truncation
((duration >> 32) as u32, duration as u32)
}
}),
start_time: start_time.map(|start_time| {
#[allow(clippy::cast_possible_truncation)]
{
// SAFETY: We're splitting in (high, low) parts, so we're not going to lose data on truncation
((start_time >> 32) as u32, start_time as u32)
}
}),
bit_rate: {
#[allow(clippy::cast_possible_truncation)]
{
// SAFETY: We're splitting in (high, low) parts, so we're not going to lose data on truncation
((bit_rate >> 32) as u32, bit_rate as u32)
}
},
chapters: chapters.into_iter().map(Into::into).collect(),
programs: programs.into_iter().map(Into::into).collect(),
metadata: metadata.into(),
}
}
}
impl From<FFmpegChapter> for super::Chapter {
fn from(
FFmpegChapter {
id,
start,
end,
time_base_den,
time_base_num,
metadata,
}: FFmpegChapter,
) -> Self {
Self {
id: {
#[allow(clippy::cast_possible_truncation)]
{
// NOTICE: chapter.id is a i64, but I think it will be extremely rare to have a chapter id that doesn't fit in a i32
id as i32
}
},
// TODO: FIX these 2 when rspc/specta supports bigint
start: {
#[allow(clippy::cast_possible_truncation)]
{
// SAFETY: We're splitting in (high, low) parts, so we're not going to lose data on truncation
((start >> 32) as u32, start as u32)
}
},
end: {
#[allow(clippy::cast_possible_truncation)]
{
// SAFETY: We're splitting in (high, low) parts, so we're not going to lose data on truncation
((end >> 32) as u32, end as u32)
}
},
time_base_num,
time_base_den,
metadata: metadata.into(),
}
}
}
impl From<FFmpegProgram> for super::Program {
fn from(
FFmpegProgram {
id,
name,
streams,
metadata,
}: FFmpegProgram,
) -> Self {
Self {
id,
name,
streams: streams.into_iter().map(Into::into).collect(),
metadata: metadata.into(),
}
}
}
impl From<FFmpegStream> for super::stream::Stream {
fn from(
FFmpegStream {
id,
name,
codec,
aspect_ratio_num,
aspect_ratio_den,
frames_per_second_num,
frames_per_second_den,
time_base_real_den,
time_base_real_num,
dispositions,
metadata,
}: FFmpegStream,
) -> Self {
Self {
id,
name,
codec: codec.map(Into::into),
aspect_ratio_num,
aspect_ratio_den,
frames_per_second_num,
frames_per_second_den,
time_base_real_den,
time_base_real_num,
dispositions,
metadata: metadata.into(),
}
}
}
impl From<FFmpegCodec> for super::codec::Codec {
fn from(
FFmpegCodec {
kind,
sub_kind,
tag,
name,
profile,
bit_rate,
props,
}: FFmpegCodec,
) -> Self {
Self {
kind,
sub_kind,
tag,
name,
profile,
bit_rate,
props: props.map(Into::into),
}
}
}
impl From<FFmpegProps> for super::codec::Props {
fn from(props: FFmpegProps) -> Self {
match props {
FFmpegProps::Video(video_props) => Self::Video(video_props.into()),
FFmpegProps::Audio(audio_props) => Self::Audio(audio_props.into()),
FFmpegProps::Subtitle(subtitle_props) => Self::Subtitle(subtitle_props.into()),
}
}
}
impl From<FFmpegAudioProps> for super::audio_props::AudioProps {
fn from(
FFmpegAudioProps {
delay,
padding,
sample_rate,
sample_format,
bit_per_sample,
channel_layout,
}: FFmpegAudioProps,
) -> Self {
Self {
delay,
padding,
sample_rate,
sample_format,
bit_per_sample,
channel_layout,
}
}
}
impl From<FFmpegSubtitleProps> for super::subtitle_props::SubtitleProps {
fn from(FFmpegSubtitleProps { width, height }: FFmpegSubtitleProps) -> Self {
Self { width, height }
}
}
impl From<FFmpegVideoProps> for super::video_props::VideoProps {
fn from(
FFmpegVideoProps {
pixel_format,
color_range,
bits_per_channel,
color_space,
color_primaries,
color_transfer,
field_order,
chroma_location,
width,
height,
aspect_ratio_num,
aspect_ratio_den,
properties,
}: FFmpegVideoProps,
) -> Self {
Self {
pixel_format,
color_range,
bits_per_channel,
color_space,
color_primaries,
color_transfer,
field_order,
chroma_location,
width,
height,
aspect_ratio_num,
aspect_ratio_den,
properties,
}
}
}
impl From<FFmpegMetadata> for super::Metadata {
fn from(
FFmpegMetadata {
album,
album_artist,
artist,
comment,
composer,
copyright,
creation_time,
date,
disc,
encoder,
encoded_by,
filename,
genre,
language,
performer,
publisher,
service_name,
service_provider,
title,
track,
variant_bit_rate,
custom,
}: FFmpegMetadata,
) -> Self {
Self {
album,
album_artist,
artist,
comment,
composer,
copyright,
creation_time,
date,
disc,
encoder,
encoded_by,
filename,
genre,
language,
performer,
publisher,
service_name,
service_provider,
title,
track,
variant_bit_rate,
custom,
}
}
}
}

12
crates/media-metadata/src/ffmpeg/program.rs Normal file
View file

@ -0,0 +1,12 @@
use serde::{Deserialize, Serialize};
use specta::Type;
use super::{metadata::Metadata, stream::Stream};
#[derive(Debug, Serialize, Deserialize, Type)]
pub struct Program {
pub id: i32,
pub name: Option<String>,
pub streams: Vec<Stream>,
pub metadata: Metadata,
}

19
crates/media-metadata/src/ffmpeg/stream.rs Normal file
View file

@ -0,0 +1,19 @@
use serde::{Deserialize, Serialize};
use specta::Type;
use super::{codec::Codec, metadata::Metadata};
#[derive(Debug, Serialize, Deserialize, Type)]
pub struct Stream {
pub id: i32,
pub name: Option<String>,
pub codec: Option<Codec>,
pub aspect_ratio_num: i32,
pub aspect_ratio_den: i32,
pub frames_per_second_num: i32,
pub frames_per_second_den: i32,
pub time_base_real_den: i32,
pub time_base_real_num: i32,
pub dispositions: Vec<String>,
pub metadata: Metadata,
}

8
crates/media-metadata/src/ffmpeg/subtitle_props.rs Normal file
View file

@ -0,0 +1,8 @@
use serde::{Deserialize, Serialize};
use specta::Type;
#[derive(Debug, Serialize, Deserialize, Type)]
pub struct SubtitleProps {
pub width: i32,
pub height: i32,
}

19
crates/media-metadata/src/ffmpeg/video_props.rs Normal file
View file

@ -0,0 +1,19 @@
use serde::{Deserialize, Serialize};
use specta::Type;
#[derive(Debug, Serialize, Deserialize, Type)]
pub struct VideoProps {
pub pixel_format: Option<String>,
pub color_range: Option<String>,
pub bits_per_channel: Option<i32>,
pub color_space: Option<String>,
pub color_primaries: Option<String>,
pub color_transfer: Option<String>,
pub field_order: Option<String>,
pub chroma_location: Option<String>,
pub width: i32,
pub height: i32,
pub aspect_ratio_num: Option<i32>,
pub aspect_ratio_den: Option<i32>,
pub properties: Vec<String>,
}

135
crates/media-metadata/src/image/mod.rs
View file

@ -1,135 +0,0 @@
use exif::Tag;
use std::path::Path;
mod composite;
mod consts;
mod datetime;
mod flash;
mod geographic;
mod orientation;
mod profile;
mod reader;
mod resolution;
pub use composite::Composite;
pub use consts::DMS_DIVISION;
pub use datetime::MediaDate;
pub use flash::{Flash, FlashMode, FlashValue};
pub use geographic::{MediaLocation, PlusCode};
pub use orientation::Orientation;
pub use profile::ColorProfile;
pub use reader::ExifReader;
pub use resolution::Resolution;
use crate::Result;
#[derive(Default, Clone, Debug, PartialEq, serde::Serialize, serde::Deserialize, specta::Type)]
pub struct ImageMetadata {
pub resolution: Resolution,
pub date_taken: Option<MediaDate>,
pub location: Option<MediaLocation>,
pub camera_data: CameraData,
pub artist: Option<String>,
pub description: Option<String>,
pub copyright: Option<String>,
pub exif_version: Option<String>,
}
#[derive(Default, Clone, PartialEq, Debug, serde::Serialize, serde::Deserialize, specta::Type)]
pub struct CameraData {
pub device_make: Option<String>,
pub device_model: Option<String>,
pub color_space: Option<String>,
pub color_profile: Option<ColorProfile>,
pub focal_length: Option<f64>,
pub shutter_speed: Option<f64>,
pub flash: Option<Flash>,
pub orientation: Orientation,
pub lens_make: Option<String>,
pub lens_model: Option<String>,
pub bit_depth: Option<i32>,
pub red_eye: Option<bool>,
pub zoom: Option<f64>,
pub iso: Option<i32>,
pub software: Option<String>,
pub serial_number: Option<String>,
pub lens_serial_number: Option<String>,
pub contrast: Option<i32>,
pub saturation: Option<i32>,
pub sharpness: Option<i32>,
pub composite: Option<Composite>,
}
impl ImageMetadata {
pub fn from_path(path: impl AsRef<Path>) -> Result<Self> {
Self::from_reader(&ExifReader::from_path(path)?)
}
pub fn from_slice(bytes: &[u8]) -> Result<Self> {
Self::from_reader(&ExifReader::from_slice(bytes)?)
}
#[allow(clippy::field_reassign_with_default)]
pub fn from_reader(reader: &ExifReader) -> Result<Self> {
let mut data = Self::default();
let camera_data = &mut data.camera_data;
data.date_taken = MediaDate::from_reader(reader);
data.resolution = Resolution::from_reader(reader);
data.artist = reader.get_tag(Tag::Artist);
data.description = reader.get_tag(Tag::ImageDescription);
data.copyright = reader.get_tag(Tag::Copyright);
data.exif_version = reader.get_tag(Tag::ExifVersion);
data.location = MediaLocation::from_exif_reader(reader).ok();
camera_data.device_make = reader.get_tag(Tag::Make);
camera_data.device_model = reader.get_tag(Tag::Model);
camera_data.focal_length = reader.get_tag(Tag::FocalLength);
camera_data.shutter_speed = reader.get_tag(Tag::ShutterSpeedValue);
camera_data.color_space = reader.get_tag(Tag::ColorSpace);
camera_data.color_profile = ColorProfile::from_reader(reader);
camera_data.lens_make = reader.get_tag(Tag::LensMake);
camera_data.lens_model = reader.get_tag(Tag::LensModel);
camera_data.iso = reader.get_tag(Tag::PhotographicSensitivity);
camera_data.zoom = reader
.get_tag(Tag::DigitalZoomRatio)
.map(|x: String| x.replace("unused", "1").parse().ok())
.unwrap_or_default();
camera_data.bit_depth = reader.get_tag::<String>(Tag::BitsPerSample).map_or_else(
|| {
reader
.get_tag::<String>(Tag::CompressedBitsPerPixel)
.unwrap_or_default()
.parse()
.ok()
},
|x| x.parse::<i32>().ok(),
);
camera_data.orientation = Orientation::from_reader(reader).unwrap_or_default();
camera_data.flash = Flash::from_reader(reader);
camera_data.software = reader.get_tag(Tag::Software);
camera_data.serial_number = reader.get_tag(Tag::BodySerialNumber);
camera_data.lens_serial_number = reader.get_tag(Tag::LensSerialNumber);
camera_data.software = reader.get_tag(Tag::Software);
camera_data.contrast = reader.get_tag(Tag::Contrast);
camera_data.saturation = reader.get_tag(Tag::Saturation);
camera_data.sharpness = reader.get_tag(Tag::Sharpness);
camera_data.composite = Composite::from_reader(reader);
Ok(data)
}
}
// TODO(brxken128): more exif spec reading so we can source color spaces correctly too
// pub enum ImageColorSpace {
// Rgb,
// RgbP,
// SRgb,
// Cmyk,
// DciP3,
// Wiz,
// Biz,
// }

32
crates/media-metadata/src/lib.rs
View file

@ -11,30 +11,28 @@
clippy::unwrap_used,
unused_qualifications,
rust_2018_idioms,
clippy::expect_used,
trivial_casts,
trivial_numeric_casts,
unused_allocation,
clippy::as_conversions,
clippy::dbg_macro
clippy::unnecessary_cast,
clippy::cast_lossless,
clippy::cast_possible_truncation,
clippy::cast_possible_wrap,
clippy::cast_precision_loss,
clippy::cast_sign_loss,
clippy::dbg_macro,
clippy::deprecated_cfg_attr,
clippy::separated_literal_suffix,
deprecated
)]
#![forbid(deprecated_in_future)]
#![forbid(unsafe_code)]
#![allow(clippy::missing_errors_doc, clippy::module_name_repetitions)]
pub mod audio;
mod error;
pub mod image;
pub mod video;
pub mod exif;
pub mod ffmpeg;
pub use audio::AudioMetadata;
pub use error::{Error, Result};
pub use image::ImageMetadata;
pub use video::VideoMetadata;
#[derive(Clone, PartialEq, Debug, serde::Serialize, serde::Deserialize, specta::Type)]
#[serde(tag = "type")]
pub enum MediaMetadata {
Image(Box<ImageMetadata>),
Video(Box<VideoMetadata>),
Audio(Box<AudioMetadata>),
}
pub use exif::ExifMetadata;
pub use ffmpeg::FFmpegMetadata;

20
crates/media-metadata/src/video.rs
View file

@ -1,20 +0,0 @@
use std::path::Path;
use crate::Result;
#[derive(
Default, Clone, PartialEq, Eq, Debug, serde::Serialize, serde::Deserialize, specta::Type,
)]
pub struct VideoMetadata {
duration: Option<i32>, // bigint
video_codec: Option<String>,
audio_codec: Option<String>,
}
impl VideoMetadata {
#[allow(clippy::missing_errors_doc)]
#[allow(clippy::missing_panics_doc)]
pub fn from_path(_path: impl AsRef<Path>) -> Result<Self> {
todo!()
}
}

1
crates/prisma/src/lib.rs
View file

@ -1,3 +1,4 @@
#![recursion_limit = "256"]
#[allow(warnings, unused)]
pub mod prisma;
#[allow(warnings, unused)]

19
crates/task-system/Cargo.toml
View file

@ -1,8 +1,8 @@
[package]
name = "sd-task-system"
version = "0.1.0"
authors = ["Ericson \"Fogo\" Soares <ericson.ds999@gmail.com>"]
rust-version = "1.75.0"
authors = ["Ericson Soares <ericson@spacedrive.com>"]
rust-version = "1.75"
license.workspace = true
edition.workspace = true
repository.workspace = true
@ -25,20 +25,17 @@ tokio = { workspace = true, features = [
tokio-stream = { workspace = true }
tracing = { workspace = true }
uuid = { workspace = true, features = ["v4"] }
# External deps
downcast-rs = "1.2.0"
pin-project = "1.1.4"
[dev-dependencies]
tokio = { workspace = true, features = ["macros", "test-util", "fs"] }
tempfile = { workspace = true }
rand = "0.8.5"
tracing-test = { workspace.dev-dependencies = true, features = [
"no-env-filter",
] }
thiserror = { workspace = true }
lending-stream = { workspace = true }
serde = { workspace = true, features = ["derive"] }
rand = { workspace = true }
rmp-serde = { workspace = true }
serde = { workspace = true, features = ["derive"] }
tempfile = { workspace = true }
thiserror = { workspace = true }
tokio = { workspace = true, features = ["macros", "test-util", "fs"] }
tracing-test = { workspace = true, features = ["no-env-filter"] }
uuid = { workspace = true, features = ["serde"] }

10
crates/utils/src/db.rs
View file

@ -63,6 +63,16 @@ pub fn inode_to_db(inode: u64) -> Vec<u8> {
inode.to_le_bytes().to_vec()
}
pub fn ffmpeg_data_field_to_db(field: i64) -> Vec<u8> {
field.to_be_bytes().to_vec()
}
pub fn ffmpeg_data_field_from_db(field: &[u8]) -> i64 {
i64::from_be_bytes([
field[0], field[1], field[2], field[3], field[4], field[5], field[6], field[7],
])
}
pub fn size_in_bytes_from_db(db_size_in_bytes: &[u8]) -> u64 {
u64::from_be_bytes([
db_size_in_bytes[0],

264
interface/app/$libraryId/Explorer/Inspector/MediaData.tsx
View file

@ -1,13 +1,13 @@
import dayjs from 'dayjs';
import customParseFormat from 'dayjs/plugin/customParseFormat'; // import plugin
import utc from 'dayjs/plugin/utc'; // import plugin
import {
capitalize,
CoordinatesFormat,
MediaDate,
ExifMetadata,
FFmpegMetadata,
humanizeSize,
int32ArrayToBigInt,
MediaLocation,
MediaMetadata,
MediaData as RemoteMediaData,
useSelector,
useUnitFormatStore
} from '@sd/client';
@ -18,38 +18,6 @@ import { Platform, usePlatform } from '~/util/Platform';
import { explorerStore } from '../store';
import { MetaData } from './index';
interface Props {
data: MediaMetadata;
}
// const DateFormatWithTz = 'YYYY-MM-DD HH:mm:ss ZZ';
// const DateFormatWithoutTz = 'YYYY-MM-DD HH:mm:ss';
// const formatMediaDate = (datetime: MediaDate): { formatted: string; raw: string } | undefined => {
// dayjs.extend(customParseFormat);
// dayjs.extend(utc);
// // dayjs.tz.setDefault(dayjs.tz.guess());
// const getTzData = (dt: string): [string, number] => {
// if (dt.includes('+'))
// return [DateFormatWithTz, Number.parseInt(dt.substring(dt.indexOf('+'), 3))];
// return [DateFormatWithoutTz, 0];
// };
// const [tzFormat, tzOffset] = getTzData(datetime);
// console.log({
// formatted: dayjs(datetime, tzFormat).utcOffset(tzOffset).format('HH:mm:ss, MMM Do YYYY'),
// raw: datetime
// });
// return {
// formatted: dayjs(datetime, tzFormat).utcOffset(tzOffset).format('HH:mm:ss, MMM Do YYYY'),
// raw: datetime
// };
// };
const formatLocationDD = (loc: MediaLocation, dp?: number): string => {
// the lack of a + here will mean that coordinates may have padding at the end
// google does the same (if one is larger than the other, the smaller one will be padded with zeroes)
@ -99,24 +67,154 @@ const UrlMetadataValue = (props: { text: string; url: string; platform: Platform
</a>
);
// const orientations = {
// Normal: 'Normal',
// MirroredHorizontal: 'Horizontally mirrored',
// MirroredHorizontalAnd90CW: 'Mirrored horizontally and rotated 90° clockwise',
// MirroredHorizontalAnd270CW: 'Mirrored horizontally and rotated 270° clockwise',
// MirroredVertical: 'Vertically mirrored',
// CW90: 'Rotated 90° clockwise',
// CW180: 'Rotated 180° clockwise',
// CW270: 'Rotated 270° clockwise'
// };
const MediaData = ({ data }: Props) => {
const ExifMediaData = (data: ExifMetadata) => {
const platform = usePlatform();
const { t } = useLocale();
const coordinatesFormat = useUnitFormatStore().coordinatesFormat;
const showMoreInfo = useSelector(explorerStore, (s) => s.showMoreInfo);
return data.type === 'Image' ? (
return (
<>
<MetaData
label="Date"
tooltipValue={data.date_taken ?? null} // should show full raw value
// should show localised, utc-offset value or plain value with tooltip mentioning that we don't have the timezone metadata
value={data.date_taken ?? null}
/>
<MetaData label="Type" value="Image" />
<MetaData
label="Location"
tooltipValue={data.location && formatLocation(data.location, coordinatesFormat)}
value={
data.location && (
<UrlMetadataValue
url={`https://www.google.com/maps/search/?api=1&query=${encodeURIComponent(
formatLocation(data.location, 'dd')
)}`}
text={formatLocation(
data.location,
coordinatesFormat,
coordinatesFormat === 'dd' ? 4 : 0
)}
platform={platform}
/>
)
}
/>
<MetaData
label="Plus Code"
value={
data.location?.pluscode && (
<UrlMetadataValue
url={`https://www.google.com/maps/search/?api=1&query=${encodeURIComponent(
data.location.pluscode
)}`}
text={data.location.pluscode}
platform={platform}
/>
)
}
/>
<MetaData
label="Resolution"
value={`${data.resolution.width} x ${data.resolution.height}`}
/>
<MetaData label="Device" value={data.camera_data.device_make} />
<MetaData label="Model" value={data.camera_data.device_model} />
<MetaData label="Color profile" value={data.camera_data.color_profile} />
<MetaData label="Color space" value={data.camera_data.color_space} />
<MetaData label="Flash" value={data.camera_data.flash?.mode} />
<MetaData
label="Zoom"
value={
data.camera_data &&
data.camera_data.zoom &&
!Number.isNaN(data.camera_data.zoom)
? `${data.camera_data.zoom.toFixed(2) + 'x'}`
: '--'
}
/>
<MetaData label="Iso" value={data.camera_data.iso} />
<MetaData label="Software" value={data.camera_data.software} />
</>
);
};
const FFmpegMediaData = (data: FFmpegMetadata) => {
const { t } = useLocale();
const streamKinds = new Set(
data.programs.flatMap((program) => program.streams.map((stream) => stream.codec?.kind))
);
const type = streamKinds.has('video')
? 'Video'
: streamKinds.has('audio')
? 'Audio'
: capitalize(streamKinds.values().next().value) ?? 'Unknown';
const bit_rate = humanizeSize(int32ArrayToBigInt(data.bit_rate), {
is_bit: true,
base_unit: 'binary',
use_plural: false
});
const duration_ms = data.duration ? int32ArrayToBigInt(data.duration) / 1000n : null;
const duration = duration_ms
? dayjs.duration(
Number(duration_ms / 1000n) + Number(duration_ms % 1000n) / 1000,
'seconds'
)
: null;
const start_time_ms = data.start_time ? int32ArrayToBigInt(data.start_time) / 1000n : null;
const start_time = start_time_ms
? dayjs.duration(
Number(start_time_ms / 1000n) + Number(start_time_ms % 1000n) / 1000,
'seconds'
)
: null;
const chapters = data.chapters
.map((chapter) => {
const num = BigInt(chapter.time_base_num);
const den = BigInt(chapter.time_base_den);
const start = dayjs.duration(
Number((int32ArrayToBigInt(chapter.start) * num) / den),
'seconds'
);
const end = dayjs.duration(
Number((int32ArrayToBigInt(chapter.end) * num) / den),
'seconds'
);
return `${start.format('HH:mm:ss')} - ${end.format('HH:mm:ss')}`;
})
.join('\n');
return (
<>
<MetaData label={t('type')} value={type} />
<MetaData label="Bitrate" value={`${bit_rate.value} ${bit_rate.unit}/s`} />
{duration && <MetaData label={t('duration')} value={duration.format('HH:mm:ss.SSS')} />}
{start_time && (
<MetaData label={t('start_time')} value={start_time.format('HH:mm:ss.SSS')} />
)}
{chapters && <MetaData label={t('chapters')} value={chapters} />}
</>
);
};
interface Props {
data: RemoteMediaData;
}
export const MediaData = ({ data }: Props) => {
const { t } = useLocale();
const showMoreInfo = useSelector(explorerStore, (s) => s.showMoreInfo);
return (
<div className="flex flex-col gap-0 py-2">
<Accordion
isOpen={showMoreInfo}
@ -124,70 +222,10 @@ const MediaData = ({ data }: Props) => {
variant="apple"
title={t('more_info')}
>
<MetaData
label="Date"
tooltipValue={data.date_taken ?? null} // should show full raw value
// should show localised, utc-offset value or plain value with tooltip mentioning that we don't have the timezone metadata
value={data.date_taken ?? null}
/>
<MetaData label="Type" value={data.type} />
<MetaData
label="Location"
tooltipValue={data.location && formatLocation(data.location, coordinatesFormat)}
value={
data.location && (
<UrlMetadataValue
url={`https://www.google.com/maps/search/?api=1&query=${encodeURIComponent(
formatLocation(data.location, 'dd')
)}`}
text={formatLocation(
data.location,
coordinatesFormat,
coordinatesFormat === 'dd' ? 4 : 0
)}
platform={platform}
/>
)
}
/>
<MetaData
label="Plus Code"
value={
data.location?.pluscode && (
<UrlMetadataValue
url={`https://www.google.com/maps/search/?api=1&query=${encodeURIComponent(
data.location.pluscode
)}`}
text={data.location.pluscode}
platform={platform}
/>
)
}
/>
<MetaData
label="Resolution"
value={`${data.resolution.width} x ${data.resolution.height}`}
/>
<MetaData label="Device" value={data.camera_data.device_make} />
<MetaData label="Model" value={data.camera_data.device_model} />
<MetaData label="Color profile" value={data.camera_data.color_profile} />
<MetaData label="Color space" value={data.camera_data.color_space} />
<MetaData label="Flash" value={data.camera_data.flash?.mode} />
<MetaData
label="Zoom"
value={
data.camera_data &&
data.camera_data.zoom &&
!Number.isNaN(data.camera_data.zoom)
? `${data.camera_data.zoom.toFixed(2) + 'x'}`
: '--'
}
/>
<MetaData label="Iso" value={data.camera_data.iso} />
<MetaData label="Software" value={data.camera_data.software} />
{'Exif' in data ? ExifMediaData(data.Exif) : FFmpegMediaData(data.FFmpeg)}
</Accordion>
</div>
) : null;
);
};
export default MediaData;

24
interface/app/$libraryId/Explorer/Inspector/index.tsx
View file

@ -27,11 +27,11 @@ import { useLocation } from 'react-router';
import { Link as NavLink } from 'react-router-dom';
import Sticky from 'react-sticky-el';
import {
byteSize,
FilePath,
FilePathWithObject,
getExplorerItemData,
getItemFilePath,
humanizeSize,
NonIndexedPathItem,
Object,
ObjectKindEnum,
@ -235,21 +235,21 @@ export const SingleItemMetadata = ({ item }: { item: ExplorerItem }) => {
});
const filesMediaData = useLibraryQuery(['files.getMediaData', objectData?.id ?? -1], {
enabled: objectData?.kind === ObjectKindEnum.Image && readyToFetch
enabled: objectData != null && readyToFetch
});
const ephemeralLocationMediaData = useBridgeQuery(
['ephemeralFiles.getMediaData', ephemeralPathData != null ? ephemeralPathData.path : ''],
{
enabled: ephemeralPathData?.kind === ObjectKindEnum.Image && readyToFetch
enabled: ephemeralPathData != null && readyToFetch
}
);
const mediaData = filesMediaData ?? ephemeralLocationMediaData ?? null;
const mediaData = filesMediaData.data ?? ephemeralLocationMediaData.data ?? null;
const fullPath = queriedFullPath.data ?? ephemeralPathData?.path;
const { name, isDir, kind, size, casId, dateCreated, dateAccessed, dateModified, dateIndexed } =
const { isDir, kind, size, casId, dateCreated, dateAccessed, dateModified, dateIndexed } =
useExplorerItemData(item);
const pubId = objectData != null ? uniqueId(objectData) : null;
@ -365,7 +365,7 @@ export const SingleItemMetadata = ({ item }: { item: ExplorerItem }) => {
</MetaContainer>
)}
{mediaData.data && <MediaData data={mediaData.data} />}
{mediaData && <MediaData data={mediaData} />}
<MetaContainer className="flex !flex-row flex-wrap gap-1 overflow-hidden">
<InfoPill>{isDir ? t('folder') : translateKindName(kind)}</InfoPill>
@ -483,7 +483,7 @@ const MultiItemMetadata = ({ items }: { items: ExplorerItem[] }) => {
getExplorerItemData(item);
if (item.type !== 'NonIndexedPath' || !item.item.is_dir) {
metadata.size = (metadata.size ?? BigInt(0)) + size.original;
metadata.size = (metadata.size ?? BigInt(0)) + BigInt(size.original);
}
if (dateCreated)
@ -529,7 +529,7 @@ const MultiItemMetadata = ({ items }: { items: ExplorerItem[] }) => {
<MetaData
icon={Cube}
label={t('size')}
value={metadata.size !== null ? `${byteSize(metadata.size)}` : null}
value={metadata.size !== null ? `${humanizeSize(metadata.size)}` : null}
/>
<MetaData
icon={Clock}
@ -638,12 +638,14 @@ interface MetaDataProps {
export const MetaData = ({ icon: Icon, label, value, tooltipValue, onClick }: MetaDataProps) => {
return (
<div className="flex items-center text-xs text-ink-dull" onClick={onClick}>
<div className="flex content-start justify-start text-xs text-ink-dull" onClick={onClick}>
{Icon && <Icon weight="bold" className="mr-2 shrink-0" />}
<span className="mr-2 flex-1 whitespace-nowrap">{label}</span>
<span className="mr-2 flex flex-1 items-start justify-items-start whitespace-nowrap">
{label}
</span>
<Tooltip
label={tooltipValue || value}
className="truncate text-ink"
className="truncate whitespace-pre text-ink"
tooltipClassName="max-w-none"
>
{value ?? '--'}

Some files were not shown because too many files have changed in this diff Show more