Software-Mac/spacedrive
Software-Mac/spacedrive
1
0
Fork 0
mirror of https://github.com/spacedriveapp/spacedrive synced 2024-07-04 13:23:28 +00:00
Code Issues Packages Projects Releases Wiki Activity
spacedrive/CONTRIBUTING.md
Vítor Vasconcellos e797b02e65
Media metadata extraction & Thumbnailer rework (#2285) 
* initial ffprobe commit

* Working slim down version ffprobe

* Auto format ffprobe and deps source

* Remove show_pixel_formats logic
- Fix do_bitexact incorrect check in main after last changes
- Fix some clangd warning

* Remove show_* and print_format options and their respective logic

* Rework ffprobe into simple_ffprobe
- Simplify ffprobe logic into a simple program that gather and print a media file metadata

* Reduce the amount of ffmpeg log messages while generating thumbnails

* Fix completly wrong comments

* mend

* Start modeling ffmpeg extracted metadata on schema
 - Start porting ffprobe code to rust
 - Rename some references to media_data to exif_data

* Finish modeling media info data
 - Add MediaProgram, MediaStream, MediaCodec, MediaVideoProps, MediaAudioProps, MediaSubtitleProps to Schema
 - Fix simple_ffproble to use its custom print_codec, instead of ffmpeg's impl

* Add relation between MediaInfo and FilePath
 - Remove shared properties from MediaInfo and related structs
 - Implement Iterator for FFmpegDict

* Fix and update schema

* Data models and start populating MediaInfo in rust

* Finish populating media info, chapters and program

* Improve FFmpegFormatContext data raw pointer access
 - Implement stream data gathering

* Impl FFmpegCodecContext, retrieve codec information
 - Improve some unsafe pointer uses
 - Impl from FFmpegFormatContext to MediaInfo conversion

* Fix FFmpegDict Drop

* Fix some crago warnings

* Impl retrieval of video props
 - Fix C char* to Rust String convertion

* Impl retrieval of audio and subtitle props
 - Fill props for MediaCodec

* Remove simple_ffprobe now that the Rust impl is done

* Fix schema to match actually retrieved media info
 - Fix import some FFmpeg constants instead of directly using values

* Rework movie_decoder
 - Re-implement create_scale_string and add support anamorphic video
 - Improve C pointer access for FFmpegFormatContext and FFmpegCodecContext
 - Use newer FFmpeg abstractions in movie_decoder

* Fix incorrect props when initializing MovieDecoder

* Remove unecessary lifetimes

* Added more native wrappers for some FFmpeg native objects used in movie_decoder

* Remove FFmpegPacket
 - Some more improvements to movie_decoder

* WIP

* Some small fixes

* More fixes
Rename movie_decoder to frame_decoder
Remove more references to film_strips

* fmt

* Fix duplicate migration for job error changes

* fix rebase

* Solving segfaults, fuck C lang

Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>

* Update rust to version 1.77
 - Pin rust version with rust-toolchain.toml
 - Change from dtolnay/rust-toolchain to IronCoreLabs/rust-toolchain for rust-toolchain support
 - Remove unused function and imports
 - Replace most CString uses with new c literal string

* More segfault solving and other minor fixes

Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>

* Fix ffmpeg rotation filter breaking portrait video thumbnails #2150
 - Plus some other misc fixes

* Auto format

* Retrieve video/audio metadata on frontend

* Auto format

* First draft on ffmpeg data save on db

Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>

* Fix some incorrect changes to prisma schema

* Some fixes for the FFmpegData schema
 - Expand logic to save FFmpegData to db

* A ton of things

Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>

* Integrating ffmpeg media data in jobs and API

* Rspc can't BigInt

* 🙄

* Add initial ffmpeg metadata entries to Inspector
 - Fix ephemeral metadata api to match the files metadata api call

* Fix Inspector not showing ffmpeg metadata

* Add bitrate, start time and chapters video metadata to Inspector
- Fix backend BigInt conversion incorrectly using i32 instead of u32
- Change FFmpegFormatContext/FFmpegMetaData bit_rate to i64
- Rename byteSize to humanizeSize
- Expand humanizeSize logic to allow handling bits and Binary units
- Move capitalize to @sd/client utils

* Solving some issues

* Fix ffmpeg probe getting incorrect stream id and breaking database unique constraint
 - Fix humanizeSize breaking when receiving floating numbers
 - Fix incorrect equality in StatCard
 - Fix unhandled error in Dialog when trying to remove an unknown dialog

* fmt

* small improvements
 - Remove some unecessary recursion_limit directive
 - Remove unused app_image releated functions
 - Fix metadata query enabled flag

* Add migration for ffmpeg media data

* Fix cypress test

* Requested changes

* Implement feedback
 - Update locale keys for all languages
 - Add pnpm command to update all language keys

* Fix thumb reactivity in non indexed locations

---------

Co-authored-by: Ericson Soares <ericson.ds999@gmail.com>
Co-authored-by: Vítor Vasconcellos <HeavenVolkoff@users.noreply.github.com>
2024-05-09 02:20:28 +00:00

175 lines
10 KiB
Markdown
Raw Blame History

# Welcome to the Spacedrive Contributing Guide
Thank you for investing your time in contributing to our project!
Please read our [Code of Conduct](./CODE_OF_CONDUCT.md) to keep our community approachable and respectable.
This guide will provide an overview of the contribution workflow, including opening an issue, creating a pull request (PR), and the review and merge process.
## New Contributor Guide
To familiarize yourself with the project, please read the [README](README.md). Here are some resources to help you get started with open-source contributions:
- [Finding ways to contribute to open-source on GitHub](https://docs.github.com/en/get-started/exploring-projects-on-github/finding-ways-to-contribute-to-open-source-on-github)
- [Setting up Git](https://docs.github.com/en/get-started/quickstart/set-up-git)
- [GitHub flow](https://docs.github.com/en/get-started/quickstart/github-flow)
- [Collaborating with pull requests](https://docs.github.com/en/github/collaborating-with-pull-requests)
- [Getting started with Tauri](https://tauri.app/v1/guides/getting-started/prerequisites)
- [pnpm CLI](https://pnpm.io/pnpm-cli)
## Getting Started
### Issues
#### Creating a New Issue
If you come across an issue or have a feature request for Spacedrive, please [search if a related issue has already been reported](https://docs.github.com/en/github/searching-for-information-on-github/searching-on-github/searching-issues-and-pull-requests#search-by-the-title-body-or-comments). If no relevant issue exists, you can open a new issue using the appropriate [issue form](https://github.com/spacedriveapp/spacedrive/issues/new/choose).
#### Solving an Issue
To find an issue that interests you, you can browse through our [existing issues](https://github.com/spacedriveapp/spacedrive/issues) and use the available `labels` to narrow down your search (See [Labels](https://github.com/spacedriveapp/spacedrive/labels) for more information). As a general rule, if you find an issue you want to work on, you are welcome to open a PR with a fix.
### Making Changes
#### Making Changes Locally
This project uses [Cargo](https://doc.rust-lang.org/cargo/getting-started/installation.html) and [pnpm](https://pnpm.io/installation). Make sure you have them installed before proceeding.
To make changes locally, follow these steps:
1. Clone the repository: `git clone https://github.com/spacedriveapp/spacedrive`
2. Navigate to the project directory: `cd spacedrive`
3. Configure your system environment for Spacedrive development
1. For Linux users, run: `./scripts/setup.sh`
> This [script](https://github.com/spacedriveapp/spacedrive/blob/main/scripts/setup.sh#L133) will check if Rust and pnpm are installed then proceed to install Clang, NASM, LLVM, libvips, Gstreamer's Plugins, FFmpeg, Perl, [Tauri essentials](https://tauri.app/v1/guides/getting-started/prerequisites/#setting-up-linux) and any other required dependencies for Spacedrive to build.
2. For macOS users, run: `./scripts/setup.sh`
> This [script](https://github.com/spacedriveapp/spacedrive/blob/main/scripts/setup.sh#L108) will check if Rust, pnpm and Xcode are installed and proceed to use Homebrew to install NASM, [Tauri essentials](https://tauri.app/v1/guides/getting-started/prerequisites/#setting-up-macos) and install any other required dependencies for Spacedrive to build.
3. For Windows users, run in PowerShell: `.\scripts\setup.ps1`
> This [script](https://github.com/spacedriveapp/spacedrive/blob/main/scripts/setup.ps1#L81) will install pnpm, LLVM, FFmpeg, C++ build tools, NASM, Rust + Cargo, Rust tools, Edge Webview 2, Strawberry Perl, [Tauri essentials](https://tauri.app/v1/guides/getting-started/prerequisites/#setting-up-windows) and any other required dependencies for Spacedrive to build.
4. Install dependencies: `pnpm i`
5. Prepare the build: `pnpm prep` (This will run all necessary codegen and build required dependencies)
To quickly run only the desktop app after `prep`, you can use:
- `pnpm tauri dev`
If necessary, the [webview devtools](https://tauri.app/v1/guides/debugging/application/#webview-console) can be opened by pressing `Ctrl + Shift + I` (Linux and Windows) or `Command + Option + I` (macOS) in the desktop app.
Also, the react-devtools can be launched using `pnpm dlx react-devtools`.
However, it must be executed before starting the desktop app for it to connect.
You can download a bundle with sample files to test the app by running:
- `pnpm test-data`
Only for Linux and macOS (Requires curl and tar).
The test files will be located in a directory called `test-data` in the root of the spacedrive repo.
To run the web app:
- `pnpm dev:web`
This will start both the server and web interface.
You can launch these individually if you'd prefer:
- `cargo run -p sd-server` (server)
- `pnpm web dev` (web interface)
To run the e2e tests for the web app:
- `pnpm web test:e2e`
If you are developing a new test, you can execute Cypress in interactive mode with:
- `pnpm web test:interactive`
To run the landing page:
- `pnpm landing dev`
If you encounter any issues, ensure that you are using the following versions of Rust, Node and Pnpm:
- Rust version: **1.78**
- Node version: **18.18**
- Pnpm version: **9.0.6**
After cleaning out your build artifacts using `pnpm clean`, `git clean`, or `cargo clean`, it is necessary to re-run the `setup-system` script.
Make sure to read the [guidelines](https://spacedrive.com/docs/developers/prerequisites/guidelines) to ensure that your code follows a similar style to ours.
##### Mobile App
To run the mobile app:
- Install Java JDK <= 17 for Android
- Java 21 is not compatible: https://github.com/react-native-async-storage/async-storage/issues/1057#issuecomment-1925963956
- Install [Android Studio](https://developer.android.com/studio) for Android and [Xcode](https://apps.apple.com/au/app/xcode/id497799835) for iOS development.
- Run `./scripts/setup.sh mobile`
- This will set up most of the dependencies required to build the mobile app.
- Make sure you have [NDK 26.1.10909125 and CMake](https://developer.android.com/studio/projects/install-ndk#default-version) installed in Android Studio.
- Run the following commands:
- `pnpm mobile android` (runs on Android Emulator)
- In order to have locations working on Android, you must run the following command once the application has been installed for the first time. Otherwise, locations will not work.
- `adb shell appops set --uid com.spacedrive.app MANAGE_EXTERNAL_STORAGE allow`
- Run the following commands to access the logs from `sd-core`.
- `adb shell`
- Then `run-as com.spacedrive.app` to access the app's directory on device.
- Run `cd files/logs` and then select the logs with the timestamp of when you ran the app. Ex: `sd.log.2023-11-28`.
- You can view the logs using `tail -f [log-name]`. Ex: `tail -f sd.log.2023-11-28`.
- `pnpm mobile ios` (runs on iOS Emulator)
- `xcrun simctl launch --console booted com.spacedrive.app` allows you to view the console output of the iOS app from `tracing`. However, the application must be built in `debug` mode for this.
- `pnpm mobile start` (runs the metro bundler only)
### Pull Request
Once you have finished making your changes, create a pull request (PR) to submit them.
- Fill out the "Ready for review" template to help reviewers understand your changes and the purpose of your PR.
- If you are addressing an existing issue, don't forget to [link your PR to the issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue).
- Enable the checkbox to [allow maintainer edits](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/allowing-changes-to-a-pull-request-branch-created-from-a-fork) so that the branch can be updated for merging.
- Once you submit your PR, a team member will review your proposal. They may ask questions or request additional information.
- You may be asked to make changes before the PR can be merged, either through [suggested changes](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/incorporating-feedback-in-your-pull-request) or pull request comments. You can apply suggested changes directly through the UI. For other changes, you can make them in your fork and commit them to your branch.
- As you update your PR and apply changes, mark each conversation as [resolved](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/commenting-on-a-pull-request#resolving-conversations).
- If you run into any merge issues, refer to this [git tutorial](https://lab.github.com/githubtraining/managing-merge-conflicts) to help you resolve merge conflicts and other issues.
### Your PR is Merged!
Congratulations! 🎉🎉 The Spacedrive team thanks you for your contribution! ✨
Once your PR is merged, your changes will be included in the next release of the application.
### Common Errors
#### `xcrun: error: unable to find utility "xctest", not a developer tool or in PATH`
This error occurs when Xcode is not installed or when the Xcode command line tools are not in your `PATH`.
To resolve this issue:
- Install Xcode from the macOS App Store or directly from [here](https://xcodereleases.com/) (requires Apple Account).
- Run `xcode-select -s /Applications/Xcode.app/Contents/Developer`.
This command will use Xcode's developer tools instead of macOS's default tools.
#### `unable to lookup item 'PlatformPath'`
If you run into this issue, or similar:
```
error: terminated(1): /us/bin/xcrun --sdk macos --show-sdk-platform-path output :
xcrun: error: unable to lookup item 'PlatformPath' from command line tools installation xcrun: error: unable to lookup item 'PlatformPath' in SDK '/Library/Developer /CommandLineTools/SDKs/MacOSX.sdk'
```
Ensure that macOS is fully updated, and that you have Xcode installed (via the app store).
Once that has completed, run `xcode-select --install` in the terminal to install the command line tools. If they are already installed, ensure that you update macOS to the latest version available.
Also ensure that Rosetta is installed, as a few of our dependencies require it. You can install Rosetta with `softwareupdate --install-rosetta --agree-to-license`.
### Translations
Check out the [i18n README](interface/locales/README.md) for more information on how to contribute to translations.
### Credits
This CONTRIBUTING.md file was inspired by the [github/docs CONTRIBUTING.md](https://github.com/github/docs/blob/main/CONTRIBUTING.md) file, and we extend our gratitude to the original author.
Reference in a new issue View git blame Copy permalink