element-desktop/scripts/hak/README.md

# hak

This tool builds native dependencies for element-desktop. Here follows some very minimal
documentation for it.

Goals:

-   Must build compiled native node modules in a shippable state
    (ie. only dynamically linked against libraries that will be on the
    target system, all unnecessary files removed).
-   Must be able to build any native module, no matter what build system
    it uses (electron-rebuild is supposed to do this job but only works
    for modules that use gyp).

It's also loosely designed to be a general tool and agnostic to what it's
actually building. It's used here to build modules for the electron app
but should work equally well for building modules for normal node.

# Running

Hak is invoked with a command and a dependency, eg. `yarn run hak fetch matrix-seshat`.
If no dependencies are given, hak runs the command on all dependencies.

# Files

There are a lot of files involved:

-   scripts/hak/... - The tool itself
-   hak/[dependency] - Files provided by the app that tell hak how to build each of its native dependencies.
    Contains a hak.json file and also some script files, each of which must be referenced in hak.json.
-   .hak/ - Files generated by hak in the course of doing its job. Includes the dependency module itself and
    any of the native dependency's native dependencies.
-   .hak/[dependency]/build - An extracted copy of the dependency's node module used to build it.
-   .hak/[dependency]/out - Another extracted copy of the dependency, this one contains only what will be shipped.

# Workings

Hak works around native node modules that try to fetch or build their native component in
the npm 'install' phase - modules that do this will typically end up with native components
targeted to the build platform and the node that npm/yarn is using, which is no good for an
electron app.

It does this by installing it with `--ignore-scripts` and then using `yarn link` to keep the
dependency module separate so yarn doesn't try to run its install / postinstall script
at other points (eg. whenever you `yarn add` a random other dependency).

This also means that the dependencies cannot be listed in `dependencies` or
`devDependencies` in the project, since this would cause npm / yarn to install them and
try to fetch their native parts. Instead, they are listed in `hakDependencies` which
hak reads to install them for you.

Hak will _not_ install dependencies for the copy of the module it links into your
project, so if your native module has javascript dependencies that are actually needed at
runtime (and not just to fetch / build the native parts), it won't work.

Hak will generate a `.yarnrc` in the project directory to set the link directory to its
own in the .hak directory (unless one already exists, in which case this is your problem).

# Lifecycle

Hak is divided into lifecycle stages, in order:

-   fetch - Download and extract the source of the dependency
-   link - Link the copy of the dependency into your node_modules directory
-   build - The Good Stuff. Configure and build any native dependencies, then the module itself.
-   copy - Copy the built artifact from the module build directory to the module output directory.

# hak.json

The scripts section contains scripts used for lifecycle stages that need them (fetch, fetchDeps, build).
It also contains 'prune' and 'copy' which are globs of files to delete from the output module directory
and copy over from the module build directory to the output module directory, respectively.

# Shortcomings

Hak doesn't know about dependencies between lifecycle stages, ie. it doesn't know that you need to
'fetch' and 'fetchDeps' before you can 'build', etc. You get to run each individually, and remember
the right order.

There is also a _lot_ of duplication in the command execution: we should abstract away
some of the boilerplate required to run commands & so forth.