element-desktop/scripts/hak

hak

This tool builds native dependencies for element-desktop. Here follows some very minimal documentation for it.

Goals:

• Must build compiled native node modules in a shippable state (ie. only dynamically linked against libraries that will be on the target system, all unnecessary files removed).
• Must be able to build any native module, no matter what build system it uses (electron-rebuild is supposed to do this job but only works for modules that use gyp).

It's also loosely designed to be a general tool and agnostic to what it's actually building. It's used here to build modules for the electron app but should work equally well for building modules for normal node.

Running

Hak is invoked with a command and a dependency, eg. yarn run hak fetch matrix-seshat. If no dependencies are given, hak runs the command on all dependencies.

Files

There are a lot of files involved:

• scripts/hak/... - The tool itself
• hak/[dependency] - Files provided by the app that tell hak how to build each of its native dependencies. Contains a hak.json file and also some script files, each of which must be referenced in hak.json.
• .hak/ - Files generated by hak in the course of doing its job. Includes the dependency module itself and any of the native dependency's native dependencies.
• .hak/[dependency]/build - An extracted copy of the dependency's node module used to build it.
• .hak/[dependency]/out - Another extracted copy of the dependency, this one contains only what will be shipped.

Workings

Hak works around native node modules that try to fetch or build their native component in the npm 'install' phase - modules that do this will typically end up with native components targeted to the build platform and the node that npm/yarn is using, which is no good for an electron app.

It does this by installing it with --ignore-scripts and then using yarn link to keep the dependency module separate so yarn doesn't try to run its install / postinstall script at other points (eg. whenever you yarn add a random other dependency).

This also means that the dependencies cannot be listed in dependencies or devDependencies in the project, since this would cause npm / yarn to install them and try to fetch their native parts. Instead, they are listed in hakDependencies which hak reads to install them for you.

Hak will not install dependencies for the copy of the module it links into your project, so if your native module has javascript dependencies that are actually needed at runtime (and not just to fetch / build the native parts), it won't work.

Hak will generate a .yarnrc in the project directory to set the link directory to its own in the .hak directory (unless one already exists, in which case this is your problem).

Lifecycle

Hak is divided into lifecycle stages, in order:

• fetch - Download and extract the source of the dependency
• link - Link the copy of the dependency into your node_modules directory
• build - The Good Stuff. Configure and build any native dependencies, then the module itself.
• copy - Copy the built artifact from the module build directory to the module output directory.

hak.json

The scripts section contains scripts used for lifecycle stages that need them (fetch, fetchDeps, build). It also contains 'prune' and 'copy' which are globs of files to delete from the output module directory and copy over from the module build directory to the output module directory, respectively.

Shortcomings

Hak doesn't know about dependencies between lifecycle stages, ie. it doesn't know that you need to 'fetch' and 'fetchDeps' before you can 'build', etc. You get to run each individually, and remember the right order.

There is also a lot of duplication in the command execution: we should abstract away some of the boilerplate required to run commands & so forth.