Fact: 8 out of 10 native node module packages published to npm are broken. Actual fact: npm has a library called pacote that handles resolving the package specifier you put in your dependencies to files on disk, and it's actually really nice. Change hak to use that library so now it's much simpler, handles versions correctly and supports git / github etc. urls so we can specify dependencies from git when a package author only publishes half of the package to npm. |
||
---|---|---|
.. | ||
build.js | ||
check.js | ||
clean.js | ||
copy.js | ||
fetch.js | ||
fetchDeps.js | ||
hakEnv.js | ||
index.js | ||
link.js | ||
README.md | ||
target.js |
hak
This tool builds native dependencies for element-desktop. Here follows some very minimal documentation for it.
Goals:
- Must build compiled native node modules in a shippable state (ie. only dynamically linked against libraries that will be on the target system, all unnecessary files removed).
- Must be able to build any native module, no matter what build system it uses (electron-rebuild is supposed to do this job but only works for modules that use gyp).
It's also loosely designed to be a general tool and agnostic to what it's actually building. It's used here to build modules for the electron app but should work equally well for building modules for normal node.
Running
Hak is invoked with a command and a dependency, eg. yarn run hak fetch matrix-seshat
.
If no dependencies are given, hak runs the command on all dependencies.
Files
There are a lot of files involved:
- scripts/hak/... - The tool itself
- hak/[dependency] - Files provided by the app that tell hak how to build each of its native dependencies. Contains a hak.json file and also some script files, each of which must be referenced in hak.json.
- .hak/ - Files generated by hak in the course of doing its job. Includes the dependency module itself and any of the native dependency's native dependencies.
- .hak/[dependency]/build - An extracted copy of the dependency's node module used to build it.
- .hak/[dependency]/out - Another extracted copy of the dependency, this one contains only what will be shipped.
Workings
Hak works around native node modules that try to fetch or build their native component in the npm 'install' phase - modules that do this will typically end up with native components targeted to the build platform and the node that npm/yarn is using, which is no good for an electron app.
It does this by installing it with --ignore-scripts
and then using yarn link
to keep the
dependency module separate so yarn doesn't try to run its install / postinstall script
at other points (eg. whenever you yarn add
a random other dependency).
This also means that the dependencies cannot be listed in dependencies
or
devDependencies
in the project, since this would cause npm / yarn to install them and
try to fetch their native parts. Instead, they are listed in hakDependencies
which
hak reads to install them for you.
Hak will not install dependencies for the copy of the module it links into your project, so if your native module has javascript dependencies that are actually needed at runtime (and not just to fetch / build the native parts), it won't work.
Hak will generate a .yarnrc
in the project directory to set the link directory to its
own in the .hak directory (unless one already exists, in which case this is your problem).
Lifecycle
Hak is divided into lifecycle stages, in order:
- fetch - Download and extract the source of the dependency
- link - Link the copy of the dependency into your node_modules directory
- fetchDeps - Fetch & extract any native dependencies required to build the module.
- build - The Good Stuff. Configure and build any native dependencies, then the module itself.
- copy - Copy the built artifact from the module build directory to the module output directory.
hak.json
The scripts section contains scripts used for lifecycle stages that need them (fetch, fetchDeps, build). It also contains 'prune' and 'copy' which are globs of files to delete from the output module directory and copy over from the module build directory to the output module directory, respectively.
Shortcomings
Hak doesn't know about dependencies between lifecycle stages, ie. it doesn't know that you need to 'fetch' and 'fetchDeps' before you can 'build', etc. You get to run each individually, and remember the right order.
There is also a lot of duplication in the command execution: we should abstract away some of the boilerplate required to run commands & so forth.