tunnckoCore/gibon

View on GitHub
@packages/glob-cache/.verb.md

Summary

Maintainability
Test Coverage
## API

<!-- docks-start -->

{%= include(process.cwd() + "/docs/src/index.md") %}

<!-- docks-end -->

## Context and how it works

Each context contains a `{ file, cacheFile, cacheLocation, cacache }` and more
properties. The `file` one represents the fresh file loaded from the system, the
`cacheFile` represents the file from the cache. Both has `path`, `size` and
`integrity` properties, plus more.

The `cacheFile` can be `null` if it's the first hit (not found in cache), in
such case the `ctx.notFound` will be `true` and on next runs this will be
`false`. When using the Hooks API, the `options.hooks.notFound()` or
`options.hooks.found()` will be called.

Important to note is that `cacheFile` don't have a `contents` property, but has
`path` which points to the place of the cache file on the disk.

The interesting one is the `ctx.changed`. This one is the reason for the whole
existance of this module. If both the "source" file and cache file are the same
(based on [cacache][]), e.g. same size and integrity (which means the
contents/shasum are equal), then `ctx.changed === false`, otherwise this will be
`true`. Simply said, when you change your file(s) matched by a the given glob
pattern(s), then it will be `ctx.changed === true` and the
`options.hooks.changed()` will be called. Depending on whether it's the first
call or not, either `options.hooks.found` or `options.hooks.notFound` will also
be called.

If you are using the Hooks API (e.g. `globCache.promise` plus `options.hooks`),
there is also one more key point and that's that we have `options.hooks.always`
hook function, which might be useful if you want more control, and so you can
decide what to do or make more additional checks - for example, listen the
`mtime` - or track the dependencies of the file. Tracking dependencies is
something that some test runner may benefit.

Because all that, we also expose [cacache][] to the Context, so you can update
or clean the cache - it's up to you.

Example Context (the `options.hooks.changed`, `options.hooks.notFound` and
`options.hooks.always` hooks are called)

```
{
  file: {
    path: '/home/charlike/github/tunnckoCore/opensource/packages/glob-cache/test/index.js',
    contents: <Buffer 27 75 73 65 20 73 74 72 69 63 74 27 3b 0a 0a 63 6f 6e 73 74 20 70 61 74 68 20 3d 20 72 65 71 75 69 72 65 28 27 70 61 74 68 27 29 3b 0a 63 6f 6e 73 74 ... 350 more bytes>,
    size: 427,
    integrity: 'sha512-p5daDYwu9vhNNjT9vfRrWHXIwwlPxeqeub4gs3qMZ88J//ONUH7Je2Muu9o+MxjA1Fv3xwbgkBdjcHgdj7ar4A=='
  },
  cacheFile: null,
  cacheLocation: '/home/charlike/github/tunnckoCore/opensource/packages/glob-cache/test/fixture-cache',
  cacache: { /* cacache instance */ },
  changed: true,
  notFound: true
}
```

And when you run it more times (with no changes), the `cacheFile` will not be
`null` anymore, like so

```
{
  file: {
    path: '/home/charlike/github/tunnckoCore/opensource/packages/glob-cache/test/index.js',
    contents: <Buffer 27 75 73 65 20 73 74 72 69 63 74 27 3b 0a 0a 63 6f 6e 73 74 20 70 61 74 68 20 3d 20 72 65 71 75 69 72 65 28 27 70 61 74 68 27 29 3b 0a 63 6f 6e 73 74 ... 350 more bytes>,
    size: 427,
    integrity: 'sha512-p5daDYwu9vhNNjT9vfRrWHXIwwlPxeqeub4gs3qMZ88J//ONUH7Je2Muu9o+MxjA1Fv3xwbgkBdjcHgdj7ar4A=='
  },
  cacheFile: {
    key: '/home/charlike/github/tunnckoCore/opensource/packages/glob-cache/test/index.js',
    integrity: 'sha512-p5daDYwu9vhNNjT9vfRrWHXIwwlPxeqeub4gs3qMZ88J//ONUH7Je2Muu9o+MxjA1Fv3xwbgkBdjcHgdj7ar4A=='
    path: '/home/charlike/github/tunnckoCore/opensource/packages/glob-cache/fixture-cache/content-v2/sha512/78/84/a154130fdefee002a708cee1ae570db54b1a278fed9b7a3847c73b2545bd48947c2cd192d365f9d87653f098f80d98b4ee37923ba467dbc314acf0f42e39',
    size: 427,
    stat: Stat {}
    time: 1579561781331,
    metadata: undefined
  },
  cacheLocation: '/home/charlike/github/tunnckoCore/opensource/packages/glob-cache/fixture-cache',
  cacache: { /* cacache instance */ },
  changed: false,
  notFound: false
}
```

As you can see above, both the `file.integrity` and `cacheFile.integrity` are
the same, also the `size`, so the both files are equal (and so
`ctx.changed: false`) - the `options.hooks.notChanged` will be called.

Below example shows usage of `changed` hook and Workers.

```js
const globCache = require('glob-cache')
const JestWorker = require('jest-worker');

let worker = null;

(async () => {
  await globCache.promise({
    include: 'packages/*/src/**/*.js'
    hooks: {
      async changed(ctx) {
        // If we are here, it's either the first run, or
        // only when there's a difference between the actual file and the cache file.
        // So we can, for example, call our worker/runner or whatever here.
        worker =
          worker ||
          new JestWorker(require.resolve('./my-awesome-worker-or-runner.js'), {
            numWorkers: 7,
            forkOptions: { stdio: 'inherit' },
          });

        await worker.default(ctx);
        await worker.end();
      },
    }
  });
})();
```

Above you're looking on a basic solution similar to what's done in Jest with the
difference that Jest can detect changes only if it's a Git project. At least the
`--onlyChanged` works that way (with Git requirement) - which isn't a big
problem of course since mostly every project is using Git, but anyway.

The point is, that you can do whatever you want in custom conditions based on
your preferences and needs.

In above example you may wonder why we are instatiating JestWorker inside the
`if` statement. That's because if you instantiate it before the call of
`globCache` (where is the `let worker` assignment) then you have no way to end
the worker in any meaningful and easy way.

Similar implementation you can see in the
[`hela-eslint-workers`](https://github.com/tunnckoCore/opensource/tree/hela-eslint-workers/%40hela/eslint/src)
branch where using `glob-cache` we are trying to speed up ESLint a bit, by
putting `eslint.executeOnFiles` or `eslint.executeOnText` inside a worker. The
thing is that it doesn't help much, because ESLint is just slow - for the same
reason even the `jest-runner-eslint` doesn't help much with performance. The
complexity in ESLint is O(n) - the more configs and plugins you have in your
config, the more slow it will run even on a single file - it's inevitable and a
huge problem. I'm not saying all that just to hate. It's just because of the
synchornous design of ESLint and the way it works. A big pain point is not only
that it exposes & uses only sync methods, but also the architecture of resolving
huge amount of configs and plugins. That may change if
[RFC#9](https://github.com/eslint/rfcs/pull/9) is accepted, for which I have big
hopes. Even if it's accepted it will take few major releases.