Openfn compile for unit tests by mtuchi · Pull Request #1443 · OpenFn/kit

mtuchi · 2026-06-04T15:33:52Z

Short Description

Update openfn compile to compiles workflows job expressions to standard JavaScript, writing output to compiled/ by default.

Fixes #1424

Implementation Details

openfn compile now compiles all workflows in the project to compiled/ by default (no flag needed)
openfn compile <workflow-name> looks up a workflow by name in the project and compiles it to compiled/ use -O for stdout or -o <dir> to redirect output
openfn compile <file.js> and openfn compile <workflow.json> retain their existing behaviour (stdout by default)
Added new exports-only transformer (src/transforms/exports-only.ts) with a single responsibility: keep imports, named export declarations, and their transitive non-exported dependencies; drop everything else (operations, export default, bare declarations). Runs at order: 0 (before
all other transformers). Is a no-op unless explicitly enabled.
Added --exports-only flag: strips adaptor operation calls and keeps only exported declarations — intended for unit testing job code
When --exports-only is active the CLI sets exports-only: true and explicitly disables ensure-exports and top-level-operations, so the output is clean module code with no empty export default [].

QA Notes

Setup

Check out the 1424-unit-tests branch and install openfnx:

pnpm install:openfnx

Verify the correct version is installed — openfnx --version should output branch/1424-unit-tests.

Testing openfn compile

Navigate to a local OpenFn project containing an openfn.yaml file and test the new openfn compile command. Available options:

# Compile all workflows in the project (full compilation, preserves operations)
openfn compile

# Compile all workflows, stripping operation calls (useful for unit testing)
openfn compile --exports-only

# Compile a single workflow by name
openfn compile my-workflow

# Compile a single workflow to a custom directory
openfn compile my-workflow -o tests/

# Compile a single job expression (prints to stdout)
openfn compile workflows/my-workflow/step-a.js --exports-only

# Watch mode: recompile whenever source files change
openfn compile --exports-only --watch

AI Usage

Please disclose whether you've used AI anywhere in this PR (it's cool, we just
want to know!):

I have used Claude Code
I have used another model
I have not used AI

You can read more details in our
Responsible AI Policy

Release branch checklist

Delete this section if this is not a release PR.

If this IS a release branch:

Run pnpm changeset version from root to bump versions
Run pnpm install
Commit the new version numbers
Run pnpm changeset tag to generate tags
Push tags git push --tags

Tags may need updating if commits come in after the tags are first generated.

- Add --test flag: compiles job expressions for unit testing, writing output to tests/ by default (reads dirs.tests from openfn.yaml) - Add --strip (default on with --test): tree-shakes compiled output, keeping only export const/function declarations and their transitive dependencies; use --no-strip to keep all compiled code - Add --watch flag: watches source files and recompiles on change (uses chokidar) - Strip mode removes injected _defer import when operations are stripped - Skip writing files with no exportable code after stripping; log when skipping - Auto-clean stale step files in tests/ that were skipped in the current run without touching user-added files at other paths - Fix --test --no-strip skipping pure-operation jobs (hasExportableCode guard now only applies when strip is active) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- remove export default [] from strip mode output (not needed for unit testing) - add --no-strip flag to keep full compiled output including operations - remove --strip standalone flag (stripping is always on with --test by default) - auto-derive output path for single-file --test using tests/ dir - skip writing files with no exportable code in strip mode - auto-clean stale step files in tests/ after project-wide strip runs - fix --test --no-strip skipping pure-operation jobs - update help examples and option descriptions Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

josephjclark · 2026-06-04T16:58:42Z

@mtuchi I'm not liking the --tests argument. What we really want here is just to easily compile some or all of a project to disk. Stripping the operations is an optional step. As mentioned in the issue I'd call that compile --exports-only.

When you do openfn compile workflow.json today, it'll compile that workflow and print the result to stdout.

When you do openfn compile my-workflow, what will happen? It should lookup the workflow called my-workflow in the project, compile that, and print it to stdout. It doesn't, but that would be consistent.

When you do openfn compile it should compile the whole project. It can't print that to stdout because it'll compiling multiple files. But it can compile to dist and write all the files there. That is what I expect the default behaviour to be.

I'd also suggest that openfn compile my-workflow should compile to disk by default (the user can pass -O to write to stdout). That's a little inconsistent with openfn compile workflow.yaml but it IS consistent with compiling through the project. So I'm OK with it. Maybe later, in CLI 2.0, we'll always compile to disk for consistency. But since this stuff is very rarely used I wouldn't worry too much.

So drop the --tests flag please and just make openfn compile compile a project to a folder on disk.

…mpile - Drop --test and --strip/--no-strip flags from openfn compile - Add --exports-only flag (opt-in) to strip operation calls, keeping only exported constants and functions for unit testing - openfn compile (no args) now compiles the whole project to compiled/ by default - openfn compile <workflow-name> looks up a workflow by name/id and compiles it - Extract stripping logic into a new exports-only transformer (order: 0) that runs before all others; remove strip option from top-level-operations - Update unit-testing-jobs.md to document the new workflow Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mtuchi · 2026-06-05T13:03:44Z

QA walk through video for @hunterachieng
https://www.loom.com/share/033dc7f75c5642a1b5e8edb8a9a35f46

josephjclark · 2026-06-05T15:36:59Z


 export type CompiledJob = { code: string; map?: SourceMapWithOperations };

+export const hasExportableCode = (code: string): boolean =>


what is this here for?

josephjclark · 2026-06-05T15:41:23Z

+        (step as any).name ?? step.id
+      );
+
+      if (opts.exportsOnly && !hasExportableCode(code)) {


The compiler should decide whether there's any exportable code - not the CLI.

The CLI could choose to not write empty files, that would be fine. But we can't have it do compiler stuff.

josephjclark · 2026-06-05T15:43:22Z

+
+  for (const workflow of workflows) {
+    for (const step of workflow.steps) {
+      const expression = (step as any).expression;


typing is very messy here. Can we continue if there's no expression in step, and otherwise cast step properly?

or, if we have to cast to any, let's just do it once at the top of the loop. But I don;t think that should be neccessary

josephjclark · 2026-06-05T15:44:53Z

+        (step as any).adaptor ?? (step as any).adaptors?.[0];
+      const stepOpts: CompileOptions = {
+        ...opts,
+        adaptors: adaptor ? [adaptor] : opts.adaptors ?? [],


This is very complex! Can we simplify?

josephjclark · 2026-06-05T15:45:29Z

+        const stalePath = compiledDir
+          ? path.join(compiledDir, workflow.id, `${step.id}.js`)
+          : null;
+        log.info(


This is a debug log at most

josephjclark · 2026-06-05T15:51:23Z

+  watcher.on('change', async (changedPath: string) => {
+    logger.info(`${changedPath} changed, recompiling...`);
+    try {
+      if (options.workflowName) {


This looks like a duplication of the first half of the function?

josephjclark · 2026-06-05T15:52:03Z

+  const watchTargets = collectWatchTargets(options);
+  logger.info(`Watching for changes. Ctrl+C to stop.`);
+
+  const watcher = chokidar.watch(watchTargets, {


nodejs has a --watch argument now. I wonder if you could just that, alongside with --watch-path in the scripts to get the same effect more easily

josephjclark · 2026-06-05T16:01:09Z

+    (node) =>
+      n.ImportDeclaration.check(node) ||
+      n.ExportNamedDeclaration.check(node) ||
+      needed.has(node as n.Statement)


I don't understand this needed bit - what is it trying to do?

I think this should be really simple: accept import and name export statements, and discard the rest.

If we're trying to keep non exported constants and functions and dependencies and stuff, well that's out of scope and more work than I was expecting. Is there a specific use case for that you're looking at?

josephjclark · 2026-06-05T16:11:39Z

 import { namedTypes as n, namedTypes } from 'ast-types';
 import type { NodePath } from 'ast-types/lib/node-path';
 import type { Transformer } from '../transform';
-// Note that the validator should complain if it see anything other than export default []


Why are there diffs in this file? This should be unchanged now right?

josephjclark · 2026-06-05T16:18:47Z

+
+// --- exports-only transformer ---
+
+test('is a no-op when options is not true', (t) => {


I would prefer if these tests tests source code before and after

So like

const before = `export const x = () => {}; fn()'` const after = `fn()`

So much easier to read than the ASTs, and since these test aren't particularly intricate I think it'll work better

mtuchi and others added 5 commits June 4, 2026 18:01

chore: update pnpm-lock.yaml

9f0ecd7

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

docs: reference unit-testing-jobs.md in additional documentation

1023165

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

docs: add unit-testing-jobs.md guide for openfn compile --test

7c10b6e

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

taylordowns2000 added this to Core Jun 4, 2026

github-project-automation Bot moved this to New Issues in Core Jun 4, 2026

josephjclark reviewed Jun 4, 2026

View reviewed changes

Comment thread packages/compiler/src/transforms/top-level-operations.ts Outdated

mtuchi and others added 3 commits June 5, 2026 11:02

prettier format

5befc82

chore: trim noisy comments from compile and transformer files

13dc96c

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mtuchi marked this pull request as ready for review June 5, 2026 12:14

mtuchi marked this pull request as draft June 5, 2026 12:14

pnpm test:format

ae1b1b8

mtuchi mentioned this pull request Jun 5, 2026

Compile a project for unit tests #1424

Open

josephjclark reviewed Jun 5, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Openfn compile for unit tests#1443

Openfn compile for unit tests#1443
mtuchi wants to merge 9 commits into
mainfrom
1424-unit-tests

mtuchi commented Jun 4, 2026 •

edited

Loading

Uh oh!

josephjclark commented Jun 4, 2026

Uh oh!

Uh oh!

mtuchi commented Jun 5, 2026

Uh oh!

josephjclark Jun 5, 2026

Uh oh!

josephjclark Jun 5, 2026

Uh oh!

josephjclark Jun 5, 2026

Uh oh!

josephjclark Jun 5, 2026

Uh oh!

josephjclark Jun 5, 2026

Uh oh!

josephjclark Jun 5, 2026

Uh oh!

josephjclark Jun 5, 2026

Uh oh!

josephjclark Jun 5, 2026

Uh oh!

josephjclark Jun 5, 2026

Uh oh!

josephjclark Jun 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants


		export type CompiledJob = { code: string; map?: SourceMapWithOperations };

		export const hasExportableCode = (code: string): boolean =>


		// --- exports-only transformer ---

		test('is a no-op when options is not true', (t) => {

Conversation

mtuchi commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Short Description

Implementation Details

QA Notes

AI Usage

Release branch checklist

Uh oh!

josephjclark commented Jun 4, 2026

Uh oh!

Uh oh!

mtuchi commented Jun 5, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mtuchi commented Jun 4, 2026 •

edited

Loading