Skip to content

feat: pip-install quickstart with bundled --example + 3-path getting-started - #231

Closed
changliu2 wants to merge 3 commits into
mainfrom
changliu2/pip-quickstart-3tabs
Closed

feat: pip-install quickstart with bundled --example + 3-path getting-started#231
changliu2 wants to merge 3 commits into
mainfrom
changliu2/pip-quickstart-3tabs

Conversation

@changliu2

Copy link
Copy Markdown
Collaborator

What

Make pip install assert-ai the canonical install and give every onboarding shape a pip-only, zero-clone runnable path. Replaces the editable git clone + pip install -e ".[otel,langgraph]" quickstart, which was heavy boilerplate, invisible to PyPI download telemetry, and didn't match the published assert-ai package.

Three quickstart paths

The README, docs/getting-started.md, and the website now lead with pip install assert-ai and branch into three paths that map to how the user arrives:

Tab For Install
I have an agent LangGraph / CrewAI / OpenAI Agents SDK / custom pip install "assert-ai[otel]" — connect via the auto_trace 2-liner + target.callable
I have the agent spec a system prompt / description pip install assert-aiassert-ai run --example health-assistant
Help me start nothing yet pip install assert-aiassert-ai init

Bundled examples (--example)

  • New assert_ai/_examples/ subpackage ships two runnable configs in the wheel (configs + agent code only — no result artifacts):
    • health-assistant — Prompt Agent (system_prompt); runs on a base install
    • travel-planner-langgraph — LangGraph + OpenTelemetry trace capture; needs [langgraph,otel]
  • assert-ai run --example <name> resolves the bundled config. No value (or an unknown name) lists the catalog with per-example install hints. --config and --example are mutually exclusive.
  • Wheel stays 0.31 MB; verified a clean-venv install resolves --example from site-packages.

Website

  • Real <QuickstartTabs> client component (3 tabs + macOS/Linux ↔ Windows sub-toggle), injected by MarkdownContent via a <!--quickstart-tabs--> sentinel in docs/getting-started.md. GitHub renders the same three paths as a Markdown fallback (sentinels are invisible there).
  • Fixed a stale microsoft/ASSERT docs blob URL → responsibleai/ASSERT.

Notes

  • Extras are always quoted ("assert-ai[otel]") so zsh / PowerShell don't glob the brackets.
  • AGENTS.md setup commands updated to the published-package path with explicit per-scenario extras.
  • A follow-up PR removes the ~19 MB of committed examples/incident_triage_agent/artifacts/*.jsonl and gitignores result artifacts (kept separate to keep this diff reviewable).

Test plan

  • 7 new CLI/registry tests (tests/test_cli.py) pass
  • python -m build → wheel ships both configs, zero artifacts leaked
  • clean-venv pip install <wheel>assert-ai run --example resolves from site-packages
  • website tsc --noEmit + eslint + next build all pass (/docs/getting-started prerenders)
  • pre-existing test_viewer_* failures confirmed present on main (Node/TS env, unrelated)

@jakepresent jakepresent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes. The direction is good, but I found two issues that hit the new public quickstart surface directly.

  1. assert-ai run --example without a value does not work with the supported Click dependency range. The PR body, help text, and test say that --example with no value should list the catalog, but with Click 8.3.1 I get:
Error: Option '--example' requires an argument.

I reproduced it by running python -m pytest tests/test_cli.py -q; CliTest.test_example_without_value_lists_catalog fails. Since pyproject.toml allows click>=8.0, a fresh pip install assert-ai can resolve to this version, so the advertised no-value form is not safe. Either make that form actually work across the allowed Click versions, or change the CLI/docs to use an explicit catalog command/form instead.

  1. The pip-only docs still tell users to copy .env.example, but zero-clone users will not have that file. docs/getting-started.md and the website quickstart tabs show cp .env.example .env / Copy-Item .env.example .env immediately after pip install assert-ai. In a clean pip-only directory there is no .env.example, so the first-run path fails before ASSERT runs. For the zero-clone path, use direct environment-variable instructions or a command that creates a minimal .env with placeholders.

I stopped there because both are user-facing onboarding blockers for the exact surface this PR is changing.

changliu2 added a commit that referenced this pull request Jun 10, 2026
…eview)

1. assert-ai run --example no longer relies on Click flag_value (broke on Click 8.3.1 with 'Option --example requires an argument'). --example is now a plain value option; the catalog is discoverable via a new 'assert-ai examples' command that works across the supported Click range. Unknown name and the no-config/no-example paths still print the catalog.

2. Pip-only quickstart no longer tells zero-clone users to 'cp .env.example .env' (that file doesn't exist after a bare pip install). The three pip-only paths now instruct setting AZURE_API_KEY/AZURE_API_BASE as env vars or creating a .env — which the runner already auto-loads via find_dotenv(usecwd=True). The clone-based 'develop from source' path keeps cp .env.example .env.

Updates: cli.py, tests/test_cli.py (examples-command + bare-flag-errors tests), docs/getting-started.md, website QuickstartTabs.tsx. pytest test_cli.py green; website tsc+eslint clean.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@changliu2

Copy link
Copy Markdown
Collaborator Author

Thanks @jakepresent — both were real onboarding blockers on the exact surface this PR changes. Fixed in ca67d8c.

1. --example without a value broke on Click 8.3.1.
You're right — the flag_value trick wasn't safe across click>=8.0. Removed it: --example is now a plain value-taking option, and the catalog is discoverable via a new assert-ai examples command that works across the supported Click range. The unknown-name and no---config/no---example paths still print the catalog. Replaced the test_example_without_value_lists_catalog test with test_examples_command_lists_catalog + test_bare_example_flag_errors; pytest tests/test_cli.py is green (16 passed).

2. .env.example doesn't exist for pip-only users.
Correct — after a bare pip install assert-ai there's no .env.example to copy. The three pip-only paths now say: set AZURE_API_KEY / AZURE_API_BASE as env vars, or create a .env in the working dir — which the runner already auto-loads via find_dotenv(usecwd=True) (runner.py:63), so a hand-created .env works zero-clone. The clone-based "develop from source" path keeps cp .env.example .env. Fixed in docs/getting-started.md (3 tabs) and the website QuickstartTabs (the bash/PowerShell envLine).

Re-requesting review. Thanks for catching both before they hit users. 🙏

@changliu2
changliu2 requested a review from jakepresent June 10, 2026 21:23

@tangym tangym left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree pip-first is the right call. One concern: PyPI is published from GitHub releases, so the wheel will lag main between releases, and we don't have a release cadence yet, so a doc on main referencing a not-yet-released feature will make pip look broken. Could we either agree on a cadence (even loose, e.g. tag on user-visible features) or keep a short "want the latest from main?" pointer to the source install next to the pip command until cadence is settled? Also please rebase, currently conflicting with main.

changliu2 and others added 2 commits July 9, 2026 18:17
…started

Make `pip install assert-ai` the canonical install and give every onboarding
shape a pip-only, zero-clone runnable path. The previous docs only showed an
editable `git clone` + `pip install -e ".[otel,langgraph]"`, which is heavy
boilerplate, invisible to PyPI download telemetry, and doesn't match the
published `assert-ai` package.

Bundled examples:
- New `assert_ai/_examples/` subpackage ships two runnable configs in the wheel
  (configs + agent code only; no result artifacts):
  - health-assistant — Prompt Agent (system_prompt); runs on base install
  - travel-planner-langgraph — LangGraph + OTel trace capture; needs [langgraph,otel]
- `assert-ai run --example <name>` resolves bundled configs; no value / unknown
  name lists the catalog with per-example install hints; `--config` and
  `--example` are mutually exclusive.
- Wheel stays 0.31 MB; verified clean-venv install resolves --example from
  site-packages.

3-path quickstart (README + docs/getting-started.md + website):
- Hero install is now `pip install assert-ai`.
- Three paths: "I have an agent" (connect via auto_trace + target.callable,
  [otel]), "I have the agent spec" (--example health-assistant), "Help me
  start" (assert-ai init).
- Website: real <QuickstartTabs> client component (3 tabs + macOS/Linux <-> Windows
  toggle) injected by MarkdownContent via a <!--quickstart-tabs--> sentinel;
  GitHub renders the same paths as Markdown fallback.
- Extras are always quoted ("assert-ai[otel]") to survive zsh/PowerShell globbing.
- AGENTS.md setup commands updated to the published-package path with explicit
  per-scenario extras.
- Fixed a stale microsoft/ASSERT docs blob URL -> responsibleai/ASSERT.

Tests: 7 new CLI/registry tests; wheel build verified (configs ship, artifacts
excluded); website tsc + eslint + next build pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…eview)

1. assert-ai run --example no longer relies on Click flag_value (broke on Click 8.3.1 with 'Option --example requires an argument'). --example is now a plain value option; the catalog is discoverable via a new 'assert-ai examples' command that works across the supported Click range. Unknown name and the no-config/no-example paths still print the catalog.

2. Pip-only quickstart no longer tells zero-clone users to 'cp .env.example .env' (that file doesn't exist after a bare pip install). The three pip-only paths now instruct setting AZURE_API_KEY/AZURE_API_BASE as env vars or creating a .env — which the runner already auto-loads via find_dotenv(usecwd=True). The clone-based 'develop from source' path keeps cp .env.example .env.

Updates: cli.py, tests/test_cli.py (examples-command + bare-flag-errors tests), docs/getting-started.md, website QuickstartTabs.tsx. pytest test_cli.py green; website tsc+eslint clean.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@changliu2
changliu2 force-pushed the changliu2/pip-quickstart-3tabs branch from ca67d8c to 893b350 Compare July 9, 2026 22:19
@changliu2

Copy link
Copy Markdown
Collaborator Author

Rebased onto main and re-verified both of Jake's fixes hold (bare --example now errors cleanly via the new examples command, zero-clone quickstart no longer references .env.example).

On the cadence question — added a one-line pointer in "Develop from source" calling it out explicitly as the path for "just landed on main, not on PyPI yet" (62c8363). I've asked Aaron to work out an actual release-cadence proposal separately, so that doesn't need to hold up this PR.

@changliu2

Copy link
Copy Markdown
Collaborator Author

@jakepresent pushed fixes for both issues -- bare --example now surfaces the catalog via a dedicated assert-ai examples command (Click can't make the bare-flag form work cleanly across the supported version range, so this is the alternative you offered), and the zero-clone quickstart no longer references .env.example. Rebased onto main, 19/19 CLI tests pass. Mind re-reviewing when you get a chance?

@AaronAspinwall123

Copy link
Copy Markdown
Collaborator

@changliu2 closing for hygiene. Please reopen if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants