polyman/DOCUMENTATION.md at master · HamzaHassanain/polyman

Complete Technical Reference for Polyman CLI Tool

A TypeScript-based CLI tool for Codeforces problem setters that automates problem preparation workflows including test generation, validation, solution verification, and checker integration.

Architecture Overview
Core Components
Type System
Action Layer
Helper Modules
Execution Engine
Formatter System
Configuration Schema
Compilation Pipeline
Validation System
Solution Testing
Checker Integration
Generator System
Error Handling
Polygon Integration
File Structure
Development Guide
API Reference
Implementation Notes

Architecture Overview

Polyman follows a layered architecture:

┌─────────────────────────────────────┐
│     CLI Layer (cli.ts)              │
│     Commander.js command parsing    │
└─────────────────────────────────────┘
              ↓
┌─────────────────────────────────────┐
│     Action Layer (actions.ts)       │
│     High-level workflow orchestration│
└─────────────────────────────────────┘
              ↓
┌─────────────────────────────────────┐
│     Helper Layer (helpers/)         │
│     Domain-specific logic modules   │
└─────────────────────────────────────┘
              ↓
┌─────────────────────────────────────┐
│     Execution Engine (executor.ts)  │
│     Process management & I/O        │
└─────────────────────────────────────┘
              ↓
┌─────────────────────────────────────┐
│     Formatter (formatter.ts)        │
│     Terminal output styling         │
└─────────────────────────────────────┘

Core Components

The CLI entry point is src/cli.ts. It uses Commander.js to parse argv and delegate to action functions in src/actions.ts. See GUIDE.md - CLI Commands Reference for user-facing command descriptions, or read src/cli.ts for the wiring.

Type System

All TypeScript types live in src/types.d.ts. The key shapes are ConfigFile (the on-disk Config.json schema), LocalSolution, LocalGenerator, LocalChecker, LocalValidator, LocalTestset, plus SolutionTag and the source-type unions for C++/Java/Python. See src/types.d.ts for the full definitions and JSDoc, or GUIDE.md - Configuration Reference for usage.

Action Layer

Action functions in src/actions.ts orchestrate workflows by composing helpers. Each action is structured as a sequence of independent step… calls (see src/steps.ts) which compile, run, and verify components. See GUIDE.md - CLI Commands Reference for what each action does from the user perspective.

Helper Modules

Each helper module owns a single domain. Function-level documentation is on TypeDoc; this section lists the responsibility of each module so contributors know where to look.

src/helpers/utils.ts — Cross-cutting utilities: config file reading, directory management, C++ and Java compilation, error logging, and the shared exit/throw helpers used by every action.
src/helpers/generator.ts — Generator compilation and test file production. Drives Polygon-format scripts: parses, resolves to concrete test indices, then runs each generator (or copies each manual test) into the testset's output directory.
src/helpers/validator.ts — Validator compilation, validation of generated test inputs, and validator self-testing against validator_tests.json.
src/helpers/checker.ts — Checker compilation (custom or standard from assets/checkers/), checker invocation, and checker self-testing. Owns the verdict mapping from checker stdout to CheckerVerdict.
src/helpers/solution.ts — Solution compilation, execution against a testset, and verdict comparison. Responsible for enforcing the solution-tag contract (e.g. WA solutions must produce WA on at least one test).
src/helpers/create-template.ts — Materialises the bundled template/ tree into a new problem directory when polyman new runs.
src/helpers/testlib-download.ts — Fetches testlib.h from the upstream GitHub repo for the polyman download-testlib command.
src/helpers/testset.ts — Testset and group resolution: given filters from CLI options, returns the matching testsets, groups, and indices.
src/helpers/script-parser.ts — Parses Polygon-format generation scripts (gen [args] > N|$|{indices}, FreeMarker comments and <#list> loops, <#-- @group X --> headers) and resolves them, together with manualTests[], into a flat list of test-production instructions with concrete indices.
src/helpers/remote/ — Polygon integration. pulling.ts and pushing.ts implement the per-component download/upload steps; utils.ts handles credentials, line-ending normalization, and problem-id extraction; viewer.ts formats polyman remote view output.

For function-level docs see TypeDoc at https://hamzahassanain.github.io/polyman/.

Execution Engine

Executor: `src/executor.ts`

Low-level process execution with timeout and memory limit support.

`executor.executeWithTimeout(command: string, args: string[], timeoutMs: number, memoryLimitMB: number): Promise<ExecutionResult>`

Executes command with resource limits.

Parameters:

command: Executable path or command
args: Command-line arguments
timeoutMs: Maximum execution time
memoryLimitMB: Maximum memory usage

Returns:

interface ExecutionResult {
  stdout: string;
  stderr: string;
  exitCode: number;
  timedOut: boolean;
  memoryExceeded: boolean;
}

Platform Support:

Linux: Uses ulimit for memory limiting
macOS: Memory limiting not supported (warning shown)
Windows: Memory limiting not supported (warning shown)

Implementation:

// Linux memory limiting
if (process.platform === 'linux' && memoryLimitMB > 0) {
  const memoryLimitKB = memoryLimitMB * 1024;
  command = `ulimit -v ${memoryLimitKB} && ${command}`;
}

// Spawn process with timeout
const child = spawn(command, args, { shell: true });
const timeout = setTimeout(() => {
  child.kill();
  timedOut = true;
}, timeoutMs);

Formatter System

Formatter Class: `src/formatter.ts`

Terminal output styling with Codeforces theme.

The actual colors come from src/theme.ts: a dark theme and a light theme. A non-empty NO_COLOR turns all colors off. POLYMAN_THEME=dark or POLYMAN_THEME=light forces a theme. When POLYMAN_THEME is unset or auto, resolveTheme() detects the terminal background with src/helpers/terminal-background.ts: first COLORFGBG, then an OSC 11 query followed by a DA1 query (the DA1 answer tells a terminal without OSC 11 support apart from a slow one), with a 100 ms timeout, and dark as the fallback. Detection is skipped when stdin or stdout is not a TTY, when chalk has no colors, when TERM=dumb, for background jobs, and on Windows.

Detection is async, but fmt is created when the module loads, so src/cli.ts awaits resolveTheme() once before program.parse() and applies the result with fmt.setTheme(theme) and printComprehensiveHelp(program, theme). The sync getTheme(env) does no detection (unset means dark); it is the default for new Formatter() and for tests.

Colors:

Primary: #1E88E5 (blue)
Error: #FF6B6B (red)
Success: #4CAF50 (green)
Warning: #FFC107 (yellow)

Output Methods

section(title: string)

Prints section header with box.

╭─────────────────────────────────╮
│  📁 SECTION TITLE               │
╰─────────────────────────────────╯

step(stepNumber: number, title: string)

Prints numbered step header.

╭─ STEP 1: Creating Directory

stepComplete(message: string)

Prints step completion (╰─ ✓ Done).

Other methods: success, error, warning, info, log, successBox, errorBox.

Utility Methods

primary(text: string): Blue color
highlight(text: string): Cyan color
dim(text: string): Dimmed text
successIcon(): ✓ icon
errorIcon(): ✗ icon
warningIcon(): ⚠ icon
infoIcon(): ℹ icon

Configuration Schema

The on-disk Config.json schema is described in GUIDE.md - Configuration File Reference for users, and defined as TypeScript interfaces in src/types.d.ts (ConfigFile).

Compilation Pipeline

Language Support

C++:

g++ -O2 -std=<cppStandard> <compiler.flags...> -iquote <problem-root> -o output source.cpp

The executable comes from POLYMAN_CXX, then compiler.cpp in Config.json, then g++ (resolveCppCompiler in src/helpers/utils.ts). The flags come from resolveCppFlags: a -std= item in compiler.flags replaces -std=<cppStandard> and an -O item replaces -O2. The resolved executable and flags are part of the compile-cache key.

The problem root (the current working directory) is passed via -iquote so sources in subdirectories such as generators/ can #include "testlib.h" from the root-level copy without a system-wide install. Every path and argument that reaches the shell is quoted with quoteShellArgument (src/helpers/shell.ts), so problem directories containing spaces or shell metacharacters work.

Java:

javac source.java
java ClassName < input.txt > output.txt

javac comes from POLYMAN_JAVAC, then compiler.javac, then javac (resolveJavaCompiler). java is always taken from PATH.

Python:

python3 source.py < input.txt > output.txt

Compilation Flow

Detect language from file extension
Call appropriate compiler function:
- .cpp → compileCPP
- .java → compileJava
- .py → Return interpreter command
Handle compilation errors
Return executable path or command

Validation System

The validator is a C++ program built against testlib that reads a test input on stdin and exits 0 for VALID or 3 for INVALID (any other code is an error). The validation workflow compiles the validator once and runs it over each test file in the requested testset, logging per-test results. Validator self-tests live in validator/validator_tests.json as {stdin, expectedVerdict} pairs and are exercised by polyman test validator to verify the validator agrees with its own contract before being used on generated tests.

Solution Testing

Each solution is compiled, then run against every test in the active testset under the configured time and memory limits. The runner inspects the first line of the captured output for the sentinel verdicts Time Limit Exceeded, Memory Limit Exceeded, and Runtime Error; otherwise it invokes the checker on the (input, output, answer) triple to obtain OK/WA/PE. Aggregated VerdictTracker flags (didWA, didTLE, didMLE, didRTE) are then matched against the solution's declared SolutionTag so that, e.g., a WA solution must fail at least one test and a MA solution must pass them all.

Checker Integration

Standard Checkers

Standard checkers ship in assets/checkers/*.cpp (vendored testlib checkers — do not modify). Run polyman list checkers for the runtime list, or see GUIDE.md - Standard Checkers.

Custom Checkers

Location: checker/chk.cpp

Interface:

int main(int argc, char* argv[]) {
  registerTestlibCmd(argc, argv);

  // Read input
  // Read output (ouf)
  // Read answer (ans)

  // Compare and return verdict:
  // quitf(_ok, "Correct");
  // quitf(_wa, "Wrong Answer");
  // quitf(_pe, "Presentation Error");
}

Checker Execution

Command:

./checker <input_file> <output_file> <answer_file>

Output Parsing:

First word determines verdict
ok/OK → OK
wrong/WA → WA
presentation/PE → PE

Generator System

Generators are testlib C++ programs that take parameters as positional argv[] and write the test content to stdout. The pipeline is:

Parse — script-parser.ts reads the testset's generatorScript.script (or scriptFile) as Codeforces-Polygon syntax: each non-comment line is gen-name [args...] > target where target is N, $ (smallest unused), or {1-3,7} (multi-output). FreeMarker <#-- comments --> and <#list a..b as i> ... ${i} ... </#list> blocks are supported, and <#-- @group <name> --> headers tag every following line.
Resolve — script-parser.ts:resolveTests merges parsed lines with the testset's manualTests[], assigning concrete Polygon indices to every $. Manual indices and explicit indices reserve slots; $ walks the smallest free integer. Duplicate indices are an error.
Compile + run — generator.ts:executeResolvedTests compiles each referenced generator once and, per resolved test, either copies the manual .in file into testsets/<name>/test<index>.txt, runs the generator with stdout redirected to that file, or (for > {indices} lines) runs the generator with cwd set to the testset directory and verifies it wrote the promised files itself.

Generator names with extensions (gen.exe, gen.cpp) are rejected — Polygon strips them on the server side, so polyman keeps the same constraint.

Error Handling

Polyman provides comprehensive error handling with formatted output and proper exit codes.

Error Flow

Catch Error: In action or helper function
Log Error: Via logError
Handle Error:
- Exit: logErrorAndExit
- Throw: logErrorAndThrow

Error Types

Configuration Errors:

Config.json not found
Invalid JSON
Missing required fields
No main-correct solution

Compilation Errors:

Source file not found
Compilation failed
Invalid syntax

Execution Errors:

TLE (timeout)
MLE (memory exceeded)
RTE (runtime error)
Exit code ≠ 0

Validation Errors:

Test file not found
Validator failed
Invalid test input

Checker Errors:

Checker compilation failed
Unexpected verdict format
Answer file not found

Polygon Integration

Polyman provides comprehensive integration with the Codeforces Polygon system through a type-safe TypeScript SDK and remote operations.

Polygon SDK

Location: src/polygon.ts

The Polygon SDK is a complete TypeScript implementation of the Polygon API v1, providing type-safe methods for all Polygon operations.

SDK Architecture

class PolygonSDK {
  private apiKey: string;
  private apiSecret: string;
  private baseUrl: string;

  constructor(config: PolygonConfig) {
    this.apiKey = config.apiKey;
    this.apiSecret = config.apiSecret;
    this.baseUrl = config.baseUrl || 'https://polygon.codeforces.com/api';
  }

  // 54+ API methods for complete Polygon integration
}

The SDK exposes 50+ methods grouped under Problem Management, Statements, Solutions, Checker & Validator, Generators, Tests, Metadata, Packages, and Contests. See src/polygon.ts for the full method signatures and JSDoc, or TypeDoc for the rendered API.

API Authentication

Polygon API uses SHA-512 signature-based authentication.

Authentication Flow

API Credentials:

{
  apiKey: 'your-api-key',
  apiSecret: 'your-api-secret'
}

Request Signing:

// Generate 6-character random prefix
const rand = generateRandomString(6);

// Sort parameters alphabetically
const sortedParams = Object.keys(params).sort();

// Build signature string
const sigString = `${rand}/${methodName}?${paramString}#${apiSecret}`;

// Compute SHA-512 hash
const hash = crypto.createHash('sha512').update(sigString).digest('hex');

// Final signature
const apiSig = rand + hash;

Request Parameters:
- apiKey - Your API key
- time - Current Unix timestamp
- apiSig - Computed signature
- Method-specific parameters

API Call:

POST https://polygon.codeforces.com/api/{method}
Content-Type: application/x-www-form-urlencoded

apiKey=xxx&time=xxx&apiSig=xxx&problemId=xxx&...

Security Considerations

API credentials stored in ~/.polyman/credentials.json
File permissions set to 600 (owner read/write only)
Credentials never logged or displayed
Signature includes timestamp to prevent replay attacks
Each request has unique random prefix

Remote Operations Flow

Pull Operation Flow

User Command
    ↓
polyman remote pull 123456 ./my-problem
    ↓
remotePullProblemAction
    ↓
┌────────────────────────────────────┐
│ Step 1: Read Credentials           │
│ - Load from ~/.polyman/            │
└────────────────────────────────────┘
    ↓
┌────────────────────────────────────┐
│ Step 2: Initialize SDK              │
│ - Create PolygonSDK instance        │
│ - Configure authentication          │
└────────────────────────────────────┘
    ↓
┌────────────────────────────────────┐
│ Step 3: Fetch Problem Info          │
│ - Get limits, I/O files             │
│ - Get problem metadata              │
└────────────────────────────────────┘
    ↓
┌────────────────────────────────────┐
│ Step 4: Create Directory            │
│ - Create problem directory          │
│ - Create subdirectories             │
└────────────────────────────────────┘
    ↓
┌────────────────────────────────────┐
│ Step 5: Download Components         │
│ ┌──────────────────────────────┐   │
│ │ Solutions (parallel)         │   │
│ │ Checker + tests              │   │
│ │ Validator + tests            │   │
│ │ Generators                   │   │
│ │ Statements (all languages)   │   │
│ │ Tests (parallel fetch)       │   │
│ │ Metadata                     │   │
│ └──────────────────────────────┘   │
│ - Normalize line endings (→ Unix)   │
└────────────────────────────────────┘
    ↓
┌────────────────────────────────────┐
│ Step 6: Generate Config.json        │
│ - Build complete configuration      │
│ - Include all metadata              │
└────────────────────────────────────┘
    ↓
Success! Problem ready for local work

Push Operation Flow

User Command
    ↓
polyman remote push . ./my-problem
    ↓
remotePushProblemAction
    ↓
┌────────────────────────────────────┐
│ Step 1: Read Credentials           │
└────────────────────────────────────┘
    ↓
┌────────────────────────────────────┐
│ Step 2: Initialize SDK              │
└────────────────────────────────────┘
    ↓
┌────────────────────────────────────┐
│ Step 3: Read Config.json            │
│ - Get problem ID                    │
│ - Get all configurations            │
└────────────────────────────────────┘
    ↓
┌────────────────────────────────────┐
│ Step 4: Update Problem Info         │
│ - Upload time/memory limits         │
│ - Upload I/O file settings          │
└────────────────────────────────────┘
    ↓
┌────────────────────────────────────┐
│ Step 5-11: Upload Components        │
│ ┌──────────────────────────────┐   │
│ │ Solutions with tags          │   │
│ │ Checker + set active         │   │
│ │ Validator + tests + set      │   │
│ │ Generators                   │   │
│ │ Statements (all languages)   │   │
│ │ Metadata                     │   │
│ │ Testsets:                    │   │
│ │   - Clear existing           │   │
│ │   - Enable groups            │   │
│ │   - Upload tests (parallel)  │   │
│ │   - Upload script            │   │
│ └──────────────────────────────┘   │
│ - Normalize line endings (→ Win)    │
└────────────────────────────────────┘
    ↓
Success! Changes uploaded to Polygon
(Don't forget to commit!)

Performance Optimizations

Parallel Test Operations:

Both pull and push operations fetch/upload manual tests in parallel for significant performance improvements:

Pull (Parallel Fetch):

// Fetch test metadata without inputs (fast)
const tests = await sdk.getTests(problemId, testsetName, true);
const manualTests = tests.filter(t => t.manual);

// Fetch all inputs in parallel
const promises = manualTests.map(test =>
  sdk.getTestInput(problemId, testsetName, test.index)
);
const results = await Promise.all(promises);

Push (Parallel Upload):

// Create upload promises for all manual tests
const promises = manualTests.map(test =>
  sdk.saveTest(problemId, testsetName, index, input, options)
);

// Upload all in parallel
await Promise.all(promises);

Performance Impact:

50 manual tests: ~50 seconds sequential → ~2 seconds parallel
100 manual tests: ~100 seconds sequential → ~3 seconds parallel

File Structure

See GUIDE.md - Directory Structure for the canonical layout of a Polyman problem directory and the bundled template tree.

Development Guide

Building from Source

git clone https://github.com/HamzaHassanain/polyman.git
cd polyman
npm install
npm run build
npm link

Code Structure

Entry Points:

src/cli.ts - CLI commands
src/actions.ts - Action functions

Core Logic:

src/helpers/ - Domain modules
src/executor.ts - Process execution
src/formatter.ts - Output formatting

Types:

src/types.d.ts - TypeScript type definitions

Assets:

assets/checkers/ - Standard checkers
template/ - Problem template

Adding New Features

New Command:

Add action function in src/actions.ts
Register command in src/cli.ts
Document in README and DOCUMENTATION

New Helper Module:

Create file in src/helpers/
Export functions
Import in src/actions.ts

New Solution Type:

Add type to src/types.d.ts
Handle in getExpectedCheckerVerdict
Update verdict validation in src/helpers/solution.ts

Testing

# Lint code
npm run lint

# Build
npm run build

# Manual testing
polyman new test-problem
cd test-problem
# Test commands...

ESLint Configuration

See eslint.config.js for linting rules.

Key Rules:

TypeScript strict mode
Prettier integration
No unused variables (except prefixed with _)
No explicit any (warning)

API Reference

TypeDoc generates the full API reference: https://hamzahassanain.github.io/polyman/. Key entry points: src/cli.ts (commands), src/actions.ts (orchestration), src/polygon.ts (Polygon SDK), src/types.d.ts (types).

Implementation Notes

Important considerations for development, deployment, and platform-specific behavior.

Performance Considerations

Parallel Execution: Currently sequential; could parallelize test generation and solution execution
Caching: Compiled executables are not cached between runs
Memory Limiting: Only supported on Linux via ulimit

Platform Differences

Linux:

Full support for memory limiting
Recommended platform

macOS:

Memory limiting not supported
Warning shown when memory limit specified

Windows:

Memory limiting not supported
Requires MinGW or WSL for C++ compilation

Security

Command Injection: Uses spawn with shell: true; sanitize user input if accepting external configs
File System: Direct file operations; validate paths to prevent directory traversal

Future Enhancements

Parallel test execution
Compiled executable caching
Windows native memory limiting
Progress bars for long operations
Config validation schema
Interactive mode

License

MIT License - See LICENCE file for details.

This technical documentation is auto-generated alongside TypeDoc API documentation. For the latest version, visit the online documentation.

FilesExpand file tree

DOCUMENTATION.md

Latest commit

History