testing-prpm-cli

Test PRPM CLI commands against a local registry by building the package, setting the registry URL, and invoking the CLI directly. Includes comprehensive cross-format conversion testing patterns and contract testing to verify documented behavior matches implementation.

• Testing new CLI commands or features
• Debugging CLI behavior
• Verifying CLI changes before committing
• Testing against local registry data
• Validating cross-format conversions
• Testing new format converters
• Contract testing: verifying documented behavior matches implementation

| Step | Command |

|------|---------|

| Build CLI | npm run build --workspace=packages/cli |

| Start local registry | npm run dev --workspace=packages/registry |

| Set local registry | export PRPM_REGISTRY_URL=http://127.0.0.1:3111 |

| Run CLI directly | node /Users/khaliqgant/Projects/prpm/app/packages/cli/dist/index.js |

| Run via npm link | prpm (after linking) |

1. Build the CLI (Required First Step)

Always rebuild before testing to ensure you're testing current code:

```bash

npm run build --workspace=packages/cli

```

2. Start Local Registry

Start the registry server in background:

```bash

npm run dev --workspace=packages/registry &

sleep 3

lsof -i :3111 # Verify it's running

```

3. Configure Local Registry

Point CLI to local registry instead of production:

```bash

export PRPM_REGISTRY_URL=http://127.0.0.1:3111

```

Verify it's set:

```bash

echo $PRPM_REGISTRY_URL

```

4. Run CLI Commands

Option A: Direct invocation (recommended for testing)

```bash

node /Users/khaliqgant/Projects/prpm/app/packages/cli/dist/index.js search typescript

node /Users/khaliqgant/Projects/prpm/app/packages/cli/dist/index.js install some-package

```

Option B: npm link (for interactive testing)

```bash

cd packages/cli

npm link

prpm search typescript

```

Use Self-Improving Skill to Find Test Packages

Before testing conversions, use the self-improving skill to download a diverse set of packages:

```bash

# Search for packages of different types

node $CLI search "claude" --limit 10

node $CLI search "cursor" --limit 10

node $CLI search "agent" --limit 10

node $CLI search "skill" --limit 10

```

Create Test Directory and Install Diverse Packages

```bash

mkdir -p /tmp/prpm-conversion-tests

cd /tmp/prpm-conversion-tests

export PRPM_REGISTRY_URL=http://127.0.0.1:3111

CLI="/Users/khaliqgant/Projects/prpm/app/packages/cli/dist/index.js"

# Install packages of different subtypes

node $CLI install @prpm/agent-builder-skill --as claude # Skill

node $CLI install @prpm/creating-cursor-rules --as cursor # Cursor Rule

node $CLI install @camoneart/context-engineering-agent --as claude # Agent

```

Supported Formats (CLI_SUPPORTED_FORMATS)

Test conversions across ALL supported formats:

| Format | Description |

|--------|-------------|

| cursor | Cursor IDE rules (.mdc) |

| claude | Claude Code (skills, agents, commands) |

| windsurf | Windsurf rules |

| continue | Continue rules |

| copilot | GitHub Copilot instructions |

| kiro | Kiro steering files |

| agents.md | Agents.md format |

| gemini | Gemini CLI extensions |

| ruler | Ruler format |

| zed | Zed editor extensions |

| opencode | OpenCode rules |

| aider | Aider conventions |

| trae | Trae rules |

| replit | Replit agent rules |

| zencoder | ZenCoder rules |

| droid | Factory/Droid rules |

Conversion Test Matrix

Run comprehensive conversion tests:

```bash

cd /tmp/prpm-conversion-tests

mkdir -p /tmp/conversions

CLI="/Users/khaliqgant/Projects/prpm/app/packages/cli/dist/index.js"

# Claude Skill → All formats

for format in cursor windsurf kiro gemini zed continue copilot opencode aider trae replit zencoder droid; do

node $CLI convert .claude/skills/*/SKILL.md --to $format -o /tmp/conversions/skill-to-$format.md 2>&1

done

# Cursor Rule → Multiple formats

for format in claude windsurf gemini zed; do

node $CLI convert .cursor/rules/*.mdc --to $format -o /tmp/conversions/cursor-to-$format.md 2>&1

done

# Claude Agent → Multiple formats

for format in cursor gemini windsurf; do

node $CLI convert .claude/agents/*.md --to $format -o /tmp/conversions/agent-to-$format.md 2>&1

done

```

Round-Trip Testing

Verify content preservation through round-trip conversions:

```bash

# Claude → Cursor → Claude

node $CLI convert .claude/skills/*/SKILL.md --to cursor -o /tmp/conversions/step1-cursor.mdc

node $CLI convert /tmp/conversions/step1-cursor.mdc --to claude -o /tmp/conversions/step2-claude.md

# Compare file sizes (expect some reduction but not dramatic)

wc -c .claude/skills/*/SKILL.md /tmp/conversions/step1-cursor.mdc /tmp/conversions/step2-claude.md

```

Validation Checklist

For each conversion, verify:

• [ ] Command succeeds - Exit code 0, no errors
• [ ] Output file created - File exists at specified path
• [ ] Content preserved - Core markdown structure intact
• [ ] Format-specific frontmatter - Correct fields for target format
• [ ] File size reasonable - Not truncated (compare with source)

Expected File Sizes

| Conversion Type | Expected Size Ratio |

|----------------|---------------------|

| Claude → Cursor | ~95-100% |

| Claude → Windsurf | ~95-100% |

| Claude → Gemini | ~95-100% |

| Claude → OpenCode | May be smaller (format limits) |

| Claude → Droid | May be smaller (format limits) |

| Round-trip | ~50-70% (metadata loss expected) |

Test Report Template

Document results in this format:

```markdown

Test Setup

• Registry: Local (port 3111)
• Test packages: [list installed packages]

Results

| Source | Target | Status | Output Size |

|--------|--------|--------|-------------|

| Claude Skill | Gemini | Pass/Fail | X bytes |

| ... | ... | ... | ... |

Observations

• [Note any issues, truncations, or unexpected behavior]

```

Contract testing ensures documented behavior matches implementation. This is the most important type of testing for features with configurable behavior.

Why Contract Testing Matters

The eager/lazy loading bug is a case study: documentation described a precedence chain (CLI > file > package > default), but implementation only handled CLI flags. Tests passed because they only tested the CLI flag path. Contract testing would have caught this.

Contract Testing Principles

1. Test EVERY documented behavior path, not just the happy path
2. Test precedence chains completely - if docs say "A > B > C > default", test all 4 cases
3. Test behavior WITHOUT flags - verify defaults and configuration-driven behavior
4. Test WITH and WITHOUT explicit settings - don't assume flag presence

Contract Testing Checklist

For ANY feature with documented behavior:

• [ ] Read the documentation first - what does it claim to do?
• [ ] List all behavior paths - every if/else/default mentioned
• [ ] Create test for each path - one test per documented behavior
• [ ] Test WITHOUT user flags - verify package/config defaults work
• [ ] Test precedence - verify higher-priority overrides lower
• [ ] Verify error messages match - documented errors should occur

Example: Eager/Lazy Loading Contract Tests

Documentation states: "Precedence: CLI flag > package-level > default (lazy)"

Required tests:

```bash

# Setup test directory

mkdir -p /tmp/prpm-contract-tests

cd /tmp/prpm-contract-tests

export PRPM_REGISTRY_URL=http://127.0.0.1:3111

CLI="/Users/khaliqgant/Projects/prpm/app/packages/cli/dist/index.js"

# Test 1: CLI --eager flag (highest priority)

rm -rf .openskills AGENTS.md

node $CLI install @prpm/some-skill --as agents.md --eager

# VERIFY: AGENTS.md contains activation="eager" or priority="0"

grep -q 'activation="eager"\|priority="0"' AGENTS.md && echo "PASS: CLI --eager works" || echo "FAIL: CLI --eager"

# Test 2: CLI --lazy flag overrides package eager

rm -rf .openskills AGENTS.md

# Install a package that has eager:true in prpm.json with --lazy flag

node $CLI install @prpm/eager-package --as agents.md --lazy

# VERIFY: Should be lazy despite package setting

grep -q 'activation="lazy"\|priority="1"' AGENTS.md && echo "PASS: CLI --lazy overrides" || echo "FAIL: CLI --lazy"

# Test 3: Package-level eager (NO CLI flag) - THIS IS THE BUG THAT WAS MISSED

rm -rf .openskills AGENTS.md

# Install a package that has eager:true in its prpm.json WITHOUT --eager flag

node $CLI install @prpm/eager-package --as agents.md

# VERIFY: Should be eager based on package setting

grep -q 'activation="eager"\|priority="0"' AGENTS.md && echo "PASS: Package eager works" || echo "FAIL: Package eager"

# Test 4: Default (lazy) when no flags and no package setting

rm -rf .openskills AGENTS.md

node $CLI install @prpm/normal-skill --as agents.md

# VERIFY: Should be lazy by default

grep -q 'activation="lazy"\|priority="1"' AGENTS.md && echo "PASS: Default lazy works" || echo "FAIL: Default"

```

Contract Test Template

For any new feature, create tests in this format:

```markdown

Documentation Claims:

1. [Claim 1 from docs]
2. [Claim 2 from docs]
3. [Precedence/default behavior from docs]

Test Cases:

| # | Documented Behavior | Test Setup | Expected Result | Pass/Fail |

|---|--------------------|-----------| ----------------|-----------|

| 1 | [Claim 1] | [How to test] | [What to verify] | |

| 2 | [Claim 2] | [How to test] | [What to verify] | |

| 3 | Default behavior | [No flags/config] | [Default result] | |

Results:

• All tests must pass before feature is considered complete
• Document any deviations between docs and implementation

```

Anti-Patterns to Avoid

| Anti-Pattern | Why It Fails | Correct Approach |

|--------------|--------------|------------------|

| Only testing with flags | Misses config/default paths | Test without flags first |

| Assuming documentation is implementation | Docs may describe intent, not reality | Verify each claim with test |

| Testing happy path only | Misses precedence bugs | Test all documented paths |

| Skipping default behavior test | Defaults often broken | Always test "no config" case |

| Not reading docs before testing | Miss documented behaviors | Read docs, list claims, test each |

| Mistake | Symptom | Fix |

|---------|---------|-----|

| Forgetting to build | Old behavior, changes not reflected | Run npm run build --workspace=packages/cli |

| Missing registry env | Commands hit production registry | Set PRPM_REGISTRY_URL before running |

| Stale npm link | Wrong version running | Re-run npm link after rebuilding |

| Local registry not running | Connection refused errors | Start registry: npm run dev --workspace=packages/registry |

| Testing single format only | Miss format-specific bugs | Test ALL formats in CLI_SUPPORTED_FORMATS |

| No round-trip testing | Miss content loss bugs | Always verify round-trip preservation |

| No contract testing | Docs ≠ implementation | Test EVERY documented behavior |

| Only testing with CLI flags | Miss config/default bugs | Test without flags to verify defaults |

```bash

npm run build --workspace=packages/cli && export PRPM_REGISTRY_URL=http://127.0.0.1:3111

```

Then test commands:

```bash

node packages/cli/dist/index.js --help

```

Run converter unit tests:

```bash

# All converter tests

npm run test --workspace=packages/converters

# Specific test files

npm run test --workspace=packages/converters -- --testPathPattern="file-references"

npm run test --workspace=packages/converters -- --testPathPattern="security"

npm run test --workspace=packages/converters -- --testPathPattern="cross-format"

npm run test --workspace=packages/converters -- --testPathPattern="zed"

```