Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/workflows/validate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,14 @@ jobs:
run: pwsh scripts/validate.ps1 -Verbose
- name: Run Bash validator
run: bash scripts/validate.sh --verbose
- name: Check SKILL.md self-containment
shell: pwsh
run: |
$content = Get-Content SKILL.md -Raw
if ($content -match "raw.githubusercontent.com") { Write-Host "PASS: GitHub raw URLs present" } else { Write-Host "WARN: Missing GitHub raw URLs" }
if ($content -match "Top burned words") { Write-Host "PASS: Embedded burned words" } else { Write-Host "WARN: Missing embedded burned words" }
- name: List new scripts
shell: pwsh
run: |
$scripts = @("morph-check.ps1","readability-check.ps1","run-eval.ps1","clause-check.ps1","run-benchmark.sh")
foreach ($s in $scripts) { if (Test-Path "scripts/$s") { Write-Host "OK: $s" } else { Write-Host "MISSING: $s" } }
2 changes: 1 addition & 1 deletion EVAL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# EVAL - Quality Evaluation Framework for HUMAN-AI

> **NEW in v3.0:** ZeroGPT external validator integration. Run `scripts/run-benchmark.ps1` with a ZeroGPT API key to get independent AI detection scores.
> **ZeroGPT external validator integration.** Run `scripts/run-benchmark.ps1` with a ZeroGPT API key to get independent AI detection scores. See also `scripts/readability-check.ps1` and `scripts/morph-check.ps1` for multi-layer validation.

> Load this prompt alongside SKILL.md output to score the humanization result.
> The evaluator is a separate prompt to avoid self-assessment bias.
Expand Down
4 changes: 2 additions & 2 deletions KNOWN_LIMITATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ LLMs cannot reliably count exact words. The rhythm system uses clause-count and

## Language Depth

English, Russian, Ukrainian, and German have the deepest coverage with native-level tone markers and cultural notes. French, Spanish, Portuguese, Italian, and Polish have been enhanced in v4 but may still have gaps compared to the primary four languages. Community contributions from native speakers are welcome.
English, Russian, Ukrainian, and German have the deepest coverage with native-level tone markers and cultural notes. French, Spanish, Portuguese, Italian, and Polish may still have gaps compared to the primary four languages. Community contributions from native speakers are welcome.

## Edge Cases

Expand All @@ -23,7 +23,7 @@ English, Russian, Ukrainian, and German have the deepest coverage with native-le
Results vary significantly between models:
- GPT-4, Claude 3.5+, Gemini 2.0: recommended, consistent results
- Claude 3, GPT-4o-mini, DeepSeek V3: adequate, may need parameter tuning
- Smaller models (<13B parameters): may struggle with the full 6-stage pipeline, produce inconsistent self-evaluation scores, or miss subtle AI markers
- Smaller models (<13B parameters): may struggle with the full 5-stage pipeline, produce inconsistent self-evaluation scores, or miss subtle AI markers

The self-evaluation score (Stage 5.6) is generated by the same LLM that did the rewriting. It is a self-assessment, not an independent verification. Always review output before publishing.

Expand Down
Loading
Loading