Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ jobs:
run: nix develop --command bash -c "mix deps.get"

- name: Unit Tests
run: nix develop --command bash -c "mix test"
run: nix develop --command bash -c "mix test --exclude property"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets still run the properties here, just run them with the smaller max runs value.

Copy link
Contributor Author

@mkaput mkaput Feb 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH I disagree - this will keep the havoc with proptests spuriously breaking random PR unit tests


lint:
runs-on: ubuntu-latest
Expand Down Expand Up @@ -163,4 +163,3 @@ jobs:

- name: Run dialyzer
run: nix develop --command bash -c 'mix dialyzer --format github'

74 changes: 74 additions & 0 deletions .github/workflows/proptest.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
name: Property Tests
on:
schedule:
- cron: "0 0 * * *"
workflow_dispatch:

permissions:
contents: read
issues: write

concurrency:
group: proptest
cancel-in-progress: false

jobs:
proptest:
runs-on: ubuntu-latest
env:
SPITFIRE_PROPERTY_MAX_RUNS: "250000"

steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/nix-installer-action@main
- uses: cachix/cachix-action@v14
with:
name: elixir-tools
authToken: "${{ secrets.CACHIX_AUTH_TOKEN }}"

- uses: actions/cache@v4
with:
path: |
deps
_build
key: ${{ runner.os }}-mix-${{ hashFiles('**/.mise.toml') }}-${{ hashFiles('**/mix.lock') }}
restore-keys: |
${{ runner.os }}-mix-${{ hashFiles('**/.mise.toml') }}-

- name: Install Dependencies
run: nix develop --command bash -c "mix deps.get"

- name: Run property tests
shell: bash
run: |
set -euo pipefail

nix develop --command bash -c "mix test --only property" 2>&1 | tee property-test.log

- name: Build failure report
if: ${{ failure() }}
shell: bash
run: |
{
echo "## Property tests failed"
echo ""
echo "- Workflow run: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
echo "- Branch: \`${{ github.ref_name }}\`"
echo "- Commit: \`${{ github.sha }}\`"
echo "- Trigger: \`${{ github.event_name }}\`"
echo ""
echo "### Log tail"
echo '```text'
tail -n 400 property-test.log || true
echo '```'
} > property-test-failure.md

- name: Open issue on failure
if: ${{ failure() }}
env:
GH_TOKEN: ${{ github.token }}
run: |
gh issue create \
--repo "${{ github.repository }}" \
--title "Property tests failed (run #${{ github.run_number }})" \
--body-file property-test-failure.md
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as I typed in special considerations, I'm afraid of doing this as it might generate too much spam issues. we need to see how this works live

Loading
Loading