Skip to content
Performance Test

Performance Test #283

Sign in to view logs

Performance Test

Performance Test #283

Workflow file for this run

.github/workflows/performance-test.yml at e01035c
name: Performance Test
on:
push:
branches: [ main, dev ]
pull_request:
branches: [ main, dev ]
workflow_dispatch: # Enables manual triggering
schedule:
- cron: '0 2 * * *' # Runs daily at 2 AM UTC
jobs:
performance:
name: "Performance Test"
runs-on: ubuntu-latest
timeout-minutes: 60 # Set a timeout to prevent jobs from running indefinitely
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.10'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install --upgrade -r requirements.txt
python -m pip install -e . # Editable install ensures we test the actual source code
- name: Test Owlery Connectivity
run: |
echo "Testing basic connectivity to Owlery server..."
curl -v --max-time 30 "http://owl.virtualflybrain.org/kbs/vfb/subclasses?object=%3Chttp%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FFBbt_00005106%3E&direct=false&includeDeprecated=false&includeEquivalent=true" | head -20 || echo "Simple query failed or timed out"
echo ""
echo "Testing if server responds at all..."
curl -v --max-time 10 "http://owl.virtualflybrain.org/kbs/vfb" || echo "Server unreachable"
- name: Run Performance Test
run: |
python -m unittest src.test.test_query_performance -v 2>&1 | tee performance_test_output.log
- name: Run Legacy Performance Test
env:
VFBQUERY_CACHE_ENABLED: 'true'
MPLBACKEND: 'Agg'
VISPY_GL_LIB: 'osmesa'
VISPY_USE_EGL: '0'
run: |
python -m unittest -v src.test.term_info_queries_test.TermInfoQueriesTest.test_term_info_performance 2>&1 | tee -a performance_test_output.log
- name: Create Performance Report
if: always() # Always run this step, even if the test fails
run: |
# Create performance.md file
cat > performance.md << EOF
# VFBquery Performance Test Results
**Test Date:** $(date -u '+%Y-%m-%d %H:%M:%S UTC')
**Git Commit:** ${{ github.sha }}
**Branch:** ${{ github.ref_name }}
**Workflow Run:** [${{ github.run_id }}](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})
## Test Overview
This performance test measures the execution time of all implemented VFB queries organized by functionality:
### 1. Term Information Queries
- **Term Info**: Comprehensive term information retrieval with preview data
### 2. Neuron Part & Synaptic Queries
- **NeuronsPartHere**: Neurons with parts overlapping anatomical regions
- **NeuronsSynaptic**: Neurons with synapses in a region
- **NeuronsPresynapticHere**: Neurons with presynaptic terminals in a region
- **NeuronsPostsynapticHere**: Neurons with postsynaptic terminals in a region
### 3. Anatomical Hierarchy Queries
- **ComponentsOf**: Anatomical components of a structure
- **PartsOf**: Parts of an anatomical structure
- **SubclassesOf**: Subclasses of anatomical terms (can be very slow for complex terms)
### 4. Tract/Nerve & Lineage Queries
- **NeuronClassesFasciculatingHere**: Neurons fasciculating with tracts
- **TractsNervesInnervatingHere**: Tracts/nerves innervating neuropils
- **LineageClonesIn**: Lineage clones in neuropils (complex OWL reasoning)
### 5. Image & Developmental Queries
- **ImagesNeurons**: Neuron images in anatomical regions
- **ImagesThatDevelopFrom**: Developmental lineage images
- **epFrag**: Expression pattern fragments
- **ListAllAvailableImages**: All available images for a term
### 6. Connectivity Queries
- **NeuronNeuronConnectivity**: Neuron-to-neuron connectivity
- **NeuronRegionConnectivity**: Neuron-to-region connectivity
- **NeuronInputsTo**: Individual neuron inputs
### 7. Similarity Queries (NBLAST & NeuronBridge)
- **SimilarMorphologyTo**: NBLAST morphological similarity
- **SimilarMorphologyToPartOf**: NBLAST to expression patterns (NBLASTexp)
- **SimilarMorphologyToPartOfexp**: Reverse NBLASTexp
- **SimilarMorphologyToNB**: NeuronBridge matches
- **SimilarMorphologyToNBexp**: NeuronBridge for expression patterns
### 8. Expression & Transcriptomics Queries
- **ExpressionOverlapsHere**: Expression patterns overlapping regions
- **anatScRNAseqQuery**: scRNAseq clusters in anatomy
- **clusterExpression**: Genes expressed in clusters
- **expressionCluster**: Clusters expressing genes
- **scRNAdatasetData**: Cluster data from scRNAseq datasets
### 9. Dataset & Template Queries
- **PaintedDomains**: Template painted anatomy domains
- **DatasetImages**: Images in datasets
- **AllAlignedImages**: Images aligned to templates
- **AlignedDatasets**: Datasets aligned to templates
- **AllDatasets**: All available datasets
### 10. Publication & Transgene Queries
- **TermsForPub**: Terms referencing publications
- **TransgeneExpressionHere**: Transgene expression patterns in regions
## Performance Thresholds
- **Fast queries**: < 1 second (SOLR lookups)
- **Medium queries**: < 3 seconds (Owlery + SOLR)
- **Slow queries**: < 10 seconds (Neo4j + complex processing)
- **Very Slow queries**: < 31 seconds (Complex OWL reasoning - over 30 seconds)
## Test Results
\`\`\`
$(cat performance_test_output.log)
\`\`\`
## Summary
EOF
# Check overall test status
if grep -q "OK" performance_test_output.log || grep -q "Ran.*test" performance_test_output.log; then
echo "✅ **Test Status**: Performance tests completed" >> performance.md
echo "" >> performance.md
# Count successes and failures
TOTAL_TESTS=$(grep -c "^test_" performance_test_output.log || echo "0")
FAILED_TESTS=$(grep -c "FAIL:" performance_test_output.log || echo "0")
ERROR_TESTS=$(grep -c "ERROR:" performance_test_output.log || echo "0")
PASSED_TESTS=$((TOTAL_TESTS - FAILED_TESTS - ERROR_TESTS))
echo "### Test Statistics" >> performance.md
echo "" >> performance.md
echo "- **Total Tests**: ${TOTAL_TESTS}" >> performance.md
echo "- **Passed**: ${PASSED_TESTS} ✅" >> performance.md
echo "- **Failed**: ${FAILED_TESTS} ❌" >> performance.md
echo "- **Errors**: ${ERROR_TESTS} ⚠️" >> performance.md
echo "" >> performance.md
# Extract timing information for key queries
echo "### Query Performance Details" >> performance.md
echo "" >> performance.md
# Extract all timing lines
if grep -q "seconds" performance_test_output.log; then
echo "| Query | Duration | Status |" >> performance.md
echo "|-------|----------|--------|" >> performance.md
# Parse timing information
grep -E "^(get_term_info|NeuronsPartHere|NeuronsSynaptic|NeuronsPresynapticHere|NeuronsPostsynapticHere|ComponentsOf|PartsOf|SubclassesOf|NeuronClassesFasciculatingHere|TractsNervesInnervatingHere|LineageClonesIn|ListAllAvailableImages):" performance_test_output.log | while read line; do
QUERY=$(echo "$line" | sed 's/:.*//')
DURATION=$(echo "$line" | sed 's/.*: \([0-9.]*\)s.*/\1/')
if echo "$line" | grep -q "✅"; then
STATUS="✅ Pass"
else
STATUS="❌ Fail"
fi
echo "| $QUERY | ${DURATION}s | $STATUS |" >> performance.md
done
fi
echo "" >> performance.md
# Overall result
if [ "$FAILED_TESTS" -eq "0" ] && [ "$ERROR_TESTS" -eq "0" ]; then
echo "🎉 **Result**: All performance thresholds met!" >> performance.md
else
echo "⚠️ **Result**: Some performance thresholds exceeded or tests failed" >> performance.md
echo "" >> performance.md
echo "Please review the failed tests above. Common causes:" >> performance.md
echo "- Network latency to VFB services" >> performance.md
echo "- SOLR/Neo4j/Owlery server load" >> performance.md
echo "- First-time cache population (expected to be slower)" >> performance.md
fi
else
echo "❌ **Test Status**: Performance tests failed to run properly" >> performance.md
echo "" >> performance.md
echo "Please check the test output above for errors." >> performance.md
fi
echo "" >> performance.md
echo "---" >> performance.md
echo "" >> performance.md
echo "## Historical Performance" >> performance.md
echo "" >> performance.md
echo "Track performance trends across commits:" >> performance.md
echo "- [GitHub Actions History](https://github.com/${{ github.repository }}/actions/workflows/performance-test.yml)" >> performance.md
echo "" >> performance.md
echo "---" >> performance.md
echo "*Last updated: $(date -u '+%Y-%m-%d %H:%M:%S UTC')*" >> performance.md
# Also add to GitHub step summary
echo "## Performance Test Report" >> $GITHUB_STEP_SUMMARY
echo "Performance results have been saved to performance.md" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
cat performance.md >> $GITHUB_STEP_SUMMARY
- name: Commit Performance Report
if: always()
run: |
git config --local user.email "action@github.com"
git config --local user.name "GitHub Action"
git add performance.md
git diff --staged --quiet || git commit -m "Update performance test results [skip ci]"
- name: Push Performance Report
if: always()
uses: ad-m/github-push-action@master
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
branch: ${{ github.ref }}