Restore missing ArtiFinder mention on homepage tagline

src/_data/navigation.yml

@@ -10,8 +10,6 @@ main:
     url: institution_rankings.html
   - title: Systems vs. Security
     url: overview.html
-  - title: ArtiFinder Discovery
-    url: artifinder.html
   - title: Geographic Statistics
     url: statistics/
   - title: Profiles
@@ -82,6 +80,25 @@ main:
     url: security/repo_stats.html
 - title: Methodology
   url: methodology.html
+  children:
+  - title: Overview
+    url: methodology.html
+  - title: Data Collection
+    url: methodology.html#data-collection
+  - title: Pipeline
+    url: methodology.html#pipeline
+  - title: Author Metrics
+    url: methodology.html#author-metrics
+  - title: Institution Metrics
+    url: methodology.html#institution-metrics
+  - title: Repository Statistics
+    url: methodology.html#repository-statistics
+  - title: Search Keywords
+    url: methodology.html#search-keywords
+  - title: How to Cite
+    url: methodology.html#how-to-cite
+  - title: ArtiFinder Discovery
+    url: artifinder.html
 - title: Contribute
   url: about.html
   children:

src/about.md

@@ -20,7 +20,7 @@ AE results and committee lists live in the upstream community sites. Contribute
 
 - [sysartifacts.github.io](https://github.com/sysartifacts/sysartifacts.github.io) — systems conferences (OSDI, SOSP, ATC, EuroSys, FAST, …)
 - [secartifacts.github.io](https://github.com/secartifacts/secartifacts.github.io) — security conferences (USENIX Security, CCS, NDSS, S&P, …)
-- [ArtiFinder-Data](https://github.com/DistriNet/ArtiFinder-Data) — automatically discovered artifact links (open to manual corrections via pull requests). These links are unverified, carry no badges, and never affect ReproDB scores; see the [ArtiFinder discovery page](/artifinder.html) and [methodology](/methodology.html#artifinder-discovered-artifacts).
+- [ArtiFinder-Data](https://github.com/DistriNet/ArtiFinder-Data) — automatically discovered artifact links (open to manual corrections via pull requests). These links are unverified, carry no badges, and never affect ReproDB scores; see the [ArtiFinder discovery page](/methodology/artifinder.html) and [methodology](/methodology/data-collection.html#artifinder-discovered-artifacts).
 
 ### Suggest New Analyses
 

src/index.md

@@ -8,7 +8,7 @@ skip_chartjs: true
 <div id="search-hero" class="rdb-search-hero">
   <div class="rdb-search-hero-inner">
     <p class="rdb-search-tagline">
-      Research artifacts and artifact evaluation drive reproducibility and scientific impact. <strong>ReproDB</strong> aggregates and surfaces artifact evaluation outcomes across major <a href="{{ '/security/' | relative_url }}">security</a> and <a href="{{ '/systems/' | relative_url }}">systems</a> conferences, recognizing the contributions of both artifact authors and artifact evaluation service.
+      Research artifacts and artifact evaluation drive reproducibility and scientific impact. <strong>ReproDB</strong> aggregates and surfaces artifact evaluation outcomes across major <a href="{{ '/security/' | relative_url }}">security</a> and <a href="{{ '/systems/' | relative_url }}">systems</a> conferences, and also includes automatically discovered artifact links from <a href="{{ '/artifinder.html' | relative_url }}">ArtiFinder</a>. The site recognizes the contributions of both artifact authors and artifact evaluation service.
     </p>
     <div id="search-container">
       <div class="rdb-search-box-wrap">

src/artifinder.md → src/methodology/artifinder.md

@@ -1,6 +1,6 @@
 ---
 title: "ArtiFinder — Automatically Discovered Artifacts"
-permalink: /artifinder.html
+permalink: /methodology/artifinder.html
 ---
 
 [ArtiFinder](https://github.com/DistriNet/ArtiFinder) scrapes conference papers directly and identifies links to their artifacts, independently of any artifact-evaluation (AE) process. This page summarises the ArtiFinder links that ReproDB ingests from the [ArtiFinder-Data](https://github.com/DistriNet/ArtiFinder-Data) repository and how they have grown over time.
@@ -94,4 +94,4 @@ Discovered artifact links by conference and year. Darker cells indicate more dis
   ]
 }
 </script>
-<script src="{{ '/assets/js/reprodb-artifinder.js' | relative_url }}"></script>
+<script src="{{ '/assets/js/reprodb-artifinder.js' | relative_url }}"></script>

src/methodology/author-metrics.md

@@ -0,0 +1,68 @@
+---
+title: "Methodology: Author Metrics"
+permalink: /methodology/author-metrics.html
+skip_chartjs: true
+---
+
+## Author Metrics
+
+Individual author statistics are computed by matching artifact papers to DBLP records. Each metric is calculated as follows:
+
+### Artifacts
+The total number of evaluated artifacts (papers with at least one badge) authored by this person across all tracked conferences.
+
+### Total Papers
+The total number of papers this author published at tracked conferences, **counting only years when that conference was conducting artifact evaluation**. For example:
+- If ACSAC started AE in 2017, only papers from 2017-present are counted
+- If an author published at ACSAC in 2010-2024, only 2017-2024 papers contribute to the denominator
+- This prevents artificial deflation of Artifact Rate by excluding pre-AE papers
+
+The paper count is determined by matching author names to DBLP records and filtering by conference and year.
+
+**Clamping rule:** If DBLP undercounts papers (i.e., artifact_count > total_papers due to incomplete DBLP records), total_papers is clamped to equal artifact_count. This guarantees that Artifact Rate <= 100%.
+
+### Artifact Rate (AR%)
+The percentage of an author's papers (at AE-active conferences) that have artifact badges: **AR% = (Artifacts / Total Papers) x 100**.
+
+**Key point:** The denominator includes only papers from years when the venue had artifact evaluation. This ensures the rate reflects artifact adoption within the relevant time window, avoiding both over-inflation (counting only artifact papers) and under-inflation (counting all historical publications).
+
+**Cross-area handling:** For authors active in both systems and security, contributions are **summed**. If an author has 10 systems papers and 5 security papers (all in AE-active years), the denominator is 15. This additive approach is correct because systems and security conferences are disjoint publication venues.
+
+### Reproducibility Rate (RR%)
+Among papers with artifacts, the percentage achieving the highest-tier badge (Reproduced or Reusable): **RR% = (Reproduced badges / Total artifacts) x 100**.
+This measures the depth of reproducibility beyond mere artifact availability.
+
+### Artifact:Evaluation Ratio (A:E)
+
+The A:E ratio characterizes the balance between artifact production and evaluation service: **A:E = Artifact Score / AE Score**, where:
+- **Artifact Score** = sum of badge points (Available+1, Functional+1, Reproduced+1 per artifact)
+- **AE Score** = committee service points (member=3, chair=5)
+
+### AE Memberships
+The number of times this author served on an artifact evaluation committee across all tracked conferences.
+
+### Chair Count
+The number of times this author served as an AE chair or co-chair.
+
+### Combined Score
+
+A composite metric balancing artifact production, artifact quality, and AE service: **Combined Score = Sum_i(A_i + F_i + R_i) + Sum_j(3 + B_j x 2)**, summing over *n* artifacts and *m* AE terms, where:
+- **First sum** (per artifact *i*):
+  - A_i = 1 if artifact *i* is Available, 0 otherwise
+  - F_i = 1 if artifact *i* is Functional, 0 otherwise
+  - R_i = 1 if artifact *i* is Reproduced/Reusable, 0 otherwise
+  - **Maximum per artifact: 3 points** (all three badges)
+
+- **Second sum** (committee service, per AE term *j*):
+  - Each committee membership contributes **3 points**
+  - B_j = 1 if term *j* is a chair role, 0 otherwise - chairs receive a **+2 bonus** for a total of **5 points** per chair term
+
+**Minimum Score Threshold:** Only individuals and institutions with combined score >= 3 are included in rankings. Institutions with placeholder affiliations ("Unknown", etc.) are excluded.
+
+**Ranking Method:** Dense ranking with ties - authors with the same combined score receive the same rank; the next rank is incremented by the number of tied entries.
+
+**Why These Weights?**
+- **Additive badge scoring (1 point each)** reflects that each badge level requires distinct effort (availability, functionality, reproducibility)
+- **AE membership = 3 points** estimates the substantial time investment (~50 hours per evaluation cycle)
+- **Chair role = 5 points** recognizes leadership and coordination responsibilities
+- This formula balances artifact producers and evaluators, countering the traditional invisibility of evaluation labor in academic metrics

src/methodology/data-collection.md

@@ -0,0 +1,53 @@
+---
+title: "Methodology: Data Collection"
+permalink: /methodology/data-collection.html
+skip_chartjs: true
+---
+
+## Data Collection
+
+We scrape artifact evaluation results from sysartifacts/secartifacts websites, extract paper titles, authors, badges (Available, Functional, Reproducible, Reusable) and repository URLs. For USENIX conferences (ATC, FAST) we also scrape badge data from technical session pages. AE committee data is gathered from sysartifacts/secartifacts plus direct scraping (USENIX, CHES, PETS websites).
+
+In addition to AE results, we ingest **automatically-discovered** artifact links from [ArtiFinder](https://github.com/DistriNet/ArtiFinder) (via the [ArtiFinder-Data](https://github.com/DistriNet/ArtiFinder-Data) dataset) and match them to papers by normalized title and author list. These links are **unverified**, carry no badges, and are **excluded from every score**; see [ArtiFinder-Discovered Artifacts](#artifinder-discovered-artifacts) below.
+
+Repository statistics (GitHub stars/forks, Zenodo/Figshare downloads) are collected via their public APIs. Artifact URLs are checked monthly for availability (HTTP HEAD requests).
+
+Author names are matched to [DBLP](https://dblp.org) for disambiguation and total-publication counts. Author affiliations are enriched using a priority cascade:
+1. **Author Index** (canonical enriched source, when available)
+2. **DBLP** person pages (`<www>` tags)
+3. **CSRankings** faculty records
+4. **OpenAlex** API
+5. **Semantic Scholar** API (fallback)
+6. **AE committee data** (last resort)
+
+Affiliation names are normalized to canonical forms (e.g., "MIT" -> "Massachusetts Institute of Technology").
+
+All scripts are in the [reprodb-pipeline](https://github.com/ReproDB/reprodb-pipeline) repository. Full CLI reference, API documentation, and data model definitions are in the [pipeline documentation](https://reprodb.github.io/reprodb-pipeline/).
+
+---
+
+## ArtiFinder-Discovered Artifacts
+
+[ArtiFinder](https://github.com/DistriNet/ArtiFinder) scrapes conference papers directly to discover links to their artifacts, independently of any artifact-evaluation process. ReproDB ingests the published [ArtiFinder-Data](https://github.com/DistriNet/ArtiFinder-Data) set and integrates it as follows:
+
+- **Matching.** Each ArtiFinder link is matched to a ReproDB paper by **normalised title + author list** (ArtiFinder uses DBLP author names, as we do), requiring the same conference and year. When author lists are known on both sides, at least one author must overlap to guard against title collisions.
+- **No badges, no scores.** ArtiFinder links are **not manually verified** and carry **no badges**. They **never** contribute to the Artifact Rate, Reproducibility Rate, combined score, or any author/institution ranking. The list also contains many papers that never went through artifact evaluation; these do not count towards any denominator either.
+- **Configurable start year.** Only editions from **2017 onwards** (the start of the AE era) are ingested by default; this is configurable in the pipeline (`artifinder_min_year`).
+- **Repository-statistics exception.** When ArtiFinder discovers a **GitHub** repository for a paper that *did* go through AE, that repository *may* be counted in the [repository statistics](/methodology/repository-statistics.html), since those stats are descriptive and separate from scoring.
+- **Display.** Wherever an ArtiFinder link appears in search results or profiles it is marked with an **Artifinder** sign indicating it was found automatically and has not been manually verified. A dedicated [ArtiFinder discovery page](/methodology/artifinder.html) reports discovery counts and rates over time.
+
+Throughout the site, statistics focus on **AE-evaluated** artifacts; ArtiFinder figures are reported separately (here and on the ArtiFinder page) and are always distinguished from AE results.
+
+## Data Sources
+
+- **[sysartifacts.github.io](https://sysartifacts.github.io)** — Systems conference artifact evaluation results ({{ site.data.summary.systems_conferences | join: ", " }})
+- **[secartifacts.github.io](https://secartifacts.github.io)** — Security conference artifact evaluation results ({{ site.data.summary.security_conferences | join: ", " }})
+- **[ArtiFinder-Data](https://github.com/DistriNet/ArtiFinder-Data)** — Automatically discovered artifact links (unverified, no badges, excluded from all scores)
+- **[usenix.org](https://www.usenix.org)** — Badge information and AE committee data for USENIX conferences
+- **[DBLP](https://dblp.org)** — Author name matching, disambiguation, and total publication counts
+- **[OpenAlex](https://openalex.org)** — Author affiliation enrichment
+- **[Semantic Scholar](https://www.semanticscholar.org)** — Author affiliation fallback lookup
+- **[CSRankings](https://csrankings.org)** — Faculty affiliation data
+- **[GitHub](https://docs.github.com/en/rest)**, **[GitLab](https://docs.gitlab.com/ee/api/)** — Repository statistics (stars, forks)
+- **[Zenodo](https://developers.zenodo.org)**, **[Figshare](https://docs.figshare.com)** — Archive statistics (downloads, views)
+- **[Data Schemas](https://reprodb.github.io/data-schemas/)** — JSON Schema definitions for all data structures used by this site (versioned; see [all schema versions](https://reprodb.github.io/data-schemas/))

src/methodology/index.md

@@ -0,0 +1,66 @@
+---
+title: "Methodology"
+permalink: /methodology.html
+skip_chartjs: true
+---
+
+The following pages explain how we collect, process, and analyze artifact evaluation data, including detailed calculation formulas for all metrics displayed on this site. ReproDB also includes automatically discovered artifact links from [ArtiFinder](/methodology/artifinder.html), documented on the dedicated ArtiFinder page below.
+
+- [Data Collection](/methodology/data-collection.html)
+- [Pipeline](/methodology/pipeline.html)
+- [Author Metrics](/methodology/author-metrics.html)
+- [Institution Metrics](/methodology/institution-metrics.html)
+- [Repository Statistics](/methodology/repository-statistics.html)
+- [Search Keywords](/methodology/search-keywords.html)
+- [ArtiFinder Discovery](/methodology/artifinder.html)
+
+{% if site.data.summary %}
+{% assign af = site.data.artifinder_summary %}
+
+## Overview
+
+<table class="page__content">
+<tr><th></th><th>AE-evaluated</th><th>ArtiFinder-discovered</th></tr>
+<tr><td><strong>Data Schema Version</strong></td><td>{{ site.data.summary.schema_version }}</td><td>&mdash;</td></tr>
+<tr><td><strong>Artifacts / discovered links</strong></td><td>{{ site.data.summary.total_artifacts }}</td><td>{{ af.total_discovered | default: "&mdash;" }}</td></tr>
+<tr><td><strong>Conferences</strong></td><td>{{ site.data.summary.total_conferences }} ({{ site.data.summary.conferences_list | join: ", " }})</td><td>{% if af %}{{ af.conferences | size }} ({{ af.conferences | join: ", " }}){% else %}&mdash;{% endif %}</td></tr>
+<tr><td><strong>Years Covered</strong></td><td>{{ site.data.summary.year_range }}</td><td>{{ af.year_range | default: "&mdash;" }}</td></tr>
+<tr><td><strong>Authors</strong></td><td>{{ site.data.author_summary.total_authors }}</td><td>{{ af.author_count | default: "&mdash;" }}</td></tr>
+<tr><td><strong>AE Committee Members</strong></td><td>{{ site.data.committee_stats.total_members }} ({{ site.data.committee_stats.unique_members }} unique)</td><td>&mdash;</td></tr>
+<tr><td><strong>Badges / scores</strong></td><td>Yes</td><td>No (unverified)</td></tr>
+<tr><td><strong>Last Updated</strong></td><td><a href="https://github.com/ReproDB/reprodb-pipeline-results">{{ site.data.summary.last_updated }}</a></td><td>{{ af.data_updated | default: "&mdash;" }}</td></tr>
+</table>
+
+{% endif %}
+
+## Conferences Covered
+
+Data is collected from conferences tracked by [sysartifacts](https://sysartifacts.github.io) and [secartifacts](https://secartifacts.github.io):
+
+- **Systems**: {{ site.data.summary.systems_conferences | join: ", " }}
+- **Security**: {{ site.data.summary.security_conferences | join: ", " }}
+
+---
+
+## API Access
+
+The full artifact dataset is available as a public JSON endpoint for programmatic access:
+
+```
+GET {{ site.url }}{{ site.baseurl }}/assets/data/search_data.json
+```
+
+Returns an array of all {{ site.data.summary.total_artifacts }} artifacts with title, authors, affiliations, conference, year, badges, and repository/artifact URLs. No authentication required.
+
+Example using `curl`:
+
+```bash
+# Get all artifacts
+curl -s {{ site.url }}{{ site.baseurl }}/assets/data/search_data.json | python3 -c "
+import sys, json
+data = json.load(sys.stdin)
+# Filter: fuzzing papers from 2024
+results = [a for a in data if 'fuzz' in a['title'].lower() and a['year'] == 2024]
+print(json.dumps(results, indent=2))
+"
+```

src/methodology/institution-metrics.md

@@ -0,0 +1,40 @@
+---
+title: "Methodology: Institution Metrics"
+permalink: /methodology/institution-metrics.html
+skip_chartjs: true
+---
+
+## Institution Metrics
+
+Institution-level statistics aggregate contributions from all authors affiliated with that institution. Affiliations are determined from DBLP person pages and CSRankings faculty data.
+
+### How Institution Data is Aggregated
+
+All metrics are **summed across affiliated authors**:
+
+- **Artifacts**: Total artifacts from all affiliated authors
+- **Total Papers**: Total papers from all affiliated authors (AE-active years only)
+- **AE Memberships**: Total committee memberships from all affiliated authors
+- **Combined Score**: Sum of all affiliated authors' combined scores
+- **Researchers**: Count of unique authors affiliated with the institution
+
+**Artifact Rate and Reproducibility Rate** are then computed from these aggregated totals:
+
+- **Institution AR%** = (Total artifacts / Total papers) x 100
+- **Institution RR%** = (Total reproduced badges / Total artifacts) x 100
+
+### Institution Role Classification
+
+Each institution is classified based on its Artifact:Evaluation ratio (artifact_score / ae_score):
+
+- **Artifact-focused** (Producer): A:E ratio > 2.0, or artifact-only (no AE service)
+- **Evaluation-focused** (Consumer): A:E ratio < 0.5, or AE-only (no artifacts)
+- **Balanced**: 0.5 <= A:E ratio <= 2.0
+
+### Cross-Area Aggregation
+
+For institution rankings broken down by area (systems vs. security):
+
+- **Systems rankings**: Include only artifacts, papers, and AE service from systems conferences
+- **Security rankings**: Include only artifacts, papers, and AE service from security conferences
+- **Overall rankings**: Sum of systems + security contributions