Skip to content

Commit bc5084b

Browse files
authored
fix: update contributing and readme (#127)
1 parent b201129 commit bc5084b

File tree

4 files changed

+207
-41
lines changed

4 files changed

+207
-41
lines changed

.github/assets/logo.png

156 KB
Loading

.github/assets/run-switcher.png

39.1 KB
Loading

CONTRIBUTING.md

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Contributing to Base Benchmark
2+
3+
Thanks for your interest in improving the Base Benchmark tool!
4+
5+
## Ways to Contribute
6+
7+
1. **Report bugs** - Open an issue describing the problem
8+
2. **Suggest features** - Share ideas for improvements
9+
3. **Submit pull requests** - Fix bugs or add features
10+
11+
## Submitting a Pull Request
12+
13+
1. Fork the repository
14+
2. Create a feature branch (`git checkout -b my-feature`)
15+
3. Make your changes following our coding style
16+
4. Add tests for new functionality
17+
5. Ensure all tests pass
18+
6. Commit your changes with a clear message
19+
7. Push to your fork and open a pull request
20+
21+
## Code Style
22+
23+
- Write clear, readable code with comments where needed
24+
- Keep functions small and focused
25+
- Use descriptive variable and function names
26+
- Follow standard Go conventions (`gofmt`, `golint`)
27+
- Prefer functional programming patterns
28+
29+
## Testing
30+
31+
- Add unit tests for new functions
32+
- Add integration tests for end-to-end features
33+
- Ensure existing tests still pass
34+
35+
## Questions?
36+
37+
Open a discussion or issue if you have questions or need help. Join the [Base Discord server](https://discord.gg/WtdUtVsy9A) for additional support.

README.md

Lines changed: 170 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -1,54 +1,125 @@
1-
<div align="center">
2-
<h1 style="font-size:32pt">Base Benchmark</h1>
3-
<a href="https://shields.io/"><img src="https://shields.io/badge/status-beta-yellow" alt="Status: Beta"></a>
4-
<a href="https://go.dev/"><img src="https://shields.io/badge/language-Go-00ADD8" alt="Language: Go"></a>
5-
<a href="https://github.com/base/benchmark/blob/main/LICENSE"><img src="https://shields.io/github/license/base/benchmark" alt="License"></a>
6-
</div>
1+
![Base](.github/assets/logo.png)
2+
3+
# Base Benchmark
74

85
Base Benchmark is a performance testing framework for Ethereum execution clients. Compare client performance, identify bottlenecks, and ensure reliability before deployment.
96

10-
## 🚀 Features
7+
<!-- Badge row 1 - status -->
8+
9+
[![GitHub contributors](https://img.shields.io/github/contributors/base/benchmark)](https://github.com/base/benchmark/graphs/contributors)
10+
[![GitHub commit activity](https://img.shields.io/github/commit-activity/w/base/benchmark)](https://github.com/base/benchmark/graphs/contributors)
11+
[![GitHub Stars](https://img.shields.io/github/stars/base/benchmark.svg)](https://github.com/base/benchmark/stargazers)
12+
![GitHub repo size](https://img.shields.io/github/repo-size/base/benchmark)
13+
[![GitHub](https://img.shields.io/github/license/base/benchmark?color=blue)](https://github.com/base/benchmark/blob/main/LICENSE)
14+
15+
<!-- Badge row 2 - links and profiles -->
16+
17+
[![Website base.org](https://img.shields.io/website-up-down-green-red/https/base.org.svg)](https://base.org)
18+
[![Blog](https://img.shields.io/badge/blog-up-green)](https://base.mirror.xyz/)
19+
[![Docs](https://img.shields.io/badge/docs-up-green)](https://docs.base.org/)
20+
[![Discord](https://img.shields.io/discord/1067165013397213286?label=discord)](https://base.org/discord)
21+
[![Twitter Base](https://img.shields.io/twitter/follow/Base?style=social)](https://twitter.com/Base)
22+
23+
<!-- Badge row 3 - detailed status -->
24+
25+
[![GitHub pull requests by-label](https://img.shields.io/github/issues-pr-raw/base/benchmark)](https://github.com/base/benchmark/pulls)
26+
[![GitHub Issues](https://img.shields.io/github/issues-raw/base/benchmark.svg)](https://github.com/base/benchmark/issues)
1127

12-
Base Benchmark provides comprehensive testing capabilities:
28+
## Features
1329

14-
- **Performance Evaluation** - Test both block building and validation performance across execution clients
15-
- **Comparative Analysis** - Measure client behavior across various inputs and workloads
16-
- **Metric Collection** - Track critical metrics including submission times, latency, and throughput
17-
- **Flexible Workloads** - Configure transaction patterns to match your specific needs
18-
- **Visual Reports** - Generate interactive HTML dashboards of benchmark results
30+
- **Performance Evaluation:** Test both block building and validation performance across execution clients (Geth, Reth, and more)
31+
- **Comparative Analysis:** Measure client behavior across various inputs and workloads
32+
- **Metric Collection:** Track critical metrics including submission times, latency, and throughput
33+
- **Flexible Workloads:** Configure transaction patterns to match your specific needs
34+
- **Interactive Dashboard:** Generate beautiful HTML reports with charts and run comparisons
35+
- **Import & Merge:** Combine benchmark results from multiple machines with flexible tagging
1936

20-
## 📋 Quick Start
37+
## Repository Structure
2138

22-
[Install Forge](https://book.getfoundry.sh/getting-started/installation)
39+
```
40+
.
41+
├── Makefile # Build and development tasks
42+
├── go.mod # Go module dependencies
43+
├── benchmark/ # CLI application
44+
│ ├── cmd/ # Main entry point
45+
│ ├── config/ # Configuration types
46+
│ └── flags/ # CLI flags
47+
├── runner/ # Core benchmarking logic
48+
│ ├── benchmark/ # Benchmark execution
49+
│ ├── clients/ # Client integrations (Geth, Reth)
50+
│ ├── importer/ # Run import functionality
51+
│ ├── network/ # Network setup and management
52+
│ └── payload/ # Transaction payload generation
53+
├── configs/ # Benchmark configurations
54+
│ ├── examples/ # Development and testing configs
55+
│ └── public/ # Production-ready benchmarks
56+
├── contracts/ # Smart contracts for testing
57+
│ └── src/ # Solidity source files
58+
├── report/ # Interactive dashboard
59+
│ └── src/ # React TypeScript application
60+
└── clients/ # Client build scripts
61+
```
62+
63+
## Prerequisites
64+
65+
- **Go:** Version 1.21 or later. Install from [go.dev](https://go.dev/dl/)
66+
- **Foundry:** For smart contract compilation. See [installation guide](https://book.getfoundry.sh/getting-started/installation)
67+
- **Node.js:** Version 18+ for the interactive dashboard. Install from [nodejs.org](https://nodejs.org/)
2368

24-
Recursively clone github submodules:
69+
## Getting Started
70+
71+
### 1. Clone the Repository
2572

2673
```bash
74+
git clone https://github.com/base/benchmark.git
75+
cd benchmark
2776
git submodule update --init --recursive
2877
```
2978

79+
### 2. Build the Application
80+
3081
```bash
31-
# Build the application
3282
make build
83+
```
84+
85+
The binary will be located at `bin/benchmark`.
3386

34-
# Build the binaries, geth, reth, rbuilder
87+
### 3. Build Client Binaries (Optional)
88+
89+
To build Geth and Reth from source:
90+
91+
```bash
3592
make build-binaries
93+
```
3694

37-
# Run the basic benchmark
95+
Alternatively, you can specify paths to pre-built binaries when running benchmarks.
96+
97+
### 4. Run Your First Benchmark
98+
99+
```bash
38100
./bin/base-bench run \
39101
--config ./configs/public/basic.yml \
40102
--root-dir ./data-dir \
41-
--reth-bin path_to_reth_bin \
42-
--geth-bin path_to_geth_bin \
43103
--output-dir ./output
104+
```
44105

45-
# View the interactive dashboard
106+
To see available options:
107+
108+
```bash
109+
./bin/base-bench run --help
110+
```
111+
112+
### 5. View Results in the Interactive Dashboard
113+
114+
```bash
46115
cd report/
47-
npm i
116+
npm install
48117
npm run dev
49118
```
50119

51-
## 📋 Available Benchmarks
120+
Open your browser to the URL shown (typically `http://localhost:5173`).
121+
122+
## Available Benchmarks
52123

53124
Explore the comprehensive collection of benchmark configurations:
54125

@@ -59,7 +130,7 @@ Explore the comprehensive collection of benchmark configurations:
59130

60131
Choose from storage operations, precompile tests, token workloads, mainnet simulations, and more.
61132

62-
## 🏗️ Architecture
133+
## Architecture
63134

64135
### Benchmark Structure
65136

@@ -99,23 +170,16 @@ Each test executes a standardized workflow:
99170

100171
This approach allows precise measurement of performance characteristics for both block production and validation.
101172

102-
## 🔧 Configuration
103-
104-
### Build
105-
106-
```bash
107-
make build
108-
ls ./bin/base-bench
109-
```
173+
## Configuration
110174

111175
### Available Flags
112176

113177
```
114178
NAME:
115-
base-bench run - run benchmark
179+
benchmark run - run benchmark
116180

117181
USAGE:
118-
base-bench run [command options]
182+
benchmark run [command options]
119183

120184
OPTIONS:
121185
--config value Config Path ($BASE_BENCH_CONFIG)
@@ -140,16 +204,81 @@ OPTIONS:
140204
--help, -h Show help (default: false)
141205
```
142206
143-
## 📊 Example Reports
207+
## Managing Test Runs
208+
209+
### Understanding Runs and Suites
210+
211+
When you view benchmark results in the interactive dashboard, you can switch between different test runs using the run switcher:
144212
145213
<div align="center">
146-
<p><i>Performance comparison between Geth and Reth clients</i></p>
214+
<img src=".github/assets/run-switcher.png" alt="Run Switcher" width="600">
147215
</div>
148216
149-
## 🤝 Contributing
217+
### Creating Test Runs
218+
219+
Running benchmarks adds a new suite by default:
220+
221+
```bash
222+
./bin/base-bench run --config ./configs/public/basic.yml
223+
```
224+
225+
Each execution creates a new suite entry in the run list, allowing you to track performance over time or across different configurations.
226+
227+
### Combining Multiple Runs
228+
229+
Use `import-runs` to merge benchmark results from multiple machines or configurations:
230+
231+
```bash
232+
./bin/base-bench import-runs \
233+
--output-dir ./output \
234+
./results-from-server-1/metadata.json
235+
```
236+
237+
**Two import strategies:**
238+
239+
1. **Add to latest suite with tags** - Merge imported runs into your most recent suite, using tags to differentiate:
240+
241+
```bash
242+
# Add imported runs to the last suite with tags for differentiation
243+
./bin/base-bench import-runs \
244+
--src-tag "instance=server-lg" \
245+
--dest-tag "instance=server-md" \
246+
--output-dir ./output \
247+
./results-from-server-1/metadata.json
248+
249+
# --src-tag fills missing tags on existing runs (won't overwrite)
250+
# --dest-tag applies to the imported runs
251+
# Useful for comparing hardware configurations within the same test run
252+
```
253+
254+
2. **Create new separate suite** - Add imported runs as an independent suite in the list:
255+
256+
```bash
257+
# Interactive mode (recommended) - prompts you to choose strategy and configure tags
258+
./bin/base-bench import-runs \
259+
--output-dir ./output \
260+
./results-from-server-1/metadata.json
261+
262+
# Creates a new entry differentiated by BenchmarkRun ID
263+
# Useful for tracking performance across different code versions or time periods
264+
```
265+
266+
**Interactive Mode:** Without specifying tags, the tool enters interactive mode and guides you through:
267+
268+
- Choosing between adding to last suite or creating new suite
269+
- Configuring appropriate tags if needed
270+
- Confirming the import operation
271+
272+
This flexibility lets you organize benchmarks by hardware type, client version, or any dimension relevant to your analysis.
273+
274+
## Contributing
275+
276+
We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines on how to contribute to this project.
277+
278+
## License
150279

151-
Contributions are welcome! Please feel free to submit a Pull Request.
280+
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
152281

153-
## 📜 License
282+
---
154283

155-
This project is licensed under the [MIT License](LICENSE).
284+
**Built with ❤️ by [Base](https://base.org)**

0 commit comments

Comments
 (0)