feat: Add Remote Execution and Remote Caching Support

[avrabe]

🎯 Objective

Enable full remote execution and remote caching support for rules_wasm_component to achieve 3-10x faster builds with distributed build infrastructure.

📊 Current State Analysis

❌ Remote Execution Blockers

• TinyGo: "local": "1" execution requirement (go/defs.bzl:282)
• NPM: "local": "1" execution requirement (js/defs.bzl:316)
• WKG Rules: use_default_shell_env = True for registry authentication
• ctx.actions.run_shell: 7 files still use non-hermetic shell execution

✅ Already Remote-Ready

• Rust compilation: Fully hermetic with wasm transitions
• WASM validation/transformation: Pure tool execution
• WIT processing: Hermetic wit-bindgen usage
• Component composition: WAC operations are tool-only

🚀 Implementation Plan

Phase 1: Remove Remote Execution Blockers (Critical)

1.1 Fix TinyGo Remote Execution

File: go/defs.bzl:282

# Current (blocks remote execution):
execution_requirements = {
    "local": "1",  # TinyGo requires local execution
}

# Fixed (enable remote execution):
execution_requirements = {
    "no-network": "1",  # Hermetic build, no network needed
    "supports-path-mapping": "1",  # Support for path virtualization
}

Additional Changes:

• Remove hardcoded tool paths (/Users/r/.cargo/bin, etc.) in go/defs.bzl:186
• Use ctx.configuration.host_path_separator for cross-platform paths
• Replace wrapper script with direct toolchain binary execution

1.2 Fix NPM Remote Execution

File: js/defs.bzl:316

# Current (blocks remote execution):
execution_requirements = {
    "local": "1",  # NPM install requires network access
}

# Option A: Pre-download dependencies (recommended)
execution_requirements = {
    "no-network": "1",  # Pre-downloaded deps, no network needed
}

1.3 Fix WKG Registry Authentication

Files: wkg/defs.bzl (multiple locations)

# Current (breaks remote execution):
use_default_shell_env = True  # Needed for registry authentication

# Fixed: Use explicit credential passing
env = {
    "WKG_REGISTRY_TOKEN": ctx.var.get("WKG_REGISTRY_TOKEN", ""),
    "REGISTRY_CONFIG": ctx.file.registry_config.path if ctx.file.registry_config else "",
}
use_default_shell_env = False

Phase 2: Eliminate ctx.actions.run_shell

Files to modernize:

• wit/wit_bindgen.bzl
• rust/rust_wasm_component_bindgen.bzl
• wasm/wasm_signing.bzl
• wasm/multi_language_wasm_component.bzl
• wit/wit_markdown.bzl

Replace shell commands with direct tool execution using ctx.actions.run().

Phase 3: Remote Caching Optimization

3.1 Add Cache Keys for WASM Artifacts

def _rust_wasm_component_impl(ctx):
    # Add cache-friendly metadata
    cache_key_inputs = [
        ctx.attr.optimization,
        ctx.attr.wit[WitInfo].package_name if ctx.attr.wit else "",
        str(sorted(ctx.attr.features)),
        ctx.toolchains["@rules_rust//rust:toolchain_type"].rustc_version,
    ]
    
    ctx.actions.write(
        output = ctx.actions.declare_file(ctx.attr.name + ".cache_key"),
        content = "\n".join(cache_key_inputs),
    )

3.2 Optimize Large File Handling

execution_requirements = {
    "supports-workers": "1",  # Enable persistent workers
    "cpu": "2",  # Specify resource requirements
    "memory": "4G",  # Large WASM files need memory
}

3.3 Add Action Mnemonics

mnemonic = "WasmComponentBuild",  # Instead of generic names
progress_message = "Building WASM component %{label}",

Phase 4: Remote Execution Configuration

4.1 .bazelrc Configuration

# Remote execution settings
build:remote --remote_executor=grpcs://your-rbe-instance
build:remote --remote_cache=grpcs://your-cache-instance
build:remote --incompatible_strict_action_env=true
build:remote --spawn_strategy=remote
build:remote --strategy=WasmComponentBuild=remote
build:remote --strategy=TinyGoCompile=remote

# Remote caching only
build:cache --remote_cache=grpcs://your-cache-instance
build:cache --incompatible_strict_action_env=true

# Platform configuration
build:remote --extra_execution_platforms=@rbe_default//config:platform
build:remote --host_platform=@rbe_default//config:platform
build:remote --platforms=@rbe_default//config:platform

📈 Expected Benefits

🚀 Performance Improvements

• Build Speed: 3-10x faster builds with remote execution clusters
• Cache Hit Rate: 80-95% cache hits with proper action keys
• Parallelization: Unlimited parallel execution vs local CPU limits

💰 Cost Efficiency

• Local Resources: Free up developer machines
• CI/CD Speed: Faster deployments and testing
• Scalability: Handle large-scale WASM component builds

🔒 Reliability

• Hermetic Builds: Guaranteed reproducibility across environments
• Platform Independence: Build anywhere, run anywhere
• Disaster Recovery: Distributed build infrastructure

🎯 Success Criteria

• All rules execute successfully on remote execution platforms
• No "local": "1" execution requirements remain
• Zero ctx.actions.run_shell() calls in core rules
• Remote cache hit rate >80% for typical builds
• CI/CD pipeline uses remote execution
• Documentation for RBE setup

📋 Implementation Timeline

Phase	Effort	Timeline	Risk
Phase 1: Remove Blockers	3-5 days	Week 1	Medium
Phase 2: Eliminate Shell	2-3 days	Week 2	Low
Phase 3: Cache Optimization	2-3 days	Week 2-3	Low
Phase 4: RBE Configuration	1-2 days	Week 3	Medium

Total: ~2-3 weeks for complete remote execution + remote caching support

🔗 Related Issues

• Builds on existing hermetic toolchain foundation
• Complements shell script elimination efforts
• Enables enterprise-scale WASM component development

📚 References

• Bazel Remote Execution
• Remote Caching Guide
• Hermetic Builds Best Practices