Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 17 additions & 19 deletions docs/reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,11 +259,10 @@ putup # Pass 2: Build with generated configs

**How it works:**
1. Installs root config if `--config` is specified
2. Copies subdir `tup.config` files from config root to build tree (out-of-tree only)
3. Parses all Tupfiles using root `tup.config` only
4. Identifies rules where any output ends with `tup.config`
5. Executes only those rules (plus their dependencies)
6. Does not write to `.pup/index` (avoids conflict with subsequent build)
2. Parses all Tupfiles using root `tup.config` only
3. Identifies rules where any output ends with `tup.config`
4. Executes only those rules (plus their dependencies)
5. Does not write to `.pup/index` (avoids conflict with subsequent build)

**Relevant Options:**
- `-v` - Verbose output
Expand All @@ -278,7 +277,7 @@ putup # Pass 2: Build with generated configs

**Using --config for pre-made configs:**

The `--config` option copies an existing config file to the output directory as the root `tup.config`. It then continues with steps 2–6 above: copying subdir configs and running any config-generating rules. Useful for:
The `--config` option copies an existing config file to the output directory as the root `tup.config`. It then continues with steps 2–5 above, running any config-generating rules. Useful for:
- Cross-compilation with pre-made toolchain configs
- CI/CD where configs are externally managed
- Mixed workflows with a static root config + auto-generated subdir configs
Expand All @@ -288,26 +287,25 @@ putup configure -B build --config configs/arm-cross.config
putup configure -B build-debug -c debug.config
```

**Subdir tup.config copying (step 2):**
**Subdir config installation:**

For out-of-tree builds (`config_root != output_root`), configure automatically copies any `tup.config` files found in subdirectories of the config root to the corresponding locations in the build tree. The root-level `tup.config` is excluded (handled by `--config` or config-generating rules).
Subdirectory configs are installed via Tupfile copy rules — putup has no built-in config copying. Each subdirectory that needs a scoped `tup.config` ships a `defaults.config` alongside its Tupfile with a copy rule:

This enables per-component scoped configs: each subdirectory ships a `tup.config` alongside its Tupfile. At configure time, these are installed into the build tree where scoped config merging (§6.1) picks them up during the build.
```tup
: defaults.config |> cp %f %o |> tup.config
```

At configure time, these rules are discovered and executed like any other config-generating rule. The resulting `tup.config` files are placed in the build tree where scoped config merging (§6.1) picks them up during the build.

```bash
# Config root has per-component configs:
# gmp/tup.config, mpfr/tup.config, mpc/tup.config
#
# configure installs root config AND copies subdir configs:
# Each subdir Tupfile has a copy rule for its defaults.config:
putup configure --config configs/toolchain.config -C . -S ../src -B ../build
# → ../build/tup.config (from --config)
# → ../build/gmp/tup.config (copied from gmp/tup.config)
# → ../build/mpfr/tup.config (copied from mpfr/tup.config)
# → ../build/mpc/tup.config (copied from mpc/tup.config)
# → ../build/gmp/tup.config (from gmp/Tupfile copy rule)
# → ../build/mpfr/tup.config (from mpfr/Tupfile copy rule)
# → ../build/mpc/tup.config (from mpc/Tupfile copy rule)
```

For in-tree builds (`config_root == output_root`), step 2 is a no-op — the configs are already in place.

**Important:** You must run `putup configure` before `putup build`. If you skip the configure step, `putup build` will error:

```
Expand Down Expand Up @@ -1648,7 +1646,7 @@ S = ../.. GMP_DIR = gmp → $(S)/$(GMP_DIR) = ../../gmp ✓
S = .. GMP_DIR = ../gmp → $(S)/$(GMP_DIR) = ../../gmp ✓
```

**Scoped `tup.config` defaults:** Components can also ship default config values in their own `tup.config` — parent configs override child configs on collision (see §6.1 *Scoped Config Merging*).
**Scoped `tup.config` defaults:** Components ship default config values in a `defaults.config` file with a Tupfile copy rule (`cp %f %o`) that installs it as `tup.config` during configure. Parent configs override child configs on collision (see §6.1 *Scoped Config Merging*).

See `examples/bsp/gcc/` for a complete working example with three interdependent libraries.

Expand Down
32 changes: 16 additions & 16 deletions examples/bsp/Makefile.pup
Original file line number Diff line number Diff line change
Expand Up @@ -40,38 +40,38 @@ configure: resolve-mpn setup-host-configs

# Copy host-specific library configs when building for non-Linux hosts.
# Linux configs are the in-tree defaults; other hosts overlay from configs/host-$(HOST)/.
# To restore Linux defaults: git checkout -- gcc/*/tup.config binutils/tup.config
# To restore Linux defaults: git checkout -- gcc/*/defaults.config binutils/defaults.config
setup-host-configs:
@if [ "$(HOST)" != "linux" ] && [ -d configs/host-$(HOST) ]; then \
for cfg in configs/host-$(HOST)/*.config; do \
lib=$$(basename "$$cfg" .config); \
if [ -f "$$lib/tup.config" ]; then \
echo " COPY $$lib/tup.config (host-$(HOST))"; \
cp "$$cfg" "$$lib/tup.config"; \
elif [ -f "gcc/$$lib/tup.config" ]; then \
echo " COPY gcc/$$lib/tup.config (host-$(HOST))"; \
cp "$$cfg" "gcc/$$lib/tup.config"; \
if [ -f "$$lib/defaults.config" ]; then \
echo " COPY $$lib/defaults.config (host-$(HOST))"; \
cp "$$cfg" "$$lib/defaults.config"; \
elif [ -f "gcc/$$lib/defaults.config" ]; then \
echo " COPY gcc/$$lib/defaults.config (host-$(HOST))"; \
cp "$$cfg" "gcc/$$lib/defaults.config"; \
fi; \
done; \
fi

# Resolve mpn sources for the selected CPU target.
# - gcc/gmp/mpn/tup.config: source lists and per-function toggles (child scope)
# - gcc/gmp/tup.config: GMP_MPARAM and ASM_ENABLED (parent scope, visible to gmp/Tupfile)
# - gcc/gmp/mpn/defaults.config: source lists and per-function toggles (child scope)
# - gcc/gmp/defaults.config: GMP_MPARAM and ASM_ENABLED (parent scope, visible to gmp/Tupfile)
resolve-mpn:
@scripts/resolve-mpn.sh $(MPN_CPU) $(SRCDIR)/gcc/gmp/mpn > gcc/gmp/mpn/tup.config
@grep -v '^CONFIG_GMP_MPARAM=\|^CONFIG_ASM_ENABLED=\|^CONFIG_NO_ASM=' gcc/gmp/tup.config > gcc/gmp/tup.config.tmp && mv gcc/gmp/tup.config.tmp gcc/gmp/tup.config
@scripts/resolve-mpn.sh $(MPN_CPU) $(SRCDIR)/gcc/gmp/mpn > gcc/gmp/mpn/defaults.config
@grep -v '^CONFIG_GMP_MPARAM=\|^CONFIG_ASM_ENABLED=\|^CONFIG_NO_ASM=' gcc/gmp/defaults.config > gcc/gmp/defaults.config.tmp && mv gcc/gmp/defaults.config.tmp gcc/gmp/defaults.config
@if [ "$(MPN_CPU)" != "generic" ]; then \
arch=$${MPN_CPU%%/*}; \
if [ -f "$(SRCDIR)/gcc/gmp/mpn/$$arch/gmp-mparam.h" ]; then \
echo "CONFIG_GMP_MPARAM=mpn/$$arch/gmp-mparam.h" >> gcc/gmp/tup.config; \
echo "CONFIG_GMP_MPARAM=mpn/$$arch/gmp-mparam.h" >> gcc/gmp/defaults.config; \
else \
echo "CONFIG_GMP_MPARAM=mpn/generic/gmp-mparam.h" >> gcc/gmp/tup.config; \
echo "CONFIG_GMP_MPARAM=mpn/generic/gmp-mparam.h" >> gcc/gmp/defaults.config; \
fi; \
echo "CONFIG_ASM_ENABLED=y" >> gcc/gmp/tup.config; \
echo "CONFIG_ASM_ENABLED=y" >> gcc/gmp/defaults.config; \
else \
echo "CONFIG_GMP_MPARAM=mpn/generic/gmp-mparam.h" >> gcc/gmp/tup.config; \
echo "CONFIG_NO_ASM=1" >> gcc/gmp/tup.config; \
echo "CONFIG_GMP_MPARAM=mpn/generic/gmp-mparam.h" >> gcc/gmp/defaults.config; \
echo "CONFIG_NO_ASM=1" >> gcc/gmp/defaults.config; \
fi

# Multi-variant: build for multiple platforms in parallel
Expand Down
1 change: 1 addition & 0 deletions examples/bsp/binutils/Tupfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
include_rules
: defaults.config |> ^ INSTALL %o^ cp %f %o |> tup.config

# ============================================================
# binutils — cross-assembler (as) + cross-archiver (ar)
Expand Down
File renamed without changes.
1 change: 1 addition & 0 deletions examples/bsp/gcc/gcc/Tupfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
include_rules
: defaults.config |> ^ INSTALL %o^ cp %f %o |> tup.config

# ============================================================
# GCC 15.2.0 — C compiler backend
Expand Down
File renamed without changes.
1 change: 1 addition & 0 deletions examples/bsp/gcc/gmp/Tupfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
include_rules
: defaults.config |> ^ INSTALL %o^ cp %f %o |> tup.config

# ============================================================
# GMP 6.2.1
Expand Down
File renamed without changes.
1 change: 1 addition & 0 deletions examples/bsp/gcc/gmp/mpn/Tupfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
include_rules
: defaults.config |> ^ INSTALL %o^ cp %f %o |> tup.config

# MPN — low-level multiprecision arithmetic
#
Expand Down
1 change: 1 addition & 0 deletions examples/bsp/gcc/libbacktrace/Tupfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
include_rules
: defaults.config |> ^ INSTALL %o^ cp %f %o |> tup.config

# ============================================================
# libbacktrace - stack trace support (GCC 15.2.0)
Expand Down
1 change: 1 addition & 0 deletions examples/bsp/gcc/libcody/Tupfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
include_rules
: defaults.config |> ^ INSTALL %o^ cp %f %o |> tup.config

# ============================================================
# libcody - C++ modules protocol library
Expand Down
1 change: 1 addition & 0 deletions examples/bsp/gcc/libcpp/Tupfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
include_rules
: defaults.config |> ^ INSTALL %o^ cp %f %o |> tup.config

# ============================================================
# libcpp - C preprocessor library (GCC 15.2.0)
Expand Down
1 change: 1 addition & 0 deletions examples/bsp/gcc/libdecnumber/Tupfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
include_rules
: defaults.config |> ^ INSTALL %o^ cp %f %o |> tup.config

# ============================================================
# libdecnumber - BID decimal floating-point (GCC 15.2.0)
Expand Down
1 change: 1 addition & 0 deletions examples/bsp/gcc/libiberty/Tupfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
include_rules
: defaults.config |> ^ INSTALL %o^ cp %f %o |> tup.config

# ============================================================
# libiberty - GCC utility library
Expand Down
1 change: 1 addition & 0 deletions examples/bsp/gcc/mpc/Tupfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
include_rules
: defaults.config |> ^ INSTALL %o^ cp %f %o |> tup.config

# MPC 1.2.1 - generated files

Expand Down
File renamed without changes.
1 change: 1 addition & 0 deletions examples/bsp/gcc/mpfr/Tupfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
include_rules
: defaults.config |> ^ INSTALL %o^ cp %f %o |> tup.config

# MPFR 4.1.0 - generated files

Expand Down
File renamed without changes.
5 changes: 0 additions & 5 deletions include/pup/cli/context.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
#include "options.hpp"
#include "pup/core/result.hpp"
#include "pup/parser/ast.hpp"
#include "pup/parser/ignore.hpp"

#include <cstdint>
#include <filesystem>
Expand Down Expand Up @@ -127,10 +126,6 @@ auto compute_build_scopes(
[[nodiscard]]
auto make_layout_options(Options const& opts) -> LayoutOptions;

/// Load .pupignore patterns (with built-in defaults)
[[nodiscard]]
auto load_ignore_list(ProjectLayout const& layout, bool verbose) -> parser::IgnoreList;

/// Context for clean commands
struct CleanContext {
std::filesystem::path root;
Expand Down
1 change: 1 addition & 0 deletions include/pup/exec/scheduler.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ struct SchedulerOptions {
bool dry_run = false; ///< Print commands without executing
bool verbose = false; ///< Print commands as they run
std::filesystem::path source_root = {}; ///< Source tree root (where Tupfile.ini lives)
std::filesystem::path config_root = {}; ///< Config tree root (where Tupfiles live, for 3-tree builds)
std::filesystem::path output_root = {}; ///< Output tree root (where outputs/.pup go)
std::optional<std::chrono::seconds> timeout = {}; ///< Per-command timeout
};
Expand Down
5 changes: 4 additions & 1 deletion include/pup/graph/dag.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -390,12 +390,15 @@ auto expand_instruction(Graph const& graph, NodeId cmd_id, PathCache& cache) ->
/// Expand instruction with canonical path resolution for symlinked source trees.
/// When source_root is provided, build-tree paths are computed relative to the
/// canonical (physical) CWD instead of the logical CWD.
/// In 3-tree builds, config_root locates overlay files (like defaults.config)
/// that live alongside Tupfiles rather than in source_root.
[[nodiscard]]
auto expand_instruction(
Graph const& graph,
NodeId cmd_id,
PathCache& cache,
std::filesystem::path const& source_root
std::filesystem::path const& source_root,
std::filesystem::path const& config_root = {}
) -> std::string;

/// Expand instruction pattern (convenience overload, creates temporary cache)
Expand Down
1 change: 1 addition & 0 deletions src/cli/cmd_build.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1088,6 +1088,7 @@ auto build_single_variant(
.dry_run = opts.dry_run,
.verbose = opts.verbose,
.source_root = ctx.layout().source_root,
.config_root = ctx.layout().config_root,
.output_root = ctx.layout().output_root,
};

Expand Down
70 changes: 2 additions & 68 deletions src/cli/cmd_configure.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@
#include "pup/core/path_utils.hpp"
#include "pup/exec/scheduler.hpp"
#include "pup/graph/dag.hpp"
#include "pup/parser/ignore.hpp"

#include <cstdio>
#include <cstdlib>
Expand Down Expand Up @@ -44,73 +43,11 @@ auto install_config_file(
return EXIT_SUCCESS;
}

auto install_source_configs(
ProjectLayout const& layout,
std::string_view variant_name,
bool verbose,
pup::parser::IgnoreList const& ignore
) -> void
{
if (layout.config_root == layout.output_root) {
return;
}

auto config_canonical = std::filesystem::weakly_canonical(layout.config_root);
auto output_canonical = std::filesystem::weakly_canonical(layout.output_root);
auto count = 0;
auto ec = std::error_code {};

for (auto it = std::filesystem::recursive_directory_iterator(config_canonical, ec);
it != std::filesystem::recursive_directory_iterator();
it.increment(ec)) {
if (ec) {
fprintf(stderr, "[%.*s] Warning: error scanning configs: %s\n", static_cast<int>(variant_name.size()), variant_name.data(), ec.message().c_str());
break;
}

// Skip the output tree if it lives inside config_root (e.g., -B build)
if (it->is_directory() && it->path() == output_canonical) {
it.disable_recursion_pending();
continue;
}

// Skip directories matched by .pupignore
if (it->is_directory()) {
auto rel = std::filesystem::relative(it->path(), config_canonical);
if (ignore.is_ignored(rel)) {
it.disable_recursion_pending();
continue;
}
}

if (!it->is_regular_file() || it->path().filename() != "tup.config") {
continue;
}

auto rel = std::filesystem::relative(it->path(), config_canonical);
if (rel == "tup.config") {
continue;
}

auto dest = layout.output_root / rel;
std::filesystem::create_directories(dest.parent_path());
std::filesystem::copy_file(it->path(), dest, std::filesystem::copy_options::overwrite_existing);
if (verbose) {
printf("[%.*s] Copied %s -> %s\n", static_cast<int>(variant_name.size()), variant_name.data(), it->path().string().c_str(), dest.string().c_str());
}
++count;
}
if (count > 0) {
printf("[%.*s] Installed %d source config(s)\n", static_cast<int>(variant_name.size()), variant_name.data(), count);
}
}

auto configure_single_variant(
Options const& opts,
std::string_view variant_name
) -> int
{
// Discover layout once for steps 1 & 2
auto layout = discover_layout(make_layout_options(opts));
if (!layout) {
fprintf(stderr, "[%.*s] Error: %s\n", static_cast<int>(variant_name.size()), variant_name.data(), layout.error().message.c_str());
Expand All @@ -125,11 +62,7 @@ auto configure_single_variant(
}
}

// Step 2: Copy source subdir tup.config files to build tree
auto ignore = load_ignore_list(*layout, opts.verbose);
install_source_configs(*layout, variant_name, opts.verbose, ignore);

// Step 3: Run config-generating rules
// Step 2: Run config-generating rules
auto scopes = compute_build_scopes(opts, *layout);
auto ctx_opts = BuildContextOptions {
.verbose = opts.verbose,
Expand Down Expand Up @@ -200,6 +133,7 @@ auto configure_single_variant(
.dry_run = opts.dry_run,
.verbose = opts.verbose,
.source_root = ctx.layout().source_root,
.config_root = ctx.layout().config_root,
.output_root = ctx.layout().output_root,
};

Expand Down
2 changes: 1 addition & 1 deletion src/cli/cmd_parse.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ auto parse_single_variant(Options const& opts, std::string_view variant_name) ->
for (auto id : commands) {
if (ctx.graph().get_command_node(id)) {
auto display_sv = pup::graph::get_display_str(ctx.graph().graph(), id);
auto cmd_sv = pup::graph::expand_instruction(ctx.graph().graph(), id, cache, ctx.layout().source_root);
auto cmd_sv = pup::graph::expand_instruction(ctx.graph().graph(), id, cache, ctx.layout().source_root, ctx.layout().config_root);
auto label = display_sv.empty() ? cmd_sv : display_sv;
printf("[%.*s] %.*s\n", static_cast<int>(variant_name.size()), variant_name.data(), static_cast<int>(label.size()), label.data());
}
Expand Down
4 changes: 2 additions & 2 deletions src/cli/context.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -508,8 +508,6 @@ auto sort_dirs_by_depth(std::set<std::filesystem::path> const& available) -> std
return dirs;
}

} // namespace

auto load_ignore_list(ProjectLayout const& layout, bool verbose) -> pup::parser::IgnoreList
{
auto ignore = pup::parser::IgnoreList::with_defaults();
Expand All @@ -531,6 +529,8 @@ auto load_ignore_list(ProjectLayout const& layout, bool verbose) -> pup::parser:
return ignore;
}

} // namespace

auto make_layout_options(Options const& opts) -> LayoutOptions
{
auto layout_opts = LayoutOptions {};
Expand Down
2 changes: 1 addition & 1 deletion src/exec/scheduler.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -821,7 +821,7 @@ auto Scheduler::build_job_list(
}

// Expand command from instruction pattern + operands
auto cmd_str = expand_instruction(graph.graph(), id, cache, impl_->options.source_root);
auto cmd_str = expand_instruction(graph.graph(), id, cache, impl_->options.source_root, impl_->options.config_root);
auto display_str = std::string { get_display_str(graph.graph(), id) };

// Convert exported_vars from StringIds to strings
Expand Down
Loading