diff --git a/CHANGELOG.md b/CHANGELOG.md index 8fd44c5..cfaa026 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,26 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [0.13.0] 2026-08-10 + +### Added + +- Workflows register a Quarto report recipe (`preamble` + `sections`) via `register_es_data_workflow()`, retrieved with `get_es_workflow_report()`. Child paths are validated for existence at registration. +- `test_es_data()` helper for building lightweight `es_data` fixtures in tests and downstream packages. +- Relative QC stage completeness diagnostics: samples or pools missing stages that peers have receive a `qc_load` diagnostic while keeping their partial QC data. +- Samples page report-data callout listing loading and analysis-step diagnostics, with warning markers on samples affected by loading issues. +- Run info tab (renamed from Run settings) with a conditional Diagnostics section. + +### Changed + +- Breaking: `register_es_data_workflow()` now requires a `report` factory. Callers that only registered extractors in 0.12.0 must supply a valid report recipe (`preamble` + `sections`). +- The Quarto shell dispatches report sections from the registered workflow recipe. Shared children live under `inst/quarto/shared/`; workflow-owned children live under `inst/quarto/workflows//`. + +### Fixed + +- `print_metadata_table()` no longer requires `pxl_data_processed` for hashed experiments; `% of pool` is omitted when processed data is unavailable. + ## [0.12.0] 2026-08-06 ### Added diff --git a/DESCRIPTION b/DESCRIPTION index 91224ff..1821ecc 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -1,6 +1,6 @@ Package: pixelatorES Title: Proxiome Experiment Summary -Version: 0.12.0 +Version: 0.13.0 Authors@R: c( person("Max", "Karlsson", , "max.karlsson@pixelgen.com", role = c("aut", "cre"), diff --git a/DEVELOPERS.md b/DEVELOPERS.md index e209466..7daa9e7 100644 --- a/DEVELOPERS.md +++ b/DEVELOPERS.md @@ -204,37 +204,104 @@ Ingestion is resilient: a broken sample or missing QC file should not lose the w - An extractor can return an `es_data_extractor_result` (via the internal `.new_es_data_extractor_result(value, diagnostics)`) to hand back a **partial** value together with diagnostics — for example, PXL loading returns the samples that loaded plus `pxl_load` diagnostics for those that did not. - If an extractor throws instead, `run_es_data_extractors()` catches the error, records an `extractor` diagnostic, and leaves that slot `NULL` while continuing with the rest. +- After QC groups load, relative stage completeness is checked within samples and within pools. If peers have a stage that another loaded entity lacks, a `qc_load` diagnostic is recorded for that entity while keeping its partial QC data. Each diagnostic (`.new_es_data_diagnostic()`) has a `type` (`"pxl_load"`, `"qc_load"`, or `"extractor"`), a `target` (a sample alias, pool id, or slot path such as `"qc$crossing_edges"`), and a human-readable `message`. Inspect them via `es_data$diagnostics`. +On the report, diagnostics surface in two places: + +- **Samples**: a red callout lists sample/pool loading issues and analysis-step failures. Sample- or pool-targeted loading issues also add a warning marker in the metadata table. +- **Run info**: the Diagnostics section lists every recorded diagnostic when any exist. + ### Registering a workflow -Workflows are stored in a package-local registry (`R/workflow_registry.R`). `params$workflow` selects one and defaults to `"amplicon_demux"`, which is registered when the package loads. An extension package can add its own from its `.onLoad()` hook: +Workflows are stored in a package-local registry ([`R/workflow_registry.R`](R/workflow_registry.R)). `params$workflow` selects one and defaults to `"amplicon_demux"`, which is registered when the package loads. Each workflow registers: + +- `extractors`: a zero-argument factory returning the nested extractor list +- `report`: a zero-argument factory returning the Quarto report recipe (`preamble` + `sections`) + +Extension packages should call `register_es_data_workflow()` from their `.onLoad()` hook: ```r +.register_child <- function(...) { + system.file("quarto", ..., package = "myPackage", mustWork = TRUE) +} + register_es_data_workflow( - "my_workflow", - function() { + name = "my_workflow", + extractors = function() { list( samplesheet = my_extract_samplesheet, - pxl_data = my_extract_pxl_data, + pxl_data = my_extract_pxl_data # ... ) + }, + report = function() { + list( + preamble = .register_child("shared", "preprocessing.qmd"), + sections = list( + list( + id = "samples", + title = "Samples", + child = .register_child("shared", "samples.qmd") + ), + list( + id = "quality_metrics", + title = "Quality metrics", + child = .register_child("workflows", "my_workflow", "quality_metrics.qmd") + ) + ) + ) } ) ``` -Use `list_es_data_workflows()` to see what is registered. +Built-in workflows use paths relative to `inst/quarto/`. Extension packages should register absolute paths from `system.file()`. All referenced child paths are checked for existence at registration time. + +Use `list_es_data_workflows()` to see what is registered and `get_es_workflow_report(name)` to inspect a report recipe. ### Consuming `es_data` Report code should treat `es_data` as the single source of truth: `component_*()` functions, `key_metric_table()`, and the `print_*()` helpers all take `es_data` as their first argument and pull the slots they need internally. When adding a new component, accept `es_data` and read from its slots rather than threading individual objects through the `.qmd` files. +### Test fixtures with `test_es_data()` + +For unit tests of components and helpers, build lightweight `es_data` objects with [`test_es_data()`](R/es_data.R) instead of hand-rolling `structure(..., class = c("es_data", "list"))`. The helper wraps `new_es_data()` so the class and slot layout stay aligned with the real constructor, then overwrites the slots you pass: + +```r +es <- test_es_data( + samplesheet = sample_sheet, + qc = qc_metrics_tables, + pxl_data_processed = pg_data +) +``` + +Use this for partial or synthetic fixtures. Prefer `build_es_data(params)` when the test needs the full ingestion pipeline. + --- ## Quarto report rendering -The ES report is built with Quarto (see `inst/quarto/`). Components return `ggplot` objects and/or `DT::datatables` objects from [`style_table()`](R/tables.R). To place them in the HTML report, `.qmd` chunks call helper functions that emit Quarto markdown via `cat()`. +The ES report is built with Quarto (see `inst/quarto/`). `pixelatorES.qmd` is a thin dispatcher: it loads the registered report recipe for `params$workflow`, knits the preamble children, then emits the panel tabset from the recipe sections. + +### Quarto layout + +```text +inst/quarto/ + pixelatorES.qmd # dispatcher shell + shared/ # reused across workflows + preprocessing.qmd + samples.qmd + run_info.qmd + workflows/ + amplicon_demux/ # workflow-owned sections + quality_metrics.qmd + cell_annotation.qmd + abundance.qmd + spatial.qmd +``` + +Components return `ggplot` objects and/or `DT::datatables` objects from [`style_table()`](R/tables.R). To place them in the HTML report, `.qmd` chunks call helper functions that emit Quarto markdown via `cat()`. **Requirement:** any chunk that uses these helpers must set `#| results: 'asis'` so printed output is passed through as raw markdown/HTML rather than wrapped in a code block. @@ -267,8 +334,6 @@ tabset_plotlist(plots, level = 5) - **`close`:** defaults to `TRUE`; the function opens and closes the tabset div. - **Use when:** a component returns multiple plots with no paired summary table, or tables are rendered separately. -See `inst/quarto/quality_metrics.qmd` (sequencing saturation curves) and `inst/quarto/abundance.qmd`. - ### `tabset_nested_plotlist()` Like `tabset_plotlist()`, but each list element may itself be a `list` of plots, producing nested tabsets. @@ -286,8 +351,6 @@ close_tabset() - **Use when:** one table accompanies several plots that should be grouped in sub-tabs (e.g. cell recovery molecule-rank plots). -See `inst/quarto/quality_metrics.qmd` (`qc_metrics_molrank_plot`) and `inst/quarto/spatial.qmd`. - ### `tabset_figure_table()` Renders a **figure + table** pair as a two-tab set ("Figure" and "Table"). @@ -346,13 +409,14 @@ Formats a data frame as a non-interactive (or interactive) DT table for report e ### Adding a new component to the report 1. Implement `component_*()` in [`R/components.R`](R/components.R) returning `ggplot` objects and/or `style_table()` output. -2. In the appropriate `inst/quarto/*.qmd` file, add a chunk with `results: 'asis'`. +2. In the appropriate section `.qmd` under `inst/quarto/shared/` or `inst/quarto/workflows//`, add a chunk with `results: 'asis'`. 3. Choose a layout helper: - **One plot + one table** → `tabset_figure_table()` + `close_tabset()` - **Several plots, no table** → `tabset_plotlist()` - **Several plots + one table** → `tabset_figure_table(..., mode = "tabset_nested")` + `close_tabset()` - **Table or intro only** → `section_table()` / `section_intro()` -4. Guard the chunk with `#| eval: !expr ...` when data may be absent (e.g. `!is.null(qc_metrics_tables$denoising)`). +4. Guard the chunk with `#| eval: !expr ...` when data may be absent (e.g. `!is.null(es_data$qc$denoising)`). +5. If the section is new for a workflow, add it to that workflow's registered report recipe. --- diff --git a/NAMESPACE b/NAMESPACE index 0553218..d701c76 100644 --- a/NAMESPACE +++ b/NAMESPACE @@ -34,17 +34,21 @@ export(component_sequencing_saturation_curve) export(convert_png_to_webp) export(create_sample_palette) export(default_params) +export(diagnostics_to_tibble) export(displayed_cell_types) export(downsample_data) export(extract_sample_qc_metrics) export(filter_proximity_scores) export(find_stage) +export(format_sample_diagnostics_callout) +export(format_sample_diagnostics_summary) export(format_with_info_bootstrap) export(get_coreness_data) export(get_crossing_edges) export(get_degree_distribution) export(get_denoising_data) export(get_denoising_detail_data) +export(get_es_workflow_report) export(get_file_paths) export(get_hash_stats) export(get_qc_metrics) @@ -52,6 +56,7 @@ export(get_read_stats) export(get_seq_saturation) export(get_test_data) export(get_top_markers) +export(has_sample_diagnostics) export(key_metric_table) export(list_es_data_workflows) export(load_pxl_data_list) @@ -65,6 +70,7 @@ export(plot_embeddings_samplewise) export(plot_violin) export(plot_void) export(preferred_dimred_order) +export(print_diagnostics_table) export(print_metadata_table) export(print_params) export(print_pixelator_version) @@ -85,6 +91,7 @@ export(tabset_figure_table) export(tabset_nested_plotlist) export(tabset_plotlist) export(test_data_folder) +export(test_es_data) export(test_samplesheet) export(theme_violin) export(title_plotlist) diff --git a/R/diagnostics.R b/R/diagnostics.R new file mode 100644 index 0000000..8979036 --- /dev/null +++ b/R/diagnostics.R @@ -0,0 +1,230 @@ +#' Convert Experiment Summary diagnostics to a table +#' +#' Converts the diagnostics recorded while building an `es_data` object to a +#' tibble with one row per diagnostic. +#' +#' @param es_data An `es_data` object. +#' +#' @return A tibble with the columns `type`, `target`, and `message`. +#' +#' @export +#' +diagnostics_to_tibble <- function(es_data) { + pixelatorR:::assert_class(es_data, "es_data") + + if (length(es_data$diagnostics) == 0) { + return(tibble( + type = character(), + target = character(), + message = character() + )) + } + + diagnostics <- bind_rows(es_data$diagnostics) %>% + select(type, target, message) + + return(diagnostics) +} + +#' Label a diagnostic type +#' +#' Translates the internal diagnostic types to labels shown in the report. +#' +#' @param type A character vector of diagnostic types. +#' +#' @return A character vector of labels. +#' +#' @noRd +#' +.diagnostic_type_label <- function(type) { + labels <- c( + pxl_load = "PXL loading", + qc_load = "QC loading", + extractor = "Analysis step" + ) + + return(unname(labels[type])) +} + +#' Find samples affected by diagnostics +#' +#' Finds sample aliases targeted by PXL or QC loading diagnostics. Diagnostics +#' targeting a pool affect every sample assigned to that pool. +#' +#' @param es_data An `es_data` object. +#' +#' @return A character vector of affected sample aliases. +#' +#' @noRd +#' +sample_diagnostic_targets <- function(es_data) { + pixelatorR:::assert_class(es_data, "es_data") + + sample_sheet <- es_data$samplesheet + if (is.null(sample_sheet) || nrow(sample_sheet) == 0) { + return(character()) + } + + diagnostics <- diagnostics_to_tibble(es_data) %>% + filter(type %in% c("pxl_load", "qc_load")) + + if (nrow(diagnostics) == 0) { + return(character()) + } + + sample_aliases <- as.character(sample_sheet$sample_alias) + affected_aliases <- diagnostics$target[ + diagnostics$target %in% sample_aliases + ] + + if ("pool" %in% names(sample_sheet)) { + affected_pools <- diagnostics$target[ + diagnostics$target %in% as.character(sample_sheet$pool) + ] + pool_aliases <- sample_sheet %>% + filter(as.character(pool) %in% affected_pools) %>% + pull(sample_alias) %>% + as.character() + affected_aliases <- c(affected_aliases, pool_aliases) + } + + return(unique(affected_aliases)) +} + +#' Check whether samples have diagnostics +#' +#' Checks whether any PXL or QC loading diagnostic targets a sample or its +#' pool. +#' +#' @param es_data An `es_data` object. +#' +#' @return `TRUE` when at least one sample is affected; otherwise `FALSE`. +#' +#' @export +#' +has_sample_diagnostics <- function(es_data) { + return(length(sample_diagnostic_targets(es_data)) > 0) +} + +#' Format report diagnostics for an Experiment Summary +#' +#' Formats sample- and pool-targeted loading diagnostics together with +#' analysis-step (`extractor`) diagnostics as Markdown lines for the Samples +#' page callout. +#' +#' @param es_data An `es_data` object. +#' +#' @return A single Markdown string, or `NULL` when there is nothing to show. +#' +#' @export +#' +format_sample_diagnostics_summary <- function(es_data) { + pixelatorR:::assert_class(es_data, "es_data") + + diagnostics <- diagnostics_to_tibble(es_data) + if (nrow(diagnostics) == 0) { + return(NULL) + } + + sample_sheet <- es_data$samplesheet + if (is.null(sample_sheet) || nrow(sample_sheet) == 0) { + valid_targets <- character() + } else if ("pool" %in% names(sample_sheet)) { + valid_targets <- c( + as.character(sample_sheet$sample_alias), + as.character(sample_sheet$pool) + ) + } else { + valid_targets <- as.character(sample_sheet$sample_alias) + } + + diagnostics <- diagnostics %>% + filter( + type == "extractor" | + ( + type %in% c("pxl_load", "qc_load") & + target %in% valid_targets + ) + ) + + if (nrow(diagnostics) == 0) { + return(NULL) + } + + lines <- paste0( + "- **", + htmltools::htmlEscape(diagnostics$target), + "** (", + .diagnostic_type_label(diagnostics$type), + "): ", + htmltools::htmlEscape(diagnostics$message) + ) + + return(paste(lines, collapse = "\n")) +} + +#' Format a report diagnostics callout for an Experiment Summary +#' +#' Formats loading and analysis-step diagnostics as a red Quarto callout so +#' incomplete report data is immediately visible on the Samples page. +#' +#' @param es_data An `es_data` object. +#' +#' @return A single Markdown string holding the callout, or `NULL` when there +#' is nothing to show. +#' +#' @export +#' +format_sample_diagnostics_callout <- function(es_data) { + summary_lines <- format_sample_diagnostics_summary(es_data) + + if (is.null(summary_lines)) { + return(NULL) + } + + body <- paste( + "Some input data could not be loaded or some analyses could not be", + "completed, and the metrics in this report are therefore incomplete." + ) + if (has_sample_diagnostics(es_data)) { + body <- paste( + body, + "Affected samples are marked with a warning symbol in the table below." + ) + } + + callout <- paste0( + '::: {.callout-important title="Report data issues"}\n', + body, + "\n\n", + summary_lines, + "\n\nSee the Diagnostics section under Run info for the complete list.\n", + ":::\n" + ) + + return(callout) +} + +#' Print Experiment Summary diagnostics +#' +#' Prints all diagnostics recorded while building an `es_data` object as a +#' styled table. +#' +#' @param es_data An `es_data` object. +#' +#' @return A `datatables` HTML widget containing all diagnostics. +#' +#' @export +#' +print_diagnostics_table <- function(es_data) { + diagnostics <- diagnostics_to_tibble(es_data) %>% + mutate(type = .diagnostic_type_label(type)) %>% + rename( + "Type" = type, + "Target" = target, + "Message" = message + ) %>% + style_table(interactive = FALSE, escape = TRUE) + + return(diagnostics) +} diff --git a/R/es_data.R b/R/es_data.R index fb9bb1e..0d9e40d 100644 --- a/R/es_data.R +++ b/R/es_data.R @@ -87,11 +87,109 @@ new_es_data <- function(params) { return(object) } +#' Build a lightweight `es_data` fixture for tests +#' +#' Creates an `es_data` object via [new_es_data()], then overwrites selected +#' slots. Use this in package and downstream tests instead of hand-rolling +#' `structure(..., class = c("es_data", "list"))`, so fixtures keep the real +#' constructor's slots and class as `es_data` evolves. +#' +#' When `sample_aliases` is `NULL` and `samplesheet` has a `sample_alias` +#' column, aliases are derived with the same helper used in production. When +#' `effective_samplesheet` is `NULL` and `samplesheet` is provided, it defaults +#' to `samplesheet`. +#' +#' @param samplesheet An Experiment Summary samplesheet, or `NULL`. +#' @param sample_aliases A named character vector of sample (and pool) aliases, +#' or `NULL` to derive from `samplesheet` when possible. +#' @param effective_samplesheet Samplesheet reduced to samples that loaded, or +#' `NULL` to default to `samplesheet` when provided. +#' @param qc Nested list of formatted QC tables. Defaults to an empty list. +#' @param qc_raw Raw QC data, or `NULL`. +#' @param pxl_data Merged raw PXL data, or `NULL`. +#' @param pxl_data_processed Processed Seurat object, or `NULL`. +#' @param proximity Proximity scores, or `NULL`. +#' @param file_paths Discovered input file paths, or `NULL`. +#' @param params Experiment Summary parameters passed to [new_es_data()]. +#' Defaults to an empty list (`workflow` defaults to `"amplicon_demux"`). +#' @param diagnostics List of diagnostics. Defaults to an empty list. +#' @param ... Named overrides for any other existing `es_data` slot (for +#' example `extractors`). Unknown names are an error. +#' +#' @return An `es_data` object suitable for unit tests. +#' +#' @examples +#' es <- test_es_data() +#' inherits(es, "es_data") +#' prox <- data.frame(marker = "CD3") +#' identical(test_es_data(proximity = prox)$proximity, prox) +#' +#' @export +test_es_data <- function( + samplesheet = NULL, + sample_aliases = NULL, + effective_samplesheet = NULL, + qc = list(), + qc_raw = NULL, + pxl_data = NULL, + pxl_data_processed = NULL, + proximity = NULL, + file_paths = NULL, + params = list(), + diagnostics = list(), + ... +) { + object <- new_es_data(params) + + dots <- list(...) + if (length(dots) > 0) { + if (is.null(names(dots)) || any(!nzchar(names(dots)))) { + cli_abort("All {.arg ...} arguments must be named.") + } + unknown <- setdiff(names(dots), names(object)) + if (length(unknown) > 0) { + cli_abort(c( + "{.arg ...} contains unknown {.cls es_data} slots.", + "x" = "Unknown: {.val {unknown}}.", + "i" = "Allowed slots: {.val {names(object)}}." + )) + } + } + + object$samplesheet <- samplesheet + object$sample_aliases <- sample_aliases + object$effective_samplesheet <- effective_samplesheet + object$qc <- qc + object$qc_raw <- qc_raw + object$pxl_data <- pxl_data + object$pxl_data_processed <- pxl_data_processed + object$proximity <- proximity + object$file_paths <- file_paths + object$diagnostics <- diagnostics + + for (nm in names(dots)) { + object[[nm]] <- dots[[nm]] + } + + if ( + is.null(object$sample_aliases) && + !is.null(object$samplesheet) && + "sample_alias" %in% names(object$samplesheet) + ) { + object$sample_aliases <- .sample_aliases_from_samplesheet(object$samplesheet) + } + + if (is.null(object$effective_samplesheet) && !is.null(object$samplesheet)) { + object$effective_samplesheet <- object$samplesheet + } + + return(object) +} + #' Extractor registry for the amplicon_demux workflow #' #' Nested named list of functions. Top-level names map to `es_data` slots; -#' nested names under `qc` map to `es_data$qc$...`. Phase 1b replaces stubs -#' with adapters around existing loaders and getters. +#' nested names under `qc` map to `es_data$qc$...`. #' #' @return A nested named list of functions. #' @@ -120,6 +218,47 @@ new_es_data <- function(params) { return(extractors) } +#' Report recipe for the amplicon_demux workflow +#' +#' Paths are relative to `inst/quarto/`. +#' +#' @return A report recipe list. +#' +#' @noRd +.amplicon_demux_report <- function() { + return(list( + preamble = c("shared/preprocessing.qmd"), + sections = list( + list(id = "samples", title = "Samples", child = "shared/samples.qmd"), + list( + id = "quality_metrics", + title = "Quality metrics", + child = "workflows/amplicon_demux/quality_metrics.qmd" + ), + list( + id = "cell_annotation", + title = "Cell annotation", + child = "workflows/amplicon_demux/cell_annotation.qmd" + ), + list( + id = "abundance", + title = "Abundance", + child = "workflows/amplicon_demux/abundance.qmd" + ), + list( + id = "spatial", + title = "Spatial metrics", + child = "workflows/amplicon_demux/spatial.qmd" + ), + list( + id = "run_info", + title = "Run info", + child = "shared/run_info.qmd" + ) + ) + )) +} + #' Extract the experiment samplesheet #' #' @param object An `es_data` object containing `params$sample_sheet`. @@ -369,9 +508,60 @@ new_es_data <- function(params) { values[[alias]] <- parsed$value } + diagnostics <- append( + diagnostics, + .qc_stage_completeness_diagnostics(values, target_label) + ) + return(.new_es_data_extractor_result(values, diagnostics)) } +#' Flag QC stages missing relative to peers +#' +#' Compares the QC stages present for successfully loaded samples or pools. If +#' any peer has a stage that another loaded entity lacks, a `qc_load` +#' diagnostic is recorded for that entity. Entities with no QC files are +#' excluded from the comparison. +#' +#' @param values Named list of successfully loaded QC groups, each keyed by +#' stage. +#' @param target_label Label used in warning and diagnostic messages +#' (`"sample"` or `"pool"`). +#' +#' @return A list of `qc_load` diagnostics, one per incomplete entity. +#' +#' @noRd +.qc_stage_completeness_diagnostics <- function(values, target_label) { + if (length(values) < 2) { + return(list()) + } + + peer_stages <- sort(unique(unlist(lapply(values, names), use.names = FALSE))) + diagnostics <- list() + + for (alias in names(values)) { + missing_stages <- setdiff(peer_stages, names(values[[alias]])) + if (length(missing_stages) == 0) { + next + } + + message <- paste0( + "Missing QC stages: ", + paste(missing_stages, collapse = ", "), + "." + ) + cli::cli_warn( + "Incomplete QC data for {target_label} {.val {alias}}: {message}" + ) + diagnostics <- append( + diagnostics, + list(.new_es_data_diagnostic("qc_load", alias, message)) + ) + } + + return(diagnostics) +} + #' Extract formatted read statistics #' #' @param object An `es_data` object. diff --git a/R/params.R b/R/params.R index d998ed4..c652ed2 100644 --- a/R/params.R +++ b/R/params.R @@ -52,10 +52,11 @@ print_params <- #' Print metadata table #' -#' Print the experiment meta data in a table format. +#' Print the experiment meta data in a table format. For hashed experiments, +#' `% of pool` is included only when processed PXL data is available. #' -#' @param es_data An `es_data` object containing the samplesheet and processed -#' PXL data. +#' @param es_data An `es_data` object containing the samplesheet. Processed +#' PXL data is used when present to compute pool fractions. #' #' @return A printed table of sample metadata. #' @@ -65,15 +66,31 @@ print_metadata_table <- function(es_data) { pixelatorR:::assert_class(es_data, "es_data") sample_sheet <- es_data$samplesheet - if ("pool" %in% names(sample_sheet)) { + if ( + "pool" %in% names(sample_sheet) && + !is.null(es_data$pxl_data_processed) + ) { sample_sheet <- add_pct_of_pool_to_samplesheet( sample_sheet, es_data$pxl_data_processed ) } + if (has_sample_diagnostics(es_data)) { + flagged_aliases <- sample_diagnostic_targets(es_data) + sample_sheet <- sample_sheet %>% + mutate( + Issues = ifelse( + sample_alias %in% flagged_aliases, + "\u26A0", + "" + ) + ) + } + sample_sheet %>% select( + any_of("Issues"), "Pool" = any_of("pool"), "Sample Alias" = sample_alias, "Sample name" = sample, diff --git a/R/workflow_registry.R b/R/workflow_registry.R index 83209e5..a3a84d1 100644 --- a/R/workflow_registry.R +++ b/R/workflow_registry.R @@ -1,30 +1,203 @@ #' Registered Experiment Summary workflows #' -#' Mutable package-local registry mapping workflow identifiers to zero-argument -#' functions that return extractor lists. +#' Mutable package-local registry mapping workflow identifiers to workflow +#' definitions. Each definition contains zero-argument factory functions for +#' extractors and the report recipe. #' #' @noRd .es_data_workflow_registry <- new.env(parent = emptyenv()) +#' Resolve a report child path for existence checks +#' +#' Absolute paths are returned unchanged. Relative paths are resolved against +#' `inst/quarto/` in pixelatorES (built-in workflow convention). +#' +#' @param path A child document path. +#' +#' @return The path to check with [file.exists()]. +#' +#' @noRd +.resolve_es_workflow_report_path <- function(path) { + if (grepl("^(~|/|[A-Za-z]:[/\\\\]|\\\\\\\\)", path)) { + return(path) + } + + quarto_root <- system.file("quarto", package = "pixelatorES") + if (!nzchar(quarto_root)) { + cli_abort("Could not locate installed {.pkg pixelatorES} Quarto directory.") + } + + return(file.path(quarto_root, path)) +} + +#' Validate that report recipe child paths exist +#' +#' @param report A structurally valid report recipe. +#' +#' @return `report`, invisibly. +#' +#' @noRd +.validate_es_workflow_report_paths <- function(report) { + paths <- c( + report$preamble, + vapply(report$sections, function(section) { + return(section$child) + }, character(1)) + ) + resolved <- vapply(paths, .resolve_es_workflow_report_path, character(1)) + missing <- paths[!file.exists(resolved)] + if (length(missing) > 0) { + cli_abort(c( + "Report recipe references missing Quarto child documents.", + "x" = "Missing: {.val {unique(missing)}}.", + "i" = paste( + "Built-in workflows use paths relative to {.path inst/quarto/};", + "extension packages should register absolute paths from {.fn system.file}." + ) + )) + } + + return(invisible(report)) +} + +#' Validate an Experiment Summary report recipe +#' +#' @param report A report recipe list. +#' +#' @return The validated `report`. +#' +#' @noRd +.validate_es_workflow_report <- function(report) { + pixelatorR:::assert_class(report, "list") + if (is.null(names(report)) || any(!nzchar(names(report)))) { + cli_abort("{.arg report} must be a named list.") + } + + unexpected <- setdiff(names(report), c("preamble", "sections")) + if (length(unexpected) > 0) { + cli_abort(c( + "{.arg report} contains unexpected elements.", + "x" = "Unexpected: {.val {unexpected}}.", + "i" = "Allowed elements: {.val preamble}, {.val sections}." + )) + } + + if (is.null(report$preamble)) { + cli_abort("{.arg report} must contain {.field preamble}.") + } + # assert_vector(..., n) requires at least n elements (not exactly n). + pixelatorR:::assert_vector(report$preamble, "character", n = 1) + if (any(!nzchar(report$preamble))) { + cli_abort("{.arg report$preamble} must not contain empty paths.") + } + + if (is.null(report$sections)) { + cli_abort("{.arg report} must contain {.field sections}.") + } + pixelatorR:::assert_class(report$sections, "list") + if (length(report$sections) == 0) { + cli_abort("{.arg report$sections} must contain at least one section.") + } + + section_ids <- character(length(report$sections)) + for (i in seq_along(report$sections)) { + section <- report$sections[[i]] + pixelatorR:::assert_class(section, "list") + for (field in c("id", "title", "child")) { + if (is.null(section[[field]])) { + cli_abort(c( + "Report section {.val {i}} is missing {.field {field}}.", + "i" = "Each section needs {.field id}, {.field title}, and {.field child}." + )) + } + pixelatorR:::assert_single_value(section[[field]], "string") + if (!nzchar(section[[field]])) { + cli_abort("Report section {.val {i}} has an empty {.field {field}}.") + } + } + section_ids[[i]] <- section$id + } + + if (anyDuplicated(section_ids) > 0) { + duplicated_ids <- unique(section_ids[duplicated(section_ids)]) + cli_abort(c( + "{.arg report$sections} contains duplicated section ids.", + "x" = "Duplicated: {.val {duplicated_ids}}." + )) + } + + .validate_es_workflow_report_paths(report) + + return(report) +} + +#' Validate a workflow report factory +#' +#' @param report A zero-argument function returning a report recipe. +#' +#' @return `report`, invisibly. +#' +#' @noRd +.validate_es_workflow_report_factory <- function(report) { + pixelatorR:::assert_class(report, "function") + .validate_es_workflow_report(report()) + + return(invisible(report)) +} + +#' Validate a workflow extractor factory +#' +#' @param extractors A zero-argument function returning a nested named list of +#' extractor functions. +#' +#' @return `extractors`, invisibly. +#' +#' @noRd +.validate_es_workflow_extractors <- function(extractors) { + pixelatorR:::assert_class(extractors, "function") + + extractor_list <- extractors() + pixelatorR:::assert_class(extractor_list, "list") + + return(invisible(extractors)) +} + #' Register an Experiment Summary workflow #' -#' Registers a workflow identifier with a zero-argument function that returns -#' the nested extractor list used to build an `es_data` object. Extension -#' packages should call this function from their `.onLoad()` hook. +#' Registers a workflow identifier with its extractor factory and Quarto report +#' recipe. Extension packages should call this from their `.onLoad()` hook. +#' Workflows are the only supported way to build Experiment Summary data and +#' render the report. #' #' @param name A unique workflow identifier. #' @param extractors A zero-argument function returning a nested named list of -#' extractor functions. +#' extractor functions. The factory is called at registration time to verify +#' that it returns a list. +#' @param report A zero-argument function returning the report recipe used by +#' the Quarto shell. The recipe is a named list with: +#' - `preamble`: non-empty character vector of child document paths knitted +#' before the tabset (for example data ingestion). +#' - `sections`: non-empty list of tabset sections, each a list with `id`, +#' `title`, and `child`. +#' Built-in workflows use paths relative to `inst/quarto/`. Extension packages +#' should register absolute paths from `system.file()`. All referenced child +#' paths must exist at registration time. #' @param overwrite If `TRUE`, replace an existing registration for `name`. #' Defaults to `FALSE`. #' #' @return `name`, invisibly. #' #' @export -register_es_data_workflow <- function(name, extractors, overwrite = FALSE) { +register_es_data_workflow <- function( + name, + extractors, + report, + overwrite = FALSE +) { pixelatorR:::assert_single_value(name, "string") - pixelatorR:::assert_class(extractors, "function") pixelatorR:::assert_single_value(overwrite, "bool") + .validate_es_workflow_extractors(extractors) + .validate_es_workflow_report_factory(report) if ( !overwrite && @@ -33,7 +206,11 @@ register_es_data_workflow <- function(name, extractors, overwrite = FALSE) { cli_abort("Workflow {.val {name}} is already registered.") } - assign(name, extractors, envir = .es_data_workflow_registry) + assign( + name, + list(extractors = extractors, report = report), + envir = .es_data_workflow_registry + ) return(invisible(name)) } @@ -47,14 +224,14 @@ list_es_data_workflows <- function() { return(ls(envir = .es_data_workflow_registry, all.names = TRUE, sorted = TRUE)) } -#' Get extractors for a registered workflow +#' Get a registered workflow definition #' #' @param name A workflow identifier. #' -#' @return A nested named list of extractor functions. +#' @return A list with `extractors` and `report`. #' #' @noRd -.get_es_data_extractors <- function(name) { +.get_es_workflow_definition <- function(name) { pixelatorR:::assert_single_value(name, "string") if (!exists(name, envir = .es_data_workflow_registry, inherits = FALSE)) { @@ -64,13 +241,35 @@ list_es_data_workflows <- function() { )) } - extractor_factory <- get( + return(get( name, envir = .es_data_workflow_registry, inherits = FALSE - ) - extractors <- extractor_factory() - pixelatorR:::assert_class(extractors, "list") + )) +} + +#' Get extractors for a registered workflow +#' +#' @param name A workflow identifier. +#' +#' @return A nested named list of extractor functions. +#' +#' @noRd +.get_es_data_extractors <- function(name) { + return(.get_es_workflow_definition(name)$extractors()) +} - return(extractors) +#' Get the report recipe for a registered workflow +#' +#' Returns the Quarto report recipe registered for `name`. Built-in workflows +#' use paths relative to `inst/quarto/`; extension packages may register +#' absolute paths. +#' +#' @param name A workflow identifier. +#' +#' @return A report recipe list with `preamble` and `sections`. +#' +#' @export +get_es_workflow_report <- function(name) { + return(.get_es_workflow_definition(name)$report()) } diff --git a/R/zzz.R b/R/zzz.R index d1dfcbc..4529a58 100644 --- a/R/zzz.R +++ b/R/zzz.R @@ -10,6 +10,7 @@ register_es_data_workflow( name = "amplicon_demux", extractors = .amplicon_demux_extractors, + report = .amplicon_demux_report, overwrite = TRUE ) diff --git a/inst/quarto/pixelatorES.qmd b/inst/quarto/pixelatorES.qmd index b60e530..d9cb405 100644 --- a/inst/quarto/pixelatorES.qmd +++ b/inst/quarto/pixelatorES.qmd @@ -1,6 +1,6 @@ --- title: "PROXIOME EXPERIMENT SUMMARY" -subtitle: "v0.12.0" +subtitle: "v0.13.0" date: "`r Sys.Date()`" editor_options: chunk_output_type: console @@ -17,42 +17,36 @@ params: test_mode: FALSE --- -```{r, child="preprocessing.qmd"} -``` - -# - -::: {.panel-tabset .nav-pills} - -## Samples - -```{r, child="samples.qmd"} -``` - -## Quality metrics +```{r} +#| label: setup_report +#| include: false -```{r, child="quality_metrics.qmd"} +library(pixelatorES) +report <- get_es_workflow_report(params$workflow) ``` -## Cell annotation +```{r} +#| label: preamble +#| results: asis +#| echo: false -```{r, child="cell_annotation.qmd"} +for (child in report$preamble) { + cat(knitr::knit_child(child, quiet = TRUE)) +} ``` -## Abundance - -```{r, child="abundance.qmd"} +```{r} +#| label: sections +#| results: asis +#| echo: false + +cat("\n# \n\n") +cat("::: {.panel-tabset .nav-pills}\n\n") +for (section in report$sections) { + # Two newlines so the heading starts a new block even when the preceding + # child ends with a list or paragraph + cat("\n\n## ", section$title, "\n\n", sep = "") + cat(knitr::knit_child(section$child, quiet = TRUE)) +} +cat("\n:::\n") ``` - -## Spatial metrics - -```{r, child="spatial.qmd"} -``` - -## Run settings - -```{r, child="run_settings.qmd"} -``` - - -::: diff --git a/inst/quarto/run_settings.qmd b/inst/quarto/run_settings.qmd deleted file mode 100644 index 3759340..0000000 --- a/inst/quarto/run_settings.qmd +++ /dev/null @@ -1,27 +0,0 @@ - -### Pixelator version - -```{r} -#| label: run_settings_version - -# Print Pixelator version -print_pixelator_version(es_data) -``` - -### Run parameters - -```{r} -#| label: run_settings_params -# Print parameters -print_params(es_data) -``` - -### Session info - -```{r} -#| label: run_settings_session_info - -# Print session info -print_session_info() -``` - diff --git a/inst/quarto/samples.qmd b/inst/quarto/samples.qmd deleted file mode 100644 index ef4ade1..0000000 --- a/inst/quarto/samples.qmd +++ /dev/null @@ -1,8 +0,0 @@ - -### Sample description - -```{r} -#| label: samples -#| results: 'asis' -print_metadata_table(es_data) -``` diff --git a/inst/quarto/preprocessing.qmd b/inst/quarto/shared/preprocessing.qmd similarity index 100% rename from inst/quarto/preprocessing.qmd rename to inst/quarto/shared/preprocessing.qmd diff --git a/inst/quarto/shared/run_info.qmd b/inst/quarto/shared/run_info.qmd new file mode 100644 index 0000000..5ee6762 --- /dev/null +++ b/inst/quarto/shared/run_info.qmd @@ -0,0 +1,53 @@ + +```{r} +#| label: run_info_diagnostics_setup +eval_diagnostics <- length(es_data$diagnostics) > 0 +``` + +```{r} +#| label: run_info_diagnostics_intro +#| eval: !expr eval_diagnostics +#| results: 'asis' +section_intro( + "Diagnostics", + paste( + "The report was generated with incomplete data. Each entry below describes", + "a sample, pool, or analysis step that could not be loaded or computed." + ), + 3 +) +``` + +```{r} +#| label: run_info_diagnostics_table +#| eval: !expr eval_diagnostics + +# Print diagnostics recorded while building the report data +print_diagnostics_table(es_data) +``` + +### Pixelator version + +```{r} +#| label: run_info_version + +# Print Pixelator version +print_pixelator_version(es_data) +``` + +### Run parameters + +```{r} +#| label: run_info_params +# Print parameters +print_params(es_data) +``` + +### Session info + +```{r} +#| label: run_info_session_info + +# Print session info +print_session_info() +``` diff --git a/inst/quarto/shared/samples.qmd b/inst/quarto/shared/samples.qmd new file mode 100644 index 0000000..8153682 --- /dev/null +++ b/inst/quarto/shared/samples.qmd @@ -0,0 +1,15 @@ + +```{r} +#| label: samples_diagnostics_callout +#| eval: !expr length(es_data$diagnostics) > 0 +#| results: 'asis' +cat(format_sample_diagnostics_callout(es_data)) +``` + +### Sample description + +```{r} +#| label: samples +#| results: 'asis' +print_metadata_table(es_data) +``` diff --git a/inst/quarto/abundance.qmd b/inst/quarto/workflows/amplicon_demux/abundance.qmd similarity index 100% rename from inst/quarto/abundance.qmd rename to inst/quarto/workflows/amplicon_demux/abundance.qmd diff --git a/inst/quarto/cell_annotation.qmd b/inst/quarto/workflows/amplicon_demux/cell_annotation.qmd similarity index 100% rename from inst/quarto/cell_annotation.qmd rename to inst/quarto/workflows/amplicon_demux/cell_annotation.qmd diff --git a/inst/quarto/quality_metrics.qmd b/inst/quarto/workflows/amplicon_demux/quality_metrics.qmd similarity index 100% rename from inst/quarto/quality_metrics.qmd rename to inst/quarto/workflows/amplicon_demux/quality_metrics.qmd diff --git a/inst/quarto/spatial.qmd b/inst/quarto/workflows/amplicon_demux/spatial.qmd similarity index 100% rename from inst/quarto/spatial.qmd rename to inst/quarto/workflows/amplicon_demux/spatial.qmd diff --git a/man/diagnostics_to_tibble.Rd b/man/diagnostics_to_tibble.Rd new file mode 100644 index 0000000..09453a4 --- /dev/null +++ b/man/diagnostics_to_tibble.Rd @@ -0,0 +1,18 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/diagnostics.R +\name{diagnostics_to_tibble} +\alias{diagnostics_to_tibble} +\title{Convert Experiment Summary diagnostics to a table} +\usage{ +diagnostics_to_tibble(es_data) +} +\arguments{ +\item{es_data}{An \code{es_data} object.} +} +\value{ +A tibble with the columns \code{type}, \code{target}, and \code{message}. +} +\description{ +Converts the diagnostics recorded while building an \code{es_data} object to a +tibble with one row per diagnostic. +} diff --git a/man/format_sample_diagnostics_callout.Rd b/man/format_sample_diagnostics_callout.Rd new file mode 100644 index 0000000..062d45c --- /dev/null +++ b/man/format_sample_diagnostics_callout.Rd @@ -0,0 +1,19 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/diagnostics.R +\name{format_sample_diagnostics_callout} +\alias{format_sample_diagnostics_callout} +\title{Format a report diagnostics callout for an Experiment Summary} +\usage{ +format_sample_diagnostics_callout(es_data) +} +\arguments{ +\item{es_data}{An \code{es_data} object.} +} +\value{ +A single Markdown string holding the callout, or \code{NULL} when there +is nothing to show. +} +\description{ +Formats loading and analysis-step diagnostics as a red Quarto callout so +incomplete report data is immediately visible on the Samples page. +} diff --git a/man/format_sample_diagnostics_summary.Rd b/man/format_sample_diagnostics_summary.Rd new file mode 100644 index 0000000..32e5eb3 --- /dev/null +++ b/man/format_sample_diagnostics_summary.Rd @@ -0,0 +1,19 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/diagnostics.R +\name{format_sample_diagnostics_summary} +\alias{format_sample_diagnostics_summary} +\title{Format report diagnostics for an Experiment Summary} +\usage{ +format_sample_diagnostics_summary(es_data) +} +\arguments{ +\item{es_data}{An \code{es_data} object.} +} +\value{ +A single Markdown string, or \code{NULL} when there is nothing to show. +} +\description{ +Formats sample- and pool-targeted loading diagnostics together with +analysis-step (\code{extractor}) diagnostics as Markdown lines for the Samples +page callout. +} diff --git a/man/get_es_workflow_report.Rd b/man/get_es_workflow_report.Rd new file mode 100644 index 0000000..cdb4d42 --- /dev/null +++ b/man/get_es_workflow_report.Rd @@ -0,0 +1,19 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/workflow_registry.R +\name{get_es_workflow_report} +\alias{get_es_workflow_report} +\title{Get the report recipe for a registered workflow} +\usage{ +get_es_workflow_report(name) +} +\arguments{ +\item{name}{A workflow identifier.} +} +\value{ +A report recipe list with \code{preamble} and \code{sections}. +} +\description{ +Returns the Quarto report recipe registered for \code{name}. Built-in workflows +use paths relative to \verb{inst/quarto/}; extension packages may register +absolute paths. +} diff --git a/man/has_sample_diagnostics.Rd b/man/has_sample_diagnostics.Rd new file mode 100644 index 0000000..c17f345 --- /dev/null +++ b/man/has_sample_diagnostics.Rd @@ -0,0 +1,18 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/diagnostics.R +\name{has_sample_diagnostics} +\alias{has_sample_diagnostics} +\title{Check whether samples have diagnostics} +\usage{ +has_sample_diagnostics(es_data) +} +\arguments{ +\item{es_data}{An \code{es_data} object.} +} +\value{ +\code{TRUE} when at least one sample is affected; otherwise \code{FALSE}. +} +\description{ +Checks whether any PXL or QC loading diagnostic targets a sample or its +pool. +} diff --git a/man/print_diagnostics_table.Rd b/man/print_diagnostics_table.Rd new file mode 100644 index 0000000..e5361db --- /dev/null +++ b/man/print_diagnostics_table.Rd @@ -0,0 +1,18 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/diagnostics.R +\name{print_diagnostics_table} +\alias{print_diagnostics_table} +\title{Print Experiment Summary diagnostics} +\usage{ +print_diagnostics_table(es_data) +} +\arguments{ +\item{es_data}{An \code{es_data} object.} +} +\value{ +A \code{datatables} HTML widget containing all diagnostics. +} +\description{ +Prints all diagnostics recorded while building an \code{es_data} object as a +styled table. +} diff --git a/man/print_metadata_table.Rd b/man/print_metadata_table.Rd index 46166b4..e0387c7 100644 --- a/man/print_metadata_table.Rd +++ b/man/print_metadata_table.Rd @@ -7,12 +7,13 @@ print_metadata_table(es_data) } \arguments{ -\item{es_data}{An \code{es_data} object containing the samplesheet and processed -PXL data.} +\item{es_data}{An \code{es_data} object containing the samplesheet. Processed +PXL data is used when present to compute pool fractions.} } \value{ A printed table of sample metadata. } \description{ -Print the experiment meta data in a table format. +Print the experiment meta data in a table format. For hashed experiments, +\verb{\% of pool} is included only when processed PXL data is available. } diff --git a/man/register_es_data_workflow.Rd b/man/register_es_data_workflow.Rd index 1ed2a2a..838ac6f 100644 --- a/man/register_es_data_workflow.Rd +++ b/man/register_es_data_workflow.Rd @@ -4,13 +4,26 @@ \alias{register_es_data_workflow} \title{Register an Experiment Summary workflow} \usage{ -register_es_data_workflow(name, extractors, overwrite = FALSE) +register_es_data_workflow(name, extractors, report, overwrite = FALSE) } \arguments{ \item{name}{A unique workflow identifier.} \item{extractors}{A zero-argument function returning a nested named list of -extractor functions.} +extractor functions. The factory is called at registration time to verify +that it returns a list.} + +\item{report}{A zero-argument function returning the report recipe used by +the Quarto shell. The recipe is a named list with: +\itemize{ +\item \code{preamble}: non-empty character vector of child document paths knitted +before the tabset (for example data ingestion). +\item \code{sections}: non-empty list of tabset sections, each a list with \code{id}, +\code{title}, and \code{child}. +Built-in workflows use paths relative to \verb{inst/quarto/}. Extension packages +should register absolute paths from \code{system.file()}. All referenced child +paths must exist at registration time. +}} \item{overwrite}{If \code{TRUE}, replace an existing registration for \code{name}. Defaults to \code{FALSE}.} @@ -19,7 +32,8 @@ Defaults to \code{FALSE}.} \code{name}, invisibly. } \description{ -Registers a workflow identifier with a zero-argument function that returns -the nested extractor list used to build an \code{es_data} object. Extension -packages should call this function from their \code{.onLoad()} hook. +Registers a workflow identifier with its extractor factory and Quarto report +recipe. Extension packages should call this from their \code{.onLoad()} hook. +Workflows are the only supported way to build Experiment Summary data and +render the report. } diff --git a/man/test_es_data.Rd b/man/test_es_data.Rd new file mode 100644 index 0000000..95e866b --- /dev/null +++ b/man/test_es_data.Rd @@ -0,0 +1,72 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/es_data.R +\name{test_es_data} +\alias{test_es_data} +\title{Build a lightweight \code{es_data} fixture for tests} +\usage{ +test_es_data( + samplesheet = NULL, + sample_aliases = NULL, + effective_samplesheet = NULL, + qc = list(), + qc_raw = NULL, + pxl_data = NULL, + pxl_data_processed = NULL, + proximity = NULL, + file_paths = NULL, + params = list(), + diagnostics = list(), + ... +) +} +\arguments{ +\item{samplesheet}{An Experiment Summary samplesheet, or \code{NULL}.} + +\item{sample_aliases}{A named character vector of sample (and pool) aliases, +or \code{NULL} to derive from \code{samplesheet} when possible.} + +\item{effective_samplesheet}{Samplesheet reduced to samples that loaded, or +\code{NULL} to default to \code{samplesheet} when provided.} + +\item{qc}{Nested list of formatted QC tables. Defaults to an empty list.} + +\item{qc_raw}{Raw QC data, or \code{NULL}.} + +\item{pxl_data}{Merged raw PXL data, or \code{NULL}.} + +\item{pxl_data_processed}{Processed Seurat object, or \code{NULL}.} + +\item{proximity}{Proximity scores, or \code{NULL}.} + +\item{file_paths}{Discovered input file paths, or \code{NULL}.} + +\item{params}{Experiment Summary parameters passed to \code{\link[=new_es_data]{new_es_data()}}. +Defaults to an empty list (\code{workflow} defaults to \code{"amplicon_demux"}).} + +\item{diagnostics}{List of diagnostics. Defaults to an empty list.} + +\item{...}{Named overrides for any other existing \code{es_data} slot (for +example \code{extractors}). Unknown names are an error.} +} +\value{ +An \code{es_data} object suitable for unit tests. +} +\description{ +Creates an \code{es_data} object via \code{\link[=new_es_data]{new_es_data()}}, then overwrites selected +slots. Use this in package and downstream tests instead of hand-rolling +\code{structure(..., class = c("es_data", "list"))}, so fixtures keep the real +constructor's slots and class as \code{es_data} evolves. +} +\details{ +When \code{sample_aliases} is \code{NULL} and \code{samplesheet} has a \code{sample_alias} +column, aliases are derived with the same helper used in production. When +\code{effective_samplesheet} is \code{NULL} and \code{samplesheet} is provided, it defaults +to \code{samplesheet}. +} +\examples{ +es <- test_es_data() +inherits(es, "es_data") +prox <- data.frame(marker = "CD3") +identical(test_es_data(proximity = prox)$proximity, prox) + +} diff --git a/tests/testthat/test_components.R b/tests/testthat/test_components.R index 9c75204..d14ef7e 100644 --- a/tests/testthat/test_components.R +++ b/tests/testthat/test_components.R @@ -27,22 +27,17 @@ for (data_type in data_types) { qc_metrics_tables <- get_qc_metrics(pg_data, sample_qc_metrics, sample_sheet) - es_data <- structure( - list( - params = list( - control_markers = c("mIgG1", "mIgG2a", "mIgG2b") - ), - samplesheet = sample_sheet, - sample_aliases = - pixelatorES:::.sample_aliases_from_samplesheet(sample_sheet), - effective_samplesheet = sample_sheet, - file_paths = file_paths, - pxl_data_processed = pg_data, - qc_raw = sample_qc_metrics, - qc = qc_metrics_tables, - proximity = NULL + es_data <- test_es_data( + params = list( + control_markers = c("mIgG1", "mIgG2a", "mIgG2b") ), - class = c("es_data", "list") + samplesheet = sample_sheet, + effective_samplesheet = sample_sheet, + file_paths = file_paths, + pxl_data_processed = pg_data, + qc_raw = sample_qc_metrics, + qc = qc_metrics_tables, + proximity = NULL ) test_message <- paste("Components work as expected for", data_type, "data") diff --git a/tests/testthat/test_diagnostics.R b/tests/testthat/test_diagnostics.R new file mode 100644 index 0000000..976a6d5 --- /dev/null +++ b/tests/testthat/test_diagnostics.R @@ -0,0 +1,290 @@ +test_that("Sample diagnostics work as expected", { + samplesheet <- tibble( + pool = c("pool1", "pool1"), + sample = c("one", "two"), + sample_alias = c("S1", "S2"), + condition = c("A", "B") + ) + + clean <- test_es_data(samplesheet = samplesheet) + expect_equal( + list( + tibble = diagnostics_to_tibble(clean), + has_sample = has_sample_diagnostics(clean), + targets = pixelatorES:::sample_diagnostic_targets(clean), + summary = format_sample_diagnostics_summary(clean), + callout = format_sample_diagnostics_callout(clean) + ), + list( + tibble = tibble( + type = character(), + target = character(), + message = character() + ), + has_sample = FALSE, + targets = character(), + summary = NULL, + callout = NULL + ) + ) + + loading <- test_es_data( + samplesheet = samplesheet, + diagnostics = list( + list( + type = "pxl_load", + target = "S1", + message = "No PXL file was found." + ), + list( + type = "qc_load", + target = "pool1", + message = "No QC files were found." + ) + ) + ) + expect_equal( + list( + has_sample = has_sample_diagnostics(loading), + targets = pixelatorES:::sample_diagnostic_targets(loading), + summary = format_sample_diagnostics_summary(loading), + callout = format_sample_diagnostics_callout(loading) + ), + list( + has_sample = TRUE, + targets = c("S1", "S2"), + summary = paste( + "- **S1** (PXL loading): No PXL file was found.", + "- **pool1** (QC loading): No QC files were found.", + sep = "\n" + ), + callout = paste0( + '::: {.callout-important title="Report data issues"}\n', + "Some input data could not be loaded or some analyses could not be ", + "completed, and the metrics in this report are therefore incomplete. ", + "Affected samples are marked with a warning symbol in the table below.", + "\n\n", + "- **S1** (PXL loading): No PXL file was found.\n", + "- **pool1** (QC loading): No QC files were found.", + "\n\nSee the Diagnostics section under Run info for the complete list.\n", + ":::\n" + ) + ) + ) + + extractor_only <- test_es_data( + samplesheet = samplesheet, + diagnostics = list(list( + type = "extractor", + target = "pxl_data_processed", + message = "You've supplied a object." + )) + ) + expect_equal( + list( + has_sample = has_sample_diagnostics(extractor_only), + targets = pixelatorES:::sample_diagnostic_targets(extractor_only), + summary = format_sample_diagnostics_summary(extractor_only), + callout = format_sample_diagnostics_callout(extractor_only) + ), + list( + has_sample = FALSE, + targets = character(), + summary = paste0( + "- **pxl_data_processed** (Analysis step): ", + "You've supplied a <NULL> object." + ), + callout = paste0( + '::: {.callout-important title="Report data issues"}\n', + "Some input data could not be loaded or some analyses could not be ", + "completed, and the metrics in this report are therefore incomplete.", + "\n\n", + "- **pxl_data_processed** (Analysis step): ", + "You've supplied a <NULL> object.", + "\n\nSee the Diagnostics section under Run info for the complete list.\n", + ":::\n" + ) + ) + ) + + mixed <- test_es_data( + samplesheet = samplesheet, + diagnostics = list( + list( + type = "pxl_load", + target = "S1", + message = "No PXL file was found." + ), + list( + type = "extractor", + target = "proximity", + message = "Unavailable." + ) + ) + ) + expect_equal( + list( + has_sample = has_sample_diagnostics(mixed), + targets = pixelatorES:::sample_diagnostic_targets(mixed), + summary = format_sample_diagnostics_summary(mixed) + ), + list( + has_sample = TRUE, + targets = "S1", + summary = paste( + "- **S1** (PXL loading): No PXL file was found.", + "- **proximity** (Analysis step): Unavailable.", + sep = "\n" + ) + ) + ) +}) + +test_that("Relative QC stage completeness diagnostics work as expected", { + equal_values <- list( + S1 = list(analysis = list(), denoise = list()), + S2 = list(analysis = list(), denoise = list()) + ) + expect_equal( + pixelatorES:::.qc_stage_completeness_diagnostics(equal_values, "sample"), + list() + ) + + expect_warning( + incomplete <- pixelatorES:::.qc_stage_completeness_diagnostics( + list( + S1 = list(analysis = list(), denoise = list(), graph = list()), + S2 = list(analysis = list()) + ), + "sample" + ), + "Incomplete QC data for sample" + ) + expect_equal( + incomplete, + list(list( + type = "qc_load", + target = "S2", + message = "Missing QC stages: denoise, graph." + )) + ) + + sample_sheet <- read_samplesheet(test_samplesheet(type = "hashing")) + data_folder <- tempfile("pixelatorES_test_hashing_") + dir.create(data_folder) + stopifnot(all(file.copy( + list.files(test_data_folder(type = "hashing"), full.names = TRUE), + data_folder, + recursive = TRUE + ))) + file.remove(file.path(data_folder, "denoise", "S2.report.json")) + file.remove(file.path(data_folder, "graph", "pool2.report.json")) + file_paths <- get_file_paths( + data_folder = data_folder, + sample_sheet = sample_sheet + ) + + expect_warning( + sample_result <- pixelatorES:::.read_qc_groups_soft( + files = file_paths$qc_files, + aliases = sort(sample_sheet$sample_alias), + alias_column = "sample_alias", + target_label = "sample" + ), + "Incomplete QC data for sample" + ) + expect_equal( + lapply( + Filter( + function(diagnostic) { + return(grepl("^Missing QC stages", diagnostic$message)) + }, + sample_result$diagnostics + ), + function(diagnostic) { + return(diagnostic[c("type", "target", "message")]) + } + ), + list(list( + type = "qc_load", + target = "S2", + message = "Missing QC stages: denoise." + )) + ) + + expect_warning( + pool_result <- pixelatorES:::.read_qc_groups_soft( + files = file_paths$pool_qc_files, + aliases = sort(unique(sample_sheet$pool)), + alias_column = "pool_alias", + target_label = "pool" + ), + "Incomplete QC data for pool" + ) + expect_equal( + lapply( + Filter( + function(diagnostic) { + return(grepl("^Missing QC stages", diagnostic$message)) + }, + pool_result$diagnostics + ), + function(diagnostic) { + return(diagnostic[c("type", "target", "message")]) + } + ), + list(list( + type = "qc_load", + target = "pool2", + message = "Missing QC stages: graph." + )) + ) + + expect_warning( + zero_qc_result <- pixelatorES:::.read_qc_groups_soft( + files = file_paths$qc_files[ + file_paths$qc_files$sample_alias == "S1", + , + drop = FALSE + ], + aliases = c("S1", "S2"), + alias_column = "sample_alias", + target_label = "sample" + ), + "No QC files were found" + ) + expect_equal( + lapply( + zero_qc_result$diagnostics, + function(diagnostic) { + return(diagnostic[c("type", "target", "message")]) + } + ), + list(list( + type = "qc_load", + target = "S2", + message = "No QC files were found." + )) + ) + + default_sheet <- read_samplesheet(test_samplesheet(type = "default")) + default_paths <- get_file_paths( + data_folder = test_data_folder(type = "default"), + sample_sheet = default_sheet + ) + default_result <- pixelatorES:::.read_qc_groups_soft( + files = default_paths$qc_files, + aliases = sort(default_sheet$sample_alias), + alias_column = "sample_alias", + target_label = "sample" + ) + expect_equal( + Filter( + function(diagnostic) { + return(grepl("^Missing QC stages", diagnostic$message)) + }, + default_result$diagnostics + ), + list() + ) +}) diff --git a/tests/testthat/test_es_data.R b/tests/testthat/test_es_data.R index 5722c15..946da79 100644 --- a/tests/testthat/test_es_data.R +++ b/tests/testthat/test_es_data.R @@ -183,6 +183,30 @@ test_that("Sample aliases work as expected", { ) }) +test_that("test_es_data fixtures work as expected", { + expect_s3_class(test_es_data(), "es_data") + + prox <- data.frame(marker = "CD3") + expect_equal(test_es_data(proximity = prox)$proximity, prox) + + qc <- list(read_stats = data.frame(n = 1)) + expect_equal(test_es_data(qc = qc)$qc, qc) + + samplesheet <- tibble( + sample = c("sample_1", "sample_2"), + sample_alias = c("S1", "S2"), + condition = c("A", "B") + ) + object <- test_es_data(samplesheet = samplesheet) + expect_equal( + object$sample_aliases, + c(sample_1 = "S1", sample_2 = "S2") + ) + expect_equal(object$effective_samplesheet, samplesheet) + + expect_error(test_es_data(not_a_slot = 1)) +}) + test_that("Samplesheet extractors work as expected", { samplesheet <- tibble( sample = c("sample_1", "sample_2"), @@ -284,9 +308,17 @@ test_that("es_data diagnostics work as expected", { }) test_that("es_data workflow registration works as expected", { + report <- list( + preamble = c("shared/preprocessing.qmd"), + sections = list( + list(id = "samples", title = "Samples", child = "shared/samples.qmd") + ) + ) + register_es_data_workflow( "test_workflow", function() return(list(pxl_data = identity)), + report = function() return(report), overwrite = TRUE ) @@ -300,16 +332,19 @@ test_that("es_data workflow registration works as expected", { )$extractors, list(pxl_data = identity) ) + expect_equal(get_es_workflow_report("test_workflow"), report) expect_error( register_es_data_workflow( "test_workflow", - function() return(list()) + function() return(list()), + report = function() return(report) ) ) register_es_data_workflow( "test_workflow", function() return(list(proximity = identity)), + report = function() return(report), overwrite = TRUE ) expect_equal( @@ -320,6 +355,200 @@ test_that("es_data workflow registration works as expected", { ) }) +test_that("Workflow report recipes work as expected", { + report <- list( + preamble = c("shared/preprocessing.qmd"), + sections = list( + list( + id = "samples", + title = "Samples", + child = "shared/samples.qmd" + ), + list( + id = "quality_metrics", + title = "Quality metrics", + child = "workflows/amplicon_demux/quality_metrics.qmd" + ) + ) + ) + + register_es_data_workflow( + "test_report_workflow", + function() return(list(pxl_data = identity)), + report = function() return(report), + overwrite = TRUE + ) + expect_equal( + get_es_workflow_report("test_report_workflow"), + report + ) + expect_equal( + get_es_workflow_report("amplicon_demux"), + list( + preamble = c("shared/preprocessing.qmd"), + sections = list( + list(id = "samples", title = "Samples", child = "shared/samples.qmd"), + list( + id = "quality_metrics", + title = "Quality metrics", + child = "workflows/amplicon_demux/quality_metrics.qmd" + ), + list( + id = "cell_annotation", + title = "Cell annotation", + child = "workflows/amplicon_demux/cell_annotation.qmd" + ), + list( + id = "abundance", + title = "Abundance", + child = "workflows/amplicon_demux/abundance.qmd" + ), + list( + id = "spatial", + title = "Spatial metrics", + child = "workflows/amplicon_demux/spatial.qmd" + ), + list( + id = "run_info", + title = "Run info", + child = "shared/run_info.qmd" + ) + ) + ) + ) + + amplicon_report <- get_es_workflow_report("amplicon_demux") + amplicon_paths <- c( + amplicon_report$preamble, + vapply(amplicon_report$sections, function(section) { + return(section$child) + }, character(1)) + ) + quarto_root <- system.file("quarto", package = "pixelatorES") + expect_true(nzchar(quarto_root)) + expect_equal( + file.exists(file.path(quarto_root, amplicon_paths)), + rep(TRUE, length(amplicon_paths)) + ) + + expect_error( + register_es_data_workflow( + "missing_path_workflow", + function() return(list()), + report = function() { + return(list( + preamble = "shared/preprocessing.qmd", + sections = list( + list( + id = "samples", + title = "Samples", + child = "shared/does_not_exist.qmd" + ) + ) + )) + }, + overwrite = TRUE + ) + ) + + expect_error( + register_es_data_workflow( + "bad_report_workflow", + function() return(list()), + report = function() { + return(list(sections = list( + list(id = "samples", title = "Samples", child = "samples.qmd") + ))) + }, + overwrite = TRUE + ) + ) + expect_error( + register_es_data_workflow( + "bad_report_workflow", + function() return(list()), + report = function() { + return(list( + preamble = character(), + sections = list( + list(id = "samples", title = "Samples", child = "samples.qmd") + ) + )) + }, + overwrite = TRUE + ) + ) + expect_error( + register_es_data_workflow( + "bad_report_workflow", + function() return(list()), + report = function() { + return(list( + preamble = "preprocessing.qmd", + sections = list() + )) + }, + overwrite = TRUE + ) + ) + expect_error( + register_es_data_workflow( + "bad_report_workflow", + function() return(list()), + report = function() { + return(list( + preamble = "preprocessing.qmd", + sections = list( + list(id = "samples", title = "Samples") + ) + )) + }, + overwrite = TRUE + ) + ) + expect_error( + register_es_data_workflow( + "bad_report_workflow", + function() return(list()), + report = function() { + return(list( + preamble = "shared/preprocessing.qmd", + sections = list( + list( + id = "samples", + title = "Samples", + child = "shared/samples.qmd" + ), + list( + id = "samples", + title = "Again", + child = "shared/samples.qmd" + ) + ) + )) + }, + overwrite = TRUE + ) + ) + expect_error( + register_es_data_workflow( + "bad_extractors_workflow", + function() return("not a list"), + report = function() return(report), + overwrite = TRUE + ) + ) + expect_error( + register_es_data_workflow( + "bad_report_factory_workflow", + function() return(list()), + report = report, + overwrite = TRUE + ) + ) + expect_error(get_es_workflow_report("unknown_workflow")) +}) + test_that("Partial input failures work as expected", { sample_sheet <- read_samplesheet(test_samplesheet()) data_folder <- .copy_es_data_test_folder("default") diff --git a/tests/testthat/test_key_table.R b/tests/testthat/test_key_table.R index 0fa9799..7c81c73 100644 --- a/tests/testthat/test_key_table.R +++ b/tests/testthat/test_key_table.R @@ -1,12 +1,5 @@ library(Seurat) -.as_es_data <- function(qc = list()) { - return(structure( - list(qc = qc), - class = c("es_data", "list") - )) -} - data_types <- c("default", "hashing") for (data_type in data_types) { @@ -672,13 +665,13 @@ for (data_type in data_types) { } expect_no_error( - tabl <- key_metric_table(.as_es_data(sample_qc_tables)) + tabl <- key_metric_table(test_es_data(qc = sample_qc_tables)) ) expect_s3_class(tabl$sample, "datatables") expect_no_error( tabl <- key_metric_table( - .as_es_data(sample_qc_tables), + test_es_data(qc = sample_qc_tables), return_data = TRUE ) ) @@ -800,7 +793,7 @@ test_that("Key metric tables handle missing metrics as expected", { ) expect_equal( - key_metric_table(.as_es_data(partial_metrics), return_data = TRUE), + key_metric_table(test_es_data(qc = partial_metrics), return_data = TRUE), list( sample = structure( list( @@ -826,7 +819,7 @@ test_that("Key metric tables handle missing metrics as expected", { ) ) expect_equal( - key_metric_table(.as_es_data(pool_metrics), return_data = TRUE), + key_metric_table(test_es_data(qc = pool_metrics), return_data = TRUE), list( sample = NULL, pool = structure( @@ -842,11 +835,11 @@ test_that("Key metric tables handle missing metrics as expected", { ) expect_equal( - key_metric_table(.as_es_data(), return_data = TRUE), + key_metric_table(test_es_data(), return_data = TRUE), list(sample = NULL, pool = NULL) ) expect_equal( - key_metric_table(.as_es_data()), + key_metric_table(test_es_data()), list(pool = NULL, sample = NULL) ) }) diff --git a/tests/testthat/test_params.R b/tests/testthat/test_params.R index 6ea8d77..31fb742 100644 --- a/tests/testthat/test_params.R +++ b/tests/testthat/test_params.R @@ -31,12 +31,9 @@ test_that("Parameter tables work as expected", { ), row.names = c(NA, -2L), class = c( "tbl_df", "tbl", "data.frame" )) - es_data <- structure( - list( - params = params, - samplesheet = sample_sheet - ), - class = c("es_data", "list") + es_data <- test_es_data( + params = params, + samplesheet = sample_sheet ) expect_equal( @@ -61,4 +58,24 @@ test_that("Parameter tables work as expected", { expect_no_error(print_metadata_table(es_data)) expect_no_error(print_session_info()) + + hashed_sheet <- read_samplesheet(test_samplesheet(type = "hashing")) + hashed_es_data <- test_es_data( + samplesheet = hashed_sheet, + pxl_data_processed = NULL, + diagnostics = list(list( + type = "pxl_load", + target = "S1", + message = "No PXL file was found." + )) + ) + hashed_table <- print_metadata_table(hashed_es_data) + expect_equal( + names(hashed_table$x$data), + c("Issues", "Pool", "Sample Alias", "Sample name", "Condition") + ) + expect_equal( + hashed_table$x$data$Issues, + c("\u26A0", "", "", "") + ) })