Run the full curation pipeline headlessly (without Shiny UI)

Runs the complete CONCERT curation pipeline - file read, frontmatter detection, cleaning, CompTox API search, consensus classification, and multi-sheet XLSX export - from a single R script call with no Shiny session required. When harmonize=TRUE, additionally runs the numeric parsing, unit harmonization, and ToxVal schema mapping pipeline, and writes parquet/CSV output alongside the XLSX.

Usage

curate_headless(
  input_path,
  output_path,
  tag_map,
  skip_flags = NULL,
  header_row = NULL,
  reference_lists = NULL,
  reference_list_snapshot = NULL,
  activate_all_references = FALSE,
  verbose = TRUE,
  harmonize = FALSE,
  format = "parquet",
  unit_map = NULL,
  unit_map_snapshot = NULL,
  corrections = NULL,
  media = NULL,
  wqx_threshold = 0.85,
  starts_with = FALSE,
  postprocess_candidates = FALSE,
  review_overrides = NULL,
  accept_suggestions = FALSE,
  review_picks = NULL,
  row_flags = NULL,
  review_decision_evidence = NULL,
  candidate_validation = NULL,
  identity_decisions = NULL,
  site_manifest = NULL,
  site_alias_map = NULL,
  multi_analyte_resolutions = NULL,
  value_corrections = NULL,
  cleaning_steps = NULL,
  media_map = NULL,
  media_map_snapshot = NULL,
  write_files = TRUE,
  source_name = NULL,
  pubchem = FALSE,
  desalt = FALSE,
  desalt_workflows = c("qsar-ready", "ms-ready"),
  ignored_identifier_cols = character(),
  toxval_identity_mode = "lookup"
)

Arguments

input_path: Character. Path to the input file (CSV, XLSX, or XLS).
output_path: Character. Path for the output XLSX file. Parent directories are created automatically if they do not exist. Required when write_files = TRUE; ignored when write_files = FALSE.
tag_map: Named list mapping cleaned column names to tag roles. Keys must match column names after janitor::clean_names() normalization. Values are one of "Name", "CASRN", "Other", "Result", "Numeric", "Unit", "Qualifier", "ReportingLimit", "Uncertainty", "UncertaintyCoverage", etc. Example: list(chemical_name = "Name", cas_number = "CASRN", result = "Result", unit = "Unit").
skip_flags: Character vector of cleaning flag codes whose rows should skip the CompTox API search. Reserved for future use; isotope_match rows are already handled internally by run_curation_pipeline().
header_row: Integer or NULL. If NULL (default), frontmatter detection runs automatically. If an integer, that row number is used as the header row (manual override).
reference_lists: Named list of reference data. If NULL (default), the package-bundled reference cache is loaded automatically. If provided, must contain keys: stop_words, functional_categories, block_patterns, strip_terms, isotope_lookup.
reference_list_snapshot: Optional compact replay snapshot generated by generate_concert_script(). When supplied, package defaults are loaded without local user_reference_lists.rds overrides, verified against the snapshot hashes, and merged with snapshot overrides.
activate_all_references: Logical. If TRUE, activates all cleaning reference-list rows for this run only.
verbose: Logical. If TRUE (default), progress messages are printed to the console. If FALSE, all messages are suppressed.
harmonize: Logical. If TRUE, runs numeric parsing, unit harmonization, and ToxVal schema mapping after curation. Default FALSE for backward compat.
format: Character. Output format for ToxVal data when harmonize=TRUE. One of "parquet", "csv", or "both". Default "parquet". Ignored when harmonize=FALSE.
unit_map: Tibble with unit conversion mappings, or NULL (default) to load from package cache via load_unit_map().
unit_map_snapshot: Optional compact replay snapshot from generate_concert_script(); reconstructs the effective unit map from package defaults plus embedded deltas. Mutually exclusive with unit_map.
corrections: Tibble with pattern/replacement columns for one-off corrections, or NULL (default) to load from package cache.
media: Character. Media context for ppb/ppm routing: "aqueous", "air", or "solid". NULL (default) uses aqueous assumption.
wqx_threshold: Numeric WQX fuzzy matching threshold.
starts_with: Logical. If TRUE, enables CompTox starts-with fallback search for names unresolved by exact, CAS, and WQX matching.
postprocess_candidates: Logical. If TRUE, runs the same enrichment, similarity scoring, and auto-resolve pass used by the Shiny app after curation. Default FALSE for backward compatibility.
review_overrides: Optional function generated by generate_concert_script(), content-match spec from build_review_overrides(), or legacy positional override table/list to replay Review Results edits.
accept_suggestions: Logical. If TRUE, accepts every row scored as "suggested" after candidate postprocessing (same as the Review Results bulk accept).
review_picks: Optional data frame with name, dtxsid, and optional casrn columns. Rows whose cleaned Name (and CAS, when given) match are resolved to that DTXSID after validation against CompTox.
row_flags: Optional data frame with name, flag, and optional casrn and reason columns. Matching rows get the row flag (one of valid_row_flags()).
review_decision_evidence: Immutable portable decision snapshots and scoped acknowledgments from capture_review_decision(). Legacy flags never acquire a historical baseline implicitly.
candidate_validation: Structured saved validation outcomes from validate_review_candidates(). Reporting makes no network requests.
identity_decisions: Explicit source-scoped decisions passed to apply_identity_decisions(). Name-wide selectors cannot grant scope acceptance.
site_manifest: Optional curated Dataset Context site manifest to include in the workbook export.
site_alias_map: Optional Dataset Context raw-label alias map to include in the workbook export.
multi_analyte_resolutions: Optional data frame/list with row_index (or row), action, and optional value/values columns. row_index is the original_row_id as written to pending.csv, so it stays valid after synonym or multi-analyte splits. Applied after cleaning and before curation.
value_corrections: Optional data frame with column, pattern, replacement, and optional match_mode columns. Applied to the detected data before cleaning, so before Unicode folding: match Unicode hyphens with \\p{Pd}, not -. See apply_value_corrections().
cleaning_steps: Optional named list of logicals switching cleaning steps on or off: unicode, whitespace, cas, names, synonyms, isotopes, multi, chiral, truncated, bare_formula, reference_flags. Omitted names keep their defaults.
media_map: Optional media harmonization map passed to harmonize_media().
media_map_snapshot: Optional compact replay snapshot from generate_concert_script(); reconstructs the effective media map from package defaults plus embedded deltas. Mutually exclusive with media_map.
write_files: Logical. If TRUE (default), writes XLSX and optional parquet/CSV outputs. If FALSE, runs fully in memory and returns the same list without requiring or creating output files.
source_name: Optional dataset identifier for ToxVal source. Defaults to the input filename stem.
pubchem: Logical. If TRUE, searches unresolved names in PubChem and exports candidates without assigning DTXSIDs.
desalt: Logical. If TRUE, suggests salt parent candidates without assigning parent DTXSIDs, and adds structure-based parent_dtxsid_, parent_name_, parent_casrn_, and parent_status_<workflow> columns for resolved rows via the chemi standardizer. Independent of pubchem.
desalt_workflows: Standardizer workflows used when desalt = TRUE: "qsar-ready", "ms-ready", or both (default).
ignored_identifier_cols: Retained identifier columns deliberately treated as metadata. These columns never supply identity evidence.
toxval_identity_mode: ToxVal identifier policy: "lookup" preserves the existing audit export default; "accepted" gates IDs and preserves every measurement row with NA IDs when blocked.

Value

Invisibly returns a list:

When harmonize=FALSE:

$data – resolution_state tibble
$audit_trail – cleaning audit tibble

When harmonize=TRUE:

$data – 56-column ToxVal tibble (per D-05)
$audit_trail – cleaning audit tibble
$harmonize_audit – harmonization audit tibble (per D-06)
$harmonize_results – parsed and unit-harmonized measurement tables
$media_results, $duration_results, $date_results, and $detection_results – harmonization sidecar artifacts
$row_data – curated row data updated with generated detection, media, duration, and date fields