Code from data
A program that types every parameter code by hand breaks when the data changes. A new parameter gets no table, and a removed one gives an empty table. A data-driven program reads the codes from the data and makes one result for each. This is the heart of every table program that covers many parameters, visits or arms.
In R you do not write code that writes code. You make a list from the data, and a function works on each element. Chapter 8 introduced map(). This chapter uses it for real study output.
18.1 Lists from the data
Section titled “18.1 Lists from the data”# Chapter 18: lists from the data. The codes, their labels, and a countlibrary(dplyr)
advs <- pharmaverseadam::advs
# One row per parameter, in the order of the codeparams <- advs |> distinct(PARAMCD, PARAM) |> arrange(PARAMCD) |> haven::zap_label()params#> # A tibble: 9 x 2#> PARAMCD PARAM#> <chr> <chr>#> 1 BMI Body Mass Index(kg/m^2)#> 2 BSA Body Surface Area(m^2)#> 3 DIABP Diastolic Blood Pressure (mmHg)#> 4 HEIGHT Height (cm)#> 5 MAP Mean Arterial Pressure (mmHg)#> 6 PULSE Pulse Rate (beats/min)#> 7 SYSBP Systolic Blood Pressure (mmHg)#> 8 TEMP Temperature (C)#> 9 WEIGHT Weight (kg)
# A vector of the codes, and how many there arecodes <- params$PARAMCDcodes#> [1] "BMI" "BSA" "DIABP" "HEIGHT" "MAP" "PULSE" "SYSBP" "TEMP"#> [9] "WEIGHT"length(codes)#> [1] 9
# A named vector is a lookup: the label of a codelabels <- setNames(params$PARAM, params$PARAMCD)labels[["DIABP"]]#> [1] "Diastolic Blood Pressure (mmHg)"
# One value from the data: the size of the safety populationn_saf <- sum(pharmaverseadam::adsl$SAFFL == "Y", na.rm = TRUE)n_saf#> [1] 254distinct(PARAMCD, PARAM)keeps one row per parameter: the 9 parameters of the reference ADVS, with their labels.arrange()puts them in the order of the code.params$PARAMCDis a vector of the codes, andlength()counts them: 9. If a new parameter comes with the next data delivery, the vector has 10, and nothing in the program changes.setNames()gives each label the name of its code. A named vector works as a lookup:labels[["DIABP"]]is the label ofDIABP.- One value can come from the data too:
n_safis the size of the safety population, 254.
18.2 One summary per parameter
Section titled “18.2 One summary per parameter”# Chapter 18: one summary per parameter. split() makes a list, map() works on eachlibrary(dplyr)library(purrr)
# Baseline rows; blood pressure and pulse in one position (lying down)bl <- pharmaverseadam::advs |> filter(ABLFL == "Y", is.na(ATPTN) | ATPTN == 815)
by_param <- split(bl, bl$PARAMCD)length(by_param)#> [1] 9names(by_param)#> [1] "BMI" "BSA" "DIABP" "HEIGHT" "MAP" "PULSE" "SYSBP" "TEMP"#> [9] "WEIGHT"
summ <- function(data) { data |> summarise( n = n(), mean = round(mean(AVAL), 1), sd = round(sd(AVAL), 2), .by = TRT01P ) |> arrange(TRT01P)}tables <- map(by_param, summ)tables$DIABP#> # A tibble: 3 x 4#> TRT01P n mean sd#> <chr> <int> <dbl> <dbl>#> 1 Placebo 86 75.7 11.0#> 2 Xanomeline High Dose 84 77.2 9.8#> 3 Xanomeline Low Dose 84 76.3 9.77
# When one table is enough, group_by() does the same without a listbl |> filter(PARAMCD %in% c("DIABP", "SYSBP")) |> summarise(n = n(), mean = round(mean(AVAL), 1), .by = c(PARAMCD, TRT01P)) |> arrange(PARAMCD, TRT01P)#> # A tibble: 6 x 4#> PARAMCD TRT01P n mean#> <chr> <chr> <int> <dbl>#> 1 DIABP Placebo 86 75.7#> 2 DIABP Xanomeline High Dose 84 77.2#> 3 DIABP Xanomeline Low Dose 84 76.3#> 4 SYSBP Placebo 86 139.#> 5 SYSBP Xanomeline High Dose 84 140.#> 6 SYSBP Xanomeline Low Dose 84 139.- The filter keeps the baseline rows. Blood pressure, pulse and MAP are measured in more than one position in the pilot, so the program keeps one: lying down (
ATPTN == 815). Without it, each subject has more than one baseline row of these parameters. split(bl, bl$PARAMCD)cuts the data frame into a list of 9 data frames, one per parameter. Each element has the name of its code.map(by_param, summ)callssumm()on each data frame. The result is again a list with the same names, sotables$DIABPis the table of diastolic blood pressure.
Each table has the same rows. When you need one table only, summarise() with .by = c(PARAMCD, TRT01P) gives all parameters in one data frame, without a list. Use split() and map() when each parameter needs its own output: its own title, its own file, its own number of decimals or its own model.
| You want | Use |
|---|---|
| one table with a row per parameter | summarise(.by = c(PARAMCD, ...)) |
| one result per parameter, kept apart | split(), then map() |
| the same, without names | group_split(), then map() |
| one file or one printout per parameter | walk() or walk2() |
18.3 List-driven output: one file per parameter
Section titled “18.3 List-driven output: one file per parameter”A study often wants one output per parameter. The program writes them in a loop, and the list of parameters comes from the data.
# Chapter 18: list-driven output. One file per parameter, the title from the labellibrary(dplyr)library(purrr)
bl <- pharmaverseadam::advs |> filter(ABLFL == "Y", is.na(ATPTN) | ATPTN == 815)labels <- bl |> distinct(PARAMCD, PARAM)
out_dir <- "output/18_baseline"dir.create(out_dir, showWarnings = FALSE, recursive = TRUE)
write_one <- function(code, label) { tab <- bl |> filter(PARAMCD == code) |> summarise(n = n(), mean = round(mean(AVAL), 1), .by = TRT01P) |> arrange(TRT01P) file <- file.path(out_dir, paste0("t_bl_", tolower(code), ".csv")) write.csv(tab, file, row.names = FALSE) cat(sprintf("%-7s %-40s %s\n", code, label, basename(file)))}
# walk2() gives the two columns, row by row, to write_one()walk2(labels$PARAMCD, labels$PARAM, write_one)#> BMI Body Mass Index(kg/m^2) t_bl_bmi.csv#> BSA Body Surface Area(m^2) t_bl_bsa.csv#> DIABP Diastolic Blood Pressure (mmHg) t_bl_diabp.csv#> HEIGHT Height (cm) t_bl_height.csv#> MAP Mean Arterial Pressure (mmHg) t_bl_map.csv#> PULSE Pulse Rate (beats/min) t_bl_pulse.csv#> SYSBP Systolic Blood Pressure (mmHg) t_bl_sysbp.csv#> TEMP Temperature (C) t_bl_temp.csv#> WEIGHT Weight (kg) t_bl_weight.csv
list.files(out_dir)#> [1] "t_bl_bmi.csv" "t_bl_bsa.csv" "t_bl_diabp.csv" "t_bl_height.csv"#> [5] "t_bl_map.csv" "t_bl_pulse.csv" "t_bl_sysbp.csv" "t_bl_temp.csv"#> [9] "t_bl_weight.csv"read.csv(file.path(out_dir, "t_bl_weight.csv"))#> TRT01P n mean#> 1 Placebo 86 62.8#> 2 Xanomeline High Dose 84 70.0#> 3 Xanomeline Low Dose 84 67.1write_one()makes the table of one parameter, writes it to a CSV file, and prints one line: the code, the label and the file name.walk2(labels$PARAMCD, labels$PARAM, write_one)callswrite_one()once per row, with the code as the first argument and the label as the second.walk2()iswalk()for two lists side by side. It returns nothing that you need: the files are the result.file.path()joins folder and file names with the right separator.tolower()makes the file name lower case:t_bl_weight.csv.list.files()shows the 9 files. The last line reads one back: the mean baseline weight is 62.8 kg on placebo.
18.4 A table of settings
Section titled “18.4 A table of settings”Real summary programs need more than a list of names: each variable has a label and a number of decimals. Put these settings in a small table, one row per variable. The table reads like the table shell, and the code does not change when a row is added.
# Chapter 18: a table of settings drives the summary. One row per variablelibrary(dplyr)library(purrr)
adsl <- pharmaverseadam::adsl |> filter(SAFFL == "Y")
# The settings read like the table shell: variable, label, decimals of the datasettings <- tribble( ~var, ~label, ~dec, "AGE", "Age (years)", 0, "TRTDURD", "Duration of treatment (days)", 0)
# Mean with one decimal more than the data, SD with two moresumm_one <- function(var, label, dec) { adsl |> summarise( n = sum(!is.na(.data[[var]])), mean = mean(.data[[var]], na.rm = TRUE), sd = sd(.data[[var]], na.rm = TRUE), .by = TRT01P ) |> transmute( label, TRT01P, n, stat = paste0( formatC(mean, format = "f", digits = dec + 1), " (", formatC(sd, format = "f", digits = dec + 2), ")" ) ) |> arrange(TRT01P)}
# pmap() calls summ_one() once per row: the columns are the argumentspmap(settings, summ_one) |> list_rbind()#> # A tibble: 6 x 4#> label TRT01P n stat#> <chr> <chr> <int> <chr>#> 1 Age (years) Placebo 86 75.2 (8.59)#> 2 Age (years) Xanomeline High Dose 84 74.4 (7.89)#> 3 Age (years) Xanomeline Low Dose 84 75.7 (8.29)#> 4 Duration of treatment (days) Placebo 85 149.5 (60.35)#> 5 Duration of treatment (days) Xanomeline High Dose 83 98.2 (70.83)#> 6 Duration of treatment (days) Xanomeline Low Dose 84 97.3 (68.26)tribble()writes the settings row by row (chapter 16).summ_one(var, label, dec)makes the rows of one variable..data[[var]]takes the column from the text.formatC(x, format = "f", digits = 1)writes a number with exactly that many decimals, also when the last one is 0. The mean gets one decimal more than the data, the SD two more: a common rule in clinical tables.pmap(settings, summ_one)callssumm_one()once per row of the settings. The column names are the argument names, sovar,labelanddecgo to the right arguments.list_rbind()stacks the results.
The age row gives 75.2 (8.59) on placebo. TRTDURD has 85 values on placebo, not 86: the reference has no treatment duration for one subject.
18.5 A parameter file
Section titled “18.5 A parameter file”When the settings change from run to run, or someone who does not write R must change them, put them in a file. A parameter file is a small CSV file that the program reads at the start. This one says which parameters to show, with how many decimals, and with which title:
paramcd,include,dec,titleWEIGHT,Y,1,Baseline weight (kg)HEIGHT,Y,1,Baseline height (cm)BMI,Y,1,Baseline body mass index (kg/m2)SYSBP,Y,0,Baseline systolic blood pressure (mmHg)DIABP,N,0,Baseline diastolic blood pressure (mmHg)# Chapter 18: a parameter file. A CSV decides what the run makes, not the codelibrary(dplyr)library(purrr)
settings <- readr::read_csv("data/18_run_settings.csv", show_col_types = FALSE)settings#> # A tibble: 5 x 4#> paramcd include dec title#> <chr> <chr> <dbl> <chr>#> 1 WEIGHT Y 1 Baseline weight (kg)#> 2 HEIGHT Y 1 Baseline height (cm)#> 3 BMI Y 1 Baseline body mass index (kg/m2)#> 4 SYSBP Y 0 Baseline systolic blood pressure (mmHg)#> 5 DIABP N 0 Baseline diastolic blood pressure (mmHg)
bl <- pharmaverseadam::advs |> filter(ABLFL == "Y", is.na(ATPTN) | ATPTN == 815)
one_table <- function(paramcd, dec, title, ...) { cat("\n", title, "\n", sep = "") bl |> filter(PARAMCD == .env$paramcd) |> summarise( n = n(), mean = formatC(mean(AVAL), format = "f", digits = dec + 1), .by = TRT01P ) |> arrange(TRT01P) |> print() invisible(NULL)}
# Only the rows with include = "Y". The ... of one_table() takes the restsettings |> filter(include == "Y") |> pwalk(one_table)#>#> Baseline weight (kg)#> # A tibble: 3 x 3#> TRT01P n mean#> <chr> <int> <chr>#> 1 Placebo 86 62.76#> 2 Xanomeline High Dose 84 70.00#> 3 Xanomeline Low Dose 84 67.12#>#> Baseline height (cm)#> # A tibble: 3 x 3#> TRT01P n mean#> <chr> <int> <chr>#> 1 Placebo 86 162.57#> 2 Xanomeline High Dose 84 165.82#> 3 Xanomeline Low Dose 84 163.43#>#> Baseline body mass index (kg/m2)#> # A tibble: 3 x 3#> TRT01P n mean#> <chr> <int> <chr>#> 1 Placebo 86 23.63#> 2 Xanomeline High Dose 84 25.34#> 3 Xanomeline Low Dose 84 25.04#>#> Baseline systolic blood pressure (mmHg)#> # A tibble: 3 x 3#> TRT01P n mean#> <chr> <int> <chr>#> 1 Placebo 86 138.7#> 2 Xanomeline High Dose 84 140.1#> 3 Xanomeline Low Dose 84 138.8readr::read_csv()reads the file into a tibble.show_col_types = FALSEstops the message about the column types.filter(include == "Y")keeps four rows.DIABPhasN, so the run makes no table for it.pwalk(one_table)callsone_table()once per row, likepmap(), for the printout only....inone_table()takes the arguments that the function does not use, hereinclude. Without...,pwalk()stops with an error: unused argument..env$paramcdis the argument, not a column (chapter 17). ADVS has a columnPARAMCDin upper case, so the name does not clash here, but.env$makes it certain.
Change the file, not the program: set DIABP to Y, run again, and the fifth table appears. The program and its validation stay the same.
18.6 A specification from the data: the labels
Section titled “18.6 A specification from the data: the labels”Every ADaM data set goes to the regulator with define.xml: a file that describes each data set and each variable, with its name, label, type and derivation. Programmers keep a specification, often a spreadsheet with one row per variable, and generate define.xml and the labels of their data from it. One source, so the data and its description cannot disagree.
R keeps the type and the label of each column with the data. So a specification can be made from a data set, and a data set can be checked against it. Here the reference ADSL is the specification for the ADSL of chapter 9:
# Chapter 18: a specification from the data. Check the labels, then set themlibrary(dplyr)library(purrr)
# The specification: one row per variable, from the reference ADSLref <- pharmaverseadam::adslspec <- tibble( name = names(ref), type = map_chr(ref, \(x) class(x)[1]), label = map_chr(ref, \(x) attr(x, "label") %||% NA_character_)) |> mutate(across(everything(), unname))head(spec, 4)#> # A tibble: 4 x 3#> name type label#> <chr> <chr> <chr>#> 1 STUDYID character Study Identifier#> 2 USUBJID character Unique Subject Identifier#> 3 SUBJID character Subject Identifier for the Study#> 4 SITEID character Study Site Identifier
# Our ADSL of chapter 9, against the specificationadsl <- readRDS("data/adsl.rds")ours <- tibble( name = names(adsl), type = map_chr(adsl, \(x) class(x)[1]), label = map_chr(adsl, \(x) attr(x, "label") %||% NA_character_))both <- inner_join(ours, spec, by = join_by(name), suffix = c("", "_spec"))c( in_both = nrow(both), other_type = sum(both$type != both$type_spec), other_label = sum(!both$label %in% both$label_spec))#> in_both other_type other_label#> 20 0 5both |> filter(!label %in% label_spec) |> select(name, label, label_spec)#> # A tibble: 5 x 3#> name label label_spec#> <chr> <chr> <chr>#> 1 TRTSDT <NA> Date of First Exposure to Treatment#> 2 TRTEDT <NA> Date of Last Exposure to Treatment#> 3 SAFFL <NA> Safety Population Flag#> 4 EOSSTT <NA> End of Study Status#> 5 AGEGR1 <NA> Pooled Age Group 1
# The specification writes the labels: one loop over the shared variablesfor (i in seq_len(nrow(both))) { attr(adsl[[both$name[i]]], "label") <- both$label_spec[i]}sum(map_chr(adsl[both$name], \(x) attr(x, "label")) != both$label_spec)#> [1] 0attr(adsl$TRTEDT, "label")#> [1] "Date of Last Exposure to Treatment"map_chr(ref, \(x) class(x)[1])calls the function on each column of the data frame and gives one text per column: the type. A data frame is a list of columns, somap()works on it.attr(x, "label") %||% NA_character_gives the label, orNAwhen the column has none.- The check: 20 variables are in both, 0 have another type, 5 have no label in your ADSL. They are the variables that chapter 9 made itself:
TRTSDT,TRTEDT,SAFFL,EOSSTTandAGEGR1. - The
forloop sets the label of each shared variable from the specification. After it, 0 labels differ.TRTEDTis now Date of Last Exposure to Treatment.
In a study, the specification comes from the spreadsheet of the lead programmer, read with readxl::read_excel(). The code stays the same.
18.7 When code from data is too clever
Section titled “18.7 When code from data is too clever”In SAS, data-driven code is code that writes code. In R you can do that too: build the code as a text, then run it. It is almost never the right choice.
# Chapter 18: code built as text. It works, but it is the last choicelibrary(dplyr)
adsl <- pharmaverseadam::adsl |> filter(SAFFL == "Y")var <- "AGE"
# 1. Code as a text, then parse() and eval(). Hard to read, hard to checkcode <- paste0("adsl |> summarise(mean = mean(", var, "))")code#> [1] "adsl |> summarise(mean = mean(AGE))"eval(parse(text = code))#> # A tibble: 1 x 1#> mean#> <dbl>#> 1 75.1
# 2. The same without text code: .data[[ ]] takes the nameadsl |> summarise(mean = mean(.data[[var]]))#> # A tibble: 1 x 1#> mean#> <dbl>#> 1 75.1
# 3. A condition in a settings file is code by nature. Parse it once, then !!where <- "SEX == 'F' & AGE > 80"cond <- rlang::parse_expr(where)adsl |> filter(!!cond) |> nrow()#> [1] 46
# The risk: a typo in the text shows only when the code runswhere <- "SEX = 'F' & AGE > 80"adsl |> filter(!!rlang::parse_expr(where)) |> nrow()#> Error in `filter()`:#> i In argument: `SEX = "F" & AGE > 80`.#> Caused by error in `"F" & AGE > 80`:#> ! operations are possible only for numeric, logical or complex typespaste0()builds the text of a program, andeval(parse(text = code))runs it. It works: the mean age is 75.1. But nobody can read the code before it runs, a linter cannot check it, and a debugger cannot step into it..data[[var]]gives the same result, and the code is plain to read. When the varying part is a name, this is all you need.- A condition that comes from a settings file is code by nature, such as a population or a subgroup.
rlang::parse_expr()turns the text into code once, and!!puts it intofilter(): 46 women older than 80.
The last call shows the risk. The condition has one = instead of ==. The text looks fine, and the program fails only when the code runs, with an error that does not mention the typo.
Use code from data when it makes the program shorter and clearer. Stop when:
- a reviewer must run the program to know what it does,
- the settings table has more logic than the program (conditions inside conditions),
- a plain
filter()with three hand-written values would do. Three parameters that never change need no list.
18.8 Summary
Section titled “18.8 Summary”- Take lists from the data:
distinct(),unique(),split(). A named vector is a lookup. split()makes a list of data frames, one per parameter;map()gives a list of results with the same names.walk(),walk2()andpwalk()call a function for its side effect: a file or a printout per element.- A table of settings (
tribble()) or a parameter file (CSV) drives the run.pmap()andpwalk()call a function once per row. - A specification can be made from a data set:
map_chr()over the columns gives the types and labels. - Use
.data[[ ]]before code as text. Userlang::parse_expr()only for a condition that is code by nature.
18.9 Exercises
Section titled “18.9 Exercises”Exercise 18-1. Leave HEIGHT and TEMP out of the tables of program 18_2, with a vector exclude. Build the list without them, not with an if in the function. How many tables are left?
Exercise 18-2. Use iwalk() on the list by_param of program 18_2 to print one line per parameter: the code, the number of rows and the mean. iwalk() gives the function each element and its name.
Exercise 18-3. Switch DIABP on in a copy of the parameter file, and run the table for it. What is the mean baseline diastolic blood pressure on placebo?
Exercise 18-4. Which variables of the specification (the reference ADSL) are not in your ADSL, and which of your variables are not in the specification? How many are there of each?
The answers are in Answers, Parts IV and V.
Found an error or have a question? Write to datastep.studio@gmail.com· Every program runs in DataStep Studio for R, the R extension for VS Code and Positron.
Data: the CDISC pilot study (cdiscpilot01), from the R packagespharmaversesdtm andpharmaverseadam (Apache License 2.0).
R is a project of the R Foundation. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. This book is not affiliated with or endorsed by SAS Institute Inc. or the R Foundation. CDISC, SDTM and ADaM are trademarks of the Clinical Data Interchange Standards Consortium.
© 2026 DataStep Labs. All rights reserved.