Validation and QC
In a regulated study, a number is not finished when one program computes it. It is finished when a second, independent program computes the same number, and a compare tool finds no difference. In this chapter you write the second program, compare the two, and compare your ADaM data sets with the reference. Then you check the data rules and every warning, and you write down what you found for a reviewer.
Part II built the ADaM data sets in programs/data/. Part IV proves them. Run the programs of chapters 9 to 12 first, in file order.
15.1 Double programming: a second program
Section titled “15.1 Double programming: a second program”Double programming means two programmers build the same value from the same specification, each alone. The first program is the production program. The second is the QC program (QC means quality control). When two programs agree, both are much more likely to be right.
The production program here is the ADSL of chapter 9. The specification for four of its variables:
| Variable | Rule |
|---|---|
TRTSDT |
The date of the first dose: the earliest EX start date |
TRTEDT |
The date of the last dose: the latest EX end date. A subject with an open dose row gets RFENDTC |
AGEGR1 |
<65, 65-80 or >80 |
SAFFL |
"Y" for a randomised subject who took the study drug |
The QC program reads the same specification, but it does not read the production code. It also uses other code on purpose. If both programs use the same function in the same way, they can share the same mistake.
# Chapter 15: the QC program. Same specification as chapter 9, other codelibrary(dplyr)
dm <- pharmaversesdtm::dm |> filter(ARMCD != "Scrnfail")ex <- pharmaversesdtm::ex
# TRTSDT: sort the EX rows, keep the first row of each subjectfirst_dose <- ex |> arrange(USUBJID, EXSTDTC, EXSEQ) |> slice_head(n = 1, by = USUBJID) |> transmute(USUBJID, TRTSDT = as.Date(EXSTDTC))
# TRTEDT: a subject with an open dose row ends at RFENDTC, else at the# latest EX end date. Text dates in ISO 8601 sort in date orderlast_dose <- ex |> summarise( open = any(is.na(EXENDTC)), last_end = max(EXENDTC, na.rm = TRUE), .by = USUBJID )#> Warning:#> There were 2 warnings in `summarise()`.#> The first warning was:#> i In argument: `last_end = max(EXENDTC, na.rm = TRUE)`.#> i In group 86: `USUBJID = "01-705-1018"`.#> Caused by warning in `max()`:#> ! no non-missing arguments, returning NA#> i Run `dplyr::last_dplyr_warnings()` to see the 1 remaining warning.
qc <- dm |> select(USUBJID, AGE, RFENDTC) |> left_join(first_dose, by = join_by(USUBJID)) |> left_join(last_dose, by = join_by(USUBJID)) |> mutate( TRTEDT = as.Date(if_else(open, RFENDTC, last_end)), # whole years: (-Inf, 64] is "<65", (64, 80] is "65-80" AGEGR1 = as.character(cut( AGE, breaks = c(-Inf, 64, 80, Inf), labels = c("<65", "65-80", ">80") )), SAFFL = if_else(USUBJID %in% ex$USUBJID, "Y", "N") ) |> select(USUBJID, TRTSDT, TRTEDT, AGEGR1, SAFFL)
qc |> filter(USUBJID == "01-701-1015")#> # A tibble: 1 x 5#> USUBJID TRTSDT TRTEDT AGEGR1 SAFFL#> <chr> <date> <date> <chr> <chr>#> 1 01-701-1015 2014-01-02 2014-07-02 <65 Ycount(qc, AGEGR1, SAFFL)#> # A tibble: 3 x 3#> AGEGR1 SAFFL n#> <chr> <chr> <int>#> 1 65-80 Y 144#> 2 <65 Y 33#> 3 >80 Y 77saveRDS(qc, "data/15_qc_adsl.rds")The two programs differ at each step:
| Variable | Production (chapter 9) | QC (this program) |
|---|---|---|
TRTSDT |
min() of the dates |
arrange(), then the first row with slice_head() |
TRTEDT |
max() gives NA for an open row, then coalesce() |
any(is.na()) decides, if_else() chooses |
AGEGR1 |
case_when() |
cut() with break points |
SAFFL |
from SUPPDM | from EX: the subject has a dose row |
cut() puts each number into an interval and gives the interval a label. By default an interval is closed on the right: (64, 80] holds 65 to 80. Section 15.3 shows why this detail matters.
For 01-701-1015 the QC program gives TRTSDT 2014-01-02, TRTEDT 2014-07-02 and AGEGR1 = "<65", as chapter 9 did. All 254 subjects have SAFFL = "Y".
15.2 Compare the two
Section titled “15.2 Compare the two”diffdf() of the package diffdf compares two data frames. The first one is the base (the data you trust, here production). The second one is the compare data (the data you check, here QC). keys names the columns that identify a row: diffdf() pairs the rows by key, not by position.
# Chapter 15: compare production (chapter 9) with the QC programlibrary(dplyr)library(diffdf)
vars <- c("USUBJID", "TRTSDT", "TRTEDT", "AGEGR1", "SAFFL")prod <- readRDS("data/adsl.rds") |> select(all_of(vars)) |> haven::zap_label()qc <- readRDS("data/15_qc_adsl.rds")
diffdf(prod, qc, keys = "USUBJID")#> Warning message:#> In diffdf(prod, qc, keys = "USUBJID") :#> There are columns in BASE and COMPARE with differing attributes !!#> Differences found between the objects!#>#> Summary of BASE and COMPARE#> ==================================================================#> PROPERTY BASE COMP#> ------------------------------------------------------------------#> Name prod qc#> Class "tbl_df, tbl, data.frame" "tbl_df, tbl, data.frame"#> Rows(#) 254 254#> Columns(#) 5 5#> ------------------------------------------------------------------#>#>#> There are columns in BASE and COMPARE with differing attributes !!#> ===============================================================#> VARIABLE ATTR_NAME VALUES.BASE VALUES.COMP#> ---------------------------------------------------------------#> USUBJID label NULL "Unique Subject Identifier"#> ---------------------------------------------------------------#>
# The values agree; only a label differs. Compare the values onlydiffdf(prod, haven::zap_label(qc), keys = "USUBJID")#> No issues were found!The first compare finds one difference, and it is not a value. The column USUBJID of the QC data has a label, Unique Subject Identifier, which it took from DM. The program removed the labels of production with haven::zap_label(). diffdf() compares the attributes of each column too, and a label is an attribute.
The second compare removes the labels of the QC data as well: “No issues were found!”. The four variables agree for all 254 subjects.
15.3 A planned difference: a boundary
Section titled “15.3 A planned difference: a boundary”A real QC program is not always right the first time. Here is a QC program with one mistake: the break points of cut() are 65 and 80, not 64 and 80.
# Chapter 15: a QC program with one mistake. cut() closes each group on the rightlibrary(dplyr)library(diffdf)
prod <- readRDS("data/adsl.rds") |> select(USUBJID, AGE, AGEGR1) |> haven::zap_label()
qc <- prod |> mutate(AGEGR1 = as.character(cut( AGE, breaks = c(-Inf, 65, 80, Inf), labels = c("<65", "65-80", ">80") )))
diffdf(prod, qc, keys = "USUBJID")#> Warning message:#> In diffdf(prod, qc, keys = "USUBJID") :#> Not all Values Compared Equal#> Differences found between the objects!#>#> Summary of BASE and COMPARE#> ==================================================================#> PROPERTY BASE COMP#> ------------------------------------------------------------------#> Name prod qc#> Class "tbl_df, tbl, data.frame" "tbl_df, tbl, data.frame"#> Rows(#) 254 254#> Columns(#) 3 3#> ------------------------------------------------------------------#>#>#> Not all Values Compared Equal#> =============================#> Variable No of Differences#> -----------------------------#> AGEGR1 4#> -----------------------------#>#>#> =======================================#> VARIABLE USUBJID BASE COMPARE#> ---------------------------------------#> AGEGR1 01-707-1206 65-80 <65#> AGEGR1 01-709-1309 65-80 <65#> AGEGR1 01-715-1107 65-80 <65#> AGEGR1 01-715-1319 65-80 <65#> ---------------------------------------#>diffdf() writes a warning, Not all Values Compared Equal, and lists 4 subjects: 01-707-1206, 01-709-1309, 01-715-1107 and 01-715-1319. All four are exactly 65 years old. The interval (-Inf, 65] includes 65, so the QC program puts them in <65.
Four of 254 subjects is a small number. In a frequency table of AGEGR1, nobody sees it. A full compare finds it at once, and names the subjects. That is why QC compares every value, not only the totals.
15.4 Read a diffdf report
Section titled “15.4 Read a diffdf report”A diffdf report has one section for each kind of problem. This program makes three subjects with four planned changes, so that every section appears:
# Chapter 15: the parts of a diffdf() report, on three subjects with planned changeslibrary(dplyr)library(diffdf)
base <- readRDS("data/adsl.rds") |> filter(USUBJID %in% c("01-701-1015", "01-701-1023", "01-701-1028")) |> select(USUBJID, AGE, SEX, TRTSDT) |> haven::zap_label()
compare <- base |> filter(USUBJID != "01-701-1028") |> # a subject only in BASE mutate( SEX = if_else(USUBJID == "01-701-1023", "F", SEX), # a value differs AGE = as.integer(AGE), # another type COUNTRY = "USA" # a column only in COMPARE )
res <- diffdf(base, compare, keys = "USUBJID")#> Warning message:#> In diffdf(base, compare, keys = "USUBJID") :#> There are columns in BASE and COMPARE with different classes !!#> There are rows in BASE that are not in COMPARE !!#> There are columns in COMPARE that are not in BASE !!#> Not all Values Compared Equalres#> Differences found between the objects!#>#> Summary of BASE and COMPARE#> ==================================================================#> PROPERTY BASE COMP#> ------------------------------------------------------------------#> Name base compare#> Class "tbl_df, tbl, data.frame" "tbl_df, tbl, data.frame"#> Rows(#) 3 2#> Columns(#) 4 5#> ------------------------------------------------------------------#>#>#> There are columns in BASE and COMPARE with different classes !!#> ==================================#> VARIABLE CLASS.BASE CLASS.COMP#> ----------------------------------#> AGE numeric integer#> ----------------------------------#>#>#> There are rows in BASE that are not in COMPARE !!#> =============#> USUBJID#> -------------#> 01-701-1028#> -------------#>#>#> There are columns in COMPARE that are not in BASE !!#> =========#> COLUMNS#> ---------#> COUNTRY#> ---------#>#>#> Not all Values Compared Equal#> =============================#> Variable No of Differences#> -----------------------------#> SEX 1#> -----------------------------#>#>#> ======================================#> VARIABLE USUBJID BASE COMPARE#> --------------------------------------#> SEX 01-701-1023 M F#> --------------------------------------#>
# The result is a list. A program can test it and use its partsdiffdf_has_issues(res)#> [1] TRUEnames(res)#> [1] "DataSummary" "VarClassDiffs" "ExtRowsBase" "ExtColsComp"#> [5] "NumDiff" "VarDiff_SEX"diffdf_issuerows(base, res)#> USUBJID AGE SEX TRTSDT#> 1 01-701-1023 64 M 2012-08-05| Section | What it means |
|---|---|
| Summary of BASE and COMPARE | the names, classes, rows and columns of the two data frames |
| columns with different classes | the same column has another type: AGE is a decimal number in one and a whole number in the other |
| rows in BASE that are not in COMPARE | a key in one data frame only: a lost subject |
| columns in COMPARE that are not in BASE | a column in one data frame only |
| Not all Values Compared Equal | the number of differences per column, then each difference with its key |
The result of diffdf() is a list, so a program can use it:
diffdf_has_issues(res)isTRUEorFALSE. A batch QC run can test it.names(res)shows the sections.res$NumDiffis the table of differences per column, andres$VarDiff_SEXlists the differences ofSEX.diffdf_issuerows(base, res)gives the rows ofbasewith a value difference. Here it is 01-701-1023. It does not list the subject that is missing fromcompare.
15.5 Numbers: tolerance
Section titled “15.5 Numbers: tolerance”Two right formulas can give numbers that differ in the last digits. A computer stores decimal numbers in binary, so CHG / BASE * 100 and (AVAL / BASE - 1) * 100 are not always exactly equal.
# Chapter 15: two right formulas for PCHG differ in the last digits. tolerancelibrary(dplyr)library(diffdf)
advs <- readRDS("data/advs.rds") |> filter(PARAMCD == "WEIGHT", !is.na(CHG))
prod <- advs |> transmute(USUBJID, VSSEQ, PCHG = CHG / BASE * 100)qc <- advs |> transmute(USUBJID, VSSEQ, PCHG = (AVAL / BASE - 1) * 100)
# How many values are not exactly equal, and by how much at most?nrow(advs)#> [1] 1542sum(prod$PCHG != qc$PCHG)#> [1] 1236max(abs(prod$PCHG - qc$PCHG))#> [1] 1.421085e-14
# tolerance = 0: every tiny difference countsres <- diffdf(prod, qc, keys = c("USUBJID", "VSSEQ"), tolerance = 0, suppress_warnings = TRUE)res$NumDiff#> # A tibble: 1 x 2#> Variable `No of Differences`#> * <chr> <int>#> 1 PCHG 1236
# The default tolerance, sqrt(.Machine$double.eps), is about 1.5e-08diffdf(prod, qc, keys = c("USUBJID", "VSSEQ"))#> No issues were found!1236 of the 1542 weight changes are not exactly equal. The largest difference is 1.421085e-14, a number with 13 zeros after the decimal point. No table shows it.
tolerance is the largest difference that diffdf() still counts as equal. With tolerance = 0, diffdf() reports all 1236. The default is sqrt(.Machine$double.eps), about 1.5e-08. With it, “No issues were found!”. Keep the default. Change it only when the specification says how close two values must be.
suppress_warnings = TRUE stops the warning of diffdf(). Use it when a program reads the result itself, as program 15_7 does.
15.6 Compare with the reference ADaM
Section titled “15.6 Compare with the reference ADaM”Double programming checks the code against the specification. It cannot check the specification. Two programs that follow the same wrong rule agree. A second look comes from outside: the reference ADaM data sets of pharmaverseadam, built by other people for the same study.
# Chapter 15: our ADSL against the reference ADSL of pharmaverseadamlibrary(dplyr)library(diffdf)
vars <- c("USUBJID", "TRTSDT", "TRTEDT", "AGEGR1", "SAFFL")ours <- readRDS("data/adsl.rds") |> select(all_of(vars)) |> haven::zap_label()ref <- pharmaverseadam::adsl |> filter(ARMCD != "Scrnfail") |> select(all_of(vars)) |> haven::zap_label()
res <- diffdf(ours, ref, keys = "USUBJID")#> Warning message:#> In diffdf(ours, ref, keys = "USUBJID") :#> Not all Values Compared Equalres$NumDiff#> # A tibble: 2 x 2#> Variable `No of Differences`#> * <chr> <int>#> 1 TRTEDT 6#> 2 AGEGR1 254
# 254 of 254 differ: look for a rule, not for 254 bugsinner_join(ours, ref, by = join_by(USUBJID), suffix = c("", "_REF")) |> count(AGEGR1, AGEGR1_REF)#> # A tibble: 3 x 3#> AGEGR1 AGEGR1_REF n#> <chr> <chr> <int>#> 1 65-80 >64 144#> 2 <65 18-64 33#> 3 >80 >64 77Two variables differ:
TRTEDTdiffers for 6 subjects. These are the subjects with an open dose row of section 9.3.AGEGR1differs for all 254 subjects. When every row differs, look for a rule, not for 254 bugs. The count shows the rule: the reference uses two groups,18-64and>64. Every<65of yours is18-64there (33 subjects). Every65-80(144) and>80(77) is>64. The two specifications define other age groups.
The same check for the whole study is one function, called once for each data set:
# Chapter 15: one QC run over three data sets: the number of differences per variablelibrary(dplyr)library(purrr)library(diffdf)
# The differences of one comparison as a tibble: one row per variableqc_diff <- function(ours, ref, keys, vars) { ours <- haven::zap_label(select(ours, all_of(c(keys, vars)))) ref <- haven::zap_label(select(ref, all_of(c(keys, vars)))) res <- diffdf(ours, ref, keys = keys, suppress_warnings = TRUE) tibble( variable = vars, n_ours = nrow(ours), n_diff = map_int(vars, \(v) { nrow(res[[paste0("VarDiff_", v)]] %||% tibble()) }) )}
ref_adlb <- pharmaverseadam::adlb |> filter(LBCAT == "CHEMISTRY", is.na(DTYPE))
bind_rows( ADSL = qc_diff( readRDS("data/adsl.rds"), filter(pharmaverseadam::adsl, ARMCD != "Scrnfail"), keys = "USUBJID", vars = c("TRTSDT", "TRTEDT", "AGEGR1", "SAFFL") ), ADAE = qc_diff( readRDS("data/adae.rds"), pharmaverseadam::adae, keys = c("USUBJID", "AESEQ"), vars = c("ASTDT", "AENDT", "TRTEMFL") ), ADLB = qc_diff( readRDS("data/adlb.rds"), ref_adlb, keys = c("USUBJID", "LBSEQ"), vars = c("AVAL", "BASE", "CHG") ), .id = "dataset")#> # A tibble: 10 x 4#> dataset variable n_ours n_diff#> <chr> <chr> <int> <int>#> 1 ADSL TRTSDT 254 0#> 2 ADSL TRTEDT 254 6#> 3 ADSL AGEGR1 254 254#> 4 ADSL SAFFL 254 0#> 5 ADAE ASTDT 1191 0#> 6 ADAE AENDT 1191 0#> 7 ADAE TRTEMFL 1191 4#> 8 ADLB AVAL 32740 0#> 9 ADLB BASE 32740 0#> 10 ADLB CHG 32740 777
# Where are the 777 CHG differences? At visits that are not a scheduled weekinner_join( readRDS("data/adlb.rds") |> select(USUBJID, LBSEQ, VISIT, CHG), ref_adlb |> select(USUBJID, LBSEQ, CHG_REF = CHG), by = join_by(USUBJID, LBSEQ)) |> filter(is.na(CHG) != is.na(CHG_REF)) |> count( visit = stringr::word(VISIT, 1), ours = !is.na(CHG), ref = !is.na(CHG_REF) )#> # A tibble: 3 x 4#> visit ours ref n#> <chr> <lgl> <lgl> <int>#> 1 AMBUL FALSE TRUE 18#> 2 RETRIEVAL FALSE TRUE 18#> 3 UNSCHEDULED FALSE TRUE 741qc_diff()keeps the key and the variables, removes the labels, and runsdiffdf()without a warning. It then counts the rows of eachVarDiff_section.%||%gives the right side when the left side isNULL: a variable without differences has no section.bind_rows()with names and.id = "dataset"stacks the three results and adds the name of each data set as a column.
The table is the QC summary of the study:
- ADSL: 6
TRTEDTand 254AGEGR1, as above. - ADAE:
TRTEMFLdiffers on 4 events. They are the 4 events of 01-705-1303 of section 11.4. The reference ends the treatment of this subject at its ownTRTEDT, 2013-12-30, and counts an event only up to 30 days after it. - ADLB:
CHGdiffers on 777 rows. They are all at visits that are not a scheduled week: 741 unscheduled, 18 retrieval and 18 ambulatory ECG rows. The reference has aCHGthere, yours has none (section 10.6). Chapter 10 counted more rows, because its join also matched the derived rows of the reference.
Not one of these differences is a bug in your code. Each one is a rule that differs between two specifications.
15.7 Write the finding for a reviewer
Section titled “15.7 Write the finding for a reviewer”You do not decide which rule is right. You write the finding down and ask the statistician. A QC finding is a short note: what differs, how many rows, the cause, and the question. The reviewer must understand it without running a program.
QC finding 15-01: ADSL.TRTEDT, 6 of 254 subjects differ from the reference Compared: data/adsl.rds against pharmaverseadam::adsl, key USUBJID Found: 6 differences, all subjects with an EX row without an end date (01-704-1233, 01-705-1018, 01-705-1031, 01-705-1303, 01-705-1377, 01-705-1382). For 01-705-1018 and 01-705-1382 the reference has no TRTEDT. Cause: our rule uses RFENDTC for an open dose row (section 9.3). The reference uses the latest end date that exists. Impact: ADAE.TRTEMFL differs on 4 events of 01-705-1303. Question: Which rule does the SAP want for an open dose row? Status: open. Programs 15_6 and 15_7 show the numbers.Write one note per rule, not per subject. The AGEGR1 finding is a second note: the specification of this book has three age groups, the reference has two. Keep the notes with the QC programs, and add the answer of the statistician when it comes. The next programmer then knows why the numbers differ.
15.8 Check the data rules
Section titled “15.8 Check the data rules”Some mistakes need no second program. A data rule is a fact that must be true for every row: a key is unique, ADSL has one row per subject, a required value is not missing. A rule check is a short program that counts the rows that break each rule.
# Chapter 15: a QC program for data rules. Each rule is one row: PASS or FAILlibrary(dplyr)
adsl <- readRDS("data/adsl.rds")adae <- readRDS("data/adae.rds")advs <- readRDS("data/advs.rds")
# A rule: its name, its data set, and the number of rows that break itrule <- function(name, dataset, n_bad) { result <- if_else(n_bad == 0, "PASS", "FAIL") tibble(rule = name, dataset, n_bad, result)}n_dup <- function(data, ...) sum(duplicated(select(data, ...)))
agegr1 <- c("<65", "65-80", ">80")lost <- anti_join(adae, adsl, by = join_by(USUBJID))
checks <- bind_rows( rule("one row per subject", "ADSL", n_dup(adsl, USUBJID)), rule("TRTSDT is not missing", "ADSL", sum(is.na(adsl$TRTSDT))), rule("TRTEDT >= TRTSDT", "ADSL", sum(adsl$TRTEDT < adsl$TRTSDT)), rule("AGEGR1 in its codelist", "ADSL", sum(!adsl$AGEGR1 %in% agegr1)), rule("USUBJID, AESEQ is a key", "ADAE", n_dup(adae, USUBJID, AESEQ)), rule("every subject is in ADSL", "ADAE", nrow(lost)), rule("TRTEMFL is Y or empty", "ADAE", sum(!adae$TRTEMFL %in% c("Y", NA))), rule("USUBJID, VSSEQ is a key", "ADVS", n_dup(advs, USUBJID, VSSEQ)), rule("AVAL is not missing", "ADVS", sum(is.na(advs$AVAL))))checks#> # A tibble: 9 x 4#> rule dataset n_bad result#> <chr> <chr> <int> <chr>#> 1 one row per subject ADSL 0 PASS#> 2 TRTSDT is not missing ADSL 0 PASS#> 3 TRTEDT >= TRTSDT ADSL 0 PASS#> 4 AGEGR1 in its codelist ADSL 0 PASS#> 5 USUBJID, AESEQ is a key ADAE 0 PASS#> 6 every subject is in ADSL ADAE 0 PASS#> 7 TRTEMFL is Y or empty ADAE 0 PASS#> 8 USUBJID, VSSEQ is a key ADVS 0 PASS#> 9 AVAL is not missing ADVS 8 FAIL
# A FAIL is a question for the data, not yet a bug: look at the rows in VSadvs |> filter(is.na(AVAL)) |> left_join( pharmaversesdtm::vs |> select(USUBJID, VSSEQ, VSSTAT), by = join_by(USUBJID, VSSEQ) ) |> count(PARAMCD, VSSTAT)#> # A tibble: 3 x 3#> PARAMCD VSSTAT n#> <chr> <chr> <int>#> 1 DIABP NOT DONE 2#> 2 PULSE NOT DONE 3#> 3 SYSBP NOT DONE 3rule()makes one row of the result: the rule, the data set, the number of bad rows, andPASSorFAIL.duplicated()isTRUEfor each row that has the same values as an earlier row. On the key columns, it finds a key that is not unique.anti_join()keeps the ADAE rows without a subject in ADSL (chapter 5).
Eight rules pass. One fails: 8 ADVS rows have no AVAL. A FAIL is a question for the data, not yet a bug. The rows in VS have VSSTAT = "NOT DONE": the test was not done at that visit. So the rule is too strict. Exercise 15-5 writes it the way the data needs it.
Run the rule check after every change to Part II. It takes a second, and it finds a lost key or a copied row before a table shows a wrong N.
15.9 Check every warning and message
Section titled “15.9 Check every warning and message”A program can give the right numbers and still have problems: a value that became NA, a join that copied rows. R says so in a warning or a message in the console. A reviewer reads them all, so you read them first.
This table program has three problems:
# A small table program with three problems that a reviewer must seelibrary(dplyr)
adsl <- pharmaverseadam::adsl |> filter(SAFFL == "Y")adae <- pharmaverseadam::adae |> filter(TRTEMFL == "Y")
# Problem 1: a lab-style value "<5" cannot become a numberdose <- as.numeric(c("54", "81", "<5"))
# Problem 2: summarise() by two groups keeps one of themby_sev <- adae |> group_by(TRT01A, AESEV) |> summarise(n_subj = n_distinct(USUBJID))
# Problem 3: the join copies rows: many AEs per subject, many rows per subjectboth <- adae |> select(USUBJID, AEDECOD) |> left_join(adae |> select(USUBJID, AESEV), by = join_by(USUBJID))The log check runs the program and collects everything that R says:
# Chapter 15: a log check. Run a program and collect every warning and messagelibrary(dplyr)
check_log <- function(file) { found <- tibble(kind = character(), text = character()) keep <- function(kind, cnd) { text <- gsub("\\s+", " ", conditionMessage(cnd)) found <<- add_row(found, kind = kind, text = substr(text, 1, 60)) } withCallingHandlers( source(file, local = new.env()), warning = function(w) { keep("WARNING", w) invokeRestart("muffleWarning") }, message = function(m) { keep("MESSAGE", m) invokeRestart("muffleMessage") } ) found}
log <- check_log("15_logcheck/ae_counts.R")log#> # A tibble: 3 x 2#> kind text#> <chr> <chr>#> 1 WARNING NAs introduced by coercion#> 2 MESSAGE `summarise()` has regrouped the output. i Summaries were com#> 3 WARNING Detected an unexpected many-to-many relationship between `x`count(log, kind)#> # A tibble: 2 x 2#> kind n#> <chr> <int>#> 1 MESSAGE 1#> 2 WARNING 2
# In a test, the same check is one line: this step must give no warningtestthat::expect_no_warning( pharmaverseadam::adae |> count(TRT01A, AESEV))testthat::expect_no_warning(as.numeric("<5"))#> Error : Expected `as.numeric("<5")` not to throw any warnings.#> Actually got a <simpleWarning> with message:#> NAs introduced by coercionwithCallingHandlers()runs code and calls a function of yours for each condition: a warning, a message or an error. The program goes on after each one.keep()adds a row tofound.<<-assigns tofoundin the function around it, not to a new local copy.invokeRestart("muffleWarning")stops the warning from reaching the console. It is now a row in the table instead.source(file, local = new.env())runs the program in a new, empty environment, so its objects do not mix with yours.
The check finds the three problems: 2 warnings and 1 message. NAs introduced by coercion is the text "<5" that became NA. The message says that summarise() kept a group. The many-to-many warning is the join that copied rows.
In a test, one line does the same for one step. testthat::expect_no_warning() is silent when the code gives no warning. When it does, it stops with an error that quotes the warning. Chapter 19 uses such tests for a function library.
15.10 Summary
Section titled “15.10 Summary”- Double programming: a second program, from the same specification, with other code. Then a compare must find no differences.
diffdf(base, compare, keys = ...)compares two data frames by key. It reports rows and columns in one data frame only, other types, other attributes and other values.- The result of
diffdf()is a list.diffdf_has_issues(),$NumDiffanddiffdf_issuerows()let a program use it. toleranceignores rounding noise. Keep the default unless the specification says otherwise.- A compare with the reference checks the specification, not the code. When all rows differ, look for a rule.
- Write each finding as a short note: what, how many, the cause, the question.
- Check the data rules (keys, one row per subject, required values) and every warning and message.
15.11 Exercises
Section titled “15.11 Exercises”Exercise 15-1. Compute TRTDURD = TRTEDT - TRTSDT + 1 from your ADSL and compare it with TRTDURD of the reference ADSL. Which subjects differ, and why?
Exercise 15-2. In program 15_3, add right = FALSE to cut(). How many subjects differ now, and what do they have in common?
Exercise 15-3. List the EX rows of the 6 subjects behind the TRTEDT differences. What do all six have in common?
Exercise 15-4. check_log() stops at the first error, and the error is not in its table. Change it so that an error becomes a row of kind ERROR. Test it on a program with a warning and an error.
Exercise 15-5. Write the rule “AVAL is not missing” again: AVAL may be missing only when VSSTAT is "NOT DONE", and then it must be missing. Does ADVS pass?
The answers are in Answers, Parts IV and V.
Found an error or have a question? Write to datastep.studio@gmail.com· Every program runs in DataStep Studio for R, the R extension for VS Code and Positron.
Data: the CDISC pilot study (cdiscpilot01), from the R packagespharmaversesdtm andpharmaverseadam (Apache License 2.0).
R is a project of the R Foundation. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. This book is not affiliated with or endorsed by SAS Institute Inc. or the R Foundation. CDISC, SDTM and ADaM are trademarks of the Clinical Data Interchange Standards Consortium.
© 2026 DataStep Labs. All rights reserved.