Skip to content

Solve a programming problem

A beginner starts with code. An experienced clinical programmer starts with a note. Then they test their assumptions on the data, write down each decision, and write the derivation last, in small steps. At the end they compare the result with a second source. The derivation then has fewer surprises, and the QC programmer can follow every decision.

This chapter shows the method on one real task of the pilot study. Chapter 15 found differences that come from rules. This method finds such a rule before QC does.

Derive WEIGHTBL, the baseline weight in kg, for ADSL.

It looks like one line of code. Exercise 9-2 wrote that line: take the weight with the SDTM flag VSBLFL = "Y". One subject then got no weight. This chapter does the task again, the way a senior programmer does it.

16.2 Write a plain-language note before any code

Section titled “16.2 Write a plain-language note before any code”

Write four short lines. If you cannot fill one of them, you are not ready to code. Ask first.

Part For this task
Requirement WEIGHTBL = the weight of the subject at baseline, in kg
Source data SDTM VS, the rows with VSTESTCD = "WEIGHT"; TRTSDT from ADSL
Rule The SAP rule of chapter 10: the last non-missing value on or before the first dose
Open questions A weight on the day of the first dose has no time: is it before the dose? Does the SDTM flag VSBLFL follow the SAP rule?

Then draw the flow. Plain text is enough:

VS (WEIGHT) --> step 010: weights with a date --> step 020: decide the row --> step 030: WEIGHTBL in ADSL
ADSL (TRTSDT) ---^ (WTBLSRC says why)

The note takes five minutes. It names the data, the rule and the questions. It is also the first page of the QC programmer’s work: they read it, not your code.

16.3 Test your assumptions with a check program

Section titled “16.3 Test your assumptions with a check program”

Every rule rests on assumptions about the data. Test them before you derive. A check program is short and reads the data only. It answers yes-or-no questions with a count: 0 means that the assumption holds.

programs/16_1_check.R
# Chapter 16: a check program. Test each assumption of the note before you derive
library(dplyr)
adsl <- readRDS("data/adsl.rds") |>
select(USUBJID, TRTSDT)
weight <- pharmaversesdtm::vs |>
filter(VSTESTCD == "WEIGHT") |>
select(USUBJID, VSSEQ, VISIT, VSDTC, VSSTRESN, VSSTRESU, VSBLFL) |>
mutate(ADT = as.Date(substr(VSDTC, 1, 10))) |>
inner_join(adsl, by = join_by(USUBJID))
# The rule of the note: the last weight on or before the first dose
rule <- weight |>
filter(!is.na(VSSTRESN), ADT <= TRTSDT) |>
arrange(USUBJID, ADT, VSSEQ) |>
slice_tail(n = 1, by = USUBJID)
flag <- weight |>
filter(VSBLFL == "Y")
no_weight <- anti_join(adsl, weight, by = join_by(USUBJID))
two_a_day <- count(weight, USUBJID, ADT) |> filter(n > 1)
no_rule <- anti_join(adsl, rule, by = join_by(USUBJID))
other_flag <- anti_join(rule, flag, by = join_by(USUBJID, VSSEQ))
tribble(
~check, ~n,
"1 subjects without any weight", nrow(no_weight),
"2 weights not in kg", sum(weight$VSSTRESU != "kg"),
"3 weights without a value", sum(is.na(weight$VSSTRESN)),
"4 weights without a full date", sum(nchar(weight$VSDTC) != 10),
"5 subjects with 2 weights on one date", nrow(two_a_day),
"6 subjects without a weight before dose", nrow(no_rule),
"7 rule weights on the day of the dose", sum(rule$ADT == rule$TRTSDT),
"8 subjects with VSBLFL on another row", nrow(other_flag)
)
#> # A tibble: 8 x 2
#> check n
#> <chr> <int>
#> 1 1 subjects without any weight 0
#> 2 2 weights not in kg 0
#> 3 3 weights without a value 0
#> 4 4 weights without a full date 0
#> 5 5 subjects with 2 weights on one date 0
#> 6 6 subjects without a weight before dose 0
#> 7 7 rule weights on the day of the dose 253
#> 8 8 subjects with VSBLFL on another row 1
  • tribble() makes a small tibble row by row: ~check and ~n are the column names, then each line is one row. It is the easiest way to write a table of checks by hand.
  • Each count has a name first (no_weight, two_a_day), so a reader sees what each check computes.

Six checks give 0. Every randomised subject has a weight in kg, with a value and a full date, at most one per date, and before the first dose. Two checks need a decision:

  • Check 7: 253. For 253 subjects, the last weight on or before the dose was taken on the day of the first dose. VSDTC has no time, so you cannot know whether the weight came before the dose.
  • Check 8: 1. For one subject, the SDTM flag VSBLFL is not on the row of the SAP rule.

Without the check program, you would find check 8 at QC time, or never. Look at that subject now:

programs/16_2_look.R
# Chapter 16: one check failed. Look at the subject before you decide anything
library(dplyr)
trtsdt <- readRDS("data/adsl.rds") |>
filter(USUBJID == "01-702-1082") |>
pull(TRTSDT)
trtsdt
#> [1] "2013-07-26"
pharmaversesdtm::vs |>
filter(USUBJID == "01-702-1082", VSTESTCD == "WEIGHT") |>
select(VSSEQ, VISIT, VSDTC, VSSTRESN, VSBLFL)
#> # A tibble: 6 x 5
#> VSSEQ VISIT VSDTC VSSTRESN VSBLFL
#> <dbl> <chr> <chr> <dbl> <chr>
#> 1 101 SCREENING 1 2013-07-03 54.4 <NA>
#> 2 102 WEEK 2 2013-08-08 54.4 <NA>
#> 3 103 WEEK 4 2013-08-24 54.4 <NA>
#> 4 104 WEEK 6 2013-09-06 54.4 <NA>
#> 5 105 WEEK 8 2013-09-28 54.0 <NA>
#> 6 106 WEEK 12 2013-11-17 50.8 <NA>
# Which visits does this subject have before the first dose, in any VS test?
pharmaversesdtm::vs |>
filter(USUBJID == "01-702-1082", as.Date(VSDTC) <= trtsdt) |>
count(VISIT, VSTESTCD)
#> # A tibble: 13 x 3
#> VISIT VSTESTCD n
#> <chr> <chr> <int>
#> 1 BASELINE DIABP 3
#> 2 BASELINE PULSE 3
#> 3 BASELINE SYSBP 3
#> 4 SCREENING 1 DIABP 3
#> 5 SCREENING 1 HEIGHT 1
#> 6 SCREENING 1 PULSE 3
#> 7 SCREENING 1 SYSBP 3
#> 8 SCREENING 1 TEMP 1
#> 9 SCREENING 1 WEIGHT 1
#> 10 SCREENING 2 DIABP 3
#> 11 SCREENING 2 PULSE 3
#> 12 SCREENING 2 SYSBP 3
#> 13 SCREENING 2 TEMP 1

01-702-1082 had its first dose on 2013-07-26. It had a weight at SCREENING 1, on 2013-07-03, and then not again until WEEK 2. At the BASELINE visit, the site measured blood pressure and pulse, but not weight. VSBLFL is on no weight row of this subject. The SAP rule takes the screening weight, 54.4 kg.

16.4 Write the decisions down, then derive in small steps

Section titled “16.4 Write the decisions down, then derive in small steps”

Now write the derivation. Four habits make it easy to review:

  1. A header with the decisions. The purpose, the input, the rule in words, and one line for each decision: a number, the date, what was decided and which check led to it. A change log follows.
  2. Numbered steps with gaps. wt_010, wt_020, then the result. Each step reads the one before it, and you can look at each one. A new step later fits in as wt_015.
  3. The decision apart from the computation. Step 020 decides which row is the baseline and writes the reason in WTBLSRC. Step 030 only copies the value. A reviewer reads all decisions in one place, and a new rule changes one step.
  4. A comment with the decision number on the line that follows it: # D1: on or before.
programs/16_3_derive.R
# Chapter 16: WEIGHTBL in small numbered steps, with the decisions apart
#
# Purpose: WEIGHTBL, the baseline weight in kg, for ADSL
# Input: data/adsl.rds (TRTSDT), pharmaversesdtm::vs (VSTESTCD WEIGHT)
# Rule: the last non-missing weight on or before TRTSDT (the SAP rule
# of chapter 10), not the SDTM flag VSBLFL
# Decisions: D1 2026-11-19 A weight on the day of the first dose counts:
# VSDTC has no time (check 7, 253 subjects)
# D2 2026-11-19 VSBLFL is not used: for 01-702-1082 it is on
# no row, the SAP rule takes SCREENING 1 (check 8)
# D3 2026-11-19 No weight before the dose gives NA (check 6: 0)
# Change log:
# 1.0 2026-11-19 First version.
library(dplyr)
adsl <- readRDS("data/adsl.rds")
# Step 010: the weights of the randomised subjects, with a real date
wt_010 <- pharmaversesdtm::vs |>
filter(VSTESTCD == "WEIGHT", !is.na(VSSTRESN)) |>
transmute(USUBJID, VSSEQ, VISIT, ADT = as.Date(VSDTC), AVAL = VSSTRESN) |>
inner_join(select(adsl, USUBJID, TRTSDT), by = join_by(USUBJID))
# Step 020: the decision. Which row is the baseline, and why (WTBLSRC)
wt_020 <- wt_010 |>
filter(ADT <= TRTSDT) |> # D1: on or before
arrange(USUBJID, ADT, VSSEQ) |>
slice_tail(n = 1, by = USUBJID) |>
mutate(WTBLSRC = if_else(ADT == TRTSDT, "day of dose", "before dose"))
# Step 030: the computation. Copy the value to ADSL, one row per subject
adsl_wt <- adsl |>
left_join(
wt_020 |> select(USUBJID, WEIGHTBL = AVAL, WTBLSRC, WTBLVIS = VISIT),
by = join_by(USUBJID), relationship = "one-to-one"
) # D3: no row gives NA
# The proof that the decisions ran as agreed
count(adsl_wt, WTBLSRC, WTBLVIS)
#> # A tibble: 2 x 3
#> WTBLSRC WTBLVIS n
#> <chr> <chr> <int>
#> 1 before dose SCREENING 1 1
#> 2 day of dose BASELINE 253
adsl_wt |>
filter(USUBJID %in% c("01-701-1015", "01-702-1082")) |>
select(USUBJID, TRTSDT, WEIGHTBL, WTBLSRC, WTBLVIS)
#> # A tibble: 2 x 5
#> USUBJID TRTSDT WEIGHTBL WTBLSRC WTBLVIS
#> <chr> <date> <dbl> <chr> <chr>
#> 1 01-701-1015 2014-01-02 54.4 day of dose BASELINE
#> 2 01-702-1082 2013-07-26 54.4 before dose SCREENING 1
saveRDS(adsl_wt, "data/16_adsl_weightbl.rds")

The count of the decision columns is your proof that the rules ran as agreed. 253 subjects have their weight from the day of the dose, at the BASELINE visit. 1 subject has it from before the dose, at SCREENING 1: exactly the subject of check 8. Subject 01-701-1015 gets 54.4 kg from the day of the dose.

WTBLSRC and WTBLVIS are not ADaM variables. They are decision data: columns that record why each value is what it is. Keep them while you work and QC, and drop them from the final data set, or keep them in a QC data set.

A result is not finished until a second source agrees. Use a source that does not share your code.

programs/16_4_compare.R
# Chapter 16: compare at the end. WEIGHTBL against two independent sources
library(dplyr)
library(diffdf)
ours <- readRDS("data/16_adsl_weightbl.rds") |>
select(USUBJID, WEIGHTBL) |>
haven::zap_label()
# Source 1: the baseline of your ADVS of chapter 10, another program
advs_bl <- readRDS("data/advs.rds") |>
filter(PARAMCD == "WEIGHT", ABLFL == "Y") |>
select(USUBJID, WEIGHTBL = AVAL) |>
haven::zap_label()
diffdf(ours, advs_bl, keys = "USUBJID")
#> No issues were found!
# Source 2: the reference ADVS of pharmaverseadam
ref_bl <- pharmaverseadam::advs |>
filter(PARAMCD == "WEIGHT", ABLFL == "Y") |>
select(USUBJID, WEIGHTBL = AVAL) |>
haven::zap_label()
diffdf(ours, ref_bl, keys = "USUBJID")
#> No issues were found!

Both compares find no issues:

  • Your ADVS of chapter 10 is another program, written for another purpose: its baseline flag gives the same 254 weights.
  • The reference ADVS of pharmaverseadam also gives the same 254 weights, the screening weight of 01-702-1082 too.

So the SAP rule is the rule of the reference too, and VSBLFL alone would have been wrong for one subject. The check program found that subject before any line of the derivation existed.

A decision in the header is text. A reader can check it, but the program cannot. Write each decision once more as a row of data, with the number of subjects it must change. Then the program checks itself at every run.

programs/16_5_decisions.R
# Chapter 16: the decisions as data. Each one says what it must change, and the
# program checks that it did
library(dplyr)
decisions <- tribble(
~id, ~question, ~answer, ~expect,
"D1", "Weight on the day of the dose: baseline?", "yes: on or before", 253,
"D2", "VSBLFL or the SAP rule?", "the SAP rule", 1,
"D3", "No weight before the dose?", "WEIGHTBL is NA", 0
)
adsl_wt <- readRDS("data/16_adsl_weightbl.rds")
found <- c(
D1 = sum(adsl_wt$WTBLSRC == "day of dose", na.rm = TRUE),
D2 = sum(adsl_wt$WTBLVIS != "BASELINE", na.rm = TRUE),
D3 = sum(is.na(adsl_wt$WEIGHTBL))
)
check <- decisions |>
mutate(found = found[id], ok = found == expect)
check |>
select(id, answer, expect, found, ok)
#> # A tibble: 3 x 5
#> id answer expect found ok
#> <chr> <chr> <dbl> <int> <lgl>
#> 1 D1 yes: on or before 253 253 TRUE
#> 2 D2 the SAP rule 1 1 TRUE
#> 3 D3 WEIGHTBL is NA 0 0 TRUE
# A decision that does not hold stops the run
stopifnot(all(check$ok))
  • Each row is one decision: its number, the question, the answer and expect, the count that the check program found.
  • found counts the same thing in the result of step 030. found[id] takes the count of each decision by its name.
  • stopifnot(all(check$ok)) stops the run when a count differs. It is silent here: all three decisions hold.

When the next data delivery has a new subject without a BASELINE weight, D2 finds 2 instead of 1, and the run stops. That is what you want: a new case of an old question must go back to the statistician, not slip into the data. Then update expect, and the date of the decision in the header.

The SAP, the ADaM specification and the table shells sometimes say different things. Here, SDTM said one thing (VSBLFL) and the SAP said another. Do not choose yourself, and do not code both versions “to be safe”. Write down the two readings, and ask the statistician. Add the answer to your note and to the specification, so that the QC programmer and the next study find it.

Step What you have at the end
Note the requirement, the source, the rule, the open questions
Check program a count for each assumption; the subjects behind each failure
Decisions a numbered list in the header, and a table that the program checks
Derivation small numbered steps; the decision apart from the computation
Compare diffdf() against a source that does not share your code
  • Write a plain-language note first: requirement, source, rule, open questions.
  • Test every assumption with a short check program before you derive. A count of 0 means that the assumption holds.
  • Look at the subjects behind a failed check. Then ask; do not decide alone.
  • Write each decision in the header, with a number and the check behind it. Keep it as data too, with the count it must change.
  • Derive in small numbered steps. Keep the decision (which row, and why) apart from the computation.
  • Compare the result with a source that does not share your code.

Exercise 16-1. Add check 9 to program 16_1: weights below 30 kg or above 200 kg. What are the lowest and the highest weight in VS?

Exercise 16-2. The statistician changes decision D1: only a weight strictly before the day of the first dose is a baseline. Which step of program 16_3 changes? From which visit do the baselines come now, and what is WEIGHTBL of 01-701-1015?

Exercise 16-3. Write the check program for HEIGHTBL, the height at baseline, before you derive it. At which visits is the height measured? Does every subject have exactly one?

Exercise 16-4. Add steps 040 and 050 to program 16_3: HEIGHTBL, then BMIBL = WEIGHTBL / (HEIGHTBL / 100)^2. Compare BMIBL with the baseline BMI of the reference ADVS.

The answers are in Answers, Parts IV and V.

Found an error or have a question? Write to datastep.studio@gmail.com· Every program runs in DataStep Studio for R, the R extension for VS Code and Positron.

Data: the CDISC pilot study (cdiscpilot01), from the R packagespharmaversesdtm andpharmaverseadam (Apache License 2.0).

R is a project of the R Foundation. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. This book is not affiliated with or endorsed by SAS Institute Inc. or the R Foundation. CDISC, SDTM and ADaM are trademarks of the Clinical Data Interchange Standards Consortium.

© 2026 DataStep Labs. All rights reserved.