Skip to content

Your daily loop

Every programmer repeats one loop all day: write, run, read the log, look at the data, fix. You have used this loop since chapter 2. This chapter makes each part of it shorter and safer. It is the routine behind every later chapter of Part VI. All keys are also on the cheat sheet.

Part VI uses DataStep Studio for R, the editor extension of the setup. Each tool has two forms: a key or a command in the editor, and a dsr command that you type in a terminal. The book shows the dsr output, because it is the same text that the editor’s R Console shows.

Open programs/20_1_weight.R and press F4. F4 runs the selected lines. With nothing selected, it runs the whole file. Cmd/Ctrl+Shift+Enter (Run R File in the Session) also runs the whole file. This is the program:

programs/20_1_weight.R
# Chapter 20: a small derivation whose log tells you what each step made
library(dplyr)
adsl <- pharmaverseadam::adsl |>
filter(SAFFL == "Y") |>
select(USUBJID, TRT01A, AGE, SEX)
weight <- pharmaversesdtm::vs |>
filter(VSTESTCD == "WEIGHT", VSBLFL == "Y") |>
select(USUBJID, WEIGHTBL = VSSTRESN)
adsl <- adsl |>
left_join(weight, by = "USUBJID") |>
mutate(WGTGR = if_else(WEIGHTBL >= 70, ">=70 kg", "<70 kg"))
adsl |>
count(TRT01A, WGTGR)
#> # A tibble: 7 x 3
#> TRT01A WGTGR n
#> <chr> <chr> <int>
#> 1 Placebo <70 kg 55
#> 2 Placebo >=70 kg 31
#> 3 Xanomeline High Dose <70 kg 39
#> 4 Xanomeline High Dose >=70 kg 33
#> 5 Xanomeline Low Dose <70 kg 55
#> 6 Xanomeline Low Dose >=70 kg 40
#> 7 Xanomeline Low Dose <NA> 1

R itself prints only the table at the end. DataStep Studio for R writes more: a log, a record of what each step of the run did. It is in the R Console panel. In a terminal, dsr run gives the same log:

$ dsr run programs/20_1_weight.R
4 NOTE: adsl has 254 rows and 4 columns.
8 NOTE: weight has 253 rows and 2 columns.
12 NOTE: adsl has 254 rows and 6 columns (was 254 x 4). New: WEIGHTBL (1 missing), WGTGR (1 missing).
# A tibble: 7 × 3
TRT01A WGTGR n
<chr> <chr> <int>
1 Placebo <70 kg 55
2 Placebo >=70 kg 31
3 Xanomeline High Dose <70 kg 39
4 Xanomeline High Dose >=70 kg 33
5 Xanomeline Low Dose <70 kg 55
6 Xanomeline Low Dose >=70 kg 40
7 Xanomeline Low Dose <NA> 1
Workspace: new: adsl (tbl_df 254 x 6), weight (tbl_df 253 x 2)
RESULT: CLEAN (0.02 s)

Read it from top to bottom:

  • The number at the left is the line of the program where the step starts.
  • Each step that makes or changes a data frame gets a NOTE: its rows and columns, and for a changed data frame what it was before (was 254 x 4), the new columns and their missing values.
  • The output of the program comes at the line that printed it.
  • The Workspace: line says which objects the run made or changed.
  • The last line is the verdict: RESULT: CLEAN means no error, no warning and no suspicious message.

Now look at line 12: WEIGHTBL (1 missing). The safety population has 254 subjects, but weight has only 253 rows: one subject has no baseline weight. So one subject has no weight group, and the table has a row <NA>. The verdict is CLEAN, and still a subject is missing from a group. CLEAN means “no error”, not “right”. A NOTE with missing values is often the first sign of a problem. Read every NOTE after every run.

20.2 Three verdicts: CLEAN, CHECK and ERROR

Section titled “20.2 Three verdicts: CLEAN, CHECK and ERROR”

The last line of each run is one of these:

Verdict Meaning What you do
RESULT: CLEAN no error, no warning, no suspicious message look at the result
RESULT: CHECK a warning or a suspicious message (20.3) read the FIRST CAUSE: fix it, or write down why it is right
RESULT: ERROR the code stopped at the line that the FIRST CAUSE names fix the first cause only
RESULT: PRY the run waits at a stop chapter 24

Here is a program with a typo:

programs/20_3_typo.R
# Chapter 20: a typo stops the run, and the verdict names the line
library(dplyr)
pharmaverseadam::adsl |>
filter(SAFFL == "Y") |>
group_by(TRT01A) |>
summarise(n = n(), mean_age = mean(AGEE))
#> Error in `summarise()`:
#> i In argument: `mean_age = mean(AGEE)`.
#> i In group 1: `TRT01A = "Placebo"`.
#> Caused by error:
#> ! object 'AGEE' not found

The log of the same run:

$ dsr run programs/20_3_typo.R
5 ERROR: ℹ In argument: `mean_age = mean(AGEE)`.
ℹ In group 1: `TRT01A = "Placebo"`.
Caused by error:
! object 'AGEE' not found [in summarise(group_by(filter(pharmaverseadam::adsl, SAFFL == "Y"), ]
5 PRY: The frames of the error stay open: dsr pry stack, dsr pry ls, dsr pry eval <code>.
RESULT: ERROR | FIRST CAUSE: programs/20_3_typo.R:5 (called at line 5): ℹ In argument: `mean_age = mean(AGEE)`.
ℹ In group 1: `TRT01A = "Placebo"`.
Caused by error:
! object 'AGEE' not found [in summarise(group_by(filter(pharmaverseadam::adsl, SAFFL == "Y"), ] (0.05 s)

The first cause is the first problem of the run, with its file and line. Fix the first cause only, then run again: a later error is often a result of the first one. In the editor, the cursor goes to the line of the error.

The line PRY: The frames of the error stay open is a help for later. After an error you can still look at the values where it happened. Chapter 24 shows how.

20.3 The suspicious log: only the lines that matter

Section titled “20.3 The suspicious log: only the lines that matter”

Some problems do not stop a program. R prints a warning once, and it scrolls away. Some problems give no warning at all. This program has two such problems:

programs/20_2_bili.R
# Chapter 20: two problems that do not stop the run, and the suspicious log finds both
library(dplyr)
adsl <- pharmaverseadam::adsl |>
filter(SAFFL == "Y") |>
select(USUBJID, TRT01A)
bili <- pharmaversesdtm::lb |>
filter(LBTESTCD == "BILI") |>
mutate(BILI = as.numeric(LBORRES)) |>
select(USUBJID, VISIT, BILI)
#> Warning:
#> There was 1 warning in `mutate()`.
#> i In argument: `BILI = as.numeric(LBORRES)`.
#> Caused by warning:
#> ! NAs introduced by coercion
adsl_bili <- adsl |>
left_join(bili, by = "USUBJID")
nrow(adsl_bili)
#> [1] 1814

R printed one warning and the number 1814. The verdict of DataStep Studio for R is CHECK, and the suspicious log collects each problem with its line, a rule, why it matters and a fix:

$ dsr log --filter suspicious
Suspicious log of programs/20_2_bili.R (CHECK): 2 messages
8 WARNING [coercion] There was 1 warning in `mutate()`.
why: Values that are not numbers became NA: data were lost without an error. fix: Find them: dsr query <data> --where 'is.na(as.numeric(X)) & !is.na(X)'; clean the text, or say why NA is right.
13 WARNING [join] adsl_bili: rows went from 254 to 1814 in a join: the other side has duplicate keys (check with dsr dups)
why: The other side has duplicate keys: records were copied. fix: dsr dups <other data> --key <by columns>; then remove the duplicates or say relationship = in the join.
  • Line 8, rule coercion. Some values of LBORRES are not numbers, so as.numeric() made them NA. Exercise 20-2 finds them.
  • Line 13, rule join. adsl has one row per subject. bili has one row per lab visit. The join copied each subject once for each visit: 254 rows became 1814. R gives no warning for this. DataStep Studio for R sees that the rows grew.

In the editor, open the Suspicious Log view in the DataStep R side bar. A click on a message moves the cursor to its line. The same messages are in the Problems panel, and F8 goes from one to the next. dsr log --filter rules lists all rules. The main ones:

Rule It finds
coercion values that became NA in a conversion
join a join that multiplies rows: duplicate keys on the other side
type a column whose type changed without a conversion in the code
recycling vectors of lengths that do not fit
empty_column a column that is missing on every row
missing_values a new column with missing values (information, as in 20.1)

The goal of each run is CLEAN. A CHECK with a written reason is also fine: for example, “the 5 values <0.2 are below the limit of the test, and NA is right here”.

You do not have to run a whole program to test one change. The session keeps its objects between runs. After one full run, run only what you changed.

Key or command What runs
F4 the selection; with nothing selected, the whole file
Cmd/Ctrl+Enter the selection, or the line of the cursor
Cmd/Ctrl+Shift+Enter the whole file
Cmd/Ctrl+Alt+T (Run to Line) from the top of the file to the cursor
Run from Line (command palette) from the cursor to the end

dsr run has the same parts: --to-line and --from-line. A part is always made of whole expressions, so a pipeline is never cut in half. The first part of 20_1:

$ dsr run programs/20_1_weight.R --to-line 10
4 NOTE: adsl has 254 rows and 4 columns.
8 NOTE: weight has 253 rows and 2 columns.
Workspace: new: adsl (tbl_df 254 x 4), weight (tbl_df 253 x 2)
RESULT: CLEAN (0.12 s)

Then only the join, lines 12 to 14:

$ dsr run programs/20_1_weight.R --from-line 12 --to-line 14
12 NOTE: adsl has 254 rows and 6 columns (was 254 x 4). New: WEIGHTBL (1 missing), WGTGR (1 missing).
Workspace: changed: adsl (tbl_df 254 x 6)
RESULT: CLEAN (0 s)

The session is the R process where your programs run. DataStep Studio for R starts it for the project the first time you press F4. It keeps every object that your runs made. Three views under the DataStep R icon show what it holds:

View What it shows
Data each data frame with its rows × columns; the arrow opens its columns with type, label and missing values; a click opens the data viewer
What the Last Run Made the verdict of the last run, the data frames it created or changed with their rows and columns, the columns added or dropped, the files it wrote
Values and Functions the other objects: functions with their arguments, short vectors, lists, models

If you know RStudio, DataStep R: RStudio Layout arranges the window as its four panes. The pane at the top right then has Environment and History: the same data frames, and every command that the session ran.

In a terminal, dsr ls lists the objects and dsr history the last runs:

$ dsr ls
Data frames:
adsl 254 x 6
weight 253 x 2
Objects:
adsl tbl_df 254 x 6
weight tbl_df 253 x 2

F12 opens the last data frame that a run made in the data viewer. Chapter 21 shows the viewer in depth.

  • Stop a long run. Stop the Running Code (Cmd/Ctrl+Alt+C, or dsr stop) stops the code that runs now, as Ctrl+C does. The objects of the session stay.
  • Restart the session. dsr stop --session ends the R session. DataStep R: Start the R Session (or the play button in the Session view, or dsr start) starts a new, empty one.
$ dsr stop --session
SESSION: STOPPED
$ dsr start
SESSION: RUNNING pid 86684, <the project folder>
$ dsr status
SESSION: RUNNING pid 86684, <the project folder>, R version 4.4.1 (2024-06-14), since 2026-11-19 09:25:38
Data frames: 0 (dsr datasets)

(<the project folder> stands for the folder of the project on your computer.)

Why restart? Your program can use an object that an earlier run made, and work only because of it. In a new session such a program fails. Restart before you hand a program to QC, and run it from the top. Chapter 26 does this for many programs at once with dsr batch.

  • F4 runs the selection or the whole file. The R Console shows a log with a NOTE for each data frame a step makes, at its line.
  • The verdict is CLEAN, CHECK or ERROR. CLEAN means no error, not a right result: read the NOTEs about missing values.
  • Fix the first cause of an error, then run again.
  • The suspicious log (dsr log --filter suspicious, the Suspicious Log view) lists the problems that do not stop a run, each with why and a fix: values that became NA, joins that copy rows.
  • Run part of a file with a selection, Run to Line (Cmd/Ctrl+Alt+T) or --from-line and --to-line. The session keeps its objects between runs.
  • The Data, What the Last Run Made and Values and Functions views show the session. A new session proves that a program stands on its own.

Exercise 20-1. Run lines 1 to 10 of 20_1_weight.R only. Which data frames are in the session, and how many rows does each have?

Exercise 20-2. Which values of LBORRES became NA in program 20_2? Write a short program that lists them with their counts.

Exercise 20-3. Change program 20_2 so that it joins only the baseline bilirubin (LBBLFL == "Y"). How many rows does adsl_bili have now, and what is the verdict?

Exercise 20-4. The log of 20_1 says New: WEIGHTBL (1 missing). Which subject has no baseline weight? Use dsr query adsl --where 'is.na(WEIGHTBL)', or the data viewer with the filter . under WEIGHTBL.

The answers are in Answers, Part VI.

Found an error or have a question? Write to datastep.studio@gmail.com· Every program runs in DataStep Studio for R, the R extension for VS Code and Positron.

Data: the CDISC pilot study (cdiscpilot01), from the R packagespharmaversesdtm andpharmaverseadam (Apache License 2.0).

R is a project of the R Foundation. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. This book is not affiliated with or endorsed by SAS Institute Inc. or the R Foundation. CDISC, SDTM and ADaM are trademarks of the Clinical Data Interchange Standards Consortium.

© 2026 DataStep Labs. All rights reserved.