Develop like a senior
A senior programmer does not know more keys than you. A senior has a routine: look at the data before you write code, count a number in a second way, prove what a change did, and look inside the run instead of guessing. Each workflow below uses the tools of chapters 20 to 25 together.
| Workflow | The question | Tools |
|---|---|---|
| 26.1 | What is in this domain? | describe, freq, values, profile, the data viewer |
| 26.2 | Why is my count too high? | query --count, dups, freq --subjects |
| 26.3 | Did my fix change only what I meant? | snapshot, compare --expect, verify |
| 26.4 | Does my function do what it says? | dsr test, coverage |
| 26.5 | Does every program stand on its own? | lint, batch, verify |
| 26.6 | Can an agent help, and can I see what it does? | the Agent chat, MCP, dsr next |
dsr snapshot, dsr compare --expect, dsr verify and coverage are in the Professional edition, free during Early Access until 17 February 2027. The data tools, dsr batch and dsr test are in Community.
26.1 Explore a new SDTM domain in five minutes
Section titled “26.1 Explore a new SDTM domain in five minutes”Your new task is about concomitant medications, and you have never seen CM. Spend five minutes with the data before you write the program. Run this to put CM into the session:
# Chapter 26: put SDTM CM into the session, to explore it with the data toolslibrary(dplyr)
cm <- pharmaversesdtm::cm
cm |> summarise(records = n(), subjects = n_distinct(USUBJID))#> # A tibble: 1 x 2#> records subjects#> <int> <int>#> 1 7510 229-
Read the columns. In the Data view, open the arrow of
cm. Or:$ dsr describe cmcm: 7510 rows x 22 columns "Concomitant Medications" (tbl_df)1 STUDYID character 12 Study Identifier2 DOMAIN character 2 Domain Abbreviation3 USUBJID character 11 Unique Subject Identifier4 CMSEQ numeric Sequence Number5 CMSPID character 2 Sponsor-Defined Identifier6 CMTRT character 44 Reported Name of Drug, Med, or Therapy7 CMDECOD character 24 Standardized Medication Name...18 CMSTDTC character 10 Start Date/Time of Medication19 CMENDTC character 10 End Date/Time of Medication... -
Count the values of the coded name. Click
cm, then a cell ofCMDECOD, and pressf. Or:$ dsr freq cm CMDECOD --n 5cm: 34 levels of CMDECOD, 7510 rowsCMDECOD n pctUNCODED 6085 81ACETYLSALICYLIC ACID 380 5.1ESTROGENS CONJUGATED 195 2.6CALCIUM 180 2.4AMLODIPINE 112 1.5... 29 more (--n)81 percent of the records are
UNCODED. A table byCMDECODis not possible as the data is. -
Look at the reported names. In the value picker of
CMTRT, typemultivitin Find a value. Or:$ dsr values cm CMTRT --q multivitcm: CMTRT (Reported Name of Drug, Med, or Therapy), character: 310 values, 7510 rowsvalue r n pctMULTIVITAMIN "MULTIVITAMIN" 470 6.3MULTIVITAMINS "MULTIVITAMINS" 253 3.4MULTIVIT "MULTIVIT" 13 0.2MULTIVIT B "MULTIVIT B" 5 0.1One medication, written four ways.
-
Profile the columns you need.
$ dsr profile cm --vars 'CMDECOD CMTRT CMSTDTC CMENDTC'cm: 7510 rows x 22 columns; duplicate rows: 0; unique key: USUBJID CMSEQCMDECOD character missing 0 distinct 34 UNCODED (6085) | ACETYLSALICYLIC ACID (380) | ESTROGENS CONJUGATED (195) | CALCIUM (180) | AMLODIPINE (112) | HYDROCORTISONE (105) | NIZATIDINE (53) | FUROSEMIDE (48)CMTRT character missing 0 distinct 310 MULTIVITAMIN (470) | VITAMIN E (448) | ASPIRIN (380) | MULTIVITAMINS (253) | VITAMIN C (235) | HYDROCORTISONE, TOPICAL (222) | PREMARIN (195) | CALCIUM (180)CMSTDTC character missing 21 distinct 440 2010 (368) | 2011 (366) | 2012 (352) | 2009 (333) | 2006 (301) | 2007 (283) | 2001 (173) | 2002 (168)CMENDTC character missing 6812 distinct 235 NA (6812) | 2013-06-05 (36) | 2012-12-13 (19) | 2013-04-17 (14) | 2013-11-17 (13) | 2012-12-15 (12) | 2012-11-19 (11) | 2013-11-14 (11)The key is
USUBJID CMSEQ. 21 start dates are missing, and the most frequent start dates are a year only. Most end dates are missing: the medication goes on. -
Count subjects, not records.
$ dsr freq cm CMDECOD --subjects --n 2cm: 34 levels of CMDECOD, 229 subjectsCMDECOD n pctUNCODED 221 96.5ACETYLSALICYLIC ACID 38 16.6... 32 more (--n)$ dsr query cm --where 'nchar(CMSTDTC) < 10' --countcm: 5454 of 7510 rows where nchar(CMSTDTC) < 10COUNT: 5454
Write down what you found, before you write the program:
CM: 7510 records, 229 subjects, key USUBJID CMSEQ- 6085 records (221 subjects) are UNCODED: a table by CMDECOD is not possible as it is- CMTRT spellings differ: MULTIVITAMIN / MULTIVITAMINS / MULTIVIT / MULTIVIT B- 5454 start dates are partial and 21 are missing: decide the imputation before you derive ASTDTThese are questions for the data manager and the statistician. You found them before anyone built a table on them.
26.2 Why is my count 25 and not 16?
Section titled “26.2 Why is my count 25 and not 16?”The statistician checks your table of severe treatment-emergent adverse events and writes: “Low dose: the number of subjects should be 16, not 25.” Here is the program:
# Chapter 26: subjects with a severe treatment-emergent adverse event per arm, a first draftlibrary(dplyr)
adae <- pharmaverseadam::adae
adae |> filter(TRTEMFL == "Y", AESEV == "SEVERE") |> group_by(TRT01A) |> summarise(subjects = n())#> # A tibble: 3 x 2#> TRT01A subjects#> <chr> <int>#> 1 Placebo 6#> 2 Xanomeline High Dose 10#> 3 Xanomeline Low Dose 25-
Run it and read the log. The verdict is CLEAN. A wrong number does not always leave a message.
-
Count the same thing a second way.
$ dsr query adae --where 'TRTEMFL == "Y" & AESEV == "SEVERE" & TRT01A == "Xanomeline Low Dose"' --countadae: 25 of 1191 rows where TRTEMFL == "Y" & AESEV == "SEVERE" & TRT01A == "Xanomeline Low Dose"COUNT: 2525 rows. The program counted rows.
-
Find the subjects behind the rows.
$ dsr dups adae --key USUBJID --where 'TRTEMFL == "Y" & AESEV == "SEVERE" & TRT01A == "Xanomeline Low Dose"' --n 3adae: 25 rows; key USUBJID: 7 duplicate keys in 16 rows01-708-1019: 3 rows, differ in AESEQ, AETERM, AEDECOD, AELLT, AEHLT, AEHLGT, AESTDTC, ASTDT, ASTDTM, AEENDTC, AENDT, AENDTM, ASTDY, AESTDY, AENDY, AEENDY, AOCCIFL, AESPID, LDOSEDTM01-708-1428: 3 rows, differ in AESEQ, AETERM, AEDECOD, AELLT, AEHLT, AEHLGT, AOCCIFL, AESPID01-703-1119: 2 rows, differ in AESEQ, AETERM, AEDECOD, AEBODSYS, AELLT, AEHLT, AEHLGT, AESOC, AESTDTC, ASTDT, ASTDTM, ASTDY, AESTDY, AOCCIFL, AESPID, AEDTC, LDOSEDTMDUPS: DUPLICATES7 subjects have more than one record; together they have 16 records. The other 9 records belong to 9 subjects with one record each. 7 + 9 = 16 subjects, 16 + 9 = 25 records.
-
Fix and check. Change
n()ton_distinct(USUBJID).dsr freq adae TRT01A --where 'TRTEMFL == "Y" & AESEV == "SEVERE"' --subjectsgives the counts to compare with, without a program: 16 subjects on the low dose (chapter 21).
The fixed count is the program of chapter 21: 5, 8 and 16 subjects.
26.3 Prove that a fix changed only what you meant
Section titled “26.3 Prove that a fix changed only what you meant”A fix should change what you meant to change, and nothing else. Take the age group bug of chapter 25.
-
Keep a copy before the fix. Run
25_2_adsl_agegr.R, then:$ dsr snapshot adsl --name beforeadsl 254 x 4SNAPSHOT: SAVED before (1 data set: dsr compare <data> @snapshot:before)The snapshot is a copy outside the session’s objects. A new run of the program does not change it.
-
Fix and run. Change line 5 of
25_1_age_group.Rtoage >= 65and run25_2_adsl_agegr.Ragain. The log says that the function file changed and is sourced again:$ dsr run programs/25_2_adsl_agegr.RNOTE: programs/25_1_age_group.R changed since the session sourced it; this program sources it again (line 3), so the run uses the new code.5 NOTE: adsl has 254 rows and 3 columns (was 254 x 4). Dropped: AGEGR.8 NOTE: adsl has 254 rows and 4 columns (was 254 x 3). New: AGEGR....RESULT: CLEAN (0.01 s) -
Compare with the snapshot.
--expectnames the columns you meant to change:$ dsr compare adsl @snapshot:before --expect AGEGRBase adsl: 254 x 4; compare @snapshot:before:adsl: 254 x 4; 254 rows compared by USUBJIDAGEGR: 4 values differ01-707-1206: 65-80 | NA01-709-1309: 65-80 | NA01-715-1107: 65-80 | NA01-715-1319: 65-80 | NAONLY THESE CHANGED: AGEGR (4 values, 4 subjects)NOTHING ELSE CHANGEDSNAPSHOT: AS EXPECTED -
One verdict.
dsr verifylints the program, runs it, reads the suspicious log, runs it again in a fresh R session, and compares with the snapshot:$ dsr verify programs/25_2_adsl_agegr.R --snapshot before --expect AGEGRVERIFY PASS (programs/25_2_adsl_agegr.R: nothing to fix)STEPSlint ok, 1 file (with lintr)inputs no input made by another programrun CLEAN (0.01 s)log clean: no suspicious messagefresh run CLEAN (log: programs/25_2_adsl_agegr.log)data adsl (the last data frame the run made)spec skipped: no specs/adam/adsl.yml (a spec makes this step run)conformance skipped: no specification for this data setreference skipped: no --reference FILEsnapshot AS EXPECTED: ONLY THESE CHANGED: AGEGR (4 values, 4 subjects) | NOTHING ELSE CHANGEDDONE: VERIFY PASS: the task is done when the goal is a fact in the data (look: dsr data <data>)
If anything else had changed, the verdict would be UNEXPECTED CHANGES, with examples by subject. Put the bug back in 25_1_age_group.R after this workflow: it is the example of chapter 25.
26.4 Test a function with dsr test
Section titled “26.4 Test a function with dsr test”People call a function hundreds of times and never read its code. So write what it must do as tests: small programs that call the function and say what it must return. Write them before the fix, and keep them. This is age_group() with the fix of chapter 25 and a check of its input:
# Chapter 26: age_group() with the fix of chapter 25 and a check of its inputage_group <- function(age) { if (!is.numeric(age)) { stop("age must be a number, not ", class(age)[1]) } grp <- rep(NA_character_, length(age)) grp[age < 65] <- "<65" grp[age >= 65 & age <= 80] <- "65-80" grp[age > 80] <- ">80" grp}The tests use the package testthat, in a folder of their own. A test file must be named test-....R:
# Chapter 26: the tests of age_group(), written before the fixsource("../26_3_age_group.R")
test_that("every age gets a group, also the edges 65 and 80", { expect_equal( age_group(c(64, 65, 66, 80, 81)), c("<65", "65-80", "65-80", "65-80", ">80") )})
test_that("a missing age gets no group", { expect_equal(age_group(c(70, NA)), c("65-80", NA))})-
Red first. With the bug (
age > 65on line 8),dsr test(or DataStep R: Run the Tests) fails at the line of the test, with the value that differs:$ dsr test programs/26_testsFAIL programs/26_tests/test-age_group.R:5: every age gets a group, also the edges 65 and 80Expected `age_group(c(64, 65, 66, 80, 81))` to equal `c("<65", "65-80", "65-80", "65-80", ">80")`.Differences:1/5 mismatchesx[2]: NAy[2]: "65-80"TESTS FAIL (2 tests, 2 expectations in 1 files, 1 failed, 1.3 s) | FIRST CAUSE: programs/26_tests/test-age_group.R:5: every age gets a group, also the edges 65 and 80: Expected `age_group(c(64, 65, 66, 80, 81))` to equal `c("<65", "65-80", "65-80", "65-80", ">80")`.Good: the tests can fail. A test that cannot fail tests nothing.
-
Green. With
age >= 65:$ dsr test programs/26_testsTESTS PASS (2 tests, 2 expectations in 1 files, 0 failed, 1.2 s)The tests run in a clean R process, not in your session. So they pass only when they stand on their own.
-
Ask the coverage what you did not test. Coverage counts how often each line of the project’s functions ran:
$ dsr test programs/26_tests --coverageCOVERAGE: programs/26_3_age_group.R: 6 of 7 lines ran (never: 4)COVERAGE: programs/25_1_age_group.R: 0 of 5 lines ran (never: 3, 4, 5, 6, 7)TESTS PASS (2 tests, 2 expectations in 1 files, 0 failed, 1.2 s)$ dsr coverage programs/26_3_age_group.R1 # Chapter 26: age_group() with the fix of chapter 25 and a check of its input2 age_group <- function(age) {2x 3 if (!is.numeric(age)) {##### 4 stop("age must be a number, not ", class(age)[1])5 }2x 6 grp <- rep(NA_character_, length(age))2x 7 grp[age < 65] <- "<65"2x 8 grp[age >= 65 & age <= 80] <- "65-80"2x 9 grp[age > 80] <- ">80"2x 10 grp11 }COVERAGE: programs/26_3_age_group.R: 6 of 7 lines ran (never: 4)Line 4 never ran: no test gives the function a text. A line that never ran is a line that no test checked. (
25_1_age_group.Ris the otherage_group()of the project; the tests do not use it.) Exercise 26-3 adds the missing test.
In the editor, Run R File with Coverage marks each line with its count (2x) and the lines that never ran; Clear the Coverage Marks removes the marks.
26.5 A gate before delivery
Section titled “26.5 A gate before delivery”In your session, a program can use an object that an earlier run made. In the night run of a study, each program starts in a new session. Before you deliver, prove that each program runs on its own:
$ dsr batch programs/20_1_weight.R programs/21_1_severe.R programs/23_1_lost.R programs/23_2_copies.R programs/25_2_adsl_agegr.R CLEAN programs/20_1_weight.R log: programs/20_1_weight.log (1.4 s) CLEAN programs/21_1_severe.R log: programs/21_1_severe.log (1.4 s) CLEAN programs/23_1_lost.R log: programs/23_1_lost.log (1.5 s) CHECK programs/23_2_copies.R log: programs/23_2_copies.log (1.5 s) FIRST CAUSE: programs/23_2_copies.R:8: adsl_ds: rows went from 254 to 798 in a join: the other side has duplicate keys (check with dsr dups) CLEAN programs/25_2_adsl_agegr.R log: programs/25_2_adsl_agegr.log (1.4 s)BATCH: CHECKEach program ran in a new R session, and its log is next to it, with the code, the messages and the NOTEs. A program that needs an object of your session fails here, not in the night run. 23_2_copies.R has its CHECK of chapter 23.
A gate for one program is dsr verify: lint, a run, the suspicious log and a fresh run, in one verdict (26.3). The routine before you hand a program to QC:
dsr linton the program: no typo, no missing function.dsr format --check: the house layout.dsr verify: VERIFY PASS, or a CHECK with a written reason.dsr batchon all the programs of the delivery: each one stands on its own.dsr test: the tests of your functions pass.
For a folder of programs that make data for each other (ADSL, then ADAE, then the tables), dsr batch programs/ runs them in the order of their dependencies. The manual of DataStep Studio for R explains it in its chapter on programs that depend on each other.
26.6 Work with an AI agent in the same session
Section titled “26.6 Work with an AI agent in the same session”An AI agent, for example Claude Code, Codex, opencode or GitHub Copilot, can use the same tools as you. It runs R only through dsr, so it reads the same logs, verdicts and suspicious messages. Its runs appear in your R Console, and its stops wait in your editor, where you can look.
Inside the editor. The Agent chat is a tab on the right; Ctrl+Alt+A (Mac: Cmd+Alt+A) shows and hides it. Choose your agent in the composer. Each tool call is a card with its command and its verdict, and a change of a file is a diff.
Outside the editor. One command sets up a project for your agent. --dry-run shows what it would write, and writes nothing:
$ dsr agent setup --dry-runAGENT SETUP for <the project folder> (dry run: nothing written) YOUR DATA You decide what the agent sees. With Claude Code, your questions and what the tools return, which can include study data, go to Anthropic under your own account. You are responsible for checking that you may send this data. For patient-level or confidential data, use a local model or the aggregate data mode. The aggregate data mode: dsr agent permissions data aggregate WOULD WRITE .mcp.json: MCP server dsr (added) WOULD WRITE .claude/settings.json: permissions: 159 allow rules added (0 already there), 33 ask rules added, 2 deny rules added...AGENT: DRY RUN (2 files)Then dsr agent doctor proves the connection. Every tool answer ends with a NEXT: line: the one next command. dsr next says the same from the state of the session. After the failed run of 22_2_typo.R:
$ dsr nextWHY: the run failed: programs/22_2_typo.R:8 (called at line 8): ℹ In argument: `subjects = n_distnct(USUBJID)`.NEXT: dsr log --filter errors$ dsr guide fix-runPLAYBOOK fix-run: A run failed (RESULT: ERROR or CHECK) 1. dsr log --filter errors # the first error with its line (fix only the FIRST one) 2. dsr lint <program> # typos, missing functions and arguments, with the line 3. dsr pry ls # the values where it failed (the error keeps its frames open) 4. dsr describe <data> --find <word> # the real column names when a name is not found 5. edit the file at the line the verdict names # the smallest change; one change at a time 6. dsr verify <program> # one verdict: VERIFY PASS means doneAfter each step read the NEXT: line; dsr next says the same from the state.That playbook is also a good routine for a person.
The session is shared. When an agent’s run waits at a stop, the status bar says so, and Debug the Run That Waits at Its Stop opens it in your debugger. dsr stop cancels a long run; the objects stay.
26.7 Summary
Section titled “26.7 Summary”- Explore first.
describe,freq,values,profileandfreq --subjectsshow a new domain in five minutes. Write down the questions before you write the program. - Count twice. For every count, ask what one row is.
query --count,dupsandfreq --subjectscheck a number a second way. - Prove a fix.
snapshotbefore,compare --expectafter: only the columns you meant changed.verifygives one verdict. - Test functions.
dsr testruns testthat tests in a clean process;--coverageshows the lines that no test reached. - Gate before delivery.
lint,format --check,verify,batchandtest. - An agent uses the same tools in the same session; you see its runs and its stops. You decide what data it may see.
26.8 Exercises
Section titled “26.8 Exercises”Exercise 26-1. In CM, how many records have a start date with a year only? Use dsr query with nchar(CMSTDTC) == 4, or the column filter ???? under CMSTDTC in the data viewer.
Exercise 26-2. In workflow 26.2, how many records and how many subjects are behind the High Dose count?
Exercise 26-3. Add a test to 26_tests/test-age_group.R that a text age stops with the error age must be a number. What does the coverage say now?
Exercise 26-4. Run dsr verify programs/23_2_copies.R. What is the verdict, and what does the finding tell you to do?
The answers are in Answers, Part VI.
Found an error or have a question? Write to datastep.studio@gmail.com· Every program runs in DataStep Studio for R, the R extension for VS Code and Positron.
Data: the CDISC pilot study (cdiscpilot01), from the R packagespharmaversesdtm andpharmaverseadam (Apache License 2.0).
R is a project of the R Foundation. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. This book is not affiliated with or endorsed by SAS Institute Inc. or the R Foundation. CDISC, SDTM and ADaM are trademarks of the Clinical Data Interchange Standards Consortium.
© 2026 DataStep Labs. All rights reserved.