Civic Sample

Many people claim that trials are not diverse. There are also many ongoing initiatives to increase diversity in clinical trials. There are not many publicly available tools to assess the progress of those initiatives, or get a holistic view of trial diversity. We thought this would be a great start.

We use the ClinicalTrials.gov API — it is a great resource and should be more widely used, as should programs like the Aggregate Analysis of ClinicalTrials.gov (AACT) database from the Clinical Trials Transformation Initiative. The demographic variables we display here are more difficult to parse than the more standardized variables (e.g., trial phase, total participants). We designed this dashboard based on our experience parsing these sociodemographic characteristics — namely race and ethnicity (published here) — and ongoing projects examining sex, gender, and geography.

The dashboard covers studies registered on ClinicalTrials.gov that have posted results, from the first results postings in 2009 through the present. The Year Range filter is based on the date results were first posted, not on when the study was conducted. By default the dashboard shows interventional studies; the Study Type filter can switch to observational studies or all studies.

The four summary cards on the Overview tab describe the set of studies matching your current filters: the total number of studies, and the share that explicitly reported race data, ethnicity data, or both.

An automated pipeline re-extracts the full dataset from the ClinicalTrials.gov API weekly (Sundays at 6 AM UTC). The "Last updated" date in the header reflects the most recent extraction.

Each extraction is also archived as a dated snapshot. The "View snapshot" selector in the header loads the dashboard exactly as it stood on a past extraction date, which is useful for tracking how reporting evolves over time.

Snapshots follow a tiered retention schedule: the four most recent snapshots at roughly bi-weekly spacing (covering about the last two months) are kept as complete datasets, and beyond that one snapshot per calendar month is kept as an aggregate archive — every chart renders exactly as extracted on that date, while the interactive filters and the full study-level table require a complete (bi-weekly or latest) dataset. The live dashboard always reflects the newest weekly extraction regardless of retention.

The pruning runs automatically alongside the weekly extraction, and the "View snapshot" selector lists exactly the retained dates. We keep the log tiered rather than unlimited because each snapshot is roughly 140 MB and the dashboard is served as a static site: an unbounded archive makes deployments slow and unreliable long before it adds analytical value. If you need a specific historical extraction that has aged out of the log, email us — full weekly extractions are archived off-site.

We received an API credit grant through Anthropic PBC in May 2026. So far the elements of this initiative that the API credits will enhance have not been implemented, but we expect to begin to roll them out in late July 2026. We are open to conversations about supporting this work in other ways alongside suggestions you have for our approach. Shoot us an email [email protected]

Funding source is categorized based on sponsor information:

• Industry: Lead Sponsor is Industry
• NIH: Lead Sponsor is NIH, OR (Lead Sponsor is Other/Network AND any Collaborator is NIH)
• Other U.S. Federal: Lead Sponsor is Federal, OR (Lead Sponsor is Other/Network AND any Collaborator is Federal)
• Other: All other cases

Conditions are categorized using a standardized medical hierarchy. We group specific conditions (e.g., "Congenital Heart Disease") into broader Primary Categories (e.g., "Cardiovascular"), with more granular Secondary Categories underneath. This reduces redundancy from synonyms (e.g., "congenital heart defect" and "congenital heart disease" map to the same secondary category) and allows for both broad and granular filtering.

The classification uses a two-step process:

1. Exact/Substring Match: Each condition is checked against a curated list of keywords and synonyms, matched longest-first so specific terms (e.g., "heart failure") take priority over general ones (e.g., "heart").
2. Fuzzy Match: If no exact match is found, lightweight fuzzy string matching (via rapidfuzz) catches typos and minor variations (e.g., "Type II Diabetes" vs "Type 2 Diabetes").

Note: A study may have conditions spanning multiple categories. The dashboard filters show studies that match ANY of the keywords for the selected primary and/or secondary category. You can filter by primary category alone for broad analysis, or drill down to a specific secondary category for more targeted results.

In the "Full Distribution with Data Quality" charts, the light "Not Reported (Missing)" layer is calculated as:

Not Reported = Total Enrollment - Sum(All Reported Categories)

"Explicitly Unknown" is different: it is a category sponsors actively reported (see "What is the difference between 'Not Reported' and 'Explicit Unknown'?" below). Together the layers sum to 100% of enrollment, providing a complete picture of data completeness. In rare years where reported counts exceed registered enrollment — a registry data-quality quirk — the larger of the two is used as the denominator so percentages never exceed 100%.

Geography comes from each trial's registered study sites (the facility city, state, and country listed on ClinicalTrials.gov). Trials are classified as single-site or multi-site, and as "Location Not Reported" when no sites are listed. The U.S. view aggregates sites by state and by U.S. Census region.

Note that a multi-site trial is counted in every state or country where it has a site, so map and regional totals can exceed the total number of trials.

This project is currently focused on demographics surrounding clinical trials. There are other tools that do a great job at searching unstructured data from ClinicalTrials.gov. There is a great connector for Claude Code built by the company deepsense.ai. More information on that connector is here.

Sponsor information appears in three places on the dashboard, and each works slightly differently:

• The Sponsor filter (filter bar) matches the lead sponsor's class as registered on ClinicalTrials.gov: NIH, Industry, Federal (non-NIH), Other, or Network.
• Funding Source (study detail view) is derived from the lead sponsor and collaborators together — see "How is Funding Source derived?" above.
• The sponsor dropdown on the Geography tab takes a broad approach: an organization is counted if it is listed as either the Lead Sponsor or a Collaborator, to capture the full ecosystem of organizations supporting a trial rather than just the primary administrative entity.

We standardize to the NIH/OMB reporting categories that sponsors use when posting results to ClinicalTrials.gov. Race: American Indian/Alaska Native, Asian, Black/African American, Native Hawaiian/Pacific Islander, White, More than one race, Other, and Unknown/Not Reported. Ethnicity: Hispanic/Latino, Not Hispanic/Latino, and Unknown/Not Reported.

Sponsors often report more granular labels (e.g., specific Asian subgroups); where available these are preserved and shown in the subcategory charts on the Race and Ethnicity tabs. All values reflect what sponsors reported to the registry — we do not infer or impute demographic categories.

They are extracted separately and never cross-mapped. Sex refers to biological sex as reported in a trial's baseline characteristics, standardized to Female, Male, and Unknown or Not Reported. Gender refers to gender identity, standardized to Woman, Man, Non-binary, Transgender, Other, and Unknown or Not Reported — and is only parsed from baseline tables explicitly labeled as gender.

Because the pipeline keeps these strictly decoupled (a "Female" label is never counted as "Woman", and vice versa), the Gender tab reflects only trials that genuinely collected gender identity — which remains rare in the registry.

Time to report is calculated as the difference in days between when the study results were first posted and the study's primary completion date (the date the final participant was examined for the primary outcome). If a primary completion date is not available, the overall study completion date is used instead.

Time to Report = Results First Posted Date - Primary Completion Date

A positive number (red/orange bar) indicates the results were posted after the study's primary completion. A negative number (blue bar) indicates results were posted early, before the primary completion date.

These fields indicate the level of U.S. Food and Drug Administration (FDA) oversight for a given clinical trial, as reported by the trial sponsor to ClinicalTrials.gov.

• FDA Regulated Drug: Indicates the trial is studying a drug or biological product subject to FDA regulations.
• FDA Regulated Device: Indicates the trial is studying a medical device subject to FDA regulations.
• Unapproved Device: Indicates that at least one device being studied has not been previously approved or cleared by the FDA for any use.
• Non-Regulated / Unflagged: The trial does not have these specific FDA flags set to true. This bucket contains the majority of studies, including behavioral interventions, surgical technique comparisons, international trials without a U.S. IND, and older legacy trials conducted before these fields were mandated.

Note: These fields indicate regulatory jurisdiction over the trial itself. They do not mean the FDA has ultimately approved the drug or device for public market availability.

No. Everything shown here is built from aggregate counts that sponsors post publicly to ClinicalTrials.gov — for example, the number of participants in each race, ethnicity, sex, or gender category from a trial's baseline characteristics table. The dashboard contains no individual-level or identifiable participant data, and we collect none ourselves.

The full per-study dataset is roughly 136 MB compressed — too heavy for phones. Mobile browsers instead load a small pre-computed summary: the charts show aggregates over the whole dataset (the filters don't apply), the Studies table is limited to the 500 most recently posted studies, and the filter bar and Geography tab (which need the full per-study data) are desktop-only.

For every trial, the pipeline queries manuscript APIs (EuropePMC, Unpaywall) using the NCT ID and receives a list of candidate papers. Rather than taking the first hit, each candidate is passed through an adjusted scoring function:

• Title-Based Penalty: Candidates whose titles contain tokens like protocol, secondary analysis, retrospective, observational, survey, post-hoc, design, or rationale receive a heavy score deduction — these are almost never the primary-results paper we want.
• Temporal Bonus: Candidates published 0–24 months after the trial's primary_completion_date (or completion_date as a fallback) receive a score boost, since primary-results manuscripts typically appear within that window.
• Selection: All candidates are sorted by adjusted score (ties broken by newer publication date) and the highest-scoring paper is chosen as the PDF to download and extract.

This replaces the naive "first result wins" heuristic and materially reduces the number of design/protocol papers that leak into the extraction set.

Because ClinicalTrials.gov does not always explicitly link to published results, our pipeline uses a 3-tier system to find manuscripts:

• Tier 1 (Explicit Match): The manuscript explicitly contains the NCT ID. However, this still requires curation! Researchers often link secondary analyses, protocols, or observational spin-offs to their primary NCT ID. While the link is explicit, the paper might not contain the primary trial demographics.
• Tier 2 (High-Confidence Fuzzy Match): The pipeline found a paper where the authors, publication year, and keywords strongly align with the trial's metadata.
• Tier 3 (Low-Confidence Fuzzy Match): A broad keyword search match. These require strict manual curation, as the search may pull in loosely related papers.

AI models can hallucinate or extract data from improperly matched papers (e.g., a Tier 1 protocol paper instead of the results paper). To ensure data integrity, our engine compares every LLM-extracted demographic value against the original ClinicalTrials.gov API baseline. It flags fields as a Match, an Addition (finding data the registry missed), or a Conflict (contradicting the registry). Curators must manually verify Additions and Conflicts.

In clinical trial demographics, missing data and "unknown" classifications are fundamentally different categories. "Not Reported" means the trial investigators completely failed to mention the demographic variable. "Explicit Unknown" means the researchers actively collected the data, but the specific demographic identity of certain participants could not be determined or was declined by the participant. Our dashboard explicitly visually separates these two states.

We are running a 3-way model comparison to evaluate cost, speed, and extraction quality using Anthropic's Claude family:

• Haiku 4.5: The fastest and most cost-effective model.
• Sonnet 4.6: The balanced baseline for speed and intelligence.
• Opus 4.8: The highest quality model for complex reasoning and navigating dense clinical methodologies.

A study is flagged as AI-related if any of its text fields contain one or more keywords associated with artificial intelligence or machine learning. The fields searched are: brief title, primary endpoint, conditions, primary condition category, and secondary condition category.

The keyword list includes:

• artificial intelligence, machine learning, deep learning
• neural network, large language model, LLM
• natural language processing, computer vision
• reinforcement learning, generative AI, chatbot
• predictive algorithm, clinical decision support algorithm
• algorithm-based, algorithm-driven, AI-based, AI-driven, AI-powered, ML-based, ML-driven

Matching is case-insensitive and uses whole-word boundary matching to avoid false positives (e.g., "algorithm-based" is matched, but "algorithm" appearing inside "logarithm" is not). This approach is intentionally broad to capture the evolving vocabulary around AI in clinical research, while still being precise enough to avoid spurious matches.

The Food and Drug Administration Amendments Act of 2007 (FDAAA) is the law behind most of the data on this dashboard. Its Section 801 created the legal requirement that "applicable clinical trials" of drugs, biologics, and devices be registered on ClinicalTrials.gov and — for the first time — that sponsors submit basic results, including participant demographics, generally within 12 months of the trial's primary completion date.

For its first decade the statute left key terms ambiguous and results reporting lagged. The Final Rule (42 CFR Part 11), published September 2016 and effective January 18, 2017, resolved that: it spelled out exactly which trials are covered (including trials of unapproved products), what must be submitted (baseline demographics such as age, sex, and — when collected — race and ethnicity; outcomes; adverse events), the deadlines, and the penalties for non-compliance (civil monetary fines and, for federally funded trials, grant suspension). A companion NIH policy applied the same expectations to all NIH-funded trials on the same date. ClinicalTrials.gov maintains a summary at FDAAA 801 and the Final Rule.

This is why several time-series charts on this dashboard carry a dashed marker at 2017: reporting behavior before and after that line reflects two different enforcement regimes, and comparisons that span it should keep the rule change in mind.