Civic Sample

Many people claim that trials are not diverse. There are also many on-going initiatives to increase diversity in clinical trials. There are not many publicly available tools to assess the progress of those initiatives, or get a holistic view of trial diversity. We thought this would be a great start.

We used the clinicaltrials.gov API, it is a great resource and should be more widely used. Programs like the Aggregate Analysis of ClinicalTrials.gov (AACT) Database from the clinical trials transformation initiative. The demographic variables we display here are more difficult to parse compared to some of the more standardized variables (e.g. trial phase, total participants, etc.). We designed this dashboard based on our experience parsing some of these sociodemographic characteristics (namely race and ethnicity (pre-print here), and on-going projects examining sex, gender, and geography.

No, but we are open to conversations about supporting this work. Shoot us an email [email protected]

Funding source is categorized based on sponsor information:

• Industry: Lead Sponsor is Industry
• NIH: Lead Sponsor is NIH, OR (Lead Sponsor is Other/Network AND any Collaborator is NIH)
• Other U.S. Federal: Lead Sponsor is Federal, OR (Lead Sponsor is Other/Network AND any Collaborator is Federal)
• Other: All other cases

Conditions are categorized using a standardized medical hierarchy. We group specific conditions (e.g., "Congenital Heart Disease") into broader Primary Categories (e.g., "Cardiovascular"), with more granular Secondary Categories underneath. This reduces redundancy from synonyms (e.g., "congenital heart defect" and "congenital heart disease" map to the same secondary category) and allows for both broad and granular filtering.

The classification uses a two-step process:

1. Exact/Substring Match: Each condition is checked against a curated list of keywords and synonyms, matched longest-first so specific terms (e.g., "heart failure") take priority over general ones (e.g., "heart").
2. Fuzzy Match: If no exact match is found, lightweight fuzzy string matching (via rapidfuzz) catches typos and minor variations (e.g., "Type II Diabetes" vs "Type 2 Diabetes").

Note: A study may have conditions spanning multiple categories. The dashboard filters show studies that match ANY of the keywords for the selected primary and/or secondary category. You can filter by primary category alone for broad analysis, or drill down to a specific secondary category for more targeted results.

In the "Distribution Including Unknowns" charts, the Unknown category is calculated as:

Unknown = Total Enrollment - Sum(All Known Categories)

This ensures the chart always sums to exactly 100% of total enrollment, providing a complete picture of data completeness.

This project is currently focused on demographics surrounding clinical trials. There are other tools that do a great job at searching unstructured data from clinicaltrials.gov. There is a great connector for ClaudeCode built by the company deepsense.ai. More information on that connector is here.

We take a broad approach to capturing trial involvement. In our "Sponsor" filter, we count an organization if they are listed as either the Lead Sponsor or a Collaborator in the trial record. This allows us to capture the full ecosystem of organizations supporting a trial, rather than just the primary administrative entity.

Time to report is calculated as the difference in days between when the study results were posted and when the study officially ended.

Time to Report = Results Reported Date - Study End Date

A positive number (red/orange bar) indicates the results were reported after the study ended. A negative number (blue bar) indicates a positive scenario where results were reported early, before the official study end date.

These fields indicate the level of U.S. Food and Drug Administration (FDA) oversight for a given clinical trial, as reported by the trial sponsor to ClinicalTrials.gov.

• FDA Regulated Drug: Indicates the trial is studying a drug or biological product subject to FDA regulations.
• FDA Regulated Device: Indicates the trial is studying a medical device subject to FDA regulations.
• Unapproved Device: Indicates that at least one device being studied has not been previously approved or cleared by the FDA for any use.
• Non-Regulated / Unflagged: The trial does not have these specific FDA flags set to true. This bucket contains the majority of studies, including behavioral interventions, surgical technique comparisons, international trials without a U.S. IND, and older legacy trials conducted before these fields were mandated.

Note: These fields indicate regulatory jurisdiction over the trial itself. They do not mean the FDA has ultimately approved the drug or device for public market availability.

For every trial, the pipeline queries manuscript APIs (EuropePMC, Unpaywall) using the NCT ID and receives a list of candidate papers. Rather than taking the first hit, each candidate is passed through an adjusted scoring function:

• Title-Based Penalty: Candidates whose titles contain tokens like protocol, secondary analysis, retrospective, observational, survey, post-hoc, design, or rationale receive a heavy score deduction — these are almost never the primary-results paper we want.
• Temporal Bonus: Candidates published 0–24 months after the trial's primary_completion_date (or completion_date as a fallback) receive a score boost, since primary-results manuscripts typically appear within that window.
• Selection: All candidates are sorted by adjusted score (ties broken by newer publication date) and the highest-scoring paper is chosen as the PDF to download and extract.

This replaces the naive "first result wins" heuristic and materially reduces the number of design/protocol papers that leak into the extraction set.

Because ClinicalTrials.gov does not always explicitly link to published results, our pipeline uses a 3-tier system to find manuscripts:

• Tier 1 (Explicit Match): The manuscript explicitly contains the NCT ID. However, this still requires curation! Researchers often link secondary analyses, protocols, or observational spin-offs to their primary NCT ID. While the link is explicit, the paper might not contain the primary trial demographics.
• Tier 2 (High-Confidence Fuzzy Match): The pipeline found a paper where the authors, publication year, and keywords strongly align with the trial's metadata.
• Tier 3 (Low-Confidence Fuzzy Match): A broad keyword search match. These require strict manual curation, as the search may pull in loosely related papers.

AI models can hallucinate or extract data from improperly matched papers (e.g., a Tier 1 protocol paper instead of the results paper). To ensure data integrity, our engine compares every LLM-extracted demographic value against the original ClinicalTrials.gov API baseline. It flags fields as a Match, an Addition (finding data the registry missed), or a Conflict (contradicting the registry). Curators must manually verify Additions and Conflicts.

In clinical trial demographics, missing data and "unknown" classifications are fundamentally different categories. "Not Reported" means the trial investigators completely failed to mention the demographic variable. "Explicit Unknown" means the researchers actively collected the data, but the specific demographic identity of certain participants could not be determined or was declined by the participant. Our dashboard explicitly visually separates these two states.

We are running a 3-way model comparison to evaluate cost, speed, and extraction quality using Anthropic's Claude family:

• Haiku 4.5: The fastest and most cost-effective model.
• Sonnet 4.6: The balanced baseline for speed and intelligence.
• Opus 4.7: The highest quality model for complex reasoning and navigating dense clinical methodologies.

A study is flagged as AI-related if any of its text fields contain one or more keywords associated with artificial intelligence or machine learning. The fields searched are: brief title, primary endpoint, conditions, primary condition category, and secondary condition category.

The keyword list includes:

• artificial intelligence, machine learning, deep learning
• neural network, large language model, LLM
• natural language processing, computer vision
• reinforcement learning, generative AI, chatbot
• predictive algorithm, clinical decision support algorithm
• algorithm-based, algorithm-driven, AI-based, AI-driven, AI-powered, ML-based, ML-driven

Matching is case-insensitive and uses whole-word boundary matching to avoid false positives (e.g., "algorithm-based" is matched, but "algorithm" appearing inside "logarithm" is not). This approach is intentionally broad to capture the evolving vocabulary around AI in clinical research, while still being precise enough to avoid spurious matches.