Synergistic barriers to algorithmic recourse in healthcare and administrative systems
1Nyx Dynamics LLC, Philadelphia, Pennsylvania, United States of America
*Corresponding author; email: dr.a.c.demidont@gmail.comAbstract
Algorithmic decision systems mediate access to healthcare, credit, employment and housing, yet individuals who experience adverse decisions face multi-stage barriers when seeking recourse. We formalize these barriers as a series-structured system with 11 empirically parameterized stages across three layers (data integration, data accuracy and institutional access) and prove that single-barrier interventions are bounded by baseline system success. Under baseline parameterization derived from federal datasets and peer-reviewed algorithmic audit studies, end-to-end recourse probability is 0.0018%. Removing any single barrier yields negligible improvement (<0.02%). Factorial decomposition reveals that the three-way cross-layer interaction accounts for 87.6% of achievable improvement, confirmed by Shapley attribution, Sobol sensitivity analysis and bootstrap resampling (n = 1,000). These results provide a structural explanation for the limited impact of incremental reforms and support coordinated multi-layer intervention approaches for clinical AI governance and algorithmic fairness.
Article notes
Competing Interest Statement
A.C.D. was previously employed by Gilead Sciences, Inc. (January 2020 -- October 2024) and held company stock (divested December 2024). Gilead Sciences, Inc., had NO role in the design of this study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results. A.C.D. is founder and CEO of Nyx Dynamics, LLC, a healthcare consulting firm specializing in clinical algorithm development and implementation science. The decision support tool presented in this manuscript was developed independently and is released open-source under the MIT License.
Funding Statement
This study did not receive any funding
1Introduction
Healthcare algorithms increasingly determine which patients receive timely interventions, with documented instances where cost-based proxy variables reduced identification of high-risk Black patients by approximately 62%[1]. In clinical settings, algorithmic decisions include risk stratification scores, automated triage, prior authorization determinations, resource allocation and eligibility assessments; when these produce adverse outcomes, patients must navigate error detection, data correction and institutional appeals processes that we term clinical AI recourse. Analogous barrier structures arise in credit, employment and housing: approximately 75% of US employers use recruitment management systems to filter candidates before human review[2], and credit scoring algorithms shape financial access for virtually all Americans[3]. When these systems encode discrimination, individuals face multi-stage procedural frictions when seeking recourse. We focus on healthcare AI as the primary context and use credit, employment and housing domains as supporting analogs where empirical barrier data are available[4, 5] that span institutional boundaries regulated by separate agencies, creating regulatory fragmentation[6].
The sociotechnical nature of algorithmic systems means discrimination arises not from any single component but from interactions among design choices, data practices, institutional contexts and deployment environments[7]. This parallels well-characterized phenomena in reliability engineering, where series systems exhibit failure dynamics dominated by component interactions rather than individual failure rates[8, 9], and in patient safety, where alignment of multiple barrier failures produces system-level adverse events[10]. Despite sustained regulatory effort targeting individual barriers—improving data accuracy, providing legal resources, mandating algorithmic audits—discrimination persists, suggesting current approaches may misunderstand the problem’s structure.
We hypothesized that barriers to algorithmic recourse function synergistically such that incremental single-layer interventions produce near-zero absolute gains unless coordinated across layers. Specifically, we ask: in a multi-stage barrier system governing recourse from adverse algorithmic decisions, when do single-barrier reforms have negligible absolute impact, and how much of achievable improvement is attributable to cross-layer interactions? We formalize this as a series-structured barrier system, prove closed-form bounds on single-barrier intervention effects and quantify the interaction structure using factorial decomposition, Shapley attribution and global sensitivity analysis. This framework provides a quantitative basis for coordinated intervention design in clinical AI governance and beyond.
2Results
2.1Baseline system characteristics
The 11-barrier model (Table 2) yielded a baseline success probability of 0.0018%, indicating that fewer than 2 in 100,000 individuals successfully navigate all barriers to resolve adverse algorithmic decisions.
2.2Analytical bounds on single-barrier interventions
Proposition 1 establishes that removing barrier j yields absolute improvement Δj = P · (1/pj − 1), which is linear in baseline success P. Counterfactual analysis confirmed this prediction: all individual barrier removal effects were <0.02%, with the maximum observed for Legal Knowledge Gap (Δ = 0.0055%). This value is consistent with the analytical bound P· (1/0.25 −1) ≈ 0.0054% for the barrier with lowest pass probability (Fig. 1).
2.3Strategy equivalence
Comparison of barrier removal strategies—forward, backward, greedy by marginal impact and random ordering—revealed characteristic threshold trajectories with near-zero improvement until removal of final barriers (Fig. 2). Strategy equivalence (ANOVA: F = 0.07, P = 0.98) confirms that removal ordering is irrelevant; only completeness determines outcome.
2.4Cross-layer interaction dominance
Factorial analysis revealed interaction-dominant dynamics as predicted by Proposition 2 (Fig. 3, Table 1). The three-way interaction accounted for 87.6% of total effect variance. Main effects contributed minimal variance: Layer 1 (0.0%), Layer 2 (0.0%), Layer 3 (0.3%).
2.5Sensitivity analysis and robustness
Sobol indices confirmed interaction dominance, with total-order indices substantially exceeding first-order indices for all barriers (Fig. 4). Morris screening identified Legal Knowledge Gap and Systemic Bias in Algorithms as having highest nonlinear/interaction involvement. Signal-to-noise ratio remained positive (>0 dB) up to 25% parameter noise (Fig. 5). Bootstrap validation (n = 1,000) demonstrated 100% robustness of all principal findings across plausible parameterizations.
2.6Shapley value attribution
Shapley decomposition assigned fair responsibility accounting for all possible removal orderings. Top contributors: Legal Knowledge Gap (11.3%), Rapid Data Transmission (11.1%), Correction Process Barriers (10.7%), Systemic Bias in Algorithms (10.5%) and Awareness Gap (10.0%). Unlike marginal effects—which approach zero under multiplicative blocking (Proposition 1)—Shapley values reveal causal contribution under interaction-dominant dynamics.
3Discussion
Under a series-structured barrier model with empirically derived parameters, algorithmic discrimination operates as a synergistic system where barriers reinforce each other, rendering incremental reform structurally insufficient. The dominant three-way interaction (87.6%) provides a quantitative explanation for the observed pattern whereby incremental policy reforms—including targeted improvements to data accuracy, legal access and algorithmic auditing—have had limited impact: under synergistic barrier conditions, targeting any subset of barriers leaves the multiplicative blocking structure substantially intact.
The closed-form bound established in Proposition 1 makes precise what has been intuitive in advocacy contexts: when system-level success probability is extremely low, no single-barrier intervention can produce meaningful absolute improvement. This is an algebraic property of series systems, not contingent on specific parameter values. When health systems propose that improving algorithmic audit quality or adding patient notification requirements will substantially reduce algorithmic harm, Proposition 1 demonstrates that these claims are mathematically bounded by baseline success probability regardless of the magnitude of the barrier addressed.
The distinction between probability-scale and log-scale analysis (Proposition 2) is substantively important for clinical AI governance. On the log scale, barriers contribute additively—a framing suggesting incremental reform could accumulate benefits. On the probability scale, where patient outcomes and equity metrics are evaluated, multiplicative stacking produces threshold behavior: improvement remains negligible until most barriers are removed, then increases rapidly. This threshold pattern is a generic property of series systems[8] and predicts that single-lever fairness interventions will yield small absolute effects whenever baseline recourse probability is low.
The synergistic structure parallels combination therapy in infectious disease, where incomplete antiretroviral therapy is insufficient because the virus exploits any untargeted mechanism[11]. Similarly, incomplete policy intervention addresses visible symptoms while leaving the underlying multiplicative structure intact. For healthcare AI specifically, this suggests that fairness interventions limited to the algorithmic layer (bias mitigation, audit requirements) will have negligible population-level impact without simultaneous attention to data integration practices and institutional access to recourse mechanisms. The regulatory fragmentation governing algorithmic systems—CFPB for credit reporting, EEOC for employment, HHS for healthcare algorithms—mirrors the barrier fragmentation in our model and may structurally impede the coordinated approach our analysis indicates is necessary.
Interaction-dominant barrier structures likely extend to other clinical AI domains where administrative, technical and institutional processes create series-structured requirements for patient recourse. Sociotechnical systems in which fairness depends on alignment of multiple organizational and technical components[7] may exhibit analogous dynamics. Recent work on clinical AI fairness frameworks[12] and bias recognition strategies[13] has emphasized multi-level approaches; our quantitative framework provides formal support for this intuition and a measurement methodology for evaluating whether proposed interventions address sufficient layers of the barrier system.
Several limitations should be noted. This model is a formal abstraction and does not directly observe individual-level resolution trajectories. Barrier probabilities are derived or mapped (when direct measurement is unavailable) from heterogeneous empirical sources rather than a single integrated dataset; however, each parameter is traceable to publicly available federal reports or peer-reviewed studies (Tables 2–3), with explicit derivation logic documented in the repository. The principal conclusions are analytical properties of the series structure (Propositions 1–2) that hold across wide parameter perturbations, as confirmed by bootstrap resampling and global sensitivity analysis. The model assumes multiplicative independence between barriers; correlation between barriers could produce either stronger or weaker synergistic effects depending on correlation structure. The model does not capture feedback loops or heterogeneity across populations. Future work should integrate longitudinal administrative data—including healthcare claims, appeal outcomes and algorithmic audit records—to empirically estimate joint barrier transition probabilities and validate the series-system structure against observed recourse outcomes.
In summary, under a series-structured barrier model, algorithmic discrimination involves synergistic dynamics that render incremental reform insufficient in isolation. The dominant three-way interaction provides a mathematical basis for coordinated, multi-layer intervention. Effective mitigation of algorithmic harm in healthcare and beyond likely requires simultaneous intervention across data integration, data accuracy and institutional access barriers rather than isolated single-layer reforms.
4Methods
4.2Empirical parameter derivation
Pass probabilities were derived or mapped (when direct measurement is unavailable) from publicly available federal datasets and peer-reviewed studies, with explicit uncertainty ranges (Table 3). Mapping decisions were conservative, favoring higher pass probabilities where ambiguity existed to avoid overstating interaction effects. Layer 1 probabilities were derived from CFPB and FTC data on credit reporting system dynamics[3, 14]. Layer 2 probabilities were derived from FTC and CFPB dispute outcome data[15, 16]. Layer 3 probabilities were derived from Legal Services Corporation data on civil legal needs and the Obermeyer et al. algorithmic audit[1, 17]. Full derivation logic for all 11 parameters is documented in the public repository.
4.3Mathematical framework
Under the multiplicative barrier model, the probability of successful cascade completion is:
The effect of removing barrier j is Δj = P (success | pj = 1) − P (success).
Proposition 1(Single-barrier improvement bound) Removing barrier j yields Δj = P · (1/pj − 1). Thus absolute improvement is linear in P, the baseline success. When P ≈ 1.8 × 10−5, every Δj is necessarily small regardless of which barrier is targeted. This is an algebraic property of series systems, not a simulation finding.
Proposition 2(Interaction dominance on the probability scale) Define layer aggregate probabilities Lk = ∏i∈k pi for layers k = 1, 2, 3, so that P = L1 ·L2 ·L3. On the probability scale, factorial contrasts generate higher-order interactions because the response function is multiplicative. On the log scale, log P = ∑log pi is additive and interactions vanish; however, policy outcomes and equity metrics are evaluated on the probability scale, where multiplicative stacking produces threshold behavior and interaction dominance[8].
4.4Shapley value attribution
To fairly attribute system effect to individual barriers, we employed Shapley value decomposition from cooperative game theory[18], assigning each barrier its average marginal contribution across all possible removal orderings.
4.5Sensitivity analysis
We conducted comprehensive sensitivity analysis including one-at-a-time sensitivity (± 10%), Sobol global sensitivity indices[19] with Saltelli sampling (n = 1,024) and Morris screening[20]. All barrier probabilities were perturbed across ± 33% multiplicative ranges and across empirically bounded intervals (Table 3). Bootstrap resampling (n = 1,000) confirmed robustness of all principal findings.
4.6Software and reproducibility
All analyses were conducted in Python 3.10+ using NumPy, SciPy, Matplotlib and SALib[21]. Code is available at https://github.com/Nyx-Dynamics/algorithmic-bias-epidemiology-academic (random seed: 42).
4.7Use of artificial intelligence
Large language models (Anthropic Claude and OpenAI ChatGPT) were used for literature search support and language refinement. No generative-AI–generated images or figures were used; all figures were produced programmatically using Matplotlib. Jetbrains Junie was utilized for code correction, Zotero AI for reference management and the manuscript was prepared using Overleaf AI for LaTeX code correction. All analyses, interpretation and conclusions were conducted independently by the author.
4.8Ethics statement
This study used publicly available, aggregate federal datasets and published peer-reviewed statistics. No individual-level data were collected or analyzed and no human subjects were enrolled. Institutional review board approval was not required.
Supporting information
Data Availability
all source code and data for this manuscript are available online, open source under MIT License
https://github.com/Nyx-Dynamics/algorithmic-bias-epidemiology-academic
Data availability
All data files are available in CSV format at https://github.com/Nyx-Dynamics/algorithmic-bias-epidemiology-academic/tree/main/data/processed.
Code availability
All analysis code is available at https://github.com/Nyx-Dynamics/algorithmic-bias-epidemiology-academic. Key scripts: sensitivity_analysis.py (Sobol, Morris, bootstrap); barrier_visualization.py (figure generation and counterfactual analysis). Environment: Python 3.10+, NumPy, SciPy, Matplotlib, SALib. Random seed: 42.
Acknowledgements
The author thanks the open-source software communities whose tools made this analysis possible. This work received no external funding.
Competing interests
A.C.D. reports prior employment with Gilead Sciences from January 2020 through November 2024, and prior ownership of company stock, which was fully divested in December 2024. Gilead Sciences had no role in this study. A.C.D. is the owner of Nyx Dynamics LLC, a consulting company. This research was conducted independently and was not produced as part of any paid consulting engagement. No other competing interests are declared.