Clinical theory and behavioral studies suggest that people in a suicidal crisis fail to find constructive solutions and to incorporate potentially useful information into their decisions, prematurely converging on suicide. Prior studies of suicidal behavior, however, have not formally examined how individuals resolve the tradeoff between exploiting familiar options and exploring potentially superior alternatives. We investigated exploration and exploitation in suicidal behavior from the formal perspective of reinforcement learning, conducting two case-control behavioral and fMRI studies of exploration-exploitation of a large 1-dimensional continuous space and a 21-day prospective ambulatory study of suicidal ideation. Participants were adults diagnosed with borderline personality disorder (BPD) and mid-life and late-life major depressive disorder (MDD), each sample including demographically equated groups with a history of high-lethality suicide attempts, low-lethality suicide attempts, non-attempters, and non-psychiatric controls. The MDD sample also included a group with serious suicidal ideation. Sample characteristics were as follows. BPD: 171 adults (mean age: 30.55, 79% female). MDD sample: 143 adults (mean age: 62.03, 57% female). Behaviorally, lower levels of exploration, but not exploitation, were related to the medical seriousness of suicidal behavior across the BPD (Chi2(3) = 50.68, p < .001) and MDD (Chi2(4) = 21.79, p < .001) samples. Whereas high-lethality attempters exhibited a failure to shift away from unrewarded options, low-lethality attempters were prone to excessive behavioral shifts after rewarded and unrewarded actions. Demonstrating prospective validation of the case-control results, under-exploration also predicted incident suicidal ideation (Chi2(1) = 30.16, p < .001). Computational model-based fMRI data analyses (completed for BPD sample only) revealed altered option competition dynamics in the dorsal stream of high-lethality suicide attempters vs. all comparison groups. Findings were robust to confounds, including medication exposure, affective state, and behavioral heterogeneity. We conclude that narrow exploration and failure to abandon inferior options are part of the trait predisposition toward serious suicidal behavior and chronic suicidal ideation. A divergent pattern of excessive behavioral shifts is associated with low-lethality suicide attempts.