Endogenous reward (intrinsic reward at will) is a fiat currency that is occasioned by steps toward any goals which are challenging and/or uncommon enough to prevent its debasement by inflation. A “theory of mental computational processes” should propose what properties let goals grow from appetites for endogenous rewards. Endogenous reward may be the universal selective factor in all modifiable mental processes.

Public Health Significance Statement This work reviews the 50-year discussion between Howard Rachlin and the author about the hyperbolic discounting of delayed reward, its role in addictions, and how rewards can be framed to overcome current temptations. A model of motivation for nonrealistic future beliefs is presented. Objective: To examine Howard Rachlin's hypothetical model of molar choice as a tactic for resisting addictions and to explore how some of its then-radical components can be developed to account for nonphysical and far future rewards. Method: The history of Rachlin's long dialog with the present author about molar choice is reviewed. The possible implications are described of both authors' proposal that behavior can depend entirely on reward. Results: Molar choice entails bringing wider-and thus further future-contingencies to bear on current choices. The two authors proposed mechanisms with different foci, which they respectively called teleological behaviorism and intertemporal bargaining. Laboratory results have been modest, but supplementary demonstrations by thought experiments and brain imaging are described. Both proposals have left open how the value of distant outcomes, such as sobriety, savings, and healthy aging, survives temporal discounting enough to compete with present motivational pressures. In contradiction to Rachlin, but following his suggestion that reward is behavior, it is deduced that some reward must be endogenous rather than secondary to external primary rewards. Endogenous reward is proposed as a fiat currency that can function only to the extent that it is protected from inflation by some kind of uniqueness (singularity). Such uniqueness can be provided by personal disciplines for testing reality, but also by extraneous factors such as needs, coincidences, and biases. Conclusions: Rachlin's teleological behaviorism is a valuable hypothesis, but limited by its ruling out of nonexternal rewards and of intrapersonal self-prediction, both of them useful for understanding nonsubstance addictions.

Shortcuts to goals are rewarded by faster attainment and punished by more frequent failure, so selection of the various kinds - heuristics, cached sequences (habits or macros), gut instincts - depends on reward history just like other kinds of choice. The speeds of shortcuts lie on continua along with speeds of deliberation, and these continua have no obvious separation points.

Frames group choices into categories, thus modifying the incentives for them. This effect makes framing itself a motivated choice rather than a neutral cognition. In particular, framing an inferior choice with a high short-term payoff as part of a broad category of choices recruits incentive to reject it; but this must be motivated by its being a test case.

Fine examination of our accumulated cultural knowledge is especially helpful in studying the emotions, which are only tangentially accessible to experimental manipulation. Here I use the six properties of emotions that Elster has summarized to suggest how they show a need for changes in the science of motivation. The apparent adaptive purpose of emotions lies in their action tendencies - what they add to the cold calculation of advantage. Subjectively they stand out by their intrusiveness, the duration of which often has a half-life. Emotions each have a valence, which suggests that they are not only motivating but also motivated, an implication that requires re-examination of how negativity works. Emotions are also experienced as 'triggered', but are so malleable that triggering cannot mean a simple conditioned reflexiveness. Emotions are not only triggered - or motivated - by beliefs, but motivate beliefs in turn, and can be fed back on themselves in a 'wildfire' phenomenon. These feedback effects are further evidence against emotions being reflexive. With regard to the three great revolutions, I argue that enthusiasm differs from romantic love only in its object, and urgency comes from the dysphoria of using response suppression for self-control. I can add nothing further to Elster's masterful history.

Twenty-six commentators from several disciplines have written on the assumption that choice is determined by comparative valuation in a common denominator of reward, the “competitive marketplace.” There was no apparent disagreement that prospective rewards are discounted hyperbolically, although some found that the resulting predictions could come just as well from other models, including the interpretation of delay as risk and analysis in terms of hot versus cold valuation systems. Several novel ideas emerged.

Most authors who discuss willpower assume that everyone knows what it is, but our assumptions differ to such an extent that we talk past each other. We agree that willpower is the psychological function that resists temptations - variously known as impulses, addictions, or bad habits; that it operates simultaneously with temptations, without prior commitment; and that use of it is limited by its cost, commonly called effort, as well as by the person's skill at executive functioning. However, accounts are usually not clear about how motivation functions during the application of willpower, or how motivation is related to effort. Some accounts depict willpower as the perceiving or formation of motivational contingencies that outweigh the temptation, and some depict it as a continuous use of mechanisms that interfere with reweighing the temptation. Some others now suggest that impulse control can bypass motivation altogether, although they refer to this route as habit rather than willpower. It is argued here that willpower should be recognized as either or both of two distinct functions, which can be called resolve and suppression. Resolve is based on interpretation of a current choice as a test case for a broader set of future choices, which puts at stake more than the outcome of the current choice. Suppression is inhibiting valuation of (modulating) and/or keeping attention from (filtering) immediate alternatives to a current intention. Perception of current choices as test cases for broader outcomes may result in reliable preference for these outcomes, which is experienced as an effortless habit - a successful result of resolve, not an alternative method of self-control. Some possible brain imaging correlates are reviewed.

Motivational science has necessarily studied the control of behavior by external rewards, and so has not permitted explanations that do without such rewards. Addiction was long assumed to arise from the action of a drug. The obvious exception of gambling addiction was attributed to delusions about how the laws of probability govern the ostensibly rational goal of getting money. The chapter describes the several prominent proposals for which hyperbolic discounting has the greatest implications. Short-term interests based on addictions are aggressive bargainers by definition. They may make it necessary to cut the Gordian knot of intertemporal bargains and focus concretely on a single target. The persistence of addictions despite deteriorating reward has suggested the relevance of behavioral experiments on habit—the persistence of a choice after it is no longer rewarded. Sometimes the result is long-term artistic pleasure or sublimation, but occasions can also be structured to invite hedonic importance onto faster-paying patterns that can become addictive.

“Negative” emotions are never purely negative. They attract attention at the very least and often stay attractive enough to make rehearsing them an addictive activity. As the authors point out, they also counteract a relentless tendency for positive emotions to become boring. Analysis in terms of reward suggests why this tendency occurs and how symbiosis with negative emotions may arise, in art and in life.

Impulsiveness is the tendency to shift preference from a better, later option to a poorer, earlier option as the two get closer. Explanations have extrapolated from five piecemeal elements of psychology – failure of cognitive rationality, Pavlovian conditioning, force of habit, incentive salience, and long chains of secondary reward – but in doing so have created models that stretch the properties of these elements as observed in the laboratory. The models are hard to integrate with each other, much less with the utility-maximizing structure of microeconomics. The increasing prominence of addictions, particularly addictions that do not involve a substance, have confronted behavioral science with these shortcomings. I suggest that the roles of all five phenomena follow from the hyperbolic discounting of expected reward: rationality can be seen as maximizing expected discounted reward as seen at a distance, which can be approximated by making choices in bundles; Pavlovian conditioning reduces to reward learning when a very short-term rewarding effect of conditioning stimuli is factored in; ‘habit’ subsumes various kinds of reward-seeking, especially that which has been narrowed by failures of self-control; incentive salience is a kind of short-term reward; self-reward is possible – and probably common – without being secondary to inborn turnkeys, limited rather by not being occasioned so often that the appetite(s) for it fail to build up.

The debate between disease models of addiction and moral or voluntarist models has been endless, and often echoes the equally endless debate between determinism and free will. I suggest here that part of the problem comes from how we picture the function of motivation in self-control. Quantitative experiments in both humans and non-humans have shown that delayed reward loses its effectiveness in proportion to its delay. The resulting instability of preference is best controlled by a recursive self-prediction process, intertemporal bargaining, which is the likely mechanism of both the strength and the experienced freedom of will. In this model determinism is consistent with more elements of free will than compatibilist philosophers have heretofore proposed, and personal responsibility is an inseparable, functional component of will. Judgments of social responsibility can be described as projections of personal responsibility, but normative responsibility in addiction is elusive. The cited publications that are under the author's control can be downloaded from www.picoeconomics.org.

The role of emotional trauma in psychopathology is limited. One additional mechanism is predictable from hyperbolic discounting: When a person uses willpower to control urges each success or failure takes on extra significance through recursive self-prediction, potentially motivating several constricting defense mechanisms. The need for eliciting emotion in psychotherapy is as the authors say it is, but their hypothesis about reconsolidation of memories adds no explanatory power.

Authors of non-liberal proposals experience more collegial objections than others do. These objections are often couched as criticism of determinism, reductionism, or methodological individualism, but from a scientific viewpoint such criticism could be easily answered. Underneath it is a wish to harness scientific belief in service of positive social values, at the cost of reducing objectivity.

In previous work, smokers showed steeper devaluation of delayed rewards than non‐smokers. While the neural correlates of this link between nicotine dependence and delay of discounting are not established, altered activity in executive networks may relate to impaired delayed gratification. The goal of this study was to examine neural correlates of discounting and their relation to nicotine dependence. Thirty‐nine smokers and 33 non‐smokers completed a functional magnetic resonance imaging (fMRI) intertemporal choice task in which they made individualized Hard (similarly valued), easy (dissimilarly valued), and control monetary choices. FMRI data were analyzed using a group independent component analysis and dual regression. Smokers discounted more steeply than non‐smokers, although this difference was only significant among severely dependent smokers. Intertemporal choices recruited distinct left‐ and right‐lateralized fronto‐parietal networks. A group‐by‐difficulty interaction indicated that smokers, relative to non‐smokers, exhibited less difficulty‐sensitivity in the right fronto‐parietal network. In contrast, smokers showed greater functional connectivity between the left fronto‐parietal network and the left fronto‐insular cortex. Moreover, the degree of functional connectivity between the left fronto‐parietal network and left fronto‐insular cortex was significantly correlated with individual differences in discounting. Thus, greater functional coupling between the anterior insula and left fronto‐parietal network is a candidate neural substrate linking smoking and impulsivity. Given the anterior insula's role in interfacing cognitive and interoceptive processing, this altered functional connectivity may relate to an addiction‐related bias towards immediate rewards. Hum Brain Mapp 35:3774–3787, 2014. © 2014 Wiley Periodicals, Inc.

Research on intertemporal behavior has emphasized trait-like variance. However, recent studies have begun to explore situational factors that affect intertemporal preference. In this study, we examined the associations between emotional primes and both behavior and brain function during intertemporal decision making. Twenty-two participants completed a dual task in which they were required to make intertemporal choices while holding an expressive face in memory. From trial-to-trial, the facial expression varied between three alternatives: (i) fearful, (ii) happy and (iii) neutral. Brain activity was recorded using functional magnetic resonance imaging for 16 participants. Behavioral data indicated that fearful (relative to happy) faces were associated with greater preference for larger but later rewards. During observation of fearful faces, greater signal change was observed in the right dorsolateral prefrontal cortex. During subsequent decision making, the fear prime was associated with greater signal increase in structures including the posterior sector of the anterior cingulate cortex. Individual differences in this activity correlated with the magnitude of the priming effect on behavior. These findings suggest that incidental emotions affect intertemporal choice. Increased farsightedness after the fear prime may be explained by an 'inhibition spillover' effect.

Selfish Goal Theory is compatible with a behaviorally based theory that recognizes mental processes as behaviors. Both envision choices as made by the competition of purposive processes, which are autonomous in that they are not coordinated by an agentic "self." However, the survival of mental processes-termed "goals" or "interests," respectively-depends on a well-documented active mechanism: reward.

To the extent that acting fairly is in an individual's long-term interest, short-term impulses to cheat present a self-control problem. The only effective solution is to interpret the problem as a variant of repeated prisoner's dilemma, with each choice as a test case predicting future choices. Moral choice appears to be the product of a contract because it comes from self-enforcing intertemporal cooperation.

Behavioral science has assumed that reward which is not innate is a soft currency that must be backed by the hard currency of an innately rewarding process somewhere in prospect. However, hyperbolic delay discount curves predict that there can be arbitrarily self-generated reward, which may be not only robust but addictive. Such endogenous reward depends on the appetite for it, and therefore becomes intense when occasioned by signals that are adequately singular and surprising. Because signals are often singular because they predict innate rewards, it may be hard to distinguish this predictive value from their hedonic value as occasions for endogenous reward. The conflict between an instrumental incentive to accelerate rewarding events and a hedonic incentive to refresh appetite may motivate indirect approaches to these events, which maximize these events’ total reward by creating obstacles to them. Gambling is offered as an example.