Short-term memory traces for action bias in human reinforcement learning.

Short-term memory traces for action bias in human reinforcement learning. Research Abstract Details 

Research Abstract Table of Contents

Jump to the:

Short-term memory traces for action bias in human reinforcement learning. Abstract Text:

Rafal Bogacz,Samuel M McClure,Jian Li,Jonathan D Cohen,P Read Montague,Rafal Bogacz,Samuel M McClure,Jian Li,Jonathan D Cohen,P Read Montague,

Recent experimental and theoretical work on reinforcement learning has shed light on the neural bases of learning from rewards and punishments. One fundamental problem in reinforcement learning is the credit assignment problem, or how to properly assign credit to actions that lead to reward or punishment following a delay. Temporal difference learning solves this problem, but its efficiency can be significantly improved by the addition of eligibility traces (ET). In essence, ETs function as decaying memories of previous choices that are used to scale synaptic weight changes. It has been shown in theoretical studies that ETs spanning a number of actions may improve the performance of reinforcement learning. However, it remains an open question whether including ETs that persist over sequences of actions allows reinforcement learning models to better fit empirical data regarding the behaviors of humans and other animals. Here, we report an experiment in which human subjects performed a sequential economic decision game in which the long-term optimal strategy differed from the strategy that leads to the greatest short-term return. We demonstrate that human subjects' performance in the task is significantly affected by the time between choices in a surprising and seemingly counterintuitive way. However, this behavior is naturally explained by a temporal difference learning model which includes ETs persisting across actions. Furthermore, we review recent findings that suggest that short-term synaptic plasticity in dopamine neurons may provide a realistic biophysical mechanism for producing ETs that persist on a timescale consistent with behavioral observations.

Short-term memory traces for action bias in human reinforcement learning. Publishing Authors By Initials

R Bogacz,SM McClure,J Li,JD Cohen,PR Montague,R Bogacz,SM McClure,J Li,JD Cohen,PR Montague,

For similar abstracts research abstracts see: abstracts research

PUBMED ID PMID:

MEDLINE DATE:

Short-term memory traces for action bias in human reinforcement learning. Journal Published:

PUBLICATION TYPE: Research Support, Non-U.S. Gov

Journal: Brain research

VOLUME: 1153

Page Numbers: 111-21

Journal Abbreviation: Brain Res.

ISSN: 0006-8993

DAY: 24

MONTH: 03

YEAR: 2007

Short-term memory traces for action bias in human reinforcement learning. Information

Number of References:

LANGUAGE: eng

NlmUniqueID: 45503

Short-term memory traces for action bias in human reinforcement learning. Keywords Mesh Terms:

KEYWORDS:

MESH TERMS:

Chemical & Substance for Abstract: Short-term memory traces for action bias in human reinforcement learning. Information

Substance Name:

Registry Number:

Grant and Affiliation Information for Short-term memory traces for action bias in human reinforcement learning.

AFFILIATION: Center for the Study of Brain, Mind and Behavior, Princeton University, Princeton, NJ 08544, USA. R.Bogacz@bristol.ac.uk

Country: Netherlands

AGENCY: United States NIMH

GRANT: P50 MH62196

ACRONYM: MH

MEDLINETA: Brain Res

REFSOURCE:

DATABASENAME:

ACCESSION NUMBER:

Number Hits: 0

Short-term memory traces for action bias in human reinforcement learning Related Publications

• Cause-specific noncardiac mortality in patients with congestive heart failure--a contemporary Canadian audit. Clinical Quality Improvement Network (CQIN) Investigators.
• Clinical efficacy of emergency bypass surgery for failed coronary angioplasty.
• Contemporary disease management in Quebec.
• Contemporary medical management of left ventricular dysfunction and congestive heart failure.
• Effects of long term cholesterol lowering on coronary atherosclerosis in patient risk factor subgroups: the Simvastatin/enalapril Coronary Atherosclerosis Trial (SCAT).
• Elective irradiation with the electron beam after mastectomy for breast cancer.
• Electromagnetic implantable middle ear hearing device of the ossicular-stimulating type: principles, designs, and experiments.
• Geriatric aspects of rectal diseases.
• Home training for preschool deaf children through correspondence.
• Human ecchordosis physaliphora and chick embryonic notochord. A comparative electron microscopic study.
• In vivo skeletal muscle metabolism during dynamic exercise and recovery: assessment by nuclear magnetic resonance spectroscopy.
• Long term clinical outcomes following isolated mitral valve replacement.
• Long term follow-up after isolated aortic valve replacement.
• Malignant masses in the therapeutically irradiated breast.
• Metabolism of normal skeletal muscle during dynamic exercise to clinical fatigue: in vivo assessment by nuclear magnetic resonance spectroscopy.
• Neural signature of fictive learning signals in a sequential investment task.
• Nonpharmacologic treatment of erectile dysfunction.
• Nonrheumatic atrial fibrillation: a brief overview.
• Optimizing healthcare at the population level: results of the improving cardiovascular outcomes in nova scotia partnership.
• Penile prosthesis implantation for end-stage erectile dysfunction after radical prostatectomy.
• Proceedings: Radiotherapy of breast cancer: guidelines according to features of disease.
• Q-wave infarction: pathophysiology of body surface potential map and ventriculographic patterns in anterior and inferior groups.
• Quantitative relation of electrocardiographic and angiocardiographic measures of risk in patients with coronary atherosclerosis. Simvastatin/Enalapril Coronary Atherosclerosis Trial (SCAT) Investigators.
• Rationale and design features of a clinical trial examining the effects of cholesterol lowering and angiotensin-converting enzyme inhibition on coronary atherosclerosis: Simvastatin/Enalapril Coronary Atherosclerosis Trial (SCAT). SCAT Investigators.
• Recognition memory as a function of encoding strategy and stimulus codability.
• Short-term memory traces for action bias in human reinforcement learning.
• The changing clinical profile of coronary artery bypass graft patients, 1970-89.
• The first wave.
• The treatment and prevention of coronary heart disease in Canada: do older patients receive efficacious therapies? The Clinical Quality Improvement Network (CQIN) Investigators.
• The utilization of antithrombotic prophylaxis for atrial fibrillation in a geriatric rehabilitation hospital.