Drug addiction remains one of the most important public health problems in western societies and is a rising concern for developing nations. Over the past 3 decades, experimental research on the neurobiology and psychology of drug addiction has generated a torrent of exciting data, from the molecular up to the behavioral levels. As a result, a new and pressing challenge for addiction research is to formulate a synthetic theoretical framework that goes well beyond mere scientific eclectism to deepen our understanding of drug addiction and to foster our capacity to prevent and to cure drug addiction. Intrigued by the apparent irrational behavior of drug addicts, researchers from a wide range of scientific disciplines have formulated a plethora of theoretical schemes over the years to understand addiction. However, most of these theories and models are qualitative in nature and are formulated using terms that are often ill-defined. As a result, the empirical validity of these models has been difficult to test rigorously, which has served to generate more controversy than clarity. In this context, as in other scientific fields, mathematical and computational modeling should contribute to the development of more testable and rigorous models of addiction.

lgli/2411.pdf

lgrsnf/2411.pdf

Springer series in computational neuroscience, New York, ©2012

United States, United States of America

2012, PS, 2011

{"isbns":["1461407508","9781461407508"],"last_page":356,"publisher":"Springer","source":"libgen_rs"}

Computational Neuroscience of Drug Addiction
Foreword
Acknowledgements
Acknowledgements
Contents
Contributors
Part I: Pharmacological-Based Models of Addiction
Chapter 1: Simple Deterministic Mathematical Model of Maintained Drug Self-administration Behavior and Its Pharmacological Applications
1.1 Introduction
1.1.1 Definitions of Terms
1.1.2 Principles
1.1.3 Critical Review of Existing Models
1.2 Mathematical Model of Maintenance
1.2.1 Assumptions
1.2.2 Dose-Duration Curve
1.2.3 Satiety Threshold
1.3 Application of the Model
1.3.1 Measurement of Antagonist Kdose
1.3.2 Calculation the Level of Cocaine in the Body
1.3.3 Agonist Concentration Ratios
1.3.4 Measurement of Antagonist Pharmacokinetics
1.3.5 Pharmacokinetic and Pharmacodynamic Models
1.4 Discussion
1.4.1 Indirect Agonist
1.4.2 Quasi-Equilibrium
1.4.3 Volume of Distribution
1.4.4 Summary
1.5 Conclusions
References
Chapter 2: Intermittent Adaptation: A Mathematical Model of Drug Tolerance, Dependence and Addiction
2.1 Introduction
2.2 Properties of Adaptive Regulated Physiological Processes
2.2.1 Homeostasis
2.2.2 The Properties of Adaptive Processes
2.2.3 The Detection of Exogenous Substances
2.2.4 Oral and Environmental Cues
2.2.5 The Effect of Unknown Substances
2.2.6 The Magnitude of the Compensatory Response
2.3 Modelling Tolerance Development in Physiological Processes
2.3.1 The Model
2.3.2 Different Ways in Which Drugs Disturb the Body
2.3.3 Fast and Slow Adaptation
2.4 The Mathematical Model and Its Practical Significance
2.4.1 The Model
2.4.2 The Open-Loop Gain
2.4.3 Constant Drug Effect
2.4.4 Adaptive Regulation
2.4.5 The Effect of Changes in Drug Dose
2.4.6 The Dose-Response Curve
2.4.7 The Effect of a Further Reduction in the Drug Dose
2.4.8 Sensitisation and Other Paradoxical Effects
2.5 Practical Significance of the Model
2.5.1 Anticipation and Dependence
2.5.2 Alternative Protocols for Drug Withdrawal
Appendix
A.1 The Digestive Tract
A.2 The Bloodstream
A.3 The Adaptive Regulator
A.3.1 The Fast Regulator
A.3.2 Estimation of the Drug Effect in the Adaptive Regulator
A.3.3 The Slow Regulator
A.4 The Process
A.5 Loop Control
A.6 The Sensor
References
Chapter 3: Control Theory and Addictive Behavior
3.1 Control Theory and Addiction
3.1.1 Scope
3.2 Key Concepts
3.2.1 Feedback
3.2.2 Feedback Systems Gone Awry
3.2.3 Linear and Non-linear Dynamical Systems
3.2.4 Feedback Delay
3.2.5 Control Theoretic Diagrams
3.3 A Brief Introduction to Control Theory
3.3.1 Applying Control Theory
3.3.2 Modern History
3.3.3 Reverse Engineering
3.4 Characteristics of Control Theoretic Systems
3.5 Homeostatic Theories of Addiction
3.5.1 Heroin Addiction Model
3.5.2 Control Theoretic Considerations
3.5.3 Opponent Process Theory
3.5.3.1 Gutkin et al. (2006)'s Opponent Process Model of Nicotine Addiction
3.5.4 Respondent Conditioning
3.5.5 Control Theoretic Considerations
3.5.6 Problems with Homeostatic Models
3.5.7 Cognitive Theories of Addiction
3.5.8 Internal Model
3.5.9 Direction of the Conditioned Response
3.6 Non-homeostatic Models of Addictive Behavior
3.6.1 Instrumental Conditioning
3.6.2 Control Theoretic Considerations
3.6.3 Incentive-Sensitization Model
3.6.4 Control Theoretic Considerations
3.6.5 Autoshaping
3.6.6 Control Theoretic Considerations
3.6.7 Evolutionary Psychological Theory
3.6.8 Control Theoretic Considerations
3.7 Simulations of Two Addiction Systems
3.7.1 Simulation of Opponent Process Theory
3.7.2 Opponent Process Theory Implementation
3.7.3 "Lossy Accumulator"
3.7.4 Drug Withdrawal
3.7.5 Simulation of Instrumental Conditioning Theory
3.7.6 Sensitization and Habituation
3.8 General Discussion
3.8.1 Linear
3.8.2 Dynamical
3.8.3 Systems
Noise
Feedback
Set Points
Structural Similarity
Mutual Exclusivity
3.8.4 Limitations
Alternative Models
3.8.5 Surrogate Measures
3.8.6 Issues for Further Theoretical Development
3.9 Conclusions
3.9.1 Final Comment
References
Part II: Neurocomputational Models of Addiction
Chapter 4: Modelling Local Circuit Mechanisms for Nicotine Control of Dopamine Activity
4.1 Introduction
4.2 The Ventral Tegmental Area as a Local Circuit: A Brief Overview
4.3 Global Neurocomputational Framework Shows How Receptor-Level Effects of Nicotine Result in Self-administration
4.4 Local Circuit Model of the VTA Shows the Mechanisms for Nicotine Evoked Dopamine Responses
4.5 VTA and nAChR Model
4.5.1 Mean-Field Description of Dopaminergic and GABAergic VTA Neurons
4.5.2 Modeling the nAChR Activation and Desensitization Driven by Nic and ACh
4.6 VTA Model Results
4.7 Kinetics of the Subunit Specific nAChR Model Stimulated by Nic and/or ACh
4.8 Modeling the VTA Response to Nicotine in vitro and in vivo
4.8.1 Excitatory and Inhibitory Input Increases to DA Cells in vitro Reproduced by the Model
4.8.2 Direct Stimulation (Intrinsic Cellular) vs. Disinhibition (Local Circuit) Mechanisms for Nicotine-Evoked Increase of DA Cell Activity in vivo
Direct Stimulation
Disinhibition
4.9 Predictions of the VTA Model and Experimental Protocols to Pin Down the Major Nicotinic Pathway of Action
4.10 Discussion
References
Chapter 5: Dual-System Learning Models and Drugs of Abuse
5.1 Introduction
5.2 Background: Reinforcement Learning and Behavior
5.2.1 The Markov Decision Process
5.2.2 Values and Policies
5.2.3 Algorithms for RL
5.2.3.1 Model-Free RL
5.2.3.2 Model-Based RL
5.2.4 RL and Behavioral Neuroscience
5.2.5 RL and Drugs of Abuse
5.3 Drugs and Model-Based RL
5.3.1 Drugs and Model-Based Reward
5.3.2 Drugs and Model-Based Value
5.3.3 Drugs and Model-Based Search
5.4 Conclusion
References
Chapter 6: Modeling Decision-Making Systems in Addiction
6.1 Multiple Decision-Making Systems, Multiple Vulnerabilities to Addiction
6.1.1 Temporal Difference Reinforcement Learning and the Dopamine Signal
6.1.2 Value Prediction Error as a Failure Mode
6.1.3 Pavlovian Systems
6.1.4 Deliberation, Forward Search and Executive Function
6.2 Temporal Difference Learning in a Non-stationary Environment
6.3 Discounting and Impulsivity
6.3.1 Seeing Across the Intertrial Interval
6.3.2 Precommitment and Bundling
6.4 Decision-Making Theories and Addiction
References
Chapter 7: Computational Models of Incentive-Sensitization in Addiction: Dynamic Limbic Transformation of Learning into Motivation
7.1 Introduction
7.1.1 Dopamine and Reinforcement Learning: The "Standard" Model and Critique
7.2 Previous Computational Approaches to Learning and Incentive Salience
7.3 Our Dynamic Model of Incentive Salience: Integrating Learning with Current State
7.4 Testing Model Predictions Using a Serial Conditioning Task
7.4.1 VP Neuronal Coding for Incentive Salience
7.5 Discussion
7.5.1 Multiple Motivation-Learning Systems
7.5.2 Contrasting Dynamic Incentive Salience to Cognitive Tree Goals
References
Chapter 8: Understanding Addiction as a Pathological State of Multiple Decision Making Processes: A Neurocomputational Perspective
8.1 Introduction
8.2 Normal Decision Making Mechanism
8.3 Aspects of Addictive Behavior
8.3.1 Compulsive Drug Seeking and Taking
8.3.2 Impulsivity
8.3.3 Relapse
8.4 Computational Accounts
8.4.1 S-R Models
8.4.1.1 Redish's Model
8.4.1.2 Dezfouli et al.'s Model
8.4.2 S-S and S-R Interaction: Actor-Critic Models
8.4.2.1 Dayan's Model
8.4.2.2 Piray et al.'s Model
8.4.3 S-R and S-A-O Interaction: Dual-Process Models
8.5 Conclusion
References
Part III: Economic-Based Models of Addiction
Chapter 9: Policies and Priors
9.1 Introduction
9.2 The Free-Energy Formulation
9.2.1 Free-Energy and Self-organisation: Overview
9.2.2 Free-Energy and Self-Organisation: Active Inference from Basic Principles
9.2.2.1 Action and Perception
9.2.2.2 Summary
9.2.3 Dynamic Generative Models
9.2.3.1 Perceptual Inference and Predictive Coding
9.2.3.2 Perceptual Learning and Associative Plasticity
9.2.3.3 Action
9.2.3.4 Summary
9.3 Priors and Policies
9.3.1 Set-up and Preliminaries
9.3.2 Fixed-Point Policies: The Equilibrium Perspective
9.3.2.1 The Mountain-Car Problem
9.3.2.2 Value-Based Policies
9.3.2.3 Optimal Control and Reinforcement Learning
9.3.3 Itinerant Policies
9.3.3.1 Itinerant Control and Autovitiation
9.3.3.2 The Generative Model
9.3.3.3 The Generative Process
9.3.4 Summary
9.4 Pathological Policies
9.4.1 Simulating Parkinsonism
9.4.1.1 Summary
9.4.2 Simulating Addiction
9.4.2.1 Normal Learning
9.4.2.2 Pathological Learning
9.5 Discussion
9.6 Conclusion
Appendix A: Parameter Optimisation and Newton's Method
Appendix B: Simulating Action and Perception
References
Chapter 10: Toward a Computationally Unified Behavioral-Economic Model of Addiction
10.1 Introduction
10.2 Behavioral Economics of Addiction I: Basic Demand Curve Analysis
10.3 Behavioral Economics of Addiction II: Analysis of Inter-temporal Choice
10.4 Integration of Variables I: Extension of Price to Unit Price
10.5 Integration of Variables II: "Costs" and "Benefits" in Concurrent Choice
10.6 Conclusions and Future Directions
References
Chapter 11: Simulating Patterns of Heroin Addiction Within the Social Context of a Local Heroin Market
11.1 Introduction
11.1.1 Individual and Social Patterns Impacting Heroin Addiction
11.1.2 Internal Components of Heroin Addiction
11.1.3 Social and Market Components of Heroin Addiction
11.2 The Data
11.3 The Model
11.4 Results
11.5 Limitations
11.6 Conclusion
References
Index

2024-03-20