Diving deep into the brain: a case for Deep Reinforcement Learning in Neuroscience

Interested in the interplay of artificial intelligence and neuroscience? Then you don’t want to miss Dr. Matthew Botvinick’s talk on deep reinforcement learning this Tuesday, June 15 at 4pm. Dr. Botvinick is the director of Neuroscience research at DeepMind and an Honorary Professor at the Gatsby Computational Neuroscience Unit at University College London (UCL). He completed his undergraduate studies at Stanford University in 1989, medical studies at Cornell University in 1994, and PhD in psychology and cognitive neuroscience at Carnegie Mellon University in 2001. Dr. Botvinick’s work at DeepMind sits at the boundaries between cognitive science, computational and experimental neuroscience and artificial intelligence. His pioneering work includes studying the role of anterior cingulate cortex in conflict monitoring, and modeling the role of prefrontal cortex in hierarchically structured behavior, working memory, and sequential action.   

More recently, Dr. Botvinick has been studying deep reinforcement learning and its neuroscientific applications (Botvinick et al., 2020). Recent advances in AI and machine learning have led to an increased interest in using deep learning to model brain function. Most of these research efforts have focused on deep neural networks trained using supervised learning in tasks such as image classification. More novel developments in AI that may have profound implications in studying brain function, such as deep Reinforcement Learning (RL), have received far less attention from the neuroscience field. Deep RL integrates classic Deep Learning and Reinforcement Learning  (Figure 1) . 

Figure 1. RL, Deep Learning, and Deep RL

Deep learning leverages networks of artificial neurons to perform a task. The nonlinear nature of the neurons allows the network to compute complicated functions for a given input, making the network very adaptable to a wide range of tasks. What ends up making the network really good at one task over another is the numerous connections between neurons in the network. The key innovation of deep learning is the process of algorithmically determining how to adjust the connections between neurons in the network to produce an optimal response for a given set of inputs. Conceptually, there are two main approaches to training deep networks: supervised and unsupervised learning. In supervised learning, the network is given explicit feedback on its performance (whether a given response was correct or incorrect) and uses that information to make adjustments to the connections between neurons. In unsupervised learning, the network is not given any feedback but rather uses an algorithm to autonomously evaluate the inputs and adjust connection weights to capture a “good” representation of them. Deep RL ends up somewhere in the middle of these alternatives by actively providing feedback to the network, without being so explicit that the network loses its autonomy to explore and algorithmically determine the optimal connections.

In Reinforcement Learning, an algorithm is used to produce a reward signal for a given response produced by the network. (Think giving your cat treats as positive reinforcement.) Since the reward is algorithmically generated, the network can learn autonomously and adjust its connections to produce responses that return a greater reward. Additionally, since the network is not told explicitly whether its response was right or wrong, it has the flexibility to explore alternative actions or strategies throughout the learning processes. Integrating algorithmically generated reward signals into deep network architectures so that networks explore and perfect a given task has proven to be a challenging engineering problem. 

Recent advances now allow neuroscientists to model neuronal systems within a deep RL framework. This perspective is particularly attractive, as there is evidence that the brain computes reward or prediction signals that guide its activity. A small number of vanguard studies have started to apply deep RL to neuroscientific data. For instance, when trained on a number of similar tasks, deep RL networks are found to adapt to new tasks of the same kind (Wang et al., 2018). The resulting network activity was structured to be flexible so that the network could change its internal activity and subsequent responses to produce good performance on multiple tasks without altering the connections between neurons. This phenomena is in line with neuroscientific theories of prefrontal cortex functioning, such as activity-based working memory.

Deep RL provides an agent-based framework for studying the way reward shapes representation, and how representation in turn shapes learning and decision making. This exploration of deep RL in neuroscience offers exciting new opportunities to better understand brain function. Examples of areas for next-stage research include representation learning, memory, cognitive control, action hierarchies, and social cognition amongst many others. As more applications of deep RL models to neuroscientific data emerge, an additional opportunity for neuroscience research to influence deep RL also arises. New insights into how the brain may implement RL could synergistically give rise to concepts that engineers can implement to advance AI and machine learning.

To learn more about Dr. Botvinick’s research and recent work, join us for his talk on deep reinforcement learning, this Tuesday, 6/15/2021, at 4pm on Zoom. 

References

• Botvinick, M., Wang, J. X., Dabney, W., Miller, K. J., & Kurth-Nelson, Z. (2020). Deep Reinforcement Learning and Its Neuroscientific Implications. Neuron, 107(4), 603–616. https://doi.org/10.1016/j.neuron.2020.06.014
• Princeton Neuroscience Institute Website: http://www.princeton.edu/~templeton/Templeton/People.html
• Stanford University Human-Centered Artificial Intelligence Website: https://hai.stanford.edu/people/matthew-botvinick
• Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., Hassabis, D., & Botvinick, M. (2018). Prefrontal cortex as a meta-reinforcement learning system. Nature Neuroscience, 21(6), 860–868. https://doi.org/10.1038/s41593-018-0147-8

Written by Blanca Martin-Burgos, a 1st year student in the Neuroscience PhD Program at UCSD working with the Voytek and Muotri labs.

More than reward – dopamine circuits are a force of habit

What information do dopamine circuits in the brain carry? How does the information encoded by this powerful neurotransmitter form our habits, moods, and personalities? Dr. Talia Lerner, an assistant professor at Northwestern University, is blazing the path towards finding out. Dr. Lerner is interested in how dopamine circuit architecture regulates learning, habits, and mental disorder risk by understanding how dopamine generates and disseminates information. Not only an amazing scientist, Dr. Lerner is also focused on promoting inclusivity and openness in academia as a role model for women in science. In her professorship since 2017, she has already forged a reputation as one of the top dopamine researchers in the country, with an extensive list of awards including the NIH Director’s New Innovator Award.

Her lab’s work builds on the research she undertook during her postdoc years in the Deisseroth lab at Stanford. Habits are a tricky phenomenon to process, in that they require both reward learning circuits and motor circuits. To understand the neural circuits that contribute to habits, she was interested in looking deeper into the structure required for habit formation the striatum, the input center of the sub-brain structure of the basal ganglia, and dopamine inputs to the striatum from the substantia nigra pars compacta (SNc). 

In Lerner et al. 2015, she identified two parallel SNc dopamine pathways that input into two separate areas of the striatum (Fig. 1). Using cutting-edge techniques, the authors found that these groups of dopamine neurons differed in their biophysical properties, input wiring and natural activity patterns during free behavior. In a finding that could only been seen through active circuits, she found that through optogenetic stimulation of dopaminergic neurons, inputs from the dorsolateral striatum (DLS) to SNc were much stronger than inputs from the dorsomedial striatum (DMS) (Fig. 1). Targeting SNc dopamine neurons based on their striatal target, her experiments were consistent with past findings that DMS dopamine neurons increase firing in response to aversive stimuli. These results suggest that DMS-projection SNc dopamine neurons are more active following an unfamiliar outcome, promoting learning. DMS and DLS are also reciprocally connected with the same dopamine neurons that project back to those areas. To add to the complexity of this circuit, she found interconnection between DLS projections to DMS projecting dopamine neurons, which provides a pathway for dopamine information to transfer between neurons during habit learning. 

Figure 1. Graphical summary of Lerner et al. 2015 

Dr. Lerner’s lab is currently expanding on this work, investigating how dopamine neuron subpopulations change with experience, learning, and habit formation. She hopes that this can bring about new understanding into how adverse events – such as stress – alter habits and other learned behaviors through these circuits. 

References

Dr. Lerner’s Lab Website: http://lernerlab.org/talia

Lerner, Talia N., et al. “Intact-brain analyses reveal distinct information carried by SNc dopamine subcircuits.” Cell 162.3 (2015): 635-647.

Lauren Stanwicks is a 1st year PhD  student in the Neurosciences Graduate Program at UCSD. She is currently rotating in the lab of Richard Daneman. Meet her on twitter @neuronlauren

Let there be light: linking neural circuit activity and behavior via an “all-optical” approach

Dr. Michael Hausser is a Wellcome Principal Research Fellow and Professor of Neuroscience at University College London. The Hausser lab combines cellular and systems neuroscience in order to examine neural circuit computations at the cellular level in the mammalian brain, with a special focus on the role of dendrites. In 2015, Dr. Hausser was elected a Fellow of the Royal Society for his fundamental contributions to our understanding of how dendrites contribute to computation in the mammalian brain, presenting dendrites as independent processing and signaling units that perform local computations. Dr. Hausser’s group aims to establish a causal link between neural circuit activity and behavior and is addressing this goal by employing an “all-optical” approach, enabling concurrent readout and manipulation of neural circuit activity.

The Hausser lab recently demonstrated the utility of this approach in work published in 2020, titled “Targeted Activation of Hippocampal Place Cells Drives Memory-Guided Spatial Behavior.” Hippocampal place cells are pyramidal neurons that exhibit location-specific firing, and populations of these cells form unique maps that represent a given environment. Place cell activity can encode information about an experience, and primarily correlational evidence has shown that generation of place cell firing sequences supports spatial navigation and episodic memory formation. This study demonstrated a causal role for hippocampal place cell activity in memory-guided spatial navigation. The authors utilized an “all-optical” combination of simultaneous two-photon calcium imaging and two-photon targeted optogenetics in head-fixed mice performing a virtual reality spatial navigation task (Graphical Abstract). This allowed them to both record and stimulate various populations of place cells that encode distinct behaviorally relevant locations and thus assess their role in guiding spatial behavior. The authors hypothesized that stimulating a population of similarly tuned place cells would drive mice to exhibit the behavior that is normally carried out in the location of those cells’ place fields.

Graphical Abstract

First, mice were trained to perform a virtual reality spatial navigation task that required them to stop and lick in a specific reward zone on a virtual linear track in order to receive a reward (Figures 1C-1E). As the mice executed this task, the authors performed two-photon calcium imaging to identify cells with a place field on the virtual track (Figures 1F and 1G) and then categorized place cells with fields that covered the reward zone and start zone as reward-zone place cells (Reward-PCs) and start zone place cells (Start-PCs), respectively. Reward-PC activity corresponded to high lick rate and decelerated running speed, whereas Start-PC activity was associated with low lick rate and stable high running speed. During stimulation sessions, two-photon optogenetics was used to activate specific place cell populations (Start-PCs, Reward-PCs) as the mouse crossed the central stimulation point.

Figure 1: All-Optical Manipulation of Place Cells during Spatial Navigation in Virtual Reality

The authors found that stimulating Reward-PCs in a location preceding the reward zone and their endogenous firing fields caused an increase in lick rate in comparison to the behavioral baseline in the stimulation zone (Figures 2B and 2C). Thus, the animal’s behavior was biased toward that which was normally exhibited in the reward zone. This demonstrates that reward-zone place cell activity has a causal role in driving spatial behavior. Furthermore, Start-PC stimulation, which occurs in a location beyond their endogenous firing fields, resulted in an increase in the proportion of trials where the mouse ran past the reward zone, and these trials were accompanied by a decrease in reward zone licking (Figures 2K and 2L). This shows that Start-PC stimulation has a behavioral effect that occurs after stimulation has stopped, suggesting an ongoing impact on neural activity. The authors additionally demonstrated that place cell stimulation inhibits endogenous place code expression and triggers remapping.

Figure 2: Targeted Stimulation of Reward-Zone Place Cells Drives Reward-Zone-Related Behavior

The aforementioned work demonstrates the exciting potential for “all-optical” strategies in probing neural circuit dynamics and identifying causal relationships between behavior and neural circuit activity.

To learn more about Dr. Hausser’s work, please join us for the talk, “Forging Causal Links between Neural Circuit Activity and Behavior,” on Tuesday, May 18^th at 12pm on Zoom.

Zoom: https://uchealth.zoom.us/j/81547662915

References:

1. Dalgleish HW, Russell LE, Packer AM, Roth A, Gauld OM, Greenstreet F, Thompson EJ, Häusser M. How many neurons are sufficient for perception of cortical activity? Elife. 2020 Oct 26;9:e58889. doi: 10.7554/eLife.58889. PMID: 33103656; PMCID: PMC7695456.
2. Emiliani V, Cohen AE, Deisseroth K, Häusser M. All-Optical Interrogation of Neural Circuits. J Neurosci. 2015 Oct 14;35(41):13917-26. doi: 10.1523/JNEUROSCI.2916-15.2015. PMID: 26468193; PMCID: PMC4604230.
3. Robinson NTM, Descamps LAL, Russell LE, Buchholz MO, Bicknell BA, Antonov GK, Lau JYN, Nutbrown R, Schmidt-Hieber C, Häusser M. Targeted Activation of Hippocampal Place Cells Drives Memory-Guided Spatial Behavior. Cell. 2020 Dec 10;183(6):1586-1599.e10. doi: 10.1016/j.cell.2020.09.061. Epub 2020 Nov 6. Erratum in: Cell. 2020 Dec 23;183(7):2041-2042. PMID: 33159859; PMCID: PMC7754708.
4. Russell LE, Yang Z, Tan PL, et al. The influence of visual cortex on perception is modulated by behavioural state. bioRxiv; 2019. DOI: 10.1101/706010.

Hausser Lab Website: http://www.dendrites.org/

Audrey Miglietta is a first year PhD student in the Neurosciences Graduate Program at UCSD, currently rotating in the lab of Dr. Richard Daneman.

The cost of thinking and not thinking: metabolic control of synapses

Nerve terminals in the brain must
synthesize ATP on demand to sustain
function. Here, Ashrafi, de Juan-Sanz,
et al. show that axonal mitochondria use
the brain-specific MCU regulator MICU3
to allow efficient Ca2+ uptake in order to
accelerate ATP production.

The focus of Dr. Tim Ryan’s lab is the study of the molecular basis of synaptic transmission in the mammalian brain. Dr.Ryan’s primary interests lie in understanding the molecular basis of synaptic performance.

They use biophysical tools to examine synapse function. These tools provide single synapse measurements of exocytosis, endocytosis, action potential wave forms, calcium fluxes as well as the concentration of key metabolites. The brain is acutely sensitive to metabolic compromise. They showed recently that nerve terminals represent one of the key loci of this vulnerability (Cell 2014). These studies opened up several key questions the Ryan lab is currently pursuing about synaptic metabolism. How much ATP do different processes at synapses consume? What are the biochemical rules in play to synthesize ATP in response to activity? What are the biochemical reasons synapses are so vulnerable?

Dr. Timothy Ryan was trained as a physicist, with a master’s degree in experimental particle physics from McGill University. He then studied protein mobility on cell surfaces and signal transduction with Dr. Watt Webb (Cornell University) for his PhD. With his background in physics and biochemistry, he then joined Dr. Stephen Smith (Stanford) to investigate synaptic vesicle recycling. Over the years, his work has been recognized through distinctions such as The Alfred P. Sloan Fellow in Neuroscience, McNight technological innovations in Neuroscience, NIH Javitz Neurosceince award, and Siegel Family Award for outstanding biomedical research. Now, Dr. Ryan is a Rockefeller/Sloan-Kettering/Cornell Tri-Institutional Professor at the Weill Cornell Medical College and an HHMI Senior Fellow at the Janelia Research Campus in Ashburn Virginia. His lab works in unravelling how synaptic transmission is controlled. They develop novel quantitative optical tools that give us information about “hidden variables” i.e., aspects of cell biology and physiology that play crucial roles in determining synaptic properties but have not been readily explored. For example: how local bioenergetics and metabolism impact synaptic functioning; how the local action potential waveform is; what the role of the endoplasmic reticulum in the axon is. Dr. Ryan will elaborate during his talk, for which he has kindly provided a synopsis: “The brain is a metabolically vulnerable organ: interrupting the fuel supply leads to a rapid and drastic decline in brain function. By developing an approach to measure the concentration of ATP in nerve terminals, we showed that synapses represent one of the likely loci of the brain’s vulnerability. We discovered that synapses to not store sufficient ATP molecules to carry out function when activity increases and must therefore synthesize ATP on demand locally. In the last several years we have worked out some of the mechanisms that link activity to ATP production: blocking these regulatory pathways leads to rapid block of synapse function as if they were provided no fuel. I will describe our current work unravelling both the mechanistic aspects of this regulation as well as new clues we have obtained regarding the bio-energetic costs of different aspects of presynaptic function.”

The cost of thinking and not thinking: metabolic control of synapses.

Talk on May 11th 4-5pm PST

Tim Ryan. PhD

Weill Cornell Medical School, New York, NY

The human brain is a metabolically vulnerable organ. A decrease in blood glucose of only a factor of 2 leads to rapid manifestations of neurological symptoms including delirium and coma. This sensitivity to hypometabolic conditions implies that fundamentally neurons themselves do not tolerate brief interruptions in fuel supply.  To investigate the molecular underpinnings of the metabolic control of neuron function we developed reductionist approaches in the last several years to examine the interface of metabolic and synaptic function. Our work has shown that nerve terminals are likely loci of the brain’s metabolic vulnerability, as they do not store sufficient ATP to sustain function and must synthesize this critical biochemical currency on-demand in response to electrical activity. Our most recent work now shows that in addition, nerve terminals have very high-resting metabolic rates, that are independent of electrical activity. This high resting synaptic metabolism in turn determines, in part, synaptic performance in hypometabolic states.

Synapse adhesion molecule Neuroligin-3: Linking genetics to oxytocin in autism

Dr. Peter Scheiffele is a professor at the Biozentrum, University of Basel, where his group studies the molecular mechanisms underlying formation of neuronal circuits both in health and diseases such as autism. Dr. Scheiffele’s pioneering work on how trans-synaptic signals such as neuroligin and neurexin promote synapse formation and stabilization has led the field for over two decades. Currently, the group is interested in alternative splicing as a generator of the molecular diversity underlying synaptic specificity, as well as the role of autism risk factors such as neuroligin-3 (Nlgn3) in social behaviors. In the former domain, among other discoveries, the Scheiffele group found RNA-binding protein SLM2 as a highly specific controller of glutamatergic synapse plasticity: correction of one exon target of SLM2 could restore plasticity and behavioral defects in Slm2 knockout (KO) mice. In the latter domain, the Scheiffele group’s continuing work on mice deficient in Nlgn3 in ventral tegmental area (VTA) dopaminergic (DA) neurons has connected genetic autism risk factor Nlgn3 to oxytocinergic signaling, with the common thread of translation regulation and plasticity.

In work published in Nature in August 2020, Hörnberg et al. from the Scheiffele group both identify Nlgn3 in VTA DA neurons as a key mediator of autism-related social behaviors and find a small molecule that can reverse these deficits of Nlgn3 KO mice. The key findings of this work rest on a behavioral assay of social recognition in juvenile mice (postnatal day (P) 26-28) and electrophysiology on acute midbrain slices in the same juvenile mice. For the behavioral assay, the authors wanted to recapitulate interactions characteristic of some patients with autism, which include lower scoring on face identity recognition tasks. In the task Hörnberg et al. used, experimental mice were repeated exposed to the same same-sex mouse four times, after which they were exposed to a novel same-sex mouse (Fig 1a). During the last interaction, the experimental mice were scored for time spent interacting with the novel mouse (Fig 1b). For the electrophysiology, the authors were interested in the firing rate of VTA DA neurons at baseline, with oxytocin in the bath, and with experimentally treated mice; for this they recorded spontaneous currents under a voltage-clamped cell-attached setup.

In their initial characterizations, the authors found that for Nlgn3 KO mice, VTA DA oxytocin response is altered at both cellular and behavioral levels. In wild-type (WT) mice, hypothalamic neurons release the peptide oxytocin in the VTA, and this increases the firing of VTA DA neurons projecting to the nucleus accumbens (NAc). This circuit is thought to underlie social behaviors such as social novelty response and reinforcement. Hörnberg et al. found that constitutive Nlgn3 KO mice had a decreased social novelty response (Fig 1b), that this effect was specific to VTA DA neurons (Fig 1h) (behaviors could be recapitulated with selective miRNA KD of Nlgn3 in VTA DA neurons), and that this effect could be reverse with selective re-expression of Nlgn3 in the VTA DA neurons of Nlgn3 KO mice (Fig 1e). Thus, Nlgn3 in VTA DA neurons is both necessary and sufficient for social novelty response. Furthermore, acute slice electrophysiology showed that Nlgn3 KO VTA DA neurons had lower baseline firing rate (Fig 1p) as well as no increase in firing rate induced by oxytocin in the bath (Fig 1q).

Figure 1: VTA DA neurons lacking Nlgn3 have altered social novelty and oxytocin responses

The authors then sought to understand the mechanism underlying Nlgn3’s effect on oxytocin response, in the hopes of finding a target for therapeutic restoration of social novelty response. Interestingly, both metabolic labeling studies as well as shotgun proteomics on VTA DA neurons showed dysregulation (specifically, an increase) in mRNA translation. This effect on translation is similar to that observed in other models of autism such as Fragile X Syndrome. Drawing from this literature, the authors selected MAP kinase-interacting kinases (MNKs), mediators of signaling-dependent changes in mRNA translation, as potential targets. The authors found that the highly specific MNK inhibitor ETC-168 not only crosses the blood-brain barrier, but also, when dosed orally in mice, increases the baseline firing of VTA DA neurons (Fig 2b), increases VTA DA neurons’ responsiveness to bath oxytocin (Fig 2c), and increases social novety response (Fig 2e). Given that the electrophysiology results were obtained in slices two hours after the last ETC-168 dose, the known linking of mRNA translation to neuronal plasticity, and recent research on the importance of plasticity in VTA DA neurons, the authors hypothesize that MNK inhibition restores Nlgn3 KO oxytocin responses via a plasticity effect.

Figure 2: MNK inhibitor ETC-168 restores social novelty and oxytocin responses in Nlgn3 KO mice

The above work finds a link between a genetic autism risk factor, oxytocin signaling, and social behaviors, and it further connects these factors with the finding that a therapeutic small molecule can reverse cellular and behavioral oxytocin response deficits in a genetic model of autism. More generally, the findings also suggest that common features of autism—such as dysregulated translation and synaptic properties—might be targeted as a way to overcome the genetic heterogeneity within autism spectrum disorders. This most recent advance highlights the breadth and trajectory of Dr. Scheiffele’s work: with understandings derived from basic research into the mechanisms underlying the development of neurons, we can accelerate development of therapeutics of diseases previously thought to be intractable.

To learn more about Dr. Scheiffele’s research and recent work, please join us for the talk, “Building functional circuits: From RNA splicing to Autism”, on Tuesday, April 20th at 9am on Zoom.

https://uchealth.zoom.us/j/81547662915

—

References

Hörnberg, H., Pérez-Garci, E., Schreiner, D., Hatstatt-Burklé, L., Magara, F., Baudouin, S., Matter, A., Nacro, K., Pecho-Vrieseling, E., and Scheiffele, P. (2020). Rescue of oxytocin response and social behaviour in a mouse model of autism. Nature 584, 252–256.

Scheiffele, P., Fan, J., Choih, J., Fetter, R., and Serafini, T. (2000). Neuroligin Expressed in Nonneuronal Cells Triggers Presynaptic Development in Contacting Axons. Cell 101, 657–669.

Traunmuller, L., Gomez, A.M., Nguyen, T.-M., and Scheiffele, P. (2016). Control of neuronal synapse specification by a highly dedicated alternative splicing program. Science 352, 982–986.

—

Dr. Scheiffele’s Website

https://www.biozentrum.unibas.ch/research/researchgroups/overview/unit/scheiffele/related-to-prof-peter-scheiffele/

—

James Deng is a first year PhD student in the Neurosciences Graduate Program at UCSD. He is a member of Nicola Allen’s group at the Salk Institute for Biological Studies, where he studies the role of astrocyte signaling and astrocyte-secreted proteins in neurodevelopmental disorders.  

Untangling the Brain: “Motor Cortex Embeds Muscle-like Commands in an Untangled Population Response”

Dr. Mark Churchland is an Assistant Professor in the Department of Neuroscience and the co-director of the Grossman Center for the Statistic of Mind at Columbia University. The Churchland lab studies how movement is produced by neural activity. His groups examines, using a computational approach, how motor commands might be produced by recurrently connected networks with strong internal dynamics.  

Typically, when we examine neural data, we plot the activity of neurons against time. This allows us to examine how individual cells respond in nuanced ways for a given context, but tells us little about how neural activity might fluctuate or how network level activity might be structured. Another way to look at the same data is to plot a trajectory that moves through a high dimensional space, where the number of dimensions corresponds to the number of neurons. For example, if you knew the activity of three neurons, you could plot a point in three dimensional space that represented their coactivation: the position of the point along the x-axis would represent the firing rate of neuron one, the position of the point along the y-axis would represent the position of neuron two, and so on. Points representing the activation of these three neurons can be plotted sequentially, producing a trajectory through the three dimensional space. This process can be generalized to numerous dimensions, allowing one to study the coactivation of populations of neurons, but rapidly becomes difficult to effectively visualize.

Principal components analysis (PCA) is a dimensionality reduction approach that extracts “components” form a high-dimensional space that capture the largest amount of variance within that space. Applied to high-dimensional data, PCA can extract dimensions along which neuron firing rates cofluctuate together. Through PCA, one can reduce a high-dimensional neural dataset to a coarse representation in three dimensional space that captures the majority of cofluctating activity between neurons. This allows one to effectively visualize a trajectory that evolves over time through this three dimensional space that best represents the population’s activity.

Reframing neural analyses though this lens, Dr. Churchland untangles novel population dynamics that support movement in his 2018 paper, “Motor Cortex Embeds Muscle-like Commands in an Untangled Population Response” (Russo et al.). In this paper, they trained two rhesus macaque monkeys to perform a cycling task where they had to cycle forwards or backwards through a virtual landscape to obtain a reward. During the trials, they took intramuscular EMG recordings and single neuron recordings in the cortex. They examined the population dynamics using PCA to identify the dominant signals in multi-dimensional data. Plotting the top two principal components yields a state-space trajectory, where each point on the trajectory corresponds to the neural state at one moment. As we can see in Figure 1, the neural and muscle trajectories behaved differently, suggesting that the neural activity in the cortex is encoding more than direct muscle activity.

 

Figure 1. Visualization of Population Structure via PCA

The populations of neural activity and muscle activity were compared by calculating their trajectory tangling (Figure 2). They found that muscle trajectories had higher levels of tangling than neural trajectories. This may be because muscle activity is driven by external commands rather than internal feedback dyanamics, and therefore the state of activity of a muscle at one point in time does not necessarily predict the future state.  

Figure 2. Illustration of the Trajectory Tangling Metric

To explore potential effects of low tangling, they trained a neural network to  to generate a simple idealized output with an additional parameter employed to reduce tangling (Figure 3). Networks with low tangling were more noise robust. 

Figure 3. Low Trajectory Tangling Aids Noise Robustness and Can Be Leveraged to Predict the Motor Cortex Population Response

Finally, leveraging the hypothesis that muscle-like commands are embedded in additional structure that yields low tangling, they we were able to predict motor cortex activity from muscle activity.

To learn more about Dr. Churchland’s research and recent work, join us for his talk “Primate motor cortex exerts detailed control of muscle activity”  this Tuesday, 4/13/2021, at 4pm on Zoom. 

References

• Dr. Mark Churchland’s lab website: https://churchland.zuckermaninstitute.columbia.edu/

• Russo, A. A., Bittner, S. R., Perkins, S. M., Seely, J. S., London, B. M., Lara, A. H., Miri, A., Marshall, N. J., Kohn, A., Jessell, T. M., Abbott, L. F., Cunningham, J. P., & Churchland, M. M. (2018). Motor Cortex Embeds Muscle-like Commands in an Untangled Population Response. Neuron, 97(4), 953-966.e8. https://doi.org/10.1016/j.neuron.2018.01.004

Written by Blanca Martin-Burgos, a 1st year student in the Neuroscience PhD Program at UCSD.

Unifying brain, intelligence, life, and everything

Dr. Karl Friston, one of the most influential neuroscientists in the world, is a Professor of Neuroscience at University College London. According to the Google Scholar Citations public profiles, he ranked the 20th with 267346 citations and h-index 232 among most highly cited researchers. In 1990, he invented statistical parametric mapping, one of the most fundamental and essential techniques used in functional magnetic resonance imaging. The corresponding software SPM – the latest version SPM just released in 2020 – is the most commonly used software for fMRI data analysis. Similarly, another tool he developed in 2003, called dynamic causal modeling, is a standard analysis method to understand the coupling among brain areas. In the past decade, he is actively and ambitiously developing and contributing to the unified framework of theoretical neuroscience – the free-energy principle.

The free-energy principle is proposed to provide a unified global account of action, perception, and learning, claiming that a self-organizing system, at equilibrium with the environment, must minimize its own free energy, resisting a tendency to disorder. In order to survive, the long-term goal of an agent is to maintaining homeostasis and low entropy. Correspondingly, the short-term goal is to avoid surprise. Imagine seeing a fish out of the water! That’s an example of an emotionally and mathematically surprising event. Keeping homeostasis is excellent, so just maintaining surprise low by minimizing free energy (an upper bound on the surprise)!

Free energy is the function of two accessible variables: sensory states and a recognition density. Recognition density is an internal probabilistic representation of the causes of a particular sensation. To reduce the free energy, an agent can make two kinds of changes on the world or itself (see Figure 1b): change sensory inputs by acting on the world (so that prediction errors are minimized and accuracy is maximized), or change the recognition density by updating their internal states (so that divergence is maximized and representations are optimized). Along this way, the whole brain is formalized as a Bayesian machine with active inference. Evolving for active inference, the brain should possess the corresponding anatomical structure. A proposal of perceptual inference in the thalamocortical network is shown in Figure 2.

Figure 1. A description of the free-energy principle.

FIGURE 2. The anatomy of perceptual inference.

Want to learn more about Dr. Karl Friston’s recent work? Go to the NGP seminar series talk on Tuesday, March 16, 2021, at 10:00 a.m. Enjoy your life as an inevitable and emergent property of any ergodic random dynamical system that possesses a Markov blanket!

Reference

Wiki of Karl Friston: https://en.wikipedia.org/wiki/Karl_J._Friston
Friston K. The free-energy principle: a unified brain theory?[J]. Nature reviews neuroscience, 2010, 11(2): 127-138.
Ramstead M J D, Badcock P B, Friston K J. Answering Schrödinger’s question: A free-energy formulation[J]. Physics of life reviews, 2018, 24: 1-16.
Parr T, Friston K J. The anatomy of inference: generative models and brain structure[J]. Frontiers in computational neuroscience, 2018, 12: 90.

Ji-An Li is a first year grad student currently rotating with Dr. Mikio Aoi, working on the eigenspectrum of neural manifolds of visual cortex.

Dissecting Frog Love Songs: The Role of Premotor Neurons in the Evolutionary Divergence of Frog Courtship Calls

Dr. Darcy Kelley is the Harold Weintraub Professor of Biological Sciences at Columbia University, where her lab studies the neurobiology and evolution of social communication through their investigation of vocal signaling in African clawed frogs, Xenopus. With a repertoire of vocal signals and responses produced in a manner dependent on the sex of the signaler, the sex of the recipient and the social context, the Xenopus model allows the Kelley lab to address the fascinating question of how the nervous system produces and responds to social signals. Applying electrophysiological techniques to isolated brains from various Xenopus species, researchers in the Kelley lab are able to record fictive vocal behavior—defined as patterns of vocal nerve activity that correspond to in vivo call patterns—as a means of interrogating the neural circuitry governing vocal communication and its evolution.

In a recent collaboration between Dr. Kelley and the lab of Dr. Erik Zornik, a candidate neuronal mechanism for the vocal evolution between two species of Xenopus, Xenopus laevis and Xenopus petersii, was identified in a critical premotor hindbrain nucleus, the Xenopus parabrachial area (PB_X). Important background for this finding is the species-specific nature of male Xenopus courtship vocalizations. While the male courtship calls of both X. laevis and X. petersii include trains of ~60 Hz sound pulses called fast trills, fast trills in X. laevis are longer and lower-pitched compared to those in X. petersii. Seeking to determine whether species-specific tuning of premotor neurons could contribute to these behavioral differences in courtship calls, whole-cell patch-clamp recordings were collected from PB_X neurons in dissected brains from X. laevis and X. petersii.

Application of serotonin to the PB_X elicited fictive calling from premotor vocal cells, and through comparison of important features of these fictive calls, the researchers identified two distinct functional groups of premotor neurons: early vocal neurons and fast trill neurons. Representative responses to current injections from fast trill neurons and early vocal neurons are illustrated in Figure 1A and 1C, respectively, with responses from X. laevis on the left and X. petersii on the right. Importantly, as shown in Figure 1B and 1D, only fast trill neurons were found to have voltage-dependent membrane currents that varied between the species. Corresponding to the duration of courtship calls in X. laevis versus X. petersii, the burst duration of fast trill neurons was long in X. laevis and short in X. petersii, suggesting that the intrinsic differences between fast trill neurons in these two species may encode behavioral differences observed in their courtship calls.

Figure 1. Fast trill neuron spike bursts in X. laevis were significantly longer than in X. petersii at medium and high current levels.

Next, the authors applied tetrodotoxin in order to examine NMDA-induced membrane potential oscillations from premotor vocal cells in the absence of synaptic connectivity. As shown in Figure 2, fast trill neurons displayed NMDA-induced oscillations that differed significantly across species, such that membrane oscillations observed for X. laevis were characterized by a longer depolarization duration (Figure 2B) and period (Figure 2C) when compared to X. petersii. No such interspecies differences in NMDA-induced membrane oscillations were observed in early vocal neurons (data not shown).

Figure 2. In fast trill neurons, NMDA-induced oscillation durations and periods were significantly longer in X. laevis than in X. petersii.

Together, these findings strongly suggest that the duration and period of fast trills in X. laevis and X. petersii are encoded by fast trill neurons in the PB_X. The authors also argue that the differentiation of fast trill neuron membrane properties likely contributed to the divergence of Xenopus courtship calls during evolution. Given the importance of the PB in vocal and respiratory control across vertebrates, these findings may also inform our understanding of the evolution of vocal patterns in other vertebrates.

To learn more about Dr. Kelley’s research and recent work, please join us for her talk, “How do neural circuits for vocal communication evolve?”, on Tuesday, March 9th at 4pm on Zoom.

References

Barkan CL, Kelley DB, & Zornik E (2018). Premotor neuron divergence reflects vocal evolution. Journal of Neuroscience, 38(23), 5325–5337. 10.1523/JNEUROSCI.0089-18.2018

Dr. Kelley’s Website: https://kelleylab.biology.columbia.edu/

Anousheh Bakhti-Suroosh is a first year PhD student in the Neurosciences Graduate Program at UCSD, interested in the neurobiological mechanisms underlying the development and treatment of neuropsychiatric disorders. She is currently rotating in the lab of Sreekanth Chalasani at the Salk Institute for Biological Studies.

Reveal your HAND: Uncovering the role of chronic inflammation in the pathophysiology of HIV-Associated Neurocognitive Disorder (HAND)

With doctoral training in microbiology and immunology and a postdoctoral background in the Department of Neurology, Dr. Amanda Brown bridges her expertise in the two fields in order to better uncover the pathogenesis of HIV-associated neurocognitive disorder (HAND). It has been shown that 20% of untreated individuals infected with HIV can have severe neurocognitive symptoms that manifest as gait abnormalities, memory impairment, and encephalitis.  Though antiretroviral treatment has greatly reduced severe neurocognitive comorbidities due to HIV infection, neurocognitive functional deficits still exist, afflicting about 50% of infected individuals. Thus, the Brown lab is interested in knowing whether the low-level persistence of the virus even with antiretroviral treatment leads to chronic inflammation in the central nervous system (CNS). This chronic inflammatory activation, the lab hypothesizes, might possibly result in neuronal damage and neurodegeneration which present in patients as cognitive deficits. The ultimate goal of the Brown lab is to elucidate the critical genes and pathways by which the immune response to HIV infection leads to these neurocognitive symptoms. By filling this gap in knowledge, the results will contribute to possible targeted therapies that will protect the CNS and improve the quality of life for patients with HIV.

Dr. Amanda Brown’s lab specifically focuses on the macrophages which permit the virus to continually replicate and thus incite the proinflammatory state.  

A profound finding from the lab includes that of cortical neurons as a source of the inflammatory cytokine, osteopontin, which has been shown to be elevated in the cerebrospinal fluid of patients with HAND as well as other neurological disorders like multiple sclerosis, Alzheimer’s, and Parkinson’s to name a few. The lab examined the levels of the cytokine using immunohistochemistry comparing between brain tissue of uninfected controls versus that of amyotrophic lateral sclerosis (ALS), a neuroinflammatory positive control, that of HIV-positive individuals without cognitive symptoms, and of HIV-positive cases with cognitive symptoms. Using paraffin sections, the lab was able to show that microglia-associated osteopontin levels were significantly increased in ALS and HAND cases compared to controls.

This finding is indicated in this figure from Silva et al. where a and b are controls, c is an ALS case, d is an HIV positive case without cognitive symptoms, and e and f are HAND cases. The red and brown colocalization indicates osteopontin expression in Iba1-positive microglia, the brain tissue resident macrophages. It is pertinent to note that all HIV positive cases were on antiretroviral therapies, yet this pro-inflammatory cytokine remains at higher levels.

This study also found osteopontin expressed not only in microglia in the brain, but also in cortical neurons, suggesting a multifactorial source of inflammation in the brain.

To build on such interesting findings, the lab is currently using in vitro culture to examine osteopontin effects on HIV-mediated neurotoxicity as well as mouse models of HIV infection to investigate signaling upstream of osteopontin.

Learn more about Dr. Brown’s work and the interface between immune and central nervous systems by attending her talk on March 1^st at 4pm as part of the UCSD Neurosciences Seminar Series!

References

“Amanda Brown, PhD”. Johns Hopkins University School of Medicine. Retrieved 28 February 2021. https://neuroscience.jhu.edu/research/faculty/150

Silva, K., Hope-Lucas, C., White, T. et al. Cortical neurons are a prominent source of the proinflammatory cytokine osteopontin in HIV-associated neurocognitive disorders. J. Neurovirol. 21, 174–185 (2015). https://doi.org/10.1007/s13365-015-0317-3

Dr. Brown’s website: https://abrownhopkinsresearch.com/

Iris Garcia-Pak is a first-year PhD student in Dr. Richard Daneman’s lab studying the role of the blood-brain barrier in modulating brain function and behavior.

Getting the Genes to Fit: Advancing Treatment of Parkinson’s Disease.

Parkinson’s Disease (PD) is a devastating disorder for which there is very little treatment. A progressive illness that primarily targets the movement-related dopamine neurons in the substantia nigra (SN) area of the brain, PD slowly depletes them (Fig 1). Symptoms include slowed movement, tremor, impaired posture, rigid muscles, and an impairment and eventual loss of automatic movements. By the time these symptoms are seen in patients, the loss of dopamine cells in the SN is usually quite severe. Current treatment for PD focuses on supplementing the lost dopamine, which is beneficial at the start but will lose effectiveness over time and does not cease disease progression. Experimental therapeutics for PD have focused on disease modification and structural neuroprotection. While noble efforts, trials of these methods have all failed.  

Dr. Jeffrey Kordower, the Alla V. Solomon Jesmer Professor of Neurological Sciences at Rush University and one of the foremost researchers in PD treatment, believes the answer to successfully treating PD lies in cell replacement strategies and gene delivery therapy. Dr. Kordower is particularly poised to address this goal. He is a leader in the field of movement disorders with a specialty in experimental treatments for neurodegenerative disorders. He was a founding member of The Michael J. Fox Foundation Scientific Advisory Board, and has published over 400 peer reviewed papers relating to movement disorders, particularly PD. His group’s breakthrough findings have included revolutionary treatments for individuals with Parkinson’s such as that fetal dopaminergic grafts can survive, innervate and form synapses in patients with Parkinson’s and that gene delivery of trophic factors can preclude PD pathology in non-human primate models. Currently, Dr. Kordower and his lab are focused on cell replacement strategies for treatment of PD and the use of focused ultrasound to deliver gene therapy to specific brain targets. 

Figure 1. Pathology of Parkinson’s Disease

To treat PD, reasonable targets have to be identified. Current efforts for the treatment of PD are in trophic factors as gene therapy such as glial cell-line derived neurotrophic factor (GDNF) and neurturin, which promote healthy cell growth. Unfortunately, outcomes from PD clinical trials with trophic factors have been disappointing. Dr. Kordower and colleagues have been able to identify the core reasons PD gene treatment may be failing. These include not enough coverage by the gene, poor retrograde transport within the cells that display transgene expression, and treatment during the course of disease. Studies have shown that gene delivery only reaches a small area of the intended target and once the gene is delivered to the cells, they do not express enough of it to be effective. Once PD symptoms can be clearly defined, it may be too late for gene therapy to work properly. Depending on when a diagnosis is given, dopamine transport fibers can be almost entirely gone and there is not enough substrate for successful transfection.

Dr. Kordower and colleagues believe that the solution to the problems presenting gene treatment of PD lie in focused ultrasound and nanovectors. This approach uses intravenous injection of ultrasound contrast microbubbles followed by focused ultrasound pulses to the brain region of interest. The application of ultrasound pressure waves causes the microbubbles to disrupt tight junctions and promote transport of drugs across the blood-brain barrier. The blood-brain barrier is a highly-selective border that prevents most substances from reaching brain tissue. By using a gene-delivery nanovector small and non-adhesive enough, GDNF is able to be delivered across the blood brain barrier in the ultrasound focal region and into the tissue where it is desperately needed. This method is able to successfully address the reasons why gene therapy for PD may have failed thus far. Dr. Kordower is at the forefront of the push for targeted therapeutic delivery with ultrasound, and it may not be too far in the future that his work makes it possible for PD to be a much less fatal disease. 

References

Marks WJ, Bartus RT, Siffert J, et al. Gene delivery of AAV2-neurturin for Parkinson’s disease: a double-blind, randomised, controlled trial. Lancet Neurol 2010;9:1164–1172.

 Bartus RTT, Kordower JH, Johnson EM Jr., et al. Post-mortem assessment of the short and long-term effects of the trophic factor neurturin in patients with α-synucleinopathies. Neurobiol Dis 2015;78:162–171.

Kordower JH, Olanow CW, Dodiya HB, et al. Disease duration and the integrity of the nigrostriatal system in Parkinson’s disease. Brain 2013;136:2419–2431. [PubMed: 23884810]

Kordower, Jeffrey H., and Robert E. Burke. “Disease modification for Parkinson’s disease: axonal regeneration and trophic factors.” Movement Disorders 33.5 (2018): 678-683.

Price, Richard J., et al. “Parkinson’s disease gene therapy: will focused ultrasound and nanovectors be the next frontier?.” Movement disorders: official journal of the Movement Disorder Society 34.9 (2019): 1279.

Dr. Kordower’s Lab Website: https://www.rushu.rush.edu/research/departmental-research/neurological-sciences-research/laboratory-jeffrey-h-kordower-phd

Mayo Clinic Parkinson’s Disease Resource: https://www.mayoclinic.org/diseases-conditions/parkinsons-disease/symptoms-causes/syc-20376055

Lauren Stanwicks is a 1st year PhD  student in the Neurosciences Graduate Program at UCSD interested in how dopamine modulates internal states. She is currently jointly rotating in the labs of Tim Gentner and Vikash Gilja. Meet her on twitter @neuronlauren