24 March 2013

Musing on the mind-brain problem

During a couple of recent conversations with friends and family I wrote a short summary of my current views on the mind-brain problem. The dry jargon of scientific research reports is an obvious obstacle to a generally satisfying account of the conscious self, so I try to not be dry. This is just my current perspective, I'm not providing references and I reserve the right to be wrong.

I think it's important not to think of the brain as 'just a bunch of cells', but rather as a hundred billion individual identities that want to live and grow. The ancestors of the cells of the brain were free agents; swimming, creeping, crawling, swirling their way through the waters of ancient earth; feeding, resting, sensing, fighting, fleeing, multiplying. Now they're here, living together in this civilization we call brain, but they are still feral. In a very real way they rival and tussle each day to stay alive. The neurons of the brain do not grow old and die like other cells; you have almost entirely the same brain cells now that you had when you were a child. However, tens of thousands of them are wiped out every day - only the ones that form important constellations and alliances with other neurons receive neuromodulators and grow; others shrivel and fade. Neuromodulators, what Gerald Edelman calls 'value systems', are essential to the life and growth that brain cells seek. And here is the essential fact: neuromodulators are released in the brain in response to meaningful events of various kinds, happenings internal or external that bear on the interests of the body, the person, the brain as a whole, or one of its neural communities. The brain cells, seeking neuromodulators, seeking life and growth, are therefore in constant electrical communication and structural flux, seeking to bring about, probe and explore the meaningful, important aspects of the reality in which they find themselves.

What are these aspects of interest, of meaning? What happenings attract the complexity and potential of a living human brain cell? To start with, every neuron is in constant electrical union with the sensing, moving body, and they communicate and grow about this vital fact. They share a common path through life, so they explore and probe their shared memories constantly for nuggets of intrigue. Although their communities are often in tension as each continues the ancient will to live on and grow against the daily weeding out of the least relevant of them, they nevertheless share a common mouth, a common pair of hands and eyes, and the electrical urges of hunger and need and sleep and dreams reverberate across the neural fields endlessly. How could they not share a sense of I in this circumstance? From this seeking, seething, astronomically complex swirl of electric neural energy, membrane and will to survive and grow emerges pleasure and frustration, I and not-I, hopes, plans, dreams and distinctions. The astounding communities and constellations of living, electrified tissue that constitute each of these core features of the human experience are there for us to explore, by any method we choose - introspective, statistical, fictional, spiritual, communal - and it is our tremendous fortune and grace to be alive just as the technology to express and understand all of this fully is finally beginning to become available. But at the heart of it all is an ancient electric civilization of cells, seeking nourishment and love and excitement and force, in a never-ending myriad of ways. This is what it is to be alive, a conscious human being. Electricity is key; the near-instantaneous sharing of the tremendous depth of memory, will and rich experience among these one hundred billion little lives within that one skull. It is in their nature to seek, like their cousins still independent in the sea, but the cells of the brain seek in communication and structural union with other brain cells. The subject of this electrical conversation is and feels like you.


25 December 2012

Coupled neural attractors

Towards the end of my PhD I made a second audiovisual rendering of feeding circuit data (the first is quite different and is available here)


For those of you looking to do something similar, this blog post explains how it's done (mainly Matlab). Code and data are available here: neuralAttractor.m, nineSpikeRates.txt. Suggested improvements are welcome too.


Data

The input data for the video is 1000 seconds of spike time data from nine neurons that has been converted into smooth normalized spike rates. The data was recorded from two bilateral triplets of motorneurons and one triplet of 'extra-CPG' neurons that adapt the feeding behavior of the six motorneurons to the availability of food (see Harris et al. 2012 for details).
data = load('nineSpikeRates.txt');

Video

The video is made entirely in Matlab. Use this line to initialize and set the length of the movie
videoFrames = moviein(length(data)-10); 
Then, for t = 1:length(data)-10, run
dataSegment = data(t:t+9, :);
figure(1)
plot3(dataSegment(:,1), dataSegment(:,2), dataSegment(:,3), 'o', 'Color', 'r', 'MarkerSize', 30);
hold on
plot3(dataSegment(:,4), dataSegment(:,5), dataSegment(:,6), 'o', 'Color', 'r', 'MarkerSize', 30);
plot3(dataSegment(:,7), dataSegment(:,8), dataSegment(:,9), 'o', 'Color', 'b', 'MarkerSize', 30);
axis([0 1 0 1 0 1])
grid on
videoFrames(t) = getframe(1);
close(1)
This creates and saves multiple 3D plots showing successive 10 second segments of neuron triplet activity.

Finally, to save the video, use this line
movie2avi(videoFrames, 'videoOut.avi', 'compression', 'none', 'FPS', 10);

Audio

To play music in Matlab I use the Java library JFugue. Download the library jfugue-4.0.3.jar, or whatever is the latest version, from jfugue.org. Use the following lines to make the library available to Matlab. Note that the javaaddpath command doesn't like to be executed as part of a script - you may get an IMPORT error. To get around this you can run the javaaddpath line manually before you run the script (this is possibly a newbie error on my part).
javaaddpath('jfugue-4.0.3.jar')
import org.jfugue.Player
import java.io.File
player = Player()
 
The JFugue player takes a single long string as input. Each note is represented by an integer between 0 and 127 enclosed in square brackets (JFugue in fact supports a very wide range of musical commands, like 'Cmaj'). For example, try:
notes = '[48] [50] [52] [53] [55] [57] [59] Cmaj'
player.play(notes)
JFugue can play several notes simultaneously. Each channel or 'voice' is indicated by V1, V2 etc. You can specify an instrument for each channel, e.g. I0 for piano, I24 for guitar etc. You'll also want to specify a tempo for the entire string, using T[beats per minute]. For example:

notes = 'T[120] V1 I0 [48] [50] [52] [53] [55] [57] [59] [60] V2 I24 [60] [59] [57] [55] [53] [52] [50] [48]'
player.play(notes)

To generate the audio for this video I initialized the JFugue string notes = 'T[600] ' and ran the following for n = 1:3
notes = [notes horzcat('V', num2str(n), ' I11 ')];
for t = 10:length(data)
   note = round(mean(data(t,[1 2 3]*n)*30+30));
   if note > 30
      notes = [notes horzcat('[', num2str(note), '] ')];
   else
      notes = [notes 'R '];
   end
end
This takes the mean spike rate of each neuron triplet at each time step (t) and converts it to a bracketed integer between 30 and 60 (a mid-pitch range) unless the spike rate is zero, in which case 'R ' is added, which JFugue interprets as a pause. t starts from 10 here so that the sound played at each time step will correspond to the front of the 10 second segments of activity used to generate the video frames.

Use this to save your audio as a midi file
filePath = File('audioOut.mid')
player.saveMidi(notes, filePath)

Finalize

Now you need to merge video and audio. As far as I know there's no way to do this in Matlab. JFugue only outputs midi files and unfortunately Windows Movie Maker doesn't accept midi, so if that's what you're planning to use to merge you'll need to convert the midi to wav first. I use Switch Sound File Converter for this. You may need to play around with the number of beats per minute at the head of the JFugue string to get audio that's the same duration as the video, especially if you're using many data points per second. If you play the YouTube movie above to the end you'll see that I didn't get this completely right.

An example of the video file I get after merging videoOut.avi with audioOut.wav (this is audioOut.mid converted to wav) in Windows Movie Maker is available here. Good luck.

23 August 2012

What is reward?

So I recently passed my PhD viva and got a paper published (whoop whoop!). The titles of the two texts are ‘Multi-electrode analysis of pattern generation and its adaptation to reward and ‘Multi-neuronal refractory period adapts centrally generated behaviour to reward. That last word, and my use of it in the texts, caused a fair amount of trouble. I na├»vely thought it would be OK to leave the term undefined, seeing as we’re still working out how the brain’s reward system operates. For example, the observation that midbrain dopamine neurons are sometimes activated by stimuli purely by virtue of those stimuli being new and unexpected (rather than appetizing, sexual etc) suggests that novelty itself might be thought of as a reward. And anyway we all have a fairly good intuitive understanding of what constitutes a reward, right? Wrong. I need to define reward.

Here’s what we ended up writing in the paper following extended skirmish with reviewers:
"We will refer to… a ‘reward’ in the general meaning of a stimulus that promotes approach and consummatory behaviour rather than the more specific meaning of an unconditioned stimulus used as a positive reinforcer in a classical or operant long-term conditioning paradigm." (Harris et al., 2012)
I'd like to contrast this definition with Wolfram Schultz's, who writes:
"A reward is any object or event that generates approach behavior and consumption, produces learning of such behavior, and is an outcome of decision making." (Schultz, 2007)
Schultz's second condition, that rewards produce learning of approach behaviour and consumption, begs the question: does this refer to conditioning proper, in which memory persists long after the reward is removed, or does an effect on short-term memory suffice? For example, is a food object rewarding merely by virtue of inducing a high and sustained feeding rate, or must it also increase the probability that similar food objects be eaten in the future? This question has physiological consequences: both classical and operant conditioning require brief bursts of spikes in midbrain dopamine neurons (Tsai et al., 2009; Kim et al., 2012), whereas the rate and intensity of ongoing behaviour, and the stability of working memory representations, are regulated by the tonic concentration of dopamine, which is set by the number of dopamine neurons engaged in slow pacemaker firing at any given moment (Niv et al., 2007; Cools & Robbins, 2004). In fact, Schultz's definition of reward does require persistent memory formation, i.e. bursts of dopamine. I disagree. I think a stimulus-induced increase in the rate and intensity of approach and consummatory behaviour can be thought of as a reward-response regardless of whether it produces lasting behavioural change. Yael Niv has for example argued convincingly that the average rate of reward over time modulates tonic background concentrations of dopamine, and thereby adapts the rate and intensity of foraging behaviour (Niv et al., 2007). There are many indications that this extends also to non-food rewards. This view is also in accordance with Norman White's, who writes that rewards are stimuli that elicit approach behaviour whereas reinforcers induce memory consolidation (White, 1989). Roy Wise similarly notes that 'priming' is an important effect of rewards, but one which does not find its way into long-term memory (Wise, 2009).

Schultz's third condition, that rewards be the outcome of decision making, is also problematic. If this condition is taken to mean that a reward must be the consequence of an overt motor behaviour, as many people would argue, then two objections follow. First, cases of classical conditioning where a neutral stimulus is paired with for example food, producing a subsequent preference for the neutral stimulus, do not involve any overt motor behaviour or action and so cannot according to this definition be said to involve reward. This is in stark contrast to numerous papers that describe such experiments as 'classical reward conditioning' and the food stimuli used as rewards. Second, say you give a hungry rat a food pellet, either at a randomly chosen time or as a consequence of the rat wandering into a pre-defined part of the cage. Do we really want to say that the pellet is a reward in the latter case but not in the former? Physiologically there will be no difference: the dopamine burst response and its effect on synaptic plasticity will be the same. Isn't it in fact the case that brains are always in the process of  deciding how to act, and operate by responding to correlations between their own activity states (be they sensory- or motor-states) and varying concentrations of dopamine? Whether or not a reward is in fact the causal outcome of a decision is irrelevant from the perspective of the brain.

In light of all this, I would suggest the following new definition of reward:
A reward is an object or event that induces approach and consummatory behaviour, and produces short- or long-term learning of that behaviour.
The lack of reference to rewards necessarily being the outcome of overt decision making constitutes a deviation from the way the term reward is used in everyday language (for example, an unexpected tax-return is a reward according to this new definition), but not, I think, from the way many scientists use the term. One might argue that such stimuli should be referred to as 'non-contingent rewards', but, at least in the case of the term 'reinforcement', this approach appears only to have complicated matters (Poling & Normand, 1999). Maybe then, we should drop the term reward entirely, and use 'positive stimulus' instead? However, this term has the serious disadvantage of not being a verbal noun. That is, whereas everyone understands the noun 'reward' and the associated verb 'rewarding', there is no established understanding of the (compound) verb 'positively stimulating' that is associated with the (compound) noun 'positive stimulus'. If anything, 'positive' has optimistic or ethical connotations that would jar with the amoral and downright destructive topics often discussed in relation to reward, such as addiction. The term 'appetitive stimulus' (and 'appetitively stimulating') avoids this problem but implies a focus on satisfying bodily needs, particularly hunger, whereas the key property of reward is that it can apply to any desire or goal.

Have I missed something; some word with the same meaning as 'reward' but better able to match the physiology? Is it time to make up a new word? If not, then I would suggest we stick with reward, using the definition above, accepting it as a slight neologism. The lack of a requirement that a reward necessarily be a consequence of overt decision making or motor behaviour should be appropriately tempered by the understanding that in fact the vast majority of rewards do occur as a consequence of decision making and motor behaviour - specifically as the result of exploration, trial-and-error, or more complex goal-oriented behaviours.


And here we see the fuck yeah monkey upon his mountain of treats!
(The treats are all rewards provided the monkey has an appetite)
(h/t Austen)

15 July 2012

Finding reward in the zebrafish brain

It looks I'll be working a lot with the zebrafish model over the next couple of years, so I've been trying to get my head around its dopaminergic reward system. I was hoping for some clear homologies with the human/mammalian reward system, but the situation is more complex.

Overall, fish and mammals have similar brain topologies. The fish forebrain pushes forward during development rather than wrapping itself around the lower brain regions as it does in mammals ('eversion' rather than 'evagination'). The fish pallium is nevertheless homologous to the mammalian cortex, with distinct sensory and motor regions, although in fish a disproportionate amount of visual processing takes place in the tectum, a homologue of the mammalian superior colliculus (Salas et al., 2003). Likewise, the subpallium of fish corresponds to the mammalian basal nuclei, including the striatum, the key recipient of dopaminergic reward in the mammalian brain. So far so good; most forebrain structures involved in reward processing appear to be conserved among vertebrates, and cognitive abilities previously thought of as exclusive to 'higher' animals (i.e. birds and mammals) are now being studied also in fish (Salas et al., 2003).


However, the fish dopamine supply is all over the place, quite literally. Dopamine neurons are found throughout the zebrafish brain, except for the midbrain, where almost all mammalian dopamine neurons are located. A few dopaminergic clusters in the hypothalamic region project to the subpallium/striatum and were previously thought to be homologous to the mammalian mesolimbic dopamine system, but more recent research has debunked this view (Schweitzer et al., 2011; Tay et al., 2011). There simply is no mesencephalic dopamine system in the zebrafish brain. Nevertheless, fish are capable of both classical and operant reward conditionning (Valente et al., 2011), including dopamine-dependent place preference, and even intracranial self-stimulation (Boyd & Gardner, 1962), so what gives?

(Figure adapted from the Zebrafish Brain Atlas)

As far as I can tell, dopaminergic reward mechanisms remain remarkably poorly understood in the zebrafish, despite intense research in recent years on the neurobiology and genetics of this model system. Most of the dopamine in the zebrafish subpallium/striatum appears to originate in local dopaminergic projections from neurons whose cell bodies are distributed throughout the subpallium/striatum (Tay et al., 2011). These neurons look like plausible mediators of reward, but their input, physiology and function remains unknown(!). The function of the ascending dopaminergic fibres that project to the subpallium/striatum is also not known. Moreover, pretectal dopamine neurons arborize extensively in the tectum, suggesting a possible role in visually guided reward-seeking behaviour, such as hunting.

Plenty of reward-related research to be done in other words, but what do we make of this? Hills (2006) argues that the evolution from anamniotes (fish and amphibians) to amniotes (reptiles, birds and mammals) involved a number of changes regarding dopamine and reward-processing, including:

  • The number of cortical imputs to the striatum increased significantly
  • The number of dopaminergic inputs to the striatum increased significantly
  • The synaptic machinery that allows dopamine to modulate cortical input to the striatum expanded to include DARPP-32
  • The dopaminergic signal transitioned from representing the presence of food to representing the expectation of reward more generally

As a consequence of these changes, Hills argues, amniotes were able to apply the neural mechanisms of foraging (e.g. 'area-restricted search', the ancestral function of dopamine, present even in worms and mollusks (Barron et al., 2010)) to search for any kind of information or goal, whether internal or external to the brain; a profoundly powerful adaptation. In addition to the four changes suggested by Hills I would add:

  • Dopaminergic cell clusters became centralized in the midbrain

This centralization, together with some specific adaptations, such as gap junctions connecting dopaminergic axons, allowed amniote brains to generate a single, scalar reward signal that adjusts dopamine concentrations homogenously throughout the forebrain.

I think what I need to ask now is: how does the more ancient dopamine reward system of fish actually work; what forms of reward-processing is it capable of; and does its distributed anatomy offer any advantages to the animal or to attempts to understand the neural basis of reward-based cognition?

10 July 2012

Back in Stockholm

So.. my return to blogging ran into some snags. Had to publish a paper, write my thesis and leave the UK, all in a few months, ergo blogging never got enough juice. But that's all done now: the paper will appear in PLoS ONE shortly and the thesis grew to a reasonable 119 pages in the end. I'm back in Stockholm for the time being but assuming the thesis defence goes alright I'll be starting a postdoc in the US in October. In the meantime I've got loads of reading to do, the highlights of which I'll try to summarize here.

Until next time



14 April 2012

Top 10 podcasts

Bloggingheads    |    Hour long one-on-one, usually about current affairs. Updates 2-4 times a week. Look out for episodes with the founder, Robert Wright, they are excellent.

Slate's Political Gabfest    |    Every Friday. Editor of Slate Magazine David Plotz, with Emily Bazelon and John Dickerson. The number one podcast on US politics.

In Our Times    |    From BBC Radio 4. Every Thursday morning Melvyn Bragg interviews three guests about anything from Edward Munch to Quantum Gravity. Undisputed master of the History of Ideas, only occasionally challenged by Entitled Opinions.

This Week in Google    |    Wednesdays. Great show. TWiT-founder Leo Laporte discuss the current state of cloud computing Gina Trippani, Jeff Jarvis and a guest. The Gillmor Gang covers the same beat with a different tone, less polished, tends to dig deeper, not always to great effect. This Week In Tech, the 2-hr flagship show of the TWiT network, available Monday mornings, is sometimes good but often has too much of the everything's-a-joke TWiT humour.

Friday Night Comedy    |    BBC Radio 4 pokes fun at the news every Friday evening. The excellent News Quiz alternates with the less funny Now Show every couple of months (we're currently in a News Quiz phase).

Global News    |    Also from the BBC. Updated twice daily except weekends, including at 2-3 in the morning,  which means that the truly podcast-addicted can always start the day here. Somewhat obsessed with interviewing random people about whatever plight they're in.

The Economist    |    More current affairs. The 'All Audio' edition updates sporadically throughout the week whereas the 'Editor's Highlights' is a one hour block of readings from the latest issue of the magazine released every Friday.

Triangulation    |    Near-weekly one hour show with Leo Laporte and Tom Merrit. Great stuff if the guest is good and fortunately they manage some seriously good guests, including Steve Martin, Kevin Kelly, Ray Kurzweil, Jeri Ellsworth, the list goes on, check the archive.

Start the Week with Andrew Marr    |    Monday morning. Hit-and-miss, too much artistic sentiment but occasionally very good (Monday morning, you take what you can get). Nearly got bumped off the list by Great Lives, also from the BBC.

FourCast    |    Futurism from the TWiT network, updated once or twice a month. Only just started listening to this. A bit too light-hearted for my taste but with Changesurfer Radio seemingly out of the loop a futurist has to get his fix somewhere. Futures in Biotech, also from the TWiT network, works too.