Change of seasons

The late October Sun smiled weakly today, like a devout caregiver who has been strong for too long, and is unable to hide his waning strength any longer. "Stay strong without me", he said, in an unsuccessful attempt to inspire fortitude during his absence. A yellow, perforated, autumn leaf fell to the matted brown floor lined with its recently deceased kin --- apologetic, for having overstayed its welcome, and in quiet acceptance of its fate. A solitary gull, now devoid of its cacophonous bravado that the summer warmth had inspired merely months ago, circled the Lehtisaari bridge in silent anticipation of the inevitable.

In Helsinki, the change of seasons is an everyday affair. Starting tomorrow, it's time to switch back the clocks and prepare for yet another winter.

How can the history of gravitational theory inform modern day neuroscience?

We've all heard it said before. Neuroscience is the physics of the 21st century. The vast ocean of unknowns lies before us, we've just learned to build a vessel, we've just learned to navigate. Let's unfurl the sails, go forth and discover new lands!
In this essay, I'll attempt to simplify such portentous omens by first prying open physics and then neuroscience.

The 20th century atomic physicist, Ernest Rutherford is famous for having said, "That which is not physics is stamp collecting". Let's indulge Rutherford for now, as we try to understand what he meant by physics. But first, let's start with a brief history of studying planetary motion.

Stamp collecting

Tycho Brahe, the Danish astronomer is most famous in the popular imagination for having observed the first modern supernova. However, a lesser appreciated fact about Brahe is that he extensively and systematically documented the trajectories of planets (#1).


In just a generation, along came Brahe's assistant, Johannes Kepler. Kepler deduced from all of Brahe's accurate observations that the trajectory of planets could be explained by three simple laws or regularities. First, by carefully making calculations of the Martian orbit, he noticed that all (known) planetary orbits were in fact elliptical, as opposed to the conventional Copernican belief before him that they were circular. He further noticed that the Sun was centered at one of the foci of the ellipses. Second, he noticed that the line joining the planet and the sun swept out sectors of equal areas along the orbit. Since orbits were elliptical, this implied that planets moved with variable speed---again a radical departure from Copernicus! He published both these observations in 1609 [I'm suprised he didn't write two papers on these potent mythbusters]. Third, he noticed that the square of the orbital period was proportional to the cube of the major axis of the elliptical orbit. He only published these 10 years later in 1619 [I wonder how he got tenure]. These are now known as Kepler's laws of planetary motion (#2).

Would Rutherford call Kepler a physicist? Would you call Kepler a physicist?

If we define a physicist narrowly as someone who engages in deductive reasoning based on observations of the natural world, then Kepler was not a physicist. However, if we recognize that a physicist must wear multiple hats along the journey from making observations (stamp collecting) to reasoning about them (physics a la Rutherford), then I would call him an exemplary physicist. Let's get more granluar about Kepler. What was he? Using modern terms, I would call him an applied statistician or more specifically, a curve fitter.

Kepler basically looked at the orbital trajectories and said, "wait a minute! this doesn't look circular to me. Hmmm...". He then selected a functional form to represent the trajectories (the equation of an ellipse) and simply fit the parameters to the data, the parameters being the foci and the major/minor axis of the ellipse to the orbit. Next, he looked carefully at the non-uniform speeds of the planet during the course of its revolution around the sun and said to himself, "Hmmm, what needs to be equal in order that the speeds can be unequal?". Finally he graphed the orbital period against the major axis length and mumbled, "There's a pattern here but I don't quite get it. Could it be a power law!". And it was.

In doing thus, Kelper had reduced the painstakingly detailed data recorded by Brahe into three simple regularities or laws. He made an epistemological innovation, i.e. he told us how to organize our observations neatly, in much the same way that Darwin organized species into the tree of life or Mendeleev organized the elements into a periodic table. Kepler's laws could now make accurate predictions about planetary motion.

In the parlance of modern statistics, we could interpret Kepler's laws as a descriptive statistical generative model. It described the statistics of planetary motion by generating them from underlying regularities. However, for all its genius, Kepler's work was merely descriptive, i.e. it succinctly answered the what questions but not the why questions. Why were planetary orbits elliptical? Why did they move faster when they came closer to the sun? For these answers, we had to wait another 100 years.

Newton and the why questions

Newton first formalized the concept of a force acting between two bodies. He postulated and verified the laws of motion. In posulating the gravitational force, and observing that the force was inversely proportional to squared distance, the universal theory of gravitation took shape.

Newton's theory was a causal explanation of the data observed by Brahe and the regularities captured by Kepler's laws: i.e. it could now answer the why questions. Just to take one example: as the planet comes closer to the sun, more force acts upon it, causing a greater acceleration increasing its speed!

I hazard a guess that Rutherford would have included Newton into his elite definition of a physicist!

Marr's three levels modified

From that prelude into the history of gravity, it is interesting to note that the answers to why questions also lead to how questions. How is gravitational force transmitted between two bodies without a medium? For nearly three centuries after Newton, a number of proposals were made for the mechanical explanation of gravity, all of which are known to be wrong today. The current explanation is attempted by quantum gravity, a theoretical framework that attempts to unify gravity with the other three fundamental forces, but as of today, we don't have a mechanistic explanation of gravity!

The sequence of what-why-how questions brings us to David Marr, a 20th century vision scientist and AI researcher, who postulated three necessary conditions for a computational theory of sensation or perception. Marr and his contemporaries conceived of vision as an information processing system. He said, to have a computational theory of a system, we need to understand:

1. The computational level: what computation/ task does the system intend to perform?
2. The algorithmic level: how does it represent this computation/ task and what strategies does it adopt to achieve its goal?
3. The implementational level: what is the precise sequence of steps in the physical wet brain during the execution of the above algorithm?
   

Now, the universe is not an intentional system with a well defined goal, so Marr's postulates are more suited to building nature-inspired computational systems for solving specific tasks, rather than describing nature. Let's slightly modify (#3) Marr's levels to fit our what-why-how framework:

1. What happens in the visual areas of the brain? [This is the stamp collecting task of Brahe]
2. Can we build a simple statistical model that captures its regularities and predicts some of its dynamics? [This is the applied statistics / descriptive modeling task of Kepler]
   
3. Why is it happening i.e. what is the brain trying to achieve through the observed dynamics? [This is the causal modeling task of Newton]
   
4. How does it go about achieving its goal? [This is the mechanistic modeling task of quantum gravity]
   

Whither neuroscience?

With parables from Brahe down to Quantum Gravity, along with perspective from Marr, is it possible to meaningfully contextualize the need for theory in neuroscience as a discipline?

Let's take the primary visual cortex and see whether we can analyze developments about its understanding using the above framework.

The work of Brahe and (to a large extent) Kepler was done by the early greats (1960s onwards): David Hubel and Torston Wiesel. These guys measured from single neurons in the cat V1, recognized that there were cells selective to things like orientation, spatial frequency, ocular dominance, etc. [a Brahe task]. Next, along with Horace Barlow and other contemporaries, they explained natural images as constituted by oriented edges and gratings [a Kepler task]. Barlow and his contemporaries also attempted to give causal or normative explanations of what the visual cortex was doing. They proposed that the visual cortex was efficiently coding (in the Shannon information sense) the retinal image [a Newton task]. A little later, descriptive statistical generative models of natural images were proposed using Fourier and Gabor basis functions. The models were successful in describing the retinal image in terms of a few regularities [a Kepler task]. With the advent of artificial neural network models, it became possible to take the efficient coding hypothesis one step further and build mechanistic models of neural activity which efficiently represented the retinal image. It was possible to show mechanistically that efficient coding was realized by performing decorrelation in a distributed neural network to achieve this efficiency [a quantum gravity task]. With the advent of overcomplete basis functions: robustness, not just efficiency of visual information representation could also be normatively explained [a Newton task]. With further advances in natural image statistics by Olshausen and Field, it became clearer that besides efficient, decorrelated and robust coding of the retinal image, a key function of the visual cortex was to learn and update hypotheses (Bayesian posteriors) about the statistics of natural images [yet another Newton task]. How is Bayesian inference mechanically realized in a neural network? Again, artificial neural network models have been postulated for the same [yet another quantum gravity task].

One positive example does not make a theory, you quite rightly say (#4)? Ok. As we look around in other areas of neuroscience, what do we see?

I see that on a day to day basis we are all organized as cottage-industries and guilds, learning through apprenticeships, how to solve Brahe tasks, Kepler tasks, (and less frequently) Newton tasks, and quantum gravity tasks. Wider-scale databasing efforts [Brahe tasks] are also beginning to take place. Examples include Bert Sakmann's digital neuroanatomy project, the human connectome project (HCP), etc. Parallely, wider-scale big-data mining [Kepler tasks] is gaining ground. Examples include various connectomics projects, and contests to leverage big-data (such as the ADHD fMRI/ VBM/ DTI data analysis contest). Brahe and Kepler tasks certainly seem to be the mainstream activities of the day.

What do you see around you?

What would Feynman say to Rutherford?

The above comic strip generated some brilliant discussion in the forum. A commentor succinctly summarized this to be the everlasting tension between Rutherford's famous quote and a pithy equivalent by Feynman, attributing the following quote to the latter:

Physicists often have the habit of taking the simplest explanation of any phenomenon and calling it physics, leaving the more complicated examples to other fields.

Is neuroscience ready for Newtons, or do we still need more Brahes and Keplers for now?

Here's another gem from the forum:

Physicists who do bad biology say, "It's so simple! Look, model it so." These do not understand the concept of 'unknown unknowns.' What if, for example, your insects normally behave predictably, but release an alarm pheromone when handled clumsily - for example, by a theoretical physicist?

Physicists who do good biology say, "It's so messy! Is there any way we can control for as much of that messiness as possible? How do we pry apart this noise to get at the underlying rules?" They bring their disdain for vague claims to the field, and back up their claims with data. They aren't airy anti-biologists, but intense, experiment-driven pragmatists.

Notes

(#1) Incidentally, Brahe was not the first to catalogue planetary motion. The Babylonians, the Greeks and the Chinese each built their own MySQL servers to document the movement of heavenly bodies across the sky, with the Chinese effort taking up the most servers by far.

(#2) Interestingly, laws do not come to be known as laws as soon as they are proposed. Voltaire was the first to refer to Kepler's observations as laws in 1738, more than 100 years after they were first published!

(#3) Here's a very interesting modification of Marr's levels discussing the difficulty of studying of hierarchical with emergent properties.

(#4) In a Feynman sense, you just got physicisted!

Microeconomic complexity and development

At TEDx Chennai 2010 I had the good fortune to meet Dr. Tara Thiagarajan, who heads Madura Microfinance. Tara blogs occasionally on issues peripheral to the business of microfinance. At Physics of Poverty, with her analytical background in neuroscience and complexity theory, she dissects questions about the meaning of socioeconomic development, and strives to bring those questions to the core rather than the periphery of the microfinance industry.

Her latest post, titled Productivity Line, attempts to reconceptualize the poverty line. She suggests that instead of dividing the world into segments based on their incomes, and then agreeing upon a reasonable income as the threshold, i.e. the poverty line, why not segment the world based on economic productivity?

Her post provoked a few questions about the relationship between production and consumption both at the social and the individual level. I note these below.

Firstly, I understand and acknowledge that taking a developmental design approach is more useful and socially sustainable than an approach that hand-holds the poorest of the poor just up to the threshold of the cycle of consumption.

1. What is an ideal ratio of producers to consumers in a society so that the society is economically sustainable (let's define economic sustainability as the capacity to diversify, grow and self-renew).

2. What is an ideal ratio of consumption to production in an individual's life? One could think of this ratio as an index of Marxist alienation and therefore a proxy to measure work-satisfaction or happiness.

3. In general, how do these ratios vary as a function of population size and other demographics as well as economic complexity (let's define economic complexity as the complexity of division of labor and the diversity of goods and services produced and consumed)? For instance, in a subsistence farming society where the economic complexity is relatively low, one could imagine that the ratio of producers to consumers is one, but that is scarcely an indicator of progress. Similarly in an extremely large and diversified society the ratio of consumption to production at an individual level is extremely high, but that is not an indicator of well being necessarily.

4. If one could measure these ratios from the demographic data of target communities receiving MFI, then one could go about defining bounds on these ratios and subsequently designing developmental interventions to optimize them.

Further Reading:
David Roodman's Open Book Microfinance Blog
The Institute for New Economic Thinking
A recent article comparing metrics of economic complexity using input-output measures
The building blocks of economic complexity: A white paper that applies complex network metrics to quantify macroeconomic complexity.

Naturalistic is a fate worse than a fate worse than death

Two years ago, I had written about neologisms in science. I am finally able to give a bad example of a neologism: "naturalistic". The term I am referring to has very little to do with the ideas of naturalism in philosophy of science, or the arts, but a lot more to do with stimuli used in studies of neuroimaging. "Naturalistic" is vaguely defined as: a laboratory stimulus that is an approximation of the stimuli encountered in the natural world. So a movie would be a naturalistic stimulus. It really is a niggling issue, but I have seen a certain hesitation in the scientific community to call these stimuli "natural stimuli", and a preference towards using "naturalistic stimuli".

"Natural" itself can be defined as similar to or pertaining to nature. So, are we to understand that "naturalistic" is then "similar to or pertaining to something that is similar to or pertaining to nature"?

A fate verse zan deth, I say.

Computational neuroscience vs Neuroinformatics

Below are some hastily sketched thoughts, by no means complete, on the distinction between computational neuroscience and neuroinformatics.

Computational Neuroscience: The field posits computational candidates for mechanisms by which the brain carries out a certain function. When we say computational candidates, we loosely talk about algorithms. I think algorithms have two or more theoretical aspects. I'll try to articulate those below.

First, the goal of the function performed by the brain must be articulated by a cost function. For example, if the goal is reaching out for an object and grasping it, then the cost function could minimize muscular effort, minimize the #neurons needed to encode the task, or minimize the error rate of the task assuming that it is performed several times. Sometimes, the cost function need not describe a very specific task such as grasping, but could describe a general organizing principle of the brain - such as minimize energy consumption, minimize the use of connective tissue, etc.

Second, the process by which the cost function is optimized must be articulated, keeping in mind that such a process must be feasible in the wet brain. The wet brain provides structural and functional bounds on what a candidate algorithm can and cannot do.

With these basic ingredients, the flavours then vary because the choice of level of description can be vastly different. Someone can talk about how ion channel ratios on the cell membrane are optimal for grasping, whereas someone else can talk about why the number of cortical areas devoted to grasping the brain is optimal. Both these optimalities could be treated computationally by using selective pressures during evolution, or selective pressures during brain
development as explanatory variables. To complicate matters further, optimality in the brain can be posited at the level of evolution, brain development, learning (plasticity), and adaptation.

Neuroinformatics: The field is concerned with issues of data analysis and visualization of neuroimaging data for human interpretation purposes. The algorithms applied here (such as ICA/CCA/ridge regression etc.) need not conform to any constraints posited by the wet brain. The field does not aspire to explain how the brain performs a certain function - it just aids the process of evidence accumulation, which is of course important for theoretical and CNS because otherwise we wouldn't have phenomena to explain and our theories cannot be validated. In this sense neuroinformatics is a tool for experimental neuroscience.

Deb Roy on wordling junior's life

Stumbled upon a fascinating longitudinal data-intensive, visualization-intensive series of ongoing work at Deb Roy's group at the MIT Media Lab which prompted me to share some quick thoughts. Watch it here:



The talk is brilliantly structured: starting with an emotional moment, walking through some stunning data visualizations, transitioning into a pitch for bluefinlabs (his startup) with more chutzpah, and ending with the personal yet transcendental.

Scientifically, the work itself, IMHO is meant to be treated as a glimpse into the kinds of hypothesis that can be tested, rather than a definitive statement on language acquisition patterns in children. Further, I've only watched the TED talk and 18 minutes is not enough time to point out caveats, that too obvious ones. I imagine that among the various controls, they would have in fact segmented and separately treated utterances of 'water' by adult to adult vs. by adult to child, since implicit and explicit learning presumably have different mechanisms. But I'll refrain from speculating further without digging deeper here.

Technically, managing parallel feeds of audiovisual data does not seem straightforward in the least. Further, segmenting humans from cluttered fisheye scenes or target words from natural speech, and 100 TB of it, despite 50 years of AI research, is still a pretty big deal. Besides, it's the first graph on a TED talk with error bars! Respect!

Commercially, social media is one big ticket application, but couldn't the same infrastructure be applied to support decisions in interviews or boardrooms, or evaluations in high end schools or creches? What else?

But what captured my fascination the most was the power of big data to accelerate discovery at an unprecedented rate.

On large datasets and fishing expeditions

Scientists are just beginning to appreciate the power of trawling the world for extremely large datasets and subsequently testing various hypotheses on small subsets of the data. This approach (sometimes derogatorily called a fishing expedition) is in stark contrast to classical scientific method where an apriori hypothesis dictates an experiment design and an analysis procedure. The LHC experiment is one such expedition to fish for postulated elementary particles.

To illustrate the power of the fishing expedition approach, imagine if Deb had decided apriori that he wanted to study word utterance length over time. In a conventional longitudinal experiment he might have chosen a subset of words used in natural conversation, asked several child and caregiver pairs to come into a studio for 1h/day and recorded their speech. He might then have analyzed the data, observed this U shaped phenomenon, and reported it in a journal about language acquisition, where it would have promptly gathered dust. Even potentially interested colleagues might think several times before replicating the study with a different set of words, or correlating the word utterance length with spatial context, since acquiring funding and approvals for a longitudinal study would be prohibitive. Instead, with Deb's fishing expedition dataset which might become publicly available someday, any armchair scientist with computational resources can ask their own creative follow up questions of the data with minimal entry barrier!

However, fishing expeditions have their downsides. First and most obvious perhaps is the risk involved: what if there are no fish to be found? In other words, what if a generic experiment design is not powerful enough to eliminate alternative hypotheses until the ones being tested are left standing? Second, what if something is found but it is hard to tell with any confidence whether that something is in fact, a fish? Put differently, does the subset of data relevant to a particular hypothesis being tested (such as a correlation between two variables) have enough statistical power to falsify its corresponding null hypothesis? Last (and perhaps the hardest to spot), fishing expeditions may encourage scientists to operate in a complete vacuum of hypotheses (as opposed to designing for a multiplicity of hypotheses).

What are some other fishing expeditions and their successes and failures? How is the Human Genome Project different from the LHC experiment?

Quality poverty in Indian higher education

Came across an interesting essay by Ashok Jhunjhunwala, championing quality in Indian higher education.

Here is his paper:
http://rtbi-iitm.in/Ashok/education_link.html

He argues that quality is lacking in higher education primarily because of underpaid teaching staff, thus making the vocation unattractive for the best and brightest. He claims that teachers can only be better paid (salaries tripled) through increasing fees since government budget for higher education cannot be doubled overnight. Then, he does a lot of analysis for building the case of increasing fees while keeping access open, through financial instruments and strong regulation to prevent profiteering institutions.

An interesting aspect, that he touches upon but does not elaborate, is the fundamental conflict of interest between primary/secondary education and higher education! Improving the quality of primary eduation will increase the demand for higher education, because lower drop-out rate and better exposure will mean that more teenagers are college ready. This would triple the current levels of 3 million new college entrants and put a strain on supply or quality of higher education.

I agree with him about how underpaid university lecturers are, and how difficult it is for academics in the west to even consider a return to Indian institutions. But it seems to me that he misses out on three potential pieces to the puzzle, that can shift the burden from the end user (i.e. fee paying students) to some other intermediary parties.

First, enabling third parties to enter the training and educational content domain can decrease the teacher:student ratio (e.g. 1:20 --> 1:40), enabling more students to pay for fewer teachers' wages.

Second, allowing lecturers to supplement their income through consulting gigs made easier through stronger industry-university relations can decrease the strain on the institutional wage bill. It will also offer short-term rather than long-term return for indian industries, making more companies willing to participate. Lecturers can also be given sufficient freedoms to be intrapreneurs in this regard, by identifying and creating partnerships in their field of expertise.

Third, western universities that depend to a large extent on their supply of graduate students from India, might be incentivized to fund entry-level university education in India. In this context too, university lecturers can be paid to create formal programs of student/researcher exchange, creating an alternate income stream.