Tuesday, March 15, 2011

Deb Roy on wordling junior's life

Stumbled upon a fascinating longitudinal data-intensive, visualization-intensive series of ongoing work at Deb Roy's group at the MIT Media Lab which prompted me to share some quick thoughts. Watch it here:



The talk is brilliantly structured: starting with an emotional moment, walking through some stunning data visualizations, transitioning into a pitch for bluefinlabs (his startup) with more chutzpah, and ending with the personal yet transcendental.

Scientifically, the work itself, IMHO is meant to be treated as a glimpse into the kinds of hypothesis that can be tested, rather than a definitive statement on language acquisition patterns in children. Further, I've only watched the TED talk and 18 minutes is not enough time to point out caveats, that too obvious ones. I imagine that among the various controls, they would have in fact segmented and separately treated utterances of 'water' by adult to adult vs. by adult to child, since implicit and explicit learning presumably have different mechanisms. But I'll refrain from speculating further without digging deeper here.

Technically, managing parallel feeds of audiovisual data does not seem straightforward in the least. Further, segmenting humans from cluttered fisheye scenes or target words from natural speech, and 100 TB of it, despite 50 years of AI research, is still a pretty big deal. Besides, it's the first graph on a TED talk with error bars! Respect!

Commercially, social media is one big ticket application, but couldn't the same infrastructure be applied to support decisions in interviews or boardrooms, or evaluations in high end schools or creches? What else?

But what captured my fascination the most was the power of big data to accelerate discovery at an unprecedented rate.

On large datasets and fishing expeditions

Scientists are just beginning to appreciate the power of trawling the world for extremely large datasets and subsequently testing various hypotheses on small subsets of the data. This approach (sometimes derogatorily called a fishing expedition) is in stark contrast to classical scientific method where an apriori hypothesis dictates an experiment design and an analysis procedure. The LHC experiment is one such expedition to fish for postulated elementary particles.

To illustrate the power of the fishing expedition approach, imagine if Deb had decided apriori that he wanted to study word utterance length over time. In a conventional longitudinal experiment he might have chosen a subset of words used in natural conversation, asked several child and caregiver pairs to come into a studio for 1h/day and recorded their speech. He might then have analyzed the data, observed this U shaped phenomenon, and reported it in a journal about language acquisition, where it would have promptly gathered dust. Even potentially interested colleagues might think several times before replicating the study with a different set of words, or correlating the word utterance length with spatial context, since acquiring funding and approvals for a longitudinal study would be prohibitive. Instead, with Deb's fishing expedition dataset which might become publicly available someday, any armchair scientist with computational resources can ask their own creative follow up questions of the data with minimal entry barrier!

However, fishing expeditions have their downsides. First and most obvious perhaps is the risk involved: what if there are no fish to be found? In other words, what if a generic experiment design is not powerful enough to eliminate alternative hypotheses until the ones being tested are left standing? Second, what if something is found but it is hard to tell with any confidence whether that something is in fact, a fish? Put differently, does the subset of data relevant to a particular hypothesis being tested (such as a correlation between two variables) have enough statistical power to falsify its corresponding null hypothesis? Last (and perhaps the hardest to spot), fishing expeditions may encourage scientists to operate in a complete vacuum of hypotheses (as opposed to designing for a multiplicity of hypotheses).

What are some other fishing expeditions and their successes and failures? How is the Human Genome Project different from the LHC experiment?

Wednesday, October 27, 2010

Quality poverty in Indian higher education

Came across an interesting essay by Ashok Jhunjhunwala, championing quality in Indian higher education.

Here is his paper:
http://rtbi-iitm.in/Ashok/education_link.html

He argues that quality is lacking in higher education primarily because of underpaid teaching staff, thus making the vocation unattractive for the best and brightest. He claims that teachers can only be better paid (salaries tripled) through increasing fees since government budget for higher education cannot be doubled overnight. Then, he does a lot of analysis for building the case of increasing fees while keeping access open, through financial instruments and strong regulation to prevent profiteering institutions.

An interesting aspect, that he touches upon but does not elaborate, is the fundamental conflict of interest between primary/secondary education and higher education! Improving the quality of primary eduation will increase the demand for higher education, because lower drop-out rate and better exposure will mean that more teenagers are college ready. This would triple the current levels of 3 million new college entrants and put a strain on supply or quality of higher education.

I agree with him about how underpaid university lecturers are, and how difficult it is for academics in the west to even consider a return to Indian institutions. But it seems to me that he misses out on three potential pieces to the puzzle, that can shift the burden from the end user (i.e. fee paying students) to some other intermediary parties.

First, enabling third parties to enter the training and educational content domain can decrease the teacher:student ratio (e.g. 1:20 --> 1:40), enabling more students to pay for fewer teachers' wages.

Second, allowing lecturers to supplement their income through consulting gigs made easier through stronger industry-university relations can decrease the strain on the institutional wage bill. It will also offer short-term rather than long-term return for indian industries, making more companies willing to participate. Lecturers can also be given sufficient freedoms to be intrapreneurs in this regard, by identifying and creating partnerships in their field of expertise.

Third, western universities that depend to a large extent on their supply of graduate students from India, might be incentivized to fund entry-level university education in India. In this context too, university lecturers can be paid to create formal programs of student/researcher exchange, creating an alternate income stream.

Monday, May 24, 2010

Reading wishlist


BookBox: embed book widget, share book list

Chaturashrama as a metaphor for personal and professional management

Chaturashrama, an ancient Hindu subtext (part of the Manusmriti) describes the four ideal stages of a human's (man's) life. Indeed, Chaturashrama has received a lot of flak for being patriarchal, for being the source of much determinism in modern Indian society, and for being But this post is not about such existing debates.

The four stages (or ashramas) are Brahmacharya (the stage of learning and preparation for life), Grihastha (the stage of taking responsibility and acquiring material, and emotional wealth, thus performing one's core evolutionary duty), Vanaprastha (the state of learning to let go of wordly comforts and spreading wisdom) and Sanyasa (the stage of isolated contemplation). Implicit in the Chaturashrama is that the path to each stage is through the previous. For instance, one cannot learn to let go in the right way unless one has been in complete control. Thus, these four stages represent four key functions of a person's journey through life: learning, taking control, letting go, and reflection.

While the four stages are prescribed for different periods in life, the modern world is seldom so linear. I'll proceed to claim that each stage is constantly present in our lives, and further, that each stage individually can provide both positive and negative feedback on the other. I'll argue that mastering the balance between these constraints or tensions can result in unprecedented success (spiritual, emotional, and or financial).

To be successful in the modern world, any entity (be it an individual direct the course of his/her personal life or a large organization) must continually cycle through these four functions every waking moment. To elaborate, one must continually
  • learn (update oneself about the state of the world)
  • take control (make decisions based on the knowledge gathered and take responsibility for the consequences)
  • trust and let go (be able to delegate responsibilities to those who step up and trust them to do it, as well as move on from unexpected failures)
  • reflect (create new knowledge through reevaluation and contextualization of own experience and state of the world).
Seen thus, experiences from personal life have direct parallels in management. In fact, several management functions can be (and have been) organized according to these criteria. Given that one is always playing the balancing act between these functions, it is worthwhile asking when they are in conflict with each other, and when they reinforce each other.

The G-V Balance, or how to DO well. Lessons in executive excellence
Mastering the tension between taking control and letting go (Grihastha-Vanaprastha balance), is a trait that can be seen in the best leaders of the world.

The B-S Balance, or how to THINK well. Lessons in strategic excellence
Likewise, closing the loop between learning and reflection (Brahmacharya-Sanyasa balance) can been seen in some of the world's best thinkers. Here learning refers to casting a wide net for ideas and knowledge and having the humility and curiosity to learn from anyone, while reflection refers to the courage to think on one's own reject good ideas at times, the vision to separate bad ideas from good ones, and the capacity to synthesize new knowledge.

[Footnote: Couldn't resist, but excellent thinking is mostly about knowing your way around B-S!]

The D-T Balance, or how to THINK by DOING and DO by THINKING
While the G-V balance deals with the issue of how to be a good doer [in management world, an executive; in the scientific world, an experimentalist], the B-S balance deals with how to be a good thinker [in the management world, a strategist; in the scientific world, a theorist]. And doubtlessly, acquiring each balance is a lesson in acquiring the other. By extension, it is possible to imagine that one always pursue excellence in thinking through excellence in doing and vice versa, leading us to study a third balance: the thinking-doing balance.

Mastering each of these balances is a book in itself and may be the subject of future posts!

Friday, May 21, 2010

On crossing the chasm and reshaping the adoption curve

Moore's bestseller, Crossing the chasm talks about the diffusion of innovation through the technology adoption distribution as though it is a static distribution of people ranging from early adopters to laggards. The book presents a framework to study the dynamics of adoption of a certain idea (or more often, commercial product) and helps us understand its spread from being niche meme to mass hysteria to slow fade out.

Marketing-to-geek ideologies suggest that innovators, rather than making 'average products for average people' (to paraphrase the ideologies of giants such as UniLever and P&G from the mid-20th century) should target the early adopters and recruit their faith and passion to help cross the chasm.

In reality however, the adoption distribution is not static. There is a constant traffic of people from one part of the distribution to another. Laggards sometimes open up, and tend to become mainstream adopters. In other situations, mainstream adopters can become early adopters. People evolve into some ideas, and devlove into others. This flow in turn may be steeply dependent on a few irreducible parameters. I wonder how much this kind of flow has been modeled or studied in behavioral economics, for instance.

Given this assumption of a parametrically varying adoption distribution, it is possible to conceive of a new marketing strategy, which would come up with methods prepare or pre-seed the adoption distribution by tracking the right parameters which control people flow, so that we have inherently more early adopters than mainstream users or laggards. The chasm would not be one static point in the distribution somewhere between the early adopters and the early mainstream, but shuffle around depending on this people-flow across the distribution. We could call such a marketing strategy: Where is the Chasm now?

It is my gut feeling that general purpose initiatives which aim to inspire people and make them take a proactive, curious, experimental stance to life such as TED, would play a very strong role in making the chasm move left, and would thus become a more and more integral aspect of marketing functions in insitutions worldwide.

Friday, April 30, 2010

The third thing about fame

Among the many assertions that have been made about fame are the following. Fame brings wealth, renown and a great social standing. Aspiring to be famous is for the selfish and power-hungry. Fame breeds arrogance, laziness and corruption. Etc. But are there only those two sides to fame?

Perhaps a nuanced assertion that has been made less often is the following. Aspiring to fame is a deep willingness and open invitation to be judged by the world. It is a complete submission, and a handing over of the right to be criticized, praised and talked about, to everybody in the world, regardless of their qualification or emotional stake in passing judgment. Aspiring to a public life is thus an aspiration to live in complete and utter openness. I'll argue here that there is a right way, a noble way, to aspire to be renowned, that fame is not always base. Let's call this fame 3.0 for simplicity.

Fame 3.0 is very close conceptually to ideas from several places. For instance, it is very close to the the Bhagavad Gita's "actions > rewards" prescription, or the stress of process over results from the enterpreneurial school of thought.

And it's probably fame 3.0 being confused for fame 1.0 or 2.0 that got Jesus crucified or Martin Luther King assassinated.

Wednesday, April 7, 2010

December season's reflections of a midsummer escapade

What if Garcia-Marquez had been brought up a Tam Brahm? That is the subject of this experimental fiction piece.

---------------------------x----------------------------

Gopal-Giridhar Madhusudhan looked out of the window, idly, about to roll over and go back to sleep, only to recall the promise he had made to himself this time: to get more out of the trip. It was his seventh visit back to Chennai in as many years. Each December, he had returned, if not from a faithfulness, then from a simple, acknowledged, fear of loneliness in the darkest weeks of the year.

Over Gopal’s first few visits, he had not wanted much, content to laze around in his parents’ home, get driven around, and generally fed the middle-class version of a dream vacation. In later years, he became a more conscientious vacation planner. Once, he accumulated culture-vulture oomph meticulously, attending every lec-dem session in the December sabhas, downing cups of filter coffee with dhonnais of steaming hot kesari in the mornings. Another time, he collected greenie points from wildlife trips. From a Valmik Thapar rant, through discussions with a friend about RFID collars for endangered species and the ethics of it, he had chalked out and executed a trail through Sariska. Yet another time, he sought free-spirit points by doing unplanned road trips through a randomly chosen region, performing informal case studies of microeconomic behavior. This year again, he had set himself such a goal. He would later realize that each vacation’s goal could succinctly be summarized thus: to get a sufficient dose of ‘wholesomeness’ out of the December visit to last him through the next sterile winter.

---------------------------x----------------------------


Gopal sat down, pen in hand, fresh coffee by his side, lighting to his satisfaction. He wrote:

Creativity is not a faucet.

That same tired witticism he had picked up from a Calvin and Hobbes comic strip. Not very creative. This was not some e-mail to a junior.

Lately, Gopal had been exploring the effects of the environment and elaborately designed mood scenarios on writing quality, the null hypothesis being that no morbid or weak thoughts could be forthcoming from the nourished, fresh-coffee-equipped soul, on a satisfactorily-lit Chennai morning. He wrote again:

"I'll give you 30 seconds in heaven", she said. "What, literally?" he retorted, faking a nervous laugh, by now. "Yeah, my name is heaven". He was snubbed.

No, no that’s not right. That’s terrible. He was snubbed? Snubbed? Staccato. Faking a nervous laugh? More like trying too hard and not enough altogether at once. No flow. My name is heaven?! Stand-up comedy? He scratched it out and tried again.

“I’ll give you 30 seconds in heaven”, she said. “What, literally?” he retorted, trying on a macho, assured laugh. Wrong move. The ‘Wh’ came out high pitched. He kicked himself internally, but it showed and she laughed. In future recountings of the incident to himself, he would always stress that her laughter was knowing and amused, a laughter of the eye, an ‘I get you, and that’s cool’ signal rather than an exposing, sarcastic laughter. “Yeah,” she replied with a blank stare, quickly checking her impulse to flash a smile, “my name is heaven”. Her future recountings of this moment were fond, and slightly self-congratulatory on account of her spontaneous naughtiness.

Better already. The drama was unfolding neatly. But the punchline had moved too far away for a reader to make the connection. But at this point Gopal checked himself. Too much academic writing, reader simulations, paragraph conjunctions. Humbug. More importantly, this was far from morbid. He seemed to be deviating from the task at hand. Was the null hypothesis undeniable? He tried again. Directness with two esses.

He had fucked a whore, bought himself a ticket to intimacy, barely conscious of the possibility of future remorse. He would later learn to describe this moment as a signing away of his 'claim to have always strived for the greatest good'.

Big deal. That came out just preachy, not morbid.

The aesthete in him made him cringe. The coffee had lost its steamy nip and the sun had risen too high. Imperceptibly at first, but unmistakably, he felt crippled. This was his third day in a row attempting to sketch the same scene. It was going to be yet another sultry day of inaction.