Thursday, November 27, 2008

Independence vs. Correlation

I wonder, of the many distinctions made between these statistical notions whether the following interesting one has been made, i.e., to declare two random variables independent, an observer is not required, whereas only an observer can pronounce two random variables as correlated, post-hoc. Hence, independence is a property of the distribution, (in the context of generative function as opposed to histogram), whereas correlation is a property of one single realization sampled from that distribution with respect to another such realization.

If I were Hugh Macleod, I would say at this point, "Exactly. Bayesian vs. Frequentist."

8 comments:

Pradeep Mohan said...

I guess the notion of independence could itself be defined from correlation by using a procedure similar to asking two blind men describe an elephant! I guess the whole universe is correlated and independence is just a blind man's belief! Hope I wasn't too strong!

Pavan said...

Hmm, I was referring simply to the statistical notions of independence and correlation between two random variables, whereas (if I understand right) you seem to have conjured up the debate of a determinstic vs. free universe :)

Pradeep Mohan said...

i was actually talking about the statistical notions of independence and correlation! before I came to Minnesota i used to think otherwise but my advisor made me speak like this !! Well here is a puzzle for you.. do you think my advisor is a desi ?

Pavan said...

hmm, in that case you have to elaborate on the blind man story. if i understand correctly, i suppose you are talking of conditional independence. the RVs representing the two blind men's beliefs/observations are independent, _only given that_ each is blind to the other.

the _mutual information_ between your prof. being or not being desi and the _point i am making_ is zero. however, it is certainly non-zero between the larger discourse in this thread and his desi-ness assuming he influences your views and you influence mine :) again, an issue of conditional independence (involving assumptions about the observation model)

Pradeep Mohan said...

haah... we have a paradox here!

"_only given that_ each is blind to the other" and also... there possibly exists something that they are both blind to.

Pradeep Mohan said...

btw nice posts... atleast way better than... vishal's studpid... photos and photo comments.... sometimes.... i think why don't people grow up 5 yrs of iitm + 2yrs of grad school did not change them... i still wonder whats the pt of all this if ppl don't absorb them... some ppl still seem to live in that age old PS4 days :)

Pavan said...

hmm, someone who is producing way more original and impactful work than you or me could be saying the same thing about us. anyway, nostalgia, living in the past etc. are one of several potentially evolutionary defence mechanisms. defence against a possible fate not unlike Nietzsche's last days :) anyway, thanks for looking up this space. let's stay on topic asfap :)

quoted from wikipedia:

"On January 3, 1889, Nietzsche suffered a collapse which seems to have triggered a psychotic break. Two policemen approached him after he caused a public disturbance in the streets of Turin. What actually happened remains unknown, but the often-repeated tale states that Nietzsche witnessed the whipping of a horse at the other end of the Piazza Carlo Alberto, ran to the horse, threw his arms up around the horse’s neck to protect it, and collapsed to the ground."

Pradeep Mohan said...

interesting lesson.... you seem to be more tolerant than me to things and I appreciate the sense of attachment that you have to these posts and their context... let me not disturb them :)