Wednesday, October 7, 2009

Unbiased but Probably an Underestimate

Over at mind hacks, Vaughan asks, why it always seems worse than you think? It turns out that its proper Bayesian statistical reasoning. The frequentist constructs unbiased estimates in the limit of large data. For finite data sets these estimates might be unbiased in some sense, but that doesn't mean that the true value is going to be just as likely to be greater than the estimate or less than the estimate. The median estimate has this property, by definition, but in most cases of interest the median is significantly higher than the mean or the maximum a posterori estimate. This seems to be generically the case when estimating quantities which are small. For example, consider two cases where we are estimating a small quantity. In one case, I sample from a Poisson distribution with mean obtained from a gamma distribution. In the other case, I assume a beta prior (uniform prior in particular) for the probability p of binomially distributed random variable. Because these are conjugate priors the posterior distributions are also gamma and beta respectively and we can compare the probability that the true value of the associated means conditioned on observing K examples (normalized by N samples in the Binomial case) is greater than the maximum a posteriori or mean estimates.




Clearly both MAP and MEAN estimates of rare events are likely to be underestimates given finite data. The same is true for ML estimates. In the binomial case a uniform prior was used. In the poisson case the results are independent of the prior... unless i made a silly error putting this together in 15 minutes :) For the binomial case, a Jeffrey's prior also seems to have this property despite heavily favoring small probabilities...

Monday, February 23, 2009

Change Reply to Address on Iphone

Annoyed that you iphone uses your gmail address as the reply to address. I was. Here's the fix. Goto settings, Mail on you iphone and deactivate your gmail account. Then click on add account. Now don't click Google mail, but instead click other. On the first page put the desired return email address in the Address area, but use your gmail password for the password slot. Then just configure your imap and smtp settings using your gmail address and password as usual. Note, the email address you put into the first slot will have to be a gmail authorized address. You can authorize an address from the settings page at mail.google.com

Thursday, February 19, 2009

Adding the Noise...

As i am burning out preparing for COSYNE i've decided to add some noise of a more pleasant variety. If you're in London and like to punctuate good music with beer and bayes please join me:

http://www.last.fm/event/862561
(this one is a maybe as some figures must be made today)


These are definites:

http://www.last.fm/event/915770

http://www.last.fm/event/908623

iPhone and Gmail Properly Configured

To make sure that things you trash on you iphone show up in your gmail trash and messages you send show up in your gmail sent folder follow these instructions:

http://5thirtyone.com/archives/862

Saturday, February 14, 2009

Better iPhone Calendar/Contacts Sync via Google

So to set this up you first should backup you contacts. The easiest way to do this is to sync you iphone contacts with google through itunes. Plug in your phone, open up itunes and goto the info tab. Select sync with google (and make sure your computer has internet access). Enter google username and password and then hit the sync button. This will upload all you contacts from your iphone to you google contacts. Once that is done follow the instructions found here to setup over the air syncing of google calendars and contacts.

Finally, open up the browser on you iphone and navagate to m.google.com/sync to select which of you google calendars will be synced with you phones calendar....easy as pie.

Tuesday, February 10, 2009

Science and Politics (and Religion)

I have been meaning to write on this for sometime now, but not until reading Pure Pedantry's response to the "Rightful Place Project" was i finally motivated to do so. This is not yet a completed version so i am still taking constructive criticisms and will be doing some editing but here it is so far:



Science isn't about separating facts from non-facts. A fact is something that is *known* to be true. Unfortunately, as Laplace said, "One may even say, strictly speaking, that almost all our knowledge is only probable; and in the small number of things that we *claim* to know with certainty, the principles means of arriving at the truth, i.e. induction and analogy, are based on probabilities."

Experimentation produces facts which take the form of raw data, D, or observations. Science then is the art of generating and the means of comparing various hypothesis, H, which might have given rise to observations. The conclusions of science can, in principle, take two forms: (1) the probability distribution on the hypothesis space given your data, p(H|D), or, more usefully, (2) a posterior predictive model,

p(D'|D,{H}) = sum_H p(D'|H)p(H|D),

which gives the relative likelihood of future observations, D'.

Now, one might be tempted to say that, if a particular hypothesis is much more probable than all others under consideration, then it is true. However, this ignores one of the most important features of the scientific process: the possibility of newer and better theories (and also novel domains of measurement). As such, I fell that it is best say that a hypothesis is never true, only better than another competing hypothesis.

But even this statement isn't terribly useful. The utility of science is in the prediction of future outcomes, the p(D'|D,{H}). Latching onto the single, "best" hypothesis can lead one to ignore outliers, "Black Swans" if you prefer, as well as overconfidence in one's predictions. This should be avoided if decisions are to be made. Since this is one of the two functions of policy makers in our society, it is important not that scientists provide "facts" but rather predictions complete with relative likelihood assessments that take into account ALL the data and a broad range of theories.

Which brings me to my view on the exact nature of the relationship between science and politics. As you may have guessed, i see this in a decision theoretic framework. A decision takes into account two things predicted outcome and utility.

a_opt = argmax_a = argmax_a sum_R sum_s R(o,a)*p(o|a,s,D)p(s|D)

where:

a_opt is the optimal action or decision

is the expected reward given an action and all the observational data.

R(o,a) is the Reward or negative cost of a given outcome, o, and action, a, taken with the intent of achieving that outcome.

p(o|a,s,D) is the likelihood of a particular outcome given action a, and state of the world s.

p(s|D) is the likelihood that the world is in a current state s, given all the observational data, D, obtained to date.

Science regulates only the probability functions which are conditioned upon the data.

Politics has two functions: The first is to compute the reward function R(o,a). In a democracy this might be accomplished by averaging over the reward functions of the people, but is most often accomplished by averaging over the reward functions that politicians have, slightly weighted by the reward functions that politicians think that the people "should" have, whatever that means. The second function of politics is then actually make the decision which is simply taking the arg max over the expected Reward.

Now in principle, politicians should not be telling people what they should value, each individuals personal ethic should be doing that. Since there is no normative solution to the problem of objective function selection (sry Rand) and reward starts with, R, I will call this religion, with a small 'r' to separate it from the kind where a clergy man tells you what to value. Thus properly understood and executed, politics should only be implementing the decision based upon an average over objective or reward functions. Thus,

politics = argmax_Decisions religion(a,o) * science(o|a,D)

it couldn't be simpler :P

Anyway, i will fill this out with an addenda regarding the funding loop later.

Monday, January 5, 2009

Sync files between you Computer and your iPhone (sort of)

New and free cloud computing service soonr allows you to designate a folder on you pc or mac which will then be available to you on your iphone or (when wirelessly connected to the internet) on your ipod touch. Its great for putting papers and presentations and maps and such onto your iphone, but unlike the (not so free) AirShare app it doesn't work when offline (sry iPod Touch users).

Anyway, to get things running, install the soonr app from the app store and setup an account through it. Then you will be emailed instructions on how to install the client software on you pc or mac. Careful though, by default it want to also backup all the files in your my documents directory, but you can turn this off during the setup.


UPDATE: So it turns out that dropbox has a reasonable mobile interface accessible through safari...not as nice as soonr's app, but just as effective. Not only that dropbox is fully cross platform supporting Linux, MacOSX, and Windows file syncing. So I am rating it the more powerful utility.