Newcomb’s Paradox and Bell Theorem

Today I watched a lovely presentation of Newcomb’s paradox by Norm Wildberger http://www.youtube.com/watch?v=aR5GYeZkgvY

and I could not help but notice the deep relationship with Bell theorem. In particular Bell result shows clearly the solution for Newcomb’s paradox and I am interrupting the series to present the solution. Since I did not follow Newcomb’s paradox developments, I don’t know if my take on it is new or not.

So what is Newcomb’s paradox http://en.wikipedia.org/wiki/Newcomb%27s_paradox ? This paradox was popularized by Martin Gardner and is described in his book: “The Colossal Book of Mathematics” on page 580. Let me quote its description from this book to achieve maximum clarity:

“Two closed boxes, B1 and B2, are on a table. B1 contains $1,000. B2 contains either nothing or $1 million. You do not know which. You have an irrevocable choice between to actions:

1. Take what is in both boxes.
2. Take only what is in B2.

At some time before the test a superior Being has made a prediction about what you will decide. It is not necessary to assume determinism. You only need be persuaded that the Being’s predictions are “almost certainly” correct. If you like, you can think of the Being as God, but the paradox is just as strong if you regard the Being as a superior intelligence from another planet or a supercomputer capable of probing your brain and making highly accurate predictions about your decisions. If the Being expects you to choose both boxes, he has left B2 empty. If he expects you to take only B2, he has put $1 million in it. (If he expects you to randomize your choice by say, flipping a coin, he has left B2 empty.) In all cases B1 contains $1,000. You understand the situation fully, the Being knows you understand, you know that he knows, and so on.

What should you do? Clearly it is not to your advantage to flip a coin, so that you must decide on your own. The paradox lies in the disturbing fact that a strong argument can be made for either decision. Both arguments cannot be right. The problem is to explain why one is wrong.

Let us look first at the argument for taking only B2. You believe the Being is an excellent predictor. If you take both boxes, the Being almost certainly will have anticipated your action and have left B2 empty. You will get only the $1,000 in B1. On the other hand, if you take only B2, the Being, expecting that, almost certainly will have placed $1 million in it. Clearly it is to your advantage to take only B2.

Convincing? Yes, but the Being made his prediction, say a week ago, and then left. Either he put the $1 million in B2, or he did not. If the money is already there, it will stay there whatever you choose. It is not going to disappear. If it is not already there, it is not going to suddenly appear if you choose only what is in the second box. It is assumed that no “backwards causality” is operating; that is, your present actions cannot influence what the Being did last week. So why not take both boxes and get everything that is there? If B2 is filled, you get $1,001,000. If it is empty, you get at least $1,000. If you are so foolish as to take only B2, you know you cannot get more than $1 million, and there is even a slight possibility of getting nothing. Clearly it is to your advantage to take both boxes!“

So here is the solution.  Let us call the first solution (pick only B2): trust in correlations, and the second solution (pick both B1 and B2): trust in local realism.

If you trust correlations, you obviously should pick only B2 and the video above has a nice mathematical formulation of why and when you should choose this. If you trust local realism clearly you should pick both boxes. Now here is where Bell’s Theorem comes into play. As originally written, the paradox is based on a mathematical contradiction: local realism imposes a limit on the maximum possible correlation and Bell inequalities captures precisely this limit. As stated, Newcomb’s Paradox violates Bell’s theorem.

If the correlations are higher than Bell inequality limit, this cannot be achieved with classical objects like money. Therefore the physically correct solution is to pick both boxes.

On the other hand we can introduce a variation to the problem by populate the two boxes with quantum objects instead of money. Suppose we have may identically prepared copies of the quantum objects and the “Being” perform a weak ago half measurement in an EPR-B type experiment. Then the choice for the player is to either perform the other half (and get $1 million if the correlation is above Bell limit) or to perform a random intermediate measurement (to be discarded) and then complete the other half measurement and this reduces the correlation into Bell’s limit. What happens in this case is the violation of this: “If the money is already there, it will stay there whatever you choose. It is not going to disappear. If it is not already there, it is not going to suddenly appear if you choose only what is in the second box.” because unperformed measurements have no pre-defined values in quantum mechanics.

Therefore both solutions are valid in their range of validity and the hidden contradiction in the original paradox was the incompatibility between local realism and correlations which Bell theorem explains. The original setup of the paradox is a physical and a mathematical impossibility as shown by Bell theorem.

UPDATE

I did some reading to see other people's take on this paradox, and I found a paper by Eric Cavalcanti http://philsci-archive.pitt.edu/4872/1/Newcomb_preprint.pdf who had the same idea as mine, but presented it in a physics context with emphasis on the Bayesian approach. 

After the paradox was introduced, for the two solutions of the paradox, two schools of thought developed: Causal Decision Theory and Evidential Decision Theory (see http://fitelson.org/few/few_05/egan.pdf‎)

The way to think about the paradox is to rename "Being" as "a potential Con Artist" and then assert the degree of trust into his ability to predict your mind. If the laws of physics prevents him to achieve the correlation, then pick both boxes, if not, go with the solution dictated by the Evidential Decision Theory and pick one box.

Bell theorem sets a limit on the maximum achievable correlations and Cavalcanti showed explicitly that the standard Bell analysis for local hidden variable theories (LHV) corresponding to local realism is same as the analysis advocated by the Causal Decision Theory. This being the case, Bell inequalities set up the maximum achievable correlations for local realism. 

The domain of validity of the Evidential Decision Theory is limited by the Bell limit for classical resources, or by the Tsirelson limit for quantum resources. 

In the case of this paradox, Bell limit demands p_a = p_b = 50% where p_a and p_b are the probability of "a potential Con Artist" to correctly predict your choices (see the above video for definition) 

By Bell theorem, p_a,b > 50% OR p_a,b < 50% demands that "a potential Con Artist" is definitely a "Con Artist" and if you follow the course of action suggested by Evidential Decision Theory I have a bridge to sell you. 

Is the wavefunction ontological or epistemological?

Part 3: Bell Theorem

There are several ways to understand Bell’s theorem. Let’s start with a simple but relevant coin game. Suppose two people, say Alice and Bob want to play the following game: each of them has a coin and they go in separate rooms. Then each of them flip their coin and record the outcome. Also they try to guess their partner’s experimental outcome. They win if both of them guessed right.  Suppose this game is played N times and suppose Alice and Bob agree ahead of time on a common strategy to maximize their odds of winning this games, but they cannot communicate during the actual games. What would be the best possible odds of winning the game? A natural first guess would be 25% because the coin flips are independent and each can guess right 50% of the time. However, there are two strategies of reaching 50% overall (try to figure out the solution). The 50% correlation limit is Bell’s limit (or the local hidden variable limit) in this particular case.

There are many Bell inequalities and they establish the maximum possible correlations of classical separated systems unable to send signals. However, using quantum mechanics resources one can achieve higher correlations. This can be checked experimentally and the major merit of Bell’s theorem is that it provides a way to settle the issue of the existence of local hidden variable theories by experimental means. One way to understand why quantum mechanics can achieve higher correlations is by invoking quantum superposition. But the actual root cause of superposition is the existence of Hilbert spaces which demands that the logic of nature is the logic of projections, and not the usual Boolean logic. In particular the logical operators “OR” and “NOT” are behaving strangely in quantum mechanics. To see how this happens, we need to start with the usual Venn diagrams of classical sets.

In particular, one Bell inequality (based on an idea by d’Espagnat) can be nicely explained pictorially using three overlapping circles corresponding to three sets: A, B, and C:

From the picture one can easily see that [A and not B] + [B and not C] >= [A and not C]

This is my favorite example of Bell inequality and the interesting fact is that in a particular case, Nature (and quantum mechanics) does violate it.

So how can we translate this in an actual physical system? We’ll follow one of Bell’s papers from his famous book: Speakable and Unspeakable in Quantum Mechanics (http://www.amazon.com/Speakable-Unspeakable-Mechanics-Collected-philosophy/dp/0521368693): Bertlmann’s socks and the nature of reality:

Suppose we have a source of pair particles in total spin zero as in the EPR-B experimental setting. Also suppose we measure the spin on 0, 45, and 90 degrees angles. From the inequality above:

The number of particles that could pass at 0 and not at 45

+

The number of particles that could pass at 45 and not at 90

>=

The number of particles that could pass at 0 and not at 90

Since in an EPR-B pair the spins are always anticorrelated, it is impossible to detect the left and the right particles spin oriented in the same direction. Hence we can switch the “not part” for one particle with measuring the other particle on that particular direction.

Now the inequality becomes:

The probability of one particle passing at 0 and the other at 45

+

The probability of one particle passing at 45 and the other at 90

>=

The probability of one particle passing at 0 and the other at 90

It is time to plug in quantum mechanics’ mathematical formalism and compute those probabilities. For an angle alpha between the two measurement directions, the probability is ½ [sin(alpha/2)] ^2

Plugging this in the Bell inequality yields:

½ sin(22.5)^2 + ½ sin(22.5)^2 >= ½ sin(45)^2

or

0.1464 >= 0.2500  IMPOSSIBLE!!!

And experiments confirm the violation of Bell’s inequalities precisely as quantum mechanics predicts. The explanation for nature’s strange correlations cannot be explained by any local causal mechanism. Regarding correlations, nature is nonlocal. For some time people got spooked by this, but now the emphasis is on engineering: exploiting this correlation nonlocality for practical applications.

In the next post I’ll attempt to introduce an intuition on all this and illustrate in a geometric way the Hilbert space (responsible for superposition and strange correlations).

Is the wavefunction ontological or epistemological?

Part 2: Bohm, von Neumann, and Bell

Quantum mechanics predicts only probabilities and the result of any particular experiment cannot be computed. This led Einstein to believe that quantum mechanics is incomplete. Maybe there is a “sub-quantum” world which explains the peculiarities of the quantum world and maybe quantum randomness is only a result of us “macroscopic bulls” destroying the “sub-quantum china” in the “sub-quantum shop” during measurement. This explanation is usually called a “hidden variable theory”.

Inspired by the early pilot wave theory of de Broglie, Bohm constructed a classical deterministic explanation of quantum mechanics and John Bell studied it in detail. At that time, there was popular an impossibility theorem for hidden variables by von Neumann and Bohm’s model seemed to represent a direct counterexample to von Neumann’s theorem.

As a first task, Bell proceeded to investigate the apparent contradiction and found an unjustified assumption in von Neumann’s theorem, assumption which is obeyed by quantum mechanics but may not necessarily be obeyed by hidden variable models.

The von Neumann’s assumption is as follows:

<A+B> = <A>+<B>

meaning the average of the sum is the sum of the averages for any two observables A and B.

To justify why von Neumann’s assumption is dubious, Bell noted that it means that we are comparing results from different measurements (which may correspond to incompatible experimental setups) and we are reasoning “counterfactually” which is not amenable to experimental verification. Hence the demand is ad-hoc and there is no need to be forced on a hidden variable model of quantum mechanics. Coming back to Bohm’s theory, Bell noticed that for a 2-particle system there is a very complicated dependency between particles, and he wondered if it is still possible to construct hidden variable models when imposing Einstein’s locality condition.

So Bell assumed this from the EPR paper: “On the other hand, since at the time of measurement the two systems no longer interact, no real change can take place in the second system in consequence of anything that may be done to the first system”.

The way Bell derived his famous result was using not the EPR setting, but another experimental setup proposed by Bohm (and the experiment is usually called EPR-B) of two separating electrons in a state of zero total spin. For each electron, there are deterministic hidden variable models which recover the predictions of quantum mechanics, but what can be said about correlations of measurement results? Bell showed that quantum mechanics correlations are higher than the correlation resulting from any hidden variable models, and hence, there are no locally causal hidden variable explanations possible for quantum mechanics. Sometimes this is stated as an impossibility of local realism.

In particular, the EPR argument is not a disproof of the completeness of quantum mechanics because of the faulty locality assumption which prevents achieving the complete quantum predictions.

In the next post I’ll dive more into Bell’s theorem and Bell’s inequalities.

Impressions from Europe

Since I just came back from vacation, I’ll break the wavefunction physics series for one post and share my impressions from my 2 week vacation in Europe. The two cities I visited were Paris and Bucharest.

I grew up in Bucharest Romania, and I know the city very well, but this was my first visit to Paris. Bucharest used to be called “The little Paris”, but I think the comparison works both ways and without fear of being wrong I can call Paris “The large Bucharest”. Despite the carefully projected identity as the city of fashion, Paris is a city which shares its core identity with Bucharest and I felt right at home in it. It still had several surprises in store for me.

Like any first time tourist there, I visited the big attractions and I marveled at the opulent display of art and former glory at the usual spots: Louvre, Versailles, and Notre Dame. I found the usual description of French people as rude completely false, but the “smelly” description still applies. This is because, despite the summer heat, most Parisians wore heavy formal suites for social status sake. Only tourist wore short pants and dressed appropriately to avoid body odor.

For some odd reasons, people in small restaurants in Paris love to eat outside right on the sidewalk looking at the passing traffic. After the first failure to attract attention, you will learn that you have to seat yourself. Also you will discover that each and every dish is of an exquisite quality and any humble restaurant in Paris will beat any fancy American restaurant by several orders of magnitude in taste. Also all restaurants are packed at 11 pm and I wonder if anybody is really working in this city. (The daily oddball strikes did not improve this perception either. There were strikes at the Eiffel Tower, at the airport, etc.)

The public toilets were a joke and Paris has some fancy one with a self clean cycle and here lies the problem: a full cycle (after each usage) take like 10 minutes, and a 3 person waiting line can be a big problem if you are in a hurry. Given the man-made shortage problem of public toilets, there is no wonder that about half the city smells of urine given the relative large number of homeless people.

Champs Elysees’ stores were grossly overrated, and better priced “Banana Republic” stores exists in most malls in US. And one more thing from Paris; I did not see any fat people whatsoever, and in fact most people were skinny. I think this is the result of the normal portion size in the restaurants there compared with the supersized US food culture.  

After a week in Paris it was time to go to Bucharest. Before the global economic crisis, Romania, and Bucharest in particular enjoyed huge growth and this was the place to make big money. After the subprime mortgage bubble burst, the city had only organic growth and the local general perception was that people got poorer and there was a newfound emphasis on competition and efficiency. I was pleasantly surprised by the local metro system which is I think recently became one of the best in the world.

During communist years, the dictator Nicolae Ceausescu built a huge palace (the second largest building in the world after the Pentagon) and its opulence can match any other palace including Versailles. Romanian’s Parliament is now housed there but there is still plenty left for tourists to  visit.

I took the opportunity to visit the palace, but the visiting tours are grossly mismanaged and I had the misfortune to have as an official guide a self-hating street smart young guy who “explained” the Romania revolution of 1989 in a grotesque conspiracy theory.  (I can state this with confidence because I participated myself in the 89 events and I am intimately familiar to what happened then.) To top the mismanagement, all toilets in this large palace were missing the toilet paper and the casual explanation is that the cleaning crew steals them. In this palace NATO summits took place and I am left to wonder if the building management took steps to prevent embarrassing situations, or the heads of states were told to bring their own toilet paper at the meetings. Since the politicians in the Romanian Parliament are the one in charge of the palace now, your can correctly guess that they are all crooks.