Why are unitarity violations fatal for quantum mechanics?

Last post attracted a lot of attention and criticism from Lubos Motl and I will postponed the scheduled presentation of this week and instead try to address that criticism head on. When discussing quantum mechanics one can take three points of view: physical, mathematical, and philosophical. In the past I had an argument with Lubos on Boolean vs. quantum logic in quantum mechanics, and the root cause of the disagreement then was on the mathematical vs. the physical points of view. This time the root cause of disagreement is due to the physical vs. philosophical approach. Lubos sees no value in the quantum foundations community because the proper interpretation was settled in his opinion long time ago and all quantum foundations practitioners must be crackpots (obviously there is no love lost between the quantum foundation community and Lubos). Today I will not take the philosophical point of view and instead I will attack the argument from the mathematical side and attempt to show that the measurement problem is still an open problem (not only Lubos but also some members of the foundation community disagree with this too).

The problem starts from the category theory approach in quantum mechanics reconstruction. The mathematical formalism of quantum mechanics can be derived from two physical principles:

• invariance of the laws of nature under time evolution
• invariance of the laws of nature under physical system composition

Those two physical principles are easy to understand: invariance under time evolution means that the laws of nature are the same today as they were yesterday, or they will be tomorrow. Invariance under composition means that the laws of nature do not change with additional degrees of freedom. How can you derive the formalism of quantum mechanics (Hilbert spaces, self adjoint operators, etc) from those physical principles? With the help of the algebraic formulation of quantum mechanics.There a C* algebra can be decomposed in a symmetric product (the Jordan algebra of observables) and a skew-symmetric product (the Lie algebra of generators). Since Hilbert space composition is done using the tensor product and since there is a universal property of tensor products linking products with tensor products, imposing the physical principle of invariance under composition has deep algebraic consequences. It can be shown that those algebraic consequences completely determine the C* algebra formulation of quantum mechanics and to recover the full formalism then one has to apply the GNS construction. But to prove all this you need a starting point, and that is the Leibniz identity which comes from the invariance of the laws of nature under time evolution. Violate Leibniz identity and the whole mathematical formalism of quantum mechanics becomes mathematically inconsistent. 

Why is this important? Because as it is well known by the mathematics community, the Leibniz identity in the algebraic formulation corresponds to unitarity in the usual formulation. No unitarity during collapse means no Leibniz identity which means mathematical disaster. This is why I am seeking a unitary solution to the collapse problem. This is not a fool's errand, as category theory which highlighted the problem in the first place also points to the unique way to solve it using the Grothendieck group construction. In the process the quantum mechanics interpretation which emerges is that of Copenhagen. And this follows as a necessary mathematical result and not as a crusade against other interpretations.

Since the deep relationship between Leibniz identity and unitarity is not well known outside the mathematical physical community, I will now present it below: 

Unitarity from Leibniz:

This follows from the categorical approach of quantum mechanics reconstruction. I presented this in a series of posts starting with this one.

Leibniz from unitarity:

To see the inverse implication we need to go in depth in the Jordan-Lie algebraic formulation of quantum mechanics. The algebraic formalism is a unifying framework for both quantum and classical mechanics and the only difference is that the observables bipartite product \(\sigma_{12}\) has an additional element in quantum mechanics: \(-\alpha_1 \otimes \alpha_2\). This additional element prevents the Bell locality factorization  (the observable bipartite product \(\sigma_{12}\) can no longer be factorized in terms of \(\sigma_1\) and \(\sigma_2\) and this prevents in turn the state factorization) and makes possible superposition and continuous transitions between pure states. 

Classical mechanics is described by a Poisson algebra while quantum mechanics is described by a Jordan-Lie algebra. If we add norm properties to a Jordan-Lie algebra, we get a J L B (Jordan, Lie, Banach) algebra which is the real part of a C* algebra. The C* algebra is the algebra of bounded operators on some Hilbert space arising out of GNS construction. The pure state space \(\cal{P}(\mathfrak{A})\) of a C* algebra \(\mathfrak{A}\) is a Poisson space with transition probability. Unitarity means that the Hamiltonian flow of the states generated by a given observable \(\psi\mapsto \psi(t)\) preserves the transition probability \(p\):

\(
p(\psi(t), \phi(t)) = p(\psi, \phi)
\)

This definition is more general than the usual definition of unitarity in a Hilbert space, because it works for both the Hilbert space representation and for the state space of the C* algebra. The definition also applies to the classical case. If the classical case there is no superposition and the transition probabilities are trivial: \(p(\psi, \phi) = \delta_{\psi \phi}\). 

To derive Leibniz identity from unitarity one can proceed in two steps. First the algebra of observables \(\mathfrak{A}_{\mathbb R}\) is recovered from the pure state space. Then the Hamiltonian flow \(\psi\mapsto \psi(t)\) defines a Jordan homomorphism and the derivative with respect to time of the homomorphism property yields the Leibniz identity.

Given a transition probability space we can define linear combinations of transition probabilities and this defines a real vector space \(\mathfrak{A}_{\mathbb R} (\mathcal{P})\). In the quantum mechanics case the elements of this vector space  have a spectral decomposition \(A = \sum_j \lambda_j p_{e_j}\) which allows the definition of a squaring map: \(A^2 = \sum_j \lambda^2_j p_{e_j}\). This in turn is used to define the Jordan product:

\(
A\sigma B = \frac{1}{4} ({(A+B)}^2 - {(A-B)}^2 )
\)

With the sup norm and the product \(\sigma\), \(\mathfrak{A}_{\mathbb R} (\mathcal{P})\) now becomes a J B (Jordan Banach) algebra and the first step is complete: starting from the pure state space equipped with transition probability we arrived at the Jordan algebra of observables. In the classical mechanics case the Jordan product is simply the regular function multiplication. 

For the second step, we use the Poisson structure and the Hamiltonian flow: \(\psi\mapsto \psi(t)\). For each element \(A\) of \(\mathfrak{A}_{\mathbb R} (\mathcal{P})\) (corresponding to an operator in a Hilbert space by the GNS construction) we can define a one-parameter map \(\beta_t\) given by \(\beta_t (A): \psi \mapsto A(\psi (t))\). Then we have: \(\beta_t (A \sigma B) = \beta_t(A) \sigma \beta_t(B)\) because \(\beta_t(A^2) = \beta_t(A)^2\). As such \(\beta_t\) is a Jordan homomorphism.

From the Hamiltonian flow we have:

\(
\frac{{\rm d} A}{{\rm d} t}(\psi (t)) = \{h, A\}(\psi (t))
\)

We take the time derivative of the isomorphism \(\beta_t (A \sigma B) = \beta_t(A) \sigma \beta_t(B)\) and we obtain the Leibniz identity. 

We need to point a notation difference between what we and the literature calls Leibniz rule. In  the mathematical literature the Leibniz identity defines only how the Poisson bracket \(\alpha\) acts on the observable algebra \(\sigma\) and the proof was outlined above. In my case Leibniz identity applies to all algebraic products. However the only other algebraic product is the Poisson bracket itself and because its skew-symmetry the Leibniz identity becomes the Jacobi identity. The Jacobi identity is assumed by the structure of a Poisson space and we do not need to derive it from unitarity. Both classical and quantum mechanics have a Poisson space structure.

Now back to the philosophical point of view. Various quantum mechanics interpretations are nothing but distinct frameworks for solving the measurement problem. The basic problem is that given a unitary explanation of one outcome A by introducing a particular coupling between the measurement device and the quantum system, and given a  unitary explanation of another outcome B, then by superposition you get a superposition of outcomes and this is nonsense (there are no dead and alive cats). Now which framework for solving the measurement problem is the correct one? Because the measurement problem must explain the non-unitary collapse, and since non-unitarity makes the mathematical framework of quantum mechanics inconsistent, the mathematical solution ultimately points out the right interpretation. So far everything in the category theory approach points towards the Copenhagen family of interpretation as the correct explanation. 

If I can offer an analogy, the Grothendieck approach for solving the measurement problem without spoiling unitarity is to quantum mechanics what the Higgs mechanics is to field theory (introducing mass without spoiling the gauge invariance). The problem is still open and it is work in progress. The basic idea is that instead of one Hilbert space describing the collapse there are actually many Hilbert spaces linked by an equivalence relationship.The equivalence relationship encodes mathematically the physical principle of outcome randomness in quantum mechanics (this comes in turn from operator non-commutativity). Each Hilbert space corresponds to a GNS representation of a C* algebra state corresponding to a potential outcome. The one and only outcome is realized when the equivalence relationship is spontaneously broken by a pure unitary mechanism. The observer (measurement device) plays an essential role and the result of measurement does not exist independent of measurement. However, the consciousness of the observer plays no role whatsoever. There is no need for the measurement device to be described classically.