Is the universe deterministic or does it operate by chance?

Is the universe deterministic or does is operate by chance? It's a fair question. Most people know that quantum mechanics has an inherent randomness in it and they might be inclined to think the universe operates by chance, but that's partly correct and partly incorrect.

Classical physics is in principle deterministic. If you know everything there is to know about the initial conditions, you can predict where it's all going to end up. You can argue that it's impossible to know everything about the initial conditions, and that a tiny 'butterfly effect' can throw things off, but this is a thought experiment often framed in the mind of a know-it-all demon, so let's not nit-pick.

Quantum mechanics is different. For example, Heisenberg's Uncertainty Principle specifies limits to what can be known about certain pairs of properties, such as the position and momentum of a particle. The more you know about the position, the less you know about the momentum and vice versa.

Furthermore, talking about particles alone does not describe a quantum system. It also exists as a wave or a field and how you treat things depends on the sort of result you're after. The idea that a particle has a distinct path from A to B is invalid in quantum mechanics.

This all seems to point to random chance having a hand in things.

The way around this is to take a statistical approach to a bunch of particles or waves in a quantum system. Quantum mechanics predicts probabilities and if you take an initial set of probabilities and push them through the correct equation, you can get a resulting set of probabilities with deterministic certainty. But, I hear you cry, they're only probabilities.

This is true, but if a quantum system is large enough it becomes classical in behaviour. If you walk to your local shop, you, as a classical unit, are likely to take a single path to the shop. It is unlikely that your left arm will choose to go a different way to the shops than the rest of your body, or that you'll somehow take two paths at once.

Think of it this way: if you flip an evenly balanced coin you cannot know in advance if it will be heads or tails. But, statistically speaking, you know in advance that if you flip that coin 10,000 times you'll get a roughly 50-50 balance of heads and tails. The individual coin toss is random but the bigger picture of 10,000 coin tosses is deterministic.

That’s an example with limitations, of course. Tossing a coin is a classical activity and our know-it-all demon could predict any coin toss if he knew the initial conditions (exact coin structure, force with which it’s tossed, air currents in the room etc.). It’s merely a classical illustration of a quantum situation.

Einstein always thought quantum uncertainty was only apparent because we didn’t know enough about the quantum world. He thought it was an approximation and a deeper underlying theory would restore determinism and absolute positions and momentums (simultaneously). He spent much of the latter part of his life searching for such a theory without success. Today, to the best of scientists’ knowledge, the uncertainty is real and quantum systems do exist in a mist of probability.

This is the sort of thing philosophers love to discuss in relation to free will, and they sometimes use quantum mechanics as an example of the potential for free will. I struggle to see the relationship. Probability does not equate to free will as far as I can tell. The way I see it, it’s all about perception. We’re unable to do what the know-it-all demon can do, and that renders things unpredictable. Whether we have actual free will is neither here nor there. We have apparent free will and think we’re making choices. That’s good enough for me.