We explore how financial markets combine and influence each other, especially through the role of arbitrage - the process of finding and taking advantage of price differences across markets. We describe how markets share striking similarities to thermodynamic systems. This perspective helps us see how the process of combining markets can lead to a "market entropy," a measure of lost liquidity when markets aggregate quantifying price uncertainty. Finally, we examine how this theory unifies two perspectives of a market: one as an exchange mechanism and the other as a venue for interaction among traders. The talk is aimed at a general audience.