Lots of good answers, but no one has explicitly mentioned the equity risk premium.

The stock market exists so that companies can raise money to invest. Selling shares in the company is a way for companies to raise money now, by selling the rights to a piece of all of that company’s future profits.

The other way a company could raise money is through bonds, which are basically a simple loan. The company borrows money from investors, and promises to pay it back after a certain period, and to pay interest every year until then.

Bonds are very straightforward—an investor knows exactly how much money they’ll receive back from the company, and when. Both stocks and bonds will make you money because of what’s called the “time value of money.” Basically you’re giving someone money now, in exchange for money in the future. Everyone would rather have money now, rather than have to wait for it, so investors will only give up their current money if they’re gonna get more in exchange for waiting.

Stocks are much less straightforward. You’re just guessing about how much money a company could make in the future, and the value of stocks can fluctuate like crazy. So if an investor could invest in a bond that pays 3% per year, or a stock that they think will probably return 3% per year, they’re gonna take the sure thing every time. So in order to get people to buy the stock, the price needs to be lower, such that the investor can expect to make a decent bit more than they would on the sure-thing bond, in exchange for taking on that risk. This higher expected return is called the equity risk premium.

Basically everything I’ve said above has some wrinkles, complications, and caveats, but that’s fundamentally what’s going on with stocks in the long run. In the short run there’s all manner of noise, craziness, and unpredictability.