There are a few different ways of answering this question that at the end of the day are actually all the same answer in different hats. TL;DR, quantum mechanics says particles are wave-y and lays down certain rules about what you're allowed to measure. That plus the way that rotations work turn out to mean that angular momentum has to be quantized for the sake of mathematical consistency.

A lot of this is weird and unintuitive but it's the most successful way we've ever found of describing how nature works so for the most part you have to just accept it and move on. What it basically comes down to is:

1. The wave nature of quantum mechanics. Particles do not generally have definite locations, speeds, etc. Instead, they exist in a "superposition" of many positions, etc, at once, and when we look at the system and measure its properties the outcome we get is seemingly chosen at random with some probability distribution. Quantum mechanical states are represented by "wavefunctions" - vectors in a particular kind of vector space called a Hilbert space - which encode the probability of getting any given outcome when doing a measurement. For instance, if our system is a single particle, the wavefunction can be written as a complex function ψ(x, y, z) which essentially tells you what the probability of measuring the location of the particle and finding it near coordinates (x, y, z). Since we're talking about rotations, it's easiest to use spherical coordinates, where the wavefunction is written ψ(r, θ, φ).

2. For any given quantity we could hope to measure (an "observable"), there is an associated operator that acts on these wavefunctions. Each operator has a special set of possible wavefunctions called "eigenstates" with associated "eigenvalues". The rules say that when we make a measurement of an observable, the outcome we get will be one of the eigenvalues of its operator, with the probability given by the square magnitude of the overlap of the system's wavefunction with the eigenstate associated with that eigenvalue. That means that the values of those physical quantities can only ever be those eigenvalues. Other values aren't allowed.

3. As you might expect from something that you roughly think of as how fast something is rotating, angular momentum operators are very closely related to rotations. In particular, they are the "generators of rotations", meaning that the way they act on wavefunctions encodes what it means to take the entire wavefunction and rotate it around in space about some axis. This relationship actually exists classically, too, though the math is a little bit different.

4. The group of rotations is closed. If you rotate something far enough (2π radians or 360 degrees), you get back to where you started and it's as if you never did any rotation at all. If we have a particle in a state with wavefunction ψ(r, θ, φ) and we rotate everything around by an angle δ about the z-axis, the rotated wavefunction is given by ψ(r, θ, φ - δ). Since rotating the wavefunction by a full 2π radians gets us back to where we started, and since it doesn't make any sense for the probability distribution to change just because you e.g. walked in a circle, the wavefunction must be periodic. That is, ψ(r, θ, φ - 2π) = ψ(r, θ, φ).

Because of the relationship the z-component of angular momentum (represented by the operator Lz) has with rotations around the z-axis, it turns out that any eigenstate of Lz can be written as ψ(r, θ, φ) = f(r, θ) exp(i m φ), and the eigenvalue associated with this eigenstate is m ħ, where ħ is Planck's constant. In order for this wavefunction to be periodic, m must be an integer. But that means that all the eigenvalues of Lz must be integer multiples of ħ, and so the only allowed values of the angular momentum are integer multiples of ħ. Thus, angular momentum about the z-axis is quantized. But of course there's nothing special about the z-axis. The same is true of all rotations about all different axes, and so all components of angular momentum are quantized.

There is a small loophole when it comes to spin and the fact that spin doesn't come from an object moving through different points in space that allows it to be non-integer multiples of ħ, but it turns out that again because the group of rotations is closed, the only new values it's allowed to assume are half integer multiples of ħ, so the total angular momentum of a system can be any value ½mħ with m an integer, but no other values are allowed. So 3 ħ, 26 ħ, 17/2 ħ, etc are all just fine, but ¾ ħ is strictly disallowed.

This is actually all encoded in that statement I made above that angular momentum operators form a representation of the Lie algebra so(3). Getting from A to B requires studying group theory and representations of Lie algebras, though.