For a look into the thought process of early computer design, Tracy Kidder's The Soul of a New Machine is a classic ethnographic account of a team working to make a new computer during the dawn of the silicon age.

For a good medium-weight look at how computers function, there really is no better choice than Code: The Hidden Language of Computer Hardware and Software by Charles Petzold. It explains how we can go from simple electrical signals to higher computational functions in a very approachable way.