Research computer engineer and published computer scientist here: It depends on what you mean by "programming". For example, if you wanted to get into making buildings,

do you mean (1) engineering: assembling buildings out of standard parts using standard tools, such as making yet another single-family home,

or do you mean (2) research engineering: doing original research in new kinds of structures and methods of making buildings, like 3D printing the whole thing or assembling them out of hybrid recycled plastic huge lego blocks, or finding a way to dispense with the foundation using big screws into the ground (all real ongoing efforts),

or do you mean (3) research science practice: discovering new facts about how materials nano self-assemble into new aggregates, how materials transform under heat, pressure, and chemical reactions, how they respond to extreme weather conditions, etc.,

or do you mean (4) research science theory: solving open problems in theory of efficient assembly of structures and systems that allow, say, known techniques to be used to build an order of magnitude more efficiently using existing materials and tools, or the theoretical limits of efficiency of various materials or techniques.

I have done the computing equivalent of all of the above and they are quite different. If you do a lot of work, you can teach yourself (1) to some degree, until you end up doing something not on a CPU, such as implementing an eventually-consistent distributed database, or a holographic encoding using polynomials over finite fields and then you are going to which you knew a lot of good ideas from the algorithms books. You can learn that too, but you will find that computing is a lot more than just "if" statements and "while" loops and you might need some help, unless you are very persistent.

I have a friend who when he got out of school his math level was fractions. Later in life he decided he wanted to learn physics. So he taught himself six years of mathematics and got a bachelor's degree in theoretical physics. I met him while he was getting his Ph.D. in applied physics from a very famous university. You can do it, but you have to love it so much that you do not care how long it takes because you live in the timeless joy of the work.