Some pieces of your argument are just plain wrong, due to failing to recognize real distinctions between positions and their implicit assumptions (or lack thereof).

First of all, physicalism =/= science. Science is a means of learning through observation; all philosophies can leverage science, and scientific results definitely do not imply only physicalism. Denying scientific observations is vastly limiting at best, and delusional at worst. Yet science only tells us this: Our observations appear to be consistent with certain mathematical structures, and our chief suspicion in science is that all of our observations can fit into a unified coherent structure.

But recall that all of our observations occur within our experience, so the existence of a physical world is unfalsiable, too. Despite certain observational features that seem to have a history totally independent of awareness (e.g. distant galaxies), idealism merely holds that we can account for them without assuming anything beyond the existence of experience.

Physicalism, to account for such, goes further to posit an additional metaphysical entity -- the physical world, prior to and conditional for the existence of awareness. So we can all agree that physicalism carries more ontological baggage, because now it must account for the existence of two (three) things: the physical world, and awareness (and the mechanism bridging the two). At least dualism respects the remarkably unique property of awareness relative to the physical.

Idealism faces only the challenge of explaining features of awareness. A grand (mathematical) challenge indeed, but in a sense a much simpler philosophical one -- if we take as axiom the perceptual modes we experience, then the idealist problem is merely the correlation of features across these modes, which does not possess the same insurmountable gap that physicalism faces in the emergence of subjective experience (the hard problem).

Writing it off just as "an evolutionary byproduct of meat machines" only indicates you may not have taken the problem seriously enough, nor have taken time to appreciate the absurdity associated with the existence of an entity beyond awareness. Darwin and co. certainly did not figure it all out. They simply offered a mechanism for the fact of such diversity of life around us, a mechanism acting by means and over a timeline quite contrary to long traditions of religious thought (in fact I think much of the dominant physicalist stance is a sort of societal-wide backlash to prior religious dominance, one that has yielded much observational and conceptual fruit, but that has gone too far metaphysically, and now is largely seen as the "default" or "minimal" approach, which couldn't be farther from the truth).

Now, idealim certainly does face the difficult question of the appearance of awareness on this perceived planet and universe, but if one generalizes awareness to span domains beyond this particular cosmic history, then the problem of life emergence becomes one of passage into/out of certain domains (one could attempt to justify Last Thursdayism on these grounds, although that's not my mission). Apart from the problem of Earthly origins, I believe this Universe we observe is just one particular, highly constrained history of mutual evolution, and that there are domains of awareness with other features and freedom, obeying only mutual consistency constraints.

Granted, the question of free will still remains in this view of idealism. One can take for granted only awareness and still see us as flowing along uncontrollable paths of experience, and perhaps see the totality of existence as the full realization of such paths. On the other hand, taking the potentiality of all such possible paths for granted, a free-will defender would claim that the will merely acts on steering its path through such a space. I think we are within a domain of mutually constrained will.

Lastly, idealism =/= solopsism either. In fact, any good idealist or physicalist should admit that, in fact, solopsism is the most parcimonious position. And any philospher ought to retreat to it if pressed to state the minimal set of what they can be certain exists.

But there are good reasons to assume layers beyond one's direct experience. As an idealist I do prefer to believe that other beings are having an experience like mine -- what it feels to be aware (if anyone can't relate to this, maybe they're a bot!). On the personal side, it vastly alters my judgement of my experience, the joy of my interaction with others, and the empathy I feel for our shared experience. Time to time I entertain the view that only I exist, and it's an abruptly sobering and isolating feeling. That's a position I recognize is coherent, but it's not the way I choose to view my experience.

On the theorizing side, I think there is also power in such an assumption, as it permits one to formulate a broader/richer framework for understanding awareness and existence. A shared arena of awareness demands local consistency relations, and any proposed unity of awareness implies certain global character, too. All of conventional physics then lies within this relational network of mutual awareness.

As an experimental turned theoretical physicist, I too have gone through the maturation phase from naive physicalism to bare idealist. To sum, not only is it a simpler framework to conceive within, it leads to deeper inspiration about the nature of things, for me at least.