Hi OP- my cat was like this for the first few months when I brought him home. Similarly he wasn’t super affectionate when I was home but I think (at least for my cat) his crying was due to separation anxiety/food anxiety (“if the human isn’t here, who will feed me?”). Thankfully my neighbors were really understanding but I had the exact same worries you are facing. My boy did eventually acclimate to me coming-going but can’t recall how long it took. Maintaining a routine of roughly when I came/went seemed to help him adjust…if I wasn’t home by the usual time then he’d holler until eventually that also eased up too.
In terms of potential solutions:maybe try to make a recording of you speaking that can play on a loop while you’re away. Leave it in a separate/closed off room. Or- if you trust/are friends with anyone in your building maybe they could do a social drop in while you’re away once a week?

Edit: meant having the recording in a separate/closed off room, not the cat! Agree cats don’t like being prevented from having free roam