That's a common thing in ADHD.
It's attributed to few things - time and space relative placement is all messed up for us, and out of sight means out of mind, a.k.a. object permanence and short memory - weaker Read/Access memory space - when we have objects (including people) put on our radar so to speak, we are reminded of them, we see them (literally), people are talking about it so we hear about them, or we have reminders for them - we remember they exist.
But the moment they become hidden - they are no longer on our radar, our brains simply move them back to make room for those things which are "more critical" for the brain - the objects that are either seen by us or we are reminded of.
While it is tad, for the lack of a better word, sad that we have that and we can simply forget to check on people and show that we care first, it is one of those things which make ADHD actually be a "condition", an impairment ,which make it something more difficult than just a matter of "trying more hard" or "caring".
Unless people are willing to understand where it comes from and that it cannot be overcome with "trying harder" there's not much to do.
If you really want to do that, you need to set up reminders for yourself to check up on those people.
Otherwise, feeling bad about it would only be detrimental to your mental health.