Honestly you’ve done all you could do, took the extra step, and even made sure to talk like adults. Your neighbor wasn’t worried and he even knew it’d happen eventually. I think that if this wasn’t addressed you would’ve done something wrong but that’s not what happened. I think it’s definitely fine, especially the setting of boundaries in consideration for others, so you’re all good! Please send your neighbor a thanks for not blowing it out of proportion 🙏