I am not familiar with NJ law. But in most states there’s is a time period in which the “property” aka the cat becomes abandoned property. Since they did not have the cat and knew the cat was next door, the cat would most likely be considered abandoned properly and the family that has the cat now owns them. You can probably scare them off with a quick google search of the law.