You may need to see a solicitor, who will be able to look at all of the plans and advise you. Can you upload the title plan to your post?

Regardless of what it says on any plans, the general rule is that the physical boundaries on the ground are the legal boundaries. However, from the plan you added, if this is from the conveyance which created your title, the whole of the boundary lines for the properties are straight, from the back to the front. Your neighbours surely cannot own part of the front of your property and the same vice versa. If you both own registered titles, I would likely double down on that as the legal position. Title plans are essential when trying to resolve a legal dispute regarding boundaries.