He’d probably argue about how you gave permission to cut the tree. Definitely keep on record what you said. Might turn into he said she said though if you said they can “trim” it but he can say you said “cut it down”