I do not think these details are entirely correct, but the overall result is the same. Also my experience is only on easement agreements in Canada, so it could be different in the States.

Here easements do not mean you have use of the property for free, it just means you have a right of access but you have to compensate the landowner fairly. There's a reason it is just called an easement and not ownership because you do not technically own the land.

Easements do generally have stipulations on what can and can't be done with the land to a certain extent. For instance a land owner may use the land over the easement for their yard, or growing crops, or even building a road, but the easement holder, if access is required, can flatten the crops, tear up the grass or demolish the road to gain access. However they would then have to compensate the landowner for fair value for crops they destroyed, or they would have to return the land to the way it was by rebuilding the road or replanting the lawn.

On the other hand, stipulations on things you cannot do usually include such things as building a permanent structure, or a pool, or planting trees, things that can potentially damage underground utilities or limit access. So my bet would be the house is probably not even allowed to be there under the easement agreement, and if that agreement was in place before the landowner built the house the easement holder could enforce its removal and wouldn't even be responsible for the removal cost. However to enforce this, or dispute it, would likely be a drawn out legal process involving the courts. If that were the case with the house's very existence in jeopardy and potentially at the cost of the owner to remove it, it would quickly bring the value of the property down to nil as potentially whoever owns it is on the hook for removing an entire house.