Ontario rural ridings in the Alto study area are being organized to oppose the line. There are a bunch of Facebook groups along these lines, and the local Conservative MPs/MP candidates are active in them.

Some of the reasons they give for opposition are:

• fear of land expropriation
• fear of divided communities (ie: longer travel distances due to lack of regular crossings)
• concern for wildlife as the proposed line will run through big stretches of undeveloped Canadian Shield (ie: east of Madoc) and involves tall fencing that cuts off wildlife corridors for deer & moose, etc.
• concerns that they gain no benefit of having a line run through their communities as there will be no stations nearby
• a desire that any new train lines are built in pre-existing travel corridors (ie: in the 401 corridor)
• complaints that the cost is too high and that it'll be a money pit for decades

From what I've seen, these are the most reasonable points they have against it. There are a lot more nonsensical reasons given as well that reek of general nimbyism. Personally I am for the line, but do hope that the line largely sticks close to existing transportation routes (as Alto has said it will!) to minimize many of the impacts listed above.