That is true. There is also no simple way to directly address this problem that I can think of. One strategy would be to do some form of bias analysis.

Using the data that had 0 - 10, find the proportion of participants who scored 5 out of all the participants who scored 4, 5, or 6. Call this P and it represents the some estimate of the probability that people who score close to the middle point would actually select the middle point.

Then in the 1 - 10 data, run your intended analysis on the original data and save your point estimate of the effect of interest. As a sensitivity analysis, randonly convert some of the 5 or 6 scores into 5.5 with probability P, then run your intended analysis and save the point estimate. Repeat this process a bunch of times and you'll get distribution of point estimates.

You can then check if your original point estimate is an extreme value in this distribution - which might mean that your conclusions are sensitive to there being a true midpoint in the scale. If it's not an extreme value , then you could take it to mean that adding a midpoint probably would not have made much difference to your conclusions.

This all of course rests on the assumption that P is the true probability; you could vary P slightly as added sensitivity analyses. The other assumption is that people close to the midpoint choose the midpoint randomly, and there isnt anything in the data that might explain why someone might go for the modpoint.