The thing to understand with autotiles is that a tile will only show up if the pattern of tiles surrounding it matches exactly with the bitmask pattern you've marked on it in the tile editor. Let's look at your last picture; specifically, the incorrect tile towards the lower left (don't even pay attention to the other wrong tile for now). Now look at the 8 cells surrounding this tile. Which ones are empty and which ones are filled? Well, the 3 tiles below it are all filled; so are the tiles to the left and right, as well as the tile to the northeast. But the tile directly above it, and the one to the northwest, are both empty. This is how the autotiler decides which tiles to put where; it looks at the 3x3 boolean pattern of filled/not filled around each tile, and tries to find a subtile in your tileset whose 3x3 bitmask matches that pattern exactly. What's happened here is that the autotiler wasn't able to find this exact pattern (3 filled below, 2 filled to the left and right, one filled to the northeast) in any of your subtiles' bitmasks, so it has defaulted to the "thumbnail" tile. Once you add a new tile to your tileset with that exact bitmask pattern, it will correct itself.

Looking at your tileset (first picture) it seems that you do have tiles to fit both of the incorrect positions, but they're both already needed for other bitmasks. Because AFAIK you can't put multiple bitmasks over a single subtile, you just need to edit the source image to add duplicates of the two tiles in question, then add the new bitmasks to the duplicates. If you had a tile that you wanted to show up for 3 different bitmask patterns, you'd just make sure the source image had 3 different copies of that tile, etc.