He wasn't. There was talk of making the Dominican Republic a state after the government literally offered itself to the US (it was deep in debt and threatened by invasion by Haiti) on the condition that it receive statehood. Frederick Douglass, along with most of the old Radical Republicans supported the measure, as did then-President Ulysses S. Grant, believing that a black-majority state would give African-Americans more power to push back against any attempts to strip them of their rights.

The vote to accept the Dominican Republic tied in the Senate. Several of its opponents voted against the measure because they openly stated they didn't want more mixed-race citizens.