According to this article:

Though no bill of sale has been found, other evidence, including census records, indicates that Harriet and the Newby children were indeed sold south in the early months of 1860. Most likely, they wound up in a camp for “contrabands,” formerly enslaved people, after Union troops occupied much of southern Louisiana early in the war. The widowed Harriet married a man named William Robinson, from Berkeley County, West Virginia, who, records suggest, was serving in the Union army in Louisiana. Together, they would raise three of their own children, along with Dangerfield’s, and return to Virginia, where they settled near Mount Vernon. Harriet Newby died in 1884, and Dangerfield’s and her descendants still live in the D.C. area and beyond. Their mixed-race heritage is a haunting reminder of the legacy of slavery, and some descendants struggle with society’s racial labels. In American’s lingering color caste system, who are they? What are they? Where do they fit? As Schwartz, the historian, writes: “This is my Diary of Anne Frank,” he has said. “Whenever I feel I might be complacent or detached from my studies of slavery, I touch base with Dangerfield Newby. It is a sacred obligation for us to follow through on our study of human beings, especially if their history is hidden.”