David D. Fisher is enumerated in the 1850 Hinds Co MS census as head of the following family:

David D. Fisher, 28, m, b MS
Margaret E. Fisher, 27, f, b Ireland
Rosana Fisher, 1, f, b MS
Ann Graham, 60, f, b Ireland
Ann Miller, 25, f, b Ireland
Mary A. Miller, 2, f, b MS
Margaret J. Miller, 4/12, f, b MS

The 1850 Slave Schedule of Hinds Co indicates that David D. Fisher owned one 25-year old female slave.

The family is also listed in the 1860 Hinds Co MS census:

D. D. Fisher, 37, m, Farmer, b MS
M. Fisher, 35, f, b Ireland
Rosa Fisher, 10, f, b La
John W. Fisher, 6, m, b MS

The adjacent family is Margaret Graham Fisher’s mother:

Ann Graham, 60, f, b Ireland
Ann Miller, 35, f, b DC
M. A. Miller, 12, f, b MS
Nell Miller, 9, f, b MS
M. Miller, 9, f, b MS
(beside the last two entries is written “twins”)

David D. Fisher seems to have died before 1870, as Margaret and Rosa are enumerated in the 1870 census of Hinds Co MS:

Fisher, Margarette E. 54 f w keeping house b Ireland
---, Rosa 19 f w b LA
---, John W. 16 m w farmer b MS
---, William 6 m w b MS

Margaret’s sister Ann Miller is listed next door:

Miller, Ann 49 f w keeping house b Ireland
---, Nancy 22 f w b MS
---, William M. 19 m w at home b MS Twins from the 1860 census?
---, Maggie 19 f w at home b MS Twins from the 1860 census?

In the 1880 Census, Margaret is found in Liberty Grove, Hinds Co MS:

Margaret Fisher 66 w f 66 Ireland Ireland Ireland (widowed)
John Fisher 25 w m 25 son MS Ireland* Ireland
Mary R. Fisher 23 w f 23 dau MS Ireland* Ireland
Willis Fisher 15 w m 15 adopted son MS MS MS

* Note that David D. Fisher was born in MS, not Ireland, so this must have been an error of the census taker.