Adolf Hitler Campbell, 4, and his sisters, JoyceLynn Aryan Nation, 3, and Honszlynn Hinler Jeannie, 2, were taken from their New Jersey home in 2009.
The case first came to public attention in December 2008 after a shop refused to decorate a birthday cake for Adolf.
A family court had earlier found there was insufficient evidence of abuse.
But on Thursday, the appeals panel determined that social workers had proved the need for protective services for the three children.
The panel found that the parents, who both suffer from unspecified physical and psychological disabilities, had “recklessly created a risk of serious injury to their children by failing to protect the children from harm and failing to acknowledge and treat their disabilities”
Click here to see gallery of from slavehouse to White House