"What would prevent the strong (wealthy and powerful) from enslaving the weak (the poor)?" (Snowdenn Brock)
(The fallacy is - as this very petitioner attempted to answer for himself - the assumed answer of "Nada. Nothing.")
Hold on there, laddie. Nothing's preventing slavery in statism. But the most likely answer to what would prevent it in a free society is distributed power.
When only the state has the power, then slavery is easy; resistance is easy for the centralized slave power to destroy (although even then, things like the Underground Railroad have some success). When power is spread across the population, most of which is against or apathetic to slavery, but definitely don't want to help catch or pay for fugitive slaves, it becomes difficult if not impossible to maintain slaves.
(Read first: Right vs Power.) The so-called "civil war" (so-called because it was not a struggle for power, but rather a war for independence) was fought to prevent the seceded states (new CSA nation—under their own rules, if you follow the statist religion) from exercising their rights to independence.
Note that slavery nowhere enters into it thus far as discussed. A state may rightly secede for any reason, slaves or not. Similarly, a person may rightly free slaves anywhere, on their own property or others. An invading army has the right to go somewhere and free slaves, but not to conquer territory and harm people that keep no slaves and aren't stopping them from freeing slaves.
This line of inquiry is further expounded in Dr. Block's paper Secession.