As soon as you start hand-waving away and justifying portions you don't agree with (like the slavery example you just mentioned), you have opened the door for *any* portions to be justified away in the same manner. There is nothing in the Bible that instructs "follow this part, but ignore the previous page."
The Bible cannot be used as a basis of logic and directive on human behavior for this very reason. It can be a fantastic tool for study, for allegorical lessons, for cautionary warnings, for a supplementary tool, and especially as the guiding principals of religious belief, but not as a logical basis for extrapolation into how people or cultures should act.