1. The term "Old Testament", when applied to the 39 books in the front of our Bibles is a man-made application of the term (the term itself is Biblical, but applying it to the 39 books is a man-made application of the term). It needs to be remembered that such usage is not inspired; rather, it is mere tradition. The more Biblical description for these books is "the Law and the Prophets" (Lk 16:16) or "the Law, Prophets, and Writings (Psalms)" (Lk 24:44). (In Hebrew, it would be Torah (Law), Nevi'im (Prophets), and Ketuvim (Writings). Taking the first letter of each of these you get the acronym TNK, which is often expressed in English as Tanakh, a common method of referring to what we traditionally call "the Old Testament".)
2. There are various covenants in the Bible, not just the two ("Old" and "New") that our main divisions of our Bibles would lead us to believe. For example, there's the covenant with Adam in which God instructed Adam and Eve to be fruitful and multiply and have dominion over the earth. There's the covenant God made with Noah and all the animals and all the earth to never flood the earth again like He did in Noah's day. There's the covenant God made with Abraham to make of him a great nation and to give him land and to bless all the earth through his seed. Joel made a covenant with his eyes not to look at a woman other than his wife. There are many other covenants as well, but this should suffice to make the point.
3. Putting these two points together, and studying the relevant New Testament passages, it becomes clear that it was the covenant with Moses which has been taken away and replaced, not the other covenants. Jeremiah (31:31) makes this clear, and it is reiterated by the writer of Hebrews (8:7) that it was the covenant "made with their fathers on the day I took them by their hand to lead them out of the land of Egypt" which was to be replaced by the "new covenant".
Note that this does not replace God's covenant with Abraham; in fact, Paul contrasts the Mosaic covenant with the Abrahamic covenant in Galatians 3, specifically claiming that the Mosaic does not revoke the Abrahamic (v 17), and that we are heirs because that promise still stands (v 29), and that the Abrahamic promise has now been extended to Gentiles (v 14). Looking at the structure of this chapter you see Paul talking about the Abrahamic covenant/promise being our faith (v 6-9), then the temporary addition of the Mosaic Law (v 19), then the removal of that temporary Mosaic Law revealing once again the Abrahamic covenant of which we are now heirs (v 19, 25-29). He then goes on in the next chapter to contrast these two covenants, one as a covenant of law (Hagar, Mt Sinai) and one as a covenant of promise (the free woman (Sarah), the Jerusalem above).
Jesus himself validates the use of the non-Mosaic portion of the Tanakh as authoritative in the Christian era when he was asked about marriage and divorce and he turned to the Creation account as his source of authority. Note that he did not give a "new" law; rather, he appealed to the "Old Testament", and we accept his conclusion as binding on us today. The Bereans also turn to the "Old Testament" as their source of authority when testing the claims of the New Testament, and are praised as "noble" for doing so.
To summarize this point number three, it's not all of the "Old Testament" which has been done away with, but merely the Law of Moses portion, that portion which was given to Moses and the Israelites at Mount Sinai. The rest of the Tanakh has not been removed; thus we still are to be fruitful and multiply and be stewards of the earth (Gen 1:28-30, a provision of the Adamic covenant which was renewed in the Noahic covenant - Gen 9:1-2); we're still to have marriages consisting of a man and a woman (Gen 2:24); serpents still haven't regrown their legs and/or wings (Gen 3:14); we're still to work the ground and have pain in child-bearing and die (Gen 3:16-19); women will still fall in love with sweaty, stinky, selfish men and thus be dominated by them (Gen 3:16b); we're still to execute murderers (Gen 9:6); we still see rainbows and thus know we'll never see the world inundated completely by water again (Gen 9:12-16); we can now eat meat (one provision of the Noahic covenant that supplanted the vegetarian provision of the original Adamic covenant - Gen 9:3-4); and so forth.
So we need to be careful when we say things like, "The Old Testament has been done away with." Strictly speaking, that is inaccurate. It is only a subset of the "Old Testament", the "Old Covenant between God and Israel, given at Mount Sinai (and only to the Jews anyway, never to the Gentiles)", which has been done away with.
I believe this is an important concept for the church to know, as it's foundational to a correct approach to scripture.