The Holy Spirit considered it "good" (Acts 15:28) for the Jewish Christians to continue keeping the law of Moses (as did Paul - Acts 21:24), while the Gentiles did not (Acts 15:1-29; 21:25). Thus forms two very different and distinct groups within Christianity: the Circumcision, and the Uncircumcision.
So whatever "unity" Jesus prayed for, it wasn't a unity of looking alike and having the same practices. A church on one corner of the street intersection that wore tassels on their clothes and kept the Sabbath and observed Kosher dietary restrictions was just as unified in Christ with the church on the other corner that looks just like your congregation, as yours is with its identical sister in the next town over.
Also, Paul makes it clear in Romans 14 that unity allows for one believer to absolutely be convinced that keeping a holy day is essential to salvation, whereas another believer can be absolutely convinced that keeping a holy day is nothing. He makes it clear that unity allows for one believer to be convinced that eating certain foods is damning, while another believer can be convinced that all foods are clean, just as Jesus had proclaimed years earlier (Mark 7:19).
So whatever "unity" Jesus prayed for, it wasn't a unity based on having the same doctrine on all the issues. It was based on love for one another, *despite* doctrinal differences.
Jesus didn't say his disciples would be known by their doctrine (therefore, neither should we); he said they'd be known by their love for one another.
When he separated the sheep from the goats, he didn't do so on the basis of doctrine (and therefore, neither should we); he did so on the basis of how they treated one another.
It seems "wise" and "logical" to base unity on thinking/believing exactly alike, based on a common source of authority. But the problem with that route is that the common source of authority is not nearly so clear on all the issues as one camp or the other claims it is, and therefore they come to different conclusions on the issues. Although you have likely been trained to believe that the NT clearly teaches X, some other equally-intelligent and equally-dedicated-to-Truth seeker has been trained to believe that the NT clearly teaches Not X; and the truth of the matter is that the NT does not teach either case "clearly"; it may mention it here or there, leading you to conclude that "see, here's a clear statement", but in most cases, it just ain't clear.
This is by design. The old covenant consisted of rules and regulations written in stone for all to see; but God said the new covenant was to be unlike the old covenant, based on relationship with him, not based on rules and regulations. Paul makes it clear in Col 2:20 that Christianity is not based on rules and regulations, and he says it again in Rom 14:17, and he often speaks of how the new covenant is not based on the letter of the law, like was the old covenant, but rather on the spirit. Rules and regulations are for children; but we're now adults, expected to make good decisions based on having God's mature spirit within us, not the immature spirit of a child. God no longer treats us as children, giving us rules for bedtime and rules for crossing the street and rules for saying "thank you to the nice man"; he expects us to act like adults, as Jesus would act. If we have the heart of Jesus, the externals are just that - external. Don't make religion about the externals, like the Pharisees did; make it about loving one another and taking care of the orphan and widow and bearing one another's burdens and doing the right thing and living a clean life.
When we take a position on any particular topic, and declare that it (and others like it) are the mark of authenticity of a "real Christian", making those issues tests of fellowship, insisting that if they don't arrive at "the Truth" (as we understand it), we have already failed Jesus and his plea for unity.