(For the weapons of our warfare are not carnal, but mighty through God to the pulling down of strong holds;)But I'd like to look at this verse in its broader context (from the HCSB).
1 Now I, Paul, make a personal appeal to you by the gentleness and graciousness of Christ—I who am humble among you in person but bold toward you when absent. 2 I beg you that when I am present I will not need to be bold with the confidence by which I plan to challenge certain people who think we are behaving in an unspiritual way. 3 For though we live in the body, we do not wage war in an unspiritual way, 4 since the weapons of our warfare are not worldly, but are powerful through God for the demolition of strongholds. We demolish arguments 5 and every high-minded thing that is raised up against the knowledge of God, taking every thought captive to obey Christ. 6 And we are ready to punish any disobedience, once your obedience has been confirmed.
7 Look at what is obvious. If anyone is confident that he belongs to Christ, he should remind himself of this: Just as he belongs to Christ, so do we. 8 For if I boast some more about our authority, which the Lord gave for building you up and not for tearing you down, I am not ashamed. 9 I don’t want to seem as though I am trying to terrify you with my letters. 10 For it is said, “His letters are weighty and powerful, but his physical presence is weak, and his public speaking is despicable.” 11 Such a person should consider this: What we are in the words of our letters when absent, we will be in actions when present.The preceding chapter/context, which Paul thought might cause some to "think we are behaving in an unspiritual way", was focused on the material ("non-spiritual") aspects of fund-raising for poverty-stricken saints in another part of the world.
In other words, Paul was conducting a fund-raiser, and after making his presentation, said, "I beg that I won't need to scold those who think we're being unspiritual. For even though we're living in material bodies, with material needs, our focus is not on waging war in material ways - the weapons of our warfare are not worldly. Rather, we use our words to convince people to obey Christ. But once you belong to "us" (vs 6), to "Christ (v. 7), we're ready to use our authority as needed (v 6, 8ff)."
To press Paul's statements in this context to mean that we are never to take up arms for worldly warfare is to move Paul's statements into a context which is foreign to the context at-hand. It would be like pressing Jesus' statement, "Judge not, lest ye be judged" (Matt 7:1), to mean that we should never make judgment calls, even though within the same paragraph Jesus is telling us to not cast our pearls before swine, which requires just that, a judgment call.
Paul is speaking of the church's "Mission Statement"; he's not making a prohibition. The mission of the church is to persuade people, not to convert them at the point of a sword. The business of the church is to change minds, not to subjugate them by force.
That's what the context of this passage says. The context does not address being a soldier, or policeman, etc.
However, the context of other passages does address the issue.
Acts 10 strongly indicates that being a soldier is consistent with being a Christian, as the Roman Centurion Cornelius converts to Christianity.
Also, when the soldiers came to John the Immerser and asked what they must do, John did not tell them to quit the army; he told them to not use force for selfish purposes (Luke 3:14).
If being in the army was unacceptable as a follower of Christ, it seems to me that the one paving the way for that Christ would have at least hinted that such a change was coming, and it seems to me that Luke would have made it clear that Cornelius had to quit his commission to become a Christian.
So how do I understand 2 Cor 10:4? I understand it to be a mission statement for how the church is to grow: via persuasion, not by force. But that does not address the mission statement for how human government is to punish evildoers, which, by God's decree (Gen 9:5-6; Rom 13:4), uses force. These are two different contexts, and this passage only addresses one of them.