Yeshua had just taken his disciples to Caesarea Philippi, which was built on a massive rock, and which was the headquarters for the cult of Pan, the half-man, half-goat god. In the side of this rock was a cave from which issued the beginnings of the Jordan River, and which was incorporated into Pan-worship as the Gates of Hades, through which Pan descended into the depths of the earth every Winter and ascended every Spring. Before this cave, on a flat space of rock, was a temple to Pan, in which massive orgies (of every kind of sex, including homosexuality and with goats) were conducted in worship to this god of earth fertility, to assure good crops.
... upon this rock I will build my church; and the gates of Hades shall not prevail against it.
I suspect that Yeshua and his followers were standing on this rock foundation, his young disciples awed with confusion, interest, and guilt for being at such a place of sin, when Yeshua announced, that here, on this rock, he would build his regularly-assembled community, and the gates of Hades can't do a thing about it.
It would be like your youth minister taking the senior boys to the local strip club before regular business hours, standing on-stage, and announcing that here, on this stage, the assembly of Christ will meet, and there's nothing the forces of worldliness can do about it. Everyone would be shocked, and think the youth minister was corrupting the youth, as well as being out of his mind.
Interestingly, Christians often assemble at this site of former demon-worship now, following in the footsteps of their Messiah. Pan-worship has faded away; Christ's assembly is overpowering the forces of Death.