Here's something about world building that I've never really consciously considered until reading this lovely TalkToYoUniverse blog: The architecture of a city changes over time, yes? But it doesn't just change whole-sale. No one takes out an eraser and says: 'Well, we're a bit over Gothic architecture, let's just pull it all down and start again'.
No, they just build new houses in the newer styles. Thus you can get a crumbling old Roman-era fort surrounded by, I dunno, wattle-and-daub houses with a Gothic-era church sat in the middle of the whole town*.
Sometimes they renovate older buildings, perhaps leaving the gargoyles, but replacing the Gothic windows with ultramodern tilted, computerised windows that adjust the darkness gradient depending on how much light should be let in (an extreme example, but I'm hyped up on sugar so there you go). More often they'll renovate the interior, or part of the building (especially if it's been damaged by fire or the elements) than the whole thing. Buildings in the midst of renovations would be interesting to see in Fantasy as would buildings that are very old and aren't up to code and might dump the inhabitants through three floors into the basement if they're not careful.
Sometimes they'll just build on top of / over the older construction. The blog post mentions a Roman construction underneath a small, ancient church, that can be found inside a bigger, more modern church! Sometimes it won't be that obvious. Sometimes it's only when the owners decide to open up that painted shut door, delve into the basement, renovate the property, that they locate old rooms and basements and sub-basements from where the building had been build on other buildings or rooms pasted over.
So yes, have a think about the next time you're designing a place, whether in Science Fiction, Fantasy, or Contemporary fiction, and think about just how likely it is that all the buildings in that village, town, or city look the same!
*This is based on examples to make you think. I haven't researched the architecture here and cannot vouch for the authenticity of these buildings sitting side by side! These are really, really, really not examples of my researcher credibility...