The tower and city have been hit many, many times by the Fallen. Twilight Gap, the Battle of Six Fronts, etc. the Hive drove us off of the Moon. The Vex attacked us when we were exploring our own civilization's ruins in Venus. Yes, we infiltrate the Cabal base on Mars. But it's to attack the Black Garden - a place that the Stranger, the Awoken, and the Cabal themselves want eliminated.