The only evidence to suggest the Vex were here prior to the Golden Age is found in the ruins on Venus, which would be the Citadel and the VoG. If the Vex are time travelers, its hard to argue at which point in time they came into existence and at which points in time, past and future, that they traveled.