Telemetry data tells us a few users (< 0.1%) have their XML Language Server restarting every minute. Since this is a slower rate than 5 times in less than 3 minutes, it doesn't bubble up to the user.
Still, this indicates a problem and I have no idea what can cause it. Until we find a solution, we need to get better information (from telemetry) to figure out what's causing this.