Just going off of what Midgardsormr says, Ishgard done fucked up a long time ago. The JP translation helps drive the fact home that Ishgard has "sins" to "atone for" while the english localization says, "Sons must answer for their fathers' misdeeds..." Just to kick it off, I think those misdeeds are turning traitor.
We know for a fact that Ascians are in control of Ishgard. We also know that Ishgardian policy on things like outsiders and lending aid to other city-states has been relatively unchanged for some time now. They've been religious zealots and xenophobes fora quite some time. This makes me believe that the Ascians have had control of the Holy See for a rather long chunk of history as what good would controlling the leader of a militaristic city-state be if you haven't manipulated it into some kind of tool?
We also know that Midgardsormr, if not his brood, are allies of Hydaelyn and protectors of the planet at any cost. As far as we've been shown, the greatest threat to Hydaelyn herself are the Ascians and Zodiark.
To tie the points together, I don't think it'd be too ridiculous to think that Ishgard and the Dravanian's were allied at some point in the forgotten past, possibly fighting a war against some larger threat (likely something to be revealed in Heavensward, if it's not Zodiark-related). Somewhere along the road, Saint Shiva made a pact ("lay") with the Dragons to gift some of their power to her people to either cement the alliance between the two peoples and or simply bolster the fighting force. Hence, Au Ra looking like a crossbreed between Hyur/Elezen and Dragons. Somewhere along the line, Ishgard was infiltrated by Ascian influence (or just turned into dicks) and betrayed the Dragons, pulling an Order-66 on the Dragons and exterminating a large contingent of them. Saint Shiva's people fled to Othard to escape their now "heretic" status while the Dragons swore retribution upon Ishgard itself.
Now the Dravanian Horde has sung the Dragonson to begin war anew and is intending on destroying Ishgard for the most heinous crime of all in any honor-bound society, turning traitor. The Au Ra have heard the call to war and are coming to lend aid to the races of man because they don't think Ishgard necessarily needs to be destroyed. Just those in power/under direct Ascian influence.
Midgardsormr is following you around because he wants to know if you have the strength to stand up to the Ascian influence yourself and if there's a way to destroy Ascians at all. You were chosen for a reason or something.