I'm pretty sure it's each individual class has it's own "fatigue" as well as your physical level has its own too. Obviously you can't switch your physical level in any way, so you won't be able to avoid that fatigue, but you can go ahead and continue raising your rank with a different class just don't expect to be getting a bunch of attribute/resistance points to spend while you're doing it.
edit: beat'd
I'm wondering if the timer counts down as you simply have a specific weapon/tool type equipped... if so, that's just going to require everyone carry around a tool/weapon around for a rank-1 class they don't care about and macro to it any time they aren't fighting/gathering/crafting to delay the timer for the jobs they are trying to grind exp for.
edit: nevermind, elmer cleared it up~
XI Wiki



