How loud are we talking about exactly? I try to do most of the critical work around 85dB SPL which is the level we hear relatively flat at, but even that is pretty 'loud' to me for long periods so I turn it down in between.
Taking frequent breaks helps, get up and do a lap around the house in silence. Continually drink water or tea so that you are forced to get up and go to the bathroom frequently (plus it's good for you!).
A lot of things can play into ear fatigue though, not just the sheer volume. Your monitors will play a huge part, if they tend to be fairly mid-forward and present that is going to be more fatiguing to a lot of people. And similarly, how well you're actually mixing will have an impact too. If your mix is full of overlapping elements fighting for space in the midrange and being squashed by compressors and limiters, that right there is going to get very fatiguing very quickly, and not just to you but also to the listener at the end of it... another reason to not participate to heavily in the loudness wars.