I'm gonna go against what everybody else has been saying. I used to jump around all over the place and then I would be over tweaking until the cows came home. These days I leave it on medium (80-90db) and call it a day, until the end of a mix. It's not about losing clarity at low volumes or not enough bass at low volumes. It's about finding the factual loudness where human hearing is relatively the flattest across all frequencies. No volume is perfect, but I see a lot of this low and high volume debate, and if my own experiences are to judge, the advice here is a little misleading. When you listen to a number of mixes across the same set level, you then have a basis for your own mix, but if you keep jumping around all the time than you lose that reference. And that has the potential to have you chasing your own tail.
Just my two cents.
It seems you have a way, and that's cool.
I've had numerous people, including some to have gone on to actually become professional producers and dj's, tell me that for referencing a mix it's better to keep it at a low volume.
Now, i think the low volume thing works if you have a lot of work to do or you're doing surgical sort of stuff that doesn't require headphones. But definitely seeing how it sounds loud and tweaking it there is not a bad thing.
You're getting different and varied reference frames rather than just one perspective. It's like when you're trying to get data for what ever survey you might have a better idea of what it is you're trying to find with a larger sample size than if you were to keep it at a very low number.
But, i think the problem is not knowing where the initial reference frame should be placed. And i think that's generally reserved at a low volume (on my interface that's at the 9 o clock setting, and then with dim mode it's slightly higher and i can achieve lower volumes once at that stage).