There are arguments for both sides, but the one thing you should never do is play from memory with the score in front of you (as an aide-memoire should you get stuck). The time spent having the memory lapse and then trying to work out where you are can be very embarrassing. Or you jump forward and your pageturner has to work out where you've gone to..
I am guilty of that and I agree to a point. Yes, if you've memorized the piece, and then at the last moment you decide to bring the score up to the piano with you, then you may have a problem. Like you said, if you're not used to playing from the score, then if you are playing by memory and then look up at the score you may not know right away where you are on the paper. So I used to feel like either you plan on using the score, or you don't - it's that simple and you should stick to your decision. However, I'm changing my mind now. I think I'd rather have that pause in finding my place in the score, than to be totally stuck if I've had a major memory lapse and then just sit there mortified. That's happened to me. I was playing a big piece and got stuck in the middle somewhere and my fingers just stopped and hovered over the keys for what felt like eternity. Somehow I managed to backup and restart that section and got through all the way to the end but I was so shook up mentally that I couldn't concentrate and really didn't have a clue as to what I was playing. My fingers were just sort of on automatic pilot and because of that my playing was not musical at all. I felt terrible because I knew I could have played the piece much better if all that hadn't happened.
Ok. Two comments/suggestions.
1. If you are going to play an (already memorised) piece from the score in concert, do yourself a big favour, and practice it with the score. Retain the memorisation by all means, but having run through it repeatedly from the score means you are far less likely to get lost in the event of accidents.
2. I strongly believe that if you have a memory lapse, don't go back. Go forward. Mentally select a nearby bar which constitutes something structurally significant (this shouldn't be too difficult if you have thought about the architecture of the piece and if you keep calm). Find a way to connect, harmonically or melodically, even rhythmically, wherever you currently are at on the piano to that bar, join them up and continue. It looks much more convincing to an audience than stopping and starting. Half of the time they won't even know! (Disclaimer: I'm good at improvisation, so the process seems easier to me than maybe it actually is).