[airyphary]

You would need the "Fullscreen" button, or whichever button is mapped to that feature. On a keyboard it's the Tab key that serves this purpose.

Or you could avoid pressing the back button - a more intuitive option is to stop the video entirely, do whatever you need to do, and then when you go to play the video again it will have the option to resume at the place you left it.