On Windows use Stop and exit from either the Test menu or the Tray Icon. On Linux use Ctrl C and select Exit (Option 6 - I believe). The client should stop and exit properly on normal shutdown.
If you have set the client up to use two backup files then you should also avoid one of the few remaining problems with the client, and that is a corrupted save-file. It has happened to me once in all the time I've been running Prime95, so it is fairly rare.
The client will (or should) continue from the latest save-file.
heretic