For MS-Windows

The standard MS-Windows binaries is already portable, in that you can just unzip it anywhere and run it directly. To keep your .emacs.d in a directory called emacshome on your USB stick next to the unzipped Emacs, create a runemacs.bat file in Emacs’ bin/ directory with contents:

set HOME=%~dp0..\..\emacshome
"%~dp0runemacs.exe" %*

and then use that instead of runemacs.exe.