The standard MS-Windows binaries is already portable, in that you can just unzip it anywhere and run it directly. To keep your .emacs.d in a directory called emacshome on your USB stick next to the unzipped Emacs, create a runemacs.bat file in Emacs’ bin/ directory with contents:
set HOME=%~dp0..\..\emacshome "%~dp0runemacs.exe" %*
and then use that instead of runemacs.exe.