M-x unload-feature will unload a elisp package. It can be used to free up some memory, or occasionally to undo some add-on behaviour. See Unloading for more.
Quite how well unloading works in practice can be a bit variable. A plain mode or set of a few commands is normally fine, anything fancy might not be setup to completely undo itself.
A package can make a function FEATURE-unload-function to hook into the unloading, to take special action needed on unload. Here’s some things not otherwise done by unload-feature,
defadvice with ad-remove-advice plus ad-activate (see AdvisingFunctions), as presumably it’s no longer wanted (and might even be harmful).For Emacs 21 which doesn’t have the -unload-function mechanism, the alternative to removing defadvice is to defang the additions with a check of boundp or featurep or something so that if the rest of the package is unloaded they don’t error out when left behind.