M-x unload-feature will unload a elisp package. It can be used to free up some memory, or occasionally to undo some add-on behaviour. See Unloading for more.
Quite how well unloading works in practice can be a bit variable. A plain mode or set of a few commands is normally fine, anything fancy might not be setup to completely undo itself.
A package can make a function
FEATURE-unload-function to hook into the unloading, to take special action needed on unload. Here’s some things not otherwise done by
ad-activate(see AdvisingFunctions), as presumably it’s no longer wanted (and might even be harmful).
For Emacs 21 which doesn’t have the
-unload-function mechanism, the alternative to removing
defadvice is to defang the additions with a check of
featurep or something so that if the rest of the package is unloaded they don’t error out when left behind.