How to upgrade video card correctly?

Hey hardware gurus, I really have no idea if there is something one should pay special attention to with the process. Do I somehow need to uninstall old drivers or such? I've been running a very old nvidia card and upgrading to a more current one right now. The only thing I've always been having were the drivers from their homepage. No additional tools (nvidia inspector??), no OC'ing whatsoever. Sorry if it might sound like a dumb Q to some of you. Or just rip out the ole one, plug in the new one and good to go? Advice would be appreciated.