How do you incrementally upgrade a horse and cart without the result looking silly?
Your rig as it stands is a fairly balanced set of components (chosen to meet a price-point) which work well together. Replacing just one of them with something "much better" would generally mean that the extra money you spent on "much" is wasted, because the rest of the system can't keep up.
You can put an engine in the wooden cart, but it still won't do more than 20mph, otherwise the stresses imposed by the engine would just tear it apart.
> This computer only has a 250W ATX power supply,
Top rated graphics cards would use all of that by themselves
Typically, if your rig needs X-Watts of power, then you need a good power supply (not something cheap in a "budget" machine) rated at about 2X-Watts (so it is loaded at 50%).
The max rating is to cover transients without it shutting down completely. If you run it close to it's max from the outset, it will get very hot (and inherently less efficient with it), the fans will be audible in the next room and it's life will be measured in months at best. The system will be prone to shutdowns and reboots in the meantime.
250W will only be enough to cover what you have, and replacement on a "like-for-like" basis. Significant upgrade would require a new PSU.
TBH, finding something 6 to 12 months old on ebay which has a much better base spec might be the way to go, for not a lot more than what you might pay for a spiffy new graphics card (which you couldn't use all of anyway), and a new PSU.