Straight razors need maintenance
Sure, you'd never need to buy another blade again, but the one you have will need regular care and attention. Are you really going to take a minute or two out of every morning to go through the ritual of stropping it? Are you prepared to learn to hone it (and to look after your honing stones), or send it out to get honed once or twice a year? Will you remember to oil it now and then? These aren't deal-breakers for many people, but certainly this is more effort than your cartridge razor has ever demanded.
Straight razors demand patience
With proper care and attention, a straight razor will deliver the best shave of your life, but stop paying attention, and you WILL cut yourself. Don't believe me? I have visible scar tissue on my chin from the time I was shaving in a small bathroom and managed to knock the light switch with my elbow. In my initial moment of confusion, I managed to hit myself in the face with the Wade and Butcher wedge razor pictured in this article. It was not pretty, and 4 years later I can still show you the scar. The razor and I are still on good terms, and my chin still seems to do...whatever it is chins are for... just fine. If I'm feeling impatient or hurried though, I save the straight razor shave for another day and use one of my safety razors.
On a related note:
They are less safe than safety razors (duh!)
Provided you use it properly, you'd have to be very unlucky to do anything seriously life changing/ending in the normal process of shaving with a straight razor. All the examples I can find in a quick search are due to infection rather than blood loss, and are not coincidentally from before the era of antibiotics. Still... in thousands of years of human history, someone must have managed to actually cut themselves to death with one. Nicks have the potential to be deeper as straight razors lack the safety bar that give safety razors their safety.
There is a learning curve
Getting to know a straight razor is a journey. Sure, all razors have a learning curve to some degree, but a straight razor will demand much more from you than even the most aggressive safety razor, and your very first shaves with it may not feel great. You will have to get to know the contours of your face and the growth directions of your facial hair in a level of detail you never imagined. On the bright side, if you make progress up this learning curve, then decide straight razors aren't for you, you'll be that much better at shaving when you stitch back to something less demanding.
They cost more
...initially. The real barrier here is initial cost of the razor, strop and (optionally) hones. If you're on a budget but interested in straight razors, this can be the major factor holding you back from trying. In the longer term, an entry level straight razor should pay for itself within a few months to a couple of years compared to the more expensive cartridges. Compared to a traditional safety razor with double edge blades, or a Dollar Shave Club style arrangement, a straight razor will take a little longer to pay for itself, but over a long enough time should save you $. If nothing else on this list puts you off trying one, then you probably shouldn't let cost hold you back. If it does, then maybe double edge razors are more your speed.
You may become a snob
Once you've had your first perfect shave, you won't want anything less. You've been warned. If you are used to flying hand baggage only, be prepared to start packing hold baggage just as an excuse to take your razor with you. You may find yourself sneering at products in the men's grooming aisle of your local drugstore. There is a distinct risk that you will gravitate towards specialist shaving forums to discuss the pro's and con's of particular types of razor, shaving creams, technique, etc. This may hurt productivity in other areas of your life.
You might end up with a collection
Sure, one good straight razor should be enough for anyone. But there are many really NICE designs out there; from your basic, reliable Dovo "Best Quality", to Boker's Damascus steel blade, to a Thiers Issard with genuine mammoth horn scales. And that is just new, current production razors. Vintage restoration is a whole other (and rather more complicated) ballgame. Get hooked on straight razors, and you may need either plenty of willpower, or plenty of disposable income, and quite possibly a forgiving partner.
Visitors using your bathroom may be intimidated
Using a straight razor is a little bit dangerous, albeit in a fine-if-you're-careful, vaguely-life-affirming sort of way. After all, an infected cut from a straight razor killed of Lord Carnarvon, though a mosquito, lack of antibiotics, and if you believe some people, Tutankhamun's curse share rather more of the blame. Still, some people may not understand the appeal of it. Case in point; I once, briefly left my travel shaving kit on a bathroom counter at a friends house. I later heard from my host that another guest went to use the bathroom in the meanwhile and was apparently scared by my shaving hardware. Bear in mind that my travel razor at the time was a Merkur 34 safety razor. If that intimidated them, imagine what they'd have thought of me if I'd brought one of my straight razors instead!
If after all this, you still wish to join me among the ranks of Straight Razor Shavers, may I humbly suggest taking a look at Kaliandee's selection. It currently includes popular models from Boker and Dovo, including the affordable Dovo "Best Quality" already named in the article, which is an excellent model for the beginner (I have one myself), and we have more models and brands on the way. If I've put you off straight razors, maybe a safety razor is for you?