My story is eerily similar to DN's. I cannot fully explain the psychological effect of losing your life partner due to geographical restrictions imposed by your specialty choice. I struggled with the few job offers I had out of residency, all in undesirable locations. The one I finally took, I figured I would be at least well paid and valued there. Wrong. Not only did I lose my SO but I was despised by the sociopaths that ran the place who conspired to make my life a living hell until I quit. Back to starting over 3 years later, older, and in a personally and professionally far worse situation than what you started as. I'm almost 40 and all of my possessions from the houses I owned in med school and residency are still in long term storage. I've been living in short term rentals/extended stays since due to unstable job situation. Think this is how I envisioned my life turning out post-residency?
The OP absolutely should continue with radiology. The trap he/she is falling into is still thinking like a med student regarding what is academically interesting and not viewing it as a job. Radiology will allow you to live just about anywhere while earning at a minimum double what a PCP makes, who can also live anywhere. That is worth its weight in gold. One of the few friends I made at my first job was a radiologist. He wasn't extremely enthusiastic about his career choice and having to read all day grinds on him. However, he is on vacation once a month. Just called me from a helicopter ride in the Grand Canyon. He's always doing something fun and said his philosophy was "don't kill yourself working hard so you can enjoy your retirement but instead try to live a little bit of your retirement all the while you are working." The group I work with right now doesn't really believe in vacation and tried to gaslight me telling me that "it's unheard of for a doctor to take more than 4 weeks of vacation." Radiologists regularly take 12+. As someone who previously had this much time off, its effect on your well-being cannot be understated. It's extremely rare to find a rad onc job that deviates from the 8-5 M-F 25 days of PTO model, and actually getting and scheduling time off can be sometimes impossible. I have a partner who has not been on vacation in 3 years. I'd like to take a month and go to South America and do a Spanish immersion course. Think I can do that in rad onc? Fat chance. When they ask about your hobbies on a job interview, you will get funny looks if you say you like to travel, or maybe a story about how they were able to take a week-long trip to Cancun a few years ago, like that's what you meant and to suggest that's about all you can expect. In radiology or anesthesiology, travelling extensively is a normal and common extracurricular activity. Once, on an interview, I mentioned that I am out of clinic on Fridays and often drive out and go skiing for a long weekend. The oncology manager (an RN) became visibly angry and started huffing and puffing and, said "I don't understand that, we work five days a week here, FIVE DAYS A WEEK." I did not get the job, and I obviously would not have wanted it anyway. I had similar experiences elsewhere. Silly me with the expectations of being an independent professional after a decade of training and setting my own schedule.
You will pigeon-hole yourself in rad onc. I probably can never return to the part of the country I grew up in where my parents still live. My residency location and prior work experience mean my C.V. is not considered anywhere outside of this region. After all, as a community doctor, what can you even put on your C.V. to distinguish you? I don't publish, and I can't write "I'm a really good doctor and care about my patients, trust me." So my C.V. just lists my residency location and work experience, and that's it and bouncing around jobs in a part of the country people make fun of looks really bad. I applied for a job in a very desirable location once and a partner actually emailed me saying how competitive it was and thanks for applying, but basically they would never consider somebody like me. Why not? I'm BC, extremely efficient and productive, and take pride in delivering high quality care. My patients love me. But my C.V. does not say Harvard and/or MD Anderson on it. That's the stupid game we play.
You won't have this problem in radiology. The truth is, rad onc residency can be extremely easy, especially these days. You don't have to study every night (I didn't). You can cram for boards (a brutal month of 8 hour studying days per exam, orals was more like 12). The education process is stupid. The residency doesn't prepare you to practice independently. It's mostly med-student level shadowing and note-writing with educational activities focused on preparing you to pass 4 stupid exams testing 30 year old trial data. I don't know about other people, but I don't have a great long-term memory when it comes to memorizing. If I am not constantly studying flashcards, I will forget exam trivia. I cannot tell you the outcomes of the VA Larynx trial off the top of my head anymore, let alone p-values. I cannot tell you anything beyond the basics about physics and radiobiology. Signaling pathways? Forget about it. Long gone. Yet, I can practice radiation oncology, a great deal of which I taught myself my first year out. Funny how that works. It was college all over, wasting years memorizing things you will just forget and never use. Wasting years when you are 19-22 isn't a huge loss. It is when you are 30-35. Those are years when you are supposed to be building families and careers, not acting like a college student. In radiology, your studying, while intense, will hopefully be something you retain and put to use every day.
DO NOT SWITCH INTO RADIATION ONCOLOGY.