It's a sad thing that the primary reason I can think of to regret getting into a fandom would be the other people. Unless, I guess, if a series went wanky and did something shitty that you didn't want to be associated with? Or an RPF person turned out to be an arsehole (hockey fandom had issues with that, I think?).

Good question though