When they sat side-by-side on "The X-Factor," it was all work. But now that she's left the judging panel, perhaps it's time for a little play? When others doubted, he only had nice things to say about Britney, like this gem from 2012: "There's kind of a mystique about her, which probably is what attracted me to her in the first place. I knew what I read but I didn't know what was going on in her head and I was kind of fascinated." At 53, Simon may be a bit of a father figure -- but would that really be a bad thing?