Their split comes just a few weeks after Millson attended the Emmy Awards in Los Angeles with the Best Dramatic Actress nominee, 30, who was being honored for her critically acclaimed Downton Abbey performance.
"He's heartbroken," a source close to the actor told the paper.
Millson has since taken to Twitter to lament the supposed end of his relationship with the British beauty.
"Just wait for someone worth being in a relationship with," he wrote. "Stop wasting your time on the ones who don't deserve you."