http://discovermagazine.com/2004/sep/einsteins-lonely-path Lee Smolin: "Quantum mechanics was not the only theory that bothered Einstein. Few people have appreciated how dissatisfied he was with his own theories of relativity. Special relativity grew out of Einstein's insight that the laws of electromagnetism cannot depend on relative motion and that the speed of light therefore must always be the same, no matter how the source or the observer moves. Among the consequences of that theory are that energy and mass are equivalent (the now-legendary relationship E = mc2) and that time and distance are relative, not absolute. Special relativity was the result of 10 years of intellectual struggle, yet Einstein had convinced himself it was wrong within two years of publishing it. He rejected his own theory, even before most physicists had come to accept it, for reasons that only he cared about. For another 10 years, as others in the world of physics slowly absorbed special relativity, Einstein pursued a lonely path away from it."
"Within two years of publishing" special relativity Einstein realized the speed of light varies with the gravitational potential:
It is easy to show that, if the speed of light varies with the gravitational potential, then, relative to the observer, it varies with the speed of the observer. That is, in 1907 Einstein already knew special relativity was wrong.