The biggest shakeup of Trump's Cabinet since he took office in January 2017 was announced by the president on Twitter as his administration works toward an unprecedented meeting with the leader of North Korea.
Trump tapped the CIA's deputy director, Gina Haspel, to replace Pompeo at the intelligence agency.
Tillerson's departure capped months of friction between the Republican president and the 65-year-old former Exxon Mobil chief executive, who had no diplomatic or political experience before becoming secretary of state. The tensions peaked last fall amid reports Tillerson had called Trump a "moron" and considered resigning.
"We got along actually quite well but we disagreed on things," Trump said on the White House lawn on Tuesday. "When you look at the Iran deal: I think it's terrible, I guess he thinks it was OK. I wanted to break it or do something and he felt a little bit differently. So we were not thinking the same."
Trump said he and Pompeo have "a similar thought process."
Pompeo, a former Army officer who represented a Kansas district in Congress before Trump chose him to lead the CIA, is seen as a Trump loyalist who has enjoyed a less hostile relationship with career spies than Tillerson had with career diplomats.
There had been reports in recent months that Tillerson could be replaced but senior State Department officials said he did not know why Trump pushed him out and that he had intended to stay in the job.
BLAMING RUSSIA
On Monday, Tillerson blamed Russia for the poisonings in England of a former Russian double agent and his daughter. Earlier at the White House, press secretary Sarah Sanders had refrained from saying Moscow was responsible.
A senior White House official said Trump asked Tillerson to step down on Friday but did not want to make it public while he was on a trip to Africa. Trump's Twitter announcement came only a few hours after Tillerson landed in Washington after the trip, which had been cut short.
The official said Trump wanted Pompeo in place before the U.S. president's planned talks with North Korean leader Kim Jong Un and trade negotiations.
He appeared out of the loop last week when Trump announced he had accepted Kim's invitation to meet before the end of May.
Stocks pared gains on the Tillerson news, but then stabilized. S&P futures were last up 0.3 percent.
"Mike Pompeo, Director of the CIA, will become our new Secretary of State. He will do a fantastic job! Thank you to Rex Tillerson for his service! Gina Haspel will become the new Director of the CIA, and the first woman so chosen. Congratulations to all!" Trump said on Twitter.
Tillerson joined a long list of senior officials who have either resigned or been fired since Trump took office in January 2017. Others include strategist Steve Bannon, national security adviser Michael Flynn, FBI Director James Comey, White House chief of staff Reince Priebus, health secretary Tom Price, communications directors Hope Hicks and Anthony Scaramucci, economic adviser Gary Cohn and press secretary Sean Spicer.
OUT OF THE LOOP
Trump publicly undercut Tillerson's diplomatic initiatives numerous times and Tillerson appeared out of the loop last week when Trump announced plans to meet with North Korea's Kim.
In December, Tillerson had offered to begin direct talks with North Korea without pre-conditions, backing away from a key U.S. demand that Pyongyang must first accept that any negotiations would be about giving up its nuclear arsenal.
The White House distanced itself from those remarks, and a few days later, Tillerson himself backed off.
Several months earlier in Beijing, Tillerson told reporters the United States was directly communicating with North Korea but that Pyongyang had shown no interest in dialogue. Trump contradicted Tillerson's efforts a day later.
"I told Rex Tillerson, our wonderful Secretary of State, that he is wasting his time trying to negotiate with Little Rocket Man," Trump wrote on Twitter, using a pejorative nickname for Kim. "Save your energy Rex, we'll do what has to be done!" Trump added.
Tillerson had joined Defense Secretary Jim Mattis in pressing a sceptical Trump to stick with the agreement with Iran and other world powers over Tehran's nuclear ambitions and he has taken a more hawkish view than Trump on Russia.
If confirmed by the U.S. Senate after a hearing scheduled for April, Pompeo will be taking over a State Department shaken by the departures of many senior diplomats and embittered by proposed budget cuts.
Many Democrats in Congress expressed dismay at Tillerson's firing, saying the change would sow more instability in the administration at a crucial time.
Lawmakers from both major parties have been strongly critical of cuts at the State Department and of the administration's failure to fill dozens of open jobs there.
Tillerson faced a much tougher confirmation that most nominees to be secretary of state last year as Democrats grilled him about his oil business ties to Russia and refusal to recuse himself from energy-related matters related to Exxon Mobil while in the job.
But over time, many lawmakers grew to appreciate Tillerson as a relatively steady hand in the chaotic Trump administration.
"He represented a stable view with regard to the implementation of diplomacy in North Korea, Iran and other places in the world," said Senator Ben Cardin, the top Democrat on the Senate Foreign Relations Committee during most of Tillerson's tenure at State.
