Frank Lampard and captain John Terry scored Chelsea's goals at Stamford Bridge against a Stoke side, which had knocked Arsenal and Manchester City out of the competition.
Lampard fired home the first in the 35th minute with a deflected 20-meter drive after he was set up for the shot by Terry. In the 67th minute, Lampard's corner found Terry who scored the second with a deflected header.
Trailing to two first-half strikes by Reading's Shane Long, Villa scored three times in 11 minutes early in the second half and Carew added a fourth from the penalty spot in stoppage time.
The result means Villa gets to return to Wembley in April, after losing to Manchester United in the League Cup final at the home of English football a week ago.
Struggling near the relegation zone in the League Championship, Reading had already knocked out Liverpool in the third round of the cup and another shock appeared likely after Long's first half goals at the Madejski Stadium.
Matt Mills jumped well to head a corner into the goalmouth and Long was unmarked to deflect the ball past goalkeeper Brad Friedel in the 27th minute.
Reading went further ahead just before halftime when Jimmy Kebe broke down the right and pulled the ball back for Long to beat Friedel from 15 meters.
But Villa hit back within two minutes of the restart when Carew's pass was flicked on by Carlos Cuellar to Ashley Young, who turned the ball home at the far post.
It was 2-2 in the 51st when Carew beat two defenders in the air to meet a cross from Stewart Downing and head home, and the Norwegian striker added the third in the 57th when he guided home a cross from Stephen Warnock.
Reading was close to equalizing when Ivar Ingimarsson saw a header cleared off the line by Young. But it was all over when Carew was tripped by Ingimarsson in the area and he fired the spot kick into the roof of the net.
In Saturday's FA Cup games, Portsmouth shrugged off some of its financial woes and its last place in the Premier League by reaching the other semifinal with a 2-0 victory over Burnley. It will now play either Fulham or Tottenham, who drew 0-0 at Craven Cottage, following Sunday's draw.
The games will be played over the weekend of April 10-11.
In Sunday's only Premier League game, Everton outplayed Hull in a 5-1 victory to move up two places to eighth and leave the visitor in deeper relegation trouble.
Having missed the first half of the season through injury, Mikel Arteta scored his first of the campaign in the 17th minute when he turned home a cross from Ayegbeni Yakubu.
Although Tom Cairney made it 1-1 with a left foot volley in the 32nd minute for his first Premier League goal, Arteta restored Everton's lead in the 39th and a cross from the Spanish midfielder was headed into his own net by Hull's Richard Garcia from virtually under the crossbar.
Landon Donovan, who is on loan from the Los Angeles Galaxy, showed why the Everton fans want him to stay longer when he drove home the fourth with a right footed shot in the 82nd minute and then set up the fifth for Jack Rodwell.
While Everton moved closer to a possible place in next season's European competitions, Hull remained 18th in the standings.
With Chelsea in FA Cup action, Manchester United went top of the Premier League with a 1-0 victory at Wolves on Saturday and leads the standings by two points. Arsenal moved level on points with Chelsea by beating Burnley 3-1 but Chelsea has a game in hand.
