United left back Evra was given a hostile Anfield reception after the Frenchman's involvement in a spat with Liverpool's Luis Suarez in October, which led to the Uruguayan forward's current eight-match ban for racial abuse.
Racism allegations were also the backdrop to Chelsea's 1-0 win at Queens Park Rangers where John Terry, in court next week after being charged over comments allegedly made to QPR's Anton Ferdinand in October, was kept away from his fellow defender as the clubs decided not to shake hands before kickoff.
Like Evra, Terry was constantly heckled by the home fans whose wrath switched to the referee on 62 minutes when he awarded a soft penalty to Chelsea after Clint Hill was adjudged to have pushed Daniel Sturridge. Juan Mata converted the spot kick.
Despite the lucky break, Chelsea deserved to win but even Liverpool's captain Steven Gerrard reckoned his side were fortunate to get past United after Daniel Agger's opener was canceled out by Park Ji-Sung before Kuyt's late strike.
Asked if Liverpool had merited victory, Gerrard told ITV Sport: "Not on possession, probably not on general play; we've got to give Manchester United credit, they controlled large periods of the game today but the most important thing was the result and we got there in the end." United boss Alex Ferguson, whose side are also out of the Champions League, agreed.
"I don't know how we lost it, it was a really good performance by us," he said.
Liverpool were woeful in a league defeat at struggling Bolton Wanderers last weekend but eliminated league leaders Manchester City on Wednesday to reach the League Cup final and have now beaten bitter rivals and league champions United.
Kenny Dalglish's side, still missing the verve offered by Suarez, took the lead on 21 minutes when Agger headed in a corner which struggling United goalkeeper David De Gea failed to collect after he was crowded out by the hosts.
United, without the injured Wayne Rooney, hit back through a crisp strike inside the near post from Park on 39 minutes and the visitors had other good chances, with Antonio Valencia hitting the post.
However, Liverpool had the last laugh to send Anfield into raptures and prompt more hopes in the city that Dalglish can bring back the glory days from his previous spell in charge.
Striker Andy Carroll, widely criticized for his form since a big move from Newcastle United last year, flicked on the ball to leave Evra out of position and Dutchman Kuyt slammed it in when De Gea could again have done better.
The goal avoided a replay at Old Trafford next month which Suarez would have been eligible to play in.
Tottenham Hotspur are also in round five after beating second-tier Watford 1-0 on Friday, when Everton overcame Fulham 2-1.
The remaining fourth-round fixtures were taking place later on Saturday with two games on Sunday, including Arsenal against Aston Villa.
