For most of 90 minutes, every eye at Anfield was trained on Virgil van Dijk.
The former Celtic and Groningen central defender played well for the Reds and scored the victor with a brilliant header in the 84th minute.
The Dutchman, signed last week from Southampton for a reported fee of 75 million pounds, rose inside the area to head home an Alex Oxlade-Chamberlain corner kick.
Chris Sutton was also following the match, and the former Celtic striker has pointed out the 26-year-old centre-back's development at the Glasgow giants.
Everton barely threatened in the first half but was much more risky after the break, equalizing through Gylfi Sigurdsson after a counterattack following a Liverpool corner. "To score a goal is even more special".
The most explosive incident in a typically high-octane derby came just after Milner's goal when Holgate shoved Roberto Firmino into - and over - the advertising boards.
A swift counter-attacking move saw Everton get an unlikely equaliser through Gylfi Sigurdsson after the interval.
Milner made no mistake with the spot-kick as he dispatched the ball past Everton goalkeeper Jordan Pickford with confidence.
Liverpool hasn't lost to Everton in 16 matches - a club record - and Everton's winless run at the home of its great rival extended to 21 games, dating back to 1999.
Yannick Bolasie won possession and found Rooney's replacement Ademola Lookman, who showed superb awareness to break forward before he found Phil Jagielka on the edge of the box.
The Reds progressed to the fourth round of the competition after Virgil van Dijk headed a victor on his debut.