Key points
The dominant Edinburgh side surged eight ahead of Celtic at the top of the Scottish top flight to strengthen their early title credentials and deepen the malaise around the Celtic team.
A superb goal from Kyziridis and Lawrence Shankland's penalty gave Derek McInnes' side a 3-1 lead that the reigning champions could not overcome.
Dane Murray - one of three youngsters in the Celtic starting lineup - slashed into his own net early on to put the home side ahead at a energetic their home ground, and it was the 22-year-old's foul that led to the spot-kick.
A stylish finish from McGregor of the season had swiftly equalized Hearts' opener, and Benjamin Nygren almost put Rodgers' team in front, opting for finesse over power and allowing Alexander Schwolow to save from close range.
But the unstoppable Hearts side regained dominance to consign Celtic to successive league defeats for the first time since last year.
It also gives the Hearts a commanding advantage at the summit.
The last time they were so far clear was in 2005-06 under George Burley, when they ultimately finished second - one of just four occasions that a team has not secured the title after building such an advantage in the last three decades.
The Hearts coach, whose Aberdeen side were second to Celtic on multiple times, has been at pains to play down title talk, as Hearts seek their maiden league title since the 1960s.
Rodgers has seen it all before at the club - both perfect and shaky beginnings to a season. He's yet to finish second best.
But on Sunday's evidence, this Edinburgh outfit showed they deserve serious consideration.
Hearts started with the self-assurance of a side undefeated in the league this season - the popular attacker poked a cross in for the striker to stab wide.
Celtic didn't help themselves with some unusual sloppiness in possession and they faltered under their initial test.
A headed effort from Braga was tipped onto the woodwork by the Celtic keeper and Shankland's return ball was diverted goalwards by the unfortunate Murray, in for the absent center-back.
Celtic found their rhythm and started stringing passes together. From a flowing attack, Sebastian Tounekti released Tierney to square for McGregor, who slotted home.
Aside from the opportunity for Nygren, Celtic struggled to create much for striker Kenny or his substitute Yamada amid the absence of stricken attackers their key strikers.
The home team rediscovered their energy and versatility in creating chances in the opposition half. The Greek forward, one of the finds of the club's scouting network, fired a low shot past Schmeichel from the just outside the area.
Almost straight away, the defender tripped Braga in the inside right channel and the penalty taker scored. And it could've been an more comprehensive victory, had substitute Ageu not been denied by the shot-stopper.
The visitors' performance after the break was partially encapsulated in two incidents involving big-money recruit Engles.
The midfielder's wayward free-kick delivery was in sharp opposition to his European display in the continental victory over Sturm Graz and the Belgium international was then yellow-carded for a cynical foul.
Hearts head coach Derek McInnes: "The effort from the squad is evident, the togetherness. That's not sufficient on its own to win games week in, week out.
"What we have got, is skillful players. You're seeing that from a lot of players who have been here before. Some of the new signings are eager to show that.
"We're in a positive moment at the minute, but it's just one win. It's crucial for us to maintain this form. We've got loads to do, loads to do. After the first two rounds is typically a benchmark of your standing."
Celtic manager Brendan Rodgers: "The second goal was disappointing from our perspective. We didn't move when the ball went back as a defensive unit so when they pass the ball, they ought to be caught offside but weren't. We have got to improve with the shot.
"There's still so many points to play for. It's only the beginning. It's only nine games in. Obviously, being eight adrift is not what we would want. The positive is there's numerous fixtures to play.
"We now have to focus on the here and now and look to achieve greater stability and better performance in our play."
Celtic are back at home on Wednesday against promoted side Falkirk (evening kick-off). At the same time in the top division, Hearts are at their opponents' ground.
A digital strategist with over a decade of experience in transforming brands through innovative web solutions and creative marketing.