Hearts defender John Souttar headed an 89th-minute winner as Ange Postecoglou's difficult start to his Celtic reign continued into the Premiership season.
Souttar rose to head home Michael Smith's free-kick and earn the hosts a 2-1 victory in front of 5,272 fans at Tynecastle.
Gary Mackay-Steven had given Hearts an early lead as Swedish defender Carl Starfelt endured a torrid opening to his Celtic career, 24 hours after his first training session.
Anthony Ralston netted an impressive equaliser in the 54th minute but the hosts responded well as Souttar consigned Postecoglou to a second defeat this week following Celtic's Champions League exit against Midtjylland.
Postecoglou gave starting berths in central defence to both Starfelt and Nir Bitton while James Forrest made his first start of the season.
Starfelt made a fresh-air clearance inside 60 seconds and the Swede looked suspect again as Hearts took the lead in the eighth minute.
The new arrival from Rubin Kazan failed to halt Josh Ginnelly's progress in an attempted tackle and then went to ground in a desperate attempt to stop Liam Boyce capitalising. The ball ultimately broke for Mackay-Steven to stroke home from six yards.
David Turnbull squandered Celtic's best first-half chances, twice failing to hit the target in space on the edge of the box and also hitting a free-kick into the wall.
There were other half-chances for the visitors before Celtic levelled after some positive play from Ralston with the defender getting into an attacking position, which Postecoglou demands of his full-backs, before curling home with his left foot.
Souttar's header, however, proved decisive and broke Celtic hearts.
"It's a set-piece where we should be organised," said Postecoglou.
"We kind of knew that was going to be a major threat from them and we dealt with it fairly well the whole night but we will analyse that last one.
"But you keep the opposition in the game if you don't take control of those areas where you are dominant.
"I felt in our three competitive games we are not getting the rewards for our play. We got into so many good crossing areas and crosses ended up out on the touchline, and shooting opportunities.
"For me, it's probably the area we need to improve the most. Defensively it's not like we are being put under pressure for periods of times in any of the games."
Champions Rangers made the perfect start to their title defence with a 3-0 win over Livingstone.
Ianis Hagi, Scott Wright and Kemar Roofe got the goals for Steven Gerrard's men.
In other matches, Ross County and St Johnstone played to a goalless draw while 10-man Dundee drew 2-2 with St Mirrren.
Australian Associated Press