Wales looked set to seal victory after Andy King broke the deadlock just before the hour mark, but they conceded a soft goal in stoppage-time when Riku Riski raced past a static defence and scored.
Coleman, who signed a new two-year deal this week, had been satisfied with how his players had negotiated a tough fixture and admitted the last-gasp equaliser would provide a valuable lesson.
"We knew it would be hard and they set the pace early on," he told Sky Sports.
"We found our rhythm in the second half, but when you are 1-0 up with two minutes to go you have to see the game out.
"One slip and we've been caught and conceded a goal. The lads and I are absolutely gutted.
"In the second half we did enough to win the game, it would have been another good clean sheet for us."
Coleman was also disappointed that Wales were not given a penalty just before King's goal, when Simon Church appeared to be felled by Finnish substitute Joona Toivio.
He added: "We got our noses in front. We got a good goal and we felt we should have had a penalty also.
"You should hold on to that win with a couple of minutes to go. We should have shut up shop and we didn't do it.
"At this level it doesn't matter how well you play, one lapse in concentration and you get punished. It is a harsh, harsh level of football.
"We learn our lesson."
Finland manager Mixu Paatelainen felt a draw was a just result against a Wales side he expects to be vastly improved in the Euro 2016 qualifiers.
"I thought it was well deserved," he said.
"If you look at the Wales team individually they have some fantastic players. I think Wales will do well in the next qualifiers.
"Today we played some good football, especially as for some of the players our season is over."
The former Hibernian and Kilmarnock boss also found reason to be pleased after denying Real Madrid's Gareth Bale a goal on his return to the Wales team.
"I thought we did well, although there were a couple of occasions where he got away with his pace," he said.
"All in all I thought we pressed him quite well and limited his options. It is hard to keep him 100 per cent quiet."