Just as Colombia looked like being overwhelmed by a dangerous-looking Argentina side, substitute Martinez controlled a cross-field ball on the left wing in the 71st minute before cutting across the area and smashing it into the far top corner.
His fellow substitute Duvan Zapata made doubly sure Colombia would record their first win over Argentina since 2007 by pouncing from close range to knock in a cross from Jefferson Lerma 15 minutes later.
"We knew that it was a long time since we beat Argentina and we're delighted," Zapata told reporters.
"We'll enjoy this, it's a great result but the tournament goes on and there are lot of difficult games to go."
Both sets of supporters whipped up a lively atmosphere at a near-full Arena Fonte Nova, providing a much-needed boost for tournament organisers CONMEBOL after over 40,000 seats were left empty in Peru's game with Venezuela earlier in the day.
The fans were struggling for anything to get excited about as the first half dragged on, however.
Colombia initially looked stronger but Argentina were a different beast when Rodrigo de Paul replaced Angel di Maria immediately after the interval and captain Lionel Messi was able to find space to cause some problems.
Messi wasted the best chance of the game before Martinez's piledriver broke the deadlock when he skewed a rebound wide from close range, and the Barcelona forward also went on a thrilling solo run into the area but could not get a shot away.
MASS BRAWL
A mass brawl ensued when Messi was cut down by Colombia's Juan Cuadrado in the second half while Argentina midfielder Leandro Paredes fizzed two shots at goal.
A goal from Lionel Scaloni's side looked imminent but Colombia's usual talisman James Rodriguez split their defence with an impressive long pass to Martinez, who had come on early in the game after Luis Muriel was forced off by injury.
Martinez's stunning strike floored Argentina and although they came back with a couple more attempts, they never looked like scoring.
"We were good in the first 20 mins then we had a bad end to the half,"," said Scaloni.
"We lost the ball a lot but in the second half we were very superior to Colombia. Then, when we were playing at our best, they scored on the counterattack."
The coach also complained about the state of the pitch, calling it "lamentable".
"But the thing I will take from the game is that we were able to adjust and improve in the second half," he added. "That might not sound like much but it will help us in our next games."
Argentina, who won the last of their 14 South American titles in 1993, face Paraguay in their second match in Belo Horizonte on Wednesday.
Colombia, Copa America champions once in 2001, top Group B with three points ahead of Qatar's meeting with Paraguay on Sunday. They next play the Qataris in Sao Paulo, also on Wednesday.
(Reporting by Richard Martin, editing by Nick Mulvenney)
