The sixth seed eventually won an intriguing claycourt duel 4-6 6-0 6-3 7-5 in front of an appreciative crowd on the stunning Court Simonne Mathieu, but it was a sweat.
Claycourt specialist Dellien, whose opening-round win was the first by a Bolivian player in a Grand Slam tournament since 1984, played a sensational 10th game to take the opening set despite needing treatment on his ankle.
Tsitsipas, 20, then took control with his aggressive all-round game but Dellien, who produced a succession of stunning drop shots, threatened to drag the match into a decider.
The world number 86 broke at the start of the fourth set, much to the annoyance of Tsitsipas who received a warning for slamming his racket into the net tape.
Dellien had the chance for a double break at 3-1 but Tsitsipas survived that mini-crisis and regained control.
Tsitsipas failed to convert three match points when Dellien, who quit tennis in 2016 to set up an ice company, served at 4-5 but he sealed victory two games later with a clubbing forehand.
He said the surface on the stunning new Court Simonne Mathieu had felt slower than other courts at Roland Garros, forcing him to engage in long rallies with a player well-grooved in the art of grinding it out on the dirt.
"It was much slower. Even my fitness coach told me the fastest serve today was 180 kilometres per hour, which is very rare to see," Tsitsipas told reporters.
"Clay courts require patience more than any other surface and right strategy. If you're playing the wrong way and your opponent can dictate what you're doing, then you're screwed."
Tsitsipas, tipped as the youngster most likely to have a title run this year, will face Serbia's Filip Krajinovic next.
(Reporting by Martyn Herman; Editing by Christian Radnedge and Toby Davis)
