The IOC unveiled a list of 37 refugees who are currently receiving Olympic scholarships and who will attempt to make the final cut for Japan next year.
Among those are the 10 athletes who took part at the Rio Games in 2016.
"At this stage we cannot say how big the team will finally be," said IOC President Thomas Bach. "What we can say with some confidence is that it will be bigger than the team for Rio 2016."
"All these 37 want to qualify for the final refugee Olympic team."
The IOC unveiled its first team of refugees in Rio in an effort to raise awareness of the issue and it was one of the feel-good stories of those Games.
The 10-member team from Syria, Congo, Ethiopia and South Sudan were in the spotlight after marching as the penultimate team before host nation Brazil in the Opening Ceremony at the Olympic stadium. They competed in athletics, swimming and judo.
The 37 athletes working towards a spot in Tokyo will not necessarily need to meet international qualification criteria to get a place in the team but will need to come close.
"Qualification does not mean the overall international standards," Bach said. "We want to identify those athletes who either fulfil these (international) criteria or at least come as close as possible to these criteria."
The 37 are refugees from Afghanistan, Cameroon, Democratic Republic of Congo, Eritrea, Ethiopia, Iran, South Sudan, Sudan and Syria.
They are training in athletics, badminton, boxing, judo, karate, swimming, taekwondo and weightlifting in 11 host countries, but would come together in April next year for a joint training camp, Bach said.
More than a million refugees entered Europe in 2016 alone as they fled fighting in the Middle East and elsewhere, prompting the IOC to create the first Olympic refugee team.
Tens of millions more are housed in camps in countries across the world, having escaped wars or armed conflicts in their home nations.
