The 48-year-old Sarafraz Bahman Golbarnezhad was killed during a mountainous stretch of the C4-5 event course, according to the International Paralympic Committee (IPC).
"The athlete received treatment at the scene and was in the process of being taken to the athlete hospital when he suffered a cardiac arrest," the IPC said.
"The ambulance diverted to the nearby Unimed Rio Hospital in Barra da Tijuca where he passed away soon after arrival."
Authorities said they had launched an investigation into the accident. According to the IPC, it is the first time an athlete has died during competition in the 56-year history of the Paralympic Games, Xinhua news agency reported.
Golbarnezhad was competing in his second race in Rio after finishing 14th in the C4 time trial on Wednesday.
According to his friend, Hashem Rastegarimobin, who competed for Iran in football 7-a-side in Rio, Golbarnezhad took up cycling in 2002 after losing his leg due to a war injury in the 1980s.
"He was somebody who always had a smile on his face. He was a happy guy," Rastegarimobin said.
Iran's Paralympic committee said: "He was an exemplary Paralympic sportsman who, with love and energy, tried his best to promote the name of Iran and to make all of us proud and at the end he gave his life for this.
The Iranian flag was lowered to half-mast at the athletes' village and a minute's silence will be observed during Sunday's closing ceremony.