The issue of British-Indian author Rushdie, who cancelled his visit to the festival due to assassination threats, has overshadowed the event. Muslim groups protested against his invitation and other authors accused the government of suppressing free speech.
"There are a large number of people averse to this video link inside this property. They have threatened violence," Ramtap Singh, owner of the hotel in which the festival is held, told the large crowd that had assembled to listen to the author.
"This is necessary to avoid harm to all of you."
A death threat was received against Jaipur Literature Festival director William Dalrymple ahead of Rushdie's speech.
Rushdie, whose 1988 novel "The Satanic Verses" is banned in India, cancelled his planned visit to speak at the festival in person after reported assassination threats against him.
"All of us feel hurt and disgraced. Artists have not been able to prevail," said Sanjoy Roy, the festival's producer, holding back tears.
Political parties are seen as unwilling to lend their support to the author for fear of offending Muslim voters ahead of an important state election in Uttar Pradesh, India's most populous state, next month.
After organizers announced earlier on Tuesday that Rushdie's address would go ahead, leaders of local Muslim groups began to congregate at the main entrance to the festival, vowing to protest if the author was allowed to address the event.
"This is a big defeat. It's a triumph for bigotry," said Tarun Tejpal, editor of Indian news magazine Tehelka.
Five authors have been investigated by police in Jaipur for reading from "The Satanic Verses" at the festival, and English PEN, a writer's body, issued a statement late on Monday in their support.
"We felt that it was important to show support for Salman, who is often misrepresented ... This situation has arisen in India at a time when free speech is under attack," wrote Hari Kunzru, one of the authors involved, in the Guardian newspaper.
The five-day festival, which concludes on Tuesday, has attracted over 70,000 visitors and featured best-selling authors such as Richard Dawkins, Tom Stoppard and Michael Ondaatje, and global television superstar Oprah Winfrey.