"We are already struggling to communicate our concerns about secularism, democracy and minority rights to a diaspora in which Modi enjoys considerable support," IOC vice chairman George Abraham said