Focussing on the first 100 days early in the pandemic from March to June 2020, this study provides an evolving portrait of the health, economic and social consequences of the COVID-19 pandemic on vulnerable populations in India. We analysed people's experiences recorded and shared via mobile phone on the voice platforms operated by the Gram Vaani COVID-19 response network, a service for rural and low-income populations being deployed to support India's COVID-19 response. Quantitative and visual methods were used to summarize key features of the data and explore relationships between factors. In its first 100 days, the platform logged over 1.15 million phone calls, of which 793 350 (69%) were outbound calls related largely to health promotion in the context of COVID-19. Analysis of 6636 audio recordings by network users revealed struggles to secure the basic necessities of survival, including food (48%), cash (17%), transportation (10%) and employment or livelihoods (8%). We found that pre-existing development deficits and weak social safety nets are driving vulnerability during the COVID-19 crisis. This work was important as a contribution to relief efforts and an inclusive pandemic response based on inclusive policy design and institutional reforms.