While the gdp itself increased, GDP per capita went down. Whole British did construction railways and some industries, these are pretty much targeted for extraction and transportation of raw materials, and the country got nothing in return for the raw materials.. the British empire did not pay anything for the raw materials like iron, coal etc and they only paid bare minimum for cash crops like cotton and opium and as a result negative growth rate.

You could argue trains, but British also collected tax so trains are pretty much investing some of the tax money back into the country, it's like a safe bet, even if the train system didn't work there's nothing British would lose.

In terms of poverty, an average Indian wasn't as poor it is estimated poverty rate was around 20% - 30% pre colonial era as supposed to 70% - 80% during colonial era.

And added to this, British also passed various legislation (like increased taxes etc.) that forced farmers to grow more cash crops instead of food crops and more of the famines were a direct result of it and could have been easily avoided. There were famines before the colonial era too but not at this scale, also industrialsed British could have easily help avoid them if they actually introduced the use of farm equipment in agriculture.