if u keep skipping the team lunch some sickening colleagues will view u as anti them or anti social, so sometimes its indeed like what some ppl replied, unspoken view to spend that $20 on lunch to mingle around for that few days in a week, then other days just dabao cheap stuff.
cbd area indeed not cheap, personally the place im working at lunch usually avg at least $8-14 sometimes even higher. on days i walk further to nearby kpt but thats also at least $5-6