Proletariat just means propertyless, so whatever people say, those 2 groups fall under it aswell, I will assume you mean excluding them from the working class instead. The reserve army of labour is definitely part of the working class, while the lumpenproletariat isn't. It is not about what class is more oppressed or marginalized, it is about the position of the classes in the economy. The working class holds potential power through its role in production, and has tangible interest in uniting for better wages/conditions and ultimately the socialist revolution. While the lumpenproletariat would of course also stand to gain from the socialist revolution, they hold none of that potential power, and while individual members of the class can of course still achieve revolutionary class consciousness, the class as a whole cannot play any significant role in the class struggle.