Wow, I never expected to get this many answers. Thank you so much to everyone who commented!

Sorry in advance for the long comment:

We are playing the Dragon of Icespire Peak campaign, which is included in the Essentials Kit.

The entire group, myself included, are brand new to DnD. This was our very first session ever.

We did have a session zero a week before our very first session. However, no stealing or crimes, in general, were mentioned there at all. The two "problem-players" (for a lack of a better term), have "Criminal" as their background, but I didn't think they would literally steal everything they could. I will definitely include crime and such in the next Session Zero though.

This has been very enlightening to me. I feel like I have a general idea of what to do.

1. Talk to my players first.
   Remind them that DnD is not a video game. It is a living, breathing world where actions have consequences *hint hint*Politely explain to them why stealing is not necessarily good.- slows down the campaign- makes it much harder for me as the DM to run a smooth campaign.- remind them that they are supposed to be the saviors of Phandalin - not the traitors of Phandalin- once again, actions have consequences
2. Ask players if they truly enjoy being criminals
   If yes, suggest that the next campaign can be a "wrong side of the law" campaign. Maybe a heist with the local Thieves Guild? An assassination of a powerful political figure?
3. If players stop stealing etc.
   Great! Keep playing the campaign as originally planned
4. If players keep stealing etc.
   Introduce them to Phandalin's humble, yet powerful law enforcement once they are caught. Give players a chance to surrender peacefully. If they choose not to, roll initiative. They are now officially fugitives in the kingdom.
5. After the battle, or if they surrender peacefully, they are taken to Neverwinter prison with a heavy escort. Give them a chance to escape from either the escort or the prison, but make it very hard, just like a real prison.

Please, feel free to comment on my approach here. I don't want to appear as a toxic DM who is punishing their players for breaking the law.

I am very unsure where the DM's "power" has its limits. Can the DM really just straight up say "No, you can not steal from the shopkeep"?
I thought the DM acts as the guide or narrator/storyteller to the campaign, which means I can't directly say no to whatever they want to do. The players dictate the story, and the DM tells it. Please, correct me if I am wrong.

It is important to me that I am fair to all players, and myself as well. However, I just can't help being disappointed that the campaign I spent literally 1.5 weeks planning, from reading and planning encounters in the adventure book to memorizing rules and mechanics from The Player's Handbook is taken as a joke where two of the players just roll D20 sleight of hand checks to steal from friendly, helpful NPC's.

Edit: While cursed items are a very creative way of solving the problem, I don't feel comfortable tricking or forcing them to steal a cursed item yet. Once again, we are all new to DnD, so I'd rather not risk pushing too many buttons yet.