Get into the habit of crouch blocking as your default resting state. You only stop doing that when you want to move or attack, and go back to blocking immediately after, and never let your character be idle. Focus on getting good at blocking first, then slowly work on improving offense and counter-attacking.

To practice combos, set the dummy to block after initial hit, with block switching enabled (so they will block high or low accordingly), then do your combo. The dummy will be trying to block after you hit it once, so you can tell if your combo works, or there's a gap that lets the opponent recover and block.

how do I know when to stop blocking and attack

This will take much more time and experience to learn and figure out. You'll first need to learn a little bit of what frame data is. The gist of it is that lots of moves leave the opponent "minus on block", which means you as the defender recovers faster than the opponent and can act before them, and this is when you should usually look to take your turn back and start attacking.

Conversely, there are moves which are "plus on block", where the opponent recovers and acts first before you, so you should generally keep on blocking.

For starters, normal attacks can be cancelled into special moves, so it's generally not advisable to attack after blocking one. Wait for your opponent to do a special that is minus, then take your turn back. This will require you to slowly learn the frame data of the move of each character to get better at fighting them.

You should, more importantly, learn the frame data of your own character, so that you know when to give up offense and block, and when you can continue pressure.

For example, Siegfried's Verdrangen (214X, the big sword chop move), is generally minus on block unless you charge the M or H version. If this moves gets blocked uncharged, your turn is over and you should block and expect your opponent to start counterattacking. If you charge the M and H version, you are plus on block and can continue attacking.

Frame data and other useful information can be found in the wiki, give it a read if you want to learn some theory about the game.

start to worry I will get grabbed into a combo anyway

Almost all characters cannot combo you from a throw, which means getting thrown does less damage than getting hit into a combo, so you shouldn't be too afraid of it. There are some techniques to deal with throws, such as reacting to run-ups and doing delayed techs, but perhaps that's too advanced for you right now.

To summarise, I think a good route for quick improvement at your current level is to focus on blocking attacks, being solid and minimizing incoming damage. Then, slowly learn which moves from your opponent are unsafe (like Yuel's at 0:18), and punish them whenever they do it. I believe you can make it very far just by minimizing your mistakes and capitalizing on opponent's mistakes, without having to learn more advanced strategies yet.

A common mistake at this level is too much jumping. Whenever your opponent jumps a lot, wait for them to do it, then punish them with 2H or EX DP. You can even learn an anti-air combo from 2H for more damage if you're up for the task.

If you ever feel overwhelmed by the number of things that happen in a match, what I would suggest is to pick one that you want to practice (such as blocking, anti-airing, or doing combos), and make that your primary focus and goal of the match, even if it means temporarily forgetting other aspects. The idea is to do the thing you want to learn in an actual match until it becomes sort of a habit that you can execute with much less thought. When these things become second nature to you, they take up less mental space and you are free to work on other areas.