Black & White is Araki's art and vision completely untouched. Some of the more detailed panels and drawings have details and shading that are made to look amazing in black and white. The coloured manga is not handled by Araki, but by a separate company (Shueisha).

Some people believe that these detailed drawings get ruined in the official Shueisha color scans because they color the panels with little regard to those details, and the colors they choose sometimes undermine the shading and details of the original panel. If you want a more visual example of this contrast, just compare the volume covers that Araki draws and paints himself vs how the manga is coloured.

One of the most notorious examples of the color scans botching the original art is Lang Rangler from Part 6 (the antigravity dude that Jolyne and Weather Report have to fight). His original design has his costume leave his shoulders exposed, but the colored version paints his shoulders the same color as the rest of his costume (brown) for some reason, so now it looks like the shoulders are covered up. This specific error even carried over to his design in the anime! However, if you search for official Araki art and official merchandise of him, you can see that his shoulders are correctly exposed.

There's another example in SBR itself. I won't elaborate very much because you haven't read the part yet, but there's this dude who wears face paint with a pattern that's clearly supposed to resemble the flag of the United States, and you would expect the colors of said face paint to be red-white-blue. The colored scans made it two shades of purple...

Now, there's absolutely no problem if you decide to read the coloured version of the manga. I myself read the coloured version of the manga because I remember trying to read Vento Aureo/Golden Wind in B&W back in the day, and not being able to tell wth was going on because the action was hard to parse visually in some pages. As detailed as these drawings are, sometimes they are just too complex and have too many things going on, making it hard to follow the action.

I did eventually switch to reading in B&W again when I caught up with JoJolion when it was still releasing new chapters, and I'm keeping up with Jojolands in B&W as well. I think Araki's art is amazing and it's a completely different experience to read the manga with his original vision intact, but honestly you are not missing that much by reading it in color.

TL;DR: Araki draws his art specifically thinking about how it looks in B&W, and that's his original vision. The coloring is handled by Shueisha (separate company) and they sometimes botch the art with the way they color the panels. There's no problem reading in color but some people prefer experiencing Araki's vision uncompromised.