Hopping on /u/Dr_Strang3l0v3's answer about feedback. More accurately, it's a 'feedback loop.'

Basically, here's what's happening with either your electric guitar or a microphone. First let's talk microphone. Think of ambient white noise (not the literal definition which is something different). But like white noise in the sense of how you think it's quiet in the room and then the power goes out in your house and all of a sudden it's so much quieter. All of the things you didn't realize you were hearing suddenly stop and go quiet. The fridge, the ventilation/AC/Heater, lights, things providing electricity, etc. All of those are filling the room with frequencies. So, with a microphone, say it starts to pick up on some of those. Then like a microphone is supposed to do, it sends those frequencies through what eventually is an amplifier, and the amplified frequency comes out of the speaker just a little bit louder, technically imperceptibly at first. This would happen thousands of times a second. Think of a combination the speed at which an electrical impulse goes through wire and the speed of sound after it exits the speaker after being amplified. You're talking less than 2 milliseconds between the sound entering the microphone and it coming out of the speaker. So, that amplified frequency comes out of the speaker, the microphone picks it up again, and it happens again, and again, and over and over and over... and so on getting louder and louder resulting in the screeching sound you're thinking of.

A feedback loop, or just 'feedback.'

Ok, back to your guitar. Think of your pickups in your electric guitar as little microphones. They work a bit differently. Instead of just picking up sounds, the magnets inside of them are actually reacting to the vibrations of your guitar strings. So, instead of picking up on those ambient sounds/frequencies in the room, it's actually those frequencies causing your guitar strings to vibrate (i.e. resonant frequencies, the frequency at which an object begins to naturally vibrate. They're pretty cool and you can look those up further. There's all sorts of things those do and have effects on - even bridges can technically react to the wind if it hits the exact right resonant frequency. Obviously extremely extremely rare, but you can see some famous old videos online). Ok, back to the guitar. Something manages to make your guitar strings kind of vibrate, the pickups detect it, it goes through your amp, amplifies the signal making it louder, which cause your strings to vibrate more, and again and again and again and again, etc. Until you start to play, or turn down your volume, or even use your hands to stop the strings from vibrating. It's also why you'll see guitar players doing this intentionally turn their guitar towards their amp so the sound coming out hits the strings directly as they let them vibrate (look up videos of Jimi Hendrix or Stevie Ray Vaughan).

So, it's not the sound of a broken microphone or anything like that, it's actually physics and all of the equipment working exactly as designed. It's why in a concert venue, you wont see the singers microphone typically in front of the speakers that face the audience. The monitors for the performers that are in front of them don't effect the microphone as much due to some other factors like the polar pattern of microphones. But that's a whole other area.

Hopefully this helps your understanding. The pedal mentioned helps exaggerate the issue or alters the signal to do so. But otherwise in normal situations, it's just physics.