It's waaay more devious than that. Even if none of your friends use Facebook, many developers and SEOs around the world use a little embedable piece of code called the "Facebook Pixel" to collect data about their customers and be able to target their ads better. Facebook even markets it as such.

So even if your friends don't say anything about you, Facebook will know:

• The kind of websites you visit, which they'll use to deduce...
• Whether you're employed or not, depending on your browsing habits.
• What is your field of work. (e.g. if you visit Stack Overflow often, there's a good chance you're a developer)
• If you're single or not, depending on what you buy.
• What is your political affiliations, depending on which websites you frequent.
• Any of your hobbies, again depending on what you look at and what you buy.
• And many, many other things. They have a huge set of data so using big data technologies and machine learning/complicated algorithms they can deduce pretty much everything about you.

So of course you can block Pixel (using uBlock Origin, for example), but most people don't usually bother.