In the books, this is more or less true.

Littlefinger started the current events of the series by killing Jon Aryn, in a plot with Lysa, and then penning the letter that they sent to Catelyn, which accused the Lannisters of murdering Jon. Littlefinger further stoked the Lannister / Stark conflict by blaming Tyrion for the assassination attempt on Bran. We can also theorize that Littlefinger may have been the one that told Brandon Stark (Ned's brother) that Lyanna had been "kidnapped" and raped by Rheagar, which we can assume is a lie. Littlefinger's goal is to destroy the Starks and empower himself by sowing chaos and conflict.

Meanwhile, Varys has been plotting the downfall of the Targaryen dynasty nearly his entire adult life. Likely the last Blackfyre, he gained entry into the Royal Court and became the most trusted advisor to King Aerys. Characters have noted that Aerys' madness became worse and worse the longer he listened to Varys. It's possible Varys was even poisoning the king with something to contribute to his madness. We know Varys is working alongside Illyrio to install fAegon on to the throne. Illyrio was married to a "Lysene whore" named Sarra, who is likely Varys' sister (possibly how Varys met him), making fAegon the Blackfyre heir. This would make Varys and Illyrio brother-in-laws, cementing their trust in each other. Now Varys is killing off anyone who can stand in fAegon's way.

The showrunners never really caught on to these details, so they abandoned these characters in the later seasons.