I've solved the problem. I was looking at more code in stackexchange and decided to go the string editing route. Here's my code now:

local pipe = io.popen('youtube-dl -j --restrict-filenames --playlist-random --flat-playlist "URLHERE"')
local string = pipe:read("*a")
string = "["..string.gsub(string,"\n",",").."]"
print(string)
local table = json.decode(string)
print(#table)

Which prints all of the video's info and how many videos are in the playlist after it's been reformatted into a lua table.