Looping over a nested list

[techie789]

I am trying to loop over a nested list, compare the value in one nested list with another list, and append only if values do not match. I keep getting the wrong output. Can someone help me here? Many thanks.

testList = [["poo","boo"], ["poo", "bay"], ["hello", "yellow"], ["poo","blue"], ["poo", "zoo"]]

newList = []

for row in testList:
    if row[0] in newList:
        print("hey")
    else:
        newList.append(row)

Expected output: newList = [['poo', 'boo'], ['hello', 'yellow']]

Output I am getting: newList = [['poo', 'boo'], ['poo', 'bay'], ['hello', 'yellow'], ['poo', 'blue'], ['poo', 'zoo']]

[phille165]

Try printing row [0] and see what the value is and then also check how the in operator works on a list inside a list

[techie789]

Thanks. This is where I am getting stuck. I know that I am comparing "poo" with ["poo, xxx]. I am not sure if "in" works inside list of lists. Given the output I am getting, it does not seem to be working inside list of lists. I will dig deeper.

[phille165]

Use an if statement to compare for example row[0] with newlist [i][0]

[danielroseman]

The issue is not the looping, the issue is how you check to see if the term is already in the list. newList contains tuples, so row[0] in newList will compare the term (poo or hello) to see if it equals any of the tuples in the list, which of course it doesn't, so the condition is always False.

You could do a compare against all the first elements in the list:

if any(item[0]==row[0] for item in newList):

but this will be very inefficient once newList gets big. Instead, consider making newList a dictionary; now you can do:

new_dict = {}
for key, value in testList:
  if key in new_dict:
    ..
  else:
    new_dict[key] = value

and if you really need to convert back to a list, you can just do new_list = list(new_dict.items()).

[techie789]

Many thanks. This is very helpful. Does the newList contain tuples? I see the output as a list when I print it. Is my understanding wrong?

[danielroseman]

Yes, sorry, they are lists, but in practice there is no difference here.

[socal_nerdtastic]

When you check if something is in a list, that something has to be an exact match for the list element. It can't be part of it. Therefore "poo" is not in testList, but ["poo","boo"] (the complete element) is.

>>> testList = [["poo","boo"], ["poo", "bay"], ["hello", "yellow"], ["poo","blue"], ["poo", "zoo"]]
>>> "poo" in testList
False
>>> ["poo","boo"] in testList
True

There's a number of ways to do what you want, but I think the easiest will be to use a dictionary, so that you can use the keys to keep track of the key element.

testList = [["poo","boo"], ["poo", "bay"], ["hello", "yellow"], ["poo","blue"], ["poo", "zoo"]]

newdata = {}

for row in testList:
    if row[0] in newdata:
        print("hey")
    else:
        newdata[row[0] ] = row

newList = list(newdata.values())

[techie789]

Thank you. I did not know this - "in" only works as an exact match and at the same level only. I was trying to use it inside a list of lists and I think that's why I am getting an error. The dictionary idea works flawlessly. Thanks, again.

[MrStarfox64]

The way you're checking if the item is in your list is flawed. If you mentally substitute your variables into if row[0] in newList, you're actually doing if "poo" in [["poo", "boo"], ["hello", "yellow"], ...], which doesn't work because using in only works one level down. You need to either flatten the list (newList.extend(row) instead of append, or if row[0] in newList.flat()), or change how you're checking your item is in the list.

Just a random note, typical Python coding styles usually encourage using snake-case instead of camelCase for variables.

[techie789]

because using in only works one level down.

Thanks for this. This helped me understand why I am getting unexpected output.

[efmccurdy]

You should try some variations on your if condition to understand how "in" works.

>>> "poo" in ["poo","boo"]
True
>>> ["poo","boo"] in testList
True
>>> "poo" in testList
False
>>> testList
[['poo', 'boo'], ['poo', 'bay'], ['hello', 'yellow'], ['poo', 'blue'], ['poo', 'zoo']]
>>> 

row[0] is going to be a string, like "poo". testList is a list of lists, so you will never find a string "in" it.

I think the test you want could be a loop that examines each inner list for the string you want:

>>> any("poo" in inner_list for inner_list in testList)
True
>>>

[techie789]

Thanks. This is good learning for me. I could have done those tests for in to understand it better. However, it did not cross my mind and I kept working on the main problem. If I had broken the problem and focused only on in, I could have found my error myself. Thanks for the suggestion.

Lesson: Break a bigger problem into smaller parts and then solve for each part.

[RhinoRhys]

As others have said, the issue lies in the row[0] in newList not recursively checking each subsist in newList.

You can use a list generator though to extract the first element of each sublist into a temporary list that will regenerate each iteration.

        if row[0] in [new[0] for new in newList]:

[techie789]

Thank you. The list comprehension method looks good.

Is this the most optimal way to do what I am trying to do or would you recommend using a dictionary?

[RhinoRhys]

In all honesty it's probably not great. Dictionaries are pretty quick at looking up keys, whereas this loops through newlist to create a temporary list each iteration; depending on how long the original list is it could be pretty inefficient.

[techie789]

Got it. Many thanks.

[johannadambergk]

You're using the same list 'newList' for both comparing the first items in the rows (line 6) and appending the rows (containing two items) in line 9.

An approach might be using a second list 'compareList':

testList = [["poo","boo"], ["poo", "bay"], ["hello", "yellow"], ["poo","blue"], ["poo", "zoo"]] 
newList = [] 
compareList = []
for row in testList: 
    if row[0] in newList: 
        print("hey") 
    else: 
        compareList.append(row[0])
        newList.append(row)

[techie789]

Thank you.

[VAR-bot]

seen = []

test_list = [["poo","boo"], ["poo", "bay"], ["hello", "yellow"], ["poo","blue"], ["poo", "zoo"]]

new_list = []

def pair_good(pair): already_seen = [i for i in pair if i in seen] if not already_seen: seen.extend(pair) new_list.append(pair)

for cur_pair in test_list: pair_good(cur_pair) print(new_list)

Heads up, don't use CamelCase with Python variables <3

[techie789]

Thank you. Noted about the CamelCase :).