python - Removing the lists from a list which are duplicated for some items

Question

I'm trying to remove the lists from a list which have same first and third items but only keeping the first one. Example list and output:

li=[ [2,4,5], [1,3,5], [1,6,5] ]
output_list = [ [2,4,5], [1,3,5] ]

The code I've written takes a very long time to execute as the original list contains millions of list.

b_li = []
output_list = []
for x in li:
    s = [ x[0], x[2] ]
    if s not in b_li:
        b_li.append(s)
        output_list.append(x)

How can I improve the code? Thanks in advance.

Answer 1

An improved version:

b_li = set()
output_list = []
b_li_add = b_li.add
output_list_append = output_list.append
for x in li:
    s = (x[0], x[2])
    if s not in b_li:
        b_li_add(s)
        output_list_append(x)

The changes are:

• Use a set() for b_li which makes lookups faster.
• Turn s into a tuple as there is no need to store unique first and third elements as lists.
• Reduced function lookups which speeds up the code as well.