Delete Tuples In More Dimensional List If Same

I have a list of tuples namely: [[[('p', 'u'), ('r', 'w')], [('t', 'q')]], [[('p', 'u'), ('r', 'w')], [('v', 'q')]], [[('p', 'u'), ('r', 'w')], [('t', 's')]], [[('p', 'u'), ('r', '

Solution 1:

Lists aren't hashable, tuples are hashable. You then need to take a set of these tuples. But inside these tuples, you want to disregard order. But a tuples of sets are not hashable, so instead you need to use tuples of frozenset objects:

uniques = {tuple(map(frozenset, i)) for i in doublegammas1}

print(uniques)

{(frozenset({('p', 'w'), ('r', 'u')}), frozenset({('t', 'q')})),
 (frozenset({('p', 'w'), ('r', 'u')}), frozenset({('v', 'q')})),
 (frozenset({('p', 'w'), ('r', 'u')}), frozenset({('v', 's')})),
 (frozenset({('p', 'u'), ('r', 'w')}), frozenset({('t', 's')})),
 (frozenset({('p', 'u'), ('r', 'w')}), frozenset({('t', 'q')})),
 (frozenset({('p', 'u'), ('r', 'w')}), frozenset({('v', 'q')})),
 (frozenset({('p', 'u'), ('r', 'w')}), frozenset({('v', 's')})),
 (frozenset({('p', 'w'), ('r', 'u')}), frozenset({('t', 's')}))}

You can then apply this via the itertools unique_everseen recipe, also available in 3rd party libraries as toolz.unique or more_itertools.unique_everseen:

from more_itertools import unique_everseen

def uniquekey(x):
    return tuple(map(frozenset, x))

res = list(unique_everseen(doublegammas1, key=uniquekey))

print(res)

[[[('p', 'u'), ('r', 'w')], [('t', 'q')]],
 [[('p', 'u'), ('r', 'w')], [('v', 'q')]],
 [[('p', 'u'), ('r', 'w')], [('t', 's')]],
 [[('p', 'u'), ('r', 'w')], [('v', 's')]],
 [[('p', 'w'), ('r', 'u')], [('t', 'q')]],
 [[('p', 'w'), ('r', 'u')], [('v', 'q')]],
 [[('p', 'w'), ('r', 'u')], [('t', 's')]],
 [[('p', 'w'), ('r', 'u')], [('v', 's')]]]

Input data

# input data
doublegammas1 = [[[('p', 'u'), ('r', 'w')], [('t', 'q')]],
                 [[('p', 'u'), ('r', 'w')], [('v', 'q')]],
                 [[('p', 'u'), ('r', 'w')], [('t', 's')]],
                 [[('p', 'u'), ('r', 'w')], [('v', 's')]],
                 [[('p', 'w'), ('r', 'u')], [('t', 'q')]],
                 [[('p', 'w'), ('r', 'u')], [('v', 'q')]],
                 [[('p', 'w'), ('r', 'u')], [('t', 's')]],
                 [[('p', 'w'), ('r', 'u')], [('v', 's')]],
                 [[('r', 'u'), ('p', 'w')], [('t', 'q')]],
                 [[('r', 'u'), ('p', 'w')], [('v', 'q')]],
                 [[('r', 'u'), ('p', 'w')], [('t', 's')]],
                 [[('r', 'u'), ('p', 'w')], [('v', 's')]],
                 [[('r', 'w'), ('p', 'u')], [('t', 'q')]],
                 [[('r', 'w'), ('p', 'u')], [('v', 'q')]],
                 [[('r', 'w'), ('p', 'u')], [('t', 's')]],
                 [[('r', 'w'), ('p', 'u')], [('v', 's')]]]

Solution 2:

A mutable object (such as a list or a set) cannot be a member of a set. You can use a frozenset, which is immutable.

main_list = [[[('p', 'u'), ('r', 'w')], [('t', 'q')]],
             [[('p', 'u'), ('r', 'w')], [('v', 'q')]],
             [[('p', 'u'), ('r', 'w')], [('t', 's')]],
             [[('p', 'u'), ('r', 'w')], [('v', 's')]],
             [[('p', 'w'), ('r', 'u')], [('t', 'q')]],
             [[('p', 'w'), ('r', 'u')], [('v', 'q')]],
             [[('p', 'w'), ('r', 'u')], [('t', 's')]],
             [[('p', 'w'), ('r', 'u')], [('v', 's')]],
             [[('r', 'u'), ('p', 'w')], [('t', 'q')]],
             [[('r', 'u'), ('p', 'w')], [('v', 'q')]],
             [[('r', 'u'), ('p', 'w')], [('t', 's')]],
             [[('r', 'u'), ('p', 'w')], [('v', 's')]],
             [[('r', 'w'), ('p', 'u')], [('t', 'q')]],
             [[('r', 'w'), ('p', 'u')], [('v', 'q')]],
             [[('r', 'w'), ('p', 'u')], [('t', 's')]],
             [[('r', 'w'), ('p', 'u')], [('v', 's')]]]

main_set = set(tuple(frozenset(innermost_list) for innermost_list in sublist) for sublist in main_list)

from pprint import pprint
pprint(main_set)

Output:

{(frozenset({('r', 'u'), ('p', 'w')}), frozenset({('t', 'q')})),
 (frozenset({('p', 'u'), ('r', 'w')}), frozenset({('v', 'q')})),
 (frozenset({('r', 'u'), ('p', 'w')}), frozenset({('v', 'q')})),
 (frozenset({('p', 'u'), ('r', 'w')}), frozenset({('t', 's')})),
 (frozenset({('r', 'u'), ('p', 'w')}), frozenset({('t', 's')})),
 (frozenset({('p', 'u'), ('r', 'w')}), frozenset({('v', 's')})),
 (frozenset({('r', 'u'), ('p', 'w')}), frozenset({('v', 's')})),
 (frozenset({('p', 'u'), ('r', 'w')}), frozenset({('t', 'q')}))}

To convert back to the original structure of nested lists:

new_list = [[list(frozen) for frozen in subtuple] for subtuple in main_set]
pprint(new_list)

Output:

[[[('r', 'u'), ('p', 'w')], [('t', 'q')]],
 [[('p', 'u'), ('r', 'w')], [('v', 'q')]],
 [[('r', 'u'), ('p', 'w')], [('v', 'q')]],
 [[('p', 'u'), ('r', 'w')], [('t', 's')]],
 [[('r', 'u'), ('p', 'w')], [('t', 's')]],
 [[('p', 'u'), ('r', 'w')], [('v', 's')]],
 [[('r', 'u'), ('p', 'w')], [('v', 's')]],
 [[('p', 'u'), ('r', 'w')], [('t', 'q')]]]

UPDATE:

A solution that removes the duplicate items in-place from the input data.

unique = []

for item in main_list[:]:
    frozen_item = frozenset(frozenset(innermost_list) for innermost_list in item)
    if frozen_item not in unique:
        unique.append(frozen_item)
    else:
        main_list.remove(item)