4
\$\begingroup\$

I have a dict of categories of Python keywords, with lists of keywords as the values:

Keywords_33=[('File_2', ['with', 'as']),
             ('Module_2', ['from', 'import']),
             ('Constant_3', {'bool': ['False', 'True'],
                             'none': ['None']}),
             ('Operator_4', {'boolean_operation': {'or', 'and', 'not'},
                             'comparison': {'is'}}),
             ('Container_operation_2', ['in', 'del']),
             ('Klass_1', ['class']),
             ('Function_7',['lambda', 'def', 'pass',
                            'global', 'nonlocal',
                            'return', 'yield']),
             ('Repetition_4', ['while', 'for', 'continue', 'break']),
             ('Condition_3', ['if', 'elif', 'else']),
             ('Debug_2', ['assert', 'raise']),
             ('Exception_3', ['try', 'except', 'finally'])]

I want to ensure that every keyword is in a category. This code does that:

from keyword import kwlist
s = str(Keywords_33)
intersection_with_kwlist = [keyword for keyword in kwlist if keyword not in s]
print(intersection_with_kwlist)

Expected (and actual) output:

[]

This indicates that no keywords are missing from Keywords_33.

Can I make this code more elegant?

\$\endgroup\$
  • 2
    \$\begingroup\$ An identical question has been posted at a more appropriate site (stackoverflow) stackoverflow.com/questions/47788806/… - this site is not really for these sorts of questions. \$\endgroup\$ – Luke Dec 13 '17 at 8:54
  • 1
    \$\begingroup\$ It's better, but I'm still not sure what you're doing here and why you're using the names you're using. This question is going to need more context, but could turn into a mighty fine question. Please take a look at "How to get the best value out of Code Review - Asking Questions" \$\endgroup\$ – Mast Dec 13 '17 at 9:35
  • 1
    \$\begingroup\$ I think you also have a bug. If raise is in the Keywords_33, your check says that is is also in it, even if it might not be, because the string contains is. \$\endgroup\$ – Graipher Dec 13 '17 at 11:51
7
\$\begingroup\$

I'd personally use a generic function to flatten your datatype.

  1. If an item in your collection in collections.Iterable then you will want to recurse through it.
  2. If it's not you will want to yield it.
  3. If it's a str or bytes you'll want to yield it, however this can change. And so you'll want to pass a tuple of these wanted types.
  4. You probably want to only go through dictionary values as opposed to keys. And so you should pass a function to handle dictionaries.

And so I'd make:

def flatten(obj, dict_handle=dict.keys, wanted=(str, bytes)):
    def inner(obj):
        if isinstance(obj, dict):
            obj = dict_handle(obj)
        for el in obj:
            if isinstance(el, collections.Iterable) and not isinstance(el, wanted):
                yield from inner(el)
            else:
                yield el
    return inner(obj)

From this your code becomes:

from keyword import kwlist
import collections

unknown = set(kwlist) - set(flatten(Keywords_33, dict_handle=dict.values))
print(unknown)

If you don't want to remove 'File_2' from kwlist, then you can always change Keywords_33 to a dictionary:

unknown = set(kwlist) - set(flatten(dict(Keywords_33), dict_handle=dict.values))
print(unknown)
\$\endgroup\$
  • \$\begingroup\$ By "generic function", do you mean "generator function"? \$\endgroup\$ – 200_success Dec 13 '17 at 22:28
  • \$\begingroup\$ @200_success No I mean generic, as opposed to bespoke - such as Graipher's code. \$\endgroup\$ – Peilonrayz Dec 14 '17 at 9:26
5
\$\begingroup\$

I would make this a function that recursively steps through a dictionary:

def get_keywords(d):
    keywords = set()
    for x in d.values():
        if isinstance(x, str):
            # Not needed with current example, but might be helpful
            keywords.add(x)
        elif isinstance(x, (list, tuple, set)):
            # Can simply add all elements
            keywords.update(x)
        elif isinstance(x, dict):
            # Have to recurse down to get all nested keywords
            keywords.update(get_keywords(x))
        else:
            # Unrecognized type
            raise ValueError(
                "{} is of type {}, which is currently unhandled.".format(x, type(x)))
    return keywords

For this to work, you need to start with a dictionary, but that is quite easy with your given data structure, just pass dict(Keywords_33).

Afterwards you can just do the set difference like this:

from keywords import kwlist

...

if __name__ == "__main__":
    Keywords_33 = ...
    keywords_without_category = set(kwlist) - get_keywords(dict(Keywords_33))

This also avoids the bug I mentioned above, namely that "is" is in "raise", "if" in "elif", "or" in "for", "as" in "class" and "assert", any of which might lead to false negatives.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.