2
\$\begingroup\$

Motivation

I wanted to write a simple console application to improve my vocabulary by using the Wordnik API to get words and definitions and using the SuperMemo2 spaced-repetition algorithm to schedule review sessions.

Features

  • Correctly implement the SM2 (SuperMemo2) spaced-repetition algorithm to schedule practice

  • Automatically compute the "easiness factor" for the SM2 algorithm based on the amount of time taken to answer each question

  • Automatically add new words using the Wordnik API

Implementation

Here is a small Python module that I wrote which prompts the user with a definition and asks for them to supply the correct word.

NOTE: This requires a (free) Wordnik API key in order to run.

#!/usr/bin/env python3
# -*- encoding: utf8 -*-
"""learnwords.py

Learn words from the Wordnik API using
the SuperMemo2 spaced-repetition algorithm.
"""

from datetime import datetime
from datetime import date
from datetime import timedelta
import json
import os
import requests
import sys
import time

# Define module-level variable for Wordnik API base URL
WORDNIK_API_BASEURL = "https://api.wordnik.com/v4"

def get_updated_sm2(
    repetition_iteration,
    easiness_factor,
    repetition_interval,
    response_quality,
):
    """SuperMemo2 update function"""

    # Rename the parameters for convenience
    n = repetition_iteration
    EF = easiness_factor
    q = response_quality

    # Update the iteration number
    # If the response quality was too low, restart the
    # repetitions without changing the easiness factor
    if q<3:
        new_repetition_iteration = 0
    else:
        new_repetition_iteration = n + 1

    # Compute the new easiness factor
    new_easiness_factor = max(
        EF + (0.1-(5-q)*(0.08+(5-q)*0.02)),
        1.3
    )

    # Compute the new inter-repetition interval
    if new_repetition_iteration == 1:
        new_repetition_interval = 1
    elif new_repetition_iteration == 2:
        new_repetition_interval = 6
    else:
        new_repetition_interval = \
            int(new_repetition_iteration * new_easiness_factor)

    # Return the updated SM2 values
    return (
        new_repetition_iteration,
        new_easiness_factor,
        new_repetition_interval,
    )

def get_random_word(api_key):
    """Returns a random word as a string."""

    # Construct the API URL for a random-word query
    api_url = "{baseurl}/words.json/randomWord".format(
        baseurl=WORDNIK_API_BASEURL
    )

    # Set the parameters for the query to
    # limit the list of results to a single word
    parameters = {
        'hasDictionaryDef': 'true',
        'includePartOfSpeech': 'noun',
        'minCorpusCount': 0,
        'maxCorpusCount': -1,
        'minDictionaryCount': 1,
        'maxDictionaryCount': -1,
        'minLength': 5,
        'maxLength': 15,
        'api_key': api_key,
    }

    # Perform the query and store the HTTP response object
    response = requests.get(api_url, params=parameters)

    # Convert the response content to a list
    # NOTE: The content is initially returned as a byte string
    word_object = json.loads(response.content)

    # Get the word itself
    word = word_object.get("canonicalForm", word_object["word"])

    # Return the word as a string
    return word

def get_definition(api_key, word):
    """Returns the definition of a word."""

    # Construct the API URL for a word-definition query
    api_url = "{baseurl}/word.json/{word}/definitions".format(
        baseurl=WORDNIK_API_BASEURL,
        word=word,
    )

    # Set the parameters for the query to limit the
    # list of results to a single definition
    parameters = {
        'limit': 1,
        'partOfSpeech': 'noun',
        'includeRelated': False,
        'sourceDictionaries': 'webster',
        'useCanonical': True,
        'includeTags': False,
        'api_key': api_key,
    }

    # Perform the query and store the HTTP response object
    response = requests.get(api_url, params=parameters)

    # Convert the response content to a list
    # NOTE: The content is initially returned as a byte string
    definition_list = json.loads(response.content)

    # Return the first definition as a string, else return None
    if definition_list:
        return definition_list[0]["text"]

def teach_new_word(word_dict):

    # Prompt the user with the new word
    # Repeat until they enter it correctly
    answer = None
    while answer != word_dict["word"]:

        # Clear the screen
        os.system('cls' if os.name == 'nt' else 'clear')

        # Prompt the user and get their response
        answer = input(
            "New word: {}\n\nDefinition:\n\t{}\n\nRepeat the word: ".format(
                word_dict["word"],
                word_dict["definition"],
            )
        ).strip()

        # Tell the user whether or not they entered the answer correctly
        if answer == word_dict["word"]:
            print("\nCorrect!")
        else:
            print("\nIncorrect!")

        # Pause for a second to give the user time to read
        time.sleep(1)

def review_word(word_dict, deck, update):

    # Clear the screen
    os.system('cls' if os.name == 'nt' else 'clear')

    # Prompt with the definition and time the response
    start = time.time()
    answer = input(
        "\nDefinition: {}\n\nWord: ".format(word_dict["definition"])
    ).strip()
    stop = time.time()
    duration = stop - start

    # Check to see if the answer is correct,
    # compute a reponse quality,
    # and display the correct answer
    if answer == word_dict["word"]:

        # Compute the response quality from the duration
        # NOTE: This was totally arbitrary on my part.
        if duration < 5:
            response_quality = 5
        elif duration < 10:
            response_quality = 4
        elif duration < 15:
            response_quality = 3
        elif duration < 20:
            response_quality = 2
        elif duration < 25:
            response_quality = 1
        else:
            response_quality = 0

        # Display feedback for the user
        print("\nCorrect!\n\nTime taken: {:6.2f} seconds\n\nResponse quality: {}".format(
            duration, response_quality)
        )

    else:

        # If the question was answered incorrectly
        # then the response quality is zero
        response_quality = 0

        # Display feedback for the user
        print(
            (
                "\nIncorrect!"
                "\n\nAnswer: {}"
                "\n\nTime taken: {:6.2f} seconds"
                "\n\nResponse quality: {}"
            ).format(
                    word_dict["word"], duration, response_quality
            )
        )

    # We update after the first attempt, but not on subsequent attempts
    if update:

        # Compute the new SuperMemo2 parameter values
        repetition_iteration, easiness_factor, repetition_interval = \
            get_updated_sm2(
                repetition_iteration=word_dict["repetition_iteration"],
                easiness_factor=word_dict["easiness_factor"],
                repetition_interval=word_dict["repetition_interval"],
                response_quality=response_quality,
            )

        # Set the date for the next practice for this word
        next_practice = \
            date.today() + \
            timedelta(days=repetition_interval)

        # Update the SuperMemo2 parameter values in the deck
        deck[word_dict["word"]].update({
            "repetition_iteration": repetition_iteration,
            "easiness_factor": easiness_factor,
            "repetition_interval": repetition_interval,
            "next_practice": next_practice
        })

    # Pause for a second to give the user time to read
    time.sleep(1)

    # Return the response quality (determines whether or not to review again)
    return response_quality

def learn_words(api_key, deck):

    # Store the current time (to determine which words to review)
    current_date = date.today()

    # Get all the words that need to be reviewed
    words_to_review = [
        v for k, v in deck.items()
        if datetime.strptime(v["next_practice"], "%Y-%m-%d").date()\
            <= current_date
    ]

    # If no words are due, then get a new word
    if not words_to_review:

        # Get a random word and make sure it has a definition
        # NOTE: Why isn't this always the case? I'm setting the
        #       "hasDictionaryDef" parameter to 'true'.
        definition = None
        while not definition:

            # Get a random word
            word = get_random_word(api_key)

            # Get the definition of the word
            definition = get_definition(api_key, word)

        # Construct the dictionary representation for the word
        word_dict = {
            "word": word,
            "definition": definition,
            "easiness_factor": 0,
            "repetition_interval": 0,
            "repetition_iteration": 0,
            "last_practiced": None,
            "next_practice": None,
        }

        # Teach the user the new word
        teach_new_word(word_dict)

        # Update the practice times for this word
        word_dict.update({
            "last_practiced": date.today(),
            "next_practice": date.today(),
        })

        # Add the word object to the deck
        deck[word] = word_dict

        # Add the word to the list of words to review
        words_to_review.append(word_dict)

    # Enter the main review loop
    while words_to_review:

        # Get the next word
        word_dict = words_to_review.pop()

        # Review the next word
        response_quality = review_word(
            word_dict=word_dict,
            deck=deck,
            update=True,
        )

        # If the response quality is too low, review again but don't update
        while response_quality < 4:
            response_quality = review_word(
                word_dict=word_dict,
                deck=deck,
                update=False,
            )

def run():

    # Get the Wordnik API key from a config file
    with open(os.path.expanduser('~/.learnwords/wordnik.conf'), 'r') as config_file:

        # Load the configuration parameters from a JSON file
        config_json = json.load(config_file)

        # Get the Wordnik API key
        api_key = config_json["api_key"]

    # Load the deck from a file if it exists or initialize it otherwise
    try:

        # Load the deck (local library of stored words)
        with open(os.path.expanduser('~/.learnwords/deck.json'), 'r') as deck_file:

            # Convert the JSON data to a Python dictionary
            deck = json.load(deck_file)

    except FileNotFoundError as e:

        # Create an empty deck file
        with open(os.path.expanduser('~/.learnwords/deck.json'), 'w') as deck_file:
            deck_file.write(r'{}')
            deck = {}

    # Learn the words in the deck and update the deck file (allow the user to interrupt)

    # Learn the words in the deck
    try:
        learn_words(api_key, deck)

    # Allow the user to interrupt
    except KeyboardInterrupt as e:
        print("\n\nExiting program...")
        sys.exit(0)

    # Update the deck file
    finally:

        # Update the local deck file
        with open(os.path.expanduser('~/.learnwords/deck.json'), 'w') as deck_file:
            json.dump(deck, deck_file, indent=4, sort_keys=True, default=str)

    # Print an exit message
    print("\nAll done for today!")

# Run the application
if __name__ == "__main__":
    run()

Usage

The Wordnik API key is stored in a JSON-format configuration file (~/.learnwords/wordnik.conf), e.g.

{
    "__comment__": "Wordnik API Key",
    "api_key": "idah8AeV1Shai7oosah9ayefahy1yiequa2yi0uyo2eeshaej"
}

After running the program for the first time (e.g. python learnwords.py, a local file (~/.learnwords/deck.json) should be created and a new random word should be added from the Wordnik online dictionary. The user is shown the definition of the new word and prompted to enter the word:

New word: vidding

Definition:
    Command; order; a proclamation or notifying.

Repeat the word: 

After entering the word correctly, the user is again prompted with the definition and asked for the word:

Definition:
    Command; order; a proclamation or notifying.

Word:

After answering incorrectly (or too slowly), the user is given feedback and asked the question again:

Definition: Command; order; a proclamation or notifying.

Word: not vidding

Incorrect!

Answer: vidding

Time taken:  11.37 seconds

Response quality: 0

After answering correctly, the user is asked another word or the program exits if there are no more words to review:

Definition:
    Command; order; a proclamation or notifying.

Word: vidding

Correct!

Time taken:   7.11 seconds

Response quality: 4

All done for today!
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.