This scrapes results from thefreedictionary.com:


from bs4 import BeautifulSoup as bs
import re
from requests import get

def remove_non_ascii(text):
    return re.sub(r'[^\x00-\x7F]+', '', text)

def get_soup(url):
    raw = remove_non_ascii(get(url).content)
    soup = bs(raw)
    return soup.select("#MainTxt")[0].select('.ds-single')[0].text.strip()

def lookup(word):
    base_url = "http://www.thefreedictionary.com/"
    query_url = (base_url + word)
    return get_soup(query_url)

if __name__ == '__main__':
    print lookup('linux')



Warning: this script may fail to comply with thefreedictionary.com's TOS

  1. According to PEP8, import re should come first
  2. In free dictionary there is only one div having the class ds-single. You can simplify your code to search for this div
  3. This doesn't apply to your program, but if you want to scrape multiple pages, it's better to use grequests.
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.