7
\$\begingroup\$

Description

I'm not sure if it Covid-19 but lately it is impossible to book a tennis court in my area on time. It's always full or maybe just don't check enough :)

To beat the queue and get notified earlier, I started making a scraper to ensure I can book a court on time.

But because of iframe hell and the lack (to my knowledge) of an API it was impossible to do it the easy way, hence I used selenium. I coded this today and it works pretty well, although it's already become a bit of a mess.

Todo

Dynamically detecting the hours available and starting time of the day, so I don't have to supply those values anymore, because my guess is they will change (depending on the time it will get dark etc.)

You can still flame me for it, but I just haven't found a good way to do it yet, now I just supply them in the COURTS dictionary

Code

#!/usr/bin/python3

import time

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

# At what minimal time I am willing/able to book
WEEKEND_TIME = 10
WEEK_TIME = 18

COURTS = {
    # Some courts have different hours available and the hour at which you can start can differ
    # Thus it is key -> name with values -> url, hours available, time diff start
    "Bethnal Green Gardens" : ["https://www.towerhamletstennis.org.uk/bethnal-green-gardens-court-bo", 14, 0],
    "St Johns Park"         : ["https://www.towerhamletstennis.org.uk/st-johns-park-court-boo", 14, 1],
    "Poplar Rec"            : ["https://www.towerhamletstennis.org.uk/poplar-rec-court-bookin", 12, 0],
    "King Edward's Park"    : ["https://www.towerhamletstennis.org.uk/kemp-court-bookin", 12, 0],
    "Victoria Park"         : ["https://www.towerhamletstennis.org.uk/victoria-park-court-book", 12, 0],
}

def get_week_schedule(driver, time_length, time_start, first_week=True):
    elem = driver.find_element_by_xpath('//div[@class="cal_time cal_time_time"]')
    sched = [e.text for e in elem.find_elements_by_xpath('//div[@class="inc"]')]
    times = sched[time_start:17]
    active = sched[17:]

    # When first week we only want from x till sunday and second week monday till x
    days = ["Sunday", "Saturday", "Friday", "Thursday", "Wednesday", "Tuesday", "Monday"]
    active_days = days[:len(active) // time_length][::-1] if first_week \
             else days[::-1][:len(active) // time_length]
    
    week_schedule = {}
    for i, active_day in enumerate(active_days):
        week_schedule[active_day] = {
            int(times[j].split(":")[0]) : active[i * time_length + j]
            for j in range(time_length)
        }
    return week_schedule

def get_court_schedule(driver, data):
    url, time_length, time_start = data
    driver.get(url)
    
    # Yo dawg, I heard you like iframes so I put an iframe in your iframe...
    WebDriverWait(driver, 3).until(EC.presence_of_element_located((By.XPATH, "//iframe")))    
    driver.switch_to.frame(driver.find_element_by_tag_name("iframe"))
    driver.switch_to.frame(driver.find_element_by_tag_name("iframe"))

    court_schedule = {}
    court_schedule["this"] = get_week_schedule(driver, time_length, time_start)

    # Next week is second element
    elem = driver.find_elements_by_xpath("//a[@href]/span")
    elem[1].click()
    # TODO Fix time.sleep...  but same page so how can we wait?
    time.sleep(5)
    court_schedule["next"] = get_week_schedule(driver, time_length, time_start, first_week=False)

    return court_schedule

def get_schedule():
    driver = webdriver.Chrome()

    court_schedules = {
        court_name : get_court_schedule(driver, url)
        for court_name, url in COURTS.items()
    }

    driver.quit()
    return court_schedules

def get_available_dates(schedule):
    for court_name, court_schedule in schedule.items():
        for week, week_schedule in court_schedule.items():
            for day, day_schedule in week_schedule.items():
                earliest_time = WEEKEND_TIME if day in ("Saturday", "Sunday") else WEEK_TIME
                for hour, available in day_schedule.items():
                    if hour >= earliest_time and available not in ("Full", " "):
                        print(f"Can book at court {court_name} {week} week on {day} at {hour} for price {available}")

if __name__ == "__main__":
    schedule = get_schedule()
    get_available_dates(schedule)

Example output

./tennis.py
Can book at court Poplar Rec next week on Saturday at 10 for price £6
Can book at court Poplar Rec next week on Saturday at 11 for price £6
Can book at court King Edward's Park this week on Saturday at 18 for price £6
Can book at court King Edward's Park next week on Friday at 18 for price £6
Can book at court King Edward's Park next week on Saturday at 10 for price £6
Can book at court King Edward's Park next week on Saturday at 11 for price £6
\$\endgroup\$
4
  • 1
    \$\begingroup\$ Hey. Are the various PEP 8 violations intentional? \$\endgroup\$
    – Peilonrayz
    Commented Sep 25, 2020 at 19:01
  • \$\begingroup\$ I am aware of them, yes. :) \$\endgroup\$
    – Ludisposed
    Commented Sep 25, 2020 at 19:03
  • 1
    \$\begingroup\$ Ah, I though so. :) Might be worth mentioning that in the post. \$\endgroup\$
    – Peilonrayz
    Commented Sep 25, 2020 at 19:06
  • \$\begingroup\$ The widget seems driven by bookingbug. Api here: uk.bookingbug.com/apidocs/v1/index.html Might work... \$\endgroup\$
    – S Hubble
    Commented Oct 26, 2021 at 1:34

1 Answer 1

1
\$\begingroup\$

Managing driver

Any time a resource needs to be cleaned up at the end of life, such as object.close(), a context manager can be really nice to have. This way if you hit an exception the object still gets cleaned up, which might not be the case otherwise.

from contextlib import contextmanager

@contextmanager
def get_driver():
    try:
        driver = webdriver.Chrome()
        yield driver
    finally:
        driver.quit()


def get_schedule():
    with get_driver() as driver:
        court_schedules = {
            court_name : get_court_schedule(driver, url)
            for court_name, url in COURTS.items()
        }

    return court_schedules

Courts global var

I think this could be restructured by eliminating the common base url and just using the extension. The values on the dictionary could be namedtuples for readability. A list is unnecessary, you don't need the mutability here:

from collections import namedtuple

CourtInfo = namedtuple('CourtInfo', ['uri', 'time_length', 'time_start'])

BASE_URL = 'https://www.towerhamletstennis.org.uk'

COURTS = {
    # Some courts have different hours available and the hour at which you can start can differ
    # Thus it is key -> name with values -> url, hours available, time diff start
    "Bethnal Green Gardens": CourtInfo("bethnal-green-gardens-court-bo", 14, 0),
    "St Johns Park": CourtInfo("st-johns-park-court-boo", 14, 1),
    "Poplar Rec": CourtInfo("poplar-rec-court-bookin", 12, 0),
    "King Edward's Park": CourtInfo("kemp-court-bookin", 12, 0),
    "Victoria Park": CourtInfo("victoria-park-court-book", 12, 0),
}

As a stylistic note, you don't need the extra spaces surrounding the colons in your dictionaries.

get_court_schedule args

The arguments here are implicitly an iterable, since they get unpacked in the first line of the function. Second, data is not a great variable name. Paired with a lack of type hints or docstrings, a user of this code would need to read the actual function to see what's going on. Let's expand on the namedtuple that we used earlier:

# I don't know what an actual signature for a webdriver would be
# so I'm making one up for now
def get_court_schedule(
    driver: 'WebDriver', 
    uri: str, 
    time_length: int, 
    time_start: int
) -> dict:
    # now you don't need to unpack these
    # and can format the url together
    driver.get(f"{BASE_URL}/{uri}")
    ...

And to call the function:

court_schedules = {
    court_name : get_court_schedule(driver, *info)
    for court_name, info in COURTS.items()
}

Long Lines

Any line that's continued by a \ can instead use parentheses:

# instead of this
active_days = days[:len(active) // time_length][::-1] if first_week \
     else days[::-1][:len(active) // time_length]

# do this
active_days = (
    days[:len(active) // time_length][::-1]
    if first_week else
    days[::-1][:len(active) // time_length]
)

This works for most other things as well:

# go from this
WebDriverWait(driver, 3).until(EC.presence_of_element_located((By.XPATH, "//iframe")))

# to this
WebDriverWait(driver, 3).until(
    EC.presence_of_element_located((By.XPATH, "//iframe"))
)


# you can break up dotted expressions even further
driver.switch_to.frame(driver.find_element_by_tag_name("iframe"))

(
    driver
    .switch_to
    .frame(
        driver
        .find_element_by_tag_name("iframe")
    )
)

This last example is a bit extreme, but illustrates the point.

You should certainly be using it for function calls that start to be long:

# This
court_schedule["next"] = get_week_schedule(driver, time_length, time_start, first_week=False)

# becomes this
court_schedule["next"] = get_week_schedule(
    driver, 
    time_length, 
    time_start, 
    first_week=False
)

get_available_dates

I don't love how wide this gets, even though you can't completely avoid it. I'd make a generator out of this and put the print statement in a new function:

def get_available_dates(schedule):
    for court_name, court_schedule in schedule.items():
        for week, week_schedule in court_schedule.items():
            for day, day_schedule in week_schedule.items():
                for hour, available in day_schedule.items():
                    if available in ('Full', ' '):
                        continue
                    yield (
                        court_name, 
                        week, 
                        day, 
                        hour, 
                        available
                    )


# Let's use a main function here
def main():
    schedule = get_schedule()

    # Now unpack this later
    dates = get_available_dates(schedule)
    
    for court_name, week, day, hour, available in dates:
        # I've moved this statement here and broken it
        # up to be a bit easier to read
        if day in ('Saturday', 'Sunday'):
            earliest_time = WEEKEND_TIME
        else:
            earliest_time = WEEK_TIME
    
        # now test for this calculated value
        if not hour >= earliest_time:
            continue
    
        # no else statement needed here
        print(
            f"Can book at court {court_name} {week} "
            f"week on {day} at {hour} for price {available}"
        )


if __name__ == "__main__":
    main()
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.