# Comparing web-scraped data from several sports betting APIs

I have written a python script that requests information from several Sports Betting API's, processes and standardises the results into a dictionary, Matches items between dictionaries and then performs a calculation based on the results. The code works however I am facing two problems:

1. The code is very slow taking up to an hour to execute

2. When the code is executed it seems to use my CPU instead of GPU which may be a contributing factor to how slow it is

Currently my code uses for loops and fuzzywuzzy to match names and bet types in different dictionaries I have standardised from several API's against a list of tuples. Once a match is found the code then executes a calculation which is returned if it is below 1. This works but is very slow.

from fuzzywuzzy import fuzz,process

BETennis = {'Frances Tiafoe v Roberto Bautista Agut': {'PVP': {'PlayerA/Match/Win': 3.6, 'PlayerB/Match/Win': 1.28, 'PlayerA/Set1/Win': 2.94, 'PlayerB/Set1/Win': 1.39, 'PlayerA/FirstService/Win/Yes': 1.28, 'PlayerA/FirstService/Win/No': 3.3, 'PlayerB/FirstService/Win/Yes': 1.11, 'PlayerB/FirstService/Win/No': 5}, 'ID': {'EventID': '4806539', 'PlayerA': 'Frances Tiafoe', 'PlayerB': 'Roberto Bautista Agut', 'Agency': 'BET'}}, 'Nick Kyrgios v Karen Khachanov': {'PVP': {'PlayerA/Match/Win': 1.72, 'PlayerB/Match/Win': 2.08, 'PlayerA/Set1/Win': 1.79, 'PlayerB/Set1/Win': 1.96, 'PlayerA/FirstService/Win/Yes': 1.05, 'PlayerA/FirstService/Win/No': 8, 'PlayerB/FirstService/Win/Yes': 1.06, 'PlayerB/FirstService/Win/No': 7.5}, 'ID': {'EventID': '4804932', 'PlayerA': 'Nick Kyrgios', 'PlayerB': 'Karen Khachanov', 'Agency': 'BET'}}}

TBTennis = {'Frances Tiafoe v Roberto Bautista Agut': {'PVP': {'PlayerA/Match/Win': 9, 'PlayerB/Match/Win': 1.99, 'PlayerA/Set1/Win': 2.94, 'PlayerB/Set1/Win': 1.42, 'PlayerA/FirstService/Win/Yes': 1.35, 'PlayerA/FirstService/Win/No': 3.67, 'PlayerB/FirstService/Win/Yes': 1.11, 'PlayerB/FirstService/Win/No': 5}, 'ID': {'EventID': '4806539', 'PlayerA': 'Frances Tiafoe', 'PlayerB': 'Roberto Bautista Agut', 'Agency': 'TB'}}, 'Nick Kyrgios v Karen Khachanov': {'PVP': {'PlayerA/Match/Win': 1.78, 'PlayerB/Match/Win': 2.98, 'PlayerA/Set1/Win': 1.99, 'PlayerB/Set1/Win': 1.96, 'PlayerA/FirstService/Win/Yes': 1.15, 'PlayerA/FirstService/Win/No': 8.1, 'PlayerB/FirstService/Win/Yes': 1.09, 'PlayerB/FirstService/Win/No': 7.5}, 'ID': {'EventID': '4804932', 'PlayerA': 'Nick Kyrgios', 'PlayerB': 'Karen Khachanov', 'Agency': 'TB'}}}

PVPtuples = [(('PlayerA/Match/Win'), ('PlayerB/Match/Win')), (('PlayerA/Set1/Win'), ('PlayerB/Set1/Win')), (('PlayerA/Set2/Win'), ('PlayerB/Set2/Win')), (('PlayerA/Aset/Win/Yes'), ('PlayerA/Aset/Win/No'))]

###BETennis/TBTennis###

for i in BETennis:
for e in BETennis[i]['PVP']:
for p in TBTennis:
for m in TBTennis[p]['PVP']:
for y in PVPtuples:
try:
if fuzz.token_sort_ratio(BETennis[i]['ID']['PlayerA'], TBTennis[p]['ID']['PlayerA']) > 80 and fuzz.token_sort_ratio(BETennis[i]['ID']['PlayerB'], TBTennis[p]['ID']['PlayerB']) > 80:
if e == m and y[0] == e:
c = 1/float(TBTennis[p]['PVP'][y[1]]) + 1/float(BETennis[i]['PVP'][y[0]])
if c < 1:
print(TBTennis[p]['ID']['Agency'] + '/' + TBTennis[p]['ID']['PlayerB'] + '/' + y[1] + '  ' + BETennis[i]['ID']['Agency'] + '/' + BETennis[i]['ID']['PlayerA'] + '/' + y[0] + '       ' + str(c))
except:
continue

for i in BETennis:
for e in BETennis[i]['PVP']:
for p in TBTennis:
for m in TBTennis[p]['PVP']:
for y in PVPtuples:
try:
if fuzz.token_sort_ratio(BETennis[i]['ID']['PlayerA'], TBTennis[p]['ID']['PlayerA']) > 80 and fuzz.token_sort_ratio(BETennis[i]['ID']['PlayerB'], TBTennis[p]['ID']['PlayerB']) > 80:
if e == m and y[0] == e:
c = 1/float(TBTennis[p]['PVP'][y[0]]) + 1/float(BETennis[i]['PVP'][y[1]])
#print(TBTennis[p]['ID']['Agency'] + y[0] + '/' + BETennis[i]['ID']['Agency'] + y[1] + c)
if c < 1:
print(TBTennis[p]['ID']['Agency'] + '/' + TBTennis[p]['ID']['PlayerA'] + '/' + y[0] + '  ' + BETennis[i]['ID']['Agency'] + '/' + BETennis[i]['ID']['PlayerB'] + '/' + y[1] + '       ' + str(c))
except:
continue



The code works more or less as expected however as I am new to python and programming in general I think I may be doing this in a very inefficient way. One remedy I am currently exploring is using CUDA to try run this using my NVIDIA GPU instead of my CPU as it currently does. Any and all suggestions would be greatly appreciated. One last thing to note is the dictionaries in my code are significantly longer and more extensive than the ones used in the example above.

I'm not the Python expert you might be expecting to perform some magic on your code and make it run in fractions of a second and there are lots of much more experienced guys than myself on the website however here are some general things you should consider before asking for advice/review/whatever:

• Nesting levels: I suggest you separate the nested dictionaries each having a unique name and a unique purpose.
• If you expect anyone to try to work/suggest improvements on the code, I suggest you make it more readable for human beings: a line shouldn't exceed 120 characters, variable names should be descriptive (for i for e for p for m for n) what exactly i, e, m, n, whatever_the_name_is are meant to do
• The try and except are meant to catch errors, not a decoration for the code and you're not excepting anything btw, you should specify the exception expected to occur and what you actually did is try block_of_code ... except nothing_indicated ... so I guess it's pointless.
• I'm surprised this code only takes an hour to execute with 9 nested fors and ifs, the moment you start nesting the 2nd for, you might start considering another approach for solving whatever the problem at hand or the running time is going to grow exponentially.
• When you fix some of many things that I did and I did not mention, you might post the modified working version of the code and ask for some advice.
• from fuzzywuzzy import fuzz,process ... No one is expected to decode what these things do, if you're not going to include what they contain in your code at least give us the honor of writing some comments that begin with # that indicate what these things do.
• You must include a description for what your code is exactly meant to do not some vague remarks "Once a match is found the code then executes a calculation which is returned if it is below 1." what kind of calculation that is being executed?
• Separate your dictionaries including the nested ones, make your code more readable, write comments, give a clear description for what the code is expected to do then repost this code because most probably no one will be able to decode what this is meant for.
• Hi thanks for the reply. I’ll try addresses your points. First, I tried minimise the code length and explanation to key important points as this website suggests when posting a question, apologies if it confused you. As such ill start with a more in-depth explanation. My code is fairly simple it makes requests to several Sports Book APIs for betting data. This API data is stored in what is essentially nested dictionaries. Each API uniquely stores and names bets for example one API might name the bet for the Match winner as PlayerAHeadtoHead whereas another might call it PlayerAMatchWin. – Jordan Aug 15 '19 at 8:50
• As such I created new dictionaries from each API in a standardised format, ie PlayerA/Match/Win in dictionary1 is the same as PlayerA/Match/Win in dictionary2 except obviously with different odds from respective websites. – Jordan Aug 15 '19 at 8:50
• If the same bet type (such as match winner) is contained in both dictionaries then a simple arbitrage calculation is done which is the sum of 1 divided by the odds of a bet in both directions. For example: 1/Odds for player A to win + 1/Odds for player B to win. To do this my code verifies that the matches being compared are the same by comparing PlayerA and PlayerB names in dictionary 1 and dictionary 2. – Jordan Aug 15 '19 at 8:50
• As mentioned earlier each API has slightly varied names, this extends to Match/Player names so for example Rafael Nadal may be called by his full name in one API as Rafael Nadal but may be called by an abbreviated version in another like R.Nadal. To overcome this, I used fuzzy wuzzy to compare and accept matches above 80%. If this is true matching bets are found using == as I have standardised bet names. – Jordan Aug 15 '19 at 8:50
• To find the opposite type of bet in the other dictionary I created a list of tuples that contain each standardised bet name and its opposite ie ((PlayerA/Win), (PlayerB/Win)). The matched bets are compared to this list of tuples to find the opposite bet in each dictionary and perform the arbitrage calculation. This is what the I, e, m and n are referring. – Jordan Aug 15 '19 at 8:51