I needed a script to import repositories from other Bitbucket accounts to my own Bitbucket account. I have some 250 repositories in my account and I wanted them to move under Team (which is same as Organization in Github). Though Bitbucket has APIs, I couldn't find any API to import, and instead they provide a HTML form on their site. When I couldn't get my pure Python script to automate and fill the form, I decided to go with Selenium IDE. The following Python script generates Selenium IDE script, which I later run on the IDE.
This is how this script works:
- It takes a REST call to Bitbucket, to find all the repositories
- The Bitbucket provides a paginated response, the response includes repo URL which is just enough
- I make a series of calls to get all repository URLs
- Once I have all the URLs, I fill in the Selenium script template and write it back to a file
This script also can be used to import repositories from Github/or any other version control server by importing the generate_sel_script function.
import json
import requests
from requests.auth import HTTPBasicAuth
from templates import sel_script_template
from templates import command_template
from settings import *
def generate_bb_repo_urls():
def make_rest_call(url):
auth = HTTPBasicAuth(SRC_AC_USERNAME, SRC_AC_PASSWORD)
return requests.get(url=url, auth=auth).json()
seed_url = 'https://bitbucket.org/api/2.0/repositories/' + SRC_AC_USERNAME
response = make_rest_call(seed_url)
bb_repo_urls = []
while('next' in response.keys()):
for repo in response['values']:
bb_repo_urls.append(repo['links']['html']['href'])
response = make_rest_call(response['next'])
return bb_repo_urls
def generate_sel_script(src_repo_urls):
commands = "".join(map(lambda url: command_template.substitute({
'bb_username': SRC_AC_USERNAME,
'bb_password': SRC_AC_PASSWORD,
'repo_url': url,
'owner_value': IMPORTER_ID
}), src_repo_urls))
with open('selenium-script.txt', 'w') as sel_file:
sel_file.write(sel_script_template.substitute({
'bb_username': IMPORTER_BB_USERNAME,
'bb_password': IMPORTER_BB_PASSWORD,
'commands': commands
}))
if __name__ == '__main__':
generate_sel_script(generate_bb_repo_urls())
settings.py:
SRC_AC_USERNAME = 'bitbucket'
SRC_AC_PASSWORD = 'ohlongjohnson'
IMPORTER_BB_USERNAME = 'bitbucket'
IMPORTER_BB_PASSWORD = 'ohlongjohnson'
IMPORTER_ID = '007007'
templates.py:
from string import Template
sel_script_template = Template("""<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head profile="http://selenium-ide.openqa.org/profiles/test-case">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<link rel="selenium.base" href="https://bitbucket.org/account/signin/?next=/repo/import" />
<title>New Test</title>
</head>
<body>
<table cellpadding="1" cellspacing="1" border="1">
<thead>
<tr><td rowspan="1" colspan="3">New Test</td></tr>
</thead><tbody>
<tr>
<td>open</td>
<td>/account/signin/?next=/repo/import</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>id=id_username</td>
<td>$bb_username</td>
</tr>
<tr>
<td>type</td>
<td>id=id_password</td>
<td>$bb_password</td>
</tr>
<tr>
<td>clickAndWait</td>
<td>name=submit</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>100</td>
<td></td>
</tr>
$commands
</tbody></table>
</body>
</html>
""")
command_template = Template("""<tr>
<td>pause</td>
<td>1000</td>
<td></td>
</tr>
<tr>
<td>open</td>
<td>https://bitbucket.org/repo/import</td>
<td></td>
</tr>
<tr>
<td>pause</td>
<td>100</td>
<td></td>
</tr>
<tr>
<td>open</td>
<td>https://bitbucket.org/repo/import</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>id=id_url</td>
<td>$repo_url</td>
</tr>
<tr>
<td>click</td>
<td>id=id_auth</td>
<td></td>
</tr>
<tr>
<td>type</td>
<td>id=id_username</td>
<td>$bb_username</td>
</tr>
<tr>
<td>type</td>
<td>id=id_password</td>
<td>$bb_password</td>
</tr>
<tr>
<td>select</td>
<td>id=id_owner</td>
<td>value=$owner_value</td>
</tr>
<tr>
<td>clickAndWait</td>
<td>xpath=(//button[@type='submit'])[2]</td>
<td></td>
</tr>""")
What I am looking for:
- Any general code improvements/comments
- How do I avoid
from settings import *? (or is it okay forimport *in this case) I have two options,import settingsor fiveimportstatements for each variable. - Is it possible to avoid this line
bb_repo_urls = [], but create a empty list later and append directly in while loop? - Any way to improve this line :
"".join(map(lambda url...
keyringmodule rather than storing Bitbucket credentials insettings.py. \$\endgroup\$