Make a “Web-Proxy” - step-by-step

My problem with links is solved. Text challenge is to copy pictures on remote page to my own host, then replace the links in source code. Any idea?

I started creating a web proxy.

I decided to explain my work here for two reasons:

1. For everyone who wants to start a similar project.
2. Most parts of this code are copied from Stack pages; I've just collected them.

I need experts to correct my mistakes and help me to continue. Here is what I did:

ASP (Default.aspx):

• I put a textbox named txtURL to enter the web address by user.
• I put a button named btnRun to start processing.

For now, these components are enough!

C#:

Clicking on btnRun makes the page redirect to: "www.domain.com/default.aspx?URL=(xxx)" - xxx will be replaced by web page address encrypted by a function.

This is the code for btnRun_Click:

protected void btnRun_Click(object sender, EventArgs e)
{
if (txtURL.Text.Length == 0) return;
if (!(txtURL.Text.ToLower().StartsWith("http://") || txtURL.Text.ToLower().StartsWith("https://")))
txtURL.Text = "http://" + txtURL.Text;

try
{
Response.Redirect("Default.aspx?URL=(" + Encrypt(txtURL.Text, mainKey) + ")", false);
}
catch (Exception ex)
{
ShowPopUpMsg(ex.Message);
}


I'll explain Encrypt and ShowPopUpMsg functions later.

By clicking on btnRun, this page will be refreshed and the encrypted URL will be included in the address.

Now, in Page_Load, we should read the encrypted URL (also a condition to detect postback):

protected void Page_Load(object sender, EventArgs e)
{
string url = Regex.Match(HttpContext.Current.Request.Url.AbsoluteUri, @"$$([^)]*)$$").Groups[1].Value;
if (url.Length == 0 || Page.IsPostBack) return;


From now, every code is added to Page_Load, one after other.

Decrypt the URL and read the remote web page source-code:

try
{
txtURL.Text = Server.UrlDecode(Decrypt(url, mainKey));
string TheUrl = txtURL.Text;
string response = GetHtmlPage(TheUrl);


I'll explain Decrypt and GetHtmlPage later.

Now, we have the source-code in response<string>.

Next step is find the links in this source-code. Beginning of the links is **href="xxx"** and xxx is the link. We must replace them with our links through the proxy:

            response = response.Replace("href =", "href=");
response = response.Replace("href\n=", "href=");
response = response.Replace("href\t=", "href=");

HtmlWeb hw = new HtmlWeb();
{
char[] c = { ' ', '\"' };
int from = s.IndexOf("href=");
int to = SearchString(s, from, '\"');

s = s.Substring(from + 5, to - from - 5);
s.TrimStart(c);
if (s.StartsWith("\"")) s = s.Remove(0, 1);


SearchString is a function to return the closing quotation mark of href. I'll explain this later.

There are two kind of links:

1. Links that refer to another domain-name. This links are begun with "http://" or "https://". We'll find them and replace the address:

            string corrected = "href=\"" + "Default.aspx?URL=(" + Encrypt(s, mainKey) + ")" + "\"";
if ((s.ToLower().StartsWith("http://") || s.ToLower().StartsWith("https://")))
response = response.Replace("href=\"" + s + "\"", corrected);

2. Link that refer to current domain-name. This links are begun with "/". To replace them, we should first find the domain name then the whole address:

            else
{
var uri = new Uri(txtURL.Text);
string domain = uri.GetComponents(UriComponents.Host, UriFormat.SafeUnescaped);
corrected = "href=\"" + "Default.aspx?URL=(";
if (txtURL.Text.ToLower().StartsWith("http://")) corrected += Encrypt("http://" + domain + s, mainKey);
if (txtURL.Text.ToLower().StartsWith("https://")) corrected += Encrypt("https://" + domain + s, mainKey);
corrected += ")" + "\"";
response = response.Replace("href=\"" + s + "\"", corrected);
}


Now, everything is done (refer to my current knowledge) and we should show the page with new links and finish Page_Load:

            }
Response.Write(response);
}
catch (Exception ex)
{
ShowPopUpMsg(ex.Message);
}
}


Function to search in a string:

private int SearchString(string mainString, int startLocation, char charToFind)
{
if (startLocation < 0) return -1;
bool next = false;
for (int i = startLocation; i < mainString.Length; i++)
if (mainString.Substring(i, 1) == charToFind.ToString() && next)
return i;
else
{
if (mainString.Substring(i, 1) == charToFind.ToString()) next = true;
continue;
}
return -1;
}


private string GetHtmlPage(string URL)
{
String strResult;
WebResponse objResponse;
WebRequest objRequest = HttpWebRequest.Create(URL);
objResponse = objRequest.GetResponse();
{
sr.Close();
}
return strResult;
}


Function to show a popup message:

private void ShowPopUpMsg(string msg)
{
StringBuilder sb = new StringBuilder();
sb.Append(msg.Replace("\n", "\\n").Replace("\r", "").Replace("'", "\\'"));
sb.Append("');");
}


Function to decrypt a string:

private string Decrypt(string s, string key)
{
try
{
byte[] keyArray; byte[] toEncryptArray = Convert.FromBase64String(s);
MD5CryptoServiceProvider hashmd5 = new MD5CryptoServiceProvider();
keyArray = hashmd5.ComputeHash(UTF8Encoding.UTF8.GetBytes(key)); hashmd5.Clear();
TripleDESCryptoServiceProvider tdes = new TripleDESCryptoServiceProvider();
ICryptoTransform cTransform = tdes.CreateDecryptor();
byte[] resultArray = cTransform.TransformFinalBlock(toEncryptArray, 0, toEncryptArray.Length);
tdes.Clear(); return UTF8Encoding.UTF8.GetString(resultArray);
}
catch { return null; }
}


Function to encrypt a string:

private string Encrypt(string s, string key)
{
try
{
byte[] keyArray; byte[] encryptArray = UTF8Encoding.UTF8.GetBytes(s);
MD5CryptoServiceProvider Hashmd5 = new MD5CryptoServiceProvider();
keyArray = Hashmd5.ComputeHash(UTF8Encoding.UTF8.GetBytes(key)); Hashmd5.Clear();
TripleDESCryptoServiceProvider Tdes = new TripleDESCryptoServiceProvider();
ICryptoTransform Ctransform = Tdes.CreateEncryptor();
byte[] resultarray = Ctransform.TransformFinalBlock(encryptArray, 0, encryptArray.Length);
Tdes.Clear(); return Convert.ToBase64String(resultarray, 0, resultarray.Length);
}
catch { return null; }
}


Just from a quick read-through it looks like you're trying to write some kind of secured web-based proxy. Unfortunately the approach in the code I've seen so far has major design failings which make it susceptible to a range of attacks. You'll need to have a radical rethink of the threats you're trying to mitigate before you continue.

Displaying your exception bodies on the output is not a good idea as it can reveal information about your system that might aid attackers. It's also a bad user experience to expose implementation details. Instead, log the exception details to a file or database and then redirect the user to a generic error page. Give them some kind of unique error report number if the client/tester is able to contact you.

Page_Load only attempts to modify html where it could handle a wider range of text based protocols by a more simple string search/replace looking for URL's in plain text content. Sometimes the simplest option is also the best.

SearchString seems to have reinvented the wheel somewhat - were you looking for string.IndexOf?

Functions to call the server don't include cookies, they are heavily used and your proxy will have trouble functioning without them.

ShowPopupMsg is using StringBuilder which is really the wrong tool for the job. StringBuilder is only really an advantage where you have big strings and need to avoid rewrites, or you are reusing the stringbuilder. It's a premature optimisation and the place it would be needed more is where the downloaded content is modified.

I've not looked at the encryption/decryption functions, certainly they'll obfuscate the request which might frustrate any would-be attacker. I'd want a security specialist to verify they're correct.

Designing things to be secure is hard, some thought has gone into the possible attack vectors which is a step in the right direction. As a general tip, perhaps you need to spend more time looking for common functions as parts of what you have written are already in the language. Good luck!