1
\$\begingroup\$

I want to make a http request, get the website content and check one specific element. Currently I'm doing it with lcurl (got explanations from a tutorial). Here is an excerpt of my code:

curl_global_init(CURL_GLOBAL_ALL);
CURL *myHandle;
CURLcode result;
struct BufferStruct output;
output.buffer = NULL;
output.size = 0;

//requested value is on this website
char* url = "https://www.somewebsite.com";

//set curl handle to write the website content into buffer
myHandle = curl_easy_init();
curl_easy_setopt(myHandle, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);
curl_easy_setopt(myHandle, CURLOPT_WRITEDATA, (void*)&output);
curl_easy_setopt(myHandle, CURLOPT_URL, url);
curl_easy_perform( myHandle );

//searching for position of the requested value.
uint8_t* value_position = strstr(output.buffer, "value_description");
//copy the value (8 chars) into a buffer defined earlier
strncpy(value, value_position, 8);
//... clean up etc.

The code works. But I wonder if this is the fastest way... Any ideas how to do it faster?

EDIT: Okay, more explanations: I do run this program in a loop, in my program the url parameter is changing slightly every time, trying lots of websites, not just one. The content I get is plain html. I want to request and check the value on as many websites per second as possible. I hope that information is helpful :)

\$\endgroup\$
  • 1
    \$\begingroup\$ Faster? In which dimension? \$\endgroup\$ – Stephen Rauch Jun 21 '17 at 6:16
  • \$\begingroup\$ Please tell us more about what the request is, and what kind of response you expect from the web server. Also, why is performance important to you? Are you running this code in a loop? If so, how frequently? \$\endgroup\$ – 200_success Jun 21 '17 at 6:17
  • \$\begingroup\$ thx, I edited my post for more information \$\endgroup\$ – Benedict Jun 21 '17 at 6:54
  • \$\begingroup\$ libcurl is very close to the metal and effecient (i.e. there is not much to slow it down apart from your code). Potentially raw sockets is faster but I would not bet on it because of all the extra work you need to do implementing the HTTP protocol (and its unlikely you could implement that correctly or faster than libcurl). So yes the libcurl stuff is the fastest way (and by far the easiest as you don't need to re-implement the HTTP protocol). \$\endgroup\$ – Martin York Jun 21 '17 at 7:06
  • \$\begingroup\$ But there are two other things. One your code that libcurl calls WriteMemoryCallback() and your usage of libcurl. You are only using the easy version of the libcurl interface which is basically one connection at a time. But if you want to make lots of connections you need to use the multi interface curl.haxx.se/libcurl/c/libcurl-multi.html \$\endgroup\$ – Martin York Jun 21 '17 at 7:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.