1
\$\begingroup\$

I have one application in c++ and i'm using the C++ Mysql Connector (https://dev.mysql.com/downloads/connector/cpp/)

I need to save some logs inside one table.Depending on the times I may have large amounts of data on the order of thousands (for example, 80,000).

I already implement a function that iterate my std::vector and save the std::string to my database.

For example:

std::vector<std::string> lines = explode(filedata, '\n');
for (int i = 0; i < lines.size(); i++)
{
   std::vector<std::string> elements = explode(lines[i], ';');
   ui64 timestamp = strtol(elements.at(0).c_str(), 0, 10);
   std::string pointId = elements.at(6);
   std::string pointName = elements.at(5);
   std::string data = elements.at(3);

   database->SetLogs(timestamp, pointId, pointName, data);
}

The logs come from csv file, i save all fields to my vector. After this i parse the vector (with explode) and get only the fields that i need to save.

But i have a problem. If i have e.g 80,000 i'm calling my function to save in database 80,000. It works and save correctly all data but it takes a lot of time.

Exists some way to save all data only calling one time the function to save without calling e.g 80,000 and thus optimize the time?

\$\endgroup\$
  • 3
    \$\begingroup\$ Welcome to Code Review! You'll receive better reviews if you show a complete example. For example, I recommend that you edit to show the necessary #include lines, and a main() that shows how to call your function. It's not mandatory, but it really helps! \$\endgroup\$ – Toby Speight Sep 5 '17 at 10:41
  • \$\begingroup\$ I feel this question is more suited for stackoverflow... Anyway, you may try MySQL BULK INSERT, possibly with no need to write a program: BULK INSERT db.table FROM 'log.csv' WITH FIELDTERMINATOR = ',' \$\endgroup\$ – sorush-r Sep 6 '17 at 7:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.