# Converting between std::wstring and std::string

While researching ways to convert back and forth between std::wstring and std::string, I found this conversation on the MSDN forums.

There were two functions that, to me, looked good. Specifically, these:

std::wstring s2ws(const std::string& s)
{
int len;
int slength = (int)s.length() + 1;
len = MultiByteToWideChar(CP_ACP, 0, s.c_str(), slength, 0, 0);
wchar_t* buf = new wchar_t[len];
MultiByteToWideChar(CP_ACP, 0, s.c_str(), slength, buf, len);
std::wstring r(buf);
delete[] buf;
return r;
}

std::string ws2s(const std::wstring& s)
{
int len;
int slength = (int)s.length() + 1;
len = WideCharToMultiByte(CP_ACP, 0, s.c_str(), slength, 0, 0, 0, 0);
char* buf = new char[len];
WideCharToMultiByte(CP_ACP, 0, s.c_str(), slength, buf, len, 0, 0);
std::string r(buf);
delete[] buf;
return r;
}


However, the double allocation and the need to delete the buffer concern me (performance and exception safety) so I modified them to be like this:

std::wstring s2ws(const std::string& s)
{
int len;
int slength = (int)s.length() + 1;
len = MultiByteToWideChar(CP_ACP, 0, s.c_str(), slength, 0, 0);
std::wstring r(len, L'\0');
MultiByteToWideChar(CP_ACP, 0, s.c_str(), slength, &r[0], len);
return r;
}

std::string ws2s(const std::wstring& s)
{
int len;
int slength = (int)s.length() + 1;
len = WideCharToMultiByte(CP_ACP, 0, s.c_str(), slength, 0, 0, 0, 0);
std::string r(len, '\0');
WideCharToMultiByte(CP_ACP, 0, s.c_str(), slength, &r[0], len, 0, 0);
return r;
}


Unit testing indicates that this works in a nice, controlled environment but will this be OK in the vicious and unpredictable world that is my client's computer?

-

## 9 Answers

I would, and have, redesign your set of functions to resemble casts:

std::wstring x;
std::string y = string_cast<std::string>(x);


This can have a lot of benefits later when you start having to deal with some 3rd party library's idea of what strings should look like.

-
I love the syntax. Can you share the code? – Jere.Jones Jan 31 '11 at 18:53
Oooh. That looks nice. How would one do that? Just make a template with specializations to convert between the various string types? – Billy ONeal Feb 5 '11 at 18:21
@Billy someone posted a codereview question for string_cast implementation on here if you guys are interested. – greatwolf Nov 5 '11 at 3:20

Actually my unit testing shows that your code is wrong!

The problem is that you include the zero terminator in the output string, which is not supposed to happen with std::string and friends. Here's an example why this can lead to problems, especially if you use std::string::compare:

// Allocate string with 5 characters (including the zero terminator as in your code!)
string s(5, '_');

memcpy(&s[0], "ABCD\0", 5);

// Comparing with strcmp is all fine since it only compares until the terminator
const int cmp1 = strcmp(s.c_str(), "ABCD"); // 0

// ...however the number of characters that std::string::compare compares is
// someString.size(), and since s.size() == 5, it is obviously not equal to "ABCD"!
const int cmp2 = s.compare("ABCD"); // 1

// And just to prove that string implementations automatically add a zero terminator
// if you call .c_str()
s.resize(3);
const int cmp3 = strcmp(s.c_str(), "ABC"); // 0
const char term = s.c_str()[3]; // 0

printf("cmp1=%d, cmp2=%d, cmp3=%d, terminator=%d\n", cmp1, cmp2, cmp3, (int)term);

-
I found the addition of the terminator annoying too: it also broke a string addition in my case. I ended up adding the boolean parameter includeTerminator to both methods. – reallynice Jul 30 '15 at 13:01

One thing that may be an issue is that it assumes the string is ANSI formatted using the currently active code page (CP_ACP). You might want to consider using a specific code page or CP_UTF8 if it's UTF-8.

-
This may be a silly question but, how can I tell? For my usage these will typically be filenames. – Jere.Jones Jan 31 '11 at 19:49
How do you obtain the filenames? That will determine the correct code page to use. – Ferruccio Feb 1 '11 at 1:30
@Jere.Jones: One way is to check if the string is valid UTF-8. If not, assume it's ANSI. – dan04 Feb 5 '11 at 16:31
@dan04: ANSI requires that a code page is specified. en.wikipedia.org/wiki/Code_page. – Ferruccio Feb 5 '11 at 21:33

I'd recommend changing this:

int len;
int slength = (int)s.length() + 1;
len = WideCharToMultiByte(CP_ACP, 0, s.c_str(), slength, 0, 0, 0, 0);


...to this:

int slength = (int)s.length() + 1;
int len = WideCharToMultiByte(CP_ACP, 0, s.c_str(), slength, 0, 0, 0, 0);


Slightly more concise, len's scope is reduced, and you don't have an uninitialised variable floating round (ok, just for one line) as a trap for the unwary.

-

I don't do any Windows development, so I can't comment on the WideCharToMultiByte part being safe.

The one thing I would say though is to ensure you are using the proper types for everything. For example, string.length() returns a std::string::size_type (most likely a size_t, the constructor also takes a std::string::size_type, but that one isn't as big of a deal). It probably won't ever bite you, but it is something to be careful of to ensure you don't have any overflows in other code you may be writing.

-
Well, it returns a std::string::size_type. – Jon Purdy Jan 30 '11 at 9:15
@Jon: True, but I've never it seen it not be equal to the representation of a size_t. I'll modify the answer though, thanks for your feedback. – Mark Loeser Jan 30 '11 at 15:41
@Jon: std::string::size_type is always a std::size_t. – GManNickG Jan 30 '11 at 22:02
@GMan: I was just being pedantic out of boredom. SGI says it's "an unsigned integral type that can represent any nonnegative value of the container's distance type"—that is, difference_type—and that both of these must be typedef s for existing types, but this doesn't imply that size_type has to be equivalent to size_t. Is there something else at work here? – Jon Purdy Jan 30 '11 at 23:28
@Jon: I'm not sure why SGI matters. The standard says std::string::size_type is allocator_type::size_type, and the default allocator's size_type is std::size_t. – GManNickG Jan 31 '11 at 5:31

I think you invoke undefined behavior in this section:

std::wstring r(len, L'\0');
MultiByteToWideChar(CP_ACP, 0, s.c_str(), slength, &r[0], len);


when you pass &r[0] as the output parameter to MB2WC.

I don't have the standard handy, but here is what cplusplus.com has to say:

string::operator[]

The function actually returns data()[ pos ].

and

string::data

The returned array points to an internal location which should not be modified directly in the program.

-

Here's a cross platform version I've written for a framework I'm working on, it uses the UTF8 code page but fell free to change as needed. This is a slimmed version as it doesn't contain all of the explicit macro definitions, but you can get the idea from it:

#if defined(OMNI_OS_WIN)
#include <windows.h>
#endif
#include <cctype>
#include <cwctype>
#include <string>
// not sure if these are all needed .. haven't had my midnight coffee :)

std::string omni::string::to_string(const std::wstring& str)
{
size_t sz = str.length();
#if defined(OMNI_OS_WIN)
int nd = WideCharToMultiByte(CP_UTF8, 0, &str[0], sz, NULL, 0, NULL, NULL);
std::string ret(nd, 0);
int w = WideCharToMultiByte(CP_UTF8, 0, &str[0], sz, &ret[0], nd, NULL, NULL);
if (w != sz) {
#if defined(OMNI_THROW_ON_ERR)
throw omni::string_exception("Invalid size written");
#else
OMNI_ERR_RETV("");
#endif
}
return ret;
#else
const wchar_t* p = str.c_str();
char* tp = new char[sz];
size_t w = wcstombs(tp, p, sz);
if (w != sz) {
delete[] tp;
#if defined(OMNI_THROW_ON_ERR)
throw omni::string_exception("Invalid size written");
#else
OMNI_ERR_RETV("");
#endif
}
std::string ret(tp);
delete[] tp;
return ret;
#endif
}

std::wstring omni::string::to_wstring(const std::string& str)
{
#if defined(OMNI_OS_WIN)
size_t sz = str.length();
int nd = MultiByteToWideChar(CP_UTF8, 0, &str[0], sz, NULL, 0);
std::wstring ret(nd, 0);
int w = MultiByteToWideChar(CP_UTF8, 0, &str[0], sz, &ret[0], nd);
if (w != sz) {
#if defined(OMNI_THROW_ON_ERR)
throw omni::string_exception("Invalid size written");
#else
OMNI_ERR_RETV(L"");
#endif
}
return ret;
#else
const char* p = str.c_str();
size_t len = str.length();
size_t sz = len * sizeof(wchar_t);
wchar_t* tp = new wchar_t[sz];
size_t w = mbstowcs(tp, p, sz);
if (w != len) {
delete[] tp;
#if defined(OMNI_THROW_ON_ERR)
throw omni::string_exception("Invalid size written");
#else
OMNI_ERR_RETV(L"");
#endif
}
std::wstring ret(tp);
delete[] tp;
return ret;
#endif
}


Here's an example using it:

std::string s = "here's a standard string";
std::wstring w = L"here's a wide string";
std::string sw = omni::string::to_string(w);
std::wstring ws = omni::string::to_wstring(s);

std::cout << "s = " << s << std::endl;
std::wcout << "w = " << w << std::endl;
std::cout << "sw = " << sw << std::endl;
std::wcout << "ws = " << ws << std::endl;


Hope that can help someone.

-

I've only briefly looked over your code. I haven't worked with std::string much but I've worked a lot with the API.

Assuming you got all your lengths and arguments right (sometimes making sure the terminator and wide vs multibyte lengths are all right can be tricky), I think you're on the right track. I think the first routines you posted unnecessarily allocate an additional buffer. It isn't needed.

-

No, this is dangerous! The characters in a std::string may not be stored in a contiguous memory block and you must not use the pointer &r[0] to write to any characters other than that character! This is why the c_str() function returns a const pointer.

It might work with MSVC, but it will probably break if you switch to a different compiler or STL library.

-
-1: Wrong: stackoverflow.com/questions/2256160/… – Billy ONeal Nov 5 '11 at 13:59

## protected by Jamal♦Dec 20 '13 at 1:11

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?