3
\$\begingroup\$

Due to recent need I wrote a simple main function that has the goal to convert the C-style strings and arrays into a more STL-style. Then because I also had a need for it, I created a variation that converts all the involved std::strings into std::wstrings.

So essentially this is to be seen as two parts:

  • Converting C-style main into STL-style main
  • Converting STL-style main into STL-style main with wide strings (in practice you can leave that out if you don't need wide strings)
#include <algorithm>
#include <codecvt>
#include <locale>
#include <string>
#include <vector>

int main(int argc, char* argv[]);
int stlMain(std::string&& cmd, std::vector<std::string>&& args);
int stlWMain(std::wstring&& cmd, std::vector<std::wstring>&& args);

int main(int argc, char* argv[]) {
    std::string cmd{argv[0]};

    std::vector<std::string> args;
    args.reserve(argc - 1);
    args.assign(argv + 1, argv + argc);

    return stlMain(std::move(cmd), std::move(args));
}

int stlMain(std::string&& cmd, std::vector<std::string>&& args) {
    // main with STL objects

    std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> converter;

    std::wstring wcmd = converter.from_bytes(cmd);

    std::vector<std::wstring> wargs;
    wargs.reserve(args.size());
    std::transform(args.cbegin(), args.cend(), std::back_inserter(wargs),
                   [&converter](const std::string& str) { return converter.from_bytes(str); });

    return stlWMain(std::move(wcmd), std::move(wargs));
}

int stlWMain(std::wstring&& cmd, std::vector<std::wstring>&& args) {
    // main with wide strings

    // Start you program here
}

I'm aware that this requires C++11 at least due to the codecvt header and that some compilers don't support it.

\$\endgroup\$
2
  • \$\begingroup\$ No I'm compiling this with C++17, but the code itself works up until C++11, that's why I tagged it that way. \$\endgroup\$
    – BrainStone
    Oct 14 at 10:18
  • \$\begingroup\$ That's not how the tags work here - you should tag for your target platform, so that you get reviews appropriate to C++17. I've made that change for you. \$\endgroup\$ Oct 14 at 10:40
2
\$\begingroup\$

For just upgrading the char* arguments to object types, you should use string_view instead, and this does not require recopying the data. That's true for parameters in general.

std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>> converter;

The meaning of wchar_t is implementation-dependent. I understand that on non-Windows platforms it's actually a 32-bit value. Yet you are only wanting 16-bit code points. Use char16_t instead.

I agree with the posters that said "UTF-8 everywhere" is better than wide characters. But working in UTF-16 is useful/necessary/efficient for some purposes including specific platforms (specifically Windows). I understand that the convert stuff in std is actually deprecated because it is flawed and has issues. If you're doing this because it's for a specific platform, it might be better to use the platform-specific features instead. OTOH, the supplied Win32 functions also have issues with error checking and reporting and lack of options for dealing with invalid data, so in a similar situation I used my own code.

You're not doing any error checking or trapping. It also assumes that the arguments coming into the program are encoded as UTF-8! (That's certainly not the case in Windows) It needs to check the Locale to determine how the incoming characters are encoded; or I should say how they are supposed to be encoded, as if the program was not invoked from an interactive command line it could still get it wrong.

\$\endgroup\$
1
\$\begingroup\$

This is not as portable as we'd like it to be. Instead of assuming that the arguments are supplied as UTF-8 bytes, we ought to use the local character encoding, as used by std::locale{""}.

\$\endgroup\$
0
\$\begingroup\$

Design:

I'd recommend going the UTF-8 Everywhere route instead.

This means avoiding std::wstrings and using std::strings in your program, encoded as utf-8. This requires a little care when processing strings, since you can't assume that one byte is one character, but it's better overall (see the link above for details).

On Windows, you then need to convert strings back to UTF-16 (what Microsoft calls "Unicode") when calling functions to open files, or using the Windows API. You can convert UTF-8 to UTF-16 with MultiByteToWideChar, and back again with WideCharToMultiByte.

You can get UTF-16 command line arguments on Windows with GetCommandLineW or CommandLineToArgvW, which should then be converted to UTF-8 for use internally.


On Linux there is no fixed encoding for command line arguments. Perhaps it's reasonable to simply require that arguments to your program are passed as UTF-8.

Otherwise I guess you're stuck with converting from the current locale... But you might run into issues with arguments in different encodings as described in that stackoverflow link.


Code:

I'd suggest avoiding the nested call to an inner main function. Extracting the first argument and passing it separately also seems unnecessary.

So perhaps:

std::vector<std::string> read_args(int argc, char* argv[]);
std::vector<std::wstring> convert_strings_utf8_to_utf16(std::vector<std::string> const& args);

int main(int argc, char* argv[]) {
    
    auto args = read_args(argc, argv);
    auto wargs = convert_strings_utf8_to_utf16(args);

    return run_program(wargs);
}
\$\endgroup\$
3
  • \$\begingroup\$ That's twice the allocations necessary... \$\endgroup\$ Oct 14 at 11:37
  • \$\begingroup\$ @Deduplicator Do you mean that it should convert each argument to a std::wstring first before adding directly to the std::vector<std::wstring> ? \$\endgroup\$
    – user673679
    Oct 14 at 11:44
  • \$\begingroup\$ I wanted to separate the program name from the arguments, as to me that isn't an argument. I do understand the technical reasons as to why it has been implemented like this, but personally speaking I see it as something separate enough to warrant putting it into it's own variable \$\endgroup\$
    – BrainStone
    2 days ago

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.