Google API Wrapper

Question

Playing around with the google API.

I have written demo applications to use the google gmail API. But what I am actually looking for is a review of the C++ wrappers for Authentication (OAuth2) and an initial GMail wrapper. Want input before I start going any further.

Please take your time to point out all even the slightest issues you have the code (my skin is thick). BUT also interested in if anybody can think of better ways to design the interfaces. How would you expect to use objects from Google (we don't have to map REST API to class method I would like to design a system were interacting with the objects at the code level is intuative).

CurlHelper.h

#ifndef THROSANVIL_CURL_CURLHELPER_H
#define THROSANVIL_CURL_CURLHELPER_H

#include <curl/curl.h>
#include <memory>
#include <functional>
#include <string>

namespace ThorsAnvil::Curl
{
    using CurlDeleter = std::function<void(CURL*)>;
    using CurlHDeleter= std::function<void(curl_slist*)>;

    class CurlUniquePtr: public std::unique_ptr<CURL, CurlDeleter>
    {
        public:
            CurlUniquePtr(std::string const& url)
                : std::unique_ptr<CURL, CurlDeleter>{curl_easy_init(), [](CURL* c){curl_easy_cleanup(c);}}
            {
                if (get() == nullptr) {
                    throw std::runtime_error("Failed to Create Curl Handle");
                }
                if (curl_easy_setopt(get(), CURLOPT_URL, url.c_str()) != CURLE_OK) {
                    throw std::runtime_error("Could not set a valid URL");
                }
            }
            operator CURL*() {
                return get();
            }
    };
    class CurlHeaderUniquePtr: public std::unique_ptr<curl_slist, CurlHDeleter>
    {
        public:
            CurlHeaderUniquePtr(std::initializer_list<std::string> const& list = std::initializer_list<std::string>{})
                : std::unique_ptr<curl_slist, CurlHDeleter>{nullptr, [](curl_slist* s){curl_slist_free_all(s);}}
            {
                for (auto const& head: list) {
                    append(head);
                }
            }

            void append(std::string const& header)
            {
                curl_slist* tmp = curl_slist_append(get(), header.c_str());
                if (tmp) {
                    release();
                    reset(tmp);
                }
            }
    };
}

#endif

Credentials.h

#ifndef THORSANVIL_GOOGLE_CREDENTIALS_H
#define THORSANVIL_GOOGLE_CREDENTIALS_H

#include "CurlHelper.h"

#include "ThorSerialize/Traits.h"
#include "ThorSerialize/SerUtil.h"
#include "ThorSerialize/JsonThor.h"

#include <string>
#include <vector>
#include <fstream>
#include <sstream>

extern "C" size_t outputToStream(char* ptr, size_t size, size_t nmemb, void* userdata)
{
    std::iostream& stream = *(reinterpret_cast<std::iostream*>(userdata));
    stream.write(ptr, size * nmemb);
    return size * nmemb;
}
extern "C" size_t dropHeaders(char* ptr, size_t size, size_t nmemb, void* userdata)
{
    // Drop and ignore headers.
    // Otherwise they clutter the standard output.
    return size * nmemb;
}


namespace ThorsAnvil::Google
{

struct ServiceAccount
{
    std::string     client_id;
    std::string     project_id;
    std::string     auth_uri;
    std::string     token_uri;
    std::string     auth_provider_x509_cert_url;
    std::string     client_secret;
    std::vector<std::string>    redirect_uris;
};

struct ApplicaionInfo
{
    ServiceAccount  installed;
    ApplicaionInfo(std::istream& stream)
    {
        namespace TAS=ThorsAnvil::Serialize;
        stream >> TAS::jsonImporter(*this, TAS::ParserInterface::ParserConfig{TAS::JsonParser::ParseType::Strict, false});
    }

    std::string getManualAuthURL(std::string const& scope)
    {
        // Find URL
        std::size_t uriIndix = 0;
        for (;uriIndix < installed.redirect_uris.size(); ++uriIndix) {
            if (installed.redirect_uris[uriIndix].substr(0,4) == "urn:") {
                break;
            }
        }
        if (uriIndix >= installed.redirect_uris.size()) {
            throw std::runtime_error("Failed to find URN");
        }

        std::stringstream oauth2URLStream;
        oauth2URLStream << "https://accounts.google.com/o/oauth2/v2/auth"
                        << "?client_id=" <<  installed.client_id
                        << "&redirect_uri=" << installed.redirect_uris[uriIndix]
                        << "&response_type=code"
                        << "&scope=" << scope;

        return oauth2URLStream.str();
    }

    void createCredentialFile(std::string const& credFileName, std::string const& token)
    {
        namespace TAC=ThorsAnvil::Curl;
        namespace TAS=ThorsAnvil::Serialize;

        // Find URL
        std::size_t uriIndix = 0;
        for (;uriIndix < installed.redirect_uris.size(); ++uriIndix) {
            if (installed.redirect_uris[uriIndix].substr(0,4) == "urn:") {
                break;
            }
        }
        if (uriIndix >= installed.redirect_uris.size()) {
            throw std::runtime_error("Failed to find URN");
        }

        // Convert token into OAuth2 credentials
        // Part 1: Build Request
        std::stringstream requestBodyStream;
        requestBodyStream << "client_id=" + installed.client_id
                          << "&client_secret=" + installed.client_secret
                          << "&grant_type=authorization_code"
                          << "&redirect_uri=" + installed.redirect_uris[uriIndix]
                          << "&code=" + token;

        // Open a scope so we can create an output file stream to credStore.
        // Closed at the end of this scope.
        std::string     requestBody = requestBodyStream.str();
        std::fstream    credStore(credFileName, std::ios_base::out);

        // Part 2: Send request to OAuth server
        TAC::CurlUniquePtr          curl("https://oauth2.googleapis.com/token");
        TAC::CurlHeaderUniquePtr    headerList{{"Content-Type: application/x-www-form-urlencoded"}};

        curl_easy_setopt(curl, CURLOPT_POST, 1L);
        curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerList.get());
        curl_easy_setopt(curl, CURLOPT_POSTFIELDS, requestBody.c_str());
        curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, requestBody.size());
        curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, outputToStream);
        curl_easy_setopt(curl, CURLOPT_WRITEDATA, &static_cast<std::iostream&>(credStore));
        curl_easy_setopt(curl, CURLOPT_HEADERFUNCTION, dropHeaders);

        CURLcode code  = curl_easy_perform(curl);
        if (code != CURLE_OK) {
            throw std::runtime_error("Failed to get OAuth2 credentials");
        }
    }
};

struct OAuth2Creds
{
    std::string     access_token;
    std::size_t     expires_in;
    std::string     refresh_token;
    std::string     scope;
    std::string     token_type;

    OAuth2Creds(std::istream& stream)
    {
        namespace TAS=ThorsAnvil::Serialize;
        stream >> TAS::jsonImporter(*this, TAS::ParserInterface::ParserConfig{TAS::JsonParser::ParseType::Strict, false});
    }

    void refresh(ApplicaionInfo& application)
    {
        namespace TAC=ThorsAnvil::Curl;
        namespace TAS=ThorsAnvil::Serialize;

        TAC::CurlUniquePtr   refreshRequest("https://oauth2.googleapis.com//token");
        curl_easy_setopt(refreshRequest, CURLOPT_POST, 1L);

        std::stringstream bodyStream;
        bodyStream  << "client_id=" << application.installed.client_id
                    << "&client_secret=" << application.installed.client_secret
                    << "&refresh_token=" << refresh_token
                    << "&grant_type=refresh_token";
        std::string body(std::move(bodyStream.str()));
        curl_easy_setopt(refreshRequest, CURLOPT_POSTFIELDS, body.c_str());
        curl_easy_setopt(refreshRequest, CURLOPT_POSTFIELDSIZE, body.size());

        std::stringstream stream;
        curl_easy_setopt(refreshRequest, CURLOPT_WRITEFUNCTION, outputToStream);
        curl_easy_setopt(refreshRequest, CURLOPT_WRITEDATA, &static_cast<std::iostream&>(stream));


        CURLcode code  = curl_easy_perform(refreshRequest);
        if (code != CURLE_OK) {
            throw std::runtime_error("Failed to refresh OAuth2 credentials");
        }
};

}

ThorsAnvil_MakeTrait(ThorsAnvil::Google::OAuth2Creds, access_token, expires_in, refresh_token, scope, token_type);
ThorsAnvil_MakeTrait(ThorsAnvil::Google::ServiceAccount, client_id, project_id, auth_uri, token_uri, auth_provider_x509_cert_url, client_secret, redirect_uris);
ThorsAnvil_MakeTrait(ThorsAnvil::Google::ApplicaionInfo, installed);

#endif

GoogleMail.h

#ifndef THORSANVIL_GOOGLE_MAIL_H
#define THORSANVIL_GOOGLE_MAIL_H

#include "ThorSerialize/Traits.h"
#include "ThorSerialize/SerUtil.h"
#include "ThorSerialize/JsonThor.h"

#include <string>
#include <vector>

namespace ThorsAnvil::Google
{

struct Header
{
    std::string name;
    std::string value;
};

struct MessagePartBody
{
    std::string     attachmentId;
    std::size_t     size;
    std::string     data;
};

struct MessagePart
{
    using Headers = std::vector<Header>;
    using MessageParts = std::vector<MessagePart>;

    std::string     partId;
    std::string     mimeType;
    std::string     filename;
    Headers         headers;
    MessagePartBody body;
    MessageParts    parts;
};

struct Message
{
    using Labels = std::vector<std::string>;

    std::string     id;
    std::string     threadId;
    Labels          labelIds;
    std::string     snippet;
    MessagePart     payload;
    std::size_t     sizeEstimate;
    std::string     raw;
};

struct Draft
{
    std::string     id;
    Message         message;
};

}

ThorsAnvil_MakeTrait(ThorsAnvil::Google::Header, name, value);
ThorsAnvil_MakeTrait(ThorsAnvil::Google::MessagePartBody, attachmentId, size, data);
ThorsAnvil_MakeTrait(ThorsAnvil::Google::MessagePart, partId, mimeType, filename, headers, body, parts);
ThorsAnvil_MakeTrait(ThorsAnvil::Google::Message, id, threadId, labelIds, snippet, payload, sizeEstimate, raw);
ThorsAnvil_MakeTrait(ThorsAnvil::Google::Draft, id, message);

#endif

auth.cpp

#include "Credentials.h"
#include "CurlHelper.h"

#include <iostream>
#include <fstream>
#include <iterator>

namespace TAG=ThorsAnvil::Google;
namespace TAC=ThorsAnvil::Curl;

void printIntructions()
{
        std::cout   << R"Instructions(
First Run: You need to manually authorize with Google to get an OAuth2 token.

If you already have an Account/Application/OAuth2 tokens set up you can reuse them
These instructions assume you have not set up anything yet and are starting fresh
feel free to skip instructions that you have already completed before

Step 0: Set up a google developer account
        A: https://console.cloud.google.com/
        B: Sign in with your normal google credentials
Step 1: Create an Application in Google console
        A: Open: https://console.cloud.google.com/home/dashboard
        B: Click on the Burger top left
        C: From the DropDown Menu Select: 'IAM & Admin/Create a Project'
            C1: Name the App Click Create.
Step 2: Create OAuth2 Credential
        A: Click on the Burger top left
        B: From the DropDown Menu Select: 'APIs & Services/Credentials'
        C: Click the button '+ Create Credentials'
        D: From the DropDown Menu Select: 'OAuth client ID'
            D1: Make the 'Application Type' a 'Desktop app'
            D2: Enter the Application Name
            D2: Click 'Create'
            D3: This brings you back to the 'Credentials Screen'
            D4: Clock the 'Download Arrow (on the right) next to your OAuth2 Client
Step 3: Locate the Key File.
        A: Step 3 should have downloaded a file your computer.
        B: Locate this file and put it somewhere safe and note the absolute address of this file
        C: Enter the Full path of this file below when asked
Step 4: Enable API
        A: Click on the Burger top left
        B: From the DropDown Menu Select: 'APIs & Services/Library'
        C: Search for the API you want (gmail)
        D: Select the API. Then Click 'Enable'
Step 5: Decide the scope of you application
        A: Goto https://developers.google.com/identity/protocols/oauth2/scopes
        B: Locate the scope you want to give this application: (https://www.googleapis.com/auth/gmail.compose)
        C: Enter this value blow when asked
)Instructions";
}

void oauth2()
{
    std::ifstream   credStore("creds.json");
    if (!credStore) {
        printIntructions();

        // Get the Key File
        std::cout << "\n\nPlease enter path of key file\n";
        std::string keyFilePath;
        std::getline(std::cin, keyFilePath);

        std::ifstream keyFileStream(keyFilePath);
        TAG::ApplicaionInfo  keyFile(keyFileStream);

        // Get the Scope Info
        std::cout << "\n\nPlease enter the scope for the OAuth2 token\n";
        std::string scope;
        std::getline(std::cin, scope);

        // Request the user authorize the App.
        std::string oauth2URL = keyFile.getManualAuthURL(scope);
        std::cout   << "\n\nPlease authorize this application\n"
                    << "Open the following URL in a browser and follow the instructions.\n"
                    << "Once complete copy and paste the token into the string below.\n"
                    << "\n"
                    << "Open: " << oauth2URL << "\n";
        // Get Temp Token:
        std::cout << "\n\nPlease input token generated by OAuth2\n";
        std::string token;
        std::getline(std::cin, token);

        keyFile.createCredentialFile("creds.json", token);

        // Now that we have downloaded the creds.
        // Try to open the file again. So we can attempt to load it.
        credStore.open("creds.json");
        if (!credStore) {
            throw std::runtime_error("Failed to open creds.json");
        }
    }

    // If this loads without an exception we have some credentials.
    TAG::OAuth2Creds       oathCreds(credStore);
}

int main()
{
    try {
        oauth2();
    }
    catch(std::exception const& e) {
        std::cout << "Error: " << e.what() << "\n";
        throw;
    }
}

mail.cpp

#include "GoogleMail.h"
#include "Credentials.h"
#include "CurlHelper.h"


#include <iostream>
#include <fstream>
#include <iterator>

namespace TAC=ThorsAnvil::Curl;
namespace TAG=ThorsAnvil::Google;
namespace TAS=ThorsAnvil::Serialize;

extern "C" size_t headerCallback(char *ptr, size_t size, size_t nmemb, void *userdata)
{
    return size * nmemb;
}
extern "C" size_t writeCallback(char *ptr, size_t size, size_t nmemb, void *userdata)
{
    return size * nmemb;
}

void mail()
{
    using namespace std::string_literals;

    std::ifstream       keyFileStream("KeyFile.json"); // Same as the one used in auth
    TAG::ApplicaionInfo keyFile(keyFileStream);

    std::ifstream       credStream("creds.json");
    TAG::OAuth2Creds    oathCreds(credStream);

    oathCreds.refresh(keyFile);

    const std::string createDraft="https://gmail.googleapis.com/gmail/v1/users/me/drafts";

    TAG::Draft   draft;
    draft.message.snippet           = "Test";
    draft.message.payload.body.data = "VGhpcyBpcyBhIHRlc3QgZS1tYWls"; // base64 of "This is a test e-mail";

    std::stringstream bodyStream;
    bodyStream << TAS::jsonExporter(draft, TAS::PrinterInterface::OutputType::Stream);
    std::string body(std::move(bodyStream.str()));

    TAC::CurlUniquePtr       curl(createDraft);
    TAC::CurlHeaderUniquePtr headerList({"Content-Type: application/json; charset=UTF-8",
                                         "Authorization: "s + oathCreds.token_type + " " + oathCreds.access_token
                                        });

    curl_easy_setopt(curl, CURLOPT_POST, 1);
    curl_easy_setopt(curl, CURLOPT_POSTFIELDS, body.c_str());
    curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, body.size());
    curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerList.get());
    curl_easy_setopt(curl, CURLOPT_HEADERFUNCTION, headerCallback);
    curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writeCallback);

    CURLcode code  = curl_easy_perform(curl);
    std::cerr << "Code: " << code << "\n";
}
int main()
{
    try {
        mail();
    }
    catch(char const* msg) {
        std::cout << "Error: " << msg << "\n";
    }
}

Makefile

CXXFLAGS   += -std=c++17 -I/usr/local/include
CXXFLAGS   += -L/usr/local/lib -lcurl -lThorSerialize17 -lThorsLogging17

Building

ThorsSerializer and curl are needed for this code. They can both be retrieved with brew.

> brew install thors_serializer
> brew install curl

Brew is available for mac Linux and even Windows!

Then you can build with:

> make auth
> make mail

To use. Runt ./auth to create the credentials file (only need to do this once). Then you can run ./mail to create a draft e-mail. I'll add sending mail later.

Answer 1

Spelling:

• ApplicaionInfo -> ApplicationInfo
• uriIndix -> uriIndex

---

class CurlUniquePtr: public std::unique_ptr<CURL, CurlDeleter>

class CurlHeaderUniquePtr: public std::unique_ptr<curl_slist, CurlHDeleter>

A bit dodgy. We shouldn't really inherit from std::unique_ptr and we don't need its full interface. So we could just have a std::unique_ptr member variable inside the class.

Alternatively we could typedef the pointer type, and write the 3 functions we need as free functions, something like:

using CurlUniquePtr = std::unique_ptr<CURL, std::function<void(CURL*)>>;

CurlUniquePtr initCurl()
{
    auto curl = curl_easy_init();

    if (!curl)
        throw std::runtime_error("curl_easy_init() failed");
    
    return { curlUniquePtr, [] (CURL* c) { return curl_easy_cleanup(c); } };
}

using CurlSlistPtr = std::unique_ptr<curl_slist, std::function<void(curl_slist*)>>;

CurlSlistPtr curlSlistInit(std::initializer_list<std::string> list)
{
    auto slist = CurlSlistPtr();

    for (auto const& s : list)
        curlSlistAppend(slist, s);

    return slist;
}

void curlSlistAppend(CurlSlistPtr& slist, std::string const& s)
{
    auto added = curl_slist_append(slist.get(), s.c_str());

    if (!added)
        throw std::runtime_error("curl_slist_append() failed");
    
    slist.release();
    slist.reset(added);

    return slist;
}

Note: We should probably check the return value of curl_slist_append() instead of silently failing.

---

    // Find URL
    std::size_t uriIndix = 0;
    for (;uriIndix < installed.redirect_uris.size(); ++uriIndix) {
        if (installed.redirect_uris[uriIndix].substr(0,4) == "urn:") {
            break;
        }
    }
    if (uriIndix >= installed.redirect_uris.size()) {
        throw std::runtime_error("Failed to find URN");
    }

Finding the URN is repeated and could be abstracted into a function. Consider using std::find_if instead of a plain for-loop, e.g.:

std::vector<std::string>::const_iterator findUrn(std::vector<std::string> const& uris)
{
    return std::find_if(uris.begin(), uris.end(),
        [] (std::string const& uri) { return uri.starts_with("urn:"); });
}

...

    auto uri = findUrn(installed.redirect_uris);

    if (uri == uris.end())
        throw std::runtime_error("Failed to find URI");

---

curl_easy_setopt(...)

Might be worth checking the return values of all these calls (or wrapping it in a function).

---

extern "C" size_t outputToStream(char* ptr, size_t size, size_t nmemb, void* userdata)
{
    std::iostream& stream = *(reinterpret_cast<std::iostream*>(userdata));
    stream.write(ptr, size * nmemb);
    return size * nmemb;
}

static_cast should work fine here, no need to reinterpret_cast.

    curl_easy_setopt(curl, CURLOPT_WRITEDATA, &static_cast<std::iostream&>(credStore));

Not really fond of the manual cast to base pointer. But I guess wrapping curl_easy_setopt in a type-safe layer isn't something we want to do. (Then again, we're not using that much of the library, so it might be worth it).

It might be preferable to use a lambda for the write function instead of a free-function... but that seems to have its own complications.

---

Some comments re. Credentials.h and GoogleMail.h.

• It might be more flexible to separate fetching the credentials from the server from writing the "creds.json" file (even if we can do both at once with curl).

  (As a side note, we should probably have more error handling here - checking that the file is open, and the writes succeeded, etc.)

• Not fond of typedefs like using Headers = std::vector<Header>;. It obscures the type of the container.