# Replacing tokens in a text file with replaced text

This code parses a text file, replacing special tokens marked as $(token) with some replaced text. The$ symbol can be escaped by entering a double $$. In the example a user enters a text file containing tokens to replace and a separate mapping file with, for instance, colour:red to denote how to replace a token. Some remarks I have is that is is not using any OO really. Does it need to be? #include <fstream> #include <iostream> #include <vector> #include <map> #include <string> static void readToken(std::istream& strm, std::vector<char>& token) { while(strm) { int c (strm.get()); token.push_back(c); if(')' == c) break; } } static void lookup_replacement(const std::map<std::string,std::string>& token_map, const std::vector<char>& key, std::vector<char>& value) { if(key.size() > 3) { //3 chars min are '', '(' and ')' std::map<std::string,std::string>::const_iterator it = token_map.find(std::string(key.begin()+2, key.end()-1)); if(it != token_map.end()) value.insert(value.end(), it->second.begin(), it->second.end()); } } static bool readTo(std::istream& strm, std::vector<char>& buf, const char term[]) { bool ret(false); while(strm) { int c (strm.peek()); if(strchr(term, c)) { strm.get(); int c2(strm.peek()); if(c != c2) { strm.unget(); ret = true; break; } } strm.get(); buf.push_back(c); } return ret; } static void create_mapping(std::ifstream& strm, std::map<std::string,std::string>& tokens) { if(strm.good()) { std::string line; while(std::getline(strm, line)) { std::string::size_type pos = line.find_first_of(':'); if(pos != std::string::npos) tokens.insert(std::make_pair<std::string, std::string>(line.substr(0, pos), line.substr(pos+1))); } } } int main() { std::map<std::string,std::string> token_map; typedef std::map<std::string,std::string>::iterator map_iter; std::cout << "Enter path to text file to replace\nTokens to be replaced should be of form: (token)\n"; std::string file; std::cin >> file; std::cout << "Enter path to file with target token <-> replacement token in format:\n" "<toreplace1>:<replacement1>\n<toreplace2>:<replacement2>\netc\n"; std::string mapfile; std::cin >> mapfile; std::ifstream strmmap(mapfile.c_str()); create_mapping(strmmap, token_map); // replaced text std::vector<char> replaced; std::ifstream strm(file.c_str()); if(!strm.bad()) { while(!strm.eof()) { if(readTo(strm, replaced, "")) { std::vector<char> token; readToken(strm, token); std::vector<char> replace_tok; lookup_replacement(token_map, token, replace_tok); if(!replace_tok.empty()) replaced.insert(replaced.end(), replace_tok.begin(), replace_tok.end()); } } } //print out replaced text std::cout << "your replaced text file\n"; std::vector<char>::iterator pit = replaced.begin(); while(pit != replaced.end()) std::cout << *pit++; return 0; }  The example input file I was using: Once upon a (place1) there was a (adjective1) (colour) (animal). Do you have a$$

You have enemies? Good. That means you've stood up for something, sometime in your life. by $(author) on$(date)

A fanatic is one who can't change his mind and won't change the subject. By $(author) And the mapping file: place1:time adjective1:cunning colour:brown animal:fox author:Winston Churchill date:06/06/2013  • Why are you using vector<char> instead of string for the token etc? – William Morris Jun 8 '13 at 0:39 • So what happens with $X? You replacement thing is $(<X>). So the questions are: 1) What is $() 2) what is $<Y> (Y => any character not '('). 3) What happens when the replacement is not terminated $(<Z>\n – Martin York Jun 8 '13 at 4:49

Its a little heavy on the code side.

Your definition of a token still leaves possibilities that you have not defined.

          :  Replace with $Fine$(<X>)      :  Replace with the mapped <X>                          Fine.

$(<stuff>\n : No closing ')' Error$()         : What happens with no identifier.                        Error
$<X> : Some other character after '$' that is not '(' or '$' Error  Especially your stream handling. Its usually bad practice to test for .good(), .bad() or .eof() during normal processing. You want to test these after things go wrong to generate the appropriate error message or not. Usually stream code looks like this: while(std::getline(stream, line)) { // correctly read a line } // or while(stream >> value >> value2 >> value3 >> etc) { // correctly read a value from the stream }  You can use OO to compartmentalize your mapping. Personally I would combine your mapper and mapped class into a single class. class Mapper { // Class contains a map from tokens => value // used via replace() to replace all tokens$(<X>) in string
// with the values contained in the map.

std::map<std::string, std::string>  replaceMap;
public:
Mapper(std::string const& fileName)
{
// Read all the mapping values from the file.

std::ifstream   mapFile(fileName.c_str());
std::string     line;

while(std::getline(mapFile, line))
{
// Each line in the file contains a token/value mapping
std::stringstream   linestream(line);

std::string         token;
std::string         value;

std::getline(linestream, token, ':');
std::getline(linestream, value);

// Save the token and value
replaceMap[token]  = value;
}
}

std::string replace(std::string const& line)
{
std::string result;
std::size_t last = 0;

// Search for the '$' char repeatedly. for(std::size_t find = line.find('$');find != std::string::npos;last=find, find=line.find('$', find)) { // Copy the inert text from line // into the result result += line.substr(last, (find-last)); // If the next character is a '(' then search for the closing ')' // // If we don't find a token $(<X>) then ignore the '$' and treat // it like a normal character. Note If <X> has no mapping then // we replace it with the empty string. Note if <X> is empty it is // is still value. bool hit = false if ((find + 1 < line.size()) && line[find + 1] == '(') { std::size_t end = line.find(')', find); if (end != std::string::npos) { std::string key = line.substr(find+2, end-find-2); result += replaceMap[key]; find = end + 1; hit = true; } } if (!hit) { // Token was not found. // Put the '$' on the output and move on.
result  += '\$';
find++;
}
}
// Add the rest of the inert string to the output
result  += line.substr(last);

return result;
}
};


This makes it easy to use:

int main(int argc, char* argv[])
{
Mapper  mapper(argv[1]);

std::ifstream   input(argv[2]);
std::string     line;
while(std::getline(input, line))
{
std::string result = mapper.replace(line);
std::cout << result << "\n";
}
}


This looks pretty good as it is. No project strictly "needs" OOP, especially one of this size. But here is one way to frame it in OOP -

Create a Mapped class and a Mapper class, both accepting stream references in their constructors. Perhaps constructor overloads accepting filenames. Mapped.Map could accept a Mapper instance reference, and use it to do the string replacement. When both objects are destroyed, both streams are closed.