# Making an email text-parser generic using PredicateBuilder

I have some code that reads an email from a text file, parses the text, removes some lines (the 'cc' and 'to' lines) and then finds the line numbers of various phrases (e.g. "Regards", "Thanks") and then removes all the text for the lines after the first phrase found.

It works fine, but is not generic enough as I will have a variable number of phrases to look for to remove text. I want to use PredicateBuilder to add the OR conditions but I am not sure of the syntax to use.

I have attached a cut down version of the code below. The part with 'regards', 'thanks', 'please do not forward' is that part I want to use PredicateBuilder on.

foreach (List<string> list in groups)
{
IEnumerable<string> lines3 = list.Where(line => !line.StartsWith("Cc:", StringComparison.OrdinalIgnoreCase));
lines3 = lines3.Where(line => !line.StartsWith("To:", StringComparison.OrdinalIgnoreCase));

var result = lines3.Select((r, i) => new { line = r, index = i })
.Where(r =>
r.line.StartsWith("Regards", StringComparison.OrdinalIgnoreCase)
|| r.line.StartsWith("Thanks", StringComparison.OrdinalIgnoreCase)
))
.Select(r => new { LineNo = r.index }).OrderBy(r => r.LineNo);

foreach (var item in result)
{
lineNumber = item.LineNo;
if (lineNumber > 0)
emailText += String.Join(System.Environment.NewLine, lines3.Take(lineNumber - 1));
else
emailText += String.Join(System.Environment.NewLine, lines3);
break;
}
}

• Can you please post an example input/output to clarify the problem? Should everything after the first "regards" (or "thanks", etc) be cut off? – mjolka Jul 2 '14 at 12:53

First of all, you have one too many parentheses and are missing some variable declarations, so let's fix that up.

var emailText = "";
foreach (List<string> list in groups)
{
IEnumerable<string> lines3 = list.Where(line => !line.StartsWith("Cc:", StringComparison.OrdinalIgnoreCase));
lines3 = lines3.Where(line => !line.StartsWith("To:", StringComparison.OrdinalIgnoreCase));

var result = lines3.Select((r, i) => new { line = r, index = i })
.Where(r =>
r.line.StartsWith("Regards", StringComparison.OrdinalIgnoreCase)
|| r.line.StartsWith("Thanks", StringComparison.OrdinalIgnoreCase)
)
.Select(r => new { LineNo = r.index }).OrderBy(r => r.LineNo);

foreach (var item in result)
{
var lineNumber = item.LineNo;
if (lineNumber > 0)
emailText += String.Join(System.Environment.NewLine, lines3.Take(lineNumber - 1));
else
emailText += String.Join(System.Environment.NewLine, lines3);
break;
}
}


Now let's run this on

var groups = new List<List<string>>
{
new List<string> { "Hello", "To: me", "Message body", "Regards", "me" }
};


From your problem description, I would expect the output to be Hello\nMessage body, but it's just Hello. So I'm going to assume the problem description is correct and the code is wrong.

Here are some other notes:

• lines3 is not a good variable name.
• If you find yourself concatenating a lot of strings, see if you can use a StringBuilder instead.
• OrderBy line number is redundant, as the lines are already ordered by line number.

Let's start with a skeleton of what we want the code to look like:

var result = new StringBuilder();
foreach (List<string> email in emails)
{
// TODO: Remove "to" and "cc" lines.
// TODO: Remove everything after first "thanks", "regards", etc.
result.Append(string.Join(Environment.NewLine, ???));
}


Let's handle the easy part first.

private static bool IsDestination(string line)
{
return line.StartsWith("Cc:", StringComparison.OrdinalIgnoreCase) ||
line.StartsWith("To:", StringComparison.OrdinalIgnoreCase);
}

var result = new StringBuilder();
foreach (List<string> email in emails)
{
var filteredEmail = email.Where(line => !IsDestination(line));
// TODO: Remove everything after first "thanks", "regards", etc.
result.Append(string.Join(Environment.NewLine, filteredEmail));
}


Now you want a variable number of phrases to look for, and it seems that some will need to be at the start of the line, while others can be anywhere.

We can write the tests like this

var valedictionTests = new List<Predicate<string>>
{
line => line.StartsWith("Regards", StringComparison.OrdinalIgnoreCase),
line => line.StartsWith("Thanks", StringComparison.OrdinalIgnoreCase),
line => line.IndexOf("Please do not forward", StringComparison.OrdinalIgnoreCase) != -1
};


Then you can add tests like this

valedictionTests.Add(line => line.StartsWith("Best,", StringComparison.OrdinalIgnoreCase));


And pass them in as a parameter

var result = new StringBuilder();
foreach (List<string> email in emails)
{
var filteredEmail = email.Where(line => !IsDestination(line))
.TakeWhile(line => !valedictionTests.Any(test => test(line)));
result.Append(string.Join(Environment.NewLine, filteredEmail));
}