Well, I'm working with Selenium to scrape a webpage that has obfuscated class names (example <div class="example-gHJkLM">...
)...
So I needed to create an extension method to get
public static ReadOnlyCollection<IWebElement> Descendants(this IWebElement element, bool hasChildren = false, int index = -1, string postAppend = "")
{
var web = element.GetWebDriver(); // < 1ms
var xpath = element.XPath(); // 5 ms
return web.FindElements(By.XPath(xpath + "//" + (hasChildren ? "child::node()" : "*") + (index > -1 ? $"[{index + 1}]" : string.Empty) + postAppend)); // 64 ms
}
public static ReadOnlyCollection<IWebElement> GetNodesByContainingClass(this IWebElement element, string className,
bool toLower = false, bool useFullHierarchy = false)
{
return new ReadOnlyCollection<IWebElement>(element?.Descendants()
.Where(n =>
(toLower ? n.GetAttribute("class")?.ToLowerInvariant() : n.GetAttribute("class"))
?.Contains(className) == true).ToList()); // ????
}
public static IWebElement GetNodeByContainingClass(this IWebElement node, string className)
{
return node?.GetNodesByContainingClass(className).FirstOrDefault();
}
Example usage:
var watch = Stopwatch.StartNew();
var welcomeMessage = messageBox
.GetNodesByContainingClass(
"icon-", false, true)?
.LastOrDefault();
watch.Stop();
Console.WriteLine($@"[Getting welcome message element] Ellapsed {watch.ElapsedMilliseconds} ms!", Color.Yellow);
- messageBox is a
IWebElement
- I'm using Colorful.Console package
And this is what I got:
4 and 9 seconds to materialize a Where LINQ clause, seriously? I debugged the element?.Descendants()
part and this has only 727 elements...
Can anyone guide me?
Also, I will be so thankfully if somebody suggests me a better approach to do this.
return new ReadOnlyCollection<IWebElement>(element?.Descendants().Where(n => n.GetAttribute("class").Equals(className, StringComparison.InvariantCultureIgnoreCase)));
check the performance, and report. \$\endgroup\$