1
\$\begingroup\$

Well, I'm working with Selenium to scrape a webpage that has obfuscated class names (example <div class="example-gHJkLM">...)...

So I needed to create an extension method to get

public static ReadOnlyCollection<IWebElement> Descendants(this IWebElement element, bool hasChildren = false, int index = -1, string postAppend = "")
{
    var web = element.GetWebDriver(); // < 1ms
    var xpath = element.XPath(); // 5 ms

    return web.FindElements(By.XPath(xpath + "//" + (hasChildren ? "child::node()" : "*") + (index > -1 ? $"[{index + 1}]" : string.Empty) + postAppend)); // 64 ms
}

public static ReadOnlyCollection<IWebElement> GetNodesByContainingClass(this IWebElement element, string className,
    bool toLower = false, bool useFullHierarchy = false)
{
    return new ReadOnlyCollection<IWebElement>(element?.Descendants()
        .Where(n =>
            (toLower ? n.GetAttribute("class")?.ToLowerInvariant() : n.GetAttribute("class"))
            ?.Contains(className) == true).ToList()); // ????
}

public static IWebElement GetNodeByContainingClass(this IWebElement node, string className)
{
    return node?.GetNodesByContainingClass(className).FirstOrDefault();
}

Example usage:

            var watch = Stopwatch.StartNew();

            var welcomeMessage = messageBox
                .GetNodesByContainingClass(
                    "icon-", false, true)?
                .LastOrDefault();

            watch.Stop();
            Console.WriteLine($@"[Getting welcome message element] Ellapsed {watch.ElapsedMilliseconds} ms!", Color.Yellow);
  • messageBox is a IWebElement
  • I'm using Colorful.Console package

And this is what I got:

...

4 and 9 seconds to materialize a Where LINQ clause, seriously? I debugged the element?.Descendants() part and this has only 727 elements...

Can anyone guide me?

Also, I will be so thankfully if somebody suggests me a better approach to do this.

\$\endgroup\$
2
  • \$\begingroup\$ for start, try this return new ReadOnlyCollection<IWebElement>(element?.Descendants().Where(n => n.GetAttribute("class").Equals(className, StringComparison.InvariantCultureIgnoreCase))); check the performance, and report. \$\endgroup\$
    – iSR5
    Jan 29, 2020 at 3:45
  • \$\begingroup\$ But I'm not looking for Equals comparison, I'm checking Contains. \$\endgroup\$
    – z3nth10n
    Jan 29, 2020 at 23:02

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.