I'm trying to learn F# by creating a little web scraper that will do custom scraping based on the url domain. For this, I need to create and select the correct kind of scraper. I figure I would use a Factory that would determine the correct kind of scraper for me.
So here is the interface for my scraper
type IExtractor = /// determines if the extractor will work for the url /// returns true if the extrator can handle the url abstract member Suitable : string -> bool /// name of the extractor abstract ExtractorName : string with get
Here is the instance of one of the scrapers
type CustomExtractor1() = /// <summary> /// tests whether the input url is valid /// </summary> /// <param name="url"></param> let (|ValidUrl|_|) url = let regexExpression = "^http(s)?://www\.customsite\.com.*\?fd=" // regex helper that returns string option Common.UrlMatch regexExpression url interface IExtractor with member this.Suitable url = match url with | ValidUrl url -> true | _ -> false member this.ExtractorName with get() = "Custom extractor"
Here is the Factory
module Extractor = let private _extractors: IExtractor list = [new CustomExtractor1(); new CustomExtractor2()] let create url = let isSuitable (input: string) (ex:IExtractor) = ex.Suitable input let one (input:string) (ex:IExtractor) = true match (_extractors |> List.tryFind (isSuitable url) ) with | Some ex -> Some ex | _ -> None
Here is the usage
let input = "https://www.customsite.com?fd=cWz5d" let extractor = Extractor.create input let extractorOption = match Extractor.create input with | Some ex -> Some ex | _ -> ( raise (Error ("no extractor found. Exiting")) ) let extractor = extractorOption.Value // do stuff with extractor printfn "%s" extractor.ExtractorName
The flow for this usage somehow feels more like C#, so it seems a little off. Especially in the usage where I'm using
extractorOption.Value. I am currently investigating using ROP instead of exception handling for better control flow.
What do you think — is there a way to improve this?