I'm trying to learn F# by creating a little web scraper that will do custom scraping based on the url domain. For this, I need to create and select the correct kind of scraper. I figure I would use a Factory that would determine the correct kind of scraper for me.
So here is the interface for my scraper
type IExtractor =
/// determines if the extractor will work for the url
/// returns true if the extrator can handle the url
abstract member Suitable : string -> bool
/// name of the extractor
abstract ExtractorName : string with get
Here is the instance of one of the scrapers
type CustomExtractor1() =
/// <summary>
/// tests whether the input url is valid
/// </summary>
/// <param name="url"></param>
let (|ValidUrl|_|) url =
let regexExpression = "^http(s)?://www\.customsite\.com.*\?fd="
// regex helper that returns string option
Common.UrlMatch regexExpression url
interface IExtractor with
member this.Suitable url =
match url with
| ValidUrl url -> true
| _ -> false
member this.ExtractorName with get() = "Custom extractor"
Here is the Factory
module Extractor =
let private _extractors: IExtractor list = [new CustomExtractor1(); new CustomExtractor2()]
let create url =
let isSuitable (input: string) (ex:IExtractor) = ex.Suitable input
let one (input:string) (ex:IExtractor) = true
match (_extractors |> List.tryFind (isSuitable url) ) with
| Some ex -> Some ex
| _ -> None
Here is the usage
let input = "https://www.customsite.com?fd=cWz5d"
let extractor = Extractor.create input
let extractorOption =
match Extractor.create input with
| Some ex -> Some ex
| _ ->
(
raise (Error ("no extractor found. Exiting"))
)
let extractor = extractorOption.Value
// do stuff with extractor
printfn "%s" extractor.ExtractorName
The flow for this usage somehow feels more like C#, so it seems a little off. Especially in the usage where I'm using extractorOption.Value
. I am currently investigating using ROP instead of exception handling for better control flow.
What do you think — is there a way to improve this?