I'm working on a project that requires POS Tagging of paragraphs. The text contains lot of URLs which contain various punctuation marks such as
?. This affects the accuracy of the sentence tokenization.
So I decided to clean the data by removing/replacing all the URLs. And thought regular expressions would be handy in doing that. Remember, I want to match not just the domain name but the complete URL.
For example, in the below text,
Hey Mayur, We have successfully credited the cashback amount of INR 600 in your CITRUS wallet linked to mobile - 9130977755. Please download and rate our apps here - http://smarturl.it/ixigoapps Happy travelling! Team ixigo
I want the regex to match
Here's what I came up with so far
Right now, I'm not concerned about protocols other than
https, but I would like to match URLs without
https but start with
What do you think of my approach so far, and the regular expression?