I'm working on a project that requires POS Tagging of paragraphs. The text contains lot of URLs which contain various punctuation marks such as .
?
. This affects the accuracy of the sentence tokenization.
So I decided to clean the data by removing/replacing all the URLs. And thought regular expressions would be handy in doing that. Remember, I want to match not just the domain name but the complete URL.
For example, in the below text,
Hey Mayur, We have successfully credited the cashback amount of INR 600 in your CITRUS wallet linked to mobile - 9130977755. Please download and rate our apps here - http://smarturl.it/ixigoapps Happy travelling! Team ixigo
I want the regex to match http://smarturl.it/ixigoapps
Here's what I came up with so far
https?:\/\/?[-a-zA-Z0-9@:%._\+~#=]{2,256}\.[a-z]{2,6}\b[-a-zA-Z0-9@:%_\+.~#?&=\/]*
Right now, I'm not concerned about protocols other than http
and https
, but I would like to match URLs without http
or https
but start with www
. Example: www.acttv.in
What do you think of my approach so far, and the regular expression?