Web scraping is the use of a program to simulate human interaction with a web server or to extract specific information from a web page.

Web scraping is the use of a program to simulate human interaction with a web server. A common goal is to extract specific information from a web server, especially from web pages whose contents are intended for human consumption rather than machine interpretation. It is usually considered to be a fragile technique, to be used as a last resort when no exists to accomplish the desired task.

Another use of web scraping is to make a series of HTTP requests for integration testing of a web application, as if a human were clicking on UI elements.

history | excerpt history