What is Web Data Extraction / Web Scraping?

We provide this brief introduction for those who are new to web data extraction or web scraping. If you are familiar with web data extraction then you may want to begin by reading the article Installing Sequentum Enterprise.

Web data extraction is the process of extracting data from websites and storing that data in a structured, easy-to-use format. The value of a web data extraction tool like Sequentum Enterprise is that you can easily specify and collect large amounts of source data that may be very dynamic (data that changes very frequently).

Usually, data available on the Internet has little or no structure and can only be viewed with a web browser. Elements such as text, images, video, and sound are built into a web page so that they are presentable in a web browser. It can be very tedious to manually capture and separate this data and can require many hours of effort to complete. With Sequentum Enterprise, this process can be automated to capture website data in a fraction of the time that it would take using other methods.

Web data extraction software interacts with websites in the same way you do when using your web browser. However, in addition to displaying the data in a browser on your screen, web data extraction software saves the data from the web page to a local file or database.

You can configure web data extraction agents to run on multiple websites, and you can schedule each agent to run automatically. It's easy to configure your agent to run as frequently as you like (hourly, daily, weekly, monthly) to ensure that you are capturing the very latest data.