Web Scraping

“Once a new technology rolls over you, if you're not part of the steamroller, you're part of the road.” ~Stewart Brand

A recent survey showed that the average person spends 12 to 15 hours each week on the web. During this time, they will visit more than 25 websites and spend a chunk of it on Facebook.

So much of what drives a purchasing decision on the web is what people read about a product or service from other consumers. If I read a bad review about a hotel that I was considering, clearly, I will not stay there. On the other hand, if I see great reviews about a new tech gadget, I will seriously consider purchasing it.

The more you know about what your customers are reading and what is influencing their decisions, the greater your success. This was not possible in the past. Now, however, new web-scraping software mines data on the web, automatically extracting valuable information for analysis.

One of the leaders in this field is Mozenda, which offers a neat web-scraping product at very reasonable prices. Rates start at $99 to scrape 5,000 pages or images of websites a month. There are also free web scraping programs at www.freedownloadmanager.org/downloads/web_scraping_software/.

Businesses can use this software to monitor blogs, chat rooms and social networking sites to keep tabs on what people are saying about their products. With this technology, so much more information can be learned before a problem develops, enabling a more timely response.

If Apple had been monitoring blogs, it would have known the extent of the iPhone 4's problems (dropped calls) and seen how dissatisfaction was escalating. The company could have responded more quickly to the growing negative remarks.

There is an overabundance of information available online. With astronomical amounts of data swirling around out there, shoppers rely more and more on the information provided by other consumers to make decisions. Knowing this, many firms let customers comment on their products and services right there on their websites.

Take QVC and HSN, for example. On their sites, consumers can comment on each item sold. When a product does incredibly well in consumer input, they frequently mention this on the air.

Web scraping analysis should be part of each firm's marketing budget because it will become an even more dominant marketing tool. Now go out and see if you can apply web-scraping software to your business in a cost-effective way.

You can do this!