"The Billion Prices Project @ MIT
is an academic initiative that collects prices from hundreds of online retailers around the world on a daily basis to conduct economic research. We currently monitor daily price fluctuations of ~5 million items sold by ~300 online retailers in more than 70 countries.
BPP Database Key Facts
Statistics updated every day
5 million individual items
Started in October of 2007
Supermarkets, electronics, apparel, furniture, real estate, etc.
: Our data is collected every day from online retailers using a software that scans the underlying code in public webpages and stores the relevant price information in a database. The resulting dataset contains daily prices on the full population of products sold by individual retailers, allowing us to observe every single price change that takes place. Our data includes information on product descriptions, package sizes, brands, special characteristics (e.g. “organic”), and whether the item is on sale or price control.
Example: The chart above shows daily price indexes for the U.S. from July 2008 through November 15, 2010, comparing the BPP Index to the Consumer Price Index.
See WSJ article here on the BPP.