Monitoring of the Price of Sawn Wood Through Web Scraping Techniques in Chile

Instituto Forestal, Santiago, Chile

Abstract: The purpose of this study is to present a methodology applied to monitor and collect information on prices of forest products in the Chilean market, available on company websites. Web Scraping corresponds to a technique that, through an automated process, allows selected documents and data to be downloaded from the web, and then transformed and saved in a structured format. Through this process, it is possible to monitor the evolution of the public prices of products offered in the market through the web. In this research, the automated process was implemented using the Selenium library of the Python programming language, with which a series of URLs of different forest product trading companies are accessed. Once a product catalog of company was obtained, products were filtered based on page categories or keywords related to target product types. Once the products were thus identified, the information on their price, brand, and description was captured. This information was stored in individual variables, which were purged by removing unnecessary characters, to later be consolidated in a data frame. This process was carried out for each of the companies considered to obtain prices, to later combine all the information in a CVS file that corresponds to the output of the program. Monthly price information was collected, with which price series associated with the products were built: Impregnated sawn wood, planed sawn wood, dimensioned dry sawn wood, and dimensioned sawn wood, considering the 2''4'' and 2''6' squares. The Web Scraping application helps to collect supply information, generating inputs that can be used as a basis for new studies.