What is this?

It was early January 2024, and I was trying to understand how to decide on a piece of land purchase. I have plans in the near future to build a house for my family and me.

At that point, I had only one yearly report that mentioned the market was at its lowest point. This source did not provide a dataset; it only contained references to non-disclosed privately gathered data and some Excel charts. However, it was respectable enough to trust. Additionally, news articles suggested that this was the best moment to buy.

There is a fairly good (in my Grug eyes) rule of thumb that asserts that large public mentions of a market event signal the beginning of the next typical cyclical phase. In other words, popular opinions tend to lag behind true phenomena, such as:

• lowest low –> about to rise
• highest high –> about to drop
• growing fast –> just reached its highest high
• …

Bias

My professional bias led me to build a sensor, a thermometer to validate public opinion and confirm my rule of thumb, so I could time my purchase correctly. I started thinking about setting up a couple of scrapers on some popular and high-traffic real estate websites to monitor trends.

• I focused on one site in particular, let’s call it real-state-company-webapp.com
• stack: PHP + Angular
• I began exploring how to batch scrape properties
• The query ?items_per_page=1000 work