Websites make available a lot of data, which scraping bots can download. These websites don't like you doing this, and thus started an arms race of better and better ways of detecting bots (and counteracting them)

BrightData lets you scrape websites easily by just adding a few lines of code to your project, and you can get $10 of free credit by signing up at brdta.com/abyteofcode