this post was submitted on 11 Jul 2023
49 points (100.0% liked)
Technology
4 readers
1 users here now
Talk about anything tech related!
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Amazing! I can't wait to read about how they solved this. The devs were coming up with some very creative solutions the last I looked.
Great work to all the contributors!
Just phenomenal work; how could they even scrape data? When Musk restricts the API.
Don't need API access to scrape data. That's the whole point of scraping - you're indistinguishable from regular traffic. Nitter could even impersonate a Google search crawler if they want.
I could be wrong but when looking at the changed code, it still uses a token, specifically on this line. It's possibly using user routes instead of the actual paid API, but it's hard to tell without knowing twitter's API in depth. Also it's not possible to get everything just by web scraping, especially after they locked everything behind logins. AFAIK the most you can do is impersonate search crawling and get singular tweets or profile bios.