Follow

## Web Scraping obstacle: force close browser tab when Developer's Tools are open

On the web, webpages exist on a scale between static and dynamic. For each there are various techniques to get the desired data mainly consisting of `document.querySelectorAll()`.

Some website try to be sneaky and deactivate specific events like context menu or right-clicking. This functionality will usually be placed on media like photos or videos.

Manually going to the Developer's Tools in the browser and deactivating the undesired events is how to overcome this barricade.

There is a new level of annoyance: browser tab closes if Developer's Tools are open.

Now one needs to become creative on how to continue scraping. Here are two approaches, either download the site e.g. via `curl` (hopefully the user agent is not blocked and/or checks are not in place) or use a third-party extension to be a mocked Developer's Tool e.g. origin.

The ideal result is to locate the function that force closes the browser tab and either preload a script that overwrites it or temporally deactivate it.

Sign in to participate in the conversation
Qoto Mastodon

QOTO: Question Others to Teach Ourselves
An inclusive, Academic Freedom, instance
All cultures welcome.
Hate speech and harassment strictly forbidden.