dear websites that don’t load without javascript

:blobcatpuffyfux: fuck you

@icedquinn web apps should use javascript; that's what it is for, and it is a decent language. But the VAST majority of websites are not web apps, and shouldn't try to be.

@worldsendless it's just a forums. but you can't even view a page without javascript. which means now i gotta wrangle selenium or puppeteer which is obnoxious
Follow

@icedquinn sounds like you are trying to scrape stuff, if you need automation tools?

@worldsendless ye but now the selenium driver doesn't work

HTTP/1.1 405 Method Not Allowed

:blobcattableflip:
@worldsendless nah its issues getting the webdrivers to work. i guess the julia lib i found doesn't quite work
@worldsendless i did manage to get the stack of julia->selenium->webdriver->browser to work and download a single thread page, and rip the posts. there's annoying "click to expand" parts too and some scrubbing. i got tired of working on it.

i maybe just want to read a 200 page thread on an ebook reader y'know :blobcatadorable:

@icedquinn Oh, I understand that. So much learning that probably will have 0 value after this project... that's web scraping for you.

@icedquinn How is Julia? That's one of those languages where I wouldn't even recognize its code of it was before me. Still the best for numerics, or are Go or Rust in the running? Or possibly even Python?

Sign in to participate in the conversation
Qoto Mastodon

QOTO: Question Others to Teach Ourselves
An inclusive, Academic Freedom, instance
All cultures welcome.
Hate speech and harassment strictly forbidden.