In-reply-to » Anyone know of a tool that will crawl a website, run JavaScript, and then save the resulting DOM as HTML?

@prologic@twtxt.net I’m trying to make a static local mirror of MDN Web Docs. It’s all free information on GitHub, but the whole system is extremely complicated.

<​tinfoil-hat>I think it’s so they can sell more MDN plus subscriptions, making people use their terrible MDN Offline system that uses the local storage of your browser.<​/tinfoil-hat>

At this point, I’m willing to run a local dev server and just save each generated page and its dependencies.

I really only need it to run JavaScript so it can request the browser compatibility JSON. It’s https://github.com/mdn/browser-compat-data but the MDN server, annoyingly, transforms it.

Once the BCD is rendered statically, I should be able to remove the references to the JavaScript.

That will solve another issue I’m having where the JavaScript is constantly trying to download /api/v1/whoami, which seemingly has no purpose aside from user tracking.

⤋ Read More