@quark@ferengi.one I run the cron job every 15 minutes. Running it every minute imposes quite a bit of load on the feeds of other people. While jenny requests everything with the if-modified-since header set, not all servers honor that header. Most of all, yarn does not: https://git.mills.io/yarnsocial/yarn/issues/430

@movq@www.uninformativ.de What I would really like to see if jenny could use HTTP range requests to fetch only new content.

E.g. it could refetch only last twtext line of last request to make sure it starts off at correct position.

I guess there are twtxt files that only grow, then this will save a lot bandwidth over time.

For twtxt files that “forget” older content this situation would be detected and as a fallback the whole twtxt file could then be fetched.