@prologic@twtxt.net what time in UTC?
@prologic@twtxt.net cool, I will be there! Are you going to post the regular banner notice? It will serve as a reminder, at least for me.
@lyse@lyse.isobeef.org I don’t think this is true.
@david@collantes.us Well, I wouldn’t recommend using my code for your main jenny use anyway. If you want to try it out, set XDG_CONFIG_HOME and XDG_CACHE_HOME to some sandbox directories and only run my code there. If @movq@www.uninformativ.de is interested in any of this getting upstreamed, I’d be happy to try rebasing the changes, but otherwise it’s a proof of concept and fun exercise.
@falsifian@www.falsifian.org hiya from the Sunshine State (also known as a never-ending hell, LOL)!
@falsifian@www.falsifian.org this one hits hard, as jenny was just updated today. :‘-(
@david@collantes.us Hello!
@prologic@twtxt.net Hi. i have noticed sometimes when i hit the back button i lose all the surrounding layout and just have a list of twts.

@prologic@twtxt.net Hi. i have noticed sometimes when i hit the back button i lose all the surrounding layout and just have a list of twts.

I wrote some code to try out non-hash reply subjects formatted as (replyto ), while keeping the ability to use the existing hash style.
I don’t think we need to decide all at once. If clients add support for a new method then people can use it if they like. The downside of course is that this costs developer time, so I decided to invest a few hours of my own time into a proof of concept.
With apologies to @movq@www.uninformativ.de for corrupting jenny’s beautiful code. I don’t write this expecting you to incorporate the patch, because it does complicate things and might not be a direction you want to go in. But if you like any part of this approach feel free to use bits of it; I release the patch under jenny’s current LICENCE.
Supporting both kinds of reply in jenny was complicated because each email can only have one Message-Id, and because it’s possible the target twt will not be seen until after the twt referencing it. The following patch uses an sqlite database to keep track of known (url, timestamp) pairs, as well as a separate table of (url, timestamp) pairs that haven’t been seen yet but are wanted. When one of those “wanted” twts is finally seen, the mail file gets rewritten to include the appropriate In-Reply-To header.
Patch based on jenny commit 73a5ea81.
https://www.falsifian.org/a/oDtr/patch0.txt
Not implemented:
- Composing twts using the (replyto …) format.
- Probably other important things I’m forgetting.
@lyse@lyse.isobeef.org indeed! There is no “central authority” acting as witness, and notary. The more I think of it… LOL.
@movq@www.uninformativ.de I recognise him, but yes, he has aged quite a bit. I mean, I look at myself in the mirror and can’t, often, recognise myself. Ageing is a bitch! 😅
compressed_subject(msg_singlelined) be configurable, so only a certain number of characters get displayed, ending on ellipses? Right now the entire twtxt is crammed into the Subject:. This request aims to make twtxts display on mutt/neomutt, etc. more like emails do.
I mean, really, it couldn’t get any better. I love it!
compressed_subject(msg_singlelined) be configurable, so only a certain number of characters get displayed, ending on ellipses? Right now the entire twtxt is crammed into the Subject:. This request aims to make twtxts display on mutt/neomutt, etc. more like emails do.
@movq@www.uninformativ.de perfect in every way. Configurable too! Thank you!
compressed_subject(msg_singlelined) be configurable, so only a certain number of characters get displayed, ending on ellipses? Right now the entire twtxt is crammed into the Subject:. This request aims to make twtxts display on mutt/neomutt, etc. more like emails do.
@movq@www.uninformativ.de yes, that’s perfect! <3
@eldersnake@we.loveprivacy.club I wanted to ask you, are you running Headscale and WireGuard on the same VPS? I want to test Headscale, but currently run a small container with WireGuard, and I wonder if I need to stop (and eventually get rid of) the container to get Headscale going. Did you use the provided .deb to install Headscale, or some other method?
@prologic@twtxt.net I read it. I understand it. Hopefully a solution can be agreed upon that solves the editing issue, whilst maintaining the cryptographic hash.
Can I get someone like maybe @xuu@txt.sour.is or @abucci@anthony.buc.ci or even @eldersnake@we.loveprivacy.club – If you have some spare time – to test this yarnd PR that upgrades the Bitcask dependency for its internal database to v2? 🙏
VERY IMPORTANT If you do; Please Please Please backup your yarn.db database first! 😅 Heaven knows I don’t want to be responsible for fucking up a production database here or there 🤣
@prologic@twtxt.net I know the role of the current hash is to allow referencing (replies and, thus, threads), and it also represents a “unique” way to verify a twtxt hasn’t been tampered with. Is that second so important, if we are trying to allow edits? I know if feels good to be able to verify, but in reality, how often one does it?
@movq@www.uninformativ.de could it be possible to have compressed_subject(msg_singlelined) be configurable, so only a certain number of characters get displayed, ending on ellipses? Right now the entire twtxt is crammed into the Subject:. This request aims to make twtxts display on mutt/neomutt, etc. more like emails do.
@prologic@twtxt.net I don’t trust Google with anything, sorry, pass. Oh, and you need to sign in on your Google Account (or whatever they call it these days).
@prologic@twtxt.net how about hashing a combination of nick/timestamp, or url/timestamp only, and not the twtxt content? On edit those will not change, so no breaking of threads. I know, I know, just adding noise here. :-P
@eldersnake@we.loveprivacy.club there has to be less reliance on a single point of failure. It is not so much about creating jobs in the US (which come with it, anyway), but about the ability to produce what’s needed at home too. What’s the trade off? Is it going to be a little bit more expensive to manufacture, perhaps?
@prologic@twtxt.net That’s definitely a little less depressing, when thinking of it that way 🤣 Be interesting when the hype dies down.
@quark@ferengi.one It does not. That is why I’m advocating for not using hashes for treads, but a simpler link-back scheme.
@prologic@twtxt.net where was that idea?
@prologic@twtxt.net where was that idea?
@prologic@twtxt.net the basic idea was to stem the hash.. so you have a hash abcdef0123456789... any sub string of that hash after the first 6 will match. so abcdef, abcdef012, abcdef0123456 all match the same. on the case of a collision i think we decided on matching the newest since we archive off older threads anyway. the third rule was about growing the minimum hash size after some threshold of collisions were detected.
@prologic@twtxt.net the basic idea was to stem the hash.. so you have a hash abcdef0123456789... any sub string of that hash after the first 6 will match. so abcdef, abcdef012, abcdef0123456 all match the same. on the case of a collision i think we decided on matching the newest since we archive off older threads anyway. the third rule was about growing the minimum hash size after some threshold of collisions were detected.
@prologic@twtxt.net Wikipedia claims sha1 is vulnerable to a “chosen-prefix attack”, which I gather means I can write any two twts I like, and then cause them to have the exact same sha1 hash by appending something. I guess a twt ending in random junk might look suspcious, but perhaps the junk could be worked into an image URL like
. If that’s not possible now maybe it will be later.git only uses sha1 because they’re stuck with it: migrating is very hard. There was an effort to move git to sha256 but I don’t know its status. I think there is progress being made with Game Of Trees, a git clone that uses the same on-disk format.
I can’t imagine any benefit to using sha1, except that maybe some very old software might support sha1 but not sha256.
@prologic@twtxt.net :-D Thanks! Things can come in cycles, right? This is simply another one. Another cycle, more personal than the other “alter egos”.
@aelaraji@aelaraji.com hey, hey! You are my very first reply! 👋🏻 Cheers!
Incredibly upset—more than you could imagine—because I already made the first mistake, and corrected it (but twtxt.net got it on it’s cache, ugh!) :‘-( . Can’t wait for editing to become a reality!
Alright. My first mentions—which were picked not so randomly, LOL—are @prologic@twtxt.net, @lyse@lyse.isobeef.org, and @movq@www.uninformativ.de. I am also posting my first image too, which you see below. That’s my neighbourhood, in a “winter” day. Hopefully @prologic@twtxt.net will add my domain to his allowed list, so that the image (and any other further) renders.
Alright, announce_me set to true. Now, who do I pick to be my first mention? Decisions, decisions. Next twtxt will have my first mention(s). :-)
I have configured my twtxt.txt as simple as possible. I have setup a publish_command on jenny. Hopefully all works fine, and I am good to go. Next will be setting the announce_me to true. Here we go!
@sorenpeter@darch.dk hmm, how does your client handles “a little editing”? I am sure threads would break just as well. 😉
@prologic@twtxt.net, there is a parser bug on parent. Specifically on this portion:
"*If twtxt/Yarn was to grow bigger, then this would become a concern again. *But even Mastodon allows editing*, so how
+much of a problem can it really be? 😅*"
@movq@www.uninformativ.de going a little sideways on this, “*If twtxt/Yarn was to grow bigger, then this would become a concern again. But even Mastodon allows editing, so how much of a problem can it really be? 😅*”, wouldn’t it preparing for a potential (even if very, very, veeeeery remote) growth be a good thing? Mastodon signs all messages, keeps a history of edits, and it doesn’t break threads. It isn’t a problem there.😉 It is here.
I think keeping hashes is a must. If anything for that “feels good” feeling.
@movq@www.uninformativ.de Agreed that hashes have a benefit. I came up with a similar example where when I twted about an 11-character hash collision. Perhaps hashes could be made optional somehow. Like, you could use the “replyto” idea and then additionally put a hash somewhere if you want to lock in which version of the twt you are replying to.
I’m not advocating in either direction, btw. I haven’t made up my mind yet. 😅 Just braindumping here.
The (replyto:…) proposal is definitely more in the spirit of twtxt, I’d say. It’s much simpler, anyone can use it even with the simplest tools, no need for any client code. That is certainly a great property, if you ask me, and it’s things like that that brought me to twtxt in the first place.
I’d also say that in our tiny little community, message integrity simply doesn’t matter. Signed feeds don’t matter. I signed my feed for a while using GPG, someone else did the same, but in the end, nobody cares. The community is so tiny, there’s enough “implicit trust” or whatever you want to call it.
If twtxt/Yarn was to grow bigger, then this would become a concern again. But even Mastodon allows editing, so how much of a problem can it really be? 😅
I do have to “admit”, though, that hashes feel better. It feels good to know that we can clearly identify a certain twt. It feels more correct and stable.
Hm.
I suspect that the (replyto:…) proposal would work just as well in practice.
Hey, @movq@www.uninformativ.de, a tiny thing to add to jenny, a -v switch. That way when you twtxt “That’s an older format that was used before jenny version v23.04”, I can go and run jenny -v, and “duh!” myself on the way to a git pull. :-D
@movq@www.uninformativ.de ooooh, nice! commit 62a2b7735749f2ff3c9306dd984ad28f853595c5:
Crawl archived feeds in –fetch-context
Like, very much! :-)
@movq@www.uninformativ.de to paraphrase US Presidents speech on each State of the Union, “the State of the Jenny is strong!” :-D As for the potential upcoming changes, there has to be a knowledgeable head honcho that will agglomerate and coalesce, and guide onto the direction that will be taken. All that with the strong input from the developers that will be implementing the changes, and a lesser (but not less valuable) input from users.
@lyse@lyse.isobeef.org I call upon the services of the @yarn_police@twtxt.net to further investigate this oddness!
@quark@ferengi.one Oh, sure, it would be nice if edits didn’t break threads. I was just pondering the circumstances under which I get annoyed about data being irrecoverably deleted or otherwise lost.
@falsifian@www.falsifian.org Yeah, delete requests feel very odd.
@falsifian@www.falsifian.org “I don’t really mind if the twt gets edited before I even fetch it.”, right, that’s never the problem. Editing a twtxt before anyone fetches it isn’t even editing, right? :-P The problem we are trying to fix is the havoc is causes editing twtxts that have already been replied to, often ad nauseam. That’s the real problem.
@quark@ferengi.one I don’t really mind if the twt gets edited before I even fetch it. I think it’s the idea of my computer discarding old versions it’s fetched, especially if it’s shown them to me, that bugs me.
But I do like @movq@www.uninformativ.de’s suggestion on this thread that feeds could contain both the original and the edited twt. I guess it would be up to the author.
@lyse@lyse.isobeef.org now, how am I not surprised at that reply?! Hahahahaha!
@falsifian@www.falsifian.org that would be problematic to do on a fully decentralised system. I am not disagreeing, though. That’s the reason I have stopped editing twtxts. I strive to own mistakes, as minor as they might be. Now, if trail editing can be accomplished, I am all for it!
@quark@ferengi.one None. I like being able to see edit history for the same reason.
@falsifian@www.falsifian.org what would the difference be between an edit the changes everything on the original twtxt, and a delete?
@prologic@twtxt.net Why sha1 in particular? There are known attacks on it. sha256 seems pretty widely supported if you’re worried about support.
@prologic@twtxt.net I wouldn’t want my client to honour delete requests. I like my computer’s memory to be better than mine, not worse, so it would bug me if I remember seeing something and my computer can’t find it.
There’s a simple reason all the current hashes end in a or q: the hash is 256 bits, the base32 encoding chops that into groups of 5 bits, and 256 isn’t divisible by 5. The last character of the base32 encoding just has that left-over single bit (256 mod 5 = 1).
So I agree with #3 below, but do you have a source for #1, #2 or #4? I would expect any lack of variability in any part of a hash function’s output would make it more vulnerable to attacks, so designers of hash functions would want to make the whole output vary as much as possible.
Other than the divisible-by-5 thing, my current intuition is it doesn’t matter what part you take.
Hash Structure: Hashes are typically designed so that their outputs have specific statistical properties. The first few characters often have more entropy or variability, meaning they are less likely to have patterns. The last characters may not maintain this randomness, especially if the encoding method has a tendency to produce less varied endings.
Collision Resistance: When using hashes, the goal is to minimize the risk of collisions (different inputs producing the same output). By using the first few characters, you leverage the full distribution of the hash. The last characters may not distribute in the same way, potentially increasing the likelihood of collisions.
Encoding Characteristics: Base32 encoding has a specific structure and padding that might influence the last characters more than the first. If the data being hashed is similar, the last characters may be more similar across different hashes.
Use Cases: In many applications (like generating unique identifiers), the beginning of the hash is often the most informative and varied. Relying on the end might reduce the uniqueness of generated identifiers, especially if a prefix has a specific context or meaning.
@aelaraji@aelaraji.com odd, I ran it under Ubuntu 24.04, and got the same result as @prologic@twtxt.net (which is on macOS), zq4fgq.
@prologic@twtxt.net I just realised the jenny also does what I want, as of latest commit. Simply use jenny --debug-feed <feed url>, and it will do what I wanted too!
@movq@www.uninformativ.de alright, fair, and interesting. I was expecting them to be all the same (format wise), but it doesn’t matter, for sure, as it works just fine. Thanks!
I have noticed that twtxt timestamps differ. For example:
- @prologic@twtxt.net (and I assume any Yarn user)
2024-09-18T13:16:17Z
- @lyse@lyse.isobeef.org
2024-09-17T21:15:00+02:00
- @aelaraji@aelaraji.com (and @movq@www.uninformativ.de, and me)
2024-09-18T05:43:13+00:00
So, which is right, or best?
I came across this Gallery Theme for Hugo, and @lyse@lyse.isobeef.org immediately came to mind. I think it would be a very fitting theme to use for all your photos, Lyse!
An alternate idea for supporting (properly) Twt Edits is to denoate as such and extend the meaning of a Twt Subject (which would need to be called something better?); For example, let’s say I produced the following Twt:
2024-09-18T23:08:00+10:00 Hllo World
And my feed’s URI is https://example.com/twtxt.txt. The hash for this Twt is therefore 229d24612a2:
$ echo -n "https://example.com/twtxt.txt\n2024-09-18T23:08:00+10:00\nHllo World" | sha1sum | head -c 11
229d24612a2
You wish to correct your mistake, so you make an amendment to that Twt like so:
2024-09-18T23:10:43+10:00 (edit:#229d24612a2) Hello World
Which would then have a new Twt hash value of 026d77e03fa:
$ echo -n "https://example.com/twtxt.txt\n2024-09-18T23:10:43+10:00\nHello World" | sha1sum | head -c 11
026d77e03fa
Clients would then take this edit:#229d24612a2 to mean, this Twt is an edit of 229d24612a2 and should be replaced in the client’s cache, or indicated as such to the user that this is the intended content.
@quark@ferengi.one My money is on a SHA1SUM hash encoding to keep things much simpler:
$ echo -n "https://twtxt.net/user/prologic/twtxt.txt\n2020-07-18T12:39:52Z\nHello World! 😊" | sha1sum | head -c 11
87fd9b0ae4e
@prologic@twtxt.net the real conclusion is, is it going to change, to what, and when? :-P
@prologic@twtxt.net yes, that would work, except there is no debug command on my local yarnc. Are you talking about a potential future implementation here?
@quark@ferengi.one Do you mean something like this?
$ ./yarnc debug ~/Public/twtxt.txt | tail -n 1
kp4zitq 2024-09-08T02:08:45Z (#wsdbfna) @<aelaraji https://aelaraji.com/twtxt.txt> My work has this thing called "compressed work", where you can **buy** extra time off (_as much as 4 additional weeks_) per year. It comes out of your pay though, so it's not exactly a 4-day work week but it could be useful, just haven't tired it yet as I'm not entirely sure how it'll affect my net pay
@prologic@twtxt.net I saw those, yes. I tried using yarnc, and it would work for a simple twtxt. Now, for a more convoluted one it truly becomes a nightmare using that tool for the job. I know there are talks about changing this hash, so this might be a moot point right now, but it would be nice to have a tool that:
- Would calculate the hash of a twtxt in a file.
- Would calculate all hashes on a
twtxt.txt(local and remote).
Again, something lovely to have after any looming changes occur.
@aelaraji@aelaraji.com Woah! Overkill, but nicely laid out. Hey, the ultimate goal is for it to work, so, mission accomplished! :-)
Could someone knowledgable reply with the steps a grandpa will take to calculate the hash of a twtxt from the CLI, using out-of-the-box tools? I swear I read about it somewhere, but can’t find it.
@quark@ferengi.one Mine is a little overkill 😂 but I need to do something for practice:
#!/bin/bash
set -e
trap 'echo "!! Something went wrong...!!"' ERR
#============= Variables ==========#
# Source files
LOCAL_DIR=$HOME/twtxt
TWTXT=$LOCAL_DIR/twtxt.txt
HTML=$LOCAL_DIR/log.html
TEMPLATE=$LOCAL_DIR/template.tmpl
# Destination
REMOTE_HOST=remotHostName # Host already setup in ~/.ssh/config
WEB_DIR="path/to/html/content"
GOPHER_DIR="path/to/phlog/content"
GEMINI_DIR="path/to/gemini-capsule/content"
DIST_DIRS=("$WEB_DIR" "$GOPHER_DIR" "$GEMINI_DIR")
#============ Functions ===========#
# Building log.html:
build_page() {
twtxt2html -T $TEMPLATE $TWTXT > $HTML
}
# Bulk Copy files to their destinations:
copy_files() {
for DIR in "${DIST_DIRS[@]}"; do
# Copy both `txt` and `html` files to the Web server and only `txt`
# to gemini and gopher server content folders
if [ "$DIR" == "$WEB_DIR" ]; then
scp -C "$TWTXT" "$HTML" "$REMOTE_HOST:$DIR/"
else
scp -C "$TWTXT" "$REMOTE_HOST:$DIR/"
fi
done
}
#========== Call to functions ===========$
build_page && copy_files
@prologic@twtxt.net woot! Fast! I think you need to change your nick to “fastlogic” instead. :-D
Thank you for adding the feature so fast, @prologic@twtxt.net! Look at how beautiful this one renders now. Oh my!
jenny nor yarnd support it very well. Only at a very basic level.
@prologic@twtxt.net sorry but nope. Neither jenny, nor yarnd supports it at all. This was treated as a thread because I picked one of @falsifian@www.falsifian.org’s twtxts (with the “old subject”), and replied to it (hence starting the thread).
yarnd (at least) doesn't support creating such a custom TwtSubject, but it will reply and respect and thread one if one was constructed.
Oh, and you can’t imagine the level of control I am commandeering by restraining me from editing that previous “missing-one-backtick” twtxt. LOL!
@aelaraji@aelaraji.com this is the little script I am using on my publish_command:
#!/usr/bin/env bash
twtxt2html -t "Quark's twtxt feed" /var/www/sites/ferengi.one/twtxt.txt > /var/www/sites/ferengi.one/index.html
I named it twtxtit. :-)
yarnd (at least) doesn't support creating such a custom TwtSubject, but it will reply and respect and thread one if one was constructed.
@prologic@twtxt.net based on @falsifian@www.falsifian.org’s findings, I don’t believe this is quite accurate.
“yarnd
(_at least_) doesn't support creating such a custom TwtSubject, but it will reply and respect and thread one if one was constructed."
@falsifian@www.falsifian.org yes, that happened around 2 years ago, on commit 5923078ea5.
@quark@ferengi.one It looks like the part about traditional topics has been removed from that page. Here is an old version that mentions it: https://web.archive.org/web/20221211165458/https://dev.twtxt.net/doc/twtsubjectextension.html . Still, I don’t see any description of what is actually allowed between the parentheses. May be worth noting that twtxt.net is displaying the twts with the subject stripped, so some piece of code is recognizing it as a subject (or, at least, something to be removed).
Opened a couple of issues on twtxt2html. Maybe @prologic@twtxt.net will get to them after he has completed his luxurious recharging cycle. LOL.
main.go (but it can be done on a template now, so no reason to touch the code):
@lyse@lyse.isobeef.org fully agree. I have never been a fan of relative times to begin with, so that one will go away, foh sho! :-D
@falsifian@www.falsifian.org based on Twt Subject Extension, your subject is invalid. You can have custom subjects, that is, not a valid hash, but you simply can’t put anything, and expect it to be treated as a TwtSubject, me thinks.
Hmm, but yarnd also isn’t showing these twts as being part of a thread. @prologic@twtxt.net you said yarnd respects customs subjects. Shouldn’t these twts count as having a custom subject, and get threaded together?
yarnd just doesn’t render the subject. Fair enough. It’s (replyto http://darch.dk/twtxt.txt 2024-09-15T12:50:17Z), and if you don’t want to go on a hunt, the twt hash is weadxga: https://twtxt.net/twt/weadxga
@sorenpeter@darch.dk I like this idea. Just for fun, I’m using a variant in this twt. (Also because I’m curious how it non-hash subjects appear in jenny and yarn.)
URLs can contain commas so I suggest a different character to separate the url from the date. Is this twt I’ve used space (also after “replyto”, for symmetry).
I think this solves:
- Changing feed identities: although @mckinley@twtxt.net points out URLs can change, I think this syntax should be okay as long as the feed at that URL can be fetched, and as long as the current canonical URL for the feed lists this one as an alternate.
- editing, if you don’t care about message integrity
- finding the root of a thread, if you’re not following the author
An optional hash could be added if message integrity is desired. (E.g. if you don’t trust the feed author not to make a misleading edit.) Other recent suggestions about how to deal with edits and hashes might be applicable then.
People publishing multiple twts per second should include sub-second precision in their timestamps. As you suggested, the timestamp could just be copied verbatim.
@prologic@twtxt.net I have some ideas:
- Add smartypants rendering, just like Yarn has.
- Add the ability to create individual twtxts, each named after their hash.
- Fix the formatting of the help. :-P
Woot, yes! It works perfectly. By the time you see my twtxt, it is already at the main Ferengi.one website.
@bender@twtxt.net that’s not your change, silly robot, it is mine! LOL. I am finding @prologic@twtxt.net’s tool handy to refer to previous posts (as reference, for example).
@aelaraji@aelaraji.com this is my change on main.go (but it can be done on a template now, so no reason to touch the code):
<time class="dt-published" datetime="{{ $twt.Created | date "2006-01-02T15:04:05Z07:00" }}">
{{ $twt.Created | date "2006-01-02 15:04:05 MST" }}
</time>
See https://ferengi.one. I am going to further customise things, but that’s a start.
@lyse@lyse.isobeef.org Sorry, I don’t think I ever had charset=utf8. I just noticed that a few days ago. OpenBSD’s httpd might not support including a parameter with the mime type, unfortunately. I’m going to look into it.
Now WTF!? Suddenly, @falsifian@www.falsifian.org’s feed renders broken in my tt Python implementation. Exactly what I had with my Go rewrite. I haven’t touched the Python stuff in ages, though. Also, tt and tt2 do not share any data at all.
By any chance, did you remove the ; charset=utf-8 from your Content-Type: text/plain header, falsifian?
@movq@www.uninformativ.de Non-ASCII characters were broken. Like U+2028, degrees (°), etc.
Turns out I used a silly library to detect the encoding and transform to UTF-8 if needed. When there is no Content-Type header, like for local files, it looks at the first 1024 bytes. Since it only saw ASCII in that region, the damn thing assumed the data to be in Windows-1252 (which for web pages kinda makes sense):
// TODO: change default depending on user's locale?
return charmap.Windows1252, "windows-1252", false
https://cs.opensource.google/go/x/net/+/master:html/charset/charset.go;l=102
This default is hardcoded and cannot be changed.
Trying to be smart and adding automatic support for other encodings turned out to be a bad move on my end. At least I can reduce my dependency list again. :-)
I now just reject everything that explicitly specifies something different than text/plain and an optional charset other than utf-8 (ignoring casing). Otherwise I assume it’s in UTF-8 (just like the twtxt file format specification mandates) and hope for the best.
-T/--template in case you need a custom template 👌
@bender@twtxt.net I should put the template that is used by default as a file in the repo. Look at the source for now and you’ll see 😅
@movq@www.uninformativ.de I didn’t run the command as you recommended, but, I wiped things once more, and ran jenny -f, and this time got:
david@arrakis:~$ jenny -f
Fetching archived feed https://anthony.buc.ci/user/abucci/twtxt.txt/1 (configured as abucci, https://anthony.buc.ci/user/abucci/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2024-04.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://darch.dk/twtxt-archive.txt (configured as soren, https://darch.dk/twtxt.txt)
Fetching archived feed https://www.uninformativ.de/twtxt-old_2024-04-21_6v47cua.txt (configured as movq, https://www.uninformativ.de/twtxt.txt)
Fetching archived feed https://twtxt.net/user/prologic/twtxt.txt/1 (configured as prologic, https://twtxt.net/user/prologic/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2024-03.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://www.uninformativ.de/twtxt-old_2022-12-21_2us6qbq.txt (configured as movq, https://www.uninformativ.de/twtxt.txt)
Fetching archived feed https://twtxt.net/user/prologic/twtxt.txt/2 (configured as prologic, https://twtxt.net/user/prologic/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2024-02.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://www.uninformativ.de/twtxt-old_2022-01-14_ew5gzca.txt (configured as movq, https://www.uninformativ.de/twtxt.txt)
Fetching archived feed https://twtxt.net/user/prologic/twtxt.txt/3 (configured as prologic, https://twtxt.net/user/prologic/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2024-01.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://www.uninformativ.de/twtxt-old_2021-12-23_f6y65bq.txt (configured as movq, https://www.uninformativ.de/twtxt.txt)
Fetching archived feed https://twtxt.net/user/prologic/twtxt.txt/4 (configured as prologic, https://twtxt.net/user/prologic/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2023-12.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://www.uninformativ.de/twtxt-old_2021-12-04_e4x7yba.txt (configured as movq, https://www.uninformativ.de/twtxt.txt)
Fetching archived feed https://twtxt.net/user/prologic/twtxt.txt/5 (configured as prologic, https://twtxt.net/user/prologic/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2023-11.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://www.uninformativ.de/twtxt-old_2021-11-18_42tjxba.txt (configured as movq, https://www.uninformativ.de/twtxt.txt)
Fetching archived feed https://twtxt.net/user/prologic/twtxt.txt/6 (configured as prologic, https://twtxt.net/user/prologic/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2023-10.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://www.uninformativ.de/twtxt-old_2021-11-08_i2wnvaa.txt (configured as movq, https://www.uninformativ.de/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2023-09.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://www.uninformativ.de/twtxt-old_2021-10-23_kvwn5oa.txt (configured as movq, https://www.uninformativ.de/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2023-08.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://www.uninformativ.de/twtxt-old_2021-10-11_mljudaa.txt (configured as movq, https://www.uninformativ.de/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2023-07.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://www.uninformativ.de/twtxt-old_2021-09-22_5mkqwua.txt (configured as movq, https://www.uninformativ.de/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2023-06.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://www.uninformativ.de/twtxt-old_2021-07-27_xcnzmlq.txt (configured as movq, https://www.uninformativ.de/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2023-05.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://www.uninformativ.de/twtxt-old_2021-06-16_mtedqya.txt (configured as movq, https://www.uninformativ.de/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2023-04.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://www.uninformativ.de/twtxt-old_2021-04-29_z7lvzja.txt (configured as movq, https://www.uninformativ.de/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2023-03.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://www.uninformativ.de/twtxt-old_2021-03-19_xjabvhq.txt (configured as movq, https://www.uninformativ.de/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2023-02.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://www.uninformativ.de/twtxt-old_2021-02-24_te4a6oa.txt (configured as movq, https://www.uninformativ.de/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2023-01.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://www.uninformativ.de/twtxt-old_2021-01-26_qxgigma.txt (configured as movq, https://www.uninformativ.de/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2022-12.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://www.uninformativ.de/twtxt-old_2020-12-13_igfnala.txt (configured as movq, https://www.uninformativ.de/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2022-11.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2022-10.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2022-09.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2022-08.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2022-07.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2022-06.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2022-05.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2022-04.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2022-03.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2022-02.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2022-01.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2021-12.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2021-11.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2021-10.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2021-09.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2021-08.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2021-07.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2021-06.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2021-05.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2021-04.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2021-03.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2021-02.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2021-01.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Fetching archived feed https://lyse.isobeef.org/twtxt-2020-12.txt (configured as lyse, https://lyse.isobeef.org/twtxt.txt)
Notice that @prologic@twtxt.net’s /6 is there. I found the twtxt then. Kind of odd it didn’t show before.
(#hash;#originalHash) would also work.
Maybe I’m being a bit too purist/minimalistic here. As I said before (in one of the 1372739 posts on this topic – or maybe I didn’t even send that twt, I don’t remember 😅), I never really liked hashes to begin with. They aren’t super hard to implement but they are kind of against the beauty of the original twtxt – because you need special client support for them. It’s not something that you could write manually in your
twtxt.txtfile. With @sorenpeter@darch.dk’s proposal, though, that would be possible.
Tangentially related, I was a bit disappointed to learn that the twt subject extension is now never used except with hashes. Manually-written subjects sounded so beautifully ad-hoc and organic as a way to disambiguate replies. Maybe I’ll try it some time just for fun.
@aelaraji@aelaraji.com I just added support for passing a custom template file via -T/--template in case you need a custom template 👌
prologic@JamessMacStudio
Wed Sep 18 01:27:29
~/Projects/yarnsocial/twtxt2html
(main) 130
$ ./twtxt2html --help
Usage: twtxt2html [options] FILE|URL
twtxt2html converts a twtxt feed to a static HTML page
-d, --debug enable debug logging
-l, --limit int limit number ot twts (default all) (default -1)
-n, --noreldate do now show twt relative dates
-r, --reverse reverse the order of twts (oldest first)
-T, --template string path to template file
-t, --title string title of generated page (default "Twtxt Feed")
-v, --version display version information
pflag: help requested
@quark@ferengi.one At the moment, the twt in question exists in the sixth archive:
$ jenny -D https://twtxt.net/user/prologic/twtxt.txt/6 | head
[o6dsrga] [2020-07-18 12:39:52+00:00] [Hello World! 😊]
Does that work for you? 🤔
() @falsifian@www.falsifian.org You mean the idea of being able to inline
# url =changes in your feed?
Yes, that one. But @lyse@lyse.isobeef.org pointed out suffers a compatibility issue, since currently the first listed url is used for hashing, not the last. Unless your feed is in reverse chronological order. Heh, I guess another metadata field could indicate which version to use.
Or maybe url changes could somehow be combined with the archive feeds extension? Could the url metadata field be local to each archive file, so that to switch to a new url all you need to do is archive everything you’ve got and start a new file at the new url?
I don’t think it’s that likely my feed url will change.
@prologic@twtxt.net Yeah, that thing with (#hash;#originalHash) would also work.
Maybe I’m being a bit too purist/minimalistic here. As I said before (in one of the 1372739 posts on this topic – or maybe I didn’t even send that twt, I don’t remember 😅), I never really liked hashes to begin with. They aren’t super hard to implement but they are kind of against the beauty of the original twtxt – because you need special client support for them. It’s not something that you could write manually in your twtxt.txt file. With @sorenpeter@darch.dk’s proposal, though, that would be possible.
I don’t know … maybe it’s just me. 🥴
I’m also being a bit selfish, to be honest: Implementing (#hash;#originalHash) in jenny for editing your own feed would not be a no-brainer. (Editing is already kind of unsupported, actually.) It wouldn’t be a problem to implement it for fetching other people’s feeds, though.
@aelaraji@aelaraji.com Btw, I’m also open to ideas for this tool and welcome any contributions 👌
@bender@twtxt.net It’s just a simple twtxt2html and scp … it goes like:
twtxt2html $HOME/path/to/local_twtxt_dir/twtxt.txt > $HOME/path/to/local_twtxt_dir/log.html && \
scp $HOME/path/to/local_twtxt_dir/log.html user@remotehost:/path/to/static_files_dir/
I’ve been lazy to add it to my publish_command script, now I can just copy/pasta from the twt 😅