@lyse@lyse.isobeef.org Now increase the indexes on the Twt Subject form 7 bytes to 64 bytes 😈

@lyse@lyse.isobeef.org Congrats 🙌

Hmm this question has a leading “Yes” in favor of so far with 13 votes:

Should we formally support edit and deletion requests?

Thanks y’all for voting (it’s all anonymous so I have no idea who’s voted for what!)

If you haven’t already had your say, please do so here: http://polljunkie.com/poll/xdgjib/twtxt-v2 – This is my feeble attempt at trying to ascertain the voice of the greater community with ideas of a Twtxt v2 specification (which I’m hoping will just be an improved specification of what we largely have already built to date with some small but important improvements 🤞)

Starting a couple of new projects (geez where do I find the time?!):

HomeTunnel:

HomeTunnel is a self-hosted solution that combines secure tunneling, proxying, and automation to create your own private cloud. Utilizing Wireguard for VPN, Caddy for reverse proxying, and Traefik for service routing, HomeTunnel allows you to securely expose your home network services (such as Gitea, Poste.io, etc.) to the Internet. With seamless automation and on-demand TLS, HomeTunnel gives you the power to manage your own cloud-like environment with the control and privacy of self-hosting.

CraneOps:

craneops is an open-source operator framework, written in Go, that allows self-hosters to automate the deployment and management of infrastructure and applications. Inspired by Kubernetes operators, CraneOps uses declarative YAML Custom Resource Definitions (CRDs) to manage Docker Swarm deployments on Proxmox VE clusters.

@aelaraji@aelaraji.com I think all replies are missing the fact that your auto-completion isn’t working. LOL. Or did I misunderstood?

I think that’s one of the worst aspects of the proposed idea of location-based addressing or identity. The fact that Alice reads Twt A and Bob reads Twt A at the same location, but Alice and Bob could have in fact read very different content entirely. It is no longer possible to have consistency in a decentralised way that works properly.

One could argue this is fine, because we’re so small and nothing matters, but it’s a properly I rely on fairly heavily in yarnd, a properly that if lost would have significant impact on how yarnd works I think. 🤔

Unless I”m missing something here 🤔 But a <url> <timestamp> does not for me identify an individual Twt, it only identifies its location, which may or may not have changed since I last saw a version of it hmmm 🧐

Also I’m not even sure I can validly cache, let alone index feeds anymore if we do this, because if the structure of a Twt is cuh that I can no longer trust that an individual Twt’s content hasn’t been changed at the source, what’s the point of caching or indexing individual twts at all? This makes the implementations of yarnd and yarns (the search engine, crawlers and indexer) kind of hard to reason about.

Also you’re right I guess. But still that also requires the author not to change the timestamp too. Hmmm

@movq@www.uninformativ.de I don’t think there’s any misunderstand at all. I just treat every lines in a feed as an individual entity. These are stored on their own.

@movq@www.uninformativ.de So I obviously happen to agree with you as well. However in so saying, one of my goals was also to bring the simplicity of Twtxt to the Web and for the general “lay person” (of sorts). So I eventually found myself building yarnd. Has it been successful, well sort of, somewhat (but that doesn’t matter, I like that it’s small and niche anyway).

I agree that the goal of simplicity is a good goal to strive for, which is why I’m actually suggesting we change the Twt identifiers to be a simple SHA256 hash, something that everyone understand and has readily available tools for. I really don’t think we should be doing any of this by hand to be honest. But part of the beauty of Twt Subject and Twt Hash(es) in the first place is replying by hand is much much easier because you only have a short 7 or 11 character thing to copy/paste in your reply. Switching to something like <url> <timestamp> with a space in it is going to become a lot harder to copy/paste, because you can’t “double click” (or is it triple click for some?) to copy/paste to your clipboard/buffer now 🤣

Anyway I digress… On the whole edit thing, I’m actually find if we don’t support it at all and don’t build a protocol around that. I have zero issues with dropping that as an idea. Why? Because I actually think that clients should be auto-detecting edits anyway. They already can, I’ve PoC’d this myself, I think it can be done. I haven’t (yet), and one of the reasons I’ve not spent much effort in it is it isn’t something that comes up frequently anyway.

Who cares if a thread breaks every now ‘n again anyway?

@doesnm@doesnm.p.psf.lt Like maybe you need to check something, debug a client, or whatever 😅

Sorry but i dont undestand b. New feed author? But why?

Don’t forget about the upcoming Yarn.social online meetup coming up this Saturday! 😅 See #jjbnvgq for details! – Hope to see y’all there 💪

👋 Don’t forget to take the Twtxt v2 poll 🙏 if you haven’t done so already (sorry about the confusing question at the end!)

@doesnm@doesnm.p.psf.lt I don’t even advocate for reading Twtxt in its raw form in the first place, which is why I’m in favor of continuing to use content-based addressing (hashes) and incremental improve what we already have. IMO the only reason to read a Twtxt file in it’s raw form is a) if you’re a developer b) new feed author or c) debugging a client issue.

Aggred. But reading twtxt in raw form sounds… I can’t do this

And finally the legibility of feeds when viewing them in their raw form are worsened as you go from a Twt Subject of (#abcdefg12345) to something like (https://twtxt.net/user/prologic/twtxt.txt 2024-09-22T07:51:16Z).

There is also a ~5x increase cost in memory utilization for any implementations or implementors that use or wish to use in-memory storage (yarnd does for example) and equally a 5x increase in on-disk storage as well. This is based on the Twt Hash going from a 13 bytes (content-addressing) to 63 bytes (on average for location-based addressing). There is roughly a ~20-150% increase in the size of individual feeds as well that needs to be taken into consideration (on the average case).

With Location-based addressing there is no way to verify that a single Twt actaully came from that feed without actually fetching the feed and checking. That has the effect of always having to rely on fetching the feed and storing a copy of feeds you fetch (which is okay), but you’re force to do this. You cannot really share individual Twts anymore really like yarnd does (as peering) because there is no “integrity” to the Twt identified by it’s <url> <timestamp>. The identify is meaningless and is only valid as long as you can trust the location and that the location at that point hasn’t changed its content.

Location-based addressing is vulnerable to the content changing. If the content changes the “location” is no longer valid. This is a problem if you build systems that rely on this.

So really your argument is just that switching to a location-based addressing “just makes sense”. Why? Without concrete pros/cons of each approach this isn’t really a strong argument I’m afraid. In fact I probably need to just sit down and detail the properties of both approaches and the pros/cons of both.

I also don’t really buy the argument of simplicity either personally, because I don’t technically see it much more difficult to take a echo -e "<url>\t<timestamp>\t<content>" | sha256sum | base64 as the Twt Subject or concatenating the <url> <timestamp> – The “effort” is the same. If we’re going to argue that SHA256 or cryptographic hashes are “too complicated” then I’m not really sure how to support that argument.

@sorenpeter@darch.dk Points 2 & 3 aren’t really applicable here in the discussion of the threading model really I’m afraid. WebMentions is completely orthogonal to the discussion. Further, no-one that uses Twtxt really uses WebMentions, whilst yarnd supports the use of WebMentions, it’s very rarely used in practise (if ever) – In fact I should just drop the feature entirely.

The use of WebSub OTOH is far more useful and is used by every single yarnd pod everywhere (no that there’s that many around these days) to subscribe to feed updates in ~near real-time without having the poll constantly.

Some more arguments for a local-based treading model over a content-based one:

1. The format: (#<DATE URL>) or (@<DATE URL>) both makes sense: # as prefix is for a hashtag like we allredy got with the (#twthash) and @ as prefix denotes that this is mention of a specific post in a feed, and not just the feed in general. Using either can make implementation easier, since most clients already got this kind of filtering.

2. Having something like (#<DATE URL>) will also make mentions via webmetions for twtxt easier to implement, since there is no need for looking up the #twthash. This will also make it possible to make 3th part twt-mentions services.

3. Supporting twt/webmentions will also increase discoverability as a way to know about both replies and feed mentions from feeds that you don’t follow.

@doesnm@doesnm.p.psf.lt Welcome back 😅

Finally pubnix is alive! That’s im missing? Im only reading twtxt.net timeline because twtxt-v2.sh works slowly for displaying timeline…

@aelaraji@aelaraji.com Rsync has a ton of options and I probably still haven’t scratched the surface, but I was able to memorize the options I actually need for day-to-day work in a relatively short time. I guess I’m the opposite of you, because I don’t know any scp(1) options.

x86 Embedded Controller with PC/104 Compatibility for Legacy Systems
The VDX3-6757 PC/104 family of low-power x86 embedded controllers meets PC/104 specifications, offering backward compatibility for projects facing end-of-life x86-based controllers. It is suited for applications like data acquisition, industrial automation, process control, and automotive control. Powered by a DM&P Vortex86DX3 1GHz dual-core CPU with 32KB L1 cache and 512KB L2 cache, the VDX3-6757 supports … ⌘ Read more

Been trying to get acquainted with rsync(1) but, whenever I Tab for completion and get this:

λ ~/ rsync –
zsh: do you wish to see all 484 possibilities (162 lines)?

I’m like: Nope! a scp -rpCq ... or whatever option salad will do just fine. 😅 [Insert: “Ain’t nobody got time fo’that!” Meme.]

Syncthing is also as good as everyone says it is.

@movq@www.uninformativ.de Yes, the tools are surprisingly fast. Still, magrep takes about 20 seconds to search through my archive of 140K emails, so to speed things up I would probably combine it with an indexer like mu, mairix or notmuch.

@eapl.me@eapl.me Sad to see you go, disappointed in your choice of X, but respect your decision and choice. I will never cave in myself, even if it means my “circle of friends” remains low. I guess we call ‘em internet friends right? 😅

Aunque me gusta mucho el concepto descentralizado de ‘twtxt’, este año no lo he utilizado tanto. No pude tener a mi círculo cercano, con quienes surgen las conversaciones que me gustan, y por el que se da un efecto de red significativo.

También estoy buscando un minimalismo digital, utilizando servicios que brinden alegría, valor y un uso de tiempo razonable.
Aunque es un tema controversial, ¿por qué no tener una comunidad de personas con las que sintamos que el mundo (digital al menos) es un lugar mejor?

Quizás un poco idealista el punto, aunque la intención es que el tiempo que pasamos en ‘la red’, nos ayude a crecer como personas, a disfrutar el tiempo, y a vivir esta vida digital con sentido.

Por todo esto, el poco tiempo que esté en microblogging, lo buscaré en las dos plataformas que más conversaciones significativas me generan, que por un lado es X, para todo lo profesional, y Mastodon, para lo hipster, indie, idealista, etc.

Si algo de lo que he compartido por twtxt ha sido importante para ti, o quieres que sigamos charlando, me puedes encontrar en alguna de estas otras plataformas:
https://text.eapl.mx/microblogging

#fzf is the new emacs: a tool with a simple purpose that has evolved to include an #email client. https://sr.ht/~rakoo/omail/

I’m being a little silly, of course. fzf doesn’t actually check your email, but it appears to be basically the whole user interface for that mail program, with #mblaze wrangling the emails.

I’ve been thinking about how I handle my email, and am tempted to make something similar. (When I originally saw this linked the author was presenting it as an example tweaked to their own needs, encouraging people to make their own.)

This approach could surely also be combined with #jenny, taking the place of (neo)mutt. For example mblaze’s mthread tool presents a threaded discussion with indentation.

@lyse@lyse.isobeef.org How violent is the thunderstorm? 🤔

@aelaraji@aelaraji.com LOl 😂

LMAO 🤣 … I’ve been scrolling through mutt(1) man page and found this:

BUGS

  None.  Mutts have fleas, not bugs.

A new thing LLM(s) can’t do well. Write patches 🤣

@lyse@lyse.isobeef.org Yeah I think it’s one of the reasons why yarnd’s cache became so complicated really. I mean it’s a bunch of maps and lists that is recalculated every ~5m. I don’t know of any better way to do this right now, but maybe one day I’ll figure out a better way to represent the same information that is displayed today that works reasonably well.

My point is, this is not a small trade-off to make for the sake of simplicity 😅

@movq@www.uninformativ.de Maybe I misspoke. It’s a factor of 5 in the size of the keyspace required. The impact is significantly less for on-disk storage of raw feeds and such, around ~1-1.5x depending on how many replies there are I suppose.

I wasn’t very clear; my apologies. If we update the current hash truncation length from 7 to 11. But then still decide anyway to go down this location-based twt identity and threading model then yes, we’re talking about twt subjects having a ~5x increase in size on average. Going from 14 characters (11 for the has, 2 for the parens, 1 for the #) to ~63 bytes (average I’ve worked out of length of URL + Timestamp) + 3 byte overhead for parents and space.

@lyse@lyse.isobeef.org Yes I think so.

Don’t forget about the upcoming Yarn.social meetup coming up this Saturday! See #jjbnvgq for details! Hope to see some/all of y’all there 💪

@lyse@lyse.isobeef.org And your query to construct a tree? Can you share the full query (screenshot looks scary 🤣) – On another note, SQL and relational databases aren’t really that conduces to tree-like structures are they? 🤣

In fact it depends on how many Twts there are that form part of a thread, if you take a much larger sample size of my own feed for example, it starts to approximate ~1.5x increase in size:

$ ./compare.sh https://twtxt.net/user/prologic/twtxt.txt 500
Original file size: 126842 bytes
Modified file size: 317029 bytes
Percentage increase in file size: 149.94%
...

In fact @falsifian@www.falsifian.org you had quite a lot of good feedback, do you mind collecting them in a task list on the doc somewhere so I can get to em? 🤔

Can someone make the edit?

@movq@www.uninformativ.de Tbis was just a representative sample. The real concrete cost here is a ~5x increase in memory consumption for yarnd and/or ~5x increase in disk storage.

@lyse@lyse.isobeef.org Mind sharing your schema?

@lyse@lyse.isobeef.org Not sure I’ll check

@lyse@lyse.isobeef.org My proposal is three steps:

• increase the hash length from 7 to 11
  

Then:

• Add support for changing your feed’s location without breaking g threads
  

Then much later:

• Add formal support for edits

@lyse@lyse.isobeef.org No I don’t either just say’n 😅

@movq@www.uninformativ.de That’s what I want to know 🤣

What gossip, gopherspace?!

(Da ZERO)

🌍🚗 Emissões de Gases com Efeito de Estufa: A Situação Continua Preocupante
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
Um ano depois, a ZERO voltou a analisar as emissões de gases com efeito de estufa (GEE) originadas pelos combustíveis rodoviários, e os resultados continuam alarmantes. Embora tenha havido uma ligeira redução de 1,8%, ainda estamos longe de alcançar as metas necessárias. As emissões de transporte precisam cair cerca de 5,3% ao ano para que possamos atingir os objetivos do Plano Nacional de Energia e Clima (PNEC) para 2030.
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
🚨 2025 Tem de Ser um Ano Exemplar!
O gasóleo, com 14 Mt de CO2 emitidos, continua a ser o maior vilão, e o consumo de gasolina aumentou significativamente. Para evitar o incumprimento das metas climáticas, é crucial que 2025 marque uma viragem.⠀⠀⠀⠀⠀⠀⠀⠀

📊 Barómetro da Mobilidade: Lançamos também um inquérito para criar uma base de dados sobre a mobilidade em Portugal, contribuindo para decisões mais informadas.
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
💡 A mudança é urgente e depende de todos nós, especialmente dos nossos líderes políticos. Vamos juntos construir um futuro mais sustentável!
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
Mais detalhe em https://zero.ong/noticias/emissoes-dos-transportes-continuam-a-ameacar-metas-climaticas-do-pais/
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
#mundoZERO #MobilidadeSustentável #Clima

So just to be clear, it’s not as bad as the OP in this thread, this is just a worst case scenario. With some additional analysis I did today, its closer to around ~5x the memory requirements of my pod, which would roughly go from ~22MB to ~120MB or so, probably a bit more in practise. But this is still a significant increase in memory. The on-disk requirements would also increase by around ~5x as well on average going from ~12GB to about ~60GB at current archive size.

Just out of curiosity, I inspected the yarns database (the search engine//cralwer) to find the average length of a Twtxt URI:

$ inspect-db yarns.db | jq -r '.Value.URL' | awk '{ total += length; count++ } END { if (count > 0) print total / count }'
40.3387

Given an RFC3339 UTC timestamp has a length of 20 characters with seconds precision. We’re talking about Twt Subject taking up ~63 characters/bytes on average.

Comparing a few feeds:

• @xuu@txt.sour.is would see an increase of ~20%
  
• @falsifian@www.falsifian.org would see an increase of ~8%
  
• @bender@twtxt.net would see an increase of ~20%
  
• @lyse@lyse.isobeef.org would see an increase of ~15%
  
• @aelaraji@aelaraji.com would see an increase of ~13%
  
• @sorenpeter@darch.dk would see an increase of ~8%
  
• @movq@www.uninformativ.de would see an increase of ~9%
  

Just from a scalability standpoint along I’m not seeing a switch to location-based Twt ids to support threading a good idea here. This is what I meant when I said to @david@collantes.us in a recent call that we open up a new can of worms (or new set of problems) by drastically changing the approach, rather than incrementally improving the existing approach we have today (_which has served us well for the past 4 years already_0.

Reminder to take the Twtxt (anonymous) Poll: http://polljunkie.com/poll/xdgjib/twtxt-v2

Apologies, I can’t edit the poll once it’s live, so the suggestion on feedback for supporting Markdown will have to be discussed at another time.

@xuu@txt.sour.is correct

@xuu@txt.sour.is 🤣🤣🤣

I demand full 9 digit nano second timestamps and the full TZ identifier as documented in the tz 2024b database! I need to know if there was a change in daylight savings as per the locality in question as of the provided date.

@falsifian@www.falsifian.org I believe the preserve means to include the original subject hash in the start of the twt such as (#somehash)

So I whipped up a quick shell script to demonstrate what I mean by the increase in feed size on average as well as the expected increase in storage and retrieval requirements.

$ ./compare.sh
Original file size: 28145 bytes
Modified file size: 70672 bytes
Percentage increase in file size: 151.10%
...

Thank goodness we relaxed that limit and I’ve stopped being so Puritan about it but my overall point is we would be significantly increasing the human size as well as the machine size of the identity of threads as well as twts

With the original specification of 140 character Twt length recommendation. There’s only leaves you with about 78 characters worth of anything remotely useful to say in response.

Let’s say the overhead is always three bytes two parentheses under space.

So for example, if we would use @movq@www.uninformativ.de ’s feed as an example thread ID here, his feed with a particular timestamp, were already looking at a subject length of 59 bytes +/- a couple of bytes to denote the subject in the Twt itself/

One of the reasons we wanted to originally use Contant based addressing and short hashes as our threading model was to keep individual Twts short so that they were still readable if you viewed the manually by hand.

With the proposal to switch to location based addressing using a pointer to a feed and a timestamp in that feed you’re looking at roughly 2025 characters long because both the HTTP and HTML and even URI specifications do not specify maximum length for URI(s) AFAIK only recommendations.

@bender@twtxt.net I can’t see myself personally, increasing the infrastructure and costs to run this pod to support this as we switch over potentially and as things continue to grow in scale. You would never get your infinite search and infinite timeline features that you’ve always wanted for example and I would have to drastically reduce what is visible or even searchable at any given point in time to much less than what it is today.

Another interesting side effect of changing from content-based addressing to location-based addressing is that switching from 7-byte keys to 2025-character keys for 3.5 million entries would expand the database size from 24.5 MB to about 7.09 GB—an increase of roughly 7.06 GB!

@bender@twtxt.net Ha! Maybe I should get on the Markdown train. You’re taking away my excuses.

@falsifian@www.falsifian.org No worries! Fell few to contribute to the doc directly I’d you wish 👌

@falsifian@www.falsifian.org Hmmm not sure sorry 🤔

@bender@twtxt.net

Sorry, you’re right, I should have used numbers!

I’m don’t understand what “preserve the original hash” could mean other than “make sure there’s still a twt in the feed with that hash”. Maybe the text could be clarified somehow.

I’m also not sure what you mean by markdown already being part of it. Of course people can already use Markdown, just like presumably nothing stopped people from using (twt subjects) before they were formally described. But it’s not universal; e.g. as a jenny user I just see the plain text.

@xuu@txt.sour.is Goos to know! 👌 So as long as we remain decentralized and non-commercial (I assume non/profit works too?) we’re good?

MS-CF16 Fanless Low-Power Pico-ITX SBC with Alder Lake-N and Amston Lake Processors
The MS-CF16 is a compact Pico-ITX single-board computer designed for fanless, low-power, high-performance applications in harsh environments. Powered by Intel Alder Lake-N or Amston Lake Series SoCs, the board features a 2.5GbE LAN port, a GbE LAN port, and SATA 3.0 for storage. Unlike the previously covered MS-CF17, this model offers configurable Intel processors, each

@falsifian@www.falsifian.org The GDPR does not apply to the processing of data for a purely personal or household activity that is not connected to a professional or commercial activity.

@prologic@twtxt.net Do you feel the same about published vs. privately stored data?

For me there’s a distinction. I feel very strongly that I should be able to retain whatever private information I like. On the other hand, I do have some sympathy for requests not to publish or propagate (though I personally feel it’s still morally acceptable to ignore such requests).

@lyse@lyse.isobeef.org I’d suggest making the whole content-type thing a SHOULD, to accommodate people just using some hosting service they don’t have much control over. (The same situation could make detecting followers hard, but IMO “please email me if you follow me” is still legit twtxt, even if inconvenient.)

@prologic@twtxt.net Thanks for writing that up!

I hope it can remain a living document (or sequence of draft revisions) for a good long time while we figure out how this stuff works in practice.

I am not sure how I feel about all this being done at once, vs. letting conventions arise.

For example, even today I could reply to twt abc1234 with “(#abc1234) Edit: …” and I think all you humans would understand it as an edit to (#abc1234). Maybe eventually it would become a common enough convention that clients would start to support it explicitly.

Similarly we could just start using 11-digit hashes. We should iron out whether it’s sha256 or whatever but there’s no need get all the other stuff right at the same time.

I have similar thoughts about how some users could try out location-based replies in a backward-compatible way (append the replyto: stuff after the legacy (#hash) style).

However I recognize that I’m not the one implementing this stuff, and it’s less work to just have everything determined up front.

Misc comments (I haven’t read the whole thing):

• Did you mean to make hashes hexadecimal? You lose 11 bits that way compared to base32. I’d suggest gaining 11 bits with base64 instead.

• “Clients MUST preserve the original hash” — do you mean they MUST preserve the original twt?

• Thanks for phrasing the bit about deletions so neutrally.

• I don’t like the MUST in “Clients MUST follow the chain of reply-to references…”. If someone writes a client as a 40-line shell script that requires the user to piece together the threading themselves, IMO we shouldn’t declare the client non-conforming just because they didn’t get to all the bells and whistles.

• Similarly I don’t like the MUST for user agents. For one thing, you might want to fetch a feed without revealing your identty. Also, it raises the bar for a minimal implementation (I’m again thinking again of the 40-line shell script).

• For “who follows” lists: why must the long, random tokens be only valid for a limited time? Do you have a scenario in mind where they could leak?

• Why can’t feeds be served over HTTP/1.0? Again, thinking about simple software. I recently tried implementing HTTP/1.1 and it wasn’t too bad, but 1.0 would have been slightly simpler.

• Why get into the nitty-gritty about caching headers? This seems like generic advice for HTTP servers and clients.

• I’m a little sad about other protocols being not recommended.

• I don’t know how I feel about including markdown. I don’t mind too much that yarn users emit twts full of markdown, but I’m more of a plain text kind of person. Also it adds to the length. I wonder if putting a separate document would make more sense; that would also help with the length.

Apparently, participation in this year’s @hacktoberfest@hacktoberfest won’t even grant you a “plant a tree” prize… how disappointing 😞

#MaradoWeekly #WeeklyRecord Week 38

Image

@lyse@lyse.isobeef.org Nice ! 🙏

@doesnm@doesnm.p.psf.lt Hello! 👋

Sassy pererê

Hello!

@lyse@lyse.isobeef.org Yes let’s make UTF-8 mandatory 👌