prologic

twtxt.net

"Problems are Solved by Method" 🇦🇺👨‍💻👨‍🦯🏹♔ 🏓⚯ 👨‍👩‍👧‍👧🛥 -- James Mills (operator of twtxt.net / creator of Yarn.social 🧶)

Recent twts from prologic

Summary of Discussions (as best I can):

  • @lyse@lyse.isobeef.org and @sorenpeter@darch.dk express simplicity. Both Lyse and Sorenpeter support location-based addressing.
  • @falsifian@www.falsifian.org believes we should continue to develop ideas and extensions progressively over time like we’ve always done.
  • @david@collantes.us @quark@ferengi.one and @bender@twtxt.net would like a better user experience, especially when threads break due to edits, deletions or feed location changes.
  • @anth@a.9srv.net would like to see utf-8 mandated, and the threading model remain largely the same as it is today, which is primarily based on the convention of a Twt Subject anyway, Twt Hash(es) just make the threading “more precise”. Anth also states that format, client and server specification/recommendations should be kept separate.
  • @movq@www.uninformativ.de @xuu sorry you two haven’t said too much really, so I’m not too sure?

Overall, the 22 votes we’ve had on the poll from the community (if you can call it a community?) have clearly shown that:

  • We continue to support content-based addressing. (65/35)
  • We think about formally supporting edits/deletes (60/40)
  • We do not increase the use of cryptography (thworing things like authenticity and identity out the window) (70/30)

And overall the NPS (net promoter score) of “Would I recommend Twtxt to a friend” is a whopping 7/10 (which is crazy! 🤯)

Let’s have our monthly catch up soon™ (1hr) and discuss together. My own take on the direction we should take at this point is as follows:

  • We continue to use hashing for the threading model.
    • We think about changing this to SHA-256 for simplicity.
  • We either adopt @anth@a.9srv.net’s UUID approach or @lyse@lyse.isobeef.org Dynamic URL approach.
  • We continue to incrementally/progressively improve things over time as @falsifian@www.falsifian.org suggested.
  • We think about mandating utf-8 as @anth@a.9srv.net suggests which makes things so much easier for everyone.
  • We further discuss the merits/ideas of supporting formal Edit/Delete requests or other ways to better support this in some way.

⤋ Read More

Hmm this question has a leading “Yes” in favor of so far with 13 votes:

Should we formally support edit and deletion requests?

Thanks y’all for voting (it’s all anonymous so I have no idea who’s voted for what!)

If you haven’t already had your say, please do so here: http://polljunkie.com/poll/xdgjib/twtxt-v2 – This is my feeble attempt at trying to ascertain the voice of the greater community with ideas of a Twtxt v2 specification (which I’m hoping will just be an improved specification of what we largely have already built to date with some small but important improvements 🤞)

⤋ Read More

Just out of curiosity, I inspected the yarns database (the search engine//cralwer) to find the average length of a Twtxt URI:

$ inspect-db yarns.db | jq -r '.Value.URL' | awk '{ total += length; count++ } END { if (count > 0) print total / count }'
40.3387

Given an RFC3339 UTC timestamp has a length of 20 characters with seconds precision. We’re talking about Twt Subject taking up ~63 characters/bytes on average.

⤋ Read More

My Position on the last few weeks of Twtxt spec discussions:

Feed authors that wish to change the location of their feed (once Twts have been published) must append a new # url = comment to their feed to indicate the new location and thus change the “Hashing URI” used for Twts from that point onward.

This has implications of the “order” of a feed, and we should either do one of two things, either:

  • Mandate that feeds are append-only.
  • Or amend the Metadata spec with a new field that denotes the order of the feed so clients can make sense of “inline” comments in the feed. – This would also imply that the default order is (of course) append-only. Suggestion: # direction = [append|prepend]

⤋ Read More

With a SHA1 encoding the probability of a hash collision becomes, at various k (number of twts):

>>> import math
>>>
>>> def collision_probability(k, bits):
...     n = 2 ** bits  # Total unique hash values based on the number of bits
...     probability = 1 - math.exp(- (k ** 2) / (2 * n))
...     return probability * 100  # Return as percentage
...
>>> # Example usage:
>>> k_values = [100000, 1000000, 10000000]
>>> bits = 44  # Number of bits for the hash
>>>
>>> for k in k_values:
...     print(f"Probability of collision for {k} hashes with {bits} bits: {collision_probability(k, bits):.4f}%")
...
Probability of collision for 100000 hashes with 44 bits: 0.0284%
Probability of collision for 1000000 hashes with 44 bits: 2.8022%
Probability of collision for 10000000 hashes with 44 bits: 94.1701%
>>> bits = 48
>>> for k in k_values:
...     print(f"Probability of collision for {k} hashes with {bits} bits: {collision_probability(k, bits):.4f}%")
...
Probability of collision for 100000 hashes with 48 bits: 0.0018%
Probability of collision for 1000000 hashes with 48 bits: 0.1775%
Probability of collision for 10000000 hashes with 48 bits: 16.2753%
>>> bits = 52
>>> for k in k_values:
...     print(f"Probability of collision for {k} hashes with {bits} bits: {collision_probability(k, bits):.4f}%")
...
Probability of collision for 100000 hashes with 52 bits: 0.0001%
Probability of collision for 1000000 hashes with 52 bits: 0.0111%
Probability of collision for 10000000 hashes with 52 bits: 1.1041%
>>>

If we adopted this scheme, we could have to increase the no. of characters (first N) from 11 to 12 and finally 13 as we approach globally larger enough Twts across the space. I think at least full crawl/scrape it was around ~500k (maybe)? https://search.twtxt.net/ says only ~99k

⤋ Read More

I’ve been using Codeium too the last week or so ! It’s pretty good and like @xuu said is a pretty desent Junior assistant, it helps me write good docs and the tab completion is amazing!

It of course completely sucks at doing anything “intelligent” or complex, but if you just use it as a fancier auto complete it’s actually half way decent 👌

⤋ Read More

One thing that’s on my mind over the last few days about all this Twt editing and identity stuff we’ve been having hot debates over is this…

I don’t really have a problem with editing twts, or someone changing their feed’s URL.

Personally I think the folks that do are rightfully pedantic and like a good user experience, which I don’t blame ‘em. I would expect the same too. Anyway, just wanted to get that out there, I believe we can support editing and identity in a way that is still simple, as long as we bring clients along for the ride with us. The old/legacy original client though will have to remain well, ya know 😅

⤋ Read More

Introduction to JuiceFS | JuiceFS Document Center – Thinking about using JuiceFS to solve a long-running problem I’ve always had.

  • Be able to run services on any node in my cluster and let Docker Swarm pick whatever node it likes (instead of now where I have to pin some workloads to specific nodes, as that’s where their local storage volume is)
  • Manage the scalability of data and growth over time instead of what I do now which is to extend EXT4 filesystems on my Docker Swarm nodes every few years.

⤋ Read More

Time for work™, But I quickly hacked together a bit of a better solution here. Rolling it out to my pod so we’ll see how it actually goes. Still possible to abuse if you’re a logged in user, etc, but at least now we delete the invalid/bad feed afterwards if it a) was not even a text//plain content-type or b) it errored out and was a new fetch of a HTTP feed.

⤋ Read More

I just realized, this is the last Saturday of the month. So Yarn.social meetup is up again tomorrow. Same time as last time if anyone is interested/around to join and hang out!

⤋ Read More

Dear OnlyDomains, part of Team Internet. Do you think you could stop being so incompetent when it comes to Domains, DNS and basic HTTP? I reported this to you on Friday, and you are still arguing with me over Support the legitimatecy of the claims? Seriously?! 😧

$ dig @1.1.1.1 +short onlydomains.com.au a
198.50.252.65

$ nc -vvv 198.50.252.65 443
nc: connectx to 198.50.252.65 port 443 (tcp) failed: Connection refused

#OnlyDomains

⤋ Read More

Hmmm 🧐

for url in $(jq -r '.Twters[].avatar' cache.json | sed '/^$/d' | grep -v -E '(twtxt.net|anthony.buc.ci|yarn.stigatle.no|yarn.mills.io)' | sort -u); do echo "$url $(curl -I -s -o /dev/null -w '%header{content-length}' "$url")"; done
...

😅 Let’s see… 🤔

⤋ Read More

Hmm remove the cpu limits on this pod, not even sure why I had ‘em set tbh, we decided at my day job that setting cpu limits on containers is a bit of a silly idea too. Anyway, pod should be much snappier now 😅

⤋ Read More

I had a play with LiveKit Agents Playground: KITT and I have to say it’s pretty impressive. Not the ChatGPT part of course, but the speech recognition and text to speech synthesis.

KITT is an AI voice assistant powered by LiveKit Agents, Deepgram, Eleven Labs, and ChatGPT. It is running on LiveKit Playground.

It’s too bad it relies on three cloud services, none of which can be run locally (with the exception of Ollama that you could replace the OpenAI component with).

⤋ Read More

So Belong (out retail mobile phone provider of choice), who are owned by Telstra want to increase the price of their plans by +40%.

Telstra, who own Belong, have had the following financial earnings over the past 4 years:

  • FY2021: NPAT (Net Profit After Tax): +3.4% $1.9B
  • FY2022: NPAT (Net Profit After Tax): -4.6% $1.8B
  • FY2023: NPAT (Net Profit After Tax): +13.1% $2.1B
  • FY2024: NPAT (Net Profit After Tax): +11.4% $1B

Not sure how this year’s results had a +11.4% increase, but only $1B in profits.

#Telstra #Belong #Australia #PriceHikes

⤋ Read More

I recently learned that our Australian Liberal National Party, spent 10’s of thousands of dollars on a campaign involving flyers posted around the suburbs (localities) of our local Greens federal member, Elizabeth Watson Brown. Not only was the material produced by the LNP party, distributed and paid for by the LNP party, full of lies, but they had the audacity to make the “flyers” appear as though they were from the Greens themselves! 🤦‍♂️ wtf?! #Politics #Sucks

⤋ Read More

* 8eef4d5d - (HEAD -> main, origin/main) Add DB restore capability and tools/backu-_db.sh script (19 seconds ago) <James Mills>

Oh well 😅 It works wonderfully!

In the event of a database corruption or loss:

$ URL=http://10.0.0.164:8000 ./tools/backup_db.sh > db.json
mv db.json data/db.restore.json
yarnd ...

⤋ Read More