@zvava@twtxt.net yarnd fetches the feeds roughly every ten minutes:

grep twtxt.net www/logs/twtxt.log | cut -d ' ' -f1 | tail -n 20
2025-10-04T07:00:45+02:00
2025-10-04T07:10:26+02:00
2025-10-04T07:22:43+02:00
2025-10-04T07:30:45+02:00
2025-10-04T07:40:48+02:00
2025-10-04T07:52:59+02:00
2025-10-04T08:00:07+02:00
2025-10-04T08:13:33+02:00
2025-10-04T08:23:13+02:00
2025-10-04T08:31:22+02:00
2025-10-04T08:41:29+02:00
2025-10-04T08:53:25+02:00
2025-10-04T09:03:31+02:00
2025-10-04T09:11:42+02:00
2025-10-04T09:23:11+02:00
2025-10-04T09:29:49+02:00
2025-10-04T09:36:17+02:00
2025-10-04T09:46:33+02:00
2025-10-04T09:58:40+02:00
2025-10-04T10:06:54+02:00

I suspect that the timing was just right. Or wrong, depending on how you’re looking at it. ;-)

@movq@www.uninformativ.de Luckily, I had a grep -v git at the end, so my repo is still in working order. Phew. I wish find had grep-like --exclude-dir and --exclude options (or the include variants) instead of its own weird options that I never can remember and combine properly.

@kat@yarn.girlonthemoon.xyz @movq@www.uninformativ.de @prologic@twtxt.net Yeah, I’m also having them in my repertoire for ages, so I’m used to the weird command line options. From today’s perspective, they’re not consistent with the rest of the typical shell utilities, that’s for sure.

Regarding find | grep foo, I recommend find -name '*foo*', prologic. Also, I regularly use -type d and -type f to find directories or files.

@movq@www.uninformativ.de Yeah I actually use sift a lot these days for most “searching” – at least code and text searching. For finding files by name I still use find | grep.

@bender@twtxt.net I’m not sure this is accurate, if you lookup mine:

$ whois shortcircuit.net.au 2>&1 | grep -i creat
created:      1986-03-05

I think this has to be the registrar’s creation date no? 🤔

What is it wrong with this command: :r ~/.bash_history | %!grep ‘err’ ⌘ Read more

This code displays the last 10 lines of a twtxt feed without a full dowload.

FEED_URL="https://twtxt.net/user/prologic/twtxt.txt"
MAX_RANGE=$(curl -sI $FEED_URL | grep -i 'content-length' | awk '{print $2}' | tr -d '\r')
MIN_RANGE=$((MAX_RANGE - 5000))

curl -s --range "$MIN_RANGE-$MAX_RANGE" "$FEED_URL" | grep -v -e '^#' -e '^$' | head -n 10

My self-response!

@prologic@twtxt.net @david@collantes.us

#twtxt

@andros@twtxt.andros.dev If something fits in a CSV file, it typically doesn’t require a database. I agree with that. Depending on the application, more complicated queries might benefit from a database, though. I don’t know awk very well, but I could imagine that grep, sed and cut reach their CSV processing limits rather quickly when you have to deal with escaped (multiline) fields.

I only very rarely have to deal with CSV files or databases in my day to day life. Maybe, these classic Unix tools offer some tricks I’m not aware of. When I have some more complicated CSV input, I generally reach for Python.

Bloody hell 🤦‍♂️🤦‍♂️

$ jq -r --arg host "gopher.mills.io" '. | select(.request.host==$host) | "\(.request.client_ip) \(.request.uri) \(.request.headers["User-Agent"])"' mills.io.log-au | while IFS=$' ' read -r ip uri ua; do asn="$(geoip -a "$ip")"; echo "$asn $ip $uri $ua"; done | grep -E '^45102.*' | sort | head
45102 47.251.70.245 /gopher.floodgap.com/0/feeds/democracynow/2015/Oct/14/0 ["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36"]
45102 47.251.84.25 /gopher.floodgap.com/0/feeds/voaheadlines/2014/Mar/09/voanews.com-content-article-1867433.html ["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"]
45102 47.82.10.106 /gopher.viste.fr/1/OnlineTools/hangman.cgi%3F0692937396569A52972EB2 ["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36 Edg/114.0.1823.43"]
45102 47.82.10.106 /gopher.viste.fr/1/OnlineTools/hangman.cgi%3F9657307A96569A52974634 ["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36 Edg/114.0.1823.43"]
45102 47.82.10.106 /gopher.viste.fr/1/OnlineTools/hangman.cgi%3FB7571C7896569A529E6603 ["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36 Edg/114.0.1823.43"]
45102 47.82.10.106 /gopher.viste.fr/1/OnlineTools/hangman.cgi%3FB75EF81296569A529E6617 ["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36 Edg/114.0.1823.43"]
45102 47.82.10.106 /gopher.viste.fr/1/OnlineTools/hangman.cgi%3FC6564ADB96569A5A9E660C ["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36 Edg/114.0.1823.43"]

Had to build a list of all feeds (that I follow) and all twts in them and there are two collisions already:

$ ./stats
Saw 58263 hashes
7fqcxaa
  https://twtxt.net/user/justamoment/twtxt.txt
  https://twtxt.net/user/prologic/twtxt.txt
ntnakqa
  https://twtxt.net/user/prologic/twtxt.txt
  https://twtxt.net/user/thecanine/twtxt.txt

Namely:

$ jenny -D https://twtxt.net/user/justamoment/twtxt.txt | grep 7fqcxaa

[7fqcxaa] [2022-12-28 04:53:30+00:00] [(#pmuqoca) @prologic@twtxt.net I checked the GitHub discussion, it became a request to join forces.

Do you plan on having them join?

Also for the name, how about:

• “progit” or “prologit” (prologic official hard fork)
  
• “git-stance” (git instance)
  
• “GitTree” (Gitea inspired, maybe to related)
  
• “Gitomata” (git automata)
  
• “Git.Source”
  
• “Forgor” (forgit is taken so I forgor) 🤣
  
• “SweetGit” (as salty chat)
  
• “Pepper Git” (other ingredients) 😉
  
• “GitHeart” (core of git with a GitHub sounding name)
  
• “GitTaka” (With music in mind)
  

Ok, enough fun… Hope this helps sprout some ideas from others if nothing is to your taste.]

$ jenny -D https://twtxt.net/user/prologic/twtxt.txt/5 | grep 7fqcxaa

[7fqcxaa] [2022-02-25 21:14:45+00:00] [(#bqq6fxq) It’s handled by blue Monday]

And:

$ jenny -D https://twtxt.net/user/thecanine/twtxt.txt | grep ntnakqa
[ntnakqa] [2022-01-23 10:24:09+00:00] [(#2wh7r4q) <a href="https://txt.sour.is/external?uri=https://twtxt.net/user/prologic/twtxt.txt">@prologic<em>@twtxt.net</em></a> I know, I was just hoping it might have also gotten fixed by that change, by some kind of backend miracles. 😂]

$ jenny -D https://twtxt.net/user/prologic/twtxt.txt/1 | grep ntnakqa
[ntnakqa] [2024-02-27 05:51:50+00:00] [(#otuupfq) <a href="https://txt.sour.is/external?uri=https://twtxt.net/user/shreyan/twtxt.txt">@shreyan<em>@twtxt.net</em></a>  Ahh 👌]

@aelaraji@aelaraji.com This is one of the reasons why yarnd has a couple of settings with some sensible/sane defaults:

I could already imagine a couple of extreme cases where, somewhere, in this peaceful world one’s exercise of freedom of speech could get them in Real trouble (if not danger) if found out, it wouldn’t necessarily have to involve something to do with Law or legal authorities. So, If someone asks, and maybe fearing fearing for… let’s just say ‘Their well being’, would it heart if a pod just purged their content if it’s serving it publicly (maybe relay the info to other pods) and call it a day? It doesn’t have to be about some law/convention somewhere … 🤷 I know! Too extreme, but I’ve seen news of people who’d gone to jail or got their lives ruined for as little as a silly joke. And it doesn’t even have to be about any of this.

There are two settings:

$ ./yarnd --help 2>&1 | grep max-cache
      --max-cache-fetchers int        set maximum numnber of fetchers to use for feed cache updates (default 10)
  -I, --max-cache-items int           maximum cache items (per feed source) of cached twts in memory (default 150)
  -C, --max-cache-ttl duration        maximum cache ttl (time-to-live) of cached twts in memory (default 336h0m0s)

So yarnd pods by default are designed to only keep Twts around publicly visible on either the anonymous Frontpage or Discover View or your Timeline or the feed’s Timeline for up to 2 weeks with a maximum of 150 items, whichever get exceeded first. Any Twts over this are considered “old” and drop off the active cache.

It’s a feature that my old man @off_grid_living@twtxt.net was very strongly in support of, as was I back in the day of yarnd’s design (nothing particularly to do with Twtxt per se) that I’ve to this day stuck by – Even though there are some 😉 that have different views on this 🤣

and then i have a compact version that makes things more grep’able in scripts.

How to grep Match Two Strings in One Line, or Anywhere in File or Command Output
The command line grep tool is incredibly powerful and useful for searching for matches in files, sorting text and data, finding strings in large files, and so much more. One common situation many command line users may find themselves in, is seeking to grep match for two different strings in a single line. You can … [Read More](https://osxdaily.com/2023/11/27/how-to-grep-mat … ⌘ Read more

I grepped access logs and found at least three subscribers! @apex@rawtext.club, @prologic@twtxt.net, and @darch@neotxt.dk, hi there!

@prologic@twtxt.net

#!/bin/sh

# Validate environment
if ! command -v msgbus > /dev/null; then
    printf "missing msgbus command. Use:  go install git.mills.io/prologic/msgbus/cmd/msgbus@latest"
    exit 1
fi

if ! command -v salty > /dev/null; then
    printf "missing salty command. Use:  go install go.mills.io/salty/cmd/salty@latest"
    exit 1
fi

if ! command -v salty-keygen > /dev/null; then
    printf "missing salty-keygen command. Use:  go install go.mills.io/salty/cmd/salty-keygen@latest"
    exit 1
fi

if [ -z "$SALTY_IDENTITY" ]; then
    export SALTY_IDENTITY="$HOME/.config/salty/$USER.key"
fi

get_user () {
    user=$(grep user: "$SALTY_IDENTITY" | awk '{print $3}')
    if [ -z "$user" ]; then
        user="$USER"
    fi
    echo "$user"
}

stream () {
    if [ -z "$SALTY_IDENTITY" ]; then
        echo "SALTY_IDENTITY not set"
        exit 2
    fi

    jq -r '.payload' | base64 -d | salty -i "$SALTY_IDENTITY" -d
}

lookup () {
    if [ $# -lt 1 ]; then
    printf "Usage: %s nick@domain\n" "$(basename "$0")"
    exit 1
    fi

    user="$1"
    nick="$(echo "$user" | awk -F@ '{ print $1 }')"
    domain="$(echo "$user" | awk -F@ '{ print $2 }')"

    curl -qsSL "https://$domain/.well-known/salty/${nick}.json"
}

readmsgs () {
    topic="$1"

    if [ -z "$topic" ]; then
        topic=$(get_user)
    fi

    export SALTY_IDENTITY="$HOME/.config/salty/$topic.key"
    if [ ! -f "$SALTY_IDENTITY" ]; then
        echo "identity file missing for user $topic" >&2
        exit 1
    fi

    msgbus sub "$topic" "$0"
}

sendmsg () {
    if [ $# -lt 2 ]; then
        printf "Usage: %s nick@domain.tld <message>\n" "$(basename "$0")"
        exit 0
    fi

    if [ -z "$SALTY_IDENTITY" ]; then
        echo "SALTY_IDENTITY not set"
        exit 2
    fi

    user="$1"
    message="$2"

    salty_json="$(mktemp /tmp/salty.XXXXXX)"

    lookup "$user" > "$salty_json"

    endpoint="$(jq -r '.endpoint' < "$salty_json")"
    topic="$(jq -r '.topic' < "$salty_json")"
    key="$(jq -r '.key' < "$salty_json")"

    rm "$salty_json"

    message="[$(date +%FT%TZ)] <$(get_user)> $message"

    echo "$message" \
        | salty -i "$SALTY_IDENTITY" -r "$key" \
        | msgbus -u "$endpoint" pub "$topic"
}

make_user () {
    mkdir -p "$HOME/.config/salty"

    if [ $# -lt 1 ]; then
        user=$USER
    else
        user=$1
    fi

    identity_file="$HOME/.config/salty/$user.key"

    if [ -f "$identity_file" ]; then
        printf "user key exists!"
        exit 1
    fi

    # Check for msgbus env.. probably can make it fallback to looking for a config file?
    if [ -z "$MSGBUS_URI" ]; then
        printf "missing MSGBUS_URI in environment"
        exit 1
    fi


    salty-keygen -o "$identity_file"
    echo "# user: $user" >> "$identity_file"

    pubkey=$(grep key: "$identity_file" | awk '{print $4}')

    cat <<- EOF
Create this file in your webserver well-known folder. https://hostname.tld/.well-known/salty/$user.json

{
  "endpoint": "$MSGBUS_URI",
  "topic": "$user",
  "key": "$pubkey"
}

EOF
}

# check if streaming
if [ ! -t 1 ]; then
    stream
    exit 0
fi

# Show Help
if [ $# -lt 1 ]; then
    printf "Commands: send read lookup"
    exit 0
fi


CMD=$1
shift

case $CMD in
    send)
        sendmsg "$@"
    ;;
    read)
        readmsgs "$@"
    ;;
    lookup)
        lookup "$@"
    ;;
    make-user)
        make_user "$@"
    ;;
esac

a simple Makefile for forwarding internet to your local machine:

SSH_HOST=https://xuu.me
PRIV_KEY=~/.ssh/id_ed25519
forward:
	LOCAL_PORT=$(HOST_PORT); sh -c "$(shell http --form POST $(SSH_HOST) pub=@$(PRIV_KEY).pub | grep ^ssh | head -1 | awk '{ print "ssh -T -p " $$4 " " $$5 " -R " $$7 " -i $(PRIV_KEY)"  }')"

@niplav@niplav.github.io Re: not guaranteed to see it: I wouldn’t have bothered waving hi across the Internet if I hadn’t gotten bored one day and grepped my webserver logs for twtxt users.

curl https://raw.githubusercontent.com/jointwt/we-are-twtxt/master/we-are-twtxt.txt | grep -v '^niplav ' | field 2 | xargs curl ^/dev/null | grep niplav here we go

number of GET /twtxt.txt requests against my server, grepped from my nginx access logs: https://jb55.com/s/e75071f023eeaf90.txt

for branch in git branch -r --merged | grep -v HEAD; do echo -e git show --format="%ci %cr %an" $branch | head -n 1 \t$branch; done | sort -r

Periodic reminder that, by using third party trackers, you are paying somebody to grep their http access log instead of greping your own. So, like, maybe roll your own analytics if you need them?

Use the x-use-gopher header on your http proxies.. “curl -sI https://codevoid.de | grep ^x-u” bitreich.org, r-36.net, taz.de are already there. #gopher

Where GREP Came From - Computerphile - YouTube https://www.youtube.com/watch?v=NTfOnGZUZDk

I changed the code of #goxtxt (my #twtxt #xmpp bot) a little bit. If all went good there should be no change in functionality but it is no more depending on #head and #grep.

Search linux bash history
Have you ever executed something on the linux shell and didn’t remember later how it was done? Well if you remember just part of it you can search for it:      history | grep -i “” ⌘ Read more