Interweb – Julian Foad

Running an OpenWrt Router

by Julian Foad

2021-09-23

12:36

Leave a comment on Running an OpenWrt Router

Interweb

hardware, IT, OpenSource, OpenWrt

I am running an OpenWrt open-source router, at last.

OpenWrt: Wireless Freedom

Dave kindly donated me the hardware three years ago, when I spent many happy and frustrating hours installing OpenWrt for the first time, bricking it, recovering by connecting a serial port inside it, and eventually finding the OpenWrt configuration interfaces at that time were just too complicated for me to navigate.

It sat on my desk ever since then, unused.

What changed?

This week, our noddy little ISP-provided router keeled over.

All I did was try to change its upstream DNS server addresses to point to AdGuard’s ad blocking service. There was a simple web UI to enter the addresses, but, after doing so, its web UI promptly and permanently died and would not come back. Its DNS gateway function and SSH access died too, while some functions such as its basic routing and port forwarding continued. I tried power-cycling the router, of course, but avoided doing a factory reset because then I would lose my port forwarding that provides access to my self-hosted services such as Matrix and contacts and calendar, and would not be sure I could reconfigure everything. I was able to regain internet access temporarily, by manually configuring each of our devices to use external DNS server addresses instead of the router’s local address.

Well, I didn’t like that router anyway. Its UI was slow and awkward, its features were very bare and its WiFi was weak. (It was a Sagemcom 2704N, also branded PlusNet and Technicolor.)

So it was that I took a second look at this TP-LINK TD-W8970 router.

A pleasant surprise awaited: I found that OpenWrt had just the previous week released a major update, a 2021 version, a year and a half since their previous 2019 version, and it looks much more polished. A quick in-place firmware upgrade, followed by many hours figuring out how to make and manage the configuration, resetting, starting again from defaults, and it’s now all working. ADSL WAN connection, wired, wireless, and my port forwarding rules for my servers, and some bits of static DHCP and static DNS hostname entries.

Where the previous router had hung lopsided from one screw, to make a better impression and improve its chances of acceptance by the family I screwed it neatly to the wall and tidied the wires.

The Ordinary User May Appreciate…

ad-blocking
stronger WiFi signal now covering the whole house and garden
faster

None of these benefits seen by the ordinary user are unique to OpenWrt, of course.

Ad blocking was the trigger for this whole exercise. I had previously been considering self-hosting either Pi-Hole or Adguard-Home. Recently I learned that AdGuard DNS service is currently available free of charge, simply by setting it as the router’s DNS server address (or, less conveniently, by overriding the setting in individual devices). While less comprehensive and customisable than a self-hosted ad-blocking DNS server, for the time being the convenience and simplicity of this solution wins.

The new router is faster in a few ways: faster WiFi connection speeds; faster access to self-hosted services such as backups enabled by gigabit ethernet (up from 100 Mbit) for the wired connection; and (probably) some faster software operations such as DNS where the previous router often seemed responsible for delays of several seconds.

The Self-Hoster Appreciates…

Where OpenWrt shines is in the features I use for self-hosting services, and how I will be able to manage it over time.

Because it’s open-source software:

reassurance that the software cannot be abandoned at the whim of some company;
strong support for open and standard and modern protocols, e.g. mesh WiFi, encrypted DNS standards, standard Unix admin tools;
likely to be upgraded to add new features, support new security measures;
I can keep my configuration if I need to buy new or different hardware, because the same software runs on many devices;
many optional add-on features contributed by community members;

Because it’s software for professionals:

full IPv6 support, alongside IPv4;
strong WiFi features, e.g. multiple networks (trusted vs. guest);
strong network protocols support, e.g. tagged VLANs, switch control protocols;
configuration stored as text, so can be managed by external tools like Ansible and version control, and re-configured from scratch by one automated script (“configuration as code”, “infrastructure as code”);

Things That Went Wrong

Bricking the device during initial installation

Part of the OpenWrt TD-W8970 installation instructions, which are in a linked forum post, advised me to use commands like “cat openwrt.image > /dev/mtdblock1” to install OpenWrt initially. What appears to have gone wrong is this did not successfully write all of the image file to the flash memory. Some blocks of flash remained blank. Then when rebooting the router, it just hung. I got in touch and was advised there are more reliable ways to do it. To recover, I had to buy a serial port to USB adapter, open up the router and solder on a serial header, and use the serial port recovery method.

Some web sites would not load

At first, a few ordinary web sites failed to load.

According to a note near the end of the user guide “Dnsmasq DHCP server” page:

“If you use Adguard DNS … you need to disable [DNS] Rebind protection… If not, you can see lot of this log in system.log, and have lag or host unreachable issue.”

"daemon.warn dnsmasq[xxx]: possible DNS-rebind attack detected: any.adserver.dns"

I have read a lot more about this issue since then, to understand it better. I changed the setting, as suggested, and everything seems to work OK now.

I wish this issue would be explained more clearly, and with references. I am still not entirely comfortable that disabling the rebind protection is the best that could be done: it seems to me it would be better if we could accept just the “0.0.0.0” responses that this DNS sends while still protecting against any other local addresses.

WiFi Would Not Connect

After a while I decided to change the WiFi channel selection from 11 to Auto. Next day, our devices would not connect. Some of them would briefly attempt to connect and immediately disconnect, while others would not even show our WiFi network in their list.

It turned out the router had switched to channel 13. From what I have been able to learn, this is a valid channel to choose, although in the USA there are restrictions on the power level on channels 12 and 13. A lot of writers strongly advise only choosing among 1, 6, and 11. The rationale for this advice seems to originate from one particular study that may not be relevant in today’s common scenarios; some writers disagree and it’s not really clear. I wonder if the problem is that the firmware in many devices may not “like” connecting to channels above 11.

Whatever the precise cause, switching back to manually selected channel 11 seems to have solved the problem.

Struggles

It was far from a breeze to install, and far from a breeze to configure.

The OpenWrt web UI (LUCI)

LUCI is still not clear and helpful, although much improved. Examples:

understanding how to set upstream DNS (on WAN interface, in LAN interface, in DHCP settings, in all of these?);
same for how to set local domain name (3 places to choose) and what the consequences are.

Poor documentation

I struggled with the OpenWrt “user manual”. For example, many of its pages say basically “help for FOO: to accomplish FOO, I pasted the following text into the config files in some unspecified version of OpenWrt,” without explaining what exactly FOO was meant to accomplish and its trade-offs and interactions.

Configuration as code

I discovered by accident that the LUCI can show the commands for the settings changes, if you click the mis-named “unsaved changes” button which appears after pressing “save”.

That’s a great start. It could be developed into something so much better, a real configuration-as-code methodology. Nowadays that should be promoted as the primary way to manage the router. Instead of just “backup” and “restore” there should be facilites like diff the current config against a backup and revert selected differences. Tools should be promoted for managing the config externally from e.g. a version control system or Ansible.

Inconsistent defaults

When LUCI writes a config section, it changes settings that the user didn’t change. It seems to have its own idea about what a default config looks like, and this is different from the default config files supplied at start-up. This makes it difficult to manage the settings in version control. These spurious changes are shown in the LUCI pending changes preview. (It would be helpful if that preview included the option to revert selected changes, although that would not go far enough.)

How it should be done: The LUCI settings should always match the text config defaults, and that should be tested. This would come naturally when adopting configuration-as-code as the primary management method.

Finding what (A)DSL settings to use

Finding settings to use for the ADSL connection was hard. My ISP PlusNet published a few basic settings (VPI/VCI, mux, user and password, etc.) but OpenWrt required other settings as well, and some of the settings didn’t exactly match.

The OpenWrt ISP Configurations page seems quite useful but says for example “Annex A, Tone A” whereas LUCI doesn’t have an option named exactly “Annex A”: its options include “Annex A+L+M (all)”, “Annex A G.992.1”, etc., and it doesn’t have an option for “Tone A” but instead “A43C+J43+A43”, “A43C+J43+A43+V43”, etc. This makes it really frustrating if one is not a DSL expert: I do not know which of the available options will work and which will not. When on my first try it would not connect (showing some sort of authentication error) I did not know which settings could possibly be the cause.

After a lot of reading and experimentation I noticed that the generated text configuration corresponding to each LUCI option gave me a strong clue: the generated config for tone “A43C+J43+A43” used the option code value “a” whereas for tone “A43C+J43+A43+V43” it used the code value “av”. That strongly suggested I should select the former. And similarly for “Annex”.

Finally I came across a small comment between two example configurations in that same page, that said I must also delete the ATM bridge that was set up by default. The LUCI description of “ATM Bridges” says, “ATM bridges expose encapsulated ethernet in AAL5 connections as virtual Linux network interfaces which can be used in conjunction with DHCP or PPP to dial into the provider network.” Not great. That didn’t help me at all.

After changing settings as best I could, and deleting that ATM bridge, it then worked.

How it should be made easier:

define a way of publishing a DSL configuration online as a structured code block (could be the OpenWrt config language, for a start);
make LUCI able to accept a whole DSL definition in a single cut-and-paste operation (a text config box);
start a database of these (encourage this to be maintained by the community; make it distributed);
add a “search in database(s)” function for these in LUCI.

By using a silo we exclude children

by Julian Foad

2021-07-16

14:03

Leave a comment on By using a silo we exclude children

Interweb, Matrix, politics

github, gitlab, Matrix, self-hosting, silo

Teckids e.V. held their annual summer camp for kids between 9 and 15 years. This year, after we started introducing Matrix and Element as a chat platform … Some got really excited that they could even change or add features to Element … Unfortunately, Element is developed on GitHub, so the potential young contributors are locked out by the exclusive Terms of Use there. We are trying to reach out to Element HQ to find a solution.
Klampfradler writing in This Week in Matrix

Ironic but all too common: a Libre/FOSS project falling foul of the very issue it is trying to overcome. For this post, however, let us not focus on this particular situation. Let us instead learn from it how we can better explain why we need Matrix and other Libre systems.

GitHub only allows users over a certain age. Initially when we adults sign up, this seems like no big deal: it sounds like quite a young age (13) and it says it’s to comply with the law of the USA, which sounds obvious and unavoidable.

This is a great example of how we restrict other people’s freedom every time we choose a silo. We start off thinking the silo is reasonably inclusive and gives enough freedom for what we need. After all, it’s one we use, so by definition it’s good enough for people like ourself. But:

everyone we want to connect with has to obey the same terms, and
we don’t have any control over those terms.

Even when it’s “our project”, we can’t choose who we are allowed to collaborate with.

What if our situation changes, perhaps a long time later? Perhaps our friend has children who under the supervision of their parents are competent and keen to code. We might ask them to use their parent’s account. GitHub’s terms forbid that too. Anyway that would mean they couldn’t keep a record of their contributions linked to their own identity.

GitHub, being a silo, restricts the freedom of its users and the freedom of our collaborators and would-be collaborators.

By contrast, when we have control of our own Libre system, such as a self-managed GitLab, then we can set our own terms and conditions.

That’s a key benefit of using a Libre system.

It’s the same in any Silo-vs-Libre domain: Zoom vs Jitsi, WhatsApp vs Matrix. When we use the silo, we restrict not only our own freedom (which we may not immediately care about) but also the freedom of those we contact. We probably care more about the freedom of those contacts who are our real-life friends, family, and colleagues, who in turn care about their own friends and family.

In trying to convince people to switch to Matrix and other open tech, it can be hard to find “selling points” that resonate with ordinary people who are not already familiar with these freedom issues. As above, they think the silo they are using gives them as much freedom as they need. This point about how it impacts those they care about is one that the general public may be able to relate to, and recognise as important.

Let’s use this point to spread the message:

By using a silo we exclude children

Matrix — Why?

by Julian Foad

2021-05-28

10:30

Matrix for …

Up-Scaling Infrastructures for Open Science — Generation R

Matrix needs: Bring Your Own Domain

by Julian Foad

2021-05-11

13:19

1 Comment on Matrix needs: Bring Your Own Domain

Interweb, Matrix, software design

8885

Owning Your Identity in matrix is currently not as cheap and easy as it should be. Hopefully this will be changing in the not too distant future.

The current options for using your own identity in Matrix are, roughly speaking:

rent a personal matrix homeserver as a managed service “in the cloud”
- around £1 a month for your own domain name, plus
- around £10 a month for the matrix service
run your own server
- around £1 a month for your own domain name; plus
- rent a virtual server from under £5 a month, or provide your own; plus
- your skill and time

The sticking point is that currently you have to run a whole matrix “homeserver” for each DNS domain name that you want to use in a matrix user id. To register an account on the matrix.org server, for example, you must use an identifier like @some-name:matrix.org which is an identifier owned by matrix.org. There is no option to Bring Your Own Domain and register yourself as @myself:myself.org on that server.

The one domain per server model works well where the users are already members of an organisation with its own domain name, such as a government, a school or a business. For the ordinary individual who wants to own their own online identity by bringing their own domain name, however, the requirement to run their own matrix homeserver is currently too onerous.

The need for one (virtual) server per domain is a limitation of the current server software. It could be lifted in future. Then we could see commercial services offering Bring Your Own Domain accounts on their servers. This would be cheaper to run than a server per domain, and would bring a useful hybrid model of:

terms and conditions set by the service provider; with
self-ownership of one’s matrix account’s identity, giving the ability to move one’s account to another service provider with different TOCs, (or, in the extreme, self-hosting) without any loss of service or contacts

The matrix community needs to break that barrier somehow. Options include:

making tiny simple homeservers so it’s no burden for each user to run their own;
- needed also for p2p Matrix, and beginning to happen with Dendrite
making homeservers that offer a Bring Your Own Domain option.

Besides Dendrite, the Matrix community has some other small-footprint homeservers under development which may be ready for use within the next year or so. I hope we will see Bring Your Own Domain (aka multi-tenant) capability being developed in at least one of this new generation of homeservers.

The good news is the options are looking likely to open up. The crucial fact is that you can own your identity in matrix, already.

Synapse issue tracker: Multi-domain support #8885

Gitter now speaks Matrix!

by Julian Foad

2020-12-09

14:53

Leave a comment on Gitter now speaks Matrix!

Interweb, Matrix

Matrix, OpenSource

Great news: Gitter, a chat system for gitlab and github projects, now interoperates with Matrix.

https://matrix.org/blog/2020/12/07/gitter-now-speaks-matrix

“Open standards-based interoperable communication is on the rise again.”

Decoupling Identity in Matrix

by Julian Foad

2020-04-23

20:00

Leave a comment on Decoupling Identity in Matrix

-, Interweb, Matrix

Matrix, OpenSource, self-hosting

For individuals, Matrix's identity scheme creates lock-in.
How to fix?

I love Matrix. I think it’s the way forward for libre/open (as in freedom) personal communications, with a real chance to free users from the lock-in of popular silo messaging systems like Whatsit, Facepalm and Twiddle. I run all my own messaging (except email) through Matrix, with bridges to the silos that my friends still use as well as SMS and IRC.

At present, unfortunately, there is a big obstacle to me recommending any friend or family member to sign up to Matrix: identity and server lock-in.

An Open system with lock-in? Ugh. What went wrong?

To use a silo, you register an account and either you are identified by your telephone number or you choose a username. (You can then set some account options, usually including a “display name” which you can change from time to time.) Now, what if at some point you dislike that silo’s rules or advertising or charging? You’re stuck. They deliberately designed the system so that nobody has any options other than continue or quit.

To use Matrix, you register an account on a server. You first need to choose a server, which is identified by its Internet domain name such as matrix.org, or mozilla.org, or my-own-server.my-name.me if you run your own server. You can find out which servers are available for public use. Some are free of charge and others require payment, similar to email services. Having chosen a server, you pick a username and are then identified globally as @username:servername . (You can also choose a display name.)

Matrix right now is great for an organization: running their own server on their own domain, they control their own rules and namespace for users, rooms and groups.

If you are a normal person, your default option is to register a username on ‘matrix.org‘. (In principle there will be other public servers but there are hardly any so far.) Then, that username is tied to that server forever, or at least until Matrix developers invent a way out.

This lock-in is different from a silo. At least with Matrix you can create a new account on another server, to get away if you don’t like the old one. What you can’t do (yet) is migrate your old account to the new one. Not in any way. See “Account Migration” below.

Bring Your Own Domain Name

One way to mitigate the account migration problem is to register an account under a server domain name that you control.

The point is, then, the user controls their own domain name registration, which is directly registered with a domain registrar, outside the control of and Matrix or other service provider. The user can keep their own domain and have it served by a new server in the future if the current server becomes unsuitable or unavailable.

How feasible is this, today?

A geek with time and skills can register a domain name and run their own server.
A person with some time and effort and money can register a domain name and pay for a hosted matrix server. The cost and effort is broadly similar to setting up a new phone or internet or TV service. It does require some investment of thought and learning what it’s all about.
A normal Whatsit user is used to “free and easy”, and there is currently no such option for them.

Hosted servers come with significant limitations on customizing your server. For example, on modular.im (currently the main hosting option), AFAIK you cannot run the Whatsit bridge.

What can we do to improve things for the normal user?

make it cheap (not necessarily free)
- Build a server that can serve lots of different people’s personal-domain user accounts. (This may be called a “multi-tenant” server design.) @mfilipe:matrix.org mentioned this today on #matrix-dev:matrix.org.
- Spread the word that it’s sensible to pay for a service so that you are not the product being sold, unlike the free silos.
make it easy
- Build services in which a new user can set up a domain name and a matrix server or account at that domain, and pay for both with one payment. (Major providers of some services like email offer this.)
- For people migrating from a specific silo, offer ready-to-use setups (bridging) and messaging (intro, and suggestions for how to tell the silo friends about it) that are customised for that case.
- Make it easier for geeks to run matrix servers for their friends and family.

Who should be doing this? Not necessarily the Matrix Foundation or New Vector (who make Riot and Modular.im among other Matrix things). They have limited resources and their own priorities. It’s an open-source system so anyone wanting these things should get involved and start making them.

Good places to discuss and get involved in the self-hosting side include #matrix-docker-ansible-deploy:devture.com and #matrix-self-host-onboarding:chat.weho.st .

Account Migration

It would be useful to be able to migrate an old account to a new one in ways like:

forward messages to the new address
inform all contacts of the new address
set up an auto-reply
copy account settings
copy message history
copy a list of contacts

I was thinking about what is possible in email, and what regrettably isn’t available. Migrating an email account is not at all simple, but most of the mini-features above are possible to some extent. One thing regrettably missing in the email system is a way to automatically inform senders to an old account that they should update your address and re-send to a new account. (Like an HTTP “redirect”.)

It would be useful to develop those kinds of mini-features for making the transition to a new matrix account smoother. That might be a feasible short-term mitigation.

However, there is a better long term solution: decoupling accounts from identity.

Decoupling Identity

[TODO: Write about decoupling identity.]

Jitsi-Meet: Excuse Me, Who Are You?

by Julian Foad

2020-03-26

12:31

1 Comment on Jitsi-Meet: Excuse Me, Who Are You?

Interweb, software design

jitsi-meet

I fired up https://meet.jit.si/ just to explore how it works. Following the hint, I chose a simple phrase for the meeting name, something like “MyFamily”, and hit “Go”.

What did I expect?

A blank meeting screen.

What did I see?

Six elderly people whom I don’t know, talking to each other, looking surprised to see me and saying “Oh, who is that?!”

Walking In on Strangers

I left the meeting after a few seconds. Perhaps they worked out what happened and how they could change their settings to stop strangers intruding. Perhaps they were worried. Perhaps I could have gone back in and explained that I walked in on them unintentionally and meant no harm. Perhaps.

There is a UI/UX problem here. Jitsi-Meet is insecure by default. Certainly it’s possible to use it in a more secure way, but that is not good enough.

The UI here is supposed to suggest that I can use a randomly generated phrase as a meeting name. The designers clearly intended that as a security measure: a sufficiently unguessable name is secure against accidental visits, and forms a part of the security measures against intentional attacks. I know that from my background knowledge of security practices and software design. From further background knowledge and exploration, I also know that I can press “GO” to accept the suggestion. But still I fell into this trap.

As a new user, it looks like I need to enter something, and I would expect to enter a simple and meaningful name, and would absolutely not expect that doing so would immediately remove the expected privacy.

Filed as a bug: https://github.com/jitsi/jitsi-meet/issues/5407 “Choosing a meeting name is insecure by default”

## Description
In real life just now, I chose a simple meeting name, something like "MyFamily" (but not that) and hit "Go". I saw and heard six elderly people whom I don’t know, talking to each other, looking surprised to see me and saying “Oh, who is that?!”

Their privacy was violated to me, and mine to them.

This is insecure by default, and inconsistent with claiming "security".

Reported here: https://blog.foad.me.uk/2020/03/26/jitsi-meet-excuse-me-who-are-you/

---

## Current behavior

On entering a simple meeting name there is a real chance of walking in on someone else's meeting, uninvited.

---

## Expected Behavior

On entering a simple meeting name, there should be some protections in place to prevent walking in on someone else's meeting, uninvited.

---

## Possible Solutions
There are several possible solutions and measures, including...
* explain about the random phrase and its security implications and make accepting the suggested phrase a more discoverable option;
* have the "create meeting" UI user choose whether they want a public/open meeting or a private/closed meeting;
* if a user-specified meeting name is chosen for a new meeting, then very strongly encourage setting a password (unless explicitly chose to start a public meeting);
* when "start a new meeting" UI leads to joining an existing meeting, that is a violation of expectation, so interpose a second step, e.g. "this already exists; try to join it or choose another name?"
* interpose a "new user is knocking on your door" step whereby existing participants have to explicitly accept the new user;
etc.

---

## Steps to reproduce
* one user uses the "Start a new meeting" UI using a common phrase like "MyFamily" as the name;
* another, supposedly unrelated user uses the "Start a new meeting" using the same common phrase like "MyFamily" as the name;
* see that they join, without protection;

---

# Environment details
Using https://meet.jit.si/ on 2020-03-26.

Matrix Bridges Working Together

by Julian Foad

2020-02-06

13:41

Leave a comment on Matrix Bridges Working Together

Interweb, software design

Matrix

I am testing SmsMatrix, an SMS-to-Matrix bridge on Android. It is one of many Matrix bridges. But this post is not mainly about SmsMatrix, it is mainly the start of a train of thought about how we can manage communications with one person bridged over different channels.

This is a write-up of thoughts I first posted to the SmsMatrix discussion room, which would be better continued on #bridges:matrix.org.

Let’s say my friend has telephone number 123456 and is in my phone’s address book as “Ray”.

When my phone receives an SMS from Ray, SMSMatrix sees that message arrive in the Android SMS subsystem, and (if it hasn’t previously done so) it creates a new Matrix direct-chat room which it will associate with the SMS sender’s phone number, and invites me to join the room.

The other participant in this room is the bridge bot. Its username is @sms-bot:my_hs, and it sets its own display name to “Ray”. I assume that is a per-room display name, so it will show as a different name in each SMS conversation. The bot does not set a room name, and so Riot displays it as “Ray” (the other participant’s display name), and the bot sets the room “topic” field to “123456”, which Riot displays next to the room name. The idea is that in the user interface it looks like I am having a chat with “Ray” and it works like I am having a chat with Ray.

In Riot-web’s message view, the sender of this first SMS message shows as (precisely) “sms-bot”. The display name of @sms-bot does seem to be correctly set to “Ray” — I can see that in the “users” panel. Perhaps that’s a Riot bug that it doesn’t always display the user’s display name.

This bot works without admin privileges. It can do so because it doesn’t try to create and control a new Matrix user for each SMS sender, nor does it try to find and puppet an existing “real” matrix user account corresponding to the sender.

I wonder if it would be nice to upgrade it to a puppeting bridge if I one day have enough time to devote to it. I’m wondering exactly what we could achieve if we did so.

I also have some other bridges, and let’s say Ray also has her own Matrix account on her own homeserver, where she calls herself by her full name “Radiator”. Now I have Ray’s messages coming in to me in these Matrix rooms/users:

@sms-bot:my_hs “Ray”
- in room (unnamed) displayed as “Ray 123456”;
@whatsapp_123456:my_hs “Ray (WA)”
- in room (unnamed) displayed as “Ray (WA) WhatsApp private chat”;
@telegram_123456:my_hs “Ray (TG)”
- in room (unnamed) displayed as “Ray (TG)”;
@ray:rays_hs “Radiator”, Ray’s real Matrix user account
- in room named “Ray”

The mautrix-telegram and mautrix-whatsapp bridges each create a new user id for Ray in their own namespace of matrix user-ids, which is different from the SMS bridge.

I am interested in researching how we could improve this situation. Would we want to make all bridges to the same person somehow bridge to the same Matrix user?

@pwr22 asked, “How would the bridge know which method to use when sending a message?” That’s a good UX question. The first thought coming to my mind is that’s easy if we dedicate a separate room for each bridge for that user. (Ray’s SMS chat room, Ray’s Telegram chat room, …).

“In that case what do we gain from having one puppet user instead of separate ones per service?” We gain things like,

“search for all messages from @ray [containing the word ‘zoo’]”,
“search for the rooms that @ray is in”.

I expect users would want to choose, by their own preferences and perhaps per contact, whether a contact’s messages are all grouped in the same room or a separate room per bridge.

If I want to group all Ray’s messages into one room, then the sending UX issue must be solved a different way. Perhaps the client would allow me to set a preference. (That could be stored as client-managed metadata, like how Riot stores some settings in “account data” under “im.vector.riot.xxx” keys.) And/or there could be bot commands so I can tell a bridge "!sms-bot: enable [or disable] sending outgoing messages in this room through SMS".

In response to my thought about making all bridges bridge to the same Matrix user, @swedneck replied, “i’m not sure that’d make sense. i would probably have all those bridges in the same room if possible. you’d probably want his bots to ignore each other though.” So:

one room (“My direct chat room with Ray”) containing
- myself,
- the virtual user @sms-bot:my_hs (“Ray”),
- the virtual user @telegram_123456:my_hs (“Ray”),
- …, and
- the Matrix user @ray:rays_hs (“Radiator”).

I see how that would work well for incoming messages: I would be able to see which bridge each message came through, if I wanted, while keeping all the display names the same if I mostly didn’t care to see the difference.

We would need a way to make the bots know about each other. When Ray sends me a message through one bridge, we (probably) don’t want all the other bridges to send him copies of it. They’re all “my” bots running against my Matrix home server (HS), so I could add something to their configs easily enough. Alternatively, we could define common metadata that they could each set, so they could detect the situation automatically. The latter would be more flexible, in case some of them are not configured directly alongside my HS.

What about when I send messages to Ray? If I just type without a mention, then the same applies as I said before: some way of configuring in advance which bridge(s) is/are to send it.

When I Mention Ray, there are different matrix usernames I might mention, depending how I do it: if I click on one of the received msgs, I might select a bridge username that isn’t the currently configured preferred send method; how should that all work?

Maybe bridged usernames should be specially treated: all treated as something like aliases, and all resolved to the same single “real” matrix username?

Needs more thought.

MSC2346: Bridge information state event

As part of a solution, we will need a mechanism to allow multiple bridges in a room to communicate with each other. This proposal may achieve it:

MSC2346: Bridge information state event (discussion | draft text)

Going to build a Matrix

by Julian Foad

2020-02-05

16:29

Leave a comment on Going to build a Matrix

Interweb, politics, software, software design

Matrix

I am leaving my latest job and moving on to my new passion, Matrix.

Recently I have become passionate about the need for modern communications systems that are Open in the sense of freedom to talk to anyone. The currently dominant silos like WhatsApp only let me talk to the friends who are willing and able to subscribe to that company’s terms and restrictions, with no way to get around them when they decide to display advertising to me or stop supporting my mother’s not-terribly-old iPhone. To me, that is like some historical feudal system in which I live as a tenant and I must obtain agreement from the lord of the estate if I want to invite any of my friends to visit me. We need and deserve better than that: an Open way to communicate to our friends and business contacts. Email served that role for the first part of the 21st century, and Matrix now serves that role for the era of instant messaging.

My software development in recent years has been mostly on the open source Subversion version-control system, and I have particularly enjoyed helping to create something so widely used and appreciated. Nowadays its popularity is eclipsed by Git in small to medium sized projects, while Subversion still enjoys a strong following in certain fields such as games development due to its strengths in versioning large data sets and simplicity of usage. Participating in the development of Open Source software has given me the greatest satisfaction in my professional life, and I intend to keep it that way. That is why developing the Matrix communication system is so exciting.

p.s. I am still contracting on Subversion support work, so get in touch if you need any bug fixing or problem diagnosis.

Just Another Proprietary Service

by Julian Foad

2020-01-25

12:00

Leave a comment on Just Another Proprietary Service

Interweb, politics, software

OpenSource

One of my favourite open source institutions is considering replacing their use of an open source tool with a proprietary service “donated” for “free” by its vendor.

It’s time I just said what I think: Encouraging open-source contributors to adopt another proprietary sponsored service is against the principles I want the institution to uphold.

Pootle is an open source tool that assists with human-language translation. Contributors to a project use it to write and contribute translations of open source software into their local language. As with many open source projects, it is under-resourced. Proprietary services look more attractive, if we look as measures such as the immediate user experience and the maintenance burden.

Yet, when we ask contributors to use any “donated” proprietary service, we make those users and the FOSS community bear its cost in the domains of lock-in and advertising. I am disappointed to hear that my favourite institution is seriously considering this. (This is not about translation tools specifically; I feel the same about all the user-facing tools and services we use.)

Don’t get me wrong: I am not suggesting this goes against the institution’s policies, and of course there are hard-to-ignore benefits to choosing a proprietary service. I can’t imagine exactly how much pain it is trying to maintain this Pootle instance. On the other hand I do know first-hand the pain of maintaining a lot of other FOSS that I insist on using myself, and I sometimes wonder if I’d like to switch to a commercial this-or-that. At those times I remember how much I value upholding the open source principles, and I choose to stick with what is sometimes less immediately convenient but ultimately more rewarding.

Time after time I observe the FOSS community suffering from getting sucked in to the traps of commercial interest like this. A FOSS project chooses to use a commercial service for its own convenience, and by doing so it feeds the commercial service, increasing familiarity with it and talk about it (forms of lock-in and advertising), decreasing the development going in to competing FOSS services, making it more likely that others will follow. I observe FOSS people tending to concentrate on the short-term benefit to their own project in isolation, even when they are peripherally aware that their field would benefit in the long run from working together with others on the tools and services that they all need.

What could be the cultural process led the institution to this place?

“Current tools are poor… Let’s try another ‘free’ service to quickly overcome our problem.”

I feel like there’s a cultural step missing there. Where is the step that says,

“We are hundreds of open source developers needing a good translation service. Other open source developers are trying to develop good translation services for people like us. What a great fit! Let’s work together!”?

I would rather join and contribute to a new project group whose purpose is to provide an Open service (in this case for translation) for the institution’s projects to use, doing whatever development, customization, maintenance and IT infra work it needs depending on the state of the available open solutions.

To fill in the missing step, I feel we need to introduce a culture of speaking out at a membership level to say, “Here’s a challenge; who can volunteer to form a group to solve it?” and encouraging members to think of working together on communal service provision projects as a normal part of the institution’s activity.

By working closely with the FOSS people who want to provide a service that we need, our contribution to the upstream software projects would benefit others for the public good, and more generally we would foster mutually beneficial growth and normalization of adoption of FOSS technologies.

I’m not saying it isn’t hard to get the necessary contribution level to make a difference, or that folks haven’t tried before. (Some communal service projects are used in this institution, but they tend to be small scale in-house projects rather than collaborations with other FOSS projects.)

How can we drum up support for doing it the FOSS way?