This is one of the things I want to be doing: running a tiny social network service for a few family members. This guide goes from questions like “Why should I?” and “Should I?”, through to practical advice for those with programming or I.T. expertise and for those without.
–> Run Your Own Social (runyourown.social)
How to run a small social network site for your friends
Hooray! Hooray! Matrix meets Mozilla today!
Matrix says:
https://matrix.org/blog/2019/12/19/welcoming-mozilla-to-matrix/
Mozilla says:
https://discourse.mozilla.org/t/synchronous-messaging-at-mozilla-the-decision/50620
How can I test a release process?
I’m improving the scripting of the Apache Subversion release process. Preparing and publishing a release involves things like making a branch in the source repository, updating web pages, updating buildbot configurations, and (not yet automated) sending emails.
When I’m modifying the release scripts, I needed a way to test without making bogus “live” releases.
The source code, web pages, and buildbot config are all stored in Subversion repositories, so that was a good place to start.
Where I’ve got to so far:
My initial script to set up local dummy repos is ugly and slow. But that’s OK; the important thing is it’s enough to proceed with verifying the automation while I improve it.
#!/bin/bash
# Make a local svn repo suitable for testing our release procedures
# ('release.py' etc.)
set -e
SRC_SVN_REPO_URL=https://svn.apache.org/repos/asf
SRC_DIST_REPO_URL=https://dist.apache.org/repos/dist
SRC_TRUNK_WC=$HOME/src/subversion-c
SRC_BRANCHES_WC=$HOME/src/svn/branches
SRC_SITE_WC=$HOME/src/svn/site
DUMMY_REPOS_DIR=/opt/svn/dummy-asf-repos
DUMMY_DIST_REPO_DIR=$DUMMY_REPOS_DIR/dist-repo
DUMMY_DIST_REPO_URL=file://$DUMMY_DIST_REPO_DIR
DUMMY_SVN_REPO_DIR=$DUMMY_REPOS_DIR/svn-repo
DUMMY_SVN_REPO_URL=file://$DUMMY_SVN_REPO_DIR
SVN_WC_DIR=$DUMMY_REPOS_DIR/svn-wc
#DIST_WC_DIR=$DUMMY_REPOS_DIR/dist-wc
# export_with_props SRC_WC DST_WC
function export_with_props() {
SRC_WC="$1"
DST_WC="$2"
svn export $SRC_WC $DST_WC
svn add --force --no-auto-props $DST_WC
# remove automatically added mime-type props as some of them are not an exact replica
svn pd -R svn:mime-type $DST_WC
# add an exact replica of source props
PROPS_TO_ADD=$PWD/props-to-add
(cd $SRC_WC && svn diff --properties-only --old=^/@0 --new=.) > $PROPS_TO_ADD
(cd $DST_WC && svn patch $PROPS_TO_ADD)
# rm props-to-del props-to-add
}
set -x
mkdir "$DUMMY_REPOS_DIR"
### the 'dist.a.o' repo ###
if ! [ -d $DUMMY_DIST_REPO_DIR ]; then
# create a repo
svnadmin create $DUMMY_DIST_REPO_DIR
# create skeleton dirs
svn -m "Init" mkdir --parents $DUMMY_DIST_REPO_URL/{dev,release}/subversion
fi
### the 'svn.a.o' repo
if ! [ -d $DUMMY_SVN_REPO_DIR ]; then
# create a repo
svnadmin create $DUMMY_SVN_REPO_DIR
# create skeleton dirs
svn -m "Init" mkdir --parents $DUMMY_SVN_REPO_URL/subversion/{trunk,tags,branches,site}
# check out
svn co $DUMMY_SVN_REPO_URL $SVN_WC_DIR
cd $SVN_WC_DIR
# populate trunk
rmdir subversion/trunk
export_with_props $SRC_TRUNK_WC subversion/trunk
svn revert -R subversion/trunk/{contrib,notes}/*
svn ci -m "Add trunk" subversion/trunk
# populate site
export_with_props $SRC_SITE_WC/tools subversion/site/tools
export_with_props $SRC_SITE_WC/publish subversion/site/publish
svn revert -R subversion/site/publish/docs/{api,javahl}/*
svn ci -m "Add site" subversion/site
svn cp -m "Add site staging branch" ^/subversion/site/publish ^/subversion/site/staging
# a mod on trunk
echo hello > subversion/trunk/hello.txt
svn add subversion/trunk/hello.txt
svn ci -m "Trunk mod." subversion/trunk/hello.txt
# make 1.13.x branch
svn cp -m "Branch 1.13.x" ^/subversion/trunk ^/subversion/branches/1.13.x
svn up --depth=immediates subversion/branches/1.13.x/
sed 's/1\.12\.[0-9]*/1.13.0/' < $SRC_BRANCHES_WC/1.12.x/STATUS > subversion/branches/1.13.x/STATUS
svn add subversion/branches/1.13.x/STATUS
svn ci -m "Add 'STATUS' file."
fi
Make the dummy repos:
$ cd /opt/svn
$ rm -rf dummy-asf-repos/ && svn-mk-dummy-asf-repo.sh
[...]Tell ‘release.py’ where the dummy repositories are:
$ export SVN_RELEASE_SVN_REPOS=file:///opt/svn/dummy-asf-repos/svn-repo/subversion
$ export SVN_RELEASE_DIST_REPOS=file:///opt/svn/dummy-asf-repos/dist-repo
$ export SVN_RELEASE_BUILDBOT_REPOS=file:///opt/svn/dummy-asf-repos/buildbot-repoStart testing:
$ mkdir -p test-releasing/ && cd test-releasing/
$ release.py build-env 1.13.0-alpha1
INFO:root:Creating release environment
[...]
$ ./release.py roll 1.13.0-alpha1 7
INFO:root:Rolling release 1.13.0-alpha1 from branch branches/1.13.x@7
[...]
$ ./release.py sign-candidates 1.13.0-alpha1
INFO:root:Signing /opt/svn/test-releasing/deploy/subversion-1.13.0-alpha1.zip
[...]
$ ./release.py create-tag 1.13.0-alpha1 7
INFO:root:Creating tag for 1.13.0-alpha1
[...]
$ ./release.py post-candidates 1.13.0-alpha1
INFO:root:Importing tarballs to file:///opt/svn/dummy-asf-repos/dist-repo/dev/subversion
[...]
# ... and so onInspect what the script changed in the dummy repositories:
$ svn log -v --limit=1 $SVN_RELEASE_SVN_REPOS
r8 | julianfoad | 2019-10-02 15:40:18 +0100 (Wed, 02 Oct 2019) | 1 line
Changed paths:
A /subversion/tags/1.13.0-alpha1 (from /subversion/branches/1.13.x:7)
M /subversion/tags/1.13.0-alpha1/subversion/include/svn_version.h
Tagging release 1.13.0-alpha1
------------------------------------------------------------------------
$ svn log -v --limit=1 $SVN_RELEASE_DIST_REPOS
r2 | julianfoad | 2019-10-02 15:40:56 +0100 (Wed, 02 Oct 2019) | 1 line
Changed paths:
A /dev/subversion/subversion-1.13.0-alpha1.tar.bz2
A /dev/subversion/subversion-1.13.0-alpha1.tar.bz2.asc
A /dev/subversion/subversion-1.13.0-alpha1.tar.bz2.sha512
A /dev/subversion/subversion-1.13.0-alpha1.tar.gz
A /dev/subversion/subversion-1.13.0-alpha1.tar.gz.asc
A /dev/subversion/subversion-1.13.0-alpha1.tar.gz.sha512
A /dev/subversion/subversion-1.13.0-alpha1.zip
A /dev/subversion/subversion-1.13.0-alpha1.zip.asc
A /dev/subversion/subversion-1.13.0-alpha1.zip.sha512
A /dev/subversion/svn_version.h.dist-1.13.0-alpha1
Add Subversion 1.13.0-alpha1 candidate release artifacts
------------------------------------------------------------------------and so on.
The Subversion release process had too many manual steps in it.
The manual effort required was becoming a burden, now we’re doing a “regular” release every six months, as well as a source of inconsistency and errors.
As part of an effort to improve stability and availability of Subversion releases, I’m scripting some of the manual steps to bring us closer to automated releases.
The headings below link to the descriptions in our community guide. An automated step or set of steps is indicated as “release.py <command>“; the rest are manual. Steps in strikethrough were previously manual, now automated.
Creating a new minor release branch
Rolling a release (repeat for both RC and final release)
The actual releasing (repeat for both RC and final release)
So quite a bit still to do…
Today I pushed the final “go” button, announcing Apache Subversion 1.12.2, 1.10.6, and 1.9.12. Thank-you, everyone who helped with reporting, developing, reviewing and testing the changes that went into these releases.
Release-managing was a bit of a struggle this time around. While Subversion has moved into the “mature” phase of its life with possiblly the most users ever, the number of active contributors has been dwindling, which is understandable. Recently it reached the point where the development community could no longer meet its self-imposed requirements for three members to approve each back-ported change. As the release was stalled even though we had fixes waiting that were otherwise ready to release, we — those of us who are still around to discuss such things — agreed to lower those requirements as a pragmatic move. Then I was able to merge in all the outstanding fixes in the queue. As a result, these releases have a few more changes in than they otherwise would have, albeit small changes as they are only patch releases.
Looking to future releases, it seems to me one of the most important issues we need to address at this stage of the project’s life is to streamline the release process, in two different ways — from the inside, making it easier for us to produce a release — and more importantly from the outside perspective, making it easier for end users and packagers to receive an upgrade. We are currently discussing this and other aspects of Subversion’s “community health”.
And when I say “we” — the Subversion project — that’s you too, if you want to be included. It might surprise some users to hear that there is no team dedicated to producing Subversion for you. There is just me, using a bit of my employer‘s time, and a very few others using a bit of their personal time, and … potentially you or a bit of your employer’s time? If you care about Subversion continuing to be available, and you can offer any kind of help — perhaps with building and releasing it in one of the many forms it’s distributed these days — you could make all the difference.
Could you be a part of true, free, libre, open source software development? Please come and say hello in the #svn channel (on IRC or on Matrix) or on the mailing lists.
Introducing Matrix 1.0 and the Matrix.org Foundation
Hurray!
My homeserver is up.
Yours futuristically,
— @julian:foad.me.uk
Matrix announced today the Final countdown to 1.0
Hurray!
I just got around to reading mhoye’s article The Evolution Of Open and it challenged and changed my perspective in a good way, relevant to how Matrix fits in to the quest for “open” communications systems for the benefit of society as a whole.
I was focused on decentralization tech, empowering users to technically control their identity (see: “Self-Sovereign Identity“) as the primary need to concentrate on.
After reading that post, I’m seeing how empowering users needs to be at a much more social level, having communities that can technically communicate with anyone but practically need to set loose boundaries in variety of ways, having the personal and collective ability to “tune out” messages and people and entire communities that are abusive or just uninteresting to them.
It’s not that we shouldn’t all be open to hearing view from people who are different from us, it’s that if we allow everyone on the planet to shout into our ears at an equal volume, that’s not effective communication, it’s no use at all. (And spammers use spam bots to amplify their unwanted inputs.) Rather, we need ways to go selectively and occasionally to hear different views, moderated by people we trust.
In the #synchronicity channel the Matrix founder Matthew expressed reservations about how, in the article, federated systems are positioned as detrimental to providing a safe “open space” for all participants to be free from online abuses. The author writes, “I believe that federated infrastructure – that is, a focus on distributed and resilient services – is a poor substitute for an accountable infrastructure that prioritizes a distributed and healthy community”. Matthew countered that federated systems “have an existential reason to fix” that problem. Big-corp silos have financial reasons to address the problems of safe inclusion, and indeed have “health” and “reputation” teams dedicated to it, but they are only ever going to do so in profitable ways, which could be insidiously worse. Maybe the perspective expressed was what’s available today rather than future-looking. Today, the “best” moderation is achieved in some profitable silos. For the future, it seems obvious to me that federation is part of the solution, even though it poses challenges, and that point seems to be missed.
After reading and pondering these issues, I raise the priority of my ability as a user to control the rules that my federated messaging server applies to me, and to migrate to one that suits me better, above the technical ability to run my own instance of such a server.
Mozilla, bastion of the Open Internet, intends to switch its communications from the ancient and very open IRC, to … well, something better, as Mike Hoye explains in the articles Synchronous Text and Goals And Constraints.
My concern is that the first article suggests an intention to look for a complete packaged solution — “We are not rolling our own”. it reads to me like the hope is that not only is the software packaged, but also a hosting service and also the management of the service, and that brings to my mind such things as the handling of complaints and legal bureaucracy. Such a solution is likely to be commercial and closed, because commerce has found there is a great demand for such systems and developed some.
The state of open communication systems is fragmented and rough-edged in comparison, even though great progress has been made on some fronts. Perhaps open communication systems are under-valued by the governments, universities and open-source organizations that might have power to resource their development. They might see that email works very well and many other channels are available and think that the commercial market is doing a good job in developing new alternatives and doing society a favour in making them available for use for no charge. Perhaps the decision makers have not the background, the insight and the careful consideration it takes to be able to see the other side of the equation. Anyway, the result is that the development of open systems hasn’t really been pushed by larger society, and it seems to me only now in the last few years are we (society) starting to understand an inkling what we are missing because of that.
As someone who feels aligned with Mozilla’s view of the social values and ethics of an open internet, I have been quietly wishing that they and other organizations will increasingly help push open systems into the mainstream. The possibility of them adopting a proprietary communication system quite upsets me.
I admire the foresight of the French government for boldly choosing Matrix for their chat system.
I took the opportunity to drop some hints in this direction in my response today to Mozilla’s “Reimagine the Web” Survey.
The specific issue of replacing Mozilla’s main real-time communication network is being discussed in a room/channel named Synchronicity on Matrix and on IRC, and they plan to start standing up candidate solutions and evaluating them soon. I don’t have much available hobby time to participate, but I hope to throw a few tuits into that space if I can.
My responses to Mozilla’s “Reimagine the Web” Survey; your responses are needed by 14th May.