Five developers met up in Aachen last week and had a productive and enjoyable time, reviewing the state of Apache Subversion, coming up with solutions, and discussing ideas for new directions. Here I will relate just a few of them.
Stefan Hett kindly organized this event in the Mercure hotel, and my boss at Assembla kindly agreed to sponsor the event by covering the conference room costs and daytime food and drinks.
We enjoyed evening meals at a different restaurant each night. On Wednesday, Stefan arranged for us to play a couple of games of pool afterwards, in which I happily report that Bert and myself found ourselves on the winning team.
Many topics came up in discussion over the course of the four days. We jotted down the topic headings as we thought of them, and transferred them to the Hackathon wiki page along with some notes.
WC Shape-Changing (View Specs)
Stefan works in a games company. Alongside his development work he also manages their version control needs. The working copies contain some large third-party subsystems that are only needed on certain platforms, and they would love to avoid the overhead of having the unnecessary parts checked out. They have considered using sparse working copies, or externals, or separate branches for the different platforms, but found none of the features are sufficiently capable for their needs.
Johan’s team on the other hand makes extensive use of sparse working copies. Johan works in the IT department of a group of hospitals, and he too looks after Subversion as a small part of his job. The team works on lots of projects, all versioned in Subversion in a “large trunk with all projects and modules underneath”.
This led to an animated discussion about “view specs” and ways of managing the sparseness.
One of the first things they would find really useful is to export a sparseness description from one WC and change another existing WC to match it, or check out a new WC with that same “shape”. The existing command-line svn tools are weak in this area, making scripted solutions inefficient. The only way to view which directories are set to a non-infinite depth is “svn info -R | grep ^Depth:”.
Bert suggested he could implement the “export” side quite easily by harnessing the internal “reporter” interface and expressing the report in a simple text format. The reporter normally tells the repository what “shape” the WC is, including sparseness but also mixed revisions and switched subtrees, so the repository can work out what diffs to send back to the WC to update it. Bert duly went ahead and did it the next day, printing the result in the form of a series of “svn checkout” and “svn update” commands with the “–set-depth” option that, if executed, would check out another working copy with the same shape.
There is more work needed on the “import” side, however, as executing these “–set-depth” commands directly can be slow and inefficient. If for example the WC root is described as depth “infinity” with all but one subdirectory set as “excluded”, then the script will first fetch the whole tree and then throw away each excluded subtree. An optimal solution will require enhancements to the working copy library code. Then we could implement variants of the “checkout” and “update” commands that take a viewspec file as input.
Shelving
Johan mentioned his colleagues would find “shelving” useful, which is encouraging for me in developing it. Stefan also mentioned a colleague who had come from a Perforce environment and really missed the shelving feature when moving to Subversion.
We discussed shelving and how it should evolve. I proposed that we need to be able to reference a shelved change in a common way, and do more or less all the things with that we can do with a regular modification in the working copy, including committing it directly to the repository, exporting it as a patch file, and viewing it in a side-by-side diff viewer. On top of that, we might want to look at shelving on the repository side too, as has been discussed recently on the dev mailing list, in which case we want to transfer to and from there and commit directly from there. Also we should support, at some level, sending a change to an external code review system such as Rietveld or those provided by Assembla and RhodeCode. For all of this to be reasonable, we need a degree of commonality in the syntax for referring to one of these “change sets”, whether it currently resides in the WC or a local shelf or a server shelf or a committed revision. I looked at how we could extend existing commands for moving change-sets around, such as using “svn commit” to commit a shelved change, so that we do not end up inventing a different new command for every combination of source location and target.
Then we need a “patch” format for transferring these changes between Subversion and external systems. Subversion currently supports only a limited extension to the traditional “unified diff” patch format, adding support for versioned properties and a little bit of support for creating and deleting and copying files. The current version does not support changes to directories, nor a log message and other revision properties, nor does it properly support binary files, copies and moves, although a little support exists for these in Subversion’s “git diff” format.
Releasing Subversion 1.10
The Subversion 1.10 release has been stalled for far too long. High on our priority list was asking ourselves what is to be done.
We reviewed the readiness and decided none of the technical issues are blocking the release process. There are a few tweaks we need; for example, we are not ready to promise future compatibility for the new shelving feature so we need to mark it as “experimental”. But what we need to get the release moving again is a Release Manager to drive it. Stefan Sperling drove some alpha releases at the start of the year, but he and the other volunteers in the project are short of time and there was a gap where there was more or less no-one being paid to work on Subversion either.
So I checked with my bosses and agreed that I can offer to do this in my work time, as getting 1.10 out the door is good for Assembla as well as everyone else in the community. Among other things, it contains some really good server side enhancements.