Too Many Password Managers

First I had too many passwords. (And still do.) But on top of that I now have too many password managers.

My web browser, Firefox, remembers web site passwords for me, which is very helpful indeed. (On a Mac, Safari does that too and additionally offers to generate random passwords for new sites, which is even better.)

I use KeePassX as my primary password manager. As well as username and password and web site URL, I find it useful to store the associated email address and some notes, e.g. “Dormant account: I requested to close it on 2016-01-01 but sign-in still works on 2017-01-01” or “Tried to update to my new email address on 2017-01-01 but the site refused: need to contact customer support”.

Thirdly, my operating system Ubuntu has its own “Passwords and Keys” app which stores passwords and keys on behalf of other applications.

Problems:

  • it’s a nuisance having to copy data between them (Firefox <-> KeePassX) when creating or changing a password
  • backup is complicated: I hardly know where all three stores are, or whether they are all uniformly backed up
  • three different master passwords to manage (no, don’t suggest what you’re thinking :-), and not being sure whether all three stores are similarly and sufficiently secured

Can I please have fewer password managers? OK I chose to add KeePassX into the mix, but Firefox forced its own one on me whereas I would like it to use either the operating system’s password manager or one that I designate.

Decentralized Web Summit — Lightning Talks

Here is one of the best resources I have found so far to hear about the exciting new technologies which may underpin the next phase of the World Wide Web.

Go to the Decentralized Web Summit home page, scroll down to the “June 9th, Meetup” section, and watch the lightning talks (5- to 10-minute presentations) about decentralized web technologies such as IndieWeb, micro-payments, and the Inter-Planetary File System.

 

Micropub for WordPress

Another IndieWeb milestone! Ryan Barrett has released

A Micropub server plugin, v1.0

From micropub.net: “Micropub is an open API standard that is used to create posts on one’s own domain using third-party clients.”

That means you can post things on your blog using any Micropub-supporting editor, decoupling the tools for writing a post from the particular system used to publish it.

I have just installed it, and then used the (experimental, I think) Quill editor to compose this post, so I can confirm it works. (I’ve since edited the post directly in WordPress.)

Click “Edit the Source Code”

How Open is the Source?

Each time I want to modify or fix some open-source application that I’m using in Ubuntu, it takes me a long time to find the correct source code, download and unpack it, install required dependencies, and figure out how to build and run it. I usually use either Ubuntu Software or Synaptic package manager to install an app, so can I also use those to install its source code? No, I can’t. I haven’t fully learnt to take advantage of the tools that are available to automate parts of this process, and I should. But nevertheless…

This is how easy it should be for anyone to modify an open-source software application on an open-source OS:

  • While running the app, click “Edit the Source Code”. (This button appears in the launcher, perhaps, when it is available.)
  • Ubuntu downloads the source code and runs an IDE with the source code loaded.
  • At this point, I can edit the source code if I wish.
  • The IDE is configured so that clicking its “Build” button re-builds the software, and clicking its “Run” button runs the built version in place of (or alongside) the system-installed version.

The key is zero configuration effort for the user to get to this point: running the locally built version.

The power user will want to configure the choice of IDE and other details, but that’s secondary.

Open-source Scanning

In the last few years I have been scanning most of the printed documents I receive. Bank statements, important receipts, event tickets to keep as souvenirs. I also from time to time scan batches of old documents such as my University notes.

I envision a free software solution that makes it easy.

SimpleScan, Pdf-Shuffler, gscan2pdf

The default solution for scanning documents on Ubuntu is not great. SimpleScan is very useful, and I have used it to scan the vast majority of my paper documents for the last few years. It has some good qualities, but at the same time is annoyingly crude. Main omissions:

  • OCR support.
  • Load a previously saved document and scan more pages into it. (For correcting mistakes.)
  • Scan a stack of pages and then divide them into separate documents. (For speed.)
  • Undo.
  • Automatic page size detection, rotation (right angles or de-skew), blank page skipping, and so on.
  • Basic UI usability (e.g. selecting and re-ordering pages doesn’t work well; forgets the crop size and position).
  • Generally stable but some serious bugs, e.g. deleting one page while another is scanning leads to data corruption.

To add more pages to a previously saved PDF document, I use PDF-Shuffler. It too is useful but crude, lacking:

  • Undo.
  • Basic UI usability (e.g. multiple selection of pages doesn’t work properly; doesn’t remember its own window position and size; doesn’t provide a dead-simple way to save the result back to the same file).

I tried gscan2pdf mainly for its OCR promise (see below), but I also liked that it can do automatic corrections such as de-skew. However, I found those features nearly impossible to configure because they are basically undocumented and presented as meaningless numbers with no clue how to choose useful values or how they interact.

On the plus side, gscan2pdf scans from my HP Photosmart C7280 all-in-one device at about twice the speed SimpleScan does, keeping up with the scanner’s speed, while SimpleScan makes the scanner pause several times per page. I don’t know if this is because it is using different scan settings and receiving less data, or because it is more efficient at receiving the data.

OCR

Most of the documents for which I expect OCR to work were printed by a computer in a nice regular clean typeface. In this century it seems ridiculous that the computer still can’t read what it wrote.

For OCR I have tried gscan2pdf, with both Tesseract and Cuneiform OCR engines. The OCR results I have obtained have been decidedly poor, good enough for generating an index of words found in the document but not good enough to want to display the result.

By contrast, the OCR performed by what I understand is a version of Abbyy FineReader embedded in my Fujitsu ScanSnap N1800 scanner gives excellent results, almost perfectly transcribing most printed pages. A notable exception is it swaps bold for italic and italic for bold. It only OCR’s one page a minute, and the scanner’s UI software is dreadful, but this does show what can be achieved by good OCR software.

Free Software Improvements

What’s needed for a good free software solution? Two main things:

  • Decent basic UI
  • OCR that just works

For the former, adding Undo to a combination of SimpleScan and PDF-Shuffler would form a good basis in terms of features and rough UI design, with maybe some ideas from gscan2pdf as well. This seems a run-of-the-mill software design and programming task.

For the OCR, I expect some intensive work is needed. It need not be totally out of reach for a group of keen volunteers, but might be more likely to happen if there were some investment by one of the big players such as Canonical, IBM or Google (who have improved Tesseract a lot but it’s not a complete OCR solution).

And then could come enhancements such as:

  • Automatic rotation, de-skew, blank page skip, cropping, …
  • Detecting a small palette of colours (often black, white and one other)

Some features such as automatic rotation are closely associated with OCR.

If Linux distributions would ship with such a solution ready to go, I think this would help general home and small office users to accept them as systems that do what they would regard as basic computer tasks.

FreeCAD: Structured CAD

The Structured Programming approach to CAD

I want my CAD software to remember how my model is constructed. If a shape is repeated in three places, I want to be able to modify that shape in one place and have the change propagated to all three places at once. If a face is rectangular, I want to know that its edges will remain parallel when I adjust its width. If each shelf in my boot rack is 18 mm thick, I want its thickness to stay at 18 mm and not reduce by 10% when I decrease the overall height of the rack from 1000 to 900 mm.

I don’t want my CAD software to remember only the final, external shape, as if it is the solid block of plastic that was printed by a 3D printer, and to allow me to alter it only by cutting a bit off or adding a bit on to that external shape one face at a time, like a sculptor using a chisel (and glue).

For me the ability to use structured design techniques, somewhat analogous to the structured programming methodology, is a key requirement for computer-aided design.

Compound Objects

We must be able to group objects together and treat the group as a single object for many purposes.

Constructive Solid Geometry operations such as union (called “fusion” in FreeCAD) and difference (“cut”) make a more complex solid shape from two or more simpler solid shapes.

Nyloc-NutA compound object is more than just a union of shapes. For one thing, it needs to allow shapes with different properties, such as a steel nut with a plastic insert (a “Nyloc” nut).

I need to look more closely at DeepSOIC’s Non-PartDesign Body container proposal which looks like it is on the right track for this.

Cloning

I want to design a shape and then insert more than one copy of it. Each wooden shelf of my boot rack is made of three or five parallel strips, which I model as cuboids initially. Later I decide to smooth their upper edges. To do this, I want all the strips to be based on a single template, and I want to modify that template so that the upper two corners are rounded. I don’t want to have to manually select each of the 52 individual edges in turn in order to apply a 4 mm radius.

Basic repetition like this requires the ability to clone a compound object. FreeCAD has the “Part Clone” operation which does this. However, a complication arises when I later modify the original. Many modifications, e.g. “fillet”, are done by wrapping the object with a transformed version of itself. The clones remain clones of the original object and so don’t reflect that modification. One existing way to overcome this is to wrap the object in a container such as “PartDesign Body” (though it has restrictions) and clone the container; then the fillet can be made inside the container and so is reflected by the clones. Another possible design could be that the “Fillet” tool offers to update the clones to point to the new Fillet object instead of the original if the user so chooses.

We need to think clearly about the notion of “the original”. If my table has four legs, I might designate one as the original and the other three as clones. Then if I want to modify them all in the same way I need to easily find the original, and make changes to it in such a way that the changes are reflected in the clones. Alternatively I could make an abstract (hidden) original, and make four clones of that. The abstract, hidden “original” might then be called a template. If I want to make a unique modification on each leg, this approach might be easier to work with, because the difference between the template and the first leg is completely clear, and I can easily choose whether I’m making a change to the template or to a particular instance.

One aspect to using clones is the importance of a local coordinate system that is used in the definition of the original object, since the cloned instances will be placed at different positions within the global coordinate system.

Lattices

Going beyond manually placed clones, it is useful to be able to place a regular array or grid or lattice of clones. FreeCAD provides a simple and very limited array operation.

I am pleased to see that DeepSOIC has been working on a Lattice Workbench for FreeCAD. This looks like it is done the Right Way, a vast improvement over the simple array operation.

We could say that using lattices is somewhat analogous to looping in a sequential programming language: for x in 0, 1, 2, 3: for z in 0, 1, 2: shelf(x * 900, y=0, z * 450 + 100);

Parametric Templates

Notice how a CAD program will usually assume that several simple shapes such as a cuboid and a cylinder are fundamental building blocks, and provide parametric templates for them. That is to say, if I use the provided cuboid tool instead of building one out of six separate faces, then FreeCAD will ensure all the angles remain right angles while allowing me to adjust the length and width and height.

That is all very well but the same principle must also be available for more complex shapes. The width of some strips of my shelves is different from others, so I want the template to be parametric. In this case:

  • a roughly cuboidal shape
  • 18 mm thick
  • variable width and length
  • two of the edges are rounded to a 4 mm radius

And then I want to place one instance at x=0, y=0 with length=900, width=46; and another instance at x=0, y=60 with length=900, width=80; and so on.

The width and length are parameters to be specified on each instantiation (in addition to the placement): that is what makes this a parametric template.

So I need my CAD software to let me define parametric templates for all the shapes I want to use. That is analogous to writing a function in structured programming, and calling the function several times with different parameters.

  • Is there a different professional terminology for what I’m calling “parametric templates” here?

Part Library

A Part Library facility will make it easy to manage templates: to organize and refer to the templates within one model, and to import parts into the model from a shared library of parts, and to create such a library.

  • There is a project called FreeCAD-library. It provides some shapes that may be copied. They are not “parametric templates” in the sense above — for example. There are no associated software enhancements to help with using them.

Refactoring

It must be easy to redefine the way the model is built while preserving the same result.

By refactoring, I mean the ability to replace one definition of an object (or any kind of data in the model) with another definition that produces the same or a similar result. If any other node in the model depended on the node that implemented the old definition, it shall henceforth depend instead on the new one.

Examples:

  • exchange the order of commutative transforms: e.g. change an array of unions to a union of arrays
  • move/rotate/scale a local coordinate system: apply the opposite transformation to the placement of each instance so there is no net change
  • a template specifies the material “wood”; change the template to not specify a material, and change all instances that didn’t override it to specify material “wood”
  • replace a stored transform with its current result (additional possibilities: freeze its result & inactivate but remember the transform; freeze the result & ensure the transform continues to give that result)
  • replace a 2D object with a sketch of itself, or vice versa
  • break any predefined object into its primitive parts along with the relevant constraints
  • replace constraints with equivalent ones: e.g. length & two angle constraints with two length constraints and one angle, defining a triangle

For any refactoring, the opposite refactoring should be available too, to the greatest extent possible.

Extrusion: 2D Sketches and 1D Profiles

Making a 2D sketch and extruding it along the third dimension is a powerful and convenient way of designing 3D shapes. In FreeCAD the Sketcher provides parametric or “constraint-based” 2D drawing that can be used as the basis for extrusion, as an alternative to using a normal 2D shape. (Indeed, the documentation suggests the Sketcher was only intended to be used for this purpose.)

Extruding a 2D face in FreeCAD makes a single solid of uniform thickness. I want a more general extrusion in which the 1D thickness profile is analogous to the 2D profile. I want to extrude my sketch not just in a single homogeneous span but in several segments (example: z=0 to 6 mm; gap; z=12 to 18 mm) to create several solid layers with or without gaps between them.

I want to store a 2D sketch as a template and use it in multiple places with different 1D extrusion profiles. I want to store a 1D extrusion profile as a template and use it in multiple places with different 2D sketches. An extrusion profile can thus serve as a specification for a composite sheet material.

A 1D profile should be able to be parametric (constraint-based), just as a 2D sketch can.

I want to be able to attach properties such as material and colour to the faces in my 2D sketch, and also to the segments in my 1D profile (z=0 to 6 material=plywood; gap; z=12 to 18 material unspecified).

Property specifications should be defaultable (“if not already specified then this”) and overrideable, so that I can choose to specify the material either in the 1D extrusion or in the 2D sketch or by a more complex combination of both.

  • 2D sketch — specifies 2D faces
  • 1D profile — specifies 1D segments
  • both can be parametric (constraint-based)
  • both can be multi-segment

Stored vs. Immediate Transforms

Mirroring, rotation, array — these are parametric ways of defining one part of the model in terms of a transformation of another part.

When a transformation is stored as a transform operator node in the model, then the parameters defining the transform (axis for mirroring, centre and angle(s) for rotation, etc.) can be changed later and the resulting object will be recalculated, or the base object can be changed later and the resulting object will be recalculated to reflect that base object. I will call this way of modelling the stored parametric transform.

The other possibility when applying a transform operator is to store only the resulting object and not the transform that was used. I’ll call that the immediate way of applying a transform.

Both behaviours are useful and both should be available for every kind of transform. The most likely wanted behaviour may well vary depending on the operator and other factors, but what we don’t want is an arbitrary mixture of behaviours, some operators creating a stored transform and others applying their effect immediately.

In FreeCAD today there does seem to be an arbitrary mixture.

In my ideal CAD program there would be a UI to change any operation from parametric to empirical, by removing the transform operator from the model and placing the result instead. This is somewhat analogous to removing a constraint: afterwards, modifying the base object will no longer affect the object created by the transform.

A more advanced UI should also provide ways to convert from empirical to parametric. This is tricky: if B is supposed to be a transformed version of A, then does B in fact differ from A only in respect of one simple transform? Should the UI try to guess the transform or must the user specify it? What if B differs arbitrarily from A? How do we even compare the objects objectively (given they may look the same but be constructed differently, for instance)?

Uniform Hierarchical Modelling

We need to be able to build any further geometry on any calculated geometry. For example, after extruding a flat face to produce a polyhedron, it must then be possible to use any face of that calculated polyhedron as the base for a further extrusion. To achieve this, every calculated result should be presented in the model as standard geometry and not only as a special kind of object whose result doesn’t act like standard geometry.

Requirements:

  • standard objects are composed of standard geometry
    • when I use the built in Rectangle tool, I can select two of its edges and build further geometry on them
  • transform result is standard geometry
    • when the result of a transform operator is a rectangle, I can select two of its edges and build further geometry on them
  • sketch result is standard geometry plus constraints
    • I can build further on two lines of a sketch

It should always be possible to:

  • transform an object in place: that is, replace it with a transformed version of itself, either a stored transform or an immediately applied transform
  • replace an object with a transformed version of another (especially interesting when the result is similar to the old object)

FreeCAD needs…

Here are my thoughts on the most important things that FreeCAD needs if it is to become my ideal CAD system.

I first used FreeCAD in 2015 to record the detailed dimensions of some rooms in my house and in 2016 to model a boot rack I was building out of wood. The ideas in this article are about fundamental aspects of FreeCAD’s scope and goals, and are based on a background of thinking about CAD as a software developer and hobbyist, as well as my experience using FreeCAD.

I am not talking here about the kinds of user interface features or specific modelling capabilities that are usually entered in a bug tracker as enhancement requests. I mention some ideas of that kind in FreeCAD Usability Improvements.

FreeCAD Needs

Fundamentally:

Every good piece of software that supports complex editing should have:

Interesting bonus capabilities:

  • shared editing

Ethos, paradigms, capabilities and limits of modelling in FreeCAD

In trying to understand how best to use FreeCAD for what I want to do — recording some dimensions and details of rooms in my house; modelling a wooden boot rack that I was making — I was unable to find a lot of the information I wanted. In trying to understand where the developers of FreeCAD are planning to get to eventually, and to decide whether I could hope to contribute to making FreeCAD into my ideal CAD system, I wanted to know more or less the same things. A discussion of the following topics would greatly help.

  • compare with other modelling systems, especially Blender and AutoCAD
  • what can a FC file contain? a single 3D scene? with a single view (camera position and type)? with lighting, background, and other scene rendering metadata? plus zero or more 2D drawings?
  • a scene is composed of what classes of object — CSG objects and mesh objects? can plug-in modules define new types of object?
  • what kinds of object can be combined with or converted to other kinds? what groups of operation can be used on what kinds of object? is there a group of operations that work on every kind of object?
  • to what extent are surface or solid properties supported (colour, texture, stiffness, …)? can arbitrary additional properties be attached to the model even though FreeCAD does not know how to interpret them unless a plug-in extension is loaded?
  • is there any kind of animation support or are we always building a static model?

It is important to describe the goals and aspirations as well as what has already been developed.

A “structured programming” style of modelling

Summary:

I want my CAD software to remember how my model is constructed. If a shape is repeated in three places, I want to be able to modify that shape in one place and have the change propagated to all three places at once. If a face is rectangular, I want to know that its edges will remain parallel when I adjust its width. If each shelf in my boot rack is 18 mm thick, I want its thickness to stay at 18 mm and not reduce by 10% when I decrease the overall height of the rack from 1000 to 900 mm.

For me the ability to use structured design techniques is a key requirement for computer-aided design.

Some important structural modelling requirements are:

  • compound objects
  • cloning
  • parametric templates
  • refactoring
  • extrusion: 2D sketches and 1D profiles

Main article: FreeCAD: Structured CAD

Shared Editing

FreeCAD would be much more powerful with shared editing: the ability to edit and display a model that’s simultaneously being edited and/or displayed by another application. Examples:

  • using Blender to edit a mesh-based subset of the model, while using FreeCAD to display the whole model and edit the rest of it
  • two people simultaneously editing the same shared model, both using FreeCAD

It’s not so much that shared editing is the killer feature, but rather I think that supporting the capability would open the way for some interesting developments such as using an external editor for some tasks (e.g. Blender for editing a mesh) and so freeing FreeCAD from the unattainable goal of having to be a good at everything. Blender will always be better for mesh editing so instead of forever trying to catch up and yet forever remaining a poor substitute, why not harness the power of Blender itself for that task and concentrate more of the FreeCAD development effort on doing the fundamental CAD tasks well?

Shared editing would require a particular kind of software architecture which, if FreeCAD isn’t already designed in the right sort of way, would probably require such major changes that it wouldn’t be feasible. However, from what I’ve seen so far, it looks like FreeCAD’s architecture might be not too far off what’s needed.

FreeCAD: Undo

Like any complex editing software, FreeCAD needs a good Undo system. It needs to be reliable: for example, undo followed by redo should always return to exactly the same state. And it needs to be friendly: for example, if I make a succession of tweaks to the view direction without any intervening actions, it should bunch those together as a single change.

A good Undo system

  • must be robust and reliable
    • implemented at the model database transaction level, and not rely on correctly implementing each modification function
    • the implementation of a modification function should only define how many undo steps are stored for this modification (often just one), the scope of each step, and a description of each step
    • getting this wrong should only affect how many undo steps are stored for that modification, and must not affect overall correctness and ability to undo the change
    • undo then redo, or redo then undo, should always leave the model and view exactly as it was
    • undo then redo, or redo then undo, should leave most of the state of the GUI as it was, with few exceptions
  • should show clearly and definitely that undo really has returned to the prior state
    • after undoing everything since the last save, file “modified” indicator should return to “unmodified” and exit should not prompt to save changes
  • should distinguish changes to the model itself from other state changes (e.g. to the view) so it is possible to make two separate UI functions:
    • undo the last change to the overall state (whether that was a change of model or of view or of both)
    • undo the last change to the model itself (either also undoing all view changes made since then, or keeping the view as it is)
  • should use appropriate granularity
    • a series of changes to the view should be grouped together
    • a series of changes to one parameter (e.g. width of a particular rectangle) should be grouped as a single change: the main reason is it is common to make a series of increments and decrements before deciding on the correct result, especially if using the scroll wheel; a related concern is entering a number such as 123 into some UI widgets causes a change first to 1, then to 12, then to 123
    • bigger granularity is often appropriate: for example, creating a new cuboid at a default size followed by adjusting all its dimensions could be grouped as a single change
  • should provide an Undo History list

I am pleased to read that wmayer has recently done some work on improving undo/redo in FreeCAD.

FreeCAD Usability Improvements

Some of my ideas for usability improvements in FreeCAD.

Visual feedback

Such as…

  • continuous immediate previewing of all changes
    • abolish all those Wizard-style input screens that force the user to enter the centre of the circle first, and then its radius, and then its plane of orientation: let the user freely specify and adjust all the parameters in whatever order makes most sense to them in that particular instance
  • better visibility controls on the model hierarchy
    • distinguish “this polygon is hidden because I temporarily want to see what’s behind it” from “this polygon is hidden because it is not itself a surface of the final 3D model: it is only the input data for an extrusion”
    • provide a way to temporarily hide a whole group and later return its visibility to the previous state in which some members of the group were visible and others were hidden
  • describe the selection in words in the status bar
    • Inkscape does this very well: e.g. “5 objects selected of types Rectangle, Path”
  • show the current selection
    • including selected elements whose visibility is set to “hidden”
    • including selected elements that would otherwise be hidden behind solid-rendered elements that are nearer the viewer
  • say what Undo and Redo will do
    • e.g. “Undo: Create Rectangle”
  • indicate whether the file is modified and whether the model is modified
    • two separate things, as the file includes both the model and the view settings

Sketcher

  • while editing a sketch mapped to a face, show that face as external geometry (to be used as construction lines) [did I notice something like this is now done?]
  • while editing, ensure the sketch is visible (e.g. by automatically fading everything else)
  • don’t clutter the display with tiny details: when the displayed size of an element is small, hide the associated constraint symbols