Filename: 154-automatic-updates.txt
Title: Automatic Software Update Protocol
Author: Matt Edman
Created: 30-July-2008
Status: Superseded
Target: 0.2.1.x
Superseded by thandy-spec.txt
Scope
This proposal specifies the method by which an automatic update client can
determine the most recent recommended Tor installation package for the
user's platform, download the package, and then verify that the package was
downloaded successfully. While this proposal focuses on only the Tor
software, the protocol defined is sufficiently extensible such that other
components of the Tor bundles, like Vidalia, Polipo, and Torbutton, can be
managed and updated by the automatic update client as well.
The initial target platform for the automatic update framework is Windows,
given that's the platform used by a majority of our users and that it lacks
a sane package management system that many Linux distributions already have.
Our second target platform will be Mac OS X, and so the protocol will be
designed with this near-future direction in mind.
Other client-side aspects of the automatic update process, such as user
interaction, the interface presented, and actual package installation
procedure, are outside the scope of this proposal.
Motivation
Tor releases new versions frequently, often with important security,
anonymity, and stability fixes. Thus, it is important for users to be able
to promptly recognize when new versions are available and to easily
download, authenticate, and install updated Tor and Tor-related software
packages.
Tor's control protocol [2] provides a method by which controllers can
identify when the user's Tor software is obsolete or otherwise no longer
recommended. Currently, however, no mechanism exists for clients to
automatically download and install updated Tor and Tor-related software for
the user.
Design Overview
The core of the automatic update framework is a well-defined file called a
"recommended-packages" file. The recommended-packages file is accessible via
HTTP[S] at one or more well-defined URLs. An example recommended-packages
URL may be:
https://updates.torproject.org/recommended-packages
The recommended-packages document is formatted according to Section 1.2
below and specifies the most recent recommended installation package
versions for Tor or Tor-related software, as well as URLs at which the
packages and their signatures can be downloaded.
An automatic update client process runs on the Tor user's computer and
periodically retrieves the recommended-packages file according to the method
described in Section 2.0. As described further in Section 1.2, the
recommended-packages file is signed and can be verified by the automatic
update client with one or more public keys included in the client software.
Since it is signed, the recommended-packages file can be mirrored by
multiple hosts (e.g., Tor directory authorities), whose URLs are included in
the automatic update client's configuration.
After retrieving and verifying the recommended-packages file, the automatic
update client compares the versions of the recommended software packages
listed in the file with those currently installed on the end-user's
computer. If one or more of the installed packages is determined to be out
of date, an updated package and its signature will be downloaded from one of
the package URLs listed in the recommended-packages file as described in
Section 2.2.
The automatic update system uses a multilevel signing key scheme for package
signatures. There are a small number of entities we call "packaging
authorities" that each have their own signing key. A packaging authority is
responsible for signing and publishing the recommended-packages file.
Additionally, each individual packager responsible for producing an
installation package for one or more platforms has their own signing key.
Every packager's signing key must be signed by at least one of the packaging
authority keys.
Specification
1. recommended-packages Specification
In this section we formally specify the format of the published
recommended-packages file.
1.1. Document Meta-format
The recommended-packages document follows the lightweight extensible
information format defined in Tor's directory protocol specification [1]. In
the interest of self-containment, we have reproduced the relevant portions
of that format's specification in this Section. (Credits to Nick Mathewson
for much of the original format definition language.)
The highest level object is a Document, which consists of one or more
Items. Every Item begins with a KeywordLine, followed by zero or more
Objects. A KeywordLine begins with a Keyword, optionally followed by
whitespace and more non-newline characters, and ends with a newline. A
Keyword is a sequence of one or more characters in the set [A-Za-z0-9-].
An Object is a block of encoded data in pseudo-Open-PGP-style
armor. (cf. RFC 2440)
More formally:
Document ::= (Item | NL)+
Item ::= KeywordLine Object*
KeywordLine ::= Keyword NL | Keyword WS ArgumentChar+ NL
Keyword ::= KeywordChar+
KeywordChar ::= 'A' ... 'Z' | 'a' ... 'z' | '0' ... '9' | '-'
ArgumentChar ::= any printing ASCII character except NL.
WS ::= (SP | TAB)+
Object ::= BeginLine Base-64-encoded-data EndLine
BeginLine ::= "-----BEGIN " Keyword "-----" NL
EndLine ::= "-----END " Keyword "-----" NL
The BeginLine and EndLine of an Object must use the same keyword.
In our Document description below, we also tag Items with a multiplicity in
brackets. Possible tags are:
"At start, exactly once": These items MUST occur in every instance of the
document type, and MUST appear exactly once, and MUST be the first item in
their documents.
"Exactly once": These items MUST occur exactly one time in every
instance of the document type.
"Once or more": These items MUST occur at least once in any instance
of the document type, and MAY occur more than once.
"At end, exactly once": These items MUST occur in every instance of
the document type, and MUST appear exactly once, and MUST be the
last item in their documents.
1.2. recommended-packages Document Format
When interpreting a recommended-packages Document, software MUST ignore
any KeywordLine that starts with a keyword it doesn't recognize; future
implementations MUST NOT require current automatic update clients to
understand any KeywordLine not currently described.
In lines that take multiple arguments, extra arguments SHOULD be
accepted and ignored.
The currently defined Items contained in a recommended-packages document
are:
"recommended-packages-format" SP number NL
[Exactly once]
This Item specifies the version of the recommended-packages format that
is contained in the subsequent document. The version defined in this
proposal is version "1". Subsequent iterations of this protocol MUST
increment this value if they introduce incompatible changes to the
document format and MAY increment this value if they only introduce
additional Keywords.
"published" SP YYYY-MM-DD SP HH:MM:SS NL
[Exactly once]
The time, in GMT, when this recommended-packages document was generated.
Automatic update clients SHOULD ignore Documents over 60 days old.
"tor-stable-win32-version" SP TorVersion NL
[Exactly once]
This keyword specifies the latest recommended release of Tor's "stable"
branch for the Windows platform that has an installation package
available. Note that this version does not necessarily correspond to the
most recently tagged stable Tor version, since that version may not yet
have an installer package available, or may have known issues on
Windows.
The TorVersion field is formatted according to Section 2 of Tor's
version specification [3].
"tor-stable-win32-package" SP Url NL
[Once or more]
This Item specifies the location from which the most recent
recommended Windows installation package for Tor's stable branch can be
downloaded.
When this Item appears multiple times within the Document, automatic
update clients SHOULD select randomly from the available package
mirrors.
"tor-dev-win32-version" SP TorVersion NL
[Exactly once]
This Item specifies the latest recommended release of Tor's
"development" branch for the Windows platform that has an installation
package available. The same caveats from the description of
"tor-stable-win32-version" also apply to this keyword.
The TorVersion field is formatted according to Section 2 of Tor's
version specification [3].
"tor-dev-win32-package" SP Url NL
[Once or more]
This Item specifies the location from which the most recent recommended
Windows installation package and its signature for Tor's development
branch can be downloaded.
When this Keyword appears multiple times within the Document, automatic
update clients SHOULD select randomly from the available package
mirrors.
"signature" NL SIGNATURE NL
[At end, exactly once]
The "SIGNATURE" Object contains a PGP signature (using a packaging
authority signing key) of the entire document, taken from the beginning
of the "recommended-packages-format" keyword, through the newline after
the "signature" Keyword.
2. Automatic Update Client Behavior
The client-side component of the automatic update framework is an
application that runs on the end-user's machine. It is responsible for
fetching and verifying a recommended-packages document, as well as
downloading, verifying, and subsequently installing any necessary updated
software packages.
2.1. Download and verify a recommended-packages document
The first step in the automatic update process is for the client to download
a copy of the recommended-packages file. The automatic update client
contains a (hardcoded and/or user-configurable) list of URLs from which it
will attempt to retrieve a recommended-packages file.
Connections to each of the recommended-packages URLs SHOULD be attempted in
the following order:
1) HTTPS over Tor
2) HTTP over Tor
3) Direct HTTPS
4) Direct HTTP
If the client fails to retrieve a recommended-packages document via any of
the above connection methods from any of the configured URLs, the client
SHOULD retry its download attempts following an exponential back-off
algorithm. After the first failed attempt, the client SHOULD delay one hour
before attempting again, up to a maximum of 24 hours delay between retry
attempts.
After successfully downloading a recommended-packages file, the automatic
update client will verify the signature using one of the public keys
distributed with the client software. If more than one recommended-packages
file is downloaded and verified, the file with the most recent "published"
date that is verified will be retained and the rest discarded.
2.2. Download and verify the updated packages
The automatic update client next compares the latest recommended package
version from the recommended-packages document with the currently installed
Tor version. If the user currently has installed a Tor version from Tor's
"development" branch, then the version specified in "tor-dev-*-version" Item
is used for comparison. Similarly, if the user currently has installed a Tor
version from Tor's "stable" branch, then the version specified in the
"tor-stable-*version" Item is used for comparison. Version comparisons are
done according to Tor's version specification [3].
If the automatic update client determines an installation package newer than
the user's currently installed version is available, it will attempt to
download a package appropriate for the user's platform and Tor branch from a
URL specified by a "tor-[branch]-[platform]-package" Item. If more than one
mirror for the selected package is available, a mirror will be chosen at
random from all those available.
The automatic update client must also download a ".asc" signature file for
the retrieved package. The URL for the package signature is the same as that
for the package itself, except with the extension ".asc" appended to the
package URL.
Connections to download the updated package and its signature SHOULD be
attempted in the same order described in Section 2.1.
After completing the steps described in Sections 2.1 and 2.2, the automatic
update client will have downloaded and verified a copy of the latest Tor
installation package. It can then take whatever subsequent platform-specific
steps are necessary to install the downloaded software updates.
2.3. Periodic checking for updates
The automatic update client SHOULD maintain a local state file in which it
records (at a minimum) the timestamp at which it last retrieved a
recommended-packages file and the timestamp at which the client last
successfully downloaded and installed a software update.
Automatic update clients SHOULD check for an updated recommended-packages
document at most once per day but at least once every 30 days.
3. Future Extensions
There are several possible areas for future extensions of this framework.
The extensions below are merely suggestions and should be the subject of
their own proposal before being implemented.
3.1. Additional Software Updates
There are several software packages often included in Tor bundles besides
Tor, such as Vidalia, Privoxy or Polipo, and Torbutton. The versions and
download locations of updated installation packages for these bundle
components can be easily added to the recommended-packages document
specification above.
3.2. Including ChangeLog Information
It may be useful for automatic update clients to be able to display for
users a summary of the changes made in the latest Tor or Tor-related
software release, before the user chooses to install the update. In the
future, we can add keywords to the specification in Section 1.2 that specify
the location of a ChangeLog file for the latest recommended package
versions. It may also be desirable to allow localized ChangeLog information,
so that the automatic update client can fetch release notes in the
end-user's preferred language.
3.3. Weighted Package Mirror Selection
We defined in Section 1.2 a method by which automatic update clients can
select from multiple available package mirrors. We may want to add a Weight
argument to the "*-package" Items that allows the recommended-packages file
to suggest to clients the probability with which a package mirror should be
chosen. This will allow clients to more appropriately distribute package
downloads across available mirrors proportional to their approximate
bandwidth.
Implementation
Implementation of this proposal will consist of two separate components.
The first component is a small "au-publish" tool that takes as input a
configuration file specifying the information described in Section 1.2 and a
private key. The tool is run by a "packaging authority" (someone responsible
for publishing updated installation packages), who will be prompted to enter
the passphrase for the private key used to sign the recommended-packages
document. The output of the tool is a document formatted according to
Section 1.2, with a signature appended at the end. The resulting document
can then be published to any of the update mirrors.
The second component is an "au-client" tool that is run on the end-user's
machine. It periodically checks for updated installation packages according
to Section 2 and fetches the packages if necessary. The public keys used
to sign the recommended-packages file and any of the published packages are
included in the "au-client" tool.
References
[1] Tor directory protocol (version 3),
https://tor-svn.freehaven.net/svn/tor/trunk/doc/spec/dir-spec.txt
[2] Tor control protocol (version 2),
https://tor-svn.freehaven.net/svn/tor/trunk/doc/spec/control-spec.txt
[3] Tor version specification,
https://tor-svn.freehaven.net/svn/tor/trunk/doc/spec/version-spec.txt