allow to limit file size through the max_file_size setting

[coquelicot.git] / README

About “Coquelicot”
==================

Coquelicot is a "one-click" file sharing web application with a specific
focus on protecting users' privacy.

Basic principle: users can upload a file to the server, in return they
get a unique URL which can be shared with others in order to download
the file.

Coquelicot aims to protect, to some extent, users and system
administrators from disclosure of the files exchanged from passive and
not so active attackers.

Features
--------

 * Support for different authentication methods

   In order to prevent random Internet users to eat bandwidth and
   disk space, Coquelicot limits upload to authenticated users.
   It currently ships with two authentication mechanisms:

    - "simplepass": users will need to provide a global, pre-shared,
      password;
    - "imap": users will need to provide a login and a password,
      those credentials will be used to authenticate against an existing
      IMAP server.

   It is possible to integrate more authentication mechanisms by
   implementing a single method, some Javascript, and a template partial
   to render the common fields. For more information have a look at the
   notes below.

 * Mandatory expiration

   When uploading, a time limit has to be specified. The file will be
   unavailable once this limit has been reached.

   During a configurable period of time, trying to download the file
   will return a page saying "too late" instead of "not found".

 * Support for one-time download

   An user might want to allow exactly _one_ download of a file, to more
   closely replace an email attachment. The file will be removed after
   the first complete download and concurrent downloads are prevented.

 * Upload progress bar

   If the web server tracks upload progress, users having javascript
   enabled will see a nice progress bar during the file upload.

 * Downgrade nicely

   The application works fine without javascript or CSS.

 * Download URL are hand-writing compatible

   URLs generated to download files uses the Base32 character set. This
   set is specifically designed to overcome misread of 'l', '1', '0' and
   'O' characters. Coquelicot will automatically convert case and
   ambiguous characters to facilitate URL exchanges through
   hand-writing.

 * Files are stored encrypted on the server

   While being uploaded, files are written to the disk using symmetric
   encryption. The encryption key is _not_ stored directly by
   Coquelicot. It is either generated randomly and given as part of the
   download URL, or specified by the uploader.

 * Download can be protected by a password

   When uploading, a password can be specified which will be used as
   the encryption key. In order to download the file, the password
   must be entered through in a POST'ed form, preventing the password
   from appearing in the server logs.

 * Files are stored with a random name

   To prevent disclosure of the shared file name, it is stored encrypted
   together with the file content. On the server, this encrypted file is
   stored with a random name.

 * Download URLs do not reflect stored file names

   The random names given in download URLs do not map directly to file
   names on the server. This prevent server logs from giving a direct
   mapping to the shared files.

 * File content is zero'ed before removal

   When a file has expired, it is removed from the server. In order
   to make it harder to retrieve its content through filesystem
   analysis, it is filled with zeros first.

Installation
------------

Coquelicot is written in Ruby using the Sinatra web framework. Due to
reasons detailed below, it needs to be run with the Rainbows! web
server. The later should probably be made accessible through a reverse
proxy.

### Initial setup

Coquelicot uses Bundler to manage its dependency. To install Bundler on
Debian, please issue:

    # apt-get install rubygems
    $ gem install bundler

Once Bundler is available, please issue:

    $ bundle install --deployment --binstubs

Then, to start Coquelicot use:

    $ bin/rainbows -c rainbows.conf -E none config.ru

Coquelicot is intended to be run on a fully encrypted system and
accessible only through HTTPS. To configure Apache as a reverse proxy,
you will need to add the following directives:

    ProxyPass / http://127.0.0.1:51161/
    SetEnv proxy-sendchunks 1
    RequestHeader set X-Forwarded-SSL "on"

You can also run Coquelicot with mod_passenger, Mongrel, Thin or any
Rack compatible webserver, but please read below about buffered input.

### Configuration

By default Coquelicot is configured to authenticate with the
"simplepass" mechanism and some other reasonable defaults.

It is possible to overwrite these settings from a configuration file
named `settings.yml` that will be used if it is present in the `conf`
directory of the application.

All available settings with their default values are documented in
`conf/settings-default.yml`.

Further settings example:

 * `conf/settings-simplepass.yml`: shows how to change the default
   password for the "simplepass" mechanism.

 * `conf/settings-imap.yml`: necessary configuration for the "imap"
   authentication mechanism.

You can copy one of these examples to `conf/settings.yml` and adjust
them according to your environment.

### Garbage collection

To cleanup files automatically when they expired, coquelicot comes with
a cleanup script, that does the garbage collection for you. The easiest
way is to add `ext/coquelicot_gc.rb` as a cron job that runs every 5
minutes (or so).

### Migrate from Jyraphe

[Jyraphe] is another free software web file sharing application.
Coquelicot provides a migration script to import Jyraphe 0.5
repositories in `tools/migrate_jyraphe.rb`.

[Jyraphe]: http://home.gna.org/jyraphe/

Test, development and extensions
--------------------------------

Coquelicot is written in Ruby and should be quite easy to improve for
anyone a little bit familiar with the Sinatra web framework. It is
mostly written using Behaviour Driven Development, making the test suite
a fine net to hack in confidence. So please go ahead!

### Setup a work environment

As Coquelicot uses Bundle, the first step to work on Coquelicot
is installing the proper dependencies by issuing:

    bundle install

### Basic operations

Coquelicot test suite is written using RSpec. Running the test suite is
just a matter of typing:

    bundle exec rspec

Running a test server can be done with:

    bundle exec rackup config-development.ru

To update the translation source files, use:

    bundle exec rake updatepo

This will update `po/coquelicot.pot` and merge the new strings in the various
`po/*/coquelicot.po` files.

### Authentication mechanisms

The authentication part of Coquelicot has been made modular. Adding a
new authentication mechanism should be fairly straightforward.

New authentication mechanism needs to provide the following 3 files,
with the following responsabilities:

 * `lib/coquelicot/auth/<METHOD>.rb`:

   A class implementing the actual authentication. This class must
   implement an `authenticate` method. It will receive the form fields
   as usual (params). This method should either return true if upload
   should be allowed.

 * `public/javascripts/coquelicot.auth.<METHOD>.js:`

    This file should define 'authentication' as an object with the
    following methods:

    - `getData()`: returns an object of all the necessary data
      to authenticate on the app side. Keys should have the same name
      as the input fields used to authenticate without Javascript.
    - `focus()`: set the focus on the first authentication form field.
    - (optional) `handleSuccess()`: arbitrary action upon successful
      authentication. This is called after the livebox is closed.
    - (optional) `handleReject()`: arbitrary action when access
      get rejected. One can reset authentication fields after a failed
      authentication.
    - (optional) `handleFailure()`: arbitrary action when there was
      a problem in the authentication procedure.

 * `views/auth/<METHOD>.haml`:

   Render the necessary form fields that will be used for
   authentication.

   The authentication method can be set in the application settings
   including mandatory options for this method.

### Watch for buffered inputs!

Coquelicot is written in Ruby using Sinatra. Sinatra is based on the
Rack webserver interface. Rack specification mandates that applications
must be able to seek and rewind freely in the request content.

Request data are always received as a stream through the network. So in
order to comply with the specification, webservers implementing Rack will
either buffer the input in memory (Webrick) or in a temporary file
(Thin, Passenger or Mongrel).

On top of that, when parsing `multipart/form-data` POST content,
`Rack::Request` (used by Sinatra) will create a new temporary file for
each files in the POST request.

For the specific needs of Coquelicot, those behaviours will prevent
users from uploading large files (if `/tmp` is in memory) or will be a
breach of privacy, as a clear text version will be written to disk.

To overcome these limitations, Coquelicot first uses a specific feature
of the Rainbows! webserver of streaming its input directly to
applications, and second bypass `Rack::Request` to directly handle
POST content. Usage of any other Rack webserver is strongly discouraged
and should be restricted to development and testing.

### Implementation details

Common application code lies in `Coquelicot::Application`. Except for
one specific (and important) type of requests, namely `POST /update`.
These requests are handled directly at bare Rack level by
`Coquelicot::Rack::Upload`.

This allows to work directly with POST data as the browser is sending
them, so we can directly stream the uploaded file to our encrypted
on-disk containers.

The POST data must be in a very specific order, as we need to handle
authentication, and various options prior to start recording the file
content. Thanks to the W3C, the [HTML specification] states that parts
of the POST data must be delivered in the same order the controls
appears in the `<form/>` container.

`Coquelicot::Rack::Multipart` expose a simple DSL to parse the fields as
they are delivered. The later is used by `Coquelicot::Rack::Upload` to
perform its logic pretty nicely.

[HTML specification]: http://www.w3.org/TR/html4/interact/forms.html

Future
------

 * More flexible expiration

   It might be interesting to also offer a calendar for specifying
   an exact date after which the file will be unavailable.

 * Hide file size (padding)

   There is currently a real close mapping from original file size to
   stored file size. Original file size will also be recorded in server
   logs. Padding could be used to improve this situation.

 * Make a Gem

   Most Ruby stuff is installed using Gem, so Coquelicot should be one.

 * Package for Debian

   A Debian package would be nice to spread Coquelicot setups.

Storage details
---------------

Files are stored in the directory specified by the 'depot_path' setting.
One file in Coquelicot is actually stored in two files: one for metadata and
one for the file content.

### Metadata file

The format is the following:

    --- 
    Coquelicot: "2.0"
    Salt: <8 bytes stored as Base64>
    Expire-at: <expiration time in seconds since epoch>
    --- 
    <encrypted metadata>

Encryption is done using OpenSSL. Cipher is AES-256-CBC with key and IV
created using the `pbkdf2_hmac_sha1()` implementation of PKCS5. The later
is fed using the former *Salt* and the given passphrase, using 2000
iterations.

Once decrypted, the metadata have the following format:

    --- 
    Created-at: <upload time in seconds since epoch>
    Filename: "<original file name>"
    Content-Type: "<MIME type>"
    Length: <content length is bytes>
    One-time-only: <true|false>

Headers must be parseable using the YAML standard.

### Content file

The content file contains the stored file in encrypted form. Encryption is done
with the same algorithm and keys as the encrypted metadata (see above).

The file name of the content file is the same as the one for metada, with an
added suffix of '.content'. For example, if the metadata file name is
`mqeb4pfcru2ymq3e6se7`, the associated content file will be
`mqeb4pfcru2ymq3e6se7.content`.

### Expired files

Both the content file and the metadata file are truncated to zero length when
they are "expired".

### URL mapping

In order to map download URLs to file name, a simple text file ".links"
is used. It contains a line for each file in the form:

    <URL name> <metadata file name>

### Changes history

  version 2.0
  :    Current version described above.

  version 1.0
  :    File content is in the same file as the metadata. Content is put in the
       after the metadata and an extra "--- \n".

Authors
-------

    Coquelicot © 2010-2012 potager.org <jardiniers@potager.org>
               © 2011 mh / immerda.ch  <mh+coquelicot@immerda.ch>

Coquelicot is distributed under the [GNU Affero General Public License]
version 3 or (at your option) any later version.

Background image (`public/images/background.jpg`) derived from:  
[“coquelicot”] © 2008 Jean-Louis Zimmermann  
Licensed under [Creative Commons Attributions 2.0 Generic]  

[“coquelicot”]: https://secure.flickr.com/photos/jeanlouis_zimmermann/2478019744/
[GNU Affero General Public License]: http://www.gnu.org/licenses/agpl.txt
[Creative Commons Attributions 2.0 Generic]: https://creativecommons.org/licenses/by/2.0/deed