1 
<?xml version="1.0" encoding="ISO-8859-1"?>
2 
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
3 
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
4 
<html xmlns="http://www.w3.org/1999/xhtml">
5 
<head>
6 
<title>Updated design notes on fetchmail</title>
7 
<link rev="made" href="mailto:matthias.andree@gmx.de" />
8 
<meta name="description" content="Updated design notes on fetchmail." />
9 
<meta name="keywords" content="fetchmail, POP, POP2, POP3, IMAP, ETRN, ODMR, remote mail" />
10 
<style type="text/css">
11 
/*<![CDATA[*/
12 
 h1.c1 {text-align: center}
13 
/*]]>*/
14 
</style>
15 
</head>
16 
<body>
17 
<table width="100%" cellpadding="0" summary="Canned page header">
18 
<tr>
19 
<td width="30%">Back to <a href="index.html">Fetchmail Home Page</a></td>
20 
<td width="30%" align="right">$Date$</td>
21 
</tr>
22 
</table>
23 





24

<hr />





25

<h1 class="c1">Design Notes On Fetchmail</h1>





26






27

<h2>Introduction</h2>





28






29

<p>This document is supposed to complement <a





30

    href="esrs-design-notes.html">Eric S. Raymond's (ESR's)





31

    design notes.</a> The new maintainers don't agree with some of the decisions





32

ESR made previously, and the differences and new directions will be laid





33

out in this document. It is therefore a sort of a TODO document, until





34

the necessary code revisions have been made.</p>





35






36

<h2>Security</h2>





37






38

<p>Fetchmail was handed over in a pretty poor shape, security-wise. It will





39

happily talk to the network with root privileges, use sscanf() to read





40

remotely received data into fixed-length stack-based buffers without





41

length limitation and so on. A full audit is required and security





42

concepts will have to be applied. Random bits are:</p>





43






44

<ul>





45

    <li>code talking to the network does not require root privileges and





46

    needs to run without root permissions</li>





47

    <li>all input must be validated, all strings must be length checked,





48

    all integers range checked</li>





49

    <li>all types will need to be reviewed whether they are signed or





50

    unsigned</li>





51

</ul>





52






53

<h2>SMTP forwarding</h2>





54






55

<p>Fetchmail's multidrop and rewrite options will process addresses





56

received from remote sites. Special care must be taken so these





57

features cannot be abused to relay mail to foreign sites.</p>





58






59

<p>ESR's attempt to make fetchmail use SMTP exclusively failed,





60

fetchmail got LMTP and --mda options &ndash; the latter has a lot of





61

flaws unfortunately, is inconsistent with the SMTP forwarder and needs





62

to be reviewed and probably bugfixed. --mda doesn't properly work with





63

multiple recipients, it cannot properly communicate errors and is best





64

avoided for now.</p>





65






66

<h2>Server-side vs. client-side state.</h2>





67






68

<h3>Why we need client-side tracking</h3>





69






70

<p>ESR asserted that server-side state were essential and those persons





71

repsonsible for removing the LAST command from POP3 deserved to





72

suffer. ESR is right in stating that the POP3 UID tracks which messages





73

have been read <em>by this client</em> &ndash; and that is exactly what





74

we need to do.</p>





75






76

<p>If fetchmail is supposed to retrieve all mail from a mailbox





77

reliably, without being disturbed by someone occasionally using another





78

client on another host, or a webmailer, or similar, then





79

<em>client</em>-side tracking of the state is indispensable. This is





80

also needed to match behavior to ETRN and ODMR or to support read-only





81

mailboxes in --keep mode.</p>





82






83

<h3>Present and future</h3>





84






85

<p>Fetchmail supports client-side state in POP3 if the UIDL option is





86

used (which is strongly recommended). Similar effort needs to be made to





87

track IMAP state by means of UIDVALIDITY and UID.</p>





88






89

<p>This will also mean that the UID handling code be revised an perhaps





90

use one file per account or per folder.</p>





91






92

<h2>Concurrent queries/concurrent fetchmail instances</h2>





93






94

<p>ESR refused to make fetchmail query multiple hosts or accounts





95

concurrently, on the grounds that finer-grained locks would be hard to





96

implement portably.</p>





97






98

<p>The idea of using one file per folder or account to track UIDs on the





99

client-side will make solving this locking problem easy &ndash; the lock can





100

be placed on the UID file instead.</p>





101






102

<h2>Multidrop issues</h2>





103






104

<p>Fetchmail tries to guess recipients from headers that are not routing





105

relevant, for instance, To:, Cc:, or Resent-headers (which are rare





106

anyways). It is important that fetchmail insists on the real envelope





107

operation for multidrop. This is detailed in <a





108

    href="http://home.pages.de/~mandree/mail/multidrop">my





109

    article &quot;Requisites for working multidrop





110

    mailboxes&quot;</a>.</p>





111






112

<p>As Terry Lambert pointed out in the FreeBSD-arch mailing list on





113

2001-02-17 under the subject "UUCP must stay; fetchmail sucks",





114

fetchmail performs DNS MX lookups to determine domains for which





115

multidrop is valid, on the assumption that the receiving SMTP host





116

upstream were the same as the IMAP or POP3 server.</p>





117






118

<hr />





119

<table width="100%" cellpadding="0" summary="Canned page footer">





120

<tr>





121

<td width="30%">Back to <a href="index.html">Fetchmail Home Page</a></td>





122

<td width="30%" align="right">$Date$</td>





123

</tr>





124

</table>





125






126

<br clear="left" />





127

<address>Matthias Andree <a





128

	href="mailto:matthias.andree@gmx.de">&lt;matthias.andree@gmx.de&gt;</a></address>





129

</body>





130

</html>