RFC5536

Network Working Group K. Murchison, Ed. Request for Comments: 5536 Carnegie Mellon University Obsoletes: 1036 C. Lindsey Category: Standards Track University of Manchester

                                                             D. Kohn
                                                  Healing Thresholds
                                                       November 2009

                     Netnews Article Format

Abstract

This document specifies the syntax of Netnews articles in the context of the Internet Message Format (RFC 5322) and Multipurpose Internet Mail Extensions (MIME) (RFC 2045). This document obsoletes RFC 1036, providing an updated specification to reflect current practice and incorporating incremental changes specified in other documents.

Status of This Memo

This document specifies an Internet standards track protocol for the Internet community, and requests discussion and suggestions for improvements. Please refer to the current edition of the "Internet Official Protocol Standards" (STD 1) for the standardization state and status of this protocol. Distribution of this memo is unlimited.

Copyright Notice

Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the BSD License.

This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling

the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English.

Introduction

Basic Concepts

"Netnews" is a set of protocols for generating, storing, and retrieving news "articles" (whose format is a subset of that for Email messages), and for exchanging them amongst a readership that is potentially widely distributed. It is organized around "newsgroups", with the expectation that each reader will be able to see all articles posted to each newsgroup in which he participates. These protocols most commonly use a flooding algorithm, which propagates copies throughout a network of participating servers. Typically, only one copy is stored per server, and each server makes it available on demand to readers who are able to access that server.

Scope

This document specifies the syntax of Netnews articles in the context of the Internet Message Format RFC5322 and Multipurpose Internet Mail Extensions (MIME) RFC2045. This document obsoletes RFC1036, updating the syntax of Netnews articles to reflect current practice and incorporating changes and clarifications specified in other documents such as [Son-of-1036].

This is the first in a set of documents that obsolete RFC1036. This document focuses on the syntax and semantics of Netnews articles. RFC5537 is also a Standards Track document and describes the protocol issues of Netnews articles, independent of transport protocols such as the Network News Transfer Protocol (NNTP) RFC3977. [USEAGE], "Usenet Best Practice", describes implementation recommendations to improve interoperability and usability.

This specification is intended as a definition of what article content format is to be passed between systems. Although many news systems locally store articles in this format (which eliminates the need for translation between formats), local storage is outside of the scope of this standard.

Requirements Notation

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC2119.

Syntax Notation

Header fields defined in this specification use the Augmented Backus- Naur Form (ABNF) notation (including the Core Rules) specified in RFC5234 as well as many constructs defined in RFC5322, RFC2045 as updated by RFC2231, and RFC3986. Specifically:

token = <see RFC 2045 Section 5.1> value = <see RFC 2045 Section 5.1> parameter = <see RFC 2231 Section 7> attribute = <see RFC 2231 Section 7>

FWS = <see RFC 5322 Section 3.2.2> comment = <see RFC 5322 Section 3.2.2> CFWS = <see RFC 5322 Section 3.2.2> atext = <see RFC 5322 Section 3.2.3> dot-atom-text = <see RFC 5322 Section 3.2.3> phrase = <see RFC 5322 Section 3.2.5> date-time = <see RFC 5322 Section 3.3> mailbox = <see RFC 5322 Section 3.4> mailbox-list = <see RFC 5322 Section 3.4> address-list = <see RFC 5322 Section 3.4> body = <see RFC 5322 Section 3.5> fields = <see RFC 5322 Section 3.6>

IPv6address = <see RFC 3986 Section 3.2.2> IPv4address = <see RFC 3986 Section 3.2.2>

ALPHA = <see RFC 5234 Appendix B.1> CRLF = <see RFC 5234 Appendix B.1> DIGIT = <see RFC 5234 Appendix B.1> DQUOTE = <see RFC 5234 Appendix B.1> SP = <see RFC 5234 Appendix B.1> VCHAR = <see RFC 5234 Appendix B.1>

Additionally, Section 3.1.3 specifies a stricter definition of <msg-id> than the syntax in Section 3.6.4 of RFC5322.

Definitions

An "article" is the unit of Netnews, analogous to an RFC5322 "message". A "proto-article" is one that has not yet been injected into the news system. In contrast to an article, a proto-article may lack some mandatory header fields.

A "message identifier" (Section 3.1.3) is a unique identifier for an article, usually supplied by the user agent that posted it or, failing that, by the "news server". It distinguishes the article

from every other article ever posted anywhere. Articles with the same message identifier are treated as if they are the same article regardless of any differences in the body or header fields.

A "newsgroup" is a forum having a name and that is intended for articles on a specific topic. An article is "posted to" a single newsgroup or several newsgroups. When an article is posted to more than one newsgroup, it is said to be "crossposted"; note that this differs from posting the same text as part of each of several articles, one per newsgroup.

A newsgroup may be "moderated", in which case submissions are not posted directly, but mailed to a "moderator" for consideration and possible posting. Moderators are typically human but may be implemented partially or entirely in software.

A "poster" is the person or software that composes and submits a potentially compliant article to a user agent.

A "reader" is the person or software reading Netnews articles.

A "followup" is an article containing a response to the contents of an earlier article, its "precursor". Every followup includes a "References" header field identifying that precursor (but note that non-followup articles may also use a References header field).

A "control message" is an article that is marked as containing control information; a news server receiving such an article may (subject to the policies observed at that site) take actions beyond just filing and passing on the article.

A news server is software that may accept articles from a user agent, and/or make articles available to user agents, and/or exchange articles with other news servers.

A "user agent" is software that may help posters submit proto- articles to a news server, and/or fetch articles from a news server and present them to a reader, and/or assist the reader in creating articles and followups.

The generic term "agent" is used when describing requirements that apply to both user agents and news servers.

An agent is said to "generate" a construct if it did not exist before the agent created it. Examples are when a user agent creates a message from text and addressing information supplied by a user, or when a news server creates an "Injection-Info" header field for a newly posted message.

An agent is said to "accept" a construct if some other entity generates it and passes it to the agent in question, and the agent processes it without treating it as a format or protocol error.

Structure of This Document

This document uses a cite-by-reference methodology, rather than repeating the contents of other standards, which could otherwise result in subtle differences and interoperability challenges. Although this document is as a result rather short, it requires complete understanding and implementation of the normative references to be compliant.

Section 2 defines the format of Netnews articles. Section 3 details the header fields necessary to make an article suitable for the Netnews environment.

Format

Base

An article is said to be conformant to this specification if it conforms to the format specified in Section 3 of RFC5322 and to the additional requirements of this specification.

An article that uses the obsolete syntax specified in Section 4 of RFC5322 is NOT conformant to this specification, except for the following two cases:

o Articles are conformant if they use the <obs-phrase> construct

  (use of a phrase like "John Q. Public" without the use of quotes,
  see Section 4.1 of RFC5322), but agents MUST NOT generate
  productions of such syntax.

o Articles are conformant if they use the "GMT" <zone>, as specified

  in Section 3.1.1.

This document, and specifications that build upon it, specify how to handle conformant articles. Handling of non-conformant articles is outside the scope of this specification.

Agents conforming to this specification MUST generate only conformant articles.

The text below uses ABNF to specify restrictions on the syntax specified in RFC5322; this grammar is intended to be more restrictive than the RFC5322 grammar. Articles must conform to the

ABNF specified in RFC5322 and also to the restrictions specified here, both those that are expressed as text and those that are expressed as ABNF.

  NOTE: Other specifications use the term "header" as a synonym for
  what RFC5322 calls "header field".  This document follows the
  terminology in Section 2 of RFC5322 in using the terms "line",
  "header field", "header field name", "header field body", and
  "folding", based on a belief that consistent terminology among
  specifications that depend on each other makes the specifications
  easier to use in the long run.

Header Fields

All header fields in a Netnews article are compliant with RFC5322; this specification, however, is less permissive in what can be generated and accepted by agents. The syntax allowed for Netnews article headers is a strict subset of the Internet Message Format headers, making all headers compliant with this specification inherently compliant with RFC5322. Note however that the converse is not guaranteed to be true in all cases.

General rules that apply to all header fields (even those documented in RFC5322 and RFC2045) are listed below, and those that apply to specific header fields are described in the relevant sections of this document.

o All agents MUST generate header fields so that at least one space

  immediately follows the ':' separating the header field name and
  the header field body (for compatibility with deployed software,
  including NNTP RFC3977 servers).  News agents MAY accept header
  fields that do not contain the required space.

o Every line of a header field body (including the first and any

  that are subsequently folded) MUST contain at least one non-
  whitespace character.

     NOTE: This means that no header field body defined by or
     referenced by this document can be empty.  As a result, rather
     than using the <unstructured> syntax from Section 3.2.5 of
     RFC5322, this document uses a stricter definition:

unstructured = *WSP VCHAR *( [FWS] VCHAR ) *WSP

     NOTE: The RFC5322 specification sometimes uses [FWS] at the
     beginning or end of ABNF describing header field content.  This
     specification uses *WSP in such cases, also in cases where this
     specification redefines constructs from RFC5322.  This is

     done for consistency with the restriction described here, but
     the restriction applies to all header fields, not just those
     where ABNF is defined in this document.

o Compliant software MUST NOT generate (but MAY accept) header field

  lines of more than 998 octets.  This is the only limit on the
  length of a header field line prescribed by this standard.
  However, specific rules to the contrary may apply in particular
  cases (for example, according to RFC2047, lines of a header
  field containing encoded words are limited to 76 octets).
  [USEAGE] includes suggested limits for convenience of display by
  user agents.

     NOTE: As stated in RFC5322, there is NO restriction on the
     number of lines into which a header field may be split, and
     hence there is NO restriction on the total length of a header
     field (in particular it may, by suitable folding, be made to
     exceed the 998-octet restriction pertaining to a single header
     field line).

o The character set for header fields is US-ASCII. Where the use of

  non-ASCII characters is required, they MUST be encoded using the
  MIME mechanisms defined in RFC2047 and RFC2231.

MIME Conformance

User agents MUST meet the definition of MIME conformance in RFC2049 and MUST also support RFC2231. This level of MIME conformance provides support for internationalization and multimedia in message bodies RFC2045, RFC2046, and RFC2231, and support for internationalization of header fields RFC2047 and RFC2231. Note that [Errata] currently exist for RFC2045, RFC2046, RFC2047 and RFC2231.

For the purposes of Section 5 of RFC2047, all header fields defined in Section 3 of this standard are to be considered as "extension message header fields", permitting the use of RFC2047 encodings within any <unstructured> header field, or within any <comment> or <phrase> permitted within any structured header field.

User agents MAY accept and generate other MIME extension header fields, and in particular SHOULD accept Content-Disposition RFC2183 and Content-Language RFC3282.

News Header Fields

The following news header fields extend those defined in Section 3.6 of RFC5322:

fields =/ *( approved /

                     archive /
                     control /
                     distribution /
                     expires /
                     followup-to /
                     injection-date /
                     injection-info /
                     lines /
                     newsgroups /
                     organization /
                     path /
                     summary /
                     supersedes /
                     user-agent /
                     xref )

Each of these header fields MUST NOT occur more than once in a news article.

The following header fields defined in this document do not allow <comment>s (i.e., they use FWS rather than CFWS).

Control Distribution Followup-To Lines Newsgroups Path Supersedes Xref

This also applies to the following header field defined in RFC5322:

Message-ID

Most of these header fields are mainly of interest to news servers, and news servers often need to process these fields very rapidly. Thus, some header fields prohibit <comment>s.

Mandatory Header Fields

Each Netnews article conformant with this specification MUST have exactly one of each of the following header fields: Date, From, Message-ID, Newsgroups, Path, and Subject.

Date

The Date header field is the same as that specified in Sections 3.3 and 3.6.1 of RFC5322, with the added restrictions detailed above in Section 2.2. However, the use of "GMT" as a time zone (part of <obs-zone>), although deprecated, is widespread in Netnews articles today. Therefore, agents MUST accept <date-time> constructs that use the "GMT" zone.

orig-date = "Date:" SP date-time CRLF

  NOTE: This specification does not change RFC5322, which says
  that agents MUST NOT generate <date-time> constructs that include
  any zone names defined by <obs-zone>.

Software that accepts dates with unknown timezones SHOULD treat such timezones as equivalent to "-0000" when comparing dates, as specified in Section 4.3 of RFC5322.

Also note that these requirements apply wherever <date-time> is used, including Injection-Date and Expires (Sections 3.2.7 and 3.2.5, respectively).

From

The From header field is the same as that specified in Section 3.6.2 of RFC5322, with the added restrictions detailed above in Section 2.2.

from = "From:" SP mailbox-list CRLF

Message-ID

The Message-ID header field contains a unique message identifier. Netnews is more dependent on message identifier uniqueness and fast comparison than Email is, and some news software and standards RFC3977 might have trouble with the full range of possible <msg-id>s permitted by RFC5322. This section therefore restricts the syntax of <msg-id> as compared to Section 3.6.4 of RFC5322. The global uniqueness requirement for <msg-id> in RFC5322 is to be understood as applying across all protocols using such message identifiers, and across both Email and Netnews in particular.

message-id = "Message-ID:" SP *WSP msg-id *WSP CRLF

msg-id = "<" msg-id-core ">"

                  ; maximum length is 250 octets

msg-id-core = id-left "@" id-right

id-left = dot-atom-text

id-right = dot-atom-text / no-fold-literal

no-fold-literal = "[" *mdtext "]"

mdtext = %d33-61 / ; The rest of the US-ASCII

                  %d63-90 /        ; characters not including
                  %d94-126         ; ">", "[", "]", or "\"

The <msg-id> MUST NOT be more than 250 octets in length.

  NOTE: The length restriction ensures that systems that accept
  message identifiers as a parameter when referencing an article
  (e.g., RFC3977) can rely on a bounded length.

Observe that <msg-id> includes the < and >.

Observe also that in contrast to the corresponding header field in RFC5322:

o The syntax does not allow comments within the Message-ID header

  field.

o There is no possibility for ">" or WSP to occur inside a <msg-id>.

o Even though commonly derived from <domain>s, <id-rights>s are

  case-sensitive (and thus, once created, are not to be altered
  during subsequent transmission or copying)

This is to simplify processing by news servers and to ensure interoperability with existing implementations and compliance with RFC3977. A simple comparison of octets will always suffice to determine the identity of two <msg-id>s.

Also note that this updated ABNF applies wherever <msg-id> is used, including the References header field discussed in Section 3.2.10 and the Supersedes header field discussed in Section 3.2.12.

Some software will try to match the <id-right> of a <msg-id> in a case-insensitive fashion; some will match it in a case-sensitive fashion. Implementations MUST NOT generate a Message-ID where the only difference from another Message-ID is the case of characters in the <id-right> part.

When generating a <msg-id>, implementations SHOULD use a domain name as the <id-right>.

  NOTE: Section 3.6.4 of RFC5322 recommends that the <id-right>
  should be a domain name or a domain literal.  Domain literals are
  troublesome since many IP addresses are not globally unique;
  domain names are more likely to generate unique Message-IDs.

Newsgroups

The Newsgroups header field specifies the newsgroup(s) to which the article is posted.

newsgroups = "Newsgroups:" SP newsgroup-list CRLF

newsgroup-list = *WSP newsgroup-name

                  *( [FWS] "," [FWS] newsgroup-name ) *WSP

newsgroup-name = component *( "." component )

component = 1*component-char

component-char = ALPHA / DIGIT / "+" / "-" / "_"

Not all servers support optional FWS in the list of newsgroups. In particular, folding the Newsgroups header field over several lines has been shown to harm propagation significantly. Optional FWS in the <newsgroup-list> SHOULD NOT be generated, but MUST be accepted.

A <component> SHOULD NOT consist solely of digits and SHOULD NOT contain uppercase letters. Such <component>s MAY be used only to refer to existing groups that do not conform to this naming scheme, but MUST NOT be used otherwise.

  NOTE: All-digit <component>s conflict with one widely used storage
  scheme for articles.  Mixed-case groups cause confusion between
  systems with case-sensitive matching and systems with case-
  insensitive matching of <newsgroup-name>s.

<component>s beginning with underline ("_") are reserved for use by future versions of this standard and SHOULD NOT be generated by user agents (whether in header fields or in newgroup control messages as defined by RFC5537). However, such names MUST be accepted by news servers.

<component>s beginning with "+" and "-" are reserved for private use and SHOULD NOT be generated by user agents (whether in header fields or in newgroup control messages RFC5537) without a private prior agreement to do so. However, such names MUST be accepted by news servers.

The following <newsgroup-name>s are reserved and MUST NOT be used as the name of a newsgroup:

o Groups whose first (or only) <component> is "example"

o The group "poster"

The following <newsgroup-name>s have been used for specific purposes in various implementations and protocols and therefore MUST NOT be used for the names of normal newsgroups. They MAY be used for their specific purpose or by local agreement.

o Groups whose first (or only) component is "to"

o Groups whose first (or only) component is "control"

o Groups that contain (or consist only of) the component "all"

o Groups that contain (or consist only of) the component "ctl"

o The group "junk"

  NOTE: "example.*" is reserved for examples in this and other
  standards; "poster" has a special meaning in the Followup-To
  header field; "to.*" is reserved for certain point-to-point
  communications in conjunction with the "ihave" control message as
  defined in RFC5537; "control.*" and "junk" have special meanings
  in some news servers; "all" is used as a wildcard in some
  implementations; and "ctl" was formerly used to indicate a
  <control-command> within the Newsgroups header field.

Path

The Path header field indicates the route taken by an article since its injection into the Netnews system. Each agent that processes an article is required to prepend at least one <path-identity> to this header field body. This is primarily so that news servers are able to avoid sending articles to sites already known to have them, in particular the site they came from. Additionally, it permits gathering statistics and tracing the route articles take in moving over the network.

path = "Path:" SP *WSP path-list tail-entry *WSP CRLF

path-list = *( path-identity [FWS] [path-diagnostic] "!" )

path-diagnostic = diag-match / diag-other / diag-deprecated

diag-match = "!" ; another "!"

diag-other = "!." diag-keyword [ "." diag-identity ] [FWS]

diag-deprecated = "!" IPv4address [FWS]

diag-keyword = 1*ALPHA ; see RFC5537

diag-identity = path-identity / IPv4address / IPv6address

tail-entry = path-nodot

                  ; may be the string "not-for-mail"

path-identity = ( 1*( label "." ) toplabel ) / path-nodot

path-nodot = 1*( alphanum / "-" / "_" ) ; legacy names

label = alphanum [ *( alphanum / "-" ) alphanum ]

toplabel = ( [ label *( "-" ) ] ALPHA *( "-" ) label ) /

                  ( label *( "-" ) ALPHA [ *( "-" ) label ] ) /
                  ( label 1*( "-" ) label )

alphanum = ALPHA / DIGIT ; compare RFC3696

A <path-identity> is a name identifying a site. It takes the form of a domain name having two or more components separated by dots, or a single name with no dots (<path-nodot>).

Each <path-identity> in the <path-list> (which does not include the <tail-entry>) indicates, from right to left, the successive agents through which the article has passed. The use of the <diag-match>, which appears as "!!", indicates that the agent to its left verified the identity of the agent to its right before accepting the article (whereas the <path-delimiter> "!" implies no such claim).

  NOTE: Historically, the <tail-entry> indicated the name of the
  sender.  If not used for this purpose, the string "not-for-mail"
  is often used instead (since at one time the whole path could be
  used as a mail address for the sender).

  NOTE: Although case-insensitive, it is intended that the
  <diag-keyword>s should be in uppercase, to distinguish them from
  the <path-identity>s, which are traditionally in lowercase.

A <path-diagnostic> is an item inserted into the Path header field for purposes other than to indicate the name of a site. The use of these is described in RFC5537.

  NOTE: One usage of a <path-diagnostic> is to record an IP address.
  The fact that <IPv6address>es are allowed means that the colon (:)
  is permitted; note that this may cause interoperability problems
  at older sites that regard ":" as a <path-delimiter> and have
  neighbors whose names have 4 or fewer characters, and where all
  the characters are valid HEX digits.

  NOTE: Although <IPv4address>es have occasionally been used in the
  past (usually with a diagnostic intent), their continued use is
  deprecated (though it is still acceptable in the form of the
  <diag-deprecated>).

Subject

The Subject header field is the same as that specified in Section 3.6.5 of RFC5322, with the added restrictions detailed above in Section 2.2. Further discussion of the content of the Subject header field appears in RFC5537 and [USEAGE].

subject = "Subject:" SP unstructured CRLF

Optional Header Fields

None of the header fields appearing in this section are required to appear in every article, but some of them may be required in certain types of articles. Further discussion of these requirements appears in RFC5537 and [USEAGE].

The header fields Comments, Keywords, Reply-To, and Sender are used in Netnews articles in the same circumstances and with the same meanings as those specified in RFC5322, with the added restrictions detailed above in Section 2.2. Multiple occurrences of the Keywords header field are not permitted.

comments = "Comments:" SP unstructured CRLF

keywords = "Keywords:" SP phrase *("," phrase) CRLF

reply-to = "Reply-To:" SP address-list CRLF

sender = "Sender:" SP mailbox CRLF

The MIME header fields MIME-Version, Content-Type, Content-Transfer- Encoding, Content-Disposition, and Content-Language are used in Netnews articles in the same circumstances and with the same meanings as those specified in RFC2045, RFC2183, and RFC3282, with the added restrictions detailed above in Section 2.2.

All remaining news header fields are described below.

Approved

The Approved header field indicates the mailing addresses (and possibly the full names) of the persons or entities approving the article for posting. Its principal uses are in moderated articles and in group control messages; see RFC5537.

approved = "Approved:" SP mailbox-list CRLF

Archive

The Archive header field provides an indication of the poster's intent regarding preservation of the article in publicly accessible long-term or permanent storage.

archive = "Archive:" SP [CFWS] ("no" / "yes")

                  *( [CFWS] ";" [CFWS] archive-param ) [CFWS] CRLF

archive-param = parameter

The presence of an Archive header field in an article with a field body of "no" indicates that the poster does not permit redistribution from publicly accessible long-term or permanent archives. A field body of "yes" indicates that the poster permits such redistribution.

No <parameter>s are currently defined; if present, they can be ignored. Further discussion of the use of the Archive header field appears in [USEAGE].

Control

The Control header field marks the article as a control message and specifies the desired actions (in addition to the usual actions of storing and/or relaying the article).

control = "Control:" SP *WSP control-command *WSP CRLF

control-command = verb *( 1*WSP argument )

verb = token

argument = 1*( %x21-7E )

The verb indicates what action should be taken, and the argument(s) (if any) supply details. In some cases, the <body> (as defined in RFC5322) of the article may also contain details. The legal verbs and respective arguments are discussed in the companion document, RFC5537.

An article with a Control header field MUST NOT also have a Supersedes header field.

Distribution

The Distribution header field specifies geographic or organizational limits on an article's propagation.

distribution = "Distribution:" SP dist-list CRLF

dist-list = *WSP dist-name

                  *( [FWS] "," [FWS] dist-name ) *WSP

dist-name = ALPHA / DIGIT

                  *( ALPHA / DIGIT / "+" / "-" / "_" )

The <dist-name>s "world" and "local" are reserved. "world" indicates unlimited distribution and SHOULD NOT be used explicitly, since it is the default when the Distribution header field is absent entirely. "local" is reserved for indicating distribution only to the local site, as defined by local software configuration.

"All" MUST NOT be used as a <dist-name>. <dist-name>s SHOULD contain at least three characters, except when they are two-letter country codes drawn from [ISO3166-1]. <dist-name>s are case-insensitive (i.e., "US", "Us", "uS", and "us" all specify the same distribution).

Optional FWS in the <dist-list> SHOULD NOT be generated, but MUST be accepted.

Expires

The Expires header field specifies a date and time when the poster deems the article to be no longer relevant and could usefully be removed ("expired").

  NOTE: This header field is useful when the poster desires an
  unusually long or an unusually short expiry time.

expires = "Expires:" SP date-time CRLF

See the remarks under Section 3.1.1 regarding the syntax of <date-time> and the requirements and recommendations to which it is subject.

  NOTE: The Expires header field is also sometimes used in Email
  with a similar meaning; see RFC2156.

Followup-To

The Followup-To header field specifies to which newsgroup(s) the poster has requested that followups are to be posted. The Followup-To header field SHOULD NOT appear in a message, unless its content is different from the content of the Newsgroups header field.

followup-to = "Followup-To:" SP ( newsgroup-list / poster-text )

                  CRLF

poster-text = *WSP %d112.111.115.116.101.114 *WSP

                  ; "poster" in lowercase

The syntax is the same as that of the Newsgroups (Section 3.1.4) header field, with the exception that the keyword "poster" requests that followups should be emailed directly to the article's poster (using the addresses contained in the Reply-To header field if one exists, otherwise using the addresses contained in the From header field) rather than posted to any newsgroups. Agents MUST generate the keyword "poster" in lowercase, but MAY choose to recognize case- insensitive forms such as "Poster".

As in the Newsgroups (Section 3.1.4) header field, optional FWS in the <newsgroup-list> SHOULD NOT be generated, but MUST be accepted.

Injection-Date

The Injection-Date header field contains the date and time that the article was injected into the network. Its purpose is to enable news servers, when checking for "stale" articles, to use a <date-time> that was added by a news server at injection time rather than one added by the user agent at message composition time.

This header field MUST be inserted whenever an article is injected. However, software that predates this standard does not use this header, and therefore agents MUST accept articles without the Injection-Date header field.

injection-date = "Injection-Date:" SP date-time CRLF

See the remarks under Section 3.1.1 regarding the syntax of <date-time> and the requirements and recommendations to which it is subject.

  NOTE: Since clocks on various agents are not necessarily
  synchronized, the <date-time> in this header field might not be a
  later value than that in the Date header field.  Agents MUST NOT
  alter a pre-existing Date header field when adding an Injection-
  Date header field.

This header field is intended to replace the currently used but undocumented "NNTP-Posting-Date" header field, whose use is now deprecated.

Injection-Info

The Injection-Info header field contains information provided by the injecting news server as to how an article entered the Netnews system; it assists in tracing the article's true origin. It can also specify one or more addresses where complaints concerning the poster of the article may be sent.

injection-info = "Injection-Info:" SP [CFWS] path-identity

                  [CFWS] *( ";" [CFWS] parameter ) [CFWS] CRLF

  NOTE: The syntax of <parameter> (Section 5.1 of RFC2045, as
  amended by RFC2231), taken in conjunction with the folding rules
  of RFC0822 (note: not RFC2822 or RFC5322), effectively
  allows [CFWS] to occur on either side of the "=" inside a
  <parameter>.

The following table gives the <attribute> and the format of the <value> for each <parameter> defined for use with this header field. At most, one occurrence of each such <parameter> is allowed.

<attribute> format of <value>

---

-----------------

"posting-host" a <host-value> "posting-account" any <value> "logging-data" any <value> "mail-complaints-to" an <address-list>

where

host-value = dot-atom-text / IPv4address / IPv6address /

                  (dot-atom-text ":" ( IPv4address / IPv6address ))

  NOTE: Since any such <host-value> or <address-list> also has to be
  a syntactically correct <value>, it will usually be necessary to
  encapsulate it as a <quoted-string>, for example:

   posting-host = "posting.example.com:192.0.2.1"

Other <attribute>s SHOULD NOT be used unless defined in extensions to this standard. If non-standards-based <attribute>s are used, they MUST begin with an "x-".

Although comments and folding of whitespace are permitted throughout the Injection-Info header field, folding SHOULD NOT be used within any <parameter>. Folding SHOULD only occur before or after the ";" separating <parameter>s, and comments SHOULD only be used following the last <parameter>.

  NOTE: Some of this information has previously been sent in non-
  standardized header fields such as NNTP-Posting-Host, X-Trace,
  X-Complaints-To, and others.  Once a news server generates an
  Injection-Info header field, it should have no need to send these
  non-standard header fields.

The "posting-host" <parameter> specifies the Fully Qualified Domain Name (FQDN) and/or IP address (IPv4address or IPv6address) of the host from which the news server received the article.

  NOTE: If the "posting-host" <parameter> fails to deterministically
  identify the host (e.g., dynamic IP address allocation), the
  "posting-account" or "logging-data" <parameter> may provide
  additional information about the true origin of the article.

The "posting-account" <parameter> identifies the source from which that news server received the article, in a notation that can be interpreted by the news server administrator. This notation can include any information the administrator deems pertinent. In order to limit the exposure of personal data, it SHOULD be given in a form that cannot be interpreted by other sites. However, to make it useful for rate limiting and abuse detection, two messages posted from the same source SHOULD have the same value of "posting-account", and two messages from different sources SHOULD have differing values of "posting-account". The exact definition of "source" is left to the discretion of the news server administrator.

The "logging-data" <parameter> contains information (typically a session number or other non-persistent means of identifying a posting account) that will enable the true origin of the article to be determined by reference to logging information kept by the news server.

The "mail-complaints-to" <parameter> specifies one or more mailboxes for sending complaints concerning the behavior of the poster of the article.

It is a matter of local policy which of the above <parameter>s to include. Some pieces of information have privacy implications; this is discussed in [USEAGE].

Organization

The Organization header field is a short phrase identifying the poster's organization.

organization = "Organization:" SP unstructured CRLF

  NOTE: There is no "s" in Organization.

3.2.10. References

The References header field is the same as that specified in Section 3.6.4 of RFC5322, with the added restrictions detailed above in Section 2.2 and those listed below:

o The updated <msg-id> construct defined in Section 3.1.3 MUST be

  used.

o Message identifiers MUST be separated with CFWS.

o Comments in CFWS between message identifiers can cause

  interoperability problems, so comments SHOULD NOT be generated but
  MUST be accepted.

references = "References:" SP [CFWS] msg-id *(CFWS msg-id)

                  [CFWS] CRLF

3.2.11. Summary

The Summary header field is a short phrase summarizing the article's content.

summary = "Summary:" SP unstructured CRLF

3.2.12. Supersedes

The Supersedes header field contains a message identifier specifying an article to be superseded upon the arrival of this one. An article containing a Supersedes header field is equivalent to a "cancel" RFC5537 control message for the specified article, followed immediately by the new article without the Supersedes header field.

supersedes = "Supersedes:" SP *WSP msg-id *WSP CRLF

  NOTE: There is no "c" in Supersedes.

  NOTE: The Supersedes header field defined here has no connection
  with the Supersedes header field that sometimes appears in Email
  messages converted from X.400 according to RFC2156; in
  particular, the syntax here permits only one <msg-id> in contrast
  to the multiple <msg-id>s in that Email version.

3.2.13. User-Agent

The User-Agent header field contains information about the user agent (typically a newsreader) generating the article, for statistical purposes and tracing of standards violations to specific software in need of correction. It is intended that this header field be suitable for use in Email.

user-agent = "User-Agent:" SP 1*product [CFWS] CRLF

product = [CFWS] token [ [CFWS] "/" product-version ]

product-version = [CFWS] token

This header field MAY contain multiple <product> tokens identifying the user agent and any subproducts that form a significant part of it, listed in order of their significance for identifying the application.

  NOTE: Some of this information has previously been sent in non-
  standardized header fields such as X-Newsreader, X-Mailer,
  X-Posting-Agent, X-Http-User-Agent, and others.  Once a user agent
  generates a User-Agent header field, it should have no need to
  send these non-standard header fields.

  NOTE: RFC2616 describes a similar facility for the HTTP
  protocol.  The Netnews article format differs in that "{" and "}"
  are allowed in tokens (<product> and <product-version>) and
  comments are permitted wherever white space is allowed.

3.2.14. Xref

The Xref header field indicates where an article was filed by the last news server to process it. User agents often use the information in the Xref header field to avoid multiple processing of crossposted articles.

xref = "Xref:" SP *WSP server-name

                  1*( FWS location ) *WSP CRLF

server-name = path-identity

location = newsgroup-name ":" article-locator

article-locator = 1*( %x21-27 / %x29-3A / %x3C-7E )

                  ; US-ASCII printable characters
                  ; except '(' and ';'

The <server-name> is included so that software can determine which news server generated the header field. The locations specify where the article is filed -- i.e., under which newsgroups (which may differ from those in the Newsgroups header field), and where under those newsgroups. The exact form of an <article-locator> is implementation-specific.

  NOTE: The traditional form of an <article-locator> (as required by
  RFC3977) is a decimal number, with articles in each newsgroup
  numbered consecutively starting from 1.

Obsolete Header Fields

The header fields Date-Received, Posting-Version, and Relay-Version defined in RFC0850, as well as Also-Control, Article-Names, Article-Updates, and See-Also defined in [Son-of-1036] are declared obsolete. See the cited specification documents for further information on their original use.

These header fields MUST NOT be generated and SHOULD be ignored.

Lines

The Lines header field indicates the number of lines in the <body> (as defined in RFC5322) of the article.

lines = "Lines:" SP *WSP 1*DIGIT *WSP CRLF

The line count is the number of CRLF separators in the <body>.

Historically, this header field was used by the NNTP RFC3977 overview facility, but its use for this purpose is now deprecated. As a result, this header field is to be regarded as obsolescent, and it is likely to be removed entirely in a future version of this standard. All agents SHOULD ignore it and SHOULD NOT generate it.

Internationalization Considerations

Internationalization of Netnews article header fields and bodies is provided using the MIME mechanisms discussed in Section 2.3. Note that the generation of internationalized <newsgroup-name>s for use in header fields is not addressed in this document.

Security Considerations

The Netnews article format specified in this document does not provide any security services, such as confidentiality, authentication of sender, or non-repudiation. Instead, such services need to be layered above, using such protocols as S/MIME RFC3851 or PGP/MIME (Pretty Good Privacy / MIME) RFC3156, or below, using secure versions of news transport protocols. Additionally, several currently non-standardized protocols such as [PGPVERIFY] may be standardized in the near future.

Message identifiers (Section 3.1.3) in Netnews articles are required to be unique; articles may be refused (in server-to-server transfer) if the identifier has already been seen. If a malicious agent can predict the identifier of an article, it can preempt the article by posting its own article (possibly to a quite different group) with

the same message identifier, thereby preventing the target article from propagating. Therefore, agents that generate message identifiers for Netnews articles SHOULD ensure that they are unpredictable.

MIME security considerations are discussed in RFC2046. Note that the full range of encodings allowed for parameters in RFC2046 and RFC2231 permits constructs that simple parsers may fail to parse correctly; examples of hard-to-parse constructs are:

Content-Type: multipart/mixed

 (; boundary=foo ; xyz=");bOuNdArY*=next%20part(")

Content-Type: multipart/digest;

 boundary (not=me) = ("yes ;-) simple (foo;bar") ; x-foo = xyzzy

Such deficiencies in parsing may be used as part of an attack.

Further security considerations are discussed in RFC5537.

IANA Considerations

IANA has registered the following header fields in the Permanent Message Header Field Repository, in accordance with the procedures set out in RFC3864.

  Header field name: Also-Control
  Applicable protocol: netnews
  Status: obsoleted
  Author/change controller: IETF
  Specification document(s): [Son-of-1036] (Section 6.15)

  Header field name: Approved
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.2.1)

  Header field name: Archive
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.2.2)

  Header field name: Article-Names
  Applicable protocol: netnews
  Status: obsoleted
  Author/change controller: IETF
  Specification document(s): [Son-of-1036] (Section 6.17)

  Header field name: Article-Updates
  Applicable protocol: netnews
  Status: obsoleted
  Author/change controller: IETF
  Specification document(s): [Son-of-1036] (Section 6.18)

  Header field name: Comments
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.2),
  RFC5322 (Section 3.6.5)

  Header field name: Control
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.2.3)

  Header field name: Date
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.1.1),
  RFC5322 (Section 3.6.1)

  Header field name: Date-Received
  Applicable protocol: netnews
  Status: obsoleted
  Author/change controller: IETF
  Specification document(s): RFC0850 (Section 2.2.4)

  Header field name: Distribution
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.2.4)

  Header field name: Expires
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.2.5)

  Header field name: Followup-To
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.2.6)

  Header field name: From
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.1.2),
  RFC5322 (Section 3.6.2)

  Header field name: Injection-Date
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.2.7)

  Header field name: Injection-Info
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.2.8)

  Header field name: Keywords
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.2),
  RFC5322 (Section 3.6.5)

  Header field name: Lines
  Applicable protocol: netnews
  Status: deprecated
  Author/change controller: IETF
  Specification document(s): This document (Section 3.3.1)
  Related information: RFC3977 (Section 8.1)

  Header field name: Message-ID
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.1.3)
  Related information: RFC5322 (Section 3.6.4)

  Header field name: Newsgroups
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.1.4)

  Header field name: NNTP-Posting-Date
  Applicable protocol: netnews
  Status: obsoleted
  Author/change controller: IETF
  Specification document(s): none

  Header field name: NNTP-Posting-Host
  Applicable protocol: netnews
  Status: obsoleted
  Author/change controller: IETF
  Specification document(s): RFC2980 (Section 3.4.1)

  Header field name: Organization
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.2.9)

  Header field name: Path
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.1.5)

  Header field name: Posting-Version
  Applicable protocol: netnews
  Status: obsoleted
  Author/change controller: IETF
  Specification document(s): RFC0850 (Section 2.1.2)

  Header field name: References
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.2.10),
  RFC5322 (Section 3.6.4)

  Header field name: Relay-Version
  Applicable protocol: netnews
  Status: obsoleted
  Author/change controller: IETF
  Specification document(s): RFC0850 (Section 2.1.1)

  Header field name: Reply-To
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.2),
  RFC5322 (Section 3.6.2)

  Header field name: See-Also
  Applicable protocol: netnews
  Status: obsoleted
  Author/change controller: IETF
  Specification document(s): [Son-of-1036] (Section 6.16)

  Header field name: Sender
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.2),
  RFC5322 (Section 3.6.2)

  Header field name: Subject
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.1.6),
  RFC5322 (Section 3.6.5)

  Header field name: Summary
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.2.11)

  Header field name: Supersedes
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.2.12)

  Header field name: User-Agent
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.2.13)
  Related information: RFC2616 (Section 14.43)

  Header field name: Xref
  Applicable protocol: netnews
  Status: standard
  Author/change controller: IETF
  Specification document(s): This document (Section 3.2.14)

References

Normative References

RFC2045 Freed, N. and N. Borenstein, "Multipurpose Internet

              Mail Extensions (MIME) Part One: Format of Internet
              Message Bodies", RFC 2045, November 1996.

RFC2046 Freed, N. and N. Borenstein, "Multipurpose Internet

              Mail Extensions (MIME) Part Two: Media Types",
              RFC 2046, November 1996.

RFC2047 Moore, K., "MIME (Multipurpose Internet Mail

              Extensions) Part Three: Message Header Extensions for
              Non-ASCII Text", RFC 2047, November 1996.

RFC2049 Freed, N. and N. Borenstein, "Multipurpose Internet

              Mail Extensions (MIME) Part Five: Conformance Criteria
              and Examples", RFC 2049, November 1996.

RFC2119 Bradner, S., "Key words for use in RFCs to Indicate

              Requirement Levels", BCP 14, RFC 2119, March 1997.

RFC2183 Troost, R., Dorner, S., and K. Moore, "Communicating

              Presentation Information in Internet Messages: The
              Content-Disposition Header Field", RFC 2183,
              August 1997.

RFC2231 Freed, N. and K. Moore, "MIME Parameter Value and

              Encoded Word Extensions: Character Sets, Languages,
              and Continuations", RFC 2231, November 1997.

RFC3282 Alvestrand, H., "Content Language Headers", RFC 3282,

              May 2002.

RFC3986 Berners-Lee, T., Fielding, R., and L. Masinter,

              "Uniform Resource Identifier (URI): Generic Syntax",
              STD 66, RFC 3986, January 2005.

RFC5234 Crocker, D. and P. Overell, "Augmented BNF for Syntax

              Specifications: ABNF", STD 68, RFC 5234, January 2008.

RFC5322 Resnick, P., Ed., "Internet Message Format", RFC 5322,

              October 2008.

RFC5537 Allbery, R., Ed. and C. Lindsey, "Netnews Architecture

              and Protocols", RFC 5537, November 2009.

Informative References

[Errata] "RFC Editor Errata",

              <http://www.rfc-editor.org/errata.php>.

[ISO3166-1] International Organization for Standardization, "ISO

              3166-1:1997. Codes for the representation of names of
              countries and their subdivisions -- Part 1: Country
              codes", 1997.

[PGPVERIFY] Lawrence, D., "Authentication of Usenet Group Changes

              (pgpverify)", June 1999,
              <ftp://ftp.isc.org/pub/pgpcontrol/README.html>.

RFC0822 Crocker, D., "Standard for the format of ARPA Internet

              text messages", STD 11, RFC 822, August 1982.

RFC0850 Horton, M., "Standard for interchange of USENET

              messages", RFC 850, June 1983.

RFC1036 Horton, M. and R. Adams, "Standard for interchange of

              USENET messages", RFC 1036, December 1987.

RFC2156 Kille, S., "MIXER (Mime Internet X.400 Enhanced

              Relay): Mapping between X.400 and RFC 822/MIME",
              RFC 2156, January 1998.

RFC2616 Fielding, R., Gettys, J., Mogul, J., Frystyk, H.,

              Masinter, L., Leach, P., and T. Berners-Lee,
              "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616,
              June 1999.

RFC2822 Resnick, P., "Internet Message Format", RFC 2822,

              April 2001.

RFC2980 Barber, S., "Common NNTP Extensions", RFC 2980,

              October 2000.

RFC3156 Elkins, M., Del Torto, D., Levien, R., and T.

              Roessler, "MIME Security with OpenPGP", RFC 3156,
              August 2001.

RFC3696 Klensin, J., "Application Techniques for Checking and

              Transformation of Names", RFC 3696, February 2004.

RFC3851 Ramsdell, B., "Secure/Multipurpose Internet Mail

              Extensions (S/MIME) Version 3.1 Message
              Specification", RFC 3851, July 2004.

RFC3864 Klyne, G., Nottingham, M., and J. Mogul, "Registration

              Procedures for Message Header Fields", BCP 90,
              RFC 3864, September 2004.

RFC3977 Feather, C., "Network News Transfer Protocol (NNTP)",

              RFC 3977, October 2006.

[Son-of-1036] Spencer, H., "Son of 1036: News Article Format and

              Transmission", Work in Progress, May 2009.

[USEAGE] Lindsey, C., "Usenet Best Practice", Work in Progress,

              March 2005.

Appendix A. Acknowledgments

As this document is the result of an eight-year effort, the number of people that have contributed to its content are too numerous to mention individually. Many thanks go out to all past and present members of the USEFOR Working Group of the Internet Engineering Task Force (IETF) and its accompanying mailing list.

Appendix B. Differences from RFC 1036 and Its Derivatives

This appendix contains a list of changes that have been made in the Netnews article format from earlier standards, specifically RFC1036.

o The RFC5322 conventions for parenthesis-enclosed <comment>s in

  header fields are supported in all newly defined header fields and
  in header fields inherited from RFC5322.  They are, however,
  still disallowed for performance and/or compatibility reasons in
  the Control, Distribution, Followup-To, Lines, Message-ID,
  Newsgroups, Path, Supersedes, and Xref header fields.

o Multiple addresses are allowed in the From header field.

o [FWS] is permitted in Newsgroups header fields.

o An enhanced syntax for the Path header field enables the injection

  point of, and the route taken by, an article to be determined with
  more precision.

o Only one (1) message identifier is allowed in the Supersedes

  header field.

o MIME is recognized as an integral part of Netnews.

o There is a new Injection-Date header field to make the rejection

  of stale articles more precise and to minimize spurious
  rejections.

o There are several new optional header fields defined, notably

  Archive, Injection-Info, and User-Agent, leading to increased
  functionality.

o Certain header fields, notably Lines, have been deprecated or made

  obsolete (Section 3.3).

o The convention to interpret subjects starting with the word "cmsg"

  as a control message was removed.

o There are numerous other small changes, clarifications, and

  enhancements.

Appendix C. Differences from RFC 5322

This appendix lists the differences between the syntax allowed by the Netnews article format (this document) as compared to the Internet Message Format, as specified in RFC5322.

The Netnews article format is a strict subset of the Internet Message Format; all Netnews articles conform to the syntax of RFC5322.

The following restrictions are important:

o A SP (space) is REQUIRED after the colon (':') following a header

  field name.

o A slightly restricted syntax of <msg-id> (to be used by the

  Message-ID, References, and Supersedes header fields) is defined.

o The length of a <msg-id> MUST NOT exceed 250 octets.

o Comments are not allowed in the Message-ID header field.

o The CFWS between <msg-id>s in the References header field is not

  optional.

o It is legal for a parser to reject obsolete syntax, except that:

  *  The <obs-phrase> construct MUST be accepted.

  *  The obsolete <zone> "GMT" MUST be accepted within a
     <date-time>.

o Every line of a header field body (including the first and any

  that are subsequently folded) MUST contain at least one non-
  whitespace character.  This means that an empty header field body
  is illegal.

Authors' Addresses

Kenneth Murchison (editor) Carnegie Mellon University 5000 Forbes Avenue Cyert Hall 285 Pittsburgh, PA 15213 U.S.A.

Phone: +1 412 268 2638 EMail: [email protected]

Charles H. Lindsey University of Manchester 5 Clerewood Avenue Heald Green Cheadle Cheshire SK8 3JU U.K.

Phone: +44 161 436 6131 EMail: [email protected]

Dan Kohn Healing Thresholds 211 N End Ave Apt 22E New York, NY 10282 U.S.A.

Phone: +1 415 233 1000 EMail: [email protected]