Internet-Draft Document Review and AD Workload March 2023
Nottingham Expires 1 October 2023 [Page]
Workgroup:
Network Working Group
Internet-Draft:
draft-nottingham-iesg-review-workload-00
Published:
Intended Status:
Standards Track
Expires:
Author:
M. Nottingham

IESG Document Review Expectations: Impact on AD Workload

Abstract

Arguably, IETF Area Directors are overloaded with document review duties. This document surveys the relevant background, discusses the implications, and makes a proposal for improvements.

About This Document

This note is to be removed before publishing as an RFC.

Status information for this document may be found at https://datatracker.ietf.org/doc/draft-nottingham-iesg-review-workload/.

information can be found at https://mnot.github.io/I-D/.

Source for this draft and an issue tracker can be found at https://github.com/mnot/I-D/labels/ad-workload.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 1 October 2023.

Table of Contents

1. Introduction

The job of an IETF Area Director is notoriously difficult. Beyond the impact on the individual, this reputation is widely thought to reduce the pool of potential candidates for the position, resulting in a lack of diversity, vulnerability to failure to find a candidate, or the possibility of having to accept a poor candidate. See Section 2.6.2 of [RFC3774] for further discussion.

One of the key responsibilities of an Area Director is reviewing each document that comes to the IESG for publication. Although Area Directors do much more than simply review documents, the sheer volume of pages that the IETF publishes makes this a significant component of the job, in terms of both their time and attention.

Section 2 surveys relevant background materials. Section 3 discusses the requirements for document review in the IETF process, and Section 4 makes a proposal to modify the Area Director's role in it, in order to make it possible for more people to consider putting their hands up to fill the position.

2. Background

The responsibility of an Area Director to review all documents being considered for publication originates in Section 6.1.2 of [RFC2026], which describes the responsibilities of the IESG when reviewing a specification for approval:

The criteria referred to regard the maturity of the document as indicated by its stability, its resolution of design choices, level of community review, implementation and operational experience.

Section 5 of [RFC3710] states that:

However, that document notes that "[i]t does not claim to represent consensus of the IETF" but rather "was written as a 'documentation of existing practice'".

The IESG has created internal policies to ensure that this goal of sufficient review is achieved. The IESG Ballot Procedures document describes the system used. Notably, the description of the "No Objection" position includes this statement:

This language implies that Area Directors need not read every page of every document; it is an "escape clause" for an overloaded AD.

Expectations for document review are also set by the job descriptions used by the NOMCOM. The General IESG Member Expertise desired by the 2023 NOMCOM includes:

However, it later sets a more onerous expectation:

... which implies that they should be reading every draft.

3. Discussion

IESG document review undeniably serves an important function: maintaining the output quality of IETF specifications, by making Area Directors both responsible for document review and accountable to the community (through transparency mechanisms like the Open Mic session at plenaries, and ultimately through the risk of appeal and recall). This accountability and quality enhances the legitimacy of the IETF's work, and ultimately, its success.

However, the expectations for review are not clearly stated: while there are clear affordances in the IESG-internal policies and in the NOMCOM job description for an AD to skip a detailed review of the document, many IETF participants and Area Directors alike seem to believe that ADs have a fundamental responsibility to review every document published for any concerns they may have -- to "read on the order of 500 pages [...] every two weeks."

These conflicting and ill-stated expectations arguably have the effect of discouraging many qualified candidates from applying for Area Director positions.

A solution should:

These requirements rule out many potential solutions. For example, allowing Area Directors to delegate their reviews to Directorates would be seen to harm the accountability relationship, since the person reviewing the document would no longer be directly accountable to the community.

On the other hand, it is clear that Area Directors are not expected to be expert in every technology that crosses their doorstep; per the 2023 NOMCOM guidelines:

This implies that an Area Director might rely on others in their review responsibilities, but cannot avoid responsibility for them.

But what is that responsibility, exactly? Clearly, they are not exercising only their own judgement; an Area Director who refused documents based only upon their personal beliefs would effectively be a tyrant.

This is supported by the materials. At their core, the desired criteria listed in Sections 4.1 and 4.2 of [RFC2026] are all fairly objective; for example:

Any proposal, then, should also be focused on assuring these properties.

This document makes one proposal below, as a starting point for discussion; others might also meet these requirements.

4. Proposal

To clarify the nature and role of Area Director reviews and thereby partially address workload issues as described in [RFC3774], this document recommends that the policy described below be adopted, either as an IESG Statement or through IETF Consensus. Once that takes place, supporting material (such as the NOMCOM job descriptions) should be clarified and aligned with it.

4.1. Policy

This policy sets the expectation that Area Directors are responsible for assuring that, from the perspective of their Area, documents being considered for publication on the IETF stream meet the requirements in Section 4 of [RFC2026]. This implies that an Area Director can ballot "No Objection" for a document that they judge to have no implications for their Area without further review. Furthermore, they need only review those portions of other documents that do have implications for their Area.

When reviewing a document, Area Directors may rely on expertise of others to judge the desired properties; they need not be expert in every technology in their Area. However, in doing so, they do not avoid responsibility for meeting the requirements stated in [RFC2026], and they may be held accountable if those requirements are not met, from the perspective of their Area.

This policy does not prevent an Area Director from exceeding these expectations. However, Area Director reviews should be based in the requirements of [RFC2026], as elaborated upon by the DISCUSS Criteria.

4.2. Policy Discussion

This policy is not a very large change from current practice of at least some ADs, based upon discussions I've had. As such, its most important function is to level-set expectations between the community and the IESG.

Practically, this allows an AD to delegate their review (or partially do so) to someone that they judge as appropriate, based upon their expertise. However, they cannot avoid responsibility -- which means that delegating to a review directorate of volunteers may be unwise.

Overall, the expectation is that ADs can (and perhaps should) rely on other experts in their area more than they do the reviewing themselves. They are responsible for understanding and managing any objections that come from the experts they rely on, and are expected to decide the relative importance of actually requiring that any issue raised by such an expert be addressed.

5. IANA Considerations

This document has no actions for IANA.

6. Security Considerations

Undoubtedly changing how the IESG reviews documents has potential security implications. Caveat emptor.

7. Informative References

[RFC2026]
Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, DOI 10.17487/RFC2026, , <https://www.rfc-editor.org/rfc/rfc2026>.
[RFC3710]
Alvestrand, H., "An IESG charter", RFC 3710, DOI 10.17487/RFC3710, , <https://www.rfc-editor.org/rfc/rfc3710>.
[RFC3774]
Davies, E., Ed., "IETF Problem Statement", RFC 3774, DOI 10.17487/RFC3774, , <https://www.rfc-editor.org/rfc/rfc3774>.

Appendix A. Acknowledgements

Thanks to Pete Resnick for his thoughts and a snippet of text.

Author's Address

Mark Nottingham
Prahran
Australia