M. Rose
 Invisible Worlds, Inc.
 March 17, 2001

BEEP: Building Blocks for Application Protocols


Historically, each Internet application protocol has defined its own set of rules for exchanging control and data information. Although there are a lot of similarities (e.g., dot-stuffing in SMTP and POP, octet counts in HTTP and IMAP, 3-digit reply codes in FTP and SMTP), there isn't a lot of reuse of design.

As a result, designers of new application protocols usually have to make the same set of design decisions — over and over again — to deal with the same set of trade-offs, but usually without the benefit of institutional history.

BEEP is a generic application protocol kernel for connection-oriented, request-response interactions. This particular subset supports a large class of Internet applications, and provides solutions to common design issues for those applications, including: framing, encoding, error reporting, asynchronous exchanges, along with authentication and privacy.


Table of Contents


1. Helpful Reading

BEEP RFCs 3080, 3081

Other I-Ds:


2. The Problem in Application Protocol Design

A lot of application protocols, a lot of similarities, not a lot of reuse

Need an application protocol? Three choices:

If choosing the first option involves little or no work, take it

If the application protocol's requirements are a good fit with the web or email, use it

Otherwise, let's talk about coming up with a decent 90% solution


3. draft-moore-using-http

HTTP has a lot of things to recommend it

The question is how good is the fit with a synchronous, small-request/large-response model

Experience with IPP suggests a subtle shift in existing practice has unintended consequences:

Regardless, the costs of introducing new methods, URL-prefixes, etc., probably neutralize the benefits of reuse


4. Application Protocol Taxonomies

Five operational characteristics:

For example:


5. The Target Market

Limiting the problem domain:

This doesn't accommodate things like:

That's OK... we can define alternative frameworks later, according to the demands of the standards marketplace


6. Overview of Protocol Mechanisms

There are (at least) six tasks that application protocols have to perform:

There are other important (application-specific) tasks, e.g., naming and authorization


7. Protocol Mechanisms — Framing

Three Approaches:

Octet-stuffing tends to be slower due to comparison overhead

Octet-counting, in the absence of segmentation, requires the entire message be available prior to sending

Connection-blasting, for small messages, is high overhead; regardless, endpoint-addresses must be conveyed at the application layer


8. Protocol Mechanisms — Encoding

Quite a few to choose from:

There are arguments for and against each

Sometimes we have to just pick one and move on


9. Protocol Mechanisms — Reporting

Historically, there is consensus on SMTP's "theory" of reply codes:

Some successes at having the same code mean the same thing in different protocols

Typically, paired with an ameri-centric textual diagnostic

There are also weaknesses:


10. Protocol Mechanisms — Asynchrony

Some success here (the second time around):

Doing asynchrony right requires:

Segmentation allows octet-counters to send segments without the entire message being available


11. Protocol Mechanisms — Authentication

Historically, not a problem, because we didn't do it

Hysterically, if we do it, we do it differently each time (e.g., FTP and HTTP)

The IETF has retrofitted many application protocols to use the SASL framework

SASL solves an NxN problem:


12. Protocol Mechanisms — Privacy

Historically, not a problem, because we didn't do it

Then Netscape gave us SSL (and screwed up by using two ports per application protocol)

The IETF has retrofitted many application protocols to support TLS

(Graphic courtesy of Bob "RL" Morgan, University of Washington)


13. Protocol Mechanisms — Summary

     Mechanism  ESMTP        FTP        HTTP1.1
     ---------  -----        ---        -------
       Framing  stuffing     blasting   counting 
      Encoding  822-style    binary     MIME 
     Reporting  3-digit      3-digit    3-digit 
    Asynchrony  pipelining   none       pipelining/ chunking 
Authentication  SASL         user/pass  user/pass 
       Privacy  SASL or TLS  none       TLS (nee SSL) 


14. Overview of Application Protocol Design Principles

Regardless of what the application protocol does, a well-designed protocol is:


15. Application Protocol Design Principles — Scalability

Surprisingly few application protocols are network-friendly:



16. Application Protocol Design Principles — Efficiency

Elegance is sometimes inefficient (e.g., octet-stuffing v. octet-counting)

Trade-offs are often necessary:

The SOAP guys are probably right that XML's inefficiencies are offset by printf's simplicity

The key lesson is to adjust the razor based on:


17. Application Protocol Design Principles — Simplicity

One designer's complexity is another's completeness

The pain should be proportional to the gain:

Two ways of doing the exact same thing suggests poor design

Simplistic doesn't imply simple-minded:


18. Application Protocol Design Principles — Extensibility

Anticipating the unforeseen is a problem:

Extensibility is no solution to poor design (viz. Telnet)


19. Application Protocol Design Principles — Robustness

Postel's Robustness Principle is a two-edged sword:


20. Overview of the BEEP Core

     Mechanism  BEEP
     ---------  ----
       Framing  counting with a trailer
      Encoding  MIME, defaulting to application/octet-stream
     Reporting  3-digit with localized textual diagnostic
    Asynchrony  independent channels
Authentication  SASL
       Privacy  SASL or TLS


21. The BEEP Core — Framing

The header is:

    tag SP channel SP msgno SP more SP seqno SP size CRLF

Several "consistency rules" used to validate each frame

The payload is 0 or more octets

The trailer is:

    "END" CRLF

So, what's a channel?


22. The BEEP Core — Channels and Profiles

A profile defines the messages exchanged on a given channel

At startup, channel 0 is bound to the channel management profile

RPY 0 0 . 0 124 
Content-Type: application/beep+xml

  <profile uri="" />
  <profile uri="" />
  <profile uri="" />
  <profile uri="" />

<start number="1">
  <profile uri="..."> ... </profile>
  <profile uri="..."> ... </profile>

<close number="1" code="200" />


23. The BEEP Core — Profile Definitions

A registration template defines:

Two kinds of profiles:

Protocol =

    BEEP + 1 or more profiles
         + authorization policies
         + provisioning rules


    MSG -> RPY
    MSG -> ERR
    MSG -> zero or more ANS, followed by NUL


24. The BEEP Core — Encoding

All payloads are MIME-encoded, by default:

Content-Type: application/octet-stream
Content-Transfer-Encoding: binary

BEEP defines a minimal XML media type, application/beep+xml, i.e.,


25. The BEEP Core — Reporting

Each profile defines messages for both positive and negative replies

For negative replies, the channel management profile uses 3 digit reply codes with a localized diagnostic, e.g.,

    <error code="421"
           xml:lang="en-US">service not available</error>

A list of preferred languages is passed in the <greeting> when a BEEP session is established


26. The BEEP Core — Asynchrony

Channels handle asynchrony:

Segmentation is handled by framing

So, what about flow control?


27. The BEEP Core — Transport Mappings

A BEEP session is connection-oriented, and mapped onto a transport service that defines how:

At present, one mapping is defined: onto a single TCP connection

In the future, look for transport mappings onto:


28. The BEEP Core — Flow Control

The TCP-based mapping re-introduces the window-based flow control mechanism of TCP, each channel:

When a channel is created, it has a 4K window, after that it's managed:

    "SEQ" SP channel SP ackno SP size 

BEEP peers expected to use basic fairness algorithms, e.g.,


29. The BEEP Core — Authentication

Each BEEP session has a single authentication/authorization identity:

Start a channel with a SASL profile, e.g.,

<start number="1">
    <profile uri="">

The <blob> is used to carry the binary SASL exchanges using base64 encoding

Once you successfully authenticate:

Note that the SASL mechanism may provide both user authentication and transport security:


30. The BEEP Core — Privacy

Each BEEP session has a single privacy context:

Either start a channel with a SASL profile that has a security layer, or start a channel with the TLS profile, e.g.,

<start number="1">
    <profile uri="">
        &lt;ready />

When the underlying negotiation process begins, all channels are closed

At the end of the negotiation, a new greeting is issued, regardless of the outcome

The usual STARTTLS rules apply, a BEEP peer:


31. Overview of Application Protocols in Progress

Several applications in varying states of progress:


32. Application Protocols in Progress — Metadata Management

A client/server relationship with asynchronous callbacks:

    C: establish and tune session
    C: MSG (tell me about)
    S: RPY (here is what i know right now)
    S: MSG (here is something new)
    C: RPY (ok)


33. Application Protocols in Progress — syslog

A server/client relationship:

    C: establish and tune session
    S: MSG (i'm ready to act as a sink)
    C: ANS (audit entry 1)
    C: ANS (audit entry 2)
    C: ANS (audit entry N)
    C: NUL


34. Application Protocols in Progress — intrusion detection

An interesting requirement:

Realized using a "tunneling" tuning profile:

     C: establish session to proxy #1
     C: start tunneling profile
     C: specify route (via IP or SRV)

    P1: establish session to proxy #2
    P1: start tunneling profile
    P1: specify remaining route (via IP or SRV)
    Pn: establish session to server
    Pn: start pass-through profile
    Pn: specify empty route

     S: pass back status to Pn

If successful:


35. Application Protocols in Progress — APEX

Application-layer relaying:

Behavior of mesh modified by self-contained options, e.g.,

Starting to look at end-system multicasting


36. Status

BEEP's core and TCP mapping specifications:

Other IETF activities gearing up

The Working Group: [email protected]

The Website:

Open Source Implementations (in progress):