Initial Congestion
Exposure (ConEx) Deployment ExamplesBTB54/77, Adastral ParkMartlesham HeathIpswichIP5 3REUK+44 1473 645196bob.briscoe@bt.comhttp://bobbriscoe.net/
Transport Area
ConExInternet-DraftThis document gives examples of how ConEx deployment might get
started, focusing on unilateral deployment by a single network.This document gives examples of how ConEx deployment might get
started, focusing on unilateral deployment by a single network.The ConEx mechanism document goes to great lengths to design for
incremental deployment in all the respects below. It should be referred
to for precise details on each of these points:The ConEx mechanism is essentially a change to the source, in
order to re-insert congestion feedback into the network.Source-host-only deployment is possible without any negotiation
required, and individual transport protocol implementations within a
source host can be updated separately.Receiver modification may optionally improve ConEx for some
transport protocols with feedback limitations (TCP being the main
example), but it is not a necessityProxies for the source and/or receiver are feasible (though not
necessarily straightforward)Queues and network forwarding do not require any modification for
ConEx.ECN is not required in the network for ConEx. If some network
nodes support ECN, it can be used by ConEx.ECN is not required at the receiver for ConEx. The sender should
nonetheless attempt to negotiate ECN-usage with the receiver, given
some aspects of ConEx work better the more ECN is deployed,
particularly auditing and border measurement.Given ConEx exposes information for IP-layer policy devices to
use, the design does not preclude possible innovative uses of ConEx
information by other IP-layer devices, e.g. forwarding itselfPackets indicate whether or not they support ConEx. introduces the following
components:The ConEx Wire Protocol (currently only specified for IPv6
, although a possible way to
fit ConEx into the IPv4 header has been described )Forwarding devices (unmodified)Sender (modified for ConEx)Receiver (optionally modified)AuditPolicy Devices:Rest-of-Path Congestion Monitoring Devices (using
information from the ConEx wire protocol)Congestion Policers (using rest-of-path congestion
monitoring) should be referred
to for definitions of each of these components and further
explanation.The goal of all these ConEx elements for this scenario is to expose
information about congestion on the whole-path to a
congestion-policer. A congestion-policer is nearly identical to a
traditional token-bucket-based bit-rate policer except the tokens it
fills with arrive at a rate that represents the volume of congestion
that the customer is allowed to contribute to over time and tokens
drain from the bucket at a rate dependent on the ConEx signals
representing rest-of-path congestion.
introduces congestion-policing and explains the benefits of policing
based on congestion-volume compared to methods like weighted
round-robin traditionally used in a BRAS.Network deployment-related definitions:The first IP node a packet
traverses that is outside the source's own network. In a
residential access network scenario, for traffic from a home this
is the first IP-aware node after the home access equipment. For
Internet access from an enterprise network this is the provider
edge router.The last IP node a packet traverses
before reaching the receiver's network.A network whose edge nodes
implement ConEx policy functions.Each network can unilaterally choose to use any ConEx information
given by those sources using ConEx, independently of whether other
networks use it.Typically, a network will use ConEx information by deploying a
policy function at the ingress edge of its network to monitor arriving
traffic and to act in some way on the congestion information in those
packets that are ConEx-enabled. Actions might include policing,
altering the class of service, or re-routing. Alternatively, less
direct actions via a management system might include triggering
capacity upgrades, triggering penalty clauses in contracts or levying
charges between networks based on ConEx measurements.Typically, a network using ConEx info will deploy a ConEx policy
function near the ingress edge and a ConEx audit function near the
egress edge. The segment of the path between a ConEx policy function
and a ConEx audit function can be considered to be a ConEx-protected
segment of the path. Assuming a network covers all its ingresses and
egresses with policy functions and audit functions respectively, the
network within this ring will be a ConEx-protected network.Of course, because each edge device usually serves as both an
ingress and an egress, the two functions are both likely to be present
in each edge device.In all the deployment scenarios below, we assume that deployment
starts with some data sources being modified with ConEx code. The
rationale for this is that the developer of a scavenger transport
protocol like LEDBAT has a strong incentive to tell the network how
little congestion it is causing despite sending large volumes of data.
In this case the developer makes the first move expecting it will prompt
at least some networks to move in response—so that they use the
ConEx information to reward users of the scavenger protocol.The name 'Receiving Network' for this scenario merely emphasises
that most data is arriving from connected networks and data centres
and being consumed by residential customers on this access network.
Some data is of course also travelling in the other direction.Figure is an attempt to
show the salient features of a ConEx deployment in a typical broadband
access provider's network (within the constraints of ASCII art).
Broadband remote access servers (BRASs) control access to the core
network from the access network and vice versa. Home networks (and
small businesses) connect to the access network, but only two are
shown.In this diagram, all data is travelling towards the access network
of Home-b, from the Peer network, the Data centre, the CDN and Home-a.
Data actually travels in both directions on all links, but only one
direction is shown.The data centre, core and access network are all run by the same
network operator, but each is the responsibility of a different
department with internal accounting between them. The content
distribution network (CDN) is operated by a third party CDN provider,
and of course the peer network is also operated by a third party.This operator of the data centre, core and access network is the
only one in the diagram to have deployed ConEx monitoring and policy
devices at the edges of its network. However, it has not enabled ECN
on any of its network elements and neither has any other network in
the diagram. The operator has deployed a congestion policing function
(P) on the provider-edge router where the peer attaches to its core,
on the BRAS where the CDN attaches and on the other BRAS where each of
the residential customers like Home-a attach. On the provider-edge
router where the data centre attaches it has deployed a congestion
monitoring function (M). Each of these policing and monitoring
functions handles the aggregate of all traffic traversing it, for all
destinations.The operator has deployed an audit function on each logical output
port of the BRAS for each end-customer site like Home-b. The Audit
function handles the aggregate of all traffic for that end-customer
from all sources. For traffic in the opposite direction (e.g. from
Home-b to Home-a, there would be equivalent policing (P) and audit (A)
functions in the converse locations to those shown.Some content sources in the CDN and in the data centre are using
the ConEx protocol, but others are not. There is a similar situation
for hosts attached to the Peer network and hosts in home networks like
Home-a: some are sending ConEx packets at least for bulk data
transports, while others are not.Within the BRAS there are logical ports that model the rate of
each access line from the DSLAM to each home network , . They are fed
by a shared queue that models the rate of the downstream link from
the BRAS to the DSLAM (sometimes called the backhaul network). If
there is congestion anywhere in the set of networks in Figure it is nearly always:either self-congestion in the queues into the logical ports
representing the access linesor shared congestion in the shared queue on the BRAS that
feeds them.Any ConEx sources sending data through this BRAS will
receive feedback about these losses from the destination and
re-insert it as ConEx markings into the data. shows an example plot of the loss
levels that might be seen at different monitoring points along a
path between the data centre and home-b, for instance. The top half
of the figure shows the loss probability within the BRAS consists of
0.1% at the shared queue and 0.2% self-congestion in the logical
output port that models the access line, making 0.3% in total. This
upper diagram also shows whole path congestion as signalled by the
ConEx sender, which remains unchanged along the whole path at
0.3%.The lower half of the figure shows (downstream congestion) =
(whole path) - (upstream congestion). Upstream congestion can only
be monitored locally where the loss actually happens (within the
BRAS output queues). Nonetheless, given there is rarely loss
anywhere else but within the BRAS, this limitation is not
significant in this scenario. The lower half of the figure also
shows the location of the policing and audit functions. Policing
anywhere within or upstream ofthe BRAS will be based on the
downstream congestion level of 0.3%. While Auditing within the BRAS
but after all the queues can check that the whole path congestion
signalled by ConEx is no less than the loss levels experienced
within the BRAS itself.Even a sending application that is modified to use ConEx can
choose whether to send ConEx or Not-ConEx packets. Nonetheless,
ConEx packets bring information to a policer about congestion
expected on the rest of the path beyond the policer. Not-ConEx
packets bring no such information. Therefore a network that has
deployed ConEx policers will tend to rate-limit not-ConEx packets
conservatively in order to manage the unknown risk of congestion. In
contrast, a network doesn't normally need to rate-limit
ConEx-enabled packets unless they reveal a persistently high
contribution to congestion. This natural tendency for networks to
favour senders that provide ConEx information encourages senders to
choose to use the ConEx protocol whenever they can.In particular, high volume sources have the most incentive to
deploy ConEx. This is because high volume sources (e.g. video
download sites or peer-to-peer file-sharing) can gain by
implementing a low 'weight' end-to-end transport (i.e. a less
aggressive response to congestion than other transports). Then,
although they send a large amount of volume, they need not
contribute significantly to congestion. If the ISP currently limits
data volume, or offers chargeable tiers based on data volume, such
customers stand to gain considerably if they can encourage the ISP
to limit usage based on congestion-volume instead of volume. explains why this is
the case. The plots show bit-rate on the vertical axis and time
horizontally. A file transfer (e.g. the one labelled from customer
'b') is given a simplified representation as a rectangle, implying
it runs at a set rate for a time, then completes. The maximum height
of each plot represents the maximum capacity of the shared link
across the backhaul network, which is typically the bottleneck in a
broadband network. The hatched regions represent unused capacity.
'c' represents the high volume source that we intend to show has an
incentive to deploy ConEx.In the upper half of the figure, customers 'b' & 'c' both use
transports with equal weights, which is why they are shown with
equal rates when they both compete for the capacity of the line. 'c'
sends larger files than 'b', so when 'b' completes each of its file,
'c' can use the full capacity of the line until 'b' starts the next
file. In the lower half of the figure, 'c' uses a less aggressive
(lower weight) transport, so whenever 'b' sends a file, 'c' yields
more of its rate. This allows 'b' to complete its transfer earlier,
so that 'c' can take up the full rate earlier. 'b' sends the same
volume files (same area in the graph), just faster and therefore
they complete sooner (tall & thin instead of shorter and wider).
As a result, 'c' hardly finishes any later than in the upper
diagram. However, 'c' will have contributed much less to congestion,
and 'b' completes the majority of its file transfers much faster.
'b' has also contributed less to congestion.As we have said, customer 'c' in particular stands to gain if the
ISP bases usage-limits (or usage charges) on congestion-volume
rather than volume. The ISP also has a strong incentive to reward
customers like 'c', because they make the network performance appear
far better than before for customer's like 'b' (e.g. short Web
transfers). However, the network cannot make this move until
customers like 'c' expose congestion information (ConEx) that the
ISP can use in its traffic management or contracts.Of course, in reality there would be more than two customers. But
this would mean that short transfers like 'b' stand to gain even
more, as multiple larger files would be yielding at once.We should point out that not all high-volume customers will be
prepared to temporarily shift their usage out of the way as shown
— real-time video for instance would still use a higher weight
(more aggressive) so as to ensure timely delivery. However, high
volume applications with elastic (non-real-time) requirements are
also common (e.g. video streaming, software downloads, etc)We should also point out that a transport that is less agressive
against other customers is similar but not quite the same as LEDBAT
. LEDBAT does indeed yield
more to other flows during congestion, but it is designed to only do
this if the contention for resources is at a slow link, such as the
customer's own home router. If the contention is at a fast link,
such as a BRAS, LEDBAT is designed not to yield. This is because
ISPs currently give no reward to a transport that minimises
congestion to others — because they do not have the congestion
information to be able to.This document does not require actions by IANA.This document has introduced how congestion policing could be
deployed at the broadband remote access servers in a typical broadband
access network. Congestion policing uses ConEx markings introduced by
data sources and packets discarded by the BRAS to determine rest-of-path
congestion, and police traffic accordingly. It has been shown that high-volume elastic data sources have a strong
incentive to deploy ConEx speculatively in the expectation that they
will be able to encourage their ISPs to account for their usage by
congestion-volume, not volume. They can use a less aggressive transport
and prove that they are contributing little to congestion despite
sending a lot of volume. ISPs also have a strong incentive to use this
ConEx information to encourage their elastic high-volume customers to
use less agressive transports, given they improve the performance of all
the other customers. Without ConEx information, ISPs can only use volume as a metric of
usage, which prevents the above virtuous circle from forming, perversely
discouraging high-volume elastic customers from such friendly
behaviour.Comments and questions are encouraged and very welcome. They can be
addressed to the IETF Congestion Exposure (ConEx) working group's
mailing list <conex@ietf.org>, and/or to the authors.Congestion Exposure (ConEx) Concepts and Abstract
MechanismGoogleBTConEx Concepts and Use CasesThis document provides the entry point to the set of
documentation about the Congestion Exposure (ConEx) protocol. It
explains the motivation for including a ConEx marking at the IP
layer: to expose information about congestion to network nodes.
Although such information may have a number of uses, this document
focuses on how the information communicated by the ConEx marking
can serve as the basis for significantly more efficient and
effective traffic management than what exists on the Internet
today.IPv6 Destination Option for ConexConex is a mechanism by which senders inform the network about
the congestion encountered by packets earlier in the same flow.
This document specifies an IPv6 destination option that is capable
of carrying conex markings in IPv6 datagrams.Reusing the IPv4 Identification Field in Atomic
PacketsThis specification takes a new approach to extensibility that
is both principled and a hack. It builds on recent moves to
formalise the increasingly common practice where fragmentation in
IPv4 more closely matches that of IPv6. The large majority of IPv4
packets are now 'atomic', meaning indivisible. In such packets,
the 16 bits of the IPv4 Identification (IPv4 ID) field are
redundant and could be freed up for the Internet community to put
to other uses, at least within the constraints imposed by their
original use for reassembly. This specification defines the
process for redefining the semantics of these bits. It uses the
previously reserved control flag in the IPv4 header to indicate
that these 16 bits have new semantics. Great care is taken
throughout to ease incremental deployment, even in the presence of
middleboxes that incorrectly discard or normalise packets that
have the reserved control flag set.Low Extra Delay Background Transport (LEDBAT)LEDBAT is an experimental delay-based congestion control
algorithm that attempts to utilize the available bandwidth on an
end-to-end path while limiting the consequent increase in queueing
delay on the path. LEDBAT uses changes in one-way delay
measurements to limit congestion that the flow itself induces in
the network. LEDBAT is designed for use by background
bulk-transfer applications; it is designed to be no more
aggressive than TCP congestion control and to yield in the
presence of any competing flows when latency builds, thus limiting
interference with the network performance of the competing
flows.DSL Forum Technical Report TR-059: Requirements for the
Support of QoS-Enabled IP ServicesBellSouth TelecommunicationsMigration to Ethernet-Based DSL AggregationECI TelecomPolicing Freedom to Use the Internet Resource PoolBTBT & UCLBTDetailed changes are available from
http://tools.ietf.org/html/draft-briscoe-conex-initial-deployRemoved Mobile and Data Centre scenarios, making this draft
solely cover the receiving access network scenario. It then
becomes a 'sibling' of the drafts on these two subjects, rather
than a 'parent'Consequently Dirk Kutscher is no longer a co-authorIncluded more comprehensive background information on
ConExCompleted Incentives sectionUpdated refsAdded Mobile Scenario section, and Dirk Kutscher as
co-author;Re-issued
without textual change. Merely re-submitted to correct a processing
error causing the whole text of draft-00 to be duplicated within the
file.