PM9313

PM9311 PMC | Alldatasheet

Document overview

• Manufacturer or author: Provided By ALLDATASHEET.COM(FREE DATASHEET DOWNLOAD SITE)
• PDF pages: 343

Technical content

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. PM9311/2/3/5 Enhanced TT1 Chip Set Enhanced TT1 Switch Fabric Datasheet Issue 3: August 2001

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. This document is the proprietary and confidential information of PMC-Sierra Inc. Access to this information does not transfer or grant any right or license to use this intellectual property. PMC-Sierra will grant such rights only under a separate written license agreement. Any product, process or technology described in this document is subject to intellectual property rights reserved by PMC-Sierra, Inc.and are not licensed hereunder. Nothing contained herein shall be construed as conferring by implication, estoppel or otherwise any license or right under any patent or trademark of PMC-Sierra, Inc.or any third party. Except as expressly provided for herein, nothing contained herein shall be construed as conferring any license or right under any PMC-Sierra, Inc.copyright. Each individual document published by PMC-Sierra, Inc. may contain additional or other proprietary notices and/or copyright information relating to that individual document. THE DOCUMENT MAY CONTAIN TECHNICAL INACCURACIES OR TYPOGRAPHICAL ERRORS. CHANGES ARE REGULARLY MADE TO THE INFORMATION CONTAINED IN THE DOCUMENTS. CHANGES MAY OR MAY NOT BE INCLUDED IN FUTURE EDITIONS OF THE DOCUMENT. PMC-SIERRA, INC.OR ITS SUPPLIERS MAY MAKE IMPROVEMENTS AND/OR CHANGES IN THE PRODUCTS(S), PROCESS(ES), TECHNOLOGY, DESCRIPTION(S), AND/OR PROGRAM(S) DESCRIBED IN THE DOCUMENT AT ANY TIME. THE DOCUMENT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY IMPLIED WARRANTY OR MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. ETT1, Enhanced TT1, TT1, and LCS are trademarks of PMC-Sierra, Inc. © 2000 PMC-Sierra, Inc.

8555 Baxter Place

Phone: (604) 415-6000 FAX: (604) 415-6200

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. Device Part Numbers This document contains information on the ETT1TM Chip Set, available from PMC-Sierra, Inc. The following devices comprise the ETT1 Chip Set: Device Name PMC Part Number Scheduler PM9311-UC Crossbar PM9312-UC Dataslice PM9313-HC Enhanced Port Processor PM9315-HC

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

Revision History

Issue Number Issue Date Details Of Change

1 March 2000 Creation of document

2 July 2000 Added HSTL Power Dissipation values, corrected programming

constraints for EPP, added OOB, JTAG and PLL blocks to device dataflow diagrams, added heat sink information to Characteristics section, corrected EPP register section, corrected Frame Format tables, added correct bit definition for EPP Interrupt register, corrected diagram in Appendix C, added send and receive procedure for control packets, added ESOBLCP register explanation, corrected conditions in Table 57, corrected Scheduler Refresh Procedure, added new drawing for Crossbar data flow, corrected token explanation, corrected Figures 36 and 37, corrected spelling and formatting errors throughout document.

3 August 2001 Removed bit 17 from Scheduler Status Register, updated BP_FIFO

information in Scheduler Control and Reset Register, corrected bottom view drawings for Scheduler and Crossbar, corrected signal description section in Scheduler and Crossbar for power pins, corrected tper3, and tpl3 in AC Electrical, added memory address information in EPP registers, corrected mechanical drawings, updated state diagram in Scheduler, added information about Scheduler PLL timing, updated initialization sequence for all chips, corrected Dataslice Signal Description for ibpen0, added bit 14(13th and 14th Dataslice enable) to EPP Control register, updated TDM constraints, added Output TDM Queue Overflow to the EPP’s Interrupt register, updated EPP Output Backpressure / Unbackpressure Threshold register, updated LCS2 link synchronization in appendix, modified EPP Egress Control Packet Data Format table, Modified ETT1 usage of LCS2 protocol section

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE i PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

ii PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE iii PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

iv PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE v PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

vi PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

x PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE xiii PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

xiv PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 15 PMC-Sierra, Inc.

1 Functional Description

1.1 O VERVIEW

The ETT1TM Chip Set provides a crossbar-based switch core which is capable of switching cells between 32 ports with each port operating at data rates up to 10 Gbit/s. This section describes the main features of the switch core and how cells flow through a complete system that is based on the ETT1 Chip Set. This document often refers to port rates of OC-192c or OC-48c. The ETT1 Chip Set itself operates at a fixed cell rate of 25M cells per second per port and thus is unaware of the actual data rate of the attached link. So a switch might be 32 ports of OC-192c, or it could be 32 ports of 10 Gbit/s Ethernet; it is the internal cell rate that is determined by the ETT1 Chip Set, not the link technology.

1.1.1 ETT1 Switch Core Features

The ETT1 switch core provides the following features:  320 Gbit/s aggregate bandwidth - up to 32 ports of 10 Gbit/s bandwidth each  Each port can be configured as 4 x OC-48c or 1 x OC-192c  Both port configurations support four priorities of best-effort traffic for unicast and multicast data traffic  TDM support for guaranteed bandwidth and zero delay variation with 10 Mbit/s channel resolution L C S TM protocol supports a physical separation of switch core and linecards up to 200 feet (70 m)  Virtual output queues to eliminate head-of-line blocking on unicast cells  Internal speedup to provide near-output-queued performance  Cells are transferred using a credit mechanism to avoid cell losses due to buffer overrun  In-band management and control via Control Packets  Out-of-band management and control via a dedicated CPU interface  Optional redundancy of all shared components for fault tolerance  Efficient support for multicast with cell replication performed within the switch core

1.1.2 The Switch Core Model

allow for transient congestion through the switch. Figure 1. The Basic Components of a Switch Built Around the ETT1 Chip Set. on the same physical board as the ETT1 port devices. described in the LCS Protocol section.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 17 PMC-Sierra, Inc. The second interface is between the ETT1 devices and the local CPU. The ETT1 Chip Set requires a local CPU for configuration, diagnostics and maintenance purposes. A single CPU can control a complete ETT1 core via a common Out-Of-Band (OOB) bus. All of the ETT1 devices have an interface to the OOB bus. The OOB bus is described in Section 1.1.4 “The OOB (Out-Of-Band) Bus” on page 18.

1.1.3 The LCS Protocol

The Linecard-to-Switch (LCSTM ) protocol provides a simple, clearly defined interface between the linecard and the core. In this section we introduce LCS. There are two aspects to LCS:  a per-queue, credit-based flow control protocol  a physical interface The LCS protocol provides per-queue, credit-based flow control from the ETT1 core to the linecard, which ensures that queues are not overrun. The ETT1 core has shallow (64 cells) queues in both the ingress and egress directions. These queues compensate for the latency between the linecard and the core. One way to think of these queues is simply as extensions of the queues within the linecards. The queues themselves are described further in Section 1.3 “Prioritized Best-Effort Queue Model” on page 30. The LCS protocol is asymmetrical; it uses different flow control mechanisms for the ingress and egress flows. For the ingress flow LCS uses credits to manage the flow of cells between the linecards and the ETT1 core. The core provides the linecard with a certain number of credits for each ingress queue in the core. These credits correspond to the number of cell requests that the linecard can send to the core. For each cell request that is forwarded to a given queue in the core the linecard must decrement the number of credits for that queue. The core sends a grant (which is also a new credit) to the linecard whenever the core is ready to accept a cell in response to the cell request. At some later time, which is dependent on the complete traffic load, the cell will be forwarded through the ETT1 core to the egress port. In the egress direction a linecard can send hole requests, requesting that the ETT1 core doesnotforward a cell for one celltime. The linecard can issue a hole request for each of the four best effort unicast or multicast priorities. If the linecard continually issued hole requests at all four priorities then the ETT1 core would not forward any best effort traffic to the linecard. The LCS protocol information is contained within an eight byte header that is added to every cell. The physical interface that has been implemented in the ETT1 Chip Set is based on a faster version of the Gigabit Ethernet Serdes interface, enabling the use of off-the-shelf parts for the physical link. This interface provides a 1.5 Gbit/s serial link that uses 8b/10b encoded data. Twelve of these links are combined to provide a single LCS link operating at 18 Gbaud, providing an effective data bandwidth that is in excess of an OC-192c link. NOTE: The LCS protocol is defined in the “LCS Protocol Specification -- Protocol Version 2”, available from PMC-Sierra, Inc. This version of LCS supersedes LCS Version 1. Version 2 is first supported in the TT1 Chip Set with the Enhanced Port Processor device (also referred to as the ETT1 Chip Set) and will be supported in future PMC-Sierra products. The ETT1 implementation of the LCS protocol is described further in Section 1.6 “ETT1 Usage of the LCS Protocol” on page 64.

1.1.4 The OOB (Out-Of-Band) Bus

OOB bus provides a simple mechanism whereby a local CPU can configure each device.

1.2 ARCHITECTURE AND FEATURES

1.2.1 ETT1 Switch Core

illustrates the logical relationship between these entities. Figure 2. ETT1 Switch Core Logical Interconnects

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 19 PMC-Sierra, Inc. Each ETT1 port is attached to one or more linecards. The port contains the shallow cell queues and implements the LCS protocol. The port and Scheduler exchange information about cells that are waiting to be forwarded through the Crossbar core. The Scheduler maintains local information on the number of cells that are waiting in all of the ingress and egress queues. It arbitrates amongst all cells in the ingress queues, and instructs all of the ports as to which cell they can forward through the Crossbar at each cell time. Two Crossbars are used. The first, referred to as simply ‘the Crossbar’, interconnects all of the ports with all of the other ports, enabling cells to be forwarded from the ingress port queues to the egress port queues (in a different port). The Crossbar is reconfigured at every cell time to provide any non-blocking one-to-one or one-to-many mapping from input ports to output ports. Each Crossbar port receives its configuration information from its attached port; the Crossbars do not communicate directly with the Scheduler. The second Crossbar is the flow-control Crossbar. It passes output queue occupancy information from every egress port to every ingress port. The ingress ports use this information to determine when requests should and should not be made to the Scheduler. The CPU/clock provides clocks and cell boundary information to every ETT1 device. It also has a local CPU which can read and write state information in every ETT1 device via the OOB bus. The CPU/clock entity is a necessary element of the ETT1 switch core, but does not contain any of the ETT1 devices.

1.2.2 Basic Cell Flow

The ETT1 Chip Set consists of four devices. Their names (abbreviations) are: D a t a s l i c e ( D S )  Enhanced Port Processor (EPP)  Scheduler (Sched)  Crossbar (Xbar) This section describes how cells flow through these four devices.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 21 PMC-Sierra, Inc. tells the ports which of the many source ports will be sending it a cell. 5. The source EPP sends a read command to the Dataslices, which then reads the cell from the appropriate queue and sends it to the Crossbar. At the same time, the destination ports send the routing tag information to the Crossbar. This routing tag information is used to configure the internal connections within the Crossbar for the duration of one cell time. The cell then flows through the Crossbar from the source port to the destination port. 6. The cell arrives at the destination port, and the EPP receives the LCS header of the cell. It uses this information to decide in which egress queue the cell should be stored. If this was a multicast cell which caused the egress multicast to reach its occupancy limit, then the EPP would send a congestion notification to the Scheduler. 7. At some later time, the EPP decides to forward the cell to the linecard. The EPP sends a read command to the Dataslices which read the cell from memory and forward the cell out to the linecard. The egress EPP also sends flow control to the ingress EPP, informing it that there now exists free space in one or more of the egress EPP’s output queues. Also, if the transmitted cell was a multicast cell then this may cause the egress queue to go from full to not full, in which case the EPP notifies the Scheduler that it (the EPP) can once again accept multicast cells. The above description does not account for all of the interactions that can take place between ETT1 devices, but it describes the most frequent events. In general, users do not need to be aware of the detailed interactions, however knowledge of the main information flows will assist in gaining an understanding of some of the more complicated sections.

1.2.3 Prioritized Best-effort Service

An ETT1 switch core provides two types of service. The first is a prioritized, best-effort service. The second provides guaranteed bandwidth and is described later. The best-effort service is very simple. Linecards forward best-effort cells to the ETT1 core where they will be queued. The Scheduler arbitrates among the various cells; the arbitration algorithm has the dual goals of maximizing throughput while providing fair access to all ports. If more than one cell is destined for the same egress port then the Scheduler will grant one of the cells and the others will remain in their ingress queues awaiting another round of arbitration. The service is best-effort in that the Scheduler tries its best to satisfy all queued cells, but in the case of contention then some cells will be delayed. The Scheduler supports four levels of strict priority for best effort traffic. Level 0 cells have the highest priority, and level 3 cells have the lowest priority. A level 0 cell destined for a given port will always be granted before a cell of a different priority level, in the same ingress port, that is destined for the same egress port. A ‘flow’ is a sequence of cells from the same ingress port to the same egress port(s) at a given priority. Best-effort flows are either unicast flows (cells in the flow go to only one egress port), or multicast flows (in w h i c hc a s ec e l l sc a ng ot om a n y ,e v e na l l ,o ft h ee g r e s sp o r t s ) .

1.2.4 End-to-End Flow Control

The full queueing and flow control model is shown in Figure 4. due to lack of buffer space. Figure 4. Queueing and Flow Control

1.2.5 TDM Service

separate reservations according to whether it will send and/or receive a cell at each cell time. inherently a multicast service.

24 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. A third difference is that the EPP must maintain separate LCS request counters for each of the subports. It must also maintain separate egress queues. So the number of queues can increase four-fold in order to preserve the independence of each subport. Section 1.3 “Prioritized Best-Effort Queue Model” on page 30 describes the various queuing models in great detail. 1.2.6.1 Identifying the Source Subport The EPP manages a single physical stream of cells at 25M cells/second. In subport mode the EPP must look at each incoming cell and determine which subport has sent the cell. The LCS label field is used to achieve this. Two bits within the label (referred to in the LCS Specification as MUX bits) are used to denote the source subport, numbered 0 through 3. The MUX bits must be inserted in the LCS label before the cell arrives at the EPP. The MUX bits might be inserted at the source linecards themselves. Alternatively, they might be inserted by a four-to-one multiplexer device placed between the subport linecards and the EPP. In this latter case, the multiplexer device might not be able to re-calculate the LCS CRC. For maximum flexibility, the EPP can be configured to calculate the LCS CRC either with or without the MUX bits. 1.2.6.2 Egress Cell Rate Within the EPP is an Output Scheduler process which is different from, and should not be confused with, the ETT1 Scheduler device. At every OC-192 cell time, the Output Scheduler looks at the egress queues and decides which cell should be forwarded to the attached egress linecard(s). In subport mode, the Output Scheduler will constrain the egress cell rate so as not to overflow any of the 2.5 Gbit/s links. It does this by operating in a strict round-robin mode, so that at any OC-192 cell time it will only try to send a cell for one of the subports. The four 2.5 Gbit/s subports are labeled as 0 through 3 ; at some cell time the Output Scheduler will only try to send a cell from the egress queues associated with subport 0. In the next time, it will only consider cells destined for subport 1, etc. If, at any cell time, there are no cells to be sent to the selected subport, then an Idle (empty) cell is sent. For example, if subports 0 and 3 are connected to linecards but subports 2 and 4 are disconnected, then the Output Scheduler will send a cell to 0, then an empty cell, then a cell to 3, then an empty cell, and then repeat the sequence. The effective cell rate transmitted to each subport will not exceed 6.25 M cells per second.

1.2.7 LCS Control Packets

The LCS protocol provides in-band control packets. These packets (cells) are distinct from normal cell traffic in that they do not pass through the fabric to an egress linecard, but are intended to cause some effect within the switch. There are two classes of Control Packets. The first class, referred to as CPU Control Packets, are exchanged between the linecard and the ETT1 CPU (via the EPP and Dataslices). The intention is that CPU Control Packets form the basic mechanism through which the linecard CPU and ETT1 CPU can exchange information. This simple mechanism is subject to cell loss, and so should be supplemented by some form of reliable transport protocol that would operate within the ETT1 CPU and the linecards. The second class, referred to as LCS Control Packets, are used to manage the link between the linecard and the ETT1 port. These LCS Control Packets can be used to start and stop the flow of cells on the link,

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 25 PMC-Sierra, Inc. to provide TDM synchronization event information, and can recover from any grant/credit information that is lost if cells are corrupted in transmission.

1.2.7.1 Sending a Control Packet from the OOB to the Linecard

Before sending a CPU (OOB to linecard) control packet, the OOB must first write the control packet header and payload data into the appropriate locations in the Dataslice. (See Section 3.3 “Output Dataslice Queue Memory Allocation with EPP”, Table 23, on page 164.) The header for customer-specific CPU control packets should be written into the Dataslices as shown, but the payload data is completely up to the customer. To send the CPU control packet which has been written into Dataslice queue memory, the OOB writes the ESOBLCP register with the control packet type select in bits [5:4] (see the bit-breakout), and a linecard fanout in bits [3:0]. If the port is connected to 4 subport linecards, then bits [3:0] are a subport-bitmap. If the port is connected to one OC-192c linecard, then bit 0 must be set when the OOB wishes to send a CP. When the CP has been sent to the linecard(s) indicated in bits [3:0], bits [3:0] will read back as 0. Since control packets have higher priority than any other traffic type, they will be sent immediately, unless the E P Pi sp r o g r a m m e dt os e n do n l yi d l ec e l l s . 1.2.7.2 Sending a Control Packet From the Linecard to the OOB The linecard sends control packets to OOB using the regular LCS request/grant/cell mechanism. A CPU (linecard to OOB) control packet must have the CPU bit set in its request label (See Section 1.1.3 “The LCS Protocol” on page 17.) When the EPP receives the cell payload for a CPU control packet, it stores the cell in the Dataslices’ Input Queue memories and raises the “Received LC2OOB/CPU Control Packet from The input queue for CPU control packets from each linecard is only 8 cells deep, so as soon as the OOB sees a “Received LC2OOB...” interrupt, it should read the appropriate “Subport* to OOB FIFO Status” register (0x80..0x8c). Bit [3:0] of that register will tell how many control packet cells are currently in the input queue; bits 6:4 will tell the offset of the head the 8-cell CPU CP input queue. That queue offset should be used to form addresses for the Dataslices’ Input Queue memories. See Section 3.2 “Input Dataslice Queue Memory Allocation with EPP” on page 162, for Dataslice Input Queue memory addressing. Then the OOB should read those addresses to obtain the CPU CP payload data. When the OOB has read the CPU CP payload data, it should write the appropriate “Linecard * to OOB FIFO Status” register (any value). A write to that register, regardless of the write data, will cause the head of the queue to be dequeued, freeing up that space in the CPU CP input queue. See Section 1.6.3 “Control Packets” on page 69 for more details.

1.2.8 Redundancy

An ETT1 core can be configured with certain redundant (duplicated) elements. A fully redundant core is capable of sustaining single errors within anyshareddevice without losing or re-ordering any cells. This section describes the main aspects of a redundant core. A complete switch may have two ETT1 cores,

the other link. All errors are reported to the local CPU via interrupts. that occur on the links or within the Scheduler or Crossbar will be detectable. passed to the egress Dataslice which will ignore that link and use the information from the other Crossbar. checksum error then it must effectively remove itself from the system. EPPs now only accept information from Scheduler 1. Figure 7. Simple Redundant Scheduler Configuration (much less than 1ms), but does cause best-effort traffic to be suspended for a brief time.

28 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

1.2.9 System Configuration Options

Most of the previous sections have assumed a particular switch configuration consisting of 32 ports of OC-192c, with 64 byte payload cells and full redundancy. The ETT1 Chip Set has four aspects that can be configured according to the user’s requirements. Two of these aspects have been described: quad OC48c versus single OC-192c port, and a redundant system versus a non-redundant system. The other two aspects are described in this section. NOTE: All four aspects are orthogonal and so the choice of any one particular aspect does not limit the choices for the other three aspects. 1.2.9.1 P ayload: 64 bytes or 76 bytes The ETT1 core and LCS protocol are designed to forward fixed length cells. Each LCS cell consists of an LCS header (8 bytes) and a payload. The size of this payload can be either 64 bytes or 76 bytes. The choice of payload size is a function of the linecard traffic: 53-byte ATM cells can probably use a 64 byte payload; fragmented IP packets might obtain greater efficiency with a 76 byte payload. This flexibility in cell size is due to the way the ETT1 Chip Set slices the cells. Figure 8 shows how an LCS cell is sliced across the Dataslices and Crossbars.

Figure 8. LCS Cell Sliced Across Dataslices and Crossbars supporting the seventh Dataslice. to 21 Gbaud. This is done by making the link wider, not faster.

8- 16- or 32-port core, as shown in Table 1.

1.3 PRIORITIZED BEST-EFFORT QUEUE MODEL

perhaps hundreds of cells) there may be temporary unfairness between ports.

1. Since the scheduling pipeline is shorter for lower priorities, a cell of a lower priority may emerge

after the higher-priority cell.

1. Hole requests for a higher priority can allow cells of a lower priority to pass cells of the higher
2. If the linecard responds to LCS grants faster than required to meet the system round trip time

a (programmable) delay in the EPP’s “Internal Delay Matching Adjustments” register.

1. In subport mode only: multicast output queues (VIQs) are shared by subports, so when a subport

blocking of multicast can allow unicast cells of the same priority to pass. model differs for OC-48c ports. Table 1. Crossbar Configurations

1.3.1 Unicast Traffic (OC-192)

ports (including itself), at four priorities, and so will have 128 unicast request counters and ingress queues. Figure 9 illustrates the unicast ingress queueing model for one port. Figure 9. The Unicast Ingress Queueing Model for One Port queue holds cells going to just one output port. way that it can move to the tail of the VIQ in the egress port. queue. The transfer of cells from ingress to egress queues is lossless from a queueing perspective. The best effort unicast ingress and egress queues can store up to 64 cells.

1.3.2 Multicast Traffic (OC-192)

group identifier. The ETT1 port uses this identifier to determine the list of ports to which the cell should go.

1. Head -of-line blocking is a phenomenon encountered by input queued switches in the case where a cell destined for one out-

32 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. The Scheduler will not necessarily schedule all ports in a fanout at the same cell time. In other words, the same multicast cell might pass through the Crossbar on several occasions before all egress ports have received their copy of the cell. Every ETT1 port has a single ingress queue and a single egress queue for multicast traffic at each priority, as shown in Figure 10. (There are also request counters, as for unicast cells, but these are not shown.)

Figure 10. Ingress and Egress Queue for Multicast Traffic ingress queue, then the system will experience head-of-line blocking for multicast cells at that priority. being sent to the Scheduler. NOTE: This selection is at the egress queues,notthe ingress queues. blocking to occur in any ingress multicast queue that will send to it. queues that send cells to this port. then head-of-line blockingmightoccur.

1.3.3 Unicast Traffic (Su bport Mo de)

and corresponds to a single channel of approximately 10 Gbit/s. The second mode is called subport mode. cells per second, the subport mode correspond to four channels each of up to 6.25M cells per second. Figure 11. shows the block diagram of an ETT1 port card. The six (or seven) Dataslices provide a single they appear as a single channel to the EPP/DS.

Figure 11. An ETT1 Port Operating in Subport Mode with Four OC-48c Linecards operate in normal mode (OC-192c). subporting explain these limitations. ETT1 core configured with all 32 ports in subport mode, supporting 128 OC-48c linecards.

Figure 12. The Input Request Counters

Figure 13. Full Set of Counters for a Single Priority

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 37 PMC-Sierra, Inc.

1.3.3.1 Virtual Output Queues

Ideally, the EPP/DS would have a separate VOQ which would correspond to each of the request counters. This would mean that congestion on one VOQ would not impact any other VOQ on the same OC-48c or any VOQs on different OC-48c channels. In practice, the EPP is constrained by the amount of cell buffering available in the ETT1 Dataslice device (which contains the actual cell buffers). Consequently, all four request counters that correspond to a single output channel share a single queue, as shown in Figure 14. The EPP only manages a single queue for each of the output channels (whether the output channel is OC-48c or OC-192c). If the EPP is connected to four OC-48c inputs, then those four OC-48c linecards must share the single queue used for each output. The four dotted arrows in Figure 14 show that the four request counters for output (0,0) all map to the same VOQ for output (0,0).

Figure 14. All Request Counters for a Single Output Share a Single Queue

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 39 PMC-Sierra, Inc. The complete sequence of events at the input side is shown in Figure 15. 1. Linecard a issues a request to the EPP to send a cell to output channel (0,1). The initial request count is zero, so the arrival of the request then increments the request count to one. 2. The LCS Grant Manager sees that there is at least one empty cell buffer in the virtual output queue for output (0,1), and that the request counter for (0,1) is non-zero and so it decrements the request counter (back to zero) and issues a grant to the linecard. 3. The linecard then forwards the actual cell body to the EPP which stores the cell in the virtual output queue. The EPP also issues a request to the Scheduler, indicating that there is a cell w a i t i n gt ob et r a n s f e r r e dt oo u t p u t( 0 ) . 4. Later the Scheduler issues a grant to the EPP which can forward the cell through the Crossbar to the output channel (the EPP at port 0).

Figure 15. Sequence of Events at the Input Side Subport 0 issues a request to send a cell to output (0,1). The request count for output (0,1) is incremented. The LCS Grant Manager issues a grant for queue (0,1) to subport 0.

Figure 15. (Continued) Linecard a receives the grant and forwards cell body to go to output (0,1). A request is made to the Scheduler. The Scheduler issues a grant to queue (0,1).

42 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. In the ideal scenario, each oEPP would have a separate virtual input queue for every (possibly 128) input channel. However, the same restriction applies, limiting the oEPP to just 32 queues per output OC-48c (and per priority). Figure 16 shows the virtual input queues in each oEPP when attached to four OC-48c ports. Each virtual input queue maps back to a single virtual output queue in one of the iEPPs. In abstract terms, the virtual input queue is simply an extension of the appropriate virtual output queue. Each output OC-48c channel within the oEPP has its own set of unicast virtual input queues. If an output queue (virtual input queue) is backpressured, then this will push back to the appropriate virtual output queue. However, each virtual output queue is shared by the four input OC-48c channels within that iEPP, so backpressuring a virtual output queue has the effect of asserting backpressure to all four OC-48c’s on t h es a m ei E P P . The oEPP has an Output Scheduler process which determines which cell should be forwarded to the egress linecards.

Figure 16. Virtual Input Queues

44 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

1.3.3.3 Four Priorities

The description in the preceding sections only considered a single priority. The EPP supports four priorities. Therefore, there are four times as many counters and queues as have been described so far , and this is only for unicast traffic. The four priorities can be modeled as separate planes; the input and output queue figures shown earlier can be considered as 2-D planes. The four priorities are then four copies of these planes, stacked next to each other in a third dimension. Figure 17 illustrates this model for the reference counters and virtual output queues. The virtual input queues at the oEPP are not shown. Consequently, each iEPP has 2048 request counters (4 OC-48c input channels * 128 output channels * 4 priorities) and 512 virtual output queues (128 outputs * 4 priorities). Each oEPP has 512 virtual input queues (4 output channels * 32 ports * 4 priorities). Remember, this calculation is for the configuration of 128 ports of OC-48c. Later on we describe how the number of queues and request counters differs for some other possible configurations.

amalgamated to become requests for port 0. request/grant connection. Figure 18 illustrates this concept. Figure 18. Scheduler Requests/Grants are Multiplexed at a Port Level The EPP has two decisions to make: which request to issue, and which of the four queues to grant. “Scheduler Request Modulator” on page 158 for a discussion of this algorithm. serviced. A simple round-robin algorithm is used (across non-empty queues).

1.3.4 Multicast Traffic (subport mode)

these four request counters map to a single virtual output queue (per priority), as shown in Figure 19.

memory becomes available whenever a cell is dequeued. ports that have their bit set to 1). Figure 21. EITIBM structure

to index into a table of 4-bit entries as shown in Figure 22. Figure 22. OITIBM structure bit 3 corresponds to subport 3.

1.3.5 Combinations of OC-192 c a n dQ u a dO C - 4 8c Linecards

counters for each of the OC-48c linecards. Each port always has four multicast ingress queues. in quad OC-48c mode will have four times as many queues, or 512 queues. Each port always has four multicast egress queues.

1.3.6 Summary

queues and four multicast ingress queues, and the same number of egress queues. having precedence over unicast requests at the same level, as shown in Figure 23. Figure 23. Scheduler Priorities

1.3.7 The LCS Protocol

the queueing model that was described in the previous section. comparison of the two versions of the LCS protocol is in the same document. physical queues present in the ETT1 core. Figure 24. ETT1 Port Ingress Queues mapping function simply has to map each linecard queue to the desired egress port within the ETT1 core. The mapping function must also select the appropriate priority if more than a single priority is required.

52 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. Figure 24 shows ingress cell requests and data cells being forwarded to the ETT1 port. These cells are of a fixed size and format: an LCS cell consists of an eight byte LCS header followed by a fixed length payload. The payload can be 64 or 76 bytes (depending on the number of Dataslices and Crossbars used), but all ports in the switch must use the same size payload. The TT1 Chip Setneverexamines or modifies the cell payload unless the cell is used for internal control purposes. The linecard can send a new cell request whenever a new cell arrives at the linecard and the linecard has at least one request credit remaining for the appropriate queue. So the linecard must keep track of the number of outstanding requests that it can send, and must not issue a request unless it has a corresponding credit. The ETT1 port returns grant/credit information to the linecard. This grant/credit information reflects the occupancy of theingressqueues in the ETT1 port. The linecard can only send a cell to a given ingress queue within the ETT1 port when it receives a grant from the ETT1 port. The linecard increments its corresponding credit counter when it receives a grant/credit. In order to sustain the maximum possible throughput, the linecard must be able to send a new cell within a short time of receiving a grant/credit from the ETT1 port. If the linecard cannot do this then the VOQ at the ingress ETT1 port may become empty which might, in turn, affect the throughput of the switch. The system designer must be aware of the time budget that is really available to the linecard devices. The grant/credit mechanism is not used in the egress direction. Rather, a simpler Xon/Xoff-like mechanism is used. The oEPP will forward cells to the egress linecard whenever it has cells waiting in its output queues. The linecard can request the EPP tonotsend a cell of a given priority, or any combination of priorities, by asserting thehole requestbits in the LCS header. If a hole request bit is asserted at a given priority, then it is a request from the linecard to the EPP that the EPP shouldnotforward a cell of that priority at one cell time in the future. The time between the EPP receiving the hole request and observing it is guaranteed to be no more than 64 cell times. Future LCS compliant products will have different response times. Linecard designs requiring backpressure features should accommodate all LCS products to which they will attach. If a linecard has nearly exhausted its egress buffers for priority 2 cells, then it mightcontinuouslyassert hole request for priority 2 until it can recover some additional buffers. NOTE: The hole request mechanism operates per-priority and not per-source port. Refer to the “LCS Protocol Specification - Protocol Version 2”, available from PMC-Sierra, Inc., for further details on the LCS protocol and the definition of the various fields used within the LCS header in each direction.

1.4 EXPLICIT BACKPRESSURE FROM OEPP TO IEPP

The iEPP does not make a unicast request to the Scheduler unless the destination VIQ has a sufficient number of empty buffers. For each unicast egress VIQ there is one corresponding ingress VOQ. Every iEPP maintains state information on the occupancy of the relevant egress VIQs in all of the EPPs (including itself) in order to ensure that it does not forward a cell to a full VIQ.

with sufficient frequency that the throughput of a single flow is not adversely affected. reverse flow of the update information. Figure 25. A Separate Crossbar is Used to Convey Backpressure Information to the iEPPs forwarded from the VIQ, the iEPP will reduce the counter and resume issuing requests for that flow. Port Processor - Flow Control Crossbar” on page 94.

54 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

1.4.1 Flow Control Crossbar Synchronization

The Flow Control Crossbars interconnect every EPP with every other EPP . During each cell time, the Flow Control Crossbar sets the crosspoints such that every iEPP receives update information from one other oEPP. Further, no two iEPPs receive the same update information from the same oEPP. In the next cell time the crosspoints must be re-organized so that every iEPP receives update information from a different oEPP. The system must ensure that the Flow Control Crossbars and the EPPs are all synchronized, so that every iEPP knows from which oEPP it is receiving information at any given time. This synchronization information is generated by the Scheduler using the FCCSYN register to send a periodic pulse out to all EPPs. Further, if a redundant Scheduler is used then the two Schedulers must be synchronized. This Scheduler-to-Scheduler synchronization occurs after a Scheduler Refresh operation. Refer to Section 1.9.4 “Scheduler Refresh Procedure” on page 98.

1.5 TDM SERVICE

The TDM service enables linecards to periodically reserve bandwidth within the ETT1 core. Cells that are switched via this service will have a fixed latency through the switch and will not be delayed due to best effort contention for an output port. 1-in-N idle cells and egress control packets may cause temporary delays, but these should be absorbed by the guardband and speedup. The TDM service is implemented within the ETT1 core using:  dedicated queues in the ETT1 ports  a table-based reservation mechanism in the ETT1 ports  a flexible synchronization mechanism

1.5.1 TDM Queues

The linecard uses the normal LCS request-grant mechanism for TDM cells. When TDM cells arrive at an ETT1 ingress port, they are queued in a single TDM queue which can buffer up to 96 cells. The cells are stored in the ingress queue until their reservation time occurs, at which time they are forwarded through the crossbar to the appropriate egress TDM queue. Given that there can be no contention for TDM cells then a TDM queue might seem unnecessary. The queue is present so that the linecard does not need to be closely synchronized with the ETT1 fabric. If the port is operating in subport mode then four ingress queues are used, one for each subport. Each ETT1 port has a single egress TDM queue, even when the port is in subport mode. The ETT1 can transmit cells out-of-order between the subports, and so a single shared TDM queue does not cause a problem. Figure 26 illustrates the queueing structure.

Figure 26. The Queueing Structure cells until the appropriate time.  The normal LCS credit mechanism is used for ingress TDM cells.

1.5.2 TDM Reservation Tables

entry is of the form shown in Table 2. Table 2. TDM Reservation Table

Packet is used to specify which of the two tables is being used at any given time. a best-effort cell from that port at the same time. Figure 27 shows the logic at the ingress port. Table 2. TDM Reservation Table (Continued)

Figure 27. Ingress Port Logic prevents the Scheduler from trying to schedule a best-effort cell to that port at the same time. cell will now become available for best-effort cell scheduling. linecard (provided that the Output Scheduler is not frozen).

1.5.3 TDM Frame Timing

cannot be used for TDM traffic. Figure 29. Single TDM Table referred to as the “TDM offset value”.

1.5.4 TDM Synchronization

synchronously with the ETT1 port. of the ETT1 devices operate cell synchronously. TDM table location 0, and normal TDM processing commences. which TDM table to use. This allows all ports and linecards to switch TDM tables synchronously.

As soon as the linecard receives the TDM_Sync signal it starts to forward TDM requests to the ETT1 port. send them), but must not re-order cells relative to the TDM tables. request and send in the first TDM cells. Figure 30 shows the sequence of events in more detail. Figure 30. TDM Synchronization

1. A Suggested_TDM_Sync from a counter in the EPP.
2. A counter in the Scheduler.
3. A Suggested_TDM_Sync from a designated linecard (through the EPP).

There are pros and cons to each of these methods. Each method is described in more detail below.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 61 PMC-Sierra, Inc.

1.5.4.1 TDM Sync Driven by the EPP

The EPP can be programmed to send a Suggested_TDM_Sync to the Scheduler once every N+1 celltimes by writing N to the EPP’s TDM Sync Generation Control register and writing 1 to the Enable TDM Sync Generation register. The TDM table select is also a field of the TDM Sync Generation Control register. The Scheduler is programmed to listen to the Suggested_TDM_Sync from a designated port. The advantage of this method over the third method is that the only exposure to jitter (uncertainties in delay) on the TDM_Sync is after the EPP sends the TDM Control Packet to the linecard, due to crossing asynchronous clock domains (and subport-ordering delay in subport mode). The advantage of this method over the second method is that when using two Schedulers for fault tolerance, the second method requires synchronous OOB writes to program the Schedulers at exactly the same time; this method provides a synchronous Suggested_TDM_Sync to the two Schedulers instead. A possible disadvantage of this method is that TDM Sync is synchronous to the ETT1’s core clock, not to the linecard’s clock or any other external synchronization signal. 1.5.4.2 TDM Sync Driven by the Scheduler The Scheduler can be programmed to generate its own TDM_Sync signals once every N+1 celltimes using the Scheduler’s TDM Control register. The TDM table select is also controlled via the TDM Control register. The Scheduler ignores Suggested_TDM_Sync from all ports. The advantages of this method are simplicity and low jitter, similar to the first method. The disadvantage of this method is that when using two Schedulers for fault tolerance, this method requires synchronous OOB writes to program the Schedulers at exactly the same time; if the Schedulers’ TDM Control registers are enabled to send TDM Sync at different times, then all EPPs will report mismatching frames from the two Schedulers. Scheduler refresh will not synchronize Scheduler-generated TDM Syncs. 1.5.4.3 TDM Sync Driven by a Designated Linecard Any linecard can send a Suggested_TDM_Sync to its ETT1 port, using a LCS TDM Control Packet, every TDM_Period cells. The TDM Control Packet also specifies which of the two TDM tables should be used. When the EPP receives a TDM Control Packet it sends a Suggested_TDM_Sync signal to the Scheduler. The Scheduler is programmed to listen to the Suggested_TDM_Sync from a designated port. The advantage of this method is that an external synchronization source can be used. The disadvantage of this method is that there is much more exposure to jitter in TDM_Sync. Asynchronous clock domain crossings (and subport-ordering delays in subport mode) will cause delay variations at multiple stages: the LCS request for the Suggested_TDM_Sync Control Packet, the LCS grant, the delivery of the Suggested_TDM_Sync Control Packet payload, and finally the delivery of the TDM_Sync Control Packet from the ETT1 to the linecards. Additionally, the Scheduler’s internal digital TDM_Sync PLL

for a more thorough discussion of TDM_Sync jitter and its consequences. Suggested_TDM_Sync signal. Figure 31 shows the passage of signals. Figure 31. Signal Flow to send TDM_Syncs to the ports at the current period.

1.5.5 C onfiguring the TDM Service

1. Configure the TDM tables in all ports that will send or receive TDM cells.
2. Configure the Scheduler in the appropriate mode for TDM synchronization.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 63 PMC-Sierra, Inc. Configuring the tables is straightforward, as described in Section 1.5.2 “TDM Reservation Tables”. The rules for the tables are: (a) No port can source more than one TDM cell at any cell time. (b) No port can receive more than one cell at any cell time. (c) The linecard must send the cells to the EPP: - in order, and - before the current TDM slot reaches the slot reserved for that cell. (d) Subport mode: The TDM Reservation Table can specify the same ingress subport only once every four time slots. (e) Subport mode: The TDM Reservation Table can not specify the same egress subport more than 48 time slots within a moving window of 192 time slots. This constraint is caused by the TDM output queue. Item (c) above is important to understand: the EPP will buffer all ingress TDM cells before they are sent through the ETT1 fabric. The EPP can only buffer 96 TDM cells at the ingress. The linecard must be able to send TDM cells to meet the average and the burst rate that are implied by the TDM table configuration. So, if the linecard is operating at 20M cells/sec, for example, while the ETT1 fabric operates at 25M cells/sec then the linecard must not reserve a contiguous burst of 128 TDM slots because it would be unable to supply the last few cells in time for their reservations. The next step is to configure the Scheduler. The Scheduler TDM service is controlled via its TDM Control register. The first aspect to configure is the source of the Suggested_TDM_sync signal. If a linecard or EPP will supply the Suggested_TDM_sync signal then configure the TDM Control register with the selected port and set the Current Port valid bit. If a linecard or EPP does not supply the Suggested_TDM_ sync signal then the Scheduler must be configured with the desired TDM_Period: write the desired period into the Initial Period field and toggle (0 to 1 to 0) the Load Initial Period control bit. In all cases the Enable bit (bit 31) of the Scheduler’s TDM Control register should be set to 1. The final stage of configuring the Scheduler is to write the desired value to the Enable TDM traffic register. Each of the 32 bits should be set to 1 for those ports that will send or receive TDM traffic.

1.5.6 Changing the Source of the Suggested Sync

If the Scheduler is configured to use an external (port or linecard) source for the Suggested_TDM_sync, then you can change the source port by writing a new value into the port field of the TDM Control register. Once the new port starts to supply the Suggested_TDM_sync signal then the Scheduler will adjust the frequency and phase of the transmitted TDM_Sync signal to match that of the new incoming signal. The Scheduler will change the frequency of the outgoing sync signal once it has seen three Suggested_TDM_ sync signals, but it will only gradually adjust the phase. The phase of the outgoing TDM_Sync will be adjusted by one cell time in every TDM_period. NOTE: The phase reference for the Scheduler is the received Suggested_TDM_sync signal,

64 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. which will be several cell times after the linecard has sent the Suggested_TDM_sync to the ETT1 port. If the Scheduler does not see an incoming Suggested_TDM_sync signal for three TDM_periods then it will issue an interrupt indicating that the sync source has failed. However it will continue to operate with the current values of frequency and phase.

1.6 ETT1 US AGE OF THE LCS P ROTOCOL

This section provides details of how the LCS header is used in an ETT1 system. Further information on the operation of the LCS protocol is in the “LCS Protocol Specification -- Protocol Version 2”, available from PMC-Sierra, Inc.

1.6.1 LCS Protocol Prepend

LCS segments are 72 or 84 bytes in length. The line to switch format is shown in Table 3, while Table 5 shows the field definitions of this format. Likewise, Tables 3 and 4 show the switch to line format and field definitions, respectively.

Table 3. LCS Format from Linecard to Switch Table 4. Definitions of LCS Fields from Linecard to Switch Req Valid 1 Indicates that request field is valid (Request Label_1). Label_1 14 Label that is being requested (e.g. unicast-QID/multicast-label/TDM-label). Type_1 4 Segment type that is being requested (e.g. Control Packet). Payload Valid 1 Indicates that the Payload Data field contains a valid payload. Seq_Number 10 A sequence number for synchronizing grants with payloads. Control” on page 169, bit 4.

Table 5. LCS Format from Switch to Linecard Table 6. Definitions of LCS Fields from Switch to Linecard Grnt Valid 1 Indicates that grant field is valid (Grant Label_1 and Sequence Number). Label_1 14 Label for credit (e.g. QID/mcast-label/TDM-label). Type_1 4 Segment type for credit (e.g. TDM). Seq_Num 10 Sequence Number associated with grant. Payload Valid 1 Indicates whether the egress payload is valid (Payload Label_2 and Payload Data). Label_2 14 Label for egress payload (e.g. ingress port identifier). Type_2 4 Segment type for egress payload (e.g. Control Packet). Control” on page 169, bit 4.

1.6.2 Use of Label and Type Fields

Table 7. Encoding of 4-bit Type Fields

0000 Control Packet Provides for inband control information between the

linecard and the switch core. 0001,...,0110 Reserved for future use Currently not defined for use.

0111 TDM TDM service provides preallocated bandwidth at a

Table 8. Usage of the Ingress Request Label_1 CP-category = 1 for CPU Control Packets. a. Reserved. All bits labeled Rsvd must be set to zero. will increase contiguously. If not used, must be set to zero. field will increase contiguously. If not used, must be set to zero. e. If the port has no subports (i.e. if it is connected to a single linecard without multiplexing), this field must be set to zero.

Table 9. Usage of the Egress Grant Label_1 CP-category = 1 for CPU Control Packets. a. Reserved. All bits labeled Rsvd must be set to zero. will increase contiguously. If not used, must be set to zero. field will increase contiguously. If not used, must be set to zero. e. If the port has no subports (i.e. if it is connected to a single linecard without multiplexing), this field must be set to zero. Table 10. Usage of the Egress Payload Label_2 CP-category = 1 for CPU Control Packets. a. Reserved. All bits labeled Rsvd must be set to zero. increase contiguously. If not used, must be set to zero. c. If the source port has no subports (i.e. if it is connected to a single linecard without multiplexing), this field will be zero. will increase contiguously. If not used, must be set to zero. will increase contiguously. If not used, must be set to zero.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 69 PMC-Sierra, Inc.

1.6.2.1 Subport Mode Label and MUX Fields

Label fields are arranged such that port addresses can be subdivided into four multiplexed, subport addresses. Each segment type with a subport field utilizes bits [1:0] for the subport addresses. When subports are not utilized, these bits will be set to zero. Subport multiplexing shares a common physical connection to carry segments to/from each subported linecard to a switch port. The time multiplexing between subports may be done on a quad linecard, or may be done at an intermediate stage. When time-multiplexed, segment transmissions are sequenced between each of the four subports through the use of a MUX field. When a port of an LCS device uses the time-multiplexed subport mode, two bits in the Request Label_1 field of ingress and Grant Label_1 field of egress segments are used to multiplex/demultiplex each of four subport channels. The 2-bit field is called the MUX[1:0] field. For ingress segments, the MUX[1:0] bits are carried in the Request Label_1 field; for egress segments, the MUX[1:0] bits are carried in the Grant Label_1 (but not the Payload Label_2). Note that for ingress traffic, this field is used to designate the source subport of the ingress segment and is independent of the Request Label_1 subport field. Likewise for egress traffic, the MUX field is used to designate the destination subport of the egress segment and is independent of the Grant Label_1 and Payload Label_2 subport fields. Segments entering the switch core must have the MUX[1:0] field correctly set. The MUX[1:0] field in arriving segments must follow the sequence 00,01,10,11,00,... in consecutive segments. The MUX[1:0] field in departing segments must also follow the sequence 00,01,10,11,00,... in consecutive segments. When the MUX subport sequence must be interrupted by the need to send an Idle frame, the sequence should continue with the next subport in sequence following the previously sent subport. The MUX[1:0] field is placed at the beginning of the Request Label_1 or Grant Label_1 field in all segments (i.e. both data traffic and control packets). Segments departing or arriving at the subported line card may have the MUX[1:0] field set to correspond to the linecard’s designated subport. Optionally, the linecard may set the MUX[1:0] field to zero, relying on an intermediary multiplexer device to correctly set the MUX[1:0] field. In such a configuration, the LCS prepend CRC is generated and checked masking the MUX[1:0] field to zero, enabling the intermediary multiplexer device to independently set the MUX[1:0] field without termination and regeneration of the CRC field. Note that without the requirement of setting the MUX[1:0] field, the linecard’s framing function does not have to be subport aware in a system with an intermediary, time-multiplexed data stream.

1.6.3 Control Packets

The LCS link is controlled by Control Packets (CPs) that are generated by the linecard or the switch core. When the switch core sends a CP to the linecard, it marks the correct Payload Label_2 and Type_2 fields as a Control Packet and sends the control information in the payload data. When the linecard wishes to send a CP to the switch core, it goes through the normal three phase request/grant/transmit process. Control Packets have a higher priority over other traffic; therefore, grants will be returned in response to CP request prior to other outstanding requests. There are three categories of CPs: (1) LCS Control Packets that are used to perform low-level functions such as link-synchronization, (2) CPU Control Packets which are used for communication between the

of LCS Control packets for timing sensitive information.

1.6.3.1 LCS Control Packets

the first 10-bytes of the LCS segment payload for an LCS CP format. most, only one request for each CP_Type should be outstanding. In what follows, the three CP_Types of currently defined LCS CPs are described. NOTE: The actual LCS headers for Switch-to-Linecard Control Packets are not built in to the EPP. Table 11. LCS Control Packet Format

The format of TDM CPs is described in Table 12. have different values (e.g. following a CRC error). Table 12. TDM Control Packet Format TDM Frame Sel 1 Indicates which TDM frame to use. CRC-16 16 16-bit CRC calculated over 10-byte control packet payload. Table 13. Request Count Control Packet Format Label_1 14 Label for request counter. Count 14 Update counter value, always set to zero for multicast. CRC-16 16 16-bit CRC calculated over 10-byte control packet payload.

Control Packet is sent, grants and egress data traffic resume. requests into the switch core until a Start Control Packet is sent. receive all categories of Control Packets. CPU Control Packets) and associated Type Field to 4’h0 (Type for Control Packets). data is passed to the linecard. No flow control mechanism is explicitly implemented for CPU CPs, and will be implementation dependent. Table 14. Start/Stop Control Packet Format Start/Stop 1 1=Start, 0=Stop. CRC-16 16 16-bit CRC calculated over 10-byte control packet payload.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 73 PMC-Sierra, Inc.

1.6.4 Use of CRC Fields

1.6.4.1 Use of CRC-16 Field

Control Packets and LCS protocol segment prepends use the CRC-16 polynomial: . T h i si st h es a m ep o l y n o m i a lu s e di nX . 2 5 . 1. Prior to starting the calculation, the remainder is set to 0xffff. 2. In all cases, the CRC-16 is calculated over all bits (80-bits of the CP or 64-bits of the prepend) with the CRC field set to all zeroes. 3. Transmission of the CRC-16 is done with the most significant bit sent first. Note that the use of the CRC-16 option in the ETT1 Switch to Linecard LCS protocol segment prepend requires that the first byte of the CRC-16 be masked when the CRC-16 is checked (see example). The following are some sample CPs and prepends, showing the CRC-16 generated for each. The CRC-16 field is always the last 16 bits. LCS Start CP: 0x0280000000000000C566 Request Count CP: 0x01037280240000006FDB LCS Prepend from Switch to Linecard: 0x8719362C09DCxxC7 1 (subport = 0) 0xE719362C09DCxxC7 1 (subport = 3, MUX masked to 0 for CRC) 0xE719362C09DCxxDF (subport = 3, MUX included in CRC) The use of the CRC-16 polynomial in an ETT1 system is the same as above, however only the last eight bits of the 16-bit CRC are valid from the Switch to the Linecard. Therefore, the receiving linecard should calculate the CRC for the entire prepend with the CRC field set to all 0’s and compare the last eight bits to those of the received CRC as shown here. “xx” means these eight bits should be masked from the compare. LCS Prepend from Linecard to Switch: 0x81B9409D00106039 1 (subport = 0) 1. Note the MUX[1:0] field is assumed equal to “00” in these CRC-8 calculations. x16 x12 x5 1++ +

74 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. 0xE1B9409D00106039 1 (subport = 3, MUX masked to 0 for CRC) 0xE1B9409D00103F21 (subport = 3, MUX included in CRC) The CRC-16 from the Linecard to the Switch is the same as the standard CRC-16. 1.6.4.2 Use of CRC-8 Option Optionally, an ETT1 based system can use LCS segment prepends with the CRC-8 polynomial: 1. Prior to starting the calculation, the remainder is set to 0xff. 2. In all cases, the CRC-8 is calculated over all bits (64-bits of the prepend) with the CRC field set to all zeroes. 3. Transmission of the CRC-8 is done with the most significant bit sent first. The following are some sample prepends, showing the CRC-8 generated for each. The CRC-8 field is always the last 8 bits. LCS Prepend from Switch to Linecard: 0x8719362C09DCxxFD 1 (subport = 0) 0xE719362C09DCxxFD 1 (subport = 3, MUX masked to 0 for CRC) 0xE719362C09DCxx19 (subport = 3, MUX included in CRC) The CRC-8 polynomial in an ETT1 system uses only the last eight bits of the 16-bit CRC field. Therefore, the receiving linecard should calculate the CRC for the entire prepend with the entire CRC field set to all 0’s and compare the calculated CRC-8 to those of the received CRC-8 as shown here. “xx” means these eight bits should be masked from the compare. LCS Prepend from Linecard to Switch: 0x81B9409D00100096 1 (subport = 0) 0xE1B9409D00100096 1 (subport = 3, MUX masked to 0 for CRC) 0xE1B9409D00100072 (subport = 3, MUX included in CRC) The CRC-8 from the Linecard to the Switch is sent with the leading eight bits of the CRC-16 field set to zero. 1. Note the MUX[1:0] field is assumed equal to “00” in these CRC-8 calculations. x8 x2 x 1++ +

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 75 PMC-Sierra, Inc.

1.6.5 LCS PHY Layer

The LCS PHY provides a high-speed parallel transfer of data in the form of framed, fixed-sized segments that have been channelized across multiple media connections. Higher segment rates are also supported through trunking options, while lower segment rates are supported through multiplexing options. The channelization, framing, and transmission functions of the LCS PHY have been specified to allow use of a variety of available Serdes (serialization/deserialization) and parallel fiber optics components. Portions of this specification will refer to examples of media types. The LCS PHY specification does not preclude other compatible options. Channelization requirements include the mapping of segments to multiple channelized frames, channel skew specifications, and interchannel (segment) alignment operation. Frame alignment, encapsulation, retiming and the required code-points for a parallel, full-duplex interface are included in the framing function. Transmission requirements include the clocking requirements and electrical characteristics of the parallel interface. The parallel interface used by the LCS PHY is based on widely available Serdes implementations that include their own endec (encoder/decoder) functionality. These components also fit well with the available and emerging parallel fiber optics components. Optionally, an LCS device can include the Serdes and 8B/10B Endec functions, but these are not strictly required by the LCS specification. In fact, LCS implementations that forego the use of optical transceivers or Serdes and endecs altogether for single board design are also compliant with the LCS specification. 1.6.5.1 Link Speed The LCS PHY for ETT1 supports one in link speed option, 1.5 Gbaud. This link speed has various requirements that affect channelization and framing functions, discussed here and in the LCS specification. Serdes and optical transceiver components are available to support the 1.5 Gbaud links. 1.6.5.2 Segment Payload Options The LCS Protocol for ETT1 uses an eight byte prepend and supports two segment payload sizes, a 64-byte payload with an eight byte LCS prepend (8+64 byte segment) and a 76-byte payload with an eight byte LCS prepend (8+76 byte segment). Discussion of these options is also covered in the channelization and framing sections here and in the LCS specification. 1.6.5.3 Segment Channelization The LCS Physical Layer includes the external interface for receiving and transmitting LCS segments from and to the linecard. The interface consists of parallel “ganged” interfaces that each carry a serialized

channels in a gang and the bytes per channel utilized for its particular implementation. word is sent through each channel as serialized 8B/10B encoded data. the local LCS device’s parallel bus receiver interface. serial stream. This Serdes will be a Single Data Rate (SDR) interface for the 1.5 Gbit/s interface. mapping of bytes to channels is shown Table 16. Table 15. Fiber 1.5 Gbit/s per Channel Interface Configurations Table 16. Mapping Segments to 1.5 Gbit/s Channels

1.6.5.4 Alignment, Framing, and Transmission Functions

Protocol Specification -- Protocol Version 2”, available from PMC-Sierra, Inc.

1.7 THE OUT-OF-BAND (OOB) BUS INTERFACE

mechanism such as an Ethernet network. This document describes only the “local” OOB bus. a local master device, such as a satellite controller, that initiates bus transactions. Figure 32. One Implementation: The OOB Bus Consists of Both Local and Global Buses

1.7.1 The OOB Bus

1.7.1.1 OOB Bus Signals

NOTE: Active low signals are indicated by “_L”. WAIT_L and the interrupt signals are not open collector/gate drivers.

1.7.1.2 Address Space

Table 17. OOB Control Bus Signals AD[7:0] 8 bit address/data bus. address/data during other cycles. A d d r e s sd r i v e nb ym a s t e r . VALID_L Bus cycle is valid. Active low. Master. Initially asserted to indicate slave will respond. to respond to current transaction.

Bit A[31] is used to indicate the cycle type: Write or Read. The remaining 31 bits are divided into a device select block and a sub-address block. the target device to identify a register or memory location within the device. The current multicast groups are shown in Table 18.

1.7.1.3 Individual Device Selects

indicate a board type: Port board, Crossbar board or Scheduler board. Figure 33 shows this organization. sub-address. All devices assert WAIT_L until their local write operation completes. 0x7F0 to0x7FE Multicast device selects. (15 multicast groups). See Table 18 below for groups. Table 18. Multicast Group Selects

Figure 33. The Global Device Select Defines the Board Number and the Device on Each Board

The combinations of permissible board addresses and device addresses are shown in Table 19. 0xPPe (i.e. local device select = 0x00e).

1.7.1.4 Port Board Assignment

Scheduler. That port board must then have a board address of 6.

1.7.1.5 Bus Cycles

1.7.1.6 A Write Cycle - No Wait

AD[7] is high to indicate a write operation. VALID_L is asserted by the master to indicate the new cycle. de-assert VALID_L for at least one cycle before re-asserting it for the next cycle. Whenever VALID_L is de-asserted the mastermust drive the local board address onto AD[6:0]. Table 19. Unicast Group Selects bars 1 and 7 are on the board with CCCC=0001, etc.

Figure 34. Write Cycle - No Wait recover cycle during which the target performs the write.

1.7.1.7 A Write Cycle - One Wait

consider this to be a bus error). Figure 35. shows a write cycle with the target forcing a one cycle wait. The target slave asserts WAIT_L D3. The master must maintain d7:0 until WAIT_L is de-asserted.

1.7.1.8 A Read Cycle - One Wait

master must drive the AD signals before the next rising edge of the clock after the VALID_L is deasserted. Figure 36. Read Cycle - One Wait

1.7.1.9 A Read Cycle - Two Waits

Figure 37. Read Cycle - Two Waits

1.7.1.10 Interrupts

Two active high interrupt lines are provided. INT_HI is for “emergency” interrupts such as a card failing. INT_LO is for non-emergency interrupts such as a device requesting to be polled for some reason. asserted at any time, even during an access.

Figure 38. Interrupts are Active High and Synchronous

1. The target slavemust assert WAIT_L during the final address cycle. If the master does not see

address and terminates the cycle (takes VALID_L high).

1. VALID_L must be de-asserted (high) for at least one clock cycle between accesses. The master
2. A slavemay insert up to 255 wait states before itmust de-assert WAIT_L. Failure to de-assert

1.8 INITIALIZATION PROCEDURE

but does incur some redundant operations when used at system initialization. industry standard Serdes devices. see Appendix C, Section 2 “The 8b/10b Interface to the Dataslice” on page 328.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 85 PMC-Sierra, Inc.

1.8.1 Initial Power-on:

 Wait for voltage ramp-up  Suggested:  Initialize the OOB satellite FPGA  Reset Enhanced Port Processor  Reset Dataslices  Reset Scheduler  Reset Crossbar  Enable PLLs on all devices (turn off PLL reset)  Program Dataslices' DINBY and DOUTBY to 3  Set interrupt masks (disable all except AIB ready up and external link interrupts) on EPP & DSs  Set primary Crossbar in DSs to one that exists  Program 8b10b tables of DSs  Program pre-defined switch to Linecard Control Packets location in all DSs. ( See section 3.3, table 28)  EPP Waiting Scheduler Request Count Memory EWSRCT address offset 1C000-1DFFCh: After resetting an EPP, write 00000000h to address offsets 1D000h through 1D7FCh in that EPP (a total of 512 OOB writes). Whan all EPPs are resetsimultaneously (using a broadcast OOB write to the EPP Reset and Control register), 512 broadcast OOB writes can be used to reset those locations in all EPPs.  Release VCSEL and PIN diode resets on DSs  Suggested:  enable clock phase shifting on Serdes,  configure Serdes loopback controls for normal operation,  enable Serdes synchronization,  send idle-not-rdy from linecard  Enable DS 8b10b encoder

86 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.  Enable DS 8b10b decoder  Suggested:  disable Serdes clock phase shifting when sync is established  Set Scheduler BP depth field of Control register to 0x14.  Set Scheduler action bits (Configuration dependent. See "Enhanced Port Processor - Scheduler" on page 83)  Enable the Flow Control Crossbar sync from the Scheduler  Reset DS AIB (for all ports present)  Reset Crossbar AIB (for all ports present)(includes Flow Control Crossbars)  Reset EPP AIB (for all ports present)  Reset Scheduler AIB (for all ports present)  Enable DS AIB (for all ports present)  Enable Crossbar AIB (for all ports present)(includes Flow Control Crossbars)  Release reset on DS AIB  Release reset on Crossbar AIB  Release reset on EPP AIB  Release reset on Scheduler AIB These sequences should complete with only the "ready up" interrupts enabled. When these interrupts occur (which may be during later parts of the above sequences) then perform the following: When DS AIB ready up:  Inform local processes that the DS is operational  Enable all interrupts on DS, except for Transceiver Comma Detect and 8b10b Decoder Comma Detect which are set every time an idle is received. When EPP AIB ready up for at least one Scheduler and at least one set of Flow Control Crossbars:  Inform local processes that the EPP is operational  Enable all interrupts on EPP

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 87 PMC-Sierra, Inc.  Set Fault Tolerance Control  Enable Port Processor  Define 1-in-N Idle Count value  Take EPP out of LCS Stop mode When Crossbar AIB ready up:  Inform local processes that the Crossbar is operational  Enable all interrupts on Crossbar When Scheduler AIB ready up:  Inform local processes that the Scheduler is operational  Enable all interrupts on Scheduler  Enable ports that are present and up  Enable non-TDM traffic  If TDM traffic, define TDM control and Enable TDM traffic In a system with redundant Schedulers, it is essential to perform a refresh operation as the final step, once ports are configured and operational. This synchronizes the two Schedulers. Refresh schedulers if necessary (do not refresh if only one in system)

1.8.2 Port Board being added:

 Wait for voltage ramp-up  Suggested:  Initialize the OOB satellite FPGA  If a Scheduler is present and has its Control register's CRC Action bit set, save and clear the CRC Action bit.  Reset Enhanced Port Processor  Reset Dataslices  Enable PLLs on EPP and DSs (turn off PLL reset)

88 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.  Program Dataslices' DINBY and DOUTBY to 3  Set interrupt masks (disable all except AIB ready up and external link interrupts) on EPP & DSs  Set primary Crossbar in DSs to one that exists  Program 8b10b tables of DSs  Program pre-defined switch to Linecard Control Packets location in all DSs. ( See section 3.3, table 28)  EPP Waiting Scheduler Request Count Memory EWSRCT address offset 1C000-1DFFCh: After resetting an EPP, write 00000000h to address offsets 1D000h through 1D7FCh in that EPP (a total of 512 OOB writes). Whan all EPPs are resetsimultaneously (using a broadcast OOB write to the EPP Reset and Control register), 512 broadcast OOB writes can be used to reset those locations in all EPPs.  Release VCSEL and PIN diode resets on DSs  Suggested:  enable clock phase shifting on Serdes,  configure Serdes loopback controls for normal operation,  enable Serdes synchronization,  send idle-not-rdy from linecard  Enable DS 8b10b encoder  Enable DS 8b10b decoder  Suggested:  disable Serdes clock phase shifting when sync is established If neighboring Crossbar cards are present:  Reset DS AIB and Crossbar AIB  Enable DS AIB and Crossbar AIB  Release reset on DS AIB and Crossbar AIB If neighboring Flow Control Crossbar cards are present:  Reset EPP AIB and Flow Control Crossbar AIB

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 89 PMC-Sierra, Inc.  Enable EPP AIB and Flow Control Crossbar AIB  Release reset on EPP AIB and Flow Control Crossbar AIB If neighboring Scheduler cards are present:  Reset EPP AIB and Scheduler AIB  Enable EPP AIB and Scheduler AIB  Release reset on EPP AIB and Scheduler AIB  Restore the Scheduler Control register CRC Action bit if it was set before initializing this Port Board. These sequences should complete with only the "ready up" interrupts enabled. When these interrupts occur (which may be during later parts of the above sequences) then perform the following: When DS AIB ready up:  Inform local processes that the DS is operational  Enable all interrupts on DS, except for Transceiver Comma Detect and 8b10b Decoder Comma Detect which are set every time an idle is received. When EPP AIB ready up for at least one Scheduler and at least one set of Flow Control Crossbars:  Inform local processes that the EPP is operational  Enable all interrupts on EPP  Set Fault Tolerance Control  Enable Port Processor  Define 1-in-N Idle Count value  Take EPP out of LCS Stop mode

1.8.3 Scheduler Board being added:

 Wait for voltage ramp-up  Suggested:  Initialize the OOB satellite FPGA

90 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.  Reset Scheduler  Enable PLLs on SCH (turn off PLL reset)  Set BP depth field of Control register to 0x14.  Set Scheduler action bits (Configuration dependent. See "Enhanced Port Processor - Scheduler" on page 83)  Enable the Flow Control Crossbar sync  Set interrupt masks (disable all expect ready up) If neighboring Port cards are present:  Reset EPP AIB and Scheduler AIB (for all ports present)  Enable EPP AIB and Scheduler AIB (for all ports present)  Release reset on EPP AIB and Scheduler AIB (for all ports present)  Reset ports on Scheduler These sequences should complete with only the "ready up" interrupts enabled. When these interrupts occur (which may be during later parts of the above sequences) then perform the following: When Scheduler AIB ready up:  Inform local processes that the Scheduler is operational  Enable all interrupts on Scheduler  Enable ports that are present and up  Enable non-TDM traffic  If TDM traffic, define TDM control and Enable TDM traffic In a system with redundant Schedulers, it is essential to perform a refresh operation as the final step, once ports are configured and operational. This synchronizes the two Schedulers. Refresh schedulers if necessary (do not refresh if only one in system)

1.8.4 Crossbar Board being added:

 Wait for voltage ramp-up  Suggested:

1.9 FAULT TOLERANCE

any of the cells within the fabric.

1.9.1 Fault Tolerance Model

Figure 39. These AIB links are protected by a CRC such that a failure in a device will likely result in a CRC

received from the two, supposedly identical elements and will act as though an error had occurred. Figure 39. Redundant Connections Section 1.10.3 “AIB Links (A)” on page 103).

redundant and non-redundant configurations.

1.9.2 Soft/Transient Errors

are different for non-redundant and redundant configurations. will indicate to the CPU that a CRC error has occurred. been corrupted. Dataslice 2 will indicate to the CPU that a CRC error has occurred. Dataslice 2 will store the corrupted information in its local memory and forward it to the egress linecard. discard the cell. The linecard would detect such an error with it’s own payload error detection scheme. error is always indicated to the local CPU via a maskable interrupt. Figure 40. Simple Non-Redundant Crossbar Configuration

94 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. Enhanced Port Processor - Flow Control Crossbar If CRC errors occur between the Flow Control Crossbar device and the EPP, the input EPP will not have the correct view of the available space in the egress queues of the output EPP. Thus, a refresh of the output queue credits must be performed. The invalid incremental credit interrupt register is used to indicate which input port to output port flow has lost output queue credits. For example, if port 5’s invalid incremental credit interrupt register has a value of 0x400, then there are lost credits for all Unicast flows from input port 5 to output port 10. See Section 1.9.3 “Flow Control Refresh Procedure” on page 97. Enhanced Port Processor - Scheduler In a non-redundant Scheduler configuration, CRC errors that occur between the Enhanced Port Processor and Scheduler might result in inconsistent state information. The Scheduler must have accurate information on the number of outstanding requests as well as the backpressure state of all VIQs. A lost request or grant can cause a discrepancy between the Enhanced Port Processor and Scheduler. Figure 41 describes a simple non-redundant Scheduler configuration where Enhanced Port Processor 1 and Enhanced Port Processor 2 are connected to Scheduler 0. If Enhanced Port Processor 1 receives a CRC error on information from Scheduler 0, then Enhanced Port Processor 1 does not know if that information contained a valid grant. If the corrupted grant was for unicast traffic, then the Enhanced Port Processor would have one more request than the Scheduler; this might delay the forwarding of one cell. However, if the corrupted grant was for multicast traffic, then the next multicast grant might cause a cell to be sent to the wrong destination port. This is considered unacceptable and so a CRC error from Scheduler-0 makes Enhanced Port Processor 1 ignore all subsequent Scheduler grants until the state information can be restored. As always, the CRC error is indicated to the local CPU which must refresh Scheduler 0 before traffic can continue to flow to/from this port. The Enhanced Port Processor continues to receive routing tags from the Scheduler, and will forward cells received from other ports. If Scheduler 0 receives a CRC error on information from Enhanced Port Processor- , then Scheduler 0 may have lost a new request or backpressure information. If the lost information was a unicast request, then the Scheduler would have one less request than the Enhanced Port Processor. This might delay the forwarding of one cell. If the lost information was a multicast request, the Scheduler might send a cell to the wrong destination port. If back pressure assertion was lost, the Scheduler could possibly overflow the output queues. These behaviors are considered unacceptable. Thus, when Scheduler 0 detects a CRC error it disables traffic to/from Enhanced Port Processor 1 until the CPU refreshes Scheduler 0’s state information. Traffic to/from other ports continues to flow. The CRC error is indicated to the CPU which must then refresh the state information in Scheduler 0. See Section 1.9.4 “Scheduler Refresh Procedure” on page 98.

Figure 41. Simple Non-Redundant Scheduler Configuration

1.9.2.2 Redundant Configurations

and therefore, the same information is received by the Dataslices from each of the Crossbars. corrupted. Crossbar 0 will indicate to the CPU that a CRC error has occurred. will indicate to the CPU that a CRC error has occurred. no corrective action is required. Figure 42. Simple Redundant Crossbar Configuration

EFTCTRL register in the EPP for more details. Figure 43. Simple Redundant Scheduler Configuration Scheduler and CRC errors to/from the secondary Scheduler. These are described below.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 97 PMC-Sierra, Inc. current primary Scheduler then it will automatically consider the other Scheduler as the new primary Scheduler. If Scheduler 0 (primary Scheduler) receives a CRC error on information from Enhanced Port Processor 1, then Scheduler 0 and Scheduler 1 might differ in their state information. In this situation Scheduler 0 immediately disables its AIB links to Enhanced Port Processor 1 and Enhanced Port Processor 2 (and all Enhanced Port Processors). The Enhanced Port Processors immediately detect the loss of connection to Scheduler 0 and automatically select Scheduler 1 (secondary Scheduler) to be their new primary Scheduler. Without CPU intervention, the system has disabled the primary Scheduler, and switched the Enhanced Port Processors to use the secondary Scheduler. The CPU is informed of this error, and must undertake the process to refresh the Schedulers.The EPP’s may register Scheduler mismatch interrupts until the Schedulers are refreshed. The AIB links to other EPPs are not affected. If Scheduler 1 (secondary Scheduler) receives a CRC error on information from Enhanced Port Processor 1, then Scheduler 1 immediately disables its AIB links to Enhanced Port Processor 1. The Enhanced Port Processors do not alter their primary/secondary setting, since Scheduler 0 is already selected as the primary Scheduler. The CPU is notified of this error, and must go through the steps to refresh the Schedulers. The actions the Scheduler performs upon detecting a CRC error are different in the redundant and non-redundant configurations. Consequently the Scheduler must be configured to be in redundant or non-redundant mode. This configuration is accomplished via the CRC Action bit in the Scheduler’s control register (SCTRLRS). The CRC Action bit shouldonlybe set to 1 for the primary Scheduler in a redundant configuration.

1.9.3 Flow Control Refresh Procedure

The previous section refers to the process of refreshing flow control state. The purpose of this procedure is to insure consistency of the state information between the iEPPs and the oEPPs. In the non-redundant Flow Control Crossbar configuration, the refresh procedure ensures that iEPPs will have correct queue depth information about oEPPs The refresh procedure is as follows: 1. Detect invalid incremental credit interrupt on porti.Read invalid incremental credit interrupt register on porti. The value specifies to which output portjflow could have missing output queue credits. 2. Write freeze unicast Output Scheduler flows register on portjwith a value of 1 << i. This will cause the Output Scheduler on portjto stop sending cells from porti. 3. Disable the non TDM enable for portiin the Scheduler. 4. Freeze the Scheduler Request Modulator on porti. 5. Read the output unicast queue information memory (queue length) on portjfor queues corresponding to input porti. If portjis in OC-192c mode, then 4 queue lengths are stored. If portj is in OC-48c mode, then 16 queue lengths are stored. 6. Read the waiting scheduler request count memory on portifor queues corresponding to output

98 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. portj. The number of request counts stored should be the same as above. 7. Add the stored queue length with the waiting scheduler request count and write that value into the output unicast queue debit count memory for each correspondingi,joutput queue. The number of write accesses should be 4 in OC-192c mode and 16 in OC-48c mode. 8. At this point in time, the output unicast queue debit count memory is up to date, and the unicast Output Scheduler can be unfrozen, the non TDM traffic enabled, and the scheduler request modulator unfrozen.

1.9.4 Scheduler Refresh Procedure

A previous section refers to the process of refreshing a Scheduler. The purpose of this procedure is to ensure consistency of the state information between the Enhanced Port Processors and the Scheduler(s). In the redundant scheduler configuration, the refresh procedure ensures that the two Schedulers will contain identical state information and make the same scheduling decisions. The refresh procedure is as follows: 1. Disable non-TDM traffic in the Scheduler(s): set the enable non-TDM traffic register (SNTDMEN, 0x80) to 0x0. Do not modify the Enable Port register (SENBPRT,0x38). Do not modify the TDM traffic register. 2. Issue the “Go Refresh” command to the Enhanced Port Processors. The EPPs then send their state information to the Scheduler(s). 3. Wait until the refresh process has completed: for each port that issued a “Go Refresh” command, wait until the appropriate bit in the Scheduler’s ‘Port is Refreshed’ register (SRFRS, 0x94) is set to 1. 4. Enable non-TDM traffic in the Scheduler(s): set the enable non-TDM traffic register (SNTDMEN, 0x80) to 1 for each port that is enabled. NOTE: An ETT1 Scheduler device will not schedule best-effort traffic until a “Go Scheduler” command is sent from all of the Enhanced Port Processors. NOTE: Extra writes are required at this point. Refer to the “Scheduler Device Errata, issue 3” document, available from PMC-Sierra, Inc. 5. Issue the “Go Scheduler” command to all of the Enhanced Port Processors that issued “Go Refresh”. This command synchronizes the two Schedulers to start scheduling best-effort traffic at t h es a m et i m e . 6. If the EPP’s EFTCTRL register’s “Ignore Central Scheduler Grants” or “Freeze SRM” bits are set, clear these bits. 7. The Scheduler(s) now contain the same state information as the EPPs and will start scheduling cells. In the redundant Scheduler configuration, the Schedulers will also make identical scheduling decisions.

1.9.5 Refresh Schedulers After Modifying Registers

1.9.6 Hard Errors

This section describes the sequence of events that need to take place to correctly replace a failed board. address each type of board replacement and describe how to resynchronize the board.

1.9.6.1 Port Board

required to replace the failed port board. NOTE: The failure of the port board may also cause the primary Scheduler to shut down.

1. Detect failed port board.
2. Disable TDM and best-effort traffic in the Scheduler(s) to/from this port.
3. Disable this port in the Scheduler’s SENBPRT register.
4. Disable the AIB links to/from the Crossbars and Schedulers.
5. Replace failed port board.

Table 20. Refresh-sensitive Registers in the Scheduler

100 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. 6. Bring up new board. See “Port Board being added:” on page 87. 7. Enable TDM traffic in Scheduler for this port. TDM Traffic can now flow to/from this new port board. 8. Refresh state information from this new port to Schedulers. (See Section 1.9.4 “Scheduler Refresh Procedure”) This is only necessary if the linecard sends traffic before the link to the Scheduler board comes up. Re-establish primary and secondary Schedulers in every port. 9. Best-effort traffic can now flow to/from this new port board. 1.9.6.2 Crossbar B oard The steps needed to replace a failed Crossbar board on a non-redundant Crossbar configuration and a redundant Crossbar configuration are different. For a non-redundant Crossbar configuration, a failed Crossbar board causes all traffic in the entire system to be disabled. These are the following steps required after detection of a failed Crossbar board. 1. Detect failed Crossbar board. For example, CPU is indicated of loss of connectivity from this board to any one of the existing port boards. 2. Immediately disable TDM and non-TDM traffic for all ports in the Schedulers. 3. Disable the AIB links to/from the Port board. 4. Replace failed Crossbar board. 5. Bring up new board (see “Crossbar Board being added:” on page 90) 6. Enable TDM and best-effort traffic for all ports in the Scheduler. 7. Traffic can now flow. The steps needed to cope with a failed Crossbar board on a redundant Crossbar configuration are described here. A failed Crossbar board does not require traffic to be disabled. The redundant Crossbar board carries traffic while the failed Crossbar board is replaced. These are the necessary steps to follow. 1. Detect failed Crossbar board. For example, CPU is indicated of loss of connectivity from this board to any one of the existing port boards. 2. Disable the AIB links to/from the Port board. 3. Replace failed Crossbar board. 4. Bring up new board (see “Crossbar Board being added:” on page 90) 5. Crossbars are now in fault tolerant mode.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 101 PMC-Sierra, Inc.

1.9.6.3 Scheduler Board

In the non-redundant Scheduler configuration, a failed Scheduler board will prevent all traffic from flowing in the entire system. The following steps are required to cope with this failure. 1. Detect failed Scheduler board. For example, CPU is indicated of loss of connectivity from this board to any one of the existing port boards. 2. Disable TDM and non-TDM traffic to/from ports that still have connectivity. 3. Disable the AIB links to/from the Port board. 4. Replace failed Scheduler board. 5. Bring up new board (see “Scheduler Board being added:” on page 89) 6. Configure this Scheduler to operate in non-redundant Scheduler mode. 7. Enable TDM traffic in Scheduler for connected ports. TDM traffic can now flow. 8. Refresh state information from all ports to this Scheduler. (See Section 1.9.4 “Scheduler Refresh Procedure”) 9. Best-effort traffic can now flow. The steps required for a redundant Scheduler configuration are more complicated. The redundant Scheduler can still schedule traffic while the failed Scheduler board is replaced. When the new Scheduler is inserted into the system, it must be resynchronized with the other Scheduler and the Enhanced Port Processors. This will be accomplished by the following steps. 1. Detect failed Scheduler board. For example, CPU is indicated of loss of connectivity from this board to any one of the existing port boards. This failure will cause this Scheduler to be disabled. 2. Disable the AIB links to/from the Port board. 3. Replace failed Scheduler board. 4. Bring up new board (see “Scheduler Board being added:” on page 89) 5. Configure this Scheduler to operate in redundant Scheduler mode. 6. If the Scheduler is generating the TDM sync, then TDM traffic must be stopped by writing a value of 0 to the TDM Sync Period and then writing the actual value to both Schedulers. 7. Enable TDM traffic in new Scheduler for appropriate ports. NOTE: The redundant Scheduler is still scheduling TDM and non-TDM traffic. 8. Refresh state information from all ports to this Scheduler. (See Section 1.9.4 “Scheduler Refresh Procedure”) TDM traffic still flows. 9. Best-effort traffic can now flow. Schedulers are now in fault tolerant mode and synchronized.

1.10 ETT1 SIGNALS AND INTERCONNECTIONS

shows the four types of devices required and lists the types of signals present. Figure 44. ETT1 Signals and Interconnects

1.10.1 LVCMOS (C and O)

on page 77 for more information.

1.10.2 HSTL (H)

are point-to-point signals. All signals have only one driver. signals are HSTL Class 1 (50 Ohm load).

1.10.3 AIB Links (A)

of the clock to provide 800Mb/s on each of the two data pairs as shown in Figure 45. Figure 45. AIB Links of the link is always visible to the local CPU via registers that can be accessed by the OOB bus. control/status signals that are visible to the CPU.

1.6 Gb/s

Figure 46. AIB Control/Status Signals. the bidirectional link is operational, so both receivers have trained correctly. sequence will be restarted and AIB_Ready signal will be deasserted.

1.10.4 Live Insertion

link that is not being removed and deassert AIB_Tx_Enable for the AIB drivers at both ends of the link.

1.11 SYSTEM LATENCIES

linecard interface (Serdes interface).

1.11.1 LCS Request to Grant Minimum Latency

when the EPP sends an LCS grant (2). These are relative to the Dataslice interface as shown, not the EPP. This minimum latency is 16 cell times. Figure 47. Request to Grant latency

1.11.2 LCS Grant to Linecard Cell Latency

“Synchronization,” on page 333.

Figure 48. ETT1 Event Latencies

1.11.3 Cell latency Through an Empty System

Figure 49.) to the output Dataslice (2). The minimum cell latency is 29 cell times.

Figure 49. Cell Latency

1.11.4 LCS Hole Request to Hole Grant Latency

the hole request is granted (2). The egress cell doesnotcontain a valid cell at the requested priority. This maximum latency for ETT1 is 64 cell times.

108 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

1.12 ETT1 DIAGNOSTICS

An ETT1-based switch is a highly sophisticated system which is typically required to demonstrate very high levels of uptime. In order to achieve this uptime it is important to be able to:  Ensure that a subsystem is operating correctly before it is brought online.  Identify subsystems that have failed as quickly as possible.  Diagnose failures to determine the specific entity that should be replaced. This document describes a number of diagnostics that customers might find useful in addressing these needs. The following terminology is used to describe a subsystem within a switch: offline: the subsystem is not interacting with the rest of the system except via the OOB bus. AIB links may be taken up and down but will not cause changes to other subsystems. online, inactive:the subsystem may be receiving signals from the system, and may be sending cells to itself, thus contending for fabric bandwidth. It only sends/receives diagnostic cells, not customer traffic. online, active:the subsystem is participating in the normal operation of the switch, sending and receiving customer traffic and maintenance/diagnostic cells.

1.12.1 Device Tests

All of the ETT1 devices have a number of registers. The local ETT1 CPU can read from and write to many of these registers. Some care must be taken when modifying the contents of registers as they may have side effects that will affect the operation or cell flow of system components that are already online and active. In general, configuration registers should not be modified once a subsystem is online and active. The CPU can read most registers at any time without effect, however some registers, particularly interrupt registers and statistics counters, are cleared when they are read. Some of the ETT1 devices contain memory structures (RAMs). Many of these RAMs can be written to and read from by the local CPU. Since the internal BIST mechanism is not available to the CPU, it is recommended that a customer verify the correct operation of such memory structures by a soak test in which the CPU writes and reads to all locations. Such a test should perform the widely known test patterns such as walking 1’s, 0’s, checkerboard, etc. Of course these tests are destructive to the current contents and can only be performed on RAMs that are not being used. In practice very few of the RAMs can be modified once the devices are active. The RAMs can be read non-destructively, and a great deal of internal system information is available this way; however unless the exact state of the system is known at the instance the RAM is read, then it is difficult to know if the returned value is correct. Obviously, RAMs that implement TDM and multicast tables are designed to be modified with due care during normal operation.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 109 PMC-Sierra, Inc.

1.12.2 AIB Link Tests

All of the ETT1 devices use the high-speed AIB links for inter-device communication. The AIBs are used exclusively for inter-device links between PCBs. The AIB links associated with a subsystem can be tested while the subsystem is offline. The following actions describe the procedure that should be used to test an AIB link attached to an offline subsystem (one end of the link may be attached to an online subsystem). 1. If the other end of the AIB is attached to an online (active) subsystem then ensure that the AIB is ignored by that subsystem, as follows:  Dataslice: this (test) AIB should be the secondary link.  EPP: this (test) AIB should be the secondary link.  Scheduler: disable the port associated with this (test) AIB. (Use the Enable Port register).  Xbar: no action needed. 2. Assert the reset control to both ends of the test AIB. 3. Enable the transmitters at both ends of the AIB. 4. De-assert the reset control to both ends of the test AIB. 5. The link should now come up -- the Ready bits associated with the link should be ‘1’. Interrupts will also be generated if the mask bits are set appropriately. 6. Disable the transmitter at one end of the AIB. 7. The far end of the AIB should indicate CRC errors and that the link is not ready. (A few CRC errors are generated before the link is considered down.) 8. Repeat steps 2 - 6 as needed to ensure that the link is OK. Also, leave the link up for a soak test and ensure that no CRC errors are received.

1.12.3 Linecard to ETT1 Link Diagnostics

When a new portcard in installed in the ETT1 Fabric it will be necessary to ensure that the portcard is operating correctly before bring it online. This section describes a number of tests that can be performed to verify the linecard-to-ETT1 side of the card. Figure 50 shows an example architecture of the portcard and its connection to the linecard on the left and the ETT1 CPU on the right. The Dataslice is shown with two sets of queues.

Figure 50. Illustrating the Flow of Cells from the Linecard CPU to the ETT1 CPU

1.12.3.1 Verifying the Fiber Link

Figure 51. Loopback Path out-of-band communication in order to set either the Serdes or Dataslice into loopback mode. without needing a linecard to be attached. 1.12.3.2 Establishing Linecard to ETT1 Communication. CPU and the ETT1 CPU. Figure 52 shows the flow of cells between CPUs.

Figure 52. Control Packet Exchange Between Linecard CPU and ETT1 CPU CPU can then read the Control packet directly from the Dataslice memory. flow control should be handled between the two CPUs. created the CPU issues a write command to the EPP and the cell will be transmitted to the linecard. link, although at a limited cell rate.

1.12.3.3 Testing Error Detection Logic

c a nb eu s e dt ot e s tt h eC R Cd e t e c t i o nl o g i ca tt h el i n e c a r d . used to check that the 8b/10b decode is operating correctly.

1.12.4 ETT1 Internal Datapath Tests

datapaths that will be tested. Figure 53. Testing the ETT1 Internal Datapaths of Scheduler-request, Scheduler-grant, forward-cell.

1. The CPU writes to the Dataslices, creating a cell in one of the virtual output queues corresponding

to itself -- i.e. it is going to send a cell to itself.

114 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. 2. Turn off the EPP Output Scheduler for cells comingfromitself. This will ensure that the cell will stay in the virtual input queue in the egress buffers of the Dataslice. 3. Bring up all AIB links and enable non-tdm traffic in the Scheduler for this port. Enable the EPP and take it out of LCS-2 Stop mode. 4. Write to the Outstanding Scheduler Request Counter in the EPP for this VOQ. This will cause a request to be made to the Scheduler which will immediately issue a grant to the EPP. The cell will be forwarded through the crossbar and back to the test port. The cell will then be stored in the egress queue but will not be forwarded to the linecard because the Output Scheduler was disabled in step 2. 5. The CPU can read the cell that is waiting in the egress queue and check that it passed through OK. 6. Repeat as needed. A number of variations can be done on this basic theme:  Because the Output Scheduler in each EPP can be stopped on a persourceport basis, then in step 1 the CPU can create a cell going toany egress port, not just to itself. This is possible even if the egress port is currently online with real traffic. Thus it is possible to verify the datapath from the new port to all other ports without interfering with existing flows.  Do not turn off the Output Scheduler in step 2 (or unfreeze it after Step 6). The linecard can then verify that the cell is OK, thus testing more of the system.  Send enough cells so that the egress queue fills up and backpressures the input. So after sending 64 cells (OC-192 configuration) the egress queue should be full and thus prevent any more cells being sent to this queue.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 115 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

2 Dataslice

This chapter contains information on the Dataslice device, part number PM9313-HC, available from PMC-Sierra, Inc. The Dataslice contains queue storage for all cells that have arrived at the input port from the linecard and all cells that have passed through the Crossbar to the output port before departing to the linecard. The Dataslice interfaces to the linecards with 8B/10B encoded physical links, operating at 1.5 Gbit/s. Each Dataslice device packages two independent logical Dataslice units. The ingress and egress datapath is striped between Dataslices such that the first six bytes of the LCS cell are sent through logical Dataslice 0; the next six bytes through logical Dataslice 1, etc. Six Dataslice devices are required on each port card in order to process 72-byte LCS cells. An alternative configuration uses seven Dataslice devices to process 84-byte LCS cells, carrying a 76-byte payload.

2.1 D ATASLICE BLOCKS

The following discussion assumes 12 logical Dataslices per port. (6 devices). There are 12 logical Dataslice units, packaged in six ICs, on a 72-byte LCS cell port card. Each logical Dataslice unit has a full-duplex, 10-bit wide, parallel bus connection to a Serdes device on the linecard interface, two bidirectional AIB connections to two Crossbars, two 8-bit busses for data from the ingress and egress paths to the Enhanced Port Processor, one bus for processed ingress data from the EPP, and two unidirectional 8-bit busses for ingress and egress control information from the EPP. Figure 54 shows how cells flow through the Dataslice.

Figure 54. Dataslice Data Flow queue. The same general flow occurs for the oBypass, except that all 12 Dataslices use the oBypass. initialization through the Out-Of-Band (OOB) bus by the CPU. queues, respectively. The iQM and oQM queue memories are each 8,704 cells deep and 48 bits wide.

2.1.1 OOB Interface and Control/Status Registers

CPU’s address space. These registers are described in more detail in Section 2.3 “Dataslice Registers”.

2.1.2 Dataslice Cell Flow

2.1.2.1 Input Side Data Flow

valid) frames to the EPP and iBypass. Figure 55. Slicing of a Data Cell into Logical Dataslices iBypass is written to memory. the iBypass registers is dropped.

118 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. Regardless of the Control to iDS information (on p2d_ic), the Credit Information (byte 0 through bit 5 of byte 4) of the cell header being written on Grant/data to iDS, is saved in a CR (credit register); it is used to indicate backpressure information back to the linecard. The CRC contained in the LCS header of the frame coming from the Grant/Data to iDS is computed only over the Tag portion of the LCS header. Another CRC in the frame that is computed only over the Credit portion of the LCS header is sent to DS0. This credit CRC is stored in the CR with the credits. The Control to iDS frame contains a read request and read address. If the read request is asserted, the iQM location specified by the read address is read and the cell is sent through the Crossbar AIB Interface. The routing tag carried in the Control to iDS link is placed at the head of the cell as it is sent to the Crossbar. If the “SendZero” bit is set, the DS sends an all-zero frame into the Crossbar instead of reading the cell from the queue memory.

2.1.2.2 Output Side Data Flow

Data cells entering the oDS devices from the Crossbar AIB Interface are placed into the oBypass. For DS0, DS1, and DS2, the cells are sent to the EPPs via the Data from oDS busses, d2p_d[1:0]_id. The VLD bit sent to the EPPs is the same as the VLD bit received from the Crossbars. The EPP, having decided to store this new cell, sends a control frame over the Control to oDS bus, p2d_ic. This frame contains a write address and write request. If the write request is asserted, the cell at the head of the oBypass is placed into the oQM location designated by the write address. If the write request is not asserted, the cell at the head of the oBypass is removed from the FIFO and discarded. The Control to oDS frame also contains a read request and read address. If the read request is asserted, then the cell in the oQM location specified by the read address is removed from the memory and placed in the oFIFO. If the “SendZero” bit is set, the DS inserts an all-zero frame into the oFIFO instead of reading a cell from queue memory. As the cell is inserted into the oFIFO, bytes 0 through the first part of byte 4 (the Credit Information fields of the LCS header) are overwritten by the contents of the CR (Credit Register) by DS0, and the LCS CRC is computed as the XOR of the tag CRC contained in the cell and the Credit CRC contained in the CR. If the Serdes clock rate is slightly less than the core clock rate, this oFIFO occasionally fills up. The EPP can be programmed to insert one “idle” cell periodically into the stream entering the oFIFO to avoid filling the oFIFO. The EPP does not assert either a read request or a “send zero” command to the oDS for these “idle” cells, nor does the oDS insert any cell into the oFIFO. The frequency of these idle cells is programmable and determined by the relative clock frequencies of the switch core and the linecard. 2.2 8B/10B INTERFACE The Dataslice connects to the outside world via a 10-bit wide transmit and receive interface. In the ingress direction, an 8B/10B decoder lookup table maps received 10-bit characters into LCS data bytes. In the egress direction, an 8B/10B encoder lookup table maps LCS data bytes into 10-bit characters to be transmitted. The 8B/10B encoder and decoder lookup tables can be programmed to use any arbitrary 8B/10B code.

2.3 DATASLICE REGISTERS

used to select which of the two logical Dataslices is addressed.

2.3.1 Dataslice Summary

Descriptions section for more information on individual registers. Table 21. Dataslice Register Summary

Table 21. Dataslice Register Summary (Continued)

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 121 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

2.3.2 Dataslice Register Descriptions

2.3.2.1 Status

Symbol: DSTS Address Offset: 00000h Default Value: 10000000h Access: Read Only Status register.

2.3.2.2 Reset

Symbol: DRS Address Offset: 00004h Default Value: 00000000h Access: Read/Write Reset register. Bits Description 31:28 Device ID Number.Identifies the specific device. 27:24 Device Revision Number. 23:2 Reserved. 1 High Priority Interrupt.1 = an outstanding high priority interrupt. One of the bits in the Interrupt Register is set, and is enabled via its corresponding high priority mask. 0 Low Priority Interrupt.1 = an outstanding low priority interrupt. One of the bits in the Interrupt Register is set, and is enabled via its corresponding low priority mask. Bits Description 31-1 Reserved. Reset. Writing a 1 to this location will reset the entire device. It is equivalent to a hardware reset. This register is cleared automatically when the device is reset. Writing a 0 is not necessary. Soft reset takes 1mS to complete.

122 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

2.3.2.3 Low Priority Mask

Symbol: DIRLMSK Address Offset: 00008h Default Value: 00000000h Access: Read/Write Interrupt Mask for interrupts.

2.3.2.4 High Priority Mask

Symbol: DIRHMSK Address Offset: 0000Ch Default Value: 00000000h Access: Read/Write Interrupt Mask for interrupts. Bits Description 31:24 Reserved. 23:0 Low Priority Mask. Mask bits for low priority interrupts. Each mask bit is set to 1 to enable a low priority interrupt when the corresponding bit in the Interrupt Register is 1. Bits Description 31:24 Reserved. 23:0 High Priority Mask. Mask bits for high priority interrupts. Each mask bit is set to 1 to enable a high priority interrupt when the corresponding bit in the General Interrupt Register is 1.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 123 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

2.3.2.5 Interrupt Register

Symbol: DIR Address Offset: 00010h Default Value: 00000000h Access: Read and Clear Interrupt Register. Bits Description 31:24 Reserved. 23 8B/10B Decoder Error.This is set if a 10B character received from the fiber optics interface is not a valid 8B/10B code as defined in the 8B/10B decoder lookup table. Output Fifo Overflow.This is set if the EPP attempts to push a cell into the output fifo when it is full. This output fifo can hold up to 4 cells. This is likely to be caused by incorrect programming of the “1-in-N” counter on the EPP. Input Fifo Overflow.This is set if the fiber optics receiver attempts to push a cell into the input fifo when it is full. The input fifo can hold up to 8 cells. This is likely to be caused by a malfunction in the “1-in-N” counter on the ingress linecard. 20 VCSEL Laser Active Up. This is set if the VCSEL laser array raises its normal operation status. 19 VCSEL Laser Active Down. This is set if the VCSEL laser array drops its normal operation status. 18 PIN Diode Signal Detect Up.This is set if the PIN diode receiver raises the detection of a light stream from the fiber optical link. 17 PIN Diode Signal Detect Down.This is set if the PIN diode receiver drops the detection of a light stream from the fiber optical link. 16 Transceiver Signal Detect Up.This is set if the transceiver raises the detection of a bit stream from the fiber optics PIN diode receiver. 15 Transceiver Signal Detect Down.This is set if the transceiver drops the detection of a bit stream from the fiber optics PIN diode receiver. 14 Transceiver Comma Detect. This is set if the transceiver indicates the detection of a comma character “0011111xxx”. 13 8B/10B Decoder Comma Detect . This is set if the fiber optics receiver looks up a 10B control character that maps to “0100000000” in the 8B/10B decoder lookup table. Fiber Optics Egress Ready Up.This is set if the fiber optics receiver on the egress linecard raises its ready status as a consequence of synchronization being locked on the 8B/10B control sequences. Fiber Optics Egress Ready Down.This is set if the fiber optics receiver on the egress linecard drops its ready status as a consequence of synchronization being lost on the 8B/10B control sequences.

124 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

2.3.2.6 AIB Reset

Symbol: DAIBRS Address Offset: 00030h Default Value: 00000003h Access: Read/Write Used to assert reset to the corresponding Crossbar AIB link. Bit 0 is for the Crossbar 0 AIB link, and bit 1 is for the Crossbar 1 AIB link. If reset is asserted, the link will not operate or transition to ready. Reset is asserted to 1 on power-up reset or if the reset bit in the control register is asserted. 10 Fiber Optics Ingress Ready Up.This is set if the fiber optics receiver raises its ready status as a consequence of synchronization being locked on the 8B/10B control sequences. 9 Fiber Optics Ingress Ready Down.This is set if the fiber optics receiver drops its ready status as a consequence of synchronization being lost on the 8B/10B control sequences. 8 Crossbar 1 Ready Up.This is set if the Crossbar 1 AIB link raises its ready status. 7 Crossbar 0 Ready Up.This is set if the Crossbar 0 AIB link raises its ready status. 6 Crossbar 1 Ready Down. This is set if the Crossbar 1 AIB link drops its ready status. 5 Crossbar 0 Ready Down. This is set if the Crossbar 0 AIB link drops its ready status. 4 Crossbar 0 and Crossbar 1 Ready Down Together.This is set if both Crossbar 0 and Crossbar 1 AIB links drop their ready status simultaneously. 3 Crossbar 1 CRC Error.This is set if a CRC error is flagged by the Crossbar 1 link receiver. 2 Crossbar 0 CRC Error.This is set if a CRC error is flagged by the Crossbar 0 link receiver. 1 Crossbar 0 and Crossbar 1 CRC Errors Together.This is set if CRC errors are simultaneously flagged by both Crossbar 0 and Crossbar 1 AIB link receivers. Crossbar 0 and Crossbar 1 Mismatch.This is set if both Crossbar 0 and Crossbar 1 AIB links are ready, there is no CRC error, and the data received from Crossbar 0 does not match that received from Crossbar 1. Bits Description 31:2 Reserved.

1 Crossbar 1 AIB Link Reset

0 Crossbar 0 AIB Link Reset

Bits Description (Continued)

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 125 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

2.3.2.7 AIB Ready

Symbol: DAIBRDY Address Offset: 00034h Default Value: 00000003h Access: Read Only Indicates if the corresponding Crossbar AIB link is ready. NOTE: Default value assumes that Primary and Secondary Crossbars are installed.

2.3.2.8 AIB Tx Enable

Symbol: DAIBEN Address Offset: 0003Ch Default Value: 00000000h Access: Read/Write Used to enable the transmitter of the corresponding Crossbar AIB link. If the corresponding Crossbar is not physically present in the system, the bit for the Crossbar should be set to zero to reduce electrical noise and to prevent electrical glitches when the Crossbar board is inserted or removed. Bits Description 31:2 Reserved.

1 Crossbar 1 AIB link

0 Crossbar 0 AIB link

31:2

1 Crossbar 1 AIB Link Enable Transmitter

0 Crossbar 0 AIB Link Enable Transmitter

126 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

2.3.2.9 Primary Crossbar Select

Symbol: DPRIXB Address Offset: 00080h Default Value: 00000000h Access: Read/Write Selects Crossbar 0 or Crossbar 1 as the primary Crossbar for fault tolerance.

2.3.2.10 Input Bypass Shift Register Depth

Symbol: DINBY Address Offset: 00084h Default Value: 00000002h Access: Read/Write Programs the depth of the Input Bypass Shift Register between 0 and 7 cells. This register defaults to a depth of 2 at power-up and reset time and should be changed to a value of 3 for use in a ETT1 system.. Bits Description 31:1 Reserved. 0 Primary Crossbar Select.A value of 0 selects Crossbar 0. A value of 1 selects Crossbar 1. Bits Description 31:3 Reserved. 2:0 Depth Range. Values from 0 to 7.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 127 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

2.3.2.11 Output Bypass Shift Register Depth

Symbol: DOUTBY Address Offset: 00088h Default Value: 00000002h Access: Read/Write Programs the depth of the Output Bypass Shift Register between 0 and 7 cells. This register defaults to a depth of 2 at power-up and reset time and should be changed to a value of 3 for use in a ETT1 system. 2.3.2.12 8B/10B Encoder Enable Symbol: DCODEN Address Offset: 0008Ch Default Value: 00000000h Access: Read/Write Enables the 8B/10B Encoder Bits Description 31:3 Reserved. 2:0 Depth Range. Values from 0 to 7. Bits Description 31:1 Reserved. 8B/10B Encoder Enable.This bit enables the 8B/10B encoder logic for the fiber optics transmitter. If the 8B/10B encoder is disabled, no cells are popped from the output fifo. It is generally a good idea to program the 8B/10B encoder lookup table before the 8B/10B encoder is enabled.

128 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. 2.3.2.13 8B/10B Decoder Enable Symbol: DDECEN Address Offset: 00090h Default Value: 00000000h Access: Read/Write Enables the 8B/10B Decoder.

2.3.2.14 VCSEL Laser Reset (Active-Low)

Symbol: DLSRRS Address Offset: 00094h Default Value: 00000000h Access: Read/Write Drive the active-low reset line on the Siemens PAROLI VCSEL laser array. NOTE: This is a general output pin and may be used for other purposes. Bits Description 31:1 Reserved. 8B/10B Decoder Enable.This bit enables the 8B/10B decoder logic for the fiber optics receiver. If the 8B/10B decoder is disabled, no synchronization is locked onto and therefore the fiber optics ready status is not asserted, and no tokens/cells are pushed into the token/input fifos. It is generally a good idea to program the 8B/10B decoder lookup table before the 8B/10B decoder is enabled. Bits Description 31-1 Reserved. VCSEL Laser Reset (Active-Low).This bit can be used to drive the active-low reset line on the VCSEL laser array. It defaults to a low level at power-up and reset time which causes the l a s e ra r r a yt ob es w i t c h e do f f .

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 129 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

2.3.2.15 VCSEL Laser Active Status

Symbol: DLSRACT Address Offset: 00098h Default Value: 00000000h Access: Read Only Indicates the status of the Siemens PAROLI laser array. NOTE: This is a general input pin and may be used for other purposes.

2.3.2.16 PIN Diode Reset (Active-Low)

Symbol: DPINRS Address Offset: 0009Ch Default Value: 00000000h Access: Read/Write Drives the active-low reset line on the Siemens PAROLI PIN diode array. NOTE: This is a general output pin and may be used for other purposes. Bits Description 31:1 Reserved. 0 VCSEL Laser Active Status.This bit indicates whether the VCSEL laser array is in normal operation (1) or there is a laser fault (0) or the reset line is asserted active-low (0). Bits Description 31:1 Reserved. PIN Diode Reset (Active-Low).This bit can be used to drive the active-low reset line on the PIN diode array. It defaults to a low level at power-up and reset time which causes the PIN diode array outputs to be driven low.

130 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

2.3.2.17 PIN Diode Signal Detect

Symbol: DPINDET Address Offset: 000A0h Default Value: 00000000h Access: Read Only Indicates whether the Siemens PAROLI PIN diode array detects a signal of sufficient AC power on the fiber optical link. NOTE: This is a general input pin and may be used for other purposes.

2.3.2.18 Transceiver Signal Detect

Symbol: DXCVDET Address Offset: 000A4h Default Value: 00000000h Access: Read Only Indicates whether the transceiver detects a bit stream of sufficient AC power from the PIN diode receiver. NOTE: This is a general input pin and may be used for other purposes. Bits Description 31:1 Reserved. 0 PIN Diode Signal Detect.This bit indicates whether the PIN diode array detects a signal of sufficient AC power on the fiber optical link. Bits Description 31:1 Reserved. 0 Transceiver Signal Detect.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 131 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

2.3.2.19 Transceiver Loopback Mode

Symbol: DXCVLB Address Offset: 000A8h Default Value: 00000000h Access: Read/Write Enables a local loopback mode as provided by the transceiver. NOTE: This is a general output pin and may be used for other purposes.

2.3.2.20 Fiber Optics Link Ready

Symbol: DFORDY Address Offset: 000ACh Default Value: 00000000h Access: Read Only Indicates synchronization and idle-ready state of the fiber optics transmitter. NOTE: This is a general input pin and may be used for other purposes. Bits Description 31:1 Reserved. 0 Transceiver Loopback Mode. This bit can be used to enable a local loopback mode as provided by the transceiver. Bits Description 31:2 Reserved. Egress Link Ready.This bit indicates that the linecard fiber optics transmitter transitioned to the idle-ready 8B/10B control sequence. The linecard should transition from the idle 8B/10B control sequence to the idle-ready 8B/10B control sequence when its fiber optics receiver is synchronized. Ingress Link Ready.This bit indicates that the fiber optics receiver is locked to the synchronization transmitted by the linecard. It causes the fiber optics transmitter to transition from the idle 8B/10B control sequence to the idle-ready 8B/10B control sequence.

132 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

2.3.2.21 Dataslice Loopback Mode

Symbol: DINTLB Address Offset: 000B0h Default Value: 00000000h Access: Read/Write Enables a local loopback mode on the Dataslice. 2.3.2.22 P LL Control/Status Symbol: DPLL Address Offset: 00100h Default Value: 0001447Ch Access: Read/Write Controls operation of the internal PLL (Phase Locked Loop). After a power on reset, the PLL itself is held in reset. This is reflected in bit 16 of this register. The local CPU must reset this bit to 0 to enable operation of the device, and thus should write 0000447Ch to this register. Bits Description 31:1 Reserved. 0 Dataslice Loopback Mode.This bit enables a local loopback mode on the Dataslice that causes data to be looped internally from the input fifo to the output fifo. Bit Description 31:17 PLL status.These bits reflect internal PLL operation status and should be ignored. Reset PLL.When set to 1 the PLL is held reset. The supplied reference clock will be used as the internal clock. The serial links will not be operational. This bit will be 1 after power-up reset and should be deasserted for normal operation. PLL reset takes 10mS to complete. 15:0 PLL control.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 133 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

2.3.2.23 Input Queue Memory (IQM)

Symbol: DIQM Address Offset: 40000-5FFFCh Default Value: Unknown Access: Read/Write The actual IQM address range extends from 40000h to 50FFCh. Beyond IQM address 50FFCh, aliasing occurs within the IQM 512x48 RAM bank. If the IQM is accessed at an even 32-bit aligned word address, the data bitmap consists of the following: If the IQM is accessed at an odd 32-bit aligned word address, the data bitmap consists of the following:

2.3.2.24 Output Queue Memory (OQM)

Symbol: DOQM Address Offset: 60000- 7FFFCh Default Value: Unknown Access: Read/Write The actual OQM address range extends from 60000h to 70FFCh. Beyond IQM address 70FFCh, aliasing occurs within the OQM 512x48 RAM bank. If the OQM is accessed at an even 32-bit aligned word address, the data bitmap consists of the following: If the IQM is accessed at an odd 32-bit aligned word address, the data bitmap consists of the following: Bits Description 31:16 Reserved. 15:0 This field contains the 16 MSBs of the addressed IQM 48-bit word. Bits Description 31:0 This field contains the 32 LSBs of the addressed IQM 48-bit word Bits Description 31:16 Reserved. 15:0 This field contains the 16 MSBs of the addressed OQM 48-bit word. Bits Description 31:0 This field contains the 32 LSBs of the addressed OQM 48-bit word

134 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. 2.3.2.25 8B/10B Encoder Lookup Table Symbol: D8BCOD Address Offset: 80000-807FCh Default Value: Unknown Access: Read/Write The actual 8B/10B encoder lookup table address range extends from 0x80000 to 0x804FC. Beyond 8B/10B encoder lookup table address 0x804FC aliasing will occur within the 8B/10B encoder lookup table 320x26 RAM. The 8B/10B encoder lookup table address space is defined as:  0x80000-0x803FC: Lookup of 8B data bytes into 10B data characters.  0x80400-0x80414: Lookup of idle 8B/10B control sequence (suggested 10B control sequence: 0x80400 = K28.5 and 0x80404-0x80414 = K27.7).  0x80420-0x80434: Lookup of idle-ready 8B/10B control sequence (suggested 10B control sequence: 0x80420 = K28.5 and 0x80424-0x80434 = K29.7). The 8B/10B encoder lookup table data bitmap consists of the following: NOTE: Bit 0 and bit 13 represent the bit that would be transmitted first for each respective 10B character. Bits Description 31:26 Reserved. 25:23 Weight of the positive RDS 10B character. 22:13 Positive RDS 10B character. 12:10 Absolute weight of the negative RDS 10B character. 9:0 Negative RDS 10B character.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 135 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. 2.3.2.26 8B/10B Decoder Lookup Table Symbol: D8BDEC Address Offset: 81000-81FFCh Default Value: Unknown Access: Read/Write The 8B/10B decoder lookup table data bitmap consists of the following: Bits Description 31:10 Reserved. 9 This bit flags an 8B/10B decode error. The entry that the 10B character indexes does not correspond to a valid 8B control or data byte. This bit identifies an 8B/10B control character. If the 10B character indexes an entry that has this control bit set and the data field [7:0] is 0x0, the 10B character is recognized as the comma character for synchronization. If the 10B character indexes an entry that has this control bit set and the data field [7:0] is 0xFF, the 10B character is recognized as an indication that the linecard fiber optics is ready. 7:0 This data field contains the corresponding 8B value.

2.4 DATASLICE SIGNAL DESCRIPTIONS

This section describes the Dataslice signals. PECL PECL are Pseudo-ECL (positive voltage ECL) compatible signals. that may be 1 to 15 meters apart. Table 22. Dataslice Signal Descriptions

Table 22. Dataslice Signal Descriptions (Continued)

test_di1 I CMOS T e s t( V D D )S h o u l db ed r i v e nt oG N Dd u r i n gr e s e t . All outputs are tristated when low. test_di2 I CMOS T e s t( V D D )S h o u l db ed r i v e nt oG N Dd u r i n gr e s e t . All outputs are tristated when low.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 141 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

2.5 PINOUT AND PACKAGE INFORMATION

2.5.1 Pinout Tables

Table 23. Dataslice Pinout (left side)

Table 24. Dataslice Pinout (right side)

Table 18. Dataslice Alpha Pin List

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 145 lssd_scan_in11 AE07 lssd_scan_in12 AC05 lssd_scan_in13 AE05 lssd_scan_in14 AE03 lssd_scan_in15 AA05 lssd_scan_in2 AE15 lssd_scan_in3 AC15 lssd_scan_in4 AE13 lssd_scan_in5 AC13 lssd_scan_in6 W11 lssd_scan_in7 AA11 lssd_scan_in8 AA09 lssd_scan_in9 W09 lssd_scan_out0 AC01 lssd_scan_out1 AC03 lssd_scan_out10 E11 lssd_scan_out11 G11 lssd_scan_out12 C13 lssd_scan_out13 A13 lssd_scan_out14 C15 lssd_scan_out15 A15 lssd_scan_out2 AA03 lssd_scan_out3 W03 lssd_scan_out4 P07 lssd_scan_out5 P05 lssd_scan_out6 M05 lssd_scan_out7 M07 lssd_scan_out8 G09 lssd_scan_out9 E09 mon0 AD06 mon1 AB07 mon10 U11 mon11 AB12 mon12 R11 mon13 T11 mon14 AB13 mon15 AD14 mon2 T09 mon3 R09 mon4 AB08 mon5 T10 mon6 P10 mon7 Y12 mon8 V11 mon9 AA13 No Pin A01 oob0_int_hi A19 oob0_int_lo D18 oob1_int_hi D04 oob1_int_lo D03 oob_ad0 B16 oob_ad1 F13 oob_ad2 G13 oob_ad3 B15 oob_ad4 D14 oob_ad5 H12 oob_ad6 E13 oob_ad7 B14 oob_clk D06 oob_devsel0 H07 Signal Name Pin oob_devsel1 K08 oob_devsel2 B04 oob_valid_L B08 oob_wait_L B05 p2d_d0_id0 F16 p2d_d0_id1 H15 p2d_d0_id2 H19 p2d_d0_id3 L19 p2d_d0_id4 J17 p2d_d0_id5 K13 p2d_d0_id6 G15 p2d_d0_id7 H18 p2d_d1_id0 K01 p2d_d1_id1 H02 p2d_d1_id2 F04 p2d_d1_id3 G05 p2d_d1_id4 J03 p2d_d1_id5 L01 p2d_d1_id6 H01 p2d_d1_id7 L07 p2d_ic0 E07 p2d_ic1 J09 p2d_ic2 D08 p2d_ic3 L09 p2d_ic4 F08 p2d_ic5 F07 p2d_ic6 H08 p2d_ic7 D07 p2d_oc0 L11 p2d_oc1 D13 p2d_oc2 J11 p2d_oc3 F12 p2d_oc4 D12 p2d_oc5 H11 p2d_oc6 H09 p2d_oc7 K10 pin_diode0_reset_L AB17 pin_diode0_sig_det AD18 pin_diode1_reset_L AB03 pin_diode1_sig_det AD02 plllock E05 plltest_in E03 plltest_out G03 pwrup_reset_L B06 ref_clk C03 ref_clkn C01 soc_in G07 soc_inn F06 test_di1 A05 test_di2 A03 test_lt C07 test_re C05 test_ri A07 UNUSED A09 UNUSED A11 UNUSED A17 UNUSED B07 UNUSED B09 UNUSED B10 UNUSED B11 Signal Name Pin UNUSED B12 UNUSED B13 UNUSED B18 UNUSED C09 UNUSED C11 UNUSED D05 UNUSED D09 UNUSED D10 UNUSED D11 UNUSED D15 UNUSED D16 UNUSED E19 UNUSED F05 UNUSED F09 UNUSED F10 UNUSED F11 UNUSED F14 UNUSED G01 UNUSED G19 UNUSED H10 UNUSED H13 UNUSED J07 UNUSED J13 UNUSED K12 UNUSED M01 UNUSED M02 UNUSED M03 UNUSED M04 UNUSED M06 UNUSED M14 UNUSED M16 UNUSED M17 UNUSED M18 UNUSED M19 UNUSED N01 UNUSED N03 UNUSED N05 UNUSED N07 UNUSED N13 UNUSED N15 UNUSED N17 UNUSED N19 UNUSED P01 UNUSED P02 UNUSED P03 UNUSED P04 UNUSED P06 UNUSED P14 UNUSED P16 UNUSED P17 UNUSED P18 UNUSED P19 UNUSED V10 UNUSED W01 UNUSED W19 UNUSED Y09 UNUSED Y10 UNUSED Y11 UNUSED AA01 UNUSED AA19 Signal Name Pin

146 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. UNUSED AB09 UNUSED AB10 UNUSED AB11 UNUSED AC09 UNUSED AC11 UNUSED AD01 UNUSED AD09 UNUSED AD10 UNUSED AD11 UNUSED AE09 UNUSED AE11 vcsel0_laser_act AE19 vcsel0_reset_L AB18 vcsel1_laser_act AE01 vcsel1_reset_L AB02 VDD A08 VDD A12 VDD C06 VDD C14 VDD E08 VDD G02 VDD G10 VDD G18 VDD J04 VDD J12 VDD J16 VDD L10 VDD M11 VDD N08 VDD N12 VDD P09 VDD R02 VDD R06 VDD R10 VDD R14 VDD R18 VDD U04 VDD U12 VDD U16 VDD W10 VDD AA08 VDD AC02 VDD AC06 VDD AC14 VDD AC18 VDD AE08 VDD AE12 VDDQ C18 VDDQ E16 VDDQ L14 VDDQ L18 VDDQ N16 VDDQ W18 VDDQ AA16 VDDQ U08 VDDQ W06 VDDQ W14 VDDQ AA12 VDDQ AC10 VDDQ AE04 Signal Name Pin VDDQ AE16 VDDQ C02 VDDQ E04 VDDQ L02 VDDQ L06 VDDQ N04 VDDQ W02 VDDQ AA04 VDDQ A04 VDDQ A16 VDDQ C10 VDDQ E12 VDDQ G06 VDDQ G14 VDDQ J08 VDDA E01 Vref M08 Vref K07 Vref H14 Vref K15 Vref K09 Vref K11 x2d_d0a_c W15 x2d_d0a_cn Y16 x2d_d0a_e V14 x2d_d0a_en T13 x2d_d0a_o V18 x2d_d0a_on T19 x2d_d0b_c T12 x2d_d0b_cn V13 x2d_d0b_e U13 x2d_d0b_en Y15 x2d_d0b_o AD13 x2d_d0b_on AB15 x2d_d1a_c W05 x2d_d1a_cn Y05 x2d_d1a_e V06 x2d_d1a_en U07 x2d_d1a_o T01 x2d_d1a_on V02 x2d_d1b_c AD05 x2d_d1b_cn AD08 x2d_d1b_e V08 x2d_d1b_en Y08 x2d_d1b_o Y07 x2d_d1b_on AA07 xcvr0_com_det AD16 xcvr0_loopback AB14 xcvr0_sig_det AB16 xcvr1_com_det AD04 xcvr1_loopback AB06 xcvr1_sig_det AB04 Signal Name Pin

2.5.2 Package Dimensions

NOTE: Drawings are not to scale. Figure 56. Dataslice CBGA Package Dimensions - Top and Side Views

Figure 57. Dataslice CBGA Package Dimensions - Bottom View Table 19. Dataslice CBGA Mechanical Specifications

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 149 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

2.5.3 Part Number

The PMC-Sierra Part Number for the Dataslice is: Dataslice PM9313-HC O6K2298 PQ PM9313-HC Dataslice BXXLLLLLLLL PMC Logo Source “B” Part Number PMC Part Number Device Name Revision Date Code Lot Number Terminal A01 Identifier Country of Origin Rev B BYYWW XXXXXXXXXXXX P M9313-HC Temp. range (C = Commercial 0-70 degrees C) Package code (U = CCGA, H = CBGA)Sequential part number Chipset family code, 1=TT1, 5 =TTX IC IRD prefix PMC prefix (1=SCH, 2=XBAR, 3=DS, 5=EPP)

150 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 151 PMC-Sierra, Inc.

3 Enhanced Port Processor

This chapter contains information on the Enhanced Port Processor device, part number PM9315-HC, available from PMC-Sierra, Inc. The Enhanced Port Processor (EPP) is based on the original TT1TM Port Processor, but has been greatly improved in terms of its ability to handle OC-48c channels. The original Port Processor (PP) could only support OC-48c at a single priority and required all linecards to be OC-48c. The EPP can support OC-48c at the four priority levels and can be used in OC-48c or OC-192c mode on a port-by-port basis. This enables the EPP to be used with the rest of the ETT1 Chip Set and to provide up to 128 ports of OC-48c. Also, multicast handling has been substantially improved over the PP; now the EPP can handle multicast at the OC-48c port level. However, the most important feature of the EPP is that it operates with the original TT1 Dataslice, Crossbar and Scheduler devices. T h em a i nf e a t u r e so ft h eE P Pa r e :  Four ports of OC-48c or one port of OC-192c.  Communicates with other EPPs (notPPs) which can be connected either to OC-48c linecards or OC-192c linecards.  Supports four priority levels for unicast and multicast for OC-48c or OC-192c.  Supports TDM channels for OC-48c and OC-192c.  Uses the LCS (Linecard to Switch) protocol. NOTE: The EPP itself operates on an OC-192c rate stream; therefore, a multiplexer function is needed for OC-48c mode. This multiplexer sits between the four OC-48c rate linecards and the ETT1 port card, and maps the four incoming OC-48c rate streams onto a single stream to the EPP. Each request label contains the information needed to identify its source OC-48c linecard. Figure 58 shows the interconnection of the EPP to other ETT1 devices.

Figure 58. An ETT1 Port Operating in Sub-port Mode with Four OC-48c Linecards

3.1 EPP DATA FLOWS AND BLOCKS

This section describes the functionality of the EPP and its basic structure. Figure 59. Functional Diagram of LCS Enhanced Port Processor

154 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc. The EPP is logically split up into an input/ingress EPP (iEPP) and an output/egress EPP (oEPP). The LCS Grant Manager sends grants for input queues which have some space left, and arbitrates among competing flows. The Grant Matching logic matches incoming cell bodies with their LCS grant labels. The iEPP Tag Information Base (iTIB) is a table of multicast port fanouts indexed by LCS multicast tags. The TDM Frame Tables are indexed by TDM slot and contain scheduling information. The Scheduler Request Modulator provides OC-48c granularity backpressure for unicast flows, arbitrates among competing best-effort flows, and restores OC-48c granularity to port-granularity grants from the Scheduler chip. The Output Scheduler block arbitrates among all egress flows to determine which flow may send a cell to the linecard each cell time. The following section describes cell flows within the EPP. The remaining sections describe the blocks of the EPP in more detail.

3.1.1 EPP Data Flows

3.1.1.1 iEPP Cell Arrival:LCS R equest-Grant-Cell Flow The EPP uses the LCS, version 2 (Linecard-to-Switch) protocol. This is a major enhancement to the original LCS protocol in that requests from the linecard are separated from their cell body, reducing the storage requirements of ETT1. The details of LCS are described elsewhere, but the basic operation is as follows: 1. A cell arrives at the ingress linecard and is destined for some output channel, O. 2. The linecard issues a request to the EPP, indicating the output queue (O). The cell body isnotsent to the EPP at this time. 3. The EPP returns a grant to the linecard, indicating that the cell body can now be forwarded to the switch. This grant is also an implicit credit, so that the linecard can issue a further request for this output (O). The input Dataslices, iDS0, iDS1 and iDS2, send their input request and cell data to the input EPP on d2p_d0_id[7:0], d2p_d1_id[7:0], d2p_d2_id[7:0], respectively. The LCS ingress header and first 10 bytes of cell payload are sent. The request portion of the LCS ingress header goes to the LCS Grant Manager, which arbitrates among all flows with requests (and space in the input queues) to choose the flow which will receive the next grant. The LCS Grant Manager sends the grant label and sequence number back to input Dataslice 0 via p2d_d0_id[7:0]. If the grant is unicast, its label information is put into a programmable delay line to the Scheduler Request Modulator. This allows requests to be sent to the Scheduler before the arrival of unicast cell bodies. The delay is programmed so that Scheduler grants will not come back before the arrival of the cell bodies, providing that the system round trip time constraint (64 celltimes) is met. The LCS Grant Manager also sends the grant label and sequence number to the Grant Matching block, which matches incoming cell sequence numbers with previously stored grant sequence numbers, and restores the associated label information to the cell whose payload has just arrived. When a match is made, several things happen:

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 155 PMC-Sierra, Inc.  The cell’s label information is sent to the input Queue Manager, which determines where the cell will be stored in input Dataslice queue memory. If the flow’s queue is already full, the cell is dropped and an interrupt is asserted. Otherwise, the EPP sends a write command & write address to all input Dataslices via p2d_d[6:0]_ic[7:0]. At the same time, the EPP sends egress cell label information to the input Dataslices, via p2d_d0_id[7:0] and p2d_d1_id[7:0]. This information is written into the egress cell header location of the cell.  If the cell is multicast, its tag (from the label) is used as the index into the iTIB, to look up the port fanout. If the fanout is all-0’s, the cell is dropped. Otherwise the fanout is sent to the Scheduler Request Modulator for arbitration with other unicast and multicast cells waiting to be requested. 3.1.1.2 iEPP Cell Departure: Scheduler Request-Grant Flow Unicast and multicast requests to the Scheduler come from the Scheduler Request Modulator, while TDM requests to the Scheduler come from the TDM Frames block. Requests are sent to both (redundant) Schedulers, p2s_s[1:0]. The Scheduler Request Modulator receives unicast egress queue occupancy information from all EPPs’ Output Schedulers via the Flow Control Crossbars, f2p_[a:b][1:0]. It uses this information to decide whether it can pass on requests to the Scheduler for cells destined for those queues. The Scheduler Request Modulator also keeps track of subport information for each request sent to the Scheduler, and restores the subport information to grants coming back from the Scheduler via s2p_s[1:0]. This is because the Scheduler only supports port-granularity in its arbitration. When subport information has been restored to a valid Scheduler grant, the grant info is sent to the Input Queue Manager. If the specified input queue is empty, then an interrupt is asserted and no cell is sent. Otherwise, the Input Queue Manager generates a read command and read address, sent to the input Dataslices via p2d_d[6:0]_ic[7:0]. The input Dataslices then send the cell across the Data Crossbars. The Scheduler also sends a reverse routing tag to the EPP, to indicate from which port this port will be receiving the next cell. This is passed to the input Dataslices via p2d_d[6:0]_ic[7:0]. The input Dataslices pass the tag on to the data Crossbars, to configure the Crossbar fabric. 3.1.1.3 oEPP Cell Arrival: Data Crossbar to oEPP Flow After a cell traverses the data Crossbars, it is sent out to the output Dataslices. oDS0 and oDS1 send their cell data to the oEPP via d2p_d0_od[7:0] and d2p_d1_od[7:0]. The Output Queue Manager determines which output queue the cell is destined for, and generates a write command and write address for the output Dataslices. This is sent via p2d_d[6:0]_oc[7:0]. The Output Scheduler is notified that a cell has been added to that output queue. If the incoming cell causes a multicast output queue to grow larger than the programmable backpressure threshold, then backpressure is asserted to the Scheduler via p2s_s[1:0]. Optionally, queue size can be ignored and arriving multicast cells will simply be dropped if the queue fills up.

156 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc. The Scheduler’s backpressure mechanism is not used for unicast because the Output Scheduler on each port provides the Scheduler Request Modulator on each port with subport-granularity queue information via the Flow Control Crossbars. The Scheduler’s backpressure mechanism is not used for TDM because TDM programming should ensure that the output queues cannot overflow. When multicast cells arrive at an oEPP in OC-48c mode, the source port and multicast tag are used to index the oTIB, to look up the egress subport fanout. The egress subport fanout for TDM cells is taken from the TDM Frame table entry for the slot whose cells are currently arriving at oEPPs. 3.1.1.4 oEPP Cell Departure: oEPP to Linecard Flow The Output Scheduler arbitrates among all flows with cells waiting to be sent to the linecards. When the oEPP is in OC-48 mode, it arbitrates for each output linecard in turn (subport 0, subport 1, subport 2, subport 3, subport 0,...). When the Output Scheduler decides to send a cell, it passes label information to the Output Queue Manager which sends a read command and read address to the Output Dataslices via p2d_d[6;0]_oc[7:0]. When a multicast queue size falls below the programmable unbackpressure threshold, unbackpressure is sent to the Scheduler. The Output Scheduler sends incremental credits for unicast egress queues to all EPPs’ Scheduler Request Modulators via p2f_[a.b][1:0].

3.1.2 LCS G rant Manager

The purpose of the LCS Grant Manager is to prevent any input queue from overflowing, and to arbitrate among competing flows. It consists of linecard request counters, input queue debit counters, and an arbiter.

3.1.2.1 Linecard Request Counters

Each linecard request counter is initialized to 0 at reset, incremented when a new LCS Request arrives for the corresponding flow, and decremented when the LCS Grant Manager sends an LCS Grant to the requesting linecard. Request counters can also be updated by LCS Control Packet or OOB access. They are stored in the Linecard Request Counters memory. See Section 3.4 “Enhanced Port Processor Registers” for detailed address and data formats of this memory. These counters are maintained within the EPP and no customer interaction is required. However, customers may choose to monitor request counts on various flows, or to implement request count updates using this memory instead of LCS Control Packets. For Control Packet and TDM queues, there is always one request counter per input queue. For multicast and unicast, if the EPP is in OC-48c mode, there are four request counters per input queue, one for each

this is described in Section 3.1.2.3 “LCS Grant Manager Arbitration”. request counter values depend on whether the EPP is in OC-48c mode, and are shown Table 20. auxiliary registers accessed along with CP request counters in the Linecard Request Count memory. These FIFOs are maintained within the EPP and no customer interaction is required.

3.1.2.2 Input Queue Debit Counters

queues; LCS Processed Control Packets are processed and dropped immediately). Table 20. Linecard Request Count Maximum Values

158 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc.

3.1.2.3 LCS G rant Manager Arbitration

The LCS Grant Manager arbiter makes a linecard grant decision every celltime, based on information from the Linecard Request Counters and Input Queue Debit Counters. When the EPP is in OC-48c mode, each queue is shared by 4 request counters, one per subport. Therefore, in OC-48c mode, additional state information about multicast and unicast flows is used in the decision, in order to prevent any request counter from being starved by the other request counters contending for the same queue. When the EPP is in OC-48c mode, the LCS Grant Manager arbitrates between the eligible flows requested by each subport in turn (subport 0, subport 1, subport 2, subport 3, subport 0,...). Thus each OC-48c linecard will receive at most one grant every 4 celltimes. A flow is eligible for arbitration if its request counter is nonzero and its debit counter is less than the queue size. The arbiter enforces the ETT1 service class priority scheme (Control Packets > TDM > MC0 > UC0 > order of flows considered in the round-robin is increasing {egress_subport, egress_port}. So for an egress port that is in OC-48c mode, its subports will appear once each 32 steps in the ordering. For an egress port in OC-192c mode, there is only one valid subport (0), but each egress_subport step for that port will map to subport 0. Because of the strict order in which grants are given to each subport in OC-48c mode, and the unpredictable timing of cells leaving a congested queue, it would be possible for some request counters to never receive grants if the arbitration were left as a free-for-all. In order to avoid this, the LCS Grant Manager “reserves” the last space in each unicast or multicast queue for a rotating favorite subport. When a debit counter indicates that there is only one space left in a queue, only the request counter of the favorite subport for that queue is eligible. Whenever the favorite subport gets a grant, regardless of whether the queue is down to its last space, the favorite subport changes to the next subport with a non-zero request count. This algorithm ensures that no request counter can be starved.

3.1.3 Scheduler Request Modulator

The purpose of the Scheduler Request Modulator is to prevent unicast output queues from overflowing. The Scheduler’s backpressure mechanism only supports port-granularity backpressuring, which is not acceptable for unicast (subport-granularity) queues. So instead of sending backpressure commands from the oEPP to the Scheduler, all oEPPs send queue occupancy information to all iEPPs via the Flow Control Crossbars. The Scheduler Request Modulator uses the output queue occupancy information to prevent overflowing any unicast output queue; if there is not enough room left in an output queue, no more requests for that output queue will be sent to the Scheduler until more room becomes available. Since unicast and multicast are interleaved per-priority in the overall ETT1 priority scheme, multicast requests are also arbitrated by the SRM. The SRM consists of input queue request counters, output queue debit counters, an arbiter for deciding which flow to request to the Scheduler, and state for restoring subport information to Scheduler grants.

3.1.3.1 Input Queue Request Counters

Input Queue request counters are initialized to 0 at reset. Unicast and multicast request counters behave slightly differently.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 159 PMC-Sierra, Inc. Unicast request counters are incremented when unicast request/grant labels arrive via a programmable delay line from the LCS Grant Manager. Using the programmable delay line instead of waiting for unicast cells to arrive allows unicast requests to be sent to the Scheduler ahead of arrival so that Scheduler Grants can arrive at the same time as the cells, effectively masking Scheduler latency from the round trip time. The delay line is programmed so that Scheduler grants cannot arrive before the cells arrive, provided that the system round trip time constraint is met (64 celltimes from the grant to the cell arrival). Multicast request counters are incremented when the Grant Matching block matches arriving multicast cells with their request/grant labels. Unicast and multicast request counters are decremented as requests are sent to the Scheduler.

3.1.3.2 Output Queue Debit Counters

Output queue debit counters are initialized to 0 at reset. They are incremented when unicast requests are sent to the Scheduler. When incremental output queue credits arrive via the Flow Control Crossbars, they are subtracted from the appropriate debit counters. The counters are contained in the Output Queue Debit Counters Memory. See Section 3.4 “Enhanced Port Processor Registers” for detailed address and data formats. These counters are maintained within the EPP and no customer interaction is required, unless the customer chooses to implement the Flow Control recovery procedure outlined in “Enhanced Port Processor - Flow Control Crossbar” on page 94. Output queue debit counts are maintained for unicast only, not multicast, since the Scheduler’s backpressure mechanism can be used to prevent multicast output queues from overflowing.

3.1.3.3 Scheduler Request Count Modulator Arbiter

The arbiter decides every celltime which best-effort flow, if any, to request to the Scheduler. A unicast flow is eligible for arbitration if its input queue request counter is nonzero and its output queue debit counter is less than the queue size. A multicast flow is eligible for arbitration if its request count is nonzero and if fewer than 64 requests for that multicast flow are currently waiting for grants from the Scheduler.. Multicast flows are included in the arbitration in order to enforce the ETT1 best-effort service class priority scheme (MC0 > UC0 > MC1 > ... > UC3), and to prevent 96-cell multicast input queues from exceeding the maximum Scheduler request count of 64. Within each unicast priority, a round-robin is used to arbitrate among eligible flows. The order of the round-robin is increasing (egress_subport, egress_port), like the LCS Grant Manager arbiter.

3.1.4 Input Queue Manager

The Input Queue Manager stores arriving cells when the Grant Matching block has restored their grant label information, and sends departing cells when the Scheduler Request Modulator restores subport granularity to Scheduler grants. Cells are dequeued only when the Scheduler sends a dequeue command along with a grant.

160 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc. Unicast input queues are per-priority, per-destination (port, subport). If the destination port is OC-192c, the only valid subport is 0 and the queue length is 64 cells. If the destination port is OC-48c, there is a separate queue per subport, 16 cells each. Multicast input queues are per-priority and are 96 cells in length. TDM input queues are per-subport. If this port is OC-192c, the only valid subport is 0 and the queue length is 96. If this port is OC-48c, there is a separate queue per subport, 24 cells each. When a TDM Sync command arrives from the Scheduler, TDM input queues are instantly emptied. If cells were present in any TDM input queue, the “Cells Flushed by TDM Sync” interrupt is raised. Control Packet input queues are per-subport. The length is 8 regardless of OC-48c mode. Control Packet queues never receive grants from the Scheduler. Instead, OOB must write (any value) to the appropriate “Linecard to OOB FIFO Subport [3:0] Status” register in order to dequeue a Control Packet cell. If a Control Packet arrives for a full queue, an “LC2OOB/CPU CP FIFO Overflow, LC[3:0]” interrupt is raised. If the Scheduler grants to an empty (unicast/multicast/tdm) queue, a “Scheduler Grant to Empty * Queue” is raised. If a cell of any traffic type arrives for a full queue, the cell will be dropped (no write command to the Dataslices). If there is a Scheduler grant to any empty queue, no cell will be transmitted (no read command to the Dataslices). All input queues are simple circular buffers.

3.1.5 Output Queue Manager

The Output Queue Manager stores arriving cells from the Data Crossbars, and sends/dequeues departing cells under control of the Output Scheduler. Unicast output queues are per-priority, per-source port, per-destination subport. If this EPP is OC-192c, the only valid subport is 0, and the length is 64. If this EPP is OC-48c, there is a separate output queue for each subport, and the length is 16. All unicast output queues are simple circular buffers. Multicast output queues are per-priority, shared among output subports. When a multicast cell arrives, it is stored at one location regardless of how many destination subports it is to be sent to. In OC-48c mode, a separate list of queue locations is maintained for each subport, so that each subport appears to have a different queue with up to 96 locations. Since the 96 locations are actually shared by all subports, it is possible for congestion on one subport to back up the entire queue. If multicast backpressure is enabled, then multicast traffic destined for other ports at the same priority may be blocked. The multicast output queues are not simple circular buffers, instead they use a linked-list representation. There is only one TDM output queue, length 96, shared among subports the using the same mechanism as the multicast output queues. There is no queue for egress Control Packets; instead, there is one allocated cell location in Dataslice Output Queue Memory for each type of egress Control Packet. Several of these locations must be initialized by the OOB before the switch begins to operate; see Section 3.3 “Output Dataslice Queue Memory Allocation with EPP” for details.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 161 PMC-Sierra, Inc. The Output Queue Manager maintains head and length information for unicast queues in the Output Unicast Queue Information Memory, which is documented in Section 3.4 “Enhanced Port Processor Registers”. This information is maintained within the EPP and no customer interaction is required. However, in order to implement the Flow Control recovery procedure outlined in “Enhanced Port Processor - Flow Control Crossbar” on page 94, the customer will need to read the length of each affected unicast queue. The Output Queue Manager is the source of the interrupts “Output UC or MC Queue Overflow” and “output TDM Queue Overflow”.

3.1.6 Output Scheduler

The purpose of the Output Scheduler is to arbitrate among cells in the output queues waiting to be sent to the output linecard(s). When the EPP is in OC-48c mode, the Output Scheduler arbitrates among cells destined for each subport in turn (subport0, subport1, subport2, subport3, subport0,...). Unicast traffic can be shut off on a per-source-port basis in the Output Scheduler, by the “Freeze Unicast Output Scheduler Flows” register (see Section 3.4 “Enhanced Port Processor Registers”). Hole requests from the linecard are sent to the Output Scheduler. When a hole request for a given priority is valid, the Output Scheduler will not send any unicast or multicast cell of that priority during that celltime. In OC-48c mode, separate hole requests come from each subport, and are delayed until that subport is due to receive a cell. When in LCS Stop mode, the Output Scheduler does not send cells to the linecard. Since the LCS Start/Stop status is per linecard, subports on the same OC-48c mode port could have different Start/Stop status. Even though cells are not sent to the linecard, TDM cells continue to be dequeued from the TDM output queues. This is to prevent one stopped subport from blocking traffic to other subports, since the output TDM/MC queues are shared among subports. If LCS Lossy mode is in effect, then multicast cells will also continue to be dequeued. The Output Scheduler also sends Control Packets to the linecard(s) under control of the “Send OOB2LC Control Packet” register. When the Scheduler sends a TDM Sync command, the Output Scheduler sends a TDM Sync Control Packet to the linecard(s). Several types of egress Control Packet must be written into the correct locations in the Dataslice Output Queue Memory by software before the switch begins to operate; see Section 3.3 “Output Dataslice Queue Memory Allocation with EPP” for details.

3.1.7 OOB Interface and Control/Status Registers

All of the devices have an OOB interface. This interface allows a single local CPU to control and monitor all of the devices within a core fabric. Internally, each device provides registers that can be mapped into the CPU’s address space. These registers are described in more detail in Section 3.4 “Enhanced Port Processor Registers”.

3.2 INPUT DATASLICE QUEUE MEMORY ALLOCATION WITH EPP

offsets, and then those offsets should be read/written for each Dataslice. 10, and is used to denote offsets within queues that are 24 or 96 cells in length.

3.3 OUTPUT DATASLICE QUEUE MEMORY ALLOCATION WITH EPP

offsets, and then those offsets should be read/written for each Dataslice. Table 21. Input Dataslice Queue Memory Allocation

locations in output Dataslice queue memory before starting EPP operation. 10, and is used to denote offsets within queues that are 96 cells in length. Table 22. Output Dataslice Queue Memory Allocation

Table 23. EPP Egress Control Packet Data Format and Dataslice OOB Addressing

3.4 ENHANCED PORT PROCESSOR REGISTER S

Selects” on page 79 for more information.

3.4.1 Enhanced Port Processor Summary

following Descriptions section for more information on individual registers. Read and Clear means that reading the register causes it to be cleared (reset to zero). Table 24. Enhanced Port Processor Register Summary

Table 24. Enhanced Port Processor Register Summary (Continued)

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 169 PMC-Sierra, Inc.

3.4.2 Enhanced Port Processor Register Descriptions

Read and Clear means that reading the register causes it to be cleared (reset to zero). All bits labeled as Reserved should be set to 0.

3.4.2.1 Status

Symbol: ESTS Address Offset: 00000h Default Value: 51000000h Access: Read Only

3.4.2.2 Reset and Control

Symbol: ETKNRS Address Offset: 00004h Default Value: 00000031h Access: Read/Write Reset and Control register. Bits Description 31:28 Chip ID Number.Identifies the specific ETT1 device. 27:24 Chip Revision Number. 23:2 Reserved. 1 High Priority Interrupt.1 = an outstanding high priority interrupt. One of the bits in the Interrupt Register is set, and is enabled via its corresponding high priority mask. 0 Low Priority Interrupt.1 = an outstanding low priority interrupt. One of the bits in the Interrupt Register is set, and is enabled via its corresponding low priority mask. Bits Description 31-15 Reserved. 14 13th and 14th Dataslice Enable.This bit must be set before bringing up the Dataslices’ fiber-optics links in systems that use 14 Dataslices. Note: This bit will read out in bit position 6. 13:6 Reserved Generate Truncated 16-bit CRC.For Switch to Linecard LCS headers, generate a 16-bit CRC (0x11021) over the header, but only send the lower 8 bits, if this bit is 1. If this bit is 0, generate an 8-bit CRC (0x107) over the header and send that.

170 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc.

3.4.2.3 Low Priority Mask

Symbol: EIRLMSK Address Offset: 00008h Default Value: 00000000h Access: Read/Write Interrupt Mask for interrupts.

3.4.2.4 High Priority Mask

Symbol: EIRHMSK Address Offset: 0000Ch Default Value: 00000000h Access: Read/Write Check full 16-bit CRC.For Linecard to Switch LCS headers, check a 16-bit CRC (0x11021) over the header, if this bit is 1. If this bit is 0, check an 8-bit CRC (0x107). This applies only to the LCS header and not to Processed Control Packet payload data. Include MuxChip’s Subport ID in CRC gen.If this bit is 1, generate the Switch to Linecard LCS header CRC including the MUX field in the grant label. If 0, generate the CRC with the MUX field masked to 0. Include MuxChip’s Subport ID in CRC check.If this bit is 1, check the Linecard to Switch LCS header CRC including the MUX field in the request label. If 0, check the CRC with the MUX field masked to 0. Small System Mode. If this bit is set, only Flow Control Crossbar A is used; FC Crossbar B is ignored. Port numbers must be even numbers. This corresponds to the Data Crossbars’ 16x16 and 8x8 modes. Reset. Writing a 1 to this location will reset the entire chip. It is equivalent to a hardware reset. This register is cleared automatically when the chip is reset. Soft reset takes 1mS to complete. NOTE: After resetting an EPP, write 0 to offsets 0x1d000 through 0x1d7fc in that EPP (a total of 512 OOB writes). When all EPPs are reset simultaneously (using a broadcast OOB write to the EPP Control register), 512 broadcast OOB writes can be used to reset those locations in all EPPs. Bits Description 31:0 Low Priority Mask. Mask bits for low priority interrupts. Each mask bit is set to 1 to enable a low priority interrupt when the corresponding bit in the Interrupt Register is 1. Bits Description (Continued)

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 171 PMC-Sierra, Inc. Interrupt Mask for interrupts.

3.4.2.5 Interrupt Register

Symbol: EIR Address Offset: 00010h Default Value: 00000000h Access: Read and Clear Interrupt Register. Bits Description 31:0 High Priority Mask. Mask bits for high priority interrupts. Each mask bit is set to 1 to enable a high priority interrupt when the corresponding bit in the Interrupt Register is 1. Bits Description 31 Scheduler Grant to Empty TDM Queue. This is set if a Scheduler grant arrives for a TDM input queue (VOQ) which has no cells waiting. 30 Scheduler Grant to Empty MC Queue. This is set if a Scheduler grant arrives for a MC input queue (VOQ) which has no cells waiting. 29 Cells Flushed by TDM Sync.This is set if a TDM Sync clears waiting cells from TDM input queues. If a TDM Sync arrives and the TDM input queues are already empty, this is not set. 28 Linecard Requests Flushed by TDM Sync. This is set if a TDM Sync clears nonzero TDM Subport request counters. 27 Scheduler Grant to Nonwaiting UC Queue.This is set if a Scheduler Grant arrives for a priority and port for which no subport queues are expecting grants. 26 oDS to oEPP Valid Mismatch.This is set if the three oDS to oEPP frame’s valid bit does not match. 25 iDS to iEPP Valid Mismatch.This is set if the three iDS to iEPP frame’s valid bit does not match. 24 Grant or Cell Body Dropped.This is set if 128 OC-192c celltimes have passed since a grant, but no cell payload has arrived with the grant sequence number. 23 Invalid Cell Sequence Number from Subport 3.This is sent if Subport 3 has sent a cell payload with a sequence number which does not match any outstanding grant. 22 Invalid Cell Sequence Number from Subport 2.This is sent if Subport 2 has sent a cell payload with a sequence number which does not match any outstanding grant. 21 Invalid Cell Sequence Number from Subport 1.This is sent if Subport 1 has sent a cell payload with a sequence number which does not match any outstanding grant. 20 Invalid Cell Sequence Number from Subport 0.This is sent if Subport 0 has sent a cell payload with a sequence number which does not match any outstanding grant.

172 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc. 19 Received Multicast Cell with Invalid Fanout.T h i si ss e ti fam u l t i c a s tc e l la r r i v e sa n di t s tag’s fanout is all-0s. CRC Error in LCS Header from LC??.This is set if an incoming cell has an LCS header CRC error or has a code error in input dataslices 0, 1, or 2. Since the MUX field of the request label could be corrupted, the source subport is unknown. 17 CRC Error in Processed Control Packet from Subport 3.T h i si ss e ti ft h e r ei saC R C Error over the first 10 bytes of payload data for a PCP from Subport 3. 16 CRC Error in Processed Control Packet from Subport 2.T h i si ss e ti ft h e r ei saC R C Error over the first 10 bytes of payload data for a PCP from Subport 2. 15 CRC Error in Processed Control Packet from Subport 1.T h i si ss e ti ft h e r ei saC R C Error over the first 10 bytes of payload data for a PCP from Subport 1. 14 CRC Error in Processed Control Packet from Subport 0.T h i si ss e ti ft h e r ei saC R C Error over the first 10 bytes of payload data for a PCP from Subport 0. 13 Received LCS Start Control Packet.This notifies OOB that a Subport has sent a Processed Control Packet of type “LCS Start”. 12 Received LCS Stop Control Packet.This notifies OOB that a Subport has sent a Processed Control Packet of type “LCS Stop”. 11 Received LC2OOB/CPU Control Packet from Subport 3.This notifies OOB that Subport 3 has sent a CPU Control Packet. 10 Received LC2OOB/CPU Control Packet from Subport 2.This notifies OOB that Subport 2 has sent a CPU Control Packet. 9 Received LC2OOB/CPU Control Packet from Subport 1. This notifies OOB that Subport 1 has sent a CPU Control Packet. 8 Received LC2OOB/CPU Control Packet from Subport 0.This notifies OOB that Subport 0 has sent a CPU Control Packet. 7 LC2OOB/CPU Control Packet FIFO Overflow, Subport 3.T h i si ss e ti fS u b p o r t3 overflows its 8-cell FIFO of CPU Control Packets to OOB. 6 LC2OOB/CPU Control Packet FIFO Overflow, Subport 2.T h i si ss e ti fS u b p o r t2 overflows its 8-cell FIFO of CPU Control Packets to OOB. 5 LC2OOB/CPU Control Packet FIFO Overflow, Subport 1.T h i si ss e ti fS u b p o r t1 overflows its 8-cell FIFO of CPU Control Packets to OOB. 4 LC2OOB/CPU Control Packet FIFO Overflow, Subport 0.This is set if Subport 0 overflows its 8-cell FIFO of CPU Control Packets to OOB. 3 Line Card Request Count Overflow.This is set if a Subport sends too many requests (the counter will stick at max value). 2 Scheduler Grant to Empty UC Queue.This is set if the scheduler grants to an empty UC input queue (VOQ). 1 Output MC or UC Queue Overflow.This is set if a cell from the crossbar overflows a MC or UC output queue. 0 Output TDM Queue Overflow.This is set if a cell from the Crossbar overflows a TDM output queue. Bits Description (Continued)

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 173 PMC-Sierra, Inc.

3.4.2.6 Low Priority AIB Interrupt Mask

Symbol: ELPAIBIM Address Offset: 00014h Default Value: 00000000h Access: Read/Write Enables low priority interrupts.

3.4.2.7 High Priority AIB Interrupt Mask

Symbol: EHPAIBM Address Offset: 00018h Default Value: 00000000h Access: Read/Write Enables low priority interrupts. Bits Description 31:24 Reserved. 23:0 Mask bits for low priority interrupts.Each mask bit is set to 1 to enable a low priority interrupt when the corresponding bit in the AIB Interrupt register is 1. Bits Description 31:24 Reserved. 23:0 Mask bits for high priority interrupts.Each mask bit is set to 1 to enable a high priority interrupt when the corresponding bit in the AIB Interrupt register is 1.

174 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc.

3.4.2.8 AIB Interrupt Register

Symbol: EAIBIR Address Offset: 0001Ch Default Value: 00000000h Access: Read and Clear Interrupt Register Bits Description 31:24 Reserved.

23 Flow Control Crossbar A1 CRC Error

22 Flow Control Crossbar A0 CRC Error

21 Flow Control Crossbar A1 Ready Down

20 Flow Control Crossbar A0 Ready Down

19 Flow Control Crossbar A1 Ready Up

18 Flow Control Crossbar A0 Ready Up

Flow Control Crossbar A Both Links CRC Error.If set, neither A0 nor A1 provided valid incremental credits during at least one celltime, so check the Invalid Incremental Credits interrupt register to see which ports’ output queue credits need repair. Flow Control Crossbar A Frame Mismatch.Though apparently valid, the frames received from A0 and A1 were not identical. In this case, the Primary Flow Control Crossbar A is selected.

15 Flow Control Crossbar B1 CRC Error

14 Flow Control Crossbar B0 CRC Error

13 Flow Control Crossbar B1 Ready Down

12 Flow Control Crossbar B0 Ready Down

11 Flow Control Crossbar B1 Ready Up

10 Flow Control Crossbar B0 Ready Up

Flow Control Crossbar B Both Links CRC Error. If set, neither B0 nor B1 provided valid incremental credits during at least one celltime, so check the Invalid Incremental Credits interrupt register to see which ports’ output queue credits need repair. Flow Control Crossbar B Frame Mismatch. Though apparently valid, the frames received from B0 and B1 were not identical. In this case the Primary Flow Control Crossbar B is selected.

7 Scheduler 1 CRC Error

6 Scheduler 0 CRC Error

5 Scheduler 1 Ready Down

4 Scheduler 0 Ready Down

3 Scheduler 1 Ready Up

2 Scheduler 0 Ready Up

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 175 PMC-Sierra, Inc.

3.4.2.9 Low Priority Incremental Credit Interrupt Mask

Symbol: ELPICIR Address Offset: 00020h Default Value: 00000000h Access: Read/Write Enables a low priority interrup

3.4.2.10 High Priority Incremental Credit Interrupt Mask

Symbol: EHPICIR Address Offset: 00024h Default Value: 00000000h Access: Read/Write Enables a high priority interrupt. 1 Scheduler Both Links CRC Error.If set, traffic for this port is frozen and OOB must initiate Scheduler Refresh to repair Scheduler state for this port. 0 Scheduler Frame Mismatch. Though apparently valid, the frames received from Sched0 and Sched1 were not identical. In this case the Primary Sched is selected. Bits Description 31:0 Mask bits for low priority interrupts.Each mask bit is set to 1 to enable a low priority interrupt when the corresponding bit in the Invalid Incremental Credit Interrupt register is 1. Bits Description 31:0 Mask bits for High priority interrupts.Each mask bit is set to 1 to enable a high priority interrupt when the corresponding bit in the Invalid Incremental Credit Interrupt register is 1. Bits Description (Continued)

176 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc.

3.4.2.11 Invalid Incremental Credit Interrupt Register

Symbol: EIICIR Address Offset: 00028h Default Value: 00000000h Access: Read and Clear Interrupt register.

3.4.2.12 AIB Reset

Symbol: EAIBRS Address Offset: 00030h Default Value: 0000003Fh Access: Read/Write Resets AIB link for each port. Bits Description 31:0 Each bit corresponds to a single port. Invalidincremental credits were received from either Flow Control Crossbar A or Flow Control Crossbar B. Bits Description 31:6 Reserved. 5 Flow Control Crossbar A1 AIB Reset. 4 Flow Control Crossbar A0 AIB Reset. 3 Flow Control Crossbar B1 AIB Reset. 2 Flow Control Crossbar B0 AIB Reset. 1 Scheduler 1 AIB Reset. 0 Scheduler 0 AIB Reset.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 177 PMC-Sierra, Inc.

3.4.2.13 AIB Ready

Symbol: EAIBRDY Address Offset: 00034h Default Value: 00000000h Access: Read Only Indicates if the corresponding AIB link is ready.

3.4.2.14 Enable Port Processor

Symbol: EEN Address Offset: 00038h Default Value: 00000000h Access: Read/Write Indicates that the Port Processor is enabled (1) or disabled (0). Bits Description 31:6 Reserved. 5 Flow Control Crossbar A1 AIB Ready. 4 Flow Control Crossbar A0 AIB Ready. 3 Flow Control Crossbar B1 AIB Ready. 2 Flow Control Crossbar B0 AIB Ready. 1 Scheduler 1 AIB Ready. 0 Scheduler 0 AIB Ready. Bits Description 31:1 Reserved This bit indicates that the port processor is enabled (1) or disabled (0). The input port processor drops all cells (including Control Packets) from the linecard or crossbar and ignores all scheduler control information if this bit is 0.

178 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc.

3.4.2.15 AIB Enable Transmitter

Symbol: EAIBEN Address Offset: 0003Ch Default Value: 00000000h Access: Read/Write Used to enable the transmitter of the corresponding AIB link. If the corresponding central Scheduler is not physically present in the system, this bit should be set to zero to reduce unwanted electrical noise and to minimize unwanted effects when either Scheduler board is inserted

3.4.2.16 LCS S tarted/Stopped Status/Control

Symbol: ELCSSSS Address Offset: 00040h Default Value: 00000000h Access: Read/Write This register resets to 0 (all subports stopped). Traffic must be started by Processed Control Packet or OOB after reset. Bits Description 31:6 Reserved 5 Flow Control Crossbar A1 AIB Enable Tx. 4 Flow Control Crossbar A0 AIB Enable Tx. 3 Flow Control Crossbar B1 AIB Enable Tx. 2 Flow Control Crossbar B0 AIB Enable Tx. 1 Scheduler 1 AIB Enable Tx. 0 Scheduler 0 AIB Enable Tx. Bits Description 31:8 Reserved

7 Subport 3 egress MC queues are “lossy” if LCS Stopped

6 Subport 2 egress MC queues are “lossy” if LCS Stopped

5 Subport 1 egress MC queues are “lossy” if LCS Stopped

4 Subport 0 egress MC queues are “lossy” if LCS Stopped

3 Subport 3 is started if 1, stopped if 0

2 Subport 2 is started if 1, stopped if 0

1 Subport 1 is started if 1, stopped if 0

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 179 PMC-Sierra, Inc. NOTE: The “lossy” behavior is defined in the “LCS Protocol Specification”.

3.4.2.17 Lost Grant or Cell Body Debug Info

Symbol: ELGCBD Address Offset: 00044h Default Value: 00000000h Access: Read and Clear Indicates the number of lost grants or cells.

3.4.2.18 Scheduler Grant to Empty Queue Debug Info

Symbol: ESGEQD Address Offset: 00048h Default Value: 00000000h Access: Read and Clear Indicates the number of grants to an empty queue. 3.4.2.19 LCS H eader CRC Error Count Symbol: ELCSECT Address Offset: 0004Ch Default Value: 00000000h Access: Read and Clear

0 Subport 0 is started if 1, stopped if 0

31:30 Reserved 29:12 The first LCS grant label for which a cell was expected but lost. 11:0 The total number of cells which have been expected but lost. Bits Description 31:27 Reserved 26:16 The first QID for which there was a scheduler grant to empty queue 15:0 The total number of scheduler grants to empty queues Bits Description (Continued)

180 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc. Indicates the number of CRC Errors in Linecard to Switch LCS headers

3.4.2.20 LCS P rocessor CP Subport 0 CRC Error Count

Symbol: ELCSP0CT Address Offset: 00050h Default Value: 00000000h Access: Read and Clear Indicates the number of CRC Errors in processed CP from Subport 0. 3.4.2.21 LCS P rocessor CP Subport 1 CRC Error Count Symbol: ELCSP1EC Address Offset: 00054h Default Value: 00000000h Access: Read and Clear Indicates the number of CRC Errors in processed CP from Subport1. 3.4.2.22 LCS P rocessor CP Subport 2 CRC Error Count Symbol: ELCSP2EC Address Offset: 00058h Default Value: 00000000h Access: Read and Clear Indicates the number of CRC Errors in processed CP from Subport 2. Bits Description 31:0 The total number of detected CRC Errors in Linecard to Switch LCS headers Bits Description 31:0 The total number of detected CRC Errors in Processed CPs from Subport 0. Bits Description 31:0 The total number of detected CRC Errors in Processed CPs from Subport 1. Bits Description 31:0 The total number of detected CRC Errors in Processed CPs from Subport 2.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 181 PMC-Sierra, Inc.

3.4.2.23 LCS P rocessor CP Subport 3 CRC Error Count

Symbol: ELCSP3EC Address Offset: 0005Ch Default Value: 00000000h Access: Read and Clear Indicates the number of CRC Errors in processed CP from Subport 3.

3.4.2.24 Input

Symbol: EIS0VCT Address Offset: 00060h Default Value: 00000000h Access: Read and Clear Indicates the number of cells matched with grant sequence numbers

3.4.2.25 Input

Symbol: EIS1VCT Address Offset: 00064h Default Value: 00000000h Access: Read and Clear Indicates the number of cells matched with grant sequence numbers Bits Description 31:0 The total number of detected CRC Errors in Processed CPs from Subport 3. Bits Description 31:0 The total number of cells matched with grant sequence numbers for Subport 0. Bits Description 31:0 The total number of cells matched with grant sequence numbers for Subport 1.

182 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc.

3.4.2.26 InputSubport 2 Valid Count

Symbol: EIS2VCT Address Offset: 00068h Default Value: 00000000h Access: Read and Clear Indicates the number of cells matched with grant sequence numbers

3.4.2.27 Input

Symbol: EIS3VCT Address Offset: 0006Ch Default Value: 00000000h Access: Read and Clear Indicates the number of cells matched with grant sequence numbers

3.4.2.28 Output

Symbol: EOS0VCT Address Offset: 00070h Default Value: 00000000h Access: Read and Clear Indicates the number of cells matched with grant sequence numbers Bits Description 31:0 The total number of cells matched with grant sequence numbers for Subport 2. Bits Description 31:0 The total number of cells matched with grant sequence numbers for Subport 3. Bits Description 31:0 The total number of cells sent to Subport 0.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 183 PMC-Sierra, Inc.

3.4.2.29 Output Subport 1 Valid Count

Symbol: EOS1VCT Address Offset: 00074h Default Value: 00000000h Access: Read and Clear Indicates the number of cells matched with grant sequence numbers

3.4.2.30 Output

Symbol: EOS2VCT Address Offset: 00078h Default Value: 00000000h Access: Read and Clear Indicates the number of cells matched with grant sequence numbers

3.4.2.31 Output

Symbol: EOS3VCT Address Offset: 0007Ch Default Value: 00000000h Access: Read and Clear Indicates the number of cells matched with grant sequence numbers Bits Description 31:0 The total number of cells sent to Subport 1. Bits Description 31:0 The total number of cells sent to Subport 2. Bits Description 31:0 The total number of cells sent to Subport 3.

184 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc.

3.4.2.32 Subport 0 to OOB FIFO Status

Symbol: ES0OOBS Address Offset: 00080h Default Value: 00000000h Access: Read/Write Write of any value will dequeue the head of the FIFO. 3.4.2.33 Subport 1 to OOB FIFO Status Symbol: ES1OOBS Address Offset: 00084h Default Value: 00000000h Access: Read/Write Write of any value will dequeue the head of the FIFO. Bits Description 31:7 Reserved 6:4 The offset of the head of the 8-deep FIFO in DS queue memory. 3:0 The length (0 to 8) of the FIFO in DS queue memory. Bits Description 31:7 Reserved 6:4 The offset of the head of the 8-deep FIFO in DS queue memory. 3:0 The length (0 to 8) of the FIFO in DS queue memory.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 185 PMC-Sierra, Inc.

3.4.2.34 Subport 2 to OOB FIFO Status

Symbol: ES2OOBS Address Offset: 00088h Default Value: 00000000h Access: Read/Write Write of any value will dequeue the head of the FIFO. 3.4.2.35 Subport 3 to OOB FIFO Status Symbol: ES3OOBS Address Offset: 0008Ch Default Value: 00000000h Access: Read/Write Write of any value will dequeue the head of the FIFO Bits Description 31:7 Reserved 6:4 The offset of the head of the 8-deep FIFO in DS queue memory. 3:0 The length (0 to 8) of the FIFO in DS queue memory. Bits Description 31:7 Reserved 6:4 The offset of the head of the 8-deep FIFO in DS queue memory. 3:0 The length (0 to 8) of the FIFO in DS queue memory.

186 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc.

3.4.2.36 Send OOB to Linecard Control Packet

Symbol: ESOBLCP Address Offset: 00090h Default Value: 00000000h Access: Read/Write Before sending a CPU (OOB to linecard) control packet, the OOB must first write the control packet header and payload data into the appropriate locations in the Dataslice. See section 3.3, Table 28, “EPP Egress Control Packet Data Format and Dataslice OOB Addressing.” The header for customer-specific CPU control packets should be written into the Dataslices as shown, but the payload data is completely up to the customer. To send the CPU control packet which has been written into Dataslice queue memory, the OOB writes the ESOBLCP register with the control packet type select in bits 5:4 (see the bit-breakout), and a linecard fanout in bits 3:0. If the port is connected to 4 subport linecards, then bits 3:0 are a subport-bitmap. If the port is connected to one OC-192-rate linecard, then bit 0 must be set when the OOB wishes to send a CP . When the CP has been sent to the linecard(s) indicated in bits 3:0, bits 3:0 will read back as 0. Since control packets have higher priority than any other traffic type, they will be sent immediately, unless the EPP is programmed to send only idle cells. Bits Description 31:6 Reserved 5:4 Control Packet type select 0: Generic OOB to Subport Control Packet 2: LCS Stop Control Packet 3: LCS Start Control Packet

3 Subport 3 fanout for OOB to Subport Control Packet

2 Subport 2 fanout for OOB to Subport Control Packet

1 Subport 1 fanout for OOB to Subport Control Packet

0 Subport 0 fanout for OOB to Subport Control Packet

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 187 PMC-Sierra, Inc.

3.4.2.37 Suggested TDM Sync Subport Select

Symbol: ESTDMSSS Address Offset: 00094h Default Value: 00000000h Access: Read/Write Sync Support Select

3.4.2.38 Multicast Invalid Fanout Count

Symbol: EMCIFCT Address Offset: 00098h Default Value: 00000000h Access: Read and Clear Fanout count Bits Description 31:2 Reserved 1:0 Subport select.Subport from which (suggested) TDM Sync Processed Control packets will be accepted and forwarded to the scheduler, if this port is not enabled to internally generate suggested TDM Syncs. Bits Description 31:30 Reserved 29:28 First subport from which a multicast cell with all-0 fanout was received. 27:16 First tag which yielded the all-0 fanout. 15:0 Total number of cells with invalid fanouts that have been received (and dropped).

188 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc.

3.4.2.39 Internal Delay Matching Adjustments

Symbol: EIDMA Address Offset: 0009Ch Default Value: 03A8D66Bh Access: Read/Write Delay matching Bits Description 31:29 Reserved 28:27 DS iFIFO Token Mechanism Delay.Specifies the depth of the token shift register. Default at 26:23 Flow Control Crossbar oEPP->iEPP Delay.Specifies the time it takes for a Flow Control frame to reach the iEPP after it sent from any oEPP . The value of this register will not require adjustment unless there is a change in timing in a future revision of the EPP or Crossbar. 22:18 Unicast Scheduler Request to Waiting Delay.Specifies the minimum delay for a priority 0 UC scheduler request to be granted. The value of this register will not require adjustment unless there is a change in timing in a future revision of the EPP or Scheduler. 17:12 TDM Ingress Subport to Scheduler Grant Delay.Specifies the delay between a valid input TDM slot and the corresponding scheduler TDM grant. The value of this register will not require adjustment unless there is a change in timing in a future revision of the EPP or Scheduler. 11:6 TDM Egress Fanout Delay. Specifies the delay between a valid output TDM slot and the arrival of the TDM cell. The value of this register will not require adjustment unless there is a change in timing in a future revision of the EPP, DS, Crossbar or Scheduler. 5:0 LCS Unicast Grant to Scheduler Request Delay.The value of this register will not require adjustment, provided that the customer can meet the RTT constraint given in Section 1.1 “Preventing Underflow of the VOQ” on page 327.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 189 PMC-Sierra, Inc.

3.4.2.40 Output Backpressure/Unbackpressure Threshold

Symbol: EOPBPTHR Address Offset: 000A0h Default Value: 00001C46h Access: Read/Write These thresholds are for Multicast queues only.

3.4.2.41 Egress OC-48/OC-192c Mode Map

Symbol: EEMM Address Offset: 000A4h Default Value: 00000000h Access: Read/Write Chooses OC-48c/OC-192c Mode. Bits Description 31:15 Reserved 14:8 Output Unbackpressure Threshold.This specifies when the output port processor de-asserts backpressure. When the Output Scheduler grants to an output queue, the output port processor checks if the number of cells in that queue is equal to the threshold value. If the two values are equal, then an unbackpressure signal is sent to the central scheduler. If the EPP is in OC-48c mode, then this value should be changed to 1Fh.

7 Reserved

6:0 Output Backpressure Threshold.This specifies when the output port processor asserts backpressure. When a cell from crossbar enters an output queue, the output port processor checks if the number of cells already in that queue is equal to the threshold value. If the two values are equal, then a backpressure signal is sent to the central scheduler. Bits Description 31:0 Port Mode select. Bits 0:31correspond to individual ports. If the bit is set to 1, the port is OC-48; if set to 0, the port is OC-192c.

190 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc.

3.4.2.42 TDM Offset Value

Symbol: ETDMO Address Offset: 000A8h Default Value: 00000080h Access: Read/Write Determines TDM offset value.

3.4.2.43 Freeze Control

Symbol: EFCTL Address Offset: 000ACh Default Value: 00000000h Access: Read/Write Freezes TDM grants, requests and cells. Bits Description 31:8 Reserved 7:0 Specifies TDM Offset Value.The number of cell times from the arrival of TDM sync from the central scheduler to the recognition of the first time slot in the TDM table. Note that this value must be identical in all ports participating in TDM traffic. Bits Description 31:6 Reserved 5 Freeze TDM grants from LCS Grant Manager.If this bit is 1, the LCS Grant Manager sends no TDM grants to the linecard. 4 Freeze TDM Scheduler Requests. If this bit is 1, TDM input requests are not sent to the Scheduler. 3 Freeze TDM cells from Output Scheduler.If this bit is 1, the Output Scheduler does not schedule any TDM cells onto the output line. 2 Freeze MC/UC grants from LCS Grant Manager.If this bit is 1, the LCS Grant manager sends no MC or UC grants to the linecard. 1 Freeze Scheduler Request Modulator.If this bit is 1, the SRM sends no MC or UC requests to the scheduler. 0 Freeze MC/UC cells from Output Scheduler.If this bit is 1, the Output Scheduler does not schedule any MC or UC cells onto the output line.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 191 PMC-Sierra, Inc.

3.4.2.44 Freeze Unicast Output Scheduler Flows

Symbol: EFUOS Address Offset: 000B0h Default Value: 00000000h Access: Read/Write Determines whether or not the Output Scheduler sends unicast cells from specific ports.

3.4.2.45 Multicast Priority Drop

Symbol: EMPD Address Offset: 000B4h Default Value: 00000000h Access: Read/Write This specifies if a given priority should allow multicast cells to be dropped. Bits Description 31:0 Bits 31:0 correspond to specific ports.If the bit for the port is set to 1, the Output Scheduler sends no unicast cells from that port. Bits Description 31:4 Reserved 3:0 This specifies if a given priority should allow multicast cells to be dropped.If a bit is 1, then the output port processor does not assert backpressure for the corresponding multicast queue. If a bit is 0, then the output port processor prevents dropping of multicast cells by asserting backpressure to the central scheduler. Bit 3 is for priority 3, bit 2 is for priority 2, bit 1 is for priority 1, and bit 0 is for priority 0.

192 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc.

3.4.2.46 Fault Tolerance Status/Control

Symbol: EFTCTRL Address Offset: 000B8h Default Value: 00000000h Access: Read/Write Fault Tolerence Status/Control Register. Bits Description 31:5 Reserved Primary Flow Control Crossbar B Select.This bit specifies which Flow Control Crossbar B is primary. If this bit is 0, then Flow Control Crossbar B 0 is primary. If this bit is 1, then Flow Control Crossbar B 1 is primary. This bit is both a status and control bit. When the primary Flow Control Crossbar B goes down, the EPP must switch over to the secondary Flow Control Crossbar B (causes this bit to flip). OOB can also change the configuration. Primary Flow Control Crossbar A Select.This bit specifies which Flow Control Crossbar A is primary. If this bit is 0, then Flow Control Crossbar A 0 is primary. If this bit is 1, then Flow Control Crossbar A 1 is primary. This bit is both a status and control bit. When the primary Flow Control Crossbar A goes down, the EPP must switch over to the secondary Flow Control Crossbar A (causes this bit to flip). OOB can also change the configuration. Ignore Central Scheduler Grants.This bit indicates that the EPP is ignoring grants from both primary and secondary central schedulers. The EPP asserts this bit if both schedulers have transmitted CRC errors in the same frame, if the only operational scheduler has transmitted a CRC error, or if the only operational scheduler’s AIB link dropped ready. Note that t h eO O Bm u s tw r i t ea0t ot h i sb i tt oc l e a ri t . Freeze Scheduler Request Modulator (Multicast & Unicast only).This bit indicates that the SRM is frozen (TDM traffic can still flow) due to fault tolerance errors. The EPP asserts this bit if both schedulers have transmitted CRC errors in the same frame, if the only operational scheduler has transmitted a CRC error, or if the only operational scheduler’s AIB link dropped ready. Note that the OOB must write a 0 to this bit to clear it. Primary Scheduler Select.This bit specifies which scheduler is primary. If this bit is 0, then scheduler 0 is primary. If this bit is 1, then scheduler 1 is primary. This value must be consistent on all the port processors. This bit is both a status and control bit. When the primary scheduler goes down, all EPPs must switch over to the secondary scheduler (causes this bit to flip). The OOB processor can also change the configuration.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 193 PMC-Sierra, Inc.

3.4.2.47 Refresh Schedulers Command

Symbol: ERFRSSC Address Offset: 000BCh Default Value: 00000000h Access: Write Only Causes the EPP to start the refresh operation.

3.4.2.48 Go Schedulers Command

Symbol: ESTRTSC Address Offset: 000C0h Default Value: 00000000h Access: Write Only A write to this address causes an EPP to end the refresh operation. Bits Description 31:0 Start Refresh Schedulers Operation.A write to this address (any data) causes the EPP to start the scheduler refresh operation. The EPP freezes the Scheduler Request Modulator, freezes the Output Scheduler, and waits for in-flight cells to arrive before transferring state information to the scheduler(s). Note that OOB must must first disable the scheduler’s non-TDM traffic register for this port before issuing this command. Bits Description 31:0 Synchronously Start Schedulers Operation.A write to this address (any data) causes the EPP to end the refresh operation. Since the primary and secondary scheduler must transmit the same control in lock step, this command is used to synchronize the two schedulers. Note that OOB must first enable the scheduler’s non-TDM traffic register for this port before issuing this command.

194 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc.

3.4.2.49 Idle Count Value

Symbol: EIDLCT Address Offset: 000C4h Default Value: 00000000h Access: Read/Write This register is used to compensate for the clock frequency variation between the ETT1 switch and the linecard.

3.4.2.50 Enable TDM Sync Generation

Symbol: ETDMEN Address Offset: 000C8h Default Value: 00000000h Access: Read/Write Indicates if the TDM sync generation is enabled (1) or disabled (0). Bits Description 31:0 Idle Count Value.This represents the frequency of sending idle frames to the line card. This register is used to compensate for the clock frequency variation between the ETT1 switch and the linecard. If this register is set to 0x0, only idle frames are transmitted to the linecard. If this register is set to 0xFFFFFFFF, no idle frames are transmitted. If this register is set to N, an idle frame is sent every N + 1 frame times. Note that this register resets to 0x0. Bits Description 31:1 Reserved. Enable TDM Sync Generation.This bit indicates if the TDM sync generation is enabled (1) or disabled (0). If the linecard chooses not to provide the TDM sync signal, any one of the EPPs can be programmed to generate this signal.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 195 PMC-Sierra, Inc.

3.4.2.51 TDM Sync Generation Control

Symbol: ETDMCTRL Address Offset: 000CCh Default Value: 00000000h Access: Read/Write TDM Sync Generation Control register. 3.4.2.52 No Grant Count Symbol: ENOGCT Address Offset: 000DOh Default Value: FFFFFFFFh Access: Read/Write No Grant Count register. Bits Description 31:13 Reserved. 12 TDM Frame Select.This bit specifies which TDM frame table to select. 11:0 TDM Sync Frequency. This represents the frequency of the TDM sync pulse. If this register is set to N, a TDM sync signal is sent to the central Scheduler every N + 1 frame times. Bits Description 31:0 Controls the frequency of sending cells without grants to the line card.This register is used to guarantee that the linecard gets a gap in the stream of grants often enough so that it can send idle cells without holding up cell payloads. If this register is set to 0x0, no grants are sent to the linecard. If this register is set to 0xFFFFFFFF, no grants are held up. If this register is set to N, a cell without a grant is sent every N+1non-idleframe times. Note that this register resets to 0xFFFFFFFF. In OC-48 mode, (N+1) should be odd so that no-grant celltimes are distributed equally to all four linecards, so the value written to this register (N) should be even.

196 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc.

3.4.2.53 End of Sch Refresh/Sch OC-48 Sync ModCount

Symbol: EESR48SD Address Offset: 000D4h Default Value: 00000000h Access: Write Only Specifies the value of an internal OC-48 Sync counter at which the OOB Go Schedulers command should trigger the end of Scheduler refresh.

3.4.2.54 PLL Status/Control

Symbol: EPLLREG Address Offset: 00100h Default Value: 0001447Ch Access: Read/Write For more details, refer to the IBM SA-12E databook. Bits Description 31:5 Reserved. 4:0 Prevents OC-48 Sync phase change after Scheduler refresh.The value of this register will not require adjustment unless there is a change in timing in a future revision of the EPP or Scheduler. Bits Description 31:21 Reserved. 30:20 PLL_OBSERVE (read only). 19:17 Reserved 16 PLL_RESET. When set, VCO is held at minimum frequency and PLL is bypassed. PLL reset takes 10mS to complete. 15:10 PLL_TUNE. 9:6 PLL_MULT. 5:3 PLL_RANGE_B. 2:0 PLL_RANGE_A.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 197 PMC-Sierra, Inc.

3.4.2.55 TDM Frame Table 0 Memory

Symbol: ETDMFT0M Address Offset: 02000-02FFCh Default Value: 000000000h Access: Read/Write Each entry represents a time slot in the TDM table.

3.4.2.56 TDM Frame Table 1 Memory

Symbol: ETDMFT1M Address Offset: 03000-03FFCh Default Value: 000000000h Access: Read/Write Each entry represents a time slot in the TDM table. Bits Description 31:23 Reserved. 22 Input Valid.This bit is 1 if the input side TDM entry is valid. 21:20 Input Subport.(OC-48 only) Input linecard which provides the TDM cell to send. 19:10 Input Tag .The tag that the head of the input TDM tag FIFO must match. 9 Output Valid.This bit is 1 if the output side TDM entry is valid. 8:4 Input Port.The input port from which the output expects to receive a TDM cell. 3:0 Egress Subport Fanout. Specifies which linecard(s) will receive the TDM cell. Bits Description 31:23 Reserved. 22 Input Valid.This bit is 1 if the input side TDM entry is valid. 21:20 Input Subport.(OC-48 only) Input linecard which provides the TDM cell to send. 19:10 Input Tag.The tag that the head of the input TDM tag FIFO must match. 9 Output Valid.This bit is 1 if the output side TDM entry is valid. 8:4 Input Port.The input port from which the output expects to receive a TDM cell. 3:0 Egress Subport Fanout. Specifies which linecard(s) will receive the TDM cell.

198 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc.

3.4.2.57 ITIB Memory

Symbol: EITIBM Address Offset: 04000-07FFCh Default Value: 000000000h Access: Read/Write ITIB memory. 3.4.2.58 Linecard Request Count Memory Symbol: ESRCTM Address Offset: 08000-0FFFCh Default Value: 000000000h Access: Read/Write This memory is addressed by {source_subport, QID}. Each traffic type has a different address and data format. For Unicast Traffic:Each request count is 10 bits wide if OC-192c (9:0), 8 bits wide if OC-48(7:0) Address Data Bits Description 31:0 ITIB Memory. Each entry is a multicast port fanout value, addressed by a12-bit multicast tag. Note that an all-zero fanout is considered invalid. Bits Description 19:15 Must be set to 0x1 14:13 Source Subport 12:11 Must be set to 0x2 for Unicast Traffic type 10:9 Priority 8:4 Destination Port 3:2 Destination Subport 1:0 Must be set to 0 Bits Description 31:10 Reserved. 9:0 Request Count

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 199 PMC-Sierra, Inc. For Multicast traffic:Each request count is 10 bits wide if OC-192c, 8 bits wide if OC-48. Each request count corresponds to a FIFO in Linecard Multicast Tag FIFO Memory; if the request count is changed by OOB, the FIFO length also changes. OOB should specify a new head address for the FIFO in the upper data bits. OOB should also write the correct number of tags into the Linecard Multicast Tag FIFO memory so that the state of the request counter and its tag FIFO are consistent Address Data For TDM traffic:Each request count is 10 bits wide if OC-192c, 8 bits wide if OC-48. Each request count corresponds to a FIFO in Linecard TDM Tag FIFO Memory; if the request count is changed by OOB, the FIFO length also changes. OOB should specify a new head address for the FIFO in the upper data bits. OOB should also write the correct number of tags into the Linecard TDM Tag FIFO memory so that the state of the request counter and its tag FIFO are consistent. Address Bits Description 19:15 Must be set to 0x1 14:13 Source Subport 12:11 Must be set to 0x3 for Multicast Traffic type 10:9 Priority 8:0 Must be set to 0 Bits Description 31:20 Reserved. 19:10 Linecard Multicast Tag Head 9:0 Request Count Bits Description 19:15 Must be set to 0x1 14:13 Source Subport 12:9 Must be set to 0x7 for TDM Traffic type 8:4 Must be set to 0 3:2 Destination Subport 1:0 Must be set to 0

200 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc. Data For Control Packet:Each request count is 3 bits wide. Each request count corresponds to a 7-deep, 1-bit-wide FIFO. Whenever OOB changes a request count value, it should also supply any tags associated with requests in the upper data bits. The LSB of the “Control Packet Tag FIFO” field is the head of the tag FIFO, i.e. the next tag that will be sent with a grant. Address Data 3.4.2.59 Input Queue Information Memory Symbol: EIQIM Address Offset: 16000-17FFCh Default Value: 000000000h Access: Read/Write This memory is addressed by QID. This memory contains input queue information for 4 TDM, 4 multicast, 128 unicast, and 4 control packet queues. For Unicast Traffic:128-512 queues depending on Egress OC48 Mode Map register; each OC-192 queue holds 0-64 cells, each OC-48 queue holds 0-16 cells. Bits Description 31:20 Reserved. 19:10 Linecard TDM Tag Head 9:0 Request Count Bits Description 19:15 Must be set to 0x1 14:13 Source Subport 12:9 Must be set to 0x0 for Control Packet Traffic type 8:4 Must be set to 0 3:2 Destination Subport 1:0 Must be set to 0 Bits Description 31:10 Reserved. 9:3 Control Packet Tag Head 2:0 Request Count

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 201 PMC-Sierra, Inc. Address Data For Multicast traffic:4 queues regardless of OC-48 modes; each queue holds 0-96 cells. Address Data Bits Description 19:13 Must be set to 0xB 12:11 Must be set to 0x2 for Unicast Traffic type 10:9 Priority 8:4 Destination Port 3:2 Destination Subport 1:0 Must be set to 0 Bits Description 31:13 Reserved. 12:7 Head 6:0 Length Bits Description 19:13 Must be set to 0xB 12:11 Must be set to 0x3 for Multicast Traffic type 10:9 Priority 8:0 Must be set to 0 Bits Description 31:15 Reserved.

14 Full

13:7 Head 6:0 Tail

202 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc. For TDM traffic:1 or 4 queues depending on this EPP’s OC-48 mode. If OC-192, up to 96 cells; if OC-48, up to 24 cells each queue. Address Data For Control Packet:1 or 4 queues depending on this EPP’s OC-48 mode. 8 cells per queue regardless of mode Address Data Bits Description 19:13 Must be set to 0xB 12:9 Must be set to 0x7 for TDM Traffic type 8:4 Must be set to 0 3:2 Source Subport 1:0 Must be set to 0 Bits Description 31:15 Reserved. 13:7 Head 6:0 Tail Bits Description 19:13 Must be set to 0xB 12:9 Must be set to 0x0 for Control Packet Traffic type 8:4 Must be set to 0 3:2 Source Subport 1:0 Must be set to 0 Bits Description 31:7 Reserved. 6:4 Head 3:0 Length

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 203 PMC-Sierra, Inc.

3.4.2.60 Outstanding Scheduler Request Count Memory

Symbol: EOSRM Address Offset: 1A000-1BFFCh Default Value: Unknown Access: Read/Write The Outstanding Scheduler Request Count Memory contains the number of cells in a VOQ that the EPP must request a grant from the Scheduler. This memory is addressed by QID, but only Unicast and Multicast addresses are valid. For Unicast traffic:128-512 queues depending on Egress OC-48c Mode Map register; maximum 64 requests for OC-192c, max 16 requests for OC-48. Address Data For Multicast traffic:4 queues regardless of OC-48 modes; max 96 requests. Address Bits Description 19:13 Must be set to 0xD 12:11 Must be set to 0x2 for Unicast traffic type 10:9 Priority 8:4 Destination Port 3:2 Destination Subport 1:0 Must be set to 0 Bits Description 31:7 Reserved. 6:0 Requests Bits Description 19:13 Must be set to 0xD 12:11 Must be set to 0x3 for Multicast traffic type 10:9 Priority 8:0 Must be set to 0

204 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc. Data

3.4.2.61 Waiting Sched ulerReq uest Count Memory

Symbol: EWSRCT Address Offset: 1C000-1DFFCh Default Value: Unknown Access: Read/Write The Waiting Scheduler request Count Memory contains the number of requests that the EPP is waiting to get a grant back from the Scheduler. This memory is addressed by QID, but only Unicast and Multicast addresses are valid. After resetting an EPP, write 00000000h to address offsets 1D000h through 1D7FCh in that EPP (a total of 512 OOB writes). Whan all EPPs are reset simultaneously (using a broadcast OOB write to the EPP Reset and Control register), 512 broadcast OOB writes can be used to reset those locations in all EPPs. For Unicast traffic:128-512 queues depending on Egress OC-48c Mode Map register; maximum 64 requests for OC-192c, max 16 requests for OC-48. Address Data Bits Description 31:7 Reserved. 6:0 Requests Bits Description 19:13 Must be set to 0xE 12:11 Must be set to 0x2 for Unicast traffic type 10:9 Priority 8:4 Destination Port 3:2 Destination Subport 1:0 Must be set to 0 Bits Description 31:7 Reserved. 6:0 Requests

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 205 PMC-Sierra, Inc. For Multicast traffic:4 queues regardless of OC-48 modes; max 96 requests. Address Data Bits Description 19:13 Must be set to 0xE 12:11 Must be set to 0x3 for Multicast traffic type 10:9 Priority 8:0 Must be set to 0 Bits Description 31:7 Reserved. 6:0 Requests

206 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc.

3.4.2.62 Output UC Queue Debit Count Memory

Symbol: EOUCQDCT Address Offset: 1E000-1E7FCh Default Value: 000000000h Access: Read/Write This memory contains debits for UC only. Address Data

3.4.2.63 Output UC Queue Information Memory

Symbol: EOUCQI Address Offset: 20000-21FFCh Default Value: 000000000h Access: Read/Write This memory is addressed by QID, but only UC qid addresses are valid. 128-512 queues depending on Egress OC-48c Mode Map register; each OC-192c queue holds 0-64 cells, each OC-48 queue holds 0-16 cells. Bits Description 19:13 Must be set to 0xF 12:11 Must be set to 0x0 for Unicast traffic type 10:9 Priority 8:4 Destination Port 3:2 Destination Subport 1:0 Must be set to 0 Bits Description 31:7 Reserved. 6:0 Debits.128-512 queues depending on Egress OC-48c Mode Map register; maximum 64 debits for OC-192c, max 16 debits for OC-48.

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 207 PMC-Sierra, Inc. Address Data

3.4.2.64 Output MC Queue Drop Counters

Symbol: EOMCQDCT Address Offset: 3E000-3FFFCh Default Value: 000000000h Access: Read/Write This set of registers is addressed by QID, but only MC addresses are valid. Each register contains the number of MC cells dropped by the output EPP for this QID. Note that this value will stick at 0xFFFFFFFF. There is only one register for each MC output queue (total of 4 registers, regardless of OC-48 mode). Address Bits Description 19:13 Must be set to 0x10 12:11 Must be set to 0x2 for Unicast traffic type 10:9 Priority 8:4 Destination Port 3:2 Destination Subport 1:0 Must be set to 0 Bits Description 31:13 Reserved. 12:7 Head 6:0 Length Bits Description 19:13 Must be set to 0x1F 12:11 Must be set to 0x3 for Multicast traffic type 10:9 Priority 8:2 Must be set to 0 1:0 Must be set to 0

208 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC PRELIMINARY Data Sheet PMC-Sierra, Inc. Data

3.4.2.65 OTIB Memory

Symbol: EOTIBM Address Offset: 08000-FFFFCh Default Value: 000000000h Access: Read/Write OTIB memory. Each entry is a multicast egress subport fanout value, addressed by {multicast_tag, source_port}. Note that if the egress port is OC-48, an all-0 fanout will cause the cell to be dropped. If the egress port is OC-192c, this memory is not used. Bits Description 31:0 Drop Count. Bits Description 31:4 Reserved. 3:0 Fanout

3.5 ENHANCED PORT PROCESSOR SIGNAL DESCRIPTIONS

This section describes the Enhanced Port Processor signals. PECL PECL are Pseudo-ECL (positive voltage ECL) compatible signals. that may be 1 to 15 meters apart. Table 25. Enhanced Port Processor Signal Descriptions

Table 25. Enhanced Port Processor Signal Descriptions (Continued)

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 213 PMC-Sierra, Inc.

3.6 PINOUT AND PACKAGE INFORMATION

3.6.1 Pinout Tables

Table 26. Enhanced Port Processor Pinout (left side)

Table 27. Enhanced Port Processor Pinout (center)

Table 28. Enhanced Port Processor Pinout (right side)

Table 35. Enhanced Port Processor Alpha Pin List

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. 218 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE lssd_scan_out16 G01 lssd_scan_out17 E01 lssd_scan_out18 C01 lssd_scan_out19 E12 lssd_scan_out2 AC25 lssd_scan_out3 AC23 lssd_scan_out4 AC21 lssd_scan_out5 AC19 lssd_scan_out6 AA14 lssd_scan_out7 W12 lssd_scan_out8 AE07 lssd_scan_out9 AE05 No Pin A01 oob_ad0 G21 oob_ad1 K17 oob_ad2 G22 oob_ad3 F23 oob_ad4 G20 oob_ad5 G19 oob_ad6 D23 oob_ad7 C24 oob_clk K25 oob_devsel0 M15 oob_devsel1 L24 oob_devsel2 L25 oob_devsel3 L17 oob_int_hi M25 oob_int_lo M23 oob_valid_L L16 oob_wait_L J25 p2d_d0_ic0 V17 p2d_d0_ic1 U16 p2d_d0_ic2 AD21 p2d_d0_ic3 AA19 p2d_d0_ic4 AE22 p2d_d0_ic5 AA20 p2d_d0_ic6 W18 p2d_d0_ic7 Y19 p2d_d0_id0 AC08 p2d_d0_id1 AB09 p2d_d0_id2 AE11 p2d_d0_id3 AD11 p2d_d0_id4 W10 p2d_d0_id5 U11 p2d_d0_id6 R12 p2d_d0_id7 AC09 p2d_d0_oc0 V01 p2d_d0_oc1 U02 p2d_d0_oc2 W04 p2d_d0_oc3 Y03 p2d_d0_oc4 V07 p2d_d0_oc5 U08 p2d_d0_oc6 AA02 p2d_d0_oc7 AB01 p2d_d1_ic0 AE20 p2d_d1_ic1 R17 p2d_d1_ic2 W17 p2d_d1_ic3 AA18 p2d_d1_ic4 AE18 p2d_d1_ic5 AD17 p2d_d1_ic6 AB19 Signal Name Pin p2d_d1_ic7 AC20 p2d_d1_id0 W09 p2d_d1_id1 U09 p2d_d1_id2 AE06 p2d_d1_id3 AD07 p2d_d1_id4 AE10 p2d_d1_id5 AE09 p2d_d1_id6 AA09 p2d_d1_id7 Y09 p2d_d1_oc0 U07 p2d_d1_oc1 V05 p2d_d1_oc2 U01 p2d_d1_oc3 T01 p2d_d1_oc4 W02 p2d_d1_oc5 Y01 p2d_d1_oc6 R10 p2d_d1_oc7 W05 p2d_d2_ic0 AD15 p2d_d2_ic1 AE15 p2d_d2_ic2 AB17 p2d_d2_ic3 AC18 p2d_d2_ic4 Y17 p2d_d2_ic5 AA17 p2d_d2_ic6 AE17 p2d_d2_ic7 AE16 p2d_d2_oc0 U03 p2d_d2_oc1 P11 p2d_d2_oc2 U06 p2d_d2_oc3 T07 p2d_d2_oc4 R02 p2d_d2_oc5 R01 p2d_d2_oc6 U04 p2d_d2_oc7 V03 p2d_d3_ic0 V15 p2d_d3_ic1 AA16 p2d_d3_ic2 AC14 p2d_d3_ic3 AE14 p2d_d3_ic4 AC16 p2d_d3_ic5 AC17 p2d_d3_ic6 R14 p2d_d3_ic7 U15 p2d_d3_oc0 N01 p2d_d3_oc1 R03 p2d_d3_oc2 R04 p2d_d3_oc3 P09 p2d_d3_oc4 U05 p2d_d3_oc5 R08 p2d_d3_oc6 P03 p2d_d3_oc7 P01 p2d_d4_ic0 AA13 p2d_d4_ic1 T13 p2d_d4_ic2 W15 p2d_d4_ic3 U14 p2d_d4_ic4 AC13 p2d_d4_ic5 AD13 p2d_d4_ic6 AC15 p2d_d4_ic7 AB15 p2d_d4_oc0 L05 p2d_d4_oc1 L06 p2d_d4_oc2 R06 p2d_d4_oc3 R05 Signal Name Pin p2d_d4_oc4 N07 p2d_d4_oc5 N06 p2d_d4_oc6 T05 p2d_d4_oc7 R07 p2d_d5_ic0 U12 p2d_d5_ic1 W11 p2d_d5_ic2 Y13 p2d_d5_ic3 W13 p2d_d5_ic4 AA11 p2d_d5_ic5 Y11 p2d_d5_ic6 Y15 p2d_d5_ic7 AA15 p2d_d5_oc0 N10 p2d_d5_oc1 L04 p2d_d5_oc2 L03 p2d_d5_oc3 N02 p2d_d5_oc4 N03 p2d_d5_oc5 L07 p2d_d5_oc6 K05 p2d_d5_oc7 N05 p2d_d6_ic0 AE12 p2d_d6_ic1 AC12 p2d_d6_ic2 AA10 p2d_d6_ic3 V11 p2d_d6_ic4 U13 p2d_d6_ic5 AB11 p2d_d6_ic6 AC11 p2d_d6_ic7 AE13 p2d_d6_oc0 K07 p2d_d6_oc1 L09 p2d_d6_oc2 N09 p2d_d6_oc3 J03 p2d_d6_oc4 K03 p2d_d6_oc5 M01 p2d_d6_oc6 M03 p2d_d6_oc7 L08 p2f_a0_c H17 p2f_a0_cn J16 p2f_a0_e G18 p2f_a0_en F19 p2f_a0_o B23 p2f_a0_on A24 p2f_a1_c H21 p2f_a1_cn J19 p2f_a1_e H23 p2f_a1_en J22 p2f_a1_o K19 p2f_a1_on J20 p2f_b0_c L21 p2f_b0_cn L20 p2f_b0_e R20 p2f_b0_en R21 p2f_b0_o T21 p2f_b0_on R19 p2f_b1_c U22 p2f_b1_cn V23 p2f_b1_e U19 p2f_b1_en V21 p2f_b1_o W24 p2f_b1_on Y25 p2s_s0_c B13 Signal Name Pin

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 219 p2s_s0_cn C13 p2s_s0_e E11 p2s_s0_en F11 p2s_s0_o F15 p2s_s0_on E15 p2s_s1_c J15 p2s_s1_cn G16 p2s_s1_e D17 p2s_s1_en C18 p2s_s1_o A17 p2s_s1_on A16 plllock E23 plltest_in C25 plltest_out C23 pwrup_reset_in_L A22 ref_clk C21 ref_clkn A21 s2p_s0_c C09 s2p_s0_cn C10 s2p_s0_e E09 s2p_s0_en F09 s2p_s0_o C08 s2p_s0_on D09 s2p_s1_c C16 s2p_s1_cn C17 s2p_s1_e D13 s2p_s1_en A13 s2p_s1_o J13 s2p_s1_on H15 soc_in D21 soc_inn C22 soc_out E20 test_di1 E05 test_di2 A03 test_lt E13 test_re H13 test_ri E14 UNUSED A02 UNUSED A04 UNUSED A06 UNUSED A08 UNUSED A09 UNUSED A10 UNUSED A11 UNUSED A12 UNUSED A14 UNUSED A15 UNUSED A20 UNUSED A25 UNUSED B01 UNUSED B03 UNUSED B05 UNUSED B07 UNUSED B09 UNUSED B11 UNUSED B15 UNUSED B19 UNUSED B21 UNUSED B25 UNUSED C02 UNUSED C04 Signal Name Pin UNUSED C06 UNUSED C11 UNUSED C12 UNUSED C14 UNUSED C15 UNUSED D01 UNUSED D03 UNUSED D05 UNUSED D07 UNUSED D11 UNUSED D15 UNUSED D25 UNUSED E02 UNUSED E04 UNUSED E06 UNUSED E07 UNUSED E08 UNUSED E10 UNUSED E16 UNUSED E17 UNUSED E18 UNUSED E19 UNUSED E22 UNUSED E24 UNUSED F01 UNUSED F05 UNUSED F07 UNUSED F13 UNUSED F17 UNUSED F21 UNUSED G05 UNUSED G06 UNUSED G07 UNUSED G08 UNUSED G09 UNUSED G10 UNUSED G11 UNUSED G13 UNUSED G15 UNUSED H09 UNUSED H11 UNUSED J05 UNUSED J08 UNUSED J11 UNUSED J12 UNUSED J14 UNUSED J21 UNUSED K11 UNUSED K13 UNUSED K15 UNUSED L02 UNUSED L12 UNUSED L14 UNUSED L18 UNUSED M17 UNUSED N04 UNUSED N16 UNUSED N17 UNUSED N18 UNUSED N19 UNUSED N20 Signal Name Pin UNUSED N21 UNUSED N22 UNUSED N23 UNUSED N24 UNUSED N25 UNUSED P15 UNUSED P17 UNUSED P19 UNUSED P23 UNUSED P25 UNUSED R18 UNUSED R24 UNUSED R25 UNUSED T03 UNUSED T09 UNUSED T15 UNUSED T25 UNUSED U21 UNUSED U24 UNUSED U25 UNUSED V19 UNUSED V25 UNUSED W16 UNUSED W19 UNUSED W20 UNUSED W21 UNUSED W22 UNUSED Y21 UNUSED Y23 UNUSED AA08 UNUSED AA22 UNUSED AA24 UNUSED AB13 UNUSED AB21 UNUSED AB23 UNUSED AB25 UNUSED AC10 UNUSED AC22 UNUSED AC24 UNUSED AD19 UNUSED AD23 UNUSED AD25 UNUSED AE24 UNUSED AE25 VDD B04 VDD B14 VDD B22 VDD D10 VDD D16 VDD F04 VDD F08 VDD F18 VDD F22 VDD H02 VDD H24 VDD K12 VDD K14 VDD L11 VDD L15 VDD M02 VDD M06 Signal Name Pin

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc. 220 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE VDD M10 VDD M13 VDD M16 VDD M20 VDD M24 VDD N12 VDD N14 VDD P06 VDD P10 VDD P13 VDD P16 VDD P20 VDD R11 VDD R15 VDD T12 VDD T14 VDD V02 VDD V24 VDD Y04 VDD Y08 VDD Y18 VDD Y22 VDD AB10 VDD AB16 VDD AD04 VDD AD14 VDD AD22 VDDA E25 VDDQ D24 VDDQ H20 VDDQ K18 VDDQ K22 VDDQ P24 VDDQ T18 VDDQ T22 VDDQ V20 VDDQ AB24 VDDQ V10 VDDQ V16 VDDQ Y12 VDDQ Y14 VDDQ AB06 VDDQ AB20 VDDQ AD08 VDDQ AD12 VDDQ AD18 VDDQ D02 VDDQ H06 VDDQ K04 VDDQ K08 VDDQ P02 VDDQ T04 VDDQ T08 VDDQ V06 VDDQ AB02 VDDQ B08 VDDQ B12 VDDQ B18 VDDQ D06 VDDQ D20 VDDQ F12 Signal Name Pin VDDQ F14 VDDQ H10 VDDQ H16 Vref R16 Vref V13 Vref N08 Vref L10 Vref T17 Vref U18 Vref U17 Vref T11 Vref R09 Vref M11 Vref J10 Vref J09 Signal Name Pin

3.6.2 Package Dimensions

package is a 624 ceramic ball grid array (CBGA). NOTE: Drawings are not to scale. Figure 60. Enhanced Port Processor CBGA Package Dimensions - Top and Side Views

Figure 61. Enhanced Port Processor (EPP) CBGA Package Dimensions - Bottom View Table 36. Enhanced Port Processor CBGA Mechanical Specifications

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 223 PMC-Sierra, Inc.

3.6.3 Part Number

The PMC-Sierra Part Number for the Enhanced Port Processor is: Enhanced Port Processor PM9315-HC 06K1581 PQ PMC Logo Source “B” Part Number Terminal A01 Identifier Enhanced Port Processor BXXLLLLLLLL Device Name Revision Date Code Lot Number Country of Rev B BYYWW XXXXXXXXXXXX PM9315-HC PMC Part Number Origin P M931 5-HC-* P =prototype, Blank =production part Temp. range (C = Commercial 0-70 degrees C) Package code (U = CCGA, H = CBGA)Sequential part number Chipset family code, 1=TT1, 5 =TTX IC IRD prefix PMC prefix (1=SCH, 2=XBAR, 3=DS, 5=EPP)

224 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC Released Data Sheet PMC-Sierra, Inc.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 225 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

4 Crossbar

This chapter contains information on the Crossbar device, part number PM9312-UC, available from PMC-Sierra, Inc. The Crossbar device is a multicast-capable, 32-port, reverse-routed Crossbar. Each Crossbar is connected to the matching logical Dataslice in every port. For example, XBAR0 connects to DS0. Each Crossbar switches an eight byte cell each frame time. The device sets its crosspoints according to the reverse routing tags supplied by the Scheduler, via the EPPs and Dataslices. If the routing tag supplied is non-valid, the frame containing cell data being sent to this DS had a CRC error, or the frame containing the routing tag from this DS had a CRC error, then the VLD bit isnots e to nt h ef r a m et ot h eD S .

4.1 CROSSBAR BLOCKS

Figure 62 shows the basic data flow through the Crossbar. The routing tag sent in the AIB frame from the Dataslice is registered in the routing tag register (RTR). The RTR selects one of the 32 incoming byte-wide busses and the output of the 32:1 byte-wide mux is driven to the transmitter side of the asymmetric serial link dumb end. The total delay from d2x_p[31:0] to x2d_p[31:0] is four cell times.

Figure 62. The Basic Dataflow Through the Crossbar

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 227 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

4.1.1 OOB Interface and Control/Status Registers

All of the devices have an OOB interface. This interface allows a single local CPU to control and monitor all of the devices within a core fabric. Internally, each device provides registers that can be mapped into the CPU’s address space. These registers are described in more detail in Section 4.4 “Crossbar Registers”.

4.2 M ODES OF OPERATION

The default mode of operation is for the Crossbar to act as a 32-port switch. In this mode, each ETT1 port has one connection to each of the Crossbars. If an 8- or 16-port switch is required, the Crossbar can be configured so that fewer Crossbar devices are required. In these modes, the term superport represents a system port. In 8-port mode, the Crossbar considers four of its ports to be a single superport. For example, ports 0, 1, 2, and 3 make up superport 0, ports 28, 29, 30, and 31 are superport 7. A single superport is connected to only three Crossbars. For example, superport 0 has its first 4 Dataslices connected to ports 0, 1, 2, and 3 of Crossbar 0, its next 4 Dataslices connected to ports 0, 1, 2, and 3 of Crossbar 1, and its last 4 Dataslices connected to ports 0, 1, 2, and 3 of Crossbar 2. In 16-port mode, the Crossbar considers two of its ports to be a single superport. Ports 0 & 1 make up superport 0, and ports 30 & 31 make up superport 15 for example. A single superport is connected to six Crossbars. In this case, superport 0 has its first 2 Dataslices connected to ports 0 and 1 of Crossbar 0, and its last Dataslices are connected to ports 0 and 1 of Crossbar 5. The mode is set via the OOB bus. The Crossbar acts as if the superports make up a ‘trunk’, so that either two or four of the inputs are routed as a single port.

4.3 OOB ACCESS

The Crossbars can be written and read via the OOB bus. In addition to the device select addresses 0x7FF (all devices) and 0x7B (all Crossbars), each Crossbar can be addressed with an individual device select specified by {000100, CCCC, 000, C} where CCCCC is the Crossbar number in the range 0-31 (these correspond to device selects 0x200 to 0x2F1, not including all addresses within that range). The Registers section of this chapter illustrates the usage of the subaddress field to specify registers within the Crossbar.

4.4 CROSSBAR REGISTERS

4.4.1 Summary

Descriptions section for more information on individual registers. Table 37. Crossbar Register Summary

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 229 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

4.4.2 Crossbar Register Descriptions

Read and Clear means that reading the register causes it to be cleared (reset to zero). All bits labeled as Reserved should be set to 0.

4.4.2.1 Status

Symbol: XSTS Address Offset: 00000h Default Value: 40000000h Access: Read Only The top 8 bits are device_id [3:0] and revision [3:0]. OOB interrupt bits are [1:0] (high, low), all other bits are reserved.

4.4.2.2 Control/Reset

Symbol: XMODERS Address Offset: 00004h Default Value: 00000004h Access: Read/Write This register configures the number of ports supported by this Crossbar. It also resets the entire device, equivalent to a hardware reset. Bits Description 31:28 Device ID Number.Identifies the specific device. 27:24 Device Revision Number. 23:2 Reserved. 1 High Priority Interrupt.1 = There is an outstanding high priority interrupt. One of the bits in the General Interrupt Register is set, and is enabled via its corresponding high priority mask. 0 Low Priority Interrupt.1 = There is an outstanding low priority interrupt. One of the bits in the General Interrupt Register is set, and is enabled via its corresponding low priority mask. Bits Description 31-3 Reserved.

230 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

4.4.2.3 Low Priority Mask

Symbol: XIRLMSK Address Offset: 00008h Default Value: 00000000h Access: Read/Write Interrupt Mask for interrupts.

4.4.2.4 High Priority Mask

Symbol: XIRHMSK Address Offset: 0000Ch Default Value: 00000000h Access: Read/Write Interrupt Mask for interrupts. 2:1 Port Mode.These 2 bits select the number of ports the Crossbar is supporting. A zero value (00) corresponds to 8 ports, a one value (01) corresponds to 16 ports, and a two value (10) corresponds to 32 ports (the default). The number of slices per port are then given by 32/num_ ports_selected. 10 = 32 ports x 1 slice 01 = 16 ports x 2 slices 00 = 8 ports x 4 slices Reset. Writing a 1 to this location will reset the entire device. It is equivalent to a hardware reset. This register is cleared automatically after the device has been reset. Writing a 0 is not necessary. Soft reset takes 1mS to complete. Bits Description 31:3 Reserved. 2:0 Low Priority Mask.Mask bits for low priority interrupts. Each mask bit is set to 1 to enable a low priority interrupt when the corresponding bit in the General Interrupt Register is 1. Bits Description 31:3 Reserved. 2:0 High Priority Mask.Mask bits for high priority interrupts. Each mask bit is set to 1 to enable a high priority interrupt when the corresponding bit in the General Interrupt Register is 1. Bits Description (Continued)

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 231 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

4.4.2.5 General Interrupt Register

Symbol: XIR Address Offset: 00010h Default Value: 00000000h Access: Read only Interrupt Register.

4.4.2.6 CRC Error Interrupts Mask

Symbol: XCRCMSK Address Offset: 00014h Default Value: 00000000h Access: Read/Write Interrupt Mask for per-port CRC errors Bits Description 31:3 Reserved. Ready Active.This is the logical OR of the Ready Active Interrupt register, after it has been masked. This is set if a serial link goes from inactive to active and the corresponding mask bit is 1. This bit is not cleared when read. You must clear the Ready Active Interrupt register. Ready Inactive.This is the logical OR of the Ready Inactive Interrupt register, after it has been masked. This is set if a serial link goes from active to inactive and the corresponding mask bit is 1. This bit is not cleared when read. You must clear the Ready Inactive Interrupt register. CRC Error.This is the logical OR of the CRC register, after it has been masked. This is set if a CRC error occurs and the corresponding mask bit is 1. This bit is not cleared when read. You must clear the CRC Error Interrupts register. Bits Description 31:0 CRC Error Interrupts Mask.Each bit is used to mask (enable) interrupts when the corresponding bit in the CRC Error Interrupts register is set to 1. The interrupt is enabled when the bit is 1.

232 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

4.4.2.7 CRC Error Interrupts

Symbol: XCRC Address Offset: 00018h Default Value: 00000000h Access: Read and Clear Interrupt Register for per-port CRC errors

4.4.2.8 RDY Inactive Interrupts Mask

Symbol: XRDYDMSK Address Offset: 0001Ch Default Value: 00000000h Access: Read/Write Interrupt Mask for per-port RDY down signals. Bits Description 31:0 CRC Error Interrupts.Each bit indicates if a CRC error has occurred on the appropriate link (DS to XBAR) since the last time this register was read. Mask off unwanted (unused ports) bits via the CRC Error Interrupt Mask. Bits Description 31:0 RDY Inactive Interrupts Mask.Each bit is used to mask (enable) interrupts when the corresponding bit in the Link Ready Inactive register is set to 1. The interrupt is enabled when the bit is 1.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 233 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

4.4.2.9 Link RDY Inactive Interrupts

Symbol: XRDYDN Address Offset: 00020h Default Value: 00000000h Access: Read and Clear Interrupt Register for per-port RDY down signals. NOTE: The link may be up again (active) by the time the CPU reads this register. Mask off unwanted (unused ports) bits via the Link Ready Inactive Interrupts Mask. Reading this register clears all bits.

4.4.2.10 Link RDY Active Interrupts Mask

Symbol: XRDYUMSK Address Offset: 00024h Default Value: 00000000h Access: Read/Write Interrupt Mask for per-port RDY up signals. Bits Description 31:0 Link RDY Inactive Interrupts.Each bit indicates if the Ready signal from a link has transitioned from Active to Inactive (the link has gone down for some reason). Bits Description 31:0 Link RDY Active Interrupts Mask.Each bit is used to mask (enable) interrupts when the corresponding bit in the Link Ready Active register is set to 1. The interrupt is enabled when the bit is 1.

234 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

4.4.2.11 Link RDY Active Interrupts

Symbol: XRDYUP Address Offset: 00028h Default Value: 00000000h Access: Read and Clear Interrupt Register for per-port RDY up signals. NOTE: The link may be down again (inactive) by the time the CPU reads this register. Mask off unwanted (unused ports) bits via the Link Ready Active Interrupt Mask. This register is cleared to 0 when read.

4.4.2.12 AIB Reset

Symbol: XAIBRS Address Offset: 00030h Default Value: FFFFFFFFh Access: Read/Write Resets AIB link for each port. Bits Description 31:0 Link RDY Active Interrupts.Each bit indicates if the Ready signal from a link has transitioned from Inactive to Active (the link has come up). Bits Description 31:0 AIB Reset.Each bit is used to assert reset to the AIB link for each port. Reset is asserted when the bit is 1. If reset, the link will not operate or transition to ready. This register is set to FFFFFFFFh on power-up reset or if the reset bit in the control register is asserted.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 235 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

4.4.2.13 AIB Ready

Symbol: XAIBRDY Address Offset: 00034h Default Value: 00000000h Access: Read Only Indicates the status of the AIB link.

4.4.2.14 AIB Tx Enable

Symbol: XAIBEN Address Offset: 0003Ch Default Value: 00000000h Access: Read/Write Enables the transmitter of the corresponding AIB link. Bits Description 31:0 AIB Ready. Each bit indicates if the corresponding AIB link is ready (1 = active) or not (0 = inactive). Bits Description 31:0 AIB Tx Enable.If the corresponding port is not physically present in the system then this bit should be set to zero to reduce unwanted electrical noise and to minimize unwanted effects when the port board is inserted.

236 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

4.4.2.15 PLL Control/Status

Symbol: XPLL Address Offset: 00100h Default Value: 0001447Ch Access: Read/Write Controls operation of the internal PLL (Phase locked loop). After a power on reset the PLL itself is held in reset. This is reflected in bit 16 of this register. The local CPU must reset this bit to 0 to enable operation of the device, and thus should write 0000447Ch to this register. Bit Description 31:17 PLL status.These bits reflect internal PLL operation status and should be ignored. Reset PLL.When set to 1 the PLL is held reset. The supplied reference clock will be used as the internal clock. The serial links will not be operational. This bit will be 1 after power-up reset and should be deasserted for normal operation. PLL reset takes 10mS to complete. 15:0 PLL control.

4.5 CROSSBAR SIGNAL DESCRIPTIONS

This section describes the Crossbar signals. PECL PECL are Pseudo-ECL (positive voltage ECL) compatible signals. that may be 1 to 15 meters apart. Table 38. Crossbar Signal Descriptions

Table 38. Crossbar Signal Descriptions (Continued)

reset. All outputs are tristated when low. reset. All outputs are tristated when low.

4.6 PINOUT AND PACKAGE INFORMATION

4.6.1 Pinout Tables

Table 39. Crossbar Pinout (left side)

Table 40. Crossbar Pinout (center)

Table 41. Crossbar Pinout (right side)

Table 33. Crossbar Alpha Pin List

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 244 d2x_p8_cn K29 d2x_p8_e J28 d2x_p8_en L25 d2x_p8_o G32 d2x_p8_on F33 d2x_p9_c L29 d2x_p9_cn L30 d2x_p9_e L27 d2x_p9_en M25 d2x_p9_o L32 d2x_p9_on K33 GND B02 GND B06 GND B10 GND B14 GND B20 GND B24 GND B28 GND B32 GND D04 GND D08 GND D12 GND D16 GND D18 GND D22 GND D26 GND D30 GND F02 GND F06 GND F10 GND F14 GND F20 GND F24 GND F28 GND F32 GND H04 GND H08 GND H12 GND H16 GND H18 GND H22 GND H26 GND H30 GND K02 GND K06 GND K10 GND K14 GND K20 GND K24 GND K28 GND K32 GND M04 GND M08 GND M12 GND M16 GND M18 GND M22 GND M26 GND M30 GND P02 GND P06 Signal Name Pin GND P10 GND P14 GND P17 GND P20 GND P24 GND P28 GND P32 GND R15 GND R19 GND T04 GND T08 GND T12 GND T16 GND T18 GND T22 GND T26 GND T30 GND U14 GND U17 GND U20 GND V04 GND V08 GND V12 GND V16 GND V18 GND V22 GND V26 GND V30 GND W15 GND W19 GND Y02 GND Y06 GND Y10 GND Y14 GND Y17 GND Y20 GND Y24 GND Y28 GND Y32 GND AB04 GND AB08 GND AB12 GND AB16 GND AB18 GND AB22 GND AB26 GND AB30 GND AD02 GND AD06 GND AD10 GND AD14 GND AD20 GND AD24 GND AD28 GND AD32 GND AF04 GND AF08 GND AF12 GND AF16 GND AF18 GND AF22 Signal Name Pin GND AF26 GND AF30 GND AH02 GND AH06 GND AH10 GND AH14 GND AH20 GND AH24 GND AH28 GND AH32 GND AK04 GND AK08 GND AK12 GND AK16 GND AK18 GND AK22 GND AK26 GND AK30 GND AM02 GND AM06 GND AM10 GND AM14 GND AM20 GND AM24 GND AM28 GND AM32 jtag_tck E31 jtag_tdi C33 jtag_tdo A31 jtag_tms C31 jtag_trst_L G31 lssd_ce1_a AG31 lssd_ce1_b AJ31 lssd_ce1_c1 AJ29 lssd_ce1_c2 AL33 lssd_scan_in0 AN31 lssd_scan_in1 AL31 lssd_scan_in10 AL07 lssd_scan_in11 AN07 lssd_scan_in12 AL05 lssd_scan_in13 AN05 lssd_scan_in14 AL03 lssd_scan_in15 AN03 lssd_scan_in2 AN29 lssd_scan_in3 AL29 lssd_scan_in4 AN27 lssd_scan_in5 AL27 lssd_scan_in6 AE18 lssd_scan_in7 AG18 lssd_scan_in8 AG16 lssd_scan_in9 AE16 lssd_scan_out0 V09 lssd_scan_out1 V07 lssd_scan_out10 G16 lssd_scan_out11 G18 lssd_scan_out12 J18 lssd_scan_out13 C27 lssd_scan_out14 A27 lssd_scan_out15 C29 lssd_scan_out2 T07 lssd_scan_out3 T09 Signal Name Pin

245 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. lssd_scan_out4 A29 lssd_scan_out5 A05 lssd_scan_out6 C05 lssd_scan_out7 A07 lssd_scan_out8 C07 lssd_scan_out9 J16 M_ackreg_L J24 M_pending_L H25 M_xreset_L L22 No Pin A01 oob_ad0 K19 oob_ad1 L18 oob_ad2 F19 oob_ad3 G19 oob_ad4 G15 oob_ad5 F15 oob_ad6 L16 oob_ad7 K15 oob_clk J10 oob_devsel0 N14 oob_devsel1 M13 oob_devsel2 L12 oob_devsel3 H09 oob_int_hi K13 oob_int_lo L14 oob_valid_L E08 oob_wait_L M15 plllock C03 plltest_in E03 plltest_out G03 pwruprst_L D07 ref_clk E05 ref_clkn C01 soc_in E07 soc_inn C06 soc_out G08 test_di1 V27 test_di2 A03 test_lt U27 test_ri V25 UNUSED A02 UNUSED A09 UNUSED A11 UNUSED A13 UNUSED A15 UNUSED A16 UNUSED A17 UNUSED A18 UNUSED A19 UNUSED A21 UNUSED A23 UNUSED A25 UNUSED A32 UNUSED A33 UNUSED B01 UNUSED B03 UNUSED B17 UNUSED B31 UNUSED B33 UNUSED C02 UNUSED C04 Signal Name Pin UNUSED C16 UNUSED C17 UNUSED C18 UNUSED C28 UNUSED C30 UNUSED C32 UNUSED D03 UNUSED D05 UNUSED D17 UNUSED D27 UNUSED D29 UNUSED D31 UNUSED E04 UNUSED E06 UNUSED E15 UNUSED E16 UNUSED E17 UNUSED E18 UNUSED E19 UNUSED E26 UNUSED E27 UNUSED E28 UNUSED E29 UNUSED E30 UNUSED E33 UNUSED F03 UNUSED F05 UNUSED F07 UNUSED F17 UNUSED F27 UNUSED F29 UNUSED F31 UNUSED G01 UNUSED G04 UNUSED G05 UNUSED G06 UNUSED G07 UNUSED G17 UNUSED G26 UNUSED G27 UNUSED G28 UNUSED G29 UNUSED G30 UNUSED G33 UNUSED H05 UNUSED H07 UNUSED H17 UNUSED H27 UNUSED H29 UNUSED J01 UNUSED J08 UNUSED J09 UNUSED J17 UNUSED J25 UNUSED J26 UNUSED J33 UNUSED K09 UNUSED K17 UNUSED K21 UNUSED K25 UNUSED L01 Signal Name Pin UNUSED L11 UNUSED L13 UNUSED L15 UNUSED L17 UNUSED L19 UNUSED L20 UNUSED L21 UNUSED L23 UNUSED L33 UNUSED M11 UNUSED M17 UNUSED M19 UNUSED M21 UNUSED M23 UNUSED N01 UNUSED N10 UNUSED N11 UNUSED N12 UNUSED N15 UNUSED N16 UNUSED N17 UNUSED N18 UNUSED N19 UNUSED N20 UNUSED N22 UNUSED N23 UNUSED N24 UNUSED N33 UNUSED P11 UNUSED P13 UNUSED P15 UNUSED P19 UNUSED P21 UNUSED P23 UNUSED R01 UNUSED R05 UNUSED R06 UNUSED R07 UNUSED R10 UNUSED R11 UNUSED R12 UNUSED R13 UNUSED R14 UNUSED R16 UNUSED R17 UNUSED R18 UNUSED R20 UNUSED R21 UNUSED R22 UNUSED R23 UNUSED R24 UNUSED R27 UNUSED R28 UNUSED R29 UNUSED R33 UNUSED T01 UNUSED T03 UNUSED T05 UNUSED T11 UNUSED T13 UNUSED T15 Signal Name Pin

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 246 UNUSED T19 UNUSED T21 UNUSED T23 UNUSED T29 UNUSED T31 UNUSED T33 UNUSED U01 UNUSED U02 UNUSED U03 UNUSED U04 UNUSED U05 UNUSED U06 UNUSED U07 UNUSED U08 UNUSED U09 UNUSED U10 UNUSED U11 UNUSED U12 UNUSED U13 UNUSED U15 UNUSED U19 UNUSED U21 UNUSED U22 UNUSED U23 UNUSED U24 UNUSED U25 UNUSED U26 UNUSED U28 UNUSED U29 UNUSED U30 UNUSED U31 UNUSED U32 UNUSED U33 UNUSED V01 UNUSED V03 UNUSED V05 UNUSED V11 UNUSED V13 UNUSED V15 UNUSED V19 UNUSED V21 UNUSED V23 UNUSED V29 UNUSED V31 UNUSED V33 UNUSED W01 UNUSED W05 UNUSED W06 UNUSED W07 UNUSED W10 UNUSED W11 UNUSED W12 UNUSED W13 UNUSED W14 UNUSED W16 UNUSED W17 UNUSED W18 UNUSED W20 UNUSED W21 UNUSED W22 UNUSED W23 Signal Name Pin UNUSED W24 UNUSED W27 UNUSED W28 UNUSED W29 UNUSED W33 UNUSED Y11 UNUSED Y13 UNUSED Y15 UNUSED Y19 UNUSED Y21 UNUSED Y23 UNUSED AA01 UNUSED AA10 UNUSED AA11 UNUSED AA12 UNUSED AA14 UNUSED AA15 UNUSED AA16 UNUSED AA17 UNUSED AA18 UNUSED AA19 UNUSED AA20 UNUSED AA22 UNUSED AA23 UNUSED AA24 UNUSED AA33 UNUSED AB11 UNUSED AB13 UNUSED AB15 UNUSED AB17 UNUSED AB19 UNUSED AB21 UNUSED AB23 UNUSED AC01 UNUSED AC11 UNUSED AC12 UNUSED AC13 UNUSED AC14 UNUSED AC15 UNUSED AC16 UNUSED AC17 UNUSED AC18 UNUSED AC19 UNUSED AC20 UNUSED AC21 UNUSED AC22 UNUSED AC23 UNUSED AC33 UNUSED AD09 UNUSED AD13 UNUSED AD15 UNUSED AD17 UNUSED AD19 UNUSED AD21 UNUSED AD25 UNUSED AE01 UNUSED AE08 UNUSED AE09 UNUSED AE10 UNUSED AE17 UNUSED AE24 Signal Name Pin UNUSED AE25 UNUSED AE26 UNUSED AE33 UNUSED AF05 UNUSED AF07 UNUSED AF09 UNUSED AF17 UNUSED AF25 UNUSED AF27 UNUSED AF29 UNUSED AG01 UNUSED AG03 UNUSED AG04 UNUSED AG05 UNUSED AG06 UNUSED AG07 UNUSED AG08 UNUSED AG15 UNUSED AG17 UNUSED AG19 UNUSED AG26 UNUSED AG27 UNUSED AG28 UNUSED AG29 UNUSED AG30 UNUSED AG33 UNUSED AH03 UNUSED AH05 UNUSED AH07 UNUSED AH15 UNUSED AH17 UNUSED AH19 UNUSED AH27 UNUSED AH29 UNUSED AH31 UNUSED AJ01 UNUSED AJ03 UNUSED AJ04 UNUSED AJ05 UNUSED AJ06 UNUSED AJ07 UNUSED AJ08 UNUSED AJ15 UNUSED AJ16 UNUSED AJ17 UNUSED AJ18 UNUSED AJ19 UNUSED AJ26 UNUSED AJ27 UNUSED AJ28 UNUSED AJ30 UNUSED AJ33 UNUSED AK03 UNUSED AK05 UNUSED AK07 UNUSED AK17 UNUSED AK27 UNUSED AK29 UNUSED AK31 UNUSED AL01 UNUSED AL02 Signal Name Pin

247 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. UNUSED AL04 UNUSED AL06 UNUSED AL16 UNUSED AL17 UNUSED AL18 UNUSED AL28 UNUSED AL30 UNUSED AL32 UNUSED AM01 UNUSED AM03 UNUSED AM17 UNUSED AM31 UNUSED AM33 UNUSED AN01 UNUSED AN02 UNUSED AN09 UNUSED AN11 UNUSED AN13 UNUSED AN15 UNUSED AN16 UNUSED AN17 UNUSED AN18 UNUSED AN19 UNUSED AN21 UNUSED AN23 UNUSED AN25 UNUSED AN32 UNUSED AN33 VDD B04 VDD B12 VDD B18 VDD B26 VDD D10 VDD D20 VDD D28 VDD D32 VDD F04 VDD F08 VDD F16 VDD F22 VDD H02 VDD H14 VDD H24 VDD H28 VDD K08 VDD K12 VDD K18 VDD K30 VDD M06 VDD M24 VDD M32 VDD N13 VDD N21 VDD P04 VDD P16 VDD P18 VDD P26 VDD T02 VDD T10 VDD T14 VDD T17 Signal Name Pin VDD T20 VDD T28 VDD U16 VDD U18 VDD V06 VDD V14 VDD V17 VDD V20 VDD V24 VDD V32 VDD Y08 VDD Y16 VDD Y18 VDD Y30 VDD AA13 VDD AA21 VDD AB02 VDD AB10 VDD AB28 VDD AD04 VDD AD16 VDD AD22 VDD AD26 VDD AF06 VDD AF10 VDD AF20 VDD AF32 VDD AH12 VDD AH18 VDD AH26 VDD AH30 VDD AK02 VDD AK06 VDD AK14 VDD AK24 VDD AM08 VDD AM16 VDD AM22 VDD AM30 VDD F30 VDD H32 VDD K26 VDD M28 VDD P22 VDD P30 VDD T24 VDD T32 VDD V28 VDD Y22 VDD Y26 VDD AB24 VDD AB32 VDD AD30 VDD AF28 VDD AK32 VDD AB14 VDD AB20 VDD AD12 VDD AD18 VDD AF14 VDD AF24 Signal Name Pin VDD AH08 VDD AH16 VDD AH22 VDD AK10 VDD AK20 VDD AK28 VDD AM04 VDD AM12 VDD AM18 VDD AM26 VDD D02 VDD H06 VDD K04 VDD M02 VDD M10 VDD P08 VDD P12 VDD T06 VDD V02 VDD V10 VDD Y04 VDD Y12 VDD AB06 VDD AD08 VDD AF02 VDD AH04 VDD B08 VDD B16 VDD B22 VDD B30 VDD D06 VDD D14 VDD D24 VDD F12 VDD F18 VDD F26 VDD H10 VDD H20 VDD K16 VDD K22 VDD M14 VDD M20 VDDA E01 x2d_p0_c G09 x2d_p0_cn K11 x2d_p0_e B05 x2d_p0_en A04 x2d_p0_o E09 x2d_p0_on C08 x2d_p10_c N25 x2d_p10_cn L28 x2d_p10_e N32 x2d_p10_en M33 x2d_p10_o L31 x2d_p10_on M29 x2d_p11_c P29 x2d_p11_cn P31 x2d_p11_e P25 x2d_p11_en N27 x2d_p11_o N30 x2d_p11_on N31 Signal Name Pin

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 248 x2d_p12_c AA28 x2d_p12_cn Y27 x2d_p12_e W31 x2d_p12_en W30 x2d_p12_o W25 x2d_p12_on W26 x2d_p13_c AA29 x2d_p13_cn AB31 x2d_p13_e Y33 x2d_p13_en W32 x2d_p13_o AB27 x2d_p13_on AA26 x2d_p14_c AC30 x2d_p14_cn AC29 x2d_p14_e AD33 x2d_p14_en AC32 x2d_p14_o AB25 x2d_p14_on AC27 x2d_p15_c AD29 x2d_p15_cn AE30 x2d_p15_e AH33 x2d_p15_en AG32 x2d_p15_o AC25 x2d_p15_on AE28 x2d_p16_c AK25 x2d_p16_cn AJ24 x2d_p16_e AM27 x2d_p16_en AN28 x2d_p16_o AH25 x2d_p16_on AE23 x2d_p17_c AJ23 x2d_p17_cn AK23 x2d_p17_e AM23 x2d_p17_en AN24 x2d_p17_o AG23 x2d_p17_on AE22 x2d_p18_c AL22 x2d_p18_cn AJ21 x2d_p18_e AM19 x2d_p18_en AN20 x2d_p18_o AF21 x2d_p18_on AG22 x2d_p19_c AG20 x2d_p19_cn AH21 x2d_p19_e AK19 x2d_p19_en AL19 x2d_p19_o AF19 x2d_p19_on AE19 x2d_p1_c H11 x2d_p1_cn G10 x2d_p1_e B09 x2d_p1_en A08 x2d_p1_o C09 x2d_p1_on C10 x2d_p20_c AL14 x2d_p20_cn AJ14 x2d_p20_e AG13 x2d_p20_en AE14 x2d_p20_o AL13 x2d_p20_on AK13 x2d_p21_c AH11 Signal Name Pin x2d_p21_cn AE13 x2d_p21_e AN12 x2d_p21_en AM13 x2d_p21_o AJ12 x2d_p21_on AL11 x2d_p22_c AG10 x2d_p22_cn AF11 x2d_p22_e AN08 x2d_p22_en AM09 x2d_p22_o AL10 x2d_p22_on AL09 x2d_p23_c AD11 x2d_p23_cn AG09 x2d_p23_e AN04 x2d_p23_en AM05 x2d_p23_o AL08 x2d_p23_on AJ09 x2d_p24_c AE04 x2d_p24_cn AD05 x2d_p24_e AG02 x2d_p24_en AH01 x2d_p24_o AE06 x2d_p24_on AC09 x2d_p25_c AC05 x2d_p25_cn AC04 x2d_p25_e AC02 x2d_p25_en AD01 x2d_p25_o AC07 x2d_p25_on AB09 x2d_p26_c AB03 x2d_p26_cn AA05 x2d_p26_e W02 x2d_p26_en Y01 x2d_p26_o AA08 x2d_p26_on AB07 x2d_p27_c Y07 x2d_p27_cn AA06 x2d_p27_e W04 x2d_p27_en W03 x2d_p27_o W08 x2d_p27_on W09 x2d_p28_c P03 x2d_p28_cn P05 x2d_p28_e N07 x2d_p28_en P09 x2d_p28_o N03 x2d_p28_on N04 x2d_p29_c L06 x2d_p29_cn N09 x2d_p29_e M01 x2d_p29_en N02 x2d_p29_o M05 x2d_p29_on L03 x2d_p2_c J13 x2d_p2_cn F11 x2d_p2_e B13 x2d_p2_en A12 x2d_p2_o C11 x2d_p2_on E12 x2d_p30_c K07 x2d_p30_cn L08 Signal Name Pin x2d_p30_e H01 x2d_p30_en J02 x2d_p30_o K03 x2d_p30_on J03 x2d_p31_c L10 x2d_p31_cn J07 x2d_p31_e D01 x2d_p31_en E02 x2d_p31_o H03 x2d_p31_on J05 x2d_p3_c E14 x2d_p3_cn C14 x2d_p3_e J14 x2d_p3_en G13 x2d_p3_o D13 x2d_p3_on C13 x2d_p4_c F21 x2d_p4_cn G20 x2d_p4_e C19 x2d_p4_en D19 x2d_p4_o J19 x2d_p4_on H19 x2d_p5_c E21 x2d_p5_cn C22 x2d_p5_e A20 x2d_p5_en B19 x2d_p5_o G22 x2d_p5_on H21 x2d_p6_c D23 x2d_p6_cn E23 x2d_p6_e A24 x2d_p6_en B23 x2d_p6_o J22 x2d_p6_on G23 x2d_p7_c E24 x2d_p7_cn D25 x2d_p7_e A28 x2d_p7_en B27 x2d_p7_o J23 x2d_p7_on F25 x2d_p8_c J27 x2d_p8_cn L24 x2d_p8_e E32 x2d_p8_en D33 x2d_p8_o J29 x2d_p8_on H31 x2d_p9_c L26 x2d_p9_cn K27 x2d_p9_e J32 x2d_p9_en H33 x2d_p9_o J31 x2d_p9_on K31 Signal Name Pin

4.6.2 Package Dimensions

NOTE: Drawings are not to scale. Figure 63. Crossbar CCGA Package Dimensions - Top and Side Views

Figure 64. Crossbar CCGA package Dimensions - Bottom View Table 34. Crossbar CCGA Mechanical Specifications

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 251 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

4.6.3 Part Number

The PMC-Sierra Part Number for the Crossbar is: Crossbar PM9312-UC 09K0372 PQ PMC Logo Source “B” Part Number Terminal A01 Identifier Crossbar BXXLLLLLLLL Device Name Revision Date Code Lot Number Country of Rev A BYYWW XXXXXXXXXXXX PM9312-UC PMC Part Number Origin PM 9 3 12 - U C Temp. range (C = Commercial 0-70 degrees C) Package code (U = CCGA, H = CBGA)Sequential part number Chipset family code, 1=TT1, 5 =TTX IC IRD prefix PMC prefix (1=SCH, 2=XBAR, 3=DS, 5=EPP)

252 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 253 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

5 Scheduler

This chapter contains information on the Scheduler device, part number PM9311-UC, available from PMC-Sierra, Inc. The Scheduler communicates directly with all of the fabric ports. Its purpose is to arbitrate the flow of cells through the fabric from the input ports, through the fabric, to the required output ports. The Scheduler has information about all cells that are waiting in queues at the input ports. The output ports tell the Scheduler when they can accept new cells from the input ports. At every cell time, the Scheduler uses all of this information to arbitrate between all of the inputs to establish the set of cells that can be transferred through the crossbar fabric to the output queues. The effectiveness of the Scheduler at performing this arbitration determines the throughput of the fabric under any given cell load. The Scheduler must have accurate, up to date information in order to arbitrate efficiently and fairly. Consequently, the EPPs send new state information to the Scheduler at every cell time.

5.1 BLOCK STRUCTURE

The block diagram of the Scheduler is shown in Figure 65 on page 255.

5.1.1 Port Block

The Scheduler can arbitrate between as many as 32 ports of OC-192c or 128 ports of OC-48c. The state associated with each port is maintained in a port block. Each port block contains a serial link, a port control unit, and some queue information memory. The serial link provides a full duplex, 1.6 Gbit/s, communications channel to the appropriate EPP . This channel is used to send new request and backpressure information to the Scheduler, and to send grant and routing tag information to the EPP. The port control unit provides a set of registers that indicate the current status of many elements of the port block, and that can also be used to control the behaviour of the port block. It is described further in Section 5.2 “Port State”. The port block also has some local memory which contains the number of outstanding requests for every queue associated with this port.

5.1.2 Arbiter Block

The arbiter receives information from each of the port blocks. This information indicates which of the virtual output queues in each port have cells waiting to be forwarded to their appropriate destination port. At every cell time, the arbiter combines these inputs to decide which ports can forward a cell at some fixed time in the future. The resulting set of grants is then passed back to the port blocks.

254 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. The arbiter has a number of control registers which are used primarily for diagnostics. Except for initial configuration after reset, the arbiter does not require any intervention from the OOB CPU.

5.1.3 OOB Interface and Control/Status Registers

All of the devices have an OOB interface. This interface allows a single local CPU to control and monitor all of the devices within a core fabric. Internally, each device provides registers that can be mapped into the CPU’s address space. These registers are described in more detail in Section 5.4 “Scheduler Registers”.

5.1.4 TDM Service

At every cell time the Scheduler may receive a TDM request from each port that intends to transfer a TDM cell (according to the TDM tables in the Port Processor). If a TDM request is received then the Scheduler will return a TDM Grant to that port, and that port will not be used in the arbitration of the best-effort traffic for that cell time. The Scheduler may also receive notification that a port is expecting to receive a TDM cell. This notification identifies the port that will source the TDM cell. If the Scheduler receives a TDM request from the designated source port then the destination port will receive an appropriate routing tag. However, if the source port has not made a TDM request then the destination port will be eligible to receive a best-effort cell. The Scheduler is the point of synchronization for the TDM service provided by the core fabric. The TDM control block generates synchronization signals at regular intervals. The period of these signals is determined either by internal registers, or by signals received from an EPP. The Scheduler sends the synchronization pulses to all 32 EPPs at the same time to indicate the start of a new TDM frame. See Section 1.2.5 “TDM Service” on page 22 for more information on the TDM service.

Figure 65. Scheduler Block Diagram

5.2 PORT STATE

is added to the system, it can be tested offline without affecting existing traffic flows. determines the type of cells it can, and cannot, forward through the fabric. Figure 66. Port State Machine After reset, the port is in thedisabledstate. No cells will flow to or from this port when it is disabled. arbiter and passes those grants to the EPP.

traffic and non-TDM traffic are enabled independently. prevent other ports from sending to this port. Table 35 shows whether non-TDM traffic can flow, based on the port state. blocking in the case where one port is sending a multicast cell to a port which is disabled.

5.3 FAULT TOLERANCE

dual scheduler system, otherwise it is not set. NOTE: The Enable Port register will not reflect this change. Table 35. Non-TDM Traffic

258 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.  The appropriate register (CRC or Link Ready Inactive) is updated. This port asserts backpressure to all other ports. Once the CPU determines that this port can continue operation, then a refresh operation must be performed on this port and then the port can be enabled. If the CRC_Action bit is set, then in addition to the above actions, the Scheduler also asserts reset to all of its serial links to all EPPs. This turns off all of the links, forcing all of the EPPs to switch to the secondary Scheduler. All of the primary Scheduler’s ports are set to the disabled state and the enable_port register is set to 0. The Scheduler must be reset by the CPU and have its queue information refreshed before it is re-enabled.

5.4 SCHEDULER REGISTERS

5.4.1 Summary

Descriptions section for more information on individual registers. Table 36. Scheduler Register Summary

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 261 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

5.4.2 Scheduler Register Descriptions

Read and Clear means that reading the register causes it to be cleared (reset to zero). All bits labled as Reserved should be set to 0.

5.4.2.1 Status

Symbol: SSTS Address Offset: 00000h Default Value: 30000000h Access: Read Only Status register.

5.4.2.2 Control and Reset

Symbol: SCTRLRS Address Offset: 00004h Default Value: 00000130h Access: Read/Write Control and Reset register. Bits Description 31:28 Chip ID Number.Identifies the specific device. 27:24 Chip Revision Number. 23:2 Reserved. 1 High Priority Interrupt.1 = an outstanding high priority interrupt. One of the bits in the Interrupt Register is set, and is enabled via its corresponding high priority mask. 0 Low Priority Interrupt.1 = an outstanding low priority interrupt. One of the bits in the Interrupt Register is set, and is enabled via its corresponding low priority mask. Bits Description 31-9 Reserved. 8-4 BP_FIFO. Specifies the depth of the backpressure FIFO. This is an internal FIFO and users should set these bits to a value of 14h after reset. The default value is 13h 3 Reserved.

262 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

5.4.2.3 Low Priority Mask

Symbol: SIRLMSK Address Offset: 00008h Default Value: 00000000h Access: Read/Write Interrupt Mask for interrupts.

5.4.2.4 High Priority Mask

Symbol: SIRHMSK Address Offset: 0000Ch Default Value: 00000000h Access: Read/Write Interrupt Mask for interrupts. CRC Action. This specifies what action is to be taken when an enabled port (port enable register bit is 1) encounters a CRC error from its attached Port Processor. If set to 0 then the port with the error is shut down and the Scheduler continues to arbitrate among the remaining ports. If set to 1 then the Scheduler will reset all serial links, effectively disabling all 32 ports. This bit should only be set to 1 for the primary Scheduler within a core with redundant Schedulers. Subport mode . If set to 1 then the Scheduler operates in subport mode, in which each port is considered to be four subports with only one priority. This bit is not used in the ETT1 Scheduler a n dm u s tb es e tt o0 . Reset. Writing a 1 to this location will reset the entire device. It is equivalent to a hardware reset. This register is cleared automatically when the chip is reset. This bit will always read as 0 and it is not necessary to write a 0 having just written a 1. Soft reset takes 1mS to complete. Bits Description 31:5 Reserved. 4:0 Low Priority Mask. Mask bits for low priority interrupts. Each mask bit is set to 1 to enable a low priority interrupt when the corresponding bit in the Status Register is 1. Bits Description 31:5 Reserved. Bits Description (Continued)

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 263 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

5.4.2.5 Interrupt Register

Symbol: SIR Address Offset: 00010h Default Value: 00000000h Access: Read and Clear Interrupt Register.

5.4.2.6 CRC Error Interrupt Mask

Symbol: SCRCMSK Address Offset: 00014h Default Value: 00000000h Access: Read/Write 4:0 High Priority Mask. Mask bits for high priority interrupts. Each mask bit is set to 1 to enable a high priority interrupt when the corresponding bit in the Status Register is 1. Bits Description 31:5 Reserved. All Ports Refreshed.During a refresh operation, this bit will be set when all enabled ports have been refreshed. See also Refreshed Status register. This bit is cleared on reading this register. TDM Sync Lost. This bit is set if the TDM feature has been configured, and no Sync signal is received for three consecutive sync periods. This would suggest that a failure has occurred in either the Port Processor or its attached linecard. This bit is cleared on reading this register. Ready Inactive.This is the logical OR of the Link Ready Inactive Interrupt register, after it has been masked by the Link Ready Inactive Interrupt Mask register. This bit is set if a serial link goes from active to inactive and the corresponding mask bit is 1. This bit is not cleared when read - the user must clear the Link Ready Inactive Interrupt register. Ready Active.This is the logical OR of the Ready Active interrupt register, after it has been masked. This is set if a serial link goes from inactive to active and the corresponding mask bit is 1. This bit is not cleared when read - the user must clear the Ready Active interrupt register. CRC Error.This is the logical OR of the CRC interrupt register, after it has been masked. This is set if a CRC error occurs and the corresponding mask bit is 1. This bit is not cleared when read - you must clear the CRC register. Bits Description

264 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. Interrupt mask for CRC errors.

5.4.2.7 CRC Errors

Symbol: SCRC Address Offset: 00018h Default Value: 00000000h Access: Read and Clear CRC Interrupt Error Register

5.4.2.8 Link Ready Inactive Interrupt Mask

Symbol: SRDYDMSK Address Offset: 0001Ch Default Value: 00000000h Access: Read/Write Enables interrupts when the corresponding bit in the Link Ready Inactive register is set to 1. Bits Description 31:0 CRC Error Interrupt Mask .Each bit is used to mask (enable) interrupts when the corresponding bit in the CRC Error register is set to 1. The interrupt is enabled when the bit is 1. Bits Description 31:0 CRC Errors.Each bit indicates if a CRC error has occurred on the appropriate link (Port Processor to Scheduler) since the last time this register was read. Mask off unwanted (unused ports) bits via the CRC Error Interrupt Mask. Bits Description 31:0 Link Ready Inactive Interrupt Mask.Each bit is used to mask (enable) interrupts when the corresponding bit in the Link Ready Inactive register is set to 1. The interrupt is enabled when the bit is 1.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 265 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

5.4.2.9 Link Ready Inactive

Symbol: SRDYDN Address Offset: 00020h Default Value: 00000000h Access: Read and Clear Each bit indicates if the Ready signal from a link has transitioned from Active to Inactive (the link has gone down for some reason). NOTE: The link may be up again (active) by the time the CPU reads this register. Mask off unwanted (unused ports) bits via the Link Ready Inactive Interrupt Mask. Reading this register will clear all bits.

5.4.2.10 Link Ready Active Interrupt Mask

Symbol: SRDYUMSK Address Offset: 00024h Default Value: 00000000h Access: Read/Write Enables interrupts when the corresponding bit in the Link Ready Active register is set to 1. Bits Description 31:0 Link Ready Inactive.Each bit indicates if the Ready signal from a link has transitioned from Active to Inactive. Bits Description 31:0 Link Ready Active Interrupt Mask.Each bit is used to mask (enable) interrupts when the corresponding bit in the Link Ready Active register is set to 1. The interrupt is enabled when the bit is 1.

266 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

5.4.2.11 Link Ready Active

Symbol: SRDYUP Address Offset: 00028h Default Value: 00000000h Access: Read and Clear Each bit indicates if the Ready signal from a link has transitioned from Inactive to Active (the link has come up). NOTE: The link may be down again (inactive) by the time the CPU reads this register. Mask off unwanted (unused ports) bits via the Link Ready Active Interrupt Mask. Reading this register will clear all bits.

5.4.2.12 AIB Reset

Symbol: SAIBRS Address Offset: 00030h Default Value: FFFFFFFFh Access: Read/Write Used to assert reset to the AIB link for each port. Bits Description 31:0 Link Ready Active.Each bit indicates if the Ready signal from a link has transitioned from Inactive to Active. Bits Description 31:0 AIB Reset.Each bit is used to assert reset to the AIB link for each port. Reset is asserted when the bit is 1. If reset the link will not operate or transition to ready. This register is set to 0xFFFF_FFFF on power-up reset or if the reset bit in the Control Register is asserted.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 267 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

5.4.2.13 AIB Ready

Symbol: SAIBRDY Address Offset: 00034h Default Value: 00000000h Access: Read Only Indicates if the corresponding AIB link is ready.

5.4.2.14 Enable Port

Symbol: SENBPRT Address Offset: 00038h Default Value: 00000000h Access: Read/Write This register is used to enable a scheduler port to carry traffic. After reset this register is 0 - all scheduler ports are disabled. While a scheduler port is disabled it will not respond to requests from the attached port card, nor will it issue grants. NOTE: Reading this register only reflects the last value written to it. A CRC error, for example, might cause a Scheduler port to become inactive, but that change will not be reflected in this register. Similarly, if a port is active, then setting the port to inactive by writing to the Reset Port register will not be reflected in this register even though the port will be inactive. Whenever a port is reset the appropriate bit in this register should also be cleared. Bits Description 31:0 AIB Ready. Each bit indicates if the corresponding AIB link is ready (1 = active) or not (0 = inactive). Bits Description 31:0 Enable Port.Each bit indicates if the corresponding port should be enabled.

268 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

5.4.2.15 AIB Tx Enable

Symbol: SPORTEN Address Offset: 0003Ch Default Value: 00000000h Access: Read/Write This is used to enable the transmitter of the corresponding AIB link.

5.4.2.16 Flow Control Crossbar Sync

Symbol: FCCSYN Address Offset: 00040h Default Value: 00000000h Access: Read/Write NOTE: This register is only used in conjunction with the Enhanced Port Processor. It is not used with the Port Processor. The counter is reloaded at reset or whenever a refresh_go occurs. Thus, the counter can be synchronized between two Scheduler devices. To force the counter to reload, set the Run/Stop bit to 0, write a new count value to the Synch_period, and then set the run/stop bit to 1. Dual Schedulers should always be refreshed (synchronized) after this register has been modified. Bits Description 31:0 AIB Tx Enable .If the corresponding port is not physically present in the system then the appropriate bit should be set to zero to reduce unwanted electrical noise and to minimize unwanted effects when the port board is inserted Bits Description 31:16 Synch_period . This value denotes the period at which OC48 synch pulses will be issued. Setting this to the recommended value of 32 means that the OC48_synch bit will be set to 1 for every one in 32 frames sent to the Port Processor. 15:1 Reserved. Run/Stop. This bit enables the OC-48 Synch counter. When 0 the counter does not run and is continuously reloaded with the value in the synch_period field. When this bit is set to 1 the counter counts down by one for each cell time. When the counter reaches the value one a synchronization pulse is issued in the scheduler to Port Processor frame on all ports (a bit is set in the frame).

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 269 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

5.4.2.17 Enable non-TDM Traffic

Symbol: SNTDMEN Address Offset: 00080h Default Value: 00000000h Access: Read/Write I ndicates if the corresponding port can send and receive non-TDM traffic.

5.4.2.18 Enable TDM Traffic

Symbol: STDMEN Address Offset: 00084h Default Value: 00000000h Access: Read/Write Indicates if the corresponding port can send and receive TDM traffic. Bits Description 31:0 Enable non-TDM Traffic.Each bit indicates if the corresponding port can send and receive non-TDM traffic (i.e. best-effort cells of priority 0,1,2 or 3). This bit should be de-asserted (set to 0) before telling the EPP to refresh the Scheduler’s state. Bits Description 31:0 Enable TDM Traffic.Each bit indicates if the corresponding port can send and receive TDM traffic (bit set to 1). Each bit should be 0 if the appropriate port does not send or receive TDM traffic.

270 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

5.4.2.19 Reset Port

Symbol: SPORTRS Address Offset: 00088h Default Value: 00000000h Access: Read/Write Resets each port.

5.4.2.20 Freeze Port

Symbol: SPORTFRZ Address Offset: 0008Ch Default Value: 00000000h Access: Read/Write Freezes Scheduler ports.

5.4.2.21 Port is Refreshing

Symbol: SRFRSNG Address Offset: 00090h Default Value: 00000000h Access: Read Only Bits Description 31:0 Reset Port.Each bit indicates if the corresponding port is held in reset (1). By asserting and then de-asserting this bit for a port, the internal state associated with that port is reset so that t h ep o r ti si nt h eDisabledstate (see Section 5.2 on page 256). This register should not be left asserted. Do not use this register to keep ports in the inactive state. Instead, use the SENBPRT register to enable or disable ports. Bits Description 31:1 Reserved. Freeze Port.This is used for diagnostic purposes only! Asserting this bit will cause all ports to halt operation and to send a freeze command to their attached Port Processor. De-asserting this bit will allow traffic flow to be resumed. Caution: Cell loss may occur when modifying this bit.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 271 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. Refreshing status. Bits Description 31:0 Port is Refreshing.During a refresh operation the port transitions fromactivetorefreshing when it receives a refresh cell from the Port Processor. This bit is 1 if the port is currently refreshing (has received a refresh cell) but has not yet completed the refresh operation.

272 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

5.4.2.22 Port is Refreshed

Symbol: SRFRS Address Offset: 00094h Default Value: 00000000h Access: Read Only Refreshed status.

5.4.2.23 TDM Status

Symbol: STDMSTS Address Offset: 00098h Default Value: 00000000h Access: Read Only Indicates TDM status.

5.4.2.24 TDM Control

Symbol: STDMCTRL Address Offset: 0009Ch Bits Description 31:0 Port is Refreshed.During a refresh operation the port transitions fromactivetorefreshing when it receives a refresh cell from the Port Processor. Once all refresh cells have been received the port is considered to be “refreshed”. This bit is 1 if the port is currently refreshed - i.e. it has been refreshed but has not yet restarted normal operation. The refreshed bit in the Interrupt Register is set when all enabled ports have this bit set to one, and at least one port is enabled. Bits Description 31:17 Reserved.

16 Indicates the TDM frame (0 or 1) that was received in the most recent cell from the current port

that is providing the suggested TDM sync. 15:11 Reserved. 10:0 The current period minus one at which TDM sync signals will be issued to the Port Processors. So if the current TDM Synch is being generated every 512 cell times then this field will be read as 511.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 273 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. Default Value: 00000000h Access: Read/Write TDM Control Register Bits Description Enable TDM Sync. Set this bit to 1 to enable TDM sync to be sent out to all the Port Processors. Default is 0 (TDM sync is off). This bit only controls the propagation of TDM Synch out of the Scheduler, and does not affect the internal TDM PLL. Load Initial Period.Set this bit to 1 and then 0 to cause the Scheduler to use the Initial Period field in this register as the current TDM sync period. (Only used if the Scheduler generates TDM Synch itself) 29:27 Reserved. 26:16 Initial Period.This can be used in conjunction with the Load Initial Period bit to set an initial TDM period. Only use this if there is not a currently valid port supplying TDM sync and TDM frame. (Only used if the Scheduler generates TDM Synch itself). This value is the initial period minus one at which TDM sync signals will be issued to the Enhanced Port Processors 15:11 Reserved. 10:8 FIFO Depth.Leave this value set to zero. 7 Transmitted TDM Frame (0 or 1). This is the value of the TDM Table select that is sent to the PPs when bit 6 is 0. (Only used if the Scheduler generates TDM Synch itself) C u r r e n tP o r tV a l i d .Indicates that the Current Port field is valid, i.e. there is a port that is currently sending correct TDM sync and frame information to the Scheduler. This bit is 0 if the Scheduler generates TDM Synch itself. 5 Reserved. 4:0 Current Port.Specifies the port which is currently sending suggested TDM sync signal and TDM frame (which is echoed back out to all Port Processors). This field is irrelevant unless bit 6 is set. If these bits are modified then bit 6 should be set to 0 and then back to 1.

274 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

5.4.2.25 PLL Control/Status

Symbol: SPLL Address Offset: 00100h Default Value: 0001447Ch Access: Read/Write Controls operation of the internal PLL (Phase locked loop). After a power on reset the PLL itself is held in reset. This is reflected in bit 16 of this register. The local CPU must reset this bit to 0 to enable operation of the device, and thus should write 0000447Ch to this register. Bit Description 31:17 PLL status.These bits reflect internal PLL operation status and should be ignored. Reset PLL.When set to 1 the PLL is held reset. The supplied reference clock will be used as the internal clock. The serial links will not be operational. This bit will be 1 after power-up reset and should be deasserted for normal operation. PLL reset takes 10mS to complete. 15:0 PLL control.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 275 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

5.4.3 Input Queue Memory (IQM)

The Scheduler has internal memory which stores the number of waiting cells at every input port. This memory can be read by the CPU. Caution: while the memory can also be modified, doing so may result in cell loss. CPU access to this memory is provided for diagnostic purposes only. After reset the memory is cleared to zero. Each Scheduler port (0:31) has four IQM units, one for each best-effort priority. Each IQM is organized as two independent memories: the first is 64 words of 32 bits and stores multicast fanout information. The second is 32 words of 7 bits and stores the number of waiting cells at each unicast virtual output queue. The individual entries in each IQM are addressed by the CPU as shown below: Port is the port number 0:31. Pri is the selected best effort priority 0:3. M/U is 1 for the multicast fanout memory and 0 for the unicast request count memory. Addr/QID is the multicast fanout address or a unicast VOQ identifier (only 0:31). So, for example, unicast VOQ 10 contains the number of requests (cells) that are waiting to go from this port to output port 10. The multicast fanout addresses are organized as a circular list. However the head of the list cannot be determined by the CPU except immediately after a refresh operation, at which time the second entry in the multicast fanout queue is at location zero. The first entry is held in a private register and is not visible to the CPU. Although the IQM should not be modified during normal operation, it is prudent to verify the integrity of the memory after the Scheduler is reset. 19 0315 7 11 Addr/QIDM/U0PriPort100 00

5.5 SCHEDULER SIGNAL DESCRIPTIONS

This section describes the Scheduler signals. PECL PECL are Pseudo-ECL (positive voltage ECL) compatible signals. that may be 1 to 15 meters apart. Table 37. Scheduler Signal Descriptions

Table 37. Scheduler Signal Descriptions (Continued)

5.6 PINOUT AND PACKAGE INFORMATION

5.6.1 Pinout Tables

Table 38. Scheduler Pinout (left side)

Table 39. Scheduler Pinout (center)

Table 40. Scheduler Pinout (right side)

Table 41. Scheduler Alpha Pin List

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 283 mon1 F07 mon10 N14 mon11 M13 mon12 M15 mon13 N15 mon14 P15 mon15 J13 mon2 E06 mon3 E07 mon4 A04 mon5 B05 mon6 H09 mon7 G08 mon8 L11 mon9 J09 No Pin A01 oob_ad0 B01 oob_ad1 C02 oob_ad2 D03 oob_ad3 E04 oob_ad4 G06 oob_ad5 H07 oob_ad6 F05 oob_ad7 G05 oob_clk J08 oob_devsel0 U09 oob_devsel1 P13 oob_devsel2 R12 oob_devsel3 R14 oob_int_hi K09 oob_int_lo E02 oob_valid_L N12 oob_wait_L D01 p2s_p0_c E08 p2s_p0_cn A06 p2s_p0_e G09 p2s_p0_en L12 p2s_p0_o A09 p2s_p0_on B09 p2s_p10_c L29 p2s_p10_cn L30 p2s_p10_e P23 p2s_p10_en N24 p2s_p10_o N31 p2s_p10_on M33 p2s_p11_c N29 p2s_p11_cn N30 p2s_p11_e N27 p2s_p11_en N26 p2s_p11_o R32 p2s_p11_on P33 p2s_p12_c AA25 p2s_p12_cn AB27 p2s_p12_e AA28 p2s_p12_en AB29 p2s_p12_o AA32 p2s_p12_on AA33 p2s_p13_c AB25 p2s_p13_cn AC27 p2s_p13_e AC28 p2s_p13_en AD29 Signal Name p2s_p13_o AC33 p2s_p13_on AB31 p2s_p14_c AE28 p2s_p14_cn AD27 p2s_p14_e AE29 p2s_p14_en AG32 p2s_p14_o AD31 p2s_p14_on AD33 p2s_p15_c AC24 p2s_p15_cn AB23 p2s_p15_e AG30 p2s_p15_en AH31 p2s_p15_o AF33 p2s_p15_on AE31 p2s_p16_c F01 p2s_p16_cn H05 p2s_p16_e J07 p2s_p16_en L09 p2s_p16_o J02 p2s_p16_on J01 p2s_p17_c J04 p2s_p17_cn H03 p2s_p17_e N11 p2s_p17_en L08 p2s_p17_o L03 p2s_p17_on L02 p2s_p18_c L04 p2s_p18_cn L05 p2s_p18_e N10 p2s_p18_en P11 p2s_p18_o M01 p2s_p18_on N03 p2s_p19_c N04 p2s_p19_cn N05 p2s_p19_e N08 p2s_p19_en N07 p2s_p19_o P01 p2s_p19_on R02 p2s_p1_c C08 p2s_p1_cn D09 p2s_p1_e L13 p2s_p1_en G10 p2s_p1_o B11 p2s_p1_on C11 p2s_p20_c AB07 p2s_p20_cn AA09 p2s_p20_e AB05 p2s_p20_en AA06 p2s_p20_o AA01 p2s_p20_on AA02 p2s_p21_c AC07 p2s_p21_cn AB09 p2s_p21_e AD05 p2s_p21_en AC06 p2s_p21_o AB03 p2s_p21_on AC01 p2s_p22_c AD07 p2s_p22_cn AE06 p2s_p22_e AE05 p2s_p22_en AG02 p2s_p22_o AD01 Signal Name p2s_p22_on AD03 p2s_p23_c AB11 p2s_p23_cn AC10 p2s_p23_e AH03 p2s_p23_en AG04 p2s_p23_o AE03 p2s_p23_on AF01 p2s_p24_c AJ08 p2s_p24_cn AK07 p2s_p24_e AC12 p2s_p24_en AG09 p2s_p24_o AM09 p2s_p24_on AN09 p2s_p25_c AL08 p2s_p25_cn AM07 p2s_p25_e AG10 p2s_p25_en AC13 p2s_p25_o AL11 p2s_p25_on AM11 p2s_p26_c AK11 p2s_p26_cn AJ11 p2s_p26_e AD13 p2s_p26_en AG11 p2s_p26_o AN12 p2s_p26_on AL13 p2s_p27_c AK13 p2s_p27_cn AJ13 p2s_p27_e AE13 p2s_p27_en AG12 p2s_p27_o AN14 p2s_p27_on AM15 p2s_p28_c AH23 p2s_p28_cn AC20 p2s_p28_e AJ22 p2s_p28_en AH21 p2s_p28_o AN21 p2s_p28_on AM21 p2s_p29_c AE22 p2s_p29_cn AF23 p2s_p29_e AK25 p2s_p29_en AJ24 p2s_p29_o AL22 p2s_p29_on AN23 p2s_p2_c E11 p2s_p2_cn D11 p2s_p2_e G11 p2s_p2_en K13 p2s_p2_o C13 p2s_p2_on A12 p2s_p30_c AH25 p2s_p30_cn AE23 p2s_p30_e AN28 p2s_p30_en AJ25 p2s_p30_o AN24 p2s_p30_on AL24 p2s_p31_c AD23 p2s_p31_cn AE24 p2s_p31_e AM29 p2s_p31_en AL28 p2s_p31_o AL25 p2s_p31_on AN26 Signal Name

284 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. p2s_p3_c B15 p2s_p3_cn A14 p2s_p3_e E14 p2s_p3_en C14 p2s_p3_o H13 p2s_p3_on G12 p2s_p4_c L20 p2s_p4_cn F23 p2s_p4_e F21 p2s_p4_en E22 p2s_p4_o B21 p2s_p4_on A21 p2s_p5_c H23 p2s_p5_cn J22 p2s_p5_e E24 p2s_p5_en D25 p2s_p5_o A23 p2s_p5_on C22 p2s_p6_c J23 p2s_p6_cn F25 p2s_p6_e E25 p2s_p6_en A28 p2s_p6_o C24 p2s_p6_on A24 p2s_p7_c J24 p2s_p7_cn K23 p2s_p7_e C28 p2s_p7_en B29 p2s_p7_o A26 p2s_p7_on C25 p2s_p8_c H29 p2s_p8_cn F33 p2s_p8_e L25 p2s_p8_en J27 p2s_p8_o J33 p2s_p8_on J32 p2s_p9_c H31 p2s_p9_cn J30 p2s_p9_e L26 p2s_p9_en N23 p2s_p9_o L32 p2s_p9_on L31 plllock E05 plltest_in G03 plltest_out E03 pwrup_reset_in_L A02 ref_clk A05 ref_clkn C05 s2p_p0_c K11 s2p_p0_cn J10 s2p_p0_e C09 s2p_p0_en A08 s2p_p0_o C06 s2p_p0_on D07 s2p_p10_c L27 s2p_p10_cn M25 s2p_p10_e M31 s2p_p10_en L33 s2p_p10_o K29 s2p_p10_on L28 s2p_p11_c M27 Signal Name s2p_p11_cn N25 s2p_p11_e N33 s2p_p11_en N32 s2p_p11_o M29 s2p_p11_on N28 s2p_p12_c AA30 s2p_p12_cn AA29 s2p_p12_e Y33 s2p_p12_en W32 s2p_p12_o AA26 s2p_p12_on AA27 s2p_p13_c AC30 s2p_p13_cn AC29 s2p_p13_e AB33 s2p_p13_en AA31 s2p_p13_o AA24 s2p_p13_on Y23 s2p_p14_c AE30 s2p_p14_cn AF31 s2p_p14_e AC31 s2p_p14_en AC32 s2p_p14_o AA23 s2p_p14_on AC26 s2p_p15_c AH33 s2p_p15_cn AF29 s2p_p15_e AE32 s2p_p15_en AE33 s2p_p15_o AE27 s2p_p15_on AC25 s2p_p16_c L10 s2p_p16_cn M11 s2p_p16_e H01 s2p_p16_en J03 s2p_p16_o G04 s2p_p16_on F03 s2p_p17_c J06 s2p_p17_cn K07 s2p_p17_e K03 s2p_p17_en K01 s2p_p17_o J05 s2p_p17_on G02 s2p_p18_c M09 s2p_p18_cn L07 s2p_p18_e L01 s2p_p18_en M03 s2p_p18_o L06 s2p_p18_on K05 s2p_p19_c N09 s2p_p19_cn M07 s2p_p19_e N02 s2p_p19_en N01 s2p_p19_o N06 s2p_p19_on M05 s2p_p1_c F09 s2p_p1_cn J11 s2p_p1_e A10 s2p_p1_en C10 s2p_p1_o B07 s2p_p1_on E09 s2p_p20_c AA05 s2p_p20_cn AA04 Signal Name s2p_p20_e W02 s2p_p20_en Y01 s2p_p20_o AA07 s2p_p20_on AA08 s2p_p21_c AC05 s2p_p21_cn AC04 s2p_p21_e AA03 s2p_p21_en AB01 s2p_p21_o Y11 s2p_p21_on AA10 s2p_p22_c AF03 s2p_p22_cn AE04 s2p_p22_e AC02 s2p_p22_en AC03 s2p_p22_o AC08 s2p_p22_on AA11 s2p_p23_c AF05 s2p_p23_cn AH01 s2p_p23_e AE01 s2p_p23_en AE02 s2p_p23_o AC09 s2p_p23_on AE07 s2p_p24_c AE10 s2p_p24_cn AD11 s2p_p24_e AN08 s2p_p24_en AL09 s2p_p24_o AL06 s2p_p24_on AM05 s2p_p25_c AE11 s2p_p25_cn AH09 s2p_p25_e AL10 s2p_p25_en AN10 s2p_p25_o AJ09 s2p_p25_on AN06 s2p_p26_c AF11 s2p_p26_cn AE12 s2p_p26_e AN11 s2p_p26_en AL12 s2p_p26_o AJ10 s2p_p26_on AK09 s2p_p27_c AC14 s2p_p27_cn AH11 s2p_p27_e AM13 s2p_p27_en AN13 s2p_p27_o AH13 s2p_p27_on AJ12 s2p_p28_c AJ21 s2p_p28_cn AK21 s2p_p28_e AM19 s2p_p28_en AN20 s2p_p28_o AG22 s2p_p28_on AE21 s2p_p29_c AJ23 s2p_p29_cn AK23 s2p_p29_e AL21 s2p_p29_en AN22 s2p_p29_o AG23 s2p_p29_on AD21 s2p_p2_c J12 s2p_p2_cn H11 s2p_p2_e C12 Signal Name

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 285 s2p_p2_en A11 s2p_p2_o E10 s2p_p2_on F11 s2p_p30_c AM27 s2p_p30_cn AL26 s2p_p30_e AM23 s2p_p30_en AL23 s2p_p30_o AC21 s2p_p30_on AG24 s2p_p31_c AK27 s2p_p31_cn AJ26 s2p_p31_e AN25 s2p_p31_en AM25 s2p_p31_o AG25 s2p_p31_on AC22 s2p_p3_c E13 s2p_p3_cn D13 s2p_p3_e A13 s2p_p3_en B13 s2p_p3_o E12 s2p_p3_on F13 s2p_p4_c D21 s2p_p4_cn E21 s2p_p4_e A20 s2p_p4_en B19 s2p_p4_o J21 s2p_p4_on G22 s2p_p5_c D23 s2p_p5_cn E23 s2p_p5_e A22 s2p_p5_en C21 s2p_p5_o K21 s2p_p5_on G23 s2p_p6_c C26 s2p_p6_cn B27 s2p_p6_e C23 s2p_p6_en B23 s2p_p6_o G24 s2p_p6_on L21 s2p_p7_c E26 s2p_p7_cn D27 s2p_p7_e B25 s2p_p7_en A25 s2p_p7_o L22 s2p_p7_on G25 s2p_p8_c M23 s2p_p8_cn L24 s2p_p8_e J31 s2p_p8_en H33 s2p_p8_o F31 s2p_p8_on G30 s2p_p9_c K27 s2p_p9_cn J28 s2p_p9_e K33 s2p_p9_en K31 s2p_p9_o J29 s2p_p9_on G32 soc_in C04 soc_inn D05 soc_out B03 test_di1 G18 Signal Name test_di2 A03 test_lt C27 test_ri J18 UNUSED A15 UNUSED A16 UNUSED A17 UNUSED A18 UNUSED A19 UNUSED A30 UNUSED A32 UNUSED A33 UNUSED B17 UNUSED B31 UNUSED B33 UNUSED C15 UNUSED C16 UNUSED C17 UNUSED C18 UNUSED C19 UNUSED C20 UNUSED C30 UNUSED C32 UNUSED D15 UNUSED D17 UNUSED D19 UNUSED D29 UNUSED D31 UNUSED D33 UNUSED E15 UNUSED E16 UNUSED E17 UNUSED E18 UNUSED E19 UNUSED E20 UNUSED E27 UNUSED E28 UNUSED E30 UNUSED E31 UNUSED E32 UNUSED E33 UNUSED F15 UNUSED F17 UNUSED F19 UNUSED F27 UNUSED F29 UNUSED G01 UNUSED G13 UNUSED G14 UNUSED G15 UNUSED G17 UNUSED G19 UNUSED G20 UNUSED G21 UNUSED G26 UNUSED G27 UNUSED G28 UNUSED G29 UNUSED G31 UNUSED G33 UNUSED H15 UNUSED H17 Signal Name UNUSED H19 UNUSED H21 UNUSED H25 UNUSED H27 UNUSED J14 UNUSED J15 UNUSED J17 UNUSED J19 UNUSED J20 UNUSED J25 UNUSED J26 UNUSED K15 UNUSED K17 UNUSED K19 UNUSED K25 UNUSED L14 UNUSED L15 UNUSED L16 UNUSED L17 UNUSED L18 UNUSED L19 UNUSED L23 UNUSED M17 UNUSED M19 UNUSED M21 UNUSED N16 UNUSED N17 UNUSED N18 UNUSED N19 UNUSED N20 UNUSED N22 UNUSED P03 UNUSED P05 UNUSED P07 UNUSED P09 UNUSED P19 UNUSED P21 UNUSED P25 UNUSED P27 UNUSED P29 UNUSED P31 UNUSED R01 UNUSED R03 UNUSED R04 UNUSED R05 UNUSED R06 UNUSED R07 UNUSED R08 UNUSED R09 UNUSED R10 UNUSED R11 UNUSED R13 UNUSED R16 UNUSED R17 UNUSED R18 UNUSED R20 UNUSED R21 UNUSED R22 UNUSED R23 UNUSED R24 UNUSED R25 Signal Name

286 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. UNUSED R26 UNUSED R27 UNUSED R28 UNUSED R29 UNUSED R30 UNUSED R31 UNUSED R33 UNUSED T01 UNUSED T03 UNUSED T05 UNUSED T11 UNUSED T13 UNUSED T15 UNUSED T19 UNUSED T21 UNUSED T23 UNUSED T29 UNUSED T31 UNUSED T33 UNUSED U01 UNUSED U02 UNUSED U03 UNUSED U04 UNUSED U05 UNUSED U06 UNUSED U07 UNUSED U08 UNUSED U10 UNUSED U11 UNUSED U12 UNUSED U13 UNUSED U15 UNUSED U19 UNUSED U21 UNUSED U22 UNUSED U23 UNUSED U24 UNUSED U25 UNUSED U26 UNUSED U27 UNUSED U28 UNUSED U29 UNUSED U30 UNUSED U31 UNUSED U32 UNUSED U33 UNUSED V01 UNUSED V03 UNUSED V05 UNUSED V11 UNUSED V13 UNUSED V15 UNUSED V19 UNUSED V21 UNUSED V23 UNUSED V29 UNUSED V31 UNUSED V33 UNUSED W01 UNUSED W03 UNUSED W04 Signal Name UNUSED W05 UNUSED W06 UNUSED W07 UNUSED W08 UNUSED W09 UNUSED W10 UNUSED W11 UNUSED W12 UNUSED W13 UNUSED W14 UNUSED W16 UNUSED W17 UNUSED W18 UNUSED W20 UNUSED W21 UNUSED W22 UNUSED W23 UNUSED W24 UNUSED W25 UNUSED W26 UNUSED W27 UNUSED W28 UNUSED W29 UNUSED W30 UNUSED W31 UNUSED W33 UNUSED Y03 UNUSED Y05 UNUSED Y07 UNUSED Y09 UNUSED Y13 UNUSED Y15 UNUSED Y19 UNUSED Y21 UNUSED Y25 UNUSED Y27 UNUSED Y29 UNUSED Y31 UNUSED AA12 UNUSED AA14 UNUSED AA15 UNUSED AA16 UNUSED AA17 UNUSED AA18 UNUSED AA19 UNUSED AA20 UNUSED AA22 UNUSED AB13 UNUSED AB15 UNUSED AB17 UNUSED AB19 UNUSED AB21 UNUSED AC11 UNUSED AC15 UNUSED AC16 UNUSED AC17 UNUSED AC18 UNUSED AC19 UNUSED AC23 UNUSED AD09 UNUSED AD15 Signal Name UNUSED AD17 UNUSED AD19 UNUSED AD25 UNUSED AE08 UNUSED AE09 UNUSED AE14 UNUSED AE15 UNUSED AE17 UNUSED AE19 UNUSED AE20 UNUSED AE25 UNUSED AE26 UNUSED AF07 UNUSED AF09 UNUSED AF13 UNUSED AF15 UNUSED AF17 UNUSED AF19 UNUSED AF21 UNUSED AF25 UNUSED AF27 UNUSED AG01 UNUSED AG05 UNUSED AG06 UNUSED AG07 UNUSED AG08 UNUSED AG13 UNUSED AG14 UNUSED AG15 UNUSED AG17 UNUSED AG19 UNUSED AG20 UNUSED AG21 UNUSED AG26 UNUSED AG27 UNUSED AG28 UNUSED AG29 UNUSED AG33 UNUSED AH05 UNUSED AH07 UNUSED AH15 UNUSED AH17 UNUSED AH19 UNUSED AH27 UNUSED AH29 UNUSED AJ01 UNUSED AJ02 UNUSED AJ04 UNUSED AJ06 UNUSED AJ07 UNUSED AJ14 UNUSED AJ15 UNUSED AJ16 UNUSED AJ17 UNUSED AJ18 UNUSED AJ19 UNUSED AJ20 UNUSED AJ27 UNUSED AJ28 UNUSED AJ30 UNUSED AJ32 Signal Name

PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 287 UNUSED AJ33 UNUSED AK01 UNUSED AK03 UNUSED AK05 UNUSED AK15 UNUSED AK17 UNUSED AK19 UNUSED AK29 UNUSED AK31 UNUSED AK33 UNUSED AL02 UNUSED AL04 UNUSED AL14 UNUSED AL15 UNUSED AL16 UNUSED AL17 UNUSED AL18 UNUSED AL19 UNUSED AL20 UNUSED AL30 UNUSED AL32 UNUSED AM01 UNUSED AM03 UNUSED AM17 UNUSED AM31 UNUSED AM33 UNUSED AN01 UNUSED AN02 UNUSED AN04 UNUSED AN15 UNUSED AN16 UNUSED AN17 UNUSED AN18 UNUSED AN19 UNUSED AN30 UNUSED AN32 UNUSED AN33 VDD B04 VDD B12 VDD B18 VDD B26 VDD D10 VDD D20 VDD D28 VDD D32 VDD F04 VDD F08 VDD F16 VDD F22 VDD H02 VDD H14 VDD H24 VDD H28 VDD K08 VDD K12 VDD K18 VDD K30 VDD M06 VDD M24 VDD M32 VDD N13 Signal Name VDD N21 VDD P04 VDD P16 VDD P18 VDD P26 VDD T02 VDD T10 VDD T14 VDD T17 VDD T20 VDD T28 VDD U16 VDD U18 VDD V06 VDD V14 VDD V17 VDD V20 VDD V24 VDD V32 VDD Y08 VDD Y16 VDD Y18 VDD Y30 VDD AA13 VDD AA21 VDD AB02 VDD AB10 VDD AB28 VDD AD04 VDD AD16 VDD AD22 VDD AD26 VDD AF06 VDD AF10 VDD AF20 VDD AF32 VDD AH12 VDD AH18 VDD AH26 VDD AH30 VDD AK02 VDD AK06 VDD AK14 VDD AK24 VDD AM08 VDD AM16 VDD AM22 VDD AM30 VDD F30 VDD H32 VDD K26 VDD M28 VDD P22 VDD P30 VDD T24 VDD T32 VDD V28 VDD Y22 VDD Y26 VDD AB24 VDD AB32 Signal Name VDD AD30 VDD AF28 VDD AK32 VDD AB14 VDD AB20 VDD AD12 VDD AD18 VDD AF14 VDD AF24 VDD AH08 VDD AH16 VDD AH22 VDD AK10 VDD AK20 VDD AK28 VDD AM04 VDD AM12 VDD AM18 VDD AM26 VDD D02 VDD H06 VDD K04 VDD M02 VDD M10 VDD P08 VDD P12 VDD T06 VDD V02 VDD V10 VDD Y04 VDD Y12 VDD AB06 VDD AD08 VDD AF02 VDD AH04 VDD B08 VDD B16 VDD B22 VDD B30 VDD D06 VDD D14 VDD D24 VDD F12 VDD F18 VDD F26 VDD H10 VDD H20 VDD K16 VDD K22 VDD M14 VDD M20 VDDA E01 Signal Name

5.6.2 Package Dimensions

NOTE: Drawings are not to scale. Figure 67. Scheduler CCGA package Dimensions - Top and Side Views

Figure 68. Scheduler CCGA package Dimensions - Bottom View Table 42. Scheduler CCGA Mechanical Specifications

290 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

5.6.3 Part Number

The PMC-Sierra Part Number for the Scheduler is: Scheduler PM9311-UC 45L9374 PQ PMC Logo Source “B” Part Number Terminal A01 Identifier Scheduler BXXLLLLLLLL Device Name Revision Date Code Lot Number Country of Rev A BYYWW XXXXXXXXXXXX PM9311-UC PMC Part Number Origin PM 9 3 11 - U C Temp. range (C = Commercial 0-70 degrees C) Package code (U = CCGA, H = CBGA)Sequential part number Chipset family code, 1=TT1, 5 =TTX IC IRD prefix PMC prefix (1=SCH, 2=XBAR, 3=DS, 5=EPP)

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 291 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

292 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

6 Characteristics

6.1 SIGNAL ASSOCIATIONS

Table 43. Signal Associations

Table 43. Signal Associations (Continued)

200 Mbps HSTL (EPP/DS Interface)

800 Mbps STI (AIB Interfaces)

6.2 ABSOLUTE MAXIMUM RATINGS

Table 44. Absolute Maximum Ratings Table 45. Absolute Maximum Ratings for 3.3V-tolerant 2.5V CMOS (OOB & Serdes Interface) Table 46. Absolute Maximum Ratings for 2.5V CMOS (JTAG Interface) Table 47. Absolute Maximum Ratings for 200 Mbps HSTL (EPP/DS Interface)

Table 48. Absolute Maximum Ratings for 800 Mbps STI (AIB Interface) Table 49. Absolute Maximum Ratings for 2.5V 200 Mbps PECL (CLK and SOC Signals)

6.3 RECOMMENDED OPERATING CONDITIONS

primarily external to the device package, through the termination resistors, see Table 51. Table 50. Recommended Operating Conditions

300 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

resistors used for termination. Table 51. Additional Power Design Requirements

6.4 DC ELECTRICAL CHARACTERISTICS

Table 52. DC Electrical Characteristics for 3.3V-tolerant 2.5V CMOS (OOB Interface) Table 53. DC Electrical Characteristics for 2.5V CMOS (JTAG Interface)

a. VDDQ is recommended to be 1.6 V to provide additional noise margin above 0.75 V for HSTL signals. Table 54. DC Electrical Characteristics for 3.3V-tolerant 2.5V CMOS (Serdes) Table 55. DC Electrical Characteristics for 200 Mbps HSTL (EPP/DS Interface)

Table 56. DC Electrical Characteristics for 800 Mbps STI (AIB Interfaces) Table 57. DC Electrical Characteristics for 2.5V 200 Mbps PECL (CLK and SOC Signals)

6.5 AC ELECTRICAL CHARACTERISTICS

Table 58. AC Electrical Characteristics for 3.3V-tolerant 2.5V CMOS (OOB Interface) Table 59. AC Electrical Characteristics for 2.5V CMOS (JTAG Interface)

Table 60. AC Electrical Characteristics for 3.3V-tolerant 2.5V CMOS (Serdes Interface) Table 61. Reference Clock for 200 Mbps HSTL (EPP/DS Interface)

Table 62. AC Electrical Characteristics for 800 Mbps STI (AIB Interface) Table 63. AC Electrical Characteristics for 2.5V 200Mbps PECL (CLK Signals)

6.6 TIMING DIAGRAMS

Figure 69. Setup and Hold for 3.3V-tolerant 2.5V CMOS and 2.5V CMOS (OOB Interface) Table 64. Jitter and Static Phase Offset for PLL

Figure 80. Class 1 HSTL Test Loading

Figure 83. AIB Test Loading

Table 65. Request/Data from iDS Frame Format Table 66. Data from oDS Frame Format Table 67. Grant/Data to iDS Frame Format

used from the Crossbar to the Dataslices.

8 CRC over the credit portion of the LCS header on DS1, reserved on DS0, DS2 --DS11

Table 68. Control to xDS Link Format Table 69. Dataslice to Crossbar Frame Format

special control frame for the Scheduler. Table 70. Crossbar to Dataslice Frame Format Table 71. Scheduler to EPP Frame Format Freeze 1 1 if the EPP should be frozen, 0 if not.

1 Flow Control Crossbar Sync bit

Sel 1 if 0, use TDM table 1, if 1, use TDM table 2. Table 71. Scheduler to EPP Frame Format (Continued)

Table 72. EPP to Scheduler Frame Format Req. 1 Asserted when input has TDM cell to send. TDM Sync 1 1 if this port is currently sending a suggested TDM Sync. VLD 1 1 if the best-effort request to follow is valid. control message to the Scheduler (set to 1) or just a frame with no request (set to 0). bits reserved if the request is unicast.

322 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 323 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. Appendix B. Common Pinout Configuration B.1 JTAG INTERFACE The ETT1 Chip Set supports the following JTAG public instructions in conformance with the IEEE Std 1149.1 specification:  IDCODE = b100  BYPASS = b111  EXTEST = b000  SAMPLE/PRELOAD = b010 As part of that standard, the following pins are used on each of the ETT1 Chip Set devices. jtag_tck, jtag_tdi, jtag_tdo, jtag_tms, jtag_trst_L The id codes used for the ETT1 Chip Set are as follows: Dataslice 14367049h Enhanced Port Processor 14581049h Crossbar 14372049h Scheduler 14374049h B.2 RESERVED MANUFACTURING TEST PINS Each device in the ETT1 Chip Set has a number of manufacturing test pins. These pins are reserved for use during the manufacturing process. For functional operation, input test pins must be driven by a voltage source, while output test pins should be left unconnected. Depending on conventions used during PCB assembly, test inputs can be tied with a 1k ohm resistor to the appropriate VDD or GND supply. Such a convention allows for onboard test operation for unusual diagnostic purposes. Test outputs should still be soldered to the PCB for proper thermal and mechanical operation, and these outputs are often brought to the solder (bottom) side of the board through vias for standard test fixtures. For correct functional operation of the ETT1 devices, the following test inputs should be tied to VDD through a resistor: lssd_ce1_a, lssd_ce1_b, lssd_ce1_c1, lssd_ce1_c2, lssd_ce1_c3,

324 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. test_ri, test_lt, lssd_scan_in[0:15], plltest_in The following test inputs should be tied to GND through a resistor: ce0_io, ce0_scan, ce0_test, ce0_tstm3, test_re The following test outputs should be left floating: lssd_scan_out[0:15], mon[0:15], plltest_out The following test inputs must be driven low during reset and high during operation. These can be connected to the same reset signal used for the pwrup_reset_in_L input. NOTE: These inputs can also driven low during board assembly tests to tristate all outputs. Testing of board continuity can done during ICT. test_di1, test_di2 The following test output could be left floating. Optionally, it may be used to monitor that the internal PLL has locked to the ref_clk/ref_clkn inputs by checking the signal level during ETT1 operation. plllock B.3 POWER SUPPLY CONNECTIONS All supply inputs labeled VDD must be driven by a common 2.5 V (nominal) supply. All supply inputs labeled GND must be tied to a common (0 V) ground. All pins labeled UNUSED must not be tied to a power or ground plane. The supply input labeled VDDA must be connected to the common 2.5 V supply through a noise isolation circuit. One possible method is shown in Figure 86.

The voltage ramp-up time is the time needed for the power to the device to stabilize above 2.3 V. Figure 88. Voltage Ramp-up

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 327 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. Appendix C Interfacing Details forETT1 C.1 COMPENSATING FOR ROUND TRIP DELAY BETWEEN LINECARD AND ETT1 (LCS) A key benefit of the LCSTM protocol is the physical separation of the ETT1 fabric and the linecards. This separation introduces some latency in the communication channel between the linecard and ETT1. The credit mechanism used in LCS ensures that this latency does not adversely impact the throughput of the system. However, the linecard must compensate for the latency in the communication channel so that the credit mechanism can function correctly. This section describes the operations that the linecard should perform. C.1.1 Preventing Underflow of the VOQ The linecard must respond to grants from the ETT1Chip Set within a certain period of time in order to sustain maximum throughput. In this section we explain how much time the linecard has in which to respond to grants. Consider the case where the virtual output queue in the port card in Figure 89 is full, with 64 cells waiting to be forwarded to the appropriate output. Now, assume that the ETT1 Scheduler starts to forward those cells, one per cell time. When the first cell is forwarded, a grant is sent to the linecard, indicating that another cell can be sent to that output. If that cell does not arrive before the VOQ is empty, then the throughput for that flow cannot be equal to the link rate. Therefore, the new cell forwarded by the linecard must arrive at the EPP/DS before the VOQ is emptied. The round trip time for the whole process must be less than or equal to the number of cells in the VOQ; 64 cells for an OC-192c flow and 16 cells for an OC-48c flow. Figure 89 shows this time divided between the EPP/DS devices and “everything else”. The EPP/DS uses 22 cells plus the programmed_token_delay. Refer to Section 3.4.2.39 “Internal Delay Matching Adjustments” on page 188. Subtract the total (22 cells + programmed_token_delay) from 64 to get the budget for “everything else” (OC-192c cells). “Everything else” includes any delay on the port card (Serdes), a mux device, the propagation delay of the signal down the fiber, the Serdes at the linecard and the linecard device delay. To calculate the time available to the linecard device, subtract the other elements from the “for everything else” budget. The fiber delay is approximately 15 cell times (7.5 cells in each direction), assuming a 70m link.

Figure 89. The Time Available to the Linecard

Figure 90. Cells are Striped Across the 12 Physical Links interface, initialization, and errors.

Figure 91. The 10-bit Data Paths map an arbitrary 8b/10b control word to its own decoded control word.

Figure 92. Initial Sequence Expected at Port Card timing so that the K28.5 word corresponds to the first word in the cell. the ETT1 CPU that the link is up and locked in both directions. Table 74 shows the state machine operated by the transmit side of the Dataslice. Table 73. Suggested 8b/10b Decode Map Within the Dataslice Table 74. Dataslice Egress Truth Table

332 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. If the encoder is not enabled then all zeros are sent. If the Dataslice output FIFO is empty then the Dataslice sends either Idle-ready or Idle-not-ready, depending on the state of the receiver. Otherwise the Dataslice will send the data cell at the head of its transmit FIFO. Why K28.5, K27.7 etc? The Dataslice doesn’t care what the actual 10b control words are. We have chosen K28.5 as it is the 8b/10b comma character which the Serdes receiver will try to lock to in order to determine the word boundary in the bit stream. The choice of K27.7 and K29.7 is somewhat arbitrary but they comply with the 8b/10b transmission rules. (Indeed any control characters can be used provided that the Dataslice decoder table is configured appropriately.) To summarize, the steps required to bring up the ingress and egress linecard to Dataslice links are:  Program the 8b/10b codec tables on each Dataslice;  Transmit Idle-not-ready cells from the linecard to each Dataslice;  Enable the 8b/10b codecs on each Dataslice causing the Dataslices to start sending Idle-not-ready cells to the linecard;  Each Dataslice will transition to sending Idle-ready cells when it has acquired cell synchronization from the linecard;  Likewise the linecard should transition to sending Idle-ready cells when it has acquired cell synchronization from each Dataslice. Eventually both the linecard and the Dataslice will be receiving the Idle-ready sequence, indicating that both ends of the link have locked up. The linecard transmitter should then switch to sending data cells. The Dataslice has a receive FIFO which stores the incoming words only if they are decoded as a data word. Idle-not-ready and Idle-ready are not stored in the FIFO. A data cell is a sequence of six 8b/10b data words (Dxx.x). These data cells are stored in the receive FIFO. The Dataslice reads from the receive FIFO at a nominal 150MHz which is derived from a local oscillator. This local oscillator will operate at a slightly different frequency to that used on the linecard transmitter. To avoid overruns in the case where the linecard is at a slightly faster frequency than the Dataslice, the linecard must send Idle cells at regular periods. The period is determined by the maximum difference in frequency that is possible. Assuming +/-200 ppm oscillators, and allowing for storage of one extra cell in t h eF I F O ,t h e n1i n2 , 5 0 0c e l l sm u s tb ea nI d l ec e l lt oa v o i do v e r r u n . The oscillator on the ETT1 port card might be faster than that used on the linecard. Therefore, the Port Processor contains a register that will determine the period between Idle cells sent from the ETT1 port, to avoid overrunning the receive FIFO in the linecard.

PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE 333 PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. C.2.1.3 Link Errors If the link has no errors then the Dataslice will always see valid sequences of either six control words or six data words. In reality, errors will occur. The system designer must be aware of the implications of the different errors.  If a single word is corrupted to map to an illegal 10b word then the Dataslice will set DIR bit 23 and assert an interrupt signal to the OOB bus. If the error occurs at one of the Dataslices that deals with the LCS header or magic packets (i.e., Dataslices 0, 1, or 2) then the error is communicated to the Enhanced Port Processor which will ignore that cell.  If a word is corrupted to look like another valid word, then this willnotbe detected, unless it corrupts the LCS header in which case the header CRC should detect it. If the user payload is corrupted then the cell will still be carried through the switch to the egress linecard.  The Serdes recognizes the bit sequence 0011111xx as a comma character for byte alignment. This bit sequence occurs in the K28.5 comma character and it is used for link synchronization as described above. An error can potentially cause the 0011111xx bit sequence to appear anywhere in the serial bit stream, causing the Serdes to re-align to the corrupted 0011111xx comma and thus lose byte synchronization. This condition will cause a sequence of 8B/10B errors to be encountered by the 8B/10B decoder and the Dataslice will raise the DIR23 decoder error interrupt. The Serdes should re-acquire lock when the linecard next sends an Idle cell. To prevent fiber optics bit errors from throwing the Serdes out of byte synchronization the Serdes byte sync enable pin can be connected to a pin on the Dataslice that is accessible through the OOB. Once the Serdes has locked onto the right byte boundary software can make sure that the Serdes will no longer look for the 0011111xx comma character.  A subtle yet serious error can occur if an idle cell is corrupted in such a way that the comma character as programmed in the Dataslice 8B/10B decoder lookup table appears anywhere other than at the start of the cell or several data words in the same cell are corrupted into valid control words. This can cause the entire cell to appear as a cell of the other type (data -> Idle or Idle -> data). While improbable, this can have serious consequences that are discussed in the section on Channel errors. C.2.2 Channel A channel consists of 121links and thus 12 logical Dataslices, each slice dealing with six bytes of the 72 byte cell. C.2.2.1 S ynchronization Each of the 12 channels carry a part (6-bytes) of the same LCS cell. Although the 12 channels are all operating at the same frequency, their phase may be offset with respect to each other. In other words, each Dataslice may receive its 6-byte portion of the cell at a slightly different time from its neighboring Dataslices on the same switch port. So that the 12 parts of a cell remain intact, and are all switched at the same time, 1. The EPP can support fourteen logical slices (i.e. seven Dataslice chips)

EPP allows for one cell to be removed from the input FIFO of each Dataslice and sent to the EPP. programmed token delay, in units of 150MHz clock cycle times. Possible sources of skew include linecard launch timing, fiber, connector, serializer/deserializer, mux chip. decreases the budget for the rest of the system (see C.1.2). resynchronize the incoming slices to form the complete cell.

1. All ETT1 devices operate from the same core clock of 200MHz.

Table 75. Programmed Token Delay vs. Inter-link Skew

Figure 93. Cells May Be Skewed Between Dataslices Each link in the channel can, of course, suffer bit errors as discussed in the section on link errors above. However, the channel can incur a particular type of error which the system designer must be aware of. it is read from the input FIFO and passed on to the port processor. recovery actions from this rare condition require the port to be taken down and brought back up again. is true for the ETT1 port card: it should not send cells at a rate that will cause cell loss at the linecard.

336 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. The Enhanced Port Processor ensures this by periodically inserting an Idle cell in the egress cell stream (to the linecard). Clearly, if the linecard can accept a cell rate that is fundamentally greater than the maximum rate at which cells are sent, then there is no problem. In practice, however, the rate at which cells are transmitted and the rate at which cells can be accepted are determined by the clocks at each end. These clocks have some inherent imprecision, usually expressed as some number of parts per million. This means that the true rate at which cells can be transmitted or accepted might be slightly greater or smaller than the nominal rate. The remainder of this section is an example to explain this in more detail. Example: We assume that the ETT1 port and Linecard operate at a nominal frequency of 200MHz with an imprecision of +/-200ppm. One (OC-192c) cell time is eight clocks or 40ns, corresponding to a nominal rate of 25M cells/s +/-5000 cells/s. 1. Linecard to ETT1 Port. Assuming a slow clock at ETT1, then the ETT1 port may only be able to accept 24,995,000 cells/s. The linecard must not exceed this rate. So, if the linecard has maximum frequency clock (200.04MHz) then it could send 25,005,000 cells/s. So, to avoid overrunning the ETT1 port the lin- ecard must send 10,000 Idle cells per second to bring its effective cell rate down to that of the ETT1 port. 2. ETT1 Port to Linecard. In this direction the worst case is where the ETT1 port has the highest frequency clock and the lin- ecard has the lowest frequency. So the ETT1 port must insert 10,000 Idle cells per second. This would be done by programming the value 2,499 into the Idle Count Register in the Enhanced Port Processor. This causes the Enhanced Port Processor to insert one Idle cell for every 2,499 cells transmitted. C.4 AVOIDING DELAYS DUE TO IDLE CELLS Figure 94 shows a simple view of a linecard. A grant is received from the ETT1 port and the linecard must immediately respond with the corresponding cell body. However, if the Idle counter expires just at that moment, then the linecard cannot send the cell body; it must send an Idle cell instead. This behavior can cause a cell body to be delayed, thus diminishing the effective round-trip time available to the linecard, as well as introducing extra queuing into the linecard.

338 PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS’ INTERNAL USE PM9311/2/3/5 ETT1™ CHIP SET PMC-2000164 ISSUE 3 ENHANCED TT1™ SWITCH FABRIC RELEASED Data Sheet PMC-Sierra, Inc. 2. Whenever the linecard receives an LCS cell that doesnothave a grantand F = 1 then reset F to 0 and send an Idle cell. This very simple algorithm ensures that an Idle cell can be sent whenever the linecard does not have to respond to a grant. The only requirement here is that the ETT1 port must send a minimum rate of non-grant cells. This can be achieved in one of two ways: 1. Program the Idle Count register in the EPP to send Idles slightly more frequently than the linecard. 2. Program the NoGrant register in the EPP to restrict the rate of grants coming from the EPP. NOTE: The maximum number of cells between Idles sent by the linecard, if it follows this algo- rithm, is then set by the following equation: max_non_idle = linecard_idle_count + min(EPP_idle_count, EPP_NoGrant_count) This means that the just setting the Idle count at the linecard is not sufficient to ensure a minimum Idle cell rate; the rate at which non-grant cells are received must be included in the equation. The only further complication is that the linecard also sends out requests for new cells. However, it is assumed that requests (new cells) arrive at the linecard at a slower rate than they can be forwarded to the ETT1 fabric due to speedup within the fabric. Given this, then the linecard just needs to provide a one-deep queue for outbound requests, as shown in Figure 96. Given the higher rate at which requests are sent, then the request FIFO will never overflow.

Figure 96. One-deep Queue for Outbound Requests