Tom Demchak (SIAC)

May 10, 2001

As requested, I have documented the key points that I discussed at the March 26th Langevoort sub-committee meeting on Alternative Models. At that meeting, in addition to reviewing the CTS and CQS processing, I reviewed four considerations that could have technical implications for disseminating real time market data in a competitive consolidator environment. The four considerations are: (1) potential differences in sequencing of information; (2) different validation tolerances; (3) capacity considerations; and (4) differences in protocols and data formats. Each of these considerations is addressed below along with the general description of the systems.

CTS/CQS Background

On behalf of CTA, SIAC operates and maintains two separate computer environments to process trade and quote information. Trade information is process by the Consolidated Tape System; quote information by the Consolidated Quotation System. These systems are operated on fault tolerant computer platforms (Tandem hardware) at different physical computer sites; CTS is operated at the MetroTech complex in Brooklyn; CQS is operated at the 55 Water Street complex in lower Manhattan, thereby providing redundancy in the event of a site disaster. If a site disaster should occur at either location, all of the computer processing would be transferred to the surviving site at reduced capacity (65% of normal capacity). SIAC’s service level agreements with CTA, state that an event that causes a single system disaster can be recovered in the surviving system within a 2 hour time period; a full site disaster can be recovered on a next day basis. The actual recover time frame for a full site disaster is under several hours

SIAC's two operating sites are located on two separate power grids and have multiple redundant communications paths connecting the two facilities. The sites have uninterrupted power supplies to smooth out Con Edison power as well as emergency generator back-ups. All of our fault tolerant Tandem systems utilize a SIAC developed software environment, Common Software, which allows for a common operations interface and internal processing infrastructure. This software product provides our Tandem systems many levels of error recovery thereby preventing duplication or loss of data by utilizing different functions (e.g, safestoring of messages, internal message management).

CTS and CQS receive their data from the nine market centers over network based TCP/IP connections. Each market center has redundant communication paths into the two operating environments and each uses diverse common telephone carriers to send their trade and quote data to SIAC. SIAC uses a state-of–the-art high bandwidth (155MB) router backbone network to simultaneously distribute, via IP Multicast, trade and quote information over 10 MB routers to the 66 CTS and 62 CQS subscribers (data recipients). These data recipients receive their data over T-1 and T-3 communications facilities from both sites through diverse common carriers. Independent of where the system is actually located (e.g., CTS at MetroTech), both streams of data are simultaneously distributed out of both sites using a SIAC developed Multicast Packet Replicator (MPR) thus providing the Data Recipients "live" redundant streams. The use of IP Multicast, introduced to the market data industry in 1997 was the first widespread implementation of such technology. SIAC’s design and implementation of this technology has been recognized and accepted into the permanent of the Smithsonian Institute. This technology allows trade and quote data to be distributed in a "broadcast" mode over a network and eliminates any dependency where one data recipient having a problem might impact another data recipient.

In situations where a data recipient has experienced data loss in receiving the information (e.g., due to a system problem at the receiving site) an automated retransmission facility is available to allow that data recipient to automatically request and receive message retransmissions.

CTS and CQS receive trade and quote information, respectively from the nine market centers using a standard message format. Each system validates its respective message formats, verifies the information against its databases (e.g., valid symbol, etc.), consolidates the information with the other market centers information, and disseminates the information to the data recipients over its respective common standard message formats via the IP Multicast network. Included in every trade and quote message is a "time stamp" which represents the time that the message is disseminated.

Every trade and quote is stored in the system for both on-line and after hours processing. Each system maintains a master database by symbol. CTS maintains in its database, by symbol, a consolidated high, low, last price and volume; and for each market center that trades that symbol, the market’s last sale and volume information. This information is updated with each trade. Market centers are required, per the CTA Plan, to report their trade activity within 90 seconds of execution time to CTS otherwise the trade report must be designated as a late report. It is the responsibility of the SRO to determine when a trade is late. Late trades do not impact the national last sale price.

CTS provides an automated correction processing capability in the event that a market center incorrectly reported their information. When a market center issues a correction message, CTS processes the correction and disseminates the revised trade report along with the updated consolidated and market center information that is maintained in the database. CTS also disseminates at end-of-day, closing summary messages that provides by stock the summary information from its database.

For every quote message received from a market center, CQS calculates a National Best Bid and Offer (NBBO) based on a price, size and time priority scheme. If the quote is a NASDAQ market maker quote, CQS also calculates an NASDAQ BBO. CQS disseminates the Market center’s "root" quote with an appendage that includes the National and NASDAQ BBOs. In the event that a market center is experiencing technical difficulties in providing quote information, CQS also has a facility that, at the direction of the market center, disseminates zero quotes in their securities thus eliminating any stale quotes and taking that market center out of the BBO calculations.

Competing Consolidator Risks

There are four potential risks that we have identified associated with a competing consolidator model.

1. Sequencing of Information

Once CTS and CQS receive a message, the systems guarantee proper sequencing of information throughout the internal system processing. The Tandem operating environment "services" the market center’s TCP/IP input lines (i.e., logical connections) on a time slicing arrangement based on a round robin approach. The messages from a market center are framed in TCP/IP packets. The number of messages in a packet, up to 1000 bytes per packet, is determined by the level of activity of the market center and the efficiency by which its system blocks the messages.

In a competitive consolidator model, there are several considerations that could result in messages being processed in a different sequence among consolidators. Multiple consolidators would be using different hardware, software and communications platforms to process the data. Different hardware platforms’ operating systems such as a Tandem and a Unix platform (or even the same hardware platforms used by different consolidators (Unix)) could be servicing the nine market centers’ input differently, potentially resulting in one consolidator’s system processing the TCP/IP packets in a different time slice than another consolidator’s system thus processing messages in different sequences for the same symbol. The servicing differences between different platforms are most likely in the sub-second range. Consequently, the frequency of occurrence of out of sequence messages among competitive consolidators due to differences in operating system environments should be minimal dependent on how the market centers send the data to the multiple consolidators.

Other factors that could result in incorrect sequencing among competitive consolidators are message gapping, internal system software design, and the overall choice of dissemination technology utilized. One consolidator could experience message gapping in either receiving a message from a market center or distributing a message to a client due to numerous factors such as a communication line problem while other consolidators process the messages normally. There could be internal system software design differences such as the number of times a consolidator’s system safestores the message thus resulting in processing dwell time differences. The internal processing path designs could be different such as how and when the messages are validated or corrections to trades are processed.

With this use of TCP/IP technology as the communication protocol layer for receiving messages from a market center, the frequency of gaps due to communication line problems are minimal. Such gaps are probably more attributable to system problems at a market center. With the use of IP Multicasting to distribute the data to the data recipients and the dual streams of redundant data, the frequency of gaps experienced by the data recipients due to communications line problems are also minimal. In fact, we have noticed a decrease in the number of gaps experienced with the use of IP multicasting.

2. Validation Tolerances

CTS and CQS validate all market center messages to verify that the messages are being forwarded utilizing the correct message structures. Incorrect message formats will result in the messages being rejected back to the originating market center. CTS and CQS calculate various information for the industry such as the NBBO, NASDAQ BBO, trade summary information such as high, low, volume information and the processing of trade corrections. A zero quote capability is provided to eliminate stale quotes when a market center is experiencing technical difficulties. In a competitive consolidator environment, there should be standards established to verify consistency across calculated information assuming that this information is required. With standards, the impact of competitive consolidator’s generating inconsistent information is likely not great. Today, each CTS/CQS data recipient calculates its own open, high, low, volume information since that information is not currently provided with every trade report. Most if not all of the nine market centers calculate their own NBBO for ITS purposes to eliminate NASDAQ non-CAES market maker quotes from the NBBO calculation.

3. Capacity Considerations

CTS and CQS must be sized to handle the required capacity levels of the combined market centers. There are numerous components within a consolidator’s environment that must be sized to handle the required capacity levels. The components include: the network capacity, input and output line capacities, system capacity, internal system threading capacity, storage and memory capacity and database size. If any one of these components can not handle the required capacity levels then queuing will result thus delaying messages. Message capacity is usually designated by the number of messages per second (mps) that must be handled.

In a competitive consolidator model, if one consolidator experiences problems in handling the required message capacity, their data recipients will experience delays. This situation exists today at the data recipient level. If one of the CTS/CQS data recipients can’t handle the capacity requirements, their clients experience delays.

4. Protocols/Data Formats

CTS and CQS receive the nine-market centers’ information utilizing standard input formats over a network-based protocol, TCP/IP. The systems disseminate the consolidated data streams to the data recipients using standard output formats using IP multicasting. When changes are made to either the formats or protocols, SIAC provides extensive testing support for the industry. For major changes, we normally provide a conversion arrangement where we support both the old and new service simultaneously. This arrangement allows the data recipients to convert to the new environment independently.

In a competitive model, there may be different protocols, message formats and technologies used. These differences could make it more difficult for the industry to handle trade and quote information especially when converting to new environments. To mitigate this potential risk, message format and protocol determination might be handled via an industry technology standards committee. There is an effective example of such an arrangement with the FIX (Financial Information Exchange) protocol that is used by many global and domestic institutional trading firms and is now being adopted by many of the market centers for order handling.

Today, most of the state-of-the-art protocols are based on TCP/IP protocols that the CTS and CQS market centers and data recipients are used to handling. New distribution technologies for may provide a competitive consolidator a technology advantage over a consolidator that is using an older technology.