The FCIP Standard for SAN Connectivity Across TCP/IP Networks

Abstract

As organizations increasingly face an enormous influx of data that must be stored, protected, backed up, and replicated, they have turned to Storage Area Networks (SANs) to manage these tasks in an efficient, cost-effective manner. Fibre Channel SANs address the need to protect data while making it available to a wide variety of users who demand anywhere, anytime data access. To ensure that this data reaches all the users who need it, organizations are now looking for new ways to transport data throughout the enterprise—locally over the SAN as well as over much longer distances.

One of the best ways to achieve this goal is to interconnect geographically dispersed SANs through reliable, high-speed links. This approach involves transporting Fibre Channel block data over existing IP infrastructures currently used throughout the enterprise. The new FCIP protocol standard has rapidly gained acceptance as a manageable, cost-effective way to blend the best of both worlds: Fibre Channel block data storage and proven, widely deployed IP infrastructure. As a result, organizations now have an excellent way to protect, store, and move their data while leveraging existing technology investments.

The Rapid Emergence of Storage Area Networks

For organizations around the world, the advent of the Internet and e-business—and the resulting flurry of new applications—has created unprecedented growth in the amount of data they must process, store, and transport throughout the enterprise. As these data storage and processing requirements continue to grow, so does the value of that data—as well as the need to protect its integrity, security, and availability. Moreover, organizations must make this data available to all of the users who have come to expect anytime, anywhere data access. To address these requirements and better utilize their information assets, many organizations have turned to Storage Area Networks (SANs).

SANs have quickly proven themselves ideal for handling the enormous data storage and backup requirements resulting from this information explosion. In particular, by using SANs, customers can now efficiently and independently scale the amount of storage required by their application servers—all of which greatly improves the efficiency and utilization of storage resources.

Today's SANs are primarily based on the Fibre Channel protocol, a gigabit networking technology specifically developed for interconnecting servers and storage devices. Fibre Channel is an industry-standard, field-proven technology that provides very high performance and rapid transport of block data based on the SCSI protocol. As a result, Fibre Channel SANs have been widely deployed as a means to efficiently scale storage systems that meet the requirements of data-intensive businesses.

The Growing Need for SAN Interconnectivity

Because of their high reliability and performance, SANs solve a wide range of technology challenges. SANs are especially adept at supporting business continuance and disaster tolerance through high-availability solutions that enable organizations to:

• Archive tapes remotely without the need for physical tape transport.

• Mirror disk storage at remote locations in redundant configurations.

• Share storage at the block level (versus file level) over extended distances.

• Deploy disaster recovery sites in remote locations to guard against natural disasters and other types of outages.

• Optimize personnel and equipment resources by centrally managing distributed environments. Implement a dynamic storage capacity utility for servers located throughout the enterprise.

In most cases, organizations can implement these types of solutions by interconnecting SANs over existing network infrastructures—introducing "distance" as a key design element and spreading IT equipment over multiple locations. By combining Fibre Channel SAN technology and traditional IP networks customers can extend the benefits of the SAN across much longer distances. For instance, Storage Service Providers (SSPs) are already extending the value of SANs by consolidating disaster recovery and offsite backup services in a single location with specially trained staff. These types of SSP implementations can provide immediate time and cost advantages to customers while enhancing data protection and availability.

In order to meet these growing distance requirements, organizations need new ways to interconnect geographically distributed SANs. The choice of SAN interconnectivity technologies depends on several factors, including distance, bandwidth, price, performance, and latency (synchronous versus asynchronous applications). In general, each technology has certain characteristics and tradeoffs to consider—and the value of the particular technology must be weighed against its cost. Today, organizations can choose from multiple existing and emerging technologies to interconnect disparate SANs over a variety of distances:

• Fibre Channel over dark fiber or DWDM.

• Fibre Channel over SONET.

• Fibre Channel over ATM.

• Fibre Channel over IP.

Fibre Channel over dark fiber or DWDM is capable of spanning distances beyond the native Fibre Channel 10 km and is predominantly used in Metropolitan Area Networks. The transport distance can be extended by adding buffers at both ends of the long-distance link—enabling full Fibre Channel bandwidth up to approximately 120 km. Additional buffers and dark fiber or DWDM can be used to extend SAN connections over even longer distances without performance degradation. Although this type of solution is ideal for remote mirroring operations that require extremely high bandwidth, it is also expensive. Moreover, access to dark fiber lines may be limited.

Emerging Fibre Channel-over-SONET solutions are expected to function in similar fashion to DWDM. The network would appear as a single long-distance SAN with additional buffers required for full bandwidth operation—making it useful for remote mirroring operations.

Fibre Channel over ATM encapsulates Fibre Channel in an ATM transport with the benefits of different levels of service guarantees and varying bandwidth. Current solutions can support OC-3 up to OC-12 and can specify the bandwidth to be utilized on each connection. Depending on the ATM Class of Service (CoS), these solutions can be used for latency-sensitive (synchronous) applications such as mirroring as well as latency-tolerant (asynchronous) applications such as tape backup. These solutions are limited to OC-12 bandwidth and are relatively expensive to implement.

Fibre Channel over IP is an ideal combination of technologies to address the dual requirements of storage networking and networking over distance. A mature technology optimized for storage-data movement within the campus and data center, Fibre Channel represents a major investment in software compatibility, interoperability, and proven applications for campus-based storage networking. Likewise, IP is a mature technology optimized for data movement across WAN distances. It represents a major investment in software compatibility, equipment interoperability, and proven applications for WAN-based data networking.

Today's Fibre Channel-over-IP solutions encapsulate Fibre Channel and transport it over a TCP socket. As in all IP networks, performance can vary based on the types of switches and routers, the number of hops the packets must traverse, and the level of congestion in the network. Today, storage transport performance over IP networks—especially over public networks—is limited due to the variable latency of service provider networks. As IP and Ethernet equipment continues to evolve, higher levels of Quality of Service (QoS), Cost of Service (CoS), provisioning, and circuit emulation should provide the latency guarantees required by synchronous storage applications. In controlled environments, these technologies might even improve the performance of IP networks. Regardless, Fibre Channel over IP is currently a very cost-effective technology for asynchronous applications such as remote data backup.

The Benefits of Fibre Channel Over IP (FCIP)

While multiple technologies are capable of interconnecting SANs, very few can be widely deployed in a cost-effective manner today. Because most organizations already have IP connections and significant experience with Ethernet and IP networks, they can usually leverage this equipment and expertise to help manage data in conjunction with Fibre Channel SANs. For example, IP connectivity provides the greatest flexibility at the lowest cost for latency-tolerant applications. As a result it can be used to back up data across a campus network, Metropolitan Area Network (MAN), or WAN. Moreover, this flexible technology can be deployed within a single enterprise or in an SSP multi-tenant environment. Today, many types of organizations are beginning to transport SAN storage over IP, especially for non–real-time data transfer. By combining the best of two mature technologies (as shown in Figure 1), FCIP solutions provide a more standardized and lower cost way to increase SAN interconnectivity for a variety of applications. In fact, FCIP includes full support for the Fibre Channel set of equipment and software. Organizations can seamlessly extend existing and planned Fibre Channel SANs over long distances through IP networks—thereby protecting significant investments in both technologies. In addition, FCIP provides a cost-effective way to achieve business protection (enabling such solutions as remote tape archiving).

One of the first implementations of FCIP is the interconnection of disparate SAN islands. FCIP can transport existing Fibre Channel services across the IP network such that two or more interconnected SANs can appear as a single large SAN and can be managed by traditional SAN management applications. In addition, FCIP enables SAN applications to support additional protocols without modification. These applications might include disk mirroring between buildings in a campus network or remote replication over the WAN. The type of applications utilized will be based on the distance the data must travel, the network bandwidth, and the QoS requirements and/or abilities of the network connection. While some implementations of FCIP are point-to-point "tunnels," the protocol does not require that the "gateways" support only point-to-point tunneling. The FCIP standard supports all Fibre Channel services, including FSPF routing algorithms, such that multiple logical links created from a single gateway can route Fibre Channel packets over the IP infrastructure. Not only is FCIP routable, but IP networks do not need to know anything about the packets being routed. The Fibre Channel services handle all routing between logical links, while the TCP protocol handles the delivery of packets to the specific gateway device.

The Emergence of the FCIP Standard

FCIP is defined as a tunneling protocol for connecting geographically distributed Fibre Channel SANs transparently over IP networks. It uses TCP/IP as the transport while keeping Fibre Channel fabric services intact. The FCIP standard is a cost-effective technology that uses widely deployed IP standards while enabling management of remote SANs through existing SAN management tools. This extremely powerful standard is designed to leverage the rapid advances in Gigabit Ethernet and emerging 10 Gigabit Ethernet technologies as well as tested local-loop and long-haul technologies. As bandwidth increases in local networks and prices continue to fall, FCIP becomes even more cost-effective.

FCIP is being standardized by the Internet Engineering Task Force (IETF) IPS Working Group, whose main objectives are to:

• Specify the encapsulation for Fibre Channel frames being transported by TCP/IP.

• Specify the use of the encapsulation to create a virtual Fibre Channel link that connects Fibre Channel devices and fabric elements.

• Specify the TCP/IP environment for supporting virtual Fibre Channel links and providing the capabilities for tunneling Fibre Channel traffic over an IP-based network, which include security, data integrity, congestion, and performance.

To establish connectivity between remote SANs over LANs, MANs, or WANs, the FCIP standard relies on IP-based network services. FCIP also relies upon TCP/IP for congestion control and management and upon both TCP/IP and Fibre Channel for data error and data loss recovery.

A Powerful Standard Designed for Growth

Today, many organizations are deploying FCIP to enable SAN interconnectivity over longer distances for synchronous applications such as remote data backup (see Figure 2). As higher link speeds—including 10 Gigabit Ethernet and 10 Gigabit Fibre Channel—emerge, FCIP will be ideal for handling synchronous applications such as real-time mirroring. New applications such as the distribution of time-sensitive multimedia data and interactive multimedia Web applications will also be possible. Over the long term, SAN interconnectivity through FCIP is well positioned to provide a high-performance, high-reliability infrastructure that can help a wide range of organizations:

• Leverage their existing technology investments.

• Expand their service offerings to existing customers.

• Bring services closer to end users while improving response times and quality of delivery.

While other technologies are still at the planning stages, FCIP is currently available. By leveraging existing Fibre Channel and TCP/IP investments and an organization's collective expertise, FCIP provides a much more cost-effective solution that can be widely deployed today.

For more information about the FCIP standard, visit www.ietf.org.

...SNIA profile