click to visit home page
leading the way to the new storage frontier .....
the SSD Buyers Guide - click to see article
SSD buyers guide ...
high availabaility SSD arrays
read the article on SSD ASAPs
auto tiering SSDs ..
fibre channel compatible SSDs
pcie  SSDs - click to read article
PCIe SSDs ..
click to see directory of SAS SSD companies
image shows megabyte waving the winners trophy - there are over 200 SSD oems - which ones matter? - click to read article
top SSD oems ..

The FCIP Standard for SAN Connectivity Across TCP/IP Networks

See also:- storage ORGs
the Top 20 SSD companies
Storage Architecture Guide
an introduction to enterprise SSD market silos
Editor:- May 5, 2002 - The interval between new computer standards first being proposed, and then finding their way into commercial products, just keeps getting shorter. This white paper was first published by the SNIA in June 2001. SNIA has kindly given us permission to republish it here and bring these ideas before a wider audience.


As organizations increasingly face an enormous influx of data that must be stored, protected, backed up, and replicated, they have turned to Storage Area Networks (SANs) to manage these tasks in an efficient, cost-effective manner. Fibre Channel SANs address the need to protect data while making it available to a wide variety of users who demand anywhere, anytime data access. To ensure that this data reaches all the users who need it, organizations are now looking for new ways to transport data throughout the enterprise—locally over the SAN as well as over much longer distances.

One of the best ways to achieve this goal is to interconnect geographically dispersed SANs through reliable, high-speed links. This approach involves transporting Fibre Channel block data over existing IP infrastructures currently used throughout the enterprise. The new FCIP protocol standard has rapidly gained acceptance as a manageable, cost-effective way to blend the best of both worlds: Fibre Channel block data storage and proven, widely deployed IP infrastructure. As a result, organizations now have an excellent way to protect, store, and move their data while leveraging existing technology investments.

The Rapid Emergence of Storage Area Networks

For organizations around the world, the advent of the Internet and e-business—and the resulting flurry of new applications—has created unprecedented growth in the amount of data they must process, store, and transport throughout the enterprise. As these data storage and processing requirements continue to grow, so does the value of that data—as well as the need to protect its integrity, security, and availability. Moreover, organizations must make this data available to all of the users who have come to expect anytime, anywhere data access. To address these requirements and better utilize their information assets, many organizations have turned to Storage Area Networks (SANs).

SANs have quickly proven themselves ideal for handling the enormous data storage and backup requirements resulting from this information explosion. In particular, by using SANs, customers can now efficiently and independently scale the amount of storage required by their application servers—all of which greatly improves the efficiency and utilization of storage resources.

Today's SANs are primarily based on the Fibre Channel protocol, a gigabit networking technology specifically developed for interconnecting servers and storage devices. Fibre Channel is an industry-standard, field-proven technology that provides very high performance and rapid transport of block data based on the SCSI protocol. As a result, Fibre Channel SANs have been widely deployed as a means to efficiently scale storage systems that meet the requirements of data-intensive businesses.

The Growing Need for SAN Interconnectivity

Because of their high reliability and performance, SANs solve a wide range of technology challenges. SANs are especially adept at supporting business continuance and disaster tolerance through high-availability solutions that enable organizations to:

  • Archive tapes remotely without the need for physical tape transport.
  • Mirror disk storage at remote locations in redundant configurations.
  • Share storage at the block level (versus file level) over extended distances.
  • Deploy disaster recovery sites in remote locations to guard against natural disasters and other types of outages.
  • Optimize personnel and equipment resources by centrally managing distributed environments. Implement a dynamic storage capacity utility for servers located throughout the enterprise.

In most cases, organizations can implement these types of solutions by interconnecting SANs over existing network infrastructures—introducing "distance" as a key design element and spreading IT equipment over multiple locations. By combining Fibre Channel SAN technology and traditional IP networks customers can extend the benefits of the SAN across much longer distances. For instance, Storage Service Providers (SSPs) are already extending the value of SANs by consolidating disaster recovery and offsite backup services in a single location with specially trained staff. These types of SSP implementations can provide immediate time and cost advantages to customers while enhancing data protection and availability.

In order to meet these growing distance requirements, organizations need new ways to interconnect geographically distributed SANs. The choice of SAN interconnectivity technologies depends on several factors, including distance, bandwidth, price, performance, and latency (synchronous versus asynchronous applications). In general, each technology has certain characteristics and tradeoffs to consider—and the value of the particular technology must be weighed against its cost. Today, organizations can choose from multiple existing and emerging technologies to interconnect disparate SANs over a variety of distances:

  • Fibre Channel over dark fiber or DWDM.
  • Fibre Channel over SONET.
  • Fibre Channel over ATM.
  • Fibre Channel over IP.

Fibre Channel over dark fiber or DWDM is capable of spanning distances beyond the native Fibre Channel 10 km and is predominantly used in Metropolitan Area Networks. The transport distance can be extended by adding buffers at both ends of the long-distance link—enabling full Fibre Channel bandwidth up to approximately 120 km. Additional buffers and dark fiber or DWDM can be used to extend SAN connections over even longer distances without performance degradation. Although this type of solution is ideal for remote mirroring operations that require extremely high bandwidth, it is also expensive. Moreover, access to dark fiber lines may be limited.

Emerging Fibre Channel-over-SONET solutions are expected to function in similar fashion to DWDM. The network would appear as a single long-distance SAN with additional buffers required for full bandwidth operation—making it useful for remote mirroring operations.

Fibre Channel over ATM encapsulates Fibre Channel in an ATM transport with the benefits of different levels of service guarantees and varying bandwidth. Current solutions can support OC-3 up to OC-12 and can specify the bandwidth to be utilized on each connection. Depending on the ATM Class of Service (CoS), these solutions can be used for latency-sensitive (synchronous) applications such as mirroring as well as latency-tolerant (asynchronous) applications such as tape backup. These solutions are limited to OC-12 bandwidth and are relatively expensive to implement.

Fibre Channel over IP is an ideal combination of technologies to address the dual requirements of storage networking and networking over distance. A mature technology optimized for storage-data movement within the campus and data center, Fibre Channel represents a major investment in software compatibility, interoperability, and proven applications for campus-based storage networking. Likewise, IP is a mature technology optimized for data movement across WAN distances. It represents a major investment in software compatibility, equipment interoperability, and proven applications for WAN-based data networking.

Today's Fibre Channel-over-IP solutions encapsulate Fibre Channel and transport it over a TCP socket. As in all IP networks, performance can vary based on the types of switches and routers, the number of hops the packets must traverse, and the level of congestion in the network. Today, storage transport performance over IP networks—especially over public networks—is limited due to the variable latency of service provider networks. As IP and Ethernet equipment continues to evolve, higher levels of Quality of Service (QoS), Cost of Service (CoS), provisioning, and circuit emulation should provide the latency guarantees required by synchronous storage applications. In controlled environments, these technologies might even improve the performance of IP networks. Regardless, Fibre Channel over IP is currently a very cost-effective technology for asynchronous applications such as remote data backup.

The Benefits of Fibre Channel Over IP (FCIP)

While multiple technologies are capable of interconnecting SANs, very few can be widely deployed in a cost-effective manner today. Because most organizations already have IP connections and significant experience with Ethernet and IP networks, they can usually leverage this equipment and expertise to help manage data in conjunction with Fibre Channel SANs. For example, IP connectivity provides the greatest flexibility at the lowest cost for latency-tolerant applications. As a result it can be used to back up data across a campus network, Metropolitan Area Network (MAN), or WAN. Moreover, this flexible technology can be deployed within a single enterprise or in an SSP multi-tenant environment. Today, many types of organizations are beginning to transport SAN storage over IP, especially for non–real-time data transfer. By combining the best of two mature technologies (as shown in Figure 1), FCIP solutions provide a more standardized and lower cost way to increase SAN interconnectivity for a variety of applications. In fact, FCIP includes full support for the Fibre Channel set of equipment and software. Organizations can seamlessly extend existing and planned Fibre Channel SANs over long distances through IP networks—thereby protecting significant investments in both technologies. In addition, FCIP provides a cost-effective way to achieve business protection (enabling such solutions as remote tape archiving).

iSCSI on
Terrorbyte liked to test out hot new backup technologies.
SSD ad - click for more info
SSD ad - click for more info
popular SSD articles on

SSD Myths - "write endurance" - In theory the problems are now well understood - but solving them presents a challenge for each new chip generation.

SSDs replacing HDDs? - That's a gross simplification.

the Top 20 SSD companies - updated quarterly.

the Fastest SSDs - in each form factor. Speed is still the #1 reason for buying SSDs.

the SSD Buyers Guide - summarizes key SSD market developments in the past quarter and has a top level directory of SSD content.

PCIe SSDs - news and market commentary. We've reported on PCIe SSDs since the first products shipped in 2007.

SSD market history - If you're new to the market it provides a clue to how much things have changed - and how fast (or how slowly).

is eMLC the true successor to SLC in enterprise flash SSD?- which so called "enterprise MLC" tastes the sweetest?

Clarifying SSD Pricing - where does all the money go? - Also includes SSD price examples.

the SSD Reliability Papers - links and abstracts of articles related to the subject of SSD reliability and data integrity.

the problem with Write IOPS in flash SSDs - this classic article helps you understand why all flash SSD benchmarks incorrectly suggest you're going to get much higher performance from some types of flash SSDs than you will actually see in your application..

Data Integrity Challenges in flash SSD Design - looks into the detailed techniques to achieve data reliability.

What's the best / cheapest - PC SSD? - I often get emails from readers who ask the above question.

this way to the Petabyte SSD - This article describes the future storage architecture of the datacenter, explains the economics of SSDs replacing HDDs for bulk storage and suggests a roadmap for getting there.

the 3 fastest PCIe flash SSDs list - or is it really lists?
"...the SSD market will be bigger in revenue than the hard drive market ever was."
How will hard drives fare in an SSD world?

SSD ad - click for more info
Figure 1
Figure 1: FCIP combines the best benefits of both IP and Fibre Channel technologies

One of the first implementations of FCIP is the interconnection of disparate SAN islands. FCIP can transport existing Fibre Channel services across the IP network such that two or more interconnected SANs can appear as a single large SAN and can be managed by traditional SAN management applications. In addition, FCIP enables SAN applications to support additional protocols without modification. These applications might include disk mirroring between buildings in a campus network or remote replication over the WAN. The type of applications utilized will be based on the distance the data must travel, the network bandwidth, and the QoS requirements and/or abilities of the network connection. While some implementations of FCIP are point-to-point "tunnels," the protocol does not require that the "gateways" support only point-to-point tunneling. The FCIP standard supports all Fibre Channel services, including FSPF routing algorithms, such that multiple logical links created from a single gateway can route Fibre Channel packets over the IP infrastructure. Not only is FCIP routable, but IP networks do not need to know anything about the packets being routed. The Fibre Channel services handle all routing between logical links, while the TCP protocol handles the delivery of packets to the specific gateway device.

The Emergence of the FCIP Standard

FCIP is defined as a tunneling protocol for connecting geographically distributed Fibre Channel SANs transparently over IP networks. It uses TCP/IP as the transport while keeping Fibre Channel fabric services intact. The FCIP standard is a cost-effective technology that uses widely deployed IP standards while enabling management of remote SANs through existing SAN management tools. This extremely powerful standard is designed to leverage the rapid advances in Gigabit Ethernet and emerging 10 Gigabit Ethernet technologies as well as tested local-loop and long-haul technologies. As bandwidth increases in local networks and prices continue to fall, FCIP becomes even more cost-effective.

FCIP is being standardized by the Internet Engineering Task Force (IETF) IPS Working Group, whose main objectives are to:

  • Specify the encapsulation for Fibre Channel frames being transported by TCP/IP.
  • Specify the use of the encapsulation to create a virtual Fibre Channel link that connects Fibre Channel devices and fabric elements.
  • Specify the TCP/IP environment for supporting virtual Fibre Channel links and providing the capabilities for tunneling Fibre Channel traffic over an IP-based network, which include security, data integrity, congestion, and performance.

To establish connectivity between remote SANs over LANs, MANs, or WANs, the FCIP standard relies on IP-based network services. FCIP also relies upon TCP/IP for congestion control and management and upon both TCP/IP and Fibre Channel for data error and data loss recovery.

A Powerful Standard Designed for Growth

Today, many organizations are deploying FCIP to enable SAN interconnectivity over longer distances for synchronous applications such as remote data backup (see Figure 2). As higher link speeds—including 10 Gigabit Ethernet and 10 Gigabit Fibre Channel—emerge, FCIP will be ideal for handling synchronous applications such as real-time mirroring. New applications such as the distribution of time-sensitive multimedia data and interactive multimedia Web applications will also be possible. Over the long term, SAN interconnectivity through FCIP is well positioned to provide a high-performance, high-reliability infrastructure that can help a wide range of organizations:

  • Leverage their existing technology investments.
  • Expand their service offerings to existing customers.
  • Bring services closer to end users while improving response times and quality of delivery.

While other technologies are still at the planning stages, FCIP is currently available. By leveraging existing Fibre Channel and TCP/IP investments and an organization's collective expertise, FCIP provides a much more cost-effective solution that can be widely deployed today.

For more information about the FCIP standard, visit

...SNIA profile

storage search banner

STORAGEsearch is published by ACSL