Copyright 2003 Storage Networking Industry Association. All Rights Reserved. SCSI - The Protocol for all Storage Architectures David Deming, President Solution Technology SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 2 SNIA Legal Notices The material contained in this tutorial is copyrighted by SNIA Member companies and individuals may use this material in presentations and literature under the following conditions: Any slide or slides used must be reproduced without modification SNIA must be acknowledged as the source of any material used in the body of any document containing material from these presentations This presentation is a project of the SNIA Education Committee SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 3 Objectives Brief understanding of SCSI and its history Understanding of industry standards and the role of the SCSI Architecture Understanding the SCSI Architecture model Understanding the I/O Operation and the importance behind SCSI Protocol Learn how parallel SCSI, Fibre Channel, and iSCSI rely on the SCSI Architecture to support storage applications SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 4 Brief History SCSI-1: In the beginning 1987 SCSI was adapted from the SASI Interface (1979) Specified a physical interface, transport protocol, and standard command for disk, tape, and other storage devices SCSI-2: Second coming of SCSI Speed and bus width enhancements Multi-threading commands and transport protocol cleanup Added more storage device type command sets SCSI-3: Today and beyond More speed: Ultra-2/Ultra-160/Ultra-320 Separation of Physical Interface, Transport Protocols, and the SCSI Command Sets SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 5 SCSI Today and the Future SCSI Language SCSI is the Language of Love Every architecture uses the SCSI command sets Every architecture uses the SCSI upper layer protocol SCSI Transport Defines protocol mapping and control of SCSI language Fibre Channel Protocol (FCP), Serial Storage Architecture (SSA), Serial Bus Protocol (SBP), iSCSI, Serial ATA (SATA), Serial Attached SCSI (SAS) SCSI Physical Architectures Serial SCSI the future of the SCSI physical interface Fibre Channel, SSA, 1394, ATAPI, TCP/IP Page 6 SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Introduction to Standards SCSI Standards and their impact on all storage interfaces SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 7 Standards Organizations T13 ATA T13 ATA IEEE-1394 (Firewire) IEEE-1394 (Firewire) International Organization for Standards (ISO) International Organization for Standards (ISO) American National Standards Institute (ANSI) American National Standards Institute (ANSI) INCITS INCITS Ethernet (802.x) Ethernet (802.x) IEEE IEEE iFCP iFCP IETF IETF T11 FC T11 FC T10 SCSI T10 SCSI Internet Protocol Internet Protocol iSCSI iSCSI FCIP FCIP SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 8 SCSI Standards Physical Interfaces Shared Command Set Architectural Model Device Specific Command Sets Transport Protocols SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 9 Protocol Service Model SCSI Application SCSI Application SCSI Application SCSI Application SCSI Transport Protocol Services SCSI Transport Protocol Services SCSI Transport Protocol Services SCSI Transport Protocol Services Interconnect Services Interconnect Services Interconnect Services Interconnect Services Application Layer Transport Protocol Layer Interconnect Layer Command Set Standards Transport Protocol Standards Interconnect Standards SCSI Transport Protocol SCSI Application Protocol Protocol Service Interface Interconnect Service Interface Client I/O System Server I/O System Application layer: Clients and servers that originate and process SCSI I/O operations by means of a SCSI application protocol Transport protocol layer: Services and protocols through which clients and servers communicate Interconnect layer: Services, signaling mechanism and interconnect subsystem needed for the physical transfer of data from sender to receiver. SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 10 SCSI Command Sets one of the most appealing feature behind the SCSI Architecture, i.e. the SCSI language includes every type of storage device manufactured today; disk, tape, CD, storage enclosures, etc manufacturer independent and are common for all types of devices SCSI device drivers and operating systems have utilized SCSI Command Sets since the late 80s can be used in any storage architecture including Fibre Channel, iSCSI, InfiniBand, ATA/IDE, SATA (Serial ATA), SAS (Serial Attached SCSI), 1394 (Firewire) Page 11 SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. SCSI Architectural Model Characteristics of the SCSI-3 Architecture SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 12 The SCSI Distributed Model Protocol Service Interface Service Delivery Subsystem SCSI is a client-server protocol. The client is called the Initiator (usually the OS I/O subsystem) and issues requests to the server. The server is called the Target (usually the SCSI controller that is part of a storage device) and receives, executes and returns Initiator requests and their associated responses. Server Request Server Response Client Server Client-Server Transaction Initiator Target SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 13 Client-Server Model A single Initiator can have multiple Application Clients. Targets have ONE Task Manager and one or more Logical Units (LU), which are numbered (LUN). The Task Manager: controls the sequencing of one or more tasks within a LU carries out the task management functions has the authority to modify Service Requests that have already been received by the target The Device Server processes operations and directs them to a specific LUN. Task Management Request Task Management Response Initiator Target Application Application Application Application Clients Task Manager Device Server Device Server Request Device Server Response Logical Units SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 14 Re-cap SCSI Terminology SCSI is a standard that defines an interface between an Initiator (usually a computer) and a Target (usually a storage device such as a hard disk, tape backup, or storage array). Interface refers to connectors, cables, electrical signals, optical signals and the command protocol that allow initiators and targets to communicate. Logical Units are a subset of Target devices which can allow for scalability. Initiator Target Interface Logical Units SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 15 SCSI Structural Model SCSI Device SCSI Device SCSI Device SCSI Device SCSI Device SCSI Device SCSI Device SCSI Device Service Delivery Subsystem Service Delivery Subsystem Domain I/O System Represents a view of the elements comprising a SCSI I/O system as seen by the application clients interacting with the system. The fundamental object is the SCSI Domain that represents an I/O system. a Domain is made up of SCSI devices and a Service Delivery Subsystem that transports Commands, Data, and related information SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 16 Initiator Device Model SCSI Initiator Device SCSI Initiator Device Initiator Device Name Initiator Device Name SCSI Initiator Port SCSI Initiator Port Application Client Application Client Initiator Port Name Initiator Port Name Initiator Port Identifier Initiator Port Identifier A SCSI Initiator device contains: Zero or more Initiator device names One or more SCSI Initiator ports each containing an Initiator Port Identifier and an optional Initiator Port Name One or more Application Clients SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 17 Target Device Model A SCSI Target device contains: Zero or more Target device names One or more SCSI Target ports each containing a Task Router, SCSI Target Port Identifier, and an optional Target Port Name One or more Logical Units SCSI Target Device SCSI Target Device Target Device Name Target Device Name SCSI Target Port SCSI Target Port Logical Unit Logical Unit Target Port Name Target Port Name Target Port Identifier Target Port Identifier Target Router Target Router SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 18 Logical Unit Model A Logical Unit contains: A Logical Unit Number If access controls are not in effect, one Logical Unit Number per Logical Unit; or If access controls are in effect, one Logical Unit Number per SCSI Initiator port that has access rights plus one default Logical Unit Number per Logical Unit; One or more Logical Unit names A Device Server A Task Manager; and One or more Task Sets each of which may contain zero or more untagged tasks or a combination of zero or more tagged tasks and zero or more untagged tasks. Logical Unit Logical Unit Device Server Device Server Task Set (Queue) Task Set (Queue) Task Manager Task Manager Untagged Task Untagged Task Tagged Task Tagged Task Logical Unit Name Logical Unit Name Logical Unit Number Logical Unit Number SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 19 Tasks Task = the work to be performed by the Logical Unit A single Command A group of linked Commands Two types of SCSI Tasks Tagged Untagged Untagged Task Represented by an I_T_L nexus Tagged Task Represented by an I_T_L_Q nexus Nexus Methodology used to uniquely identify an I/O Operation between an Initiator, Target, Logical Unit, and an optional Queue Tag Initiator Target Logical Units Task Sets Queue SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 20 Task Attributes There are four types of Task attributes that can affect how the Target executes each task Simple Target can execute in any order Target will typically apply a performance algorithm to numerous simple tasks Ordered Target must execute all ordered tasks in the order they are received Any task prior to ordered must be executed before ordered task Head of queue Informs Target to insert the task into the front of the queue Auto Contingent Allegiance (ACA) Used when the Target enters into an error condition for a command that has previously executed Page 21 SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. I/O Operations Characteristics of SCSI I/O Operations SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 22 I/O Operation Model There are two categories of Protocol Services: - Execute Command and Confirmation Services - Data Transfer Services There are three main phases of an I/O operation that includes a data transfer: 1.Command: Send required command and parameters via a Command Descriptor Block (CDB) 2.Data: Transfer data in accordance with the command 3.Status: Receive confirmation of command execution SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 23 SCSI I/O Operations At a minimum, SCSI I/O Operations consist of: An Initiator issuing a SCSI Command A Target returning completion Status There is no Data transfer between Initiator and Target Types of Commands that do not move Data Test Unit Ready Start/Stop Unit Rewind Interconnecting Subsystem Interconnecting Subsystem SCSI Command SCSI Status Initiator Target SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 24 Operations that move Data When an Initiator and Target need to exchange information They utilize a Data phase Data In transmits information from the Target to the Initiator Data Out transmits information from the Initiator to the Target Data can be transmitted all at once or take numerous Data phases to complete information transfers Types of Commands Read or Write Inquiry Interconnecting Subsystem Interconnecting Subsystem SCSI Command SCSI Status Read or Write Data Initiator Target SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 25 Command Descriptor Block A Command is executed by sending a Command Descriptor Block (CDB) to the Target For each CDB The first byte of the CDB is the Operation Code The last byte of the CDB is the Control Byte The format of the Operation Code and Control Byte is identical for every SCSI Command in the SCSI Universe bit 7 bit 6 bit 5 bit 4 bit 3 bit 2 bit 1 bit 0 byte 0 Operation code byte 1 Reserved (MSB) byte 2 Logical block address (if required) byte 3 (LSB)
byte 4 Transfer length (if required) Parameter list length (if required) Allocation length (if required) byte 5 Control byte
Example Six Byte CDB Informs the Target where to start the transfer How many blocks (512 bytes) to transfer How many bytes to transfer SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 26 Other CDB Formats bit 7 bit 6 bit 5 bit 4 bit 3 bit 2 bit 1 bit 0 byte 0 Operation code byte 1 Reserved Service Action (if required) byte 2 (MSB) byte 3 Logical block address (if required) byte 4 byte 5 (LSB) byte 6 Reserved byte 7 (MSB) Transfer length (if required) Parameter List Length (if required) byte 8 Allocation length (if required) (LSB) byte 9 Control byte
bit 7 bit 6 bit 5 bit 4 bit 3 bit 2 bit 1 bit 0 Ten Byte CDB Twelve Byte CDB CBDs can be: 10 bytes 12 bytes 16 bytes or even variable bytes in length byte 0 Operation code byte 1 Reserved Service Action (if required) byte 2 (MSB) byte 3 Logical block address (if required) byte 4 byte 5 (LSB) byte 6 (MSB) byte 7 Transfer length (if required) byte 8 Parameter list length (if required) byte 9 Allocation length (if required) (LSB) byte 10 Reserved byte 11 Control byte
SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 27 SCSI Commands all Devices There are standard SCSI Commands for: Disk Tape Storage Enclosures Disk Arrays CD WORM Media Changers Every device imaginable or not yet created Op. Code Command Name Type 00h TEST UNIT READY M 03h REQUEST SENSE Z 12h INQUIRY M 15h MODE SELECT(6) Z 18h COPY O 1Ah MODE SENSE(6) Z 1Ch RECEIVE DIAGNOSTIC RESULTS O 1Dh SEND DIAGNOSTIC O 39h COMPARE O 3Ah COPY AND VERIFY O 3Bh WRITE BUFFER Z 3Ch READ BUFFER O 4Ch LOG SELECT O 4Dh LOG SENSE O 55h MODE SELECT(10) Z 5Ah MODE SENSE(10) Z
SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 28 SCSI Status Any time a SCSI Command is sent to a Target The Initiator expects a completion Status This status can reflect successful or unsuccessful completion of the command The Status may indicate Busy or Not Ready Error condition exists for another command Targets task set is full Page 29 SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Parallel SCSI SCSI Application SCSI Application SCSI Application SCSI Application SCSI Transport Protocol Services SCSI Transport Protocol Services SCSI Transport Protocol Services SCSI Transport Protocol Services Interconnect Services Interconnect Services Interconnect Services Interconnect Services Command Set Standards SCSI-3 Parallel Interface SCSI-3 Parallel Interface SCSI Transport Protocol SCSI Application Protocol Protocol Service Interface Interconnect Service Interface Initiator Target SCSI-3 Parallel Bus SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 30 Parallel SCSI Characteristics Half-duplex, daisy chained, shared bus, requires signal termination Separate control and data signals Information can only go in one direction at a time Each piece (byte) of information if acknowledged Information transfers are interlocked A connection protocol is used before any information transfers Arbitration, Selection, and Message phases Uses SCSI IDs to identify devices Utilizes protocol bus phases to accomplish information transfers Command (CDB), Data, or Status Can multi-task by disconnecting SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 31 Read Command Example Initiator Target C o nn ectio n A rbitrate and S elect C D B 0x28 00 00 10 00 00 00 0A 00 00 C o m m an d P hase R equ ested R ead Info rm atio n D ata P h ase C o m p letio n Info rm ation 0x00 S tatu s P hase R equ ested R ead Info rm atio n D ata P h ase D iscon nectio n 0x00 M essag e P h ase Initiator connects to Target Once Initiator connects to Target, the Target is in control of the I/O Process Initiator sends CDB information via Command Phases Target returns requested information via Data Phase Target returns completion information via Status Phase Target Disconnects via Message Phase Page 32 SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Fibre Channel SCSI Application SCSI Application SCSI Application SCSI Application SCSI Transport Protocol Services SCSI Transport Protocol Services SCSI Transport Protocol Services SCSI Transport Protocol Services Interconnect Services Interconnect Services Interconnect Services Interconnect Services Command Set Standards Fibre Channel Protocol (FCP) Fibre Channel) SCSI Transport Protocol SCSI Application Protocol Protocol Service Interface Interconnect Service Interface Originator Responder Fibre Channel Serial Interface SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 33 FC Characteristics Full-duplex, bidirectional serial links 1 or 2 Gb/s now 4 and 10Gb/s later Up to 10km links some cases allow 100km Interconnected via switches and/or hubs Switched fabric if switches are used Hubs connect loop devices and requires arbitration Typically referred to as a Storage Area Network (SAN) Requires login/logout procedures Process Login for SCSI Layer Fabric Login if switches are used All information is transferred in packets (frames) SCSI CDB, Data, and Status information is wrapped in Fibre Channel frames Utilizes Sessions, Exchanges, and Sequences SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 34 Fibre Channel Hierarchy Session Frame 1 Sequence 1 Frame 2 Frame n Exchange =I/O Process Start of Frame 4 bytes Header 24 bytes Payload - up to 2112 bytes CRC 4 bytes Optional Header 64 Bytes StandardPayload 2048 Bytes End of Frame 4 bytes Frame 1 Sequence 2 Frame 1 Sequence 3 Frame 2 Frame n Frame 1 Sequence n Frame 1 Last Sequence Word bits 31 to 24 bits 23 to 16 bits 15 to 8 bits 7 to 0 0 R_CTL D_ID 1 Reserved S_ID 2 TYPE F_CTRL 3 SEQ_ID DF_CTL SEQ_CNT 4 OX_ID RX_ID 5 PARAMETER Mandatory 6 words used to: identifies type of frame, where it came from, where its going to, contents of payload, and other Sequence and Exchange identifiers SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 35 FCP Characteristics Session - no actual parallel SCSI equivalent Exchange = SCSI I/O Operation Sequence - no actual parallel SCSI equivalent FCP_CMD frame Equivalent to parallel SCSI Command Phase FCP_XFER_RDY frame No actual parallel SCSI Equivalent FCP_DATA frame Equivalent to parallel SCSI Data Phase FCP_RSP frame Equivalent to parallel SCSI Status Phase SCSI Function FCP Equivalent I/O Operation Exchange Protocol Service Request and Response Sequence Send SCSI Command Request Unsolicited command IU (FCP_CMD) Data deli very request Data descriptor IU (FCP_XFR_RDY) Data deli very action Solicited data IU (FCP_DATA) Send Command Complete Response Command status IU (FCP_RSP) REQ/ACK for Command Complete Confirmation IU (FCP_CONF) SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 36 FCP_CMD frame The FCP_CMND frame is normally sent as a single-frame sequence, which is the first sequence of an exchange. The FCP_CMND contains all the necessary information to start the I/O Process. Exec Mgmt Codes Task Mgmt Flags Task Codes reserved Single Level LUN Byte 3 Byte 2 Byte 1 Byte 0 Byte 3 of CDB Byte 2 of CDB Byte 1 of CDB Byte 0 of CDB Byte 7 of CDB Byte 6 of CDB Byte 5 of CDB Byte 4 of CDB Byte 11 of CDB Byte 10 of CDB Byte 9 of CDB Byte 8 of CDB Byte 15 of CDB Byte 14 of CDB Byte 13 of CDB Byte 12 of CDB Data Length LSB Data Length Data Length Data Length MSB FCP_LUN 0 FCP_CNTL FCP_CDB FCP_DL 1 3 4 5 6 2 7 Word SOF 4 bytes Header 24 bytes Payload 24 bytes CRC 4 bytes EOF 4 bytes SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 37 FCP_XFER_RDY frame FCP_XFER_RDY specifies the relative offset and length of any following data sequence Optional for READ operations Required for WRITE operation SOF 4 bytes Header 24 bytes Payload 12 bytes CRC 4 bytes EOF 4 bytes reserved reserved reserved reserved lsb LENGTH BURST msb lsb OFFSET RELATIVE msb Byte 3 Byte 2 Byte 1 Byte 0 0 1 2 Word SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 38 FCP_DATA frame The Data Payload can be variable up to 2048 bytes in length. SOF 4 bytes Header 24 bytes Payload 128 to 2048 bytes CRC 4 bytes EOF 4 bytes FCP_DATA 128 to 2048 bytes FCP_DATA 128 to 2048 bytes SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 39 FCP_RSP frame The FCP_RSP frame defines the data structure containing the SCSI status byte, SCSI autosense data, and the FCP response code. This frame signals the completion of a SCSI command or task management function. SOF 4 bytes Header 24 bytes Payload Up to 256 bytes CRC 4 bytes EOF 4 bytes SCSI Status byte Validity Flags reserved reserved Byte 3 Byte 2 Byte 1 Byte 0 LSB MSB LSB MSB LSB MSB RSP_CODE reserved reserved reserved reserved reserved reserved reserved Reserved 0 FCP_STATUS FCP_RESID 1 3 4 5 6 2 7 Word FCP_SNS_LEN FCP_RSP_LEN FCP_RSP_INFO FCP_SNS_INFO ... ... Byte 1 Sense Data Byte 0 Sense Data Byte n Sense Data ... ... ... 8 n SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 40 Write Command Example Before devices can communicate in FC they must establish a session via login protocol Originator sends Write command to responder Responder replies with a ready to receive response Originator sends Data frames When all Data is transferred, Responder sends Status to Originator Session could end at this point but typically doesnt Originator Responder Establish Session (outside scope of this presentation) F C P _C M D F ram e (W rite) C o m m an d P hase F C P _X F E R _R D Y F ram e (R espo nd er is read y fo r D ata) F C P _R S P F ram e S tatu s P hase F C P _D A T A F ram e D ata P hase F C P _D A T A F ram e D ata P hase Remove Session (outside scope of this presentation) E x c h a n g e S e q u e n c e Page 41 SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. iSCSI IP Network IP Network FC SAN FC SAN iSCSI Gateway Interconnect Services Interconnect Services Data Link Data Link IP IP TCP TCP Interconnect Services Interconnect Services IP Network Data Link Data Link SCSI Application SCSI Application SCSI Application SCSI Application iSCSI Transport Protocol Services iSCSI Transport Protocol Services iSCSI Transport Protocol Services iSCSI Transport Protocol Services Command Set Standards iSCSI Protocol Standard Internet Engineering Task Force Standards iSCSI Transport Protocol SCSI Application Protocol Protocol Service Interface Interconnect Service Interface Initiator Target IP IP TCP TCP SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 42 iSCSI Characteristics iSCSI is a SCSI transport protocol for mapping of block oriented storage data over TCP/IP networks. Storage generally identified for this application include disk, tape arrays and tape libraries. IP networks most applicable for this purpose are Gigabit Ethernet and in the future 10 Gigabit Ethernet, however for low performance applications 10/100bT will work. The iSCSI layer encapsulates the SCSI CDB into a iSCSI Protocol Data Unit (PDU) and forwards it to the Transmission Control Protocol (TCP) layer. The communications between the Initiator and Target will occur over one or more TCP connections. The TCP connections form a session and will carry the iSCSI PDUs. The sessions are given an ID called a Connection ID (CID). There are two parts of the ID, Initiator Session ID (ISID) and Target ID (TSID) and together make up an I_T nexus . SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 43 iSCSI Encapsulation Data Data Initiator Target Data Interconnect Services Interconnect Services Data Link Data Link IP IP TCP TCP Interconnect Services Interconnect Services Data Link Data Link SCSI Application SCSI Application SCSI Application SCSI Application iSCSI Transport Protocol Services iSCSI Transport Protocol Services iSCSI Transport Protocol Services iSCSI Transport Protocol Services iSCSI Transport Protocol SCSI Application Protocol Protocol Service Interface Interconnect Service Interface IP PDU Header PDU Data IP Network IP TCP TCP Ethernet Header Ethernet Frame IP Header IP Packet (Datagrams) F C S TCP Header TCP Segment C H K SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 44 SCSI Command PDU Byt e 0 Byt e 1 Byt e 2 Byt e 3 0 .|I| 0x01 F|R|W|. .| ATTR Reserved 8+ Logical Unit Number (LUN) 16 Initiator Task Tag 48 AHS (if any) + Header-Digest (Optional) + DataSegment (Command Data)+ Data-Digest (Optional) 28 ExpStatSN 32+ SCSI Command Descriptor Block (CDB) 4 TotalAHSLength DataSegmentLength 20 Expected Data Transfer Length 24 CmdSN Byt e 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 45 SCSI Response PDU Byt e 0 Byt e 1 Byt e 2 Byt e 3 32 MaxCmdSN 36 ExpDataSN or Reserved 40 Bi-directional Read Residual Count or Reserved Byt e 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 4 8+ 16 20 24 28 44 48 + . |. | 0x21 1|. . |o|u|O|U|. Response St at us TotalAHSLength DataSegmentLength Reserved Initiator Task Tag SNACK Tag or Reserved StatSN ExpCmdSN Residual Count or Reserved Header-Digest (Optional) Data Segment (Optional) + Data-Digest (Optional) SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 46 SCSI Data-out PDU Byt e 0 Byt e 1 Byt e 2 Byt e 3 28 ExpStatSN 32 Reserved 40 Buffer Offset Byt e 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 4 8+ 16 20 24 36 44 48 + . |. | 0x05 F| Reserved TotalAHSLength DataSegmentLength LUN or Reserved Initiator Task Tag Target Transfer Tag or 0xFFFFFFFF Reserved DataSN Reserved Header-Digest (Optional) DataSegment + Data-Digest (Optional) SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 47 SCSI Data-in PDU Byt e 0 Byt e 1 Byt e 2 Byt e 3 28 ExpCmdSN 32 MaxCmdSN 40 Buffer Offset Byt e 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 4 8+ 16 20 24 36 44 48 + . |. | 0x25 F|A|0 0 0|O|U|S Reserved Status or Rsvd TotalAHSLength DataSegmentLength LUN or Reserved Initiator Task Tag Target Transfer Tag or 0xFFFFFFFF StatSN or Reserved DataSN Residual Count Header-Digest (Optional) DataSegment + Data-Digest (Optional) SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 48 Ready To Transfer (R2T) Byt e 0 Byt e 1 Byt e 2 Byt e 3 0 .|.| 0x31 1| Reserved 16 Initiator Task Tag 28 ExpCmdSN 32 MaxCmdSN 36 R2TSN 48 Header-Digest (Optional) 40 Buffer Offset 44 Desired Data Transfer Length 4 TotalAHSLength DataSegmentLength 8+ LUN 20 Target Task Tag 24 StatSN Byt e 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 49 Write Command Example Before devices can communicate in iSCSI they must establish a session via login protocol Initiator sends Write command to Target Target replies with a ready to transmit R2T response Initiator sends Data frames When all Data is transferred, Target sends Status to Initiator Session could end at this point but typically doesnt Initiator Target Establish Session (outside scope of this presentation) C o m m an d P D U (W rite) C o m m an d P hase R 2T P D U (Targ et is read y fo r D ata) R espo n se P D U S tatu s P hase D ata-O ut P D U D ata P hase D ata-O ut P D U D ata P hase Remove Session (outside scope of this presentation) S e q u e n c e SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 50 Read Command Example Before devices can communicate in iSCSI they must establish a session via login protocol Initiator sends Read command to Target Target sends Data frames When all Data is transferred, Target sends Status to Initiator Session could end at this point but typically doesnt Initiator Target Establish Session (outside scope of this presentation) C o m m an d P D U (R ead ) C o m m an d P hase R espo n se P D U S tatu s P hase D ata-In P D U D ata P h ase D ata-In P D U D ata P h ase Remove Session (outside scope of this presentation) S e q u e n c e SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 51 Summary SCSI is the language of LOVE Every storage architecture utilizes the SCSI model SCSI Commands sets are available for every type of storage device even including host-to-host An I/O Process is interface independent and is made up of: Command Phase Optional Data Phase and a Status Phase SCSI language is here to stay and will be integrated in every storage architecture For more information on standards see: incits.org, t10.org, t11.org, t13.org, ietf.org, scsita.org, fibrechannel.org SCSI The Protocol for All Storage Architectures Copyright 2003 Storage Networking Industry Association. All Rights Reserved. Page 52 Credit Slide Please send any questions or comments on this presentation to: snia- snw-scsi@snia.org Many thanks to the following SNIA members for their contributions to this tutorial: Developer: David Deming, Solution Technology, ddeming@soltechnology.com Reviewers: Rob Elliott, HP, elliott@hp.com LeRoy Budnik, Knowledge Transfer, lbudnik@ktilwf04.knowledgetransfer.net Charles Curtis, HP, charles.curtis@hp.com Antolin Agatep, Xilinx, Antolin.Agatep@xilinx.com