MMUSIC Working Group X. Mingqiang Internet-Draft D. Komiya Expires: April 27, 2006 S. Kawaguchi M. Rahman B. Kumar E. Shim Panasonic October 24, 2005 Extensions of Session Description Protocol (SDP) for Seamless Session Mobility draft-mingqiang-mmusic-session-mobility-attribute-01.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on April 27, 2006. Copyright Notice Copyright (C) The Internet Society (2005). Abstract Session mobility allows a user to transfer an ongoing communication session from one device to another device. Seamless session mobility is important for real time multimedia applications. This Mingqiang, et al. Expires April 27, 2006 [Page 1] Internet-Draft Seamless Session Mobility October 2005 document describes requirements for seamless session mobility, which will guide development of seamless session transfer mechanisms and associated Session Description Protocol (SDP) extensions. 1. Introduction IP-based multimedia streaming applications are important usages of mobile devices. Mobile devices provide convenience of mobility but have limited capability compared to stationary devices; for example, smaller screens and computing power than stationary devices. Session mobility allows a user to transfer an ongoing communication session from one device to another device. Session mobility between mobile devices and stationary devices enable users to use best devices at hand without terminating existing sessions and starting new ones. There can be various forms of disruptions in media play during session transfer. Some portion of media may be lost or skipped. There can be noticeable and annoying pause or delay until media play starts in the target device. The general approach to achieve session mobility in SIP [1] including discovery process to locate transfer target devices, and signaling needed for the transfer of session has been described in [6] . In [6], the following is described as a requirement of session mobility: "Session transfer should be as seamless as possible. It should involve minimum disruption of media flow and should not appear to the remote participant as a new call." 3GPP proposed seamless session mobility as a part of the vision of 3GPP All-IP Networks in [7]. Starting a streaming application at the receiver usually takes some time which MAY involve reading system parameters and initializing local buffer. Streaming clients typically have a receiver buffer that is capable of storing a relatively large amount of data [5] initially, when a streaming session is established, a client does not start playing the stream back immediately. Rather, it typically buffers the incoming data in a play back buffer for a short duration. This duration could range from a few milliseconds to a few seconds depending upon the property of the decoder at the receiver. This buffering at the receiver helps maintain continuous playback. In the event of occasionally increased transmission delays or network throughput drops, the client can decode and play buffered data. Otherwise, without initial buffering, the client has to freeze the display, stop decoding, and wait for incoming data. Therefore, in order for a session transfer to be seamless, we need to parallelize the process of selecting the transfer target device and creating the necessary playback buffer at the target device to eliminate pre-roll delay. We define pre-roll delay as the time that Mingqiang, et al. Expires April 27, 2006 [Page 2] Internet-Draft Seamless Session Mobility October 2005 it takes for the receiver buffer to fill to a desired level so that playback can begin instantly. The SIP REFER based session transfer mechanism as specified in [4] clearly is inadequate to deal with the lapse caused by the disruption of media stream at the new receiver. The purpose of this document is to identify requirements of seamless session transfer and extensions of Session Description Protocol (SDP) [3] useful to implement seamless session transfer. It will take development of thorough seamless session transfer mechanisms to identify all necessary extensions of SDP and other protocols, which is a challenge requiring more efforts. The current version of this document defines performance metrics that can be used to assess sesson transfer mechanisms and requirements of seamless session transfer mechanisms. Extenstions of SDP will be described in the future version. 2. Terminology In this document, the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" are to be interpreted as described in BCP 14, RFC 2119 [2]. Device: A software or hardware appliance, which is represented by a SIP UA. PAN (Personal Area Network): PAN is a collection of one or more logically associated devices that share the same physical communication medium within a close proximity of each other (e.g., network formed by devices that can be accessed by near field radio like Bluetooth). Session: A session is a set of senders and receivers and the data stream flowing from senders to receivers. A multimedia conference is an example of a session. Seamless: A user experience that is unaffected by changes in the mechanisms used to provide services to a user. Note: The determination of whether something satisfies the requirement for being seamless or not is dependent on the user's perception of the service being received and not necessarily the technology used to provide the service. Session mobility: The ability for a communication session to be moved from one device to another under the control of the user. Mingqiang, et al. Expires April 27, 2006 [Page 3] Internet-Draft Seamless Session Mobility October 2005 Session Transfer: The act of moving a communication session from one device to another under the control of the user. Seamless Session Transfer: Session transfer in which a seamless session with no perceivable interruption from a user perspective is maintained during a change in access system. Session Source: The device that was holding the session before session transfer. Session Destination: The device to which the session is transferred. Correspondent Node (CN): The device participating in the session with the Session Source. Media Synchronization: Coordination of the media playback timings. Time Gap: The time difference between the Session Destination's playback start and the Session Source's playback stop. Time Shift: The time difference between the Session Source and the Session Destination for the same point of the stream. Media Gap: The difference of the first frame played by the Session Destination and the last frame played by the Session Source. Response Delay: The time duration from when the human user requests session transfer to when the Session Destination plays its first frame. Pre-roll Delay: The time duration a media player spends to buffer media data before starting playback. Mingqiang, et al. Expires April 27, 2006 [Page 4] Internet-Draft Seamless Session Mobility October 2005 +--------------+ | | 3 | | +--------------+ | | 4 | ------------------------------------------|Tc +--------------+ | | 5 | | +--------------+ | | 6 | | +--------------+ | | 7 | | +--------------+ Session Source stops playback ------------|Ts | (8) | | +--------------+ ------------------------------------------|Tv Session Source | | | +--------------+ -----------|Td | 9 | | +--------------+ | | 10 | | +--------------+ | | 11 | | +--------------+ | | 12 | | +--------------+ | | | v | +--------------+ | | | | +--------------+ | Session Destination | | | V time Figure 1: A media playback scenario during session transfer Figure 1 illustrates a media playback scenario during session transfer, in which a command to execute a session transfer was issued by a human user at time Tc, the Session Source stopped media playback at Ts and its last frame was frame 7, and the Session Destination started playback at Td and its first frame was frame 9. Frame 8 was not played by either device. Tv is the time the frame 9 should be played if the media stream was played continously by the Session Mingqiang, et al. Expires April 27, 2006 [Page 5] Internet-Draft Seamless Session Mobility October 2005 Source. In this scenario, Time Gap is (Td - Ts), Time Shift is (Td - Tv), Media Gap is 9 - 7 + 1 = 1 (frame), and Response Delay is (Td - Tc). 3. Issues with Session Transfer This section describes potential problems in session transfer that can cause disruption to the user experience. If the Session Destination does not start playing media a noticeable amount of time after the Session Source stopped playing media, continuity of media breaks. In particular, if the session is for interactive communications like VoIP, media during the delay will be lost. If a noticeable amount of media is skipped, i.e., neither the Session Source nor the Session Destination plays some portion of media, the user experience can degrade substantially. For media like music, the impact of media loss is significant. If the Session Source and the Session Destination play different portion of media at the same time, it causes a problem, too. For example, different sounds from the two devices will mix and hamper user's reception significantly. In an ideal session transfer, the Session Destination will start playing media without any media loss or duplication right after the Session Source stops playing media. Even if the Session Source plays media until the Session Destination is ready to play media and thus there is no loss or duplication or media or time gap, a user may not tolerate if the Session Destination's playing media is delayed beyond a certain time period. How much response delay is tolerable varies depending on situations and users. Studies show that synchronous end-to-end (round-trip) delays that consistently exceed about 250 ms are often unacceptable from the user point of view when the interaction is conducted in the key-stroke-by- keystroke mode [8][9][10]. [11] states that time delay or high response time is a major problem of most user interfaces on the World Wide Web and gives the following guidelines regarding response times: Approximately one tenth of a second is the limit for having the user feel that the system is reacting instantaneously. Consequently, no special feedback is necessary except to display the result; Approximately one second is the limit for the user's flow of thought to remain uninterrupted, even though the delay is noticed by the user. With delays of more than 0.1 but less than 1.0 Mingqiang, et al. Expires April 27, 2006 [Page 6] Internet-Draft Seamless Session Mobility October 2005 second, the user does lose the feeling of operating directly on the data, but no special feedback is necessary; Approximately ten seconds is the limit for keeping the user's attention focused on the dialogue. For longer delays, users turn to other tasks while waiting for the computer to finish. It is difficult to decide the tolerable response delay limit of session transfer before conducting an experiment with a sample system and a significant number of users. But the above studies show it is preferable to bound the response delay in session transfer below a second. MPEG families such as MPEG1,2 and 4 are widely used for video streaming applications these days and have significant commonality with RealVideo from Real Networks, Inc. and Windows Media Video from Microsoft that are widely used in the Web. All these video compression techniques use motion prediction and generate video frames that depend on adjacent video frames for decoding. These MPEG-like video formats raise another challenge for session transfer. What happens if a MPEG video streaming session is transferred at a random point of time? If the Session Destination receives video data from an intermediate frame of a GOP but not the first frame (I frame), the Session Destination cannot decode the video frames within the GOP. In such a simple session transfer scenario, video play is disrupted until the Session Destination reaches the beginning of the next GOP. The span of damage on video play in the Session Destination depends on the location of the first video frame it received within the first GOP. In average, we can say a half of GOP won't be played properly in the Session Destination. The degree of this damage depends on the length of the GOP, which depends on specific applications or encoder settings. Literature says a GOP consists of 15 to 35 frames or 8 to 24 frames. The frame rate for NTSC is 30 frames/sec and for PAL is 24 frames/sec. One can infer that a typical GOP is from a half second to a second in time length from the above literature. But there are lots of low frame rate video streams as well in the Internet, which makes it difficult to infer typical time length of a GOP. Nevertheless, we can notice that a GOP time length is easily longer than half a second and a simple session transfer can cause at least a quarter second of video disruption. In the worst cases, there can be disruption in video play for a second or longer, which will be noticeable to most users. The problem of media synchronization in session transfer is how to coordinate the Session Destination and the Session Source, in particular, their handling of media data, to minimize the disruption. Mingqiang, et al. Expires April 27, 2006 [Page 7] Internet-Draft Seamless Session Mobility October 2005 4. Requirements for Seamless Session Transfer This section describes requirements for media synchronization in multimedia streaming session transfer. 4.1 Minimal Time Gap A large time gap means that there is a long pause in media playback. There SHOULD be minimal time gap between the first moment of video display by the Session Destination and the last moment of video display by the Session Source. 4.2 Minimal Media Gap There SHOULD be minimal media gap between the first frame to be displayed by the Session Destination and the last frame by the Session Source. One can think of a negative media gap. For example, if the last frame for the Session Source is frame 9 and the first frame for the Session Destination is frame 6, the media gap becomes -3, which means three frame overlap between the two devices. A positive media gap means some portion of media is skipped without being played by either the Session Source or the Session Destination. The idea media gap is zero. There should be minimal media gap (in the absolute value) between the Session Destination and the Session Source, that is, minimal media skipping or overlapping. 4.3 Minimal Response Delay There SHOULD not be too much response delay from the human user's perspective in session transfer. The above three requirements are fundamental and common requirements for seamless session transfer in general. Time Gap, Media Gap and Response Delay can be common criteria in evaluating the performance of session transfer mechanisms. Minimal Time Shift can be another requirement of seamless session transfer. But Time Shift can be derived from Time Gap and Media Gap. Any two of these three metrics can be chosen as performance metrics of session transfer in addition to Response Delay. Each of the following four requirements can be an issue in some cases but not in every case. 4.4 Supporting MPEG-like media Seamless session transfer schemes SHOULD support MPEG-like video that uses prediction-based coding. As described earlier, MPEG video requires more sophisticated coordination between the Session Source Mingqiang, et al. Expires April 27, 2006 [Page 8] Internet-Draft Seamless Session Mobility October 2005 and the Session Destination than other simpler video formats. The Session Destination is supposed to handle from the beginning of a GOP or should able to decode even from an intermediate frame of the first (partial) GOP it handles. 4.5 Supporting Different Buffer Sizes The Session Source and the Session Destination may have different capabilities. An important capability is the media playback buffer size. Simply transferring the whole buffer content from the Session Source to the Session Destination does not work if the Session Destination has a smaller buffer because it will cause buffer overflow in the Session Destination and eventually media data loss. Seamless session transfer schemes SHOULD work when the Session Destination and the Session Source have different playback buffer sizes. 4.6 Supporting Different Network Characteristics The network environment of the devices affects media streaming. An example of such a situation is when the Session Source is attached to the network backbone via WLAN and a corporate broadband access network and the Session Destination (mobile handset) is attached via a 3G cellular network which provides smaller bandwidth. This access network bandwidth difference affects the time to achieve buffering in the media buffer. Seamless session transfer schemes SHOULD work when the Session Source and the Session Destination have different network environments. 4.7 Supporting Media Characteristics Change Transcoding technologies enable change of media characteristics of the same media content such as resolution or bit rate to optimize the media quality in heterogeneous network environments and device capabilities. Transcoding techniques were widely exploited to support adjustable media streaming in heterogeneous networks like the Internet. If the streaming application system has the capability of changing dynamically media characteristics during a session and performs the change when the Session Destination takes over the session from the Session Source, the media data sent to the Session Source may not fit into the Session Destination at all.Seamless session transfer schemes SHOULD work when media characteristics change during session transfer. Mingqiang, et al. Expires April 27, 2006 [Page 9] Internet-Draft Seamless Session Mobility October 2005 5. IANA Considerations There is no need for IANA consideration from the current version of this document. 6. Security Considerations There are some security concerns that must be addressed while a Mobile Node intends to establish communication sessions with devices in PAN. The Mobile Node needs to authenticate itself with the devices in PAN in order to limit the accessibility of devices in PAN by unauthorized users. Some of the security concerns are already addressed in [2] and additional security concerns will be addressed in the future revision. 7. References 7.1 Normative References [1] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, "SIP: Session Initiation Protocol", RFC 3261, June 2002. [2] Bradner, S., "Key words for use in RFCs to Indicate Requirements Levels", RFC 2119, March 1997. [3] Handley, H. and V. Jacobson, "SDP: Session Description Protocol", RFC 2327, April 1998. [4] Handley, H., "The Session Initiation Protocol (SIP) REFER Method", RFC 3515, April 2003. [5] Wenger, S., Hannuksela, M., Westerlund, T., and D. Singer, "RTP Payload Format for H.264 Video", RFC 3984, February 2005. 7.2 Informative References [6] Shacham, R., Schulzrinne, H., Thakolsri, S., and W. Kellerer, "Session Initiation Protocol (SIP) Session Mobility", draft-shacham-sippin-session-mobility-01 (work in progress), July 2005. [7] Sancho, ed., "All-IP Network (AIPN) Feasibility Study, Release 7", 3GPP TR22.978, 2005. [8] Bitzer, M. and D. Bitzer, "Teaching nursing by computer: an evaluation study", Computers in Biology and Medicine 3, 187- 204, 1973. Mingqiang, et al. Expires April 27, 2006 [Page 10] Internet-Draft Seamless Session Mobility October 2005 [9] Kauer, P., "An Analysis of North Carolina State University's Network Performance", M.S. Thesis Department of Computer Science, NCSU, 1995. [10] Dixit, P., "Quality of Service Modeling ofr Wide Area Network- Based Systems,", Ph.D. Thesis North Carolina State University, May 1998. [11] Nielsen, J., "Designing Web Usability: The Practice of Simplicity", New Riders Publishing Indianapolis, USA, 1999. Authors' Addresses Xu Mingqiang Matsushita Electric Industrial Co., Ltd.(Panasonic) 4-5-15 Higashi-shinagawa Shinagawa-ku, Tokyo Japan Email: xu.mingqiang@jp.panasonic.com Daisaku Komiya Matsushita Electric Industrial Co., Ltd.(Panasonic) 4-5-15 Higashi-shinagawa Shinagawa-ku, Tokyo Japan Email: komiya.daisaku@jp.panasonic.com Sachiko Kawaguchi Matsushita Electric Industrial Co., Ltd.(Panasonic) 4-5-15 Higashi-shinagawa Shinagawa-ku, Tokyo Japan Email: kawaguchi.sachiko@jp.panasonic.com Mingqiang, et al. Expires April 27, 2006 [Page 11] Internet-Draft Seamless Session Mobility October 2005 Mahfuzur Rahman Panasonic Digital Networking Laboratory Two Research Way, 3rd Floor Princeton, NJ 08540 USA Email: mahfuz@research.panasonic.com Brijesh Kumar Panasonic Digital Networking Laboratory Two Research Way, 3rd Floor Princeton, NJ 08540 USA Email: kumar@research.panasonic.com Eunsoo Shim Panasonic Digital Networking Laboratory Two Research Way, 3rd Floor Princeton, NJ 08540 USA Email: eunsoo@research.panasonic.com Mingqiang, et al. Expires April 27, 2006 [Page 12] Internet-Draft Seamless Session Mobility October 2005 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Disclaimer of Validity This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Copyright Statement Copyright (C) The Internet Society (2005). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society. Mingqiang, et al. Expires April 27, 2006 [Page 13]