Ensuring high quality video is a critical component of any successful IP-based video offering. Managing video quality is essential for differentiating services, reducing customer churn and enabling faster service deployments. Service providers face a number of challenges, not only in rolling out video services as such, but also in selecting and deploying the most appropriate monitoring solution to meet their specific needs. To assist this effort, this practical guide outlines the top ten things to consider before deploying a video monitoring solution, providing recommendations and approaches for each issue.
1. Implement a QoE-based Monitoring Solution
Most triple-play service assurance solutions available today were originally designed with the goal of network performance monitoring in mind. These offerings have recently tried morphing themselves into video quality monitoring solutions by analyzing network issues and attempting to predict actual end-user video quality. Impairments to video quality however, come from a wide range of sources (source video, encoders, network, and decoders/CPE), and monitoring only network metrics renders these solutions ineffective. Moreover, even if network issues exist, raw network metrics do not provide accurate assessment of service quality, leaving you with little insight into your customers’ actual video experience. Real time video presents new challenges that traditional network monitoring solutions do not address.
Network QoS approaches, like MDI, only look at network metrics (delay, jitter, loss, and bandwidth) with some basic transport stream (TS) header analysis and they are not an accurate representation of end-user experience. Video QoE (quality of experience) solutions should dig deep into the content, determine impacts of network impairments on content, and model the human vision system for an accurate assessment of end-user video quality.
Unlike QoS approaches, which report network metrics, QoE approaches report metrics that are relevant to the end-user experience and meaningful to operators attempting to understand them. In addition to Mean Opinion Score (MOS) – a high level 1-5 representation of overall video quality – some examples of QoE metrics are listed in Figure 1 above.
Figure 2 below shows the most common video impairments observed by IPTV operators. As the chart indicates, the most common and major issues seen today turn out to be rather basic problems (black screen, video freeze, lost audio, etc.). Operators should be sure to deploy a QoE solution that addresses these issues first and foremost.
2. Understand the End User Experience
One of the foremost challenges in controlling video service quality is understanding what’s actually happening with the video as the user views it. Unfortunately today, short of conducting surveys and interviews, operators lack the tools necessary to obtain an accurate assessment of video quality as perceived by the end user. In addition, diagnosing and troubleshooting are equally challenging tasks since relevant information generally doesn’t exist.
The main objective in any monitoring program is to understand what’s actually happening with the video when it reaches the customers’ home. This includes all aspects of the user experience – first and foremost, these are video and audio quality, but they also include lip-sync and user interactions such as zap times. After understanding the issues, subsequent steps can be implemented to diagnose, troubleshoot and isolate root causes. A video monitoring solution should achieve all these objectives.
3. Monitor the Headend
Headend problems are significant since these generally represent major issues and service degradations affecting many, if not all, customers and need adequate detection mechanisms. Furthermore, a lot of processing takes place at the headend that can potentially affect the video. It is here that most of the video encoding, transcoding, rate shaping, program insertion, multiplexing, DRM/encryption, etc. take place. As a result, common problems at the headend include video content impairments, some of which are listed below.
Common Headend Issues
Pre-Encoder (From content provider)
— Color issues
— Video freeze, jerkiness
— Missing audio
— Audio loudness
— Degraded video quality
— Compression impairments
— Bitstream (News - Alert)/transport stream issues
— Codec specific issues
— Streamer impairments
— Audio/video out of sync
— Degraded audio quality
High performance headend analyzers are needed to examine, in real time, the quality and integrity of source video and detect transcoding and encoding impairments. Such monitoring solutions can also reduce the need for “golden eye” operations personnel by automatically validating and highlighting the presence of major issues. Network probes cannot perform these measurements.
4. Home: The Missing Link
Last mile and home network problems account for up to 75% of all quality issues seen by service providers and must be effectively addressed. A monitoring solution is not end-to-end unless it effectively covers this network demarcation with a CPE-based solution.
Hand-held solutions used by technical field personnel are reactive approaches which require truck rolls – a requirement these end-to-end monitoring solutions should be designed to reduce. Deploying hardware probes in all DSLAM
sites is not cost effective either. More proactive solutions are needed to scale seamlessly as subscribers are added to the network.
Monitoring solutions that address CPE monitoring requirements effectively have to sit in the STB and home gateways for complete coverage of the problematic last mile and the home. They should proactively monitor the CPE and ideally be able to identify problems even before they are noticed or reported by the end-user. They can be resident in the device or deployed remotely in an on-demand fashion for problem diagnosis. This greatly reduces truck rolls and site visits. Due to the potentially large number of clients, such a solution must scale well to handle a growing network without prohibitive cost or complexity. To achieve cost effective scalability, monitoring solutions for the home should involved little or no hardware to minimize extra equipment installations and ultimately capital and operational expenditures.
5. Monitor Encrypted Environments
Video content being distributed over an IP
network is often encrypted, making it difficult to dig into video content to determine the impact of network impairments on actual video traffic. Unfortunately, the impact of network impairments is content dependent, so it is vital to understand how the video is affected by any network impairment.
While the IP network in general is not the largest culprit in terms of contributing to overall video quality issues, there are various reasons why monitoring this part of the network is important. Operations teams, responsible for network demarcations where encrypted traffic flows, need to validate the behavior of their equipment and in an encrypted network they often struggle to do so. Also, being part of the end-to-end delivery chain, the IP network does contribute to video errors and tools are needed that can operate in encrypted environments to provide visibility into where errors are coming from and information for diagnosis and troubleshooting purposes.
Solutions must be implemented to provide operators with the right visibility into the performance of their network areas of responsibility. To provide the most accurate assessment of video quality based on errors from the IP network, solutions must be considered that effectively determine the impact of network impairments on video content – even in an encrypted environment.
6. Pay Attention to Standards
The key QoE standard activities focus on the definition of methods and metrics for QoE measurements. The need for standardization comes from the fact that such metrics have to be meaningful, reliable and reproducible. Meaningful refers to the selection of the right metrics and impairments that are important for video monitoring. Reliable means that the metrics can be trusted to measure video QoE correctly and accurately, similar to the QoE reported by an actual subscriber. Reproducible means that the values report by the metrics have to be well-defined and can be compared between different tools and vendors.
Some of the standardization work focuses on the prediction of MOS (Mean Opinion Score), which is a quality rating obtained viewers watching many different video clips. Evaluation is done using large databases and standardized subjective experiments. Just as important, however, is the standardization of measurements of individual video impairments such as the ones mentioned earlier, covering both content and network issues, as these are particularly useful in determining the root cause of potential problems with the service.
These activities are ongoing within the Video Quality Experts Group (VQEG), the International Telecommunications Union (ITU), the ATIS IPTV (News - Alert) Interoperability Forum (IIF), the Video Services Forum (VSF), and represent convergence of the industry around the most relevant and important QoE standards, which are to be expected by 2009 or 2010.
What does this mean for a service provider? Most importantly, providers should give input to the standards groups in terms of their requirements and needs. They should use a QoE monitoring solution that already works well today (providing reliable, reproducible video quality measurements), and they should select a QoE vendor that is active in the key standards groups to ensure that the solutions they deploy are flexible to align with the future standards.
7. Deploy a Comprehensive End-to-End Solution
End-to-end monitoring is not just about having a range of equipment that can be used in various network locations. An effective end-to-end monitoring solution requires specialized devices for each network demarcation, all the way from the source content to the customer premises, combined with a mechanism to consolidate and correlate all information resulting in accurate and timely results. The figure below describes the three key demarcations when considering network monitoring solutions for video services:
The key demarcation points are:
- Video head-ends (national or regional) where all the content is being captured/encoded/trans coded and then streamed – the demarcation point being the first multicast router
- IP network which carries unicast as well as multicast content between the video headends and the access network - the demarcation point is the termination of the IP core network (edge QAM, Broadband Access Server, etc.)
- Last mile and CPE that can be made up of different technologies (DSL, PON, FTTC, etc.) including local in-house networking (including local wiring, home gateway and STB)
The chart below, which outlines percentage of problems coming from each demarcations, suggests the home, access network and headend are all critical components which a comprehensive end-to-end network monitoring solution must consider to provide comprehensive QoE solution.
In addition to deploying a solution to cover end-to-end issues, many operations teams need “point solutions” for monitoring their domain of responsibility (headend, last mile, home, etc). While it is important to ensure these demarcations have optimal solutions deployed, it is also important, for cost and operations purposes, that a deployed solution ties together end-to-end network monitoring into one comprehensive monitoring package.
8. Implement Multi-Play Monitoring Solutions
Monitoring solutions should support all requirements and monitoring metrics for IPTV/digital video, VoD, VoIP
, and IP performance monitoring to ensure optimum effectiveness from an operations and capital cost perspective. Deploying separate hardware platforms, especially in the core IP network, to monitor each service is not practical. Multiple devices increase both capital and operators costs and create network complexity and management challenges. In addition, integration and interoperability issues increase as more hardware devices are deployed.
Despite this, some demarcations dealing with specialized functionality may not need multi-service monitoring capabilities. For instance, headends or CPEs, due to their unique functionality as well as operational domains of responsibility, a video-only monitoring solution may oftentimes be preferred. In many cases however, especially with core and access network infrastructure, huge cost and operational efficiencies can be attained by implementing devices that perform monitoring functionality across all triple and quad play services.
9. Consider a Monitoring Solution that Works with Loss Recovery
Packet loss recovery approaches like forward error correction (FEC) and packet re-transmission techniques such as reliable UDP
(RUDP) can overcome a large percentage of packet loss in some network scenarios. These solutions can be helpful in mitigating impact to video quality from packet loss in network elements. These approaches, however, are not sufficient as their “static” nature (they are often tuned for specific cases) often only protects against pre-determined loss probabilities; there are theoretical and practical limitations to their effectiveness.
While such solutions can make video delivery more robust, operators will still require monitoring tools to understand the quality of their video service. Furthermore, FEC and RUDP are network-centric solutions that can protect the content to some extent as it is delivered over an IP network; however, like network QoS probes, they do not address video quality issues at the headend.
10. Build in Video Monitoring from the Start
Monitoring is a critical component needed to deliver high-quality IP-based video services. Based on years of history watching video, end-user expectations are extremely high when it comes to quality and if these expectations are not met, users get frustrated and complain, and usually churn. Monitoring is essential to gaining a better understanding of the service quality so steps can be taken to diagnose and troubleshoot issues and improve overall service levels. In fact, just as carriers deploy other networking equipment as part of video architectures (set-top-boxes, middleware, encoders, transcoders, etc), QoE-based video monitoring should also be considered an integral part of any IP-based video network deployment. Above all, video monitoring should NOT be an afterthought!
In addition, packet loss is not the only source of QoE problems as issues come from other sources besides the network (encoder impairments, source quality problems, home network issues, etc.); and as such, there still exists a great need for monitoring solutions. However, packet loss, corrected with enhanced error-correction or re-transmission techniques, cause monitoring solutions to falsely report lower video quality scores (when no problems actually exist); and as such, operators must deploy QoE solutions that work seamlessly with these approaches.
Summary and Conclusions
The real-time, high bandwidth nature of video traffic, combined with escalating end-user quality expectations, makes network monitoring for IP video challenging. Network operators are faced with what often seems to be an overwhelming amount of information and approaches for video monitoring. Let’s summarize the top 10 considerations once again:
1. Implement a QoE-based monitoring solution
2. Understand the end-user experience
3. Monitor the headend
4. Monitor the home
5. Monitor encrypted environments
6. Pay attention to standards
7. Deploy a comprehensive end-to-end solution
8. Implement multi-play monitoring solutions
9. Consider a monitoring solution that works with loss recovery
10. Build in video monitoring from the start
Boiling it all down, these are perhaps some of the most important factors when it comes to deploying video quality monitoring solutions in operational Telco and Cable operator networks.
S. Winkler: Digital Video Quality. John Wiley & Sons (News - Alert), 2005. http://stefan.winkler.net/book.html
IPTV Video Quality Study, Symmetricom, 2007.
Available at http://qoe.symmetricom.com/lp/IPTV_Study1/
Cable Operator Video Quality Study, Symmetricom, 2008.
Available at http://qoe.symmetricom.com/lp/cable_operators_study/
S. Winkler: Quality of Experience (QoE): An Important Measure of Success for IP-based Video Services. Symmetricom white paper, 2007.
Available at http://qoe.symmetricom.com/pdf/WP_QoE_IPVS.pdf
Video Quality Experts Group (VQEG): http://www.vqeg.org/
International Telecommunications Union (ITU): http://www.itu.int/
ATIS (News - Alert) IPTV Interoperability Forum (IIF): http://www.atis.org/iif/
Video Services Forum (VSF): http://www.videoservicesforum.org/
About the Author
Dr. Stefan Winkler is Principal Technologist at Symmetricom’s Quality of Experience Assurance Division (QAD). One of the world’s top experts in digital media quality, he was formerly chief scientist and co-founder of Genista Corporation, a provider of quality assurance solutions for IPTV and mobile media. He has also held assistant professor positions at the National University of Singapore (NUS) and the University of Lausanne, Switzerland, and has worked for Hewlett-Packard (News - Alert), Siemens, German Aerospace and Andersen Consulting. Dr. Winkler has published more than 40 papers on perceptual quality measurement and is the author of the book, “Digital Video Quality.” He has also been a member and contributor of the Video Quality Experts Group (VQEG) since its founding in 1997. Dr. Winkler holds a Master of Science in Electrical Engineering from the University of Technology in Vienna, Austria and a doctorate from the Swiss Federal Institute of Technology Lausanne (EPFL), Switzerland.