Ultra High Definition Video Formats and Standardisation - BT Media ...

BT Media & Broadcast

Research Paper

IN STRICTEST CONFIDENCE

IN STRICTEST CONFIDENC

BT Media and Broadcast IN STRICTEST CONFIDENCE

Research Paper IN STRICTEST CONFIDENCE


Ultra High Definition Video Formats and Standardisation STRICTEST CONFIDENCE

Version 1.0 IN STRICTEST CONFIDENC

April 2015 Mike Nilsson


IN STRICTEST CONFIDEN






Page | 1

Contents


Page

1

Introduction

6

2

Enhanced Resolution

7

3

High Dynamic Range


10

3.1

The Benefit of High Dynamic Range

3.2

The dynamic range of the human visual system

3.3

The non-linearity of the human visual system

3.4

The mapping of linear light to code levels

14

3.5

The mapping of pixel code levels back to linear light IN STRICTEST CONFIDENCE

15

3.6

Black level: how dark should displays be?

16

3.7

The mapping of pixel code levels to linear light in the presence of ambient light

17

3.8

The current state of high dynamic range capture and display technology

19

3.9

Interest and Experience in Hollywood

21

3.10

Dolby Cinema

4

10 IN STRICTEST CONFIDENCE


STRICTEST CONFIDENCE

Wider Colour Gamut

4.1

The CIE RGB Colour Space

4.2

The CIE XYZ Colour Space

11 13

22

23 23 24 IN STRICTEST CONFIDENC

4.3

Perceptually Uniform Colour Spaces

25

4.4

Colour Television

26

4.5

The need for a Wider Colour Gamut

4.6

Wider Colour Gamut Standards

4.7

Conversion between Colour Gamuts

4.8

How large a Colour Gamut is needed?

30

4.9

Display Technology for Wider Colour Gamut

31

5

Higher Frame Rate

5.1

The artistic impact of frame rate

5.2

Historical choices of frame rate

5.3

The motion blur – jerkiness trade-off


27 29




30

34 34 34 34


5.4

Subjective evaluation of moving picture quality



Page | 2

5.5

6

The impact of lighting frequency on frame rate


Standardisation

38

39

6.1

ITU-R

39

6.1.1

Parameters for Digital Television

6.1.2

High Dynamic Range

41

6.1.3

Colorimetry conversion

41

6.1.4

Higher Frame Rates

6.2

SMPTE

6.2.1

UHD Parameter Values

6.2.2

High Dynamic Range

6.2.3

Colour Volume Metadata

43

6.2.4

Digital Cinema

43

6.2.5

Colour Equations

44

6.2.6

On-going Activities

44

6.3

MPEG

6.3.1

High Efficiency Video Coding

45

6.3.2

The Future of Video Coding Standardisation

46

6.3.3

High Dynamic Range and Wide Colour Gamut

46

6.4

DVB

40


IN STRICTEST CONFIDENCE 41


42 42



45

47 IN STRICTEST CONFIDENC

6.4.1

UHD-1 Phase 1

48

6.4.2

UHD-1 Phase 2

49

6.4.3

UHD-2

6.4.4

Eco-design requirements for electronic displays

6.5

EBU

6.6

DTG

51

6.7

Blu-ray Disk Association

51

6.8

HDMI Forum

6.9

The Digital Cinema Initiatives (DCI)

6.10

The Forum for Advanced Media in Europe (FAME)

6.11

UHD Alliance and UHD Forum

7

Abbreviations

49 IN STRICTEST CONFIDENCE 50 IN STRICTEST CONFIDEN


50

52 53


53 54




55 Page | 3










BT Media and Broadcast contacts

Name

Business Area / Role

Phone

Email

John Ellerton

Head of Media Futures

020 7432 5224

[email protected]

Jonathan Wing

Head of Sales

020 7432 5347 [email protected] IN STRICTEST CONFIDENCE

BT Research and Innovation contacts


Name

Business Area / Role

Phone

Email

Mike Nilsson

Research and Innovation

01473 645413

[email protected]

Steve Appleby

Research and Innovation

01473 645743 [email protected] IN STRICTEST CONFIDENCE



Page | 4

Executive summary


BT Media and Broadcast provides services to broadcast and media organisations worldwide, including the international carriage of Ultra High Definition Television signals. We commissioned our colleagues in BT Research and Innovation to examine the current international status of technology and standardisation of Ultra High Definition Television to aid in our own understanding and future product development. This paper is the result, which we were so impressed with, we decided to release as a resource to the professional broadcast community. We hope it is useful – do get in touch if youINhave feedback orCONFIDENC comments. STRICTEST UHD televisions are now retailing in significant numbers, and services, such as those offered by Netflix and Amazon, are starting to appear in the market. But while these services offer higher resolution than HD services, further improvement could be made in due course to provide an even better viewing experience. STRICTEST CONFIDENCE Future television systems should be capable of producing an experience that isIN either closer to real life or is capable of more accurately recreating the artistic intent of the storyteller. To this end, increased resolution, wider colour palette, higher frame rate and an improvement in the dynamic range of the images when used together, have the potential to provide viewers with a better IN STRICTEST CONFIDENCE visual experience compared to current television applications and provide a viewer with a stronger sense of “being there”. Many standardisation organisations around the world have been, and are continuing to be, very active in the area of UHD TV standardisation. IN STRICTEST CONFIDENCE DVB has created standards for what it terms UHD-1 phase 1, effectively the parameters of HDTV but with enhanced resolution, similar to the deployments of Netflix and Amazon. DVB is now working on the commercial requirements for two subsequent phases, the first of which is expected to add support for higher dynamic range, wider colour gamut, and higher frame rates, and the second of which is planned to add support for even higher, 8K, resolution. The HEVC compression standard, almost essential to make delivery of UHDTV commercially feasible, has been approved, with the recently approved second version including support for higher bit-depths and enhanced chroma formats. MPEG is currently studying whether HEVC is optimal, as currently standardised, to support higher dynamic range and wider colour gamut, and if not, is expected to launch a standardisation activity during 2015 to address the deficiencies. STRICTEST CONFIDENCE There is widespread interest in backwards compatibility of future UHDTV services, which may have higher dynamic range, wider colour gamut, and higher frame rates, with first generation UHDTV services with only enhanced resolution. Technology to achieve this is still under development, and the ultimate success or failure of backwards compatibility will depend on how efficiently it could be implemented compared to simple but inefficient simulcasting. There are, at the writing, many problems and few agreed solutions, but there are many lively discussions taking place in the standards community. 2015 could be the year in which significant advances are made to the technical standardisation of the second phase of UHDTV. IN STRICTEST CONFIDENC This report begins with sections describing the science and technology of UHD television: enhanced resolution, higher dynamic range, wider colour gamut and higher frame rate. This is followed by a section describing the current state of standardisation in various bodies that are essential to the standardisation of UHD television services, ITU-R, SMPTE, MPEG and DVB, while also providing a brief status update on EBU, DTG, Blu-ray Disk Association, HDMI Forum, Digital Cinema Initiatives (DCI), the Forum for Advanced Media in Europe (FAME), the UHD Alliance and the UHD Forum. IN STRICTEST CONFIDENCE We consider that while the enhanced resolution of the first phase of deployments of UHDTV services could provide undoubtedly better picture quality than current HDTV services when viewed from an optimal viewing distance, the combination of human visual IN STRICTEST CONFIDEN acuity, screen size, and home viewing distances could make the improvement over HDTV in the home environment less pronounced. We have observed the growing feeling within the standardisation community that to make UHDTV reach the mass market, additional features would be needed beyond the enhanced resolution achieved with the first phase of deployment. In particular, enhancements that that do not depend so critically on viewing distance would be beneficial. Fortunately these features, higher dynamic range, wider colour gamut, and higher frame rates, are becoming technologically feasible and are generating much interest IN STRICTEST CONFIDENCE within the industry. This has perhaps been most noticeable in the film industry. Dolby Cinema™, which was announced in December 2014, and which is IN STRICTEST CONFIDENC claimed to deliver high dynamic range with enhanced colour, is planned for theatrical exhibition in 2015. The Blu-ray Disc Association is developing a next generation format that will support UHD, a wide colour gamut, and high dynamic range, with the first players and titles being expected before the end of 2015. The broadcast TV industry is close behind, with DVB aiming to develop specifications to enable deployment of second phase UHD IN STRICTEST CONFIDENCE services in the 2017-18 timeframe. IN STRICTEST CONFIDENCE


Page | 5

1

Introduction


Ultra High Definition TV has attracted a good deal of attention lately as a result of a push from TV vendors and some content providers, notably Netflix. However, to date, the focus has been solely on increased resolution. Ultra High Definition though, is about more than just increasing the number of pixels, being also about increasing the dynamic range, widening the colour gamut and increasing the frame rate. When viewing TV on a popular size of screen in a typical home environment, it is these additional improvements which may ultimately provide the most benefit to the viewer. IN STRICTEST CONFIDENC The quality of television is therefore set to improve in multiple dimensions over the next few years. There is still heated debate about the standards that will be used, but it seems clear that managing legacy and dealing with multiple dimensions of TV improvements will be a challenge for the broadcast industry. The purpose of this document is to provide a thorough review of the current status of UHD standards with a view to obtaining a IN STRICTEST CONFIDENCE clearer understanding of the challenges and opportunities ahead. IN STRICTEST CONFIDENCE











Page | 6

2

Enhanced Resolution


Display and capture technology is advancing. Today there are both video cameras and displays capable of resolutions higher than full High Definition TV. These resolutions are frequently referred to as either 4K or Ultra High Definition TV (UHDTV) and have resolution 3840x2160, that is, four times the resolution of full High Definition TV. Figure 1 shows the resolution of UHD relative to the earlier formats of Standard Definition (SD), HD-ready (720p HD), and High Definition (HD). IN STRICTEST CONFIDENC




Figure 1. The resolution of UHD relative to earlier, lower, resolutions. 1

UHD televisions are now retailing in significant numbers , enabling potentially much better picture quality than current HD televisions. STRICTEST CONFIDENCE BT worked with the BBC during 2014 to deliver coverage of the 2014 football World Cup Final in UHD resolution live to the BT Tower, using HEVC video compression and MPEG DASH technology, delivering content from BT Wholesale Content Connect over BT Infinity superfast fibre optic broadband through BT Home Hub 5 routers to three set top boxes connected to UHDTVs and directly to two internet-enabled UHDTVs. The picture quality achieved was very impressive. The quality of the UHD signal was undoubtedly much better than HD delivered by Freeview, when viewed near to the screen. IN STRICTEST CONFIDENC 2 The ITU-R describe optimal viewing distance in image heights (H) for various digital image systems , recommending 6H for Standard Definition, 4.8H for 720p HD, 3.2H for HD, and 1.6H for UHD. Such optimal viewing distances are often calculated by considering normal human visual acuity to be 20/20, meaning that a letter, such as the letter E, can be identified when top to bottom it subtends an angleIN of STRICTEST 5 arc minutes. CONFIDENCE This corresponds to 1 arc minute per horizontal feature (three lines and two spaces). It also corresponds to one dark to light cycle every 2 arc minutes. One dark to light cycle every 2 arc minutes equals 30 cycles per degree, which can be represented with 60 pixels per degree. This IN STRICTEST CONFIDEN has been taken as standard acuity for TV applications, although clearly some viewers have worse eye sight and some have better eye sight. The screen diagonal size, S, measured in inches, required to achieve a resolvable number of horizontal pixels, R, when viewed from a distance D, measured in centimetres, is given by the following equation, assuming visual acuity of 60 pixels per degree and 16:9 picture aspect ratio. IN STRICTEST CONFIDENCE

(

) IN STRICTEST CONFIDENC

1

http://advanced-television.com/2014/11/26/large-screen-tv-sales-up-48/

2

"Guidelines on metrics to be used when tailoring television programmes to broadcasting applications at various image quality IN STRICTEST CONFIDENCE levels, display sizes and aspect ratios", Recommendation ITU-R BT.1845-1 (03/2010), http://www.itu.int/rec/R-REC-BT.1845-1201003-I IN STRICTEST CONFIDENCE


Page | 7

However, there is reason to believe that viewers will not sit as close as 1.6H to IN their television, and hence would not achieve the STRICTEST CONFIDENCE ITU-R optimal viewing experience. 3

A survey of 102 BBC employees in 2004 reported that the median viewing distance for viewing the main TV in the home was 2.7m. 4 The BBC carried out a more extensive survey of the UK population during the summer of 2014 , collecting information from 2185 people who have a television in their home, and finding the median viewing distance was 2.63m. While the 2004 survey found a median screen height of 32.5cm, the median height from the 2014 survey was 49cm, which, assuming a 16:9 picture aspect ratio, corresponds to a screen diagonal size of 39.3 inches. The 2014 survey also found that the IN STRICTEST CONFIDENC median relative viewing distance is 5.5 times the screen height (5.5H). The 2014 survey, by asking respondents how they would expect the size of their next television to compare to that of their current one, found that about half the respondents expected to buy a larger screen next time. When asked to estimate the ideal television size for their current home, assuming that money were no object, they responded with a median ideal diagonal size of 48 inches. IN STRICTEST CONFIDENCE By assuming that people would need to sit three picture heights (3H) or closer to a UHD screen to get a resolution improvement over HD, the BBC deduced from the 2014 survey that 10.2% of viewers would be able to observe a benefit from UHD with their current television, and 22.9% would if they acquired their ideal television. IN STRICTEST CONFIDENCE These two surveys show no significant change in television viewing distances over a period of 10 years. As this is likely to be significantly influenced by the size of rooms in UK homes, it would seem reasonable to expect little change in the coming years. IN STRICTEST CONFIDENCE Currently 55 inch and 65 inch diagonals are popular for UHD televisions. There would appear little reason to believe that this would change in the near future, as it is in part determined by the size and layout of rooms in which people watch television. A viewing distance of 1.6H, which the ITU-R consider to be optimal for viewing UHD content, when calculated for a television with 65 inch diagonal, corresponds to a viewing distance of about 1.3m. It seems unlikely given the results of the two BBC surveys that many people would sit this close to their UHDTV. Table 1 shows the resolution of various popular TV formats and the screen size that would be needed for optimal viewing at a distance of 2.63m, the median viewing distance found in the 2014 BBC survey. It is unlikely that viewers would want to, and be able STRICTEST CONFIDENCE to, acquire screens with diagonal size of 148 inches, so that they could both maintain a viewing distance of 2.63m and achieve optimal, as defined by the ITU-R, viewing of UHD content. While some viewers may compromise on viewing distance and screen size to get such a UHD experience, it seems likely that the majority would not, and hence they may not get the full benefit of UHDTV resolution.

Format

Horizontal Resolution

Screen Size (Inches) IN STRICTEST CONFIDENC 25

SD

720

720p HD

1280

HD

1920

IN STRICTEST68 CONFIDENCE

UHD

3840

148 IN STRICTEST CONFIDEN

45

Table 1. The screen size needed for optimal viewing of popular TV formats from 2.63m.

The screen would not need to be so large if the requirement were relaxed from achieving optimal viewing of UHD content to achieving a better experience than watching HD content. Again referring to Table 1, any screen with diagonal size larger than 68 inches would enable more resolution than HD to be observed from 2.63m. 5 IN STRICTEST CONFIDENCE However, there is some disagreement with this whole methodology. Spencer has reported results of subjective tests in which viewers assessed content from a distance of 30cm, controlled using a head mount. By using the methodology described above, the IN STRICTEST CONFIDENC 3

“Results of a survey on television viewing distance”, http://downloads.bbc.co.uk/rd/pubs/whp/whp-pdf-files/WHP090.pdf, N. E. Tanton, June 2004 4

IN STRICTEST CONFIDENCE “A Survey of UK Television Viewing Conditions”, http://downloads.bbc.co.uk/rd/pubs/whp/whp-pdf-files/WHP287.pdf, Katy C. Noland and Louise H. Truong, January 2015 IN STRICTEST CONFIDENCE


Page | 8

viewers should have been limited to seeing benefits as the pixel density increased to about 300 pixels per inch, and then seeing no IN STRICTEST CONFIDENCE further improvement as the pixel density was increased. However, the presented results show improvements at 500 pixels per inch, and yet further improvements at 1000 pixels per inch. Spencer explains these results by stating that the human eye is able to detect differences represented by even finer structures, referencing tests of vernier acuity, which involve distinguishing the relative alignment of parallel lines, that show that humans can distinguish details five to ten times smaller than standard resolution measurements predict. While this is an isolated study, it does suggest that further investigation of the benefits of the increased resolution of UHDTV, when IN STRICTEST CONFIDENC viewed on televisions of desirable size at normal home viewing distances, would be worthwhile.










5

IN STRICTEST CONFIDENCE “How much higher can mobile display resolution go?”, Lee Spencer, http://www.eetasia.com/STATIC/PDF/201312/EEOL_2013JAN03_OPT_TA_01.pdf?SOURCES=DOWNLOAD IN STRICTEST CONFIDENCE


Page | 9

3

High Dynamic Range


The human visual system has a significantly larger dynamic range than that supported by current television systems. Display devices that are able to support a much larger dynamic range than conventional televisions systems are starting to appear in the market. They achieve the larger dynamic range by extending the range both towards darker as well as lighter values as a result of fitting a conventional LCD panel with a spatially varying backlight, which could be either a projector or a panel of LEDs which are individually addressable. IN STRICTEST CONFIDENC A key technical challenge for video delivery is how to signal the higher dynamic range while also supporting legacy standard dynamic range displays. Some proprietary technology is already entering the market, and various standardisation bodies, including SMPTE, the EBU and MPEG are considering the issues. This section provides an overview of the benefits of using a higher dynamic range for images andCONFIDENCE video, the capability of the human IN STRICTEST visual system, and the capability of emerging display technology. Additionally, the issues and proposed solutions for integrating higher dynamic range into existing end to end television systems are discussed. IN STRICTEST CONFIDENCE

3.1

The Benefit of High Dynamic Range

Figure 2 is an example of an image where the use of a higher dynamic range would improve the CONFIDENCE viewing experience. With a standard IN STRICTEST dynamic range, it is not possible to have visible details in both the shaded parts of the image and the parts that are in full sunshine.




IN STRICTEST Figure 2. An image that would benefit from high dynamic range. CONFIDEN The use of a high dynamic range is not simply making images brighter, which in itself can be effective, but enabling detail to be seen in both dark and light areas. Figure 3 shows an example of a histogram of the pixel luminance values of an image, represented with both standard and high dynamic range. Many of the pixels have about the same luminance in both cases, and only some pixels, the lowlights and the highlights in the image, have lower or greater luminance in the high dynamic range variant. IN 2STRICTEST CONFIDENCE 2 Luminance is measured in the derived SI units of candela per square metre (cd/m ), also known as the nit (1 nit = 1 cd/m ). Typically 2 displays are limited to a dynamic range of 100:1, and a typical maximum luminance of 100 cd/m due to legacy limitations of CRT display technology and the derived current specification for display interfaces. The human visual system, as described later, is IN STRICTEST CONFIDENC capable of perceiving a much higher dynamic range and higher levels of luminance. Figure 4 illustrates the wide range of luminance values that occur in real world scenes, labelling specific features with approximate values of light level. IN STRICTEST CONFIDENCE



Page | 10

Relative Frequency




IN STRICTEST CONFIDENCE 0.01

0.1

1

10 Luminance (cd/m2)

Standard Dynamic Range Image

100

1000

IN STRICTEST High Dynamic Range Image

10000

CONFIDENCE

Figure 3. An example pixel luminance histogram for a standard and high dynamic range image.

4,000 nits

10,000 nits


50,000 nits 100 nits 40 nits

IN STRICTEST CONFIDENC 2 nits

Figure 4. Examples of approximate light levels in real world scenes.

3.2

The dynamic range of the human visual system

IN STRICTEST CONFIDENCE -4

2

5

The human visual system has a huge dynamic range, being able to cope with starlight at 10 cd/m and bright sunshine at 10 2 6 -6 2 cd/m . Hood and Finkelstein report values ranging from 10 to 10 cd/m for scotopic light levelsCONFIDEN where light transduction is IN STRICTEST 8 2 mediated by the rod cells in the eye, and from 0.01 to 10 cd/m for the photopic range where the cone cells are active. And in the overlap, called the mesopic range, both rods and cones are involved. At any one time, the human visual system is able to operate over only a fraction of this enormous range. This subset is called the simultaneous or steady-state dynamic range. It shifts to an appropriate light sensitivity due to various mechanical, photochemical 7 and neuronal adaptive processes , so that under any lighting conditions the effectiveness of human vision is maximised. IN STRICTEST CONFIDENCE

IN STRICTEST CONFIDENC 6

From D. C. Hood and M. A. Finkelstein, Visual sensitivity, in Handbook of Perception and Human Performance, K. Boff, L. Kaufman, and J. Thomas, Eds., vol. 1. Wiley, New York, 5–1— 5–66. 7

STRICTEST CONFIDENCE J. A. Ferwerda, Elements of early vision for computer graphics, IEEE ComputerINGraphics and Applications 21, 5, 22-33, luthuli.cs.uiuc.edu/~daf/courses/Rendering/Papers3/00946628.pdf. IN STRICTEST CONFIDENCE


Page | 11

The simultaneous dynamic range over which the human visual system is able toINfunction can be CONFIDENCE defined as the ratio between the STRICTEST highest and lowest luminance values at which objects can be detected, while being in a state of full adaptation. In a given room that contains a display device, the human visual system will typically be in a steady state of full adaptation. Figure 5, taken from Hood and Finkelstein, shows the overall range of the human visual system compared to the ranges of its approximate steady state as well as the range for typical Low Dynamic Range (LDR) and High Dynamic Range (HDR) displays. While the overall dynamic range of the human visual system and these display devices is known, less is known about the range of 8 the human visual system in the steady-state. Kunkel and Reinhard have reported a sequence of psychophysical experiments, IN STRICTEST CONFIDENC carried out with the aid of a high dynamic range display device, where they determined the simultaneous dynamic range of the human visual system, finding the human visual system to be capable of distinguishing contrasts over a range of 3.7 log units, equivalent to a range of about 1:5000, under specific viewing conditions. They also found that the dynamic range is affected by stimulus duration, the contrast of the stimulus and the background IN STRICTEST CONFIDENCE illumination, which they claimed accounts for the different dynamic ranges being reported in the literature. IN STRICTEST CONFIDENCE


STRICTEST CONFIDENCE Figure 5. The dynamic range of the human visual system. 9

Jenny Read, neuroscientist at the University of Newcastle, talked at DVB-EBU HDR Workshop in June 2014 on how the Human Visual System reacts to high dynamic range screens, stating that encoding schemes must be defined by taking into account human contrast sensitivity. It is also necessary to take into account the human visual system ‘adaptation states’, where light to dark adaptation is slow, typically taking between 8 and 30 minutes, and where dark to light adaptation is much quicker, typically taking just a few minutes. IN STRICTEST CONFIDENC 10

Daly et al report studies to find the dynamic range that is preferred by human observers. They used a Dual Modulation Research 11 2 Display, as shown in Figure 6, and referenced by Hammer , which is capable of supporting a dynamic range from 0.004 cd/m to 2 20,000 cd/m , that is, a contrast ratio of 5,000,000:1; and which supported the DCI P3 colour gamut. The experiment used several realistic and synthetic stimuli in a dark viewing environment. IN STRICTEST CONFIDENCE 2

They found for diffuse reflective images a dynamic range between 0.1 and 650 cd/m matched the average preferences, but to 2 satisfy 90% of the population, a dynamic range from 0.005 to about 3000 cd/m is needed. They claim that since a display should be IN STRICTEST CONFIDEN able to produce values brighter than the diffuse white maximum, as in specular highlights and emissive sources, they conclude that 2 the average preferred maximum luminance for highlight reproduction satisfying 50% of viewers is about 2500 cd/m , increasing to 2 marginally over 20000 cd/m to satisfy 90%. They conclude that there would be a benefit from more capable displays, as the preferred luminances found in their study exceed even the best of consumer displays today. IN STRICTEST CONFIDENCE T. Kunkel and E. Reinhard, A reassessment of the simultaneous dynamic range of the human visual system, Proceedings of the 7th Symposium on Applied Perception in Graphics and Visualization, ISBN 978-1-4503-0248-7, pp. 17–24, July 2010. http://www.cs.bris.ac.uk/Publications/Papers/2001238.pdf IN STRICTEST CONFIDENC 8

9

DVB-EBU HDR Workshop, IRT, Munich, Germany, 17 June 2014, https://tech.ebu.ch/docs/tech-i/ebu_tech-i_021.pdf

10

S. Daly, T. Kunkel, S. Farrell & Xing Sun: Viewer Preferences for Shadow, Diffuse, Specular, and Emissive Luminance Limits of High Dynamic Range Displays. SID Display Week 2013. http://onlinelibrary.wiley.com/doi/10.1002/j.2168-0159.2013.tb06271.x/abstract IN STRICTEST CONFIDENCE 11

Hammer, “High-Dynamic-Range Displays”, http://alexandria.tue.nl/extra2/773243.pdf IN STRICTEST CONFIDENCE


Page | 12






Figure 6. The experimental set up used by Daly et al. to assess preferred dynamic range. 12

Hanhart, Korshunov, and Ebrahimi report a set of subjective experiments to investigate the added value of higher dynamic range. Seven test video sequences at four different peak luminance levels were assessed using the full paired comparison methodology. Pairs of the same sequence with different peak luminance levels were displayed side-by-side on a Dolby Research HDR RGB 2 backlight dual modulation display (aka ‘Pulsar’), which is capable of reliably displaying video content at 4000 cd/m peak luminance. STRICTEST CONFIDENCE Their results, as shown in Figure 7, show that the preference of an average viewer increases logarithmically with the increase in the 2 maximum luminance level at which the content is displayed, with 4000 cd/m being the most attractive option of those tested.



IN STRICTEST CONFIDEN Figure 7. Results of subjective evaluation of HDR Video using pair comparison by Hanhart, Korshunov, and Ebrahimi.

3.3

The non-linearity of the human visual system

IN STRICTEST CONFIDENCE Ernst Heinrich Weber studied of the human response to a physical stimulus in a quantitative fashion, finding, in what is now known as Weber's law, that the just-noticeable difference between two stimuli is proportional to the magnitude of the stimuli, that is, an increment is judged relative to the previous amount. Gustav Theodor Fechner used Weber's findings to construct a psychophysical IN STRICTEST CONFIDENC scale in which he described the relationship between the physical magnitude of a stimulus and its (subjectively) perceived intensity.

12

Subjective Evaluation of Higher Dynamic Range Video, Philippe Hanhart, Pavel Korshunov, and Touradj Ebrahimi, SPIE Optical Engineering + Applications, Applications of Digital Image Processing XXXVII. IN STRICTEST CONFIDENCE http://infoscience.epfl.ch/record/200538/files/article.pdf IN STRICTEST CONFIDENCE


Page | 13

Fechner's law states that subjective sensation is proportional to the logarithm of stimulus intensity. Fechner scaling has been IN the STRICTEST CONFIDENCE found to apply to the human perception of brightness, at moderate and high brightness, with perceived brightness being 13 proportional to the logarithm of the actual intensity . At lower levels of brightness, a more accurate description is given by the de Vries-Rose law which states that the perception of brightness is proportional to the square root of the actual intensity.

3.4

The mapping of linear light to code levels

IN STRICTEST CONFIDENC 2 2 Current television systems typically support content with a range of brightness from about 0.1 cd/m to about 100 cd/m . Consequently, it is the de Vries-Rose characteristic, described above, that has been designed into television systems to date, where a function known as ‘gamma’ or ‘gamma correction’, is used to map brightness (linear light) to a scale in which each increment corresponds to a constant perceptual change. This function is known as the Opto-Electrical IN STRICTEST Transfer CONFIDENCE Function (OETF). In analogue television systems this characteristic made the whole range of intensities equally sensitive to noise, whereas in digital 14 television systems it enabled the number of bits needed to represent each sample to be minimised . This characteristic is shown in IN STRICTEST CONFIDENCE Figure 8, where mappings, as specified in ITU-R Recommendation BT.709, to both 8 and 10 bit code levels are shown. The use of 10 bit codes, as increasingly used in video production, gives little benefit over 8 bits if BT.709 is still used, as the extra bits are the least significant bits, which are often discarded for display on 8 bit devices, and which at best could reduce the minimum black level, that IN STRICTEST CONFIDENCE is, make the blacks even blacker. While using BT.709, the extra two bits cannot increase the brightness, even though this would often be more desirable than increasing the maximum darkness. ITU-R Recommendation BT.2020, a more recent specification than BT.709, defines higher frame rates and a wider colour space, but the same OETF as BT.709. These and other ITU-R Recommendations are discussed in further later in this research paper. The dynamic range of a video signal can be increased by using an alternative Opto-Electrical Transfer Function (OETF) to that specified in BT.709. Digital Imaging and Communications in Medicine (DICOM) is a standard for handling, storing, printing, and transmitting information 15 STRICTEST in medical imaging. This includes in Part 14 an Opto-Electrical Transfer Function from linearCONFIDENCE light to 10 bit code levels, also shown in Figure 8, and valid between 0.05 and 4000 cd/m2 whereby each 1-bit luminance increment is equally visible according to 16 Barten . Jenny Read stated at the DVB-EBU HDR Workshop in September 2014 that the film industry has used log-curves for many years that are quite near the human visual system and are much more efficient than the BT.709 Transfer Function used for TV today. A logarithmic OETF is used to map 14 stops of linear light, that is, a dynamic range of about 16,000:1, to a 10 bit signal, and is satisfactory to capture the “Cineon” format. IN STRICTEST CONFIDENC 17 Miller et al of Dolby Laboratories have proposed a new OETF to provide higher dynamic range for video and movie production and distribution, which has recently been standardised by the Society of Motion Picture Engineers as SMPTE ST 2084:2014, “High Dynamic Range Electro-Optical Transfer Function of Mastering Reference Displays”, which is described later in this research paper. This is also based on the work of Barten, and is also shown in Figure 8. IN STRICTEST CONFIDENCE 18 But there is no industry consensus on the relevance of the work of Barten. The BBC in its white paper 283 argues that Miller’s proposal links the camera OETF to the absolute brightness of the display and that this has potentially far reaching consequences. IN STRICTEST CONFIDEN 13

Weber–Fechner law, http://en.wikipedia.org/wiki/Weber%E2%80%93Fechner_law

14

C. A. Poynton, Digital Video and HDTV: Algorithms and Interfaces, Electronics & Electrical, Morgan Kaufmann series in computer graphics and geometric modelling, ISBN 1558607927, 9781558607927 15

http://medical.nema.org/Dicom/2011/11_14pu.pdf


16

Barten, P.G.J., Physical model for the Contrast Sensitivity of the human eye. Proc. SPIE 1666, 57-72 (1992); Barten, P.G.J., Spatiotemporal model for the Contrast Sensitivity of the human eye and its temporal aspects. Proc. SPIE 1913-01 (1993); and P. G. J. IN STRICTEST CONFIDENC Barten, “Formula for the contrast sensitivity of the human eye”, Proc. SPIE-IS&T Vol. 5294:231-238, Jan. 2004 17

Scott Miller, Mahdi Nezamabadi and Scott Daly. Perceptual Signal Coding for More Efficient Usage of Bit Codes. SMPTE Motion Imaging Journal. 2013. 122:52-59. doi:10.5594/j18290. https://www.smpte.org/sites/default/files/23-1615-TS7-2-IProc02-Miller.pdf 18

STRICTEST CONFIDENCE BBC White Paper WHP 283, Non-linear Opto-Electrical Transfer Functions forIN High Dynamic Range Television, T. Borer, July 2014. http://downloads.bbc.co.uk/rd/pubs/whp/whp-pdf-files/WHP283.pdf IN STRICTEST CONFIDENCE


Page | 14

The BBC argues that previously the photographic, movie and television industries have all alwaysCONFIDENCE worked with relative, rather than IN STRICTEST absolute, luminance levels. The BBC argues that changing to absolute luminance levels will require significant changes to the way 7 television is produced and viewed; and that it is not clear that the large dynamic range of 10 is needed for image display, when the 4 simultaneous dynamic range of the eye is about 10 . They have developed their own proposal for an OETF, as shown in Figure 8, and which is imagined to be consistent with their submissions to ITU-R WP6C. There is on-going discussion of high dynamic range in standardisation bodies. The formal standardisation process for high dynamic range signal and exchange format, including the consideration of suitable Opto-Electrical Transfer Functions, is on-going in ITU-R WP6C (RG-24) and in SMPTE 10-E as described later in this report. IN STRICTEST CONFIDENC MPEG is currently investigating the effectiveness of existing compression technologies with high dynamic range video, and the means to support standard and high dynamic range video within a single coded representation. IN STRICTEST CONFIDENCE 10000


1000


Linear Luminance (cd/m2)

100

10

1 0

200

400

600

800

1000

0.1 BT.709 with 8 bit code levels


0.01

BT.709 with 10 bit code levels DICOM Part 14

0.001

Perceptual Quantiser, SMPTE ST 2084:2014

BBC HDR Proposal - 10 bit code levels 0.0001

Code Levels


Figure 8. The mapping of linear light to code levels.

3.5

The mapping of pixel code levels back to linear light

IN television, STRICTEST CONFIDENCE While Cathode Ray Tubes (CRT) were the only or most popular display device for there was no need to specify an inverse to the OETF, known as the Electro-Optical Transfer Function (EOTF), as the inverse function was provided implicitly from the physics of cathode ray tubes. Fortunately, this function was well matched to the human visual system. IN STRICTEST CONFIDEN It was only in 2011, by which time the consumer market for CRTs had almost completely disappeared, that a standard for an EOTF, 75 ITU-R Recommendation BT.1886 , was finally agreed. This effectively documents the characteristics of CRT displays. The EOTF is not necessarily best defined as the inverse of the OETF: the combination of the OETF and the EOTF yields the total transfer function, sometimes known as the “end-to-end gamma” or “system gamma”. Some believe that this total transfer function should be a power law function with the exponent value dependent on the lighting condition, for example, with values of 1.0, 1.25, 19 and 1.5 being appropriate for bright, dim, and dark surrounding environments IN. STRICTEST CONFIDENCE The fundamental basis for interpreting any visual signal is the transfer function, the description of how to convert the signal, that is, the digital code values, to optical energy. It is therefore this EOTF and not the OETF that truly defines the intent of visual signal code IN STRICTEST CONFIDENC values. The vast majority of content is colour graded (either live in the camera, or during post production) according to artistic preference while viewing on a reference standard display.

19

INtelevision”. STRICTEST CONFIDENCE Report ITU-R BT.2246-3, (03/2014), “The present state of ultra-high definition http://www.itu.int/dms_pub/itur/opb/rep/R-REP-BT.2246-3-2014-PDF-E.pdf IN STRICTEST CONFIDENCE


Page | 15

The EOTF, when applied to code values with a given bit depth, 8, 10 or 12 bits etc., should ideallyCONFIDENCE avoid the viewing of IN STRICTEST discontinuities in tone reproduction, and hence the EOTF and bit depth should be matched to the contrast sensitivity function of the human visual system. 16

A well respected model for the contrast sensitivity function of the human visual system was developed by Peter Barten , and has been referenced by many electronic imaging studies and standards. This complex model, based on physics, optics, and some experimentally determined parameters, has been shown to align well with many visual experiments spanning several decades of research. IN STRICTEST CONFIDENC 17 Dolby Laboratories have used the Barten model directly to compute an optimized perceptual EOTF . This function is defined in Figure 9. This is defined in terms of absolute luminance levels viewed on the display screen, not in terms of absolute luminance at capture. This Perceptual Quantiser curve has nearly a square root behaviour (slope = -1/2) at the darkest light levels, consistent with the Rose-DeVries law, and then rolls off to a constant zero slope for the highest light levels, consistent with the log behaviour of Weber’s law. Between those extreme luminance regions, it exhibits varying slopes, and throughout the mid luminance levels it IN STRICTEST CONFIDENCE exhibits a slope similar to the gamma non-linearities of BT.1886. IN STRICTEST CONFIDENCE ⁄

⁄

(

⁄

)


Figure 9. The EOTF derived from the perceptual quantiser function, as proposed Dolby. 17

2

Dolby Laboratories reported the results of subjective tests with real images with peak luminance up to 600cd/m , comparing 2 2 BT.1886 scaled for a peak luminance of 1000cd/m with the perceptual quantiser based EOTF scaled to 1000cd/m , and scaled to 2 10,000cd/m . They reported finding that with both variants of the perceptual quantiser, 10 bits of bit depth were sufficient to avoid STRICTEST CONFIDENCE visible quantisation steps on all eight test images, whereas the use of BT.1886 required less bits for light (white) images, typically one bit less of bit depth, but needed more bits for the dark (black) images, including needing more than 12 bits of bit depth for one image. These results are consistent with the known limitations of BT.1886 that it has greater precision for lighter regions than darker regions, and effectively ‘wastes’ code values at the light end, and does not have enough at the dark end. It is generally thought that it would be difficult to operate legacy infrastructures with bit depth greater than 12 bits, and most live production and broadcast environments still operate at the 10 bit level. Hence it is commonly agreed that applying BT.1886 to a higher dynamic range is not feasible as the bit depth required would be too high. However, there is no consensus that the IN STRICTEST CONFIDENC perceptual quantiser proposed by Dolby Laboratories is the best solution: there are concerns about its being defined in terms of absolute light levels, and other functions have been proposed, as are discussed elsewhere in this report.

3.6

Black level: how dark should displays be?

IN STRICTEST CONFIDENCE How dark or black a region of a display can appear depends on two factors: the minimum emission from the display and the amount of ambient light that is reflected. The effective display black level, Lblack, can be calculated, as in the equation below, as the sum of IN STRICTEST the display minimum light emission, Lmin, known as dark current in the days of CRT, and meaningCONFIDEN the lowest level of luminance that comes out of the display; and a product of the display screen reflectivity, Rdisplay, and the ambient light level, Eambient. The impact therefore of higher levels of ambient light is to raise the minimum black level, and consequently to reduce the dynamic range of the image, as the maximum intensity is mostly unchanged with ambient light.

(

) IN STRICTEST CONFIDENCE

20

Mantiuk et al. have reported an experiment to determine the highest luminance level that cannot be discriminated from ‘absolute black’ as the surrounding luminance is varied. They showed viewers a patch in the centre of a screen with ‘absolute black’ on one IN STRICTEST CONFIDENC side of the patch and non-zero luminance on the other, and asked viewers to choose the side that was brighter, or choose randomly

20

Mantiuk, R. and Daly, S. and Kerofsky, L. (2010), “The luminance of pure black: exploring the effect of surround in the context of IN STRICTEST CONFIDENCE electronic displays”. In: Proc. of Human Vision and Electronic Imaging XXI, IS&T/SPIE's Symposium on Electronic Imaging. http://pages.bangor.ac.uk/~eesa0c/pdfs/mantiuk10lpb.pdf. IN STRICTEST CONFIDENCE


Page | 16

if they looked the same. Different values of non-zero luminance and of surrounding luminance were tested. Two viewing distances IN STRICTEST CONFIDENCE were used, 1.4m and 4.7m, so that the size of the square patch corresponded to 6.1 and 1.8 visual degrees. They converted these results of just detectable differences from absolute black so they could be plotted as a function of ambient light rather than the luminance of surrounding pixels. The results are shown in Figure 10, with the experimental results labelled as ‘HVS Small Patch’ and ‘HVS Larger Patch’. It can be seen that as the ambient illumination is increased, the lowest level of black that can be distinguished from absolute black increases. Also shown in Figure 10 is the black level of a black diffuse surface with reflectance of 3%, such as black velvet, and the performance 2 IN STRICTEST CONFIDENC of three displays: a CRT with minimum light emission of 1cd/m and 3% reflectance; a conventional CCFL (cold cathode fluorescent 2 lamp) backlight LCD with minimum light emission of 0.8cd/m and 1% reflectance; and a modern LED-backlight LCD with spatially 2 uniform back-light dimming with minimum light emission of 000163cd/m and 1% reflectance.1.3/2.4 The CRT appears grey compared to the diffuse black (velvet) for ambient light below 300lux (about 2.5 on the horizontal axis of IN STRICTEST CONFIDENCE 21 Figure 10), a level of brightness found in an office or a very well lit room in a home . For the CCFL-LCD, this threshold is 100lux (2.0 in Figure 10), typical of a room in a home. This is because the display effective black level is higher than the luminance of a diffuse black surface, due to a combination of reflectance and the minimum light emissions of the displays. IN STRICTEST CONFIDENCE The experimental results indicate that the eye can appreciate even deeper black than a diffuse black surface, and that of the considered display technologies, only the LED-LCD display can satisfy the demands of the human visual system, and only at levels below about 1.6lux (0.2 in Figure 10, where the LED-LCD curve crosses the HVSIN Larger Patch curve), an indoor illumination level that STRICTEST CONFIDENCE could be considered near pitch black. The problem is not the minimum light emissions which are very low, but the reflectance of ambient light from the screen. It appears unlikely that very low reflectance coatings will be possible, but this is unlikely to matter, as a reflectance of about 1% is likely to acceptable to almost all viewers as there are not many objects in the real-world that would have lower reflectivity and thus appear darker than a display.

Black Level (Logarithm to the base 10 of values measured in cd/m2)

1.5 1.0


0.5 0.0 -0.5 -1.0

IN STRICTEST CONFIDENC CRT -1.5

CCFL-LCD LED-LCD

-2.0

Diffuse Black HVS Small Patch IN STRICTEST CONFIDENCE

-2.5

HVS Larger Patch

Deep Twilight

-3.0 -0.5

0

Home

Dark Room

Office


0.5 1 1.5 2 2.5 Ambient Luminance (Logarithm to the base 10 of values measured in Lux)

3

3.5

Figure 10. Comparison of the black levels of displays and human detection capability.

3.7

The mapping of pixel code levels to linear light in the presence of ambient light IN STRICTEST CONFIDENCE 22

Although standards including BT.1886 and IEC 61966-2-1 (sRGB ) specify the mapping of pixel code values to light levels, the actual perceived light level on a screen also depends on the ambient light and the reflectivity of the screen. IN STRICTEST CONFIDENC

21

http://www.tfc-group.co.uk/assets/graphics/static/Recommended_light_levels1.pdf IN STRICTEST CONFIDENCE 22 http://www.color.org/srgb.pdf IN STRICTEST CONFIDENCE


Page | 17

23

Mantiuk et al. have developed a display model that combines the standardised mapping with a factor to include the IN gamma STRICTEST CONFIDENCE reflections of ambient light. This display model, shown in the equation below, includes the maximum brightness and dynamic range of the display device, and the viewing conditions, the amount of the ambient light that is reflected from the screen. Such reflected light increases the luminance of the darkest pixels shown on the display, thus reducing the available dynamic range.

( )

( )

(

)

(

)

IN STRICTEST CONFIDENC They claim that most CRT and LCD displays can be modelled with this equation, where the displayed luminance, Ld, is calculated as a function of the luma (pixel value), L’, and the display gamma, γ, the peak display luminance, Lmax, the display black level, which is the luminance of a black pixel displayed in a perfectly dark room, L black, the display screen reflectivity, Rdisplay, and the ambient light level, Eambient. IN STRICTEST CONFIDENCE Figure 11 shows the mapping of pixel code levels to linear light in the presence of different levels of ambient light using this 2 2 equation from Mantiuk et al. for a display with peak display luminance 80cd/m , display black level of 1cd/m , reflectivity of 1% and IN STRICTEST CONFIDENCE gamma of 2.2. It can be seen that the effective dynamic range of a display gets compressed due to screen reflections, making lower pixel values almost indistinguishable. The dynamic range of nearly 2600:1 of sRGB is reduced only 75:1 in aCONFIDENCE dimly lit room, and to only 6:1 in IN to STRICTEST sun light.

Pixel Luminance (Logarithm to the base 10 of values measured in cd/m2)

2.0

1.5


1.0

0.5

0.0

sRGB Range 2584:1 IN: Dynamic STRICTEST CONFIDENC

-0.5

Dim Room (20 lux) : Dynamic Range 75:1 Light Room (150 lux) : Dynamic Range 54:1

-1.0

CONFIDENCE SunIN lightSTRICTEST (5000 lux) : Dynamic Range 6:1 -1.5 0.0

0.1

0.2

0.3

0.4 0.5 0.6 0.7 0.8 0.9 Luma pixel value (0.005 to 1.000) IN STRICTEST CONFIDEN

1.0

Figure 11. The mapping of pixel code levels to linear light in the presence of ambient light.

Mantiuk et al. addressed this issue with adaptive tone mapping. As ambient light increases, the image gets brighter to avoid dark tones, which are the most affected by the display reflections. For the outdoors illumination, many of the bright pixels are clipped to the maximum value. IN STRICTEST CONFIDENCE An example of the actual result of their tone mapping, and how it differs for different ambient light conditions, is shown in Figure 12. IN STRICTEST CONFIDENC

23

IN STRICTEST CONFIDENCE R. Mantiuk, S. Daly and L. Kerofsky, Display Adaptive Tone Mapping, ACM Transactions on Graphics, Vol. 27, No. 3, Article 68, Publication date: August 2008. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.178.751&rep=rep1&type=pdf IN STRICTEST CONFIDENCE


Page | 18





Figure 12. An image tone-mapped for three different ambient illumination conditions. IN STRICTEST CONFIDENCE 24

Kunkel and Daly stated that current displays are very thin and highly reflective perpendicular to the display: they are similar to mirrors! The earlier LCD displays had a matte, diffuse effect, with relatively low reflectivity, but the matte caused some image blur. Glossy LCDs eliminate the blurring effect, but at the expense of more reflections, making it important to position the screen and the viewer to minimise light sources affecting the experience. While this is controllable to some extent in the home environment, it is not so for outdoor use and signage. They state that as screens get brighter and brighter, the reflection of the viewer, illuminated by the light from the display, becomes more of an issue, but screen manufacturers may be reluctant to address this issue in the short term because such a TV looks good when turned off: the current design popular for marketing. 25

STRICTEST Sharp have developed technology to minimize screen reflection, which they refer to as “mothCONFIDENCE eye™” . This technology, showcased in October 2012 at CEATEC, Japan's largest consumer electronics show, involves applying an anti-reflecting coating to LCD panels based on technology similar to the nanostructure of a moth’s eyes: the surface of a moth's eyes is covered with bumps and valleys that absorb oncoming light, enhancing night vision. Unlike conventional anti-reflection technology, Sharp’s claimed its new LCD offers more vivid colour images and higher contrast. It demonstrated 60, 70 and 80 inch moth eye panels at CEATEC based on its Aquos large-screen TVs. Sharp said its panel technology is ready for deployment in commercial products for indoor use. But Kunkel and Daly comment that although moth-eye displays are diffuse, with even lower reflectivity than the earlier matte screens, they currently suffer from not being scratch resistant. IN STRICTEST CONFIDENC Thicker displays were demonstrated at CES 2014. Kunkel and Daly claim these allow better audio because larger speakers can be included, and that they also allow for better backlight modulation. They also comment on curved screens, stating that curvature may help with screen surface side effects, and may enhance immersion, and that curved displays may be good for mobile use as they may allow the user to more easily avoid reflections from external light sources. IN STRICTEST CONFIDENCE

3.8

The current state of high dynamic range capture and display technology

IN STRICTEST CONFIDEN Modern digital motion imaging sensors can originate linear video signals having dynamic ranges up to about 14 stops, that is, ranges 18 up to about 16,000:1. This dynamic range is similar to the simultaneous dynamic range of the human visual system . Such cameras include the Arri Alexa and Amira, various models by Canon including the EOS™ C300 and EOS C500, the Red Epic™, and various models by Sony including the A7S and the F65. 26

Cinema5D have reported results of tests carried out on a selection of cameras . They used the DSC labs XYLA-21, a high quality LEDSTRICTEST CONFIDENCE backlit transmissive chart that displays 21 stops of dynamic range: each verticalINbar in the chart represents one stop of light. The chart is filmed with each camera in turn in a completely dark room using the same very sharp Zeiss 50mm CP2 T/2.1 makro lens with interchangeable mount adjusted for the camera bayonet. Each camera was set to its native ISO and the F-stop of the lens was IN STRICTEST CONFIDENC 24

Timo Kunkel and Scott Daly, Dolby Labs, Inc., SMPTE Monthly Webcast: Lessons in Light: From Reality via Display to the Eye.

25

http://www.eetimes.com/document.asp?doc_id=1262615

26

IN STRICTEST CONFIDENCE http://www.cinema5d.com/dynamic-range-sony-a7s-vs-arri-amira-canon-c300-5d-mark-iii-1dc-panasonic-gh4/ IN STRICTEST CONFIDENCE


Page | 19

adjusted accordingly. The Intra frames were extracted from the recorded videoINand tested with CONFIDENCE software from IMATEST. The results STRICTEST are reproduced in Figure 13. It can be seen that the Arri and Sony cameras produced a usable dynamic range of about 14 stops, and the Canon cameras about 11-12 stops. Figure 13 also shows that these cameras use a variety of transfer functions to map linear light to digital code values. The RED Epic with a Mysterium-X™ sensor is claimed to support 13.5 stops of dynamic range, and up to 18 stops when using RED 27 HDRx . 28

The Panasonic VariCam™ 35 is claimed to support 14+ stops of dynamic range IN . STRICTEST CONFIDENC Dynamic range is considered more important to some camera manufacturers than increased spatial resolution. For example, Arri 29 has stated that it will not move to 4K until it can do so without compromising the dynamic range on its range of cameras, including the Alexa and the newly introduced Amira. They believe that they have the best dynamic range on the market today, and that there IN STRICTEST is interest from the creative community to create better pixels, not just more pixels. The Alexa isCONFIDENCE among the most widely-used digital cinematography cameras. IN STRICTEST CONFIDENCE



Figure 13. Camera dynamic range measurements by Cinema5D.

IN STRICTEST CONFIDENC Miller et al of Dolby Laboratories report that typical displays today are now achieving peak levels of 500cd/m or more, and that 2 there are several commercial examples above 1000cd/m . As an example, the Samsung OL46B - OL Series 46" Outdoor High 30 2 Brightness High Definition Display is stated to have a peak brightness of 1500cd/m . 17

2

2

However, typical televisions appear to have peak brightness levels up to about 350cd/m . For example the Toshiba 48" LED-backlit 31 2 32 IN STRICTEST LCD HDTV, 48L1435DB , has a brightness of 300cd/m ; and the Sharp 50" LED-backlit LCD HDTV,CONFIDENCE LC 50LE651K , has a brightness of 2 350cd/m . MPEG plans to use two high dynamic range displays in analysis of the submissions to its “Call forCONFIDEN Evidence” of new tools that may IN STRICTEST improve the performance of HEVC when used to encode high dynamic range and wide colour gamut video. Subjective testing will be done using a SIM2™ monitor in two locations and a Pulsar monitor in another location.

27

http://www.red.com/products/epic-mx

28

http://www.panasonic.com/business/provideo/varicam-models.asp


29

http://www.hollywoodreporter.com/behind-screen/nab-wrap-life-pi-cinematographer-695432 IN STRICTEST CONFIDENC 30 http://www.samsung.com/us/business/displays/digital-signage/LH46OLBPPGC/ZA 31

http://www.dabs.com/products/toshiba-48--full-high-definition-led-tv9GGH.html?refs=51570000&q=brightness&src=16#usedstock 32

IN STRICTEST CONFIDENCE http://www.dabs.com/products/sharp-aquos-50le651k-50--1920x1080-full-hd-smart-3d-tv-8XGK.html?refs=5469000051570000&src=3 IN STRICTEST CONFIDENCE


Page | 20

33

The sim2 is a 47inch HD display with 2202 individually controllable white LEDIN backlights, peak brightness of 4000cd/m2 and BT.709 STRICTEST CONFIDENCE colour gamut. It supports two input modes: HDR Yu’v’ where 12 bits are assigned for Y in a log representation, and 10 bits are assigned for u’ and v’ using linear encoding, with 4:2:2 sampling; and DVI plus mode, in which the input comprises an LED signal (2202 values) and the LCD signal (1920x1078, 8 bit RGB values). This display is known to have some issues: when using the HDR Yu’v’ input mode: chroma subsampling and different sample locations are issues. And when using DVI plus mode, direct knowledge of the display and the ability to accurately compute both the backlight and the RGB sample values is needed. IN 12 STRICTEST CONFIDENC The Dolby Pulsar has been used for demonstration and experimental purposes . It is a 42inch HD display with peak brightness of 2 34 4000cd/m , DCI-P3 colour gamut, and 12 bits per colour sample . It is likely that ‘higher dynamic range’ televisions will not reach the market as a step change, but that over time, televisions with increasing peak brightness will become available. As stated above, televisions are already available that support peak brightness IN STRICTEST CONFIDENCE 2 2 levels up to about 350cd/m . It is possible that this could steadily increase to 1000 cd/m and beyond. But a change in broadcast or storage media format will be necessary for these screens to benefit from true higher dynamic range rather than just increased brightness. IN STRICTEST CONFIDENCE

3.9

Interest and Experience in Hollywood

IN STRICTEST CONFIDENCE There’s a growing number in Hollywood who want high dynamic range, which, as it is independent of the number of pixels in a frame, can be used with any picture resolution. There is a growing number of industry veterans that say, given a choice of high dynamic range or ultra high definition, it is high dynamic range that creates the more noticeable improvement, and hence it is this 29 option that is generating a lot of interest . Emmanuel “Chivo” Lubezki, director of photography of the movie “Gravity”, is reported to have stated, “I think every cinematographer will have an interest in high dynamic range,” and that he would explore this potential 31 for all his future projects . A short film, entitled “Emma” was shot in high dynamic range by director Howard Lukk and cinematographer Daryn Okada. The first 35 four minutes of the 13 minute film was shown at the SMPTE 2014 Symposium, October 2014 . STRICTEST CONFIDENCE 36 An Alexa camera was used in ‘Open Gate’ mode , capturing at a resolution of 3414x2198, and capturing into the ARRIRAW format using the ACES colour space. It was reported that one of the biggest problems on set was monitoring: they were shooting a high dynamic range movie and using a standard dynamic range monitor. Consequently they had to rely on a light meter to estimate light drop-off in certain shots. Shooting in high dynamic range is reported to have added no extra time, but spending more time with the make-up artist would be beneficial. With high dynamic range it is possible to see so much more, not only in the actors’ faces but also details in the IN STRICTEST background, which might not be desirable. With high dynamic range it was reported to be more CONFIDENC difficult to hide things and hence they spent more time on masks to get the right compositions. On the subject of whether it was possible to convert previously shot projects into high dynamic range, it was reported: “If you shot on a very good dynamic range element, meaning film, you can probably pull out a high dynamic range out of it. If you’re trying to go back and pull it out of a video camera from 10 years ago it will be impossible.” IN STRICTEST CONFIDENCE



33

http://www.sim2.com/HDR/hdrdisplay/hdr47e_s_4k


34

“Better Pixels: Color Volume and Quantization Errors”, Robin Atkins, Dolby Labs. http://hollywoodpostalliance.org/wpcontent/uploads/2014/02/RAtkins_Th1030-noon_Color-Volume4.pdf 35

http://celluloidjunkie.com/2014/10/22/cjsmpte-conference-howard-lukk-presents-hdr-footage-emma/ IN STRICTEST CONFIDENCE 36 “Alexa XT Open Gate. Record it all: Open Gate sensor mode”. http://www.arrimedia.com/news/view/48/alexa-xt-open-gate IN STRICTEST CONFIDENCE


Page | 21

It was reported that the following would help with future high dynamic range productions: IN STRICTEST CONFIDENCE    

Finding a high dynamic range display to use on screen for monitoring Locking down a transfer function for the high dynamic range master such as ACES or OCES Settling aspect ratio, since scope on Ultra-HD has to be downsized and letterboxed (otherwise pan and scan) Settling on a standard light level

3.10 Dolby Cinema IN STRICTEST CONFIDENC Dolby Cinema™ was announced in December 2014. It will feature the Dolby Vision™ projection system, which is claimed to be able to deliver high dynamic range with enhanced colour: ‘its amazing contrast, high brightness, and colour range closely matches what the human eye can see’. They claim that the blacks are truly black, colours are vibrant, and the contrast ratio far exceeds that of any other image technology on the market today. The first Dolby Vision projectors and titles for theatrical exhibition are expected in IN STRICTEST CONFIDENCE 2015. 37

Early Dolby Cinema locations will be equipped with high-brightness laser projection technology available today with 4K, high-frameSTRICTEST CONFIDENCE rate 2D and 3D capabilities. When Dolby Vision content becomes available, theIN Dolby Cinema laser projection systems will be replaced with Dolby Vision projection systems. IN STRICTEST CONFIDENCE








http://investor.dolby.com/releasedetail.cfm?releaseid=885979 IN STRICTEST CONFIDENCE


Page | 22

4

Wider Colour Gamut


The human visual system is sensitive to electromagnetic radiation in the approximate range of wavelengths from 400nm (blueviolet) to 700nm (red), using three types of cone cell that have different spectral response.

4.1

The CIE RGB Colour Space

INofSTRICTEST CONFIDENC In the 1920s, W. David Wright and John Guild independently conducted a series experiments on human colour vision. The experiments used a circular screen, of size two degrees to the observers, showing on one side a single wavelength test colour and on the other side a viewer adjustable mix of three primary colours, red, green and blue. The viewers were asked to adjust the mixture of primary colours until they considered a match with the single wavelength test colour to have been achieved. IN of STRICTEST CONFIDENCE This was not possible for all test colours, where instead a variable amount of one the primaries was added to the test colour, and a match with the remaining two primaries was carried out. The CIE RGB colour matching functions were consequently defined using theseIN experimental results, using three monochromatic STRICTEST CONFIDENCE primaries at standardized wavelengths of 700nm (red), 546.1nm (green) and 435.8nm (blue). The primaries with wavelengths 546.1nm and 435.8nm were chosen because they are easily reproducible monochromatic lines of a mercury vapour discharge. The 700 nm wavelength, which in 1931 was difficult to reproduce as a monochromatic beam, was chosen because the eye's perception IN STRICTEST CONFIDENCE of colour is rather unchanging at this wavelength, and therefore small errors in wavelength of this primary would have little effect on the results. The colour matching functions are the amounts of primaries needed to match the monochromatic test colour. These colour matching functions, known as the "1931 CIE standard observer", are shown in Figure 14. The wavelengths at which one of the colour matching functions goes negative correspond to the test cases where a variable amount of one of the primaries was added to the test colour, and a match with the remaining two primaries was carried out: the test colour is effectively being matched by a positive contribution from two primaries and a negative contribution from the other. STRICTEST CONFIDENCE

0.40 0.35

R(λ) 0.30

G(λ) B(λ)

0.25 0.20


0.15 0.10

IN STRICTEST CONFIDENCE 0.05 0.00


-0.05 -0.10 400

450

500

550 Wavelength (nm)

600

650

700

IN STRICTEST CONFIDENCE Figure 14. The CIE 1931 RGB Colour Matching Functions.

The RGB tristimulus values, (R, G, B) for a colour with spectral power distribution, I(λ), can be calculated as the integral of the IN STRICTEST CONFIDENC product of the colour matching function (R(λ ), G(λ ), B(λ)) and the spectral power distribution, I(λ ), as shown below. Note that many different spectral power distributions correspond to the same RGB tristimulus values, and also appear to the human visual system to be the same colour.

∫

( ) ( )

∫

( ) ( ) IN STRICTEST ) ( ) ∫ ( CONFIDENCE IN STRICTEST CONFIDENCE


Page | 23

IN STRICTEST CONFIDENCE The chromaticity of a spectral power distribution, (r,g), can be calculated by normalising the RGB tristimulus values:

This results in the CIE rg chromaticity space as shown in Figure 15. The curve shown represents the colours of single wavelength. Colours within this locus can only be made by mixing light of different wavelengths together, including those colours that lie along IN STRICTEST CONFIDENC the line g=0. Points outside of the locus are not real colours and cannot be observed. 2.0

Green

IN STRICTEST CONFIDENCE 1.5


g


1.0

0.5

Blue

0.0 -1.5

-1.0

-0.5

Red STRICTEST CONFIDENCE 0.0

0.5

1.0

r

Figure 15. The CIE rg Chromaticity Diagram.

4.2

The CIE XYZ Colour Space

IN STRICTEST CONFIDENC 38 The CIE decided to map the RGB colour space onto a new derived colour space, XYZ, with derived colour matching functions, and derived chromaticity space, with the benefits that the derived colour space would satisfy the following constraints:  

The colour matching functions would be positive or zero at all wavelengths; and STRICTEST CONFIDENCE The Y colour matching function would be exactly equal to the photopicINluminous efficiency function V(λ) defined by the CIE in 1926 for the "CIE standard photopic observer”;

The RGB colour space is mapped onto the XYZ colour space using the followingIN linear transformation, which has been standardised STRICTEST CONFIDEN 39 by the CIE :

[ ]

(

)[ ] IN STRICTEST CONFIDENCE

IN STRICTEST CONFIDENC “How the CIE 1931 Color-Matching Functions Were Derived from Wright–Guild Data”, Hugh S. Fairman, Michael H. Brill, Henry Hemmendinger, http://infocom.uniroma1.it/~gaetano/texware/Full-How%20the%20CIE%201931%20ColorMatching%20Functions%20Were%20Derived%20from%20Wright-Guild%20Data.pdf 38

39

STRICTEST CONFIDENCE Publication CIE No. 15.2, Colorimetry, Second Edition, Central Bureau of the IN Commission Internationale de l’Éclairage, Vienna, Austria, 1986. IN STRICTEST CONFIDENCE


Page | 24

As described above for the RGB colour space, and shown in the equation below, chromaticityCONFIDENCE of a spectral power distribution, INthe STRICTEST (x,y), can be calculated by normalising the XYZ tristimulus values. And when this is done, by the requirement of positive values of X, Y and Z, the gamut of all colours lies inside the triangle [1,0], [0,0], [0,1].

40

The CIE 1931 xy chromaticity diagram, taken from Wikipedia , is shown in Figure 16, together with the RGB primaries: 700nm (red), 546.1nm (green) and 435.8nm (blue). All colours within this triangle defined byIN the three primaries can be made by mixing the three STRICTEST CONFIDENC primaries in the appropriate non-zero combination. Colours outside of this triangle can be made by mixing the primaries, but only if negative quantities are allowed. This is why in the Guild and Wright experiments, some single wavelength test colours had to be colour matched by mixing the test colour with one primary, and matching against a mixture of the other two primaries. Both the XYZ and the xyY colour spaces can be used in practice. The latter consists of the pure chromaticity values x and y, and the IN STRICTEST CONFIDENCE luminous value Y. IN STRICTEST CONFIDENCE




IN STRICTEST CONFIDENCE Figure 16. The CIE 1931 xy chromaticity diagram showing the CIE RGB primaries.

4.3

Perceptually Uniform Colour Spaces


The CIE 1931 XYZ colour space is not perceptually uniform, that is, a constant magnitude change in the tristimulus values does not correspond to a constant perceptual change. The CIE has developed colour spaces that are closer to being perceptually uniform. These include the Lab and Luv colour spaces standardised by the CIE in 1976. Both of these colour spaces are derived from the ‘master’ XYZ colour space, with dimension L for lightness and ‘a’ and ‘b’ or ‘u’ and ‘v’ for the colour dimensions. Both share the same definition of lightness, L, with L=0 indicating black, L=100 indicating diffuse white, and higher values indicating specular highlights, STRICTEST CONFIDENCE but they have different definitions for the colour dimensions. It is believed thatINLab is the more widely used, and that both were standardised as the standardisation committee could not reach agreement on a single one. IN STRICTEST CONFIDENC

40

IN STRICTEST CONFIDENCE http://en.wikipedia.org/wiki/CIE_1931_color_space#Color_matching_functions IN STRICTEST CONFIDENCE


Page | 25

4.4

Colour Television


Colour television has taken advantage of the above observations on the mixing of primary colours: by choosing three primary colours, and mixing them appropriately, all colours within the triangle on a chromaticity diagram defined by the three primary colours can be created. Cathode ray tube displays, the primary means of watching colour television until relatively recently, use three different phosphors which emit red, green, and blue light respectively, when excited by one of three beams of electrons. IN STRICTEST CONFIDENC LCD displays achieve colour by applying three colour filters, red, green, and blue, to the backlight. In both cases of CRT and LCD displays, the colour primaries are not single wavelength sources, but have quite spread spectra, as 41 42 shown in Figure 17, where the CRT image is from Wikipedia and the right image is recreated from Chen at al . IN STRICTEST CONFIDENCE



400

450

500

550 Wavelength (nm)

600

650

700

Figure 17. The spectra of display primary colours, typical CRT on the left, typical LCD with colour filters on the right.

The spectra of the colour primaries of display devices limit the range (gamut) ofSTRICTEST colours that can be shown on the display. The ITU-R CONFIDENCE Recommendation BT.709, and the earlier BT.601, defined a set of colour primaries, in terms of the CIE 1931 x and y chromaticity diagram, to be used in television systems. These colour primaries, and the colour space defined by them, are shown in the CIE 1931 43 xy chromaticity diagram of Figure 18, taken from Wikipedia . This system is still in use today, although as described fully below, there is significant interest in extending display specifications and consequently producing devices to support wider colour spaces. It can be seen clearly in Figure 18 that this colour space leaves a large set of colours that could not be displayed. Care should be taken when quantifying this, because, as pointed out above, CIE 1931 is not a perceptually uniform colour space. IN STRICTEST CONFIDENC





http://en.wikipedia.org/wiki/Cathode_ray_tube#Color_CRTs

42

“Quantum-Dot Displays: Giving LCDs a Competitive Edge through Color”, Jian Chen, Veeral Hardev, and Jeff Yurek. http://informationdisplay.org/IDArchive/2013/JanuaryFebruary/FrontlineTechnologyQuantumDotDisplays.aspx IN STRICTEST CONFIDENCE 43 http://en.wikipedia.org/wiki/Rec._709 IN STRICTEST CONFIDENCE


Page | 26






Figure 18. The primary colours of the BT.709 colour space, and its primary colours on the CIE 1931 xy chromaticity diagram.

4.5

The need for a Wider Colour Gamut

Current television systems do not support the wide range of colours that the human eye can perceive. Future television and other video distribution environments are hoped to give a viewing experience that is closer to a real life experience, to provide a user with a stronger sense of “being there”. This requires supporting both significantly higher dynamic ranges and wider colour gamuts than supported today. STRICTEST CONFIDENCE As stated above, Figure 18 shows the primary colours of the BT.709 colour space. These are joined to form a triangle, which is often referred to as the bounds of the colour space. But this is only true for low values of luminance. For higher values of luminance, the range of colours is reduced, such that for the maximum relative luminance of 1.0, the chromaticity gamut is only a single point, 44 being the white point of the colour space on the chromaticity plane . Hence in order to describe the full range of colours that can be represented in a colour space, for example that described in BT.709, it is necessary to consider a 3D colour volume. Assuming this 3D colour volume to be drawn with the x and y chromaticity coordinates in the horizontal plane, and the luminance, Y, vertically, then the horizontal cross section of the 3D colour volume is the INthe STRICTEST CONFIDENC triangle shown on Figure 18 up to a relative luminance of 0.0722, at which point blue primary cannot add to the relative luminance. In order to increase relative luminance of blue beyond this level, it is necessary to desaturate it by adding red or green or a mixture of the two. Hence from 0.0722 upwards, the cross section becomes a quadrilateral, with two vertices directly above the red and green primaries, and with two vertices inward from the blue primary, to an increasing extent as relative luminance is increased. At relative luminance of 0.2126, the red primary similarly cannot add to the relative luminance, and hence to increase IN STRICTEST CONFIDENCE relative luminance of red beyond this level, it is necessary to add blue or green or a mixture of the two. The cross section consequently becomes pentagonal, but only until relative luminance reaches 0.0722+0.2126 = 0.2848, at which the blue plus red vertex meets the red plus blue vertex, and the luminance of either can only be increased by adding green, and the cross section IN STRICTEST CONFIDEN returns to being a quadrilateral. At a relative luminance of 0.7152, the green primary can no longer add to the relative luminance on its own, and the cross section again becomes pentagonal. As relative luminance increases further, the cross section becomes quadrilateral yet again, and finally triangular at relative luminance of 0.9278, reaching an apex at relative luminance of 1.0000, and the D65 white point given by chromaticity coordinates (x=0.3127, y=3290). This is shown on the CIE 1931 xy chromaticity diagram in Figure 19, where in addition to the showing the spectral locus, labelled as ‘CIE 1931’, the cross section of the BT.709 3D colour volume is shown for various of luminance, showing that the cross IN values STRICTEST CONFIDENCE sectional area reduces as the relative luminance increases. IN STRICTEST CONFIDENC

44

IN STRICTEST CONFIDENCE http://www.openphotographyforums.com/forums/showthread.php?t=12322 IN STRICTEST CONFIDENCE


Page | 27


0.9

CIE 1931

0.8

Y = 0.072 Y = 0.142

0.7

Y = 0.249

0.6

Y = 0.500 IN STRICTEST CONFIDENC

0.5

Y = 0.858

Y = 0.751

y

Y = 0.964

0.4


0.3


0.2


0.1 0.0

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

x STRICTEST CONFIDENCE Figure 19. The horizontal cross section of the BT.709 3D colour volume for various values of luminance. Figure 20 shows another view of the 3D colour volume, this time showing a vertical cross section, with chromaticity coordinate, x, plotted horizontally and the relative luminance, Y, plotted vertically, for chromaticity y = 0.3290. The colours shown on the figure are illustrative, and clearly not precise. But it is can be seen, particularly on the right hand side, that as the relative luminance is increased, the range of colours that can be represented is reduced.

1.0


0.9 0.8 IN STRICTEST CONFIDENCE

0.7

Out of Gamut Colour

Y

0.6


0.5

0.4 0.3 0.2


0.1

0.0


0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

x Figure 20. A vertical cross section of the BT.709 3D colour volume for y =CONFIDENCE 0.3290. IN STRICTEST



Page | 28

24

This illustrates the need for both a wider colour gamut and a higher dynamic range, as stated byCONFIDENCE Kunkel and Daly , who quote an IN STRICTEST 45 example of a volcano at night, such as that illustrated in Figure 21 . Emissive colour, such as the molten lava of Figure 21, can be both very bright and very saturated, and outside of the 3D colour volume, as shown on Figure 20. Without a change to colour gamut or dynamic range, the orange of the volcano would have to be moved into the colour volume. This could be accommodated by making it darker, effectively moving it down in Figure 20 so that it moves into the colour volume or by making it less saturated, effectively moving it horizontally into the colour volume, or some combination of both. When using a higher dynamic range and a wider colour gamut, the colour volume increases in all three dimensions, allowing such a point to be represented: just having one of these aspects may not be enough. IN STRICTEST CONFIDENC The shape of the colour volume is such that at high luminance, it is not possible to have high saturation: while a display may be able 2 2 to display white with 1000cd/m , it may not be able to display green with 1000cd/m because of the way the display is built. Combining a wider colour gamut with a higher dynamic range will create a larger colour volume which can then transport more colours. IN STRICTEST CONFIDENCE



STRICTEST CONFIDENCE Figure 21. Emissive colour can be both bright and saturated.

4.6

Wider Colour Gamut Standards

The colour space defined in BT.709 has been widely adopted, and is common to other standards, including BT.1361, IEC 61966-2-1 (sRGB or sYCC), IEC 61966-2-4, and SMPTE RP 177 (1993) Annex B. IN STRICTEST CONFIDENC The DCI P3 colour space is used by digital cinema. It is a part of the SMPTE Recommended Practice RP 431-2 for “Digital Cinema 88 Quality – Reference Projector and Environment”, first released by the SMPTE in 2007, and most recently updated in 2011 . It defines the Reference Projector and its controlled environment, along with the acceptable tolerances around critical image parameters for Review Room and Theatre applications. Although the colour gamut is specified in terms of XYZ, a Digital Cinema IN STRICTEST CONFIDENCE projector is required to support a gamut of at least DCI-P3. The DCI-P3 colour space uses the same blue primary as BT.709, but it uses different green and red primaries. The red primary of DCIP3 is monochromatic 615 nm, a slightly deeper hue of red than BT.709. The green primary is a slightly more yellowish hue of green IN STRICTEST CONFIDEN but is more saturated. In CIE 1931 (x,y) coordinates, the DCI-P3 primaries are red (0.680, 0.320), green (0.265, 0.690) and blue (0.15, 0.060). Additional colour spaces with wider gamut have been defined with the aim of supporting wider colour gamuts in the future. The ITU-R has specified a colour space in BT.2020 using single wavelength primaries. In addition, specifications have been defined that cover the full visible gamut. For example, SMPTE specified in ST 428-1 usage of the CIE 1931 XYZ colour space, and the Academy INXYZ, STRICTEST Color Encoding System (ACES) defined the ACES colour space, which is similar to but slightlyCONFIDENCE smaller while still encompassing the whole spectral locus. In CIE 1931 (x,y) coordinates, the ACES primaries are red (0.73470, 0.26530), green (0.00000, 1.00000) and blue (0.00010, -0.07700). IN STRICTEST CONFIDENC Figure 22 shows the BT.709, DCI-P3, BT.2020, ACES and XYZ colour gamuts on the CIE 1931 xy chromaticity diagram.

45

IN STRICTEST CONFIDENCE http://commons.wikimedia.org/wiki/File:Paricutin_30_613.jpg?uselang=en-gb IN STRICTEST CONFIDENCE


Page | 29

4.7

Conversion between Colour Gamuts


As colour gamuts are specified relative to CIE standards, conversion between colour gamuts is possible. When converting from one colour gamut to another, where the target gamut fits fully within the originating gamut, is straightforward, being essentially a mathematical transformation. For example, as described later in this report, the ITU-R is understood to be developing a new Recommendation, to specify a method of colorimetry conversion from BT.709 to BT.2020. Such a conversion could be needed for example when HDTV programme content is included within UHDTV programmes. However, other conversions, for example from BT.2020 to BT.709, are more problematic as there may be colours in the originating IN STRICTEST CONFIDENC colour space that cannot be represented in the target colour space. These ‘out of gamut’ colours cannot be represented accurately, and so some loss in quality must be tolerated. The problem is that while out of gamut colours need to be changed to bring them into the gamut, colours that are not outside the gamut do not need to be changed, and the larger any change to them is, the more degradation occurs, but if they are not changed, they will no longer have an appropriate relationship to colours that are moved into IN STRICTEST CONFIDENCE the gamut. A high quality gamut mapping algorithm should minimise the change of colours in the original representation that are within the gamut of the target colour space, minimise the perceived changes of hue, minimise changes to brightness, contrast and saturation, IN STRICTEST CONFIDENCE avoid loss of spatial detail, and avoid creating visible discontinuities. 46

Frohlich et al report that colour gamut conversion is a problem today in the cinema industry where newer digital cinema IN STRICTEST CONFIDENCE projectors are capable of a wider colour space than older projectors, and where content is now being produced for these newer projectors, but where the issue of supporting older projectors with a narrower colour space remains. Current practice consists of simply clipping each colour component (RGB) to fit within the target colour space, but this is reported to result in hue shifts and loss of detail. Frohlich et al take the common view that a perceptually uniform colour space is best suited for performing such a conversion: a perceptually uniform colour space is one in which equal (Euclidean) distances in the colour space correspond to equal perceived colour differences. But rather than using the popular CIE L*a*b* uniform colour space, they prefer one defined by Ebner and 47 Fairchild known as IPT. STRICTEST CONFIDENCE They report the results of subjective tests comparing algorithms that they refer to as PCLIP and WmindE. PCLIP, short for Post-CLIP is assumed to be implemented in projectors today, and consists of simple clipping of each colour component (RGB) in the projector’s native colour space. This is reported to result in hue shifts and loss of detail. The tested alternative, WmindE, which is short for weighted minimum delta E, selects the in-gamut colour with the minimum colorimetric distance for each out-of-gamut colour, but where a different weight is used for each of the components, lightness, chroma and hue. They report that in their subjective tests, WmindE in the IPT colour space was found to offer better detail preservation than PCLIP while retaining hue; and that WmindE was preferred over PCLIP by 80% of the viewers. They claim that all of the algorithms that they tested except PCLIP are too complex to render in real-time in current projectors, but that they could be implemented in INaSTRICTEST 3D lookup table. CONFIDENC

4.8

How large a Colour Gamut is needed?

While the chromaticity diagrams referred to above indicate the full gamut of colours that the human visual system can perceive, it is IN STRICTEST CONFIDENCE difficult if not impossible to produce displays that show the full gamut. Particularly, if displays are limited to three primary colours, the largest gamut that could be created is one defined by the largest triangle that could be placed within the chromaticity diagram. 48

IN STRICTEST CONFIDEN But there is still a question of how large the gamut needs to be, to provide a realistic viewing experience. The work of Pointer addresses this question. A maximum gamut for real surface colours, known as “Pointer’s Gamut”, was derived from the analysis of the colour coordinates of 4089 samples of real surfaces. Figure 22 shows Pointer’s Gamut of real colours on the 1931 CIE xy chromaticity diagram, and the standardised colour spaces, BT.709, DCI P3, BT.2020, ACES and XYZ. It can be seen that while BT.709 and even DCI P3 do not cover the whole of Pointer’s Gamut, IN STRICTEST CONFIDENCE 46

Jan Frohlich, Andreas Schilling, and Bernd Eberhardt, "Gamut Mapping for Digital Cinema," SMPTE Conf. Proc. vol. 2013, no. 10, IN STRICTEST CONFIDENC pp. 1-11 (2013). 47

F. Ebner and M. Fairchild, Development and Testing of a Colour Space(IPT) with Improved Hue Uniformity, in the proceedings of the Sixth Color Imaing Conference: color Science, Systems, and Applications, page 8-13, 1998. 48

IN STRICTEST CONFIDENCE The Gamut of Real Surface Colours, M. R. Pointer, Color Research & Application, Volume 5, Issue 3, pages 145–155, Autumn (Fall) 1980 IN STRICTEST CONFIDENCE


Page | 30

49

BT.2020 does in fact cover 99.9% of it . Hence, although there are many colours could not be described in BT.2020, there are IN that STRICTEST CONFIDENCE very few naturally occurring colours that could not be described. So BT.2020 would appear to be a sufficient colour space for television systems to display realistic images.

1.0 0.9

CIE 1931 Pointer's Gamut


0.8

BT.709 and sRGB

0.7

DCI P3 BT.2020

0.6


ACES

0.5

y

XYZ IN STRICTEST CONFIDENCE

0.4 0.3


0.2 0.1 0.0 -0.1

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 x

STRICTEST Figure 22. Comparison of Pointer’s Gamut of real colour and standardisedCONFIDENCE colour spaces.

4.9

Display Technology for Wider Colour Gamut

LCD displays produce images by selectively filtering a backlight. Many individual LCD shutters, arranged in a grid, open and close to allow a metered amount of the light through, each shutter being paired with a coloured filter to remove all but the red, green or blue portion of the light from the original white source. The shade of colour is controlled by changing the relative intensity of the 50 light passing through the LCD shutters . IN STRICTEST CONFIDENC The colour gamut that can be produced by a conventional CCFL (cold cathode fluorescent lamp) backlight LCD is dependent on the spectrum produced by the backlight and the bandwidth of the filters used to produce the primaries. The gamut can me made larger by using narrower band filters on the backlight. However this also lowers screen luminance by IN STRICTEST CONFIDENCE decreasing the proportion of the backlight that passes through the filters. Increasing the luminance of the cold cathode to counter 51 this effect tends to shorten the life of the device and often results in lighting irregularities . Wide gamut cold cathode fluorescent lamps are reported to be under development by many manufacturers, IN STRICTEST CONFIDEN but not yet widely 52 adopted because of the shorter lifetime and lower optical efficiency . The use of white LED backlighting is now popular. Typically, white light is produced by using blue LEDs with YAG (yttrium aluminium garnet) phosphor. Such ‘white’ light typically has a spectral peak in blue, from the LED, and a broad peak around yellow from the phosphor. After red, green and blue colour filtering, with relatively wide band filters to maintain a reasonable amount of brightness in the red and green primaries, the result is usually a smaller colour gamut than CCFL backlit displays. However, such displays can IN STRICTEST CONFIDENCE

49

http://www.tftcentral.co.uk/articles/content/pointers_gamut.htm

50

http://en.wikipedia.org/wiki/LCD_television

51

http://www.eizoglobal.com/library/basics/lcd_monitor_color_gamut/


52

IN STRICTEST CONFIDENCE “Five-Primary-Color LCDs”, Hui-Chuan Cheng, Ilan Ben-David, and Shin-Tson Wu, Journal of display technology, Vol. 6, No. 1, January 2010, http://ieeexplore.ieee.org/iel5/9425/5350900/05350902.pdf IN STRICTEST CONFIDENCE


Page | 31

achieve high dynamic range, alternatively referred to as high contrast, by dimming the LEDs in the dark areas of the image being IN STRICTEST CONFIDENCE displayed. RGB LED backlighting can also be used. Small groups of pixels in the LCD panel are backlit using triads of controllable red, green and blue LEDs. This increases the display’s colour gamut and the bit depth, as well as providing deep black levels due to localised 53 dimming of the backlight . The Dolby Professional Reference Monitor PRM-4200 is an example, using dual modulation technology, where the backlight unit consists of approximately 1,500 RGB LED triads, modulated on a frame-by-frame basis, that directly 54 55 illuminate the LCD panel . In 2013, it was reported that this was the only commercially available monitor capable of displaying the DCI P3 colour gamut. However since then other products have been announced. IN STRICTEST CONFIDENC Another means to widen the gamut is to use more than three colour primaries. This has been demonstrated in projection displays and some direct-view LCDs. However, the display brightness is reduced and cost is increased because of the increased number of 52 colour pixels and fabrication complicities . IN STRICTEST CONFIDENCE Quantum dots is another backlight technology, where blue LEDs are used, but instead of using a phosphor to produce white light, a layer of quantum dots is used. These act in a similar way to a phosphor, absorbing the blue light and emitting light with a different spectrum, which is determined by the size of the nanostructures. This allows narrow bandwidth light at the desired colour primary IN STRICTEST CONFIDENCE frequencies to be produced. Jeff Yurek of Nanosys, one of a number of organisations developing quantum dot technology, has reported that it is practical to produce a display that covers over 97% of BT.2020 using quantum dot technology, and has claimed to demonstrate over 91% IN STRICTEST CONFIDENCE BT.2020 just by modifying an off-the-shelf, standard LCD TV set with a specially tuned sheet of Quantum Dot Enhancement Film 56 (QDEF) . The first TV manufacturer shipping TVs of this kind was Sony in 2013, using the trade mark Triluminos™, and incorporating QD 57 58 Vision’s quantum dot technology Color IQTM™ . Quantum dot technology is also used in the Kindle Fire™ HDX 7 . Samsung Electronics, LG Electronics, the Chinese TCL Corporation and Sony showed quantum dot enhanced LED-backlighting of LCD TVs at 59 the Consumer Electronics Show 2015 . 60

Samsung introduced an extensive range of SUHD™ TVs at the Samsung Forum in Monaco in February 2015 . These UHD displays STRICTEST use quantum dot technology, referred to by Samsung as nano-crystal technology, to achieve CONFIDENCE a wide colour gamut, assumed to be consistent with DCI P3 due to their announcement of collaboration with 20th Century Fox who re-mastered multiple scenes from its film, “Life of Pi”, specifically for the SUHD TV. Another technology that promises to be able to achieve wide colour gamut is organic light-emitting diode (OLED) technology. Unlike LCD technology, OLED does not require backlighting to function. Light is produced by a process known as electrophosphorescence 61 when current flows through an emissive layer of organic molecules, such as polyaniline . The colour of light emitted depends on the exact organic makeup of the molecules. As there is no backlight, an OLED can display deep black levels and can consequently IN STRICTEST CONFIDENC achieve a higher contrast ratio than an LCD. Active Matrix OLED technology (AMOLED) is used in a number of smartphones, including leading models produced by Samsung. 62

63

Sony announced the BVM-X300 monitor at the Hollywood Post Alliance Tech Retreat, 9-13 February 2015 . Sony claims that this is its first OLED master monitor to combine UHD resolution, high dynamic range and wide colour gamut display. Sony claims that this IN STRICTEST CONFIDENCE 53

“Color Correction Handbook: Professional Techniques for Video and Cinema”, Alexis Van Hurkman IN STRICTEST CONFIDEN 54 http://www.dolby.com/us/en/professional/cinema/products/dolby-professional-reference-monitor-prm-4220-white-paper.pdf 55

http://www.noteloop.com/kit/display/color-space/dci-p3/

56

http://dot-color.com/2014/05/22/is-rec-2020-really-practical/

57

https://blog.sony.com/press/sony-announces-2013-bravia-tvs/

58 59

IN STRICTEST CONFIDENCE http://www.tested.com/tech/tablets/459137-kindle-fire-hdx-7-and-nexus-7-handily-beat-retina-ipad-mini-display-shoot-out/ http://en.wikipedia.org/wiki/Quantum_dot_display

IN STRICTEST CONFIDENC http://www.samsung.com/uk/news/local/samsung-reinforces-tv-leadership-in-europe-with-new-tizen-powered-suhd-and-smarttvs 60

61

https://pcmonitors.info/articles/oled-monitors/

IN STRICTEST CONFIDENCE https://pro.sony.com/bbsc/assetDownloadController/BVM-X300_spec_sheet.pdf?path=Asset%20Hierarchy$Professional$SEL-yfgeneric-153697$SEL-yf-generic-153718SEL-asset-473447.pdf&id=StepID$SEL-asset-473447$original&dimension=original 62



Page | 32

30-inch monitor has the ability to display ITU-R BT.709 and DCI-P3 colour gamuts accurately than any previous Sony IN more STRICTEST CONFIDENCE Trimaster™ display, and in addition, can display 80% of the ITU-R BT.2020 colour gamut. 64

The Mitsubishi launched the LaserVue® Series of DLP TVs in 2008 that were powered by lasers and which they claim was able to achieve 91.4% of the BT.2020 colour gamut. They also claim that of as of 2014, no other commercially available display has been able to achieve such a wide colour gamut. However, the displays, which retailed around $6000 were discontinued in December 2012 due to the inability to compete with the shrinking size and prices of LCD TVs. Displays that rely on a laser backlight system are understood to be very power-hungry and consequently impractical, and IN STRICTEST CONFIDENC considering the European Commission eco-design directive, which is discussed further later in this research paper, unlikely to be legal in Europe due to low power efficiency. NHK laboratories are believed to have developed a laser projector, which is believed to be the only system currently capable of achieving the BT.2020 colour space, but it is likely to be very demanding of power and hence impractical for home use. IN STRICTEST CONFIDENCE









63

http://blog.sony.com/press/sony-expands-trimaster-el-series-with-first-oled-designed-for-pro-video-production/ IN STRICTEST CONFIDENCE 64 http://www.noteloop.com/kit/display/wide-gamut/mitsubishi-laservue-dlp-tv/ IN STRICTEST CONFIDENCE


Page | 33

5

Higher Frame Rate


Motion reproduction is an important feature of television: to make a sequence of still images appear to be natural movement. When the frame rate is too low, artefacts are perceived by the viewer, which can be classified into flicker, motion blur, and jerkiness (stroboscopic effect). Flicker is a phenomenon in which the whole screen or a large area of the screen “flickers” due to low frame frequency. Motion blur is due to the accumulation mechanism of the capturing devices. It also happensINwhen moving images are tracked on the screen of a STRICTEST CONFIDENC hold-type display. Non-smooth reproduced motion at low frame frequencies is called jerkiness. The use of the shutter during acquisition can produce a multi-exposure-like image even at high frame frequencies. This phenomenon is sometimes called jerkiness as well. Motion blur and jerkiness depend on the object speed. The object speed (arc-degree/second) generally tends to become faster as the IN field of view of the system becomes wider. For STRICTEST CONFIDENCE example, given the same shooting angle, an object on an HDTV screen will move faster than the one on an SDTV screen, assuming that screen sizes and viewing distances are such that the extra detail of HDTV can be observed, that is, the HD screen fills a larger part of the viewer’s field of view than the SD screen. Consequently high definition TV systems have stricter motion portrayal IN STRICTEST CONFIDENCE requirements and UHDTV systems higher still. SDTV, HDTV, and current UHD systems are specified at frame rates up to 60 frames per second. However, it is claimed that the UHDTV viewing experience could be improved by the use of higher frame rates.IN STRICTEST CONFIDENCE

5.1

The artistic impact of frame rate

The choice of frame rate may sometimes be determined by whether the scene contains normal or fast movement, but it is more frequently due to the director of photography’s selection of a programme “look”: low frame rates are often used to create a “film like” look, regardless of the speed of movement or action in the scene. High frame rates are often used to obtain a more contemporary look with increased motion clarity, and/or to obtain sharper slow motion images. 65

The cinema industry still favours 24 frames per second. Daly et al. comment on this, statingCONFIDENCE that in addition to the historical STRICTEST association of motion judder with cinema, the motion blur associated with this frame rate allows the viewer’s attention to be directed to parts of the scene in a similar way to the use of a shallow depth of field; motion blur is useful for hiding cinema craft (e.g., fake beards); and aids the suspension of disbelief, as realistic imaging could hinder the viewer’s imagination.

5.2

Historical choices of frame rate 66

In the early days of television, it was considered sufficient for the frame rate to be high enough merely to exceed the threshold for IN STRICTEST CONFIDENC “apparent motion”, above which a sequence of recorded images appear to the eye as containing moving objects rather than being a succession of still photographs. Another factor taken into consideration was that the frame rate should be high enough that flicker was imperceptible on contemporary televisions. However, priority was not given to the elimination of motion artefacts such as 67 smearing and jerkiness, possibly as contemporary technology may have limited the benefits of a higher frame rate anyway . IN STRICTEST CONFIDENCE

5.3

The motion blur – jerkiness trade-off

Current television frame rates cause problems for motion portrayal. StationaryIN objects STRICTEST are sharp,CONFIDEN provided they are in focus, but objects that move with respect to the camera smear due to the integration time of the camera’s sensor. Shuttering the camera to shorten the integration time reduces the smearing, but the motion breaks up into a succession of still images, causing jerkiness. The perceptual difference between moving and stationary subjects is increased with the increasingly sharper images due to new television systems with successively higher spatial resolutions, so long as the temporal resolution remains unchanged. The problems smearing, jerkiness or a combination of the two, are more noticeable with larger displays where the eye tends to follow the motion across the scene. IN STRICTEST CONFIDENCE

IN STRICTEST CONFIDENC "A Psychophysical Study Exploring Judder Using Fundamental Signals and Complex Imagery", Scott Daly, Ning Xu, Jim Crenshaw, and Vickrant Zunjarrao, SMPTE Annual Conference, 2014 65

66

Zworykin, V. K. and Morton, G. A., 1940. Television: The Electronics of Image Transmission. Wiley. New York.

67

IN STRICTEST CONFIDENCE BBC White Paper WHP 209: “Higher Frame rates for more Immersive Video and Television”, R.A. Salmon, Mike Armstrong, Stephen Jolly. http://downloads.bbc.co.uk/rd/pubs/whp/whp-pdf-files/WHP209.pdf IN STRICTEST CONFIDENCE


Page | 34

67

Figure 23, copied from the BBC White Paper 209 , shows the problem in termsINofSTRICTEST the movement of a ball across a plain CONFIDENCE background. In the top illustration, the trajectory of the ball is shown as if captured by a video camera with a very short shutter: each frame would show the ball “frozen in time”, and the motion would appear jerky when the video sequence was replayed. In the middle illustration, the effect of a half-open shutter is depicted: camera integration smears the motion of the ball out over the background, removing any spatial detail and making it partially transparent. These effects would be clearly visible in the final video sequence. The bottom image shows the effect of doubling the frame rate: both the smearing and jerkiness are reduced. A substantial further increase in frame rate would still be required in this example to eliminate their effects. IN STRICTEST CONFIDENC





Figure 23. The effects of frame rate and shuttering on motion portrayal.

Without a change in frame rate and shutter speed, it takes only a small amount of motion, estimated by the BBC as three pixels per displayed image, to lose the benefits of HD over SD resolution. The problem becomes more severe as resolution increases to UHD. Just as shuttering in the camera reduces the extent of smearing, a sample-and-hold characteristic in the final display increases it in a IN STRICTEST CONFIDENC directly comparable fashion. This smearing arises with motion that is tracked by the viewer, where the viewer’s eye is following the object across the screen, but where within each displayed image the object remains stationary for duration of the frame or field. LCD televisions have this characteristic, which is one reason why these displays have a reputation for representing fast-moving material, such as sport, poorly. LCD television manufacturers have added functionality to perform a motion-compensated frame IN STRICTEST CONFIDENCE rate increase, which ameliorates the problem to some extent at the cost of introducing other artefacts when the motion becomes too hard to predict, and during cuts and cross-fades.

5.4

Subjective evaluation of moving picture quality


68

Emoto et al have reported the results of experiments using a high-speed HD camera and projector that operate up to 240 frames per second, using typical television content including sports scenes. The content was presented at 60, 120 and 240 frames per second and viewers were asked to score on the five grade quality scale. The content was presented on a 100inch display, and the 60 viewers observed from 3.7m from the screen, that is, about three screen heights away. The results, as illustrated in Figure 24, show IN STRICTEST that the scores increase as the frame frequency increases, while the extent of the improvement CONFIDENCE depends on the content. The increase in scores when going from 60 to 120 frames per second range from 0.14 to 1.04, while the average increases across all of 19 the content are 0.46 from 60 to 120 frames per second, and 0.23 between 120 and 240 frames per second . IN STRICTEST CONFIDENC

68

IN STRICTEST CONFIDENCEIDW'09, pp. 1233M. Emoto and M. Sugawara, “Flicker perceptions for wide-field-of-view and hold-type image presentations”, 1234, 2009 IN STRICTEST CONFIDENCE


Page | 35







IN STRICTEST CONFIDENC Figure 24. Results of subjective evaluations of picture quality at higher frame rates. 69

The Broadcast Technology Future’s Group (BTF) , a collaboration of the EBU, IRT, BBC, Rai, and NHK, looked at benefits of higher frame rates. This work used five test clips (train, bike, runner, football, and carousel, and shown in Figure 25), filmed at 240 fps with 100% shutter, and created lower frame rate versions from these, with different simulated shutter settings. The lowest frame rate IN STRICTEST CONFIDENCE was 30 fps. A 55 inch 1080p display was used as a higher resolution display would not support the higher frame rates. 26 viewers observed the content at 60, 120 and 240 frames per second at a distance of 3H, the point at which the 1080 HD pixels can just be discerned. IN STRICTEST CONFIDEN The BTF reported a statistically significant improvement for all the test sequences when the frame rate was increased from 60 fps to 120 fps and when it was increased from 120 fps to 240 fps. The results averaged over the five test clips are shown in Figure 26. The difference between 60 fps and 120 fps was one grade, and the difference between 60 fps and 240 fps was nearly two grades. The 19 BTF stated that there was some perceptible improvement in using a shorter (50%) shutter, but the results were not conclusive . IN STRICTEST CONFIDENCE



https://tech.ebu.ch/groups/btf IN STRICTEST CONFIDENCE


Page | 36




Figure 25. BTF Test Material.




Figure 26. BTF Average Experimental Results. 70

Kuroki et al. report results on the subjective impact of frame rate, where motion images from a high-speed camera and computer graphics were shown on a high-speed display. The camera captured at 1000 frames per second, and the resulting images were IN STRICTEST processed to simulate various frame rates and shutter speeds. A 480Hz CRT display was used to CONFIDENC present the content. Viewers were asked to evaluate the difference in quality between motion images at various frame rates. Their results show that a frame rate of 120 frames per second provides good improvement compared to 60 frames per second, and that the maximum improvement beyond which evaluation is saturated is found at about 240 frames per second for representative standard-resolution natural images. IN STRICTEST CONFIDENCE 71

Nolan described a theoretical approach to frame rates and the human visual system, combining sampling theory and the human contrast sensitivity function model, for viewing content at 1.5 picture heights from the display, for tracked and non-tracked motion. IN STRICTEST When using a 100% shutter, Nolan reported that to match the temporal resolution to the spatialCONFIDEN resolution, assuming classical sampling, for UHD content, the frame rate required was 140 frames per second for non-tracked motion and 700 frames per second for tracked motion. But it was also noted that an acceptable balance of motion blur and strobing is likely to be possible at a lower frame rate, and that there is still a need to understand the right balance between the visibility of blur (tracked) and strobing (nontracked) for a 100 fps frames per second system. IN STRICTEST CONFIDENCE

IN STRICTEST CONFIDENC “A psychophysical study of improvements in motion-image quality by using high frame rates”, Y. Kuroki, T. Nishi, S. Kobayashi, H. Oyaizu, and S. Yoshimura, Journal of The Society for Information Display, Volume 15, Issue 1, pages 61–68, January 2007. http://www.researchgate.net/publication/245414179_A_psychophysical_study_of_improvements_in_motionimage_quality_by_using_high_frame_rates IN STRICTEST CONFIDENCE 70

71

"The Human Visual System and High Frame Rate Television", Katy C. Noland, UHDTV: Voices & Choices, 25th November 2013 IN STRICTEST CONFIDENCE


Page | 37

72

The BBC demonstrated higher frame rate technology during the Commonwealth 2014, atCONFIDENCE the Glasgow Science Centre , to INGames, STRICTEST show the potential benefits of watching sport at higher frame rates. Content was captured using a For-A FT-One camera that is designed for use in slow-motion action replays: it is a 4K resolution camera, capable of recording a 10 second shot at 900 frames per second into its internal memory uncompressed. Content was captured and stored at 400 frames per second, and converted to HD resolution at 50 and 100 frames per second for display using two ceiling mounted Barco F85 projectors. A large portion of the public were reported to be able to see the benefits of higher frame rate straight away and were very impressed; others needed some prompting on where to look and eventually could appreciate the difference, while others couldn’t see the differences at all, evenIN with lots of prompting. However, the latter were STRICTEST CONFIDENC reported to be a small minority, about 5% of the visitors.

5.5

The impact of lighting frequency on frame rate

IN STRICTEST CONFIDENCE The BBC have reported to the ITU-R the results of experiments on capturing content at 120 frames per second when illuminated with lighting at 50Hz. The results of these tests have led to the conclusion that 120 frames per second acquisition within 50 Hz 19 territories would give rise to significant visual artefacts . IN STRICTEST CONFIDENCE While it would be desirable to have a worldwide agreement on a single higher frame rate for UHDTV, this work demonstrated that within 50 Hz territories there would be significant interaction between the camera settings and the different lighting types that are currently in common use. IN STRICTEST CONFIDENCE Many locations are equipped with legacy lighting systems that are not under the control of the broadcaster, and many events use lighting primarily for the benefit of the audience at the location rather than for the television coverage. These lights are suitable for 25/50 fps and 29.97/59.94 fps systems, but have been found to have shortcomings when using higher frame rates that are not related to the mains supply frequency in use at the acquisition location. NHK reported results on the same topic, reporting on measuring the flickering characteristics of lighting equipment for programme production as well as the waveform of the captured image at 120 fps, finding that the flickering frequency of the light must be equal to or higher than the frame frequency in order not to generate the flicker artefact by the flickering lights with any temporal characteristics. They also reported the trend of lighting equipment which is shifting to higher CONFIDENCE flickering frequency than the power STRICTEST 19 frequency . In the latest version of BT.2020, table 2, “Picture temporal characteristics”, does not include the frame rate of 100 frames per second, but, perhaps due to lack of consensus within the ITU-R on the inclusion of this higher frame rate, instead states in a footnote to the table “The additional frame rate of 100 Hz is used in a number of 50 Hz countries”.






72

“Higher frame rate television for future broadcasts at the Commonwealth Games – “But I already have a 100Hz TV…?””, IN STRICTEST CONFIDENCE http://www.bbc.co.uk/rd/blog/2014/08/higher-frame-rate-television-for-future-broadcasts-at-the-commonwealth-games-but-ialready-have-a-100hz-tv IN STRICTEST CONFIDENCE


Page | 38

6

Standardisation


A number of standardisation bodies are involved in the specification of digital television standards. Figure 27 shows a simplified view of this from a UK perspective for distribution of television to end users. The DTG publishes and maintains the technical specifications for Freeview, known as the D-Book. This primarily references DVB standards, and TS 101 154 in particular. This DVB standard makes use of media coding and multiplexing from ISO/IEC, notably the so-called MPEG Transport Stream format and video compression codecs, MPEG-2, H.264/AVC and HEVC. This DVB standard also makes use of ITU-R Recommendations on TV signal IN colour STRICTEST CONFIDENC formats, including BT.709, BT.1886, and BT.2020 which define transfer functions, primaries and frame rates. Other standardisation bodies are also important but not shown. SMPTE produces standards that are of primary interest for the film industry and for contribution networks, being those networks used for moving content between production locations up to the point of final encoding for distribution to end users. IN STRICTEST CONFIDENCE

DTG UK Digital TV Standards IN STRICTEST CONFIDENCE (DTG D-Book)


DVB Digital TV Standards (TS 101 154)


MPEG Audio-visual Compression (ISO/IEC 13818-1, ISO/IEC 14496-10, etc.)

ITU-R TV Signal Formats IN STRICTEST (ITU-R BT.709, CONFIDENC ITU-R BT.1886, etc.)


Figure 27. The relationship between standards bodies involved in UK digital television standards.

6.1

ITU-R


ITU-R SG6 WP6C is a Working Party within the International Telecommunication Union Radiocommunication Sector (ITU-R) concerned with signal formats for the making and exchange of television and radio programmes, and ways to evaluate picture and sound quality. WP6C currently has responsibility for standardisation of UHDTV, including high dynamic range, higher frame rates and wider colour gamut. IN STRICTEST CONFIDENCE





Page | 39

IN STRICTEST CONFIDENCE 6.1.1

Parameters for Digital Television 73

ITU-R Recommendation BT.601 , “Studio encoding parameters of digital television for standard 4:3 and wide-screen 16:9 aspect ratios”, includes the definition of colour and opto-electronic transfer (OETF) characteristic for digital television. In CIE 1931 (x,y) coordinates, the colour primaries for 625 line television systems red (0.640,CONFIDENC 0.330), green (0.290, 0.600), and IN are STRICTEST blue (0.150, 0.060), and for 525 line television systems are red (0.630, 0.340), green (0.310, 0.595), and blue (0.155, 0.070). The reference white is D65 (0.3127, 0.3290), for both systems. The opto-electronic transfer function, commonly known as ‘gamma’, is defined as below as a combination of a linear segment near black and a power law segment for the majority of the range. IN STRICTEST CONFIDENCE { IN STRICTEST CONFIDENCE 74 ITU-R Recommendation BT.709 , “Parameter values for the HDTV standards for production and international programme exchange”, defines the image format parameters and values for HDTV. It defines the same transfer function as BT.601, above, and defines similar colour primaries, which again in CIE 1931 (x,y) coordinates, are red (0.640, 0.330),CONFIDENCE green (0.300, 0.600), and blue IN STRICTEST (0.150, 0.060). The reference white is D65 again. 75

ITU-R Recommendation BT.1886 , “Reference electro-optical transfer function for flat panel displays used in HDTV studio production”, specifies the reference electro-optical transfer function (EOTF) that the displays used in HDTV programme production should follow in order to facilitate consistent picture presentation. The reference EOTF is specified as a simple equation, with exponent function, as shown below, based on measured characteristics of the Cathode Ray Tube (CRT). The parameters ‘a’ and ‘b’ correspond to the legacy user controls, with ‘a’ representing the gain or ‘contrast’ control, and ‘b’ representing the black level lift or ‘brightness’ control. (

[(

) ]) STRICTEST CONFIDENCE

It was only in 2011, by which time the consumer market for CRTs had almost completely disappeared, that this standard for an EOTF was finally agreed. It effectively documents the characteristics of CRT displays. It does not specify a mathematical inverse of the OETF specified in BT.601 and BT.709, which by taking into account the linear portion, is approximately a power law function with exponent 0.5. The combined transfer function, sometimes known as the “end-to-end gamma” or “system gamma”, is approximately a power law function with exponent 1.2, which has been stated to have been designed to compensate for the subjective effects of 18 viewing pictures in a dark surround at relatively low brightness . IN STRICTEST CONFIDENC 76 ITU-R Recommendation BT.2020 , “Parameter values for ultra-high definition television systems for production and international programme exchange”, defines the image format parameters and values for UHDTV. It specifies two resolutions, 3840x2160 and 7680x4320, frame rates up to 120 fps, and a wider colour gamut than BT.601 and BT.709. In CIE 1931 (x,y) coordinates, the colour primaries are red (0.708, 0.292), green (0.170, 0.797), and blue (0.131, 0.046). The reference white is D65 again. IN STRICTEST CONFIDENCE BT.2020 specifies the same transfer function as the earlier BT.601 and BT.709. However, it also states: “that if it is shown that an alternative electro-optical transfer function will provide significant benefits without also imposing significant disadvantages, then this Recommendation should be extended to enable use with an improved EOTF”. IN STRICTEST CONFIDEN

73

IN STRICTEST ITU-R Recommendation BT.601, “Studio encoding parameters of digital television for standardCONFIDENCE 4:3 and wide screen 16:9 aspect ratios”. http://www.itu.int/rec/R-REC-BT.601/en 74

ITU-R Recommendation BT.709, “Parameter values for the HDTV standards for production and international programme IN STRICTEST CONFIDENC exchange”. http://www.itu.int/rec/R-REC-BT.709/en 75

ITU-R Recommendation BT.1886, “Reference electro-optical transfer function for flat panel displays used in HDTV studio production”. http://www.itu.int/rec/R-REC-BT.1886/en 76

STRICTEST CONFIDENCE ITU-R Recommendation BT.2020, “Parameter values for ultra-high definitionIN television systems for production and international programme exchange”. http://www.itu.int/rec/R-REC-BT.2020/en IN STRICTEST CONFIDENCE


Page | 40

6.1.2

High Dynamic Range


ITU-R WP6C is understood to be considering this issue of alternative non-linear transfer functions to support high dynamic range. 77 Proposals have been received from Philips, Technicolor, Dolby, and the BBC . It has been reported that ITU-R WP6C is planning to 78 perform quality evaluations that will allow the proposals to be ranked for effectiveness . The differences between these proposals are believed to include the dynamic range that needs to be supported, whether the specification should be written in terms of absolute or relative brightness levels, and the importance of some degree of IN STRICTEST CONFIDENC compatibility with the transfer function of BT.709. 18

The BBC, in its white paper 283 , has argued that the use of absolute luminance levels would require significant changes to the way 85 television is produced and viewed, and that the very large dynamic range supported by the SMPTE standard ST 2084 , imagined to be consistent with the Dolby/USA proposal into ITU-R WP6C, is not needed for image display. IN STRICTEST CONFIDENCE The BBC white paper, which is imagined to be consistent with BBC submissions to ITU-R WP6C, describes a transfer function that is ‘broadly compatible’ with BT.709, and which is claimed to facilitate compression and video processing using systems designed for BT 709, and which allows the display of pictures on existing displays with a qualityIN at STRICTEST least sufficientCONFIDENCE for monitoring purposes. ITU-R WP6C held a meeting 16-20 February 2015. No public statements appear to have been made about the progress or finalisation of this work. The next meeting is scheduled for 13-17 July 2015. IN STRICTEST CONFIDENCE 6.1.3

Colorimetry conversion

ITU-R WP6C is understood to be considering the development of a Recommendation on colorimetry conversion from BT.709 to 79 BT.2020 . Unlike a conversion in the reverse direction, this should be relatively straightforward to specify, as all colours that could be represented with BT.709 colorimetry could be specified with BT.2020 colorimetry. Colour conversion from BT.2020 to BT.709 is more problematic, as there may be colours in the originating BT.2020 representation than cannot be represented in BT.709, while there is a desire to avoid changingSTRICTEST colours that can be represented in BT.709. CONFIDENCE BT.2020 specifies a colour gamut for UHDTV that is extended with respect to that specified in BT.709 for HDTV. There is therefore a need to tailor the colour gamut of UHDTV content to the HDTV colour gamut, in situations where content produced in UHDTV is needed for HDTV production or broadcasting. Colour gamut mapping is not trivial. Unless due care is taken in colour space conversion, there is a concern that material that takes advantage of the extended UHDTV gamut could look inferior on an HDTV display compared to material originated directly in HDTV using BT.709. IN STRICTEST CONFIDENC 80 ITU-R WP6C is understood to have created a Rapporteur Group in March 2014 to consider the subject of colour gamut tailoring . No public statements appear to have been made about the progress of this work. 6.1.4

Higher Frame Rates


Following the identification by the United Kingdom of issues of flicker when cameras capturing at 120 frames per second are used 19 IN STRICTEST CONFIDEN under 50 Hz lighting , ITU-R Recommendation BT.2020 was revised in March 2014 to add support for 100 frames per second and 120/1.001 frames per second, but only as a footnote to the table of frames rates, presumably due to opposition to inclusion at all. The footnote, rather than mandating support for these frame rates, simple states: “The additional frame rate of 100 Hz is used in a number of 50 Hz countries” and “The additional frame rate of 120/1.001 Hz is used in a number of 60 Hz countries, while it is still under study in a number of other countries”. 77

IN STRICTEST CONFIDENCE Contributions to ITU-R WP6C, "Programme Production and Quality Assessment". http://www.itu.int/md/R12-WP6C.AR-C/en

78

http://www.hollywoodreporter.com/behind-screen/high-dynamic-range-might-be-698330 IN STRICTEST CONFIDENC 79 Report on the meeting of Working Party 6C (Geneva, 10-14 November 2014), Annex 06 - Preliminary draft new Recommendation ITU-R BT.[709to2020] - Colorimetry conversion from Recommendation ITU-R BT.709 to Recommendation ITU-R BT.2020. http://www.itu.int/md/R12-WP6C-C-0380/en 80

IN STRICTEST CONFIDENCE Report on the meeting of Working Party 6C (Geneva, 24-28 March 2014), Annex 05 - Appointment of a Rapporteur Group - Colour gamut tailoring. http://www.itu.int/md/R12-WP6C-C-0315/en IN STRICTEST CONFIDENCE


Page | 41

The meeting of ITU-R WP6C in February 2015 received at least one contributionINproposing to move the 100 frames per second value STRICTEST CONFIDENCE 81 into the main table .

6.2

SMPTE

The Society of Motion Picture and Television Engineers (SMPTE) is an international professional association, based in the United States of America, of engineers working in the motion imaging industries. SMPTE has produced over 600 Standards, Recommended Practices and Engineering Guidelines for television production, filmmaking, digital cinema, audio recording, information technology, 82 IN STRICTEST CONFIDENC and medical imaging . 6.2.1

UHD Parameter Values

IN STRICTEST CONFIDENCE The SMPTE standard ST 2036-1, “Ultra High Definition Television — Image Parameter Values for Program Production”, defines a family of progressive image sample structures for Ultra High Definition Television, with resolution 3840 x 2160 (termed UHDTV1) and 7680 x 4320 (termed UHDTV2), with an aspect ratio of 16x9, and with RʹGʹBʹ YʹCʹ BCʹR colour encoding and digital INor STRICTEST CONFIDENCE 83 representation . The most recent version was approved in October 2014. Colorimetry is defined consistently with BT.2020. In addition, for UHDTV1 at frame rates up to 60 frames per second, BT.709 IN STRICTEST colorimetry can be used. The transfer function (‘gamma’) is consistent with BT.2020 and BT.709.CONFIDENCE The 2013 revision of the standard listed, for both resolutions, various frame rates up to 60 frames per second, but also 120 frames per second. This was intended as a single worldwide higher frame rate for UHDTV. However, following publication, requests were received to also include 100 and 120/1.001 frames per second. Consequently the 2014 revision also lists the frame rates 100 and 120/1.001 frames per second for both resolutions, although it does state “It is not necessary for an implementation to support all formats to be compliant with this standard”. It would appear that there is still no consensus among SMPTE members on which of the higher frame rates should be supported. 84 The “UHDTV Ecosystem Study Group Report” produced by SMPTE in March 2014 , and hence before the latest revision of ST 2036STRICTEST CONFIDENCE 1, states “SMPTE TC members were divided on whether to include 120 fps, 120/1.001, or both of these frame rates for UHDTV”, and reports the arguments made for and against 120/1.001 and 120 frames per second, where the major factor in favour of the fractional frame rate is interoperability with the massive library of existing content, and where the major factor in favour of the integer frame rate is simplicity, particularly in respect to time related labels. No argument is reported against inclusion of 100 frames per second, which as described earlier in this report, was found to achieve better performance with 50Hz lighting than 120 frames per second. 6.2.2


High Dynamic Range

SMPTE published its first HDR standard, SMPTE ST 2084, “High Dynamic Range Electro-Optical Transfer Function of Mastering 85 Reference Displays”, in September 2014 . It specifies an electro-optical transfer function (EOTF) characterising high dynamic range IN STRICTEST CONFIDENCE reference displays used primarily for mastering non-broadcast content. It also specifies an Inverse-EOTF derived from the EOTF. 2

The EOTF has a high luminance range capability from 0 to 10000cd/m , and as it is referenced to absolute luminance, the display is assumed to be operating in a specified reference viewing environment. It is intended to enable the creation of video images with an IN STRICTEST CONFIDEN increased luminance range, and not the creation of video images with overall higher luminance levels.

IN STRICTEST CONFIDENCE Document 389, “Soliciting inclusion of a 100 Hz frame rate among those listed in Table 2 of Recommendation ITU-R BT.2020-1”. http://www.itu.int/md/R12-WP6C-C-0389/en 81

82

http://en.wikipedia.org/wiki/Society_of_Motion_Picture_and_Television_Engineers IN STRICTEST CONFIDENC

83

http://standards.smpte.org/content/978-1-61482-765-8/st-2036-1-2013/SEC1

84

http://origin.library.constantcontact.com/download/get/file/1109962569416729/SMPTE+UHDTV+Ecosytem+Study+Group+Report+28+March+2014.pdf IN STRICTEST CONFIDENCE 85 http://standards.smpte.org/content/978-1-61482-829-7/st-2084-2014/SEC1 IN STRICTEST CONFIDENCE


Page | 42

16

The EOTF is derived from the contrast sensitivity function of the human visual system that was developed by Peter Barten . It is IN STRICTEST CONFIDENCE claimed to allow the mapping of digital code values containing as few as 10 bits to the large, absolute luminance range from 0 to 2 10000cd/m . It is shown in the equation below.

(

[(

⁄

) ⁄

]

⁄

N denotes non-linear colour value L denotes linear colour value

) IN STRICTEST CONFIDENC

6.2.3

Colour Volume Metadata IN STRICTEST CONFIDENCE

SMPTE published SMPTE ST 2086, “Mastering Display Color Volume Metadata Supporting High Luminance and Wide Color Gamut 86 Images” in October 2014 . This standard specifies the metadata items to specify the colour volume, that is, the colour primaries, IN STRICTEST CONFIDENCE white point, and luminance range, of the display that was used in mastering video content. The metadata is specified as a set of values independent of any specific digital representation. It is applicable to three-color additive display systems, such as RGB displays. IN STRICTEST CONFIDENCE 6.2.4

Digital Cinema

SMPTE has produced a number of standards for Digital Cinema: the ST 428, ST 429, ST 430, ST 431, ST 432, and ST 433 suites, along with the ST 2048 suite. These document the image formats and workflows that are optimised for theatrical presentation and not 84 television reproduction . The SMPTE standard ST 428-1, “D-Cinema Distribution Master - Image Characteristics”, approved in 2006, describes the image 87 characteristics of the Digital Cinema Distribution Master (DCDM) . It describesSTRICTEST three ‘operational levels’, being 2048x1080 at 24 CONFIDENCE and 48 frames per second, and 4096x2160 at 24 frames per second. CIE XYZ colorimetry is specified. The transfer function is a power 2 law function (similar to ‘gamma’ of BT.709), with exponent 1/2.6, and peak luminance of 52.37cd/m . The SMPTE Recommended Practice RP 431-2 for “Digital Cinema Quality – Reference Projector and Environment”, first released by 88 the SMPTE in 2007, and most recently updated in 2011 as RP 431-2:2011 . It defines the Reference Projector and its controlled environment, along with the acceptable tolerances around critical image parameters for Review Room and Theatre applications. The transfer function is a power law with exponent 2.6. The colour gamut is specified in terms of XYZ. A projector is required to support a gamut of at least DCI-P3, specified by the primaries red (0.680, 0.320), (0.265, 0.690) and blue (0.15, 0.060), but a INgreen STRICTEST CONFIDENC 2 projector may have a larger gamut. A maximum luminance of 48cd/m shall be supported. The SMPTE standard ST 2048-1, "2048 × 1080 and 4096 × 2160 Digital Cinematography Production Image Formats FS/709", approved in 2011, defines a family of progressive sample structures of 2048 × 1080 and 4096 × 2160 images for Digital Cinema 89 content creation . IN STRICTEST CONFIDENCE The standard also defines what it refers to as Free Scale-Gamut (FS-Gamut), allowing images having arbitrary chromaticity to be conveyed. However, images can also be conveyed using BT.709 colorimetry, as suggested by the ‘709’ in the title of the standard. In IN STRICTEST CONFIDEN CIE 1931 (x,y) coordinates, the colour primaries are red (0.73470, 0.26530), green (0.14000, 0.86000), and blue (0.10000, -0.02985), and the reference white is D65. It has been suggested that the primaries relate to the Sony “wide gamut” delivered by the F23, F35, 90 and F65 cameras . The standard also defines a transfer function which it refers to as Free Scale-Log (FS-Log) curve. This has a logarithm characteristic over the middle part of the range, and optionally over the whole range, and is specified by four parameters and an ‘exposure’ IN STRICTEST CONFIDENCE 86

http://standards.smpte.org/content/978-1-61482-833-4/st-2086-2014/SEC1

87

http://standards.smpte.org/content/st-428-1-2006/SEC1.abstract.html

88

http://standards.smpte.org/content/978-1-61482-243-1/rp-431-2-2011/SEC1.abstract

89

http://standards.smpte.org/content/978-1-61482-641-5/st-2048-1-2011/SEC1.abstract


90

IN STRICTEST CONFIDENCE "Free Scale Gamut, Free Scale Log (FS-Gamut, FS-Log)", Charles Poynton. http://www.poynton.com/notes/misc/Free-ScaleGamut-Free-Scale-Log.html IN STRICTEST CONFIDENCE


Page | 43

parameter. The top and bottom of the curve can optionally be specified as a non-linear function,CONFIDENCE defined as a smooth curve through IN STRICTEST three indicated points. The standards states that the FS-Log curve has an affinity with the sensitivity of human eye and can specify a much wider dynamic range than the curve defined in BT.709. This standard also defines the ‘Color VANC’, which is ancillary data that conveys the parameter values of the user-defined colour space and Log curve. SMPTE ST 2048-2 states that the Color VANC data, when transported through a single-link or a multiple-link HD-SDI shall be transmitted in link A or the first link of a multi-link interface transport once per frame. The recommended mapping location of the Color VANC packet is in the vertical ancillary data space of line 18 of the Y data stream. IN STRICTEST CONFIDENC 6.2.5

Colour Equations

The SMPTE Recommended Practice RP 177:1993, “Derivation of Basic Television Color Equations”, defines the numerical procedures INadditive STRICTEST for deriving basic colour equations for colour television and other systems using displayCONFIDENCE devices. In Annex B, an example of 91 these calculations is given for the D65 white point and the colour primaries, as also specified in BT.709 . 6.2.6


On-going Activities

STRICTEST CONFIDENCE SMPTE has a Study Group that is compiling a report on the High Dynamic RangeINEcosystem. It also has a new project defining 92 Dynamic Metadata for Color Transforms of High Dynamic Range and Wide Colour Gamut Images . The Study Group on “High Dynamic Range Ecosystem” aims to identify the specific parameters that constitute “High Dynamic Range”. The Study Group will identify and report on requirements for new SMPTE Standards, Recommended Practices, and Engineering Guidelines; identify necessary revisions of current SMPTE Engineering Documents, and provide recommendations for 93 the prioritisation of work within SMPTE . High dynamic range technology proposals have been reported to have been received from the BBC, Philips, Technicolor and NHK. The Drafting Group on “Dynamic Metadata for Color Transforms of HDR and WCG Images” plans to develop standards for specifying the semantics and representation of content-dependent metadata needed for STRICTEST colour volumeCONFIDENCE transformation of high dynamic range 94 and wide colour gamut imagery to smaller colour volumes (e.g. BT.709 or Digital Cinema) in mastering applications . It is stated that content dependent metadata is required to ensure creative intent is maintained: if colour mapping were performed without this metadata, the resulting out of gamut content could suffer severely in visual quality due to clipping. It is reported that although the science of colour volume mapping has advanced significantly recently, there is no single method to map adequately between different colour volumes. Instead, a colour transformation with content-dependent parameters could be used. For example, a very dark scene should be mapped differently from a very bright scene when transforming from a large colour INbe STRICTEST CONFIDENC volume to a more restricted colour volume, but it is claimed that this could only achieved effectively when this transformation is guided by metadata. The Study Group on “Integer and Fractional Frame Rate Conversion” is considering the issue of whether it would be practical to achieve real time high quality conversion between integer and fractional frame rates so that the use of fractional frame rates above IN STRICTEST CONFIDENCE 60 frames per second could be discontinued. 92

It was reported in December 2014 that a “Request For Information” had been agreed and that it would be sent to organisations that could comment on the performance that could be achieved with availableIN conversion technology. It was hoped that technology STRICTEST CONFIDEN demonstrations could be provided in the March 2015 timeframe.


http://standards.smpte.org/content/978-1-61482-191-5/rp-177-1993/SEC1.abstract IN STRICTEST CONFIDENC 92 “Standards Quarterly Report December 2014”, https://www.smpte.org/sites/default/files/December%202014%20Standards%20Outcome%20Report.pdf 93 94

“10E Study Group on HDR Ecosystem”. https://kws.smpte.org/kws/public/projects/project/details?project_id=241

“10E Dynamic Metadata for Color Transforms of HDR and WCG Images”. https://kws.smpte.org/kws/public/projects/project/details?project_id=294




Page | 44

6.3

MPEG


The Moving Picture Experts Group (MPEG) is a working group of ISO/IEC, specifically ISO/IEC JTC 1/SC 29/WG 11, with the mission to develop standards for coded representation of digital audio and video and related data. MPEG, working jointly with the ITU-T, has developed the well-known video compression standards MPEG-2 (officially known as ISO/IEC 13818-2 and ITU-T Recommendation H.262), H.264/AVC (Advanced Video Coding, officially known as ISO/IEC 14496-10 and ITU-T Recommendation H.264), and most recently HEVC (High Efficiency Video Coding, officially known as ISO/IEC 23008-2 and ITU-T Recommendation H.265). IN STRICTEST CONFIDENC 6.3.1


IN STRICTEST CONFIDENCE HEVC, High Efficiency Video Coding, is the latest compression standard to be developed jointly by MPEG and ITU-T, in the Joint Collaborative Team on Video Coding (JCT-VC). It enables the bit rate for video services to be approximately halved compared to the previous standard, H.264/AVC, while achieving the same video quality. IN STRICTEST CONFIDENCE The first version of this standard was approved in January 2013, where three profiles were specified: “Main Profile”, “Main 10 Profile”, and “Main Still Picture Profile”. It is expected that “Main Profile” will be used for mass market video services that historically require only 8 bits of precision in their processing, whereas “Main 10 which CONFIDENCE supports up to 10 bits of processing INProfile”, STRICTEST precision, is expected to be used where additional accuracy is needed, notably for encoding video that has a high dynamic range. The second version of this standard was approved in July 2014, where three amendments were added to the original specification. The first, known as the Range Extensions amendment, added support for colour sampling formats beyond 4:2:0 and up to 16 bits of processing precision. The second, known as Multiview HEVC (MV-HEVC), added support for providing efficient representation of video content with multiple camera views and optional depth map information, such as for 3D stereoscopic and auto stereoscopic video applications. The third, known as Scalable HEVC (SHVC), added support for embedded bitstream scalability in which different levels of encoding quality are efficiently supported by adding or removing layered subsets of encoded data. The second version of the HEVC compression standard includes features to support high dynamic range and wide colour gamut. STRICTEST CONFIDENCE In the Video Usability Information (VUI), a code point was added to indicate the colour primaries CIE 1931 XYZ, as specified in SMPTE ST 428-1, adding to a number of existing colour primary code points, including those indicating BT.709 and BT.2020. And code points were added to indicate the transfer characteristics specified in SMPTE ST 2084 and SMPTE ST 428-1, adding to a number of existing code points including those indicating BT.709 and BT.2020. HEVC version one specifies a tone mapping Supplemental Enhancement Information (SEI) message, which allows multiple mappings from the encoded dynamic range to a different, typically smaller, dynamic range, for display on devices that support a different IN STRICTEST CONFIDENC dynamic range to that of the encoded representation. The mappings are defined as one dimensional look up tables, which apply in either the luma or RGB colour space domain, and which should be applied to the luma component or to each RGB component. A related SEI message, “knee function information”, was added in the second version of the HEVC compression standard. This allows one or more mappings from the dynamic range of the encoded stream to the dynamic range of the display to be specified, allowing IN STRICTEST CONFIDENCE the dynamic range of the encoded stream to be compressed or decompressed, that is, supporting both use cases of displaying high dynamic range content appropriately on a standard dynamic range display and displaying standard dynamic range content appropriately on a high dynamic range display. The mapping is specified as a piecewise linear function, where the ‘pieces’ join at soIN STRICTEST CONFIDEN 2 called knee-points, mapping normalised encoded values to normalised display values, and also signalling, in units of cd/m , the peak luminance level of the encoded representation and the peak luminance level of the representation for display. A “mastering display colour volume” SEI message was also added in the second version of the HEVC compression standard. This allows the colour volume (the colour primaries, white point, and luminance range) of the mastering display for the video content to be indicated. This information is intended to be adequate for purposes corresponding to the use of SMPTE ST 2086. However, it IN STRICTEST CONFIDENCE does not provide information on the colour transformations that would be appropriate to preserve creative intent on displays with colour volumes different from that of the described mastering display. This functionality is however supported by the “colour remapping information” SEI message which was also added in the second IN STRICTEST CONFIDENC version of the HEVC compression standard. This allows one or more mappings from the colour space of the encoded stream to the colour space of the display to be specified, so as to preserve the artistic intent, that is, it specifies mappings from the colour graded encoded content to content suitable for the display, and approximating the fidelity of the content had it been specifically colour graded for the display. For example, it could specify a mapping from BT.2020 to BT.709 to enable high fidelity reproduction of IN STRICTEST CONFIDENCE content, mastered for BT.2020, on the BT.709 capable display. The colour remapping model is composed of a first piecewise linear



Page | 45

function applied to each colour component, followed by a three by-three matrix to the three colour components, followed INapplied STRICTEST CONFIDENCE by a second piecewise linear function applied to each colour component. 6.3.2

The Future of Video Coding Standardisation

A brainstorming session of the Future of Video Coding Standardisation was held during the meeting of MPEG in October 2014 to explore use cases, requirements and potential timelines for the development of future video coding standards. IN STRICTEST CONFIDENC Guest speakers included Roger Bolton of Ericsson, Harald Alvestrand of Google, Zhong Luo of Huawei, Anne Aaron of Netflix, Stéphane Pateux of Orange, Paul Torres of Qualcomm, and JeongHoon Park of Samsung, who each provided presentations 95 discussing the needs of applications in their industry segments . A range of opinions was expressed by the panellists and other participants from and academia, who shared a common view INindustry STRICTEST CONFIDENCE that further increases in compression efficiency remain a fundamental need. MPEG set up two ad-hoc groups to continue the study of future application requirements and to begin to establish a roadmap for future video coding standardisation. 96 STRICTEST CONFIDENCE The presentations at the brainstorming session are available on the MPEG webIN site . 6.3.3

High Dynamic Range and Wide Colour Gamut


MPEG is currently in the process of determining whether extensions to the HEVC standard would provide significant benefits for the compression of video with high dynamic range and wide colour gamut, compared to the performance that could be achieved with the current version of the standard. To achieve this, immediately following its meeting in February 2015, MPEG issued a “Call for Evidence” of new tools that may improve the performance of HEVC when used to encode high dynamic range and wide colour gamut video. The results of the Call for Evidence will be analysed to determine if MPEG should launch a new standardisation effort to extend the current capabilities of 97 HEVC . STRICTEST CONFIDENCE In order to allow the various technology proposals to be compared, MPEG has received test content with high dynamic range and/or wide colour gamut from a number of companies. This content was submitted in one of the following formats: OpenEXR file format, linear light, and BT.709 colour gamut; and TIFF files, 12 bit Perceptual Quantiser (SMPTE ST 2084) non-linear samples, and DCI-P3 colour gamut. The test content is RGB with 4:4:4 sampling, and resolution 1920x1080 progressive. The test content is defined with respect to the BT.2020 container, and in addition three of the test sequences submitted with BT.709 colour gamut are additionally defined with respect to the BT.709 container. While it is expected that high dynamic range IN STRICTEST CONFIDENC content would be delivered using the BT.2020 colour gamut in commercial services, the additional use of the BT.709 container for some test sequences would allow the impact of colours at the edge of the colour gamut to be assessed. While it could have been possible to generate artificial content with colours near the edge of the BT.2020 colour gamut, there are no available screens that could display the processed content, and hence to investigate edge of colour gamut issues, the BT.709 container had to be used. STRICTEST CONFIDENCE The MPEG developed software tools known as HDRConvert have been used to IN convert the test content into the format that was used to generate what MPEG calls the anchors, that is, the base case against which proposals will be compared and against which an improved performance will be necessary for standardisation work to be performed. This format is YCbCr, 4:2:0, 10 bit Perceptual IN STRICTEST CONFIDEN Quantiser, and two colour containers (BT.2020 and BT.709), encoded with various fixed quantisation settings with HEVC using the Main 10 Profile and the Scalable Main 10 Profile. The timeline for the work is as follows. The Call for Evidence and the anchors were made available during February 2015. Expressions of interest in responding to the call must be registered by the end of March. Encoded content must be submitted by 26 May, with documents registered and submitted by 8 June. Evaluation of the submissions, including subjective assessment, will be carried out from 8-26 June. IN STRICTEST CONFIDENCE


MPEG Press Release, 24 October 2014. http://mpeg.chiariglione.org/sites/default/files/files/meetings/docs/w14813.docx

96

“Presentations of the Brainstorming Session of the Future of Video Coding Standardization “. http://mpeg.chiariglione.org/standards/exploration/future-video-coding/presentations-brainstorming-session-future-video-coding IN STRICTEST CONFIDENCE 97 MPEG Press Release, 20 February 2015. http://mpeg.chiariglione.org/sites/default/files/files/meetings/docs/w15066_0.docx IN STRICTEST CONFIDENCE


Page | 46

The Call for Evidence defines the following test conditions related to the different forms of scalability between standard and high IN STRICTEST CONFIDENCE 98 dynamic range video coding . 







1. Compression efficiency improvement over HEVC Main 10 Profile. This case is not concerned with compatibility with standard dynamic range equipment, and is simply concerned with how efficiently high dynamic range video could be encoded, and specifically whether a significant gain could be achieved over what could be done with the currently defined HEVC Main 10 Profile. 2a and 2b. Backward compatibility with legacy standard dynamic range decoders. In this case, the encoder produces two streams: a compressed high dynamic range stream and a standard dynamic range stream that could be decoded by a Main IN STRICTEST CONFIDENC 10 decoder. The former stream could consist simply of metadata to enhance the latter, or could also or alternatively include enhancement data. The aim is to quantify the benefits relative to simulcast of standard and high dynamic range bitstreams, and relative to the use of SHVC with Colour Gamut Scalability using the Scalable Main 10 profile. There are two variants of this test case. In case 2a, the encoder is provided with the source in both high and standard dynamic range IN STRICTEST CONFIDENCE formats, and is provided with an encoded standard dynamic range bitstream; and in case 2b, the encoder is provided only with a video source with high dynamic range and the encoder is responsible for generating and encoding a standard dynamic range stream. IN STRICTEST CONFIDENCE 2c and 2d. Backward compatibility with legacy standard dynamic range displays. This case is concerned with new, high dynamic range capable decoders, being connected to legacy displays. In this case, the encoder produces two streams: a compressed high dynamic range stream and a metadata stream to help a new receiver to create standard dynamic range STRICTEST CONFIDENCE video content from the high dynamic range stream. The primary aim isINnot to test or standardise the tone mapping algorithms, but to determine the type of metadata that is needed and the bit rate that would be needed for it. There are two variants of this test case. In case 2c, the encoder is provided with both high and standard dynamic range video content and in case 2d it is provided only with high dynamic range video content. In both cases the encoder must generate the metadata to enable the high dynamic range to standard dynamic range conversion to be performed in a receiver by a tone mapping process using that metadata. 3. Optimisation of the existing Main 10 Profile and/or Scalable Main 10 Profile. This case is concerned with nonnormative changes to the HEVC standard (Main 10 Profile and/or Scalable Main 10 Profile) that would provide better coding efficiency than the anchors used in cases 1 or 2a. Such changes include, but are not limited to, improved pre- or STRICTEST post-processing, better colour sub-sampling, the use of different colour domains forCONFIDENCE encoding, and the use of constant luminance.

Submissions received for cases 1 and 2a will be subject to both objective performance assessment and subjective testing. Submissions received for the other cases will be classified as “Technology Under Consideration” where testing and next steps will depend on the submissions received: submissions will be viewed by experts to investigate the relationship between visual quality and bit rate. Objective performance assessments are to be made using the three objective metrics defined inCONFIDENC the Call for Evidence. Subjective IN STRICTEST testing will be done using a SIM2 monitor in two locations and a Pulsar monitor in another location. Cropped versions of the reference and the compressed video will be presented side by side on a single monitor, with viewers asked to assess the “visual impairment”. The action MPEG will eventually take on this work will depend on the performance of the technology submitted. A first possibility is IN STRICTEST CONFIDENCE that no standardisation activity is required at all, if efficient high dynamic range and wide colour gamut coding could be provided without any normative changes. A second possibility is that minor changes to the HEVC standard would be required to optimise the performance of the existing Main 10 Profile for high dynamic range and wide colour gamut content, for example, by enabling the IN STRICTEST CONFIDEN carriage of additional signalling. A third possibility would involve normative changes to the HEVC standard that would result in a new profile; such normative changes would probably need to be justified by a significant decrease in the bit-rate required to give a comparable visual quality to that of the Main 10 profile. It is also possible that more than one solution will be developed.

6.4

DVB

IN STRICTEST CONFIDENCE The Digital Video Broadcasting Project is an industry-led consortium of over 200 broadcasters, manufacturers, network operators, software developers and regulators from around the world committed to designing open technical standards for the delivery of digital television. IN STRICTEST CONFIDENC

98

IN STRICTEST CONFIDENCE Call for Evidence (CfE) for HDR and WCG Video Coding. http://mpeg.chiariglione.org/standards/exploration/high-dynamic-rangeand-wide-colour-gamut-content-distribution/call-evidence IN STRICTEST CONFIDENCE


Page | 47

76

ITU-R recommendation BT.2020 specifies two sets of picture spatial characteristics, with resolutions 3840x2160 and 7680x4320. IN STRICTEST CONFIDENCE The DVB is currently working on standards for UHDTV, relating to this specification. The DVB concluded that initial client implementations would not be able to support the full set of parameters of BT.2020 at the lower of the two resolutions, and hence split their standardisation work into three phases. The first of these three phases, known as UHD-1 Phase 1, supports a subset of the full set of parameters of BT.2020 at the lower of the two resolutions, the second, known as UHD-1 Phase 2, is planned to support the full set of parameters of BT.2020 at the lower of the two resolutions, and the third, known as UHD-2, is expected to support the full set of parameters of BT.2020 at the higher of the two resolutions. This is illustrated in Figure 28, where it should be noted that UHD-1 Phase 1 is fully defined, the IN while STRICTEST CONFIDENC descriptions of the subsequent phases are tentative. UHD-2 2020+ 7680x4320

UHD-1 Phase 2 2016/7 3840x2160, Up to 120 fps 4:2:0 10 bit BT.2020 - Wide Colour Gamut Higher Dynamic Range More Advanced Audio

UHD-1 Phase 1 2014 3840x2160 Up to 60 fps 4:2:0 10 bit BT.709 and BT.2020 Existing Audio Toolbox




Backward Compatibility?


Figure 28. The parameters proposed for each of the three phases of UHD in DVB.


6.4.1

UHD-1 Phase 1

The ‘commercial requirements’ for UHD-1 Phase 1 were prepared in autumn 2013. The specification was completed in July 2014, IN STRICTEST CONFIDENCE when the DVB Steering Board approved a new version of TS 101 154, the DVB specification for the use of Video and Audio Coding in Broadcasting Applications based on the MPEG-2 Transport Stream. The new specification has been passed to ETSI for publication as 99 TS 101 154 v2.1.1 which can be downloaded from the DVB website . IN STRICTEST CONFIDEN The main change in this, UHD-1 Phase 1, revision of TS 101 154 is the addition of High Efficiency Video Coding (HEVC), for which DVB has defined five decoder conformance points, four at HD resolution (the four combinations of the 25 and 30 frames per second families and Main and Main 10 profile), and one at UHD resolution (all frame rates with Main 10 profile). The conformance point at UHD resolution is specified as follows. IN /STRICTEST CONFIDENCE HEVC video encoding shall be used, as specified in ITU-T Recommendation H.265 ISO/IEC 23008-2, using the Main 10 profile at levels up to and including 5.1, unless the HEVC bitstream is an HEVC temporal video sub-bitstream. The aim in the latter case is to enable temporal scalability with a subsequent version of the specification. IN STRICTEST CONFIDENC In addition to the spatial resolutions that HDTV HEVC receivers shall support, UHDTV HEVC receivers shall be able to support the spatial resolutions, 3840x2160, 2880x2160, 3200x1800, and 2560x1440, with display aspect ratio of 16:9, displaying received

99

IN STRICTEST CONFIDENCE http://www.etsi.org/deliver/etsi_ts/101100_101199/101154/02.01.01_60/ts_101154v020101p.pdf IN STRICTEST CONFIDENCE


Page | 48

pictures full screen size. These resolutions are progressively scanned at any of the following frame rates: 24/1.001, 24, 25, 30/1.001, IN STRICTEST CONFIDENCE 30, 50, 60/1.001 and 60 frames per second. Receivers shall be capable of decoding bitstreams that use BT.709 or BT.2020 non-constant luminance colorimetry. The specification acknowledges the problem of correctly handling BT.2020 colorimetry in the short term, and recommends that any conversion to BT.709 is not done badly, specifically identifying two methods not to use: receivers should not impose a hard limit such that all colours outside of the gamut of the display are placed on the outer boundary of the display gamut; and receivers should not linearly scale the wider gamut of the bitstream to fit within the gamut of the display. IN STRICTEST CONFIDENC 100 The first commercial service compliant with UHD-1 Phase 1 is believed to be that offered by DirecTV Error! Bookmark not efined.. This requires an Internet-connected Genie® HD DVR and a ‘DIRECTV 4K Ready TV’, being a UHD TV that supports RVU 101 technology and has UHD decoding capability. The service allows a selection of UHD movies to be viewed instantly . 6.4.2

UHD-1 Phase 2


The DVB is already working on UHD-1 Phase 2, with the Commercial Module discussing the relevant parameters. While UHD-1 Phase IN STRICTEST CONFIDENCE 1 focused on resolution, UHD-1 Phase 2 will bring higher frame rates, high dynamic range, a wider colour space as well as immersive 102 audio . IN2014 STRICTEST CONFIDENCE A large number of interested parties are participating in this process and in June over 90 people came together, in a combined EBU/DVB workshop, to discuss the relevant parameters for High Dynamic Range. Although UHD-1 Phase 2 is still very much a work in progress, it is expected to support frame rates up to 120 frames per second, at least 10 bits per sample, a bitstream format specified relative to the BT.2020 colour primaries, although display technology may only be capable of displaying a subset of the colours that could be signalled, and more advanced audio. It is planned to finish the 103 specification in time to allow services to be launched in the 2017 / 2018 time frame . The main topics of discussion for UHD-1 Phase 2 include the degree of compatibility with DVB UHD-1 Phase 1; the details of how high dynamic range would be signalled and supported, how sub-titles are supported, advanced audio, content protection, and the 100 impact of other organisations and specifications, for example the Blu-Ray Disc STRICTEST Association .CONFIDENCE DVB has been in communication with Digital Europe on the likely capabilities of televisions in the 2016/17/18 timeframe, and in particular the capabilities to support higher dynamic range. This may be a ‘chicken and egg’ situation with DVB wanting to know the likely capabilities of televisions in the future so that they could draft suitable standards, and the television industry, represented by Digital Europe, wanting to know the likely content of future standards in order to plan the development of next generation televisions. There appears to be some desire within DVB for UHD-1 Phase 1 receivers to beIN able to access UHD-1 Phase 2 services. While this STRICTEST CONFIDENC should be technologically feasible, with HEVC compression supporting scalable modes of operation, the cost of this in terms of increased bit rate over non-scalable solutions is not yet known. Hence whether scalability is worth supporting at all and whether both compatible and non-compatible solutions for UHD-1 Phase 2 need to be specified are questions that are still open. IN STRICTEST CONFIDENCE 6.4.3

UHD-2

IN STRICTEST UHD-2 is a placeholder for a possible future specification with a resolution of 7680x4320, with a CONFIDEN tentative timeframe of 2020 or later. DVB has not discussed UHD-2 yet, at least in part because DVB knows of no plans to make UHD-2 consumer displays available.


UHDTV: Television a point”, David Wood, Consultant, EBU Technology and Innovation. http://www.hdforum.fr/sites/default/files/calendrierdelultrahd_ultrahdcalendar.pdf IN STRICTEST CONFIDENC 101

http://www.directv.com/technology/4k

102

“DVB Scene Issue 44”. https://www.dvb.org/news/dvbscene/issue/44

103

IN STRICTEST CONFIDENCE “DVB Developments and Future Standards Trends”, Peter Siebert, DVB Project Office. http://www.digitag.org/wpcontent/uploads/2014/12/4.Peter-Siebert_DigiTAG_GA_2014.pdf IN STRICTEST CONFIDENCE


Page | 49

However, Japanese broadcaster NHK has confirmed that it wants to push for 8kINbroadcasting as CONFIDENCE soon as possible and will begin STRICTEST testing a service by 2016. NHK wants to offer an 8k broadcasting service by 2020, the year of the Tokyo Olympics and 104 Paralympics . 6.4.4

Eco-design requirements for electronic displays 105

There is concern within the DVB and within Digital Europe on whether a new draft European Commission regulation 106 IN STRICTEST CONFIDENC implementing the eco-design directive with regard to eco-design requirements for electronic displays would allow manufacturers to improve luminescence and move towards higher dynamic range. Figure 29 shows an interpretation of the draft regulation, plotting the power constraint against the screen size, measured as the screen diagonal in inches, for 12, 36 and 60 months after the publication of the Regulation in the Official Journal of the European Union. IN STRICTEST CONFIDENCE The draft regulation states that power measurements shall be made using “a dynamic broadcast-content video signal representing typical broadcast content for electronic displays”, with the measurement being the average power consumed over ten consecutive minutes after the display has been switched on sufficiently long, typically an hour, to achieve stable operation. IN STRICTEST CONFIDENCE DVB has asked Digital Europe for their opinion on the consequences of this draft regulation. Digital Europe is believed to be continuing to discuss internally the possible impact of these energy limitations on the feasibility of higher dynamic range displays. IN STRICTEST CONFIDENCE DVB is considering whether specifications should be designed to limit or optimise the display energy consumption. For example, should the statistical distribution of luminance be constrained?

250 12 Months

Power (W)

200

36 Months 60 Months


150 100 50


0

30

35

40

45

50 55 60 Screen Diagonal (Inches)

65

70

75

80

Figure 29. The proposed future power constraints on displays. CONFIDENCE IN STRICTEST

6.5

EBU

IN STRICTEST CONFIDEN The European Broadcasting Union is an alliance of public service media entities. The EBU's technical activities include provision of technical information to Members via conferences and workshops, as well as in written form (such as the EBU Technical Review, and the EBU tech-i magazine).


http://www.pocket-lint.com/news/127997-japan-plans-8k-tv-broadcast-testing-in-2016-with-full-service-by-2020-tokyo-olympics IN STRICTEST CONFIDENC 105 “Possible requirements for electronic displays”. http://www.energimyndigheten.se/Global/F%C3%B6retag/Ekodesign/Ekodesign/TV-apparater/Ecodesign_Draft_reg-V01.pdf 106

“Directive 2009/125/EC of the European Parliament and of the Council of 21 October 2009 establishing a framework for the IN STRICTEST CONFIDENCE setting of ecodesign requirements for energy-related products”. http://eur-lex.europa.eu/legalcontent/EN/TXT/?uri=CELEX:32009L0125 IN STRICTEST CONFIDENCE


Page | 50

The EBU Technical Committee, which comprises 13 members, and which coordinates and manages the EBU's technical work, IN STRICTEST CONFIDENCE 107 produced a policy statement on UHDTV in July 2014 . This stated the opinion that the EBU Technical Committee believes that the current focus of the consumer electronics industry to provide only an increased resolution (“4k”) and ignoring other enhancements, of for example, DVB UHD-1 Phase 2, is not a sufficiently large step for the introduction of successful new broadcasting services. The report notes that an enhanced, 1080p-based HD service that includes a certain combination of UHDTV parameters, higher frame rate, higher dynamic range, wider colorimetry and advanced sound system audio, but which does not include higher resolution, is not yet standardised. The report states that such a 1080p-based HD format could be an appealing option for some broadcasters and should be taken into account in the standardisation and investigation process.CONFIDENC The EBU proposes that an IN STRICTEST enhanced 1080p format be developed for broadcasting.

6.6

DTG

IN STRICTEST CONFIDENCE The Digital TV Group (DTG) publishes and maintains the technical specifications for Freeview, known as the D-Book. The DTG also runs DTG Testing, the ISO17025 accredited digital TV test centre which provides benchmark test and conformance services for the Freeview and Freesat services and international platforms. IN STRICTEST CONFIDENCE The DTG established the UK UHD Forum in August 2013 to bring together the entire UK UHD ecosystem of content owners, platform 108 operators, broadcasters, CE makers and silicon vendors, to try and agree what is meant by UHD beyond the number of pixels . The aim is to define a conformance specification for 4K and higher definitions, also IN considering high dynamic range, wider colour gamut STRICTEST CONFIDENCE and immersive 3D audio. It is co-chaired by Chris Johns, chief engineer for broadcast strategy at Sky and Andy Quested, head of 109 technology for HD and UHDTV at the BBC . The DTG has run two 4k plugfests to date, and is planning another. A first 4k plugfest was held in October 2014. Testing focussed on the 4k HDMI interoperability of 12 UHD displays from ten different manufacturers, all of which displays were available in retail at the time, two set-top boxes and a reference set top box design from a silicon vendor, with HDMI test tools supplied by Rohde & Schwarz and Quantum Data. Participants tested different combinations of outputs and 4k resolutions, frame rate and bit depth to build a picture of which HDMI features were supported. It also inaugurated 110 the DTG's 4k receiver and set top box collection, known as the '4k Zoo' . STRICTEST CONFIDENCE A second 4k plugfest was held in January 2015 to investigate HEVC UHD interoperability. Testing was carried out on a range of 4k receivers, integrated televisions and set top boxes, representing the available 4k receiver market of 2013 and 2014, as well as new 2015 models and prototypes from leading silicon vendors. Broadcasters, satellite operators, test labs and test equipment providers contributed UHD HEVC test signals encoded in main and main 10 profiles, covering a variety of resolutions, frame rates, bit depths and colour space. These were delivered to the 4k receivers via USB, modulated as satellite and DTT signals, and using MPEG111 DASH . IN STRICTEST CONFIDENC This testing showed that there was still limited support for the HEVC compression standard. While all of the 2015 models that were tested supported UHD at 50 and 60 frames per second with HEVC encoding, only 60% of the older models did so. Additionally, only 112 half the models tested were found to support MPEG-DASH adaptive bit-rate streaming . The DTG plan another plugfest in April 2015 with the aim of testing HDCP 2.2 copy protection, looking at whether all legacy sources IN STRICTEST CONFIDENCE will work with HDCP 2.2 displays and what experience viewers will get with HDCP 2.2 into lower version displays.

6.7

Blu-ray Disk Association


The Blu-Ray Disc Association (BDA) is developing the next generation optical disc package media format, launched as Ultra HD BluRay at CES 2015. This new Blu-Ray format will support UHD with support for High Dynamic Range. It has been reported that the

107

“Tech Report 028 - EBU Policy Statement on UHDTV”. https://tech.ebu.ch/publications/tr028 IN STRICTEST CONFIDENCE

108

“UK will not go it alone with UHD – DTG”. http://www.dtg.org.uk/dtg/press_releases/dtg_csi_201401p08.pdf

109

“DTG UK UHD Forum welcomes formation of US UHD Alliance”. http://www.dtg.org.uk/news/news.php?id=5313 IN STRICTEST CONFIDENC

110

“4K HDMI plugfest inaugurates DTG Testing's Ultra HD testing zoo”. http://www.dtg.org.uk/news/news.php?id=5271

111

“DTG's 4k Plugfests turn spotlight on HEVC streams”. http://www.dtg.org.uk/news/news.php?id=5327

112

IN STRICTEST CONFIDENCE “Ultra HD technology alignment now close to “tipping point” for services”. http://www.digitaltveurope.net/321122/ultra-hdtechnology-alignment-now-close-to-tipping-point-for-services/ IN STRICTEST CONFIDENCE


Page | 51

technical specification will be released to licensees in mid-2015, and that the first titles are expected to be on the market before the IN STRICTEST CONFIDENCE 113 end of the year . 114

It has been reported that the new disk format will support resolutions up to 3840 x 2160, frame rates up to 60 frames per second, 115 10-bit colour depth, BT.2020 colour space , and HEVC (H.265) video compression technology, on disks from 66GB (dual layer) up to 100GB (triple layer). 116

It has been reported that at least three HDR technologies will be supported, including what has been reported to be an open HDR specification, which is believed to be SMPTE ST 2084, and which is believed to have also been reported as ‘Dolby Vision’, as well as IN STRICTEST CONFIDENC optional support for proposals originating from Philips and Technicolor. 117

2

Display Daily reports that the BDA recommends that frame average light levels should not exceed 400 cd/m and only specular 2 118 86 highlights are allowed to go above 1000 cd/m . It is also reported that SMPTE ST 2086 , a standard that describes the metadata items to specify the colour volume (the colour primaries, white point, and luminance range) of the display that was used in IN STRICTEST CONFIDENCE mastering video content, would be used.

6.8

HDMI Forum


The HDMI Forum is a trade association comprised of over 80 companies, including manufacturers of consumer electronics, personal computers, mobile devices, cables and components. The HDMI Forum, and previously, the HDMI Founders, have developed versions IN STRICTEST CONFIDENCE of the High-Definition Multimedia Interface, a proprietary audio/video interface for transferring uncompressed video data and compressed or uncompressed digital audio data from an HDMI-compliant source device, such as a set top box, Blu-ray player, 119 camera or tablet, to a compatible digital television, monitor, video projector, or digital audio device . Version 2.0 of the HDMI® specification was released in September 2013. It offers a significant increase in bandwidth, up to 18GBit/s, 120 allowing support of 4K resolution at up to 60 frames per second with 24 bits per pixel . This limits the number of bits to 8 bits per sample if 4:4:4 sampling is used, but allows 10 or 12 bits per sample if 4:2:2 or 4:2:0 sampling is used. The signalling of device capabilities and data formats for HDMI is specified in the Consumer Electronics Association (CEA) standard, 121 CEA-861-F, “A DTV Profile for Uncompressed High Speed Digital Interfaces” . STRICTEST CONFIDENCE 122 Support for UHD resolution, 4:2:0 sampling, and BT.2020 colorimetry was added in the ‘F’ version of this standard in August 2013 : previously only 4:4:4 and 4:2:2 sampling were supported, and resolutions up to 1920x1080. This was extended further in January 2015 with the publication of CEA-861.3, “HDR Static Metadata Extensions”, which adds support for high dynamic range by extending the metadata definitions. This now allows the signalling of traditional gamma with a 2 maximum luminance of 100cd/m , as for standard dynamic range, traditional gamma with a maximum luminance of that of the sink (typically display) device, and SMPTE ST 2084. It also allows signalling the SMPTE ST 2086 Mastering Display Metadata, which, IN STRICTEST similarly to the HEVC “mastering display colour volume” SEI message, as described earlier, allowsCONFIDENC the colour volume (the colour primaries, white point, and luminance range) of the mastering display for the video content to be signalled. It also allows signalling of Maximum Content Light Level (MaxCLL) and Maximum Frame-Average Light Level, as defined in an annex to the standard.

113

IN STRICTEST CONFIDENCE http://www.hollywoodreporter.com/behind-screen/ces-ultra-hd-blu-ray-761728

114

http://www.rethinkresearch.biz/articles/uhd-capable-blu-ray-players-will-appear-end-year/ IN STRICTEST CONFIDEN 115 http://hexus.net/ce/news/audio-visual/79333-ultra-hd-blu-ray-specs-unveiled-4k-hdr-support/ 116

http://advanced-television.com/2015/02/16/uhd-blu-ray-standard-agreed/

117

http://www.displaydaily.com/using-joomla/extensions/components/searchcomponent/search?searchword=2084&ordering=newest&searchphrase=all&limit=20 IN STRICTEST CONFIDENCE 118 http://www.displaydaily.com/using-joomla/extensions/components/searchcomponent/search?searchword=2086&ordering=newest&searchphrase=all&limit=20 119

http://en.wikipedia.org/wiki/HDMI

120

HDMI Forum releases version 2.0 of the HDMI specification. http://www.hdmi.org/press/press_release.aspx?prid=133

121

http://www.ce.org/Standards/Standard-Listings/R4-8-DTV-Interface-Subcommittee/CEA-861-E.aspx


122

IN September STRICTEST CONFIDENCE “CEA Technology and Standards Activities”, in SMPTE Motion Imaging Journal, 2013. http://journal.smpte.org/content/122/6/local/complete-issue.pdf IN STRICTEST CONFIDENCE


Page | 52

It is stated in the scope section of the standard that it is anticipated that the data structures will CONFIDENCE be extended to include additional IN STRICTEST EOTF and HDR metadata capabilities in future versions. In April 2015, the HDMI Forum announced the completion and release of Version 2.0a of the HDMI Specification enables transmission of HDR formats by making reference to CEA-861.3.

6.9

123

. This update

The Digital Cinema Initiatives (DCI)

In July 2014, the Digital Cinema Initiatives (DCI) issued a statement including new projection light sources, as quoted below.

124

about three topics that they were currently working on, IN STRICTEST CONFIDENC

New projection light sources, such as laser, supporting wider colour space and higher dynamic range have emerged that hold the promise of an improved cinematic presentation. DCI is creating additional specifications in order to take advantage of these capabilities, while retaining one of its core objectives: to ensure that distribution are interoperable on all systems. DCI IN packages STRICTEST CONFIDENCE seeks to ensure that next-generation Digital Cinema Packages (DCPs) will be accurately reproduced on any system with this capability. IN STRICTEST CONFIDENCE 125 The current DCI specification , which references numerous SMPTE documents, specifies the following:         

The Material eXchange Format (MXF) shall be used Resolution is 4096x2160 at 24 frames per second, or 2048x1080 at 24IN or STRICTEST 48 frames per CONFIDENCE second X’Y’Z’ colour space, with gamma of 2.6, and 12 bits per non-linear colour sample Images to be encoded according to ISO 15444-1:2004/AMD1, i.e. JPEG-2000 The entire image shall be encoded as a single tile Each compressed frame of a 2K distribution shall have exactly 3 tile parts, one for each colour component Each compressed frame of a 4K distribution shall have exactly 6 tile parts, the first 3 representing one 2K colour component, and the next 3 containing additional data to decompress each of the three 4K colour components A 2K decoder shall decode the 2K data from a 4K data stream and ignore the rest Bytes per image constraints limit the bit rate to 200MBit/s for 2K and 250MBit/s for 4K STRICTEST CONFIDENCE

6.10 The Forum for Advanced Media in Europe (FAME) The Forum for Advanced Media in Europe (FAME) is co-chaired by the Digital Interoperability Forum (DIF) and the European Broadcasting Union (EBU). It provides a venue for device manufacturers, platform operators, software and middleware vendors, applications developers and broadcasters engaged in advanced media, and their relevant industry associations, to share knowledge 126 and experience that will support the successful, industry-led introduction of these services in Europe . IN STRICTEST CONFIDENC FAME aims to reach a voluntary consensus on issues concerning interoperability and standardization related to advanced media experiences. Although FAME does not develop standards by itself, it provides a venue to examine the feasibility of establishing guidelines that could encourage the technical availability of content and applications. The first FAME UHDTV Roadmap workshop was held in Lucca, Italy, on 5 and 6 June 2014, and was attended by over 65 delegates, IN STRICTEST CONFIDENCE 127 representing professional and consumer industries, network operators, and private and public broadcasters . Presentations from broadcasters NHK from Japan (who are focussed on the 2020 Olympics in Tokyo) and Kobeta from South Korea STRICTEST CONFIDEN highlighted the possibility that Europe could lag behind in introducing UHD TV IN if significant progress is not made soon. It was concluded that Europe had a 12 month window of opportunity in which to agree the technical requirements and implementation scenarios for what is called UHD 1 Phase 2. The aim of this second phase of UHDTV development is to offer a comprehensive set of advanced parameters: Higher Frame Rate (HFR), Higher Dynamic Range (HDR), higher resolution (3840x2160 pixels) and an extended colour space. It is regarded as a major IN STRICTEST CONFIDENCE 123

http://www.hdmi.org/press/press_release.aspx?prid=138

124

http://dcimovies.com/DCI_Initiatives_2014-0709.pdf

125

http://www.dcimovies.com/specification/


126

UHD TV WORKSHOP, LUCCA FORUM 2014. http://www.difgroup.eu/uploads/DocsAndMediaManager/documents/FAME%20Conference%202014%20-Eng.pdf IN STRICTEST CONFIDENCE 127 12 months left to lay out UHDTV 'Roadmap'. http://www3.ebu.ch/contents/news/2014/06/tech--innovation-12-months-left.html IN STRICTEST CONFIDENCE


Page | 53

step beyond the HDTV services that the majority of European broadcasters areIN currently transitioning to; and an advance on the 4k STRICTEST CONFIDENCE technology that is already being sold in shops but which concentrates 'only' on an increase in resolution. A further meeting was planned for Wednesday 12 November in Geneva

128

.

6.11 UHD Alliance and UHD Forum A group of organisations led by Harmonic started discussions on UHDTV, taking an end to end viewpoint, in December 2014, using 129 the name “Ultra HD Alliance” . The plan was to discuss with different UHD forums during December, before getting together a IN STRICTEST CONFIDENC core group of organisations during CES in January 2015, and making a public launch at NAB in April 2015. It was claimed that the UHD market is confused, being pulled in different directions by different interests, with some standards already existing and others in development. It was claimed that an industry wide focus was needed. The scope of the Alliance was stated to be for the use and promotion of UHD for: contribution and distribution; production andCONFIDENCE post-production; playout; IN STRICTEST distribution on managed networks; OTT distribution for live and VOD; and CPEs. The Alliance was stated to consist of the following working groups: Specifications (to collaborate with standards bodies, and to IN STRICTEST CONFIDENCE describe and point to all relevant standards); Interoperability (to define the interoperability points at the system level, to organise plug fests, and to publish the results); Business Model (to publish generic business models); Forecast; Applications; and Promotion. IN STRICTEST However, during CES in January 2015, another group of companies, led by Samsung, announced CONFIDENCE a “UHD Alliance” the former organisation renamed itself the “Ultra HD Forum”.

130

. Consequently,

The UHD Alliance claims to be a global coalition of leading film studios, TV brands, content distributors, post-production and technology companies that aims to create a unified criterion for premium UHD platforms, from devices to content including next generation features like as 4K resolution, High Dynamic Range, Wide Colour Gamut, High Frame Rate and Immersive Audio. The group is reported to be composed of DIRECTV, Dolby Laboratories, LG Electronics Inc., Netflix, Panasonic Corporation, Samsung Electronics Co., Ltd., Sharp Corporation, Sony Visual Product Inc., Technicolor, The Walt Disney Studios, 20th Century Fox and Warner Bros. Entertainment. STRICTEST CONFIDENCE The UHD Alliance launch press announcement mentions establishing new standards to support innovation in video technologies including 4K and higher resolutions, high dynamic range, wider colour gamut and immersive 3D audio, while also suggesting UHD branding by stating that premium Ultra-HD content and devices will be clearly identified so consumers can easily recognize them in store. 131

Thierry Fautier of Harmonic suggests that the UHD Alliance is focussing on a single blocking factor at the moment, that is, the specification of high dynamic range, wide colour gamut and immersive audio, but will have a broader marketing and evangelisation remit. IN STRICTEST CONFIDENC He states that the UHD Forum, a group of about 40 companies led by Harmonic, is considering the end-to-end ecosystem impact. In the longer term he thinks that there is no reason why the two entities might not merge, but currently that it seems most efficient to have the two bodies with the different focuses. He also stated that discussions are on-going to ensure that the two groups (UHD Alliance and UHD Forum) work closely together. IN STRICTEST CONFIDENCE In April 2015, just before the NAB (National Association of Broadcasters) Show in Las Vegas, the UHD Alliance issued a call for 132 companies to join the UHD Alliance as contributing members to define the "Next-Generation Entertainment Experience" . IN STRICTEST CONFIDEN


“Shaping the UHDTV roadmap for Europe”. https://tech.ebu.ch/docs/tech-i/EBU%20tech-i%20issue%2021_web.pdf

129

“Ultra HD Alliance”. http://www.hdforum.fr/sites/default/files/ultra_hd_alliance__hd_forum.pdf IN STRICTEST CONFIDENC

130

http://www.uhdalliance.org/wp-content/uploads/UHD-Alliance-Announcement-Press-Release.pdf

131

http://www.ctoic.net/blog/ecosystem-getting-organized-for-real-uhd-deployments-in-2016/

132

IN STRICTEST CONFIDENCE http://www.uhdalliance.org/uhd-alliance-calls-for-contributor-members-to-define-the-next-generation-entertainmentexperience/#more-540 IN STRICTEST CONFIDENCE


Page | 54

7

Abbreviations


ACES

Academy Color Encoding System

AMOLED

Active Matrix Organic Light Emitting Diode

ARRIRAW

A data format produced by the ARRI ALEXA camera

AVC

Advanced Video Coding

BBC

British Broadcasting Corporation

BDA

Blu-Ray Disc Association

BT

British Telecommunications plc

BTF

Broadcast Technology Future’s Group

CCFL

Cold Cathode Fluorescent Lamp

CEA

Consumer Electronics Association

CEATEC

Combined Exhibition of Advanced Technologies

CES

Consumer Electronics Show

CIE CPE

International Commission on Illumination (or Commission Internationale de l´Eclairage) IN STRICTEST CONFIDENCE Customer Premises Equipment

CRT

Cathode Ray Tube

DASH

Dynamic Adaptive Streaming over HTTP

DCI

Digital Cinema Initiatives

DCP

Digital Cinema Package

DICOM

Digital Imaging and Communications in Medicine

DIF

Digital Interoperability Forum

DLP

Digital Light Processing (Video Projector)

DTG

Digital TV Group

DTT

Digital Terrestrial Television

DVB

Digital Video Broadcasting Project

DVI

Digital Visual Interface

EBU

European Broadcasting Union

EOTF

Electro-Optical Transfer Function

ETSI

European Telecommunications Standards Institute

FAME

Forum for Advanced Media in Europe

HD

High Definition

HDCP

High-bandwidth Digital Content Protection

HDMI

High-Definition Multimedia Interface

HDR

High Dynamic Range

HDTV

High Definition Television

HEVC


HFR

Higher Frame Rate

HVS

Human Visual System

IEC

International Electrotechnical Commission

IRT

Institut für Rundfunktechnik

ISO

International Organization for Standardization









ITU-R

IN STRICTEST CONFIDENC International Telecommunication Union Radiocommunication Sector

ITU-T

International Telecommunication Union Telecommunication Standardization Sector

JPEG

Joint Photographic Experts Group

LCD

Liquid Crystal Display

LDR

Low Dynamic Range




Page | 55

LED

Light Emitting Diode

MPEG

Moving Picture Experts Group

MV-HEVC

Multiview HEVC

MXF

Material Exchange Format

NAB

National Association of Broadcasters

NHK

Japan's national public broadcasting organisation

OCES

Output Color Encoding Specification

OETF

Opto-Electrical Transfer Function

OLED

Organic Light Emitting Diode

OpenEXR

A high dynamic-range image file format

OTT

Over The Top (content delivery)

RGB

Red Green Blue

RVU

Remote Viewing (pronounced "R-view")

SD

Standard Definition

SDTV

Standard Definition Television

SEI

Supplemental Enhancement Information message IN STRICTEST CONFIDENCE

SHVC

Scalable HEVC

SMPTE

Society of Motion Picture and Television Engineers

TIFF

Tagged Image File Format

TV

Television

UHD

Ultra High Definition

UHDTV

Ultra High Definition Television

USB

Universal Serial Bus

VANC

Vertical Ancillary data

VOD

Video On Demand

VUI

Video Usability Information

WCG

Wider Colour Gamut

YAG

Yttrium Aluminium Garnet














Page | 56

About BT Media & Broadcast


BT Media & Broadcast (M&B) is the digital media and broadcast services arm of BT Wholesale. All of the UK’s digital terrestrial TV is underpinned by M&B’s specialist media network, whilst it is also responsible for delivering outside broadcast services from over 150 sports and news locations around the UK. M&B’s global media network links every major location around the world where broadcast or film content is created or distributed. www.mediaandbroadcast.bt.com IN STRICTEST CONFIDENC About BT Research and Innovation BT Research and Innovation provides world-class technical expertise to the rest of the BT business. We enable them to create new IN STRICTEST CONFIDENCE services and find new ways of doing things to improve efficiency and reduce costs for BT. Our open innovation model gives us access to thinking from outside BT from small, start-up companies, to some of the best universities around the world. We have teams working with customers, partners and universities in the US, Asia, Europe and the Middle East. IN STRICTEST CONFIDENCE BT is one of the largest investors in research and development (R&D) in the UK and, over the last 10 years, is the second largest investor in R&D in the telecoms sector in the world. Since 1990, we have had over 10,000 patents granted and we are active in a number of standards bodies. IN STRICTEST CONFIDENCE http://www.btplc.com/innovation



IN STRICTEST CONFIDENCE If you want to reproduce or redistribute any of the material in this publication, you should first get BT’s permission in writing. IN STRICTEST CONFIDEN This research paper contains a summary of industry analysis carried out in Q1 2015. BT cannot guarantee the completeness or accuracy of the information in this research paper and shall not be responsible for errors or inaccuracies. You should not rely on any opinion, research or analysis provided by BT as being a recommendation or advice. Under no circumstances shall BT be liable for any reliance by you on any information in this research paper. IN STRICTEST CONFIDENCE

IN STRICTEST CONFIDENC Offices worldwide © British Telecommunications plc 2015 Registered office: 81 Newgate Street, London EC1A 7AJ Registered in England No: 1800000

IN STRICTEST CONFIDENCE http://www.btplc.com/innovation IN STRICTEST CONFIDENCE


Page | 57