Metadata Lifecycle and Framework Whitepaper - Consumer ...

Consumer Electronics Association Metadata Study Group

Metadata Lifecycle and Framework Whitepaper

Abstract This whitepaper describes metadata use cases; identifies issues around the loss, corruption, versioning, formats, and rights for metadata; and makes recommendations for CEA and other stakeholders as to how to resolve these issues to the benefit of CE devices. October 29, 2010

TABLE OF CONTENTS TABLE OF CONTENTS .............................................................................................................................. I EXECUTIVE SUMMARY ........................................................................................................................... 1 BACKGROUND AND PURPOSE ............................................................................................................. 1 NOMENCLATURE ...................................................................................................................................... 2 ACRONYMS ................................................................................................................................................. 2 INFORMATIVE REFERENCES ................................................................................................................ 3 SECTION 1 FRAMEWORK OVERVIEW ......................................................................................... 4 1.1 CREATION .................................................................................................................................................. 4 1.2 DISTRIBUTION ........................................................................................................................................... 5 1.3 AGGREGATION AND ENHANCEMENT ..................................................................................................... 6 1.4 CONSUMPTION .......................................................................................................................................... 7 SECTION 2 CASE STUDIES ............................................................................................................... 9 2.1 MUSIC MEDIA AND METADATA LIFECYCLE ...................................................................................... 10 2.1.1 Media Lifecycle in Music ................................................................................................................... 10 2.1.2 Metadata Lifecycle in Music ........................................................................................................... 10 2.2 PHOTO MEDIA AND METADATA LIFECYCLE ..................................................................................... 12 2.2.1 Media Lifecycle in Photos ................................................................................................................. 12 2.2.2 Metadata Lifecycle in Photos ......................................................................................................... 12 2.3 THEATRICAL FILM MEDIA AND METADATA LIFECYCLE ................................................................. 14 2.3.1 Media Lifecycle in Theatrical Film............................................................................................... 14 2.3.2 Metadata Lifecycle in Theatrical Film ....................................................................................... 14 2.4 BROADCAST TELEVISION MEDIA AND METADATA LIFECYCLE ..................................................... 16 2.4.1 Media Lifecycle in Broadcast Television ................................................................................... 16 2.4.2 Metadata Lifecycle in Broadcast Television ............................................................................ 16 SECTION 3 IDENTIFIED ISSUES .................................................................................................. 18 3.1 LOSS OF METADATA .............................................................................................................................. 18 3.1.1 Conversion ............................................................................................................................................... 18 3.1.2 Distribution ............................................................................................................................................ 18 3.1.3 Unexpected Creation .......................................................................................................................... 18 3.1.4 Not Captured .......................................................................................................................................... 18 3.2 METADATA CORRUPTION .................................................................................................................... 19 3.2.1 Editing ....................................................................................................................................................... 19 3.2.2 Conversion ............................................................................................................................................... 19 3.3 MEDIA AND METADATA VERSIONING ................................................................................................ 19 3.4 MISALIGNED FORMATS......................................................................................................................... 19 3.5 METADATA RIGHTS MANAGEMENT ................................................................................................... 19 SECTION 4

RECOMMENDATIONS FOR CE DEVICES ............................................................. 20

SECTION 5 RECOMMENDATIONS FOR COLLABORATION WITH OTHER STAKEHOLDERS .................................................................................................................................... 21

i

EXECUTIVE SUMMARY This report to the CEA Technology and Standards Council describes the results of the Metadata Study Group (MSG) research. It reviews metadata and its usage throughout the life of content. It contains specific recommendations for CEA standards, bulletins, and liaisons with third‐party organizations. It has been formatted so that it can be repurposed as the starting point for future work. The metadata used by consumer electronics (CE) products comes from many different sources, such as content creators, distribution channels, and viewing platforms – all of which may be related to one or more audio or video media types. This metadata may originate from or be delivered by the service or content provider, or may be created by users themselves. The complex interactions and wide variations between these different sources of metadata have lead to the development of a framework (described in Section 1) that can be used describe the interactions between media and metadata at an abstract level, so that common issues that arise for various stakeholders can be identified (such as those in Section 2). A wide range of specific issues related to metadata are identified, ranging from users having difficulty finding and playing media, to issues around creating and sharing metadata, to rights management issues for metadata itself. The broad themes of issues surrounding the use of metadata in CE devices include loss, corruption, versioning, formatting, and rights management (described in Section 3). These generalized issues apply across multiple media types and distribution channels and ultimately impact a consumer’s abilities to access media and / or access and modify metadata through their CE devices. Some metadata‐related issues are within CEA's scope, as they are related to either CE device interface standards or to standards or recommended practices related to the behavior of CE devices.

BACKGROUND AND PURPOSE On August 31, 2009, CEA's Technology & Standards Council created the Metadata Study Group ("MSG") to study the applications and use cases of metadata and make recommendations concerning the metadata requirements of CE devices, the need for metadata‐related CEA relationships with other organizations, the need for metadata‐related coordination of standards development, as well as possible standards and/or recommended practices that define how CE products receive, share, process and display metadata. The MSG conducted its work in two steps. First, it identified the ways metadata is used by consumer electronics products by collecting descriptions of use cases. Second, it drafted this white paper, which describes how metadata is used by CE

1

products and identifies areas where the CE industry might benefit from metadata collaboration, new standards or new recommended practices. This whitepaper documents, clarifies and advocates the resolution of problems that were identified during the initial phase of work. Specific goals of this document include: • Educating CE OEMs about the need for technical mechanisms for redistribution of metadata between CE devices. • Highlighting use cases where CE devices may create metadata that can be used outside the CE ecosystem, so that those use cases can be realized by both CE and non‐CE stakeholders. • Highlighting use cases that may be problematic for CE devices but must be solved by non‐CE stakeholders, so that those issues may be realized and evaluated for resolution by non‐CE stakeholders. The wide variation in media types, distribution channels, CE devices and other factors make it difficult to illustrate all the issues that CE devices may face. To that end, this document describes a common framework for describing the relationships between media and metadata. The purpose of the framework is to create a common structure and language for discussing issues around metadata, and especially for identifying, documenting, clarifying and proposing resolutions to issues in the metadata lifecycle.

NOMENCLATURE Media – The essence of the work being referenced, such as the pixels of a photo or the sound wave of an audio file.

ACRONYMS CDS CE CEA DLNA DVD EBU IMDB MP3 MPAA MPEG MSG MVPD OEM PC RIAA

Content Directory Service Consumer Electronics Consumer Electronics Association Digital Living Network Alliance Digital Versatile Disc European Broadcasting Union Internet Movie Database MPEG Layer 3 Motion Picture Association of America Moving Picture Experts Group Metadata Study Group Multichannel Video Programming Distributor Original Equipment Manufacturer Personal Computer Recording Industry Association of America 2

SCTE SD SMPTE TMS TV UPnP VOD XML

Society of Cable Telecommunications Engineers Secure Digital Society of Motion Picture and Television Engineers Tribune Media Services Television Universal Plug and Play Video On Demand Extensible Markup Language

INFORMATIVE REFERENCES 1. SMPTE / EBU Definitions of Metadata and Essence – Task Force for Harmonized Standards for the Exchange of Program Material as Bitstreams, July 1998: http://www.smpte.org/standards/tf_home/Final_Report__Findings.pdf 2. EXIF – http://www.cipa.jp/english/hyoujunka/kikaku/pdf/DC-0082010_E.pdf 3. MP3 – http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?cs number=22412 4. ID3 – http://www.id3.org/id3v2.3.0 5. UPnP Content Directory Service (CDS) – http://upnp.org/specs/av/UPnPav-ContentDirectory-v3-Service.pdf

3

SECTION 1

FRAMEWORK OVERVIEW

There are four basic stages to the lifecycle of metadata: creation; distribution; aggregation and enhancement; and consumption. Figure 1 shows the relationship between the media lifecycle and the metadata lifecycle. The bold arrows (in green and blue at the top and bottom of the diagram, respectively) show the most commonly thought about relationships between media and metadata – when media is created, metadata is created; when media is consumed, metadata is consumed. However, the diagram also shows a number of other relationships between media and metadata, such as the multiple areas where metadata is created (green arrows), the areas where metadata may be aggregated and enhanced, secondary relationships for metadata, and the areas where metadata may be consumed (blue arrows). The diagram also highlights another important relationship between media and metadata, which is that they may be distributed together or separately (as denoted by the dotted line between the two distribution stages).

Figure 1 – Relationships between the Media Lifecycle and the Metadata Lifecycle. Closed headed arrows represent potential data flows; Openheaded arrows represent a potential relationship.

This framework is anticipated to be useful to all metadata stakeholders; however, this specific document focuses on the applications and implications for CE devices. The rest of this document goes on to describe the lifecycle of metadata, with some comments on how the metadata lifecycle relates to media.

1.1 Creation The most commonly thought of phase of the creation stage of metadata is during the media creation stage. As a piece of media is created, the data around its creation is captured and forms the beginning stages of the metadata. For example, when

4

recording a film information such as location, actors, film type and working title may be captured and stored as metadata for future reference. As metadata is created in conjunction with the media, the metadata may vary widely based on the media that is being created. Types of media to consider include: • Video (Film, Broadcast TV) • Audio (Music, Audiobooks) • Print Publication (Books, Magazines, Newspapers) • Photography Although the creation of metadata is most commonly thought of as being associated with the creation of content, each additional stage in the life of the metadata can also result in the creation of content. During the distribution stage, metadata may be created about the media in relation to the distribution channel, such as price of the media or scheduling information of when the media will be available. The aggregation and enhancement stage may create metadata in the form of cross‐ referencing media and metadata to other types of metadata. For example, actor’s biographies and headshots may be acquired from secondary sources and linked to the primary asset. Aggregation serves the purpose of creating relationships between existing sets of metadata. For example, while the biographies or headshots already existed, as did the metadata for the media being distributed, the link between these two sets of metadata serves as a new form of metadata in itself. Enhancement comes in the form of entirely new metadata about the media asset, such as professional reviews or parental controls ratings. Finally, the consumption stage may also create metadata. For example, end users may create bookmarks, playlists or ratings as they are consuming the media.

1.2 Distribution Distribution of metadata may take the form of metadata that is co‐located with the media, such as being part of the media format, or the metadata may be independent of the media. Examples include cover‐art and title of an audio asset being inserted into a MP3 ID3 tag as part of the same file, which persists wherever the MP3 is transferred; or cover‐art and title information for a Compact Disc being transferred as part of an XML file separate from the physical media. Note that the instance where metadata takes a different distribution path than the media (that is, where the metadata isn’t co‐located with the media), may lead to some or all of the metadata not being distributed. The distribution of metadata is closely related to 1) the type of media being

5

distributed; 2) the stage of the lifecycle during which the metadata was created; and 3) the medium being used to distribute the media. The types of media and the various places where metadata can be created are described in the previous section. Different types of media for distribution include: • Broadcast (Radio, Television) • Physical (Discs, Printed) • Digital On‐Demand (Internet, MVPD VOD) Since the three different considerations in distribution create a complex set of scenarios of what metadata is relevant for distribution, it may be worth considering a few illustrative examples: • Metadata about television media being distributed by broadcast may be associated with scheduling metadata and frequency / channel metadata. • Metadata about audio media being distributed by broadcast (a/k/a radio) may be associated with frequency / channel metadata and may or may not have associated scheduling metadata. • Metadata about television media being distributed by physical medium may be associated with price and physical format of the physical medium. • Reviews of television media may be distributed by physical medium (i.e. – TV Guide Magazine). • Reviews of film media may be distributed by Digital On‐Demand medium (i.e. – RottenTomatoes.com). • Music metadata created by the consumer (such as playlist information) may be distributed by Digital On‐Demand (i.e. – DLNA)

1.3 Aggregation and Enhancement The aggregation and enhancement stage of the metadata lifecycle is where metadata from different sources may become associated with the primary metadata. For example, metadata about actors, cast, and crew may be aggregated by a user of metadata to help describe the media asset. The primary media asset may have already contained the names of actors, cast and crew, but the aggregated data may include biographies, headshots, additional media that the people are associated with, relationships between the people, etc. During this aggregation, the metadata created is not about the media itself, but rather about the relationship between the media and other items in the real world. Enhancement serves much the same purpose of aggregation – the defining of relationships between media and other assets – but it typically comes about through the formation of new metadata. Examples of enhancement include the creation of parental control ratings (MPAA, RIAA, TV Guidelines, etc) or professional reviews where an independent entity creates metadata about that specific media for the purpose of enhancing the understanding of the media.

6

There are multiple ways to link different sets of metadata. The traditional way of aggregating metadata and linking it to enhancements is by title. For example, the title of a film is used to link it to its reviews and ratings. This is obviously an imperfect system, since titles of films are not guaranteed to be unique and there may be issues around the wrong links occurring. In more modern systems, linking may occur directly through URLs that point to additional information, or indirectly through unique identifier systems. Specific examples of metadata aggregation and enhancement include: • TV Scheduling Information (i.e. – Rovi, TMS) • Physical and Digital Media Description (i.e. – Rovi, Gracenote) • Critics and Reviewers (i.e. – RottenTomatoes, Roger Ebert) • Web Indexers (i.e. – IMDB, Google)

1.4 Consumption The consumption of metadata is primarily thought of as an act by an end‐user who is consuming the media. For example, a user who is listening to music may use the cover‐art to browse through his catalog, or may search for a specific asset by title and artist metadata. Metadata may also be consumed at virtually every other stage of the media’s life. Studios may use metadata to catalog their assets and retailers may use metadata to represent the items they are selling. The metadata consumed may vary widely based on where and how it is being consumed. Audio media that is being consumed through an audio/visual device may make use of cover‐art, whereas the same media being consumed through an audio only device would not have the opportunity to make use of the cover‐art. The different points of consumption for metadata include: • Video • Audio • Print Note that the points of consumption for metadata parallel the types of media that were created; however, there is not always a direct correlation between the type of media and how it is consumed. For example, audio and print media and metadata may be consumed through a video point of consumption, and video metadata may be consumed through print. Also note that the consumption of metadata need not take place in same place or time as media consumption, and that “informational services” may make use of metadata to make users aware of the availability of media. Such informational

7

services may include movie theater listings, electronic program guides, news paper listings, music catalogs, etc.

8

SECTION 2

CASE STUDIES

This section shows the generalized metadata lifecycle as applied to specific instances, especially with a focus on consumer electronics devices. Some diagrams include some mention of media workflows and / or distribution, but this is an informative reference and clarification as to the source of data. These descriptions are not normative or authoritative with regard to content creation or distribution, as that is the purview of organizations such as SMPTE, SCTE and others. It should also be recognized that these use cases are not exhaustive of all the different instances of media types, distribution channels, ecosystem players or other factors, which could create an exponentially large set of use cases. Rather, these use cases focus on useful examples of how the generic metadata framework and lifecycle can be applied, especially to highlight issues that arise related to consumer electronics devices. Each case study below contains a description of a typical media flow and a typical metadata flow. Note that these flows are not the only flows possible, and are simplified to demonstrate the relevant points. The metadata lifecycle portion of each use case has numbered points that refer back to the corresponding diagram, and noteworthy points referenced later in the document are in bold.

9

2.1 Music Media and Metadata Lifecycle

Figure 2 – An example of the relationship between music media and metadata to highlight the availability of metadata (1,4); and the redistribution and use of user created metadata (8).

2.1.1 Media Lifecycle in Music • • •

Music media is created by a label in a recording studio, or by other means. Media is downloaded by a consumer, from an online store or through other means, onto a device. Media is consumed on an end device in the form of a user listening to music.

2.1.2 Metadata Lifecycle in Music The numbers below correspond to the red‐circled callouts in Figure 2: 1. During and after the recording of music, information is captured about the artists, attributes and names of songs, along with other information. Some information, such as cover art for albums, is created after the recording process. 2. Additional information about the music, such as artist biographies or discographies, is crossreferenced and linked to the album 3. Metadata is distributed to the end device, either embedded in the media file (shown as 3a in Figure 2; for example, ID3 tags in a MP3 file), or through a

10

4.

5. 6. 7. 8.

parallel and independent distribution path (shown as 3b in Figure 2; for example, an XML file delivered after the file has been received). Metadata is consumed as an enduser browses or searches through their catalog of available music. Note that at this point, only metadata that has been created at recording time or linked to the original metadata is available to help the enduser browse their music catalog. In the process of consuming or organizing music, an end‐user may create playlists, modify metadata, create bookmarks in audiobooks, create ratings or reviews, or create and modify metadata in a number of other ways. Metadata created by end‐users may be embedded back into the original files, or may persist in a separate database independent of the media files. Media may be redistributed to a secondary device. This may include transferring files from a PC to a portable music player, or between stereo components in a house, using protocols such as DLNA or UPnP. When media is redistributed in the home to a second device, the metadata on the device will also be shared if the metadata is embedded in the file. If the metadata was stored in an independent database, it may be distributed through protocols such as UPnP Content Directory Service (CDS) or it may not be redistributed at all.

11

2.2 Photo Media and Metadata Lifecycle

Figure 3 – An example of the creation of photos and the resulting metadata to highlight the distribution of metadata embedded in the photo (2,8); and the modification and redistribution of both photos and metadata through the editing process (7).

2.2.1 Media Lifecycle in Photos • • • • •

Take Photo – A consumer creates a series of photos using a consumer‐grade camera Card or Network – The photos are transferred by either removing a card from the camera, or by using networking functionality built into the camera to transmit the photos to another device. PC – Photos are stored and managed on the PC Editing / Other Tools – On the PC, a consumer can use editing software to crop, rotate, remove red‐eye or perform other modifications on the photos. Viewing Device – After editing, the media may be transferred by card or network to a viewing device, such as a digital picture frame or a television.

2.2.2 Metadata Lifecycle in Photos The numbers below correspond to the red‐circled callouts in Figure 3:

12

1. When the original photo is taken, metadata is created at the same moment to capture information about the media, which may include information about resolution, location, creation time, camera settings, etc. 2. It is common for EXIF metadata to be embedded directly into the photos. 3. As the photo is transferred using a SD card or network, the metadata embedded into the photo is transferred as well. 4. Photos are copied onto the PC, or may be manipulated on their original storage medium (such as editing photos directly on the SD card). 5. Photos may be organized on the PC using the embedded metadata. While some metadata isn’t especially useful for end‐user navigation, other information (such as location or time of creation) may be especially relevant. 6. Photos are edited to the user’s preferences using software such as Photoshop, as depicted by 6a. Also note that photos may bypass the PC and be edited directly in a CE device, such as the camera that created the photo, as depicted by 6b. 7. As photos are edited, metadata may be created, altered or removed. If photos are resized, the embedded metadata might be updated to reflect new resolutions or creation / modification timestamps may be updated to the time that changes are made. 8. The updated metadata, if any, is then embedded back into the media. 9. The photos and embedded metadata are transferred again by card or network. Note that the transfer may or may not use the same mechanisms as before. 10. The device used for viewing the photos may use the metadata embedded in photos for manual navigation in searching and selecting photos to display, or may use the metadata automatically to create slideshows.

13

2.3 Theatrical Film Media and Metadata Lifecycle

Figure 4 – An example of the relationships between theatrical film media and the related metadata to highlight the loss of metadata resulting from the use of thirdparty media (2); the alternate versions of media and corresponding metadata created during the editing process (3); and the availability and formats of metadata available to enduser devices (8).

2.3.1 Media Lifecycle in Theatrical Film • • • •

Filming – The original film is created by a professional movie crew and actors, using the location and equipment of their choosing. Editing – After the film is created, it may be edited to include sound tracks, alternate language tracks, remove culturally objectionable material, and / or formatted in any number of different ways. Assets – Each asset created during the editing phase may be distributed to different geographic locations or market segments through broadcast, physical or digital distribution channels. Device – While original distribution of an asset is to a theater, eventually the film may reach a consumer electronics device through any number of companies that have negotiated access to the asset.

2.3.2 Metadata Lifecycle in Theatrical Film The numbers below correspond to the red‐circled callouts in Figure 4:

14

1. During the creation of the film, metadata is created about the film and its contents. 2. As part of the editing process new media may also be introduced as part of the film, such as still images, likenesses, soundtracks and voiceovers. Each asset acquired from a thirdparty for inclusion into the film may have a complex set of rights about where it may and may not be used. Note that neither the rights around the new media are captured as a form of metadata, nor is the metadata of the thirdparty media preserved as a subset of the film metadata. 3. As alternate versions of the film are created during the editing process, a complex hierarchy of metadata is created for each new version of the film. Metadata may describe new lengths, resolutions, CODECs, actors, features or other information. 4. Each version of a film may be submitted (using mechanisms not described here) to a wide variety of ratings bodies that judge the appropriateness for various audiences based on age, geography, or common belief systems. These ratings are distributed as independent metadata that may be used to legally determine the availability or rules of viewing for various assets. 5. Ratings and other metadata are linked into a common set of metadata that can be used to describe each asset. The metadata surrounding alternate versions of assets may be linked into a common container, if it is available. 6. The metadata is distributed either embedded in the media file, as a separate data feed, or as some combination. For example, title and duration may be embedded directly into the media file, while synopsis and actors may be distributed as a secondary data file. 7. Metadata is used by end devices to browse through catalogs and select which asset to view. 8. Multiple versions of the asset are made available to consumers. Some variants may be easily accessible, such as DVD or digital download version; other versions may not be as easily identified, such as variants that have been made for different geographies. If metadata is available for each of the variants, it may be used to help the consumer make the optimal choice. Also note that different distribution channels may have metadata in different formats or to different levels of completeness. 9. As films are viewed on the CE device, the users may give their own feedback about the media in the form of ratings, reviews, recommendations, alternate synopses, etc. This user generated metadata may be aggregated by other sources and fed back into distribution channels, creating a feedback loop for the consumption of media and the creation of metadata.

15

2.4 Broadcast Television Media and Metadata Lifecycle

Figure 5 – An example of the relationships between broadcast television media and metadata to highlight the parallel path for distributing metadata (5); and the creation of usergenerated reviews that may be aggregated and redistributed (8).

2.4.1 Media Lifecycle in Broadcast Television • • • •

Filming – The creation of television media Scheduling – After media is received by a broadcaster, the media is scheduled for a time when it will be broadcast to end consumers Broadcast – The actual transmission of television media to end consumers Viewing – The consumption of television media

2.4.2 Metadata Lifecycle in Broadcast Television The numbers below correspond to the red‐circled callouts in Figure 5: 1. Metadata about programs is created at the same time as the filming of the media. Examples of this metadata include human‐created metadata, such as title, and metadata that is inherently created during the creation of the media, such as time of creation, casting and credits, etc.

16

2. As broadcasters determine the airing times and dates of shows, the scheduling metadata may be captured internally and / or by third parties. 3. Secondary information about the media is captured and created. This includes information such as biographies of casting and credits, information about the broadcasters themselves, professional and non‐professional reviews and descriptions of the media, ratings and parental advisories, etc. Some of this information is created through viewing the media (before and after it is received through the distribution channels described in this document) and some is created without having viewed the media. 4. Program data, scheduling data and secondary information is aggregated and linked to create a single record about a specific instance of the airing of a show. The linking mechanisms in this specific instance are typically done through proprietary mechanisms. 5. Metadata is distributed through a path that is parallel to the media – it is not included in the media itself. The path may be entire schedule line ups that are inserted into the broadcast of a single channel, or may be the distribution of scheduling information through the Internet directly to devices. 6. At the point of viewing, metadata may be used to inform consumers about the availability of programming. 7. Metadata may also be consumed independent of the point of viewing, such as on websites that give information about upcoming programming or inform users about schedules. 8. After viewing media, consumers may create their own reviews, descriptions, or other information. This data becomes secondary information, that may inturn be aggregated, linked and redistributed. Note that this creates a cycle between metadata that a user creates, and the distribution of metadata back to the same or other consumers through existing metadata distribution channels.

17

SECTION 3

IDENTIFIED ISSUES

This section focuses on the aspects of the use cases described in Section 2 where CEA can take direct or indirect action. These issues drive the subsequent recommendations around next steps and proposed solutions in Section 4 and Section 5.

3.1 Loss of Metadata The most common issue in the lifecycle of metadata is where the relationship between metadata and media gets lost. This may happen for multiple reasons:

3.1.1 Conversion As media is converted from one format to another, metadata that was embedded in the media may be lost. This includes not only transcoding of media (Section 2.3.2, Paragraph 3), but also media that is used as a subcomponent of a larger work (Section 2.3.2, Paragraph 2).

3.1.2 Distribution In the case of metadata that isn’t embedded in media, the assumption is typically that the metadata takes a parallel distribution path to the media. In the cases where this assumption falls short, media may reach its destination without any access to the metadata (Section 2.1.2, Paragraph 8; Section 2.3.2, Paragraph 8; Section 2.4.2, Paragraph 5).

3.1.3 Unexpected Creation As Figure 1 shows, there are multiple points where metadata can be created, and typically just one point is considered (the point where media is created). These secondary areas of metadata creation can create pockets of metadata that would be useful, but becomes trapped because of unavailable distribution or formats for consumption that are poorly understood (Section 2.1.2, Paragraph 8; Section 2.4.2, Paragraph 8).

3.1.4 Not Captured During the creation process, there is potentially a large amount of metadata that may not be captured. While much of this metadata may not have value associated with it, there may be some that has value that isn’t captured. In the case of internal editing, this metadata would probably have no external value; however, if both the original and the new version of the media are distributed, it may be very important to understand the relationships between the two pieces of media (Section 2.1.2, Paragraph 4; Section 2.3.2, Paragraph 2).

18

3.2 Metadata Corruption If metadata is received by a consumer, there may be instances where the metadata received is incorrect.

3.2.1 Editing Media may be altered as part of an editing process, such as to create shorter versions of videos, creating geographically targeted media, etc. If this happens and the original metadata isn’t updated as part of the editing process, the metadata may end up being incorrect (Section 2.2.2, Paragraph 7). This may also be the case for interstitial advertising, which may artificially change the overall duration of a video, or may disturb the cross‐references of metadata that points to specific time‐codes.

3.2.2 Conversion Media may be transcoded or reformatted without updating metadata about codec or resolution information (Section 2.2.2, Paragraph 7; Section 2.3.2, Paragraph 3).

3.3 Media and Metadata Versioning As media is altered in editing processes, it may be relevant to track and relate the end‐products of editing that are distributed externally from the content creator. This means not only creating metadata to track revisions of the media, but also creating and maintaining the different versions of metadata that is intended to describe revised media assets (Section 2.3.2, Paragraph 3; Section 2.3.2, Paragraph 8). Metadata may also go through its own revisions, without any changes to the media, such as the loop of creating and modifying metadata described in Section 2.4.2, Paragraph 8.

3.4 Misaligned Formats When metadata is distributed by a mechanism that is different than the media distribution, there is a potential for any stakeholder receiving the metadata to not understand the format of the metadata that is being received. Variations in the format of the metadata may be as simple as different formats for dates, or as complex as different syntactical definitions of genres (Section 2.1.2, Paragraph 8; Section 2.3.2, Paragraph 8).

3.5 Metadata Rights Management In the case of metadata that is protected by copyright (i.e. – non‐factual metadata, such as reviews, photos, trailers, etc), tracking the rights and allowed usages of metadata may become problematic (Section 2.1.2, Paragraph 1; Section 2.1.2, Paragraph 2; Section 2.3.2, Paragraph 2; Section 2.4.2, Paragraph 8).

19

SECTION 4

RECOMMENDATIONS FOR CE DEVICES

As some of the use cases and issues identified within this document are specific to CE devices, it is recommended that CEA consider the following: 1. CEA device standards should include a section on metadata, if appropriate to the scope of the standard. This section should specify what metadata available to the device might be sharable with other devices, a format for carrying that metadata, and interface specifications for delivering such metadata to other devices. For more information, refer to Issues 3.1.2, 3.1.3, and 3.4. 2. CEA standards formulating groups that develop interface standards for CE devices should develop a list of metadata elements created by other stakeholders that are requested to be made available to CE devices across such interfaces. For more information, refer to Issue 3.4. 3. CEA R7 should consider whether existing CEA home networking standards including CEA‐2033 adequately cover the range of possible metadata that may want to be transported in a standardized way in the home network. If other needs are identified, the development of new standards should be considered. 4. CEA device standards for devices that edit or convert media should include a section on metadata integrity, and should require or recommend methods for assuring such integrity. For more information, refer to Issues 3.1.1, 3.2.1, and 3.2.2. 5. CEA standards for devices that copy or create metadata should specify protocols for such devices to assure that metadata remains associated with its underlying media during the distribution of said metadata. For more information, refer to Issues 3.1.2, 3.1.3, and 3.4. 6. CEA standards for use by devices that incorporate one piece of media into another piece of media should include methods for such devices to assure that, on output, the metadata of the incorporated media is preserved as part of the composite media (see Section 2.3.2, Paragraph 2). For example, CEA standards for devices that incorporate photos into video should include methods to assure that the photo metadata is still accessible to someone watching the video. For more information, refer to Issue 3.2.2. 7. CEA interface standards for delivering media from one device to another should include protocols for carrying and transporting metadata associated with such media, and methods for assuring that such metadata remains associated with its underlying media.

20

SECTION 5 RECOMMENDATIONS FOR COLLABORATION WITH OTHER STAKEHOLDERS The scope of CEA metadata‐related standards work is limited to how CE products use and create metadata. The creation and use of metadata within professional content creation facilities and distribution channels is outside the scope of CEA standards. Nevertheless, it is important for professional systems and CE devices to work well together, and thus it is important for CEA and other standards organizations to communicate well and understand each others' needs. With this is mind this section of the whitepaper is dedicated to identifying issues that could be the subject of communications to other standards organizations, consortia, or other stakeholders. Specific recommendations for points of collaboration include: 1. As CE devices' ability to create and share metadata continues to increase, it should be better understood what sorts of metadata are valuable outside of the CE ecosystem, and the types of metadata that should be created by CE devices and the mechanisms for sharing beyond CE devices (for example, sharing user reviews with content providers). For more information, refer to Issue 3.1.3 and its related use cases in Sections 2.1.2 and 2.4.2. 2. As more media is being made available through various distribution channels to CE devices, the metadata describing the media arrives in various formats. CEA members would like to collaborate on the formats of this metadata to try and encourage commonalities wherever possible. For more information, refer to Issues 3.3 and 3.4. 3. With various kinds of metadata being aggregated and linked from different sources, it would be helpful to set up a mechanism for understanding the usage rights of metadata that is being shared. For more information, refer to Issue 3.5. 4. In order to ensure that the metadata that is reaching CE devices is correct, CEA might communicate with other stakeholders about the need to have mechanisms for retaining metadata and ensuring that metadata remains valid through editing and conversion processes should be established. For more information, refer to Issues 3.1.1, 3.2.1, and 3.2.2. 5. To help facilitate the flow of media and ensure that it may be rendered correctly CE devices may also offer metadata about themselves to services so that the services can provide media in the right formats. If information about devices (such as metadata about screen resolutions or other device capabilities) or audiences (such as demographics) would be helpful to third‐

21

parties, it would be useful to collaborate around the specific information and formats that should be provided to service providers. For more information, refer to Issues 3.1.1, 3.1.2, and 3.1.4.

22