Category Archives: Technology Trends

HFR: Are We the Problem?

Posted on by | Leave a reply

Hobbit HFRPeter Jackson’s brave use of High Frame Rate (HFR) in “The Hobbit” has sparked intense debate over the past two months. The most prevalent questions have been:

“Does HFR look ‘better’ than traditional 24 fps, or does it just look ‘different’?”

“Will audiences eventually get used to HFR and then prefer it over traditional 24 fps, as with many other advances in motion imagery?”

Clearly HFR looks different. Anyone who saw “The Hobbit” in HFR can agree with that. And I think it’s fair to say that Jackson and his team succeeded in their goal to produce a more lifelike visual experience. But many viewers and critics found this new hyper-real experience off-putting, reducing the so-called “suspension of disbelief” that is considered essential to storytelling. Critics widely panned Jackson’s use of HFR, comparing the look of HFR to television and video games, deriding it as “non-cinematic,” “plasticine,” and “fake.”

If HFR didn’t work well for a cinematic adventure film like “The Hobbit,” there must be a scientific explanation. Or, as it turns out, maybe there isn’t. Yet…

At today’s Hollywood Post Alliance (HPA) Tech Retreat, Charles Poynton presented a fairly comprehensive overview of the challenges with motion imagery. His conclusion is that we don’t know enough about how motion is perceived by humans to explain with scientific certainty why HFR looks different. Poynton provided a thoughtful and detailed overview of how image capture and display technologies interact with the human perceptual system. Without going into detail about pulse width modulation or bit splitting (!) here are some of Poynton’s main points from today’s session:

  • We are still trying to fully understand how motion is decoded in the brain. Some scientists believe the brain processes chunks of input (20 to 50 milliseconds each) while others believe perception is more like a continuous stream.
  • Despite advances in our learning about human perception, along with advances in algorithms that use these learnings to automatically assess visual quality, the only way to conclusively assess visual quality remains subjective testing by humans.
  • The way that humans track motion, by moving their eyes several times per second, presents a challenge for motion imaging systems.
  • Different display technologies (LCD, Plasma, DLP, CRT) emit light in different ways over time, creating an additional factor that must be carefully optimized.
  • Along with wide gamma and other advances in motion image technology, HFR needs further study and refinement before it can be successfully adopted.

As a fitting postscript to Poynton’s presentation, Mark Schubin suggested that not only do we not yet know why HFR looks so different from traditional 24 fps, we also don’t know where this will all land. Many other advances in cinema technology were initially rejected by viewers, but eventually became standard fare.

In my opinion, HFR will eventually be incorporated as an additional, valuable tool for storytellers.  Higher frame rates are not inherently superior to 24 fps for all content genres, but they may meaningfully enhance the audience experience when used for scenes that require fast motion or hyper-reality. It may turn out that some television genres like news, live entertainment and sports may be better suited to high frame rates, because the goal of these productions is to approximate live attendance of the event. Dramatic programs with high production values, such as The Hobbit, may ultimately benefit from the suspension of disbelief induced by the cinematic look of 24 fps, incorporating HFR more sparingly as a visual effect.

So maybe HFR is not the problem – maybe we are. Notwithstanding the creativity and skill of Peter Jackson and his team, perhaps we don’t know how to do HFR well yet. Maybe we don’t yet know when to use HFR and when not to. But at least Peter Jackson has forced the discussion by boldly pushing into this new frontier, for which we can all be grateful.

Ultra HD TV: CES 2013 Update

Posted on by | Leave a reply

TV’s will be available this year, but adoption will be slow.

Not surprisingly, Ultra High Definition Television (UHDTV for short) was the big buzz at CES last week. All of the major TV manufacturers were demonstrating UHDTV sets running at 4K resolution, with Sharp showing an updated prototype 85” TV running at 8K resolution. But even as the TV sets themselves are approaching availability, the picture is becoming increasingly clear (pun intended) that uptake will be slow.

  • Although UHDTV sets will be available as soon as March of this year, initial pricing will be prohibitively high (>$10,000). According to CEA analysts, UHDTV units will account for only 5% of TV’s sold in 2016.
  • No 4K or 8K content is available currently to view on UHDTV sets and likely won’t be available in quantity for years to come.
  • There is currently no distribution method for UHDTV content. Data rates for UHDTV can be at least four to sixteen times higher than for 1080p HD.
  • Broadcasters are generally unconvinced that a viable business model will emerge for UHDTV.
  • Most industry analysts and TV vendors now acknowledge that UHDTV has little (if any) value for displays smaller than 50 inches. For my thoughts on the relative value of higher spatial resolution, please read my earlier post on this topic.

In short, UHDTV is a really impressive technology that is desperately in search of a market opportunity. It’s hard to dispute that UHDTV represents a future we can all get excited about, but (like HD before it) the market for UHDTV will likely take several years to develop. Read on if you want more of the gory details. If you’re a UHDTV novice, feel free to check out the primer on UHDTV I posted last September.

Samsung S9000 85-inch 4K UHDTV at CES 2013
(photo courtesy of pocket-lint.com)

UHDTV’s Will Be Available Soon

Virtually all of the major TV vendors demonstrated UHDTV sets at CES last week. Note that none of the sets is currently available, but some will begin shipping as soon as March 2013. Early units will include 94-inch displays from Sony and LG that will be priced at $20,000-$25,000. Smaller models will cost less – the Consumer Electronics Association (CEA) estimates that the average wholesale cost of 4K televisions will drop to $7,000 by late 2013, then to $2,800 in 2014. Even with this steep decline in price, only 1.4 million unit sales are projected for the U.S. in 2016, equivalent to roughly 5% of the market.

“It’s a very, very limited opportunity,” said Steve Koenig, director of industry analysis at the CEA. “The price points here are in the five digits (in U.S. dollars) and very few manufacturers, at least at this stage, have products ready.”

Content is (Lac)King

No UHDTV content is available to consumers currently. But the technology is available to create 4K content. Increasingly, feature films are shot with 4K cameras and are mastered at 4K for theatrical distribution. 4K consumer camcorders are already on the market. Classic films archived on 35mm film could be retransferred at 4K for distribution. And gaming engines could fairly easily render images in 4K or 8K. So the technology is available, but until a compelling market opportunity exists, content owners may not jump on board.

Upscaling and Other Sources

Early UHDTV sets from vendors like Samsung, Sony and Toshiba, will support upscaling of HD content to 4K. For example, Sony recently announced plans for a “Mastered in 4K” Blu-ray library that will offer content mastered in 4K that is downscaled to 1080p HD for Blu-ray release.

“When upscaled via the Sony 4K Ultra HD TVs, these discs serve as an ideal way for consumers to experience near-4K picture quality,” according to Sony.

While this may seem like a sketchy marketing ploy (because the discs are no different from today’s Blu-ray titles) I have to admit that standard def DVD’s look much better when upscaled to HD, so there is a possibility that Blu-ray discs will look even better when upscaled to UHDTV. Still, upscaling of 1080p content is at best a stopgap until UHDTV-native content becomes available.

One source of UHDTV-native content could be your computer. Graphics cards have long since surpassed the 1920×1080 resolution of HD. Home movies and photo montages will look great on a huge UHDTV. I can also imagine UHDTV’s becoming commercially viable as high-end displays for retail and corporate applications. But for television viewing, the lack of native UHDTV content will definitely be a barrier to adoption.

Bandwidth and Distribution

At 4K, UHDTV content has more than four times as many pixels as HD. At 8K, that multiplier jumps to >16x. Carrying that much data will require significant advances in compression technology as well as increases in network bandwidth. Ironically, at a time when sales of physical media are plummeting, these challenges may make physical media the most practical method for distributing UHDTV for the next few years. Recently, the Blu-Ray Disk Association has formed a task force to study the viability of extending the Blu-ray format to support 4k.

As for television, some broadcasters seem interested in exploring UHDTV. Recently, satellite operator Eutelsat Communications launched a demonstration 4K channel in Europe. And an experimental Ultra HD channel is also being planned in Korea. NHK in Japan hopes to begin experimental 8K broadcasts in 2020. But these technology experiments are primarily focused on exploring the technical viability of UHDTV broadcast.

Business Model

Market viability is perhaps the biggest question surrounding UHDTV broadcast. As with the 3D hype two years ago, some industry analysts expect sports programming to drive consumer interest in UHDTV.

The Hollywood Reporter recently stated that “BSkyB in the UK, Sky Deutschland in Germany, Japan’s Sky Perfect Jsat, and Brazil’s TV Globo have all started to explore the potential of 4K, which would include coverage of events such as sports…with an eye toward offering the 2014 FIFA World Cup and 2016 Olympics (both of which will be held in Brazil) in [UHDTV].”

But other broadcasters have gone on record as much more jaundiced about the market opportunity. During the Broadband Unlimited conference at CES last Monday, Sheau Ng, VP of research and development for NBC Universal says there’s no business model for UHD TV yet.

“Therein lies the rub,” Ng said. “It’s not the technology, it’s the business model. Where is the money? Unlike the previous revolution of HD, we have the device manufacturers selling the device when people are still scratching their head and saying ‘What do I do?’ That’s something we’re wrestling with every day. For us to say ‘We’re going to do this,’ we need somebody to say ‘here’s the business model, here’s the number of devices in the market, here’s how we’re going to make money.’”

And Bryan Burns, VP of strategic business planning and development at ESPN, hinted that some broadcasters will wait for 8k.

“By the time we get [to 4K] we will be on to 8K or whatever. I don’t want to make the capital investment [in 4K]. There might be a gradual evolution…but I don’t see us heading to 4K production or an ESPN 4K channel.”

In Summary…

With the proliferation of high resolution cameras and displays, the question surrounding UHDTV is no longer “how,” so much as “why” and “when.” From my perspective, until there is critical mass of UHDTV sets installed in homes, content owners won’t spend the extra dollars to make UHDTV content available. Until the content is available, broadcasters won’t create UHDTV channels. And until a lot of compelling content is both available and affordable, consumers won’t pay extra for UHDTV sets or service.

If this circular “chicken and egg” dynamic sounds to you a lot like the uphill battle faced by HD technology fifteen years ago, or 3D TV two years ago, I think you’ve gotten my point: UHDTV is definitely coming, but not quickly.

HEVC: Half the Bandwidth

Posted on by | Leave a reply

What the world needs now is… a new video codec? Back in 2004, when most of us video geeks were trying to figure out how to make H.264 work in the real world, plans were already afoot to develop its replacement. In case you didn’t notice, a draft specification for H.265 — also known as HEVC, which stands for High Efficiency Video Coding — was released in July by the JCT-VC, a joint collaborative group co-sponsored by MPEG and the ITU. The main benefits of HEVC are as follows:

  1. Greater Efficiency. H.265 should require only half the data rate of H.264 to deliver equivalent perceived quality.
  2. Increased Spatial Resolution. Supported image sizes span from as small as QVGA (320×240) all the way up to 8K (7680×8320) for UHDTV.
  3. Improved noise level, color gamut, and dynamic range as compared to H.264.
  4. Improved methods for parallel processing.

In subjective testing, HEVC has delivered equivalent perceived image quality at half the data rate, when compared to H.264 (High Profile). This means that HEVC could enable web and mobile devices to consume one half their current network bandwidth when streaming video. Or this increase in efficiency could be used to significantly improve image quality at the same data rate. Either way, H.265 will be a boon to web and mobile delivery.

Comparison: H.265 and H.264
(Source: Qualcomm at 2012 Mobile World Congress)

How does HEVC accomplish this drastic increase in efficiency? Although based on H.264, HEVC offers many small tweaks that add up, including the following:

  • HEVC replaces macroblocks with a more efficient (but also complex) hierarchical system for partitioning frames.
  • HEVC provides larger block sizes for higher coding efficiency.
  • HEVC supports tiling, allows multiple encoder instances to work on the same frame simultaneously.
  • HEVC supports wavefront parallel processing, so multiple threads can process different slices of frames more efficiently.
  • HEVC is progressive-scan only, simplifying decoder implementations.
  • HEVC includes entropy coding algorithmic enhancements that enable hardware decoders to run more efficiently.
  • HEVC includes higher precision filtering for improved motion compensation.

HEVC is expected to be fully ratified and published by early 2013. And, although chip vendors may not finalize hardware implementations until the standard is fully baked, software implementations will likely see commercialization in 2013. At IBC earlier this month, several vendors publicly demonstrated HEVC prototypes and announced HEVC support in future products:

What’s the catch? Licensing is still an unknown at this point. MPEG-LA has recently issued a call for patents that read on HEVC, so there is a possibility that H.265 will require a licensing fee. While MPEG-LA has not charged a license for H.264 when used to distribute video for free on the web, other uses of H.264 have incurred licensing costs. In any case, it seems likely that most H.265 implementations will require licensing fees.

There’s never a good time to introduce a new video codec, but it’s easy to argue that HEVC comes at just the right time. Video distribution now consumes the vast majority of all network traffic. Video-capable smart phones and tablets are ubiquitous. Unlimited mobile data plans are a thing of the past. Here’s hoping that licensing hurdles do not prevent the adoption of a new technology whose time has come!

Best Feature of iOS6

Posted on by | Leave a reply

Apple’s free update (iOS 6) has several new features, including a new Maps app that actually offers turn-by-turn navigation. But to me, the new Do Not Disturb feature is reason enough to upgrade.

Do Not Disturb Settings
(iOS 6)

Like most people, I have a love/hate relationships with my smartphone. It’s incredibly versatile and helpful, but it also makes me accessible all the time, even when I don’t want to be. International travel is especially challenging with notifications chiming away at all hours of the night. Sometimes I have turned on Airplane Mode to get a little peace and quiet, but that means I would miss an emergency call. And sometimes I would forget to turn off Airplane Mode and miss important notifications for a few hours. When I’m at home, I usually leave my iPhone downstairs at night, so it can happily ping away all night without disturbing anyone (other than the dog).

Now, with iOS 6, it’s easy to disconnect from the endless pinging of notifications. With the new feature Do Not Disturb, you can choose to silence the less urgent notifications – either manually or for scheduled intervals. I easily configured my phone to stop chirping between 10 PM and 7 AM. But I still get phone calls from my favorites, in case of emergencies.

Although it’s a great feature as is, I can think of a few things that would make it even better. For example, location-based settings would make the phone behave differently when I’m at home or away from home. And calendar integration would make it possible for the phone to automatically stop chirping during important meetings. But these enhancements might over-complicate what is an elegant and helpful feature in its current form.

Finally, with iOS 6 and Do Not Disturb I can now stand to keep my iPhone on my nightstand, instead of banishing it to the downstairs for the night. Who knows, maybe my dog will sleep better too!