3D Just the Way You Like It
Umezu-san, did you think "Uh-oh…" because you hadn't expected to use a stereoscopic LCD?
Right. I knew about liquid crystal for 3D, of course, but I hadn't seen the test with Luigi's Mansion, so I had no idea what it would be like to make a game in 3D. When Konno-san suddenly told me that, I thought it might have just been a momentary impulse. (laughs)
(laughs)
To be honest, at first I thought, "What a problem!" It had, however, turned out really well when we decided on two screens partway through development of the Nintendo DS system.
So you thought there might be a point in trying?
Yes. And I thought, "At least it's better than two screens!" (laughs)
(laughs)
I had, after all, been able to adapt to two screens at the last minute.
I suppose an upending of the tea table at the last minute can make you hold yourself together. (laughs) But it can't have been easy to switch to 3D.
No. Because stereoscopic display means that you have to simultaneously display two images on one screen.
While only one physical screen exists, you have to make separate images for the left and right eyes, so it's like having to display two screens.
And if you add in the lower screen, that makes three.
I had to render the upper, 3D screen at twice the usual speed, which meant I needed to rethink the SoC I'd been designing.
Which brings us back to how you had designed the first SoC with some leeway in power.
Yes. Making it 3D increased the power needed, and I ended up using all the extra power.
Programmable shaders19 for showing shadows in 3D graphics have come into general use, but for Nintendo 3DS we adopted a technology from Digital Media Professionals (DMP).20 Could you explain that?
19. Programmable shader: A shader is a software tool mainly for representing light and shadow in graphics. A programmable shader allows programmers to determine how they would like the effects of light and shadow to be calculated. The high degree of freedom makes a wider variety of effects possible.
20. Digital Media Professionals Inc.: A company that mainly develops and sells graphics processors. Headquarters: Mitaka, Tokyo.
What eats more electricity than anything in a handheld gaming device is the LCD backlight.
Nintendo 3DS has two of those—and the upper one is for 3D.
Right. And the 3D one has to delivery separate images to the left and right eyes. Which means that in 3D the amount of light delivered to each eye is halved. In order to make it look just as bright as usual, you have to increase the brightness of the backlight, which increases the power used by even more.
Choosing 3D led to the need to double the graphics and increase the brightness of the backlight, so there must have been a lot of challenges with regard to power.
I knew from the very start that the LCDs would require lots of power, so I thought we couldn't use that much for graphics. I spent quite a lot of time looking into programmable shading, but when you use that method for a handheld game system, you also have software processing in the SoC, and a number of steps become necessary for calculating the shading to render one point on the screen, so it can't keep up if the operating frequency21isn't fast enough.
In other words, that method eats up too much power.
Yes. We knew that DMP's technology had the merit of calculating with the hardware, so we could cut down on the power used. 21. Operating frequency: Also known as clock rate. It signifies the rate for synchronizing the parts of a circuit.
DMP's technology achieves with the hardware most of the shadow calculation usually done by a programmable shader, so even a slower operating frequency can achieve the same results. With today's semiconductor technology, the frequency has a direct influence on power use, so it's very beneficial in that regard.
Yes. I thought DMP's technology was best for saving power, given the limits of a handheld gaming system. What's more, to save power just a little more, we prepared a power save mode for the backlight.
Could you tell us what that does?
As mentioned earlier, what uses the most power in a handheld gaming device is the LCD backlight, and making one screen 3D presented the problem of even more power used by the backlight. The power save mode uses a technology called active backlight. It precisely controls the brightness of the backlight according to the brightness of the screen being displayed. When the screen as a whole is dark, the backlight itself gets darker, which saves power.
In principle, it may not be much use in a game with lots of bright screens, but if you set the backlight to bright, the more darker screens there are in a game, the greater the effect of the power save mode.
Right. But you can't say exactly how much of an effect the power save mode is having on game processing because of the various factors that influence power use, such as how constantly the SoC is running, whether you're using the camera or wireless, and how loud the volume is set.
Maybe you can't be exact, but how about an approximation?
We announced the other day that the battery duration for playing Nintendo 3DS games on it is about three to five hours. When I measured it by playing several Nintendo games, with the backlight set to the brightest level and the power save mode turned off, battery duration was about three hours. But if you use the power save mode under the same conditions, it gets about 10-20% longer. And if you set the backlight to the darkest setting, the battery lasts five hours, but the power save mode makes less of a difference then.
In other words, the brightness of the backlight has the greatest effect on battery duration.
Right. What's more, with the backlight set as high as it will go, battery life changes about 25% according to whether you're playing in 2D or 3D.
Yes, 3D is a battle against power use. How about wireless use? Some players may be worried about that.
In StreetPass mode, the system isn't always communicating, so there isn't much of a drain on the battery, but games that communicate a lot through local play and online play—with the backlight set to high—have an over 10% effect on the battery.
So it will be necessary for this system to be charged more often than for the previous systems. For that reason, we include a special charging cradle. We recommend that when players get home, they put their system there.
Another challenge we faced this time, after adopting the 3D LCD, is how it will look different to different people. The space between people's eyes differs, among other individual differences.
Yes. We solved that with the 3D Depth Slider, which allows you to adjust the 3D effect.
But, Sugino-san, as you were receiving all those requests to increase the number of switches, such as "we need to add 3D Depth Slider" and "there's got to be a mechanical switch to shut off the wireless," weren't you and the other designers worried that this would turn out to be a product that felt messy if you could not place all these switchers properly.
Yes, that's right. For example, about the 3D Depth Slider, at first, we thought that if we were going to put that, a +/- switch would look smart and serve just fine.
If you put in a physical mechanism for the slide, it takes up space. I suppose you wanted to avoid that since we were seeking a small, thin device.
Yes. But when I asked if a +/- switch would be fine, they said it had to be a slide that could be freely adjusted however the user wanted. Miyamoto-san and Konno-san were especially particular about that. I said, "But isn't the function the same?" Konno-san, however, made an actual prototype and showed it to me.
I wanted him to try it out, so in a relatively short period of time the tech staff put a slide on a Wii Remote. If you moved the slide halfway, the depth of 3D would change by half. I showed it to him and said, "See how great 3D is when you can adjust the degree of stereoscopy however you want?"
Did it immediately win him over?
Yeah. (laughs)
I thought it was definitely good.
With a slide, you know just where it's at. It's incredibly important to be able to go straight to where you want.
That's right. You can't do that with a +/- switch. That analog aspect is great, so I knew we needed it.
In this project, Konno-san would come to me with a new suggestion and something that actually worked—like for the 3D screen and 3D Depth Slider—and say, "I made this. What do you think?" Actually trying something out, rather than just hearing an explanation, is much more convincing. Then I'd be like, "All right, let's do it!"
So for the 3D screen and other elements, you persuaded him by making a test model and having him try it out. (laughs)
Yeah. (laughs)
Then, after you decided to put on the 3D Depth Slider, the idea came up of putting on two cameras. How did you decide on that?
A 3D camera was indispensable.
It was just a matter of course?
Yeah. I thought, "If we can display in 3D, then we can add a 3D camera."
It was a fast decision.
Of course, the Nintendo DSi system has a camera, too, but taking 3D photos is a different kind of enjoyment, so I wanted to find some way to do it. Unlike the time we made Mario Kart Wii run in 3D, however, it took quite awhile before we could take stereoscopic photos and display them on the screen. We tried a number of approaches, and when we finally took something that looked 3D, I was really happy.
What's fun about 3D photos is how the people in them make all kinds of poses.
Yeah. When I turned the newly finished 3D camera on Miyamoto-san, he made a boxing pose and was like, "Whoa, it really sticks out!" (laughs)
(laughs)