ExtremeTech: Maybe each of you could quickly go over what your role is in the development and implementation of DirectX 10.
David Blythe: My job is basically to spend a lot of time talking to game developers and to hardware developers and to talk about what problems we're trying to solve and what's practical to do in hardware, and then hammer out an architecture or design that encompasses as many of those things as we can in a coherent way. Then it turns into a much more detailed discussion about every little itty-bitty detail and writing specifications and then working with Microsoft's development team that builds the run-time that interfaces with the driver the IHVs write. And working with our SDK team to build samples and demos that show off how to use the system.
Chris Donahue: David mentioned working with the IHVs and ISVs…there's a graphics advisory board that's made up of the big graphics brains around the industry. David and a lot of the guys on the DirectX team and the Graphics Platform Unit group spend time talking to, to find out, you know, five years from now, where are things going to be? And that stuff they take and turn into actual API code and ends up not only in the API but also in hardware. It's a really collaborate effort between Microsoft, the ISVs that actually consume the API and make applications, and the hardware guys.
ExtremeTech: DirectX is of course more than just graphics. There's DirectSound, DirectInput, and so on. Will DirectX 10 change any of those things to a significant degree?
Donahue: Not to DirectSound or DirectInput really. We actually have done a few things. If you look at some of the recent things we've talked about, for example the common controller. The Xbox 360 controller and all the peripherals work on Windows. To enable that, we've taken XInput, which is the API they use for Xbox, and converted that, or modded that to make it work on Windows. So now XInput is actually the preferred API if you wanted to take advantage of all the controls and components involved in the peripherals.
As far as DSound, yeah, we're updating it to keep it sort of on track, but it's not continuing to invest in. There's other audio components and low-level and high-level stuff that's in the works, but it's not anything that we have an announcement about.
ExtremeTech: What are the big bullet points—the main, big changes in DirectX 10 graphics over what we have in DX9?
Blythe: That's actually a fairly complicated question in the sense that there's a lot that has changed. I want to be careful not to point out anything as, "oh, this is the defining feature." Some aspects of it are to do with the implementation that, one of the key things that we were trying to solve with DirectX 10 was to make it more efficient. And so there's some structural things in the way the API works where what we tried to improve is what people have commonly referred to as the "small batch problem." So that applications can send a lot more material changes or changes to the description of geometry a lot more frequently. But that doesn't show up at any "bullet point" feature in the API, like "oh turn on this switch and you get better performance." It sort of underpins the design.
And then there's a lot of things about feature consistency in hardware implementations that allow developers to have an easier job of targeting a wider variety of platforms. That doesn't show up as any single feature, but it's a big deal to get agreement on what this common set of features is and it will be available everywhere.
Then when you start getting into individual features, things like adding an integer instruction set, the new geometry shader is a fairly big deal. In terms of adding that in the middle of the pipeline and having access to all the parameters of a primitive—for example the three vertices of a triangle or the two endpoints of a line—and being able to operate on that as a whole, sort of changes the game. There are things that we've done where, part of the objective was to make it possible to do more processing on the GPU and to do this in a way where there wasn't a lot of mediation that needed to be done on the CPU. So you can do iterative types of computations on the GPU without having to sort of return data to the CPU, do a little bit more work there, and then return the data to the GPU. The way data formats work and can be moved back and forth between different parts of the pipeline show up as being important features for enabling this, but they're not…
ExtremeTech: You don't look at a screenshot and say "Wow, look at the interative instructions in that screenshot!"
Blythe: Right. And so other things we've done, like the geometry shader we've added, you can stream the output of that to memory, and then turn around and use that chunk of memory as a vertex buffer. It enables something like being able to implement particle systems a lot more easily than it had been to do on a system like D3D9. Continued...
ExtremeTech: Part of what you said about making the whole thing more efficient relies on the new driver model in Vista, is that right?
Blythe: That is correct.
ExtremeTech: A lot of people are complaining, "Oh, why won't we have DirectX 10 for Windows XP." There's a good technical explanation for that, where it's really not possible to do what DX10 does in the Windows XP driver model.
Blythe: Definitely in the long term there's a vision in how we think the applications and the runtime and the hardware need to work together. What we've done in Vista was to make some major changes to try to improve that. We've got a number of additional things we wanted to do over the next few years to try to make that even better, and it's just hard to say, "oh well we'll just retrofit all that into Windows XP." That kind of put us in this position of saying, do we really want to get all these big improvements, and what do we have to give up in order to do it?
ExtremeTech: So if the decision had been made, "Yes, we're going to try to make all this work on XP," you'd really have to sort of hamstring DirectX 10. You'd have to say, "Then we can't do this, we can't do that…"
Blythe: That's right. We would probably end up with something that, from an API point of view it might not be that different, but what we ultimately get for performance and things like that, you might not get nearly as dramatic an improvement as we're hoping to see over the next few years.
Donahue: There are a bunch of other benefits to the new driver model, as well. Not just the performance and graphics component. It all has to do with graphics, obviously. But the stability and other parts that the new driver model brings in are a big advantage.
ExtremeTech: Can you elaborate on that a little bit? That has to do with not as much of the driver running in kernel mode, right?
Blythe: One of our key reasons for doing the new driver model, and it wasn't necessarily just to improve the performance of DirectX 10, was to improve stability of the system. The way that came about was first by trying to do a better match between how the hardware worked and how the driver model thinks about hardware, because hardware has continued to evolve and there weren't as many opportunities to make wholesale changes to the driver model. Part of those changes included moving more of the processing out of the kernel and into User Mode, and that just reduces the surface area for bad things to happen inside the kernel. So if there happen to be bugs in the driver, it won't take down the entire system.
There are other changes we wanted to do; our vision was around having more and more applications trying to use the graphics hardware at the same time. So we wanted to be able to virtualize applications so that they could all seamlessly and equitably share the graphics hardware. So that's a lot like the evolution of the CPU with time-sharing systems, where you are able to run multiple applications at the same time and they time-slice on the CPU, and memory gets virtualized so that applications look like they have all the available memory resources. We wanted to do the same thing for the graphics side of an application, and so the driver model was designed to incorporate that kind of thinking.
ExtremeTech: Speaking to that point, we know that the desktop in Vista is drawn using DirectX 9. What happens if you have a DirectX 10 card? Does the desktop still use DX9, or does that switch over and use DX10?
Blythe: It continues to use DirectX 9. Largely the reason for that is when we built the desktop, that was being done concurrently with the design of DirectX 10. It becomes somewhat more complicated to build both the low-level technology and the thing on top of it, concurrently. It's better to sort of have a time gap between those. At the same time, we were making some minor tweaks to DirectX 9 to accommodate new features that were needed to do the desktop. For us, it's best to have one consistent platform. Even though we could imagine there being benefit to the desktop using DX10, it's better to do all the debugging and get it to work with DX9 and ship that. Then over time, as the hardware base builds up for DX10, by the time we do the next major release, we'd be looking at trying to move the entire desktop onto 10. Continued...
ExtremeTech: You spoke about Geometry Shaders briefly. Perhaps you could elaborate a bit on exactly what a Geometry Shader is. We sort of know what a Vertex Shader or Pixel Shader is, but many people say, "Hey what's a Geometry Shader?"
Blythe: The traditional model is: Vertices come in at the front of the pipeline and get transformed, then get re-grouped to produce triangles. Then the rasterization system goes in and converts the triangle into fragments and the individual fragments get processed by the Pixel Shader.
The Geometry Shader sits in-between those two pieces, and it has access to the vertices for one of those triangle primitives, right after it's been assembled, and can operate on all the vertices at once. It can do a couple different things. It can amplify the number of triangles, so it can take those vertices and generate a new set of triangles. Or it can just generate a new set of points, or a new set of lines, and send those to the rasterizer for generation of pixel fragments. We can do things like take a point and generate a set of triangles around that point and expand it into a sprite. Or you could decompose a triangle into a smaller set of triangles that you could then think of as tessellation. Or you could extrude the edges of the triangle and turn it into a volume or a tetrahedron.
One of the problems that we've often seen is being able to get enough data from the CPU into the graphics processor, and so we can generate data internally in the graphics processor, and we've sort of eliminated this transfer bottleneck.
ExtremeTech: You could even see the graphics card having a big hand in doing some of the stuff that was traditionally done on the CPU. Things like collision detection, or calculating obscured geometry so you don't have to render it. You start to see a lot of flexibility in how developers can use both the geometry shader and the stream-out-to-memory function together.
Blythe: That's what makes the system really interesting. It isn't a point feature that says, "We just gave you this new feature that draws this new rendering effect." No, you have a whole bunch of new flexibility where you can do a ton of different things that you may not have thought of doing before, that you would have done on the CPU, but the amount of transfer back and forth between the CPU and the graphics processor would have been too much to make it really useful in a game application.
ExtremeTech: A lot of people are wondering how DX10 compares with what we're seeing on the next-gen consoles. Certainly the PlayStation 3 and Xbox 360 have sort of DirectX 9–class graphics hardware. In the case of the PS3, it's a really clear lineage from the GeForce 7 GPUs. It's a little different on the Xbox 360; it seems to have some of what's in DX10 in there. Could you clarify for people what DX10 is going to enable on the PC compared with what we see in the next-generation console?
Blythe: That's kind of a tricky question in the sense of saying "oh well it will enable any particular kind of effect." The things that we had mentioned in terms of being able to do much more data generation inside the pipeline is something that can't be done well [on next-gen consoles], even though the consoles sort of have their unique capabilities. And then there are things like the integer instruction set that neither of the two console [graphics] processors have available. They can try to fake it with floating-point operations, but it becomes really complicated, and at some point people just say it's too much work. Having a real integer instruction set makes it possible to do much more interesting address calculations, to do bit packing and unpacking. These things are not features in themselves where it'll show up as some rendering effect. It allows people to do more clever things inside the [graphics] processor.
Donahue: Also, I think Crysis recently proved that you can do some pretty amazing things on DirectX 9.
ExtremeTech: Right. You even see online, some people mislabeling the screenshots, "Oh, look at DirectX 10." Because everyone knows they intend to do DX10 stuff in Crysis. But as far as I've seen, those have all been DX9 screenshots, and they just look so incredible that people assume that it's gotta be DX10.
Donahue: Yep. So that's one of the challenges the Crytek guys have. Is to take the things that they have now that are running—and those are screenshots, they're not artist's renderings, y'know—and do something significantly step-up with DirectX 10. But we work really closely with them and some of the other guys that are doing DX10 versions of their titles. As David mentioned, the ability to generate more stuff, more clutter, more vegetation, more characters, more particles, more chunks. I think those are some of the things you're going to see. And you think about the way that the PC platform is changing with physics chips and some of the other things, the level of complexity required for physics not just to move objects better, but to have more objects to move. I think DirectX 10 is a better fit for that as well. Continued...
ExtremeTech: One of the things coming to be a fight in the PC world is separate physics vs. GPU-accelerated physics. Is DirectX 10 better suited to having physics computed on the GPU than DX9 is?
Blythe: Well, it's a more general-purpose processor. To a large extent, that's been our motivation, to generalize the capability. To the extent that physics can take advantage of this more generalized capability, it's a better match for it. And that extends to doing other kinds of game processing that doesn't have to be limited just to physics, but that's the one that's got everybody's attention right now.
Donahue: AI, and you know, any other sub-calculation that goes on, it's a great general-purpose processor, like David said.
ExtremeTech: Video is another area where you're starting to see the graphics card manufacturers doing a lot of fun stuff with their video processing using the power of the GPU. And you could see DX10, especially with the reduced overhead, enabling more powerful video processing on the graphics unit.
Blythe: Oh certainly. It's an area where our hardware developer partners continue to do a lot of innovation and there's a big market for being able to do video encode, and an increasing market for video transcode. There's a lot more interesting things that can be done, particularly as we generalize the processor and make it possible to do more interesting kinds of bit-twiddling operations or also being able to do more sophisticated image-processing operations. The kinds of changes that we've made to the processor, through the API, really do enable a bunch of things there.
ExtremeTech: Maybe you could bring it up a level, and discuss some of the things you've seen people working on. Obviously you can't name too many names or make announcements for other companies here, but describing some of the effects, some of the differences you've seen in titles as they work on DirectX 10 hardware. Like, if I'm looking at a DX10 game, what am I going to see that is going to be substantially different from what I see in a DX9 game?
Blythe: Chris started to talk about these things already. The biggest thing you'll probably see is greater complexity and more detail in the game, and that can take several forms. Better foliage, better grass, more rocks, more trees. That may seem like a minor thing, but being able to generate a lot more data makes the scene a lot richer. The tricky part is making those elements look different. And that's the place where being able to generate data inside the [graphics] processor and then be able to add specific shading parameters or even change the geometry on a per-instance basis, it really adds to the level of complexity in the scene.
Then to build richer characters, we've looked at techniques like displacement mapping to add additional silhouette complexity along characters or be able to have more geometry in the characters. And then being able to do different kinds of illumination effects, like doing a better job with shadows. We're looking at better soft-shadow techniques that can be enabled using the geometry shader. The support that we've put in for HDR formats to more compactly represent HDR data, make it possible to use HDR more efficiently. Continued...
ExtremeTech: The processing power of the GPU is skyrocketing, but memory bandwidth is not. It's creeping up and up but it's not doubling and tripling as fast as the power of the graphics processor. As you worked with IHVs to determine what DirectX 10 is going to do, was more efficient use of memory bandwidth a big part of it?
Blythe: It certainly has come up as a bigger and bigger issue, particularly, as you mention, with more game applications taking up high dynamic range lighting and greatly increasing the amount of texture data. We've also seen the move to higher resolution screens and that puts more pressure to have that kind of detail in the content that's being generated for the game and having an efficient way to do it. So we've increased the maximum texture dimensions in DX10. They were 2048x2048 or 4096x4096 in DirectX 9, and in DX10 they're 8192x8192. The way we see some of the things playing out is that if there's a consistent feature set across the range of product lines, from enthusiast parts down to value parts, the game developers will do more scaling on performance [rather than features] and just change the resolution. But they'll provide all these high-resolution assets with the game when it's distributed, and then choose the right resolution to render at for different versions of platforms that the end user has.
ExtremeTech: Consistent feature set is an issue that is not brought up much but seems to be a pretty big deal. DirectX 9 was kind of a mess in that you had Pixel Shader 2.0 and then Pixel Shader 3.0, support for different floating-point formats and so on. It's a pain for developers and it's a pain for customers, because they don't know what they're buying when they buy a "DirectX 9 graphics card." But the hardware vendors like it, because they like to differentiate. They like to add features. Where do you strike that balance in DirectX 10, between the hardware vendors being able to add features or add value, and the developers and customers getting a unified feature set?
Blythe: We've definitely sort of leaned toward the guidance we got from the game developer community in terms of really wanting a consistent feature set. So the way things have ended up in DirectX 10 is that there are a very small number of optional things. In fact I can count them on part of one hand: The amount of multisample antialiasing supported is variable. Whether they can support blending of 32-bit floating-point render targets or color buffers is optional, but all the other formats aren't. What these amount to are, the most expensive features are the ones that end up being optional. And there's a similar one for being able to texture filter 32-bit texture formats. The big deal is, doing 32-bit floating-point processing inside the texture processing unit or the blend unit is a relatively large hardware cost. So not every hardware vendor wanted to build that.
So it's those three things. And of those things, multisampling is the one thing we'd be most concerned about in the future, having that be more consistently available, even though we would probably try to lock in all three of those things in the future. It just turns out for this generation that being able to do 32-bit blending or texture filtering just wasn't as high on the list of priorities for game developers, compared to consistency amongst all these other features that we had done.
ExtremeTech: So is it fair to say that DX10 eliminates all those capabilities bits, those "cap bits," and now you just have these three things?
Blythe: That's correct. And to a large extent, they don't show up in the traditional "caps bit" mechanism. They show up as properties on some formats.
ExtremeTech: And when there are only three of them, it's not something that's really hard for developers to worry about.
Blythe: I don't want to make any promises that it will just stay at three forever. We expect that, over time, there will be newer versions of DirectX and things will come in big blocks of features. Continued...
ExtremeTech: That leads to my next question. We saw several iterations of DirectX 9 that were still "DX9," but had different capabilities. Are we going to see that happen with DirectX 10, or is there going to be a clear delineation where you'll say, "There won't be any major changes until DirectX 11." The hardware guys, every year when they come out with a new chip they want some new features in DirectX, so they can say "We're the first ones to support it."
Blythe: Generally, we try to be responsive to the development community, and that means both the software developers and the hardware developers. So if there's enabling that they need to have, we want to get that in and get that in as quickly as we can. I'm not making any product announcements or talking about schedules or anything, but the way we've been thinking about it is alternating major and minor changes to the API. So now that we have the major changes out, what did we learn about it and what are the tweaks or small things we could do. You could think of them as oversights, or late-breaking new thinking that came along that we could accommodate in a minor revision to the API.
ExtremeTech: Is there a high-level roadmap for the successor the DirectX 10 yet? I mean in the timeframe of DirectX 11, looking five years down the road.
Blythe: We certainly have a lot of thinking invested in it. How solid things are sort of varies with how far out we're thinking.
ExtremeTech: Knowing that it's not solid at all, could you run down what some of the "big picture" ideas are you have for that timeframe?
Blythe: There are a couple problems that we're worried about. One is: There's this move to build richer and more complex content in the games, and so we're really worried about how much work it is for the end developer to achieve that. How many hours do artists have to spend to produce this much clutter in the scene? And how do I store those assets in memory—if they're higher resolution, do they just take up more and more space? Does it just keep going up at the square of the resolution? So that's moving us more and more towards some procedural techniques with the idea that artists spend more time "roughing it out" as it were. Building the form and the volume of the assets, and then rely on procedural techniques to fill in the detail.
And that fits in nicely with the problems of not having memory bandwidth not scaling up at the same rate as computation. So we'll transfer a lot of the complexity generation process into doing more and more computation inside the graphics processor. So the question becomes, "What are the forms of this computation?" And one way is improving the complexity of organic characters by having better silhouettes and better detail. We've seen the rise of normal maps for greatly improving the shading realism, but we'd like to see the actual geometric complexity of those objects improve. One of the ways of doing that is having more of a pipeline that can generate that complexity inside the graphics hardware. The way we've been thinking about that is by doing higher-order surfaces like Catmull-Clark surfaces and mixing that with displacement mapping. That's something we've been talking about for a long time.
ExtremeTech: So it's fair to say that, at a high level, the successor to DirectX 10…one of the main things you're looking at is solving the problem where we have content that is approaching the level of pre-rendered movies, but game developers don't have that much time and that much manpower to make all that stuff. So how do we make the hardware better deal with generating and storing content?
Blythe: That's right. It's all about making sure that we can get high-quality games out in a reasonable amount of time, and that seems to be the bottleneck that we're hearing the most about.
ExtremeTech: Having unified pixel and vertex (and geometry) shaders on the API level does not necessarily mean that, on a hardware level, we would need to see that. It would still be possible to have a DirectX 10 graphics card that has separate vertex and pixel shader units. Right?
Blythe: That's correct. That's what's in it for the hardware developer guys. It's an opportunity for them, that the specification doesn't dictate implementation. So if they can find a way, and it makes sense for their implementation and the trade-offs that they're making to keep them distinct, they'll do that. Or if they see an opportunity to unify them and feel that does a better job of load balancing vs. the cost of that generality, they can do that.
ExtremeTech: In your opinion, is that the more optimal way to go, to have a unified vertex and pixel shader from a hardware point of view?
Blythe: I think that's the trend, that's the direction we're going to. The 10,000-foot view is that the graphics processor is this bank of parallel processors that are really good at doing parallel programming on data-parallel kinds of applications. And that there's a layer on top that is all about graphics processing which sort of provides these logical abstractions of vertex shaders and pixel shaders and geometry shaders and anything else that we think of that makes sense. Continued...
ExtremeTech: What are some of the things that didn't make the cut for DX10? That early on you maybe wanted to do, but weren't able to get into the API?
Blythe: Well there's been this interest in being able to handle higher-order surfaces and subdivision surfaces for a long time, and we looked at that pretty early in DX10. We decided we just weren't there. We didn't have enough manpower resources to really solve the problem and do a good job of the other things we were trying to do. It wasn't clear that the hardware could really be there yet, either. There were other things along those lines, where we thought it was a good idea and after we did more investigation, we'd back off on it. Like having the front-end of the vertex gathering unit be more programmable, so you could walk multiple vertex arrays at the same time and do that in a programmable way. It turns out it just wasn't worth the trouble—the costs versus the benefit we would have seen.
ExtremeTech: With every new DirectX, it takes some time before the games really start to make use of it, and then even more time before they become a required least-common-denominator. Do you see that transition happening any faster with DirectX 10?
Donahue: Part of this is a business consideration as well. At the launch of Vista and DX10 hardware, there's fundamentally zero installed base, and it grows exponentially over time. If you look at history and how rapidly they evolved from having DX8 hardware move over to DX9 hardware, it was pretty quick. My guess is DX10 will be the same way and I think that you will see…I can't predict a good time-frame, but there will be titles in the not-too-distant future that are DirectX 10 only.
ExtremeTech: It didn't take long for games to take advantage of DirectX 9, but it took a really long time before any games required it. Do you think that's going to happen more quickly with DirectX 10?
Donahue: It's difficult to say if only because the consoles, as you mentioned before, Xbox 360 and PS3, are fundamentally DX9 parts. We're kind of back into a situation we were in before, where there's a lowest common denominator across hardware. If someone is going to do a game that is multiplatform and they want it to run on the consoles, they're going to have to do a DX9-style pipeline anyway. So from that perspective, there are companies that do just one generic cross-platform component and try to hit everything, then there's the guys that do the really exploitive type stuff.
I think you'll see DX10-only in the next two years, certainly. That's about as close as I can pin it, because part of it is going to depend on adoption and installed base of DirectX 10 hardware.
ExtremeTech: Have either of you seen DirectX 10 stuff up and running yet on real hardware yet? Or are you still stuck in the reference rasterizer, exploratory mode?
Donahue: Nice try, Jason! (laughs) Those are questions for the hardware companies, you know how that is.
Graphics Tools 
