查看: 2414|回复: 9

[Anark] Using Images Responsibly

Asen

867 主题	0 听众	1万积分

外协人员

Rank: 7 Rank: 7 Rank: 7

纳金币: 17488
精华: 1

电梯直达

楼主

发表于 2011-9-29 08:52:43 |只看该作者 |倒序浏览

Summary
A primer on the use of images in realtime 3d graphics, especially with respect to Anark Media. It is split into 3 sections. The first section is an introduction to basic image properties, the second section is a description of the pipeline between an image editor and realtime 3d output, and the third section is some hard facts/data/numbers on images.


         Category
         Tutorials


         Author
         Harold Hausman


         Difficulty
         Intermediate


         Time to Complete
         15 minutes


Section one: A Basic Primer
Images have a few basic properties. (Many of you may know all this already.)

Resolution - This is usually given as two numbers. (e.g. 640x480 or 1600x1200 or 512x512 or 32x32 etc.) This tells you the dimensions of the image. The first number is how many columns of pixel data there are, and the second number is the number of rows of pixel data. The higher the numbers the more space there is to have definition in your image, but this extra space comes at a cost, increased file size & memory usage.
Bit depth - This is usually given as a single number. (e.g. 32bit or 24bit or 16bpp etc.) This tells you how many different colors can be in the image. Higher bit depths provide better color but take up more space. Lower bit depths can't represent the same number of colors, but are smaller and much faster for the computer to work with.
Format - This can often be deduced from the file extension (e.g. .jpg or .png or .psd or .tif etc.) Different formats are suitable for different things. Jpegs (.jpg) are good because they are highly compressed and therefore take up very little hard disk space when compared with uncompressed formats. Though they still represent 24 bits per pixel of color data so they work well all types of images. Png files are also highly compressed but tend to be a bit bigger than Jpegs because they represent a full 32 bits per pixel. (they store 8 bits per pixel for transparency data)

This information is very basic, and if you are interested in learning more on this topic use your favorite search engine to look up terms like "image formats" or "jpg vs png". None the less it's fairly important that you understand these concepts thoroughly, because choices you make here will drastically affect the appearance and performance of your realtime 3d application. Our goal is of course to make beautiful and speedy apps.
Section two: An image's lifecycle
There are abundant sources of image data. Digital cameras, stock photos, screenshots, texture libraries, scanners, digital painting/photo software and many other sources are creating tons and tons of image data all the time. These images are all of varying formats, bit-depths, and resolutions. Unsurprisingly, not all image files are created equal and not all images are suitable for realtime 3d.

Image data source and Anark Studio import
To begin with, your image must be in the right format. Anark Studio imports the vast majority of image formats so you're not limited much there. (See section three for a full list as of the time of writing.) Your main consideration with file format is going to be wether or not your image has any opacity information in it. If part of your image needs to be transparent you need to use a format which supports that, so look into using png or psd files. If your image has no transparency data associated with it there is no reason not to use a jpg. Its small file sizes wont strain your hard drive and it has relatively few options when you're creating it. Just crank the quality slider all the way up. Less choices always means less opportunity for error.
Anark Studio also makes importing images of different bit depths completely transparent to the user. The thing you need to keep in mind here is that images with a high bit depth are going to be big on your hard drive. So if there are space limitations on your image library remember to use as highly compressed images as possible. As we'll see later, all images are equal in bit depth by the time they make it to your screen so this area is of slightly less importance.
By far the most important aspect of an image in the realm of realtime 3d is its resolution. This is the fulcrum of the appearance/performance scale, arguably the primary factor which will make or break your application. I cannot stress the importance of choosing appropriately resolute images. Luckily this is pretty easy to do, and armed with the information in this paper you will make better applications. When choosing image resolution you are basically weighing, per image, visual appearance and memory usage. If there was no limit on memory usage (video memory) we could just merrily go on our way and use all 2048x2048 images and have photo-realistic realtime apps. Unfortunately this is not the case. Each image you add to your presentation needs to make its way to the video card's memory if it's going to be rendered (seen) and that pool is far from unlimited.
Anark Studio controls
Once an image is imported into Anark Studio several controls are provided to affect certain properties of that image. Keep in mind that, as with all things digital, it's best to have the highest quality source material and affect it as little as possible. As is often the case with digital processing some operations can have lossy, destructive affects on your data. Process your image as much as you can/need to before bringing it into Anark Studio. These tools are not a substitute for good data.
When an image is selected the follow parameters are editable in the inspector palette:

Image Size Reduction - This slider allows you to lower the resolution of your image on the fly. It is good practice to have your image appropriately scaled before ever bringing it into Anark Studio, so rarely do I ever have this set to anything besides 100%. That's not to say there aren't edge cases and final polish type situations where you would adjust this slider. Keep in mind, and I'll probably mention this in other places (and I say it more than once because it's really important) that all images on the video card (e.g. in video memory) have dimensions that are powers of 2.

Compression Type - This drop down allows you to choose between jpg and lossless compression. This refers to the image that will be saved into your .am file. Image compression at this stage affects only two things, the file size of your .am file and the visual quality of your image, regardless of the settings you make here or on the next slider; your runtime memory usage will not be affected. Lossless images have the best appearance in your application but take up the most space in your .am file. If you are delivering your application on the web large file sizes can be a problem as your users have to download the file before they can see it and if it's big that might take a while. If, on the other hand, you're delivering on a cd-rom or over an intranet file sizes might be less of a concern for you.

Compression Quality - If you've chosen jpeg in the above drop down you can set the quality of the compression with this slider. Again, this only affects visual quality and .am file size, nothing you do to this slider will affect how much memory you are using. As a general rule of thumb I set this to 100 for any textures I'm going to see up close and around 75 for anything that the camera is not going to get that close too. If the camera gets real close to an image that is jpeg compressed artifacts can be seen which are very distracting and can detract from the visual quality and perceived value of your application.

Images on the client side
What does all this mean? Where's the bottom line? How do these images finally end up on the screen?
It's an orderly process, observe:

The image stored in the .am file is compressed, this keeps the file size down. As the image comes out of the .am file it has to be decompressed.
The video card only knows how to deal with one image format. That format has the following properties:

      The dimensions have to be powers of 2 (32, 64, 128, 256, 512, 1024, or 2048). The x and y dimensions can be different, but they both must be powers of two. At this point, if your image has one or both dimensions not set to powers of two it is resized. It it always scaled up, as scaling down would destroy information. When you scale an image up a process called interpolation is performed and new image data is "made up" to fill in the gaps between the pixels that are "actually there" but get spread apart due to the up sizing. For specific examples of what this can mean for your images see section three.
      The bit depth is 32 bits per pixel. Every pixel has 8 bits for shades of red, 8 bits for shades of green, 8 bits for shades of blue and 8 shades of alpha (i.e. transparency or opacity)
      This format is totally uncompressed. All this data is stored in very fast video memory. When the camera sees a material with an image on it needs to access this information at lightning speeds so it can know what to draw.
      This can sound very limiting, but it's not all bad; there is one very important upshot which stems from this concept: The video memory used by any image is easily calculable. See section 3 for the math and resultant numbers.

Now, at this point your images are automatically converted to that format.
Once your images are converted and pushed down onto the video card they are ready to be rendered.

That's how it goes, and all this happens with a quickness. There are a few pitfalls to watch out for though which may or may not be obvious in light of this information. If you have lots of big images you're going to run out of video memory. Running out of video memory is a Bad Thing�. Also, resizing images takes time. There is a lot of math involved. If you have a bunch of images that need to be resized this math can be "felt" by users of your application. This can manifest itself as long loading times or hiccups/drops in frame rate while using the app.
In the next section we will see some hard numbers which should help you make good decisions about images.

Section three: Facts
You want numbers? We've got numbers.

Image dimensions and powers of two
I mentioned earlier briefly the concept of image dimensions having to be powers of two. What does that mean? As you know, you can import images of any size into Anark Studio, but Anark Studio and your video card have made an agreement on a single image format that they are going to use. (its properties are listed up in section two) A key property of that image format is that all the image dimensions are going to be powers of two. The relevant powers are two are listed here:


         32
         64
         128
         256
         512
         1024
         2048


Numbers lower than 32 are not listed because the smallest images inside Anark are 32x32, anything smaller than that is scaled up to that point. Numbers over 2048 are not listed because images that are bigger than that are not suitable for realtime 3d graphics. (even 2048x2048 is pushing it and images of that size should be saved for special applications/effects)
I think examples are going to be the best way to get the point across. If you have an image that is 512x512 (or any other square power of two, i.e. 64x64, 128x128, or 256x256) Anark will sing your praises. These are best case scenarios, no time will be wasted resizing the image so that the video card can understand it, it's already ready to go. On the other hand, if your image is 1030x1200 (which is just a hair bigger than 1024x1024) you've got a nightmare situation on you hands. In order for Anark to convert that image into the format that your video card can understand it needs to scale its dimensions to powers of two. You cannot scale down because you lose pixels that way (if the scaled image has less pixels than the original image you've lost some pixels) and your pixels are valuable, far be it from Anark to want to lose them. So instead it scales up. In fact it scales that image up to 2048x2048. This has several detrimental effects. First, there are a lot of pixels in that image that were created by interpolation. Interpolation algorithms tend to err on the side of caution, so this is probably going to be a pretty blurry image. What's worse is this image is now taking 16 megabytes of video ram!
Those describe the best and worst cases, in this last example I'll talk about a more common case and its repercussions. A lot of digital cameras take pictures at 640x480. In order to get such an image onto a video card its dimensions would be scaled to 1024x512. (dimensions don't have to be square, but they do have to be powers of two) This image has been scaled up so it contains a fair amount of interpolated data which may make it appear fuzzy, and a little math tells us that it is taking 2mb of video ram. This may not sound like a lot, but in a 100 photo gallery type application this can add up fast. What I would do in a situation like this is to take the images into some image editing software and scale them down to 512x384. Doing this will set them up so that by the time they make it to the video card they are 512x512 and have the least amount of interpolated data possible and they are only taking up 1mb of video ram per image. (That's half as much!)

Calculating image memory usage
Now would be a good time to start thinking about your target platform, assuming you haven't already. Specifically, how much video ram does the (weakest) machine that needs to run your application have? Is it a blazing fast monster of computation sporting the top of the line video hardware? (at the time of writing 512mb is the most video memory you can reasonably expect a machine to have, with most hardcore machines only having 256mb) Or is it some remedial laptop that your client's boss has been using since before the turn of the millennium which may only have 32mb, not to mention the video memory is shared with the system ram and performs like crap. Remember, everything that the user sees, everything, passes through the video card. Vertex buffers, face lists, material settings, and other things are all fighting for space on that video card, but by far the majority of your video memory is used by image data. (textures)
Using some simple math you can make fairly accurate guesses at how much video memory you're using, and running out of video memory can cause some of the nastiest errors in all of 3d graphics; so lets learn.
Your number of pixels = X resolution * Y resolution

If your image is 32x32 it has 1024 pixels. (32*32=1024) If your image is 64x512 it has 32768 pixels. (64*512=32768)
32 bits * Your number of pixels = Image memory usage

Because, we know that we've got 32 bits per pixel.
Here's a table that shows video memory usage for different image sizes. (Keep in mind that there are 8 bits in a byte.)



         Image Size


         Memory Usage



         32x32
         4096 bytes (4kb)


         64x64
         16384 bytes (16kb)


         128x128
         65536 bytes (64kb)


         256x256
         262144 bytes (256kb)


         512x512
         1048576 bytes (1Mb)


         1024x1024
         4194304 bytes (4Mb)


         2048x2048
         16777216 bytes (16Mb!)


What this means is that on that monster machine with 256Mb of video ram you can have basically as many 512x512 images as you want, but on your client's boss's laptop with 32Mb, if you have 2 images bigger than 1024x1024 (remember image dimensions are always scaled up to the next power of 2) you're going to fill up the video memory, which we've already established is a Bad Thing� to do. Knowing your target machine is key here. You'll be amazed how much better you sleep knowing that your application is singing on the client's machines, trust me.

Image format import options
Anark has comprehensive image import capabilities, I'll discuss them here:

JPEG - This is my image file format of choice. I've extolled the benefits of it quite a bit in this document, but I'll mention something else here. Rendering transparent images is a lot harder (read: slower) than rendering ones that are opaque. Since jpegs have no opacity channel data there's no chance of your image being mistaken for a transparent image. If you have accidentally one transparent pixel in your png or psd file the material/model with that image on it is going to render measurably slower than the same image with no transparent pixels.
PNG - If I have an image that's transparent this is the format I'll use. This is the smallest format that supports transparency. Taking this into consideration it's clear when using this format is appropriate.
PSD - Anark Studio supports Adobe's psd format. This is excellent news if you're a photoshop user as you can leave your documents in a very complicated state as far as layers and such are concerned and use them directly in your realtime 3d app. Naturally, this power comes at a cost. Psd files are huge, so if disk space is at a premium watch the sizes of these files.
TGA, TIF, BMP - Anark Studio also supports Targa, Tiff, and Windows Bitmap. This can be convenient, but as we've learned today it's important to know what's going on with your images, and to be precise. For real heavy-duty work in realtime 3d these formats are less common due to their inefficiencies.

In conclusion,
It's not as hard as it seems. These are fairly simple problems, with obvious symptoms and easy solutions. If you application is having trouble running on a certain machine find out as much as you can about that machine. Often, problems with frame rate or even instability can be solved by using more appropriate images. The old rule of, "your application is only as good as the data you put into it" definitely applies here. Hopefully this paper has given you some ideas of things to look out for and introduced you to some concepts that will help you make better 3d applications.