[webraider]

I've noticed that the upcoming release of Mac OS X will have Core Image and Core Video. The purpose of this is to offload work to a compatible GPU right? If the GPU can't handle it then it's offloaded to the processor and is treated like Floating Point, or Veclicty Engine Calculations right? My question is will this make it possible for Apple to have Hardware encoding with it's current Line of GPU's, or for Legacy Macs to get better at encoding via software??? Does anyone know the answer to this question?

[SS3 GokouX]

No, you're confusing encoding and compositing. Core Image and Video are just compositing, by adding effects and filters to the video, and then the raw video is displayed immediately on your screen. To encode a video, your machine takes the raw video and applies numerous algorithms on it and then out to a file.

There used to be (and maybe still are) dedicated video encoding cards that you could install, and those would be used specifically for encoding video (typically mpeg-2). But those are probably useless nowadays since processor speeds are so much higher.

[webraider]

Originally posted by SS3 GokouX:
No, you're confusing encoding and compositing. Core Image and Video are just compositing, by adding effects and filters to the video, and then the raw video is displayed immediately on your screen. To encode a video, your machine takes the raw video and applies numerous algorithms on it and then out to a file.

There used to be (and maybe still are) dedicated video encoding cards that you could install, and those would be used specifically for encoding video (typically mpeg-2). But those are probably useless nowadays since processor speeds are so much higher.

This is the quote I was reading from the Apple Website regarding Core Video:

"Core Video, joining Core Image in Mac OS X Tiger, delivers a modern foundation for video services, providing a bridge between QuickTime and the GPU for hardware-accelerated video processing. This highly optimized pipeline for video presentation increases performance and reduces CPU load, freeing up resources for other operations.

And Core Video allows developers to apply all the benefits of Core Image to video — blazingly-fast performance of filters and effects, per-pixel accuracy and hardware scalability."

Now it's the paragraph in the quote above that has me interested... Why.. becauase Encoding and even decoding of High Definition video is Extremely Processor intensive. This is why Elgato's eye TV has built in Hardware Encoding. Yeah.. you can still buy hardware accelerated Graphics cards specifically for encoding. Steve Jobs has admitted how powerful GPU's have come but quite honestly, the Macintosh really doesn't take advantage of it as much as it should. If it courd off load encoding and decoding to the GPU, We would see huge performance benefits... even in a Powerful Dual G5. The reason I don't believe that I'm misuderstanding this is because of the second Paragraph of the above quote. "And Core Video allows... benefits of Core Image to Video...Filters, Effects, Per Pixel accuracy etc..."

This could very well be what Steve Jobs means when he says this will be the year of High Definition Video. If we can pull off hardware accelerated Encoding with just the built in GPU's of our graphics cards then this would be a huge accomplishment.

[SS3 GokouX]

Originally posted by webraider:
"This highly optimized pipeline for video presentation increases performance and reduces CPU load, freeing up resources for other operations.

Your video card is just that, for video presentation, it can't encode/decode video formats.

"And Core Video allows... benefits of Core Image to Video...Filters, Effects, Per Pixel accuracy etc..."

None of those have anything to do with encoding or decoding. Your processor would still have to take the raw video (with all of your core video effects/filters applied) and encode it into a format. Unless you want to use raw uncompressed HD video and fill up all of your remaining hard drive space.

[Laurence]

I can't find it now, but I could swear that I read somewhere (Toms Hardware?) that the newer NVidia and ATI cards had circuitry onboard for hardware based decoding AND encoding of MPEG 1, 2 & 4 video. The article said that applications just had to be written in such a way to access the features. There was also talk that the benefits would be minimal unless using a PCI-Express bus as with normal AGP, bandwidth is only 1x back from the card even if it is 8x to the card.

I realize that even if this is correct, Apple would have to actually support this functionality in core image/video, but it would make real-time encoding possible.

[Millennium]

CoreVideo and CoreImage don't actually accelerate the way that images are drawn. What they accelerate is filtering that video, usually to apply some sort of special effect like sepia-tones.

It is possible to accelerate playback of video using hardware. That technology has existed in Macs for a long time, predating OSX itself. It remains in OSX, but it is a much older technology, and isn't part of CoreImage or CoreVideo.

Hardware-based encoders and decoders for MPEG-1 and MPEG-2 have existed for many years. Hardware-based decoders for MPEG-4 are newer, but they're also out there. I don't know of any hardware-based encoders for MPEG-4, but they are certainly possible. However in no cases are these hardware-accelerated; they are cases where the actual chips have been made expressly for decoding or encoding MPEG video from a specific stream format.

This particular approach would not work well in QuickTime, which can take many stream formats as input. Particularly in the encoding case, most video would have to first be converted to the format which the card wants, and this cuts deeply into the benefits of hardware implementation. Also, graphics cards are optimized not to convert video, but simply to render it, and while there is some overlap in these areas, each requires things which the other does not. Coding around the limitations in the graphics cards -which are only limitations because this isn't what they were made for- would once again eat into the performance benefits.

If you want to speed up video encoding through hardware, you need hardware which does common encoding tasks. GPUs aren't made for that, and so there would be little to no benefit as far as that goes. A better solution would be to work with AltiVec, which was geared for exactly this sort of thing.