[Scrod]

Now, we all know how window scrolling in OS X doesn't yet have hooks to allow 2D hardware acceleration, correct? However, window dragging is accelerated. How difficult would it be to create a new scrolling class that essentially used its own borderless, shadowless window that was moved up and down above the portion of the app that would normally be scrolling? And how might the edges be obscured so that it didn't overlap onto the rest of the app? Is this possible? If so, how feasible might this be?

[AirSluf]

I think the concept itself isn't that difficult. Your description sounds similar in concept to the Java JScrollPane Class Viewport. But the advantages to gain acceleration I am not so sure about. Making the contents inside the window change requires an extra layer of processing over the whole unchanged window being dragged. If you don't have any dragged window transparency, you can cache the internal bits of the dragged window and just overwrite them wherever it is going. Now you only have to re-compute the vacated areas, vice vacated and new areas. Unchanged opaque window dragging would be faster even without any calls for hardware acceleration.

The accelerating effect of not having to re-compute both is illustrated by a simple experiment. Open a transparent terminal window and a same sized opaque window then whip them around the screen a bit. With colors at thousands the windows have to be pretty big (at millions much smaller) to see a marked difference in redraw rates, but the difference in rate is very noticeable.

If anyone has any brilliant ideas around those limitations, Apple needs to hire them ASAP!

[Angus_D]

Originally posted by Scrod:
<STRONG>Now, we all know how window scrolling in OS X doesn't yet have hooks to allow 2D hardware acceleration, correct? However, window dragging is accelerated. </STRONG>

I don't think it is hardware accelerated. It's just double-buffered.

[Scrod]

No, window dragging is definitely hardware accelerated. Double buffering doesn't really have much to do with it. Dragging was much slower in the Public Beta than it is now. In earlier versions such as 10.0.1 the borders of windows (which were drawn by the CPU and not the video card) would often lag behind the actual body of the window. And if you STILL don't believe me, you can open up QuartzDebug and note for yourself the absence of flashing while dragging a window around. There's no question that window dragging is hardware accelerated.

[lindberg]

QuartzDebug doesn't tell you whether drawing is hardware accelerated or not -- it has nothing to do with the video card.

On MacOS X, all drawing (both higher-level Quartz and QuickDraw) goes into a buffer located in shared memory. This buffer is shared with the WindowServer. When the buffer needs to be displayed on the screen, it's the WindowServer actually doing the blitting to the screen. This way, for window moves, only the WindowServer is involved, displaying the window being moved and also displaying regions of other windows that become visible. This means that the actual applications don't have to be constantly notified (and possibly swapped in) just to redraw regions, etc.

QuartzDebug shows you any drawing activity done by the *application* in the higher levels of Quartz, i.e. the drawing that goes *into* the shared buffer. Its intent is to show the developer how much displaying/flushing their application is doing to see if there are any obvious areas for drawing optimization. The fact that QuartzDebug doesn't show anything during window moves means that the WindowServer is doing all the work, and the applications themselves aren't redrawing anything. It's certainly possible that the WindowServer does use hardware acceleration and/or AltiVec to do its blitting, but QuartzDebug won't tell you that.

Scrolling on the other hand does mean that the application has to do some drawing, both redrawing the slider as it moves and obviously the region being scrolled. NSClipView in Cocoa by default actually caches regions already drawn, so that re-exposing those areas are a simple bitmap blit rather than a full redraw, but for regions that have never been exposed before they will of course have to be redrawn by the application (possibly also requiring swapped-out memory to be paged in -- the biggest slowdown of all most likely).

Mostly, scrolling is slower since more work has to be done, and there's no getting around it. All flushing to the screen would still have to go through the WindowServer anyways, as you won't have direct access to the hardware from a client application. I don't really know enough about this area to say it *can't* be sped up, but it will be difficult, and the gains likely won't be all that dramatic.

[3.1416]

QuartzDebug doesn't tell you whether drawing is hardware accelerated or not -- it has nothing to do with the video card.

That has not been my experience. QuartzDebug flashes the entire window content area on my Wallstreet which doesn't have graphics acceleration, and only flashes the outline on my G4, which does. Also my G4 has a second monitor attached driven by a Rage II, and when I drag windows on that monitor QuartzDebug does flash the entire window. Same thing for OpenGL views; no flashing on the G4's main monitor, constant flashing on the second monitor and Wallstreet.

[BusErrorDev]

What makes the scrolling slow is that the memory for the window needs to be pushed from the main memory on the PCI/AGP bus to the VRAM to get drawn. It eats lots of bus bandwidth to do so.

That is the main problem of OSX drawing; since it doesn't allocate offscreen VRAM memory for common drawing, it it overloading it's bus with meaningless copies.

Say, open a window a draw a circle in it. On OS9, the circle 'call' will be accelerated by the graphic card, since the drawing actualy is onscreen.

On OSX, the circle is drawn 'manualy' using the main processor into the main memory, without being accelerated at all. THEN the whole window content is moved on the BUS to the card's VRAM and is then blitted onscreen.

The result is *slow*. Of course it allows transparency etc etc, but it's also the worst way of using memory and bus bandwidth. Not only that, but it makes *zero* use of the super extra nifty graphic accelerators.

The solution is to use the 'offscreen' VRAM to draw, instead of main memory. While this slows down the few applications that genuinly needs to draw *bitmaps* into windows, it speeds the rest incredibly, since it allows the graphic card to do what it does best. The final blitting is instantaneous too, since the window content is already there.

As a side effect, it also will speed up the remaining of the OS, since it's cache won't be full of useless pixels and will be able to be used for real data

As another quirks, we could also use OpenGL 3D texture memory to do lots of the drawing, like the system fonts, the various Aqua widgets and textures. Using texture memory would allows the graphic accelerator to do the drawing, and the texture manipulation processors in those graphic cards is what is usualy heating up underneath your desk, so it would be used for something useful for once :&gt;

[Scrod]

Originally posted by BusErrorDev:
<STRONG>What makes the scrolling slow is that the memory for the window needs to be pushed from the main memory on the PCI/AGP bus to the VRAM to get drawn. It eats lots of bus bandwidth to do so.

That is the main problem of OSX drawing; since it doesn't allocate offscreen VRAM memory for common drawing, it it overloading it's bus with meaningless copies.
</STRONG>

I had always suspected this was the problem!

The solution is to use the 'offscreen' VRAM to draw, instead of main memory. While this slows down the few applications that genuinly needs to draw *bitmaps* into windows, it speeds the rest incredibly, since it allows the graphic card to do what it does best. The final blitting is instantaneous too, since the window content is already there.

This sounds like the proper solution, but--correct me if I'm wrong--wouldn't this require access to Quartz' private APIs?

[BusErrorDev]

Heck, it's *quartz* thats suposed to do that, not developers! it's it's job as low level drawing engine!

[Scrod]

To clarify:
Well yes, but your point was that for things to be drawn on the screen, they have to go through main memory, correct? If so, how might an application, in its quest for the greatest drawing speed possible, go about bypassing this? Wouldn't this also require bypassing Quartz to some degree? Or if not, why then do so many apps out there appear to lack hardware-accelerated scrolling?

[BusErrorDev]

You can't bypass Quartz; to get accelerated drawing, quartz will need to be able to draw in VRAM instead of RAM.
When I say 'be able to draw' it's a manner of speaking, if an app does a 'draw a line' kind of operation, it is suposed to be resolved as one opcode given to the graphic card to proceed/accelerate in it's own memory.

To make that works for real you *need* to change the API, of Carbon in particular, to prevent applications accessing directly the pixel data of their drawing environment. Until we have that, we will *never* have the speed of windows computer with the same graphic card.

As for the problem of scrolling, it is quite well known these days that the old scrolling method of 'move that square of pixels by x,y delta' and 'redraw what has changed' is generaly a bad idea, because it regulary involves non-rectangular clipping, and also involves partial drawing (bad for antialiased text for example) so most application just redraw everything; it's much easier to handle that way.

[AirSluf]

While the last few posts on screen draw speed bottlenecks are generally true. The opinion that Apple screwed up by not doing it in the accelerator card is fairly misleading. It currently CAN'T be done on the card for whole screen mode because no currently shipping card has an API that will support Quartz. Any opinion, based on the on the correctness of making that trade-off is very valid, but just saying Apple doesn't know how to program graphics isn't the least bit helpful or accurate.

The tradeoffs are open to as much debate as anyone could possible want and I am not supporting either side of the argument, but once the decision to base the future of the OS screen display on Quartz was made, currently shipping accelerator hardware was obsoleted for a large portion of the basic OS full-screen desktop drawing environment. Cards still work as advertised for the 3D drawing they were primarily designed to support. They can't support a new technology they hadn't seen before. I don't know if driver magic and OpenGL can eventually help already installed cards or not, but do think it is an area Apple is working.

Gawd, I either need to learn to type or get some sleep...

[ 08-09-2001: Message edited by: AirSluf ]

[BusErrorDev]

All that fluf about Quartz being so advanced that the poor 32/64Mb card just *can't* keep up makes me laughs all the time.

Quartz doesn't make anything complicated. It is not 'advanced'; PDF/Poscript has been around for 20 years. The new fluf is to hide the fact they use PDF not to pay the license for poscript.
And it's *peanuts* compared to what a current graphic card can do.

Transparency? LOL
Rotations? Arf
Scaling? Wow!
Drawing rotated text? Oh dear!

Whats that compared to lighting projection, particle systems and mipmap ?

Stop eating the marketing bits, that doesn't stand up to a developer.

[AirSluf]

You obviously aren't a very good developer if you don't know the difference between hardware design capability and the software that allows a programmer to access those abilities (the API). It's got nothing to do with marketing, it is a technology shift.

I have no argument if you feel Quartz is not advanced, by your definition, none of the *nix offspring or any hardware out there is either. It's all derivative since 1981.

I also agree (without doing a detailed comparison of Quartz's guts against NVidia's or ATI's instruction sets for hardware level OpenGL implementation to know if it is even physically possible on the current chips) that the physical throughput of the cards to push pixels would not be a bottleneck. But if you are suggesting a software implementation of OpenGL running to display Quartz, i'm gonna fall off my chair laughing.

You know a little about pushing 3D pixels, but show significant limitations outside that. Learn about the integration issues inside the rest of a box-(hardware and OS level software) before you spout off so ineloquently.

Hardware acceleration will eventually come with the benefits your earlier posts point out, but if it was a stupidly easy as you say, how come you haven't made your millions selling a turn-key solution to Apple for 10.1 integration?

[BusErrorDev]

You know, when you boot OSX and hold 'v', you see a colorful message saying something like "COLOR console at... etc"

Well, I wrote that line, a LONG time ago, when I was the only external developer working on mach in MkLinux.
That was when I rewrote the frame buffer driver in mach.

So, no, of course not I don't know what I'm talking about!

[AirSluf]

You know, when you boot OSX and hold 'v', you see a colorful message saying something like "COLOR console at... etc"

Well, I wrote that line, a LONG time ago, when I was the only external developer working on mach in MkLinux.
That was when I rewrote the frame buffer driver in mach.

So, no, of course not I don't know what I'm talking about!

If that's true good for you, it would also mean you should know enough about the inner workings of a machine to examine specific implementation issues on a system-wide basis vs. "the way I may have programmed something a long time ago is the only valid method anyone else can ever use, even if the hardware can't currently support it."

You haven't passed that test, and as for the issues involved in the discussion at hand you failed miserably there as well. Knowledge of some implementation on a different architecture implies nothing about knowledge of the inner workings and relationships about another. That knowledge though, should give you the tools to be able to accurately discover what is going on in another architecture. A task which you couldn't, didn't think to, or didn't want to accomplish.

When you understand Quartz as well as Apple, ATI and NVidia who are working to come up with some way or another to get the graphics cards involved in at least partial Quartz acceleration to offload the CPU cores and Altivec units, then let us in on your solution. But without any knowledge of WHY Quartz doesn't work like YOU want (or the rest of us either for that matter) your dismissal of how things are is useless. Even if the technical descriptions you give are accurate.

Be an old dog that learns a new trick, not just another potential has-been. The kids need some leadership, not whining about "in my day we knew how to ___________. Today, ..."

[BusErrorDev]

LOL

Well, sorry, I'm a professional developer, for 15 years or more: when I read "enough about the inner workings of a machine to examine specific implementation issues on a system-wide basis vs..." I run the other way.

You SURE you don't work for microsoft?
if you don't, you should!

You have an example of a driver YOU wrote? I mean, your vocabulary is imcompatible with any form of expression a techy would use to express himself; way too inneficient. my own analysis of your noisy barrage is that you try to hide the fact you don't know anything about what you are talking about.

I know dogs like that, they bark even louder when cornered.

[Angus_D]

Now now, guys, let's play nice.

And actually I think Apple is investigating ways that OpenGL could be used to accelerate Quartz performance in some areas. And I also read that theoretically, there should be no reason why Quartz isn't just as fast as DPS, anyway.

[maxuser]

&lt;personal commentary&gt;
First off, it's my personal belief that spouting off experience and accomplishments does not a valid point make, unless your experience shows proof of a relevent point. The purpose of such an act often is to warrant respect, but it rarely has that effect. Rather, it is usually more effective to use those experiences and accomplishments to fashion sound points, as they are not valid points in and of themselves. There are of course exceptions, for instance if someone says, "Look kid, my name is John Carmack and I've written the best 3D computer games in history," then I might back off as I don't wish to waste his time with my trifles, but I would find it far more noble for him to tear me to pieces with knowledge and intelligence.
&lt;/personal commentary&gt;

I have more experience with OpenGL and 3D than I do with Quartz or OS X's WindowServer, so of course my first thought is to address these performance issues with OpenGL. Obviously it makes no sense to use OpenGL's software renderer, as that will surely be far slower than the current implementation. Unless you have performance to spare, lax performance requirements, or need to debug visuals, as a developer you simply don't use GL's software renderer.

So what would it mean for Quartz to use OpenGL hardware rendering? From the GL side, it probably means that Quartz graphical elements are rendered to textures, and those textures are rendered onto screen-aligned rectangles using GL. The extent to which GL is used depends on the ambition of those providing the implementation and the actual need, e.g. there's little need to hardware-accelerate the rendering of check boxes.

But hardware rendering works best as a black box with many inputs (geometry, textures, rendering state) and minimal outputs (display rendered image on screen). It's essentially "fire and forget," where you fire graphics instructions, but never (or rarely) look at the rendered output. As soon as you feed the output back to the CPU for additional "custom" processing before it's displayed, i.e. rendering features that the hardware doesn't support, your performance advantage will disappear because the rendered image sitting in video memory needs to be copied to main memory for CPU manipulation, then back to video memory for display. Because memory bandwidth is one of the most crucial bottlenecks today, this approach is probably slower than today's software implementation, which manipulates images in main memory and copies them to video memory (one way vs. a round trip).

Scrolling of cached views appears to be a straightforward enough use of hardware rendering. The full view would be copied to an OpenGL texture that is mapped to scroll-view's rectangle, and when the view needs to scroll, you'd simply apply a translation to the texture matrix. It's not nearly as clear to me how to apply hardware rendering to window resizing. The application determines the behavior of resizing (e.g. does text need to rewrap to the new width, do images scale, do the number of columns change), and hardware rendering can assist only particular cases at the Quartz level. However, it's an entirely different story if an app is rewritten to use OpenGL for rendering. For instance, Apple could rewrite the Finder to use OpenGL directly to render the contents of folder windows (as opposed to using OpenGL-assisted Quartz). But honestly, I think something more than slow software rendering is going with those Finder windows. From what I've heard of 10.1, many of these slow GUI problems will soon be history anyway.

[AirSluf]

You have an example of a driver YOU wrote? I mean, your vocabulary is imcompatible with any form of expression a techy would use to express himself; way too inneficient. my own analysis of your noisy barrage is that you try to hide the fact you don't know anything about what you are talking about.

Nope, I don't do drivers. I also like the ability to write in something other than "techy" [sic] lingo and that makes me anything but way too inefficient. Oh, well somebody who pay's the contracts likes my writing. So do the guys that work with/for me.

Well, sorry, I'm a professional developer, for 15 years or more:

Congratulations.

when I read "enough about the inner workings of a machine to examine specific implementation issues on a system-wide basis vs..." I run the other way.

Those words do sound kind of Dilbert-ish don't they? I think they do get the point across though. If you don't know why something is broken, you can't fix it. Don't let the door get in the way.

[Scrod]

Hey, don't feel so bad, guys. If you want to see important people flaming each other with regards to the Mac and Apple technologies, I suggest you take a look at the current round table discussion at SilliconValley.com

[AirSluf]

Is it really that bad? I figured they wouldn't be full of praise as all the panelists have left Apple with an axe to grind. I actually figured I could get better balance reading the boards every couple days!

Sorry, that was just a little off topic...