Comments on: Why Bubblemark is a poor ui benchmark

By: Craftymind » Blog Archive » Introducing GUIMark - An RIA benchmark for Flex, Silverlight, HTML and more

Thu, 22 May 2008 16:04:18 +0000

[…] shares alot in common with another RIA benchmark Bubblemark. I’ve written a bit about Bubblemark and why I think an alternative is necessary, but I do believe Bubblemark and […]

By: Jacob

Jacob — Thu, 08 May 2008 02:11:55 +0000

I just found the bubblemark page this evening and I have to agree. In fact I quickly scouted the code, making a few changes that completely changed the results.

Run this in FireFox for best performance and you’ll see how far off the bubblemark results are: http://kong.arcanecoder.com/misc/bubblemark.html (IE limits to under 70 fps).

More of my comments are here: http://www.kongregate.com/forums/1/topics/7411?page=1#posts-144487

By: Metalink » Bubblemark update

Metalink » Bubblemark update — Wed, 16 Apr 2008 17:43:25 +0000

[…] made the one, which is geared towards measuring complex UI performance. You can check it out here along with an explanation on why bubblemark is not particularly relevant for the […]

By: Mark Finkle’s Weblog » Buublemark

Mark Finkle’s Weblog » Buublemark — Tue, 15 Apr 2008 01:52:15 +0000

[…] balls (or non-balls too, I guess), this test has you covered. Thankfully, I’m not the only one. The Flash guys have more than a few gripes with the test. Sean Christmann sums up my biggest […]

By: Sean Christmann

Sean Christmann — Fri, 11 Apr 2008 22:43:47 +0000

Thanks for leaving a comment Alexey, I hope I didn’t ruffle your feathers too much with the post, as I really do appreciate the effort you’ve put in to Bubblemark. In terms of relevance, I’m mostly concerned with testing from an RIA framework point-of-view. Most of those use cases deal with animation involving data manipulation, box constraints, charts and maybe even gaming.

By: Alexey Gavrilov

Alexey Gavrilov — Fri, 11 Apr 2008 22:06:10 +0000

Good points, and a nice test case (although for Silverlight it shows fps anywhere between 6 and 100 — you should probably use a longer time window for fps calculations). I really happy to see that other people care, and we are getting more ways to compare RIA frameworks performance. As for relevance of bubblemark results — it depends on what you are trying to do of course. I’m not a Flash expert — that’s why all source code is up there for you guys to tell me, if I’m wrong.

By: Sean Christmann

Sean Christmann — Fri, 11 Apr 2008 20:46:58 +0000

That must be why my cpu stays so low during the test. I really think its necessary that tests get developed that are so taxing on these runtimes that they can’t get above 25-30 fps, then we’ll really see which vendor can crunch out those few extra fps over the competition.

By: Chet Haase

Chet Haase — Fri, 11 Apr 2008 20:10:44 +0000

Sean,

Good points, similar to the beefs that James Ward and I had with the test. Note, however, in your first point that just changing the timing mechanism to use an ENTER_FRAME handler instead of Timer won’t fix the timing resolution issue I mentioned in my blog post; they’re both capped in the browser to a much lower rate than they could handle (basically, the refresh rate of the monitor). So updaing the values in ENTER_FRAME might be more realistic from a Flash rendering perspective, but it doesn’t get at one of the serious constraints of the original benchmark.

Chet.