I’ve been doing a bit of experimenting with the Canvas and Video tags in HTML5 lately, and found some cool features hiding in plain sight. One of those is the Canvas.drawImage() api call. Here is the description on the draft site.
To draw images onto the canvas, the drawImage method can be used.
This method can be invoked with three different sets of arguments:
-
drawImage(image, dx, dy) -
drawImage(image, dx, dy, dw, dh) -
drawImage(image, sx, sy, sw, sh, dx, dy, dw, dh)
HTMLImageElement, an HTMLCanvasElement, or an HTMLVideoElement for the image argument.
The api lets you take the contents of specific HTML elements and draw them into a canvas, and the 3rd element in that list is just begging to be abused. Copying video into a canvas element means opening up the ability to manipulate or process video frames at runtime. Two concepts instantly came to mind that seemed like fun to try and figure out, here they are below.
Blowing apart fragments of video
Click around the video frame to blow up that part of the video, the exploded pieces will continue to play the video inside them. After a while they retract back to their original place. One feature I didn’t have time to figure out was adding depth to the explosion, so pieces that are closest to ground zero fly up into the air as they sail outward. With full shadow effects this could look really cool.
3D Video
This demo in particular runs really well inside webkit based browsers, but not so much in Firefox. Firefox doesn’t appear to have any hardware acceleration for Ogg decoding so I had to drop the video size in half in order to run at acceptable framerates. Even still, Firefox chokes pretty badly on my Macbook Pro.
*Update* – I’ve changed the ogg video to be 640 x 360, prepare to see firefox weep
Lessons learned
There’s a couple hints I found out along the way that are good to know if you want to play around with drawing video. First, you need a bit of hackish code to get this to work effeciently and it flows like this.
[Video playing] -> [Draw Video onto Canvas 1] -> [Draw fragments of Canvas 1 onto Canvas 2]
Don’t ask me why, but copying pixel data out of a video tag is expensive, so expensive that drawing it into a temporary canvas, and then drawing pieces of that temp canvas onto a final canvas is faster then just referencing the video tag repeatedly within the same loop. That’s why you’ll see 2 Canvases in the source code for the demos. I’m sure there’s a technical reason for this duplication process, but it’s a lazy reason.
Secondly, don’t try copying individual pixels around. You can still see the remnants of my first code attempt inside the explosion demo with getPixel() and setPixel(). This turned out to be horribly slow and completely unnecessary. Canvas.drawImage() + matrix transforms on the canvas space is far more efficient then handcrafted pixel pushing. On the other hand, pixel manipulation allows you to do things like runtime chroma keying, get ready for a new wave of “clippy” style videos with full transparency popping over websites to help you out.
Lastly, I’m learning very quickly that not all browsers are created equal when it comes to performance, it’s a crapshoot when it comes to heavy video+image manipulation. Safari and Chrome work well with h.264, Firefox slogs along with Ogg Theora, and Opera is somewhere in the middle.
This is so awesome. 3D space will be apart of the Web3.0 age.
great plugin I love it
Being able to blow up video files is definitely why we all need to switch over. This is like the cure for cancer. We can all happily blow our video files from now on with our new html5 overlords. Thank goodness me!
fantastic,but what is its pupose
Firefox 4b5pre runs even the 3D demo at a flawless speed on an intel pentium dual core running vindows vista.
Also, hardware acceleration is turned off (like is is by default) and antialiasing is used (unlike chrome which does not use antialiasing)
[...] video? http://www.craftymind.com/2010/04/20/blowing-up-html5-video-and-mapping-it-into-3d-space/ [...]
thanks for sharing. great news
can this be done with the videos playing through embed(embed tag) source?
wow awesome! html5 is really rocks the 3d effects :>
Flawless on IE 9 Beta.
Very cool demo!
[...] ????30???????????HTML5? Blowing up HTML5 video and mapping it into 3D space??HTML5???????3D?????? ???HTML 5??Canvas ?Video [...]
[...] //A } // ]]> Block based destruction of HTML5 video, best viewed in webkit based browsers. Return to Craftymind Article Due to popular request, the ogg video is now 640 x 360, don’t hold me responsible for [...]
[...] Blowing up HTML5 video and mapping it into 3D space [...]
[...] Blowing up HTML5 video and mapping it into 3D space [...]
[...] ??????????. ? ????????? ???????? ??? ????? ???????? (1, 2, 3, 4, 5, 6) ?????? ??, ??? ?? ??????????? ????? ?? [...]
[...] This is a site that allows you to click on certain videos and see how it blows up in front of your very eyes. Plus, the site will teach users on how to create such blowing up videos. Learn how to make a blowing up effect in videos with HTML5. [...]
[...] 3. Blowing up HTML5 video and mapping it into 3D space [...]
[...] 3. Blowing up HTML5 video [...]
I tried it with my webkit based Safari on iPad iOS 4.2. I also tried Skyfire.
None of this works on the iPad, sadly, but true.
If this would work on iOS, we could watch 3-D videos in true colors.
[...] Blowing up HTML5 video and mapping it into 3D space??HTML5???????3D??? [...]
??? ??? ???
I’m having problems with a much simpler video example.
I have two versions of a bouncing video. In one, the video element is moved directly.
http://faculty.purchase.edu/jeanine.meyer/html5workshop/videobounce1.html
In the other, the video is drawn on the canvas and shapes drawn on top to serve as a mask.
http://faculty.purchase.edu/jeanine.meyer/html5workshop/videobounce3.html
The first works in Firefox and Chrome.
The second works in Firefox. The video does not appear in Chrome.
Also, neither work with a tag, and they don’t appear to work with your doctype either.
Any help most appreciated.
please give me!
[...] Demo | Website:http://www.craftymind.com/2010/04/20/blowing-up-html5-video-and-mapping-it-into-3d-space/ [...]
[...] Link: Blowing up HTML5 video and mapping it into 3D space [...]
[...] 25. Blowing up HTML5 video [...]
Would you happen to know if this technique could be used to create an interactive video for the ipad?
We want to create a tutorial video with “hot spots” where a user can click on a section of the video an an action would take place…
[...] 6 Jogos em HTML 5 HTML 5 e o fim do Flash Efeitos com HTML 5 [...]
[...] not even mentionning how gorgeous the player is or how cool the full-window functionality is.3. Blowing up HTML5 videoThis will probably not only blow a video in your browser, it will also blow your mind. It’s [...]
[...] 8. Blowing Up HTML5 Video [...]
[...] Blowing up HTML5 video and mapping it into 3D space??HTML5???????3D??? [...]
[...] Blowing Up HTML5 Video [...]
Thats one of the awesome ones HTML5 work presentations.
[...] Blowing Up HTML5 Video [...]
[...] Blowing Up HTML5 Video This tutorial is about a blowing up HTML5 video and mapping it into 3D space. Check it out: [...]
[...] 10. Coding A HTML 5 Layout From Scratch [...]
[...] Blowing Up HTML5 Video This tutorial is about a blowing up HTML5 video and mapping it into 3D space. Check it out: [...]
I’m running an ubuntu laptop I bought in 05 with 128mg of RAM. Demos went flawless in the developers version of chrome. Gecko is light years behind webkit in canvas performance, but if any box would get ugly in chrome, it’s mine. I’d say you did an amazing job of conserving resources. It ain’t your fault Firefox is underfunded compared to chrome. Brilliant proof of concept. Bookmarked your sight you crafty mind.
?
[...] ????? brainy ????? html5&css3 Blowing up HTML5 video and mapping it into 3D space??HTML5???????3D?????????HTML 5??Canvas ?Video [...]
Veri nice, we are want develov our web with flatform HTML 5
very useful tips about HTML5, love it ^_^
[...] P6 C# |# G! o. ^# ]0 ?# b + y4 A& X( A+ s Blowing up HTML5 video and mapping it into 3D space??HTML5???????3D?????????HTML 5??Canvas ?Video [...]
[...] Blowing up HTML5 video and mapping it into 3D space [...]
[...] ????30???????????HTML5? Blowing up HTML5 video and mapping it into 3D space??HTML5???????3D?????????HTML 5??Canvas ?Video [...]
[...] Blowing Up HTML5 Video [...]
[...] Blowing Up HTML5 Video [...]
[...] Blowing up HTML5 video and mapping it into 3D space [...]
[...] Blowing up HTML5 video and mapping it into 3D space??HTML5???????3D?????????html 5??Canvas ?Video [...]
[...] ??http://www.craftymind.com/2010/04/20/blowing-up-html5-video-and-mapping-it-into-3d-space/ [...]
[...] Blowing up HTML5 video and mapping it into 3D space [...]
[...] Blowing up HTML5 video and mapping it into 3D space [...]
[...] Blowing up HTML5 video and mapping it into 3D space [...]
[...] Blowing Up HTML5 Video This tutorial is about a blowing up HTML5 video and mapping it into 3D space. Check it out: [...]
[...] Block based destruction of HTML5 video, best viewed in webkit based browsers. Return to Craftymind Article Due to popular request, the ogg video is now 640 x 360, don’t hold me responsible for [...]
[...] Blowing up HTML5 video and mapping it into 3D space??HTML5???????3D?????????HTML 5??Canvas ?Video ?????????????????????Canvas.drawImage() api???????? [...]
[...] ????????30???????????HTML5? Blowing up HTML5 video and mapping it into 3D space??HTML5???????3D?????????HTML 5??Canvas ?Video [...]
[...] 23. Blowing up HTML5 video and mapping it into 3D space [...]
[...] 22. Blowing up HTML5 video and mapping it into 3D space [...]
[...] View Tutorial [...]
[...] 1.Blowing up HTML5 video and mapping it into 3D space??HTML5???????3D??? [...]
[...] This is a site that allows you to click on certain videos and see how it blows up in front of your very eyes. Plus, the site will teach users on how to create such blowing up videos. Learn how to make a blowing up effect in videos with HTML5. [...]
[...] Blowing up HTML5 video and mapping it into 3D space??HTML5???????3D??? [...]
[...] View Tutorial [...]
[...] Blowing Up HTML5 Video [...]
I Like this~~~~~~
i Like it …………….!!!!!!!!!!!!
[...] Block based destruction of HTML5 video, best viewed in webkit based browsers. Return to Craftymind Article Due to popular request, the ogg video is now 640 x 360, don’t hold me responsible for [...]
[...] ???? [...]
that is great.
I remember double-buffering technics of 90’s. . .
PERO SI ESA BASURA LA HAGO IGUALITA EN FLASH 8 Y MAS RAPIDA Y MENOS NPESADA_
CUAL ES LA VENTAJA DE ESSA RIDICULES DE HTML5_
]YA LOE STUVE PROBANDO Y ES UN PROBLEMA PARA TODO
CREO QUE LOS QUE LO ESTAN APOYANDO SON GAYS TRISTES HAHAAHH
EL FLASH NUNCA MORIRA
HTML5 IS FUCK GARBAGE HAHAHAH
Pretty cool. Might try using this somewhere in my website later when HTML5 is supported by more browsers
Works with Opera without any problem (full support, no crashes, very smooth even on my slow computer).
??????????
wo bu hui ying wen ,neng fen yi cheng zhong wen bu
It is a wonderful work!!
Looks great!!
Actually it works pretty fast with the newest version of Opera (v 12.00)!
WOW Very Impressive effect.Very nice.
amazing style , nice post
[...] Blowing up HTML5 Video and Mapping it into 3D space [...]
[...] Blowing Up HTML5 Video and Mapping it into 3D Space [...]
[...] Blowing up HTML5 video and mapping it into 3D space [...]