Forum Archive

I'm trying to overlay a SceneView on top of a WebView. The goal is to have the WebView show mp4 video using html while the Scene in SceneView renders game objects and processes touch events. (I got the mp4 and html part working.)

However, the Scene always have a solid background_color that blocks the webview content and I cannot make the Scene background transparent.

Here is a toy example:

from scene import *
import ui

class GameScene(Scene):
    def setup(self):
        self.background_color = None  
        # the background_color here only accepts RGB but not RGBA
        self.view.alpha = 0.2   
        # this changes the alpha of the entire SceneView, making both the background and the contents transparent. 
        sp = SpriteNode('emj:Christmas_Tree', anchor_point=(0,0), position=(100,100), parent=self)

w, h = ui.get_window_size()
frame = (0,0,w,h)
v = ui.View(frame=frame)

webview = ui.WebView(frame=(w/4,0,w/2,h))
webview.load_url('http://google.com')
v.add_subview(webview)

gameview = SceneView()
gameview.scene = GameScene()
gameview.frame = (w/4, 0, w/2, h/2)
v.add_subview(gameview)

# overlay SceneView() on top of WebView()
gameview.bring_to_front()
v.present('full_screen')

Thanks!

I tried to use objc_utils to change the background color, but have not figured out a solution. This is what I've tried so far:

from objc_util import ObjCInstance, ObjCClass
import objc_util

UIColor = ObjCClass('UIColor')
clear_color = UIColor.color(red=0.0, green=0.0, blue=0.0, alpha=0.0)

objv = ObjCInstance(self.view)
objv.backgroundColor = clear_color  
# this make the **View** background of SceneView transparent, but does not alter the **Scene** background. 

v1 = objv.subviews()[0]   # A GLKView instance
v1.alpha = 0.2
# this makes both the background and the content of the scene transparent
v1.backgroundColor = clear_color
# this does not do anything

I need some help on how to set the background of SceneView() to be transparent, so that I can see the content of the WebView beneath it. Thanks!

I saw some threads on how to make background transparent in Objective-C using glClearColor: http://stackoverflow.com/questions/10636176/opengl-es-2-0-glkit-with-transparent-background . But I'm not sure how to do it in Pythonista.

I've tried to help you but without success 😭, sorry

@cvp Thanks for trying! I'm stuck at the point where I couldn't find a way to access scene.background_color from ObjC.

A few thoughts, have not tried yet:
SpriteKit has a SKVideoNode, although this might be tricky to integrate into a SceneView. I'm not sure whether the underlying SKScene business is exposed.

Have you tried ObjCInstance(scene) and see if it returns an objc object? Or, perhaps ObjCInstance(scene.view). If so, then you should have access to background_color (though you might also need to set ObjCInstance(scene.view).allowsTransparency=true.

I've tried but still without success.
ObjCInstance(scene.view) has no attribute allowsTransparency.
I also tried to set its background color as ObjCClass('UIColor').clearColor().

Okay, played around with this a little.

From what I can tell, we may need to swizzle the glkView_drawInRect_ to call the glClearColor. Though I'm not even sure that will work. You also have to set opaque=False on the sceneview and perhaps glkView.. but again since glClearColor is not called in the drawInRect, I don't think that helps us.

Another approach which does seem to work but will be super annoying is to set the mask() on the ObjCInstance(sceneview).layer(), or perhaps ObjCInstance(sceneview).glkView().layer(). You would then create a CALayer positioned over your sprites. You'd have to keep repositioning the layers as the sprites move, which might not be reliable, and at least would take some math.

gist.github.com/144fba5549ece5df62550dd456272bf1

is a simple working example of a SceneView on top of a webview, with a mask layer. Touches work over the original scene size (note you can touch outside the tiny pythonista logo). I didn't try getting the mask size exactly right, or try repositioning from within the scene... we'll call that an exercise for the reader!

@JonB thanks for him
@yueqiw good luck, seems really not easy for us 😥

@JonB The scene module isn't based on SpriteKit at all, even though the API is very similar (and I originally wanted to use SpriteKit). It's all just OpenGL basically.

@cvp @JonB Thanks a lot! I'll see if the mask layer is good enough for my purpose.

I'm trying to implement some GLKit similar to this: https://forum.omz-software.com/topic/2066/python-opengles
@omz Any suggestion?

@Cethric has a whole set of bindings for doing low level opengl work.
https://github.com/Cethric/OpenGLES-Pythonista

Update: Now I add webview as a background subview of the SceneView, by gameview.add_subview(webview) and webview.send_to_back().

As long as the GLKView in the front has glClearColor(0, 0, 0, 0) in its glkView_drawInRect_() delegate method, the GLKView will be transparent.

However, as you can see from the following code, once I plug in a new delegate to the GLKView of SceneView, its connection to Scene is also lost (I got the transparent background, but no longer able to draw SpriteNode on it.)

from objc_util import *
from scene import *
import ui

glClearColor = c.glClearColor
glClearColor.restype = None
glClearColor.argtypes = [c_float, c_float, c_float, c_float]
glClear = c.glClear
glClear.restype = None
glClear.argtypes = [c_uint]
GL_COLOR_BUFFER_BIT = 0x00004000

def glkView_drawInRect_(_self, _cmd, view, rect):
    glClearColor(1, 0, 0, 0.3)
    glClear(GL_COLOR_BUFFER_BIT)

MyGLViewDelegate = create_objc_class('MyGLViewDelegate', methods=[glkView_drawInRect_], protocols=['GLKViewDelegate'])

class ChristmasScene(Scene):
    def setup(self):
        objv = ObjCInstance(self.view)

        delegate = MyGLViewDelegate.alloc().init()
        objv.glkView().setDelegate_(delegate)
        objv.glkView().setOpaque_(False)

        sp = SpriteNode('emj:Christmas_Tree', anchor_point=(0,0), position=(500,300), parent=self)

w, h = ui.get_window_size()
webview = ui.WebView(frame=(w/4,0,w/2,h))
webview.load_url('http://google.com')
gameview = SceneView()
gameview.scene = ChristmasScene()

gameview.add_subview(webview)
webview.send_to_back()

gameview.present('full_screen')

Just figured out the solution!

It's very simple. Just need to call glClearColor(0, 0, 0, 0) and glClear(GL_COLOR_BUFFER_BIT) at every frame by in draw()

class ChristmasScene(Scene):
    def setup(self):
        objv = ObjCInstance(self.view)
        objv.glkView().setOpaque_(False)
        sp = SpriteNode('emj:Christmas_Tree', anchor_point=(0,0), position=(500,300), parent=self))
    def draw(self):
        glClearColor(0, 0, 0, 0)
        glClear(GL_COLOR_BUFFER_BIT)

# The rest of the code is the same as the most recent reply. No need to set up a separate delegate. 

For this to work, the WebView has to be a subview of the SceneView. Adding WebView and SceneView as subviews of the same SuperView does not work well. Hope this doesn't break other drawing steps. Thanks again @JonB and @cvp for the help! Previous code examples from @omz and @Cethric also helped a lot for me to understand how things work.

@yueqiw Whaaaaaaaa 🎉🥂🍾