Forum Archive

Applying a GLSL filter to a photo

rodolpheg

Hello,

I'm quite new here, and hope this topic hasn't been beaten to death (I looked a bit and it doesn't seem so, so here we go).

I'd like to make a simple script that take a picture with the camera (or picks one from the roll), apply a simple shader (GLSL), which basically turn the image B&W, and then save it to the iOS roll.

The example I found (filters.fsh in the games directory, and Simplexnoise) was meant to apply the effects in real time. The document scaning script seems to be closer to what I want, but instead of the enhance_contrast() function using CIFilter, I'd like to use a similar function with a custom shader.

How should I go about doing this? Here's my code so far:

Shader (bwShader.fsh) :

```precision highp float;
varying vec2 v_tex_coord;
uniform sampler2D u_texture;
uniform float u_scale;
uniform vec2 u_sprite_size;
uniform float u_time;

vec4 grayscale() {
vec4 color = texture2D(u_texture, v_tex_coord);
float contrast = 1.7;
color = (color - 0.5) * contrast + 0.5;
float gray = (color.r + color.g + color.b) / 3.0;
return vec4(gray, gray, gray, color.a);
}

void main() {
gl_FragColor = grayscale();
}


Python script, adapted from the scaning example :

import photos
import console
from objc_util import *
from scene import *

CIFilter, CIImage, CIContext, CIDetector, CIVector = map(ObjCClass, ['CIFilter', 'CIImage', 'CIContext', 'CIDetector', 'CIVector'])

def take_photo(filename='.temp.jpg'):
img = photos.capture_image()
if img:
img.save(filename)
return filename

def pick_photo(filename='.temp.jpg'):
img = photos.pick_image()
if img:
img.save(filename)
return filename

def load_ci_image(img_filename):
data = NSData.dataWithContentsOfFile_(img_filename)
if not data:
raise IOError('Could not read file')
ci_img = CIImage.imageWithData_(data)
return ci_img

def enhance_contrast(ci_img):
filter = CIFilter.filterWithName_('CIColorControls')
filter.setDefaults()
filter.setValue_forKey_(2.0, 'inputContrast')
filter.setValue_forKey_(0.0, 'inputSaturation')
filter.setValue_forKey_(ci_img, 'inputImage')
ci_img = filter.valueForKey_('outputImage')
filter = CIFilter.filterWithName_('CIHighlightShadowAdjust')
filter.setDefaults()
filter.setValue_forKey_(1.0, 'inputShadowAmount')
filter.setValue_forKey_(1.0, 'inputHighlightAmount')
filter.setValue_forKey_(ci_img, 'inputImage')
ci_img = filter.valueForKey_('outputImage')
return ci_img

def bw(ci_img,shader):
# PROBLEM HERE?
ci_img.shader = shader
return ci_img

def write_output(out_ci_img, filename='.output.jpg'):
ctx = CIContext.contextWithOptions_(None)
cg_img = ctx.createCGImage_fromRect_(out_ci_img, out_ci_img.extent())
ui_img = UIImage.imageWithCGImage_(cg_img)
c.CGImageRelease.argtypes = [c_void_p]
c.CGImageRelease.restype = None
c.CGImageRelease(cg_img)
c.UIImageJPEGRepresentation.argtypes = [c_void_p, CGFloat]
c.UIImageJPEGRepresentation.restype = c_void_p
data = ObjCInstance(c.UIImageJPEGRepresentation(ui_img.ptr, 0.75))
data.writeToFile_atomically_(filename, True)
return filename

def main():

with open('bwShader.fsh') as f:
    src = f.read()
    shader = Shader(src)

console.clear()
i = console.alert('Info', '...', 'Take Photo', 'Pick from Library')
if i == 1:
    filename = take_photo()
else:
    filename = pick_photo()
if not filename:
    return

ci_img = load_ci_image(filename)

out_img = bw(ci_img,shader)
#out_img = enhance_contrast(ci_img)

out_file = write_output(out_img)
console.show_image(out_file)
print('Tap and hold the image to save it to your camera roll.')

if name == 'main':
main()
```

Thanks a lot. Once again, I hope I'm not too far off (or too imprecise) with this question.

mikael

@rodolpheg, if you do not specifically need to use a shader, taking a picture and grayscaling it is very straightforward:

import photos

image = photos.capture_image()
image = image.convert('LA')
photos.save_image(image)

print('Saved image to photos')

Replace capture_image with pick_asset if you want to select the picture from your photos instead.

Sorry, cross-posted with your edit.

rodolpheg

Thanks!

Although I want to eventually use more personal effects, this is why I want to start with a simple b&w shader... :)

The photos.save_image trick is great to know btw.

mikael

@rodolpheg, then I think the easiest could be:

  • Capture image
  • Create scene.Texture(image)
  • Create scene.SpriteNode(texture)
  • Set spritenode.shader with your shader
  • Snapshot the scene’s view and save the image

Snapshot is a semi-custom thing:

def snapshot(view):
  with ui.ImageContext(view.width, view.height) as ctx:
    view.draw_snapshot()
    return ctx.get_image()

Continuing on the path you have started on is probably feasible as well, but I have no experience applying shaders to images directly.

rodolpheg

Cool. I'm exploring that way. If anyone has experience with this, I'd also be happy to try their suggestions :)

Thanks again !