I'm trying to implement a Photoshop style magic wand that will create image masks of islands of colors given a cursor position. My current implementation runs out of memory on the stack. The scene is very bare-bones, just a Sprite, ColorRect, and a Label. I'm using an image pulled out of the viewport, (via
get_texture().get_data()) and I made sure to lock and pass the Image data around instead of creating it again for each recursion.
Stack maxes at 1024. The crash (right now) occurs inside my Util singleton, but I'm guessing it shouldn't be near the limit at all if I were to write this properly -- I just don't know how to do that yet.
I figured I should put my code in a Pastebin since it's fairly long:
Any help or guidance is appreciated.
(Util.gd: https://pastebin.com/MAXMYFhV )
Edit: the caller for this procedure is as follows:
given input trigger:
var mask = magic_wand(get_viewport().get_mouse_position(), OCEAN)
OCEAN aliases to a const var at the top of the file but it gets overwritten anyway since it was a mistake to pass the desired
color_to_match instead of setting it in
magic_wand() via getting the color of the clicked pixel.
Edit2: Would it be more effective to do this via shaders (and, I'm guessing, offload the task to the GPU)? Would that be possible?