How to sample a texture in a script?

Question

I'm trying to create a patch of grass with height controlled by a texture. My idea was to extend MultiMeshInstance with a script to take a texture as an input and then transform the individual instances of the grass blades according to the texture values.

I can successfully add a texture parameter to the MultIMeshInstance by just adding this to the beginning of the script:

export(Texture) var height;

and I think I can transform the blades by looping over the instances and setting the transforms using multimesh.set_instance_transform(). But how do I read values used for the transformation from the texture?

I also thought that maybe the grass height texture should be a property of the target surface and not the MultiMeshInstance, but I can't really think of any way to sample the texture from there either.

Answer 1

I'm not sure what exactly you're asking with

"But how do I read values used for the transformation from the texture?"

Textures are stored on the GPU. So if your question is how to write a GPU fragment shader which samples a texture, you can read the official shader programming tutorial which does exactly that with a noise texture.

If you want to read texel values on the CPU from a texture, you can't, since the texture memory isn't accessible from the CPU. This is not a Godot limit, it's a hardware limit. You first need to create an image from the texture using the get_data method, and then you can use the image's get_pixel method (or it's get_data if you want to process values in bulk ) to read specific pixel values.

Note that while sampling textures in fragment shaders is very efficient and fast, sampling them on the CPU will likely bring your framerate down to something unacceptable.

Comments

I wanted a property of MultiMeshInstance to be affected by the texture on the target surface. Reading pixels from an image won't work, because the information of how the texture is mapped to the target surface is missing there. To be clear, this doesn't need to be updated every frame, but just when MultiMeshInstance is created (and, optionally, when explicitly requested in the code).

This would be similar to how I can, in Blender, modify the properties of particles according to a texture applied to the emitter surface. I wonder if I should just use Godot particles instead of MultiMeshes (I'm a bit unclear on when I should use one or the other anyway) and would the behaviour I want be easier to implement on particles?