I want to roll my own network protocol, so was looking for a way to efficiently output a RawArray from many pieces without knowing initially how long it will need to be (similar to C++ output streams). How should I do this, purely using GDScript? Ideally it would work in 2.1, but 3.0 answers are helpful too.
Using one RawArray and appending all elements to it using push_back / append is not a good solution because RawArray has to be reallocated for every element that is appended, leading to bad performance (quadratic time?).
A solution I considered is to build up an Array of RawArrays and then concatenate them at the end, since Array has efficient push_back and when I'm done adding RawArrays I will know the total length and can just make one giant RawArray to copy elements to. However, I don't know a good way to copy elements from the small arrays to the final array. Here are some ideas I considered:
- Don't preallocate RawArray and just use append_array: Same problems as using lots of appends / push_backs, and will still be quadratic time since most of the component RawArrays will be small.
- In GDScript, copy each byte individually into the right place in a loop: This is very inefficient because several GDScript operations need to be done for every byte (index into RawArray, assign element of final RawArray, loop to beginning of loop). This may or may not be better than the first idea, depending on average size of component RawArrays and the number of component RawArrays. For smaller streams it will probably be less efficient.
I'm looking for either an efficient way to copy a RawArray into part of another RawArray (similar to C memcpy, though hopefully with more memory safety), or a completely different way to handle building up a RawArray that will work for me. Everything should be implementable in GDScript, because if I was using C++ the second idea I described above would be efficient.