Styjun

I am using the Go flatbuffers interface for the first time. I find the instructions sparse.

I would like to write a vector of uint64s into a table. Ideally, I would like to store numbers directly in a vector without knowing how many there are up front (I'm reading them from sql.Rows iterator). I see the generated code for the table has functions:

func DatasetGridAddDates(builder *flatbuffers.Builder, dates flatbuffers.UOffsetT) 
 builder.PrependUOffsetTSlot(2, flatbuffers.UOffsetT(dates), 0)

func DatasetGridStartDatesVector(builder *flatbuffers.Builder, numElems int) flatbuffers.UOffsetT 
 return builder.StartVector(8, numElems, 8)

Can I first write the vector using (??), then use DatasetGridAddDates to record the resulting vector in the containing "DatasetGrid" table?

(caveat: I have not heard of FlatBuffers prior to reading your question)

If you do know the length in advance, storing a vector is done as explained in the tutorial:

name := builder.CreateString("hello")

q55310927.DatasetGridStartDatesVector(builder, len(myDates))
for i := len(myDates) - 1; i >= 0; i-- 
 builder.PrependUint64(myDates[i])

dates := builder.EndVector(len(myDates))

q55310927.DatasetGridStart(builder)
q55310927.DatasetGridAddName(builder, name)
q55310927.DatasetGridAddDates(builder, dates)
grid := q55310927.DatasetGridEnd(builder)
builder.Finish(grid)

Now what if you don’t have len(myDates)? On a toy example I get exactly the same output if I replace StartDatesVector(builder, len(myDates)) with StartDatesVector(builder, 0). Looking at the source code, it seems like the numElems may be necessary for alignment and for growing the buffer. I imagine alignment might be moot when you’re dealing with uint64, and growing seems to happen automatically on PrependUint64, too.

So, try doing it without numElems:

q55310927.DatasetGridStartDatesVector(builder, 0)
var n int
for rows.Next() // use ORDER BY to make them go in reverse order
 var date uint64
 if err := rows.Scan(&date); err != nil 
 // ...
 
 builder.PrependUint64(date)
 n++

dates := builder.EndVector(n)

and see if it works on your data.