
- - - - - - - - - - - - - - - - - -
arrayvalue - nativearray discussion
- - - - - - - - - - - - - - - - - -

1. when reading arrays off the wire (tagdata input) the binding creates arrayvalue objects.

2. question whether we can substitute our nativearray across the board.
   a. probably best not, since code up the line that is now mostly ported,
      would instead have to be rewritten for nativearray.
      
3. should we however plan to always use 1-dim nativearray as the base of our arrayvalue, 
   since if we need to be able to access elements, we can't simply reflect to an array as
   does java, we need the subscripting and primitive object instantiation facilities of the 
   nativearray.
   
   a. however the arrayvalue is an arraylist under the covers. perhaps we should let the
      arrayvalue have either or both backing store types, arraylist or nativearray.
   b. that could create inconsistencies of access, in that accessing nativearray always
      returns a disposable, and accessing arrayvalue always returns a non-disposable.
   c. however we could well want to not store an object for every array element, a large
      byte array case in point. 
   d. let's store only as arraylist for now, leaving the option open to store as
      nativearray if we need it. 
   e. another possibility is to "lazy-load" the arraylist from the natarray. 
      in this way, we always return a non-disposable from arrayvalue.get().
      when item[i] is requested, we look in the arraylist, lazy-allocating it if
      needed. if arraylist[i] is null, we populate it from natarray[i]. in either
      case we return the non-disposable arraylist[i].
   
4. our arrayvalue has *both* an arraylist *and* a nativearray member.
   the natarray member currently is saved there when an arrayvalue was created *from* a nativearray.
   the natarray can be owned by the arrayvalue or not.
   a. arrayvalue has from() and to() nativearray methods
   b. arrayvalue_to_nativearray is limited to three dimensions of course.  
   
5. java code freely references the native array or the array value and expects both to be available.
   ArrayValue has getArray() which returns ArrayValue.array, the native array. TaggedData has
   Object fromArrayValue(ArrayValue) which returns the native array representation of the ArrayValue.
   So we really need to keep the 1-dim nativearray in the c arrayvalue. let's do it the lazy load way
   descibed above. the arrayvalue then handles the multi-dimensional aspect. 
   
   
- - - - - - - - - - - - - - - - - -
serialization notes 1
- - - - - - - - - - - - - - - - - -  
let's walk through writing some arrays.
first, a 2-dimensional array.
int x[2][3] = { {1,2,3}, {4,5,6} };

0. we start with a nativearray, and call writeValue(ARRAY, nativearray);

1. writeValue always puts the supplied byte typecode to the buffer first.
   so in this case, that is ARRAY. (1)
   
2. next, switch(typecode) to ARRAY. 
   a. convert the nativearray to an arrayvalue, passing ownership of na to av: 
        etch_arrayvalue* av = new_arrayvalue_from
           (nativearray, typecode, customtype, size(?), 0, TRUE); 
   b. call writeArray(av, vtor);  
   
3. writeArray calls startArray(arrayvalue);  
   a. put the arrayvalue's content type byte. (2)
   b. if content type is custom, put array's custom struct type. (3)
   c. put intval arrayvalue's dimensions (4)
   d. put intval arrayvalue's itemcount  (5)
   
4. writeArray calls writeValues(arrayvalue); 
   a. elementVtor ev = av.eltvtor;
   b. foreach entry val in ev
      writeValue(ev, val);  

5. writeValue writes each value in the arrayvalue.
   this will be recursive for multidim arrays.
   a. in our case, the first arravalue has dim 2 and has 2 entries, 
      which are the two 3-integer dimensions of x
      so writeValue(arravalue.value[i]); goes back to step 0. 
   b. our inner recursion of writeValue (arravalue.value[i]) writes the int content (6)

6. writeArray calls endArray(arrayvalue);   
   a. endArray writes the EOD marker. (7)  
   
ARRAY INTEGER 2 2            (a1) (a2) (a4) (a5)
   ARRAY INTEGER 1 3    (a6):(b1) (b2) (b4) (b5)
   1                    (b6) 
   2                    (b6)
   3                    (b6)
   EOD                  (b7)
   ARRAY INTEGER 1 3    (a6):(b1) (b2) (b4) (b5)
   4                    (b6) 
   5                    (b6)
   6                    (b6)
   EOD                  (b7)
EOD                     (a7)
   
- - - - - - - - - - - - - - - - - -
deserialization notes 1
- - - - - - - - - - - - - - - - - - 
looking at the serialization, it seems as if we should do things a bit differently from the java version.
when java creates an arrayvalue in which to read an array off the wire, it first allocates a java native
array, and then wraps an arrayvalue around that. with the scanty information available, and without lookahead,
we can't allocate our version of a nativearray, since (a) ours is top down, i.e. we need the entire byte blob
first, and (b) we don't know the item count up front (we know the itemcount for the serialized *array*, in 
our example above that would be 2, i.e. 2 arrays of type and length not yet read. all we know at the time
of array container allocation is the number of dimensions, and the actual serialized item count for that
dimension.

if it turns out we need data not yet read, we could conceivably look ahead for it. however the better approach
might be to read into arrayvalues, and when done, construct the nativearray. lets explore that avenue here.

0. we start with a serialized arrayvalue of dim 2, type int, whose members are 2 arrayvalues of dim 1.

1. readValue finds ARRAY (a1)
   a. calls readArray()
   
2. readArray 
   a. calls startArray() getting a new arrayvalue back. 
   
3. startArray()   
   a. reads the content type byte = INTEGER (a2)
   b. reads the dimensions count = 2  (a4) 
   c. reads the item count = 2 (a5)  
   d. allocates an arrayvalue (java code is lost)
   e. returns the new arrayvalue.
   
4. readArray receives arrayvalue from startArray.
   readArray calls readValues(arrayvalue, vtor)
   
5. readValues gets the element vtor for this dimension count
   value = readValue(arrayvalue); (a6)
   
   until value is EOD
      arrayvalue.add(value)
      value = readValue() ;
      
6. readValue 
   a. ARRAY (b1) 
   
7. readArray()
   arrayvalb = startArray();
   
8. startArray()
   a. reads the content type byte INTEGER (b2)
   b. reads the dimensions count = 1 (b4) 
   c. reads the item count = 3 (b5)
   d. allocates an arrayvalue (java code is lost)
   e. returns the new arrayvalb.   
   
   
9. readArray()          
   a. readValues(arrayvalb);  
   
10. readValues gets the element vtor for this dimension count
    value = readValue(arrayvalb) = tiny 1; (b6)        
    arrayvalb.add(value)
    value = readValue(arrayvalb) = tiny 2; (b6)        
    arrayvalb.add(value)
    value = readValue(arrayvalb) = tiny 3; (b6)        
    arrayvalb.add(value)             
    value = readValue(arrayvalb) = EOD (b7)  
    return; 
    
10. readArray() calls endArray which is a nop
    and returns arrayvalb. 
    
11. readValues (a) gets back Object arrayvalb and puts it to arrayvalue[0] 
    (repeat 5-10)
     readValues (a) gets back Object arrayvalc and puts it to arrayvalue[1]         
   
   
999. readValue processing ARRAY receives arrayvalue result of readArray(),
   presumably an arrayvalue with 2 members, each member being an array dim1 with 3 members. 
   and converts the arrayValue to nativearray.