Skip to content

Stream

Jump to bottom
Allen edited this page May 28, 2020 · 1 revision

English | 中文

json-iterator provides Stream to control the json encoding output. Through the API provided by it, in conjunction with a custom Extension or ValEncoder, we can customize how our data is encoded and output into json.

Stream instance

There are two ways to create a Stream instance:

  1. Borrow one from the Stream instance pool of the API object

    c := jsoniter.ConfigDefault
s := c.BorrowStream(os.Stdout)
defer c.ReturnStream(s)

// xxxxxx
// ......

    Call BorrowStream to "borrow" a Stream instance from the pool, remember to "return" it when you don't need it any more

  2. Call NewStream to create a new one

    s := jsoniter.NewStream(jsoniter.ConfigDefault, os.Stdout, 1024)

// xxxxxx
// ......

    Call it with your configuration, the underlying writer and the initial size of the internal buffer used by Stream

Custom encoding behavior

When registering a Extension or ValEncoder, you need to use Stream to customize how your fields are output as json

type sampleExtension struct {
    jsoniter.DummyExtension
}

type wrapEncoder struct {
    encodeFunc  func(ptr unsafe.Pointer, stream *jsoniter.Stream)
    isEmptyFunc func(ptr unsafe.Pointer) bool
}

func (enc *wrapEncoder) Encode(ptr unsafe.Pointer, stream *jsoniter.Stream) {
    enc.encodeFunc(ptr, stream)
}

func (enc *wrapEncoder) IsEmpty(ptr unsafe.Pointer) bool {
    if enc.isEmptyFunc == nil {
        return false
    }

    return enc.isEmptyFunc(ptr)
}

func (e *sampleExtension) CreateEncoder(typ reflect2.Type) jsoniter.ValEncoder {
    if typ.Kind() == reflect.Int {
        return &wrapEncoder{
            func(ptr unsafe.Pointer, stream *jsoniter.Stream) {
                // Add the value to 1000 and encode it
                stream.WriteInt(*(*int)(ptr) + 1000)
            },
            nil,
        }
    }

    return nil
}

func streamTest(){
    jsoniter.RegisterExtension(&sampleExtension{})
    j, _ := jsoniter.MarshalToString(1000)
    fmt.Println(j)
    // output:2000
}

In the example above, we registered an Extension. In the CreateEncoder function of Extension, we add the value pointed by ptr to 1000 and then called the WriteInt function of Stream to encode it into json. When registering ValEncoder, we also use the function provided by Stream to customize the output of our field

type testStructForStream struct{
    Field int
}

jsoniter.RegisterFieldEncoderFunc(reflect2.TypeOf(testStructForStream{}).String(), "Field",
func(ptr unsafe.Pointer, stream *jsoniter.Stream) {
    // Convert int type value to string type
    stream.WriteString(strconv.Itoa(*(*int)(ptr)))
}, nil)

j, _ := jsoniter.MarshalToString(testStructForStream{1024})
fmt.Println(j)
// output:{"Field":"1024"}

In this example, we registered a ValEncoder for the Field field of testStructForStream, this ValEncoder calls the WriteString method provided by Stream, and the int type value pointed by ptr is converted into string type

Stream provides various types of writing functions, which allow us to easily customize how our data is encoded into json:

  • WriteBool
  • WriteInt
  • WriteFloat32
  • WriteString
  • WriteArrayStart, WriteArrayEnd
  • WriteObjectStart, WriteObjectEnd
  • WriteEmptyArray
  • WriteEmptyObject
  • ......

For detailed description of each function, please refer to godoc

Construct json Manually

With the functions provided by Stream, you can manually construct an output json

s := jsoniter.ConfigDefault.BorrowStream(nil)
// Remember to return the Stream instance
defer jsoniter.ConfigDefault.ReturnStream(s)

s.WriteObjectStart()
s.WriteObjectField("EmbedStruct")
s.WriteObjectStart()
s.WriteObjectField("Name")
s.WriteString("xxx")
s.WriteObjectEnd()
s.WriteMore()
s.WriteObjectField("Id")
s.WriteInt(100)
s.WriteObjectEnd()

fmt.Println(string(s.Buffer()))
// output:{"EmbedStruct":{"Name":"xxx"},"Id":100}

However, we usually don't need to do this. More often, these functions are used to customize the encoding behavior when registering Extension or ValEncoder.

Another serialization api

Stream also provides a fuction that work exactly the same as the Encode method of Encoder

type testStructForStream struct{
    Field int
}

s := jsoniter.NewStream(jsoniter.ConfigDefault, nil, 1024)
s.WriteVal(testStructForStream{300})
result := s.Buffer()
buf := make([]byte, len(result))
copy(buf, result)
fmt.Println(string(buf))
// output:{"Field":300}

In the above example, we call NewStream to create a Stream instance, then call the WriteVal function to finish the serialization. Actually that's how jsoniter.Marshal work internally. You can also try this function to serialize your data and if you do use it, there are two points to note here:

  • When calling NewStream to create Stream instance, you can set the size of Stream's internal buffer. If your usage scenario has extreme performance requirements and you know how large your serialized json will be, we recommend you to use this method instead of directly calling Marshal for serialization. By specifying the right initial size of the internal buffer, the performance loss bring by the growth of the buffer can be avoid
  • Remember to copy the result returned by calling Buffer to your slice since stream will reuse its buffer and the data in it will be overwritten.

Reusing Stream instances

If you need to reuse the same Stream instance, remember to reset it before next serialization

type testStructForStream struct{
    Field int
}

s := jsoniter.ConfigDefault.BorrowStream(os.Stdout)
defer jsoniter.ConfigDefault.ReturnStream(s)
s.WriteVal(testStructForStream{300})
result := s.Buffer()
tmp := make([]byte, len(result))
copy(tmp, result)

// xxxxxx
// ......

if s.Error != nil{
    return
}

// Remember to reset your stream
s.Reset(nil)
s.WriteVal(testStructForStream{400})

Please note that when an error occurs during serialization, the Stream instance can not be used any more even you call Reset to reset it. The only thing you can do is to create a new one

Flush

If you specified the io.Writer when creating the Stream, you need to call Flush to flush the buffer in Stream to your io.Writer

type testStructForStream struct{
    Field int
}

s := jsoniter.NewStream(jsoniter.ConfigDefault, os.Stdout, 1024)
s.WriteVal(testStructForStream{300})
// Without this call, your serialized results will not be flushed to the underlying device
s.Flush()
// output:{"Field":300}
Clone this wiki locally