首页 > 其他分享 >Go - Parsing JSON Data Streams Into Structs

Go - Parsing JSON Data Streams Into Structs

时间:2023-10-03 11:33:08浏览次数:47  
标签:nil color Into json Parsing JSON time data

Problem: You want to parse JSON data from a stream.


Solution: Create structs to contain the JSON data. Create a decoder using NewDecoder in the encoding/json package, then call Decode on the decoder to decode data into the structs.

 

In this first JSON file you have an array of three JSON objects (part of the data is truncated to make it easier to read):

[{ 
"name" :   "Luke  Skywalker" , 
"height" :   "172" , 
"mass" :   "77" , 
"hair_color" :   "blond" , 
"skin_color" :   "fair" , 
"eye_color" :   "blue" , 
"birth_year" :   "19BBY" , 
"gender" :   "male" 
}, 
{ 
"name" :   "C - 3PO" , 
"height" :   "167" , 
"mass" :   "75" , 
"hair_color" :   "n/a" , 
"skin_color" :   "gold" , 
"eye_color" :   "yellow" , 
"birth_year" :   "112BBY" , 
"gender" :   "n/a" 
}, 
{ 
"name" :   "R2 - D2" , 
"height" :   "96" , 
"mass" :   "32" , 
"hair_color" :   "n/a" , 
"skin_color" :   "white,  blue" , 
"eye_color" :   "red" , 
"birth_year" :   "33BBY" , 
"gender" :   "n/a" 
}]

To read this, you can use Unmarshal by unmarshalling into an array of Person structs:

func   unmarshalStructArray ()   ( people   [] Person )   { 
      file ,   err   :=   os . Open ( "people.json" ) 
      if   err   !=   nil   { 
          log . Println ( "Error  opening  json  file:" ,   err ) 
      } 
      defer   file . Close () 

      data ,   err   :=   io . ReadAll ( file ) 
     if   err   !=   nil   { 
          log . Println ( "Error  reading  json  data:" ,   err ) 
      } 

      err   =   json . Unmarshal ( data ,   & people ) 
      if   err   !=   nil   { 
          log . Println ( "Error  unmarshalling  json  data:" ,   err ) 
      } 
      return 
}

This will result in an output like this:

[]json.Person{
    {
        Name:       "Luke  Skywalker",
        Height:     "172",
        Mass:       "77",
        HairColor:  "blond",
        SkinColor:  "fair",
        EyeColor:   "blue",
        BirthYear:  "19BBY",
        Gender:     "male",
        Homeworld:  "",
        Films:      nil,
        Species:    nil,
        Vehicles:   nil,
        Starships:  nil,
        Created:    time.Date(1,  time.January,  1,  0,  0,  0,  0,  time.UTC),
        Edited:     time.Date(1,  time.January,  1,  0,  0,  0,  0,  time.UTC),
        URL:        "",
    },
    {
        Name:       "C - 3PO",
        Height:     "167",
        Mass:       "75",
        HairColor:  "n/a",
        SkinColor:  "gold",
        EyeColor:   "yellow",
        BirthYear:  "112BBY",
        Gender:     "n/a",
        Homeworld:  "",
        Films:      nil,
        Species:    nil,
        Vehicles:   nil,
        Starships:  nil,
        Created:    time.Date(1,  time.January,  1,  0,  0,  0,  0,  time.UTC),
        Edited:     time.Date(1,  time.January,  1,  0,  0,  0,  0,  time.UTC),
        URL:        "",
    },
    {
        Name:       "R2 - D2",
        Height:     "96",
        Mass:       "32",
        HairColor:  "n/a",
        SkinColor:  "white,  blue",
        EyeColor:   "red",
        BirthYear:  "33BBY",
        Gender:     "n/a",
        Homeworld:  "",
        Films:      nil,
        Species:    nil,
        Vehicles:   nil,
        Starships:  nil,
        Created:    time.Date(1,  time.January,  1,  0,  0,  0,  0,  time.UTC),
        Edited:     time.Date(1,  time.January,  1,  0,  0,  0,  0,  time.UTC),
        URL:        "",
    },
}

This is an array of Person structs, which you get after unmarshalling a single JSON array.

 

However, when you get a stream of JSON objects, this is no longer possible. Here is another JSON file, one that is representative of a JSON data stream:

{ 
"name" :   "Luke  Skywalker" , 
"height" :   "172" , 
"mass" :   "77" , 
"hair_color" :   "blond" , 
"skin_color" :   "fair" , 
"eye_color" :   "blue" , 
"birth_year" :   "19BBY" , 
"gender" :   "male" 
} 
{ 
"name" :   "C - 3PO" , 
"height" :   "167" , 
"mass" :   "75" , 
"hair_color" :   "n/a" , 
"skin_color" :   "gold" , 
"eye_color" :   "yellow" , 
"birth_year" :   "112BBY" , 
"gender" :   "n/a" 
} 
{ 
"name" :   "R2 - D2" , 
"height" :   "96" , 
"mass" :   "32" , 
"hair_color" :   "n/a" , 
"skin_color" :   "white,  blue" , 
"eye_color" :   "red" , 
"birth_year" :   "33BBY" , 
"gender" :   "n/a" 
}

Notice that this is not a single JSON object but three consecutive JSON objects. This is no longer a valid JSON file, but it’s something you can get when you read the Body of a http.Response struct. If you try to read this using Unmarshal you will get an error:

Error  unmarshalling  json  data:  invalid  character  '{'  after  top - level  value

However, you can parse it by decoding it using a Decoder:

func   decode ( p   chan   Person )   { 
      file ,   err   :=   os . Open ( "people_stream.json" ) 
      if   err   !=   nil   { 
          log . Println ( "Error  opening  json  file:" ,   err ) 
      } 
      defer   file . Close () 

      decoder   :=   json . NewDecoder ( file ) 
      for   { 
          var   person   Person 
          err   =   decoder . Decode ( & person ) 
          if   err   ==   io . EOF   { 
              break 
          } 
          if   err   !=   nil   { 
              log . Println ( "Error  decoding  json  data:" ,   err ) 
              break 
          } 
          p   < -   person 
      } 
      close ( p ) 
}

First, you create a decoder using json.NewDecoder and passing it the reader, in this case, it’s the file you read from. Then while you’re looping in the for loop, you call Decode on the decoder, passing it the struct you want to store the data in. If all goes well, every time it loops, a new Person struct instance is created from the data. You can use the data then. If there is no more data in the reader, i.e., you hit io.EOF , you’ll break from the for loop.
In the case of the preceding code, you pass in a channel, in which you store the Person struct instance in every loop. When you’re done reading all the JSON in the file, you’ll close the channel:

func   main ()   { 
      p   :=   make ( chan   Person ) 
      go   decode ( p ) 
      for   { 
          person ,   ok   :=   < - p 
          if   ok   { 
              fmt . Printf ( "%#  v\n" ,   pretty . Formatter ( person )) 
          }   else   { 
              break 
          } 
      } 
}

Here’s the output from the code:

json.Person{
    Name:       "Luke  Skywalker",
    Height:     "172",
    Mass:       "77",
    HairColor:  "blond",
    SkinColor:  "fair",
    EyeColor:   "blue",
    BirthYear:  "19BBY",
    Gender:     "male",
    Homeworld:  "",
    Films:      nil,
    Species:    nil,
    Vehicles:   nil,
    Starships:  nil,
    Created:    time.Date(1,  time.January,  1,  0,  0,  0,  0,  time.UTC),
    Edited:     time.Date(1,  time.January,  1,  0,  0,  0,  0,  time.UTC),
    URL:        "",
}
json.Person{
    Name:       "C - 3PO",
    Height:     "167",
    Mass:       "75",
    HairColor:  "n/a",
    SkinColor:  "gold",
    EyeColor:   "yellow",
    BirthYear:  "112BBY",
    Gender:     "n/a",
    Homeworld:  "",
    Films:      nil,
    Species:    nil,
    Vehicles:   nil,
    Starships:  nil,
    Created:    time.Date(1,  time.January,  1,  0,  0,  0,  0,  time.UTC),
    Edited:     time.Date(1,  time.January,  1,  0,  0,  0,  0,  time.UTC),
    URL:        "",
}
json.Person{
    Name:       "R2 - D2",
    Height:     "96",
    Mass:       "32",
    HairColor:  "n/a",
    SkinColor:  "white,  blue",
    EyeColor:   "red",
    BirthYear:  "33BBY",
    Gender:     "n/a",
    Homeworld:  "",
    Films:      nil,
    Species:    nil,
    Vehicles:   nil,
    Starships:  nil,
    Created:    time.Date(1,  time.January,  1,  0,  0,  0,  0,  time.UTC),
    Edited:     time.Date(1,  time.January,  1,  0,  0,  0,  0,  time.UTC),
    URL:        "",
}

You can see that three Person structs are being printed here, one after another, as opposed to the earlier one that was an array of Person structs.

 

A question that sometimes arises is when should you use Unmarshal and when should you use Decode ?
Unmarshal is easier to use for a single JSON object, but it won’t work when you have a stream of them coming in from a reader. Also, its simplicity means it’s not as flexible; you just get the whole JSON data at a go.
Decode , on the other hand, works well for both single JSON objects and streaming JSON data. Also, with Decode you can do stuff with the JSON at a finer level without needing to get the entire JSON data out first. This is because you can inspect the JSON as it comes in, even at a token level. The only slight drawback is that it is more verbose.
In addition, Decode is a bit faster.

 

标签:nil,color,Into,json,Parsing,JSON,time,data
From: https://www.cnblogs.com/zhangzhihui/p/17740915.html

相关文章

  • JSON基础
    概述JavaScriptObjectNotation(JavaScript对象表示法)简称JSON是一种轻量级的数据交换格式。它基于ECMAScript的一个子集。JSON采用完全独立于语言的文本格式,但是也使用了类似于C语言家族的习惯(包括C、C++、C#、Java、JavaScript、Perl、Python等)。这些特性使JSON成为理......
  • 前端JSON.stringify,JSON.parse函数
    JSON.stringify将对象转为JSON字符串;JSON.parse将JSON字符串转为对象;对象:{productId:129}JSON字符串:"{\"productId\":129}"***JSON使用场景***1. localStorage/sessionStorage存储对象  localStorage/sessionStorage只可以存储字符串,当我们想存储对象的时候,需要使用JSON.s......
  • 爬取豆瓣电影,保存到json文件中
    importurllib.requesturl='https://movie.douban.com/j/chart/top_list?type=5&interval_id=100%3A90&action=&start=0&limit=20'headers={'User-Agent':'Mozilla/5.0(WindowsNT10.0;Win64;x64)AppleWebKit/537......
  • Go每日一库之186:sonic(高性能JSON库)
    介绍我们在日常开发中,常常会对JSON进行序列化和反序列化。Golang提供了encoding/json包对JSON进行Marshal/Unmarshal操作。但是在大规模数据场景下,该包的性能和开销确实会有点不够看。在生产环境下,JSON序列化和反序列化会被频繁的使用到。在测试中,CPU使用率接近10%,其中极端情况......
  • Mac部署Python语言json模块(Anaconda)
      本文介绍在Mac电脑的Anaconda环境中,配置Python语言中,用以编码、解码、处理JSON数据的json库的方法;在Windows电脑中配置json库的方法也是类似的,大家可以一并参考。  JSON(JavaScriptObjectNotation)是一种轻量级的数据交换格式,常用于数据的序列化和传输。而Python中的json库,......
  • 对象转JSON 遇到的BigDecimal 科学计数法的问题,json转化字段单独处理
    问题描述:项目需要发送JSON数据,BigDecimal转成json仍然显示科学计数法,如果使用BigDecimai的toPlainString()需要将数据格式转为String,所以找了一下fastjson的自定义序列化内容,记录一下,以免以后忘记解决方案:方案一:JSONObject.toJSONString(vo,SerializerFeature.WriteBigDecimalA......
  • VScode对于json格式文件允许添加注释设置(永久有 效)
    如果你想让VSCode永久地将所有的.json文件都识别为JSONC,你可以通过修改VSCode的全局设置来实现。以下是具体步骤:在VSCode中按下Ctrl+,来打开设置(或者在菜单中选择"File"->"Preferences"->"Settings")。在搜索框中输入“files.associations”。在"Files:Associations......
  • SyntaxError: "undefined" is not valid JSON
    今天在写组件的一个接受JSON字符串的prop时,不知道为什么会报以下错误。file:[console]SyntaxError:"undefined"isnotvalidJSONfile:[Example.vue]<SVGv-if="data.length"v-for="itemindata":width="item.width":height="item......
  • xStream完美转换XML、JSON
    xStream框架xStream可以轻易的将Java对象和xml文档相互转换,而且可以修改某个特定的属性和节点名称,而且也支持json的转换;前面有介绍过json-lib这个框架以及Jackson这个框架它们都完美支持JSON,但是对xml的支持还不是很好。一定程度上限制了对Java对象的描述,不能让xml完全体现到对Java......
  • json
    JSON(JavaScript ObjectNotation)是一种轻量级的数据交换格式。它基于ECMAScript的一个子集。JSON采用完全独立于语言的文本格式,但是也使用了类似于C语言家族的习惯(包括C、C++、C#、Java、JavaScript、Perl、Python等)。这些特性使JSON成为理想的数据交换语言。易于人阅读和编......