json文件处理以及将数据切分成训练集、验证集和测试集.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 读取json文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import random\n",
    "from stanfordcorenlp import StanfordCoreNLP\n",
    "\n",
    "nlp = StanfordCoreNLP(r'/home/jinhanqi/program/stanford-corenlp-full-2016-10-31/')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "path = '/home/jinhanqi/tf_program/summarization_data/review_data/Toys_and_Games_5.json'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def parse(Str):\n",
    "    try:\n",
    "        s = nlp.word_tokenize(Str)\n",
    "        return ' '.join(s)\n",
    "    except ValueError:\n",
    "        print \"Error.............................................................\"\n",
    "        print Str\n",
    "        print \"Error.............................................................\"\n",
    "        return Str"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "I like the item pricing . My granddaughter wanted to mark on it but I wanted it just for the letters .\n",
      "2000\n",
      "I saw this game on the cover of a magazine , and in several stores , and it was all the awards it received that got my attention . I needed a game that me and my husband could play with our kids . ( ages 6 and 14 ) . That is not an easy task to find a game for such a broad spectrum of ages . We usually stick with Uno . But this looked promising , so we picked it up . We played it wrong the first couple of times , but with each play we figured out the rules better . ( it is not one that you can just skim over the rules and \" get it \" right away . But once you do \" get it \" it is super simple . ) It kind of works like Uno crossed with the strategy of Scrabble . We love it .\n",
      "4000\n",
      "This game has great potential to teach kids strategy , tactics and arithmetic combinations , if they can bear the disappointment of being sent back to the start by their opponent . That is why it scores low on fun . However if the kids can overcome that disappointment , which is not easy from my experience with my 6 year old , and learn to be sportive there are enormous skills for the picking that can be useful as building blocks for academic skills .\n",
      "6000\n",
      "Love this case - it actually stays closed , even when fully loaded with heavy cars , has tons of space - actually holds more than 48 cars due to several central compartments that are larger than the others . You can fit 2 cars in the smaller slots and up to 6 or 7 in the large ones . Sturdy and attractive .\n",
      "8000\n",
      "My autistic son has played with this faithfully & aggressively moved dirt , clay , sand & gravel , and even his smaller metal earth movers for over 30 days straight now . He weighs OVER 60lb and this toy has yet to show signs of stress , bending , or dents . He has shared it with all his friends as well.I read the posts about the quality of this item and how well it holds up . And I have to totally disagree with all the negative issues.However , I guarantee should a grownup try to play kiddo and get on it , it will indeed bend & or break . It 's a toy made for a small child and must be appreciated for what it is . Just keep daddy & uncle off of it , and the big neighborhood kids and it should last years.And honestly evey rideon toy for a toddler is made WITHOUT seatbelts & harnesses . You would n't think twice when your child falls off his little tikes pony or rocking horse so do n't complain about this . It 's barely off the ground so if there is atumble it 's not nothing to freak out about . But you can pile sand and or dirt around the base if you feel the need to secure it some . Or get a welder to beef it up some so that Daddy can play on it too .\n",
      "10000\n",
      "Maybe 10 different puzzle patterns to satisfy little minds . My 2 and 4 year old granddaughters never seem to tire of this toy . Sturdy , wood pieces . Large enough for little hands . Shapes and colors are fun to identify while putting together brightly colored puzzles . Great learning toy .\n",
      "12000\n",
      "Bought this for my eight year old niece . She was n't impressed , but I had a lot of fun with it .\n",
      "14000\n",
      "I 'd recommend Bohnanza to anyone looking for a light , interactive card game for the family , or even a filler for experienced game groups . The fixed hand mechanic is fairly unique and clever . The trading element can be a fantastic way to teach fairness and equity to kids.I 'd recommend checking out the BoardGameGeek.com entry for this game . There 's a fascinating and fun variant available as a downloadable PDF in the files section called the Futures Market that I highly recommend . It adds some cool new options for trading , while not really making the game much more complex .\n",
      "16000\n",
      "I 'm a big Mod Podge fan , but this is my choice for puzzles after trying it.It 's thinner , which means using the foam applicator top , you do n't get brush marks . It does need at least 2 applications on a puzzle . Let them cure/dry for an hour or two between coats.The foam applicator top covers a small button that opens the bottle to distribute glue . It wo n't open without pressure , and unless the bottle is inverted , it wo n't glue even when the button is depressed . So even though the cap does come off easily , you do n't need to worry about it spilling . The glue will not spill just because it tips over . The foam applicator does need to be rinsed clean of glue after each use or it will dry out and be unusable.It dries to a very nice gloss which works very well on good quality puzzles.It 's a good puzzle glue by a good puzzle company .\n",
      "18000\n",
      "As a teacher and mom of three , I was very happy with this toy . The kids enjoy handling the chunky beads . It can easily be adapted for toddler through kindergarten . I love all the Melissa and Doug toys . They are easy to store away for a rainy day or something special to pull out when the babysitter arrives .\n",
      "20000\n",
      "not the same colors as shown in picture but that doesnt matter , my one year old and i both fit in it last summer ( 2011 ) and used it pretty much everyday . brought it in this winter to use as a ball pit and it works pretty good as one . still no holes or rips .\n",
      "22000\n",
      "My 3 year old grandchild loved these . She plays with barbies too , but found these even more fun , because of the large family , including Grandma and Grandpa , and a baby . These seem to be very sturdy . I would buy them again , or add to them in the future .\n",
      "24000\n",
      "So i have had this tent now for several months , and i must say its been worth every penny !!!!! my kids who are 2 and four love it . the tent is made of good quality material . its sturdy and can be easily assembled and moved around . My kids play very rough and so far it has met up to the standards of being able to take a beating . I highly recommend this tent to those with two young ACTIVE kids .\n",
      "26000\n",
      "I bought this for my twin 2 year old grandsons . They love it !! It is teaching them amounts , shapes , and colors ... all in one product . They will be playing with this for a long time ! Highly recommend !!\n",
      "28000\n",
      "I was so excited to get this for my daughter for Xmas . She got to use it 1 time . yes , that 's right 1 time , then the prongs in the connector from the hand control to the video reader broke when we tried to connect it a second time.I really think they should have used a more durable connector material or prongs that do not bend like aluminum foil.Very disappointed and have to try to find just the hand controller . Wish me luck .\n",
      "30000\n",
      "This is a very very handy product , we love to color in our house and this offers crayons in a wide variety of colors , perfect !\n",
      "32000\n",
      "I already had a set of Unifix cubes which were well loved and well-worn , so when my mother in law recommended these to go with a workbook , I turned her down at first . But then when she bought us a set anyway , I was thrilled . I did n't realize from the picture online that they could snap together any way you want . But this makes them far more versatile than the Unifix ( which only snap top/bottom ) . You can make different shapes out of these , including rectangles or capital letters . You can make 3-D shapes too . You can illustrate many basic mathematical concepts more creatively . My four year old even liked lacing them onto a shoelace for fun!I am a big fan of cubes in general , but these are such an upgrade .\n",
      "34000\n",
      "Cute baby figurines . The price was great and my 5 year old loves them . They are a little small , so I would n't purchase if you have young toddlers who like everything in their mouth . I could see them being a choking hazard .\n",
      "36000\n",
      "We bought a couple 5 &#34; balls for our baby for his first birthday . We are completely happy with this ball . He loves playing with them . They are sized perfect for tiny hands , and not too firm so they hurt him . He enjoys the bouncing sounds and plays with these balls each and every day .\n",
      "38000\n",
      "I bought this because my almost 2 year old pretty much destroyed regular crayons . These are perfect and extremely durable and the case is great to .\n",
      "40000\n",
      "Our 4 or 6 year old grandchildren loved the Big Flyer . It is very well built and seems extra sturdy . They ride it down our slightly inclined driveway and have a ball . It was a great find . The adjustable seat works perfectly for the two children . I would recommend it highly .\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "42000\n",
      "8 yr old likes this toy for short periods , but it gets boring for him . His parents enjoyed it more than he does .\n",
      "44000\n",
      "My son loves this toy . The toy is a good size . . but if you are going to let your kids play with it unattended , be sure to knot the string or cut it . The string is REALLY LONG . . the picture doesnt show exactly how long it really is . My son tried to swallow it . But the Puppy is cute ... it shakes its &#34; booty &#34; and head as you pull it along the floor . . fun !\n",
      "46000\n",
      "The plants in this expansion deck tend to be \" better \" than the original plants . For instance , they will have higher capacity and be more efficient than the original plants . It is n't reccomended to play a game with simply all new plants because this would make the game too easy . Or I guess if you wanted a shorter game , then this might be ok . Generally , what we do is simply mix in a few of these \" better \" plants to spice things up . It 's a fun little thing to have some new plants in the game , but if I had to choose , I 'd rather have a new double sided map to play on such as the many that are available like Italy/France , etc .\n",
      "48000\n",
      "If you have a devout Lego builder , you need doors and windows . My granddaughter like to make houses from Legos . Needed these pieces very badly , she told grandma . They filled her need and her houses look great .\n",
      "50000\n",
      "Incredible ! This is one of the few toys that my two boys ( 5yo and 2yo ) would play with together . There are lots of things to do so it keeps them busy for hours . In fact , I wish I had one of these growing up -- watching my sister play with her three-story Barbie house while I used shoe boxes for my superheroes was definitely unfair!As a parent , it 's nice that it does n't even take up that much space . The bonus is that folds up and is easy to carry . Lastly , it 's seems very durable so the batcave will survive the my boys ' rough playstyle!If you need more figures , theImaginext DC Super Friends Exclusive 5-Packis a good choice to accompany this set .\n",
      "52000\n",
      "What a cool idea ! I really like the idea of this bike . The only thing I do n't like is the place where the child is supposed to rest their feel seems a bit awkward . Hopefully my child will get used to it .\n",
      "54000\n",
      "The best Lauri Puzzle . The objects with each letter are large to see and hold by a 6 year old student .\n",
      "56000\n",
      "The Ravensburger Xoomy Animals set is a unique children 's introduction to drawing . It comes with 20 different transparent animal images which your children will be able to learn to copy/trace . 10 sheets of paper , and a black felt tip pen . The Xoomy allows you to mix and match these different images as well as magnify to create small picture or larger ones . The Xoomy is recommended for children ages 7 + however , my 5 year old daughter is quite able to use this on her own and loves it . In fact she quickly ran out of paper ( 10 is n't a lot ) so I had to get some of my basic white printer paper and cut her some more.You can also buy other stencils for it includes a boy 's pattern and girls pattern set . It does require 4 AA batteries which are not included , but other than that it is ready to go straight out of the box.Overall this is a unique toy that will help stimulate your child 's artistic imagination .\n",
      "58000\n",
      "This doll is the first baby doll I have given her . She loves taking the binky in and out of the babies mouth . It is soft and you can change the babies clothes .\n",
      "60000\n",
      "I very much like board games of all sorts , and will hold no discretion to even the simplest of games ( e.g. , Go Fish ) . If it entertains the other party members , I 'm all for it.Personally , I like the meatier games stemming from various parts of Europe , namely Germany . Which is why I was attracted to FITS . As well as being a 4-player Tetris , the game excited me more so in that it was designed by Dr. Reiner Knizia , a prolific board game designer . His games may not hold my # 1 board game title , but they are all intricately detailed and great , and are easily some of the best out there.Okay , so enough of that mumbo-jumbo . FITS is remarkably fun , even if you 've played Tetris all your life ( which I have ) . It has 4 different rounds , and the rounds become more difficult each time . Dr. Knizia makes the game interesting by making empty spaces in the tiles good and bad , depending on which round you play . The pieces themselves are a combination of Tetris and Blokus pieces ( which are themselves enhanced Tetris pieces ) and are very sturdy . They make a nice \" clink \" sound when you hit them together . The player boards are good , and they even hold the pieces inside when storing.The instructions could 've been a wee bit clearer for the first couple rounds , but my sister and I picked up quickly . And that 's the only drawback I have to the game : it 's short . With only 4 rounds , my sister and I played a full game in about 15-20 minutes , and that 's with reading the instructions and all . It could be a good thing if you 're looking for a quick game to play , and all players know how Tetris works . The extra rules are n't that hard to learn .\n",
      "62000\n",
      "These showed up and i was surprised at how hefty the guns are . These feel like sturdy products . I like a lot about them.Pros : Sturdy , works outside and inside , day and night . Accurate line of sight firing , lost of lights and noises.Con : 6 batteries - not a deal breaker but seems like a lot but there is a lot of things going on in the gun . going rechargable with these seems like the right thing to do.Kids love em , adults love em too .\n",
      "64000\n",
      "The grandkids go through these in about three month and then they need more . How can you play without a lightsaber !\n",
      "66000\n",
      "I would not recommend this toy house . After you put it together it falls apart . All the pieces , from the porch and railings to the second floor . Waste of money . The Mystery Machine is a better alternative .\n",
      "68000\n",
      "My toddler loves this set . Much better quality then I was expecting . The drill actually drills the bits which is pretty cool . Good price and will last a long time .\n",
      "70000\n",
      "This is one of my favorite toys to give to 1 year olds for their birthday . These blocks are big and easy for even very small children to build with . The wagon is also fun for storage and transporting the blocks .\n",
      "72000\n",
      "What a cool thing !! For those of us that ca n't fold a straight line , what a gem . I want to make my own sketchbooks with Arches paper so I checked out a few bookbinding videos on youtube and ordered a few things and I am good to go.This scoring board is great . It comes with an envelope tool and it has envelope sizes marked on it plus the scoring lines you need to make to get that size . It has a little slot for the envelope tool to slide in & out of the way . What a good idea and it fits snug enough that you wo n't lose it . It comes with it 's own boning tool with a little place at the top that opens up to store it . Inside the little storage spot it has sizes of cards and boxes & the lines to score to get them done . All for $ 16.00 !! I LOVE IT !!\n",
      "74000\n",
      "I bought this for my son who is 2 and an avid Thomas the Train fan . It works great in that it connects easily to his other Take N Play track pieces.The only problem I ran into with this particular pieces is that when you go by the passenger pick up station on the loop track you push the train over this kind of turning wheel thing that makes the passengers disappear from the passenger pickup station . The problem with this pieces is that the turn table thing sits so high up off the track that it almost derails the train . Also my son never just plays with one train , he is always connecting several trains together usually a good 5 or 6 together at a time . If you try to push more than one train over that turn table all the the other trains derail and become disconnected . Overall though except for this minor fault the station is great .\n",
      "76000\n",
      "This is a cute toy , and it goes well with all the M&D beach toys . My ( almost ) two year old has n't taken to this toy as much as the crab bucket and shovel , and she loves the cultivator , but maybe she 'll enjoy this one soon . It 's a little lighter in weight than I expected , but it 's cute .\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "78000\n",
      "It is another LEGOS set so you can expect high price for some plastic blocks but at least this one comes with 3 different designs .\n",
      "80000\n",
      "We first tried Left Center Right at Christmas at a relatives house and found it to be a wonderful game to be played by our whole family ages 4-87 ! Everyone really had fun with it . We liked it so much that we came home and immediately ordered one for ourselves and have played it several times with our grandchildren . This game is also one that would be great for an RV because it does n't take up hardly any space ( always a good thing in an RV ) and all ages have fun playing it . It is a fast playing game that keeps younger kids interested . I recommend this to anyone who loves playing games !\n",
      "82000\n",
      "And I love New York City too ! This Ravensburger Puzzle of the Yellow New York Taxi in a Black and White city was fun and challenging.The black and white puzzle areas present a bit more of a challenge I enjoy . I spent 5 days working on the puzzle off and on throughout each day.I love the quality of Ravensburger puzzles , the feel of the pieces in my hands , and the Softclick-Technology that allows only the right pieces to fit together and those that fit , fit well.Ravensburgher has become my favorite puzzle company !\n",
      "84000\n",
      "My 1 1/2 year old daughter likes it . It 's friendly looking and a good hug-able size for her.The doll feels durable and well made . I like that the bottom has a bean bag weight in it so that it can sit up while on a table or chair .\n",
      "86000\n",
      "Its chalk . I received 2 colored boxes instead of a white one which would have been nice . None of it was broken so thats a plus .\n",
      "88000\n",
      "This is the first Lalaloopsy that I purchased for my granddaughter , and it is still one of her favorites . She loves these dolls and has expanded her interest to the \" littles \" and other items from the lalaloopsy collections , such as clothing . She does n't mind the big heads , and enjoys dressing , and undressing them , pushing them in her doll stroller , etc . These dolls have been worth the money for us . I have not noticed a problem with the clothing , but everything seems very durable . We have had Dot for almost 2 years now , and she has told me the dolls she wants for her next birthday .\n",
      "90000\n",
      "Learning Resources is a great brand and this super sorting pie is no exception . I bought it for my 2.5 year old who loves to sort items . The pieces of the pie are sturdy : they 're plastic but not hollow like so many other pretend food items . All the pieces of the pie fit inside the pie for storage , except for the tongs . My little one is not using the tongs yet , though , so I just put them away . I like the different wheels that go in the bottom of the pie , allowing for several dimensions of play . This is a great learning toy : my only complaint is that the many pieces are a pain to pick up when they inevitably get thrown around in excitement . I like this pie so much I bought a second one for my niece .\n",
      "92000\n",
      "Great for improving spatial skills , visual skills , problem solving , and planning ahead . We love it . Fantastic game , one that you will enjoy very much as it improves your skills .\n",
      "94000\n",
      "Recently my oldest son , 5 years old , received the new LeapPad from LeapFrog to review for free along with Disney Pixar Cars 2 . The great thing about this learning tablet is that it takes the same cartridges as our Leapster Explorer . So when one boy is n't playing Disney Pixar Cars 2 on the LeapPad , he 's playing it on the Leapster Explorer.About the Game : In Disney Pixar Cars 2 , children will join Mater and Lightening McQueen in a global race-turned-espionage tale . Along the way , children will practice measurements , problem-solving , graphs , sequencing and patterns.I had to pry the game out of my son 's hands to learn more about it to share with you . So that said , you can be sure he loved it on the LeapPad . And when he was n't playing with Cars 2 on the LeapPad , my 4 year old was playing with it on the Leapster Explorer . But once I got my hands on it , I chose to play it on the LeapPad to share my thoughts with you.From the Main Menu of the game there are 5 places to visit : Adventure Mode : You drive around Radiator Springs finding collectibles . You can meet characters along the way to talk too . It lets you know when you can talk or stop somewhere . There are activities along the way , like customizing cars or keep Mater disguised using number sequencing . As you complete activities you can get badges.Activity Mode : There looks like there are eight activities in this mode - but two of ours were locked by the time I started playing . Activities in this mode include Airport Chase ( matching , fine motor skills ) , Finn 's Photo Mission ( graphing ) , Secret Decoder ( matching ) , Finn 's Jump ( fine motor skills , tracking ) , Oil Rig Chase ( matching , reading signs , fine motor skills ) , Disguise Device ( number sequencing ) . Salvage Yard : Where collectibles from Radiator Springs adventure mode are stored.Badges : The different badges earned.Micromods : I have n't ever used micromods . You download them LeapWorld to change or enhance the game somehow . It could be new game music , character upgrades , game modes or moves , etc.The only thing I was disappointed in was the packaging . I miss having a way to easily organize my games . I kept all previous Leapster Explorer Games in their boxes on a shelf , so the boys can easily find what they are looking for and put their games back . The new packaging has to be ripped apart to remove the game , leaving you with a loose game to easily misplace.That is everything I discovered ! Overall the game is pretty cute and I am impressed with the skills it 's teaching . Cartridge games for Leapster Explorer and LeapPad usually retail for $ 24.99 . If you find them less , it 's a bargain .\n",
      "96000\n",
      "My daughter has been asking for a pillow pet forever ( the ladybug in-particular ) . She finally received one for Christmas and she loves it ! It is great as a pillow or as a large stuffed animal and it is wonderful to cuddle with . I can not comment on the durability as she has only had it a few weeks . Very cute , very soft , very pleased with this pillow pet .\n",
      "Error.............................................................\n",
      "It was our twins' favorite toy from when they were about 14 months old until 24 months when lost the dumps but they still play with it once in a while... it's really cool and functional toy and helps the kids with their coordination at a younger age without making unnecessary noises or movements. %100 recommended.\n",
      "Error.............................................................\n",
      "98000\n",
      "The double bed is gorgeous . The colors are cute and go great with my \" purple \" loving daughter . I do wish the Monster High dolls would n't be so delicate but it 's a very cute set . Do n't regret buying it . Just get it ... you know you want to : )\n",
      "100000\n",
      "My kids got this last Xmas ( Dec 2011 ) , and it has been working great for over 5 months now . This one never seems to get old , as we always refill it with new candy or junk toys . What bugs me the most about this toy is the noise . I ca n't believe more people do n't mention how annoying the music can be . It is loud , and there is one song .... doot doot doodle oodle doot doo doo doot ... . I 've heard it a jillion times now . No volume control , no music off switch . It 's like the makers thought it would be a funny way to torture parents . My kids love it , as do their friends that come over . I have no issue with the quality or it breaking , just the music that plays non-stop . Other than that , 2 thumbs way up !\n",
      "102000\n",
      "I purchased this game for my 4-yr old daughter and she does n't really play it that much . It seems to be a little challenging for her so she definately chooses it over her other games . I 'm assuming its because of the age range but I would recommend this game for 5-7 year olds and not so much for the 4 year old .\n",
      "104000\n",
      "My daughter ( 12 months ) loves her Weebles . We are constantly playing with them . But not so much on this toy . The mechanism to make the bridge move is very stiff and she easily gets distracted to play with her other Weebles toys . I would not recommend this too greatly except if it is on sale and even then , think twice . The Weeble itself gives my daughter a ton of enjoyment . The playground , well , it gets ignored .\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "106000\n",
      "We bought this to replace a similar toy ( from a diff mfg ) and my daughter loves it ! It 's a good size and is easy to use and fun to play with !\n",
      "108000\n",
      "These little beauties really fly , Great for those Long Rang Attacks . They are hard to lose and very durable . Maybe a bit much for indoor use . Highly Recommended!ProsDurableHard to loseLong Range\n",
      "110000\n",
      "This Lego toy was bought for my 6 year old grandson who really enjoys the challenge of completing the construction of his lego projects independently.The Town Tripper Truck provide an excellent opportunity to follow instructions ( he can read a little but uses mostly picture clues ) , brain storm when faced with a construction problem and the satisfaction and pride experieced when the project is completed.The truck is cute when built , however , like all lego constructions it is fragile and the truck is n't meant for \" Tonka Truck \" style of play.He has kept the instructions so he may rebuild it again .\n",
      "112000\n",
      "This is an excellent , gateway abstract piece placement game . Easily taught in under two minutes , yet complex enough to get one thinking . The premise is simple . Each turn place three houses in terrain matching the card you have drawn . The two catches being that , you must place the three together and if you can place them adjacent to pieces you already have on the board , you must . Each game features a standard scoring that stays the same ( for placing pieces adjacent to special spaces ) and three ways to score that are randomly drawn from a deck of cards . In addition , the boards along with some special action tokens can be different each game , leading to a new game each time . With one expansion out already , the possibilities become endless without the rules becoming convoluted.The one drawback in the game is that it is very unforgiving at the beginning . Your first three turns can make or break your game . A bad initial placement can make it very difficult to recover . While experience can help you overcome this , there 's no help for you if you receive a bad card draw . ( Getting two or three of the same terrain in a row at the beginning can be a particularly bad situation ) . For this reason , we play with a minor variant . Each player starts with two cards instead of one , replenishing their hand each turn . This greatly reduces the luck factor and , in my opinion , greatly increases the enjoyment due to having more choice . Some argue this lengthens the game , however , in my experience the length of the game is far more dependent on the playing style of those at the table . When approached about this , the designer preferred to keep the game as it was and won the Spiel des Jahres so it 's obviously a quality game as written . In my opinion , it truly shines with the variant .\n",
      "114000\n",
      "Bought this for 2 yr old grandson . He loved the gadgets and getting pushed around . Also great for going outside . Grandson felt like abig deal opening the door , climbing in and turning the key .\n",
      "116000\n",
      "This Ravensburger \" Dragon Nursery \" puzzle is very colorful and has large kid friendly pieces that makes it just about perfect for the target age group of 3 + . We have several Ravensburger puzzles and they do tend to be very good quality , with this one meeting those expectations.The puzzle is a respectable 2x3 foot , making this best built on the floor , at least in our house . There are only 24 large pieces that make up the fun fantasy scene featuring colorful dragons and a cool wizard . The pieces are fairly sturdy and can be easily wiped off with a damp cloth when soiled.This was a big hit and it has quickly become a requested fun time activity.Highly Recommended!CFH\n",
      "118000\n",
      "These are perfect for the girl who like these type of posters . The markers are good quality.May buy another for my daughter !\n",
      "120000\n",
      "My four year old niece had a blast with the Polly Pocket Playtime Set ( hooray for alliterations ! ) . Aside from it 's bright colors , the play set has a lot of interchangeable , modular elements so you 're not stuck with one shape or design.The play set comes with a Polly doll and a pet store that comes with a slide and lift to take Polly and her pets up and down . It 's not an overly large play set , thankfully , so it sits perfectly comfortably on the kitchen table ( which may not be a good thing , actually , come dinner time ) . The only drawback is that there are lot of little parts which could easily get lost . So , if a piece is missing , check under the couches and in the vents !\n",
      "122000\n",
      "This little set has been a great addition to the castle with the other pieces that go with it . Recommend you also buy the castle with them . .\n",
      "124000\n",
      "This set was so fun to put together and is chock full of fun little unexpected extras . I really love the vampire and vampiress minifigs , and also the fact that 6 power crystals are included with this set . This is a must-have for any fan of the Monster Fighters series . Beautifully designed and super fun to play with .\n",
      "126000\n",
      "Got this for my son last year for his birthday . Got it again for a present for another little boy what kid does n't like Angry Birds ??\n",
      "128000\n",
      "I actually boaght this for 2 girls , ages 5 and 6 . I thought since it was Avengers they would still think it was cool and fun . They have spent several hours on it and wo n't let me dismantle it to store it . We had a family/friend party for the holidays and between 8-10 kids , this item was never untouched . It is a solid design ( although not too flashy ... as in no loop-de-loops ) and the cars are very durable . I have heard good things about Carrera , this is my first purchase from them and wo n't be my last .\n",
      "130000\n",
      "My twin toddlers have had this for several weeks , and for the entire time they only played w / the lion for perhaps 15 mins combined . They had no interest in the lion.On the other hand , they are still obsessed w / the balls and continue to fight over them.Although they understand the popping mechanism , they had no interests in the lion other than carrying it around like a bucket for a brief time.Overall , its a cute product , but does n't hold a toddler 's attention span long .\n",
      "132000\n",
      "Simple but beautiful , all the details of the dolls are unique . the picture does n't show the beauty of this doll . Love all the little things that doll have of Paris .\n",
      "134000\n",
      "I got this for $ 20 cheaper on Amazon than at Toys R US and I am glad because it is defintely not worth $ 50 . It 's a fun tent do n't get my wrong but has about a 2-3 month lifetime , less if you have multiple kids using it . It is structured with some type of bendy wire that is hard to shape back once a child has played with it . It 's possible , just takes forever . My son likes the different entry points but it does collapse very easily . I would buy it again because he loved it so much but would only pay $ 30 .\n",
      "136000\n",
      "Okay , I got these for super cheap and I 'm happy with them for the price I paid.They are bright and colorful and the paint job on them is pretty good . I like that the little magnets do n't stick out , so if they do get in the kids mouth I wo n't have to worry about them biting the magnet off .\n",
      "138000\n",
      "This bath toy absolutely does not float . If even a tiny amount of water gets inside ( and it is not sealed mind you so it WILL happen ) it sinks in a heartbeat ! It 's very frustrating for my three-year old and she is always asking me to \" fix \" it . To add to this toy 's not so great features , the swings fall off super easy as well AND the middle part that holds the swings up comes off pretty easily too . The only nice part is the Ariel figure . What a piece of junk . I ended up getting the \" Barbie Splash and Slide Bath Playset \" which I absolutely have zero issues with and my girl loves.Save your money and save yourself the headache and do n't get this toy !\n",
      "140000\n",
      "My daughter is having a great time with her Furby . My only compliant so far would be the difficulty we had changing the personality . For whatever reason , ours turned into the mean and evil version , which really confused my 6 year old . After a few tears and an hour of searching the internet and trying different things , we finally got her changed back to the nice version . This really should be easier , especially for younger kids .\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "142000\n",
      "This set is for my son 's birthday , so he has n't played with it yet . But , I opened it up and looked it over . The wood pieces look good and sturdy , as do the plastic accessories . This set has enough pieces to make a decent set-up , but you 'll probably want more to add to it down the road . I 'm sure my son will like it .\n",
      "144000\n",
      "This one is from new series and itis a good beginning.very well painter and sculpted , making a good impression as a toy and as a collectible as well.No extra accesories ( I 'm not counting the leg ) , but there is no need for that.Articulation points are placed smart , there is not to many or not to little.For me great piece of work , and not overpriced : )\n",
      "146000\n",
      "This is another great Safari Ltd. animal . The quality is amazing and the details are great !!! Do n't hesitate to add this dinosaur to your collection .\n",
      "148000\n",
      "Since this set works with all other Knex , the possibilities are endless . The Angry birds are cute and work well with the other parts - packaging was a bit of an overkill with everything being in an individual bag , but that 's the only negative thing about this set . Clear instructions helped my 6 yr old play right away and he has gone back to it over and over .\n",
      "150000\n",
      "I love that this plane is foam so that my son can shot it around the house without damaging furniture . It flies pretty far and has held up really well .\n",
      "152000\n",
      "The Barbie Royal Doll Giftset is a fun set for little girls ! Each of the 3 dolls have permanent clothing , so these gals are rather modest compared to most Barbies ! Although they are clothed partially in permanent items , the removable pieces of clothing are made to mix and match with each doll.A fairy skirt , and gown and a mermaid skirt/tail will keep little girls busy for quite some time ! Each doll also comes with a removable crown , and shoes that will actually stay on the doll.My nieces were delighted when I let them play with these dolls , and using the 18 + mix and match looks along with items I already had kept them busy for quite some time.This set is called a Giftset for a reason , it really does make for a wonderful gift .\n",
      "154000\n",
      "I bought this for my niece for Christmas . She 's in her 2nd year of ballet and she really loves this ballerina Barbie .\n",
      "156000\n",
      "My 5 year old son got this for Christmas from Santa this year . He 's been asking for it for months now ! This one is a little harder to change than Broc is and for obvious reasons , Jagger is much larger , yet once you get the hang of it , it 's a pretty quick change . He speaks to you , walks ( rolls ) and drives on his own , he can take certain commands from you . His voice and volume can be changed by the push of a button . Jagger came with the little bulldozer dinosaur my son named Bite . Sometimes Jagger does not.All of the Switch n Go Dinosaurs are built sturdily . These are great for children even at 2 years old and upwards of 9 or maybe even 10 . Granted a 2 year old could n't change them about but there is n't small parts that could be choked on . Except for the Atilla the Anklyosaurus because he comes with a small gumball sized wrecking ball attachment.Last night was the second night with him , and he did n't start talking or walking all by himself during the night , which was a concern of mine . There is an on/off switch on his leg . Broc 's batteries had lasted for a good 5 months of nearly daily play and he turned off on his own , therefore I think Jagger 's will last almost as long . I say almost as long because Jagger can do much more than Broc can.The price is a little steep but I think as sturdy as Jagger is built he 'll last for years which makes the price worth it . 6 months ago Broc was $ 60 then around Christmas-time he went down to $ 30 . I can see this happening to Jagger in as much time.So far my son has Broc , Silver , Jagger , Attila , Quiver , McRoar , SkySlicer , and Blister . The last 6 , he got for Christmas , which I will submit reviews for as well .\n",
      "158000\n",
      "To start , this item is labeled for ages 8 + . Follow that suggestion - it 's too complicated to assemble for younger kids.As a Super Mario themed building set , it 's not bad . Pieces generally assemble correctly and reasonably sturdily . There is certainly some learning value in the building process . There is some looming anticipation of the pending Super Mario playtime results.But then on completion , as a play toy , it seems to disappoint . Although too complicated to properly assemble for younger kids , it fails to hold attention in the older ones . And although reasonably rugged , the design and quality control result in herky-jerky crude operating mechanisms , and a constant rattle of plastic pieces bouncing off the floor ( after they have fallen or been knocked off ) . Another drawback . . . . Many of the latest generation of playhouse type toys can fold up and provide a sort of carrying case for the associated action figures and sub-parts . Not this one . It 's a floor space eater -- doomed to loss of interest in a day or two and then a subsequent scattering and loss of pieces , likely to never be reassembled again.A kid 's enthusiasm for the electronic Super Mario game does not necessarily translate into similar fun time with this toy . It did n't for my grandsons . I can not really recommend this set and certainly not at the prevailing price .\n",
      "160000\n",
      "Purchased for my 9 year old granddaughter , she loved it ! Mom said she did n't have the patience to wait and open one box a day , so it was complete before Christmas Eve . This was n't a problem , she enjoyed it . Great LEGO quality , I will definitely be buying the advent calendars each year now for three of my grandchildren .\n",
      "162000\n",
      "Excellent addition to my husband 's collection . The front crossbow piece is easy to put on and take off . It does n't serve a function but looks cool . You load 4 darts and it shoots one per pull of the trigger . Love the color scheme on the Zombie crossfire pieces . Serious power for this one . My husband shot me point blank and it hurts ! I would advise making sure players shoot each other from farther away .\n",
      "164000\n",
      "Awsome ! Fast delivery .\n",
      "166000\n",
      "This loom works well for my kids . they have three different kinds and they all do the job . This is a really good price and includes a handy carry bag and extra bands . The kids love the added charms and bands . The bands and loom are durable and work well in this family . The is the best activities of all their toys . It keeps them occupied for hours , the older children help the younger . they can watch the teaching videos on-line to learn new patterns . It is wholesome , coordination building , social , builds self esteem ... a really good activity . Good customer service , fast shipping .\n"
     ]
    }
   ],
   "source": [
    "with open(path,'r') as f:\n",
    "    content = f.readlines()\n",
    "dics = []\n",
    "for i, line in enumerate(content):\n",
    "    dic = json.loads(line)\n",
    "    dic['reviewText'] = parse(dic['reviewText'])\n",
    "    dic['summary'] = parse(dic['summary'])\n",
    "    if i % 2000 == 0:\n",
    "        print i\n",
    "        print dic['reviewText']\n",
    "    dics.append(dic)\n",
    "    #print dic\n",
    "    #print dic['reviewText']\n",
    "    #print dic['summary']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 将文件小写"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "for dic in dics:\n",
    "    dic['reviewText'] = dic['reviewText'].lower()\n",
    "    dic['summary'] = dic['summary'].lower()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 将分词后的结果写入文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "out_path = '/home/jinhanqi/tf_program/summarization_data/review_data'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(out_path+'/Toys_and_Games_5_tokenized.json',\"w\") as dump_f:\n",
    "    for dic in dics:\n",
    "        json.dump(dic,dump_f)\n",
    "        dump_f.write('\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 将数据随机打乱"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "167597"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(dics)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "random.shuffle(dics)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 写入json文件"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 写训练数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(out_path+'/Toys_and_Games_5_train.json',\"w\") as dump_f:\n",
    "    for dic in dics[0:150000]:\n",
    "        json.dump(dic,dump_f)\n",
    "        dump_f.write('\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 写验证数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(out_path+'/Toys_and_Games_5_val.json',\"w\") as dump_f:\n",
    "    for dic in dics[150000:157597]:\n",
    "        json.dump(dic,dump_f)\n",
    "        dump_f.write('\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 写测试数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(out_path+'/Toys_and_Games_5_test.json',\"w\") as dump_f:\n",
    "    for dic in dics[157597:167598]:\n",
    "        json.dump(dic,dump_f)\n",
    "        dump_f.write('\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}