Added functionalities

Author: avidLearnerInProgress

memrise-scraper/memrise_mapper.txt

@@ -1,10 +1,64 @@
 abate => to decrease; reduce
+In an hour or so the storm would abate and they could leave.
+I took Laura some homemade cookies, hoping to abate her sadness.
+Taking pain medicine can cause headaches to abate.
+The crowd's roar began to abate after a few minutes of excitement.
+!-------------------!
+
 abdicate => to give up a position, right, or power
+He agreed to abdicate and retire in favor of the next in line for the throne.
+The King decided to abdicate in favor of his son.
+Edward VIII was the only British monarch who voluntarily abdicated the throne.
+He abdicated government here, by declaring us out of his protection and waging war against us.
+!-------------------!
+
 aberrant => deviating from what is normal
+Except in a few aberrant types, which are mentioned below, the mouth is surrounded by a much developed lip like a funnel directed downwards, and is armed with a horny beak not unlike that of a cuttle-fish.
+In the Malacostraca, an elongated heart with numerous segmentally arranged ostia is found only in the aberrant group of Stomatopoda and in the transitional Phyllocarida.
+They are not tales of aberrant individuals but of societal norms.
+Eyestalks and the antennules in this aberrant group correspond to the primitive head somites or not, their distinctness is certainly a secondarily acquired character, for it is not found in the larvae, nor in any of the more primitive groups of Malacostraca.
+!-------------------!
+
 abeyance => temporary suppression or suspension
+This regulation fell into abeyance after the 12th century, and such inscriptions are very rare.
+It is more probably abeyance of external function during a periodic internal assimilatory phase.
+Then the Sudan was abandoned and the railway remained in abeyance until 1905-1906, when the line was at length built.
+The last may be considered in abeyance as there has not been any coronation banquet since that of George IV.
+!-------------------!
+
 abject => miserable; pitiful
+The poor surroundings caused abject misery.
+The situation created abject terror.
+He died in abject poverty in 1961 at the age of 57 years.
+Abject humiliation followed the scene with the elders.
+!-------------------!
+
 abjure => to reject; abandon formally
+With a crusade, Ninoslav was baptized, only to abjure Christianity in 1233.
+He was asked whether certain books had been written by him and whether he was prepared to maintain or to abjure what he had written.
+At the council of Pisa, where he was compelled to abjure his errors and was sentenced to imprisonment.
+Among the propositions he could heartily abjure was that relating to transubstantiation; among those he felt constrained unflinchingly to maintain was one which had given great offence, to the effect that Christ, not Peter, is the head of the church to whom ultimate appeal must be made.
+!-------------------!
+
 abscission => the act of cutting; the natural separation of a leaf or other part of a plant
+It consists of (a) those churches which have accepted all the decrees of the first seven general councils, and have remained in full communion with one another, (b) such churches as have derived their origin from these by missionary activity, or by abscission without loss of communion.
+!-------------------!
+
 abscond => to depart secretly
+No sentences.. :( 
+!-------------------!
+
 abstemious => moderate in appetite
+He went thoroughly into the practice as well as the theory of Stoicism, and lived so abstemious and laborious a life that he injured his health.
+Browne was a man of abstemious habits, charitable disposition, and impressive eloquence.
+Abstemious in his habits, he possessed great physical endurance.
+You know me: I am busy from morning till night and abstemious, so of course I am well.
+!-------------------!
+
 abstinence => the giving up of certain pleasures
+A nearly vegetarian diet and a complete abstinence from alcoholic stimulants is the ideal in such cases, but it must be modified to suit individuals, as sometimes very strict limitations prove injurious.
+In the above countries, therefore, abstinence from matrimony may be said to have been a factor of some importance in the decline.
+With the growth of the Oxford Movement in the English Church, the practice of observing Lent was revived; and, though no rules for fasting are authoritatively laid down, the duty of abstinence is now very generally inculcated by bishops and clergy, either as a discipline or as an exercise in self-denial.
+A fourth was subsequently added, for the sake of symmetry, to make them correspond with the four seasons, and they became known as the jejunium vernum, aestivum, autumnale and hiemale, so that, to quote Pope Leo's words, "the law of abstinence might apply to every season of the year."
+!-------------------!
+

memrise-scraper/memrise_scraper.py

@@ -3,10 +3,26 @@
 import time
 
 memrise_url = "https://www.memrise.com/course/121215/barrons-800-essential-word-list-gre/"
-sentences_url = "sentence.yourdictionary.com/"
+sentences_url = "http://sentence.yourdictionary.com/"
 
 mapping = dict()
 combine = []
+glob_s = []
+
+sentence_map = dict()
+
+def get_sentences(ur):
+	url = ur
+	print(url)
+	response = req.urlopen(url)
+	data = response.read()
+	soup = BeautifulSoup(data, 'lxml')
+	sentences = []
+	for s in soup.find_all("li", class_ = 'voting_li'):
+		sentence = s.find("div", class_ = 'li_content')
+		#print(sentence.text)
+		sentences.append(sentence.text)
+	return sentences
 
 for i in range(1,81):
 	url = memrise_url + str(i) + "/"
@@ -16,14 +32,34 @@
 	for w in soup.find_all("div", class_ = "thing text-text"):
 		word = w.find("div", class_ = "col_a col text")
 		meaning = w.find("div", class_ = "col_b col text")
+		sentence = sentences_url + word.text
+		sentences = get_sentences(sentence)
+		sentences = sentences[:4]
+		
 		combine.append((word.text, meaning.text))
+		#glob_s.append(sentences)
+		sentence_map[word.text] = sentences
+
 		if word.text not in mapping:
 			mapping[word.text] = meaning.text
 			antimapping = dict((y, x) for x,y in mapping.items())
-	time.sleep(2)
-	break
+	time.sleep(3)
+
+with open("memrise_mapper.txt", 'r') as f:
+	contents = f.read()
 
 with open("memrise_mapper.txt", 'a') as mfile:
 	for ele in combine:
-		writetotxt = str(ele[0]) + " => " + str(ele[1]) + "\n"
-		mfile.write(writetotxt)
+		if ele[0] in contents:
+			print("Mapping already exists.")
+		else:
+			writetotxt = str(ele[0]) + " => " + str(ele[1]) + "\n"
+			#mfile.write("!-------------------!\n")
+			mfile.write(writetotxt)
+			if ele[0] in sentence_map.keys() and len(sentence_map[ele[0]]) != 0:
+				for s in sentence_map[ele[0]]:
+					mfile.write(s + "\n")
+				mfile.write("!-------------------!\n\n")
+			else:
+				mfile.write("No sentences.. :( \n")
+				mfile.write("!-------------------!\n\n")