Skip to content

Commit bf75db5

Browse files
method to to dump reader easily (#255)
1 parent 3d82022 commit bf75db5

File tree

3 files changed

+13
-4
lines changed

3 files changed

+13
-4
lines changed

README.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,9 +14,9 @@ An End-To-End Closed Domain Question Answering System.
1414

1515
## cdQA in details
1616

17-
If you are interested in understanding how the system works and it is implemented, we wrote an [article on Medium](https://towardsdatascience.com/how-to-create-your-own-question-answering-system-easily-with-python-2ef8abc8eb5) with a high-level explanation.
17+
If you are interested in understanding how the system works and its implementation, we wrote an [article on Medium](https://towardsdatascience.com/how-to-create-your-own-question-answering-system-easily-with-python-2ef8abc8eb5) with a high-level explanation.
1818

19-
We also made a presentation during the \#9 NLP Breakfast organised by [Feedly](feedly.com). You can check out the video of the presentation [here](https://blog.feedly.com/nlp-breakfast-9-closed-domain-question-answering/).
19+
We also made a presentation during the \#9 NLP Breakfast organised by [Feedly](feedly.com). You can check it out [here](https://blog.feedly.com/nlp-breakfast-9-closed-domain-question-answering/).
2020

2121
## Table of Contents <!-- omit in toc -->
2222

@@ -130,6 +130,10 @@ cdqa_pipeline = QAPipeline(model='bert_qa_vGPU-sklearn.joblib')
130130
cdqa_pipeline.fit_reader('path-to-custom-squad-like-dataset.json')
131131
```
132132

133+
Save the reader model after the fine-tune:
134+
```python
135+
cdqa_pipeline.dump_reader('path-to-save-bert-reader.joblib')
136+
``
133137
### Making predictions
134138

135139
To get the best prediction given an input query:

cdqa/pipeline/cdqa_sklearn.py

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -179,7 +179,7 @@ def predict(self, X=None, return_logit=False, n_predictions=None):
179179
)
180180

181181
def to(self, device):
182-
""" Send reade to CPU if device=='cpu' or to GPU if device=='cuda'
182+
""" Send reader to CPU if device=='cpu' or to GPU if device=='cuda'
183183
"""
184184
if device not in ("cpu", "cuda"):
185185
raise ValueError("Attribute device should be 'cpu' or 'cuda'.")
@@ -202,6 +202,11 @@ def cuda(self):
202202
self.reader.device = torch.device("cuda")
203203
return self
204204

205+
def dump_reader(self, filename):
206+
""" Dump reader model to a .joblib object
207+
"""
208+
joblib.dump(self.reader, filename)
209+
205210
@staticmethod
206211
def _expand_paragraphs(df):
207212
# Snippet taken from: https://stackoverflow.com/a/48532692/11514226

setup.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ def read(file):
88

99
setup(
1010
name="cdqa",
11-
version="1.1.3c",
11+
version="1.1.4a",
1212
author="Félix MIKAELIAN, André FARIAS, Matyas AMROUCHE, Olivier SANS, Théo NAZON",
1313
description="An End-To-End Closed Domain Question Answering System",
1414
long_description=read("README.md"),

0 commit comments

Comments
 (0)