Skip to content

Commit

Permalink
Add files via upload
Browse files Browse the repository at this point in the history
  • Loading branch information
@finlay-liu
finlay-liu authored Jun 20, 2023
1 parent eb011e3 commit 918d679
Showing 1 changed file with 76 additions and 52 deletions.
128 changes: 76 additions & 52 deletions competition/科大讯飞AI开发者大赛2023/基于论文摘要的文本分类与关键词抽取挑战赛_baseline.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"execution_count": 2,
"id": "9fadb9cd-fe0b-49c8-b62d-b6ae656d543c",
"metadata": {
"tags": []
Expand All @@ -23,9 +23,17 @@
"simplefilter(\"ignore\", category=ConvergenceWarning)"
]
},
{
"cell_type": "markdown",
"id": "12dcda70-7854-4dd3-b044-9978a0f63c33",
"metadata": {},
"source": [
"## TASK1"
]
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 3,
"id": "2b0cb4c6-ba3e-4be5-b6a0-c3fe821b3cfd",
"metadata": {
"tags": []
Expand All @@ -43,7 +51,7 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": 4,
"id": "829bfa2f-4e38-40bf-a2c7-127c0fbf96e5",
"metadata": {
"tags": []
Expand Down Expand Up @@ -75,95 +83,101 @@
" <th>author</th>\n",
" <th>abstract</th>\n",
" <th>Keywords</th>\n",
" <th>label</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>0</td>\n",
" <td>Monitoring Changes in Intracellular Reactive O...</td>\n",
" <td>Al-Hassan M Mustafa,Ramy Ashry,Oliver H Krämer...</td>\n",
" <td>Reactive oxygen species (ROS) are induced by s...</td>\n",
" <td>Flow cytometry; HDACi; Leukemia; ROS.</td>\n",
" <td>Accessible Visual Artworks for Blind and Visua...</td>\n",
" <td>Quero, Luis Cavazos; Bartolome, Jorge Iranzo; ...</td>\n",
" <td>Despite the use of tactile graphics and audio ...</td>\n",
" <td>accessibility technology; multimodal interacti...</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>1</td>\n",
" <td>Source Printer Classification Using Printer Sp...</td>\n",
" <td>Joshi, Sharad; Khanna, Nitin</td>\n",
" <td>The knowledge of the source printer can help i...</td>\n",
" <td>Printer classification; local texture patterns...</td>\n",
" <td>Seizure Detection and Prediction by Parallel M...</td>\n",
" <td>Li, Chenqi; Lammie, Corey; Dong, Xuening; Amir...</td>\n",
" <td>During the past two decades, epileptic seizure...</td>\n",
" <td>CNN; Seizure Detection; Seizure Prediction; EE...</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>2</td>\n",
" <td>Plasma-processed CoSn/RGO nanocomposite: A low...</td>\n",
" <td>Omelianovych, Oleksii; Larina, Liudmila L.; Oh...</td>\n",
" <td>The high cost of state-of-the-art Pt counter e...</td>\n",
" <td>Plasma reduction; Bimatalic alloy CoxSn1-x; Re...</td>\n",
" <td>Fast ScanNet: Fast and Dense Analysis of Multi...</td>\n",
" <td>Lin, Huangjing; Chen, Hao; Graham, Simon; Dou,...</td>\n",
" <td>Lymph node metastasis is one of the most impor...</td>\n",
" <td>Histopathology image analysis; computational p...</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>3</td>\n",
" <td>Immediate Antiretroviral Therapy: The Need for...</td>\n",
" <td>Mgbako, Ofole; E. Sobieszczyk, Magdalena; Olen...</td>\n",
" <td>Immediate antiretroviral therapy (iART), defin...</td>\n",
" <td>HIV; antiretroviral therapy; rapid; health equity</td>\n",
" <td>Long-Term Effectiveness of Antiretroviral Ther...</td>\n",
" <td>Huang, Peng; Tan, Jingguang; Ma, Wenzhe; Zheng...</td>\n",
" <td>In order to assess the effectiveness of the Ch...</td>\n",
" <td>HIV; ART; mortality; observational cohort stud...</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>4</td>\n",
" <td>Design and analysis of an ultra-low-power LC q...</td>\n",
" <td>Lee, Kin Keung; Bryant, Carl; Tormanen, Markus...</td>\n",
" <td>This paper presents the design of an ultra-low...</td>\n",
" <td>Varactor; Spiral inductor; Quadrature generati...</td>\n",
" <td>Real-Time Facial Affective Computing on Mobile...</td>\n",
" <td>Guo, Yuanyuan; Xia, Yifan; Wang, Jing; Yu, Hui...</td>\n",
" <td>Convolutional Neural Networks (CNNs) have beco...</td>\n",
" <td>facial affective computing; convolutional neur...</td>\n",
" <td>0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" uuid title \\\n",
"0 0 Monitoring Changes in Intracellular Reactive O... \n",
"1 1 Source Printer Classification Using Printer Sp... \n",
"2 2 Plasma-processed CoSn/RGO nanocomposite: A low... \n",
"3 3 Immediate Antiretroviral Therapy: The Need for... \n",
"4 4 Design and analysis of an ultra-low-power LC q... \n",
"0 0 Accessible Visual Artworks for Blind and Visua... \n",
"1 1 Seizure Detection and Prediction by Parallel M... \n",
"2 2 Fast ScanNet: Fast and Dense Analysis of Multi... \n",
"3 3 Long-Term Effectiveness of Antiretroviral Ther... \n",
"4 4 Real-Time Facial Affective Computing on Mobile... \n",
"\n",
" author \\\n",
"0 Al-Hassan M Mustafa,Ramy Ashry,Oliver H Krämer... \n",
"1 Joshi, Sharad; Khanna, Nitin \n",
"2 Omelianovych, Oleksii; Larina, Liudmila L.; Oh... \n",
"3 Mgbako, Ofole; E. Sobieszczyk, Magdalena; Olen... \n",
"4 Lee, Kin Keung; Bryant, Carl; Tormanen, Markus... \n",
"0 Quero, Luis Cavazos; Bartolome, Jorge Iranzo; ... \n",
"1 Li, Chenqi; Lammie, Corey; Dong, Xuening; Amir... \n",
"2 Lin, Huangjing; Chen, Hao; Graham, Simon; Dou,... \n",
"3 Huang, Peng; Tan, Jingguang; Ma, Wenzhe; Zheng... \n",
"4 Guo, Yuanyuan; Xia, Yifan; Wang, Jing; Yu, Hui... \n",
"\n",
" abstract \\\n",
"0 Reactive oxygen species (ROS) are induced by s... \n",
"1 The knowledge of the source printer can help i... \n",
"2 The high cost of state-of-the-art Pt counter e... \n",
"3 Immediate antiretroviral therapy (iART), defin... \n",
"4 This paper presents the design of an ultra-low... \n",
"0 Despite the use of tactile graphics and audio ... \n",
"1 During the past two decades, epileptic seizure... \n",
"2 Lymph node metastasis is one of the most impor... \n",
"3 In order to assess the effectiveness of the Ch... \n",
"4 Convolutional Neural Networks (CNNs) have beco... \n",
"\n",
" Keywords \n",
"0 Flow cytometry; HDACi; Leukemia; ROS. \n",
"1 Printer classification; local texture patterns... \n",
"2 Plasma reduction; Bimatalic alloy CoxSn1-x; Re... \n",
"3 HIV; antiretroviral therapy; rapid; health equity \n",
"4 Varactor; Spiral inductor; Quadrature generati... "
" Keywords label \n",
"0 accessibility technology; multimodal interacti... 0 \n",
"1 CNN; Seizure Detection; Seizure Prediction; EE... 1 \n",
"2 Histopathology image analysis; computational p... 1 \n",
"3 HIV; ART; mortality; observational cohort stud... 0 \n",
"4 facial affective computing; convolutional neur... 0 "
]
},
"execution_count": 3,
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"test.head()"
"train.head()"
]
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 5,
"id": "a4bc6c89-002e-4941-bfb9-ca7406c1ae9e",
"metadata": {
"tags": []
Expand All @@ -176,7 +190,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 6,
"id": "371f23ec-03c9-4003-8666-479359bfe6d2",
"metadata": {
"tags": []
Expand All @@ -190,7 +204,7 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 7,
"id": "b4b555b6-373d-43ae-99c8-51b85aec385a",
"metadata": {
"scrolled": true,
Expand Down Expand Up @@ -224,7 +238,7 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 8,
"id": "fd736331-1b2b-42e4-aa4c-fd6f2b8436e5",
"metadata": {
"tags": []
Expand All @@ -237,9 +251,17 @@
"test[['uuid', 'Keywords', 'label']].to_csv('submit.csv', index=None)"
]
},
{
"cell_type": "markdown",
"id": "7ce75c4d-915e-4f0b-a142-b16567fda726",
"metadata": {},
"source": [
"## TASK2"
]
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 9,
"id": "2b493756-886a-4a08-81ed-b33db69a1e72",
"metadata": {},
"outputs": [],
Expand Down Expand Up @@ -361,6 +383,8 @@
"for row in test.iterrows():\n",
" prediction_keywords = extract_keywords_by_freq(row[1].title, row[1].abstract)\n",
" prediction_keywords = [x.title() for x in prediction_keywords]\n",
" if len(prediction_keywords) == 0:\n",
" prediction_keywords = ['A', 'B']\n",
" test_words.append('; '.join(prediction_keywords))"
]
},
Expand All @@ -380,7 +404,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "ca2ebf9b-7e1c-4927-bbcf-67b6e13d3cd2",
"id": "b0627479-60ba-45ef-9902-e22abab982df",
"metadata": {},
"outputs": [],
"source": []
Expand Down

0 comments on commit 918d679

Please sign in to comment.