Release v0.1.3 (#23)

* Test realase (#12) * File and directory creation * Adding proxy scrapper script. Result: list of elite proxy with port * Script edited. Deleted unneeded lines, deleted bs, function returns clear dataframe. * Saving to csv added * Delete duplicates script with update of list in file added * Exception FileNotFound added. Checking status code. * Result of proxy scrapper * Convert proxy to list. Files reorganization. * Header list. * ImportError fix csv file added to gitignore * Scrapping world's list from tibia.com * Scrapping character information from tibia.com. * Temporary data file * Highscores scrapper. Early version * Deleted proxy usage. Deleted unneeded code. * Deleted unneeded code. * Main page html code added. Footer changing font size. Added YouTube, Twitch buttons to nav bar * Fixing spacing, typo and content information for future work * Test news added * Database added. Creating models. * New features (#22) * File and directory creation * Adding proxy scrapper script. Result: list of elite proxy with port * Script edited. Deleted unneeded lines, deleted bs, function returns clear dataframe. * Saving to csv added * Delete duplicates script with update of list in file added * Exception FileNotFound added. Checking status code. * Result of proxy scrapper * Convert proxy to list. Files reorganization. * Header list. * ImportError fix csv file added to gitignore * Scrapping world's list from tibia.com * Scrapping character information from tibia.com. * Temporary data file * Highscores scrapper. Early version * Deleted proxy usage. Deleted unneeded code. * Deleted unneeded code. * Main page html code added. Footer changing font size. Added YouTube, Twitch buttons to nav bar * Fixing spacing, typo and content information for future work * Test news added * Database added. Creating models. * Pages/main page (#14) * Test commit * Init added to scripts * Gitignore updated * Models managed changed to true * Functions to get data from tibiadata API * Checking data from api and compare with database. If not exist, insert. * Views at main page for news ticker * Main page news ticker data loaded from database. * Added condition to selecting data for news ticker * Deleted unneeded code. * Adding condition to check if list is not empty. * Variable renamed * Getting data about boosted boss. * Boosted creature models created. * Adding boss to database. Replace function added * Display boosted boss. * Get boosted creature information * Adding boosted creature to database * Display information about boosted creature. * Fixed naming and spaces * Added getting news history * Html formating of tibia.com news. Inserting news into database. Some name changes. * Added column * Views for news * Added news on main page with testimonials * Pages/main page (#16) * Test commit * Init added to scripts * Gitignore updated * Models managed changed to true * Functions to get data from tibiadata API * Checking data from api and compare with database. If not exist, insert. * Views at main page for news ticker * Main page news ticker data loaded from database. * Added condition to selecting data for news ticker * Deleted unneeded code. * Adding condition to check if list is not empty. * Variable renamed * Getting data about boosted boss. * Boosted creature models created. * Adding boss to database. Replace function added * Display boosted boss. * Get boosted creature information * Adding boosted creature to database * Display information about boosted creature. * Fixed naming and spaces * Added getting news history * Html formating of tibia.com news. Inserting news into database. Some name changes. * Added column * Views for news * Added news on main page with testimonials * Names changed * Names changed * About page added. (#17) * .format changed to f * Typo fix * Collecting data from tibia.com about worlds, vocation, beprotection and categories with values used on tibia.com. (#18) Saved to json file. * Pages/worlds (#19) * Adding script to scrap world type values from tibia.com * Adding script to get worlds list and details from tibiadata api * Models, added auto increment * urls to worlds list added * Added loading settings from django * Added scripts for datatables to base * Added views for all worlds page * Added html page for all worlds * Orm added * Online history added, small changes in other models * creation date added * Charts deleted. Added loop for table creation with queryset data * Changed function name * Main() added for develop purpose * Charts added, fixed js inside django template with variables * Online history data for chart added. Creation history data for chart added. * JSON file with creation history added * Custom tag added. * Pages/single world (#20) * Url changed * Urls, added variable threw url * Dynamic views for single world added. * Urls added to single world * Single world site template added * Custom tag added ( iso date to y, m, d) * Import clean up * Arguments clean up * Test table added * Search button connected to views. * Search button connected to views. * Pages/find character (#21) * Get character information from api * Changed icon * Html site, display information from api. * Views for character search added. * Adding comment to unused buttons * Custom tag for data edited. * Forms file added * Updated working features.
jakubg89 · Dec 15, 2022 · 5dc2c00 · 5dc2c00
1 parent f8624c8
commit 5dc2c00
Show file tree

Hide file tree

Showing 24 changed files with 1,442 additions and 95 deletions.
diff --git a/.gitignore b/.gitignore
@@ -5,3 +5,5 @@ ignore/
 static/root/
 scripts/proxy_scrapper/proxy_list.csv
 scripts/tibiacom_scrapper/temp
+main/migrations/
+
diff --git a/README.md b/README.md
@@ -7,10 +7,10 @@ Django project that provides tools and statistics for players of Tibia.
 ## Features
 
 1. **Worlds informations.**
-    - [ ] Basic information.
+    - [x] Basic information.
     - [ ] Online time counter.
 2. **Character information.**
-    - [ ] Basic information.
+    - [x] Basic information.
     - [ ] World transfers.
     - [ ] Name changes.
 3. **Experience change list.**

diff --git a/main/forms.py b/main/forms.py
@@ -0,0 +1,3 @@
+from django import forms
+from .models import World
+
diff --git a/main/models.py b/main/models.py
@@ -8,14 +8,16 @@ class News(models.Model):
     type = models.CharField(max_length=20, blank=True, null=True)
     content = models.TextField(blank=True, null=True)
     content_html = models.TextField(blank=True, null=True)
+    date_added = models.DateTimeField()
+    news_title = models.CharField(max_length=150, blank=True, null=True)
 
     class Meta:
-        managed = False
+        managed = True
         db_table = 'news'
 
 
 class World(models.Model):
-    world_id = models.SmallIntegerField(primary_key=True)
+    world_id = models.AutoField(primary_key=True)
     name = models.CharField(max_length=45, blank=True, null=True)
     name_value = models.CharField(max_length=45, blank=True, null=True)
     pvp_type = models.CharField(max_length=30, blank=True, null=True)
@@ -25,7 +27,47 @@ class World(models.Model):
     battleye_value = models.IntegerField(blank=True, null=True)
     location = models.CharField(max_length=45, blank=True, null=True)
     location_value = models.IntegerField(blank=True, null=True)
+    creation_date = models.DateField(blank=True, null=True)
 
     class Meta:
-        managed = False
+        managed = True
         db_table = 'world'
+
+
+class Boosted(models.Model):
+    boosted_id = models.AutoField(db_column='Boosted_id', primary_key=True)  # Field name made lowercase.
+    name = models.CharField(max_length=50, blank=True, null=True)
+    image_url = models.CharField(max_length=100, blank=True, null=True)
+    type = models.CharField(max_length=15, blank=True, null=True)
+    date_time = models.DateTimeField(blank=True, null=True)
+
+    class Meta:
+        managed = True
+        db_table = 'boosted'
+
+
+class WorldOnlineHistory(models.Model):
+    id_online = models.AutoField(primary_key=True)
+    world = models.ForeignKey(World, models.DO_NOTHING)
+    players_online = models.SmallIntegerField(blank=True, null=True)
+    date = models.DateTimeField(blank=True, null=True)
+
+    class Meta:
+        managed = True
+        db_table = 'world_online_history'
+
+
+class Highscores(models.Model):
+    category = models.CharField(max_length=45, blank=True, null=True)
+    rank = models.IntegerField(blank=True, null=True)
+    name = models.CharField(max_length=45, blank=True, null=True)
+    vocation = models.CharField(max_length=45, blank=True, null=True)
+    world_id = models.CharField(max_length=45, blank=True, null=True)
+    level = models.IntegerField(blank=True, null=True)
+    value = models.BigIntegerField(blank=True, null=True)
+    date = models.DateTimeField(blank=True, null=True)
+
+    class Meta:
+        managed = True
+        db_table = 'highscores'
+
diff --git a/main/templatetags/__init__.py b/main/templatetags/__init__.py
diff --git a/main/templatetags/custom_tags.py b/main/templatetags/custom_tags.py
@@ -0,0 +1,15 @@
+from django import template
+from datetime import datetime
+
+
+register = template.Library()
+
+
+@register.filter(name='put_space')
+def put_space(text):
+    return text.replace('_', ' ')
+
+
+@register.filter(name='format_date')
+def format_date(date):
+    return datetime.strptime(date, "%Y-%m-%dT%H:%M:%SZ")
diff --git a/main/views.py b/main/views.py
@@ -1,9 +1,36 @@
 from django.shortcuts import render
-
+from django.db import connection
+from .models import News, Boosted, World, WorldOnlineHistory
+from datetime import datetime
+import datetime
+from pathlib import Path
+import os
+import json
+import pandas as pd
+import scripts.tibiadata_API.get_data as dataapi
 
 # Main page
 def main_page(request, *args, **kwargs):
-    return render(request, "sites/index.html")
+
+    # news ticker
+    latest_tickers = News.objects.filter(type='ticker').order_by('-news_id')[:3]
+
+    # boosted boss
+    boosted_boss = Boosted.objects.filter(type='boss').order_by('-boosted_id')[:1]
+
+    # boosted creature
+    boosted_creature = Boosted.objects.filter(type='creature').order_by('-boosted_id')[:1]
+
+    # news
+    latest_news = News.objects.filter(type='news').order_by('-news_id')[:4]
+
+    content = {
+        'news_ticker': latest_tickers,
+        'boosted_boss': boosted_boss,
+        'boosted_creature': boosted_creature,
+        'latest_news': latest_news,
+    }
+    return render(request, "sites/index.html", content)
 
 
 # Error 404
@@ -26,11 +53,146 @@ def sign_up(request, *args, **kwargs):
     return render(request, "sites/sign_up.html")
 
 
-# About
-def about(request, *args, **kwargs):
-    return render(request, "sites/about.html")
+# # # # # # # # # # # # Worlds # # # # # # # # # # # #
+
+# World main
+def worlds_main(request, *args, **kwargs):
+
+    # All worlds
+    all_worlds = WorldOnlineHistory.objects\
+        .select_related('world')\
+        .filter(date__gte=datetime.datetime.now() - datetime.timedelta(seconds=260))
+
+    # Load data for creation chart from json
+    path_to_json = Path(__file__).resolve().parent.parent
+    creation_chart = json.load(open(os.path.join(path_to_json, 'scripts\\json_files\\creation_chart.json')))
+
+    # Online history chart
+    get_online_history = WorldOnlineHistory.objects \
+        .select_related('world') \
+        .filter(date__gte=datetime.datetime.now() - datetime.timedelta(hours=24)) \
+        .values('world__location', 'world_id', 'players_online', 'date')
+
+    df = pd.DataFrame(get_online_history)
+    df['hour'] = df['date'].dt.round('H').dt.hour
+    df = df.sort_values(by=['date'])
+    world = df['world__location'].unique()
+    hours = df['hour'].unique()
+    online_history = {}
+    online_history_all = {}
+
+    for i in world:
+        mean_location = {}
+        for hour in hours:
+            mean_location.update({
+                hour: int(df.loc[(df['world__location'] == i) & (df['hour'] == hour), 'players_online'].mean())
+            })
+            if i == world[-1]:
+                online_history_all.update({
+                    hour: int(df.loc[df['hour'] == hour, 'players_online'].mean())
+                })
+        online_history.update({i: mean_location})
+    online_history.update({'Total': online_history_all})
+    content = {
+        'all_worlds': all_worlds,
+        'created': creation_chart,
+        'online_history': online_history
+    }
+    return render(request, "sites/worlds/all_worlds.html", content)
+
+
+# Single world
+def single_world(request, name):
+
+    # get data from api
+    world_information = dataapi.get_world_details(name)
+
+    # data for chart ( number of online players of each vocation )
+    knight, druid, paladin, sorcerer, none = 0, 0, 0, 0, 0
+    if world_information['players_online'] != 0:
+        for i in world_information['online_players']:
+            if i['vocation'] == 'Paladin' or i['vocation'] == 'Royal Paladin':
+                paladin += 1
+            elif i['vocation'] == 'Druid' or i['vocation'] == 'Elder Druid':
+                druid += 1
+            elif i['vocation'] == 'Knight' or i['vocation'] == 'Elite Knight':
+                knight += 1
+            elif i['vocation'] == 'Sorcerer' or i['vocation'] == 'Master Sorcerer':
+                sorcerer += 1
+            else:
+                none += 1
+
+    # data to dictionary
+    online_counter = {
+        'Knights': knight,
+        'Sorcerers': sorcerer,
+        'Paladins': paladin,
+        'Druids': druid,
+        'None': none
+    }
+
+    # sort by value (highest > lowest)
+    online_counter = {key: value
+                      for key, value in sorted(online_counter.items(),
+                                               key=lambda item: item[1],
+                                               reverse=True)}
+
+    # get world list with id
+    world_list = World.objects.all().values('name', 'name_value')
+
+    content = {
+        'world': world_information,
+        'online': online_counter,
+        'world_list': world_list,
+    }
+    return render(request, "sites/worlds/single_world.html", content)
+
+
+# # # # # # # # # # # # Worlds # # # # # # # # # # # #
+
+# Search character
+def search_character(request, *args, **kwargs):
+
+    # check if it's not empty
+    if request.GET:
+        query = request.GET['q']  # <input name="q"> return dictionary { q : item }
+
+        # check if it's only characters
+        if query.replace(' ', '').isalpha():
+            character_information = dataapi.get_character_info(query)
+
+            # check if exist on tibia.com
+            if character_information['character']['name'] != '':
+                exist = True
+            else:
+                character_information = f'Character {query} does not exist.'  # if not exist
+                exist = False
+
+
+        else:
+            character_information = f'Character {query} does not exist.'  # if contains numbers or special characters
+            exist = False
+
+        # if there is no data from search request
+    else:
+        character_information = ''
+        exist = False
+
+    content = {
+        'exist': exist,
+        'char': character_information,
+    }
+    return render(request, "sites/characters/search_character.html", content)
 
 
 # Discords
 def discord(request, *args, **kwargs):
     return render(request, "sites/discords.html")
+
+
+# About
+def about(request, *args, **kwargs):
+    return render(request, "sites/about.html")
+
+
+
diff --git a/scripts/__init__.py b/scripts/__init__.py
diff --git a/scripts/database/__init__.py b/scripts/database/__init__.py