GSoC Idea: Creating Quality models using GrimoireLab and CHAOSS metrics

[valeriocos]

GrimoireLab is a powerful open source platform that provides support for monitoring and in-depth analysis of software projects. It produces a rich set of dashboards, which can be easily inspected by decision makers to help them understanding the evolution and health of their projects. Despite the large set of dashboards available in GrimoireLab, comparing projects between each others is not straightforward since it requires navigating and drilling down the data in different dashboards.

The GrimoireLab module Prosoul is a web application that empowers decision makers with the means to create and manage their own quality models, which are useful means to evaluate and compare software projects. This project idea is about supporting the definition of Quality Models using GrimoireLab data and Prosoul. You will work with
Python, Django and ElasticSearch.

The aims of the project are as follows:

• Learning about data analytics in the context of open source communities.
• Understanding the GrimoireLab components (Perceval, ELK, Mordred) and the corresponding tool-chain.
• Understanding the overall approach of Prosoul.
• Designing an approach to shape GrimoireLab data in a format that can be easily consumed by Prosoul.
• Implementing the approach with GrimoireLab data obtained from git, github, mailing lists repositories to obtain simple quality models.
• Improving the Prosoul UI to simplify the management of quality models.

The aims will require working with Python, Django and ElasticSearch.

• Difficulty: Medium
• Requirements: Python programming. Interest in software analytics. Willingness to understand GrimoireLab internals.
• Recommended: Experience with ElasticSearch and Django would be convenient, but can be learned during the project.
• Mentors: @dlumbrer, @Polaris000, @sduenas, @valeriocos

Microtasks

For becoming familiar with GrimoireLab, you can start by reading some documentation. You can find useful information at:

• GrimoireLab Tutorial
• Perceval: Software Project Data at Your Will
• The GitHub repositories hosting the tools (https://github.com/chaoss/grimoirelab-perceval, https://github.com/chaoss/grimoirelab-sirmordred, https://github.com/chaoss/grimoirelab-...) and https://github.com/Bitergia/archimedes.

Once you're familiar with Grimoirelab, you can have a look at the following microtasks.

• Microtask 0:
  Download PyCharm and get familiar with it (for instance, you can follow this tutorial).

• Microtask 1:
  Set up Perceval to be executed from PyCharm.

• Microtask 2:
  Create a Python script to execute Perceval via its Python interface using the GitLab and GitHub backends. Feel free to select any target repository.

• Microtask 3:
  Based on the JSON documents produced by Perceval and its source code, try to answer the following questions:

  • What is the meaning of the JSON attribute 'timestamp'?
  • What is the meaning of the JSON attribute 'updated_on'?
  • What is the meaning of the JSON attribute 'origin'?
  • What is the meaning of the JSON attribute 'category'?
  • How many categories do the GitHub and GitLab backends have?
  • What is the meaning of the JSON attribute 'uuid'?
  • What is the meaning of the JSON attribute search_fields?
  • What is stored in the attribute data of each JSON document produced by Perceval?

• Microtask 4:
  Set up a dev environment to work on GrimoireLab. Have a look to https://github.com/chaoss/grimoirelab-sirmordred#setting-up-a-pycharm-dev-environment.

• Microtask 5:
  Execute micro-mordred to collect, enrich and visualize data from Git and GitHub repositories.

• Microtask 6:
  Using the dev tools in Kibiter, create a query that counts the number of unique authors on a Git repository from 2018-01-01 until 2019-01-01.

• Microtask 7:
  Install and use elasticdump to download the mapping and data of an ElasticSearch index (it can be anyone created in Microtask 5).

• Microtask 8:
  Set up Prosoul to be executed from PyCharm.

• Microtask 9:
  Create your first Quality Model with Prosoul. Optionally, try perform an assessment with the data at https://github.com/valeriocos/prosoul-input-data (it contains a dump of an index and its mapping) to be imported in your local instance of ElasticSearch.

• Microtask 10:
  Submit at least a PR to one of the GrimoireLab repositories to fix an issue, improve the documentation, etc.

[valeriocos]

For all students interested in this idea, please comment on this issue to get in touch with the mentors. This is the main communication channel.

[vchrombie]

Hi @valeriocos

I am interested to apply for this idea. 🙂

I am currently working on the microtasks.

Actually I started working on this idea, 287. But (out of slight confusion) I commented on 286 by mistake and I just realised it right now. 😅

Sorry for all the references. I just got panicked and messed it up all. 🙁

[valeriocos]

ok, no worries! thank you for working on this idea :)

[killerpanda101]

Hello @valeriocos

I am a B.tech student from India and an interested to apply for this idea.
I have a descent understanding of Django and a currently working on the micro-tasks
will let you know once that's done.

[valeriocos]

Hi @killerpanda101! Don't hesitate to write here if you need feedback/clarifications on the microtasks. Thanks

[jacintoj]

Hi @valeriocos

I am interested in this idea for Google Summer of Code 😃 I´m a student from Rey Juan Carlos University.
Can I start now with the microtasks or I have to wait until my student apply for GSoC?

Thanks! 👍

[valeriocos]

Hi @jacintoj !

Great! Please start with the microtasks, you should include them as part of your application. Don't hesitate to write if you need help or feedback.

Thanks!

[sp35]

Hi @valeriocos
I'm interested to apply for this idea. As stated, I'll start with the microtasks :)

[valeriocos]

Great @sp35 ! thank you for your interest in this idea!

[imnitishng]

Hi! This project strikes my interest would love to work on this, I will start off with the microtasks ASAP and update my progress here.

[valeriocos]

Hi @imnitishng ! thanks for your interest and don't hesitate to write if you need feedback/clarifications

[imnitishng]

Create a Python script to execute Perceval via its Python interface using the GitLab and GitHub backends

Don't understand this microtask.
As far as I know I see we need to call perceval on some github repo but not by the default way given on perceval readme via terminal.
How do I do the same with a "python interface" please guide.

[animeshk08]

@imnitishng . You can refer this documentation for a better understanding. In the given script a git backend is used to fetch the commits from a repository. The given microtask require us to create a similar script using GitHub and GitLab backend to retrieve the data from repositories of our choice.