GSOC/2023 ideas list: Difference between revisions

From Open Food Facts wiki
No edit summary
(added Impact and contribution metrics for Open Food Facts)
Line 2: Line 2:


= Server-side =
= Server-side =
<span id="a-new-api-simplify-and-unify-the-open-food-facts-api-used-by-200-apps"></span>
<span id="a-new-api-simplify-and-unify-the-open-food-facts-api-used-by-200-apps"></span>
=== A New API: Simplify and unify the Open Food Facts API used by 200+ apps ===
=== A New API: Simplify and unify the Open Food Facts API used by 200+ apps ===
<span id="description"></span>
<span id="description"></span>
==== Description ====
==== Description ====


Open Food Facts provides a free and open API to read, search and write products data and photos. This API powers more than 200 apps developed by 3rd parties, and is also used by many tools like our AI platform Robotoff and our power contribution tool Hunger Games. Our API is unfortunately very difficult to understand and use as it evolved organically in the last 10 years. It's not 1 API in fact, it's several APIs to do different things, and we have multiple versions of those APIs, each with a different state of features and documentation. This project aims to bring &quot;A New API&quot; to our many reusers. An API that is simple, unified, easy to understand and use, well documented (with OpenAPI) and well tested.
Open Food Facts provides a free and open API to read, search and write products data and photos. This API powers more than 200 apps developed by 3rd parties, and is also used by many tools like our AI platform Robotoff and our power contribution tool Hunger Games. Our API is unfortunately very difficult to understand and use as it evolved organically in the last 10 years. It's not 1 API in fact, it's several APIs to do different things, and we have multiple versions of those APIs, each with a different state of features and documentation. This project aims to bring &quot;A New API&quot; to our many reusers. An API that is simple, unified, easy to understand and use, well documented (with OpenAPI) and well tested.<span id="expected-outcomes"></span>
 
<span id="expected-outcomes"></span>
==== Expected outcomes ====
==== Expected outcomes ====


Line 28: Line 24:
* Taxonomy read
* Taxonomy read
* User creation and profile edition
* User creation and profile edition
* Authentification
* Authentification<span id="project-information"></span>
 
<span id="project-information"></span>
==== Project information ====
==== Project information ====
* repository: [https://github.com/openfoodfacts/openfoodfacts-server <u>https://github.com/openfoodfacts/openfoodfacts-server</u>]
* repository: [https://github.com/openfoodfacts/openfoodfacts-server <u>https://github.com/openfoodfacts/openfoodfacts-server</u>]
Line 41: Line 35:
<span id="bring-taxonomy-editor-to-a-fully-usable-tool"></span>
<span id="bring-taxonomy-editor-to-a-fully-usable-tool"></span>
=== Bring Taxonomy editor to a fully usable tool ===
=== Bring Taxonomy editor to a fully usable tool ===
<span id="description-1"></span>
<span id="description-1"></span>
==== Description ====
==== Description ====
Line 51: Line 44:
While effective for the application, this format is quite cumbersome to edit for contributors.
While effective for the application, this format is quite cumbersome to edit for contributors.


Taxonomy editor was started during last Google Summer of code. Thanks to the work of Aadarsh, we got a first demonstration of the product, and it’s been in continuous development since then. But a boost would be needed to make it a really usable project.
Taxonomy editor was started during last Google Summer of code. Thanks to the work of Aadarsh, we got a first demonstration of the product, and it’s been in continuous development since then. But a boost would be needed to make it a really usable project.<span id="expected-outcomes-1"></span>
 
<span id="expected-outcomes-1"></span>
==== Expected outcomes ====
==== Expected outcomes ====


Line 60: Line 51:
From there, according to the pace of the projects many more features could be added which would be of great benefit: embarked assistance integrating tools (eg. searching / linking to wikidata, or wiktionary), integration of the products database (eg. seeing which entries need to be added or translated first), navigation in the taxonomy.
From there, according to the pace of the projects many more features could be added which would be of great benefit: embarked assistance integrating tools (eg. searching / linking to wikidata, or wiktionary), integration of the products database (eg. seeing which entries need to be added or translated first), navigation in the taxonomy.


An important aspect: as this tool is meant to live a long time quality and maintainability of the project is important.
An important aspect: as this tool is meant to live a long time quality and maintainability of the project is important.<span id="project-information-1"></span>
 
<span id="project-information-1"></span>
==== Project information ====
==== Project information ====
* repository: [https://github.com/openfoodfacts/taxonomy-editor/ <u>https://github.com/openfoodfacts/taxonomy-editor/</u>]
* repository: [https://github.com/openfoodfacts/taxonomy-editor/ <u>https://github.com/openfoodfacts/taxonomy-editor/</u>]
* Slack channels: #taxonomy-editor
* Slack channels: #taxonomy-editor
Line 71: Line 58:
* Project duration: 350 hours
* Project duration: 350 hours
* Skills required: Reactjs / Python / Neo4j
* Skills required: Reactjs / Python / Neo4j
* Difficulty rating: Medium
* Difficulty rating: Medium<span id="improve-our-producer-platform-to-the-next-level"></span>
 
<span id="improve-our-producer-platform-to-the-next-level"></span>
=== Improve our producer platform to the next level ===
=== Improve our producer platform to the next level ===


Line 83: Line 68:
Thanks to the support of the national French public health agency Santé Publique France, the platform is already alive and working. Though it is a bit rough for our users that are not tech savvy, also there are a lot of features which could help adoption and lessen the support needed to operate it.
Thanks to the support of the national French public health agency Santé Publique France, the platform is already alive and working. Though it is a bit rough for our users that are not tech savvy, also there are a lot of features which could help adoption and lessen the support needed to operate it.


Technically the producer platform uses the same code base as the main website.
Technically the producer platform uses the same code base as the main website.<span id="expected-outcomes-2"></span>
 
<span id="expected-outcomes-2"></span>
==== Expected outcomes ====
==== Expected outcomes ====


The project will bring many new features to the producer platform. Here are some of the current priorities: add more inline help, make UX more convenient, support advanced updates patterns on fields, add a workflow to product edition, connect to our internal CRM (allow automating many tasks), collect more statistics on usage, support more fields and format for data import, more robust import/publishing workflows and add more tests.
The project will bring many new features to the producer platform. Here are some of the current priorities: add more inline help, make UX more convenient, support advanced updates patterns on fields, add a workflow to product edition, connect to our internal CRM (allow automating many tasks), collect more statistics on usage, support more fields and format for data import, more robust import/publishing workflows and add more tests.


Extended goals include adding rules to help producers make products with better Eco-score, advanced features to make editing products easier (like picture upload).
Extended goals include adding rules to help producers make products with better Eco-score, advanced features to make editing products easier (like picture upload).<span id="project-information-2"></span>
 
<span id="project-information-2"></span>
==== Project information ====
==== Project information ====
* presentation of the platform: https://world.pro.openfoodfacts.org/
* presentation of the platform: https://world.pro.openfoodfacts.org/
* repository: https://github.com/openfoodfacts/openfoodfacts-server/
* repository: https://github.com/openfoodfacts/openfoodfacts-server/
Line 104: Line 83:
* Difficulty rating: Medium
* Difficulty rating: Medium


<span id="data-quality-moderation-hub"></span>
=== Data Quality &amp; Moderation Hub ===
=== Data Quality &amp; Moderation Hub ===
<span id="description-3"></span>
<span id="description-3"></span>
==== Description ====
==== Description ====
Line 113: Line 89:
Open Food Facts infrastructure is currently focused on reflecting the current state of the database, which is its primary goal. However, it would be beneficial to introduce a queue/backlog system to coordinate the efforts of users in data cleaning and moderation.
Open Food Facts infrastructure is currently focused on reflecting the current state of the database, which is its primary goal. However, it would be beneficial to introduce a queue/backlog system to coordinate the efforts of users in data cleaning and moderation.


The system currently provides some level of coordination through the facets, indicating missing nutritional data and providing a list of products that need completion. However, this system lacks flexibility.
The system currently provides some level of coordination through the facets, indicating missing nutritional data and providing a list of products that need completion. However, this system lacks flexibility.<span id="expected-outcomes-3"></span>
 
<span id="expected-outcomes-3"></span>
==== Expected outcomes ====
==== Expected outcomes ====


Line 130: Line 104:


The next steps would involve supporting questions from users and allowing moderators to answer them. This would include:
The next steps would involve supporting questions from users and allowing moderators to answer them. This would include:
* Extending the database to save questions
* Extending the database to save questions
* Send emails with the answer from moderation interface
* Send emails with the answer from moderation interface
* Add buttons in main website to send questions
* Add buttons in main website to send questions<span id="project-information-3"></span>
 
<span id="project-information-3"></span>
==== Project information ====
==== Project information ====
* repository: https://github.com/openfoodfacts/hunger-games/
* repository: https://github.com/openfoodfacts/hunger-games/
* Slack channels: #hunger-games #robotoff #data-quality
* Slack channels: #hunger-games #robotoff #data-quality
Line 145: Line 113:
* Project duration: 350 hours
* Project duration: 350 hours
* Skills required: PostgreSQL, Python, Frontend (React/ReactNative or Flutter)
* Skills required: PostgreSQL, Python, Frontend (React/ReactNative or Flutter)
* Difficulty rating: Medium
* Difficulty rating: Medium<span id="impact-metrics-for-open-food-facts"></span>
=== Impact and contribution metrics for Open Food Facts ===
 
==== Description ====
Open Food Facts is the hub of a lot of interactions: visitors, various re-use, contributions, data import, machine learning predictions and human validation, moderation, etc. We currently have different systems interacting together, but we strive at having precise and reliable real time indicators. We would like to improve metrics (specially on contributions) to be able to measure our global impact as well as news coverage or new features impacts but also to fuel other important features like gamification or moderation.
 
==== Expected Outcomes ====
The project is a bit of a patchwork of elements to assemble together but a very clear goal:
 
* fix and upgrade openfoodfacts-events project to be really usable, it will be a very useful data source. Also extends its use in the existing tools, if relevant.
* make openfoodfacts-server logs more significant events to either influxdb or to openfoodfacts-events
* write and run scripts to import data from previous years (as much as possible)
* build flexible and informative dashboards for the project
* (stretch goal 1) bootstrap the gamification
* (stretch goal 2) build a personal contributor dashboard
 
==== Project information ====
As the project involves several competences you will need to be a fast learner able to quickly crawl an existing project and documentations. Also you have to be motivated by the ultimate goal of all those metrics to be able to grasp the righ way of doing them.


<span id="impact-metrics-for-open-food-facts"></span>
* main repository: https://github.com/openfoodfacts/openfoodfacts-events
== Impact metrics for Open Food Facts ==
* secondary repositories: https://github.com/openfoodfacts/openfoodfacts-server https://github.com/openfoodfacts/openfoodfacts-infrastructure/tree/develop/docker/metrics 
Description coming soon
* Slack channels: #productopener #gamification
* Potential mentors: Vova, Alex, Stephane, Pierre
* Project duration: 350 hours
* Skills required (some might be learned): PostgreSQL, Python, InfluxDB,  Grafana, Perl
* Difficulty rating: Easy
<span id="mobile-side"></span>
<span id="mobile-side"></span>
= Mobile-side =
= Mobile-side =
Coming soon
Coming soon

Revision as of 16:51, 13 February 2023

Ideas for GSOC 2023 will be posted here. There are just ideas, and are non limitative.

Server-side

A New API: Simplify and unify the Open Food Facts API used by 200+ apps

Description

Open Food Facts provides a free and open API to read, search and write products data and photos. This API powers more than 200 apps developed by 3rd parties, and is also used by many tools like our AI platform Robotoff and our power contribution tool Hunger Games. Our API is unfortunately very difficult to understand and use as it evolved organically in the last 10 years. It's not 1 API in fact, it's several APIs to do different things, and we have multiple versions of those APIs, each with a different state of features and documentation. This project aims to bring "A New API" to our many reusers. An API that is simple, unified, easy to understand and use, well documented (with OpenAPI) and well tested.

Expected outcomes

We started to build a new API v3 to provide a clean, generic and unified way to query all API features of Open Food Facts. This new API v3 is currently available for only a very small subsets of our existing APIs: product read, a very small subset of product write, and taxonomy suggestions.

This project is about revisiting one by one all existing APIs to:

  1. Decide which feature should be added, changed (or maybe even removed)
  2. Define the API for each feature (e.g. method, path, parameters, query and response headers and bodies etc.) and document it using OpenAPI. (This step is critical to ensure that the API will be both useful and used, and will need to be done in coordination with existing users of the API.)
  3. Implement the API for the feature and add integration tests for it

APIs that needs to be converted to API v3:

  • Product image upload, selection and crop
  • Full support for product write (should be done incrementally)
  • Search
  • Taxonomy read
  • User creation and profile edition
  • Authentification

Project information

Bring Taxonomy editor to a fully usable tool

Description

Taxonomies are at the heart of Open Food Facts in many aspects. It helps identify components (ingredients, labels, brands,…) and link them to useful properties, at the base of nutri-score, eco-score, allergens identification and some other properties.

Each taxonomy is a DAG (directed acyclic graph) where leaves have one or more parents. Currently the taxonomy is in a raw text file in our repository: https://github.com/openfoodfacts/openfoodfacts-server/tree/main/taxonomies.

While effective for the application, this format is quite cumbersome to edit for contributors.

Taxonomy editor was started during last Google Summer of code. Thanks to the work of Aadarsh, we got a first demonstration of the product, and it’s been in continuous development since then. But a boost would be needed to make it a really usable project.

Expected outcomes

Minimum feature set to be added would be: good inline documentation to on board users, an ergonomic UI, searching for potential issues in the taxonomy, tracking modification to enable undoing changes and comparison to original taxonomy, helping find entries to translate. The idea is to enable a wide public of contributors to participate in adding translation or simple items (like ingredients) while ensuring quality and helping moderators validate changes. Another important technical feature will be to augment significantly test coverage (at least on backend part).

From there, according to the pace of the projects many more features could be added which would be of great benefit: embarked assistance integrating tools (eg. searching / linking to wikidata, or wiktionary), integration of the products database (eg. seeing which entries need to be added or translated first), navigation in the taxonomy.

An important aspect: as this tool is meant to live a long time quality and maintainability of the project is important.

Project information

Improve our producer platform to the next level

Description

One lesser known but very important aspect of Open Food Facts is that, not only do we encourage citizens to contribute to the database (crowdfunding) but we also propose producers to do so. The incentive for them is to ensure they provide up to date and accurate information, that their products are well represented (eg. with studio photography). They also get hints on data quality to make their product better, which is an important goal to Open Food Facts.

Thanks to the support of the national French public health agency Santé Publique France, the platform is already alive and working. Though it is a bit rough for our users that are not tech savvy, also there are a lot of features which could help adoption and lessen the support needed to operate it.

Technically the producer platform uses the same code base as the main website.

Expected outcomes

The project will bring many new features to the producer platform. Here are some of the current priorities: add more inline help, make UX more convenient, support advanced updates patterns on fields, add a workflow to product edition, connect to our internal CRM (allow automating many tasks), collect more statistics on usage, support more fields and format for data import, more robust import/publishing workflows and add more tests.

Extended goals include adding rules to help producers make products with better Eco-score, advanced features to make editing products easier (like picture upload).

Project information

Data Quality & Moderation Hub

Description

Open Food Facts infrastructure is currently focused on reflecting the current state of the database, which is its primary goal. However, it would be beneficial to introduce a queue/backlog system to coordinate the efforts of users in data cleaning and moderation.

The system currently provides some level of coordination through the facets, indicating missing nutritional data and providing a list of products that need completion. However, this system lacks flexibility.

Expected outcomes

Currently, robots send messages to Slack channels that the admins need to clean. A queue of remaining problems to moderate would simplify the workload for the team and make it easier to track progress. It would also allow users to request moderation or ask questions.

In a perfect world, users would be able to create their own tasks, such as "extract nutritional table," and associate barcodes with each step.

    • select nutritional table image
    • crop nutritional table image
    • fill nutriments

The MVP would include a back-end with an API for moderation, providing a table with information such as barcode, image_id, type_of_problem, author (including Robotoff), and moderation status (has_been_moderated). The front-end would allow users to delete pictures and mark elements as moderated. The next steps would involve supporting questions from users and allowing moderators to answer them. This would include:

  • Extending the database to save questions
  • Send emails with the answer from moderation interface
  • Add buttons in main website to send questions

Project information

  • repository: https://github.com/openfoodfacts/hunger-games/
  • Slack channels: #hunger-games #robotoff #data-quality
  • Potential mentors: Raphael, Charles, Pierre, Alex
  • Project duration: 350 hours
  • Skills required: PostgreSQL, Python, Frontend (React/ReactNative or Flutter)
  • Difficulty rating: Medium

Impact and contribution metrics for Open Food Facts

Description

Open Food Facts is the hub of a lot of interactions: visitors, various re-use, contributions, data import, machine learning predictions and human validation, moderation, etc. We currently have different systems interacting together, but we strive at having precise and reliable real time indicators. We would like to improve metrics (specially on contributions) to be able to measure our global impact as well as news coverage or new features impacts but also to fuel other important features like gamification or moderation.

Expected Outcomes

The project is a bit of a patchwork of elements to assemble together but a very clear goal:

  • fix and upgrade openfoodfacts-events project to be really usable, it will be a very useful data source. Also extends its use in the existing tools, if relevant.
  • make openfoodfacts-server logs more significant events to either influxdb or to openfoodfacts-events
  • write and run scripts to import data from previous years (as much as possible)
  • build flexible and informative dashboards for the project
  • (stretch goal 1) bootstrap the gamification
  • (stretch goal 2) build a personal contributor dashboard

Project information

As the project involves several competences you will need to be a fast learner able to quickly crawl an existing project and documentations. Also you have to be motivated by the ultimate goal of all those metrics to be able to grasp the righ way of doing them.

Mobile-side

Coming soon