Eco-Score - Agribalyse coverage: Difference between revisions
No edit summary |
|||
| (8 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
[[Category:Environment]] | |||
= Introduction = | = Introduction = | ||
| Line 15: | Line 16: | ||
{| class="wikitable" | {| class="wikitable" | ||
|- | |- | ||
! [https://fr.openfoodfacts.org/categories-properties FR products] !! 2020-07-22 !! | ! [https://fr.openfoodfacts.org/categories-properties FR products] !! 2020-07-22 !! 2020-08-28 !! 2020-10-21 | ||
|- | |- | ||
| All-products || 697812 - 100%|| | | All-products || 697812 - 100% || 708863 || 724653 | ||
|- | |- | ||
| Categories-known || 362464 - 52% || | | Categories-known || 362464 - 52% || 365522 - 52% || 373380 | ||
|- | |- | ||
| Agribalyse-food-code-known || 106549 - 15% || | | Agribalyse-food-code-known || 106549 - 15% || 113053 - 16% || 121291 | ||
|- | |- | ||
| Agribalyse-proxy-food-code-known || 1431 - 0% || | | Agribalyse-proxy-food-code-known || 1431 - 0% || 14327 - 2% || 57720 | ||
|- | |- | ||
| Agribalyse-known || 107913 - 15% || | | Agribalyse-known || 107913 - 15% || 124573 - 18% || 167746 | ||
|} | |||
{| class="wikitable" | |||
|- | |||
! [https://fr.openfoodfacts.org/popularity/top-10000-fr-scans-2019/categories-properties Top 10k FR products] !! 2020-07-22 !! 2020-07-26 !! 2020-08-28 !! 2020-10-21 | |||
|- | |||
| All-products || 9798 - 100% || 9813 - 100% || 9813 - 100% || 9821 | |||
|- | |||
| Categories-known || 9759 - 100% || 9774 - 100% || 9788 - 100% || 9800 | |||
|- | |||
| Agribalyse-food-code-known || 3696 - 38% || 3701 - 38% || 3985 - 41% || 4415 | |||
|- | |||
| Agribalyse-proxy-food-code-known || 110 - 1% || 386 - 4% || 622 - 6% || 2669 | |||
|- | |||
| Agribalyse-known || 3801 - 39% || 4041 - 41% || 4487 - 46% || 6402 | |||
|} | |} | ||
| Line 40: | Line 56: | ||
Unfortunately, in many cases, the Agribalyse categories differ from the OFF categories, in which case the Agribalyse categories have been added to the OFF taxonomy. In that case, the categories are empty and do not have products yet. | Unfortunately, in many cases, the Agribalyse categories differ from the OFF categories, in which case the Agribalyse categories have been added to the OFF taxonomy. In that case, the categories are empty and do not have products yet. | ||
== | == Implementation of products coverage metrics - May-July 2020 == | ||
In order to measure the coverage (% of products with an associated Agribalyse category), we are computing a new /categories-properties facets that shows which products have an associated Agribalyse category. | In order to measure the coverage (% of products with an associated Agribalyse category), we are computing a new /categories-properties facets that shows which products have an associated Agribalyse category. | ||
| Line 70: | Line 86: | ||
</pre> | </pre> | ||
=== How to identify potential Agribalyse proxy categories === | === How to identify and add potential Agribalyse proxy categories === | ||
'''Detailed guide in French: [https://docs.google.com/document/d/1MimfhVuFw1Xd1VRvqyduaHMiqr6eoxlOaWR_86yLr4Q/edit#heading=h.ihogua6y37an Guide d’ajout de correspondances Agribalyse - OFF] | |||
''' | |||
Ask Pierre or Stéphane for the Agribalyse list of categories | Ask Pierre or Stéphane for the [https://drive.google.com/file/d/15KrNf_RETm90lrpM-bnzaDDJbaH4gMWM/view?usp=sharing Agribalyse list of categories] | ||
* Search for groups of Agribalyse categories that correspond to different versions (e.g. flavoured, sugared etc.) of similar products | * Search for groups of Agribalyse categories that correspond to different versions (e.g. flavoured, sugared etc.) of similar products | ||
** It can be useful to look at the current list of [https://fr.openfoodfacts.org/categories-properties/agribalyse-unknown/categories categories of products for which we don't have an Agribalyse association] in order to see which of those categories could have good potential Agribalyse categories matches | ** It can be useful to look at the current list of [https://fr.openfoodfacts.org/categories-properties/agribalyse-unknown/categories categories of products for which we don't have an Agribalyse association] in order to see which of those categories could have good potential Agribalyse categories matches | ||
*** [https://fr.openfoodfacts.org/popularity/top-10000-fr-scans-2019/categories-properties/agribalyse-unknown/categories unmatched categories for Top 10k products] | |||
* Check if the environmental impact for the categories are the same or almost the same | * Check if the environmental impact for the categories are the same or almost the same | ||
* If the environmental impact is the same, look in the OFF categories taxonomy to see if we have a parent category that matches well the group of Agribalyse taxonomies | * If the environmental impact is the same, look in the OFF categories taxonomy to see if we have a parent category that matches well the group of Agribalyse taxonomies | ||
| Line 84: | Line 104: | ||
If a product does not have a category, we cannot match it to the Agribalyse categories. So it is very important to add categories to products. The more specific the category, the better chance we have that it will match either an Agribalyse category or an Agribalyse proxy category. | If a product does not have a category, we cannot match it to the Agribalyse categories. So it is very important to add categories to products. The more specific the category, the better chance we have that it will match either an Agribalyse category or an Agribalyse proxy category. | ||
=== Tools to easily add categories to hundreds of products === | |||
* [https://hunger.openfoodfacts.org/questions?type=category&countries=en%3AFrance Hunger Games] shows you category predictions that you can very easily and quickly validate if they are correct. | |||
** To specifically validate categories for which we already have an Agribalyse category, you can use the links from this file: [https://docs.google.com/spreadsheets/d/1nldTEVoVVZ_c-VgiwhzdawThq4L4f0U4E6UxYMLG0e0/edit?usp=sharing Hunger Games links to validate Agribalyse categories]</big> | |||
= Tags rules to automatically add more specific categories = | = Tags rules to automatically add more specific categories = | ||
(not coded yet) | (not coded yet) | ||
Latest revision as of 14:31, 23 February 2023
Introduction
The main factor for computing the Eco-Score of a product is the measure of its environmental impact in the Agribalyse database from ADEME (French national environmental agency).
Agribalyse contains data for 2500 categories of products (CIQUAL categories, different from the Open Food Facts categories).
This page:
- tracks the % of products in the Open Food Facts database that we can associate with an Agribalyse category
- defines a plan to improve that %
Metrics
The /categories-properties shows the products that have associated Agribalyse properties.
| FR products | 2020-07-22 | 2020-08-28 | 2020-10-21 |
|---|---|---|---|
| All-products | 697812 - 100% | 708863 | 724653 |
| Categories-known | 362464 - 52% | 365522 - 52% | 373380 |
| Agribalyse-food-code-known | 106549 - 15% | 113053 - 16% | 121291 |
| Agribalyse-proxy-food-code-known | 1431 - 0% | 14327 - 2% | 57720 |
| Agribalyse-known | 107913 - 15% | 124573 - 18% | 167746 |
| Top 10k FR products | 2020-07-22 | 2020-07-26 | 2020-08-28 | 2020-10-21 |
|---|---|---|---|---|
| All-products | 9798 - 100% | 9813 - 100% | 9813 - 100% | 9821 |
| Categories-known | 9759 - 100% | 9774 - 100% | 9788 - 100% | 9800 |
| Agribalyse-food-code-known | 3696 - 38% | 3701 - 38% | 3985 - 41% | 4415 |
| Agribalyse-proxy-food-code-known | 110 - 1% | 386 - 4% | 622 - 6% | 2669 |
| Agribalyse-known | 3801 - 39% | 4041 - 41% | 4487 - 46% | 6402 |
Facets can be combined, for instance to see the categories of products for which we don't have an Agribalyse association.
Plan to improve the Agribalyse coverage
Initial matching of Agribalyse categories to OFF categories - April-June 2020
As part of the Data4Good program, volunteers have added all the Agribalyse categories inside the OFF categories taxonomy.
When there was an exact match (the Agribalyse category exists in OFF), the agribalyse_food_code:en property was added to the existing category.
Unfortunately, in many cases, the Agribalyse categories differ from the OFF categories, in which case the Agribalyse categories have been added to the OFF taxonomy. In that case, the categories are empty and do not have products yet.
Implementation of products coverage metrics - May-July 2020
In order to measure the coverage (% of products with an associated Agribalyse category), we are computing a new /categories-properties facets that shows which products have an associated Agribalyse category.
Adding Agribalyse proxy categories in the OFF taxonomy
There are many cases where the Agribalyse categories are much more specific than the OFF categories, while that specificity is not needed to estimate environmental impacts.
This is because the Agribalyse categories are in fact the CIQUAL categories (the French national database for nutrition). For instance, many categories specify if something is flavoured, sugared, un-sugared, or with artificial sweeteners (e.g. yogurts). That distinction is very important for nutrition, but has often little or no impact on environmental impacts.
For example, CIQUAL define 4 categories for drinkable yogurts:
Boisson lactée, lait fermenté ou yaourt à boire, aromatisé, sucré Boisson lactée, lait fermenté ou yaourt à boire, nature, sucré Boisson lactée, lait fermenté ou yaourt à boire, nature, sucré, au L Casei Boisson lactée, lait fermenté ou yaourt à boire, aux fruits, sucré
But those 4 categories in Agribalyse have the same or mostly the same environmental impact.
In that case, we can associate one of the specific Agribalyse category to the broader "Drinkable yogurts" category on OFF. We call it a "proxy" Agribalyse category.
en:Drinkable yogurts agribalyse_proxy_food_code:en:19508 agribalyse_proxy_food_name:en:Dairy drink or fermented milk or yogurt, flavoured, with sugar agribalyse_proxy_food_name:fr:Boisson lactée, lait fermenté ou yaourt à boire, aromatisé, sucré
How to identify and add potential Agribalyse proxy categories
Detailed guide in French: Guide d’ajout de correspondances Agribalyse - OFF
Ask Pierre or Stéphane for the Agribalyse list of categories
- Search for groups of Agribalyse categories that correspond to different versions (e.g. flavoured, sugared etc.) of similar products
- It can be useful to look at the current list of categories of products for which we don't have an Agribalyse association in order to see which of those categories could have good potential Agribalyse categories matches
- Check if the environmental impact for the categories are the same or almost the same
- If the environmental impact is the same, look in the OFF categories taxonomy to see if we have a parent category that matches well the group of Agribalyse taxonomies
- Add the agribalyse_proxy_food_code:en: property on the parent OFF category
Adding categories or more specific categories to products
If a product does not have a category, we cannot match it to the Agribalyse categories. So it is very important to add categories to products. The more specific the category, the better chance we have that it will match either an Agribalyse category or an Agribalyse proxy category.
Tools to easily add categories to hundreds of products
- Hunger Games shows you category predictions that you can very easily and quickly validate if they are correct.
- To specifically validate categories for which we already have an Agribalyse category, you can use the links from this file: Hunger Games links to validate Agribalyse categories
Tags rules to automatically add more specific categories
(not coded yet)