Taxonomy editor: Difference between revisions

From Open Food Facts wiki
Tag: New redirect
 
 
(14 intermediate revisions by 5 users not shown)
Line 1: Line 1:
#REDIRECT [[GSOC 2022 - Taxonomy editor]]
[[Category:Taxonomy editor]]
Welcome to the Taxonomy Editor documentation.
 
This editor is a tool designed to make editing taxonomies easy for everyone.
 
== Where is it ? ==
 
* https://ui.taxonomy.openfoodfacts.org/
* Source code: https://github.com/openfoodfacts/taxonomy-editor
 
== Important links & files ==
* Taxonomy Editor page in the Mobile app Figma file (there's another web dedicated file): https://www.figma.com/design/nFMjewFAOa8c4ahtob7CAB/Mobile-App-Design-(Quentin)?node-id=3811-11147
* Empty folder on Canva for Marketing and visuals: https://www.canva.com/folder/FAF7FAwiA6E
 
== But first, what is a taxonomy ? ==
Taxonomies are at the heart of Open Food Facts in various aspects :
 
* They facilitate the identification of components such as ingredients, labels, brands, …
* They establish connections between these components and their relevant properties
* They enable the calculation of useful metrics such as nutri-score, eco-score, and allergen identification, …
You can learn more in [[Global taxonomies|Global taxonomies.]]
 
Technically, a taxonomy is represented as a DAG (directed acyclic graph) where each leaf node has at least one parent.
 
== Why is this editor important ? ==
Currently the taxonomies are stored in raw text files in our [https://github.com/openfoodfacts/openfoodfacts-server/tree/main/taxonomies repository].
 
As you can imagine, contributing to improving taxonomies requires significant effort and understanding. Moreover, manual editing can easily result in inconsistencies between taxonomies and even errors within a single taxonomy due to the lack of a format checker.
 
This is where the Taxonomy Editor comes in: it aims to help the contribution process for improving taxonomies and enhance overall quality.
 
== Contributing to taxonomies ==
Contributing to taxonomies greatly help Open Food Facts (and all applications and researchers using the database).
 
You can improve ingredients parsing in your language, translation of products informations, add categories specific to your culuture (only if there already is a significant amount of products).
 
=== Contribution guidelines ===
 
* Do not make a big edition. If you want to change a lot of entries, spread your changes across different projects. Also try to have a short timeline, it avoids having too much conflicts with other projects. It's better to propose a smaller (yet consistent) set of changes, and then continue opening a new project.
* Only add items in the taxonomy if occurrences exists in the database.
* Don't make big changes to the taxonomy structure without first discussing it with others (in #taxonomies in slack, or eventually on the forum)
* Always use english (en) as primary language for a new entry, unless no translation exists in english (cultural peculiarity)
 
'''FIXME''': add more
 
=== Contribution process ===
Contributing a set of changes will go through the following process:
 
{{#mermaid:flowchart TD
    Create[Create a project] --> Edit[edit entries]
    Edit --> Ready{Ready ?}
    Ready -->|No| Edit
    Ready -->|Yes| Export[export to a Pull Request]
    Export --> Review[Pull Request is reviewed by maintainers]
    Review --> Correct{All is ok}
    Correct -->|No| Edit
    Correct -->|Yes| Accepted[Pull Request is accepted]
    Accepted --> Release[Your changes are in the code]
    Release --> Visible[After a release changes are visible in production]
}}
 
As you can see, changes are not immediate and go under a review in a Pull Request. This is a request for changes in GitHub (the tool we use to manage source code contributions). In the future, this validation step might take place directly in taxonomy editor.
 
Note that currently when your changes go to production, they do not necessarily affect products in the database, which have to be refreshed first. This will happen on next edit or once in a while.
 
==Using Taxonomy Editor==
First you arrive at the home page where you have two options:
 
*create a new project for taxonomy editing : Start with a currently used taxonomy of your choice and begin editing.
*open an existing project for taxonomy editing : Continue your editions on a project, or collaborate with others on editing a taxonomy together.
[[File:ImageHomePage.png|thumb|alt=|400x400px]]
 
===Create a new project===
As you create a new project, you have some important informations to provide:
 
*the taxonomy you want to edit ;
*the GitHub branch name: this is a name you choose but be as explicit as you can. You can't use spaces, but you can add "_" instead ;
*a description: please try to explain what is your contribution about. Maybe you are adding synonyms to recognize an ingredient, maybe you are adding translations in a specific language, maybe you are adding wikidata links, etc. Describe what you intend to do, and why. This description is important also to maintainers who will inspect your changes before they are accepted.
 
After your project is created, you can edit nodes. But beware, if something wrong happen while importing the taxonomy, you will be notified. You should not attempt to edit a taxonomy with error, but inspect errors instead, and eventually talk with the Open Food Facts team.
 
[[File:Existing projects.png|left|thumb|400x400px|Existing projects page ]]
 
===Collaborate with others (or continue editing you project)===
To join an existing project, simply click on "OPEN EXISTING PROJECT" from the home page. You will then be directed to the existing projects page.
 
Please note that it's better to coordinate your effort with the initiator of a project. '''Don't edit a project without first asking permission'''!
 
Here, you'll find a list of projects that contributors have already initiated. Each project includes information such as the taxonomy being edited and a description outlining its goals, ensuring clarity for potential contributors. The status indicates the progress of the project:
 
*'''''OPEN''''': The project has been created, but no changes have been exported to GitHub yet
*'''''EXPORTED''''': Changes have been exported to GitHub. Normally at this point, the changes are under review by others, and contributions should only concern fixing issues.
*'''''CLOSED''''': The project has been accepted (merged), and further contributions are not possible<br />
[[File:Nodes page.png|thumb|300x300px|Nodes page : list of root nodes]]
While additional information such as the GitHub branch name and error count encountered by the parser are provided in the table, they are not essential for contributing.
 
To contribute to a project, simply click on the pencil icon!
 
===Edit the taxonomy===
First you arrive on a page listing all the root nodes, which are nodes without parents. Most probably you would want to use the search page to search for specific nodes using a simple search query.
 
As you have a list of nodes, you can choose one of them by clicking on the pencil button.
 
 
[[File:Edit a node page.png|left|thumb|400x400px|Edit a node]]
 
===Edit a node===
Once you've selected the node you wish to edit, you'll be directed to the edit page.
 
Here, you'll find the parent nodes and children of the selected node. You navigate through the taxonomy graph by clicking on their parent or child nodes.
 
Next, you'll encounter the translations for the node. The primary language of the node and any "universal" translations (if they exist) are automatically displayed. To add or check translations for a specific language, simply click on "LANGUAGES SHOWN" and select the desired languages. If "All languages" is not visible and you wish to add a universal translation, simply check the corresponding checkbox.
 
For each language, you can add a translation by clicking in the designated area. You can also adjust the order of translations within a section, allowing you to rearrange them as needed, even setting a new first translation.
 
Finally, you'll find a list of properties associated with the node. Here, you can edit existing properties, add new ones, or remove unnecessary ones. Property names must be unique and adhere to the format "name:of:the:property:lc" (where "lc" represents the language code). These names should contain only letters and numbers, excluding special characters.
[[File:Add a node.png|thumb|Add a node to the taxonomy]]
Once you've completed editing the node, remember to save your changes by clicking the blue button; otherwise you would lose your modifications.
 
===Add a node===
You can either:
 
*add a root node by going to Nodes page and clinking on the + icon. Always consider English as main language, if possible.
*even better, you can also add a child node by navigating to a node, and using the "+" icon
 
At creation time, you will have to choose a main language code (two letter code) and a name for the node.
 
 
[[File:Add a children node.png|left|thumb|400x400px|Add children to a node]]
 
===Adding relations===
If you want to put a node under a new parent, navigate to the parent node and use the "+" icon next to children, then enter main language code of the entry and it's name. The existing node will be retrieved and put under the parent node.
 
 
 
 
 
 
[[File:Export.png|thumb|400x400px|Export your changes to GitHub]]
 
===Publish your contributions===
Navigate to the export tab in the header bar to export your edited taxonomy.
 
From there, you have the option to download the file locally or create a pull request (PR), either initiating a new one or adding a commit to an existing PR, ensuring that your modifications are visible on GitHub !
 
Upon submission, your PR will undergo review, and upon approval, it will be merged. Through this process, you'll have contributed to improving the quality of the taxonomy and gave an important contribution to food transparency.
== Get in touch ==
{{Box
| 1    =  Slack channel
| 2    =  [https://openfoodfacts.slack.com/messages/C03MEM1CMJ4/ #taxonomy-editor]
}}
 
[[Category:Taxonomies]]

Latest revision as of 13:25, 15 August 2024

Welcome to the Taxonomy Editor documentation.

This editor is a tool designed to make editing taxonomies easy for everyone.

Where is it ?

Important links & files

But first, what is a taxonomy ?

Taxonomies are at the heart of Open Food Facts in various aspects :

  • They facilitate the identification of components such as ingredients, labels, brands, …
  • They establish connections between these components and their relevant properties
  • They enable the calculation of useful metrics such as nutri-score, eco-score, and allergen identification, …

You can learn more in Global taxonomies.

Technically, a taxonomy is represented as a DAG (directed acyclic graph) where each leaf node has at least one parent.

Why is this editor important ?

Currently the taxonomies are stored in raw text files in our repository.

As you can imagine, contributing to improving taxonomies requires significant effort and understanding. Moreover, manual editing can easily result in inconsistencies between taxonomies and even errors within a single taxonomy due to the lack of a format checker.

This is where the Taxonomy Editor comes in: it aims to help the contribution process for improving taxonomies and enhance overall quality.

Contributing to taxonomies

Contributing to taxonomies greatly help Open Food Facts (and all applications and researchers using the database).

You can improve ingredients parsing in your language, translation of products informations, add categories specific to your culuture (only if there already is a significant amount of products).

Contribution guidelines

  • Do not make a big edition. If you want to change a lot of entries, spread your changes across different projects. Also try to have a short timeline, it avoids having too much conflicts with other projects. It's better to propose a smaller (yet consistent) set of changes, and then continue opening a new project.
  • Only add items in the taxonomy if occurrences exists in the database.
  • Don't make big changes to the taxonomy structure without first discussing it with others (in #taxonomies in slack, or eventually on the forum)
  • Always use english (en) as primary language for a new entry, unless no translation exists in english (cultural peculiarity)

FIXME: add more

Contribution process

Contributing a set of changes will go through the following process:

As you can see, changes are not immediate and go under a review in a Pull Request. This is a request for changes in GitHub (the tool we use to manage source code contributions). In the future, this validation step might take place directly in taxonomy editor.

Note that currently when your changes go to production, they do not necessarily affect products in the database, which have to be refreshed first. This will happen on next edit or once in a while.

Using Taxonomy Editor

First you arrive at the home page where you have two options:

  • create a new project for taxonomy editing : Start with a currently used taxonomy of your choice and begin editing.
  • open an existing project for taxonomy editing : Continue your editions on a project, or collaborate with others on editing a taxonomy together.

Create a new project

As you create a new project, you have some important informations to provide:

  • the taxonomy you want to edit ;
  • the GitHub branch name: this is a name you choose but be as explicit as you can. You can't use spaces, but you can add "_" instead ;
  • a description: please try to explain what is your contribution about. Maybe you are adding synonyms to recognize an ingredient, maybe you are adding translations in a specific language, maybe you are adding wikidata links, etc. Describe what you intend to do, and why. This description is important also to maintainers who will inspect your changes before they are accepted.

After your project is created, you can edit nodes. But beware, if something wrong happen while importing the taxonomy, you will be notified. You should not attempt to edit a taxonomy with error, but inspect errors instead, and eventually talk with the Open Food Facts team.

Existing projects page

Collaborate with others (or continue editing you project)

To join an existing project, simply click on "OPEN EXISTING PROJECT" from the home page. You will then be directed to the existing projects page.

Please note that it's better to coordinate your effort with the initiator of a project. Don't edit a project without first asking permission!

Here, you'll find a list of projects that contributors have already initiated. Each project includes information such as the taxonomy being edited and a description outlining its goals, ensuring clarity for potential contributors. The status indicates the progress of the project:

  • OPEN: The project has been created, but no changes have been exported to GitHub yet
  • EXPORTED: Changes have been exported to GitHub. Normally at this point, the changes are under review by others, and contributions should only concern fixing issues.
  • CLOSED: The project has been accepted (merged), and further contributions are not possible
Nodes page : list of root nodes

While additional information such as the GitHub branch name and error count encountered by the parser are provided in the table, they are not essential for contributing.

To contribute to a project, simply click on the pencil icon!

Edit the taxonomy

First you arrive on a page listing all the root nodes, which are nodes without parents. Most probably you would want to use the search page to search for specific nodes using a simple search query.

As you have a list of nodes, you can choose one of them by clicking on the pencil button.


Edit a node

Edit a node

Once you've selected the node you wish to edit, you'll be directed to the edit page.

Here, you'll find the parent nodes and children of the selected node. You navigate through the taxonomy graph by clicking on their parent or child nodes.

Next, you'll encounter the translations for the node. The primary language of the node and any "universal" translations (if they exist) are automatically displayed. To add or check translations for a specific language, simply click on "LANGUAGES SHOWN" and select the desired languages. If "All languages" is not visible and you wish to add a universal translation, simply check the corresponding checkbox.

For each language, you can add a translation by clicking in the designated area. You can also adjust the order of translations within a section, allowing you to rearrange them as needed, even setting a new first translation.

Finally, you'll find a list of properties associated with the node. Here, you can edit existing properties, add new ones, or remove unnecessary ones. Property names must be unique and adhere to the format "name:of:the:property:lc" (where "lc" represents the language code). These names should contain only letters and numbers, excluding special characters.

Add a node to the taxonomy

Once you've completed editing the node, remember to save your changes by clicking the blue button; otherwise you would lose your modifications.

Add a node

You can either:

  • add a root node by going to Nodes page and clinking on the + icon. Always consider English as main language, if possible.
  • even better, you can also add a child node by navigating to a node, and using the "+" icon

At creation time, you will have to choose a main language code (two letter code) and a name for the node.


Add children to a node

Adding relations

If you want to put a node under a new parent, navigate to the parent node and use the "+" icon next to children, then enter main language code of the entry and it's name. The existing node will be retrieved and put under the parent node.




Export your changes to GitHub

Publish your contributions

Navigate to the export tab in the header bar to export your edited taxonomy.

From there, you have the option to download the file locally or create a pull request (PR), either initiating a new one or adding a commit to an existing PR, ensuring that your modifications are visible on GitHub !

Upon submission, your PR will undergo review, and upon approval, it will be merged. Through this process, you'll have contributed to improving the quality of the taxonomy and gave an important contribution to food transparency.

Get in touch

Slack channel