Jump to content

Data fields: Difference between revisions

Data fields (view source)

Revision as of 15:28, 27 February 2024

9,830 bytes added ,  27 February
Nutrition: Entering values: expand the trace-amounts entry guidance
VisualWikitext
(→‎Serving size: serving_quantity)
(Nutrition: Entering values: expand the trace-amounts entry guidance)
 
(24 intermediate revisions by 5 users not shown)
Line 1: Line 1:
== Data fields ==
A '''data field''' is a structured information that has at least a specific usage. For example, the "product name" field allows us to easily recognize the main name printed on the packaging.
 
A "data field" is a structured information that has at least a specific usage. For example, the "product name" field allows us to easily recognize the main name printed on the packaging.


Open Food Facts manages different kinds of data fields:
Open Food Facts manages different kinds of data fields:
Line 8: Line 6:
# fields that are sometimes computed based on other fields, such as the Nutri-Score, the Nova score, etc.
# fields that are sometimes computed based on other fields, such as the Nutri-Score, the Nova score, etc.


=== Fields completed by users ===
This page deals only with "Fields completed by users". All these fields can be entered or modified by hand by the users.
All these fields can entered or modified by hand by the users.
 
[to be completed]
== Product characteristics ==


==== Product name ====
=== Product name ===
The product name is the main name printed on the packaging. It can be a registered trademark such as Nutella. This data is important and useful as it's one of the most used data.
The product name is the main name '''printed on the packaging'''. It can be a registered trademark such as Nutella. This data is important and useful as it's one of the most used data.


If it's not a part of the name, it shouldn't contain the number of portions or a quantity: bad examples are "1 Onglet", "10 Burgers", "1L Sirop cerise"; "100% Cacao", "1848 Lait noisette" (name of a product); it shouldn't contain registered trademark symbols ®, HTML code such as <code>&amp;quot;</code>; it shouldn't be in capital letters except if they are used on the product; it shouldn't contain brands except if it's included in the name ("Kinder Bueno" is good while "Kronembourg 1664", "Stella" or "Vodka Smirnoff" are not); it shouldn't contain price.
If it's not a part of the name, it shouldn't contain the number of portions or a quantity: bad examples are "1 Onglet", "10 Burgers", "1L Sirop cerise"; "100% Cacao"; it shouldn't contain registered trademark symbols ®, HTML code such as <code>&amp;quot;</code>; it shouldn't be in capital letters except if they are used on the product; it shouldn't contain brands except if it's included in the name ("Kinder Bueno" is good while "Kronembourg 1664", "Stella" or "Vodka Smirnoff" are not); it shouldn't contain price.


At the beginning of 2020, more than 95% of Open Food Facts products have a product name:
At the beginning of 2020, more than 95% of Open Food Facts products have a product name:
Line 23: Line 21:
The product name shouldn't include any other information such as the brand of the product, the weight, etc.
The product name shouldn't include any other information such as the brand of the product, the weight, etc.


Good examples:
In the database, the technical name for this field is <code>product_name</code>.
 
==== Good examples ====
* <code>Nesquick</code> ([https://world.openfoodfacts.org/product/3033710065967/nesquik-poudre-cacaotee-boite-nestle link])
* <code>Nesquick</code> ([https://world.openfoodfacts.org/product/3033710065967/nesquik-poudre-cacaotee-boite-nestle link])
* <code>111 Filini</code> ([https://world.openfoodfacts.org/product/8008845071119/111-filini-maltagliati link])
* <code>Yazoo strawberry</code> ([https://world.openfoodfacts.org/product/5410438044845/yazoo-strawberry link]) => in this example, "strawberry" has a lower font size than "Yazoo" but the flavour here distinguishes between other products, so we can consider it is part of the name


Bad examples:
==== Bad examples ====
* <code>Petit déjeuner Nesquick</code> => you don't have to explain, just put the name from the packaging
* <code><u>Petit déjeuner</u> Nesquick</code> => you don't have to explain, just put the name from the packaging
* <code>Nutella by Ferrero</code> => you shouldn't fill the brand here, there's a field for that :)
* <code>Nutella <u>by Ferrero</u></code> => you shouldn't fill the brand here, there's a field for that :)
* <code>Nesquick<u>®</u></code> => Don't use symbols ®, ™, © or similar in product name data field or even in other fields.


In the database, the technical name for this field is <code>product_name</code>.
=== Common name ===
[[File:Coca-Cola cherry example.jpg|thumb|Example in Coca-Cola Cherry: '''"Sparkling Cherry Flavour Soft Drink with Vegetables Extracts"'''. This info should be in "Common name" filed.]]


==== Common name ====
The common name defines the product. It is the name used when you don't want or can't use the product name. This is the place where you say <code>Cocoa and hazelnuts spreads</code> instead of <code>Nutella</code>. This name is very useful for our AI (artificial intelligence): it helps to guess the category of the product.
The common name defines the product. It is the name used when you don't want or can't use the product name. This is the place where you say <code>Cocoa and hazelnuts spreads</code> instead of <code>Nutella</code>. This name is very useful for our AI (artificial intelligence): it helps to guess the category of the product.


Line 40: Line 42:
In the database, the technical name for this field is <code>generic_name</code>.
In the database, the technical name for this field is <code>generic_name</code>.


==== Quantity ====
=== Quantity ===
This is the quantity of the product, with the corresponding number of portions or unit. The best way to fill it is to enter the value as indicated on the product. Don't forget the units! If we can deduce the quantity in grams it can be used to calculate some things such as the carbon impact.
This is the quantity of the product, with the corresponding number of portions or unit. The best way to fill it is to enter the value as indicated on the product. Don't forget the units! If we can deduce the quantity in grams it can be used to calculate some things such as the carbon impact.


Line 51: Line 53:
A complete wiki page is dedicated to [[products quantities]].
A complete wiki page is dedicated to [[products quantities]].


In the database, the technical name for this field is <code>quantity</code>.
In the database, the technical name for this field is <code>quantity</code>. If the system recognizes the product's quantity, it stores it as a numeric value in the <code>product_quantity</code> field: e.g. <code>quantity</code> = <code>10 * 9.8 ml</code> will lead to <code>product_quantity</code> = <code>98</code>.


See [https://world.openfoodfacts.org/data-quality/quantity-not-recognized quantities that are not recognized].
See [https://world.openfoodfacts.org/data-quality/quantity-not-recognized quantities that are not recognized].
Line 57: Line 59:
See [https://github.com/openfoodfacts/openfoodfacts-server/issues?q=is%3Aissue+is%3Aopen+quantity+label%3Aquantity issues related to <code>quantity</code>].
See [https://github.com/openfoodfacts/openfoodfacts-server/issues?q=is%3Aissue+is%3Aopen+quantity+label%3Aquantity issues related to <code>quantity</code>].


==== Packaging ====
=== Packaging ===
This is the packaging of the product. Multiple values are allowed. There is no taxonomy for this field, so you can enter anything you find relevant including:
This is the packaging of the product. Multiple values are allowed. There is no taxonomy for this field, so you can enter anything you find relevant including:
* the substance of the packaging: glass, metal, plastic, etc.
* the substance of the packaging: glass, metal, plastic, etc.
Line 74: Line 76:
See [https://github.com/openfoodfacts/openfoodfacts-server/issues?q=is%3Aissue+is%3Aopen+packaging+label%3Apackaging issues related to <code>packaging</code>].
See [https://github.com/openfoodfacts/openfoodfacts-server/issues?q=is%3Aissue+is%3Aopen+packaging+label%3Apackaging issues related to <code>packaging</code>].


==== Brands ====
=== Brands ===
This is the brands of the product. The main brand, generally clearly displayed on the front pack, should be entered first. A product can have other brands:
This is the brands of the product. The main brand, generally clearly displayed on the front pack, should be entered first. A product can have other brands:
* when a product is a brand sold by a big company: <code>Actimel</code> is sold by <code>Danone</code>, see https://world.openfoodfacts.org/product/4009700036810/actimel-granatapfel
* when a product is a brand sold by a big company: <code>Actimel</code> is sold by <code>Danone</code>, see https://world.openfoodfacts.org/product/4009700036810/actimel-granatapfel
Line 85: Line 87:
In the database, this field is called <code>brands</code>.
In the database, this field is called <code>brands</code>.


See [https://github.com/openfoodfacts/openfoodfacts-server/issues?q=is%3Aissue+is%3Aopen+ingredients+label%3Abrands issues related to <code>brands</code>].
See: [https://github.com/openfoodfacts/openfoodfacts-server/issues?q=is%3Aissue+is%3Aopen+ingredients+label%3Abrands issues related to <code>brands</code>].
 
=== Categories ===
[to be completed]
 
This field list the specific [[Global_categories_taxonomy|category]] of a product and its related parents. '''This field is very important''': it needs to be completed for Nutri-Score computation, as this computation makes differences between types of products (beverages, ...).
 
→ Indicate only the most specific category. "Parents" categories will be automatically added.
 
Examples: <code>Sardines in olive oil</code>, <code>Orange juice from concentrate</code>
 
See also:
* [https://github.com/openfoodfacts/openfoodfacts-server/blob/master/taxonomies/categories.txt our categories taxonomy on our repository].
* [https://github.com/openfoodfacts/openfoodfacts-server/issues?q=is%3Aissue+is%3Aopen+category+label%3Acategories issues related to <code>categories</code>].
 
=== Labels ===
Here we put any characteristic of the product which is factual and different from the other fields: organic, vegetarian, religious, etc.
 
Provenance: many contributors add the origins here: eg, <code>Made in Belgium</code>, <code>Produced in Brittany</code>, ...
 
Religious labels: namely halal and kosher labels.
 
Rating labels when they are displayed on the product (even if it's not the same as we compute): Nutri-Score (6+ European countries), Five Start Rating system (Australia, New-Zealand), etc.
 
[to be completed]
 
"[[Labels]], certifications, awards" on the website.
 
Example: Organic.
 
See the [[Labels]] page for more informations.
 
See also:
* [https://github.com/openfoodfacts/openfoodfacts-server/blob/master/taxonomies/labels.txt our labels taxonomy on our repository].
* [https://github.com/openfoodfacts/openfoodfacts-server/issues?q=is%3Aissue+is%3Aopen+category+label%3Alabels issues related to <code>labels</code>].
 
=== Manufacturing or processing places ===
[to be completed]
 
This field lists the places where the product has been manufactured or processed.
 
It is often displayed on the packaging by the traditional "Made in XXX". The packaging can also mention the region or even the address of the place. The origins of the ingredients have nothing to do with this: they can come from all over the world without any impact on the place where they have been processed.
 
You should enter this data like an address: the most precise information first and the most general at the end.
 
Eg.
 
* France
* Bretagne, France
* Laiterie de Saint-Denis-de-l'Hôtel (LSDH) - 10 Route de l'Aérodrome - Les Grandes Beaugines - 45550 Saint-Denis-de-l'Hôtel, Loiret, Centre-Val de Loire, France
 
In the database, this field is called <code>manufacturing_places</code>; it has a normalized version (small caps, spaces removed) under the name <code>manufacturing_places_tags</code>.
 
=== EMB code ===
[to be completed]
 
This field is dedicated for various codes related to packaging marks, identification marks or health marks:
 
* [https://en.wikipedia.org/wiki/EC_identification_and_health_marks EC identification and health marks] in use in the European Community to identify food producers or food packagers. Example: <code>FR 72.264.002 CE</code>.
* [https://fr.wikipedia.org/wiki/EMB_(code) EMB codes] (fr): packaging codes in use in France. Example: <code>EMB 72264</code>.
 
Open Food Facts gathered [https://world.openfoodfacts.org/packager-codes 25,000+ codes] as of 2021-08-18. These codes allow to localize the places on a map: https://world.openfoodfacts.org/packager-code/fr-72-264-002-ce
 
We use it to produce the world map of products: https://cestemballepresdechezvous.fr/ (fr) and https://madenear.me/ (en).
 
=== Best before date (expiration date) ===
The expiration date is a way to track product changes over time and to identify the most recent version. It's a data for manual usages. At this moment (2020-03), Open Food Facts apps and website don't make any usage of this field. An issue is open to [https://github.com/openfoodfacts/openfoodfacts-server/issues/76 throw off very old products in averages], it could be useful for it.
 
Be aware that, for the moment, this field is NOT normalized, so it probably contains dates in various formats that can be ambiguous (31/12/2019, 12/31/2019, 13 mai 2018, etc.). Also, the meaning of "best before", "expiry date", "use-by" notions might be different from different countries. You can help us to gather information about it in your own country, participating to the page [[Dates on the products]].
 
It is possible to see:
*[https://world.openfoodfacts.org/state/expiration-date-completed how many products do have an expiration date] (a bit more than 10% at the beginning of 2020)
* and [https://world.openfoodfacts.org/state/expiration-date-to-be-completed how many don't]
 
In the database and in Product Opener software, the technical name for this field is <code>expiration_date</code>.
 
=== Countries where sold ===
This field contains all the countries where the product is widely available (not including stores specialising in foreign products). If the field contains France, the product will be listed on the https://fr.openfoodfacts.org/ website.


==== Ingredients ====
The list of all existing countries can be found here: https://world.openfoodfacts.org/countries
 
In 2021, the [https://en.wikipedia.org/wiki/List_of_sovereign_states United Nation recognize 193 countries] but Open Food Facts also recognize other territories, such as overseas regions/territories such as French Guiana, Guadeloupe, French Polynesia, etc. This field is taxonomized ([https://github.com/openfoodfacts/openfoodfacts-server/blob/main/taxonomies/countries.txt source code]). Though, Open Food Facts accepts all values and some people or some bogus tools enter other values, which leads to have bad values.
 
The users or third party apps should enter:
 
* either the name of the country under the same locale as the web site:
** on fr.openfoodfacts.org (French locale) you should enter <code>Belgique</code>,
** while on world.openfoodfacts.org (English locale) you should enter <code>Belgium</code>
* either the name of the country with the locale as a prefix; e. g. <code>en:Belgium</code> or <code>fr:Belgique</code>.
 
In the database, this field is called <code>countries</code>.
 
This field allows to compute the <code>countries_tags</code> field, the normalized version of countries, which allows to show the name of a country in the desired language, thanks to countries taxonomy. Eg: when <code>Italy</code> is entered from a page in english (say,  world.openfoodfacts.org), <code>countries_tags</code> becomes <code>en:italy</code>.
 
== Ingredients ==
This field lists the [[ingredients]] of the product. This field is '''one of the most important''' as it is used for:
This field lists the [[ingredients]] of the product. This field is '''one of the most important''' as it is used for:
* Nova calculation
* Nova calculation
Line 102: Line 196:
See [https://github.com/openfoodfacts/openfoodfacts-server/issues?q=is%3Aissue+is%3Aopen+ingredients+label%3Aingredients issues related to <code>ingredients</code>].
See [https://github.com/openfoodfacts/openfoodfacts-server/issues?q=is%3Aissue+is%3Aopen+ingredients+label%3Aingredients issues related to <code>ingredients</code>].


 
=== Substances or products causing allergies or intolerances ===
==== Substances or products causing allergies or intolerances ====
The substances are ingredients that are actually in the product, which could cause common allergies. This field can be filled by hand, but is also completed by automatic ingredients analysis.
The substances are ingredients that are actually in the product, which could cause common allergies. This field can be filled by hand, but is also completed by automatic ingredients analysis.


Line 114: Line 207:


See [https://github.com/openfoodfacts/openfoodfacts-server/issues?q=is%3Aissue+is%3Aopen+traces+label%3Aallergens issues related to <code>allergens_tags</code>].
See [https://github.com/openfoodfacts/openfoodfacts-server/issues?q=is%3Aissue+is%3Aopen+traces+label%3Aallergens issues related to <code>allergens_tags</code>].
=== Traces ===
The traces are ingredients which are not used for the product itself but lay in the factory or the production process: the product might contain traces of these ingredients. Traces are really important if you are allergic.


==== Traces ====
The traces are ingredients which are not used for the product itself but lay in the factory or the production process: the product might contains traces of these ingredients. Traces are really important if you are allergic.
This field can be filled by hand, but is also completed by automatic ingredients analysis.
This field can be filled by hand, but is also completed by automatic ingredients analysis.


Line 129: Line 221:
See [https://github.com/openfoodfacts/openfoodfacts-server/issues?q=is%3Aissue+is%3Aopen+traces+label%3Atraces issues related to <code>traces</code>].
See [https://github.com/openfoodfacts/openfoodfacts-server/issues?q=is%3Aissue+is%3Aopen+traces+label%3Atraces issues related to <code>traces</code>].


== Nutrition facts ==


==== Best before date (expiration date) ====
=== Nutrition facts not specified ===
The expiration date is a way to track product changes over time and to identify the most recent version. It's a data for manual usages. At this moment (2020-03), Open Food Facts apps and website don't make any usage of this field. An issue is open to [https://github.com/openfoodfacts/openfoodfacts-server/issues/76 throw off very old products in averages], it could be useful for it.
Sometimes nutrition facts are not specified on the packaging or on a document given with the product. In this case, and only in this case, you have to fill the checkbox called <code>Nutrition facts are not specified on the product</code>.
 
Be aware that, for the moment, this field is NOT normalized, so it probably contains dates in various formats that can be ambiguous (31/12/2019, 12/31/2019, 13 mai 2018, etc.).
 
It is possible to see:
* [https://world.openfoodfacts.org/state/expiration-date-completed how many products do have an expiration date] (a bit more than 10% at the beginning of 2020)
* and [https://world.openfoodfacts.org/state/expiration-date-to-be-completed how many don't]


In the database and in Product Opener software, the technical name for this field is <code>expiration_date</code>.
It's often (if not always) the case for [https://world.openfoodfacts.org/category/aromatic-herbs aromatic herbs].


In the database, the technical name for this field is <code>no_nutrition_data</code>.


==== Serving size ====
=== Serving size ===
Serving size has a specific goal: to let Open Food Facts app make a proportional calculation of each nutrient per serving size. If a candy's weight is 5 g, it can be chosen as the serving size: if these candies has 66 g of sugar per 100 g, it has about 3 g per candy. [https://github.com/openfoodfacts/openfoodfacts-server/blob/f25308b7d47255be83210f699f897cba87c9517f/lib/ProductOpener/Food.pm#L3835 Allowed units] are: <code>kg, g, mg, µg, oz, l, dl, cl, ml, fl.oz, fl oz, г, мг, кг, л, дл, кл, мл, 毫克, 公斤, 毫升, 公升, 吨</code>.
Serving size has a specific goal: to let Open Food Facts app make a proportional calculation of each nutrient per serving size. If a candy's weight is 5 g, it can be chosen as the serving size: if these candies has 66 g of sugar per 100 g, it has about 3 g per candy. [https://github.com/openfoodfacts/openfoodfacts-server/blob/f25308b7d47255be83210f699f897cba87c9517f/lib/ProductOpener/Food.pm#L3835 Allowed units] are: <code>kg, g, mg, µg, oz, l, dl, cl, ml, fl.oz, fl oz, г, мг, кг, л, дл, кл, мл, 毫克, 公斤, 毫升, 公升, 吨</code>.


Line 149: Line 237:
Decimals can be written with a comma (<code>,</code>) or a point (<code>.</code>).
Decimals can be written with a comma (<code>,</code>) or a point (<code>.</code>).


Good:
In the database and in Product Opener software, the technical name for this field is <code>serving_size</code>. Based on <code>serving_size</code>, Open Food Facts computes a <code>serving_quantity</code> float number for 100g or 100ml. The <code>serving_quantity</code> can be found in the API or the data exports.
 
See [https://github.com/openfoodfacts/openfoodfacts-server/issues?q=is%3Aissue+is%3Aopen+serving+label%3Aportions issues related to <code>serving_size</code>].
 
==== Good examples ====
* <code>60 g</code> (preferred, for readability reasons)
* <code>60 g</code> (preferred, for readability reasons)
* <code>30g</code>
* <code>30g</code>
Line 156: Line 248:
* <code>1L</code>
* <code>1L</code>


Possible (while not recommended):
==== Possible examples (while not recommended) ====
* <code>cookie 25g</code>
* <code>cookie 25g</code>
* <code>One Slice (50g)</code>
* <code>One Slice (50g)</code>
* <code>97 g (0.5 cup)</code>
* <code>97 g (0.5 cup)</code>


Bad:
==== Bad examples ====
* <code>30 gr</code> => <code>gr</code> is not a correct unit
* <code>30 gr</code> => <code>gr</code> is not a correct unit
* <code>9 candies and 2 biscuits</code> => it's not possible to calculate a ratio because we don't know the weight of this portion
* <code>9 candies and 2 biscuits</code> => it's not possible to calculate a ratio because we don't know the weight of this portion
* <code>30</code> => there is no unit
* <code>30</code> => there is no unit


In the database and in Product Opener software, the technical name for this field is <code>serving_size</code>. Based on <code>serving_size</code>, Open Food Facts computes a <code>serving_quantity</code> float number for 100g or 100ml. The <code>serving_quantity</code> can be found in the API or the data exports.
=== Entering values ===
'''Question:''' is there any difference between comma "," and dot "."? (example "2,5" vs "2.5")
 
'''Answer:''' no
 
'''Question:''' if the number on the product ends with ".0" ("8.0", for example). Does it make any difference, if we enter "8.0" or "8"?
 
'''Answer:''' yes
 
'''Question:''' if the number on the products starts with "<". ("<0.5", for example). Does it make any difference, if we enter "<0.5" or "0.5"?
 
'''Answer:''' yes
 
'''Question''': what do I enter when the label says something like "contains trace amounts of ..."?
 
'''Answer''': you can enter the word "traces" as the value, and the system recognizes this as "approximately zero".
 
=== Missing value ===
When the nutrition facts table is missing some values, enter the "-" (hyphen symbol) for that field. For example the '''''Fiber''''' value is sometime missing.
 
== Photos and data check ==
At the end of the page, there is a specific section called "Photos and data check", documented as follow:
 
"Product pages can be marked as checked by experienced contributors who verify that the most recent photos are selected and cropped, and that all the product data that can be inferred from the product photos has been filled and is correct."
 
When <code>[x] Photos and data checked</code> is checked, Product Opener save the following data into the database:
 
* the value <code>on</code> in the field <code>checked</code>
* the last checked date (Unix timestamp format) into <code>last_checked_t</code>. Eg. <code>last_checked_t: 1704906819</code>.
* the last user who checked the product into <code>last_checker</code>. Eg. <code>last_checker: benbenben</code>.
* the users who have checked the products are stored in the field <code>checkers_tags</code>. Eg. <code>checkers_tags: ["benbenben","stephane"]</code>.
 
When <code>[x] I checked the photos and data again</code> is checked, Product Opener update the following data:
 
* the last checked date (Unix timestamp format) into <code>last_checked_t</code>.
* the last user who checked the product into <code>last_checker</code>.
* the users who have checked the products are stored in the field <code>checkers_tags</code>.
 
 
When a product is checked, this information is given on the product page, in the '''Data sources''' section, eg. "Last check of product page on January 10, 2024, 6:13:39 PM CET by charlesnepote".
 
These data can also be retrived via the API.
 
It's possible to explore the checked products with the dropdown menu "Explore products by...", selecting "Last checked dates", leading to the page: https://world.openfoodfacts.org/last-check-dates


See [https://github.com/openfoodfacts/openfoodfacts-server/issues?q=is%3Aissue+is%3Aopen+serving+label%3Aportions issues related to <code>serving_size</code>].
It's possible to combine facets, for example, to get last checked dates of the TOP-1000 products in a country, eg. https://world.openfoodfacts.net/popularity/top-1000-fr-scans-2022/last-check-dates
Jayaddison
10

edits

Retrieved from "https://test-wiki.openfoodfacts.org/Special:MobileDiff/15347...20588"