Ingredients Analysis Quality Evaluation - June 2022: Difference between revisions
No edit summary |
|||
| (2 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
Evaluation of the quality of [[Ingredients Extraction and Analysis]] so that we can measure the improvements. | Evaluation of the quality of [[Ingredients Extraction and Analysis]] so that we can measure the improvements. | ||
[[Category:Metrics]] | |||
== Ingredients parsing and recognition == | == Ingredients parsing and recognition == | ||
| Line 416: | Line 416: | ||
[[Category:Project:Personalized_Search]] | [[Category:Project:Personalized_Search]] | ||
[[Category:Data quality]] | |||
[[Category:Ingredients]] | |||
Latest revision as of 10:44, 7 August 2024
Evaluation of the quality of Ingredients Extraction and Analysis so that we can measure the improvements.
Ingredients parsing and recognition
For each product, the ingredients list is parsed to separate each ingredient. Ingredients that we can match to our multilingual ingredients taxonomy as marked as "known", others as "unknown".
There are different reasons an ingredient can be marked as unknown:
- The input ingredient list is incorrect (misspellings etc.)
- The ingredients list contains other things than ingredients and we have not been able to split it at the right place.
- We have not been able to parse a particular sentence structure or formatting (e.g. "a drop of delicious honey with a shower of powder sugar")
- The ingredient is not yet present in our taxonomy (or not with the right synonym or in the right language)
In the table below,
- the first column of numbers corresponds to the number of unique ingredients across all products.
- And the second column of number corresponds to the number of occurrences of those ingredients. e.g. if a specific ingredient appears in 5 products, it is counted 5 times.
EN - English - India
| 2022-06-23 | 2022-06-27 | |
|---|---|---|
| All products: |
known 776 (23.07%) 8444 (69.62%) unknown 2588 (76.93%) 3685 (30.38%) all 3364 (100.00%) 12129 (100.00%) |
known 780 (23.10%) 9062 (71.49%) unknown 2596 (76.90%) 3614 (28.51%) all 3376 (100.00%) 12676 (100.00%) |
| Top 10k most scanned products: |
known 407 (35.83%) 2149 (69.23%) unknown 729 (64.17%) 955 (30.77%) all 1136 (100.00%) 3104 (100.00%) |
known 407 (36.02%) 2284 (71.38%) unknown 723 (63.98%) 916 (28.62%) all 1130 (100.00%) 3200 (100.00%) |
EN - English - UK
| 2022-06-23 | ||
|---|---|---|
| All products: |
known 2591 (6.22%) 406797 (88.32%) unknown 39034 (93.78%) 53807 (11.68%) all 41625 (100.00%) 460604 (100.00%) |
|
| Top 10k most scanned products: |
known 1664 (21.69%) 77617 (91.14%) unknown 6008 (78.31%) 7546 (8.86%) all 7672 (100.00%) 85163 (100.00%) |
EN - English - US
| 2022-06-23 | ||
|---|---|---|
| All products: |
known 3027 (1.77%) 8121359 (91.22%) unknown 167758 (98.23%) 781910 (8.78%) all 170785 (100.00%) 8903269 (100.00%) |
|
| Top 10k most scanned products: |
known 1619 (18.24%) 97755 (88.84%) unknown 7255 (81.76%) 12283 (11.16%) all 8874 (100.00%) 110038 (100.00%) |
FR - French
| 2022-07-13 | ||
|---|---|---|
| All products: |
known 4440 (1.05%) 5069577 (88.03%) unknown 419374 (98.95%) 689143 (11.97%) all 423814 (100.00%) 5758720 (100.00%) |
|
| Top 10k most scanned products: |
known 2436 (21.84%) 210027 (94.37%) unknown 8718 (78.16%) 12527 (5.63%) all 11154 (100.00%) 222554 (100.00%) |
HR - Croatia
| 2022-09-20 | ||
|---|---|---|
| All products: |
known 680 (8.64%) 8476 (46.93%) unknown 7187 (91.36%) 9584 (53.07%) all 7867 (100.00%) 18060 (100.00%) |
|
| Top 10k most scanned products: |
known 369 (12.05%) 2394 (42.61%) unknown 2693 (87.95%) 3224 (57.39%) all 3062 (100.00%) 5618 (100.00%) |
PL - Polish - PN
| 2022-07-08 | ||
|---|---|---|
| All products: |
known 1215 (11.38%) 68425 (78.48%) unknown 9457 (88.62%) 18758 (21.52%) all 10672 (100.00%) 87183 (100.00%) |
|
| Top 10k most scanned products: |
known 742 (16.75%) 21685 (77.06%) unknown 3689 (83.25%) 6455 (22.94%) all 4431 (100.00%) 28140 (100.00%) |
Ingredients analysis
For each product, once we have parsed the ingredients list to separate each ingredient and match them to our taxonomy, we can analyze if the products are vegan, vegetarian, without palm oil etc.
To mark a product as non-vegan etc. we only need to find one non-vegan ingredient. But to mark a product as vegan, we need to be sure that we have been able to correctly recognize all ingredients so that we can verify that all of them are vegan.
EN - English - IN
| 2022-06-23 | ||
|---|---|---|
| All products: |
Vegetarian status unknown 477 Vegan status unknown 389 Palm oil content unknown 370 Non-vegan 184 Palm oil free 160 Vegetarian 160 Vegan 80 Palm oil 74 May contain palm oil 47 Non-vegetarian 20 Maybe vegan 8 Maybe vegetarian 7 |
|
| Top 10k most scanned products: |
Vegetarian status unknown 127 Vegan status unknown 100 Palm oil content unknown 93 Non-vegan 43 Palm oil free 37 Vegetarian 27 Palm oil 22 Vegan 13 May contain palm oil 5 Non-vegetarian 3 Maybe vegan 2 Maybe vegetarian 2 |
EN - English - UK
| 2022-06-23 | ||
|---|---|---|
| All products: |
Vegetarian status unknown 8631 Palm oil content unknown 8379 Non-vegan 7699 Vegetarian 7295 Vegan status unknown 6193 Palm oil free 5730 Vegan 5023 Non-vegetarian 2756 Palm oil 2663 May contain palm oil 1651 Maybe vegetarian 912 Maybe vegan 559 |
|
| Top 10k most scanned products: |
Vegetarian 1742 Palm oil content unknown 1474 Vegetarian status unknown 1413 Palm oil free 1406 Non-vegan 1359 Vegan 1139 Vegan status unknown 1121 Palm oil 511 Non-vegetarian 434 May contain palm oil 270 Maybe vegetarian 159 Maybe vegan 112 |
EN - English - US
| 2022-06-23 | ||
|---|---|---|
| All products: |
Vegetarian status unknown 212874 Palm oil content unknown 176193 Non-vegan 146466 Vegan status unknown 126232 Palm oil free 85546 Vegetarian 51118 Non-vegetarian 45192 Vegan 44944 Palm oil 37920 May contain palm oil 30916 Maybe vegetarian 21744 Maybe vegan 13281 |
|
| Top 10k most scanned products: |
Vegetarian status unknown 2914 Palm oil content unknown 2486 Vegan status unknown 1954 Non-vegan 1552 Palm oil free 1226 Vegetarian 944 Vegan 847 Palm oil 405 Non-vegetarian 391 May contain palm oil 360 Maybe vegetarian 248 Maybe vegan 143 |
HR - Croatia
| 2022-09-20 | ||
|---|---|---|
| All products: |
en:Vegetarian status unknown 706 en:Palm oil content unknown 656 en:Vegan status unknown 556 en:Non-vegan 206 en:Vegetarian 189 en:Vegan 165 en:Palm oil free 152 en:May contain palm oil 50 en:Palm oil 50 en:Non-vegetarian 29 en:Maybe vegetarian 8 en:Maybe vegan 5 |
|
| Top 10k most scanned products: |
en:Vegetarian status unknown 267 en:Palm oil content unknown 248 en:Vegan status unknown 215 en:Non-vegan 65 en:Vegetarian 50 en:Palm oil free 45 en:Vegan 43 en:May contain palm oil 18 en:Palm oil 13 en:Non-vegetarian 6 en:Maybe vegetarian 2 en:Maybe vegan 2 |
PL - Polish - PL
| 2022-07-08 | ||
|---|---|---|
| All products: |
en:Vegetarian status unknown 3442 en:Palm oil content unknown 2571 en:Vegan status unknown 2297 en:Non-vegan 1756 Bez oleju palmowego 1336 en:Vegetarian 699 en:Vegan 581 en:May contain palm oil 435 en:Non-vegetarian 413 Olej palmowy 311 en:Maybe vegetarian 137 en:Maybe vegan 51 |
|
| Top 10k most scanned products: |
en:Vegetarian status unknown 1382 en:Palm oil content unknown 1032 en:Vegan status unknown 964 en:Non-vegan 599 Bez oleju palmowego 491 en:Vegetarian 244 en:Vegan 187 en:May contain palm oil 142 en:Non-vegetarian 109 Olej palmowy 92 en:Maybe vegetarian 32 en:Maybe vegan 16 |