Ingredients Analysis Quality Evaluation - June 2022: Difference between revisions

Revision as of 12:51, 13 July 2022

Evaluation of the quality of Ingredients Extraction and Analysis so that we can measure the improvements.

Ingredients parsing and recognition

For each product, the ingredients list is parsed to separate each ingredient. Ingredients that we can match to our multilingual ingredients taxonomy as marked as "known", others as "unknown".

There are different reasons an ingredient can be marked as unknown:

The input ingredient list is incorrect (misspellings etc.)
The ingredients list contains other things than ingredients and we have not been able to split it at the right place.
We have not been able to parse a particular sentence structure or formatting (e.g. "a drop of delicious honey with a shower of powder sugar")
The ingredient is not yet present in our taxonomy (or not with the right synonym or in the right language)

In the table below,

the first column of numbers corresponds to the number of unique ingredients across all products.
And the second column of number corresponds to the number of occurrences of those ingredients. e.g. if a specific ingredient appears in 5 products, it is counted 5 times.

EN - English - India

2022-06-23

2022-06-27

All products:

known	776 (23.07%)	8444 (69.62%)
unknown	2588 (76.93%)	3685 (30.38%)
all	3364 (100.00%)	12129 (100.00%)

known	780 (23.10%)	9062 (71.49%)
unknown	2596 (76.90%)	3614 (28.51%)
all	3376 (100.00%)	12676 (100.00%)

Top 10k most scanned products:

known	407 (35.83%)	2149 (69.23%)
unknown	729 (64.17%)	955 (30.77%)
all	1136 (100.00%)	3104 (100.00%)

known	407 (36.02%)	2284 (71.38%)
unknown	723 (63.98%)	916 (28.62%)
all	1130 (100.00%)	3200 (100.00%)

EN - English - UK

2022-06-23

All products:

known	2591 (6.22%)	406797 (88.32%)
unknown	39034 (93.78%)	53807 (11.68%)
all	41625 (100.00%)	460604 (100.00%)

Top 10k most scanned products:

known	1664 (21.69%)	77617 (91.14%)
unknown	6008 (78.31%)	7546 (8.86%)
all	7672 (100.00%)	85163 (100.00%)

EN - English - US

2022-06-23

All products:

known	3027 (1.77%)	        8121359 (91.22%)
unknown	167758 (98.23%)	        781910 (8.78%)
all	170785 (100.00%)	8903269 (100.00%)

Top 10k most scanned products:

known	1619 (18.24%)	97755 (88.84%)
unknown	7255 (81.76%)	12283 (11.16%)
all	8874 (100.00%)	110038 (100.00%)

FR - French

2022-07-13

All products:

known	4440 (1.05%)	5069577 (88.03%)
unknown	419374 (98.95%)	689143 (11.97%)
all	423814 (100.00%)	5758720 (100.00%)

Top 10k most scanned products:

known	2436 (21.84%)	210027 (94.37%)
unknown	8718 (78.16%)	12527 (5.63%)
all	11154 (100.00%)	222554 (100.00%)

PL - Polish - PN

2022-07-08

All products:

known	1215 (11.38%)	68425 (78.48%)
unknown	9457 (88.62%)	18758 (21.52%)
all	10672 (100.00%)	87183 (100.00%)

Top 10k most scanned products:

known	742 (16.75%)	21685 (77.06%)
unknown	3689 (83.25%)	6455 (22.94%)
all	4431 (100.00%)	28140 (100.00%)

Ingredients analysis

For each product, once we have parsed the ingredients list to separate each ingredient and match them to our taxonomy, we can analyze if the products are vegan, vegetarian, without palm oil etc.

To mark a product as non-vegan etc. we only need to find one non-vegan ingredient. But to mark a product as vegan, we need to be sure that we have been able to correctly recognize all ingredients so that we can verify that all of them are vegan.

EN - English - IN

2022-06-23

All products:

Vegetarian status unknown	477	
Vegan status unknown	        389	
Palm oil content unknown	370	
Non-vegan	                184	
Palm oil free	                160	
Vegetarian	                160	
Vegan	                        80	
Palm oil	                74	
May contain palm oil	        47	
Non-vegetarian	                20	
Maybe vegan	                 8	
Maybe vegetarian	         7

Top 10k most scanned products:

Vegetarian status unknown	127	
Vegan status unknown	100	
Palm oil content unknown	93	
Non-vegan	43	
Palm oil free	37	
Vegetarian	27	
Palm oil	22	
Vegan	13	
May contain palm oil	5	
Non-vegetarian	3	
Maybe vegan	2	
Maybe vegetarian	2

EN - English - UK

2022-06-23

All products:

Vegetarian status unknown	8631	
Palm oil content unknown	8379	
Non-vegan	7699	
Vegetarian	7295	
Vegan status unknown	6193	
Palm oil free	5730	
Vegan	5023	
Non-vegetarian	2756	
Palm oil	2663	
May contain palm oil	1651	
Maybe vegetarian	912	
Maybe vegan	559

Top 10k most scanned products:

Vegetarian	1742	
Palm oil content unknown	1474	
Vegetarian status unknown	1413	
Palm oil free	1406	
Non-vegan	1359	
Vegan	1139	
Vegan status unknown	1121	
Palm oil	511	
Non-vegetarian	434	
May contain palm oil	270	
Maybe vegetarian	159	
Maybe vegan	112

EN - English - US

2022-06-23

All products:

Vegetarian status unknown	212874	
Palm oil content unknown	176193	
Non-vegan	146466	
Vegan status unknown	126232	
Palm oil free	85546	
Vegetarian	51118	
Non-vegetarian	45192	
Vegan	44944	
Palm oil	37920	
May contain palm oil	30916	
Maybe vegetarian	21744	
Maybe vegan	13281

Top 10k most scanned products:

Vegetarian status unknown	2914	
Palm oil content unknown	2486	
Vegan status unknown	1954	
Non-vegan	1552	
Palm oil free	1226	
Vegetarian	944	
Vegan	847	
Palm oil	405	
Non-vegetarian	391	
May contain palm oil	360	
Maybe vegetarian	248	
Maybe vegan	143

PL - Polish - PL

2022-07-08

All products:

en:Vegetarian status unknown	3442	
en:Palm oil content unknown	2571	
en:Vegan status unknown	2297	
en:Non-vegan	1756	
Bez oleju palmowego	1336	
en:Vegetarian	699	
en:Vegan	581	
en:May contain palm oil	435	
en:Non-vegetarian	413	
Olej palmowy	311	
en:Maybe vegetarian	137	
en:Maybe vegan	51

Top 10k most scanned products:

en:Vegetarian status unknown	1382	
en:Palm oil content unknown	1032	
en:Vegan status unknown	964	
en:Non-vegan	599	
Bez oleju palmowego	491	
en:Vegetarian	244	
en:Vegan	187	
en:May contain palm oil	142	
en:Non-vegetarian	109	
Olej palmowy	92	
en:Maybe vegetarian	32	
en:Maybe vegan	16

Observations