Metrics: Difference between revisions

Revision as of 11:07, 18 August 2024

Why ?

What ?

🎯 Roadmap for metrics harvesting

How ?

Reporting dashboard

An InfluxDB/Grafana stack has been set up to allow Open Food Facts team to be able to follow essential metrics. The dashboards can be viewed at https://metrics.openfoodfacts.org.

There is a read-only account, you can access the credentials on Slack to have access to it.

Metrics is different from monitoring (https://monitoring.openfoodfacts.org), we don't store on metrics.openfoodfacts.org infrastructure-related data.

We have a single InfluxDB bucket (=similar to SQL database) off_metrics. There are two measurements (=similar to SQL table): facets and insights.

`insights`

Save metrics about Robotoff product_insight PostgreSQL table. The export is performed daily at night time by Robotoff.

Columns:

annotation: annotation status of the insight, either 0, -1, 1 or <nil>.
automatic_processing: whether the insight will be (or has been) automatically processed, either "True" or "False".
predictor: the predictor of the insight (=model or method that generated the insight)
reserved_barcode: either "False" or "True", if True the product barcode is a reserved barcode, mostly used for variable weight products
type: the insight type, see InsightType class in Robotoff codebase for a complete list.
count: the number of insights with the previous characteristics.
percent: the % of insights with the previous characteristics, over the total number of insights.

`facets`

Save metrics about Product Opener facets, using public facet API. The export is performed daily at night time by Robotoff by calling Product Opener API.

Current saved facets:

ingredients-analysis
data-quality
ingredients
states
misc

Columns:

country: the ISO 2-letter code of the country (ISO_3166-1 alpha-2), or "world" for metrics on the full database. Only a selected subset of country is available.
facet: the name of the facet, it's the lower-case facet name with '-' replaced with '_' (ex: `data_quality` instead of `data-quality`)
tag_name: name of the tag
tag_id: identifier of the tag (ex: en:alcoholic-beverages-category-without-alcohol-value), this is the field you will probably have to use
products: number of products with the given tag
percent: % of products with the given tag over the total number of product for the country

@@ Line 3: / Line 3: @@
 == 🎯 Roadmap for metrics harvesting ==
+== How ? ==
-== Reporting dashboard ==
+=== Reporting dashboard ===
 An InfluxDB/Grafana stack has been set up to allow Open Food Facts team to be able to follow essential metrics. The dashboards can be viewed at https://metrics.openfoodfacts.org.
@@ Line 13: / Line 13: @@
 We have a single [https://en.wikipedia.org/wiki/InfluxDB InfluxDB] bucket (=similar to SQL database) ''<code>off_metrics</code>''. There are two measurements (=similar to SQL table): ''<code>facets</code>'' and ''<code>insights</code>''.
-=== <code>insights</code> ===
+==== <code>insights</code> ====
 Save metrics about Robotoff <code>product_insight</code> PostgreSQL table. The export is performed daily at night time by Robotoff.
@@ Line 26: / Line 26: @@
 * <code>percent</code>: the % of insights with the previous characteristics, over the total number of insights.
-=== <code>facets</code> ===
+==== <code>facets</code> ====
 Save metrics about Product Opener facets, using public facet API. The export is performed daily at night time by Robotoff by calling Product Opener API.