Metrics: Difference between revisions

Revision as of 15:55, 9 January 2023

An InfluxDB/Grafana stack has been set up to allow Open Food Facts team to be able to follow essential metrics. The dashboards can be viewed at https://metrics.openfoodfacts.org.

There is a read-only account, you can access the credentials on Slack to have access to it.

Metrics is different from monitoring (https://monitoring.openfoodfacts.org), we don't store on metrics.openfoodfacts.org infrastructure-related data.

We have a single InfluxDB bucket (=similar to SQL database) off_metrics. There are two measurements (=similar to SQL table): facets and insights.

`insights`

Save metrics about Robotoff product_insight PostgreSQL table. The export is performed daily at night time by Robotoff.

Columns:

annotation: annotation status of the insight, either 0, -1, 1 or <nil>.
automatic_processing: whether the insight will be (or has been) automatically processed, either "True" or "False".
predictor: the predictor of the insight (=model or method that generated the insight)
reserved_barcode: either "False" or "True", if True the product barcode is a reserved barcode, mostly used for variable weight products
type: the insight type, see InsightType class in Robotoff codebase for a complete list.
count: the number of insights with the previous characteristics.
percent: the % of insights with the previous characteristics, over the total number of insights.

`facets`

Save metrics about Product Opener facets, using public facet API. The export is performed daily at night time by Robotoff by calling Product Opener API.

Current saved facets:

ingredients-analysis
data-quality
ingredients
states
misc

Columns:

country: the ISO 2-letter code of the country (ISO_3166-1 alpha-2), or "world" for metrics on the full database. Only a selected subset of country is available.
facet: the name of the facet, it's the lower-case facet name with '-' replaced with '_' (ex: `data_quality` instead of `data-quality`)
tag_name: name of the tag
tag_id: identifier of the tag (ex: en:alcoholic-beverages-category-without-alcohol-value), this is the field you will probably have to use
products: number of products with the given tag
percent: % of products with the given tag over the total number of product for the country

@@ Line 1: / Line 1: @@
-An InfluxDB/Grafana stack has been set up to allow Open Food Facts team to be able to follow essential metrics.
+An InfluxDB/Grafana stack has been set up to allow Open Food Facts team to be able to follow essential metrics. The dashboards can be viewed at https://metrics.openfoodfacts.org.
-The dashboards can be viewed at https://metrics.openfoodfacts.org. There is a read-only account, you can access the credentials on Slack to have access to it.
+There is a read-only account, you can access the credentials on Slack to have access to it.
 Metrics is different from monitoring (https://monitoring.openfoodfacts.org), we don't store on metrics.openfoodfacts.org infrastructure-related data.
-We have a single InfluxDB bucket (=similar to SQL database) ''off_metrics''.
+We have a single [https://en.wikipedia.org/wiki/InfluxDB InfluxDB] bucket (=similar to SQL database) ''<code>off_metrics</code>''. There are two measurements (=similar to SQL table): ''<code>facets</code>'' and ''<code>insights</code>''.
-There are two measurements (=similar to SQL table): ''facets'' and ''insights''.
-=== insights ===
+=== <code>insights</code> ===
-Save metrics about Robotoff ''product_insight'' PostgreSQL table. The export is performed daily at night time by Robotoff.
+Save metrics about Robotoff <code>product_insight</code> PostgreSQL table. The export is performed daily at night time by Robotoff.
 Columns:
-* annotation: annotation status of the insight, either 0, -1, 1 or <nil>.
+* <code>annotation</code>: annotation status of the insight, either 0, -1, 1 or <nil>.
-* automatic_processing: whether the insight will be (or has been) automatically processed, either "True" or "False".
+* <code>automatic_processing</code>: whether the insight will be (or has been) automatically processed, either "True" or "False".
-* predictor: the predictor of the insight (=model or method that generated the insight)
+* <code>predictor</code>: the predictor of the insight (=model or method that generated the insight)
-* reserved_barcode: either "False" or "True", if True the product barcode is a reserved barcode, mostly used for variable weight products
+* <code>reserved_barcode</code>: either "False" or "True", if True the product barcode is a reserved barcode, mostly used for variable weight products
-* type: the insight type, see [https://github.com/openfoodfacts/robotoff/blob/9598d29cc2a23b802ef6eff1c7c154607a6ccc93/robotoff/types.py#L51 InsightType] class in Robotoff codebase for a complete list.
+* <code>type</code>: the insight type, see [https://github.com/openfoodfacts/robotoff/blob/9598d29cc2a23b802ef6eff1c7c154607a6ccc93/robotoff/types.py#L51 InsightType] class in Robotoff codebase for a complete list.
-* count: the number of insights with the previous characteristics.
+* <code>count</code>: the number of insights with the previous characteristics.
-* percent: the % of insights with the previous characteristics, over the total number of insights.
+* <code>percent</code>: the % of insights with the previous characteristics, over the total number of insights.
-=== facets ===
+=== <code>facets</code> ===
 Save metrics about Product Opener facets, using public facet API. The export is performed daily at night time by Robotoff by calling Product Opener API.
 Current saved facets:
-* ingredients-analysis
+* <code>ingredients-analysis</code>
-* data-quality
+* <code>data-quality</code>
-* ingredients
+* <code>ingredients</code>
-* states
+* <code>states</code>
-* misc
+* <code>misc</code>
 Columns:
-* country: the ISO 2-letter code of the country (ISO_3166-1 alpha-2), or "world" for metrics on the full database. Only a selected subset of country is available.
+* <code>country</code>: the ISO 2-letter code of the country (ISO_3166-1 alpha-2), or "world" for metrics on the full database. Only a selected subset of country is available.
-* facet: the name of the facet, it's the lower-case facet name with '-' replaced with '_' (ex: `data_quality` instead of `data-quality`)
+* <code>facet</code>: the name of the facet, it's the lower-case facet name with '-' replaced with '_' (ex: `data_quality` instead of `data-quality`)
-* tag_name: name of the tag
+* <code>tag_name</code>: name of the tag
-* tag_id: identifier of the tag (ex: `en:alcoholic-beverages-category-without-alcohol-value`), this is the field you will probably have to use
+* <code>tag_id</code>: identifier of the tag (ex: <code>en:alcoholic-beverages-category-without-alcohol-value</code>), this is the field you will probably have to use
-* products: number of products with the given tag
+* <code>products</code>: number of products with the given tag
-* percent: % of products with the given tag over the total number of product for the country
+* <code>percent</code>: % of products with the given tag over the total number of product for the country