951
edits
Reusing Open Food Facts Data: Difference between revisions
Reusing Open Food Facts Data (view source)
Revision as of 14:58, 30 November 2021
, 30 November 2021Merging delta exports documentation
(Eg: Filtering barcodes which are different from a code containing 1 to 13 digits) |
(Merging delta exports documentation) |
||
| Line 85: | Line 85: | ||
Beware, each line is a product and some variable need to be unnest: tidyverser::unnest_wider() | Beware, each line is a product and some variable need to be unnest: tidyverser::unnest_wider() | ||
=== JSONL delta exports === | |||
Every day, Open Food Facts exports all the products created during the last 24 hours. The documentation of this export can be found in the /data page. | |||
If you don't have MongoDB and just want to use these delta exports to build an up-to-date database, you can merge each export with the help of <code>[https://stedolan.github.io/jq/manual/v1.6/ jq]</code> tool. | |||
$ gunzip products_1638076899_1638162314.json.gz # will decompress the file | |||
$ wc -l products_1638076899_1638162314.json # will count the number of products in this export (in JSONL each line is a JSON object) | |||
$ jq -c '. + .' 2021-11-30.json products_1638162314_1638248379.json > 2021-12-01.json # merge the delta with previous complete data | |||
=== JSONL export === | === JSONL export === | ||