Infrastructure Planning: Difference between revisions

Latest revision as of 06:55, 19 August 2024

Welcome :-)
This page describes the hardware/software infrastructure for the Open Food Facts, Open Beauty Facts, Open Pet Food Facts and Open Products Facts projects, as well as all the complementary deployments.
It is not updated anymore, as we have moved to GitHub for planning, postmortems…
Please check our coordination repo: https://github.com/openfoodfacts/openfoodfacts-infrastructure
Note, we also have an #infrastructure channel on Slack, and a monthly video call. We can arrange ad-hoc calls if you'd like to get started faster.

Current infrastructure

Details on https://en.wiki.openfoodfacts.org/Infrastructure#Current_infrastructure

off1: frontend + backend for the web site and API
off2: mongodb + off1 backup
dev: dev server, robotoff, remote backup

Hardware needs 2020

Production

Frontend + backend for the web site and API
- off1

MongoDB
- off2

Robotoff
- New server
  - Models are memory hungry, at least 32 GB of RAM is needed. Ideally, 64 GB.
  - Ideally, 1 GPU server. Otherwise, high-performance CPU are suitable, 8 cores at least (the more, the better).
  - Disk: speed is only important when loading model from disk, which only occurs at startup. Ideally NVMe, classic SSDs are fine. 500 GB of space should be ok.

New producer platform
- New server
  - Little web traffic, lots of image data (sometimes in huge PDFs or png)
  - mid-level cpu and ram, at least 2x4 Tb disks

New content platform (wiki, blog...) ??
- New server for different kind of tools (example: Zammad, MediaWiki, Wordpress, Matomo, Rserver?, Nextcloud?)
  - Little web traffic, not much disk needed (except for videos if needed?)
  - mid-level cpu and ram, 2x500 Gb SSD disks probably ok

Backup

New backup server or cloud storage

Dev

OFF + producer platform dev
- New server
  - mid level cpu and ram, at least 2x8 Tb disks

@@ Line 1: / Line 1: @@
+[[Category:Infrastructure]]
+[[Category:Roadmap]]
+Welcome :-) <br>
+This page describes the hardware/software infrastructure for the Open Food Facts, Open Beauty Facts, Open Pet Food Facts and Open Products Facts projects, as well as all the complementary deployments.<br>
+'''It is not updated anymore, as we have moved to GitHub for planning, postmortems…'''<br>
+Please check our coordination repo: https://github.com/openfoodfacts/openfoodfacts-infrastructure<br>
+Note, we also have an #infrastructure channel on Slack, and a monthly video call. We can arrange ad-hoc calls if you'd like to get started faster.
 == Current infrastructure ==
@@ Line 7: / Line 15: @@
 * dev: dev server, robotoff, remote backup
-== Hardware needs 2019 ==
+== Hardware needs 2020 ==
 === Production ===
@@ Line 13: / Line 21: @@
 * Frontend + backend for the web site and API
 ** off1
 * MongoDB
 ** off2
-* Robotoff
+* '''Robotoff'''
 ** New server
-*** cpu? ram? disk?
+*** Models are memory hungry, at least 32 GB of RAM is needed. Ideally, 64 GB.
-* New producer platform
+*** Ideally, 1 GPU server. Otherwise, high-performance CPU are suitable, 8 cores at least (the more, the better).
+*** Disk: speed is only important when loading model from disk, which only occurs at startup. Ideally NVMe, classic SSDs are fine. 500 GB of space should be ok.
+* '''New producer platform'''
 ** New server
-*** Little web trafic, lots of image data (sometimes in huge PDFs or png)
+*** Little web traffic, lots of image data (sometimes in huge PDFs or png)
-*** mid level cpu and ram, at least 2x4 Tb disks
+*** mid-level cpu and ram, at least 2x4 Tb disks
+* '''New content platform''' (wiki, blog...) ??
+** New server for different kind of tools (example: Zammad, MediaWiki, Wordpress, Matomo, Rserver?, Nextcloud?)
+*** Little web traffic, not much disk needed (except for videos if needed?)
+*** mid-level cpu and ram, 2x500 Gb SSD disks probably ok
 === Backup ===
@@ Line 29: / Line 47: @@
 === Dev ===
-* OFF + producer platform dev
+* '''OFF + producer platform dev'''
 ** New server
 *** mid level cpu and ram, at least 2x8 Tb disks