Events system: Difference between revisions
Line 2: | Line 2: | ||
This is a brainstorm page, to find a good solution to monitor events in Open Food Facts. | This is a brainstorm page, to find a good solution to monitor events in Open Food Facts. | ||
* *We need an event and action tracking and management system for the Platform for Producers: to monitor imports, verify new users etc.* | |||
* We can also consider other use cases to see if we can create a solution that could work for them too | |||
= What kind of events do we want to track = | = What kind of events do we want to track = |
Revision as of 11:32, 23 June 2020
Introduction
This is a brainstorm page, to find a good solution to monitor events in Open Food Facts.
- *We need an event and action tracking and management system for the Platform for Producers: to monitor imports, verify new users etc.*
- We can also consider other use cases to see if we can create a solution that could work for them too
What kind of events do we want to track
Info events
Something happened.
- User actions
- e.g. an user registered, a product was added or edited...
- could be useful to have recent changes filterable by facets
- Processes: some process started, failed, or completed
- -> this is the most pressing need, for which we don't have a good solution today
- Regular maintenance tasks (e.g. daily processing of something)
- Adhoc maintenance tasks (e.g. re-processing of products to apply a new taxonomy)
- User requested tasks
- Imports and exports on the Producers Platform
- Errors
- Software errors
- MongoDB down
- Product quality issues
- Software errors
Events that require an action
Something needs to be done.
- User request that needs to be manually validated
- exporting products from the Producers Platform to the public database (at least for the first time)
Desirable features
- Event log: being able to see individual events
- Filtering by event type, country, product, organization, user etc.
- Integration with OFF
- It's not just about generating events, but also being able to display events directly into OFF
- e.g. show status of user generated tasks
- It's not just about generating events, but also being able to display events directly into OFF
- Aggregated metrics, graphs etc. of events
- Nice to have, but not the main focus.
- Event status for events that require an action (e.g. open / close)
- Notifications (by mail, Slack channel etc.) for specific types of events
- Public events and private events
Possible options
External events system
- Sentry
- Perl client: https://docs.sentry.io/clients/perl/
- Matomo
- Minion
- We could turn maintenance tasks into Minion jobs
OFF events system
- Events collection in MongoDB
- Makes it easy to have lots of OFF specific facets for events
Notes
- We probably do not want to turn the events system in an analytics system
- Keep things (e.g. the events database) relatively small