Jump to content

Product Opener/Installation/Debian or Ubuntu: Difference between revisions

Product Opener/Installation/Debian or Ubuntu (view source)

Revision as of 13:38, 7 August 2024

11,393 bytes added ,  7 August
no edit summary
VisualWikitext
No edit summary
 
(24 intermediate revisions by 6 users not shown)
Line 1: Line 1:
Also see [[Product Opener Installation]]
Also see [[Product Opener Installation]]
[[Category:ProductOpener]]
[[Category:Infrastructure]]
This page lists all the steps necessary to install Product Opener on Debian, including a development environment.
= Warning! Those instructions are outdated and kept for reference only =
Please see http://en.wiki.openfoodfacts.org/Infrastructure for up-to-date install instructions.
= Installation on Jessie =
We start from a clean Jessie install.
== Get the code and configure it ==
<pre>
apt-get install git
git clone <repo> /srv/openfoodfacts
mkdir /srv/openfoodfacts/users
mkdir /srv/openfoodfacts/products
chown -R www-data:www-data /srv/openfoodfacts
cd /srv/openfoodfacts/cgi
rm Blogs
ln -s . Blogs
ln -s SiteLang_off.pm SiteLang.pm
</pre>
Edit a few variables in Config2.pm:
<pre>
$server_domain = "yourdomain.com";
$www_root="/srv/openfoodfacts/html";
$data_root="/srv/openfoodfacts";
</pre>
Also in Config_off.pm:
<pre>
$domain = 'yourdomain.com';
$contact_email = 'webmaster@yourdomain.com';
$admin_email = 'webmaster@yourdomain.com';
</pre>
And in startup.pl:
<pre>
use lib "/srv/openfoodfacts/cgi/";
</pre>
== Install packaged prerequisites ==
<pre>
apt-get install apache2 build-essential geoip-bin geoip-database imagemagick libbarcode-zbar-perl libcache-memcached-fast-perl libclone-perl libcrypt-passwdmd5-perl libdatetime-format-mail-perl \
libencode-detect-perl libgeo-ip-perl libgraphics-color-perl libgraphviz-perl libhtml-defang-perl libimage-magick-perl libjson-perl libmime-lite-perl libmime-lite-perl libmongodb-perl libperl-dev \
libtest-nowarnings-perl libtext-unaccent-perl liburi-escape-xs-perl liburi-find-perl libwww-perl libxml-encoding-perl libxml-feedpp-perl memcached mongodb tesseract-ocr tesseract-ocr-fra zlib1g-dev
</pre>
== Prepare CPAN ==
If you never used CPAN before, you can automatically configure it:
<pre>
(echo y;echo o conf prerequisites_policy follow;echo o conf commit)|cpan
</pre>
== Install a few Perl modules from CPAN ==
<pre>
cpan Net::IDN::Punycode
cpan Encode::Punycode
cpan GraphViz2
cpan Image::OCR::Tesseract
cpan Algorithm::CheckDigits
cpan XML::Simple
</pre>
Note: on Debian Jessie, you might need to install the follwoing to be able to get cpan to install GraphViz2
<pre>
apt-get install libtest-version-perl
</pre>
== Rebuild Apache 2.2 and mod_perl ==
<pre>
cd /usr/local/src/
wget http://apache.crihan.fr/dist/httpd/httpd-2.2.31.tar.gz
tar xvfz httpd-2.2.31.tar.gz
cd httpd-2.2.31
./configure --with-mpm=prefork --prefix=/usr/local  --enable-rewrite --enable-proxy --enable-proxy_http --enable-deflate --disable-userdir --enable-headers
make
make install
cd /usr/local/src/
wget http://mir2.ovh.net/ftp.apache.org/dist/perl/mod_perl-2.0.9.tar.gz
tar xvfz mod_perl-2.0.9.tar.gz
cd mod_perl-2.0.9
perl Makefile.PL MP_APXS=/usr/local/bin/apxs
make
make install
cat >> /usr/local/conf/httpd.conf <<EOF
Listen 8001
LoadModule perl_module modules/mod_perl.so
PerlWarn Off
PerlRequire /srv/openfoodfacts/cgi/startup.pl
<VirtualHost *:8001>
  DocumentRoot /srv/openfoodfacts/html
  ServerName yourdomain.com
  ErrorLog /var/log/apache2/error.log
  CustomLog /var/log/apache2/access.log combined
  ScriptAlias /cgi/ /srv/openfoodfacts/cgi/
  <Location /cgi>
    SetHandler perl-script
    PerlResponseHandler ModPerl::Registry
    PerlOptions +ParseHeaders
    Options +ExecCGI
    Order allow,deny
    Allow from all
  </Location>
</VirtualHost>
PerlPostReadRequestHandler My::ProxyRemoteAddr
EOF
sed -i -e 's/^Listen 80$/#Listen 80/' /usr/local/conf/httpd.conf
sed -i -e 's/^User.*/User www-data/' /usr/local/conf/httpd.conf
sed -i -e 's/^Group.*/Group www-data/' /usr/local/conf/httpd.conf
/usr/local/bin/apachectl -k start
</pre>
== Configure standard Apache 2 ==
<pre>
a2enmod proxy_http
a2enmod rewrite
cat >> /etc/apache2/sites-available/product-opener-proxy.conf <<EOF
<VirtualHost *>
  DocumentRoot /srv/openfoodfacts/html
  ServerName yourdomain.com
  ServerAlias *.yourdomain.com
  <Directory "/srv/openfoodfacts/html">
    Options Indexes FollowSymLinks
    Require all granted
  </Directory>
  ProxyPreserveHost On
  RewriteEngine on
  RewriteCond  %{REQUEST_URI}  !/./
  RewriteRule ^(/cgi/.*)$ http://localhost:8001$1 [P,L]
  RewriteMap escape int:escape
  RewriteRule ^/favicon.ico$ /favicon.ico [L]
  RewriteCond  %{REQUEST_URI}  !^/images/
  RewriteCond  %{REQUEST_URI}  !^/js/
  RewriteCond  %{REQUEST_URI}  !^/foundation/
  RewriteCond  %{REQUEST_URI}  !^/rss/
  RewriteCond  %{REQUEST_URI}  !^/robots
  RewriteCond  %{REQUEST_URI}  !^/clicks/
  RewriteCond  %{REQUEST_URI}  !^/data/
  RewriteCond  %{REQUEST_URI}  !^/files/
  RewriteRule  ^(.*)$ http://localhost:8001/cgi/display.pl?${escape:$1} [P,L,QSA]
</VirtualHost>
EOF
a2ensite product-opener-proxy.conf
a2dissite 000-default.conf
service apache2 restart
</pre>
You should now have a working Productopener instance running at http://yourdomain.com/. Enjoy!
= Installation on Debian Jessie with stock Apache and nginx =
<pre>
Vous êtes sur le point d'installer un OS sur le serveur ks3095298.kimsufi.com. Choisissez le type d'installation que vous souhaitez utiliser :
-> Installer à partir d'un template OVH
Debian 8.1 stable (Jessie) (stable) - debian8
root@ks3095298:~# uname -a
Linux ks3095298.kimsufi.com 3.14.32-xxxx-grs-ipv6-64 #5 SMP Wed Sep 9 17:24:34 CEST 2015 x86_64 GNU/Linux
perl -v
This is perl 5, version 20, subversion 2 (v5.20.2) built for x86_64-linux-gnu-thread-multi
apt-get update
apt-get upgrade
apt-get install fail2ban
apt-get install sudo
apt-get install mongodb


This page lists all the steps necessary to install Product Opener on Debian, including a development environment.
MongoDB shell version: 2.4.10
 
vi /etc/mongodb.conf
- change dbpath
- uncomment :
#port = 27017
#nohttpinterface = true
 
 
apt-get install git
 
apt-get install apache2
apt-get install libapache2-mod-perl2
apt-get install modperl
 
adduser off
 
apt-get install imagemagick
apt-get install memcached
apt-get install tesseract-ocr tesseract-ocr-fra
apt-get install exim4
dpkg-reconfigure exim4-config
apt-get install geoip-bin geoip-database
apt-get install zlib1g-dev
 
apt-get install libwww-perl libimage-magick-perl libxml-encoding-perl libtext-unaccent-perl libmime-lite-perl libcache-memcached-fast-perl libjson-perl libclone-perl  libgraphviz-perl libmime-lite-perl libcrypt-passwdmd5-perl libencode-detect-perl libgraphics-color-perl libbarcode-zbar-perl libxml-feedpp-perl libmongodb-perl liburi-find-perl libxml-simple-perl
 
 
cp -a /home/openfoodfacts-bitbucket/product-opener/cgi /home/off/cgi/
 
/home/off/cgi# rm Blogs
root@ks3095298:/home/off/cgi# ln -s /home/off/cgi Blogs
 
ln -s SiteLang_off.pm SiteLang.pm
 
vi /home/off/startup.pl
# Needs to be configured
use lib "/home/off/cgi/";
 
 
ln -s ../sites-available/001-off.conf  001-off.conf
service apache2 restart
 
To see errors, missing perl modules etc.
 
/var/log/apache2# tail -f error.log
 
cpan
install URI::Escape::XS
install Encode::Punycode
install GraphViz2
install HTML::Defang
install Algorithm::CheckDigits
install Geo::IP
install Image::OCR::Tesseract
DateTime::Format::Mail
DateTime::Format::CLDR
DateTime::Locale
 
 
 
 
Issues:
   
MongoDB does not start:
   
    service mongodb start
root@ks3095298:/home/off/cgi# ps -ef | grep mongo
root    21814 23471  0 23:56 pts/0    00:00:00 grep mongo
 
 
in /var/log/syslog:
   
    Jan 11 00:05:05 ks3095298 systemd[1]: Starting An object/document-oriented database...
Jan 11 00:05:05 ks3095298 systemd[1]: Started An object/document-oriented database.
Jan 11 00:05:05 ks3095298 mongod[22404]: Mon Jan 11 00:05:05.434 terminate() called, printing stack (if implemented for platform):
Jan 11 00:05:05 ks3095298 mongod[22404]: 0xa6d3c6 0x57e204 0x7212186b7bb6 0x7212186b7c01 0x7212186b7e19 0x7212192b666d 0xa6070d 0xa61a67 0xa5f8ca 0xa5f979 0x5a626c 0x5a65d2 0x5a6a93 0x5a6ab9 0x5a6e5c 0x56647b 0x721217dbab45 0x57e05c
Jan 11 00:05:05 ks3095298 mongod[22404]: /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x26) [0xa6d3c6]
 
 
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com --recv EA312927
 
 
Text::Unaccent was causing Apache to segfault, replaced the module with pure perl
 
remote_ip has been replaced in Apache 2.4, need to check if we do get the X-Forwarder-For ip directly now
 
 
Loading products from /products in the mongodb database:
   
update_all_products_from_dir_in_mongodb.pl
   
   
 
Trying to use nginx for the reverse proxy:
   
apt-get install nginx  (need port 80 to be available, stop apache or change its port first)
 
 
 
root@ks3095298:/etc/nginx/sites-enabled# more /etc/nginx/sites-available/off
##
# You should look at the following URL's in order to grasp a solid understanding
# of Nginx configuration files in order to fully unleash the power of Nginx.
# http://wiki.nginx.org/Pitfalls
# http://wiki.nginx.org/QuickStart
# http://wiki.nginx.org/Configuration
#
# Generally, you will want to move this file somewhere, and start with a clean
# file but keep this around for reference. Or just disable in sites-enabled.
#
# Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples.
##
 
# Default server configuration
#
server {
        listen 80 default_server;
        listen [::]:80 default_server;
 
        server_name openfoodfacts.ovh *.openfoodfacts.ovh;
 
        # SSL configuration
        #
        # listen 443 ssl default_server;
        # listen [::]:443 ssl default_server;
        #
        # Self signed certs generated by the ssl-cert package
        # Don't use them in a production server!
        #
        # include snippets/snakeoil.conf;
 
        root /home/off/html;
 
        # Add index.php to the list if you are using PHP
        index index.html index.htm index.nginx-debian.html;
 
        location ~ ^/(images|js|rss|data|files|resources|foundation)/ {
                # First attempt to serve request as file, then
                # as directory, then fall back to displaying a 404.
                try_files $uri $uri/ =404;
        }
 
        location = /robots.txt {
                try_files $uri $uri/ =404;
        }
 
        location / {
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_pass http://127.0.0.1:8001/cgi/display.pl?;
        }
 
        location /cgi/ {
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_pass http://127.0.0.1:8001;
        }
 
        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        #
        #location ~ /\.ht {
        #      deny all;
        #}
}
 
 
</pre>
 
= Old notes =


All instructions below are for user "stephane", please use your own name. :-)
All instructions below are for user "stephane", please use your own name. :-)
Line 205: Line 571:


make
make
make install
sudo make install
cd ~
</pre>
</pre>


==== Configuration ====
==== Configuration ====


in proxy/conf/httpd.conf :
in ~/proxy/conf/httpd.conf :
* uncomment #LoadModule rewrite_module modules/mod_rewrite.so
* uncomment #LoadModule rewrite_module modules/mod_rewrite.so
* uncomment #LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
* uncomment #LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
Line 225: Line 592:
CustomLog /home/stephane/logs/proxy_access_log combined
CustomLog /home/stephane/logs/proxy_access_log combined
ServerAdmin stephane@openfoodfacts.org
ServerAdmin stephane@openfoodfacts.org
<Location "/foundation/foundation-icons" >
Header set Access-Control-Allow-Origin "*"
</Location>
<Location "/images/products" >
Header set Link "<http://creativecommons.org/licenses/by-sa/3.0/>; rel='license'; title='CC-BY-SA 3.0'"
</Location>


<Directory "/home/stephane/product-opener/html">
<Directory "/home/stephane/product-opener/html">
Line 243: Line 619:
RewriteCond  %{REQUEST_URI}  !^/data/
RewriteCond  %{REQUEST_URI}  !^/data/
RewriteCond  %{REQUEST_URI}  !^/files/
RewriteCond  %{REQUEST_URI}  !^/files/
RewriteCond  %{REQUEST_URI}  !^/resources/
RewriteCond  %{REQUEST_URI}  !^/foundation/
RewriteRule  ^(.*)$ http://localhost:8001/cgi/display.pl?${escape:$1} [P,L,QSA]
RewriteRule  ^(.*)$ http://localhost:8001/cgi/display.pl?${escape:$1} [P,L,QSA]
</VirtualHost>
</VirtualHost>
Line 264: Line 642:


</pre>
</pre>


=== Install Perl modules ===
=== Install Perl modules ===
Line 329: Line 705:
ln -s ./ Blogs
ln -s ./ Blogs
ln -s S
ln -s S
</pre>
=== Update IP to country GeoIP database ===
Product Opener uses the free legacy GeoIP Lite database from Maxmind:
* http://search.cpan.org/~maxmind/Geo-IP-1.45/lib/Geo/IP.pm
* http://dev.maxmind.com/geoip/legacy/geolite/
The database installed with apt-get install geoip-database can be very old.
To update manually, download the new databases to:
<pre>
/usr/share/GeoIP# ls -lrt
total 7256
-rw-r--r-- 1 root root 1103675  5 déc.  2013 GeoIPv6.dat
-rw-r--r-- 1 root root  600352  5 déc.  2013 GeoIP.dat
wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
gzip -d GeoIP.dat.gz
</pre>
Retrieved from "https://test-wiki.openfoodfacts.org/Special:MobileDiff/1152...22042"