Jump to content

Product Opener/Installation/Debian or Ubuntu: Difference between revisions

Product Opener/Installation/Debian or Ubuntu (view source)

Revision as of 13:38, 7 August 2024

11,414 bytes added ,  7 August
no edit summary
VisualWikitext
(→‎Install Perl modules: MongoDB perl module HAS to be installed with cpan)
No edit summary
 
(28 intermediate revisions by 6 users not shown)
Line 1: Line 1:
Also see [[Product Opener Installation]]
Also see [[Product Opener Installation]]
[[Category:ProductOpener]]
[[Category:Infrastructure]]
This page lists all the steps necessary to install Product Opener on Debian, including a development environment.
= Warning! Those instructions are outdated and kept for reference only =
Please see http://en.wiki.openfoodfacts.org/Infrastructure for up-to-date install instructions.
= Installation on Jessie =
We start from a clean Jessie install.
== Get the code and configure it ==
<pre>
apt-get install git
git clone <repo> /srv/openfoodfacts
mkdir /srv/openfoodfacts/users
mkdir /srv/openfoodfacts/products
chown -R www-data:www-data /srv/openfoodfacts
cd /srv/openfoodfacts/cgi
rm Blogs
ln -s . Blogs
ln -s SiteLang_off.pm SiteLang.pm
</pre>
Edit a few variables in Config2.pm:
<pre>
$server_domain = "yourdomain.com";
$www_root="/srv/openfoodfacts/html";
$data_root="/srv/openfoodfacts";
</pre>
Also in Config_off.pm:
<pre>
$domain = 'yourdomain.com';
$contact_email = 'webmaster@yourdomain.com';
$admin_email = 'webmaster@yourdomain.com';
</pre>
And in startup.pl:
<pre>
use lib "/srv/openfoodfacts/cgi/";
</pre>
== Install packaged prerequisites ==
<pre>
apt-get install apache2 build-essential geoip-bin geoip-database imagemagick libbarcode-zbar-perl libcache-memcached-fast-perl libclone-perl libcrypt-passwdmd5-perl libdatetime-format-mail-perl \
libencode-detect-perl libgeo-ip-perl libgraphics-color-perl libgraphviz-perl libhtml-defang-perl libimage-magick-perl libjson-perl libmime-lite-perl libmime-lite-perl libmongodb-perl libperl-dev \
libtest-nowarnings-perl libtext-unaccent-perl liburi-escape-xs-perl liburi-find-perl libwww-perl libxml-encoding-perl libxml-feedpp-perl memcached mongodb tesseract-ocr tesseract-ocr-fra zlib1g-dev
</pre>
== Prepare CPAN ==
If you never used CPAN before, you can automatically configure it:
<pre>
(echo y;echo o conf prerequisites_policy follow;echo o conf commit)|cpan
</pre>
== Install a few Perl modules from CPAN ==
<pre>
cpan Net::IDN::Punycode
cpan Encode::Punycode
cpan GraphViz2
cpan Image::OCR::Tesseract
cpan Algorithm::CheckDigits
cpan XML::Simple
</pre>
Note: on Debian Jessie, you might need to install the follwoing to be able to get cpan to install GraphViz2
<pre>
apt-get install libtest-version-perl
</pre>
== Rebuild Apache 2.2 and mod_perl ==
<pre>
cd /usr/local/src/
wget http://apache.crihan.fr/dist/httpd/httpd-2.2.31.tar.gz
tar xvfz httpd-2.2.31.tar.gz
cd httpd-2.2.31
./configure --with-mpm=prefork --prefix=/usr/local  --enable-rewrite --enable-proxy --enable-proxy_http --enable-deflate --disable-userdir --enable-headers
make
make install
cd /usr/local/src/
wget http://mir2.ovh.net/ftp.apache.org/dist/perl/mod_perl-2.0.9.tar.gz
tar xvfz mod_perl-2.0.9.tar.gz
cd mod_perl-2.0.9
perl Makefile.PL MP_APXS=/usr/local/bin/apxs
make
make install
cat >> /usr/local/conf/httpd.conf <<EOF
Listen 8001
LoadModule perl_module modules/mod_perl.so
PerlWarn Off
PerlRequire /srv/openfoodfacts/cgi/startup.pl
<VirtualHost *:8001>
  DocumentRoot /srv/openfoodfacts/html
  ServerName yourdomain.com
  ErrorLog /var/log/apache2/error.log
  CustomLog /var/log/apache2/access.log combined
  ScriptAlias /cgi/ /srv/openfoodfacts/cgi/
  <Location /cgi>
    SetHandler perl-script
    PerlResponseHandler ModPerl::Registry
    PerlOptions +ParseHeaders
    Options +ExecCGI
    Order allow,deny
    Allow from all
  </Location>
</VirtualHost>
PerlPostReadRequestHandler My::ProxyRemoteAddr
EOF
sed -i -e 's/^Listen 80$/#Listen 80/' /usr/local/conf/httpd.conf
sed -i -e 's/^User.*/User www-data/' /usr/local/conf/httpd.conf
sed -i -e 's/^Group.*/Group www-data/' /usr/local/conf/httpd.conf
/usr/local/bin/apachectl -k start
</pre>
== Configure standard Apache 2 ==
<pre>
a2enmod proxy_http
a2enmod rewrite
cat >> /etc/apache2/sites-available/product-opener-proxy.conf <<EOF
<VirtualHost *>
  DocumentRoot /srv/openfoodfacts/html
  ServerName yourdomain.com
  ServerAlias *.yourdomain.com
  <Directory "/srv/openfoodfacts/html">
    Options Indexes FollowSymLinks
    Require all granted
  </Directory>
  ProxyPreserveHost On
  RewriteEngine on
  RewriteCond  %{REQUEST_URI}  !/./
  RewriteRule ^(/cgi/.*)$ http://localhost:8001$1 [P,L]
  RewriteMap escape int:escape
  RewriteRule ^/favicon.ico$ /favicon.ico [L]
  RewriteCond  %{REQUEST_URI}  !^/images/
  RewriteCond  %{REQUEST_URI}  !^/js/
  RewriteCond  %{REQUEST_URI}  !^/foundation/
  RewriteCond  %{REQUEST_URI}  !^/rss/
  RewriteCond  %{REQUEST_URI}  !^/robots
  RewriteCond  %{REQUEST_URI}  !^/clicks/
  RewriteCond  %{REQUEST_URI}  !^/data/
  RewriteCond  %{REQUEST_URI}  !^/files/
  RewriteRule  ^(.*)$ http://localhost:8001/cgi/display.pl?${escape:$1} [P,L,QSA]
</VirtualHost>
EOF
a2ensite product-opener-proxy.conf
a2dissite 000-default.conf
service apache2 restart
</pre>
You should now have a working Productopener instance running at http://yourdomain.com/. Enjoy!
= Installation on Debian Jessie with stock Apache and nginx =
<pre>
Vous êtes sur le point d'installer un OS sur le serveur ks3095298.kimsufi.com. Choisissez le type d'installation que vous souhaitez utiliser :
-> Installer à partir d'un template OVH
Debian 8.1 stable (Jessie) (stable) - debian8
root@ks3095298:~# uname -a
Linux ks3095298.kimsufi.com 3.14.32-xxxx-grs-ipv6-64 #5 SMP Wed Sep 9 17:24:34 CEST 2015 x86_64 GNU/Linux
perl -v
This is perl 5, version 20, subversion 2 (v5.20.2) built for x86_64-linux-gnu-thread-multi
apt-get update
apt-get upgrade
apt-get install fail2ban
apt-get install sudo
apt-get install mongodb


This page lists all the steps necessary to install Product Opener on Debian, including a development environment.
MongoDB shell version: 2.4.10
 
vi /etc/mongodb.conf
- change dbpath
- uncomment :
#port = 27017
#nohttpinterface = true
 
 
apt-get install git
 
apt-get install apache2
apt-get install libapache2-mod-perl2
apt-get install modperl
 
adduser off
 
apt-get install imagemagick
apt-get install memcached
apt-get install tesseract-ocr tesseract-ocr-fra
apt-get install exim4
dpkg-reconfigure exim4-config
apt-get install geoip-bin geoip-database
apt-get install zlib1g-dev
 
apt-get install libwww-perl libimage-magick-perl libxml-encoding-perl libtext-unaccent-perl libmime-lite-perl libcache-memcached-fast-perl libjson-perl libclone-perl  libgraphviz-perl libmime-lite-perl libcrypt-passwdmd5-perl libencode-detect-perl libgraphics-color-perl libbarcode-zbar-perl libxml-feedpp-perl libmongodb-perl liburi-find-perl libxml-simple-perl
 
 
cp -a /home/openfoodfacts-bitbucket/product-opener/cgi /home/off/cgi/
 
/home/off/cgi# rm Blogs
root@ks3095298:/home/off/cgi# ln -s /home/off/cgi Blogs
 
ln -s SiteLang_off.pm SiteLang.pm
 
vi /home/off/startup.pl
# Needs to be configured
use lib "/home/off/cgi/";
 
 
ln -s ../sites-available/001-off.conf  001-off.conf
service apache2 restart
 
To see errors, missing perl modules etc.
 
/var/log/apache2# tail -f error.log
 
cpan
install URI::Escape::XS
install Encode::Punycode
install GraphViz2
install HTML::Defang
install Algorithm::CheckDigits
install Geo::IP
install Image::OCR::Tesseract
DateTime::Format::Mail
DateTime::Format::CLDR
DateTime::Locale
 
 
 
 
Issues:
   
MongoDB does not start:
   
    service mongodb start
root@ks3095298:/home/off/cgi# ps -ef | grep mongo
root    21814 23471  0 23:56 pts/0    00:00:00 grep mongo
 
 
in /var/log/syslog:
   
    Jan 11 00:05:05 ks3095298 systemd[1]: Starting An object/document-oriented database...
Jan 11 00:05:05 ks3095298 systemd[1]: Started An object/document-oriented database.
Jan 11 00:05:05 ks3095298 mongod[22404]: Mon Jan 11 00:05:05.434 terminate() called, printing stack (if implemented for platform):
Jan 11 00:05:05 ks3095298 mongod[22404]: 0xa6d3c6 0x57e204 0x7212186b7bb6 0x7212186b7c01 0x7212186b7e19 0x7212192b666d 0xa6070d 0xa61a67 0xa5f8ca 0xa5f979 0x5a626c 0x5a65d2 0x5a6a93 0x5a6ab9 0x5a6e5c 0x56647b 0x721217dbab45 0x57e05c
Jan 11 00:05:05 ks3095298 mongod[22404]: /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x26) [0xa6d3c6]
 
 
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com --recv EA312927
 
 
Text::Unaccent was causing Apache to segfault, replaced the module with pure perl
 
remote_ip has been replaced in Apache 2.4, need to check if we do get the X-Forwarder-For ip directly now
 
 
Loading products from /products in the mongodb database:
   
update_all_products_from_dir_in_mongodb.pl
   
   
 
Trying to use nginx for the reverse proxy:
   
apt-get install nginx  (need port 80 to be available, stop apache or change its port first)
 
 
 
root@ks3095298:/etc/nginx/sites-enabled# more /etc/nginx/sites-available/off
##
# You should look at the following URL's in order to grasp a solid understanding
# of Nginx configuration files in order to fully unleash the power of Nginx.
# http://wiki.nginx.org/Pitfalls
# http://wiki.nginx.org/QuickStart
# http://wiki.nginx.org/Configuration
#
# Generally, you will want to move this file somewhere, and start with a clean
# file but keep this around for reference. Or just disable in sites-enabled.
#
# Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples.
##
 
# Default server configuration
#
server {
        listen 80 default_server;
        listen [::]:80 default_server;
 
        server_name openfoodfacts.ovh *.openfoodfacts.ovh;
 
        # SSL configuration
        #
        # listen 443 ssl default_server;
        # listen [::]:443 ssl default_server;
        #
        # Self signed certs generated by the ssl-cert package
        # Don't use them in a production server!
        #
        # include snippets/snakeoil.conf;
 
        root /home/off/html;
 
        # Add index.php to the list if you are using PHP
        index index.html index.htm index.nginx-debian.html;
 
        location ~ ^/(images|js|rss|data|files|resources|foundation)/ {
                # First attempt to serve request as file, then
                # as directory, then fall back to displaying a 404.
                try_files $uri $uri/ =404;
        }
 
        location = /robots.txt {
                try_files $uri $uri/ =404;
        }
 
        location / {
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_pass http://127.0.0.1:8001/cgi/display.pl?;
        }
 
        location /cgi/ {
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_pass http://127.0.0.1:8001;
        }
 
        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        #
        #location ~ /\.ht {
        #      deny all;
        #}
}
 
 
</pre>
 
= Old notes =


All instructions below are for user "stephane", please use your own name. :-)
All instructions below are for user "stephane", please use your own name. :-)
Line 120: Line 486:
make
make
make install
make install
cd ~


cd ~/src
wget http://mir2.ovh.net/ftp.apache.org/dist/perl/mod_perl-2.0.8.tar.gz  
wget http://mir2.ovh.net/ftp.apache.org/dist/perl/mod_perl-2.0.8.tar.gz  
tar xvfz mod_perl-2.0.8.tar.gz
tar xvfz mod_perl-2.0.8.tar.gz
Line 127: Line 495:
make
make
make install
make install
cd ~
</pre>
</pre>


==== Configuration ====
==== Configuration ====


In apache/conf/httpd.conf:
In ~/apache/conf/httpd.conf:


Change the port to something else than 80 (the reverse proxy will listen on 80 and forward to the backend on another port).
Change the port to something else than 80 (the reverse proxy will listen on 80 and forward to the backend on another port).
Line 177: Line 546:
PerlPostReadRequestHandler My::ProxyRemoteAddr
PerlPostReadRequestHandler My::ProxyRemoteAddr
</pre>
</pre>


=== Apache 2.4 - light weight reverse proxy ===
=== Apache 2.4 - light weight reverse proxy ===
Line 196: Line 564:
tar xvfz apr-1.5.1.tar.gz
tar xvfz apr-1.5.1.tar.gz
tar xvfz apr-util-1.5.4.tar.gz  
tar xvfz apr-util-1.5.4.tar.gz  
ln -s apr-1.5.1 ap
ln -s apr-1.5.1 apr
ln -s apr-util-1.5.4 apr-util
ln -s apr-util-1.5.4 apr-util
cd ..  
cd ..  
Line 203: Line 571:


make
make
make install
sudo make install
cd ~
</pre>
</pre>


==== Configuration ====
==== Configuration ====


in proxy/conf/httpd.conf :
in ~/proxy/conf/httpd.conf :
* uncomment #LoadModule rewrite_module modules/mod_rewrite.so
* uncomment #LoadModule rewrite_module modules/mod_rewrite.so
* uncomment #LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
* uncomment #LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
Line 223: Line 592:
CustomLog /home/stephane/logs/proxy_access_log combined
CustomLog /home/stephane/logs/proxy_access_log combined
ServerAdmin stephane@openfoodfacts.org
ServerAdmin stephane@openfoodfacts.org
<Location "/foundation/foundation-icons" >
Header set Access-Control-Allow-Origin "*"
</Location>
<Location "/images/products" >
Header set Link "<http://creativecommons.org/licenses/by-sa/3.0/>; rel='license'; title='CC-BY-SA 3.0'"
</Location>


<Directory "/home/stephane/product-opener/html">
<Directory "/home/stephane/product-opener/html">
Line 241: Line 619:
RewriteCond  %{REQUEST_URI}  !^/data/
RewriteCond  %{REQUEST_URI}  !^/data/
RewriteCond  %{REQUEST_URI}  !^/files/
RewriteCond  %{REQUEST_URI}  !^/files/
RewriteCond  %{REQUEST_URI}  !^/resources/
RewriteCond  %{REQUEST_URI}  !^/foundation/
RewriteRule  ^(.*)$ http://localhost:8001/cgi/display.pl?${escape:$1} [P,L,QSA]
RewriteRule  ^(.*)$ http://localhost:8001/cgi/display.pl?${escape:$1} [P,L,QSA]
</VirtualHost>
</VirtualHost>
Line 262: Line 642:


</pre>
</pre>


=== Install Perl modules ===
=== Install Perl modules ===
Line 327: Line 705:
ln -s ./ Blogs
ln -s ./ Blogs
ln -s S
ln -s S
</pre>
=== Update IP to country GeoIP database ===
Product Opener uses the free legacy GeoIP Lite database from Maxmind:
* http://search.cpan.org/~maxmind/Geo-IP-1.45/lib/Geo/IP.pm
* http://dev.maxmind.com/geoip/legacy/geolite/
The database installed with apt-get install geoip-database can be very old.
To update manually, download the new databases to:
<pre>
/usr/share/GeoIP# ls -lrt
total 7256
-rw-r--r-- 1 root root 1103675  5 déc.  2013 GeoIPv6.dat
-rw-r--r-- 1 root root  600352  5 déc.  2013 GeoIP.dat
wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
gzip -d GeoIP.dat.gz
</pre>
Retrieved from "https://test-wiki.openfoodfacts.org/Special:MobileDiff/1105...22042"