Dedicated producion VM setup

System setup

Making things work behind the proxy

• git git config --global http.proxy http://fifi.arz.oeaw.ac.at:8080/
• docker mkdir /etc/systemd/system/docker.service.d cat > /etc/systemd/system/docker.service.d/http-proxy.conf <<EOL [Service] Environment="HTTP_PROXY=http://fifi.arz.oeaw.ac.at:8080/" Environment="HTTPS_PROXY=http://fifi.arz.oeaw.ac.at:8080/" Environment="NO_PROXY=127.0.0.1,localhost,10.3.6.33" EOL systemctl daemon-reload systemctl restart docker

Deployment

A custom systemd service using docker-compose with the docker-compose.yaml in '/opt/prod`.

``` cat > /etc/systemd/system/oeawresourcesprod.service <<EOL [Unit] Description=OEAW Resources Production Instance Wants=docker.service After=docker.service

[Service] Type=simple Restart=always WorkingDirectory=/opt/prod ExecStart=/usr/bin/docker-compose up

[Install] WantedBy=multi-user.target EOL

systemctl daemon-reload systemctl start oeawresourcesprod systemctl enable oeawresourcesprod ```

Cron jobs

Harvesting and indexing cron jobs set up on root user account.

```

Redeployment

*/5 * * * * /opt/prod/pull_redeploy.sh

Thumbnails

20 * * * * docker run --rm -v prodthumbnails:/var/www/cache/covers --entrypoint /bin/sh nondef/thumbnailfetcher:latest -c 'node dist/bin/index.js fetch -p https://raw.githubusercontent.com/acdh-oeaw/AkSearchWeb/main/local/config/thumbnailfetcher/basisarchive.json -a && chown -R 33:33 /var/www/cache/covers' > /opt/prod/log/fetch-thumbnails.log 2>&1

Harvesting

05 * * * * docker exec prodweb1 /bin/sh -c 'flock -n /tmp/harvestarchive-basis.lock -c "/var/www/vufind/harvest/harvestoai.sh archivebasis"' > /opt/prod/log/harvestarchivebasis.log 2>&1 10 * * * * docker exec prodweb1 /bin/sh -c 'flock -n /tmp/harvestarchive-oeai.lock -c "/var/www/vufind/harvest/harvestoai.sh archiveoeai"' > /opt/prod/log/harvestarchive-oeai.log 2>&1 15 * * * * docker exec prodweb1 /bin/sh -c 'flock -n /tmp/harvestbib-alma.lock -c "/var/www/vufind/harvest/harvestoai.sh bibalma && chown -R 8983:www-data /var/www/local/harvest/bibalma && [ -f /var/www/local/harvest/bibalma/.delete ] && { sed -i \"s/oai:alma.43ACCOEAW://g\" /var/www/local/harvest/bibalma/.delete; /usr/local/vufind/harvest/batch-delete.sh ../../../../var/www/local/harvest/bibalma/; } || echo \"No deletions submitted.\""' > /opt/prod/log/harvestbib-alma.log 2>&1 00 0 1 * * docker exec prodweb1 /bin/sh -c 'flock -n /tmp/harvestdegruyter.lock -c "/var/www/vufind/harvest/harvestdegruyter.sh /var/www/local/harvest/harvestdegruyter.json $DEGRUYTERPSWD && chown -R 8983:www-data /var/www/local/harvest/bibdegruyter"' > /opt/prod/log/harvestdegruyter.log 2>&1

Indexing

35 * * * * docker exec prodsolr1 /bin/sh -c 'flock -n /tmp/indexarchive-basis.lock -c "cd /opt/aksearch && harvest/batch-import-marc.sh -d -p /opt/local/import/importarchivebasis.properties /opt/harvest/archivebasis"' > /opt/prod/log/indexarchive-basis.log 2>&1 40 * * * * docker exec prodsolr1 /bin/sh -c 'flock -n /tmp/indexarchive-oeai.lock -c "cd /opt/aksearch && harvest/batch-import-marc.sh -d -p /opt/local/import/importarchiveoeai.properties /opt/harvest/archiveoeai"' > /opt/prod/log/indexarchive-oeai.log 2>&1 45 * * * * docker exec prodsolr1 /bin/sh -c 'flock -n /tmp/indexbib-alma.lock -c "cd /opt/aksearch && harvest/batch-import-marc.sh -d -p /opt/local/import/importbibalma.properties /opt/harvest/bibalma"' > /opt/prod/log/indexbib-alma.log 2>&1 30 0 1 * * docker exec prodsolr1 /bin/sh -c 'flock -n /tmp/indexdegruyter.lock -c "cd /opt/aksearch && harvest/batch-import-marc.sh -d -p /opt/local/import/importbibdegruyter.properties /opt/harvest/bibdegruyter"' > /opt/prod/log/indexbib-degruyter.log 2>&1 ```

Redeployment script:

``` cat > /opt/prod/pull_redeploy.sh <<EOL

!/bin/bash

for i in `docker ps --format '{{.Image}}' | grep -E '/|:'` ; do docker pull \$i done

if any running container runs an untagged image, restart

docker ps -f name=prod* --format '{{.Image}}' | grep -q -v -E '/|:' if [ "\$?" == "0" ] ; then systemctl restart oeawresourcesprod docker image prune -f fi EOL chmod +x /opt/prod/pullredeploy.sh ```

Reverse proxy

Ask ARZ to proxy the https://www.oeaw.ac.at/resources/ to the http://10.3.6.33:8080/resources/