Nginx with uWSGI serving dynamic Ical-Feeds generated by Python script

This weekend I invested some time to create a Python script, which parses calendars on several websites and generates a dynamic Ical feed that you can subscribe to. In this example the events are fetched from a MediaWiki table from the Hackerspace in Karlsruhe (Entropia). You can test the resulting feed here.
The server, which hosts and serves the Python uWSGI script needs following packages:

yaourt -S uwsgi uwsgi-plugin-python nginx python-dateutil python-requests python-beautifulsoup4 python-icalendar-git

The configuration file inside the uwsgi-directory defines the path to the Python scripts:

[uwsgi]
plugins = cgi
socket = 127.0.0.1:9000
chdir = /var/www/onny.project-insanity.org/calendars/
module = pyindex
cgi=/calendars=/var/www/onny.project-insanity.org/calendars/
cgi-helper =.py=python

Here’s an example script which prints out a generated Ical-feed to std.out:

#!/usr/bin/python

# This script should parse the calendar at the Entropia public wiki
# and export and subscribeable .ics iCalendar-file

# Dependencies: yaourt -S python-dateutil python-requests python-beautifulsoup4 python-icalendar-git

import requests
import json
from bs4 import BeautifulSoup
from icalendar import Calendar, Event
from datetime import datetime
import pytz
import re
import random

known_locations = {
    "Entropia": "",
    "JuBeZ": "",
    "Hochschule für Gestaltung, Karlsruhe": "",
    "PH Karlsruhe": "",
    "PH Ludwigsburg": "",
    "Karlsruhe": "",
    "HfG Karlsruhe": "",
    "CCH Hamburg": "",
    "Carl-Engler-Schule Karlsruhe": ""
}

def stripHtmlTags(htmlTxt):
    if htmlTxt is None:
        return None
    else:
        return ''.join(BeautifulSoup(htmlTxt).findAll(text=True))

class termin:
    def __init__(self, cols):
        self.date = cols[0].renderContents().decode("utf-8").lstrip().rstrip()
        self.time = cols[1].renderContents().decode("utf-8").lstrip().rstrip()
        self.location = stripHtmlTags(cols[2].renderContents().decode("utf-8").lstrip().rstrip())
        self.desc = stripHtmlTags(cols[3].renderContents().decode("utf-8").lstrip().rstrip())
        self.start = self.get_start()
        self.end = self.get_end()
    def get_start(self):

        # Check date for start/end
        if " - " in self.date:
            matchObject = re.search(r'(\d{2}.)', self.date.split(' - ')[0])
            if matchObject:
                day = matchObject.group(1)
            matchObject = re.search(r'(\d{2}.\d{4})', self.date.split(' - ')[1])
            if matchObject:
                startdate = day + matchObject.group(1)
        else:
            matchObject = re.search(r'(\d{2}.\d{2}.\d{4})', self.date)
            if matchObject:
                startdate = matchObject.group(1)

        # Check time for start/end
        if self.time:
            if " - " in self.time:
                return "critical error"
            else:
                starttime = self.time
        else:
            starttime = "00:00"

        if starttime and startdate:
            return datetime.strptime(startdate + ' ' + starttime, '%d.%m.%Y %H:%M')
        else:
            return "critical error"

    def get_end(self):

        # Check date for start/end
        if " - " in self.date:
            matchObject = re.search(r'(\d{2}.\d{2}.\d{4})', self.date.split(' - ')[1])
            if matchObject:
                enddate = matchObject.group(1)
        else:
            matchObject = re.search(r'(\d{2}.\d{2}.\d{4})', self.date)
            if matchObject:
                enddate = matchObject.group(1)

        # Check time for start/end
        if self.time:
            if " - " in self.time:
                return "critical error"
            else:
                endtime = "00:00"
        else:
            endtime = "00:00"

        if endtime and enddate:
            return datetime.strptime(enddate + ' ' + endtime, '%d.%m.%Y %H:%M')
        else:
            return "critical error"

    def show(self):
        print(self.start.strftime("%d.%m.%Y %H:%M") + " - " + self.end.strftime("%d.%m.%Y %H:%M") + " | " + self.location + " | " + self.desc)

cal = Calendar()
termine = []

# Fetch calendar entries from MediaWiki

kalender = requests.get("https://entropia.de/wiki/api.php?action=parse&format=json&page=Vorlage:Termine")
kalender_raw = kalender.text
kalender_json = json.loads(kalender_raw)
kalender_soup = BeautifulSoup(kalender_json['parse']['text']['*'])
kalender_rows = kalender_soup.find("table").findAll('tr')
for row in kalender_rows[1:]:
    cols = row.findAll('td')
    termine.append(termin(cols))

# Start generating ical feed

cal.add('prodid', '-//Entrpoia.de Events//onny.project-insanity.org//')
cal.add('version', '2.0')

for termin in termine:
    event = Event()
    event.add('summary', termin.desc)
    event.add('dtstart', termin.start)
    event.add('dtend', termin.end)
    event.add('location', termin.location)
    event.add('dtstamp', datetime(2005,4,4,0,10,0,tzinfo=pytz.utc))
    event['uid'] = datetime.now().strftime("%Y%m%dT%H%M%S") + '/' + str(random.randrange(100000)) + '@onny.project-insanity.org'
    cal.add_component(event)

print("Content-type: text/calendar; charset=utf-8")
print("Content-Disposition: inline; filename=entropia.ics\n")
print(str(cal.to_ical(),"utf-8").replace('\r',''))

The output of this script will then be redirected to the Nginx web server:

server {
        server_name .onny.project-insanity.org;
        access_log /var/log/nginx/onny.project-insanity.org.access.log;
        error_log /var/log/nginx/onny.project-insanity.org.error.log;
        root /var/www/onny.project-insanity.org/;

        location / {
                index index.htm index.html index.php;
        }

        location /calendars {
                include uwsgi_params;
                uwsgi_modifier1 9;
                uwsgi_pass 127.0.0.1:9000;
        }
[...]
systemctl restart nginx uwsgi@calendars
systemctl enable nginx uwsgi@calendars

The final feed can be validated online.

No Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

* Checkbox GDPR is required

*

I agree

Software
Importing playlists to Jellyfin media server

In the following post I would like to describe a way to import music playlists to Jellyfin media server, for example in case you want to migrate them from a former Plex or Emby installation. Usually, Jellyfin is already able to detect playlist files within the media library. Unfortunately my …

Software
Jellyfin media server on Archlinux ARM

In this post, I want to share some insights on building Jellyfin media server for Archlinux ARM. The PKGBUILD for Jellyfin one can find on the AUR, is specifically made for 64 bit architectures. Nevertheless Microsoft released the dotnet runtime, which Jellyfin relies on instead of Mono, also for Linux …

Software
2
Voice control Archlinux with Amazon Alexa

I was interested to see how commercial voice recognition software would behave on an usual Linux laptop and tried to deploy an Amazon Alexa instance. There are some installation scripts and tutorials available for Ubuntu and Raspberry PI using the avs-device-sdk from Github. Even though some of them are official …