Nginx with uWSGI serving dynamic Ical-Feeds generated by Python script

This weekend I invested some time to create a Python script, which parses calendars on several websites and generates a dynamic Ical feed that you can subscribe to. In this example the events are fetched from a MediaWiki table from the Hackerspace in Karlsruhe (Entropia). You can test the resulting feed here.
The server, which hosts and serves the Python uWSGI script needs following packages:

yaourt -S uwsgi uwsgi-plugin-python nginx python-dateutil python-requests python-beautifulsoup4 python-icalendar-git

The configuration file inside the uwsgi-directory defines the path to the Python scripts:

[uwsgi]
plugins = cgi
socket = 127.0.0.1:9000
chdir = /var/www/onny.project-insanity.org/calendars/
module = pyindex
cgi=/calendars=/var/www/onny.project-insanity.org/calendars/
cgi-helper =.py=python

Here’s an example script which prints out a generated Ical-feed to std.out:

#!/usr/bin/python

# This script should parse the calendar at the Entropia public wiki
# and export and subscribeable .ics iCalendar-file

# Dependencies: yaourt -S python-dateutil python-requests python-beautifulsoup4 python-icalendar-git

import requests
import json
from bs4 import BeautifulSoup
from icalendar import Calendar, Event
from datetime import datetime
import pytz
import re
import random

known_locations = {
    "Entropia": "",
    "JuBeZ": "",
    "Hochschule für Gestaltung, Karlsruhe": "",
    "PH Karlsruhe": "",
    "PH Ludwigsburg": "",
    "Karlsruhe": "",
    "HfG Karlsruhe": "",
    "CCH Hamburg": "",
    "Carl-Engler-Schule Karlsruhe": ""
}

def stripHtmlTags(htmlTxt):
    if htmlTxt is None:
        return None
    else:
        return ''.join(BeautifulSoup(htmlTxt).findAll(text=True))

class termin:
    def __init__(self, cols):
        self.date = cols[0].renderContents().decode("utf-8").lstrip().rstrip()
        self.time = cols[1].renderContents().decode("utf-8").lstrip().rstrip()
        self.location = stripHtmlTags(cols[2].renderContents().decode("utf-8").lstrip().rstrip())
        self.desc = stripHtmlTags(cols[3].renderContents().decode("utf-8").lstrip().rstrip())
        self.start = self.get_start()
        self.end = self.get_end()
    def get_start(self):

        # Check date for start/end
        if " - " in self.date:
            matchObject = re.search(r'(\d{2}.)', self.date.split(' - ')[0])
            if matchObject:
                day = matchObject.group(1)
            matchObject = re.search(r'(\d{2}.\d{4})', self.date.split(' - ')[1])
            if matchObject:
                startdate = day + matchObject.group(1)
        else:
            matchObject = re.search(r'(\d{2}.\d{2}.\d{4})', self.date)
            if matchObject:
                startdate = matchObject.group(1)

        # Check time for start/end
        if self.time:
            if " - " in self.time:
                return "critical error"
            else:
                starttime = self.time
        else:
            starttime = "00:00"

        if starttime and startdate:
            return datetime.strptime(startdate + ' ' + starttime, '%d.%m.%Y %H:%M')
        else:
            return "critical error"

    def get_end(self):

        # Check date for start/end
        if " - " in self.date:
            matchObject = re.search(r'(\d{2}.\d{2}.\d{4})', self.date.split(' - ')[1])
            if matchObject:
                enddate = matchObject.group(1)
        else:
            matchObject = re.search(r'(\d{2}.\d{2}.\d{4})', self.date)
            if matchObject:
                enddate = matchObject.group(1)

        # Check time for start/end
        if self.time:
            if " - " in self.time:
                return "critical error"
            else:
                endtime = "00:00"
        else:
            endtime = "00:00"

        if endtime and enddate:
            return datetime.strptime(enddate + ' ' + endtime, '%d.%m.%Y %H:%M')
        else:
            return "critical error"

    def show(self):
        print(self.start.strftime("%d.%m.%Y %H:%M") + " - " + self.end.strftime("%d.%m.%Y %H:%M") + " | " + self.location + " | " + self.desc)

cal = Calendar()
termine = []

# Fetch calendar entries from MediaWiki

kalender = requests.get("https://entropia.de/wiki/api.php?action=parse&format=json&page=Vorlage:Termine")
kalender_raw = kalender.text
kalender_json = json.loads(kalender_raw)
kalender_soup = BeautifulSoup(kalender_json['parse']['text']['*'])
kalender_rows = kalender_soup.find("table").findAll('tr')
for row in kalender_rows[1:]:
    cols = row.findAll('td')
    termine.append(termin(cols))

# Start generating ical feed

cal.add('prodid', '-//Entrpoia.de Events//onny.project-insanity.org//')
cal.add('version', '2.0')

for termin in termine:
    event = Event()
    event.add('summary', termin.desc)
    event.add('dtstart', termin.start)
    event.add('dtend', termin.end)
    event.add('location', termin.location)
    event.add('dtstamp', datetime(2005,4,4,0,10,0,tzinfo=pytz.utc))
    event['uid'] = datetime.now().strftime("%Y%m%dT%H%M%S") + '/' + str(random.randrange(100000)) + '@onny.project-insanity.org'
    cal.add_component(event)

print("Content-type: text/calendar; charset=utf-8")
print("Content-Disposition: inline; filename=entropia.ics\n")
print(str(cal.to_ical(),"utf-8").replace('\r',''))

The output of this script will then be redirected to the Nginx web server:

server {
        server_name .onny.project-insanity.org;
        access_log /var/log/nginx/onny.project-insanity.org.access.log;
        error_log /var/log/nginx/onny.project-insanity.org.error.log;
        root /var/www/onny.project-insanity.org/;

        location / {
                index index.htm index.html index.php;
        }

        location /calendars {
                include uwsgi_params;
                uwsgi_modifier1 9;
                uwsgi_pass 127.0.0.1:9000;
        }
[...]
systemctl restart nginx uwsgi@calendars
systemctl enable nginx uwsgi@calendars

The final feed can be validated online.

No Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

* Checkbox GDPR is required

*

I agree

Software
Virtual 3D online exhibition with MapBox GL JS

For my last semester in university (summer semester 2018) at the KIT, I was part of a project to create an “online art exhibition”. We planned to produce different media formats in smaller groups. One for video, another for text and promotion and one for the online presence. I’ve figured …

Software
Host your own Mapbox GL JS vector tiles map

I’ve done some research recently on how I could host my own online map viewer with a MapBox GL JS instance, an excellent and modern open-source alternative for Google Maps. The server should also serve own preprocessed map data from OpenStreetmap planet extracts. No external or third-party service will be …

Software
1
Easily setup Signal 2FA on Nextcloud 14

Two-factor authentication (short 2FA) is an important security concept to secure unauthorized access to your web applications. Popular online services like Google Mail, Instagram or Facebook already provide this mechanism to secure user accounts with an additional one-time token. Considering someone is able to obtain your username and password combination, …