Nginx with uWSGI serving dynamic Ical-Feeds generated by Python script

This weekend I invested some time to create a Python script, which parses calendars on several websites and generates a dynamic Ical feed that you can subscribe to. In this example the events are fetched from a MediaWiki table from the Hackerspace in Karlsruhe (Entropia). You can test the resulting feed here.
The server, which hosts and serves the Python uWSGI script needs following packages:

yaourt -S uwsgi uwsgi-plugin-python nginx python-dateutil python-requests python-beautifulsoup4 python-icalendar-git

The configuration file inside the uwsgi-directory defines the path to the Python scripts:

[uwsgi]
plugins = cgi
socket = 127.0.0.1:9000
chdir = /var/www/onny.project-insanity.org/calendars/
module = pyindex
cgi=/calendars=/var/www/onny.project-insanity.org/calendars/
cgi-helper =.py=python

Here’s an example script which prints out a generated Ical-feed to std.out:

#!/usr/bin/python

# This script should parse the calendar at the Entropia public wiki
# and export and subscribeable .ics iCalendar-file

# Dependencies: yaourt -S python-dateutil python-requests python-beautifulsoup4 python-icalendar-git

import requests
import json
from bs4 import BeautifulSoup
from icalendar import Calendar, Event
from datetime import datetime
import pytz
import re
import random

known_locations = {
    "Entropia": "",
    "JuBeZ": "",
    "Hochschule für Gestaltung, Karlsruhe": "",
    "PH Karlsruhe": "",
    "PH Ludwigsburg": "",
    "Karlsruhe": "",
    "HfG Karlsruhe": "",
    "CCH Hamburg": "",
    "Carl-Engler-Schule Karlsruhe": ""
}

def stripHtmlTags(htmlTxt):
    if htmlTxt is None:
        return None
    else:
        return ''.join(BeautifulSoup(htmlTxt).findAll(text=True))

class termin:
    def __init__(self, cols):
        self.date = cols[0].renderContents().decode("utf-8").lstrip().rstrip()
        self.time = cols[1].renderContents().decode("utf-8").lstrip().rstrip()
        self.location = stripHtmlTags(cols[2].renderContents().decode("utf-8").lstrip().rstrip())
        self.desc = stripHtmlTags(cols[3].renderContents().decode("utf-8").lstrip().rstrip())
        self.start = self.get_start()
        self.end = self.get_end()
    def get_start(self):

        # Check date for start/end
        if " - " in self.date:
            matchObject = re.search(r'(\d{2}.)', self.date.split(' - ')[0])
            if matchObject:
                day = matchObject.group(1)
            matchObject = re.search(r'(\d{2}.\d{4})', self.date.split(' - ')[1])
            if matchObject:
                startdate = day + matchObject.group(1)
        else:
            matchObject = re.search(r'(\d{2}.\d{2}.\d{4})', self.date)
            if matchObject:
                startdate = matchObject.group(1)

        # Check time for start/end
        if self.time:
            if " - " in self.time:
                return "critical error"
            else:
                starttime = self.time
        else:
            starttime = "00:00"

        if starttime and startdate:
            return datetime.strptime(startdate + ' ' + starttime, '%d.%m.%Y %H:%M')
        else:
            return "critical error"

    def get_end(self):

        # Check date for start/end
        if " - " in self.date:
            matchObject = re.search(r'(\d{2}.\d{2}.\d{4})', self.date.split(' - ')[1])
            if matchObject:
                enddate = matchObject.group(1)
        else:
            matchObject = re.search(r'(\d{2}.\d{2}.\d{4})', self.date)
            if matchObject:
                enddate = matchObject.group(1)

        # Check time for start/end
        if self.time:
            if " - " in self.time:
                return "critical error"
            else:
                endtime = "00:00"
        else:
            endtime = "00:00"

        if endtime and enddate:
            return datetime.strptime(enddate + ' ' + endtime, '%d.%m.%Y %H:%M')
        else:
            return "critical error"

    def show(self):
        print(self.start.strftime("%d.%m.%Y %H:%M") + " - " + self.end.strftime("%d.%m.%Y %H:%M") + " | " + self.location + " | " + self.desc)

cal = Calendar()
termine = []

# Fetch calendar entries from MediaWiki

kalender = requests.get("https://entropia.de/wiki/api.php?action=parse&format=json&page=Vorlage:Termine")
kalender_raw = kalender.text
kalender_json = json.loads(kalender_raw)
kalender_soup = BeautifulSoup(kalender_json['parse']['text']['*'])
kalender_rows = kalender_soup.find("table").findAll('tr')
for row in kalender_rows[1:]:
    cols = row.findAll('td')
    termine.append(termin(cols))

# Start generating ical feed

cal.add('prodid', '-//Entrpoia.de Events//onny.project-insanity.org//')
cal.add('version', '2.0')

for termin in termine:
    event = Event()
    event.add('summary', termin.desc)
    event.add('dtstart', termin.start)
    event.add('dtend', termin.end)
    event.add('location', termin.location)
    event.add('dtstamp', datetime(2005,4,4,0,10,0,tzinfo=pytz.utc))
    event['uid'] = datetime.now().strftime("%Y%m%dT%H%M%S") + '/' + str(random.randrange(100000)) + '@onny.project-insanity.org'
    cal.add_component(event)

print("Content-type: text/calendar; charset=utf-8")
print("Content-Disposition: inline; filename=entropia.ics\n")
print(str(cal.to_ical(),"utf-8").replace('\r',''))

The output of this script will then be redirected to the Nginx web server:

server {
        server_name .onny.project-insanity.org;
        access_log /var/log/nginx/onny.project-insanity.org.access.log;
        error_log /var/log/nginx/onny.project-insanity.org.error.log;
        root /var/www/onny.project-insanity.org/;

        location / {
                index index.htm index.html index.php;
        }

        location /calendars {
                include uwsgi_params;
                uwsgi_modifier1 9;
                uwsgi_pass 127.0.0.1:9000;
        }
[...]
systemctl restart nginx uwsgi@calendars
systemctl enable nginx uwsgi@calendars

The final feed can be validated online.

No Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

* Checkbox GDPR is required

*

I agree

Software
2
Bye bye Play Store! Mirroring Play Store Apks to private F-Droid repo

This small tutorial will show you how to setup your own F-Droid repository containing automatically mirrored Play Store apps. Using this private repo, you don’t have to relay on Google Play Store or third party stores anymore to keep non-free apps up-to-date. All you need to have is a Linux …

Software
Ebook reader app for Nextcloud

Because of lack of altenatives I decided to write a small ebook reader plugin for Nextcloud. The task of creating such an app was quite easy: I just forked the files_pdfviewer extension and replaced PDF.js with the Epub.js library. In the app template file, I used the reference ebook reader …

Software
Hacking replay gain audio normalization into Jellyfin

There is already a feature request for audio normalization in Jellyfin media server. This is important if you want to listen to your music collection while always having the same loudness level. Usually, in different recordings or music genres some tracks are louder and others are more quiet. The standard …