Nginx with uWSGI serving dynamic Ical-Feeds generated by Python script

This weekend I invested some time to create a Python script, which parses calendars on several websites and generates a dynamic Ical feed that you can subscribe to. In this example the events are fetched from a MediaWiki table from the Hackerspace in Karlsruhe (Entropia). You can test the resulting feed here.
The server, which hosts and serves the Python uWSGI script needs following packages:

yaourt -S uwsgi uwsgi-plugin-python nginx python-dateutil python-requests python-beautifulsoup4 python-icalendar-git

The configuration file inside the uwsgi-directory defines the path to the Python scripts:

[uwsgi]
plugins = cgi
socket = 127.0.0.1:9000
chdir = /var/www/onny.project-insanity.org/calendars/
module = pyindex
cgi=/calendars=/var/www/onny.project-insanity.org/calendars/
cgi-helper =.py=python

Here’s an example script which prints out a generated Ical-feed to std.out:

#!/usr/bin/python

# This script should parse the calendar at the Entropia public wiki
# and export and subscribeable .ics iCalendar-file

# Dependencies: yaourt -S python-dateutil python-requests python-beautifulsoup4 python-icalendar-git

import requests
import json
from bs4 import BeautifulSoup
from icalendar import Calendar, Event
from datetime import datetime
import pytz
import re
import random

known_locations = {
    "Entropia": "",
    "JuBeZ": "",
    "Hochschule für Gestaltung, Karlsruhe": "",
    "PH Karlsruhe": "",
    "PH Ludwigsburg": "",
    "Karlsruhe": "",
    "HfG Karlsruhe": "",
    "CCH Hamburg": "",
    "Carl-Engler-Schule Karlsruhe": ""
}

def stripHtmlTags(htmlTxt):
    if htmlTxt is None:
        return None
    else:
        return ''.join(BeautifulSoup(htmlTxt).findAll(text=True))

class termin:
    def __init__(self, cols):
        self.date = cols[0].renderContents().decode("utf-8").lstrip().rstrip()
        self.time = cols[1].renderContents().decode("utf-8").lstrip().rstrip()
        self.location = stripHtmlTags(cols[2].renderContents().decode("utf-8").lstrip().rstrip())
        self.desc = stripHtmlTags(cols[3].renderContents().decode("utf-8").lstrip().rstrip())
        self.start = self.get_start()
        self.end = self.get_end()
    def get_start(self):

        # Check date for start/end
        if " - " in self.date:
            matchObject = re.search(r'(\d{2}.)', self.date.split(' - ')[0])
            if matchObject:
                day = matchObject.group(1)
            matchObject = re.search(r'(\d{2}.\d{4})', self.date.split(' - ')[1])
            if matchObject:
                startdate = day + matchObject.group(1)
        else:
            matchObject = re.search(r'(\d{2}.\d{2}.\d{4})', self.date)
            if matchObject:
                startdate = matchObject.group(1)

        # Check time for start/end
        if self.time:
            if " - " in self.time:
                return "critical error"
            else:
                starttime = self.time
        else:
            starttime = "00:00"

        if starttime and startdate:
            return datetime.strptime(startdate + ' ' + starttime, '%d.%m.%Y %H:%M')
        else:
            return "critical error"

    def get_end(self):

        # Check date for start/end
        if " - " in self.date:
            matchObject = re.search(r'(\d{2}.\d{2}.\d{4})', self.date.split(' - ')[1])
            if matchObject:
                enddate = matchObject.group(1)
        else:
            matchObject = re.search(r'(\d{2}.\d{2}.\d{4})', self.date)
            if matchObject:
                enddate = matchObject.group(1)

        # Check time for start/end
        if self.time:
            if " - " in self.time:
                return "critical error"
            else:
                endtime = "00:00"
        else:
            endtime = "00:00"

        if endtime and enddate:
            return datetime.strptime(enddate + ' ' + endtime, '%d.%m.%Y %H:%M')
        else:
            return "critical error"

    def show(self):
        print(self.start.strftime("%d.%m.%Y %H:%M") + " - " + self.end.strftime("%d.%m.%Y %H:%M") + " | " + self.location + " | " + self.desc)

cal = Calendar()
termine = []

# Fetch calendar entries from MediaWiki

kalender = requests.get("https://entropia.de/wiki/api.php?action=parse&format=json&page=Vorlage:Termine")
kalender_raw = kalender.text
kalender_json = json.loads(kalender_raw)
kalender_soup = BeautifulSoup(kalender_json['parse']['text']['*'])
kalender_rows = kalender_soup.find("table").findAll('tr')
for row in kalender_rows[1:]:
    cols = row.findAll('td')
    termine.append(termin(cols))

# Start generating ical feed

cal.add('prodid', '-//Entrpoia.de Events//onny.project-insanity.org//')
cal.add('version', '2.0')

for termin in termine:
    event = Event()
    event.add('summary', termin.desc)
    event.add('dtstart', termin.start)
    event.add('dtend', termin.end)
    event.add('location', termin.location)
    event.add('dtstamp', datetime(2005,4,4,0,10,0,tzinfo=pytz.utc))
    event['uid'] = datetime.now().strftime("%Y%m%dT%H%M%S") + '/' + str(random.randrange(100000)) + '@onny.project-insanity.org'
    cal.add_component(event)

print("Content-type: text/calendar; charset=utf-8")
print("Content-Disposition: inline; filename=entropia.ics\n")
print(str(cal.to_ical(),"utf-8").replace('\r',''))

The output of this script will then be redirected to the Nginx web server:

server {
        server_name .onny.project-insanity.org;
        access_log /var/log/nginx/onny.project-insanity.org.access.log;
        error_log /var/log/nginx/onny.project-insanity.org.error.log;
        root /var/www/onny.project-insanity.org/;

        location / {
                index index.htm index.html index.php;
        }

        location /calendars {
                include uwsgi_params;
                uwsgi_modifier1 9;
                uwsgi_pass 127.0.0.1:9000;
        }
[...]
systemctl restart nginx uwsgi@calendars
systemctl enable nginx uwsgi@calendars

The final feed can be validated online.

No Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

* Checkbox GDPR is required

*

I agree

Software
Officepad – Etherpad alternative based on Onlyoffice

During the Chaos Communication Camp 2019 I was working on a project called Officepad. The aim was to build a web interface comparable to Etherpad. Etherpad is a well established tool for online text collaboration used by software developers, activists and so on. Etherpad is relatively stable and works well …

Software
2
Bye bye Play Store! Mirroring Play Store Apks to private F-Droid repo

This small tutorial will show you how to setup your own F-Droid repository containing automatically mirrored Play Store apps. Using this private repo, you don’t have to relay on Google Play Store or third party stores anymore to keep non-free apps up-to-date. All you need to have is a Linux …

Software
Ebook reader app for Nextcloud

Because of lack of altenatives I decided to write a small ebook reader plugin for Nextcloud. The task of creating such an app was quite easy: I just forked the files_pdfviewer extension and replaced PDF.js with the Epub.js library. In the app template file, I used the reference ebook reader …