You don't need a data service, you just need an object store and some JSON files

Speaker: Gordon Inggs

Track: Web

Type: Talk

Room: Lefthand Room (Seminar Room 3)

Time: Oct 04 (Fri): 14:00

Duration: 0:45

What are you talking about?

In this talk, I will describe an approach to building highly scalable services that is both extremely reliable and dirt cheap to run. The essence of our approach is eschewing popular service frameworks such as Flask, and instead sweating the HTTP API of object stores, such as those made available by large cloud providers, or on-premise options such as Minio or Ceph, along with supporting infrastructure such as data pipelines and content delivery networks.

To illustrate this concept, I'll talk through the architecture of the City's open source Service Alert API, and how we provide a near real time data API which is well specified using an OpenAPI spec, but is really just a few JSON files in AWS's S3.

Who is this for?

This should be interesting to anyone that is nervous about having to support a large user base, but has important information that they need to make available. For small teams or those lacking DevOps capacity, we believe this is an extremely viable approach. It's also potentially a cautionary tale about what happens when the hubris of Man is left unchecked.


Thinkst Canary
Python Software Foundation SARAO
AWS City of Cape Town
Afrolabs Centre for High Performance Computing
Black Python Devs