We are delighted to be able to share our eighth monthly architects' newsletter with you, which we will send on the last Friday of every month. InfoQ strives to facilitate the spread of knowledge and innovation within this space, and in this newsletter we aim to curate and summarise key learnings from news items, articles and presentations created by industry peers, both on InfoQ and across the web. We aim to keep readers informed and educated about emerging trends, peer-validated early adoption of technologies, and architectural best practices, and are always keen to receive feedback from our readers.

News

The Challenges of Developing Event-Driven Serverless Systems

In a recent "A Cloud Guru" blog post, Rob Gruhl discusses the design considerations of Nordstrom's event-sourced architecture working off a distributed ledger. In the fall of 2014, Nordstrom experienced several "wall of traffic" incidents when the organisation sent out millions of new, personalised marketing emails. At that time AWS Kinesis and Lambda had just been released, and so the team decided to investigate utilising a real time stream of data using a combination of Kinesis and Lambda.

Gruhl discusses the use of a "distributed ledger" pattern -- an immutable, replicated, partitioned, distributed, append-only log with multi-reader capability -- via Kinesis Data Streams. There are several benefits from leveraging this pattern, one example being able to replay streams when issues emerge, "from moving the backpressure you have from your consumers off of your producers, from having multiple consumers of the same stream, even new versions of old systems running in parallel while in development". A key takeaway from the article is the benefits from embracing event-driven architecture when working with FaaS technologies: "you just naturally start thinking in terms of events when you go serverless. It's very impedance-matched to the technology".

Use the Serverless Cloud for ETL Scenarios

Jeremy Likness has created a GitHub repository with code samples that demonstrate several capabilities and workflows associated with serverless application development for Extract, Transform, and Load (ETL) using Azure Functions. Likness believes that ETL is a common scenario in enterprise applications that serverless applications are uniquely positioned to address. Developers are able to handle file uploads, set timers and schedules, and run transformations using serverless without touching hardware, installing third-party services or writing custom polling software.

The State of OpenWhisk

The Apache OpenWhisk serverless functions platform, borne out of IBM Research, is currently gearing up for its first official Apache release -- this is an important milestone on the road to graduating from the incubation phase. Rodric Rabbah discusses "The State of OpenWhisk" in a recent post on Medium, where the current pluggable platform architecture is also described alongside recommended best practices for application design. For engineers keen to experiment with the platform, the full OpenWhisk stack may be deployed in several ways. For local development, the project supports quick start options via Docker Compose (60 seconds quick start), Docker for Mac, and Vagrant. It may also be deployed on top of virtual machines, DC/OS (Mesos) and, of course, Kubernetes.

Announcing Gloo: The Function Gateway

Solo have released Gloo, a "Function Gateway". Gloo was designed on the principle that gateways should build APIs out of functions rather than services. By allowing engineers to build APIs from functions -- arguably the smallest unit of compute -- Gloo equips them with a high level of control over how APIs are built. Gloo is built on top of Lyft's Envoy Proxy, the data plane of the increasingly popular Istio service mesh, and competes with other open source projects like the Ambassador API Gateway (also built on Envoy) and Spring Cloud Gateway, and commercial offerings like Amazon API Gateway and Azure API Management.

Gloo provides a unified entry point for access to all services and serverless functions, translating from any client-facing API to any backend function. The gateway also aggregates HTTP/REST APIs and events from clients, "glueing" together functions "in-cluster, out of cluster, across clusters, along with any provider of serverless functions". Gloo features a pluggable architecture, with an extendable configuration language, pluggable upstream types, discovery services, and internal components.

AWS Makes Serverless Application Repository Generally Available

After a brief preview period since re:Invent 2017, Amazon Web Services (AWS) recently made its new Serverless Application Repository generally available. Users can now discover, configure, and deploy serverless applications and components via the AWS Lambda console.

AWS introduced the Serverless Application Model (SAM) at re:Invent 2016 with the goal to make it easier to "manage and deploy serverless applications on AWS". AWS then followed up with the Docker-based SAM Local so that developers can test functions locally via simulating an API Gateway and enabling the generation of sample payloads for various event sources. AWS has now added the Serverless Application Repository as a dedicated packaging, discovery and provisioning service for SAM applications. Similar to the still available blueprints feature, it integrates with the AWS Lambda console's create function wizard to guide users through the discovery and deployment steps.

This edition of The Software Architects' Newsletter is brought to you by:

How Microservices Differ Among Different Platforms

Microservices is a conceptual approach, and as such it is handled differently in each language. This is a strength of the architecture because developers can use the language they are most familiar with. Older languages can use microservices by using a structure unique to that platform. Here are some of the characteristics of microservices on different platforms:

• Java - Avoids using Web Archive or Enterprise Archive files; Components are not auto-deployed. Instead, Docker containers or Amazon Machine Images are auto-deployed.
• .NET - Building a .NET self-hosted web service using Open Web Interface for .NET (OWIN). You can then use it to incorporate microservices.
• Python - It’s easy to create a Python service that acts as a front-end web service for microservices in other languages such as ASP or PHP.

Case Study

Service Security -- What's Left to Protect?

At the recent QCon London conference, Guy Podjarny presented an updated version of his "Securing Serverless - By Breaking In" talk. This is also complemented by a full length InfoQ article, "Serverless Security: What's Left to Protect?" in which Podjarny enumerates and discusses the security responsibilities for developers deploying "serverless" Functions-as-a-Service (FaaS) applications.

Although the term "serverless" is a misnomer -- serverless technologies still use servers (and OS containers) behind the scenes -- FaaS vendors do take on the responsibility for "patching" the underlying servers, freeing engineers from OS patching. Known vulnerabilities in unpatched servers and apps are the primary vector through which systems are exploited, due to their frequency and broad deployment, along with the fact that updating apps and servers at scale is hard.

The elastic nature of FaaS platforms means that Denial of Service (DoS) attacks are naturally thwarted by the (presumed) infinite capacity offered by serverless solutions, although Podjarny warned in his talk that this can have serious financial implications without proper account scaling limits in place.

The finer granularity of application code (functions versus microservices/monoliths) lets engineers implement the "least privilege" concept well, minimizing the damage a compromised function can cause -- in theory. In practice, however, managing granular permissions for hundreds or thousands of functions is very hard to do unless automation is used. Podjarny recommends watching Aaron Kammerer's talk explaining (amongst other Serverless Ops topics) how iRobot auto tunes Lambda permissions.

While FaaS platforms handle the server-level security concerns, it cannot be expected that attackers will simply give up! Instead, they will shift their attention to the areas that remain exposed - and first amongst those would be the application itself. The chance a given function will have a SQL Injection, Cross-Site Scripting or Command Injection vulnerability is neither higher or lower with Serverless, but these vectors may get extra attention from attackers in the FaaS world. API gateways also offer an opportunity to create schemas and whitelists that reduce the chance of a malicious input making its way in.

A key takeaway for function developers is that the responsibility for addressing vulnerable third-party application libraries and dependencies lies firmly with them. Podjarny suggests that when developing functions, the developer must continuously monitor for and fix known vulnerabilities within libraries, using tools such as Snyk, Victims DB, or the OWASP Dependency check. More can be learnt about this in Podjarny's book, "Securing Open Source Libraries", which is also available in a series of posts on the O'Reilly blog.

Additional information on securing serverless applications can be found in Jeremy Daly's blog post "Securing Serverless: A Newbie's Guide", which discusses implementation details in more depth, and also provides several useful references. For engineers looking for a practical introduction to securing their own serverless platform using OpenFaaS, Daniel Shapira has written "Securing Kubernetes for OpenFaaS and beyond", which demonstrates how to override basic platform defaults and enable isolation between pods and implement authentication.

The upcoming introduction of the General Data Protection Regulation (GDPR), which will apply to any organisation working with data from the European Union (EU), adds even more impetus for ensuring infrastructure is secure. In a recent InfoQ article, "GDPR for Operations", Jon Topper has explored this topic in depth.

To get notifications when InfoQ publishes content on this topic follow Serverless on InfoQ.

This is the eighth issue of a monthly newsletter, focusing exclusively on software architecture. We thought it would be valuable for you to get a quick overview of things you might like to keep an eye on. If this is not the case, you can unsubscribe using the link below.

Forwarded email? Subscribe and get your own copy.