Best Practices for Serverless – Connection Pooling Your Database

Connection Pooling Is Key For Speedy DB Calls; Here’s How To Do It With Functions

Wait, what’s connection pooling?

Let’s refresh our memories on connection pooling. In short, connection pooling helps reduce the number of resources required for connecting to a database, speeding up the connection. Instead of the old way, where you would need to open a connection with the correct credentials each time you make a request to the DB, connection pooling does just that. It pools the connections to the DB, maintaining a connection to the DB and greatly reducing the number of connections that must be opened.

The pooler maintains ownership of the physical connection with the DB by keeping a set of active connections alive. Whenever a user calls on a connection, the pooler searches for an available connection, using that one rather than opening a new one. When the application calls to Close the connection, that connection is instead returned to the pool of connections. So next time there’s a call to the DB, that one will already be opened.

By maintaining an open pool of connections to the DB, connection pooling is essential for speedy database connectivity. But on Serverless, many consider connection pooling to be impossible. So we’re here to bust that myth.

Why connection pooling “can’t be done” on Serverless

Well obviously, we say it can (and we’ve done it), thus the quotation marks on “can’t be done”. That said, there are a lot of smart people using a lot of strong logic to explain why it can’t be done. Let’s address that first.

In that article we linked to above, Rowan Udell says the following:

“Lambda functions are stateless and asynchronous which is great, except that it would be wonderful to share a few things like connection pools, that are expensive to setup. Connection pooling isn’t properly supported. Setting up and tearing down database connections for each request increases latency and affect performance.”

In other words, because Serverless functions are supposed to remain Stateless, each DB connection also must be Stateless. So instead of connecting once with connection pooling, you should need to connect to the DB from scratch every time you call to the DB, because with Serverless, each session is independent of the other. Makes sense in theory, but is it true?

How connection pooling CAN be done on Serverless

The proof, as always, is in the pudding. You can check out my Github repository to see examples of connection pooling working just fine on Serverless. We understand the misconception as it’s quite common. In fact, Umer Mansoor over at CodeAhoy, addressed this misconception quite well over a year ago in his blog post, saying:

“A common misconception that keeps coming up is that it’s not an appropriate choice for things that use a database connection because you have to create a database connection for every request you receive, and that’s just not the case!”

So how is it done then?

Simple. You ARE able to store variables outside the scope of our handler function. This means that you are able to create your DB connection pool outside of the handler function, which can then be shared with each future invocation of that function. This allows for pooling to occur. Again, you can check out my GitHub for examples.

const mysql = require('mysql');

const pool = mysql.createPool({
    host: {Your Host},
    user: {Your Username},
    password: {Your Password},
    database: {Your Database},
    port: 3306
});

exports.main = function main (req) {
  let query = ""
  return new Promise(function(resolve, reject){
    return resolve({
      statusCode: 200,
      body: "Success"
    });
         })
}

Note again that the connection to this DB is outside of the function.

Update: If you want to see the power of connection pooling in action check out these screenshots shown below. The first image is a graph from the Spotinst console showing the number of times that our MySQL function with connection pooling has been invoked. As you can see the graph spikes up to almost 80 invocations! But if you look at the second image taken from AWS, the connection to the database for the same time frame only goes up to 1. This goes to show that our function was able to maintain a single connection to the database and still successfully make multiple calls.