Creating a Cloud Function to publish messages to Pub/Sub

Every time I want a cloud managed message queue, I would look at AWS SQS service. It’s simple. Create a SQS queue, get the HTTP endpoint for the queue, start posting the messages using any HTTP client like curl.

This time I made up my mind to give GCP’s counterpart a try.

The GCP’s counterpart is Pub/Sub. On a high level, the queue is known as topic in the Pub/Sub terms. There will be one or more publishers who will publish to the topic and one or more consumers who will subscribe to the topics. There are other differences with GCP’s Pub/Sub and AWS’ SQS but thats not relevant to this experiment.

GCP allows different ways to publish messages to a topic but not simple HTTP endpoint. Searching online on how to add a http endpoint to the Pub Sub gave multiple (useless) answers. Since GCP documentation has used very good SEO techniques, you tend to NOT get the answers for any GCP related queries easily.

Finally, I found that Cloud Functions is the simplest way to attach a HTTP endpoint. Here’s how I did it:

1. Create a GCP Service Account and assign it the role “Pub/Sub Publisher”. This will be a Google managed service account.

2. In Pub/Sub section, create a topic with Encryption set to “Google-managed key”

3. In Cloud Function section, click on “Create function”. For this example I would be using Python runtime with the following code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

import os
from google.cloud import pubsub_v1

def hello_world(request):
    try:
        testAttribute = request.get_json().get('test')
        topic_name = 'projects/badshah-project/topics/pubsub-publisher'.format(
            project_id=os.getenv('GOOGLE_CLOUD_PROJECT'),
            topic='pubsub-publisher',
        )
        publisher.publish(topic_name, b'', testAttribute=testAttribute)
        return f"OK"
    except:
        return f"Not OK"

Make sure you add google-cloud-pubsub to the requirements.txt section

Click on “Environment variables, networking, timeouts and more”. Under “Service account” select the service account that was created in step 1.

To trigger this cloud function, use the following curl command:

1
2
3

curl -X POST https://asia-east2-badshah-project.cloudfunctions.net/pubsub-publisher \
  -H "Content-Type:application/json" \
  -d '{"test":"Hello World"}'

When this returns a message “OK”, then an empty message with attribute test = Hello World is published into the PubSub topic.