Odtnhj

We're using Firebase for our app that needs to process a some data and then send out a series of e-mails after their data has been decided.

Right now I'm triggering a single handler via CRON (which uses pub/sub) that processes the data and then publishes a series of messages to a different pub/sub topic. That topic in turn has a similar trigger function that goes through a few processes and then sends an single email per execution.

// Triggered by CRON task
const cronPublisher = functions.pubsub.topic('queue-emails').onPublish(async () => 
 //processing
 ... 
 // Publish to other topic
 await Promise.all(
 emails.map((email) =>
 publisher.queueSendOffer(email)
 )
 );
);

// Triggered by above, at times twice
const sendEmail = functions.pubsub.topic('send-email').onPublish(async () => 
 //processing and send email
);

The issue I'm running into is that the 2nd topic trigger at times is executed more than once, sending two identical emails. The main potential cause I've come across by way of Google just involves long execution times resulting in timeouts, and retries. This shouldn't be the case since our acknowledgment timeout is configured to 300 seconds and the execution times never exceed ~12 seconds.

Also, the Firebase interface doesn't seem to give you any control over how this acknowledgment is sent.

This CRON function runs everyday and the issue only occurs every 4-5 days, but then it duplicates every single email.

Any thoughts?

Appreciated.

If 'every single message' is duplicated, perhaps it is your 'cronPublisher' function that is being called twice? Cloud Pubsub offers at least once semantics, so your job should be tolerant to this https://cloud.google.com/pubsub/docs/subscriber#at-least-once-delivery.

If you were to persist some information in a firebase transaction that this cron event had been received, and check that before publishing, you could prevent duplicate publishing to the "send-email" topic.