Enterprise Integration - Async vs Sync

Introduction

Manhattan Active® Platform is API-first, cloud-native, and microservices-based. All Manhattan Active® Solutions are born in the cloud, hosted on Manhattan Active Platform, and are highly available, elastic, scalable, secure, and resilient. For enterprise systems to integrate and communicate, Manhattan Active Platform provides both Synchronous and Asynchronous communication options. This document describes the integration landscape and compares options for enterprise integration with Manhattan Active® Solutions.

Integration Landscape

As depicted here, you may integrate with Manhattan Active Solutions using a variety of options. You do not need to use just one option, and can pick between them based on the use case, the nature and volume of the data being transmitted, and the amount of infrastructure you would like to manage. Some pros and cons are listed later in this document.

Note: Manhattan Integration Framework (MIF) is an optional offering. MIF is based on Software AG webMethods and is an enterprise integration tool that will help convert both inbound and outbound messages between Manhattan Active Solutions and any enterprise host system.

Synchronous REST Integration

All Manhattan Active® APIs follow the same calling mechanisms and uniformly communicate via JSON payloads. All APIs in Manhattan Active Solutions are accessed via HTTPS REST endpoints. So, an authenticated user can directly call any Manhattan Active API endpoint. With REST-based integration, feedback to the caller is immediate. However, the caller must have the necessary infrastructure to handle failures and scale calls to achieve the required throughput.

Asynchronous Queue Integration

Queue-based integration with Manhattan Active Solutions follows the same integration pattern for all interfaces. You build these asynchronous queue integrations using Google Pub/Sub and may choose from a wide range of integration options provided by Google that includes Pub/Sub APIs and a wide variety of client libraries. Once messages are posted into the dedicated Google Pub/Sub queue, Manhattan Active® Platform’s messaging infrastructure picks up and routes these messages to the target endpoints while supporting a full-featured message queue management system to manage the delivery of messages.

Note: Asynchronous Queue Integration delivers JSON messages to the same REST endpoints as Synchronous REST Integration with the added benefits of message queue management capabilities.

Integration Factors to Consider

This section provides guidance for choosing the appropriate approach for individual use cases by comparing Synchronous and Asynchronous capabilities against key integration requirements.

Scaling

Auto-scaling is the ability to automatically scale up or down to meet the fluctuations in demand for interface calls. This is a critical requirement for large-volume communication between external host systems and Manhattan Active Solutions.

With Asynchronous Queue Integration, Manhattan Active Platform dynamically scales up and down the queue processors to meet the demands of large message inflows based on the response times of the target Manhattan Active API endpoints. It does this to account for variance in the response times stemming from the non-uniform nature of inbound payloads (for example: the save order endpoint will generally process an order with 10 lines faster than an order with 250 lines).

With Synchronous REST Integration, downstream systems will need to handle dynamic scaling by closely monitoring API response times and scaling up/down the number of threads making API calls to achieve the desired throughput. Manhattan Active Platform will automatically scale REST services based on the call rates but cannot guarantee response times due to the variability of the inbound payload. Also, note that there are appropriate guard rails in place to throttle the maximum number of calls allowed per second per API. In general, the rate limits set are higher than most reasonable business needs. Rate limits are necessary to protect the environment and maintain the overall health of the environment.

Typically, downstream systems integrating synchronously will require more infrastructure as they will need to wait for the synchronous calls to respond from the upstream system.

We recommend that if high throughput is a criterion, Asynchronous Queue Integration will typically achieve better overall throughput and require less infrastructure and management in the downstream systems.

Error Handling

Error handling is the ability to capture and handle errors. Errors can occur for various reasons, and such instances need proper handling.

With Asynchronous Queue Integration, Manhattan Active Platform provides comprehensive error handling capabilities which allow automatic and manual resolution of errors. In the event of a message posting failure, Manhattan Active Platform automatically retries posting the message multiple times after a configurable interval. After exhausting the retry limit, the message is posted into a staged queue for remediation.

Customers have multiple options to monitor staged queues:

• Monitoring dashboard to periodically check for any error build-up.
• Purpose-built user interface to manage failed messages including capabilities to correct and repost data.
• API endpoints to programmatically manage failed messages as above.
• Capability to configure alerts based on customizable data conditions to get notified of any failures as they happen.
• Detailed logs of failures.

With Synchronous REST Integration, it is recommended that the caller captures and corrects errors, stages failed payloads and reposts payloads to continue business operations.

When immediate feedback is not a requirement, Asynchronous Queue Integration will provide better error handling and retry mechanisms. If the downstream system already has sophisticated error handling and retry mechanics, both approaches may be appropriate.

Traceability

Traceability is the ability to capture logs of service calls and the ability to debug/troubleshoot messages.

Like error handling above, traceability and audits of message inflow and status are your responsibility when Synchronous REST Integration is chosen. With Asynchronous Queue Integration, Manhattan Active Platform preserves detailed logs and payloads in addition to a full trace of errors encountered during message processing.

System Isolation

System isolation refers to the ability to independently operate one application regardless of the availability of another system to ensure systems don’t directly impact one another.

Acknowledgement

The ability to receive a response or an acknowledgment after calling an API or posting a message.

Overall

Our experience shows that Asynchronous Queue Integrations typically provide better system isolation, error handling, traceability, and scaling whereas Synchronous REST Integration is the appropriate choice when an immediate response is needed for a business requirement.