Table of Contents
In software development, efficiency often hinges on choosing between building from scratch or leveraging pre-built solutions. For Cashfree Payments, an API-first company, this decision has shaped our journey in building Software Development Kits (SDKs) to simplify and enhance our merchants’ integration experience. Initially, we took the manual route, crafting SDKs line by line to align with our API updates. But as we grew and the demand for seamless integration increased, we saw the need to evolve—moving from fully manual development to a more automated, streamlined approach.
Introduction
Elevating Software Development Kits as First-Class Citizens
Cashfree Payments, as an API-first company, constantly writes new APIs and modifies existing ones. These APIs are made available to merchants, enabling them to integrate our products seamlessly. A merchant can write their own network layer and independently integrate all required APIs. However, this approach involves the complexity of building all necessary data models to handle requests and responses.
What if we could simplify this process for them while also allowing them to modify network-level configurations (such as setting up proxies, timeouts, and connection pooling)? By taking the other route, integration would save development time and expedite production. The developer will just end up writing code that is as simple as Cashfree.PGFetchOrder() to get the details of an order.
First Approach: Building SDKs Manually
When we moved to new-generation APIs, we also built development kits for the backend. Our first approach was completely manual. A single developer worked on building an SDK for Java (to be published to Maven Repository). This involved spending all his bandwidth setting up the entire project from scratch and manually writing the code for every API. This approach did not scale.
Second Approach: Combining Automation with Manual Effort
To improve the developer experience, we were determined to build all the necessary SDKs so that our merchants’ developers can easily integrate Cashfree. This time, we used the OpenAPI specification to generate SDKs.
The OpenAPI Specification (OAS) defines a standard, language-agnostic interface to REST APIs. This interface allows both humans and computers to discover and understand the service without access to source code, or documentation.
An OpenAPI definition can then be used by documentation generation tools to display the API, code generation tools to generate servers and clients in various programming languages, testing tools, and many other use cases.
OpenAPI specification, along with the openapi-generator tool (the dark knight) (openapi-generator), we built our first SDK automatically and published it to respective repositories.
While the automatically generated code works, it still lacks seamless code integration. Every time we generated a new version, we manually changed the code before deployment to make it fit our needs. For instance, we wanted to collect a config to set the environment and network layer configuration
This approach was not scalable as it always included more manual intervention to make changes that adhered to our needs, mostly to make integration easy and seamless. Generating multiple languages and deploying them to repositories would also end up taking developer bandwidth more than expected.
Third Approach: Exploring Companies for End-to-End Integration Support
We were still adamant about making integration with Cashfree easy for the developers. This was when we started exploring software companies that provided services where they generated the SDK for us using our OpenAPI specification file and also automated the process of releasing the SDK to their respective repositories. While this approach would improve our time to market for the SDKs, we were still exploring the possibility of automating this in-house. This led to our fourth attempt to create an in-house development kit that was automated from creating SDK to publishing with minimal manual intervention.
Fourth Approach: In-House Automation (Resilience Pays Off)
From our previous attempts, one thing was clear with the data that we had collected, developers did use SDKs. One trend we observed was that the usage for nodejs sdk had skyrocketed. This helped us decide which language to target as our first deployment.
Developing nodejs SDK
- The first challenge was to perfect the OpenAPI specification file. The operationId present in the OAS file became the name of the method. We had to make it simple to use and understand. For example, the operationId for creating an order at Cashfree became PGCreateOrder → <product><action><entity>
- The SDK is generated using the OAS file along with an open API. This time, the approach was to make the SDK generation automated.
- CI/CD pipelines to the rescue. All the steps to generate the SDK were written as commands in the bitbucket-pipeline.yml file.
- The OAS file was also added to the same repository, with a condition that the entire pipeline run if and only if there was a change made to this OAS file.
- With some things sorted, the SDK still was not easy to use. With the generated SDK at this point, the integration steps looked like
const configuration = ClientConfigiration()
const orderService = OrderService(configuration)
orderService.PGCreateOrder()- We wanted to make it something as easy as cashfree.PGCreateOrder()
- Mustache templating became the way forward. (https://openapi-generator.tech/docs/usage/#template)
Mustache’ing Our Way Through
When we dived deeper into templating provided by the openapi-generator tool, we found that the tool internally uses mustache files, and the code is generated based on that.
The templates can be generated by using the command openapi-generator author template -g typescript-axios -o node template
openapi-generator author template -g this part of the command is fixed. -g is short for generate and what follows this command is the language that we want to build the SDK for (list of languages mentioned above)
-o node template generates the template and stores it in a folder named node template
This template has all files and language-specific code templates with placeholders that the tool uses to generate the SDK.
The openapi-generator tool provides commands to pass templates while creating the SDK. With this figured out, we downloaded the templates, made necessary changes, added extra code wherever needed, passed this template information as command line arguments, and generated SDKs.
Initially, we experimented with the NodeJS SDK. Once we reached a final solution for the SDK, we moved on to creating SDKs for other languages as well (Python, C #, Go, PHP, and Java).
Before Using Templates
After Using Templates
Mustache Template Variables
The OpenAPI specification file, written in YAML or JSON format and adheres to OpenAPI specification rules, gets converted into a mustache template structure when it passes through the generator. A basic structure looks like:
The mustache structure also consists of path parameters, function parameters, model information, and many more. For more information regarding mustache variables, visit here → https://github.com/swagger-api/swagger-codegen/wiki/
Node SDK template for the above structure will look like
Operation with all parameters
Example: Fetching an order
OAS File
Method that is created from mustache file
public static PGFetchOrder(x_api_version: string, order_id: string, x_request_id?: string, x_idempotency_key?: string, options?: AxiosRequestConfig) {
}Command to generate sdk
Automatically Pushing to Github
All our SDKs are public. Bitbucket Pipeline helped generate the SDK, but it had to be pushed to GitHub for the next steps (releasing it to the public).
Further, we wrote steps to raise a PR in GitHub repositories from bitbucket-pipelines.
The steps involved pulling the master branch from the respective repository, copying the newly generated SDK, creating a new branch, and raising a PR.
Running tests on Pull-Requests and release SDK
Once a pull request is created, unit tests run in Git Hub. These steps are written as part of GitHub actions. One of the team members has to merge this Pull Request, and another GitHub action runs that pushes the SDK to its respective repository.
- NodeJS SDK is pushed to npm
- Go is pushed to pkg.go.dev
- Dotnet is pushed to Nuget
- Python is pushed to Pipy
- Java is pushed to Maven
- Php is pushed to packagist
- GitHub Actions to build and push nodejs sdk to npm
Github Actions to build and push nodejs SDK to npm
Conclusion
While there is still a little manual intervention in developing the SDK, it has reduced months of developer bandwidth in developing and releasing SDKs. With improved test cases and improved test coverage, we aim to automate the merge process as well. It has been almost a year since we started deploying SDKs with this approach, and we have seen a tremendous increase in the number of developers using our SDKs. While nodejs still stands at the top, being the most used SDK, other languages are catching up as we speak. I hope this was helpful. This was us documenting our journey to creating our in-house SDK generation process.
https://github.com/OpenAPITools/openapi-generator
