I have been asked many times in my career about my thoughts around using middleware versus writing custom code to integrate systems that need to communicate with each other. My thoughts have changed over the years as my experience has grown significantly. There are a lot of challenges that need to be thought about and considered when you are making a decision between middleware or writing custom code and what is best solution for your situation.
With whatever solution you go with, the solution will have to govern or handle the following areas.
Internal Applications or SaaS that have changing API’s: A lot of the applications that you might be using have APIs available to allow other applications to integrate with them. These APIs can change with new releases of the application as a company enhances their software to accommodate new functionality and features. If you decide to write custom code to handle the integration you will need to get skilled programmers who will then have to get a firm understanding on how the API works (transaction rules, capabilities, schema, authentication, etc) for each application, after each new revision or deployment. If you are using middleware the configuration of the integration process can be changed easily and quickly when APIs change.
Handling business process changes: As a company grows business process will most likely change. These changes can include new or different business rules, data transformations, and business logic. With constant changes to the business and if you choose to write custom code, that code can grow to become unmanageable. With the use of middleware all changes are done through a graphical user interface which includes connections, transformations, business rules, flow logic, and data mappings. Most if not all these changes are done without writing any code.
Monitoring: More complex integrations require the integration between the systems to be monitored. If you have choose to do the system integration with custom code the developers will need to write all the monitoring and error handling code which can take a significant time to write possibly pushing your project longer then expected. With most middleware solutions monitoring, error handling, and logging is implemented with some point and clicks and an update to the configuration file.
Optimizing Processes: Most API’s have stipulations within them to help manage the integrity and performance of the application. These stipulations can add to the complexity of the integrations by restricting the amount that can be sent or received in each call, required authentication, how many times you can make the call to just name a few. With just some of the stipulations that I have name, the complexity of the custom code goes way up and a significant amount of code must be written and tested to handle them. Middleware usually contains functionality (tested functionality) to handle these stipulations and makes sure all the processes are optimized and the data is delivered faster.
My Conclusion: I have built many middleware solutions in my career and the codebase at times was very large depending on the complexity of the integrations. Within each project the business owner always thought that integration should be easy by saying “Hey you are just moving data from one point to another point”. I can tell you that is usually not the case at all. Based on my experience using a middleware solution is usually the most cost effective solution to go with in delivering a speedy ROI back to the business. Most if not all the functionality is already been developed and tested thoroughly so there is no need to reinvent functionality that has already existed.