It’s the new year! If you are wondering what 2022 has in store for all of us in the world of data, then read below as we discuss the trends and challenges in data & enterprise integration that organizations may face in the coming year.
Data & Enterprise Integration Trends: What is in Store?
1. Explosive Growth Of Data
We live in a time where data about almost everything is being recorded and stored. With more sensors and business systems, more data is created. This exponential growth of data presents a unique challenge for organizations to process and distill data to drive decisions.
The benefits for organizations that tame data in the face of this growth are many. From increases in productivity and efficiency to being able to serve customers better. Data-driven decisions are paramount to the competitive business landscape of today. The more data is being used to drive decisions, the better those decisions will be.
Source: https://www.red-gate.com/blog/database-development/whats-the-real-story-behind-the-explosive-growth-of-data
[Further Reading] What’s the real story behind the explosive growth of data?
[Further Reading] 6 Predictions About Data In 2020 And The Coming Decade
Data Quality and Better Decisions
Data quality is another challenge organizations will face in 2022. As data continues to grow exponentially, determining and ensuring data quality gets more difficult.
What is data quality? Data quality is a measure of how accurate, complete, reliable, and suitable data is, depending on its purpose.
Data quality is something every organization works to improve. Universally, every organization that has adopted our platform is or is looking to leverage its data quality capabilities. This improves the quality of their data to make even better decisions. As you’ve all heard, “bad data is expensive”!
The two most popular workflows are:
- Checking data as it enters the system to ensure that the data meets quality standards. This ensures that new data doesn’t lower the quality of data. Data quality checks are important.
- Tasks that run to improve the quality of existing data assets. These include looking for missing fields, identifying and processing duplicates, and validating various data types.
As volumes of data increase, the system must be both highly efficient and scalable in both cases.
Efficiency and Scalability Means Fast Decisions
Whatever the task, the faster you can get it done, the bigger of an advantage you have over a competitor. This allows you to give your customers a better experience.
From improving data quality to making data-driven decisions, we strive to deliver data to organizations easier and faster than ever before by:
- Making the speed of our data processing engine faster so that a single engine can do more in the same amount of time. This saves time, money and frees people up for higher level tasks.
- Making it easy for users to do parallel processing on FME Server through data partitioning. Our “no-code” interface enables massive parallel processing capabilities. In the past, this has been the domain of high end developers.
2. Growth of Spatial Data
Spatial data and its importance across all business sectors will continue to grow. With the growing penetration of 5G, IoT, AR and other new technologies, “location” is critical to a new area of decision-making and business efficiency workflows. Here are a few ways we predict spatial data will grow in ubiquity and importance:
- Augmented Reality: While still in its infancy, AR is gaining traction. Persuading businesses to unlock the value of AR and augmented reality data requires the understanding of location. Everything is someplace all the time, every event occurs someplace. AR is all about augmenting the real world so without comprehensive understanding of location you cannot understand AR in its entirety.
- Enterprise Solutions: More and more enterprise solutions are building in spatial data support. Spatial data support offers a competitive edge over competitors. Snowflake, for example, added vector support to their data storage solution. Read this success story highlighting how the FME Platform integration with Snowflake makes new things possible. In 2022, we expect more Snowflake-like announcements from other companies. Spatial data importance is growing.
- Enterprise Integration: Enterprise integration is now expected to support spatial data. When companies adopt new technology, they make decisions based on the future. Missing comprehensive spatial data support is a big influence on those decisions and will only grow.
The ability to combine data, including spatial, across all systems gives an organization a leg up on its competitors. Safe Software is “the only Enterprise Integration Solution with comprehensive spatial data support” and delivers what no other solution can.
Our understanding of spatial data and its importance was recognized by Gartner in the Magic Quadrant for Data Integration Tools. Spatial data is our strength and a growing number of clients of all sizes are making the switch to our solution from other data integration and data quality tools. This trend shows no sign of abating. If your enterprise integration is without spatial data capabilities, you may find your decisions coming up short.
3. Augmented Reality
As mentioned above, Augmented Reality is continuing to gain traction. New handsets and underlying technology from Apple and Google continue to improve their native support. Augmented Reality is truly an example of next generation technology that is not possible without spatial data. Augmented Reality can’t be done with technology that doesn’t understand spatial. For example, you need to be able to pull together all sorts of spatial data (imagery, lidar, 3D models, etc.) to represent infrastructure or 3D buildings in a realistic manner.
New technologies like AR gives organizations solutions that lower the cognitive load on the mobile workforce by magnitudes. This makes arduous tasks like locating underground infrastructure simple, easy, and fun. Here is an example of how easy it is with AR.
Augmented Reality lowers cognitive load for locating and identifying assets in the field
4. Agile Enterprise Integration and CPU-time based Processing
Modern enterprise integration serves many different workflows:
- Connecting Applications: Reduce or remove any double keying. Derive decisions that require information from multiple systems. These workflows put more load on the system when the applications that are being connected are busy.
- Processing Data on a Schedule: This could be any data workflow, such as ETL to move data between systems, or produce reports. These workflows can often be controlled when they are run to avoid busy times. However, there are still definite deliveries that have to be met.
- On-Demand Data for Decision Makers: This workflow enables decision makers to request the data when they need it. For our system, these are delivered through “no-code” FME Server Apps via a web browser. Here, the load on the system is when the decision makers need the data. Users have certain expectations since data is delivered through a web browser interface.
- Real-Time Data Stream Processing: With the growth of sensors, 5G and enterprise systems that produce real-time logging, these systems must be able to handle the variability of the message stream and thus its subsequent load.
“Time to decision” is a competitive advantage. Organizations find that the demands on their system vary greatly over time. In the workflows identified above 1), 3), and 4) are loads that are difficult to predict. Hence, the deployment must be agile so that it adapts to changing load quickly. Traditional licensing models where organizations purchase a “maximum” processing capability is giving way to a new approach where organizations purchase “work that is done”. This trend is clear. Licensing models based on “work that is done” enable organizations to meet and adjust to their changing loads more easily. They also enable organizations that meet customers’ expectations.
Paying for “work that is done” is also more cost effective. It provides enhanced value across organizations’ decision making systems. It is all about “delivering information when, where, and how it is wanted”.
5. Real-Time Data Streaming
With the growing adoption of 5G and IOT, more real-time data solutions are being built with our technology for both users in the field and vehicles. We expect this to continue to grow as the case for improved efficiency, reduced costs, and improved customer experience continue to be the drivers. Real-time data and real-time decision making is critical in situational awareness. Knowing what is happening can save lives in a disaster or event management. It also ensures that resources are deployed to where they are most needed and have the biggest impact.
6. Hybrid Solution Deployment
This 2021 trend continues into 2022. Organizations needing solutions deployed to both the cloud and on premises. The world is now firmly in this hybrid mode, and organizations are looking for integration technology that best supports this. Enterprise Integration Solutions that run as efficiently on premises as in the cloud give organizations flexibility. This flexibility allows businesses to control speed, which systems are migrated and to decide where systems are run and makes the most sense.
While the trend is towards the cloud, many organizations need to run enterprise applications both on premises and in the cloud. Processing must be close to the data for good performance. There are many stories of poor performance when data is on a remote network. Here, latency is higher and bandwidth is lower. At Safe, we are working to optimize our technology across deployments from on premises and to all the major cloud platforms (AWS, Azure, and Google Cloud). We will continue to embrace these platform offerings and solutions so that processing can always be as close to the data as possible.
Our FME Cloud regions with more being added when requested by customers.
7. Graviton and ARM
Infrastructure costs in the cloud can quickly grow as your deployment grows. The Graviton processor family aims to offer a higher performance/cost ratio.
Amazon has just announced Graviton 3. With this announcement the efficiencies of this processor are more compelling than ever. We may be a bit ahead of the trend here, but we believe it will not be long until we see clients requesting or inquiring about deploying on ARM.
With both the Mac and AWS now embracing ARM you will see us release our server product on Graviton with FME 2022, with native Mac Apple Silicon support to follow for FME Desktop.
2022
And there you have it… Some of the data & enterprise integration trends and exciting things we saw in 2021 that we feel are going to continue to grow in 2022. If you have any thoughts, we always enjoy talking about the world of data and enterprise integration. Where do you see things going?
Please reach out!
We also have a webinar coming up about the above-mentioned trends. Attend the webinar to find out what we learned from 2021, how we adapted to unexpected data challenges and gain more insight into the data landscape in 2022.
One trend is clear. The world of data will continue to grow at an accelerated pace with spatial data getting more and more important. We at Safe, work tirelessly to ensure that we meet all your needs now and in the future so that you always get the full value from all the data to which you have access.
Don Murray
Don is the co-founder and President of Safe Software. Safe Software was founded originally doing work for the BC Government on a project sharing spatial data with the forestry industry. During that project Don and other co-founder, Dale Lutz, realized the need for a data integration platform like FME. When Don’s not raving about how much he loves XML, you can find Don working with the team at Safe to take the FME product to the next level. You will also find him on the road talking with customers and partners to learn more about what new FME features they’d like to see.
