Don Murray is the co-founder of secure software And has spent his career helping organizations bring data to life to make better decisions.
How organizations store and access data has been undergoing significant change and evolution over the past decade. On the one hand, the cloud computing market is expected to reach more than $2.4 trillion USD by 2030, indicating an ongoing trend toward public and private clouds. On the other hand, many businesses and industries still choose to store some data on-premises or otherwise so that it cannot be accessed remotely, with security concerns and protecting intellectual property being the main reasons behind keeping data away from the cloud. It has been told.
I’ve been in technology a long time and know that there is no silver bullet. The world of data is no different. While it’s unlikely we’ll see a cloud-only future, it’s also unlikely that companies will go back entirely to on-premises. The future of data is hybrid, with companies leveraging both data storage approaches that best fit their business model and the regulatory landscape they operate in.
As we embrace a hybrid future in which data resides in multiple locations, this has a significant impact on how we process data. Today data is so distributed and so large that it is not possible to move it to a processing location. Now, businesses must find ways to move processing closer to the data – and the closer, the better. Doing this improves performance (reduces latency and increases bandwidth) to get derived values faster. This also reduces any transfer costs. From a security perspective, it also reduces the risk of data interception during transfer.
However, what does this look like in practice?
Reducing data processing costs
When you move the data you have stored out of a cloud service, you will have to pay for its exit. This can grow rapidly and moving the data to be processed to another location can be costly. Avoiding this cost is another reason to shift processing capabilities to data.
Think about growing sensor networks that constantly monitor systems to help identify if something is out of the ordinary and needs to be flagged. This could be a water level sensor in a river basin or a security camera on the shore. These sensors can send all their data over the network to a remote processor, which will flag when the river goes above a certain point or an unusual shape is seen on camera.
A better approach is to move processing to where the data is collected and then send only critical or unusual event data to connected systems. This results in a system that requires less network bandwidth, performs better, and scales better.
This is nothing new and that is what databases are all about. Databases are not just places to store data; They come with powerful query/processing capabilities to perform processing in the database. No one would even consider taking all the data out of the database for processing, when it can be done within.
Whenever you know where your data is stored, you should consider starting your processing as early as possible. If you’re combining data from multiple locations, consider an architecture that results in as little data as possible being delivered for processing. In fact, you are likely to find that you need a data processing environment that directly supports the distributed processing framework.
Minimizing the risks of data processing
Moving processing to data can also help from a security perspective. There are many organizations that have critical infrastructure for which they are responsible or have proprietary information that they do not want transferred or accessed remotely.
For remote access to data, you need to provide access and open holes in your data center. These disclosures introduce the risk that a third party gains access to that information. The same is true when data is in motion. Even with a strong data security strategy for data in transit, bad actors are looking for ways to intercept and steal information. In other words, the more data and paths, the larger the attack surface.
Installing processing within data centers helps businesses gain the insights they need to be strategic without compromising data.
Introduction to speed and flexibility
As I talk to our customers and partners, I see a lot of enthusiasm for putting processing agents closer to the data. For teams that have a central processing environment, this will give them the ability to distribute processing capabilities and do more with the data. Local processing will enable all organizations that have data in databases and applications to do more with their data and get more value from it.
Make smart choices about where you store (and process) your data
What does all this mean for your business? For starters, you can take a more discretionary approach to determining where you store your data. Ask yourself these things:
• Where is the best place for your data—public cloud, private cloud, or on-premises? The ability to move processing gives you greater flexibility.
• If you’re integrating data from a web-based SaaS organization, find out where the data center is. Often, the organization will share it, and running your processing in the same data center can make a difference.
• It is always good to consider whether there is an opportunity to move your processing to data.
Learn which hybrid models make the most sense for your business. There are a number of ways you can cut costs and better secure your data, as well as take advantage of all the insights data provides – allowing you to take your business to the next level .
The Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs, and technology executives. Am I eligible?
