Can the Current Database Technologies Cope Up with the Intense Growth of the Internet?

Big data is getting bigger and bigger each year with the advent of mobile phone / gadget use.

Big data is certainly getting bigger.

McKinsey Global Institute forecasts that the volume of data you get on the Internet will grow at around 40% a year, representing a growth of 44x in 11 years, between 2009 to 2020.

The thing with big data is that it is not only the volume that you need to consider. As more and more machines become smarter, there is now an influx of machine generated data that you have to store and analyze. Then you have more people with Internet access creating a whole lot of content that you should capture.

Aside from volume, you should also check out velocity or frequency that data is coming in. Case in point, social media allows your customers to share their opinions and you need to capture these opinions because these are important to your customer relationship management.

Then you also have to consider variety. In the old days, data came to you pretty much structured and follows a specified data format. These formats are pretty much standard and changes slowly over time. Today, you get unstructured data using different formats and is constantly changing.

Further, you need to consider value. Not all data sets are gold mines. You need to identify which sets of data are valuable and you need to know how to capture, extract, transform and analyze the data for it to be useful.

Oracle gives you an idea of just how fast data is growing. A single jet engine generates up to 10 terabytes of data in just half an hour. Considering that there are 25,000 flights each day, jet engines all around the world can generate petabytes and petabytes of data daily. Meanwhile, Twitter might be limiting data to only 140 characters, but the speed with which tweets are sent out means that Twitter as a data source adds around 8 terabytes of data every day.

As you can imagine older databases will not be enough to handle the volume, variety and velocity of big data. What are the database technologies you need?

• Column-oriented databases. If you are running a database for online transaction processing, then you would do well with traditional databases because these are row-oriented and updates are very fast. However, row-oriented databases often falter when data volumes grow. This is why a column-oriented database is needed for the speediest query times and data compression.

• NoSQL. NoSQL databases works best with unstructured data, because it does away with restrictions that are imposed by traditional databases, allowing you to scale easily.

• Hadoop. This is an open source platform that is geared towards managing big data. It helps you work with data coming from different sources.

• SkyTree. It helps you with data analytics and machine learning when dealing with big data.

• Hive. This allows you to use your traditional BI applications to query Hadoop clusters. Other similar technologies that you should know include PIG, WibiData, and Platfora.

If you are thinking about how to get your big data initiative right, then you should call Four Cornerstone. We can help you determine all the right technologies that would allow you to get your databases ready for the unrelenting growth of the Internet.

Photo by Christoph Scholz.


Security Trends For 2024: SBOMs

A recent article on Forbes, linked below, lists five security trends that are likely to be important in 2024. Of course, AI-related security topics top...

Keep Reading


Getting started with your first…

Anyone looking at writing their first Generative AI application should read this to save time. In June 2023, Google Cloud published a useful blog post,...

Keep Reading


Read-Write and Read-Only query splitting…

Optimize the usage of the standby (secondary) read-only MySQL InnoDB Cluster servers by automatically and transparently distributing read-queries to them. A great feature in MySQL...

Keep Reading


Considering Cloud Diversification?

Moving an enterprise to a Public Cloud can quickly feel like moving into a locked-in relationship with the Cloud vendor. One solution to diminish this...

Keep Reading


Gartner: Global cloud consumption will…

Gartner forecasts that 2024 will see a 20% global cloud consumption increase from 2023 to $679B, while 2023 has seen an 18% increase from 2022...

Keep Reading


ByteDance: Use AI for tuning…

ZDNet is reporting that a ByteDance (the maker of TikTok) Linux kernel developer has proposed to implement AI to tune the performance of Linux systems...

Keep Reading

AI Governance

What is AI TRiSM?

If reading now about AI TRiSM for the first time, it might be good read on and learn what it’s about because it leads the...

Keep Reading

Cloud Blog

4 Ways To Benefit from…

One of the benefits you get when you work with cloud applications is that you often have quarterly updates that are packed with features. This...

Keep Reading

Artificial Intelligence

Data and Analytics: Cross the…

  Artificial intelligence is a manna sent from digital heaven. That’s how blessed your business can get if you immerse into the AI of things....

Keep Reading

Business Intelligence Blog

How Brand Names Survive in…

  The age of digital marketplace has made it possible for unknown and smaller companies to compete with better-known and well-established brands. Take for example...

Keep Reading

Live Chat | Emergency